Skip to main content

I can do an initial query of our board & items but I am having trouble using the built in pagination in Azure data factory. The issue is that this API works differently than most and I have to do a different query and specify a value for the cursor rather than simply an absolute URL for the next page. Can anyone show me how to do this?

here’s a quick approach:


Initial setup: Use a Copy Data activity to call the API.


Custom pagination: Instead of the usual URL for the next page, use the cursor-based pagination.




  • In your dataset, set up the initial query (e.g., fetch items from a board).




  • In the request body, add the cursor like this:






{



"query": "query { boards { items { id name } pageInfo { hasNextPage endCursor } } }"



}

Pagination settings:




  • Go to Source settings in your Copy Data activity.




  • Under pagination, set the Next Page Token as $.data.pageInfo.endCursor and Has More Pages as $.data.pageInfo.hasNextPage == true.




Dynamic cursor: Use @{activity('YourCopyActivity').output.paging.endCursor} to move to the next page.

Reply


Terms & ConditionsCookie settings