Skip to main content

Using next_items_page in Azure data factory

  • September 23, 2024
  • 1 reply
  • 24 views
A

I can do an initial query of our board & items but I am having trouble using the built in pagination in Azure data factory. The issue is that this API works differently than most and I have to do a different query and specify a value for the cursor rather than simply an absolute URL for the next page. Can anyone show me how to do this?

1 reply

denlunev
  • denlunev
  • Participating Frequently
  • 7 replies
  • October 16, 2024

here’s a quick approach:

Initial setup: Use a Copy Data activity to call the API.

Custom pagination: Instead of the usual URL for the next page, use the cursor-based pagination.

  • In your dataset, set up the initial query (e.g., fetch items from a board).

  • In the request body, add the cursor like this:


{

"query": "query { boards { items { id name } pageInfo { hasNextPage endCursor } } }"

}

Pagination settings:

  • Go to Source settings in your Copy Data activity.

  • Under pagination, set the Next Page Token as $.data.pageInfo.endCursor and Has More Pages as $.data.pageInfo.hasNextPage == true.

Dynamic cursor: Use @{activity('YourCopyActivity').output.paging.endCursor} to move to the next page.


Terms & ConditionsCookie settingsAccessibility statement