Querying with large limit is resulting in errors

Hi @spb,

50,000 items is a lot… My guess is this puts you complexity way over the limit.

Depending on your query and the depth, I would probably look at using a limit of a few hundred to a thousand at a time.

Hi @mitchell.hudson, is there a way to get the total count of all items? This way we could break it into lots of small queries. I’ve had queries of 200 000 with complexity 6million and also only recently started getting the technical issues response.

Is there a better way to get all items?

@morneluckybeard so you are looking to get the total number of all items in a monday account?

@dipro @Scott-monday.com any info on the technical issue response?

Not necessarily for the entire account, but let’s say I create a query with defined limits, the pagination is great, but it doesn’t help me if I don’t know how many pages to iterate through. I could call sequentially and check if an empty list returns, but this will take just as long as making a large query as the calls need to wait for the previous call to complete. I was hoping for some way to get the total item count based on the page size that I provide. so for a total of 1000 items / page size(200) = 5 pages.

@morneluckybeard I don’t believe there the API returns how many pages of items there are.

You would need to keep making a request until you return an empty response

That is rather unfortunate. Hopefully that gets added at some point as waiting for each call may actually take longer than just submitting a huge query.

Appreciate your feedback!

Dipro from the monday.com team here – sorry for the late response.

Why not make N concurrent requests at a time? You can check the Nth response to see if there are more items to return. This means you can keep each request smaller (lower the chance of hitting a timeout or 500 error) and still get a lot of data at once.

Here are my responses to the two questions you have:

Queries with big limits

At the moment we don’t have a hard limit on how many items you can request at a time. However, we do have a 60 second timeout for all requests, so if you’re returning a ton of data you might see a timeout.

And of course, if you’re asking for a really big data set an error might be thrown on the infrastructure side and cause a 500 error.

Paging through responses

We don’t currently expose the number of pages when you’re paginating, but I’ve passed that as feedback.

Thank you, the pagination count would be great. I’ll give your solution a go, thanks! As for the OP’s question, the technical issues are returning html instead of an error code. Hopefully this can be solved as well.

Thanks again for all the help and feedback!