Platform Allowances

note icon

This page summarizes the key limits you need to be aware of when building your solution.

For further discussion on these limits, and how to monitor your usage see the Understanding Limits guide.

Analysis tasks

The total number of analysis tasks you can process per hour for your account will depend on the package you subscribe to.

For query filters we currently limit CSDL to 30 logical operators, each with a maximum of 100 arguments.

For frequency distributions you can return up to 200 items.

For nested frequency distributions the maximum depth of nesting is three levels - a parent, with a child and grandchild.

For nested frequency distributions the overall 'threshold product' for a nested query is limited to 80,000, except for queries that analyze user skills for which the limit is 1000.

For time series analysis the maximum period you can analyze depends on the interval size you use:

  • minute - Maximum period is 60 minutes
  • hour - Maximum period is 336 hours (2 weeks)
  • day - Maximum period is 32 days
  • week - Maximum period is 4 weeks
  • month - Maximum period is 1 month

API rate limit

Your account package determines your allowance, specified in credits per hour. For example you may be allowed 10,000 credits per hour. Each call to an API endpoint consumes a small number of credits which are taken from this allowance.

If you consume all of your credits you will be 'rate limited' and your API calls will no longer be accepted. You will need to wait until the next hour begins, when your credits are reset.

You can find the cost of calling each endpoint in the Resource Information box on its documentation page. For example, the cost of each call to GET /pylon/{service}/task endpoint is 5 credits.

Full details on platform rate limiting are available on our API Rate Limiting summary page.

Note that this rate limit applies to your general API usage, including the DataSift STREAM product if you have access to this. However it does not apply to calls to the POST /pylon/{service}/task endpoint which has a separate rate limit, explained next.

API rate limit for POST /pylon/{service}/task

As well as a general API allowance your account package also determines how many tasks you can submit to the POST /pylon/{service}/task endpoint.

The number of tasks you can submit, and rate at which the tasks are processed is determined by the number of 'slots' included in your account package. A typical account is allocated one slot.

For each 'slot':

  • you can queue up to 1,000 analysis tasks at any time
  • tasks are processed at a best effort rate of 160 tasks per hour
  • if you have tasks on your queue then they will be processed in turn until the queue is empty (at the best effort rate)