Managed Sources 2016+ Architecture

Why has DataSift built a new Managed Sources architecture?

With ever increasing usage of our Managed Sources services, we began to reach the limits of how far the existing Managed Sources architecture could scale, and the number of Managed Sources and resources we could support. In some cases, this caused data collection to become delayed, and some of our customers began to let us know that the service was not reliable enough.

We built the new architecture to handle this increased load, scale horizontally, and resolve a number of bug reports, usability complaints and feature requests we'd been unable to implement since we introduced Managed Sources all those years ago.

What are the differences between the new and existing architectures?

This is a large topic, so take a look at Managed Sources 2016+ Architecture: What's Changed, for full details of all the changes you should expect.

Supported Managed Source types

The following Managed Source types will be supported:


Tracks Instagram users, tags, or location-based content.

Facebook Pages

Tracks activity on a set of Facebook Pages.

Getting started

You can configure Managed Sources via the API or dashboard.

Configuring sources using the API

Learn how to configure sources using the API in our step-by-step guide.

Configuring sources in the dashboard

Learn how to configure sources in our how-to guides:

Best practices

It's easy to get started with Managed Sources but there are important points to keep in mind.

Advice for each source can be found on the source's overview page.

In addition, take time to read our best practice guide.