Targets vs Output Data

DataSift filters data and delivers the results to you:

  • A target is something that you can filter on
  • Output data is something that you receive from DataSift

Thus, a key point to understand is that targets are not the same as output data

On this page, we'll look at the three different cases you will encounter:

  • when data is available in both a target and in the output
  • when it is available only as a target
  • when it is available only in the output

Target and Output

The most common case you will encounter is when information is available both as a target and in the output. For example, when you filter for people on Twitter who have more than 1,000,000 followers, the output that you receive will include their exact follower count.

twitter.user.followers_count > 1000000

In this CSDL code, twitter.user.followers_count is the target.

You can see the output element, in JSON format, in the following code snippet:

"twitter": {
        "created_at": "Wed, 08 May 2013 10:20:52 +0000",
        "filter_level": "medium",
        "id": "332077635188884496",
        "lang": "en",
        "text": "This is just an example...",
        "user": {
            "name": "dummyuser",
            "url": "http://example.com",
            "statuses_count": 81785,
            "followers_count": 1547316,

In this example, your CSDL filter tells you nothing about the content that you will receive. The filter simply finds all Tweets posted by people with a large number of followers.

Target only

You will see cases where information is available as a target so you can filter on it, but it is not in the output. For example:

interaction.sample < 10

Output only

Finally, you will find cases where information is not available as a target, so it is not filterable, but it does appear in the output. For example: the created_at field is often presented in the output data but it is not a target.

Conclusion

The structure of the JSON objects in your output data closely matches the namespaces in the targets. In many cases, they look the same; for example, there is a target called twitter.text and there is a corresponding output element in the JSON output:

{
    "twitter": {
        "text": "Hello!"

However, they do not always match exactly, so remember that the target documentation describes the targets you can use for filtering rather than the JSON output that DataSift delivers.

Data Dictionary

Our Data Dictionary provides an overview of all the targets and output data elements.