Getting Started - Node.JS

This Quickstart Guide

What you'll learn: How to connect and stream live data from the DataSift platform to your machine.
Duration: 10 minutes

Table Of Contents

Before You Start

Before you get started make sure you've completed the Developer Quick Start guide. This will take you through:

  • Activating the Tumblr data source
  • Activating the Links augmentation

You'll need to have done these steps to continue.

Once complete, sign-in to DataSift and make a note of your username and API key on your dashboard. You'll need them in a minute.

Step 1: Install The Node Package

The Node library is available as a package on NPM here.

You can install the package at the command line:

npm install datasift-node

Or, of course you can specify the package in your package.json file!

Step 2: Create A DataSift Client

With the package installed, now you can write a script to access the API. Firstly we need to create a client object that will access the API for us.

Start a new Node script with the following code (inserting your username and api key):

// Include the library
var DataSift = require('datasift-node');

// Create a client
var ds = new DataSift('YOUR_USERNAME', 'YOUR_APIKEY');

Step 3: Compile A Filter

In order to stream data from the platform, you need to create a filter in CSDL. You compile this filter using the API and receive a hash that represents the filter.

Add a function that performs this task to your script. Notice we call connect (which we'll declare in a second) once the filter has been compiled:

var filter = 'interaction.content contains_any "Calvin Klein, GQ, Adidas"';

// Compiles a stream from a CSDL definition:
function compileFilter(csdl) {
ds.compile({ 'csdl': csdl }, function (err, response) {
        if (err)
            console.log("Filter hash: " + response.hash);
            connect(response.hash); // Connects to DataSift

Here we're using the interaction.content property (or target) of the piece of content. This is the text of the Tumblr post in this case.

Step 4: Streaming Data

With your filter compiled you can now start streaming data. To do so you need to declare event handlers for the stream connection and kick off the connection.

Warning: When you are running a stream you are consuming your platform credit. Your free credit gives you plenty to play with, but always remember to stop your stream if you're not using to make the most of your credit.

Add the following to your script:

// Connects to DataSift and starts streaming data:
function connect(hash) {

// Set up a 'connect' event handler, which will fire when a connection is established. When connected we compile our CSDL filter and subscribe to streaming data.
ds.on('connect', function () {
        console.log('Connected to DataSift');

// Set up 'error' handler to alert us of any errors. For more details on possible errors see [](
ds.on('error', function (error) {
        console.log('ERROR: ' + error);

// Set up 'delete' handler. Depending on the data sources you are using, you may need to delete this data to stay compliant.
ds.on('delete', function (data) {
        console.log('Data deleted: ', data); // TODO: Do something useful with the data!

// Set up 'interaction' handler - this receives our data! This is triggered each time we receive a new interaction - a piece of data on the live stream.
ds.on('interaction', function (data) {
        console.log('Recieved data: ', data);

// Now all handlers are set up, connect to DataSift!


// Initiate our script by comiling the filter, which in turn will connect us to DataSift

Step 5: Give It A Whirl

With your script now complete, you can run the example and see data pouring into your console.

node [yourscript].js

There's a complete version of the script here.

Step 6: Classifying Data

To help you understand, integrate and act upon the data, DataSift allows you to add custom metadata through tags and scores. Read more about DataSift VEDO to find out more.

As a quick example, you could use tags to identify where music is being shared from. Try replacing the filter variable in your script with the following, and run the script again.

// Declare a filter in CSDL, looking for content mentioning brands, tagging each mentioned
var filter  = 'tag.brand "Calvin Klein" { interaction.content contains "Calvin Klein" } \
   tag.brand "GQ" { interaction.content contains "GQ" } \
   tag.brand "Adidas" { interaction.content contains "Adidas" } \
   return \
   { \
    interaction.content contains_any "Calvin Klein, GQ, Adidas" \

You can see the tags assigned to the data, under the interaction.tag_tree property of each item.

Learn More

That's the end of the quick start guide. To learn more about the platform please take a look at the following resources: