New Release of the Query Builder

Ed Stenson's picture
Ed Stenson
Updated on Thursday, 25 September, 2014 - 12:06

Have you tried our Query Builder yet? It's a visual tool that makes it easy for newcomers to get started with DataSift quickly, before they even begin to learn our query language, CSDL. Despite its simplicity, the Query Builder offers very nearly all the features on offer in the full language. It includes every CSDL operator and logical operator, together with very nearly all the targets and augmentations. Recently we added the ability to use parentheses and, with the latest release, we've added the NOT logical operator.

Let me give you an example. In CSDL, you can write a filter that includes one keyword and excludes another like this:

 

To date, it has not been possible to perform logical inversion using the Query Builder but now you can do it with a single click:

Adding a NOT in the Query Builder

To create rules that use NOT:

  1. Click Create New Filter.
     
  2. Choose IMDb -> Title.
  3. Choose Contains words as your operator.
     
  4. Type "Star" as the filter keyword.
     
  5. Click Save.
     
  6. Click Create New Filter again and build a second rule that looks for "Trek" in the IMDb title.
     
  7. Click Save and your Query Builder screen will show the two rules like this:


    The will filter for titles that include "Trek" AND "Star". We need to adjust it to filter for "Trek" AND NOT "Star".
     
  8. Click Advanced.

    Notice how the two rules now have numbers?


     
  9. Click NOT.



    The Query Builder adds the NOT in front of rule 1. This is exactly what we want, because rule 1 filters for "Star". If you wanted to apply the NOT to the rule for "Trek" you could drag and drop it in front of the "2".

  10. Click Save and Preview. The Query Builder saves your work and automatically generates code.

 

// JCSDL_MASTER 4bce1d2f67166ea38e0875cc79750c85 !1&2

// JCSDL_VERSION 2.1

NOT

// JCSDL_START 8a1624733bb708222fab239bcb5d8aaf imdb.title,contains_any,25-4 1

imdb.title contains_any "Star"

// JCSDL_END

AND

// JCSDL_START ea352777ca8a4b1e80c3f4cb60e22dfc imdb.title,contains_any,25-4 2

imdb.title contains_any "Trek"

// JCSDL_END

// JCSDL_MASTER_END

 

Several of the lines are commented out because they contain internal information for the Query Builder itself.  If we remove those lines, we're left with the original CSDL that I included at the beginning of this blog.

 

NOT imdb.title contains_any "Star"

AND

imdb.title contains_any "Trek"

 

Using NOT with parentheses

Let's look at an example that uses parentheses as well as NOT. Suppose you wanted to filter for blogs about mention the NASDAQ, but exclude mentions of Microsoft and Intel, two of the tech-laden index's most well-known components:

  1. Click Create New Filter.
     
  2. Choose Blog -> Content.
     
  3. Choose Contains words as the operator.
     
  4. Add "NASDAQ" as the filter keyword.
     
  5. Click Save.
     
  6. Click Create New Filter again and build a second rule that looks for "Microsoft" in the blog content.
     
  7. Finally, build one more rule that looks for "Intel" in the blog content.



    Now we need to add logical operators, parentheses, and a NOT operator.
     
  8. Click Advanced.
     
  9. Select the parentheses.



    By default, the Query Builder places the parentheses around your entire query definition.


     
  10. Drag the left parenthesis so that the parentheses surround rules 2 and 3.
     
  11. Click the second AND operator and it will change to OR.


     
  12. We need to make one final modification to make the query exclude Microsoft and Intel. Click NOT and drag it so that it operates on the clause defined within the parentheses.


     
  13. Click Save and Preview. The Query Builder saves your work and automatically generates code.

 

// JCSDL_MASTER c24e74b9aa6c3aed7c09d36aae51b661 1&!(2|3)

// JCSDL_VERSION 2.1

// JCSDL_START cfc90f784c8274788a2bb3984ba42ee1 blog.content,contains_any,27-6 1

blog.content contains_any "NASDAQ"

// JCSDL_END

AND  NOT (

// JCSDL_START 13a0d4a23c57744568df3c58dde08ec4 blog.content,contains_any,27-9 2

blog.content contains_any "Microsoft"

// JCSDL_END

OR

// JCSDL_START 18b98e1b4d1b7c312d6ac8827be9350c blog.content,contains_any,27-5 3

blog.content contains_any "Intel"

// JCSDL_END

)

// JCSDL_MASTER_END

 

If we strip out the comments, the code becomes easy to read:

 

blog.content contains_any "NASDAQ"

AND  NOT (

blog.content contains_any "Microsoft"

OR

blog.content contains_any "Intel"

)

 

Summary

To try this new functionality out, you'll need to grab a copy of the Query Builder. It's Open Source and you can find it on Github. We provide full documentation to show you how to embed the Query Builder on your own pages.