New Release of the Query Builder

Have you tried our Query Builder yet? It's a visual tool that makes it easy for newcomers to get started with DataSift quickly, before they even begin to learn our query language, CSDL. Despite its simplicity, the Query Builder offers very nearly all the features on offer in the full language. It includes every CSDL operator and logical operator, together with very nearly all the targets and augmentations. Recently we added the ability to use...

Read New Release of the Query Builderimageimage

Historics - We Keep Making It Better

A year ago, Datasift released Historics, a product that enabled users to access content from the past. Its demand has grown massively over the past year. We have had to make many optimizations in order to keep up with not just the demand, but the scale of our ever-growing archive. Our Historics archive is very close to one petabyte in size now and we are adding about two terabytes to it each day. We run over 2,000 Hadoop jobs every month that scan over a total of nine trillion records...

Read Historics - We Keep Making It Betterimageimage

Writing CSDL in Vim

Like any developer-friendly company, DataSift too has fans of the good old Vim editor working for us and with us. And since we spend so much time inside Vim, it is no wonder that we use it to write CSDL too. Which is why today I'm especially happy to announce that CSDL syntax highlighiting has been added to the Vim source code repository and should be shipping with all major operating systems worth using soon. (An OS worth using is the one that ships with Vim, of course.) In the...

Read Writing CSDL in Vimimageimage

Split-Second Social Media Analysis with DataSift and Redis

Social media gives us a way to sample trends and sentiment in real time. Consequently, it is very important that the analysis of the data we are looking at also happens in real time. And we want to help you, because here at DataSift we want our platform to be the Swiss Army knife of the social media analysis tools. We try to be flexible and do as much of the hard work as possible so that you can focus on analyzing the data instead of having to think how to feed it into your processing pipeline...

Read Split-Second Social Media Analysis with DataSift and Redisimageimage

The Query Builder

At DataSift we love open source. We use it and we create it. As part of our commitment, we're proud to announce that a major new component of the DataSift platform, the Query Builder, is now available. It's open source and you can download it from GitHub today. Take a look at our demo page to try out the Query Builder. What is the Query Builder? Everyone talks about Big Data, but not many people know how to handle it. We live it. We created the Query Builder...

Read The Query Builderimageimage