<p>Imagine you could get all tweets in realtime.</p>
<p>Imagine you need to count them. Imagine you need to filter them by keywords. Imagine you need to answer complex questions about them in realtime. For example, how many tweets from women, containing the word <code>clojure</code> expressing a positive sentiment during the last hour. Imagine the same question about the last year.</p>
<p>How would you do it?</p>
<p>First you’ll need to absorb tweets in realtime. The twitter streaming API is here for that. But you are limited to 1% of all twitter volume. If you want not to be limited, you need either deal directly with twitter or use GNIP.</p>
<p>Next, you’ll need to keep only tweet of interest. By example, you’ll need to filter by keyword.</p>
<p>Just after that, you need to add informations for each received tweet. You need to enrich them by information it doesn’t already possess. For example, the gender of the author of a tweet must be guessed. The same for the sentiment expressed by the tweet.</p>
<p>In the end you’ll need to display all these informations in real-time. By real-time we mean with a very low latency.</p>
<p>Under the minute is generally acceptable. But under the hood, we generally have less than half a second of latency.</p>
<p>Most of the lantency is due to twitter (about 2s) or gnip (about 15s).</p>
