How to create a Real-Time Twitter Stream with SignalR

April 29^th, 2016

With Twitter as popular as ever, with today's post, I show you how to create a real-time Twitter stream with SignalR.

SignalR has become a breakthrough technology that all ASP.NET developers want to include in all of their web applications.

It definitely blurs the lines between a web application and a desktop application and makes your users sit up and take notice of a dynamic web page that shows movement instead of text and images on a static page.

I've always found SignalR a welcome addition to my tool belt for building real-time applications. I also like to include a lot of SignalR demos to show what can be done with the technology like the WebGrid editing or the real-time Like button.

So today, I wanted to get into building an asynchronous, real-time, twitter stream that runs on your web site. This specific stream is meant to grab your real-time tweets on your timeline and display them as they come in.

Setup

Initially, I went with the standard MVC application option (File -> New Project -> ASP.NET Web Application -> ASP.NET 4.5.2 Templates -> MVC)

After completing an update-package, I installed the following packages:

SignalR (install-package Microsoft.AspNet.SignalR)
TweetInvi (install-package TweetinviAPI)

After those packages were installed, we are ready to lay the groundwork for our twitter stream.

For SignalR to work, an app.MapSignalR() is required for the Startup.cs file in the root of your project as mentioned in the readme.txt when you install-package of SignalR.

Startup.cs

using Microsoft.Owin;
using Owin;
 
[assembly: OwinStartupAttribute(typeof(SignalRTwitterDemo.Startup))]
namespace SignalRTwitterDemo
{
    public partial class Startup
    {
        public void Configuration(IAppBuilder app)
        {
            app.MapSignalR();
 
            ConfigureAuth(app);
        }
    }
}

Once the line (app.MapSignalR();) was included in the Startup, we can move towards the UI.

The Home Index

The User Interface isn't anything to get excited about. I stripped off the extra DIVs and kept the menu at the top.

The screen has two buttons on it: One to activate the Twitter stream and one to turn it off.

/Views/Home/Index.cshtml

@{
    ViewBag.Title = "Home Page";
}
 
<p>&nbsp;</p>
 
<button id="firehoseOn" class="btn btn-warning">Turn on the firehose</button>
<button id="firehoseOff" class="btn btn-danger">Turn it off! Turn it off!</button>
<label id="firehoseStatus"></label>
 
<div class="tweets">
   
</div>
 
@section scripts
{
    <script src="/Scripts/jquery-2.2.3.min.js" type="text/javascript"></script>
    <script src="/Scripts/jquery.signalR-2.2.0.min.js" type="text/javascript"></script>
    <script src="/signalr/hubs" type="text/javascript"></script>
    <script src="/Scripts/twitterLive.js" type="text/javascript"></script>
}

I also added a "firehoseStatus" to show the status of whether the stream was running or not and the div.tweets will contain all of our tweets happening in real-time.

The three section scripts are standard and required for SignalR to run properly. The fourth file, twitterLive.js is our own homebrew JavaScript.

We'll get back to that in a minute. For now, let's focus on the server-side functionality.

Stream and Data

One of the issues I ran into with this particular technique was keeping track of a twitter stream and knowing how to shut it off.

Since SignalR is asynchronous and we were making asynchronous calls to Twitter, this caused a little bit of a problem.

After doing some research, I found out there was a ConcurrentDictionary class available and decided to use it to keep track of Twitter data along with Cancellation Tokens.

CancellationTokens are notifications that propagate to other operations letting it know that it should be canceled.

So we need to contain the data in a class of some kind.

Hubs\TwitterTaskData.cs

public class TwitterTaskData
{
    public string Id { get; set; }
    public string Status { get; set; }
    public IOEmbedTweet Tweet { get; set; }
    [JsonIgnore]
    public CancellationTokenSource CancelToken { get; set; }
}

The IOEmbedTweet is a bonus from TweetInvi. When the tweet is received, TweetInvi creates a Twitter card and sends the HTML as a IOEmbedTweet. Once we create it, we just send it over to the client and have it display it.

This leads us to our TwitterStream class.

Hubs\TwitterStream.cs

public static class TwitterStream
{
    private static IUserStream _stream;
 
    private static readonly IHubContext _context = GlobalHost.ConnectionManager.GetHubContext<TwitterHub>();
 
    public static async Task StartStream(CancellationToken token)
    {
        // Go to https://apps.twitter.com to get your own tokens.
        Auth.SetUserCredentials("CONSUMER_KEY", "CONSUMER_SECRET", "ACCESS_TOKEN", "ACCESS_TOKEN_SECRET");
 
        if (_stream == null)
        {
            _stream = Stream.CreateUserStream();
 
            // Other events can be used. This is just on YOUR twitter feed.
            _stream.TweetCreatedByAnyone += async (sender, args) =>
            {
                if (token.IsCancellationRequested)
                {
                    _stream.StopStream();
                    token.ThrowIfCancellationRequested();
                }
 
                // let's use the embeded tweet from tweetinvi
                var embedTweet = Tweet.GenerateOEmbedTweet(args.Tweet);
                
                await _context.Clients.All.updateTweet(embedTweet);
            };
 
            // If anything changes the state, update the UI.
            _stream.StreamPaused += async (sender, args) => { await _context.Clients.All.updateStatus("Paused."); };
            _stream.StreamResumed += async (sender, args) => { await _context.Clients.All.updateStatus("Streaming..."); };
            _stream.StreamStarted += async (sender, args) => { await _context.Clients.All.updateStatus("Started."); };
            _stream.StreamStopped += async (sender, args) => { await _context.Clients.All.updateStatus("Stopped (event)"); };
 
            await _stream.StartStreamAsync();
        }
        else
        {
            _stream.ResumeStream();
        }
 
        await _context.Clients.All.updateStatus("Started.");
    }
}

At the beginning of this method, you'll notice we have a placeholder for your credentials for Twitter. Set your credentials using the tokens from apps.twitter.com and place those in the code (No, I'm not giving you mine). ;-)

Next, we set up an event to receive a tweet when created by anyone. However, if we receive a CancellationToken, we need to stop processing immediately.

When we receive the tweet, we create an embedded Tweet and send it to the client's method, updateTweet.

We also want our users to have a good user experience so we also add the events when a Twitter stream is paused, resumed, started, or stopped.

Finally, we kick off an asynchronous stream and send a status of "Started." to the client through the updateStatus JavaScript.

Making the Hub

The hub is the most important piece and also the easiest once everything is in place.

All that's needed for this to function is the ConcurrentDictionary defined at the top and our twitter start and stop methods.

Hubs\TwitterHub.cs

[HubName("twitterHub")]
public class TwitterHub : Hub
{
    private static ConcurrentDictionary<string, TwitterTaskData> _currentTasks;
 
    private ConcurrentDictionary<string, TwitterTaskData> CurrentTasks
    {
        get { return _currentTasks ?? (_currentTasks = new ConcurrentDictionary<string, TwitterTaskData>()); }
    }
 
    public async Task StartTwitterLive()
    {
        var tokenSource = new CancellationTokenSource();
 
        var taskId = string.Format("T-{0}", Guid.NewGuid());
 
        CurrentTasks.TryAdd(taskId, new TwitterTaskData
        {
            CancelToken = tokenSource,
            Id = taskId,
            Status = "Started."
        });
 
        await Clients.Caller.setTaskId(taskId);
 
        var task = TwitterStream.StartStream(tokenSource.Token);
 
        await task;
    }
 
    public async Task StopTwitterLive(string taskId)
    {
        if (CurrentTasks.ContainsKey(taskId))
        {
            CurrentTasks[taskId].CancelToken.Cancel();
        }
 
        await Clients.Caller.updateStatus("Stopped.");
    }
}

You may be wondering why we send the taskId to the client. This is to make sure we have a matching taskId to a Twitter stream in case they want to turn it off.

Finally, the JavaScript

The SignalR JavaScript is just a collection of the server-side C# methods.

Scripts\twitterLive.js

$(function () {
 
    var twitterHub = $.connection.twitterHub;
 
    twitterHub.client.setTaskId = function (id) {
        $("#firehoseOff").attr("data-id", id);
    }
 
    twitterHub.client.updateStatus = function (status) {
        $("#firehoseStatus").html(status);
    }
 
    twitterHub.client.updateTweet = function (tweet) {
        $(tweet.HTML)
            .hide()
            .prependTo(".tweets")
            .fadeIn("slow");
    };
 
    $("#firehoseOn").on("click", function () {
        twitterHub.server.startTwitterLive();
    });
 
    $("#firehoseOff").on("click", function () {
        var id = $(this).attr("data-id");
        twitterHub.server.stopTwitterLive(id);
    });
 
    $.connection.hub.start();
 
});

As you can see, the setTaskId targets the #firehoseOff button and sets the attribute data-id to the taskId.

We also set the updateStatus by the Html function.

Remember the OEmbedTweet? The updateTweet receives that object and includes an HTML property that we turn into a jQuery object, hide it, prepend it to the div.tweets, and fade it in slowly.

The FirehoseOn button calls the C# method startTwitterLive and kicks off the Twitter stream while the FirehoseOff button grabs the taskId off it's tag and sends it over to the C# stopTwitterLive method.

Conclusion

Today, I presented a way for you to make asynchronous calls from SignalR to show a real-time Twitter stream.

You can easily change the code to create a progress bar, a file upload, or even another task that can work in the background.

While my strong suit is not threading, I was happy to get this working to share with my readers.

Did you follow this project? Would you have done it differently? Post your comments below. Always love a good discussion.