Reading Stack Overflow Information

If you’ve been programming even just for a small amount of time, the contents of Figure 1 will most likely be very, very familiar to you:

Figure 1: Stack Overflow main page

If you’re a good Internet citizen, you may actually have a Stack Overflow account, and participate in the site by answering questions and helping others learn.

If you do, you may be interested to know that the parent company, “Stack Exchange,” actually provides a very easy to use API that allows you to read just about every bit of information from your account.

In this information are things like your ranking on the site, your profile info, and questions you’ve participated in. In this post, I’m going to show you how to access that information using C#.

We are, however, going to access it using a console mode program, rather than something like a Web application. The intent here is to show you how to, not give you a ready to rock and roll application to copy and paste.

Accessing the API

The fun part about getting at Stack Overflow is that, unlike many APIs out there, there are still a lot of areas that are simple to access and don’t require any overly complex authentication schemes. (Cough: Google & Amazon… I’m looking at you) 🙂

If you head to the main Stack Exchange documentation here, you’ll find everything is very clearly marked and at a glance you can see exactly what is and isn’t available.

For this post, we need three things: user profile info, questions the user has answered, and the ability to read information on a given question.

First, however, a little warning. Even though it’s possible to use the API without authentication, you will find that the number of requests you can make in any given period is very low. While I was researching and writing this article, I noticed the access count I had available was only 300 requests.

The documentation describes in detail how to authenticate, but for this post we’re going to use the un-authenticated end points for simplicity.

Fire up Visual Studio and start a new console project.

Stack Exchange’s API, just like most APIs these days, use JSON for data exchange, and rest over HTTP for API access. Once you’ve created a new project, head to your NuGet console, search for and add “Newtonsoft JSON.NET.”

You also might want to use a library for making rest calls, but that’s up to you. For this post, I’m just going to use the standard ASP.NET WebClient class to grab the JSON data.

The first call you’re going to want to perform is to get your user info. You can get this by calling “/users” with your Stack Overflow ID.

You can get your Stack Overflow ID by logging into your Stack Overflow account, going to your onsite profile, and looking at the URL or in the search box.

Figure 2: Places where you can see your SO ID

Once you have your ID, you can make a call directly to the API using it. For example, to get my profile and pasting the following URL into a browser, https://api.stackexchange.com/2.2/users/431111?site=stackoverflow, will give you the following:

Figure 3: My Profile obtained from the Stack Exchange API

The ‘site’ parameter is required, and is the name of the site you’re querying. In my case, I have accounts on several sites in the Stack Exchange family, For now, though, we’ll concentrate just on Stack Overflow.

Where the ‘431111’ is in the URL, replace this with your own ID and you’ll get your own profile.

Now that we know how to get the data, let’s create a couple of simple routines to help us. Make sure your program.cs file looks as follows:

using System;
using System.Net;
using Newtonsoft.Json;

namespace StackOverflowinfo
{
   class Program
   {
      static string RequestWebData(string URL)
      {
         WebClient client = new WebClient();
         return client.DownloadString(URL);
      }

      static UserInfo GetUserInfo(int userId)
      {
         string URL = String.Format("http://api.stackexchange.com/
            2.2/users/{0}?site=stackoverflow", userId);
         string jsonData = RequestWebData(URL);

         var allData = JsonConvert.DeserializeObject
            <SeUserInfoBase>(jsonData);

         return allData.Items[0];
      }

      static void Main(string[] args)
      {
      }
   }
}

Next, add a class to hold your user info. We’re not going to include everything (you can if you want); all I’m going to grab is the interesting info.

using Newtonsoft.Json;

namespace StackOverflowinfo
{
   public class UserInfo
   {
      public int Reputation { get; set; }

      [JsonProperty("user_id")]
      public int UserId { get; set; }

      [JsonProperty("accept_rate")]
      public int AcceptanceRate { get; set; }

      public string Location { get; set; }

      [JsonProperty("website_url")]
      public string Website { get; set; }

      [JsonProperty("display_name")]
      public string DisplayName { get; set; }
   }
}

The strange looking ‘JsonProperty’ attribute is used to map JSON property names that use underscores to C# properties in the class because JSON.NET won’t do that automatically.

You also need to create a base stack exchange data class, one that includes an “items” array and the quota data items. If you don’t, JSON.NET won’t be able to traverse and find your user info data.

using Newtonsoft.Json;

namespace StackOverflowinfo
{
   public class SeUserInfoBase
   {
      public UserInfo[] Items { get; set; }

      [JsonProperty("has_more")]
      public bool HasMore { get; set; }

      [JsonProperty("quota_max")]
      public int QuotaMax { get; set; }

      [JsonProperty("quota_remaining")]
      public int QuotaRemaining { get; set; }

   }
}

Once you’ve created the class, add the following code to the main method of your program:

UserInfo myInfo = GetUserInfo(431111);
Console.WriteLine("Name: {0} ({1})", myInfo.DisplayName,
   myInfo.UserId);
Console.WriteLine("Reputation: {0}", myInfo.Reputation);
Console.WriteLine("Acceptance Rate: {0}%",
   myInfo.AcceptanceRate);
Console.WriteLine("Location: {0}", myInfo.Location);
Console.WriteLine("Website: {0}", myInfo.Website);

With the above code, all should be okay, but you might actually find that your call throws an exception when you try to decode the JSON. This is because the Stack Exchange API always responds using “GZip” compressed data.

If you look in our code, we’re just using the simple “downloadstring” method from the Web client class in System.Net.

At this point, you have a couple of choices. You can derive your own custom class, or you can write a much longer download string method that uses the more complicated inner classes in System.Net. For this demo, we’ll use the simpler, first method.

Add the following class to your project:

using System;
using System.Net;

namespace StackOverflowinfo
{
   class ExtendedWebClient : WebClient
   {
      protected override WebRequest GetWebRequest(Uri address)
      {
         HttpWebRequest request =
            base.GetWebRequest(address) as HttpWebRequest;
         request.AutomaticDecompression =
            DecompressionMethods.Deflate | DecompressionMethods.GZip;
         return request;
      }
   }
}

And then, in your previous code, change the “RequestWebData” method to use your new class

static string RequestWebData(string URL)
{
   ExtendedWebClient client = new ExtendedWebClient();
   return client.DownloadString(URL);
}

If everything has worked as expected, you should see something similar to Figure 4 when you run your application.

Figure 4: The output from our first API call

Once you get this far, the rest is very simple. It’s just a case of calling the correct API, and then creating the appropriate classes so that JSON.NET can deserialize the data your retrieve.

To get a list of questions you’ve answered, use the following URL: https://api.stackexchange.com/2.2/users/431111/answers?site=stackoverflow.

As with the call to get your user info, you can see that to use it we simply need to change the ‘431111’ to your own user ID to get the correct set of answers.

This time you’ll get back much more data than you did previously.

Figure 5: There’s a larger amount of data from the questions list

You’ll note that like the info, this too is wrapped in a base class, and this time the “has_more” property is “true,” which indicates that there is more data to be gotten in further pages.

I’ll not go into that, though, The documentation on the site, really, is very simple and self explanatory.

This time, instead of a “UserInfo” object, we need to create a “QuestionInfo” object, something like the following:

using Newtonsoft.Json;

namespace StackOverflowinfo
{
   public class QuestionInfo
   {
      [JsonProperty("is_accepted")]
      public bool IsAccepted { get; set; }

      public int Score { get; set; }

      [JsonProperty("answer_id")]
      public int AnswerId { get; set; }

      [JsonProperty("question_id")]
      public int QuestionId { get; set; }

   }
}

As before, we’ll also need a base object.

using Newtonsoft.Json;

namespace StackOverflowinfo
{
   public class SeQuestionInfoBase
   {
      public QuestionInfo[] Items { get; set; }

      [JsonProperty("has_more")]
      public bool HasMore { get; set; }

      [JsonProperty("quota_max")]
      public int QuotaMax { get; set; }

      [JsonProperty("quota_remaining")]
      public int QuotaRemaining { get; set; }

   }
}

We could probably do this better using something like generics because it’s only the type of the items that changes. However, for this simple example, this will suffice for now.

Add the following method to program.cs:

   static QuestionInfo[] GetUserAnswers(int userId)
   {
      string URL = String.Format("http://api.stackexchange.com/
         2.2/users/{0}/answers?site=stackoverflow", userId);
      string jsonData = RequestWebData(URL);

      var allData = JsonConvert.DeserializeObject
         <SeQuestionInfoBase>(jsonData);

      return allData.Items;
   }

Followed by this additional code to your main method:

Console.WriteLine();
Console.WriteLine("User has answered on the following
   questions:");
QuestionInfo[] answers = GetUserAnswers(431111);
foreach (QuestionInfo questionInfo in answers)
{
   Console.WriteLine(questionInfo.QuestionId);
}

If everything has worked as expected, you should see the following output:

Figure 6: The list of questions answered by a user

The last part of the puzzle is to use the question ID to retrieve the question details for each question in turn, and display the info for that question. For that, however, I’m going to leave you, dear reader, to write the code. I will, however, give you a head start.

The URL you need to get question details is as follows: https://api.stackexchange.com/2.2/questions/25693523?site=stackoverflow. The ‘25693523’ in the URL is the ID of the question whose details you wish to retrieve.

If you plug the URL into your browser, you should get something like the following:

Figure 7: The JSON output from a question detail query

As you can see, the format is the same as for the other objects. You just need to make a couple of classes and add a method to read it.