Creating a foursquare connected app overnight: #mom code review.

We built and launched the connected apps version of #mom in one long night a few weeks ago and we’d like to walk through some of our code. Foursquare’s connected apps platform is just a few months old and still in developer preview so we wanted to help other developers get started.

Foursquare allows 3rd party apps to (1) receive a push feed of check-ins, (2) reply to check-ins with text and a link to a custom web view or your native app, and (3) post to the activity feed when an user connects or interacts with your service. For #mom, this is exciting because we can suggest that you call your mom after a check-in rather than hope you remember to include “#mom” in the message (aka a shout in foursquare data model land). There are multiple places to provide users value and delight through a backend action, simple prose, and links to your native app or web UI.

We’re going to walk through replying to check-ins, controlling your service from inside foursquare, and creating posts in the activity feed. Our goal was to build this in one night (10pm to sunrise). We’re assuming you’re already connecting users over OAuth, receiving real-time push check-ins, and familiar with foursquare’s data model as we described in our prior technical post.

First, we need to choose if we want to respond to a given check-in (we don’t want to spam!). Foursquare has an extensive venue categorization taxonomy, from broad levels like parks and recreation to granular places like volcanoes; yes, we have users checking in atop hot lava. You can browse the venue categories online, though we queried the JSON API and dumped to Google Docs for easier person and machine-readability.

We spent much of the night crafting close to 60 quirky messages based on the venue’s category. Connect the app and see if you discover them all! 

Foursquare orders each Reply based on response time so we need to be speedy. Replies contain a customizable short message with an optional click through link and are displayed asynchronously in the user’s post check-in screen. We noticed this can appear buggy as new replies are rendered as they come in.  

We use the click through link as the trigger to send a call or a text depending on our user’s preference.  This was tricky because we don’t receive any information from foursquare when the user clicks the link; we only control the URL.  Each link therefore needs to be unique per check-in so that we can look up in our database who to call or text and what to say.  The links also need to be unguessable, so you can’t go to hashtagmom.com/easy-to-guess and call your friend’s mom. We accomplish this by storing a distinct random string for every check-in, generated by Ruby’s SecureRandom library. We ensure each token is unique by, elegantly, doing mighty brute force— continuously generating tokens until we find one unused. We could improve this by pre-computing a list of tokens or using some kind of UUID if it were a performance bottleneck, but we hit zero conflicts in our testing of 5M tokens and moved on.

Once we generate a link and text, we send a reply to the API endpoint at https://api.foursquare.com/v2/checkins/checkin_id/reply.  Composing the request is easy, but we ran into a roadblock getting foursquare to accept it. Unless otherwise instructed, foursquare defaults to their oldest API, so you need to explicitly instruct it to route your request to the correct endpoint version.  To use the latest API, send a { :v => date } parameter along with your request, where `date` is the most recent date when your app worked with the newest API.  We developed this on August 17th, so we use “20120817”.  Having to provide a version date along with  “/v2/” in the URL is confusing. All set with constructing the reply? Don’t forget to use https:

After a successful call or text, we wanted to create a Post for that check-in, which looks like an auto-generated comment in the activity feed on both mobile and web. This is much simpler than sending replies and can include custom text and a custom link. There is a lot of new jargon in terms of (pun!) ways to communicate with the user but at least the Post syntax is similar to a Reply:

A brief aside— you’re required to provide a privacy and terms of service statement to be officially blessed into their directory. We used Docracy and branched their open privacy statement and adopted it for our use (feel free to branch ours too!).

We’re looking forward to the evolution of the platform and hope to see a full fledged marketplace discovery section within the application, clearer policy about how relevant connected apps are chosen and ordered in the check-in view, and more clever experiences that provide value and do not tire after a few uses. Foursquare uniquely has the attention and context of people when they are at places and we’re excited to see how others bring the joys, efficiency and serendipity of the internet to us when we most and least expect it.

To get started, review their technical documentation and guidelines which gives details and a sense of the spirit they’d like apps to embrace but it’s still very opened ended. If you have questions, please tweet or say hello at the bicoastal foursquare hackathon on Nov 3rd; @jeff_weinstein in SF and @scpike in NYC.

Foursquare API, and SSL, and OAuth. Oh, my!

Continuing our deeper dive into the nuts and bolts of how we built #mom, let’s walk through how we used the foursquare API. We we wanted the foursquare platform to: (1) handle user authentication, (2) supply general user data, (3) retrieve user checkins, and (4) promote the service by displaying #mom in the friend checkin feed.

Much like Twilio, our first step was to register our application with foursquare. They call this process “register a new consumer.” It made sense, after a while, since as a developer we are “consuming” foursquare data. Foursquare also asks you for a website and callback URL. We hadn’t settled on the hashtagmom.com domain yet (my original idea was theresafe.ly) but luckily it doesn’t matter what you put into this field.  The callback URL is more important and a not well documented. We had issues with invalid redirect uri errors when specifying the full callback path (http://www.hashtagmom.com/path/to/callback) for our app. If instead you just give foursquare the hostname (http://www.hashtagmom.com/) they will allow you to specify any path under that host in your OAuth2 request, so that seems like the way to go.  The foursquare API was also easier to work with locally than Twilio, as OAuth2 uses a browser redirect that works to localhost, so you don’t need a localtunnel set up. Please read our previous Twilio post if this distinction is unclear.

With the goal of connecting to foursquare and retrieving data from a simple standalone ruby program, I Googled “foursquare gem.” Unfortunately the first result, in my version of Google, is an out of date gem. I futzed with that until I stumbled upon a more recent gem called, wait for it, foursquare2 gem. Foursquare’s libraries page points to the latest gem so it’s always a good tip to just follow their official links rather than my brain-ingrained-let’s-Google-it approach. There is also a foursquare-api gem but I haven’t used it.

Ok, so we were able to query foursquare and retrieve data from endpoints that only required “userless” access permission. These are endpoints like get information about venues, tips, and lists. To access checkin details, user data, or many of the other interesting bits available in the platform, you need to act “as” the user— you need a user to authenticate your application with foursquare and explicitly get permission to query the platform as if you were that user.

The following is a  bit of an aside and not specifically related to any particular platform but necessary to understand. You’ve likely connected to Facebook before. You’re on SomeGreatSite.com, they have a connect to Facebook button, you click it, and now you’re redirected to a Facebook page that read, “Hi, would you like SomeGreatSite to have access to x, y, z Facebook permissions?” Yup, hit yes and you’re redirect back to SomeGreatSite.com. This flow is brought to you by OAuth2: check out this technical description and foursquare’s overview. From the developer’s perspective: you a user on your domain and want to query an API on that user’s behalf. Rather than ask for andstore a user name and password, you direct the user to the service and wait to hear back if you have permission or not. If you do get permission, you get a special token that is specific to that user. (Remember going to the arcade as a kid and getting special tokens that only fit into the video game machines?) Now, you can query the other service with this token and act as the user. The developer does not have to see or store the user’s authentication details. If the user wants to revoke your site’s access to the platform, they can just tell the platform to stop accepting that token. Change of password, no problem. Badaboom. 

We’ve shipped the user off to foursquare but does foursquare know how to return the user back to us? How do we know which user they’ve sent back to us? The callback URL! You inform foursquare which application you are with your client ID and secret on the redirect call  and foursquare calls your application back at your predefined URL. We can test this locally since a localhost callback URL will work. Just set up a route in Rails that handles the URL call: in our case this was /foursquare_callback and we handle this call in our Users controller. Foursquare returns the user’s access token in the callback URL which you can parse from the params hash. The next problem: which user are they referring to? We had a few options: (1) we thought about passing some user ID to foursquare and then parsing it back out of the callback but we’re not sure that would work (not to mention it seemed like a bad idea to pass a 3rd party an app specific user specific ID), (2) matching some foursquare user data against our user information but we already removed email address from the sign up process (and this also felt wrong at a gut level), (3) storing an ID in the browser cookie for that user’s session.

Rails easily allows you to store a session ID in the user’s browser’s cookie and access per-session data in your controllers. We bank on the fact that the callback URL will occur within the same session. Rails queries the user’s session from the cookie and then we look up which user had that session ID. We also store the OAuth2 access token (a string) in the user object and use it anytime we need to query foursquare on the user’s behalf.


It’s a windy road to get started with user authentication. You have to play by the platform’s rules. Once you have an access token, you’re ready to query!

We want the foursquare checkins as they happen. When someone includes “#mom” in his or her shout (foursquare’s field name for the 140 character message per checkin), we want to kick off a process that parses the message, looks up the user’s settings for call or text and tells Twilio to go. Foursquare has two ways to get checkins for a user: pull and real-time push. During the development process, we used the pull API to access the latest checkins and manually scheduled when to refresh this data. We’d briefly store the checkins then cycle through them looking for “#mom” and call the next step. This helped us manually test the service and we could even generate fake data within our database saving us an embarrassing number of #mom public checkins (though I generated like 20 of them in production the night we got it all working). We figured we could get push working later and guarantee that our data flow worked without the need for push. Push also requires SSL which I’m not aware you can do in local testing. 
Handling both push and pull calls from foursquare was made simple by abstracting checkin processing into a Checkin model. This model knows how to send out calls or texts based on the information contained in a checkin, and doesn’t care how that checkin got created.

Pulling data from foursquare is more straightforward than receiving pushes.  One tricky part is that foursquare doesn’t take care of de-duplicating the checkins they send you, so you have to make sure not to process the same checkin twice. Here’s our pull code, which fits nicely in the User model:



With this code, we can dump a User’s latest unprocessed checkins into our database, where we can manually process. We considered using the delayed job framework to pull and process checkins in the background, but eventually decided to take a shot at hooking into the push API.  

For push, foursquare posts to your servers whenever an authorized user has checked in to a venue.   The complicating issue that kept us from starting with push is similar to the problem we ran into with Twilio— it’s a pain to use these webhook APIs from a local development environment.  Foursquare’s API was even more complicated in this regard, as they only allow pushing of data over SSL. This ruled out the localtunnel solution from Twilio, and we never succeeded in pushing data to our local servers. It would be awesome if foursquare could allow non-SSL connections for testing (maybe restricting to only getting your personal checkins pushed). We would also be happy with a solution similar to what Stripe does with their Webhooks framework, where they send you a non-sensitive event ID that you can then use to make a secure request to their API from your server.