Google App Engine + Pusher Webhooks

So I’ve started working on a project and have decided to go with Google App Engine as my backend. I must say, I have absolutely fallen in LOVE with the service! Anyways, I learned about their experimental Endpoints API, and decided to immerse myself in the wonderful documentation and absorb the API. I quickly fell in love with the simplicity of development and how powerful this service really is – It only takes a few lines of code in Javascript to use my new API!! Nevertheless, that’s for another time.

Part of my project also involves websockets, and since I am in love with Pusher, I started using the service  in my project. As I continued developing the web app (Google App Engine) and the client app (websockets), a part of the project was implementing Pusher’s new webhooks. After viewing the documentation for webhooks and the Channel Existence Event documentation, I attempted many times (over many hours) on how to develop my endpoint to handle the webhook. After many headaches, and the help of a friend, I figured it out! Perhaps, I was overlooking the simplicity, or failing to understand what was really going on, but I wanted to share my results with you in case you are in the same position I was in, especially since my copious amounts of Googling and researching returned very little helpful results. In this tutorial, I’m going to show you how to:

1. Create the Endpoints API
2. Configure Pusher to create the webhook
3. Connect to Pusher
4. Bonus: Use your new API in Javascript

Create the Endpoints API

To begin, I’m assuming you have created a Google App Engine application and have an app.yaml. Inside the app.yaml file, you need to add the endpoints api handler:

Once you have defined this, you can go ahead and actually create your Endpoints API, which should look something like this:

As you can see, I’ve created the Class and defined the webhook method which will execute when Pusher sends a request to your app. Take a look at lines 32 – 37. For an example, I actually am triggering an event to Pusher when a user connects to say hey and welcome. A couple of things: One, you obviously are going to need the Pusher class to enable this, which I found on Pusher’s Git. Two, I don’t suggest doing triggers like this since it makes a REST API call to Pusher and is bad practices. I suggest creating a task queue which processes this Trigger.

Once you have done this, go ahead and deploy to your app on GAE and check the logs to make sure everything is working correctly and no errors were found. If that’s the case, then go to: https://developers.google.com/apis-explorer/?base=https://[your-app-id].appspot.com/_ah/api and there you can view your API, which currently only consists of your pusher webhook. You can actually use this explorer to fake a pusher webhook request to make sure you get no errors.

Configure Pusher

Now it’s time to tell Pusher your webhook URL. There are many ways to find this URL. I suggest going to the explorer I just mentioned above and you can actually execute the API called and it will show you the url it POSTs to, which should be something like: https://[your-app-id].appspot.com/_ah/api/my_api/v1/pusher/webhook. Assuming you have created your app on pusher, go to the webhook page: https://app.pusherapp.com/apps/app_id/web_hooks. Under “Define a new WebHook”, add the api link, select the “Channel existence events” option and hit save.

Once you save, your new Webhook, you should be good to go! I suggest next going to the “Debug Console” tab and leaving it open so you can debug the next step easier.

Connect to Pusher

The next step is to actually connect to Pusher. This is naturally done in whatever language your application is in, but in Javascript, it looks something like this:

This will connect to pusher when executed. Once you go ahead and run the code, check out the Pusher Debug Console you left opened previously and make sure that you get the green box that says “Webhook sent”. If it’s not green, then hit the arrow on the left to see what data they are sending. Also, you can check the GAE logs to see the POST request from Pusher and hopefully get a glance of what went wrong.

Bonus: Use your new API in Javascript

So now you should be good to go connecting the Client, Pusher, and Google App Engine. The app I’m developing is partly Chrome extension, and I want access to other methods I have defined in my API. Google makes it so incredibly easy, that I’ll simply show you my code that allows me to make API calls.

First, you need to add the remote Google API:

Next, just call the load method and wait for it to load:

For a more in-depth look at using your API in Javascipt, check out the docs.

Conclusion

I hope this tutorial has made it a little easier on understanding how to use Pusher Webhooks with Google App Engine. I know for me, I wish I would have had more documentation specifically with GAE, so here it is, in my own words!

If you have any questions/comments, you can tweet me @benbeadle and I’ll respond promptly! Have a wonderful day!

Ben Beadle

For those of you who don’t know, Google Reader is an RSS Feed aggregator. You can keep up to date with your favorite websites which provide RSS feeds.

I’m currently subscribed to 36 websites. While some of those rarely have new content, according to Google Reader I have read almost 8,500 items in the past couple of days. That’s a lot of items to shift through!! As you can expect, I can get bogged down and days behind. This really unfortunate for some items which include sales of products for a single day. If I’m days late to reading that article, then I miss the opportunity for savings! This is also frustrating when sites post invite codes to new services which run out quickly because of the number of readers said site has. Doh!

This has been a frustration for a while and I’ve been trying to find ways of getting super important content to me as fast as possible. I’m not sure why I haven’t thought of it before, but using some tweaking and IFTTT, you can get important new content to you pretty fast without having to worry about getting behind in Google Reader. By adding certain subscriptions to a certain ‘watch’ folder, you can make that folder public and provide IFTTT with the RSS Feed Google provides to that folder so IFTTT can watch for certain keywords or phrases! Of course, if you subscribe to picture websites, then adding them to your ‘watch’ folder would be pointless.

This article assumes you have a Google Reader account already set up with some subscriptions and an IFTTT account.

Here’s how to get important new content texted to you:

1. In Google Reader, roll over “Subscriptions” in the Navigation pane and click on the down arrow that shows up on the right.
2. Find a subscription that you would like to add to your ‘watch’ folder and click “Change Folders” and then at the bottom hit “New Folder”. You can call this folder whatever you want, I just chose to call it ‘watch’ since it’s basically what the folder is for: IFTTT to ‘watch’ for new updates.
3. Next, add all other subscriptions you would like to said ‘watch’ folder.
4. Once you are done adding subscriptions, you need to make the ‘watch’ folder public. Click on the “Folders and Tags” tab and click the little RSS Feed icon (next to “private”) for your ‘watch’ folder.
5. Links will appear on the same line. Click the “view public page” link which will open a new tab, and then copy the “Atom Feed” link. This is the RSS Feed link you will provide IFTTT.
6. Now, checkout this recipe on iFTTT: Text me when there’s a new (public) Google Reader item matching ‘…’. Use this Recipe to monitor your RSS Feed with whatever word / phrase you would like!

Hopefully this can help some people who’s, like me, Google Reader gets backed up and you miss important posts in time!

Ben Beadle

Probably my favorite class this semester was Information Storage and Retrieval. The class taught many ways to use data and turn it into more than just data. We learned how to classify and cluster text, rank documents in searches, and recommend items by analyzing previous data. We got some great insight as to how Google works and some of the things that makes their search engine so good. Overall, it was an awesome class and James Caverlee was an awesome professor! So good, in fact, I’m taking Big Data with him in the Spring!

Anyhow, our main project at the end of the semester was to somehow use what we’ve learned in class and apply it to something real world. So my partner, Amanda Cofsky, and I set out to find the project we wanted to work on. We wanted it to be fun and actually be useful and entertaining. We had Twitter in the back of our heads and knew that we wanted to somehow incorporate Twitter in our app.

Amanda came up with the idea to use the show My Little Pony: Friendship is Magic as the basis for our show, and so we began to ponder what we could do that related to My Little Pony. After pondering for a little bit, I figured it out. I know there were quizzes that helped you find what pony you are most similar to, but no apps that automated the process by analyzing your tweets. That, my friends, is how Pony Placement Project was born! This nifty app automatically finds what pony you are most like by analyzing your tweets!

The application, which is Ruby on Rails, uses Cosine Similarity and kNN Classification to compare your tweets to the transcripts of My Little Pony. For every tweet, it compares the text to over 7,000 lines of transcripts to find the kNN closest lines. It then adds up the similarity values and finds which pony you are most like! If you would like, you can view the source code over on Github.

So now, all that’s left is for you to find your inner pony via Pony Placement Project!

- Ben Beadle

So when it comes to my iTunes songs, I get pretty OCD and want every song to have lyrics. This way I can sing along to songs I don’t know the words to while also helping me learn the words!

Unfortunately, I haven’t found a viable application to automatically download lyrics that integrates with iTunes. I decided to develop a Chrome Extension to help simplify copying lyrics to the clipboard in one click. Enter LyricMe.

LyricMe is a Chrome Extension that allows you to search only supported sites and then allows you to either view the lyrics in a simple interface, or copy the lyrics to your clipboard in one click.

To learn more about LyricMe, check out the homepage.

I hope you find it useful!

- Ben Beadle