Google Analytics Fun

Doesn’t it stink that google analytics doesn’t have an API? Yeah, I know. It seems nearly every post I mention conductor nowadays so I’ll bring it up again. We have about 15 sites live in conductor and darn it if I’m not interested in how they are doing traffic-wise.

The Original Sin

What I thought would be handy, is one simple dashboard where I could see all the pageviews, visits, etc. for each of the live sites in conductor, both separately and collectively. Naturally, I went searching for an analytics API and found nothing. I did a bit more digging and found that there were a few rogue hackers out there who had some form of an API working. It intrigued me so I started to check out the analytics source code. You know all those fancy charts and such? Yep, they’re powered by XML. Woohoo!

Log Me In Scotty

Now, the only thing between my glorious dashboard and me was a dainty browser session. For this, I reverse engineered Google’s login, discovered their cookie monster and created an authentication base class. This made it drop dead easy to make authenticated requests to google for data (I’m also using it in a GReader wrapper I’m working on ).

If you want to make an authenticated request to google it’s as simple as gem install googlebase and the following:

require 'google/base'
Google::Base.establish_connection('username', 'password')
Google::Base.get('http://www.google.com/reader/api/0/token')

Statwhore

Ding, ding, ding. Dainty browser session down, it was now time to start wrapping up those xml responses into ruby objects that are more fun to play with. It’s pretty much just a lame start, but it is aptly named Statwhore and it’s on a github near you. For now it just has really basic stuff, like pageviews and visits for a given time period. Take the irb session below for example:

>> require 'statwhore'
=> true
>> Google::Base.establish_connection('username@gmail.com', 'tehs3cr3t')
=> #<Google::Base:0x199ae84 @password="tehs3cr3t", @email="username@gmail.com", @sid="areallylonghashofkeysthatyoucantsee">
>> Statwhore::Google::Analytics::Account.find_all
=> [#<Statwhore::Google::Analytics::Account:0x1943300 @account_id=85301, @name="addictedtonew.com">, #<Statwhore::Google::Analytics::Account:0x1943058 @account_id=344381, @name="webgroup.nd.edu">]
>> Statwhore::Google::Analytics::Profile.find_all(85301)
=> [#<Statwhore::Google::Analytics::Profile:0x12a0304 @account_id=85301, @name="addictedtonew.com", @profile_id=47912>, #<Statwhore::Google::Analytics::Profile:0x129edd8 @account_id=85301, @name="googlebase.rubyforge.org", @profile_id=5867643>, #<Statwhore::Google::Analytics::Profile:0x129e70c @account_id=85301, @name="googlereader.rubyforge.org", @profile_id=5868472>,  #<Statwhore::Google::Analytics::Profile:0x129cde4 @account_id=85301, @name="lorem.rubyforge.org", @profile_id=3027809>, #<Statwhore::Google::Analytics::Profile:0x129a080 @account_id=85301, @name="mirrored.rubyforge.org", @profile_id=5211778>, #<Statwhore::Google::Analytics::Profile:0x12999c8 @account_id=85301, @name="railstips.org", @profile_id=1897579>, #<Statwhore::Google::Analytics::Profile:0x12995b8 @account_id=85301, @name="scrobbler.rubyforge.org", @profile_id=3293580>, #<Statwhore::Google::Analytics::Profile:0x12991bc @account_id=85301, @name="snitch.rubyforge.org", @profile_id=2800544>, #<Statwhore::Google::Analytics::Profile:0x12989c4 @account_id=85301, @name="twitter.rubyforge.org", @profile_id=2865122>]
>> profile = Statwhore::Google::Analytics::Profile.find(85301, 1897579)
=> #<Statwhore::Google::Analytics::Profile:0x1c3af18 @account_id=85301, @name="railstips.org", @profile_id=1897579>
>> profile.pageviews 
=> 1743
>> profile.pageviews(:from => Time.mktime(2008, 3, 1), :to => Time.mktime(2008, 3, 10))
=> 2416
>> profile.visits(:from => Time.mktime(2008, 3, 1), :to => Time.mktime(2008, 3, 10))
=> 1862

Cool stuff, eh? So I didn’t write this post to say go use this in production for a paying client. I just thought it was fun playing around and thought I would put it out there with some explanation on the off chance other people might feel like helping the project along.

The Future

Eventually, I see Statwhore wrapping a crap load of the analytics api’s that I’m sure are out there. Feedburner has an awareness api and a feed management api. Rest assured I’ll be wrapping those up eventually, as I have a dream of only Feedburner urls being spewed from Conductor’s news section. I guess my question is, what are some other good analytics services that also have api’s?

Testing Live Web Services

Also, if you are working on a web service gem and are wondering what is the best way to test it, I don’t have the answer. :) I have tried a few different ways. My recommendation is to mock or stub in some way the response and then let your class go to town. Below are a few different ways I have done it on previous gems.

1. Live. Don’t do this as your only tests. It sucks. Good idea to have around to see if API changes though. I now horribly regret the tests I put in for the twitter gem. They are horribly fragile.
2. Mock the class that does the outside connections and instead return local fixture files of some sort.
3. Do the same thing as the previous one but use the FakeWeb gem
4. And most recently, use rSpec to stub the fetching of the response with a local file.

Anyway…it’s late and I’m heading to bed. Just felt like getting this post out before the project died a horrible open source death.