June 16, 2010

Posted by John

Tagged gems and mongomapper

Older: RailsConf 2010

Newer: Mongo Scout Plugins

MongoMapper 0.8: Goodies Galore

Let me tell you, this release has been a tough one. It is made up of 43 commits to Plucky and 92 commits to MongoMapper. Features added include a sexy query language, scopes, attr_accessible, a fancy cache key helper, a :typecast option for array/set keys, and a bajillion little improvements. Let’s run through each of them just for fun.

Sexy Query Language

This right here is all thanks to plucky. The goal for plucky is a fancy query language on top of MongoDB. It has been created in a such a way that other MongoDB projects (Mongoid, Candy, MongoDoc, etc.) can benefit from it if they wish. It still has a long way to go in covering edge cases and deeply nested queries, but the majority of queries one will do are covered quite nicely.

User.where(:age.gt => 27).sort(:age).all
User.where(:age.gt => 27).sort(:age.desc).all
User.where(:age.gt => 27).sort(:age).limit(1).all
User.where(:age.gt => 27).sort(:age).skip(1).limit(1).all

All of the above are supported out of the box. Each query method (limit, reverse, update, skip, fields, sort, where) returns a Plucky::Query object so they can be changed together until a kicker is hit, such as all, first, last, paginate, count, size, each, etc. It is fashioned in a similar form as ARel in this manner, but more simple as ARel has to handle a lot more than just simple queries (joins, etc).

Scopes

The main thing I was waiting for to do scopes was to get plucky to a point where scopes would be just a sprinkling of code to merge plucky queries. Thankfully that day has finally arrived and with this latest release, you can now scope away. The code is so compact, that I figured I would drop it in here for those that are curious:

module MongoMapper
  module Plugins
    module Scopes
      module ClassMethods
        def scope(name, scope_options={})
          scopes[name] = lambda do |*args|
            result = scope_options.is_a?(Proc) ? scope_options.call(*args) : scope_options
            result = self.query(result) if result.is_a?(Hash)
            self.query.merge(result)
          end
          singleton_class.send :define_method, name, &scopes[name]
        end

        def scopes
          read_inheritable_attribute(:scopes) || write_inheritable_attribute(:scopes, {})
        end
      end
    end
  end
end

Yep, that is it. With that bit of code, you can now do stuff like this:

class User
  include MongoMapper::Document

  # plain old vanilla scopes with fancy queries
  scope :johns,   where(:name => 'John')

  # plain old vanilla scopes with hashes
  scope :bills, :name => 'Bill'

  # dynamic scopes with parameters
  scope :by_name,  lambda { |name| where(:name => name) }
  scope :by_ages,  lambda { |low, high| where(:age.gte => low, :age.lte => high) }

  # Yep, even plain old methods work as long as they return a query
  def self.by_tag(tag)
    where(:tags => tag)
  end

  # You can even make a method that returns a scope
  def self.twenties; by_ages(20, 29) end

  key :name, String
  key :tags, Array
end

# simple scopes
pp User.johns.first
pp User.bills.first

# scope with arg
pp User.by_name('Frank').first

# scope with two args
pp User.by_ages(20, 29).all

# chaining class methods on scopes
pp User.by_ages(20, 40).by_tag('ruby').all

# scope made using method that returns scope
pp User.twenties.all

I am sure there are some edge cases, but I cannot wait to start swapping some of the code I have out for scopes. This is definitely one of the features I missed most from ActiveRecord.

attr_accessible

Previously, MongoMapper only supported attr_protected. The main reason was that I am lazy and someone from the community contributed the beginnings of the code. I spent some time today adding attr_accessible, so now you can really lock down your models if you want to.

class User
  include MongoMapper::Document

  attr_accessible :first_name, :last_name, :email

  key :first_name, String
  key :last_name, String
  key :email, String
  key :admin, Boolean, :default => false
end

Based on the example above, only first_name, last_name and email can be assigned when using mass assignment, such as in .new or #update_attributes.

Cache Key

On a recent MongoMapper project, I had to some caching. This led me to create bin, a MongoDB ActiveSupport cache store. The first thing you notice when you start to cache stuff is that you need a key to name the cached object or fragment. I dug around in AR and discovered the cache_key method. MongoMapper’s cache_key works the same with a little twist. You can pass arguments to it and they will become suffixes on the cache key. Lets look at an example:

class User
  include MongoMapper::Document
end

User.new.cache_key # => "User/new"
User.create.cache_key # => "User/:id"
User.create.cache_key(:foo, :bar) # => "User/:id/foo/bar"

It should also be noted that if the User model has an updated_at key, that will be appended after the id like so User/:id-:timestamp. This addition is definitely going to clean up some code on a project of mine.

Typecasting Array/Set values

A common idiom in MongoDB modeling is to use Array keys for many to many type relationships. You have a User model and a Site model. Sites can have many Users and Users can have many Sites. Typically, I make a key :user_ids, Array and store the ids of each user that has access to the site.

When this is done from web forms, everything comes in as a string, so you have to typecast those strings to object ids. The new :typecast option wraps this up in a single key/value.

class Site
  include MongoMapper::Document
  key :user_ids, Array, :typecast => 'ObjectId'
end

Now, whenever user_ids is assigned, each value gets typecast to an ObjectId. This will work with any class that defines the to_mongo class method, which means you can use it with custom types as well.

Conclusion

I learned more about Ruby while working on this release of MongoMapper than probably any other period in my brief history. I really feel like this release brings MongoMapper to the forefront of MongoDB/Ruby object mappers.

All the typical dressings are now in place and with a few more tweaks, I can smell 1.0. Hope you all find this stuff useful and as always, if you don’t, that is ok because I am enjoying the heck out of working on this stuff. :)

Oh, and if all of this above did not excite you, know that the new MongoMapper site, including full documentation, is well underway and should be ready for consumption soon. Hooray!

27 Comments

  1. Michael Wood Michael Wood

    Jun 16, 2010

    Awesome job! MongoMapper keeps on getting better and better. Looking forward to the future… and documentation website.

  2. jamieorc jamieorc

    Jun 16, 2010

    Excellent work, John. I’m curious about bin and some more examples of how to use it.

  3. Good Job! I’ll try in few hours.

    A question:

    Does User.all(:name => ‘John’) still work?

    Thanks

  4. This is amazing. Thank you VERY MUCH.

  5. @Michael: Thanks!

    @Adilson: Yep, fully backwords compatible in that regard. Check out the UPGRADES file for things that were changed.

    @jamieorc: I’ll update the readme file of bin soon. Just use it as you would any Rails cache store.

    @Gustavo: You are very welcome. :)

  6. That’s great news!

    Does it work on Rails 3?

  7. Amazing! thanks a lot Jhon!

    Just a quick question, is the support for multiparam arguments in rails, as they come from the date_select helper anywhere in the list of priorities? (or is it already in MM and I’m missing out?)

  8. Is mongomapper going to be working with devise again anytime? I think on the devise google group Jose posted that and orm must use:

    • ActiveModel API
    • ActiveModel::Validations for validations

    Just curious. Thanks for the great work.

  9. @John: fantastic stuff, I’ve been following HEAD and trying out each batch of awesomeness as it has appeared – looking forward to what’s next!

    @Rodrigo: I’m using it on Rails 3 without any issues.

    @Camilo: for multi-parameter attributes I’ve been using something I found on Gist a while ago, I think it was this. I’ve just packaged that up into a gem so hopefully that’s a bit easier to use until it makes it into MM proper.

  10. Ah, looks like the links got stripped!

    Gist is here: https://gist.github.com/268948/f81e3becc4767b46476dfebcecf92cec75581daf

    Gem is here: http://github.com/rlivsey/mm-multi-parameter-attributes

  11. John, timely update. I just Googled on how to typecast array values, and wondered why your instructions weren’t working until I realized, you just added them today! Thanks mate.

  12. Martijn Martijn

    Jun 17, 2010

    Thanks John, i love MongoMapper!

  13. oh yes! good job.

  14. @Brandon Martin: The AM API is already supported. The only issue is validations. I’ll be switching that before 1.0.

  15. Any plans to include accepts_nested_attributes_for ?

    http://gist.github.com/275594

  16. @Geoff: Not opposed to the idea in general, but not overwhelmed by the code in that gist. Gut reaction is it feels messy.

  17. @John: Fair enough. It’s a feature I find myself using often in projects, so it would be really nice to have!

    Thanks a lot for your hard work.

  18. Wonderful work. Thank you for all your time and effort into making MM awesome.

  19. Excellent! I love scopes, and I can’t wait to try it out :-)

  20. I love, typecast but seems that didn’t work in conjunction with :many :some, :in =>

    Ex:

    key :category_ids, Array, :typecast => ‘ObjectId’
    many :categories, :in => :category_ids

    If I try to populate some like that:


    post.category_ids = [“4c1c9ea64dd962239f00005a”, “4c1c9ea64dd962239f00005b”]

    I got:


    BSON::InvalidObjectID in ‘Post Model categories associate correctly categories’
    illegal ObjectID format


    This way help us especially in forms.


    Thanks!

  21. @DaddYE: Interesting. I’ll look into that. I think I know what it is.

  22. Michael Fairley Michael Fairley

    Jun 24, 2010

    I can’t seem to get cache_key to work on models without updated_at. Line 10 of caching.rb throws a KeyNotFound when it tries to read :updated_at.

  23. @Michael Fairley: Ah, good catch. That behavior is stupid, so I removed it. If a key does not exist, MM will now return nil, more like a hash.

  24. Kristian Mandrup Kristian Mandrup

    Jul 07, 2010

    Wow! I have been looking at all the wrong places for some good documentation of the latest Mongo Mapper API.

    Now I finally find it here and it mentions a new site with full documentation on the way!

    So I hope it will at least match the Mongoid and Data Mappers sites with regards to having enough documentation to really get started without having to spend days browsing the net for obsolete examples and having to ask too many questions on the mailing list.

    I’m so looking forward to this!!! Great work! I really want to contribute to this when I get enough knowledge and experience to be up and running… ;)
    NoSQL is the wave of the future for sure!

    Thanks again!!!

  25. holymongo holymongo

    Jul 12, 2010

    How to make many-to-many associations with mongomapper? Is this the right way?

    class Site
    key :user_ids, Array, :typecast => ‘ObjectId’
    many :users, :in => :user_ids

    class User
    key :site_ids, Array, :typecast => ‘ObjectId’
    many :sites, :in => :site_ids

  26. @holymongo: If you do that, then you are storing the relationship information in both site and user. You can do that, but if one changes you need to update the other. We just put the relationship on one side, probably users. Then you can just make a method for site that finds all users.

    def users
      User.all(:site_ids => site.id)
    end

    Hope that helps.

  27. holymongo holymongo

    Jul 12, 2010

    Thanks John!

    Storing the relations in both models is actually quite useful. Maybe MongoMapper can resolve it with a sort of dependency (when Site is deleted, remove it from User.sites)?

Sorry, comments are closed for this article to ease the burden of pruning spam.

About

Authored by John Nunemaker (Noo-neh-maker), a programmer who has fallen deeply in love with Ruby. Learn More.

Projects

Flipper
Release your software more often with fewer problems.
Flip your features.