RubyGarden

09 May 05

The Past, Present, and Future of RubyGems

part 1

by Chad Fowler

note: This is the original English version of a multipart article written for The Rubyist.

History

In year 2000, when I started using Ruby, one of the first discussions I remember on the English ruby-talk mailing list was about whether or not Ruby had some kind of equivalent to Perl’s CPAN. Credit for the first (English) mention of Ruby’s CPAN goes to Sun Microsystems’ Manpreet Singh, in early 1999:

        Subject: [ruby-talk:00247] Ruby Application Arvhive
        From: Manpreet Singh <manpreet.singh sun.com>
        Date: Wed, 10 Mar 1999 14:07:15 -0800
        Hi,
        Does anybody know if there is a plan to create a CPAN (perl) like
        archive for ruby modules and having a

        perl -MCPAN -e shell

        like facility?

        Thanks
        Manpreet

Since then, it’s been a frequent topic of discussion—especially when the subject of Ruby advocacy comes up. It usually goes something like this:

"Why isn’t Ruby popular as #{language}?"

                                "Not enough libraries."

"There are plenty of libraries. Which are you missing?"

                                "Well, I guess there are enough libraries.
                             But we can't find them.  And, they're hard
                                to install!"

"We need to create something like CPAN for Ruby! Only better! Let’s form a committee."

*time passes*

"Why isn’t Ruby as popular as #{language}?"


I don’t mean to trivialize these complaints. Things really were difficult back then. We had some rudimentary installers, but you couldn’t rely on things being laid out in a familiar way when you tried to install Ruby software. And, you’d end up in dependency hell trying to follow the recursive path of one dependency to the next. All this led to a frustrating experience—especially for new Ruby users.

In the summer of 2001, two interesting changes took place in the world of Ruby packaging. First, Minero Aoki released version 1.0.0 of his setup.rb. setup.rb, still very popular today, solves part of the Ruby packging problem. It wasn’t quite the solution that the "where is CPAN?" people were looking for, but it provided a much-needed defacto standardization of Ruby installers, leading to a uniform methodology for laying out package structures.

Second, Ryan Leavengood started work on a project he called "RubyGems". RubyGems was first introduced at the First International Ruby Conference in October, 2001. Like Java’s jar files, it created single-file archives out of Ruby libraries, and it hacked ‘require’ to enable users to load code from these files. Matz was in attendance, and took a definite interest in RubyGems, offering to give Ryan commit access to the Ruby CVS repository so that he could add RubyGems to the Ruby distribution. Matz was especially interested in the ability to "execute" one of these archives, for example:

        $ ruby --gem=myprogram.gem

Fueled by excitement over Ryan’s software and the continued desire to "create Ruby’s CPAN", the RubyConf attendees held a session to collaboratively design the features of what we were then calling RAA.succ. Like most committee-based efforts, this version of RAA.succ never saw the light of day, but the ideas that were captured there definitely continued to linger in the minds of the attendees after the conference. Particularly relevant today is the idea of "smart" packages whose attached metadata fed the central repository.

After RubyConf, when the excitement had settled, Ryan released RubyGems 0.4.0 on SourceForge, marking the end of his work on RubyGems.

2002 saw a flurry of activity, including rpkg, installpkg, and (most notably) RAA-Install. RAA-Install was a really ambitious attempt to auto-install packages based on their RAA metadata. It garnered a hefty bank of supporters, resulting in a wiki-based petition to get it included in the Ruby distribution.

Despite the efforts of many dedicated Rubyists, 2002 came and went and we still didn’t have a solid, ubiquitous package management solution in place for the Ruby community. And, the "Where is CPAN?" question continued to tumble around in its seemingly never ending cycle.

In fact, 2003 came and almost went, and things remained the same. It was in late November, at the 2003 International Ruby Conference, that David Black suggested during a roundtable that Matz, our benevolent dictator, become a "little less benevolent and a little more dictator" on the topic.

So, Matz said, in typical Matz style (paraphrase), "someone build it, and I’ll put it in the Ruby distribution". So, Rich Kilmer immediately (as in _right that moment_) pulled a few of us together, and we setup in the hotel bar and built it. After a long night of code and caffeine, we had the basic skeleton of our reincarnation of RubyGems working and remotely installing libraries and their dependencies. Though we chose to take Ryan’s original name (he gave us permission), the similarities between RubyGems of 2001 and RubyGems of 2003 exist only in their shared purpose: to simplify and standardize the way that Ruby code is distributed.

Nuts and Bolts

The most interesting (and controversial) feature of our initial RubyGems release was that we allowed it to manage multiple, simultaneously installed versions of the same library. This means, for example, that you can have RedCloth 2.0.3, 3.0.0, and 3.0.3 installed all at the same time, and you can choose on a per script basis which version to use. In fact, you can use comparison operators to specify version by criteria. As an example, if you have a library that works only with RedCloth 3 or above, you could use the following in your code:

        require_gem 'RedCloth', '>= 3.0.0'

This will properly add the Redcloth 3.0.0 distribution to your $LOAD_PATH and will automatically require any files that the gem’s author specified in the gem’s autorequire attribute of its metadata.

The version operators can also be used at install time:

        $ gem install redcloth --version ">= 3.0.0"

In either case, you can leave off the version argument, and the most recent version will be used.

What makes this controversial is that the way we chose to implement this feature was to put all of RubyGems’ stuff into its own directory structure. We did it in the interest of being polite. It also makes it easier to safely clean things up if we don’t have to worry about removing files that were not installed by RubyGems.

To be continued

Stay tuned for another installment in this 3 part series. We will continue with the current state of RubyGems. Since its release, it has been downloaded over 40,000 times and over 300,000 libraries and applications have been installed remotely via RubyGems. We will examine the evolutionary steps that have been made since its first release and will conclude with a look at RubyGems 1.0 and beyond.

02 Mar 05

Og (ObjectGraph)

Teaching Ruby objects how to persist

by George Moschovitis

Introduction

Ruby is a wonderful object oriented language featuring a well designed syntax and advanced constructs to bring the joy back to programming. Creating the object model to describe your problem domain is easy, but making this model persistent is another story: you have to deal with relational databases and the SQL language.

RDBMS systems are a proven and robust technology for storing and querying data, but after experiencing the wonders of Ruby, it is hard not to wish for a better way to integrate the OOP and Relational paradigms.

Og makes your dream come true! Og stands for ObjectGraph and provides a transparent way to make your objects persistent while leveraging the full querying power of an RDBMS system. In fact, Og is designed to use an RDBMS system like MySQL or PostgreSQL to implement the actual data store where the objects are serialized.

But, enough with the techno-babble, let’s walk through a simple example to give you a better idea of what Og can do.

Installing Og

The best way to install Og is through RubyGems. For example:

gem install og

In order to use Og with a specific RDBMS, you have to install the corresponding Ruby binding. A list of supported RDBMS’s and information about the Ruby bindings can be found in the README file.

Alternatively, you can install a .tar.gz or .zip distribution. You can find these at the following URL:

www.rubyforge.com/projects/nitro

A Basic Blog Model

Blogs are in vogue. It seems that almost everyone is running a blog, and many try to code one from scratch. We’ll review the steps necessary to generate the persistence model for a blog application using Og.

Let’s start by designing the objects we’ll use. Our simple Blog will use these three objects:

  # Blog category

  class Category
      attr_accessor :name
  end

  # Blog posting

  class Post
      attr_accessor :title
      attr_accessor :body
      attr_accessor :author
  end

  # Blog comment

  class Comment
      attr_accessor :title
      attr_accessor :body
      attr_accessor :author
  end

As you can see, this is pure Ruby code. One of the features of Ruby is dynamic typing. When defining the attributes of our objects, we don’t declare the actual type. However, in order to persist the model in SQL, we need to provide some hints to Og.

Og provides a replacement to the attr* family of methods to facilitate attaching metadata to the object’s attributes. An attribute that contains metadata is called a property. For each attr* method, there is a corresponding prop* method. That is,

    attr          => prop
    attr_accessor => prop_accessor
    attr_reader   => prop_reader
    attr_writer   => prop_writer

Here are the class definitions using the property mechanism:

  require 'og'

  class Category
      prop_accessor :name, String
  end

  class Post
      prop_accessor :title, String
      prop_accessor :body, String
      prop_accessor :author, String
      prop_accessor :create_time, Time
      prop_accessor :hits, Fixnum
  end

  class Comment
      prop_accessor :title, String
      prop_accessor :body, String
      prop_accessor :author, String
      prop_accessor :create_time, Time
  end

Notice that the prop_accessor works similar to Ruby’s attr_accessor. Here are some examples:

  prop :title, true, String
  prop_reader :title, :body, :author, String

To make the definitions look even cleaner, Og provides the property alias:

  class Category
    property :name, String
  end

  class Post
    property :title, String
    property :body, String
    property :author, String
    property :create_time, Time
    property :hits, Fixnum
  end

  class Comment
    property :title, String
    property :body, String
    property :author, String
    property :create_time, Time
  end

This is most of the information that Og needs to manage these objects. Before we continue, we need to setup the actual RDBMS data store used by Og. Currently, Og has built-in adapters for PostgreSQL, MySQL, SQLite3, and Oracle. For this example, we’ll use the PostgreSQL adapter, so add this code after the class definitions.

  db = Og::Database.new(
    :database => 'test',
    :adapter  => 'psql',
    :user     => 'postgres',
    :password => 'navelrulez'
  )

Now you are ready to save your first object into Postgres. Add the following code:

  # create the object
  p = Post.new
  p.title = 'Hello'
  p.body = 'World'
  p.author = 'tml'

  # save the object in the database
  p.save

That’s it! Og works behind the scenes doing all the work for you. This simple command, p.save, does the following:

  1. Creates the database ‘test’ if it doesn’t exist.
  2. Creates a table to store Post objects if it doesn’t exist. The table’s columns map to the object properties.
  3. Creates SQL indices.
  4. Creates any needed sequences.
  5. Serializes the object into the table.

Issue the following SQL to see the result:

  SELECT * FROM og_post

This is nice, but where does the #save method come from? Og uses Ruby’s advanced introspection features to automatically ‘enchant’ class that define properties. An enchanted class provides several methods that will be discussed in the following text. These enchanted classes are called managed classes.

Before going on, let’s look at another Og macro that eases object creation:

  p = Post.create

Create automatically calls the save method. Here is another way to save the object:

  db << p

OR

  db.save(p)

Let’s create a Category object.

  cat = Category.new
  cat.name = 'Programming'
  cat.save

If you investigate the generated og_category table, you will see an ‘oid’ column which serves as the primary key. This column is added automatically by Og. You can use the oid values to lookup objects:

  cat = Category[1] # loads the category object with oid = 1

OR

  cat = db.load(1, Category)

As a convenience, Og allows you to lookup the category using the special property ‘name’:

  cat = Category['Programming']

You can lookup objects by name only if the name property is defined.

If you want to view Og’s SQL, you can enable debug mode by setting this global debug (DBG) variable:

  $DBG = true

Customizing the Schema and Defining Relations

Og makes our blog model persistent through a simple interface. The next step is to refine the schema and define relations between the objects:

  class Post; end
  class Comment; end

  class Category
    property :name, String

    many_to_many :posts, Post

    def initialize(title = nil)
        @title = title
    end
  end

  class Post
    property :title, String, :sql => 'VARCHAR2(32) NOT NULL'
    property :body, String
    property :author, String
    property :create_time, Time
    property :hits, Fixnum, :sql_index => true

    has_many :comments, Comment

    def initialize(title = nil, body = nil, author = nil)
        @title, @body, @author = title, body, author
        @create_time = Time.now
        @hits = 0
    end
  end

  class Comment
    property :title, String, :sql => 'VARCHAR2(32) NOT NULL'
    property :body, String
    property :author, String
    property :create_time, Time

    belongs_to :post, Post

    def initialize(title = nil, body = nil, author = nil)
        @title, @body, @author = title, body, author
        @create_time = Time.now
    end
  end

Observe the :sql property option is used to refine the generated column type for the title property of Post, and how the :sql_index option is used to add an index to the generated table.

Notice that the initialize methods provide default values to all parameters. This is required for all managed objects.

Observe the many_to_many, has_many, and belongs_to macros. Og uses these macros to define the relations between standard Objects. In essence, Og defines a domain specific mini language. The following kinds of relations are supported:

    * has_one: has one object of the given type.

    * has_many: has many objects of the given type.

    * belongs_to: belongs_to an object of the given type.

    * many_to_many: defines a many-to-many relation. The corresponding
      rows in the database are linked through a join table.

    * refers_to: refers to another object.

These macros generate the constructs needed to efficiently implement the corresponding relations. For example, the belongs_to macro generates the property that links to the parent. The many_to_many relation generates the join table that links the participating classes.

Note that we have to use forward definitions of Post and Comment to satisfy Ruby’s parser. Workarounds will be provided in a future version.

After defining these relations, using and querying the object model is easy:

  cat = Category.create('Programming')

  cat.add_post { |p|
       p.title = 'Title'
       p.body = 'Body
  }

  cat.add_post { |p|
    p.title = 'Another'
    p.body = 'Hello'
  }

  cat.posts
   => [Post(Title), Post(Another)]

  cat.posts[0].title
   => Title

  cat.posts.size
   => 2

  p = Post[1]
  p.title
   => Title

  p.categories[0].title
   => 'Programming'

  c = Comment.new('hello', 'world', 'tml')
  c.post = p
  c.save

  p.comments.size
   => 1

  p.add_comment { |c|
    c.title = 'Hi there'
  }

  p.comments[1].title
  => 'Hi there'

  com = Comment.new('Hi there')
  p.add_comment(com)

All the methods used in the above examples are generated automatically. These methods transparently modify the underlying SQL schema using efficient queries.

Og provides full access to all features of the underlying RDBMS. Look at the following:

  post = Post.select("title='Title' and body='Body'")
  post.size
   => 1
  post.hits
   => 0

Updating existing objects is easy too:

  p = Post[1]
  p.title = 'Changed'
  p.save

  p = Post[1]
  p.title
  => 'Changed'

You can also update specific properties, for example:

  p = Post[1]
  p.update_properties "body='Hello world'"

  p = Post[1]
  p.body
  => 'Hello world'

If you don’t like a particular comment, you can easily delete it by doing the following:

  Comment.delete(comment)

OR

  comment.delete!

OR

  db.delete(comment)

To delete all comments for a posting, enter the following:

  p.delete_all_comments

When deleting an object that participates in relations, Og tries to delete all objects that belong to this object (ie, cascade deletes).

All the generated methods take more parameters to customize their behaviour to suit your needs.

Defining Callbacks

Og provides a detailed callback facility allowing you to hook into a managed object’s Lifecycle. This is a very useful feature that can improve your code considerably. To implement a callback, you have to define one or more of the following methods in your class:

    * og_pre_insert
    * og_post_insert
    * og_pre_update
    * og_post_update
    * og_pre_insert_update
    * og_post_insert_update
    * self.og_pre_delete

For example, the following code defines a callback for the Post class.

  class Post
    ...

    def og_post_insert(conn)
        puts 'Hey, a new post was just posted!'
    end
  end

When post.save is called, you’ll get this alert:

  p = Post.create('Hello')
   => console: Hey, a new post was just posted!

Using OOP techniques

Og’s managed objects are standard Ruby objects, so we can use class inheritance and module inclusion to minimize the code we have to write. Here’s how we can improve the blog schema:

  class Category
    property :name, String
    many_to_many :posts, Post

    def initialize(title = nil)
        @title = title
    end
  end

  class Common
    property :title, String, :sql => 'VARCHAR2(32) NOT NULL'
    property :body, String
    property :author, String
    property :create_time, Time

    def initialize(title = nil, body = nil, author = nil)
        @title, @body, @author = title, body, author
        @create_time = Time.now
    end
  end

  class Post < Common
    property :hits, Fixnum, :sql_index => true
    has_many :comments, Comment

    def initialize(title = nil, body = nil, author = nil)
        super
        @hits = 0
    end
  end

  class Comment < Common
    belongs_to :post, Post
  end

In essence, this feature allows you to create SQL tables using inheritance, saving you lots of time when using objects with similar properties. It’s also less error prone.

Defining Validation Rules

When managing large amounts of data, enforcing data integrity is important. Og provides another domain specific mini language that allows you to define validation rules in a simple manner. In the following code, the blog schema is enriched with hints that allows Og to automatically generate validation code:

  class Common
    property :title, String, :sql => 'NOT NULL VARCHAR(32)'
    property :body, String
    property :author, String
    property :create_time, String

    validate_value :title
    validate_length :body, :range => 2..100, :msg_long => 'argh'
    validate_format :author, :format => /[a-z]/, :msg => 'wrong format'

    def initialize(title = nil, body = nil, author = nil)
        @title, @body, @author = title, body, author
        @create_time = Time.now
    end
  end

This code demonstrates some validations facilities. Using the validate_value macro, we enforce that the ‘title’ property will have a value. Using the validate_length macro, we enforce the minimum and maximum lengths for the ‘body’ property. Using the validate_format macro, we enforce a required format for values assigned to the ‘author’ field.

Let’s see this validation in practice:

  c = Comment.new
  c.valid?
   => false
  c.errors.count
   => 3

  c.title = 'Hello'
  c.valid?
  c.errors.count
   => 2

The errors array contains a list of Error objects that point to the offending field and contain a descriptive message.

With Og, you can customize almost everything! More information can be found in the source code (lib/glue/validation.rb). To whet your appetite, here is a list of predefined validation macros:

    * validate_value
    * validate_format
    * validate_length
    * validate_inclusion
    * validate_confirmation

TypeMacros

If you look at the common class definition, you will notice that the :sql option looks kind of ugly:

  :sql => 'VARCHAR2(32) NOT NULL'

When building larger object models, this issue comes up frequently. Og provides a elegant solution in the form of type macros:

  def VarChar(size)
      return String, :sql => "VARCHAR2(#{size}) NOT NULL"
  end

  property :title, VarChar(30)

Switching To Another Database

While Postgres is a great database, let’s assume that the client wants to switch to MySQL at the last minute. Don’t worry, Og can easily accomodate this by simply changing the db reference in the configuration file to look like this and then re-running the example:

  db = Og::Database.new(
    :database => 'test',
    :adapter  => 'mysql',
    :user     => 'postgres',
    :password => 'navelrulez'
  )

A new MySQL database is automatically created along with all tables, indices, etc. You get all this with changing only one line of code!

Is There More?

You betcha! You can to find more about Og by reading the available RDoc documentation and browsing the examples.

For any questions regarding Og, feel free to ask on the ruby-talk mailing list (which is mirrored to comp.lang.ruby) or contact gm@navel.gr.

A Nitro specific mailing list is also available. You can post questions about Og to this list. Please subscribe to nitro-general@rubyforge.com. The homepage for this list is available here:

rubyforge.org/mailman/listinfo/nitro-general

Note that Og is still under heavy development, so new features are being added frequently. Be sure to check back for updates.


George Moschovitis is a Ph.D candidate at the National Technical University of Athens and a co-founder of Navel. He is the driving force behind www.joy.gr (a greek youth portal) and the main developer of Nitro, a powerful Web Framework for Ruby.


21 Nov 04

To the Point

One Man’s Journey of Dependency-Injected Discovery

by Jamis Buck

I, Dilbert

It was February, and it was cold. Cold outside, in a dry, wish-there-were-snow sort of way, and cold inside, in a being-forced-to-adopt-Java-at-work sort of way. Rarely had I ever felt more like Dilbert.

I had just finished a nearly-year-long stint in a project investigating how our department might best satisfy the recent "Thou Shalt Have Java" mandate. We came away from that project recommending the adoption of a framework built around Tapestry, and eventually decided to use a light-weight container over a heavy-weight EJB container like JBoss.

This was when I first heard the term "IOC". There were a handful of us, standing around, discussing our non-EJB options, when someone said they were investigating the use of an IOC container. My first thought was "IO-what?"

I was shown HiveMind, which at that time was still only at an "alpha", pre-1.0 release. Repeated attempts were made to answer my questions of "how is this useful?" and "what problem does this solve, exactly?" I realized, in some frustration, that I was never going to really get what this whole IOC buzz was about until I actually sat down and tried to use it.

More frustration. HiveMind, at that point, had virtually no documentation beyond the API Javadocs. No one in my office had done anything with it. And I just wanted to write a simple "hello, world" kind of application to see, at a high level, how HiveMind managed to do its stuff, but the whole "write-compile-test" cycle really got in my way. Java’s syntax itself obscured, for me, much of what I was trying to see in HiveMind.

More than once, I thought, if I could only do this in Ruby, I’d be able to see what this was all about… There was only one way that would ever happen. It would take someone doing something silly, like rewriting HiveMind in Ruby…

Copland

Thus, the first, rough core of Copland was born. There was a lot of stumbling and fumbling involved, since I was trying to implement a HiveMind-workalike without understanding the problems that HiveMind really solved. However, bit by bit, as I implemented each successive feature, a light went on for me and I began to see how service points related to services, what service models were all about, and how configuration points were useful.

Specifically, I began to understand that IOC (or dependency injection, as it is also called) was just a means of loosely coupling components in an application. It was a way to add a layer of indirection, allowing a component to reference another component, without knowing (or caring) specifically what that other component was. (For a more in-depth discussion of what IOC/DI is, check out Martin Fowler’s article, www.martinfowler.com/articles/injection.html, or read the article by Jim Weirich for a more Ruby-oriented description, onestepback.org/index.cgi/Tech/Ruby/DependencyInjectionInRuby.rdoc).

I implemented nearly every significant feature of HiveMind, in Copland, and was very pleased at how easy Ruby made it. What is more, I began to see how the Ruby community could benefit from a library like this, so I registered my first RubyForge project and uploaded Copland. That was in March, of 2004.

I progressed through the first few releases quickly, and despite people’s initial confusion over what Copland (and IOC in general) were for, and despite my inability to articulately explain the concepts, it was downloaded over 100 times in its first month.

Progress lagged, then, as I turned to other projects. In July I finally revisited Copland, and discovered that the code horrified me. I understood IOC much better at this point, and I could see numerous ways that Copland’s design was…well…flawed. What is more, HiveMind development had diverged from the interface it had published when I had first started Copland, and now supported features that I couldn’t really implement given the current state of the code. In discouragement, I packaged up and released 0.4, and then put it away, thinking to never look at it again.

But I couldn’t leave it alone! I kept coming back to it, thinking about it should have been done. I found myself visualizing a way to allow multiple configuration file formats, instead of simply YAML. I thought of a way to improve Copland’s error handling. I saw a cleaner way of doing interceptors, and adding schemas…

I finally gave up the fight, sat down, whipped out Poseidon, and sketched out a rough design for a rewrite of Copland. I knew it would have to be a rewrite—there was very little of the code I could reuse. It was that bad.

Copland, Anew

It took nearly a month, but what I ended up with was infinitely more presentable. Things were better organized, factories were more consistent, and all-in-all it was much easier to extend and maintain. What is more, I was able to implement the newer features of HiveMind in Copland, as well.

I was proud of this newer, better Copland.

So, when Chad and David sent out the request for presentations for the upcoming RubyConf 2004, I could not imagine a better way to "get the word out" about dependency injection. I threw together a quick summary of what I would like to present ("Dependency Injection in Ruby"), and submitted it. To my great pleasure, it was accepted, and I spent September further improving Copland, and preparing my presentation.

RubyConf

With much anxiousness and trepidation the day of my presentation arrived. I was confident in my slides, and I knew the material well, having rehearsed it and polished it over and over. What I was unsure of was how the other conference attendees would receive what I had to say.

Certainly, I knew there would be no hostility. I didn’t fear for my life, or anything as dramatic as that. But I could only hope that my presentation would clear up some of the confusion that appeared whenever IOC or dependency injection was mentioned on ruby-talk.

I took the stand, trembling inside and yet attempting to portray a sense of indomitable confidence and calm.

I took a deep breath.

And I began.

The presentation went very smoothly. Many questions were asked. There was even a fairly lively discussion about dependency injection right in the middle of my presentation, as some attendees—those with real-life IOC experience—fielded answers from those who still hadn’t seen the light. The questions ranged from the predictable ("why should I use this?") to the surprising ("why don’t you serialize the registry to ruby code to save parse time?"). I know for a fact that I came away from the presentation very edified, and from the comments that others made both during and afterwards, I think others found food for thought in it, as well.

The primary criticisms of Copland were about its strong Java taste. Ironically, I had been so close to the project for so long, I had grown used to the flavor. Many attendees of the conference disliked Copland’s use of external configuration files to specify services and their dependencies, feeling that Ruby itself was a more-than-sufficient language for specifying data. Also, Copland makes what some might call excessive use of factory classes, especially factory classes that are completely separate from the objects they produce. Java needs such constructs, but there are more elegant ways to represent those relationships in Ruby.

Due to my travel schedule, I had to leave almost as soon as the last speaker finished. As I was heading out the door, Jim Weirich approached me and asked, casually, in that gentle way of his, what a dependency injection container that was designed with Ruby’s idioms in mind might be like. Being in a hurry, I agreed that it would be an interesting exercise. Jim said he’d try to write up something, and I said I’d love to see it if he did, and I headed out the door.

A Fresh Perspective

I had completely forgotten Jim’s offer to write up something about DI in Ruby until I got an email from him just a few days later. He had attached an article he’d written about a Ruby-esque DI implementation, complete with a proof-of-concept, and he asked for my feedback.

I was, frankly, astounded. My world was shaken. I saw, for the first time, just how strong the Java slant in Copland really was.

I probably frightened Jim with my subsequently exuberant praise of his insight. (Incidentally, Jim’s article has since become one of the cornerstones in my virtual library of dependency injection articles.)

Jim’s proof-of-concept included the bare-bones of a DI implementation in which all services were singletons. I was interested in extending Jim’s implementation to include some of the more powerful features of Copland, including service models and interceptors. Jim granted me permission to use his design as a basis for a new project.

I started from scratch, cannibalizing Jim’s design for ideas and inspiration (both of which I found in spades). The first few revisions were codenamed Syringe. That name I later abandoned in favor of Needle, "syringe" being rather annoying to type frequently. After feedback from many people, I released Needle 0.5 in mid-October.

Copland versus Needle

To give you an idea of how much simpler Needle’s approach is versus Copland’s approach, consider the following example.

In some imaginary file, "calc.rb", resides the following code:

  class Adder
    def compute( a, b ); a.to_f + b.to_f; end
  end

  class Subtractor
    def compute( a, b ); a.to_f - b.to_f; end
  end

  class Multiplier
    def compute( a, b ); a.to_f * b.to_f; end
  end

  class Divider
    def compute( a, b ); a.to_f / b.to_f; end
  end

  class Calculator
    def initialize( operations )
      @operations = operations
    end

    def method_missing( sym, *args )
      @operations[ sym ].compute( *args )
    end
  end

This simply implements a generic "calculator" class that accepts a hash as a constructor parameter, mapping operation-names to their implementations. In Copland, you would use the following YAML file (package.yml) to tie all the pieces together:

  ---
  id: calc

  service-points:
    Adder:
      implementor: calc/Adder
    Subtractor:
      implementor: calc/Subtractor
    Multiplier:
      implementor: calc/Multiplier
    Divider:
      implementor: calc/Divider

    Calculator:
      implementor:
        class: calc/Calculator
        parameters:
          - !!configuration Operations

  configuration-points:
    Operations:
      type: map

  contributions:
    Operations:
      :add      : Adder
      :subtract : Subtractor
      :multiply : Multiplier
      :divide   : Divider

Briefly, all of these services are said to belong in the "calc" package. Five service points are defined, one for each of the classes in "calc.rb". Particularly, the Calculator class is told that when it is instantiated, it should be given one constructor parameter, the configuration point "Operations".

The Operations configuration point is defined to be a map (or hash), and the Adder, Subtractor, Multiplier, and Divider services are contributed to it, under the keys :add, :subtract, :multiply, and :divide.

(For more information on Copland service definitions, see the Copland user manual at copland.rubyforge.org).

Given the above definitions, and assuming that both "calc.rb" and "package.yml" are in the same directory as the following script, the following script would load the services and play with the calculator:

  require 'copland'

  registry = Copland::Registry.build
  calc = registry.service( "calc.Calculator" )

  p calc.add( 1, 2 )
  p calc.subtract( 2, 3 )
  p calc.multiply( 3, 4 )
  p calc.divide( 4, 5 )

Now, compare the above with the following, done using Needle. The "calc.rb" script is unchanged from above. The driver file would look like this:

  require 'needle'
  require 'calc'

  reg = Needle::Registry.define! do
    namespace_define! :operations do
      add { Adder.new }
      multiply { Multiplier.new }
      divide { Divider.new }
      subtract { Subtractor.new }
    end

    # the namespace becomes the hash of operations!
    calculator { Calculator.new( operations ) }
  end

  calc = reg.calculator
  p calc.add( 1, 2 )
  p calc.subtract( 2, 3 )
  p calc.multiply( 3, 4 )
  p calc.divide( 4, 5 )

Most of that file is just the registering of the services, which could be done in a separate file or module if necessary. Needle resulted in fewer lines of configuration, and (in my opinion) a much more readable configuration overall.

Copland, Deprecated

One of the most incredible things to come of this new perspective on dependency injection is that my interest in Copland has all but evaporated, boiled away by the incandescent heat of Needle. Needle is, in my opinion, Copland’s superior in every way. It is true that Copland can do some things that Needle can’t, but the things it does that Needle doesn’t are things that Needle doesn’t have to do.

Copland supports eager loading of services. Needle doesn’t, but you can accomplish the same thing by referencing the service immediately after its definition:

  reg.register( :foo ) { Foo.new }
  reg[:foo]

Copland supports "fail fast" processing, meaning that it attempts to detect and report configuration errors as soon as possible. Needle does very little explicit processing to detect configuration errors, but because Needle uses Ruby itself to register services, Ruby’s own error checking serves much the same purpose.

The one thing that (for me) the jury remains out on are external configuration files. Copland supports them (in fact, it can’t really function well without them). Needle does not. Are they useful? There are arguments both for and against. The fact of the matter is that there are people that would use them in Needle if they existed. Is that argument enough to add them to Needle? Perhaps, perhaps not. Time will tell.

Future Directions

So, what does the future hold for Copland and Needle? Will Copland whither away and disappear into the mists of history?

I have the idea of rewriting Copland a second time, this time building it on top of Needle. Whether this will happen or not is unknown, but it’s a thought.

Needle’s future seems secure, though. There are several projects out there that are already using it. I’ve been reworking my Net::SSH implementation (net-ssh.rubyforge.org) to use Needle to tie all the pieces together. RPA-Base (rpa-base.rubyforge.org/wiki/wiki.pl) will also use Needle, starting with their version 0.3 release. Christian Neukirchen is writing a new blog system called Nukumi2 which will use Needle (kronavita.de/chris/blog/2004/10/18-nukumi2-roadmap.html#x-20041018-161215). And I recently made a proposal to David Heinemeier Hansson that demonstrated how dependency injection could help Rails (ruby.jamisbuck.org/rails-injected.html).

Someday, perhaps even Matz will catch the vision of DI and want to include Needle as a standard Ruby library… Wipe that grin off your face, soldier! A fellow can dream, can’t he?

Points of Interest

Dependency Injection

Ruby DI/IOC Containers


Jamis Buck is a software engineer for Brigham Young University. He has written various bits of software over the years, most recently hacking on Needle, Copland, Net::SSH, and SQLite/Ruby. Jamis is currently the chairman of the BYU Ruby Users Group (ruby.byu.edu), living and working in Provo, Utah. He is married to Tarasine, and they have two children, Nathaniel and Katie.


26 Oct 04

Reflections on Rails

by David Alan Black

RubyConf ended just over three weeks ago, and in those three weeks I’ve written three Web applications (actually, rewritten them: two based on my own projects, one based on someone else’s) using Rails. Of those three, one is a few CSS and template tweaks away from replacing its predecessor; one is scheduled for a demonstration to the IT people at my university in a couple of weeks (and is essentially ready); and the third, the Ruby FAQ, is now in place and operational (www.rubygarden.org/faq).

I want to talk a bit about the FAQ project, in particular; but I’m going to approach it by way of some general comments about my experience so far with Rails.

For me, the biggest obstacle to using Rails — indeed, the only obstacle — was "getting it"; that is, getting my mind around the Model/View/Controller paradigm and how Rails expresses it. And the only reason this took a day or two in my case, rather than minutes, is that it was new to me. But as soon as I "got it", everything started to move very quickly indeed.

I’ve written scores of CGI scripts and applications over the years, usually with no more library or framework support than Perl or Ruby’s CGI libraries, and I’ve achieved a certain facility with that mode of programming. I can pretty much sit down and hammer out a voting booth or brute-force site-search utility in my sleep, and have, at least figuratively speaking, frequently done so. To my credit, I’m aware of the hackerliness and inelegance of most of these projects. In some cases, actually, those dubious qualities don’t matter — but what I’ve run up against a number of times that does matter is the fact that doing everything this way doesn’t scale. The little flatfile-based voting booth will work, but it doesn’t result in any conveniently reuseable code and doesn’t really provide a model for projects that can grow and breathe. Up to a point, I would actually defend some of this framework-lessness and non-reuse. I really do think there’s no reason to maintain and use a whole library instead of typing:

     name, password = line.chomp.split(/:/)

once in a while. But the "up to a point" point looms large. It’s fine to write code to parse .htpasswd thirty times — but it also means that all you’re doing is things like parsing .htpasswd.

Rails has been my way out of all of this, and it could hardly be more radical or dramatic. Nor did I intentionally avoid other Web application frameworks; I just didn’t seek them out. Rails sought me out, after a fashion: David gave his talk at RubyConf, and was available for questions and coaching; and without my quite planning it this way, I found the momentum pointing toward my really learning it, rather than away. I date my first-hand acquaintance with Rails from the last morning of RubyConf, and David was there to guide me.

Working with Rails makes me feel like a windup toy that had been stuck against the wall, and then got turned around. It’s actually very similar, in its domain, to the feeling many get from Ruby itself. For me, it also represents a kind of enforced de-spaghettification that can only be for the good. Yes, one can write spaghetti code in a Rails controller. But the organization of one’s project itself will militate against it. All your controller really has to do is make life possible for the views it controls — and that is often as easy as:

   def show
   end

An action is often more involved than that, of course, but it’s still usually very concise for what it does.

The idea for redoing the FAQ in Rails came from Chad Fowler, maintainer of Ruby Garden. Reworking the FAQ site was on Chad’s to-do list, and finding an enjoyable and useful Rails project was on mine. So I took on the FAQ.

The original FAQ implementation was an Iowa application by Dave Thomas. I borrowed some RDoc conversion code from this application — and, of course, I reused the FAQ database itself. The only changes I had to make to the database were to some SQL table and field names. Rails wants the table names to be plural, and fields have to be named according to a particular scheme in order to reap the benefits of Rails’s object associations. Thus I had to change the table name ‘section’ to ‘sections’, and in the entries table I had to change the field name ‘sec_id’ to ‘section_id’. Other than a few things along those lines, the database remained unchanged.

I started the FAQ project on a Saturday, and within a couple of hours (much of which was devoted to SQL massaging), I had a functional site for displaying and selecting FAQ sections and entries. To this I added user-related capabilities. In the past it was possible for people to leave questions and comments without any kind of authentication. Chad and I agreed that the site needed a bit more security. There was already a ‘users’ table, and I added an ‘applicants’ table (and corresponding Rails model), which points into the users table and also holds the random key that the applicant has to provide to complete registration.

I also added a very dinky — but possibly helpful — search facility. It basically does a brute-force search of the existing FAQ entries, and thus is in spirit rather akin to some of the ad hoc CGI scripting I’ve done in the past. Rails, of course, makes it easy. I have to say, though, that this particular action is probably about the scrappiest-looking I wrote for this project:

  def search
    terms = @params['search']['terms'].scan(/"(.*?)"|([^"\s]+)/).
                                       flatten.compact
    cond = "body LIKE '%#{terms[0]}%' "
    cond << terms[1..-1].map {|t| " AND body LIKE '%#{t}%'"}.join("")
    @results = Entry.find_all(cond)
  end

But most of the scrappiness has to do with the piecing together of the SQL condition. Once all that is in place, the last line of the method body does everything that’s needed to prepare the environment for the view. (Note: readers are invited to send me ideas for cleaner ways to generate that multiple condition line!)

Actually one of the most enjoyable moments in developing the FAQ came right after the site went up, when someone asked me whether I could include a view of each section with all the entries present in full. The answer to that, in the section controller, was:

  def expanded
    @section = Section.find(@params['id'])
    @entries = @section.entries.sort_by {|e| e.number.to_i }
  end

and the business part of the corresponding view:

  <% for entry in @entries %>
    <p><%= @section.number %>.<%= entry.number %>
       <%= link_to "#{entry.title}",
           :controller => "entry",
           :action     => "show",
           :id => entry.id %>
    </p>
    <%= rdoc_to_html(entry.body) %>
  <% end %>

which I wrote in about two minutes and was able to present him with. (It’s still "undocumented", so to speak, but it will probably work for you.) It’s not just the small number of lines that pleases me. It’s also the way it all fits together. As I said, it took me a bit of time to get this (I believe it was Day 2 before I noticed that the method names in the controllers and the names of the view files were the same…), but the payoff in time and productivity has already been considerable.

"Rails is boring", they say — meaning, it makes the development process almost self-evident. It’s true, in that sense, but in fact it’s making Web application development vastly more interesting for me, because it’s rescuing me from some very stale habits and letting me produce and share code that I’m confident about using and expanding. Even if your background in this area isn’t exactly like mine, I definitely recommend looking at Rails and trying it out.


David Alan Black is one of the founding directors of Ruby Central, Inc., co-organizer of the annual International Ruby Conference, contributor to Teach Yourself Ruby in 21 Days (Sams Publishing, 2001), co-author of scanf.rb in the Ruby standard library, maintainer of RCRchive (Ruby Change Request Archive), and frequent discussant on the ruby-talk mailing list and the #ruby-lang channel on irc.freenode.net. In real life David is an Associate Professor of Communication at Seton Hall University.

06 Sep 04

RubyGarden Site Changes

As old-timers here will no doubt notice, RubyGarden is undergoing some changes. Several of the functions on the old site were out of date and/or not working. So, while we have bigger and better plans, we’ve introduced a temporary "patch" that we hope will offer an incremental improvement over the previous incarnation.

You’ll also notice that users can no longer login and post news items. If you’re interested in contributing an article or interview, please email webmaster@rubygarden.org. We’ll handle these on a case by case basis for now until we get things setup for user contributions.

Happy Rubying!

Powered by RubLog