Launchpad

Often times when I come into an existing project, or even come back to a project I’ve been maintaining for a long time, I’m reminded of the importance of third party abstraction.

This type of abstraction provides your application with an insulation layer, a warm blanket if you will, against uncontrollable changes within the code which you do not directly maintain.

Most recently, I’ve been working quite a bit with message queues through Ruby. These queues are used for several reasons, but primarily to facilitate separation of function between machines of much larger systems. This separation can be very beneficial when it comes to moving off slow-running processes to worker machines, breaking your system architecture into functionality zones, or just generally sharing information between two systems without introducing a tight coupling between them.

The best way to think about message queues is as an office or dormitory tack board. For instance, someone posts up a message (“Used bike for sale!!”), with no specific concept of who or what might read it, but just that someone may see it and hopefully that the poster may hear back something about it later. At your leisure, you might glimpse at the board as you walk by and either decide you don’t care about the bike, thus allowing you to move along in your day, or that the bike is useful to you and you’re willing to purchase it. Now, maybe you need to run to the ATM for cash, make a call to the poster, check on Craigslist for a better deal, or whatever. The point is, all this happens in separate interactions with each party deciding what’s important and then what to do with the information.

In this case, I was working on a new project which collects information about a specific topic from several external sources. These sources may or may not be available, some certainly respond much faster than others, and each responds with a differing set of data. So, it is quite beneficial – for this project – to manage that gathering through individual sub-systems controlled by a queue.

Because Ruby has interface libraries to several different queues (AmazonSQS, Beakstalkd, Apache ActiveMQ, RabbitMQ, etc.) and really, you never know how long a library will be maintained or even how long the underlying system might be around, I decided it would be better to abstract the queue interface into core functions and data. Thus, queued was born. This abstraction allows me to use my own queue interface (queued) with any of the above mentioned queue systems, without losing application functionality. I can drop in a new back end, and my system continues to work. AmazonSQS is no longer viable? Ok! Let’s move over to Joe’s Queue Service. The only code I need to change is the specific implementation in the back of queued, my abstraction layer. Once done, everything else should work as advertised.

In fact, I did just this, and created a “test” queue service which performs no interaction whatsoever with an external service. This allows me to exercise my code in testing without querying against external, potentially unavailable services.

For all of the Rails people out there, this is the same idea as how ActiveRecord abstracts your specific database implementation. Pulling data from MySQL or PostgreSQL or even Oracle are all invisible to you. User.first just works, regardless of the specific database level implementation.

Certainly taking the time to abstract all of your third party libraries is not necessary or even a good use of your time. The places in your application in which abstraction may be beneficial should be decided on a larger functionality and library basis. Your decision should balance the likelihood of change against cost of re-implementation… and then that will certainly get trumped how much that library or service failing might immediately ruin your entire application. ;)