Announcing Patron: a Ruby HTTP client library
Jul 3rd, 2009 by phil
I was recently given the opportunity to release several of the projects I was working on at The Hive as open source. These projects are libraries that we built to address specific problems we ran into but were not core to our business. Most of these were always destined to be released, it just took a while before we got around to it. Everything is currently on my GitHub page, but I wanted to do a little cleanup on each project and announce them individually.
Today I am announcing the first of these projects: Patron. Patron is an HTTP client library for Ruby that features a usable API and good performance. As for the name Patron, I searched through the thesaurus looking for synonyms for the word “client”.
The performance problems with the built-in Ruby HTTP client are well documented elsewhere, so I won’t go into that here. The other alternative is the Curb gem which is based on libcurl and offers great performance. Unfortunately, curb is unfinished and doesn’t appear to have an official maintainer. Also, the API leaves much to be desired. These issues are compounded by the fact that curb is primarily implemented in C and is difficult to modify.
I tried to patch curb to get all of the features we wanted, but subtle bugs kept creeping in. Eventually I gave in and wrote Patron. The goal was to have a Ruby HTTP client built on libcurl with a reasonable API. By implementing as much as possible in Ruby I think that Patron is much more maintainable than curb.
I have not made an attempt to support every feature of libcurl. Libcurl is huge and trying to support every feature would lead to bloat and make it harder to maintain. Instead, I built a small API that did what we needed it to do. It has been in use at The Hive for some time now and seems pretty stable.
The documentation for Patron is on Rubyforge and the source code is available on GitHub.
proxy support please :]
Which features does Curb lack?
It was missing PUT and DELETE support when I first started using it. I added support for both, and it worked OK for a while. The problem is that the C code is a bit of a mess and adding features can be dangerous. There were a few situations where we wanted to add a new feature or tweak an existing one and it would cause breakage elsewhere.
So it’s similar to RestClient (http://rest-client.heroku.com/). What would be the major difference?
RestClient is a higher level DSL for interacting with RESTful web services built on top of HTTP. Patron has a bit of a lower level API and is built on top of libcurl. This means that Patron may be more appropriate for applications where you are not necessarily talking to a strictly RESTful web service or where you want a programming API as opposed to a DSL.
That having been said, the interfaces to RestClient and Patron aren’t all that different. The big difference is speed. Since RestClient is built on Net::HTTP it inherits all of the performance problems mentioned in the article. Patron is almost certainly much faster, but I haven’t specifically benchmarked RestClient.
Philipe, also if you do any serious http interaction in your Rails app, and need performance and accurate timeouts you have to use cUrl instead of Net::HTTP otherwise you’ll go nuts. Will definitely give this library a try, curb is clearly unfinished.
Here is my own benchmark to see if switching from rest-client to patron would make sense for couchrest. (CouchDB DSL/ORM) http://gist.github.com/142999
Patron is faster, way faster. We are talking about 7x faster! I didn’t yet look at the full API but it seems like it would support everything couchrest needs. However because it required libcurl to be installed, I will probably have to implement a fallback to a pure ruby solution.
Anyways, great job Phil!
Hi, can you try to use rake-compiler for creating binary gem for windows users?
http://github.com/luislavena/rake-compiler/tree/master
Hi. Thanks a lot for the library. What an amazing coincidence - I spent the last night fighting with Curb crashes in my multithreaded app and was already trying some alternatives (namely, httpclient and pure Net::HTTP). Patron surely looks interesting.
One of the servers that I query sets a cookie and redirects to the same URL. Do I understand correctly that cookies have to be handled manually in Patron - e.g. extracting key-value pairs from response.headers and putting back into the next request in the redirect chain? Or libcurl, the way you invoke it, will handle such session cookies automatically?
Oh my goodness thank you! I can’t tell you how much I’ve bashed my head against net/http in my time with ruby, the whole time thinking that in such an active and modern community somebody must have written something better. Every time I asked the question (to friends or google or whoever) it was always open-uri or curb, which have bad underlying client problems and bad api problems respectively. You’ve taken the right back-end and put a real api on top of it and based on my first little toying around, I’m guessing I’ll be greatly indebted to you for it. Nice work!
That’s really good news ! I’ve just look quickly the documentation. Based on what I’ve seen, if I download a small or big file, the whole file will be downloaded in memory, and then, I can write it to a file.
Is there a way, or will it have a way to get the chunks donwloaded as they come, without putting everything in memory ? I know Net::HTTP can do this, even if Net::HTTP as serious performance issues.
Anyway, very interesting work.
LeMarsu: Patron doesn’t currently support this, but it shouldn’t be difficult to add. I will try to get it into Patron 1.0.
I’m enjoying Patron a lot. It’s very easy to work with. However, I’m getting the following error trying to connect to https sites: “SSL: couldn’t create a context!”
Reading up on similar issues in PHP, it seems there may be a strange interaction with openssl linked in libcurl and other openssl usage in Ruby. Do you have any thoughts on that?
Heston: I recently fixed this issue. I am going to push the changes to GitHub soon and will release version 0.4 sometime this weekend.
[…] provides more background about his design choices on his blog. You can find Patron on Github or install it as a […]
Just 10 minutes ago I proposed a small project to my boss that will benefit greatly from using Patron. One of us has great timing!
@phil: I saw the change you mentioned in github changing “CURL_GLOBAL_NOTHING” to “CURL_GLOBAL_ALL.” I made this patch myself and rebuilt the extension, but I still get the context error. Is there something else I’m missing? Thanks.
@Heston, That change fixed it for me. Does your version of libcurl have SSL support compiled in? The default version on Leopard does, as does the MacPorts version, but I think you have to install extra packages under Linux. It has been a while since I have had to build Patron under Linux, so I don’t remember the details.
Nice work! Looks like we’ve been working on scratching the same itch. I and my coworkers put together CurbFu as a simple wrapper for Curb (http://github.com/gdi/curb-fu). I like Patrion’s idea of keeping an HTTP session open. I had that on my own wish list for CurbFu, but we’ve been focusing lately on integrating CurbFu with Rack::Test to make an easy cross-application testing framework. I also like the idea of going straight to libcurl instead of depending on Curb.