Fiat Developmentum » Code

Patron 0.4

phil — Wed, 22 Jul 2009 17:35:32 +0000

The 0.4 version of Patron has been released. This version addresses feedback from the initial public release and fixes a few minor bugs:

Fixed a bug in the Patron.version method
Added HTTP proxy support
Enabled SSL and Win32 support
Added ability to download data to a file with GET
Fixed a bug where header values were prefixed with a space
Made the Session#request method public
Added support for the WebDAV COPY method
Added support for uploading files with POST and PUT

In addition to all of this, Patron has a new home page at http://toland.github.com/patron/.

So, what’s next for Patron? Well, I would like to put out a 0.5 release in the next few weeks with some refactoring and API tuning. Also, I would like to implement the rest of the WebDAV methods. I am toying with the idea of creating a net/http compatible interface so that Patron can be used as a drop-in replacement. Not sure if that is going to go anywhere, though.

Once I am happy with the API and documentation I will release 1.0. Other than that, I try to respond to bug reports and feature requests in a timely manner. I originally wrote Patron to scratch my own itch, and that itch has been scratched. At this point I am looking to make Patron as broadly useful as I can, and your feedback helps with that.

Patches and contributed documentation are always welcome. In particular, if anyone feels like trying to get this working on Windows, I will be happy to integrate the resulting patches and publish a binary gem.

Using DTL templates with Nitrogen

phil — Mon, 13 Jul 2009 20:24:35 +0000

One thing that I did not like about working with the Nitrogen web framework is the way that view rendering is handled. Nitrogen calls the function main/0 in your web modules which, by convention, returns an instance of the template record. This record points to a template file that has HTML code with a little Erlang mixed in. Essentially, you can call functions in your web module and whatever content those functions return is interpolated into your template.

This all makes it sound a little more complicated than it really is. Here is a little “hello world” sample to illustrate how it works. First, the web_page Erlang module defines the main/0 function and a couple of callouts:

-module(web_page).
-include("wf.inc").
-compile(export_all).

main() -> #template { file="./wwwroot/template.html" }.
title() -> "Hello, World!".
content() -> #h1 { text="Hello, World!" }.

This will cause Nitrogen to render the template.html file.



[[[page:title()]]]


[[[page:content()]]]

The callouts in the template will call the respective functions in the web_page module and we end up with a page that displays a bold “Hello, World!”. The template file is really analogous to a Rails layout and the content that Rails puts into a view template is returned from the callouts using a markup DSL based on the Erlang record syntax.

This template mechanism is simple, but it becomes very limiting when working on a non-trivial application. Nitrogen mixes controller functionality with presentation in a way that is reminiscent of PHP or JSP. Also, you cannot compose templates and the markup DSL doesn’t cover all of the HTML elements and attributes.

I ran into this problem while working on a Nitrogen powered site and decided that I just couldn’t live with the default template system. After a little digging around I discovered ErlyDTL which implements a substantial subset of the Django Template Language in Erlang. It turns out that using ErlyDTL in a Nitrogen site is quite simple.

The first thing I needed was a utility function to render ErlyDTL templates. Nitrogen includes implicit elements for the notification flash and Javascript code for all of that AJAX goodnes which I needed to replicate in my render function.

-module(my_util).
-compile(export_all).

template_path(Template) ->
    filename:join([code:priv_dir(my_app), templates, Template]).

render_template(Name, Variables) ->
    Template = template_path(Name ++ ".html"),
    Mod = list_to_atom(Name ++ "_template"),
    ok = erlydtl:compile(Template, Mod),
    {ok, Body} = Mod:render([{flash, element_flash:render()},
                             {script, wf_script:get_script()}
                             | Variables]),
    Body.

Now, I can replace the simple module from earlier with this new one:

-module(web_page).
-include("wf.inc").
-compile(export_all).

main() ->
    my_util:render_template("template", [
          {title, "Hello, World!"},
          {content, content()}]).

content() -> "Hello, World!".

My template file is now equally simple:



{{ title }}


{{ content }}

All of the markup is now in the template file, and the module is concerned only with getting the content from somewhere and making it available to the view. This also allows us to use the full power of ErlyDTL: we can compose templates and perform simple operations on the content to be displayed. Here is an example from the ErlyDTL home page:

Welcome back, {{ name }}!

You have {{ friends|length }} friends: {{ friends|join:", " }}

Have some primes:
{# this is exciting #}
{% for i in primes %}
    {{ i }}
{% endfor %}

There is one drawback to this approach. The Nitrogen DSL includes helpers for generating links that work with its event system. When using ErlyDTL we have to create those links manually, but it isn’t hard. The key thing to remember is that the link id is the event name that will be passed to your module. Here are a couple of examples:

Do It!

The extra flexibility of DTL is certainly worth the small amount of extra work.

Announcing Patron: a Ruby HTTP client library

phil — Fri, 03 Jul 2009 20:02:50 +0000

I was recently given the opportunity to release several of the projects I was working on at The Hive as open source. These projects are libraries that we built to address specific problems we ran into but were not core to our business. Most of these were always destined to be released, it just took a while before we got around to it. Everything is currently on my GitHub page, but I wanted to do a little cleanup on each project and announce them individually.

Today I am announcing the first of these projects: Patron. Patron is an HTTP client library for Ruby that features a usable API and good performance. As for the name Patron, I searched through the thesaurus looking for synonyms for the word “client”.

The performance problems with the built-in Ruby HTTP client are well documented elsewhere, so I won’t go into that here. The other alternative is the Curb gem which is based on libcurl and offers great performance. Unfortunately, curb is unfinished and doesn’t appear to have an official maintainer. Also, the API leaves much to be desired. These issues are compounded by the fact that curb is primarily implemented in C and is difficult to modify.

I tried to patch curb to get all of the features we wanted, but subtle bugs kept creeping in. Eventually I gave in and wrote Patron. The goal was to have a Ruby HTTP client built on libcurl with a reasonable API. By implementing as much as possible in Ruby I think that Patron is much more maintainable than curb.

I have not made an attempt to support every feature of libcurl. Libcurl is huge and trying to support every feature would lead to bloat and make it harder to maintain. Instead, I built a small API that did what we needed it to do. It has been in use at The Hive for some time now and seems pretty stable.

The documentation for Patron is on Rubyforge and the source code is available on GitHub.

QLMarkdown 1.1

phil — Thu, 12 Feb 2009 03:19:31 +0000

There is a new version of the QLMarkdown generator available. I didn’t have anything to do with the new release other than zipping up the final product. Michael Dominic K. added some nice new features. You can download the goodies here or grab the source from GitHub.

Nitrogen

phil — Sat, 06 Dec 2008 17:10:35 +0000

I have been using Erlang extensively for the last 6 months or so and it is a very nice language. While I have enjoyed working with the language, I wasn’t really impressed with the existing Erlang web frameworks. Mochiweb is nice enough, and I use it to power an application with an HTTP based ReSTful API, but it is too bare bones to use for building a full-featured site.

Then I ran across Nitrogen. The first thing that struck me about Nitrogen is how it (ab)uses the Erlang record syntax to create a passable DSL for HTML generation. Check out this snippet from the included sample application:

main() ->
  Title = "Postbacks",
  Body = #template { file=onecolumn, title=Title,
                     headline=Title, section1=[

    #button { text="Press Me", postback=button_pressed },
    #p{},

    #link { text="Click Me", postback=link_clicked },
    #p{},

    #label { text="Press enter in the textbox." },
    #textbox { text="This is a message...", 
               postback=textbox_enterkey },
    #p{},

    #checkbox { text="Toggle Me", postback=checkbox_clicked },
    #p{},

    #dropdown { postback=dropdown_changed, options=[
      #option { text="Option 1" },
      #option { text="Option 2" },
      #option { text="Option 3" }
    ]},
    #p{},

    #span { text="Mouse Over Me", 
            actions=#event { type=mouseover, 
                             postback=span_mousedover } },
    #p{}

  ]},
  wf:render(Body).

The second thing that caught my attention is the event model. The postback attribute on the elements in the above snippet cause the Erlang term that is assigned to it (in this case symbols) to be posted back to the server using AJAX. This example simply shows an alert box with the posted symbol. The code is very concise:

event(EventInfo) ->
  wf:wire(#alert { text=wf:f("~p", [EventInfo]) }),
  ok.

The nice thing about this model is that you can use Erlang’s pattern matching to catch events from different elements. If you have elements that look like this:

#button { text="Next", postback=next },
#button { text="Previous", postback=previous },

then you can handle the events using pattern matching:

event(next) ->
  %% Go to the next item...
  ok;
event(previous) ->
  %% Go to the previous item...
  ok.

This is clean and easy to understand.

I have not spent as much time as I would like with Nitrogen, but it certainly has my attention. It currently addresses two of the gripes I have had with other frameworks, but there are other issues that have not yet been addressed. For example, how easy is it to deploy or update a site built with Nitrogen? What does the performance and efficiency look like? Using Erlang gives Nitrogen a head start in both of these areas, so I am optimistic.

app_version Updated

phil — Sun, 10 Aug 2008 23:47:23 +0000

I have updated my app_version Rails plugin to support getting build numbers from Git. This is of necessity a little kludgy. The plugin allows you to specify either ‘git-revcount’ or ‘git-hash’ as the build. ‘git-revcount’ will set the build number to the number of commits on the current Git branch and ‘git-hash’ will set the build number to the first six digits of the Git HEAD hash. Neither method is really equivalent to the Subversion revision number, but either might be useful in certain contexts.

The source code for the plugin is available at http://github.com/toland/app_version.

app_version plugin is now hosted on GitHub

phil — Fri, 11 Apr 2008 02:25:27 +0000

I recently changed hosting providers and I am still in the process of moving everything over. One headache that I don’t need is recreating all of the Subversion repositories that I had hosted on the old service. So, I have decided to move the code for my app_version plugin to GitHub.

The plugin’s code is now located at http://github.com/toland/app_version. You can clone the repository using the following command:

git clone git://github.com/toland/app_version.git

OOP is a tool

phil — Tue, 08 Apr 2008 00:49:34 +0000

I spotted this comment to a lengthy article about the proper use of inheritance in Object Oriented programming:

On one of the ‘communities,’ someone had a sarcastic response to the Final == Good article: To boil the article down: OOP is hard. Let’s make it as non-object-oriented as possible so that things don’t get too complicated!

I think that’s actually correct in a way. OOP actually is hard to get right. Encapsulation is not hard, it is a simplifying principle. But inheritance is seductively easy in the trivial case but hard to get just right in the wild. Thus, the admonition to favour composition.

That does’t mean we never use it of course. But I am not embarrassed to say that I think inheritance is hard to get right.

This strikes me as odd; the whole point of OOP is to make it easier for programmers to conceptualize and build complicated software systems. If inheritance is complicating the issue rather than simplifying it, maybe OOP needs a rethink.

The only “right” way to do OOP is the way that helps the programmer deliver high quality software in a reasonable amount of time. This isn’t something that strikes me as capable of being objectively right or wrong. I think that there is a danger in overthinking issues like this or in assigning too much value to being “object oriented.”

Creating a ‘returning’ macro in Lisp

phil — Sat, 25 Aug 2007 21:11:01 +0000

I have been reading Practical Common Lisp and learning quite a lot in the process. However, I began to see a pattern emerging in the code that deals with objects that is also very common in Ruby. The pattern emerges where you have a class and you want to perform some initialization beyond what the normal initializers provide you. I will refer to the following class in this example:

;;; Lisp User class
(defclass user ()
 (login
  password
  first-name
  last-name))

The recurring pattern involves code that looks like this:

(let ((u make-instance user))
  (with-slots (first-name last-name) u
    (setf first-name "Foo")
    (setf last-name "Bar"))
  u)

In other words, we want to create an instance of a class within a let form, manipulate that instance somehow and then return it from the form. This example is trivial and could have been accomplished using initializers, but I think it still illustrates the pattern.

Rails has a function that makes this pattern a little cleaner, at least to my eyes. The function is called returning, is part of the ActiveSupport library and is defined as:

def returning(value)
  yield(value)
  value
end

This is very simple and it allows you to write the Ruby equivalent of our user initialization code like this:

def create_user
  returning User.new do |u|
    u.first_name = "Foo"
    u.last_name = "Bar"
  end
end

Translating this into Lisp should be a fun and reasonable simple exercise. First, lets try a naive transliteration of the returning function from Ruby to Lisp. That might leave us with the following code:

(defun returning (value fn)
  (let ((var value))
    (funcall fn var)
    var))

This is a literal and faithful translation of the code from Ruby to Lisp and it allows us to write our user initialization code like so:

(returning (make-instance 'user)
           (lambda (u)
             (with-slots (first-name last-name) u
               (setf first-name "Foo")
               (setf last-name "Bar"))))

This code has a couple of problems. First, it is more verbose than I would like. The final term is no longer hanging off the bottom but we have added a lambda form that wasn’t there before. Second, this really isn’t very idiomatic Lisp code (at least as far as I can tell with my still limited knowledge of Lisp).

The problem is that Lisp’s lambda forms are not really the same thing as Ruby’s blocks. In fact, the closest thing Ruby has to a Lisp lambda is a Proc object which you can create, oddly enough, with the lambda keyword. The returning function in Ruby is taking advantage of blocks to implement the Template Method pattern. While you can use lambda in Lisp to create a template method, there is a more elegant solution: macros.

So, lets take a stab at creating a ‘returning’ macro in Lisp. We will need to take the name of the variable to use and its initial value as parameters to the macro. We will also need the body of the form where all of the work gets done. This gives us the following signature:

(defmacro returning (var value &body body) ...)

The &body works like &rest and collects the rest of the forms passed to the macro into the body parameter. The let form way back at the beginning of this post provides a template for what we want the body of the macro to look like. If we take the original let form, quote it and substitute var, value and body in the appropriate places we get this:

(defmacro returning (var value &body body)
  `(let ((,var ,value))
     ,@body
     ,var))

Since body is a list we have to splice it into the current form using the @ operator. This does almost what we want. We can use it to rewrite our original initialization code using returning in place of the original let:

(returning u (make-instance 'user)
  (with-slots (first-name last-name) u
    (setf first-name "Foo")
    (setf last-name "Bar")))

There is a tiny problem with this code as well. The variable name and initialization form are hanging out there at the end of the line and there is no indication that they are related. We would really like to put a pair of parenthesis around both the u and the make-instance form so that it is clear they are related. Fortunately, we can “destructure” lists that are passed to macros as parameters. If we change the signature of the returning macro to include parenthesis around var and value we can write the code like this:

(returning (u (make-instance 'user))
  (with-slots (first-name last-name) u
    (setf first-name "Foo")
    (setf last-name "Bar")))

Finally, what if we want to leave off the initialization form? Perhaps we want to initialize the variable ‘u’ in the body. This is easily accomplished by adding &optional to the macro parameter list before the value parameter:

(defmacro returning ((var &optional value) &body body)
  `(let ((,var ,value))
     ,@body
     ,var))

This allows us to initialize u in the body of the form:

(returning (u)
  (setf u (make-instance 'user))
  (with-slots (first-name last-name) u
    (setf first-name "Foo")
    (setf last-name "Bar")))

It can be helpful to see how a macro expands. The macroexpand-1 function does exactly what we want. Here are the expansions of our returning macro both with and without the initialization form (the output has been cleaned up a bit):

CL-USER> (macroexpand-1 '(returning (u (make-instance 'user))
              (with-slots (first-name last-name) u
                (setf first-name "Foo")
                (setf last-name "Bar"))))
(LET ((U (MAKE-INSTANCE 'USER))) 
  (WITH-SLOTS (FIRST-NAME LAST-NAME) U 
    (SETF FIRST-NAME "Foo") 
    (SETF LAST-NAME "Bar")) 
  U)
T
CL-USER> (macroexpand-1 '(returning (u)
              (setf u (make-instance 'user))
              (with-slots (first-name last-name) u
                (setf first-name "Foo")
                (setf last-name "Bar"))))
(LET ((U NIL)) 
  (SETF U (MAKE-INSTANCE 'USER)) 
  (WITH-SLOTS (FIRST-NAME LAST-NAME) U 
    (SETF FIRST-NAME "Foo") 
    (SETF LAST-NAME "Bar")) 
  U)
T

Making ActionWebService work with Edge Rails

phil — Thu, 31 May 2007 21:04:47 +0000

I have a Rails application with a small SOAP service that is used for integration. Today I updated the site to Edge Rails and noticed a ton of strange errors related to ActionWebService. The first error looked like this:

Expected app/controllers/my_controller.rb to define MyController

I remembered hearing that ActionWebService was going to be moved out of Rails core, and I eventually found changeset 6550 from about 1 month ago where AWS was removed from the default load path. The ChangeLog suggests adding vendor/rails/actionwebservice/lib to config.load_paths. It turns out this is not enough. The app/apis directory also needs to be added to the load path and ‘action_web_service’ needs to be required.

Here is an excerpt from my environment.rb after making these changes:

Rails::Initializer.run do |config|
  # Add additional load paths for your own custom dirs
  config.load_paths += %W(
    #{RAILS_ROOT}/app/apis
    #{RAILS_ROOT}/vendor/rails/actionwebservice/lib
  )

  # ...more stuff...
end

require 'action_web_service'

This will get AWS working again, but functional tests may fail with the following error:

NoMethodError: undefined method 'invoke' for #

There is one more change that is necessary: add require 'action_web_service/test_invoke' to the top of test/test_helper.rb.

ENV["RAILS_ENV"] = "test"
require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
require 'test_help'
require 'action_web_service/test_invoke'