With the advent of multi-core CPUs concurrency is becoming more important, but is traditional threading the way to go? The problem is that traditional threading is very difficult to get right. I am sure that are some cowboys out there who will respond to that last sentence by saying that “real programmers” don’t have a problem with current threading models. Machismo aside, I have seen many smart, capable programmers write multithreaded code that deadlocks or corrupts data (including one of the aforementioned “real programmers”).

I took the Sun Certified Java Programmer test in 2004 to see what it was like. There were a lot of threading questions on the test. I remember reading somewhere at the time that Sun felt many programmers got threading wrong and thus felt the need to make it one of the 9 major areas the test covered. Also, some of the most frequent problems that developers run into when writing Swing code are related to threading.

Last month, there was an article on InfoQ the provided a good overview of the debate about threading in Ruby. It all started when blogger Ciaran McCreesh wrote a post about how Ruby’s green (or user-space) threads and the difficulty of interfacing with 3rd party libraries that use native (or kernel) threads. In this case, Ciaran was discussing the problems that can arise when interfacing with GUI toolkits which often rely on native threads to provide a responsive user interface. The article points out that Smalltalk implementations typically use green threads and then discusses some of the strengths and weaknesses of both approaches.

There have been some discussions recently with regards to Ruby’s performance, and it seems that the lack of native threads is a common complaint. In fact, James Edward Gray II asked Matz and Koichi about threading in the third part of his Ruby VM Serial Interview and the first comment had this to say:

Ruby’s future is at risk of becoming irrelevant as 2, 4, and soon 8 core solutions become common place.

The first assumption here is that as the number of CPU cores goes up that concurrency will be more important. This is hard to argue, but the second assumption is that something radical must be done to handle these levels of concurrency. Some people argue that the something radical is native threads which would allow the VM’s workload to be spread across multiple cores. Interestingly enough, the commenter then mentions Erlang as a language that seems to get it right.

This is interesting because Erlang’s concurrency model is closer to the “shared nothing” concept that Rails espouses than to the heavyweight threading model of Java. It is also interesting because Matz and Koichi indicated in the interview that they are exploring a “Multi-VM” concurrency model that is similar to Erlang. According to Koichi:

As matz say, if we have multiple VM instance on a process, these VMs can be run in parallel. I’ll work on that theme in the near future (as my research topic).

Erlang uses lightweight processes that communicate by passing messages. This approach is very similar to the way a Rails application is deployed on Mongrel or FastCGI: multiple independent processes that do not share state and communicate by passing messages via the database (if at all). I have a production Rails application that runs on a Dell server with two dual-core Xeon CPUs. We currently have 9 Mongrel instances running and all of the cores stay busy when the site is under load.

One drawback to the Rails approach is that communication is difficult and spawning new Mongrel instances is not fast or easy. Erlang processes have other advantages as well; from the Erlang FAQ:

Erlang processes are very lightweight, much lighter than an operating system thread. Switching between Erlang processes is typically an order of magnitude or two faster than switching between OS threads.

Each Erlang process is garbage collected separately. This avoids the (relatively) long delay which would occur if there was a single large heap for all processes.

Lightweight processes in Erlang also give rise to a novel error handling technique. In languages like Ruby and Java you do everything you can to avoid exiting the program unexpectedly. Typically, an application will throw an exception which will unwind the stack looking for a handler that can deal with the error. While this approach is much better than the earlier practice of having to check return codes for errors, it is still less than ideal. If the code that raises an exception is too far from the handler it can be difficult for a programmer reading the code to understand how the errors are handled. Also, the stack unwinding can be expensive and it may be difficult to continue after many types of errors.

In Erlang, you simply let the process die and a supervisor will start a new process and log the error. This seems to achieve the goal of exception handling (separating error detection and error handling) without the drawbacks.

I didn’t really intend for this to be a post about how Erlang is a great language. In fact, I have very little experience with it myself. The point I want get across is that there are multiple ways of solving many problems and often the accepted way isn’t the best way. Concurrency is going to be more and more of an issue as CPU cores scale up, but I doubt that threading will provide the best solution. Erlang shows how a fresh approach can get great results and I hope that Koichi follows through on his Multi-VM idea for Ruby.

Update: This post generated a surprising amount of interest. There is a short discussion on programming.reddit.com and murphee has an excellent followup post. Finally, in the comments Andre points to a great resource on concurrency and Erlang.

Tags: concurrency, erlang, java, performance, ruby, threads