I interviewed David Heinemeier Hansson in Chicago last week about some of the technical and cultural trends that are affecting Ruby on Rails. I started out by asking him about Chicago, the home base for 37signals, and I then ask him some questions about how the Rails culture and community is evolving. We discuss Agile and some of the technologies that are catching his attention. Click on the video below to watch the High Definition version of this video through YouTube, or read the following.
Or, you may download the file.
Tim O'Brien: We're in Chicago, in Wicker Park, which is in a neighborhood also called Wicker Park, which is recursion.
DHH: It is a kind of recursion. I think the neighborhood took its name from the park itself. So I think that's how it started out. Its weird in the sense this area can't really figure out what it's called, whether it's called Bucktown or whether it's called Wicker Park. So most people even go for the even longer version which is "I live in Bucktown/Wicker Park." It has somewhat of an identity crisis it seems.
TOB: Speaking of Chicago, what made you choose to move to Chicago? You moved here from Denmark?
DHH: I did live in Denmark pretty much all my life. I moved to Chicago about 3 years ago and I moved because of 37signals.
The company was founded here in Chicago and has its base and half of the employees are here in Chicago. Jason Fried, my partner in the company, is living here. Everything came together 3 years ago, I was done with school, I had an opportunity to come here, my girlfriend wanted to study at a university here. So we thought hey this would be a great opportunity to come here. Everything was kind of set up and was kind of easy.
TOB: How have you found Chicago to be for technology and innovation? Does it have a rich culture of technology? Is it the same thing as San Francisco?
DHH: I think its very different from San Francisco. Its great because its different from San Francisco. I think it's great because I don't have a whole lot of sense of the scene being this place. Chicago, for me, is not about technology, per se, which is why it works so well.
I don't think being in space or in a place that is primarily about technology is a good thing, in fact. I think that is there is a lot of "group think" going on the on West Coast, and a lot of bad memes that can "infect" you when you live there.
Chicago, in some way, is more of a neutral zone. There's not a huge start-up culture, not a lot of tech parties all the time that you can go to, no ton of VCs just walking over to have lunch with you. I think those are all pluses, net positives. The thing about Chicago is this sense that its a city where you just get down to work, and I really like the fact its not about all the extra stuff. Its about just businesses that are built and are profitable. You get away from the fluff that I tend to see a lot when I go to the West Coast. The companies that come out of that, companies that are founded and operated here, have a more "real" sense. Chicago feels more real, a lot less fake, a lot less polished, a lot less... fluff, for lack of a better word.
TOB: You recently started a start-up school for Y-Combinator.
DHH: Paul Graham, I don't know whether or not he calls himself VC, probably not, but he has this early stage angel fund where he takes in a bunch of teams every spring and fall. And start-up school is a conference for people like that, people who are starting new companies and doing it with Y Combinator money.
So I got invited to speak there, and I think that it was a great opportunity for me. I like being in places like the West Coast to provide an alternate opinion, let's say it like that, on a number of topics.
I certainly did a start-up school where the rest of the conference was, in my mind, very heavily tilted towards the "let's get to VC, let's get eye-balls, let's get big, let's get bought," kind of approach to company building, which I'm personally not a huge fan of to say the least.
So, I offered an alternate idea of how to build a company, how to grow a company, how to not sell your company, how to make it more about real stuff. The basic premise of the talk was how to get rich online -- which had 3 factors - having a desirable product, the secret sauce being having a price for that desirable product, and thus making a profit without going all the opposite routes you could go, like taking VC money and relying on advertising or whatever, just charging end-users. Its an underrated model as a whole on the web 2.0 space especially in the West Coast.
TOB: Let's talk about Twitter. Do you feel its fair for people to hold Twitter as the example for Rails scaling, or not scaling. Could you just talk a little bit about the indirection there, Rails doesn't scale because Twitter doesn't scale? Prove or disprove that.
DHH: Sure. Any application, whether it scales or not, usually does not have a whole lot to do with the framework or the programming language or any of the other individual tech pieces in isolation. It usually weighs more on algorithms, architecture or how its all being put together. When people talk about Twitter not scaling, which by the way is false -- Twitters is obviously scaling, it's growing every single day. I use it every single day, lots of people use it every single day. It has problems, sure, but it's not that it's not scaling. Not scaling kind of implies that the site went off the air. Twitters has never more powerful than it is today, never been growing more faster than it is today. Its working regardless to the problems they're having.
As whether their problems pertain to Rails web frame or anything, it's always just an irrelevant question. To me it serves as a filter. If people make that connection in their mind ofTwitter having scaling issues thus Rails can't scale, to me, it tells more about that person than it tells me anything else. It just tells me they don't have a very, I wouldn't even say deep understanding, but would say they don't even have a shallow understanding of technology.
Its a basic misconception. that's not interesting to me. In any ways, I don't care. If that is the position that you have on the opinion you use that you wouldn't want to use Rails because Twitter's having some issues, well so what! I won't loose sleep or money or anything else over that. Everybody shouldn't use Rails. So if there's some percentage of people who take that as their reason not to use Rails, all the better.
TOB: So when someone runs for office in this country, they do a lot of polls and find out negative and positive sentiment. Rails was sort of built as a reaction to heavy enterprising systems. My take is that you unseated a couple of big titans. Do you think that created a population of people that are "out to get" Rails and they say well look at Twitter?
DHH: Totally. I think that's what more of what Twitter is about. Twitter is about finding some rock or stick to beat Rails over the head with from people who have had a grudge all along. There are plenty of people like that. I think it comes with the territory of rapid popularity. I don't think you can mention pretty much anything, whether it's an organizational technique, or whether it's technology, or a cause or anything that has rapidly grown in popularity that hasn't had some sort of backlash or hasn't had some sort of hater camp. I think in many ways, that just certifies that what we're doing is interesting.
If nobody is hating it, you're typically not doing very interesting work. You're typically not doing anything that's different from what already is; you're not upsetting the status quo. I take every outburst of hate or displeasure or whatever you want to call the push back against Rails as part of the validation for what we're already doing. I'd much rather have people talking about Rails in the sense that "oh, it can't scale because of Twitter," then nobody saying anything at all.
The counter weight to that is usually there is a huge camp of really, really passionate people. It kind of comes (as Kathy Sierra talks about) like balancing out the universe. You have a bunch of super passionate users who love what they're doing with the tools against Rails. You can't have order in the universe without having some sort of other camp that pushes back against that. I think it's an absolute natural progression and I think it has to happen with any kind of technology that grows and has any kind of popularity. It has to have those counterbalances.
TOB: Looking at the curve of technology, the early adopter, people who follow on the heels of people who blaze the trail, so to speak, where is Rails now?
DHH: We're definitely, in many ways, out of the early adopter phase. That means a couple of things. It means that more people are obviously using it. It means people who traditionally would not play around with technology for technology's own sake, are getting interested. But it also means that the people who are playing around with technology for technology's own sake can somewhat lose interest.
These early adopters, who were with Rails originally, were interested and are interested in new technology. Some have definitely moved on and there are other newer, shinnier things out there. Rails is 5 years old; I started working on Rails more than 5 years ago.
Its obvious not everybody will have the attention span, the interest or whatever you want to call it, to stick with it all along. They gonna find other things and they're going to be interested in that and move on, which I think is totally great. I don't think anybody being interested in Rails and then moving on is not necessarily a reflection on either the person or the framework. We have different interests at different times. A lot of these people from the early days helped Rails tremendously to get going. There are no hard feelings or anything else just because you find something that is more interesting to you. I think that's perfectly natural.
People coming on to . . .community now, people being interested, are a different kind. I just hope some of the shared culture persists, though. Even though we change some of the characters in the community, you can still maintain somewhat cohesion; somewhat of the same ideas or ideals.
TOB: What is the culture? From the outside it appears to be somewhat bohemian, iconoclastic. Rails in its introduction was a very successful challenge. Now it's moved to something that's established. Talk about the culture and how it's changing. Do you see it changing? Is it becoming less of a culture challenge and innovation and kind of clastic bohemian weirdness and more of a corporate [culture]?
DHH: I think you're definitely right. Rails started out as a rebel's cause. It was a rejection of a lot of dogma, a lot of established thinking of how things were supposed to be in other languages and camps, a lot of push back in that and thus a lot of conflict. People used these ideas, these established approaches. Not all of them took the challenge equally well.
Now there are a lot of ideas that Rails started out as being very controversial, convention over configuration to take one example. It;s almost now establish wisdom in a lot of camps. That makes it lose a little bit of it's edge; but I think these things will keep popping up, keep being new arguments where Rails will take a position and thus be somewhat rebellish.
So, one of the more recent ones is the whole thing over web services. We go to use the heavy, cumbersome stack of SOAP and WS-Deathstar and what else have you, or we're going to go with the lighter way or best approach and so on.
TOB: I'll interrupt you. It is possible to use SOAP from Rails with Soap4R, but it is an awful exercise.
DHH: It is and I don't wish it on my worst enemies. A lot of things we do and the opinions we hold are for people who get to pick. I don't look down on people who have to use SOAP or whatever because that is what they have to do to hold their job.
Its people picking new systems or the green field choice/approach as to what what technology to use, that I can't understand that people will pick these most of the time.
Rails will continue to take opinions and will continue to be a source of contention regarding that issue. At the same time, I think its' very inward looking to consider Rails as this outward, established technology, because its very well established over a very small period.
The are still so many more people doing C#, Java, PHP, whatever, is an endlessly larger group than the people who are doing Rails. Even though we've been around for almost 5 years, we have a huge user base. Its a "drop in the bucket" compared to this established scene/technologies that represent in many ways the counter of what Rails is all about. There is so much more that is still to be done.
Its easy to lose sight of that when you use the same blogs, going to the same IRC channels, go to the same conferences, etc., you have a tendency to think "oh, everybody is like me, everybody is using Rails, this is so established", but its really not. Its a small pocket of everything that is out there.
TOB: Can you talk about how Iron Ruby or J-Ruby are either co-opting the energy of .Ruby?
DHH: So its actually fun because in many ways, Sun started out as the evil empire in terms of practices, tools and approaches they pushed on people.
TOB: Many would still call them the evil empire.
DHH: I think they are, but the thing that I've realized and a lot of other people have realized is that there's not just one empire; the empire is not united. There's many fractions inside of Sun and they're doing very nice work. I think that Charles Nutter and the whole JRuby team, I hung out them a bunch of times, at a bunch of different conferences, they are awesome guys, doing interesting, good work and they're on the side of let's say "Ruby versus Java."
But, I think they're doing a pragmatic thing. There are lots of people out there who, work in environments where Java is the established technology, and they have have a huge base of existing infrastructure that they are forced to work with. It's not like there's really much of a choice.
TOB: In your start-up camp talk, you specifically talked about companies that pay people to design Frameworks and API's -- that's a big mistake.
DHH: In my mind it is. I think it's a little different on things there are very established. For example, re-implementing Ruby in Java is not a design exercise in the same way as designing a brand new API or a new language or something else like that. I think when you do that kind of work, not re-implementation, not working against spec, so to speak, just working on the frame work, etc., is not a good approach. I don't like that at all. But re-implementing Ruby in J-Ruby I think that's fine. I think you can totally work on that full-time. The scope is defined and you don't have to make those choices there are so hard to make when you're not working on something that is real.
This is an abstract distinction, but I definitely have to believe that framework such as Ruby and Rails can not be made by people who are working on it full-time. You can not design that kind of thing if you're not working on real stuff with real constraints at the same time. And I don't think you have to. Its much more enjoyable to be both the consumer and the producer at the same time. You get to make better decisions, have more fun at the same time.
TOB: Robert Lefkowitz spoke at the Open Source Convention in Portland about 3 weeks ago. He gave a great speech about the difference between Techne and Praxis, a difference between doing and the "leadership of man", the division between engineering and arts and sciences. Is it that Rails and other other successful technologies are designed in an "engineering" context with a real set of requirements and a real product to support, that makes the difference, versus just working on an API or a framework in abstract, almost scientific [sense]?
DHH: I definitely think that in my mind I grew up in scientific and engineering are somewhat together. I think they're the wrong paradigms for something like Rails and API development. In my mind, it way more the arts and crafts... the craftsmanship approach to things, that you are building tools as you are using them, and that's the only way to arrive at useful tools.
I don't like the term engineer. I don't consider myself as an engineer. I think that . . .
TOB: What would you call yourself if someone asked you to choose a word?
DHH: Craftsman, is much more an appropriate term and paradigm for what we're doing. I think that's embodies so much more the combination. Its halfway art, there's some science involved, there's some engineering involved, it is bound together by a different kind of pragmatism to get the work done.
TOB: Let's talk about process a bit. You're in Chicago, which is the home of ThoughtWorks. I know Martin Fowler is considered the grandfather of Agile (although he might be offended I called him a grandfather). Agile has seemed to latched on to Rails as one of it's demonstration technologies.
DHH: I think it's the other way around really. Rails latched on to Agile actually. In some ways its an implementation. so there's a lot of the ideas in Agile I thought were hopefully incompatible with the tools they were being used with. A lot of Agile was done using Java, etc. Sure it can be done. I don't think those tools and platforms...
[Interrupted by a somewhat aggressive flock of birds - a sign of an approaching thunderstorm...]
...don't really capsulate or try to embody the ideals, culture, thinking of Agile.
So Rails tried to explicitly say you should do testing. There's no way to opt out of the testing stubs that have been generated when you try to use the Rails generators.
There is a lot of ideas from Agile that we try to embody in the implementation of Rails. And I think that makes a difference. There's a lot of choices you can make in creating your tools/platforms that correspond to certain underlying thoughts and paradigms. And I think that Java and the C-sharps, etc. of the world embodies a different era, a different paradigm style, a more "waterfall-ish" approach, project management that is not very compatible with the Agile ideas.
TOB: There is a lot of cross-talk. Do you talk to people at ThoughtWorks about process and methods? Chicago has a culture of Agile people.
DHH: It hasn't been so much in Chicago. I talked with and hung out with the Thought Works guys and Martin Fowler a lot of times. We share a lot of the same ideas; and at the same times we work in different spheres. I'm in product development, they're doing consulting. And in some ways, those are different territories and different approaches, which work well in different scenarios. There is a lot of overlap and I think those guys have it right on a lot of issues. I'm happy they're pushing Agile and the kind of thoughts on and into "enterprisy" scenarios because a lot of their customers are very "enterprisy" in the negative sense of that word.
Its good that they're trying to spread these means, thoughts, and hopefully in a few decades we will have gotten away from the bad stuff that has been going on.
TOB: Once that happens, what's next? That's a question for you. Are there technologies that are catching your eyes that are not related to what you do where you're saying "that's interesting, I need to look at that for inspiration for Rails?"
DHH: There are a lot of technologies that are popping up that are interesting to me. One of the things that was controversial in the beginning of Rails is the role of the database. I've long treated the database as mainly being a big hash. Its interesting to see that recently things like CouchDB or BigTable, and other implementations treat the database as a big hash. Or implement a big hash as a database are popping up and that's very interesting. I haven't done a whole lot of work with it yet, but I think its interesting and very much in line with what we've been saying in the Rails camp.
I think another thing we were actively using and have changed is Memcached, especially the whole idea of using expiration in keys or key generations, whatever you want to call it, where you're not doing... one of the last hard things in computing, I forget who told me about this. There's concurrency and then there's cache expiration. In many ways Rails tries to sidestep both of them, tries to treat concurrency as not an end-programmer concern and let's get rid of Cache expiration and just deal with caches as identified by keys, I think memcached enabled that whole move. That's interesting to me.
TOB: I have a question about things like BigTable and the way Google does all of their systems. Changing the paradigm of an application, 99% are built to service and support a relational database. Are there tools available for people to work towards something else, to work towards the CouchDB model, without having to fit a square piece into a round hole?
DHH: No, I think that's one thing that hasn't been worked out quite well. I still don't have a very clear picture of how would I convert Basecamp into something that runs on CouchDB. How do you deal with permissions in scenarios like that. There are a lot of unresolved questions, which is why I'm interested in this. There is a piece of technology that shows great promise but all the questions have not been answered yet. It is on the horizon and that's why were looking at it. If all the questions were already answered, it would already be established.
TOB: BaseCamp and Write Board, you scaled these horizontally, is that correct?
DHH: Well, we kind of do both. On the application side and web side, we scale them horizontally. If we want more capacity, we add more application servers, more mongrels, more web servers, etc. On the database side, we're scaling up. We just bought a even bigger box to handle the database for BaseCamp.
TOB: There's only one or is there a cluster?
DHH: There is one that runs primary and we have a slave that is a back up machine.
TOB: You're doing a binary replication between the two?
David. Yes. Only for backup purposes. All reads and write go to a single "big ass" machine. And, we do that because it is easy. We'll do that as long as it remains easy and economical. Currently, it still is. It is a 128GB machine in terms of RAM and by the time that we need more, then we can hopefully get a 256GB and can continue to do this. At some point, it might not be feasible any more. There are tons of applications where it is not feasible anymore and then they have to do things like sharding . . .they have to do other things that make scaling beyond a single box possible but painful.
TOB: What Rail sites that you know of shard and how do they do it?
DHH: Good question. I remember being in conversation with others about this, but there's not a whole lot of applications that really need it. Basecamp is a pretty big application, have a lot of customers, a lot of usage, but we don't even need to shard yet. As technology moves on, hardware gets cheaper and cheaper. In my mind, you don't want to shard unless you positively have to, sort of a last resort approach. So I'm not sure!
[Rain interrupts... and then a Chicago Park District employee tells me to "shut off the camera". Evidently, Chicago doesn't allow filming in a public park without a permit. The park employee told us to "stay put" while he went to call his Supervisor. I had other ideas. David and I ran from the scene of the crime on foot, and I sped out of Chicago city limits to download this unauthorized recording to my MacBook.]
TOB: If you get the idea that that interview was not complete, then you're right. We had about 10 more minutes to go. I was going to ask questions about the future direction of Rails and what is beyond rails but also ask him to comment on some of the interesting applications.
We couldn't get to those questions because a Chicago Park District employee approached me and asked me to shut off the camera. I had assumed that talking about Rails was legal. But evidently one needs a permit to talk about Rails in a public park in Chicago.
Editor's Note (Tim): The fact that two people can't meet in a public park and record an interview without a permit is disturbing. Even if you are a student photographer trying to take pictures in a public park, you need to request explicit permission from the park district, pay a permit fee of $25, and maintain $1M of liability insurance. My problem isn't with the fee, it is with the idea that anyone needs to ask permission to record a conversation between two individuals on public property. Chicago's permit requirements were challenged in 2002 at the US Supreme Court - Scalia delivered an opinion supporting the permit requirements. Predictably, that case is about a NORML rally and the requirement that more than 50 people can't assemble in a Chicago public park without a permit. While that Supreme Court opinion says that the permit process is not prior restraint on free speech, the fact that a city employee walked up to David and I and told me to "shut off the camera" certainly felt like a violation of the First Amendment. The irony of that Chicago has started an aggressive push for greater surveillance; in other words, Chicago can videotape you 24/7, but don't even think of bringing a camcorder into a public park to record a conversation. The other irony here is that the interview had some very positive things to say about Chicago.
David and I will likely finish the conversation at the Web 2.0 Expo in New York this September.