Powerset to Launch Front-end on Ruby

kev — Thu, 21 Jun 2007 12:41:00 -0400

Powerset is fairly well-known in the Ruby community, but there’s a certain amount of confusion as to what we use it for. As a consequence, I’m regularly asked what the front end is going to be written in, and just as regularly have to leave the question unanswered. But today I’m happy to announce that we are, in fact, launching our front-end on Ruby.

Cool, huh? For everyone’s sanity (and in avoidance of some of the flame wars to ensue), do note that we are going to be using Ruby (the language) but not necessarily Ruby on Rails (the web framework).

In the spirit of Powerset’s new found openness, I’d like to take some time to explain why we’re making this decision where others might not.

Why Ruby?

1. We’ve already got the brains

One thing we haven’t kept secret is that we’ve hired some of the best Ruby developers around. Our total number of day in day out Ruby developers is somewhere around 10, and I’m constantly humbled to be working with this team. We’ve got the people and they have the skills, so it makes sense to apply them.

2. Ruby is already being used throughout the company

We’ve always spoken in general terms about how much Ruby is being used internally, but let’s get specific: a substantial part of our infrastructure is being written in Ruby or being accessed through Ruby services. Our scientists use Ruby to interact with our core language technology. Our packaging infrastructure is Ruby. A big portion of our system administration work is all done with Ruby. Frankly, we as an organization use Ruby a whole heck of a lot.

Additionally, all of our product demos and prototypes are also in Ruby. We’ve got an interesting mix of Rails, Merb and Camping apps (depending on the scope of the project) connecting to tiny Ruby services which hook into our various back-end systems. Day to day, the majority of the product team is hacking in Ruby in some capacity.

3. We’re not worried about scaling

So, inevitably, whenever we talk about Ruby or Rails scaling these days someone brings up Twitter and its scaling problems in the past. Twitter is right down the block from our offices and I know several of the devs personally, so before we made a final decision I arranged a sit down with Twitter’s lead developer, Blaine Cook, to talk about the situation. Blaine was kind enough to let me bring along our Search Architect (and former search architect at Yahoo!) Chad Walters , our Head of Product Scott Prevost, and our COO Steve Newcomb, to poke and prod and get their questions answered. The simple fact is that Ruby wasn’t the source of Twitter’s woes. As it often happens with rapidly growing sites, they ran into architectural problems. Some design decisions don’t hurt until they reach a massive scale and at that point you have to rethink your approach. In an email he writes:

For us, it’s really about scaling horizontally - to that end, Rails and Ruby haven’t been stumbling blocks, compared to any other language or framework. The performance boosts associated with a “faster” language would give us a 10-20% improvement, but thanks to architectural changes that Ruby and Rails happily accommodated, Twitter is 10000% faster than it was in January

This is great news for Twitter, but even better for us because we don’t have the bottle necks that they’ve struggled with – databases, instant messaging servers, and regularly recycling cache systems – which makes scaling horizontally much much smoother. At that point, our scaling issue doesn’t concern Ruby. For a search engine, the front-end is largely just a templating system and the real work happens in the back when we process your query.

What does this mean for the community?

When writing this article, at some point I had to sit down and ask myself why anyone should care we’re adopting Ruby for the front-end. For me, it comes down to the fact that we’re good for the community as a whole.

First off, the fact that Powerset is deploying on Ruby means you’ve got one more high traffic site (potentially very high traffic) using Ruby in production. It’s one more case study, and one more example that Ruby as a whole is ready for the big show.

Personally, I think the more interesting and useful thing to take away from this is that as we do the heavy lifting, building up infrastructure around all the aspects of Ruby development and deployment in the company, we’re selecting large chunks to be open-sourced. I’ve got a list of things I’d love to put out into the wild (which is encouraged, and actually suggested by my manager. Man, I love this place) as soon as I can find the time. Already Tom Werner and Dave Fayram have pushed out Ruby to Erlang bindings and a sweet little (in-development) web server called Fuzed, I’ve gotten to hack at Merb, and a fair about of Rails patches have come directly from work in-house. Hopefully the community will be able to benefit from our code as much as we have.

Obviously we don’t have a search product open to the public yet, but we’ll be launching Powerlabs in September. In Powerlabs, you’ll be able to play with our products and give us feedback. If you want to keep track of what Powerset is doing, sign up.

"Powerset to Launch Front-end on Ruby" by Kevin Clark

Mon, 25 Jun 2007 23:22:01 -0400

John: Short answer: yes. Long answer: you’ll have to talk to Dave and Tom about it. I’m always excited about it when they explain it to me, and I’ve always forgotten exactly what was so exciting the next week.

"Powerset to Launch Front-end on Ruby" by John Benjamin

Sun, 24 Jun 2007 15:50:08 -0400

Why create fuzed? Does it do something mongrel doesn’t?

"Powerset to Launch Front-end on Ruby" by Rick DeNatale

Fri, 22 Jun 2007 18:05:22 -0400

Absolutely. I’ve seen this same story before.

http://talklikeaduck.denhaven2.com/articles/2007/06/22/good-news

"Powerset to Launch Front-end on Ruby" by Gabe da Silveira

Thu, 21 Jun 2007 13:03:50 -0400

Yeah, the whole scaling debate is getting so tired. I can see how it was a legitimate question (among many) when Rails first got hot, but these days it keeps popping up as some kind of defense-mechanism from entrenched J2EE guys.

The bottom line is architectures scale, not languages.

Gluttonous: Powerset to Launch Front-end on Ruby