Powerset to Launch Front-end on Ruby

Posted by kev Thu, 21 Jun 2007 16:41:00 GMT

Spread the word on digg

Powerset is fairly well-known in the Ruby community, but there’s a certain amount of confusion as to what we use it for. As a consequence, I’m regularly asked what the front end is going to be written in, and just as regularly have to leave the question unanswered. But today I’m happy to announce that we are, in fact, launching our front-end on Ruby.

Cool, huh? For everyone’s sanity (and in avoidance of some of the flame wars to ensue), do note that we are going to be using Ruby (the language) but not necessarily Ruby on Rails (the web framework).

In the spirit of Powerset’s new found openness, I’d like to take some time to explain why we’re making this decision where others might not.

Why Ruby?

1. We’ve already got the brains

One thing we haven’t kept secret is that we’ve hired some of the best Ruby developers around. Our total number of day in day out Ruby developers is somewhere around 10, and I’m constantly humbled to be working with this team. We’ve got the people and they have the skills, so it makes sense to apply them.

The part of Powerset Ruby we could round up

2. Ruby is already being used throughout the company

We’ve always spoken in general terms about how much Ruby is being used internally, but let’s get specific: a substantial part of our infrastructure is being written in Ruby or being accessed through Ruby services. Our scientists use Ruby to interact with our core language technology. Our packaging infrastructure is Ruby. A big portion of our system administration work is all done with Ruby. Frankly, we as an organization use Ruby a whole heck of a lot.

Additionally, all of our product demos and prototypes are also in Ruby. We’ve got an interesting mix of Rails, Merb and Camping apps (depending on the scope of the project) connecting to tiny Ruby services which hook into our various back-end systems. Day to day, the majority of the product team is hacking in Ruby in some capacity.

3. We’re not worried about scaling

So, inevitably, whenever we talk about Ruby or Rails scaling these days someone brings up Twitter and its scaling problems in the past. Twitter is right down the block from our offices and I know several of the devs personally, so before we made a final decision I arranged a sit down with Twitter’s lead developer, Blaine Cook, to talk about the situation. Blaine was kind enough to let me bring along our Search Architect (and former search architect at Yahoo!) Chad Walters , our Head of Product Scott Prevost, and our COO Steve Newcomb, to poke and prod and get their questions answered. The simple fact is that Ruby wasn’t the source of Twitter’s woes. As it often happens with rapidly growing sites, they ran into architectural problems. Some design decisions don’t hurt until they reach a massive scale and at that point you have to rethink your approach. In an email he writes:

For us, it’s really about scaling horizontally - to that end, Rails and Ruby haven’t been stumbling blocks, compared to any other language or framework. The performance boosts associated with a “faster” language would give us a 10-20% improvement, but thanks to architectural changes that Ruby and Rails happily accommodated, Twitter is 10000% faster than it was in January

This is great news for Twitter, but even better for us because we don’t have the bottle necks that they’ve struggled with – databases, instant messaging servers, and regularly recycling cache systems – which makes scaling horizontally much much smoother. At that point, our scaling issue doesn’t concern Ruby. For a search engine, the front-end is largely just a templating system and the real work happens in the back when we process your query.

What does this mean for the community?

When writing this article, at some point I had to sit down and ask myself why anyone should care we’re adopting Ruby for the front-end. For me, it comes down to the fact that we’re good for the community as a whole.

First off, the fact that Powerset is deploying on Ruby means you’ve got one more high traffic site (potentially very high traffic) using Ruby in production. It’s one more case study, and one more example that Ruby as a whole is ready for the big show.

Personally, I think the more interesting and useful thing to take away from this is that as we do the heavy lifting, building up infrastructure around all the aspects of Ruby development and deployment in the company, we’re selecting large chunks to be open-sourced. I’ve got a list of things I’d love to put out into the wild (which is encouraged, and actually suggested by my manager. Man, I love this place) as soon as I can find the time. Already Tom Werner and Dave Fayram have pushed out Ruby to Erlang bindings and a sweet little (in-development) web server called Fuzed, I’ve gotten to hack at Merb, and a fair about of Rails patches have come directly from work in-house. Hopefully the community will be able to benefit from our code as much as we have.

Obviously we don’t have a search product open to the public yet, but we’ll be launching Powerlabs in September. In Powerlabs, you’ll be able to play with our products and give us feedback. If you want to keep track of what Powerset is doing, sign up.

Posted in ,  | 4 comments

Updates, Speaking, Writing

Posted by kev Fri, 06 Apr 2007 18:36:00 GMT

Hi guys. Gluttonous has been quiet lately, so I wanted to update you a bit on life in general.

First off, I haven’t yet talked about my work at Powerset. We’ve been quickly growing and I’m humbled by the type of people who join up on a daily basis. We’re about to hit 6 Ruby devs and 50 in the company, and for me it really feels like a dream.

That being said, I talk a bit more about what I’m actually doing at Powerset in an interview given over at the Working With Rails Blog in their series on Hackfest winners.

Additionally, I give an update on one of the reasons this blog has gone dry for a while at the end of the interview. You really shouldn’t miss this.

As far as speaking goes, I’ll be giving a talk on improving test coverage with Heckle at the Silicon Valley Ruby Conference in San Jose on April 21-22. The list of speakers is amazing, and I’m looking forward to the conf. If you’re there, come say hi.

Posted in , ,  | no comments

Link: Kathy Sierra Receives Death Threats

Posted by kev Mon, 26 Mar 2007 23:24:00 GMT

Death threats against bloggers are NOT “protected speech” (why I cancelled my ETech presentations)

I have cancelled all speaking engagements.

I am afraid to leave my yard.

I will never feel the same. I will never be the same.

– Kathy Sierra

So entirely not cool.

Posted in  | 2 comments

When Config Files Go Bad

Posted by kev Thu, 08 Mar 2007 22:40:00 GMT

It is often necessary to check the content of a `configure.ac' file,
but parsing it yourself is extremely fragile and error-prone.  It is
suggested that you rely upon `--trace' to scan `configure.ac'.  For
instance, to find the list of variables that are substituted, use:


     $ autoconf -t AC_SUBST
     configure.ac:2:AC_SUBST:ECHO_C
     configure.ac:2:AC_SUBST:ECHO_N
     configure.ac:2:AC_SUBST:ECHO_T
     More traces deleted

-- info autoconf

Posted in  | no comments

WHATWG WebApps 1.0 Spec

Posted by kev Fri, 02 Mar 2007 02:39:00 GMT

Specs with spunk make me optimistic.

1.2.1. How to read this specification

This specification should be read like all other specifications. First, it should be read cover-to-cover, multiple times. Then, it should be read backwards at least once. Then it should be read by picking random sections from the contents list and following all the cross-references.

Web Applications 1.0 Working Draft

Posted in  | 1 comment

REST Talk At SD.rb Is Live

Posted by kev Wed, 04 Oct 2006 23:48:00 GMT

The talk on REST basics I mentioned last month has hit the SD.rb podcast. I cover REST concepts and basics and Chris Abad has one immediately following on using web services (for free).

And if you haven’t seen Chronic, the lean, mean, completely awesome natural language date/time parser from sd.rber Tom Werner, do take a look at his presentation. It’s really a beautiful thing.

Update: You can grab the slides for my presentation here. I should have repeated the questions as they came in for the podcast viewers, but since I didn’t they were roughly:

  1. If browsers don’t support the other HTTP verbs, how does Rails pass the other methods like DELETE and PUT?
  2. Are there other HTTP verbs besides the CRUDy ones?

Posted in , ,  | 3 comments

The RubyConf Express

Posted by kev Fri, 08 Sep 2006 20:20:00 GMT

A few of the sd.rbers are planning to take the train to Denver next month. The trip will take about a day and a half. I’m planning to go. The view should be great and it’ll give me some time to work on my presentation if needed.

The train does go through LA, so if you’re in the Southern California area this may be right up your alley.

More information can be found at Patrick Crowley’s blog. So go check out the RubyConf Express!

Posted in , ,  | 2 comments

SD.rb Adds 4 New (Video) Podcasts

Posted by kev Fri, 08 Sep 2006 19:52:00 GMT

Today Chris Abad of the sd.rb podcast uploaded four new episodes:

In Episode 004: Numbers Tom Preston-Werner (of [Gravatar], Chronic) gives the first part of a series I hope he’ll continue called “Ruby is Awesome”. This talk focuses on interesting things about how Ruby handled numbers. Though this sounds really basic, he showed things that I had never come across and kept my attention. If you’re newish to Ruby this is definitely worth your attention. Go take a look.

Episode 005: ARTS Plugin is my presentation on using ARTS to build test driven RJS. This is from the second meeting of sd.rb so the first part is full of Rails basics. If you’re interested in seeing ARTS in action (used to build an ajaxified blog) you probably want to skip to about 1/3 of the way in.

Episode 006: Dynamic Domains features Chris Abad talking about his new application [Outlandish] and how they implemented dynamic domains.

Finally, in Episode 007: Rails Authentication I talk very briefly about how to use the acts_as_authenticated and restful_authentication plugins. At some point I begin rambling about how REST works, but that part is best left for when the podcast of last night’s REST presentations go up. I’ll point you there when it happens.

Posted in , , ,  | 3 comments

JRuby Team Picked Up By Sun

Posted by kev Thu, 07 Sep 2006 18:31:00 GMT

Wow. Just wow. Congrats guys.

Update: Tim Bray gives us Sun’s motiviations.

Posted in ,

SD.rb Meeting Tonight

Posted by kev Thu, 07 Sep 2006 17:13:00 GMT

If you’re in San Diego (or even Southern California) we’d love you to drop by the sd.rb meeting tonight. Chris Abad and I will be talking about RESTful stuff in Rails and Tom Preston-Werner will be debuting his new natural time parsing library, Chronic.

We have our meetings at the UCSD CSE buildng (room 1202). If you want to drop me an email at kevin dot clark at gmail dot com I’ll give you my cell in case you get lost.

I hear OC Ruby may be making a field trip. See their list if you want to get in on that.

Addtionally, several OC Rubyists and SD Rubyists are planning to meet up at Rock Bottom just off campus around 6:15 for dinner. Come hang out.

If you can’t make it the sd.rb podcast is always an option and I’m told you should be getting several updates from the last meeting in the next couple days. Keep a look out.

Posted in ,

Older posts: 1 2