Gluttonous: Why Fixtures Suck (And How We Can Fix Them)

Why Fixtures Suck (And How We Can Fix Them)

kev — Mon, 07 Aug 2006 14:24:00 -0400

I hate fixtures in Rails. Sometimes I hate them because they slow down my tests. Most often I hate Rails fixtures because they make my tests brittle. Everyone that uses fixtures has had that moment where they add another bit to their fixture file for that new test and suddenly things are breaking all over the place. It’s ugly and arguably a broken system.

I think we can fix it. I’ve had several long discussions about this and most of what I’m going to preach here isn’t original. That being said, I think this might work so pay attention and tell me what you think.

Fixtures suck because it’s all or nothing. You can either grab all of your user data or not have any of it. It does cut down on duplication because you’re not redefining data for every set of tests. However, having all the extra stuff in the scope of your test is unnecessary and leads to brittle tests.

One option is to compromise. We could modify the current fixtures method to accept syntax like fixtures :users => [:alice, :bob], :articles => :ruby. This would allow you to get only the data you need. At the same time, the old syntax could be backwards compatible and you could grab all of the data if you wanted to. I generally find that when I’m writing tests it’s only with one or two pieces of data. With Rails, everything else in the database is just hanging out.

The downside to this new syntax is that the current way of structuring test cases would make this ugly. You don’t want to define fixtures for every individual test. It’s much easier defining fixtures by test case. Here’s where the BDD folks got it right. Test cases should really be broken up by what context they’re being run in. We shouldn’t have one test case for the model. We should have several test cases with tests divided by what data we’re working with. We get smaller and more granular sets of tests this way. It’s the testing version of DHH’s “everything is REST”. Every test is about context.

Do remember that more test files don’t mean more work. We’re already generating empty integration tests. What’s one or two more generators? Really, it’ll be easy. You’ll like it.

The other option is to throw off fixtures completely. I know of people who have already taken this step and they solve the fixture problem by writing helpers to create the data in code and then run them in setup. This seems like a viable option as well and would require no changes to Rails core.

It’s time to admit that fixtures, as implemented in Rails, are broken and ugly. We’ve been taught to demand beauty and elegance. Let’s make something better.

"Why Fixtures Suck (And How We Can Fix Them)" by Tom

Fri, 11 Aug 2006 17:06:09 -0400

Maybe I should illustrate my point in code.

How it works now: fixtures :users

What you’re proposing: fixtures :users => [:alice, :bob]

What I’m proposing: fixtures :users_smallgroup

The duplication I’m referring to is the fact that in the user.yml file, you’re going to have a record for alice, one for bob (and a bunch of others). In your test you list alice and bob again – that’s the duplication.

I know above you said that test cases are “supposed to be” split up by fixtures, and if we did that – you’re right, the duplication would be minimal.

I guess I’m not completely sold on the fact that test cases should be grouped by fixtures. I’d end up testing the same methods in two different places if I wanted to test them with two different data sets. It sounds like that would make it a pain to find things. Or, maybe I’m not understanding what the layout would look like.

So, that’s why I was thinking along the lines of allowing a few different fixtures per table, and then you could call upon them when needed without having to restructure your tests around the fixtures you need.

"Why Fixtures Suck (And How We Can Fix Them)" by Kevin Clark

Fri, 11 Aug 2006 14:35:46 -0400

Tom: It doesn’t involve any duplication of fixture detail. You do it once in each test case. I’m suggesting we split of test cases as they’re supposed to be split, by what fixtures they need.

"Why Fixtures Suck (And How We Can Fix Them)" by Tom

Thu, 10 Aug 2006 00:32:10 -0400

I’m not sure I like the proposed solution because it means duplicating a bunch of fixture detail (namely, the rows you’re interested in) in the tests.

What I’d really love to see is the relationship between tables and fixtures changed from 1:1 to 1:many. For example, I think 95% of my fixture issues could be resolved if I was able to create (for example) people_lotsofusers.yml and people_oneuser.yml fixtures, and use one or the other to represent the people table, depending on what test I’m running.

"Why Fixtures Suck (And How We Can Fix Them)" by floyd

Wed, 09 Aug 2006 12:44:56 -0400

Yes, I thought you were going to say fixtures suck because THEY’RE brittle. One little change to your model and you have to rebuild all your fixtures for that model and possibly other dependent ones. The best solution for this is a plugin like Geoff’s:

http://nubyonrails.com/articles/2005/12/27/dump-or-slurp-yaml-reference-data

"Why Fixtures Suck (And How We Can Fix Them)" by floyd

Wed, 09 Aug 2006 12:44:26 -0400

http://nubyonrails.com/articles/2005/12/27/dump-or-slurp-yaml-reference-data

"Why Fixtures Suck (And How We Can Fix Them)" by Luke Redpath

Mon, 07 Aug 2006 17:35:59 -0400

“I personally don’t care how long my tests take to run.”

Tests should always run as fast as they possibly can. If your tests dont run as fast as possible then you aren’t likely to run them all as often because it becomes a chore.

"Why Fixtures Suck (And How We Can Fix Them)" by James H

Mon, 07 Aug 2006 17:16:48 -0400

I was thinking that perhaps what we really need is a way of setting up mock data based on the validations for our models, and the schema. By basing mock data on the schema and validations, we can create fixtures (or what have you) designed to actually break our apps! Perhaps it’d be as simple as running a Rake command to generate said data. Even then if the skeleton of a fixture entry could be generated by reflecting on the schema, this would save time too.

The thing I hate about fixtures is that sometimes it’s a bit of a pain assembling the fixture data when one fixture relies on another. For instance, perhaps a Post model references an author_id and we want to test this association. It’s a bit of a pain to set them both up.

I personally don’t care how long my tests take to run. I just want a mechanism by which I can see if my changes have borked some part of my app along the way.

"Why Fixtures Suck (And How We Can Fix Them)" by Lourens

Mon, 07 Aug 2006 17:01:02 -0400

Or you could just load them @ once with ‘rake RAILS_ENV=test db:fixtures:load’ and drop the following rake task in lib/tasks:

module Rake class Task def remove_prerequisite(task_name) name = task_name.to_s @prerequisites.delete(name) end end end

Rake::Task[‘test:units’].remove_prerequisite(‘db:test:prepare’) Rake::Task[‘test:functionals’].remove_prerequisite(‘db:test:prepare’)

Relieves headache of fixtures order, which ones to load etc., but we’re using this for a subdomain_as_account app for which an Account model has 30 odd model dependencies.Still no silver bullet though as fixture changes requires a full reload prior to testing.

"Why Fixtures Suck (And How We Can Fix Them)" by topfunky

Mon, 07 Aug 2006 16:51:21 -0400

My problem has been that unloaded fixtures cause problems or pass without testing every line of code that they should.

If you have a template that iterates around an array of objects and that fixture hasn’t been loaded, then the contents of the loop will never be executed. It’s sometimes hard to remember what fixtures are used by a specific test.

I’ve solved this by just loading all fixtures once at the beginning of the test cycle, but it does slow things down.

"Why Fixtures Suck (And How We Can Fix Them)" by matt lyon

Mon, 07 Aug 2006 15:36:14 -0400

here’s an idea:

with_fixtures => [:posts => [:today_first, :today_second], :authors => :all do 
  def test_todays_posts
    # this test would have only the above referenced fixtures...
  end 
end

"Why Fixtures Suck (And How We Can Fix Them)" by Luke Redpath

Mon, 07 Aug 2006 14:56:16 -0400

Good post. Of course my preference is throwing out fixtures all together and writing my own helper methods with Ruby. I’ve seen massive speed increases since throwing out fixtures.

For unit tests, they generally aren’t needed – simple helper methods suffice for creating related objects (ideally we’d use mocks but activerecord barfs on mock objects).

For controller unit tests (sorry functional tests..whatever ;) ) they are slightly more useful but nothing that can’t be fixed by promoting the use of mocks – I highly recommend Stubba:

http://mocha.rubyforge.org/

For those of you not using rSpec yet (and why not???) then I highly recommend FlexMock, though there is also Mocha (see above) which is like Stubba’s older cousin.