Top-to-bottom Testing in Ruby

kev — Fri, 14 Oct 2005 16:56:00 -0400

Presented by Francis Hwang http://fhwang.net/top_to_bottom/

First Toy Example fibonacci.rb

Testing is harder in the real world

Complexity
External components
Side-effects
- You don’t want to send a thousand emails to test
Speed
- You don’t want to drop and rebuild your schema for every single test

The Quality Elbow

You initially get lots of quality for your cost
As it goes on, you get lest quality from your code
Similarly, testing degrades as code does

Message

Simple mailer.
Just sends stuff down the pipe

For testing we don’t want to send real emails

MockMailer

Same methods, but its really really stupid
Adds emails to emails sent

We can use this to simply check our mailers.

This is too coupled however, so a message ought to automatically know what mailer to use in tests.

So we use a user, and pass in the mock mailer to the user. This still isn’t right, as its not the user’s job.

We need to bite the bullet and imitate a global variable. We’ll use contextual service.

If we need to use a mock with several outside services, we can set those up in setup.

The issue with mocking a sql query is that you’re basically writing a full sql parser.

Back to slides…

Tradeoffs of Complex Mock classes

Upsides
- Speed
  - Incredible speed increase
  - Not spending 3 hours to run sql tests
- No side-effects and no cleanup
  - No need to touch the db
Downsides
- Indirection
- Possible bugs in mock class
- Time spent to build mock

You could mock anything!

Filesystem - MockFS
Command line user entry - EasyPrompt
email
network services

Dynamicity is your friend

Overriding

May not be a good idea
Undefines FileUtil, and Dir and File
Also require

Further improvement

Test-centric libraries
- Try to include mocks in your libraries
Domain-specific test languages

Questions/Comments

DHH: Use transactions and still use the db. Saves speed. Run test cases in a transaction, rollback. Basecamp tests using db does 410 tests, runs in 60 seconds.

Austin: In 3rd code example, ContextualService, isn’t that just another way of doing dependency injection (needle, etc).

Response: Yup. I like the fact that the mailer manages its own instance.

DHH: At 0.10, Rails on Needle? Decided its not an approach, what you really want is dependency injection, without knowing anything about dependency injection. What you’re really interested in is referencing the class mailer and getting the mock back when you’re running tests. You can preload the load path so when you’re using tests it uses the mock directory before the real thing. The mock can then just be called Mailer.

Azlak: One downside is writing them.. libraries can generate mock objects dynamically.

Response: YMMV, mocking is a thing mostly on the logick of the thing, so abstracting out doesn’t work well. Most mock libraries work like message interception, but its like a black box. I personally like just writing them better.

Azlak: Mixing up mocks and stubbs, whats the difference? Mocks are actually black boxes that pretend to be something else. Rails has stubbs not mocks.

DHH: You can choose to override or not. Or turn into a whole black box if you like. You can use either mocks or stubbs.

Azlak: The major difference is that if you don’t specifically say its a mock, it blows up in your face.

Blah blah blah..

AC: If you want a mockfs, why don’t you just use a specific path as a sandbox.

Response: I have classes where they know the specific paths, so you have to intercept everywhere and it gets messy.

AC: If you’re hitting a real file system you have the potential to bork your tests in the future if things crash.

Response: You may also want to check if the hd is full.. etc.

AC: Can run mock tests without actually having the server running (mysql etc).

Response: Also nice to be able to change your structure without touching a database. Also, in theory, you could use tests to test if you’re working on code running on different machines…

Dom Tersen: Mock testing are good for tests, but they actually don’t proove the system works for real. Mock object testing doesn’t replace real testing.

Patrick: Who’s responsibility is it to write the mocks? Should we write them on the side, or should the capability to be mocked be part of every IO library?

Response: I’d be happy if library authors were writing mocks for their libraries. Those writing the library are usually in the best place to write Mocks.

I’ll clean this article up more as I have time. – Kev

Gluttonous: Top-to-bottom Testing in Ruby

Top-to-bottom Testing in Ruby