Re: [re-online] Requirements vs Agile User Tests

From: Brian Marick (
Date: Fri Apr 11 2003 - 07:17:18 EST

On Wednesday, April 9, 2003, at 09:47 AM, Jane L Huang wrote:
> "Agile methods have taught us a great deal about automated testing,
> both at the unit level and at the system level. Agile methods have
> also taught us that well written automatic tests are better at
> specifying requirements than prose documents are."
> He then goes on to claim that "a good automated test can also be a
> good requirements document."

I'm another person who's said things like that, but I'm increasingly coming to think that's not the right way to talk. I don't like the words "specifying" or "document". I think they sidetrack discussion.

I don't think that the agile people intend the test suite to mirror or represent an understanding of the problem to be solved. (I think most people who write requirements documents do intend that.)

I think the agile people intend the test suite to provoke correct actions over the course of the project (including during maintenance). One of my mottos is "An effective set of tests is one that provokes a programmer to write the right program."

Does that require that the programmer "truly understand" the problem? Maybe. But understanding isn't the goal. If the programmer could write the right program without understanding the problem, that would be fine.

To the extent there is understanding, I think it's misleading to talk as if it were contained in the test suites and transmitted to the programmer who reads them. The tests provoke understanding by participating in a whole range of activities: talking to the business experts to get enough detail to write a test, talking about old tests with the business experts, making code changes and seeing what tests complain, talking about those complaints with the business experts and other programmers, getting annoyed that a particular set of tests fails a lot and wondering why (Ward Cunningham has a wonderful story about that), reading a test and trying related ideas by hand, being puzzled by code and hunting down the tests that exercise it, and so forth. And, whenever the tests and the people around them prove insufficient, reading other documents like requirements documents. (I think of requirements documents as commentaries on the tests, explaining what might otherwise be puzzling.)

"What would Plato have thought of test-first-design?" <> "Requirements and the conduit metaphor"
< conduit>
"The conduit metaphor (continued)"
< commentaries-2>

And, behind all of this, I'm afraid, the pragmatic philosophers like William James, John Dewey, and Richard Rorty (_Philosophy and the Mirror of Nature_).

Brian Marick
Consulting, training, contracting, and research Focused on the intersection of testing, programming, and design,,

I'm program chair or cohost of these events: PLoP: <> Analogy Fest: <> Please join me.

To send a message to this mailing list send it to To unsubscribe from this mailing list, email with the message `unsubscribe re-online' in the BODY of the mail.

This archive was generated by hypermail 2.1.6 : Mon Apr 14 2003 - 09:00:09 EST