Copyright © 2015 jsd

Changing One Variable at a Time ... or Not
John Denker

1  Introduction

The idea of changing One Variable at a Time (OVaaT) is also known as changing One Factor at a Time (OFaaT). Sometimes we write OV@T or OF@T. No matter what you call it, it’s not a very good idea.

Example: Consider the famous “Twelve Coins” puzzle (reference 1). The obvious approach is to weigh each coin separately, but this is very far from optimal. The optimal solution requires arranging and re-arranging multiple coins. This is an interesting challenge at the middle-school level.

Example: Simplicio has a car where the tires are getting old and worn-out, and they were never very good tires to begin with. The question arises, would new tires work better? He decides to test this hypothesis. In obedience to the OVaaT principle, he changes one tire at a time, and observes the results. It turns out that no matter which tire he chooses, replacing that one tire makes things worse. The ordinary driving is marginally worse, ordinary braking is worse, and braking in the most demanding situations is very much worse and indeed quite dangerous. On the basis of this evidence he decides not to install any new tires.

Of course his logic is completely wrong. Installing four new tires all at once makes things markedly better.

Example: At the undergraduate level, one could compare Bevington’s GRIDLS versus GRADLS (reference 2). The latter is far from optimal, but the former, which changes only one variable at a time, is worse.

Example: When piloting a twin-engine aircraft, you almost always want to use both engines together. Each throttle could be considered an independent variable, but if you advance one throttle at a time, you will be very unhappy with the results.

Therefore I find it quite strange to find sweeping statements in favor of changing only one variable at a time, such as the appalling reference 3 through reference 8, and many many others.

Consider the contrast:

The OVaaT approach is favored by non-experts, especially in situations where the data is plentiful and cheap. It may be that the mental effort required to conduct a complex multi-factor analysis exceeds the cost of obtaining extra data, in which case OVaaT might be expedient. Inefficient, but expedient.   OVaaT is vanishingly rare in professional experimental and statistical situations. Indeed, the fact that the data is precious and needs careful analysis is probably why the job was assigned to The Professor and not to Gilligan.

OVaaT might make sense as a pedagogical starting point but never as an ending point.  

Also, OVaaT is harmless and convenient in situations where there are only a few variables, and the variables are naturally uncorrelated. For example, in the introductory physics class, you can study a pendulum one variable at a time. That is, you can vary the mass separately, vary the length separately, vary the amplitude separately, et cetera. The system is so simple that there is little to be gained by varying more than one variable at a time.   In the real world, systems are rarely so simple.

For young children doing cheap, simple experiments, it might make sense to tell them to change only one thing at a time, because the rate-limiting step is interpreting and understanding the data, and we want to make that step as easy as possible.   Skilled scientists, engineers, farmers, et cetera, routinely do complex, expensive experiments. In such situations, changing only one variable at a time would be an unnecessary burden, and often a disastrous burden.

In the introductory course, it makes sense to separate each task into simple components, and learn each component separately. This even has a name: It is part of the building block approach. If the task were not naturally simple, we would look for ways to make it simple, for pedagogical reasons.   Although the building block approach begins with learning the components one at a time, it must not end there. It is absolutely essential to put the blocks back together again, two at a time, three at a time, et cetera, to finish the overall edifice.

Changing only one variable at a time is a crutch, which may partially compensate for the investigator’s lack of skill in interpreting the data.   For anyone with ordinary ability and training, crutches are harmful, not helpful.

Bottom line: Please do not tell students – or anybody else – that scientists change only one variable at a time.

2  Discussion

I see this as somewhat of a Big Deal. I’ve seen too many “Conceptual Physics” courses where the students learn simplified concepts in isolation, but never learn to put things together. A pile of disconnected concepts is not interesting, just as a pile of disconnected Legos or full-sized building blocks is not interesting. They only become interesting – and useful – when they are put together in artful ways.

Concepts without real-world connections will be more-or-less instantly forgotten. This defeats the point of the whole course.

3  References

John Denker,
“The Twelve-Coins Puzzle”

Philip R. Bevington
Data Reduction and Error Analysis for the Physical Sciences
McGraw-Hill (1969).

«This lesson helps students understand that it is best to test only one variable at a time.»


«It is essential for students to know that only testable questions, which are used to test one variable, are suitable for scientific investigations.»

[emphasis in the original]

«change one variable at a time»

«Explain why an experiment should test only one variable at a time.»

«Because you want to see how the experimental results change due to only that one variable change»


[In other words, that seems to be saying:
Q: Explain why you should sit on a cactus.
A: Because you want to sit on a cactus.

IMHO the logic doesn’t seem very tight.]

«True or false: A scientist should choose only one variable at a time to change so that it is a fair test.»



«Second, as teachers of science, we understand the most important principle of the experimental method: “Check one variable at a time.” This rule not only guides the simplest classroom experiments but also the most sophisticated statistical statistical techniques [.....]»

NSTA Pathways to the Science Standards
Second High School Edition
https://books.google.com/books?isbn=0873552296 (page 44)

[Even if that idea were true, it wouldn’t rank as “most important”. Besides, it’s not true. Not by a mile.]


Copyright © 2015 jsd