Figure 1: Some Points to be Fitted
Copyright © 2006 jsd
It is very dangerous to live in a society where a few people have high-level thinking skills, and the rest don’t. Democracy does not work well in such a society.
Also: People who have high-level thinking skills are generally more productive than people who don’t. As a consequence, jobs that require high-level thinking generally pay better than jobs that don’t.
| Almost everybody knows how to run, after a fashion. However, if you sign up for the track team, or the soccer team, or anything like that, the coach will train you to run better, possibly a lot better. | Everybody knows how to think. It would be incorrect and insulting to tell someone they don’t know how to think. However, the fact remains that a good science class will train you to think better, possibly a lot better. |
Sometimes good thinking and good learning habits are taught implicitly, by osmosis ... but sometimes they are discussed explicitly, as a topic unto themselves. This topic is called metacognition, which means “thinking about thinking”.
That is, thinking skills are by-and-large separate from domain knowledge. To solve real-world problems in a particular domain, you need knowledge about the domain plus general thinking skills.
If you have high-level thinking skills, you can become proficient in a new domain just by learning the new domain-specific knowledge; you don’t need to learn the thinking skills all over again.
Einstein said “An education is what remains after you have forgotten everything you learned in school.” I’m pretty sure his point was that thinking skills remain, even after all the narrow domain-specific factoids have been forgotten (or have become irrelevant).
Anecdote: Once upon a time, a friend and I were conducting sea trials in a large, brand-new sailboat. The two of us had worked together before, debugging large computer programs. As you can imagine, debugging a computer program requires a detailed understanding of the computer language ... whereas debugging a boat requires considerable knowledge about how boats work, which is quite a different body of knowledge. However, both of us were struck by the fact that we used essentially the same process in both cases. We checked the typical case, then we checked the edges of the envelope, then we checked the corners of the envelope. When we observed small anomalies, we made a note of them, and then did whatever was necessary to make them reproducible. And so forth. We both knew what had to be done, and we each knew what the other guy was thinking, which helped us to work quickly and efficiently.
I call such things “game-show tests”. They cause some serious problems, as we now discuss. (Additional discussion can be found in reference 1.)
I used to say that such tests don’t predict anything at all, but if things keep going the way they are, such tests will begin to predict success in school ... for the simple reason that success in school is being measured, more and more, by such tests. This is circular in a truly ghastly way. It encourages rote learning and discourages thinking.
In particular, we need tests that measure thinking skills.
If you do it right, kids will increase their thinking skills and enjoy it.
After years of a steady diet of such problems, students will be alarmed and recalcitrant if you suddenly assign them homework that requires nontrivial thinking. You will have to explain that your course is different from other courses, past and present. Then you will have to patiently teach them the required thinking skills. Then you can assign problems that require thinking, with gradually increasing complexity.
Some parts of critical-thinking task are hard, but some of them are easy. Indeed you should have learned some of them in third grade, such as item 2, item 3, and item 9 in the following list.
Here are a few ideas that contribute to critical thinking, and can be applied to almost any field:
This is hugely important. A memory is not useful, and hardly even counts as a memory, if you cannot recall it when needed. Thinking about the connections increases the usefulness of each memory, by increasing the number of ways in which it can be recalled. See section 10.1 and reference 3.
For example:
More generally:
| With the math, you can learn a relatively small number of powerful, versatile principles. You can understand where the rules come from and how they fit together. | Without the math, you are reduced to learning a huge number of unprincipled inflexible rules that apply in special situations. These rules have to be learned by rote, because there is no way to see where they come from or how they fit together. |
| The principles are easy to remember, because they get used again and again. Every time a memory is used, it gets stronger. | Special situations arise so rarely that by the time the situation arises, you are likely to have forgotten the special-purpose rule. |
Sometimes the indirect approach is more convenient than the direct approach. Also, as discussed in section 5.2, a devious solution can serve as a valuable cross-check on the straightforward solution.
Here’s a classic example: The task is to add 198 plus 215. The easiest way to solve this problem in your head is to rearrange it as (215 + (200 − 2)) which is 415 − 2 which is 413. The small point is that by rearranging it, a lot of carrying can be avoided. The larger points are:
For the example of adding 198 plus 215, the straightforward approach would have worked; it just would have been less convenient. This stands in contrast to the nine dots puzzle (section 14.2), where the straightforward approach doesn’t work, and an imaginative approach is absolutely necessary.
In some quarters, the term “compensation” is applied to situations like this. I don’t particularly like this term, especially given that more expressive, more descriptive terms are available. I prefer to speak in terms of multiple, imaginative, devious, indirect, and/or outside-the-box approaches to the problem.
It’s important to make the work checkable. Some day you will be doing something important, where a mistake could cost millions of dollars and/or put people’s lives at risk. Important work obviously needs to be checked. Therefore you should get in the habit of making your work easy to check. Even if you are doing exercises that aren’t really important, pretend they are important, and check the work. To put it bluntly, if you cannot be trusted to check your work and to make it easily checkable by others, you won’t get hired to do anything important.
The first step toward making the work checkable is to show your work. Also, to the extent that you have checked your own work, show the checks.
The same principle applies to software: Document your code. For more on this, see reference 10.
Here’s a simple example of what it means to show your work in a useful way: When doing a calculation, whenever you write down a number, write an equation or a sentence that explains what the number means. That is to say, do not write down a bunch of unlabeled numbers:
| ☠ (1) |
but instead write equations:
| (2) |
This applies not only to calculations on paper, but also to computer programs including spreadsheets. If one cell in the spreadsheet contains a number, use the neighboring cells to document what the number means.
Rationale for labeling the numbers: This is partly so that you can go back and check your work ... and partly so that other folks can check your work.
This is one of the reasons why standardized multiple-guess tests are the enemy of critical thinking: Only the final answer is scored, so there is no incentive for making the intermediate steps checkable. (Arguably there is an indirect incentive for double-checking the result, because it improves the odds of getting the right answer ... but one major reason for grades in the first place is that students at the introductory level do not respond well to indirect incentives. They need more direct, immediate feedback.)
Also note that the architecture of a multiple-guess test rewards taking a slap-dash approach to every problem. IMHO we urgently need a different architecture, something that rewards meticulous work. That includes choosing a modest number of important problems, rather than a huge number of trivial problems.
Suggestion to teachers: One small step in the right direction is to arrange the scoring scheme on every quiz so that getting the numerically-correct answer is nowhere near sufficient for getting full credit. Another part of the score is for showing the work leading up to the answer, and yet another part is for double-checking the answer and showing that work as well.
Also: Sometimes it helps to assign the same question twice, perhaps as two items on the same assignment, or perhaps on two separate assignments a week apart. The point is to find a second, independent method of solution. Anybody who used method A the first time should use method B or C the second time. Anybody who used method B the first time should use method A or C the second time. And so forth. It must be emphasized that we are not redoing the problem because the first solution was wrong. The point is that there are multiple perfectly correct solutions, and we want to find them.
There are many, many ways that a solution can be verified.
There are almost always numerous constraints that must be satisfied at each step along the way. The details will vary from problem to problem, so it is your job to figure out what constraints apply to the problem and hand, and then verify that each step satisfies the constraints. If there are multiple elements in the solution-set, be sure to check each of them.
The same idea applies if you want to find the force experienced by the rider in a centrifuge, or practically any other situation involving rotational motion. Analyze it once in the lab frame and then analyze it again in the rotating frame.
Verifying the solution is valuable, but beware that it does not protect you from all possible errors. For example, if you have overlooked some elements of the solution-set, verifying whatever solutions you have found won’t alert you to the existence of other solutions. As another example, in the trigonometric example given above, if you inadvertently reverse the roles of a and b, you will get the wrong answer, but it will still satisfy the four trigonometric constraints itemized above.
Loosely speaking, any problem that requires thinking is called a puzzle or (equivalently) a riddle. Also, most puzzles have the further property that it is much harder to find a solution than it is to verify and understand a solution once it has been found. For example, consider the “eleven words in one” puzzle (reference 11). A given solution can be verified directly ... but a direct attack to find the solution would be thousands of times harder, since it would require searching through all the six-letter words in the English language.
Note: Easy verification is related to what computer scientists call the NP property. (If you don’t know what this means, don’t worry about it.) This is also related to what some puzzle aficionados call this the “Aha!” property, especially if the puzzle hinges on a single point that is obvious in retrospect.
Puzzles can be classified along various axes, as we now discuss.
One axis indicates how much domain knowledge the puzzle requires. Let’s call this the K axis. There are thousands of available puzzles that are near K=0. They are completely self-contained, i.e. the statement of the problem contains all the information necessary to solve it. Good starting places include the “20 questions” game (reference 12) and the “twelve coins” puzzle (reference 13). Reference 14 is a classic source; some of them are word puzzles, while others involve (in subtle ways) a fair bit of mathematical sophistication. There are also whole series of books by the likes of Raymond Smullyan and Martin Gardner. Self-contained puzzles are useful as a starting point, so that students can get accustomed to thinking even before they have much domain knowledge. As it says in reference 15, “Children lack knowledge and experience, but not reasoning ability.”
Moving along the K axis we come to problems that are “almost” self-contained, in the sense that they depend on facts that are unstated but well-known and easy to bring to mind. Farther along this axis are problems that require some amount of domain-specific knowledge. Reference 16 is a well-known source of open-ended questions and puzzles that involve modest amounts of physics knowledge.
At the far end of the K axis we find problems that require deep and broad knowledge of the world. To illustrate the range of the K axis, consider the following contrast:
| The “Who Owns the Fish” problem (reference 17) is intricate enough to scare away most people, but it is near K=0, because it is completely self-contained and well-posed. The statement of the problem contains just the information required to solve the problem ... no less, and essentially no more. | The “Mississippi Flow” problem (section 14.4) problem is very far from being self-contained. It requires you to rack your brain searching for information that might help solve the problem. A wide search is necessary, because seemingly very disparate tidbits of information turn out to be helpful. This is characteristic of a wide range of real-world problems. |
We can also define a B axis, which indicates to what extent a direct approach suffices, or not. The nine-dots puzzle (section 14.2) is the quintessential example and the source of the expression “outside-the-box thinking”. Other venerable examples where the direct approach fails include the dog-duck-grain problem and the orchard with 10 trees in five straight rows of four trees each.
If an indirect approach is needed, you need to use your imagination to find it, as discussed in section 4.
We can also define a H axis, which indicates how large is the space of hypotheses that must be considered. The number of hypotheses might be small, large, or infinite. Examples include
Similarly, the Mississippi flow problem (section 14.4) is in some sense infinite or nearly so ... not so much because there are infinitely many answers, but because there are nearly infinitely many dusty corners of your mind where you must look for potentially-useful information.
It is also worth noting that some puzzles (and many real-world problems) have multiple solutions; that is, there are multiple members of the solution set. As an elementary example, suppose the desired answer is x and we know that x2 = 81. If you find an x-value that solves the equation, you may or may not have found the desired answer.
A much more challenging example is to find the complete solution-set to the “south/east/north triangle problem” (section 14.5). Many people find one solution and express absolute certainty that it is the only solution. It’s not.
For some reason that I don’t fully understand, finding one solution creates a tremendous psychological barrier to finding another solution. Perhaps this is just a result of poor training: the students have been trained to expect that every homework problem will have only one solution.
We now turn to a topic that is somewhat related but somewhat different, namely methods of solution. (This topic was introduced in section 4.) For example, there are two completely independent ways of finding how much water flows in the Mississippi. That means we can ask questions at two different metaphysical levels:
Question (a) has essentially only one answer, but question (b) has a solution-set with at least two members.
Again it seems that finding one answer to question (b) creates a tremendous psychological barrier to finding another answer.
It must be emphasized that being able to solve question (a) in two different ways is a tremendously valuable skill, because it vastly decreases the chance of making an undetected error.
We now consider problems that are underspecified, overspecified, or otherwise ill-posed. The most troublesome kind of ill-posed problems involve inconsistencies. That is, sometimes the “facts” you’re working with are not entirely true.
To deal with such problems, you need to move beyond black-and-white notions of true-and-false; instead you need to weigh the probabilities. Similarly, you are no longer dealing with facts; instead you are weighing the evidence.
Some of the inconsistencies are exogenous, i.e. they come from what other people have told you. Other inconsistencies are endogenous, i.e. they come from assumptions that you have made on your own.
Some “recreational” puzzles, especially those that involve outside-the-box thinking, are useful for developing a subset of critical thinking skills, because they tempt you to make false assumptions, and force you to question your assumptions.
On the other hand, the overwhelming majority of “recreational” puzzles are well-posed, which means they don’t really exercise the full range of critical thinking skills.
For more discussion of ill-posed problems, see reference 18.
By way of example, suppose you were asked to fit a sine wave to a set of measured points as shown in figure 1. The obvious solution to this problem is shown in figure 2.
That looks like a good fit. The amplitude, frequency, and phase of the fitted function are determined to high precision, according to the standard formulas.
Even so, some crucial questions remain: How sure are you that this is the right answer? How well does this fitted function predict the position of the next measured point? These are tricky questions, because an unrestricted search for the sine wave that best fits the points is almost certainly not the best way to predict the next point. Figure 3 is the key to understanding why this is so.
It turns out that for almost any set of points, you can always find some sine wave that goes through the points, as closely as you please, if you make the frequency high enough. However, this can be considered an extreme example of overfitting and the over-fitted sine wave will be useless for predicting the next point. Another term that gets used in this connection is bias-variance tradeoff. These facts can be quantified and formalized using the Vapnik-Chernovenkis dimensionality and related ideas. A sine wave has an infinite VC dimensionality.
The sine wave stands in contrast to a polynomial with N adjustable coefficients, for which the VC dimensionality is at most N. That means if you fit the polynomial to a large number of points, large compared to N, the coefficients will be well determined and the polynomial will be a good predictor.
There are some deep ideas here, ideas of proof, disproof, predictive power, et cetera. For more on this, see the machine-learning literature, especially PAC learning. Reference 19 is a good place to start.
This sine-wave example calls attention to the fact that the family of fitting functions we are using (sine waves with adjustable amplitude, frequency, and phase) has an infinite VC dimensionality, even though there are only three adjustable parameters. We see that three data points – or even a couple dozen data points – are nowhere near sufficient to pin down these three parameters. This tells us that VC dimensionality is the important concept, and “number of parameters” is only an approximate concept, sometimes valid but definitely not always.
Another example of what can go wrong is shown in figure 4. The black curve represents the raw data. We have lots and lots of data points, with very high precision. We know a priori that the area under the black curve is the sum of two rectangles – a red rectangle and a blue rectangle. All we need to do is a simple fit, to determine the height, width, and center of the two rectangles. As you can see from the figure, there are two equally good solutions. There are two equally perfect fits. Alas, this leaves us with very considerable uncertainty about the area, width, and center of the blue rectangle.
Some problems in this category can be solved by introducing some sort of regularizer, as discussed in reference 18.
Additional examples to show how easy it is for people to fool themselves into “knowing” that they have “the” answer (when in fact they have not considered all the possibilities) can be found in reference 20.
| The school experience – especially the standardized “game-show” testing discussed in section 2 – gives many people the destructive idea that if it takes more then 45 seconds to solve a problem, they should give up. In the real world, you don’t get 40 questions in 30 minutes. That’s off by multiple orders of magnitude. More commonly you get 4 questions in 300 minutes, or something even beyond that. Therefore you must learn not to give up too soon. | At some point you should give up. You don’t want to spend the rest of your life stuck on some problem that you can’t solve. If you don’t want to give up entirely, you can set the problem aside temporarily, and return to it later, after you have acquired more knowledge and skill. |
| If you give up on the main goal you are admitting defeat. Many people are too quick to give up on the main goal. | Many problems require exploring the possibilities. That involves choosing tentative, hypothetical sub-goals. If such a hypothesis doesn’t work out satisfactorily, you need to backtrack and redo the analysis, choosing the next item from the list of hypotheses. Many people are too slow to give up on an untenable hypothesis (and therefore too slow to begin consideration of alternative hypotheses). |
The process of exploring the hypotheses can often be formalized as a search tree. Many chess problems involve search trees. Another example is searching a maze. Giving up on dead-end sub-goals is absolutely necessary for making progress toward the main goal.
Indeed, sometimes solving a small instance of the problem puts you in a position to solve all larger instances by induction.
We want knowledge to be useful, which means you must be able to apply the knowledge when needed. We want memory to be useful, which means you must be able to recall the memory when needed.
The key to having a good memory is to put stuff in to your memory in such a way as to facilitate getting it out of your memory when needed.
In the 1890s William James (reference 3) described memory in terms of the associations between ideas:
Each of the associates is a hook to which [the memory] hangs, a means to fish it up when sunk below the surface. Together they form a network of attachments by which it is woven into the entire tissue of our thought. The ’secret of a good memory’ is thus the secret of forming diverse and multiple associations with every fact we care to retain. But this forming of associations with a fact, – what is it but thinking about the fact as much as possible? Briefly, then, of two men with the same outward experiences, the one who thinks over his experiences most, and weaves them into the most systematic relations with each other, will be the one with the best memory.
This (along with lots of other evidence) tells us that thought and memory are inextricably intertwined. Analytical thought requires facts; otherwise there is nothing to analyze. Conversely, the process of memorizing and recalling facts is itself a thought process ... and if done right, it is a rather deep though process.
The cycle time for neurons in the human brain is on the order of 20 milliseconds. That’s 50 cycles per second. In contrast, there are many many thousands of facts that you can recall in less than a second, incluing names, phone numbers, vocabulary words, et cetera. That should make it obvious that recall is not a serial process. You do not run down a sequential list of facts, looking for the one you want. Instead, the recall process is massively parallel. Roughly speaking, it’s like posing a question to a very large audience: If any one person knows the answer, they raise their hand, and you receive the answer very quickly.
| In item 8 in section 9, the advice was to “figure it out”. Sometimes that’s exactly the right advice, especially if you are not accustomed to figuring things out, or if you didn’t realize that a given problem was figure-outable. | Once you have begun the process of figuring things out, having somebody tell you “Figure it out” is no longer of much help. |
| Learning to be smart is a long-term process. It is a lifestyle, in much the same way that the habit of healthy eating and regular exercise is a lifestyle. | By the time you are facing a hard problem with a short deadline, it is too late. You cannot make yourself significantly smarter in the time available. |
| If you want to improve your ability to recall things, you need to plan ahead. You need to make an extra effort at the time you encounter each new idea. | If you wait until you need to recall the idea, making an extra effort at that time won’t help much. It’s too late. |
The idea is to live your life in such a way that you continually get smarter. One of the key ingredients is this: Every time you hear a new idea you should turn it over in your mind, looking for ways in which it is connected to other ideas. Make a note of which old ideas are consistent with – or inconsistent with – the new idea. The process of mulling ideas and looking for connections takes time and effort. Over a period of weeks and months and years, these connections will make you smarter. They will make you more prepared to handle hard problems.
Let’s be clear that we are talking about something very odd: Forming a useful memory requires making a conscious effort to teach your subconscious what to do. The recall process is mostly subconscious ... but by consciously mulling over an idea you can form the connections that will allow the subconscious processes to work properly, later, when they are needed.
Now that we have discussed useful learning, let’s say a few words about useless learning. As a first example, relying on cramming is a bad idea. Anything you learn in a couple of days you will forget a couple of days later. Cramming might improve your grade this semester, but it guarantees you will crash and burn next semester. If a “cram” situation ever arises, it tells you something very important: It means you need to change your lifestyle and change your learning habits so you never get into that situation again.
As another example: Rote learning allows you to recall an idea in one way. As such, it counts as a memory, but it’s not a very useful memory. In contrast, mulling over an idea and establishing connections to other ideas allows you to recall the idea in 100 different ways, which is 100 times more useful.
Reference 5 explains how a scaling argument based on figure 5 can be used to figure out the formula for the area of an ellipse.
This leaves us with multiple ways of figuring out the area of an ellipse: You could just plain remember the formula from high-school geometry, and/or you could look it up, and/or you could easily reconstruct it whenever it is needed.
I know some people who have quite bad memory who are successful physicists. They carefully remember a few fundamental facts, and rederive everything else on an as-needed basis. For example, with a little practice, you can rederive the formula for the area of an ellipse faster than most people can recall it from memory (and with less probability of error).
It may be that some people develop extra-sharp thinking skills as a way of compensating for bad memory ... in analogy to the way that blind persons often develop extra-sharp hearing skills. However, I am not going to recommend bad memory any more than I would recommend blindness. Memory is a valuable skill. Obviously it is best to have a good memory and good thinking skills.
Feynman said that knowledge is like a grand tapestry. A forgotten fact is like a hole in the tapestry. You should be able to repair the hole in several different ways, by reweaving down from the top, or up from the bottom, or in from the sides. Any important fact can be rederived in numerous ways, because our knowledge has numerous interconnections.
Therefore: You should practice rederiving things. Even if it is something that you remember, rederive it anyway. This provides multiple advantages: First, it serves as a cross-check on your memory. Secondly, it builds up your thinking skills. Thirdly, it improves your understanding and recall of facts related to the one you are looking for, by exercising the all-important connections between facts.
Remember that any important formula should be derivable in multiple different ways, so if you derived it one way last time, try to derive it another way next time.
Some things can’t be derived, so you just have to remember them.
Conversely, some things can’t be remembered, so you just have to figure them out. In particular, if/when you visit unexplored territory, it is nice to be able to derive new formulas on the spot. It is a really good feeling to know that even though you are in unexplored territory, you are not lost. Based on your good thinking skills, you can move around more freely than most people do in familiar territory.
In contrast, the guy who tries to get by on memory alone, to the neglect of good thinking skills, will get seriously stuck as soon as he sets foot in unexplored territory, because the facts he needs are nowhere in his memory.
Last but not least: There is no clear-cut distinction between remembering something and figuring it out. If there is any distinction at all, it is of zero importance. Memory is itself a thought process. Sometimes it is a subconscious process, and sometimes it is a recognizably conscious process, but there is no important distinction. As an example, if I need to know the square root of 40, I can never remember the numerical value, but I know at least two ways of figuring it out in my head. I can figure it out to 1% accuracy in less time than it takes to talk about it, and figure it out to 0.1% accuracy almost as quickly. There’s thinking involved, but not much in the way of creative thinking, because I know exactly what procedures to use. You could ask whether this counts as memory, or as thinking, or both ... but the answer doesn’t matter.
Once upon a time, there was a sophomore who heard that fruits and vegetables are good for you. So he ate nothing but apples and celery for three months. Then he died.
Some members of the community reacted by saying “Apples are corruption! Celery is emblematic of everything that is wrong with society today! We must destroy all fruits and vegetables immediately!”
I beg to differ. I still think fruits and vegetables are good for you. I don’t think the problem was what the guy ate ... the problem was what they guy didn’t eat.
Let’s turn our attention now to algorithms and mnemonics.
| I’ve heard math teachers tell me, in all seriousness, that long division is evil, because it is an algorithm, and all algorithms are mindless. (If you want to know how I do long division, see reference 6.) | I get really tired of hearing that. |
| I’ve heard chemistry teachers tell me that students should not be allowed to use Gaussian elimination to balance chemical reaction equations, even when the number of variables is huge, because that would be an algorithm, and all algorithms are rote, and rote is evil. | I get really tired of hearing that, too. |
My point is that properly-chosen algorithms / mnemonics / equations / procedures / formalisms / methods are good for you. Really they are ... just like fruits and vegetables are good for you, as part of a balanced diet. If a student has some formal tools but lacks a gut feeling for how things work, the problem is not what the student has ... the problem is what the student doesn’t have.
Everyone needs a balanced diet. That is, everyone needs gut feelings and formalism.
Real understanding is represented by point B, in the upper-right corner, where there is a high level of feeling for the subject backed up by a high level of rigor.
As indicated by the red and blue arrows, you don’t get to the goal in one step. You start out with a little bit of feeling and a little bit of formalism. They reinforce each other and provide a foundation for the next step. The red leverages the blue and the blue leverages the red. And so you itsy-bitsy-spider your way up and over toward point B.
Let’s be clear:
The problem is not what the students have; the problem is what they don’t have. They don’t have a feeling for the subject.
This situation is represented by point D in figure 6. It sometimes goes by the name “rigor mortis”, which is a pretty good name for rigor without feeling.
This manifests itself in many ways. As an example, sometimes people sling buzzwords around without any real understanding. If they had checked their feelings against the theory, they would have known their feelings were nonsense.
Many additional examples are classified under the educationalese term “negative transference”. That means your gut feeling based on experience in one domain might give you the wrong answer when applied in another domain.
I’m not saying that gut feelings are bad. I’m saying that gut feelings have to be checked against the facts.
Red Queen: “Why, sometimes I’ve believed as many as six impossible things before breakfast.”
— Lewis Carroll
Also, I’m saying that sometimes having some sophistication gives you useful information about the limits of validity of your gut feelings.
Lady Thiang: “This is a man who thinks with his heart, His heart is not always wise.”
— Oscar Hammerstein
This sheds some light on the so-called “new math” and its relationship to “old math”, which has remained an unsettled issue since the 1960s. (If you’re interested in the history of this, reference 22 is a reasonably informative, non-hysterical, non-polemical news article.) This issue is commonly referred to as the “Math Wars” but I don’t like to use that term. The warlike aspects are a discredit to everyone involved. The sensible approach is to use smart, efficient algorithms2 and to understand the principles involved.
Some people object to algorithms because they can be memorized. Of course algorithms can be memorized, but that’s usually irrelevant, sometimes an advantage, and never a disadvantage. As mentioned in section 10.2, I don’t recommend doing away with memory, for the same reason I don’t recommend blindness. Memory is not the opposite of thought, nor the enemy of thought. Using an algorithm is not necessarily the non-thoughtful approach; usually it is the most thoughtful approach. Algorithms are like tools. When I tighten a bolt, I use a wrench. That does not make me any less skillful than the guy who tries to tighten the bolt with his bare hands. I’m allowed to use the wrench, even though I didn’t invent it or even manufacture it.
Continuing that thought: There have many occasions where I did invent and construct a specialized wrench or other tool to solve a specialized problem. Building custom tools and jigs requires an investment, but often this approach pays off handsomely, leading to overall faster and better results, compared to the brute-force head-on approach.
It is always possible to learn an algorithm in a mindless way, and to apply the algorithm by rote. That’s unsurprising, because any tool can be abused. Similarly equations can be abused by students who plug and chug, without any thought as to what the symbols mean. However:
| You should never use “equation” as a synonym for plug-and-chug. You should never use “algorithm” as a synonym for mindless. You should never use “systematic” as a synonym for rote. | If you mean rote, say “rote”. If you mean mindless, say “mindless”. If you mean plug-and-chug, say “plug-and-chug”. |
| Having a tool does not oblige you to abuse the tool. | You must not blame the presence of one tool for the absence of another. |
A tool that is well-suited for “Task A” might be laughably ill-suited for “Task B” – and vice versa. It’s your job to figure out which tool to use for the task at hand. This requires judgement.
As an example: If you want a numerical solution to a system of N linear equations in N unknowns, Gaussian elimination is incomparably more appropriate than Cramer’s rule. It is much less laborious and more numerically stable. See reference 23. In order of descending cleverness we have:
We should also say a few words about crutches:
| Sometimes there is a legitimate need for a crutch. That can happen if somebody has a broken leg .... after you have taken direct action to treat the underlying malady and provided you have briefed the user on the correct usage and limitations of the crutch. | On the other hand, crutches can actually cause secondary injuries, especially if overused or abused. For a person with normal abilities, a crutch is worse than useless. It gets in the way, and hinders development of normal performance. |
So ... there are upsides and downsides to crutches. We should not over-react to the upsides or the downsides. I’ve seen some algorithms – such as the infamous “density triangle” – that should be categorized as crutches. They may be useful in some rare, temporary situations, but otherwise are worse than useless.
If you see somebody using a crutch that is not really needed, it is a good idea to wean them off the crutch, sooner rather than later.
Last but not least: The right answer depends on the background and developmental level of the student. If a five year old kid asks “how does this flashlight work”, he does not want a lecture on the chemistry of batteries or the physics of LEDs. A more appropriate answer would be something purely operational, such as “you need to twist it, like so.”
If the student actually wants a more detailed answer, he can always ask a more detailed question.
In section 10.3 we argued that memory is part of thought (not the opposite of thought or the enemy of thought). Similarly we argued that algorithms and methods are part of thought (not the opposite of thought or the enemy of thought). These parts reinforce each other in a lattice, as shown in figure 6.
The same applies to creativity. Not all thinking counts as creative thinking, but if you are going to do any creative thinking, it will necessarily be based on a foundation of memories and methods, of gut feelings and algorithms.
Most inventions can be described as pushing forward the frontier of knowledge. In order to do this, you need to know where the frontier is! In almost all cases, usefully original thinking is not wildly original. For example, Beethoven is famous for breaking the rules of classical music theory ... but he did not break all the rules at once. He broke a rule here and a rule there, in crafty and purposeful ways.
Any discussion of critical thinking must necessarily cover much of the same ground as a discussion of scientific methods. See reference 8.
Consider the following scenario: I pose the “Mississippi Flow” problem to two different people who have nominally similar educational backgrounds and experience.
| The usual case is that I work with the person for 45 minutes, telling them “don’t give up” and “if you need to know that, figure it out” ... and giving a series of hints. At the end of this time, they have a solution. They realize in retrospect that in principle they could have solved the problem, in the sense that they knew everything necessary to permit a solution. At the same time, they realize that in practice they could never have found the solution on their own, because they would not have been able to organize their thinking in such a way as to call attention to the relevant facts. | In a not-very-small minority of the cases, the person can solve the puzzle very very quickly. They outline the method of solution in about four seconds, and then take another few seconds to carry out the required multiplications. |
The fact that proficiency with this sort of problem-solving is so unevenly distributed makes this sort of problem difficult to discuss in a classroom situation. The class as a whole, working as a team, can solve the problem relatively quickly, but that defeats one of the major purposes, namely giving each person experience racking their brain to find and organize the required bits of information. I don’t really know how to solve this problem. It would be ideal to spend 45 minutes with each student one-on-one, going over this puzzle, but that would be prohibitively expensive in a typical school setting.
Similar considerations apply to homework. If the purpose of the exercise is to get experience racking one’s brain, the purpose is defeated if students google the solution, or get the solution from a classmate. This problem cannot be prevented, but it can be fairly well controlled, as follows: You can separate the sheep from the goats by assigning a modified version of the puzzle on a closed-book in-class quiz. Someone who understands the method of solution will be able to solve the modified version instantly, whereas someone who merely copied the solution will not. (I don’t know of any suitable modifications of the “Mississippi Flow” problem, but others such as the “Who Owns the Fish” problem are readily modifiable.)
Let us return to the question of what is a puzzle. Consider the contrast:
| Many puzzles have the unfortunate property that even if you solve the puzzle, it’s still just a puzzle. The reward for solving it is trivial, artificial, or very indirect. Most homework problems are in this category; that is, the teacher already knows the right answer, and is not going to make any life-or-death decisions based on the student’s answer. | In many real-world situations, there is a lot riding on the question. It may truly be a life-and-death decision. |
| As my friend Larry says: If it’s not worth doing, it’s not worth doing right. | If it’s really worth doing, it’s worth double-checking to make sure you did it right. |
Consider someone who is learning to ride a bike. Why are they doing it? They typically are not doing it for the challenge; they are not doing it because the learning process is difficult. They are doing it because being able to ride a bike will empower them to go places and do things they could not do otherwise.
Consider the following four scenarios:
| Problem A is hard, and the solution is worth $10.00. | Problem B is hard, and the solution is worth $100.00. |
| Problem C is easy, and the solution is worth $10.00. | Problem D is easy, and the solution is worth $100.00. |
Given the choice, I would prefer problem B over problem A every time. That is, we should not value puzzles because they are hard; instead we should value puzzles if and when the answer is important. Homework problems have indirect value if (and only if) they teach skills that will have direct value later.
It is also true that given the choice, I would prefer problem C over problem A. Easy problems are preferable to hard problems, other things being equal.
Of course problem D is the most preferable of all.
More generally, I need to do a cost/benefit analysis. Given the choice between an easy, low-value problem and a hard, high-value problem, a tradeoff must be made. Making wise tradeoffs requires analysis and judgement.
In any case, we need to maintain a clear understanding of what is primary versus what is secondary, what is directly valuable versus what is only indirectly valuable, and what is real versus what is artificial.
Therefore do not get carried away with doing puzzles for the sake of doing puzzles. Choose puzzles that cultivate some useful general skill. Explicitly discuss what skills are being taught, and why. (See section 9 for some basic thoughts about this.)
The idea is neither to work harder, nor to work less hard. The idea is to get more done, by being clever. Things that formerly seemed difficult become easy once you know how. Above all, you should learn to solve important problems.
For more on this, see reference 24.
Some of these are interesting because they have more than one answer, i.e. the solution-set is not a singleton. Others are interesting because even though there is only one final answer, there are multiple methods of solution.
I have a quantity x such that x2=81. Please tell me the value of x. How do you know? How sure are you?
Arrange nine dots in three rows of three:
| • | • | • | ||
| • | • | • | ||
| • | • | • |
The task is to draw a path consisting of four straight contiguous line segments, such that the path goes through all of the dots.
Given a barometer, how many different ways can you think of for measuring the height of a building?
This is a classic, although the original story (reference 25) set up the question differently, not quite so directly.
Please give me an estimate of how much Mississippi River water flows past New Orleans in a year. This is a closed-book question; don’t look anything up; figure it out.
You start out at point A. You travel strictly south for one mile. You then make a right-angle turn and travel strictly east for one mile. You then make another right-angle turn and travel strictly north for one mile. It turns out that you are now back at point A. So, please tell me, where is point A? How do you know? How sure are you?
Note: For present purposes, we approximate the earth as being perfectly spherical. Point A is on the surface, and all travel takes place along the surface.
Bongard problems (reference 26) have the advantage that they don’t require much if any domain- specific knowledge, so you can use them on Day One of the course. Also, there are many dozens of them (unlike the Mississippi Flow problem, which is a one-off). The disadvantage is that Bongard problems are artificial puzzles. The downside to puzzles and games is that even if you win the game, it’s still just a game. The idea here is to use Bongard problems as scaffolding, to allow students to feel what it’s like to deal with open-ended problems. In the long run they won’t need the scaffolding, because there are plenty of real-world problems begging to be solved.
Students generally enjoy Bongard problems. They teach some useful thinking skills, including the necessity of looking at a problem from more than one viewpoint.
Let’s talk about creativity, originality, artistry, and imagination. These topics are closely related to the idea of open-ended questions and to the larger idea of critical thinking, in ways that will be discussed shortly.
| Whenever (in section 15.2 or elsewhere) I talk about “the building block approach”, please imagine a child building something out of Legos ... something complicated and imaginative. The question of what to build and how to build it is open-ended. There is an astronomical number of right answers. | In contrast, please do not imagine a menial laborer building a brick wall. In this situation, if you ask where the next brick comes from, there is usually only one right answer. Similarly, if you ask where the next brick needs to go, there is usually only one or two right answers. |
| With the Legos, reducing it to a step-by-step, multiple-choice, checklist-oriented activity would spectacularly bad. It would remove the spontaneity, creativity, and open-endedness. | With menial bricklaying, multiple-choice questions might be perfectly appropriate. |
Here’s another analogy: At many universities, the music department offers a “music appreciation for dummies” course. The students listen to music and talk about it. There is little if any open-endedness. This stands in contrast to the course in “composition and orchestration” that is taken by music majors, by real musicians, where originality and artistry are required. There is a great deal of structure, but also a great deal of open-endedness.
I mention this because all too often, the introductory physics course degenerates into “physics appreciation for dummies”. The students are on the outside looking in. The students look at physics and talk about physics, but they don’t actually do any physics. In particular,
I insist that it doesn’t have to be this way. Physics, even at the introductory level, does not have to be a mindless, joyless, multiple- guess activity.
Life is not a multiple-guess test. Teaching is certainly not a multiple- guess job. I mean, seriously, when was the last time a student came up to a teacher and said “I’m confused, and here are the four possible ways in which I could be confused. Pick one.”
Anybody who has more than a day of experience in the teaching profession (or any other profession) knows what it’s like to deal with open-ended questions. Everybody on this list knows in their bones how to do it.
There may be some questions about how best to teach this idea, but that is to be expected.
On the other hand, there are some tried-and-true ideas that we can keep in our bag of tricks, and there are some known pitfalls to be avoided. The following sections contain suggestions from the keen-grasp-of-the-obvious department.
When faced with a complex task, consider the classic “building block” approach. It consists of two phases:
The first phase of this approach (learning the individual building-blocks) is not sufficient. You also need the second phase, namely putting the blocks together to complete the overall edifice.
As an aid to the second phase, consider the “Music Minus One” approach as described in reference 27.
That is, rather than asking kids to solve a complex real-world problem ab initio, you hand them an almost-complete solution and let them provide the missing piece(s). I do this routinely with flying students, when they are learning to land the airplane:
The building-block approach stands in contrast to the “all-or-nothing” fallacy. Never allow yourself to be put in a position where the only options are:
There are lots of more nuanced approaches, whereby they gradually learn how to swim.
Suppose you don’t know anything about how to play the piano. Now imagine plopping yourself in front of a grand piano and trying to play the Hammerklavier sonata (reference 28). If you believe in the direct approach, you start by playing just the first note. That’s doable, right? Then you play just the first two notes. That’s doable, right? Then imagine working your way through the whole piece that way. If at first you don’t succeed, try harder. TRY HARDER!
Alas, that is not going to work. Instead you need to back up many many steps and approach the problem indirectly, using ultra-simple level-1 pieces and lots of scales and études. This is indirect because later, after you have mastered the masterpieces, it is OK to forget the level-1 pieces. They are are not wrong and do not need to be unlearned; they just get left behind or integrated and absorbed ... just as a chick leaves behind its shell and absorbs its yolk-sac.
Another term for this is scaffolding: To assemble a huge statue, you need scaffolding ... but afterwards you remove the scaffolding. It’s not needed anymore, and it just gets in the way. You can call this unlearning if you want, but it’s the ultra-simple non-problematic kind of unlearning, because it’s obvious which part is statue and which part is scaffolding. It’s obvious what you need to remove. There is nothing confusing or deceptive.
Remark: Only on the rarest of occasions does it help to tell a student to “TRY HARDER”. Usually the problem is that the kid doesn’t have the requisite foundation. You have to go back and build the foundation.
To say the same thing another way: Putting more effort into an approach that isn’t working is worse than useless. See section 9 for more on this.
Remark: The scales-and-études approach is indirect in another way: There is some sort of Cartesian direct product. Imagine a matrix with N rows and M columns. Studying N different scales and études prepares you to play M different repertory pieces. There is not a one-to-one relationship between a given étude and a given sonata.
That leads us to two more fallacies, which are mirror images of each other. As usual, both extremes are wrong.
| At one extreme is the repertory-only approach, aka learning-by-doing, aka the problem-solving-only approach. Imagine trying to learn a new subject, reading nothing but the Schaum’s outline. This is nuts. You need more explanation than that. You need more theory than that. | At the opposite extreme is the principles-and-concepts-only approach. This corresponds to giving the kid an enormous set of Legos, enough in principle to build almost anything ... but not allowing him to ever actually build anything. This is also nuts. If the principles and concepts are not used for interesting and important applications, there will be zero motivation and zero retention. |
As a more positive way of saying the same thing, you need teach both principles and applications, so that applications illustrate and motivate the theory, and the theory enables and explains the applications. At this point I do the itsy-bitsy-spider thing with my hands.
That leads us back to the topic of open-endedness. There are plenty of things in the world that are not open-ended. If you are the home plate umpire, for every pitch you need to call “ball” or “strike”. It’s multiple choice. You can’t split the difference.
On the other hand, there are also lots of open-ended things in the world. Most of real life consists of dealing with open-ended questions. Some examples that can be used in the classroom include:
We need to emphasize open-ended problems in order to restore balance, because in recent years closed-ended checklist-oriented multiple-guess problems have been grotesquely overemphasized.
Suggestion: Whenever selecting or creating quiz questions, include a goodly proportion of open-ended questions. Whenever I see a multiple-guess question, I wince, and then ask myself whether it could be converted into an open-ended question.
If you say multiple-choice questions save a lot of time, because they are easier to grade, then I respond as follows: If we are going to talk about savings, we need to talk about the costs, too. The cost of overemphasizing short-answer multiple-guess tests is that they defeat the purpose of the entire educational system. The cost exceeds your entire salary plus overhead and more than that besides. Ask the kids whether they would rather learn a small number of truly useful things, or be “exposed to” a huge number of useless things that they won’t remember anyway.
Besides, consider the options for a 50-minute test: You could assign 65 multiple-guess questions (45 seconds apiece) or you could assign 4 open-ended questions (12 minutes apiece). You may find that grading the 4-question version does not take much (if any) longer than grading the 65-question version.
As mentioned in section 3, the critical thinking must be baked into the thinking process, like the oatmeal in oatmeal cookies, not sprinkled on as an afterthought. So it is with the teaching of critical thinking: It has to be baked into the curriculum. It is not something you can advocate for 15 minutes on Tuesday morning and then ignore – or penalize – the rest of the time.
Combining several of the items mentioned in section 15 and elsewhere, we come back to the idea that 99.999% of reasoning is massively parallel and subconscious. Consider the Mississippi Flow problem. Most people have a very hard time with this problem. Solving it involves racking the brain and sifting the memory, searching for information that is somehow related to the problem. If on Day One of the course you ask students to do this, they can’t do it, and pestering them to “try harder” won’t help ... and the direct approach won’t work either. This problem doesn’t rise to the Hammerklavier level of complexity, but the same idea applies: Rather than attacking it directly, you need to go back and spend a long time working on scales and études so as to build up the skills necessary to attack the problem.
More specifically: You can’t fetch stuff out of your memory in a good way if you didn’t put it into your memory in a good way. This helps explain why the current overemphasis on fully-scripted problem-solving and multiple-guess quizzes is so poisonous. If there is a script for solving every problem that is going to be on the state test, then students naturally get the idea that rote learning is sufficient. The hallmark of rote learning is that each idea can be recalled in exactly one way. Technically that counts as a memory, but it is not a very useful memory. As discussed in section 10.1, the smart approach is to mull over each new idea, checking it against previously-known ideas, looking for connections ... and, conversely, checking for inconsistencies. If you do this, each idea can be recalled in 100 different ways, which makes it 100 times more useful than a rote memory.
|
The point is, you need to make it a habit to give every new idea this treatment. Don’t wait until you have cavities to start brushing your teeth. Don’t wait until you are facing an open-ended question to suddenly wish you had a more agile, effective mind. Wishing won’t help. This is one of the many things I like about the Harry Potter stories: the kids didn’t get to be powerful wizards overnight. They didn’t do it by praying or wishing. They worked hard for years to develop their skills, including reasoning and teamwork as well as the more domain-specific skills.
There is a bit of a chicken-and-egg problem here, because until the students learn how to build up richly-connected memories, they won’t be very good at solving open-ended problems ... and conversely, until they have some success at recalling off-the-wall and out-of-the box ideas, they won’t appreciate the value of richly-connected memories, and won’t be motivated to do the work – the years of work – necessary to build such memories.
Here is a bit of an exercise: The association game. Call on students, in order, so that everybody has to participate. The assignment is to come up with some word or idea that is associated with the Mississippi.
Kids who are called on later have the advantage of more time to think, but the disadvantage that the low-hanging fruit has already been picked.
Then we can go back and take those items two at a time, looking for other connections. In particular, what do riverboats and Huckleberry Finn have in common? What does that mean? Could that possibly help solve the Mississippi Flow problem?
Don’t tell me students can’t play this game. They play six degrees of Kevin Bacon for fun. The downside to 6∘KB is that even if you win, it’s still just a trivia game. What we’re doing here is just as much fun, but it’s better because it’s not just a game. We are building up the skills needed to solve important real-world problems.
More generally, you can play six degrees of physics. The law of universal gravitation is related to Coulomb’s law which is related to conservation of flux lines which is related to conservation of other things (such as the butter gun discussed in Feynman) which is related to continuity of world lines in spacetime which is ............
Students (and parents etc.) get irate if they think you are cheating, if they think you are not playing by the rules.
Therefore it is super-important to make the point to all concerned that you’re not cheating and you’re not even changing the rules ... you’re changing the game. Tell them:
“You don’t show up to play football wearing your baseball uniform and carrying your bat and mitt. It’s a different game, with different rules, different skills, and different equipment.”
“So it is with this class. You’ve spent 12 years learning how to play trivial pursuits, and there’s nothing wrong with that, but in this class we are playing a whole nother game, It has different rules, and requires different equipment and different skills. For starters, rather than touching on a large number of trivial problems, we are going to solve a small number of important problems. There will be lots of open-ended questions, and relatively few multiple- guess questions. There will be few if any questions that can be answered in 45 seconds. Creativity and originality will be encouraged.”
“In this class, we will not do hard problems. We will however do problems that would have been hard if you hadn’t learned the right techniques.”
“In the real world, some of the important problems are hard, but even so, we don’t go looking for hard problems; we go looking for important problems. The purpose of an education is to learn how to solve important problems as easily as possible.”
You’ll have to give that speech multiple times before anybody believes you ... and then you have to deliver. They’ve heard (most of) that speech before, from people who didn’t mean it and/or didn’t even understand what they were saying.
Suggestion: On occasion, assign the same exercise more than once, with instructions to find the answer by a different method. This is not easy, but it can undoubtedly be done. For example, more than 250 different proofs of the Pythagorean theorem are known.
Discuss the various solutions. Start by deciding which are correct ... but don’t stop there. It is also appropriate to evaluate the degree of originality ... but don’t stop there, either. As shown in figure 7, we don’t want to cultivate originality just for the sake of originality; we want to cultivate elegance and good style.
Originality without correctness is the domain of kooks and crackpots. Originality without good style is sometimes perverse and sometimes just weird. Good style is subjective, but it is nevertheless real and important. For more about the role of style, elegance, and artistry in science, see reference 29.
Here’s a constructive suggestion that is simple yet super-important: Start by removing the thousands of little things that reward conformity and rote regurgitation while penalizing creativity and critical thinking. You don’t have to remove all of them at once; just remove them a few at a time, as you come to them, all day every day.
For example, all too often on page 101 of the textbook there will be a definition, and then on page 105 there will be an “exercise” that calls for regurgitation of the definition, word for word. Suggestion: don’t assign that exercise. Come up with some exercise that requires applying the idea rather than memorizing empty words. If you can’t come up with such an exercise, then the idea must not be very important, and you need not assign any exercise whatsoever on that topic.
Here’s an even more obvious and more important suggestion: Don’t require students to learn things that can’t possibly be true. For example:
Another suggestion: Set up a program that rewards students for finding errors in the textbook. The reward should depend on the importance of the error: One point for simple spelling errors, and many points for fundamental misconceptions.
Similarly, assign students to find examples of nonsense in real life. There are plenty of blatant examples, including advertisements for “low calorie energy bars” and so forth. The aisles of a typical drug store contain homeopathic “drugs”, magnetic-therapy bracelets, and many other products that could not possibly work as advertised. Many news articles cannot keep straight the distinction between a millisievert and a millisievert per hour. Politicians promise to reduce taxes and balance the budget without cutting government programs. More subtle forms of nonsense are even more abundant.
Last but not least, get rid of the high-stakes game-show tests, as discussed in section 2.
Thinking is hard. Teaching and learning are hard. Teaching and learning critical thinking skills is particularly hard.
We need to do these things, even though they are hard. Some specific suggestions and hints on how to do this were presented above. See the table of contents.
I would particularly like to emphasize one step that needs to be taken. It’s a big step in the right direction. We must do away with high-stakes “standardized” multiple-choice tests. Just do away with them entirely and immediately. It is simply outrageous that we would judge students, teachers, and/or schools based on such horrible tests. For more on this, see reference 1.
Copyright © 2006 jsd
