[Contents]

Copyright © 2006 jsd

Learning, Remembering, and Thinking
John Denker

1  Thinking is Important, Learnable, and Portable

  1. More-or-less everybody should have a high level of thinking skills.

    It is very dangerous to live in a society where a few people have high-level thinking skills, and the rest don’t. Democracy does not work well in such a society.

    Also: People who have high-level thinking skills are generally more productive than people who don’t. As a consequence, jobs that require high-level thinking generally pay better than jobs that don’t.

  2. Thinking skills can be learned, and can be taught. This is the main subject of this document, as discussed at length below. First, though, consider the analogy to running:

    Almost everybody knows how to run, after a fashion. However, if you sign up for the track team, or the soccer team, or anything like that, the coach will train you to run better, possibly a lot better.   Everybody knows how to think. It would be incorrect and insulting to tell someone they don’t know how to think. However, the fact remains that a good science class will train you to think better, possibly a lot better.

    Sometimes good thinking and good learning habits are taught implicitly, by osmosis ... but sometimes they are discussed explicitly, as a topic unto themselves. This topic is called metacognition, which means “thinking about thinking”.

  3. Thinking skills are portable.

    That is, thinking skills are by-and-large separate from domain knowledge. To solve real-world problems in a particular domain, you need knowledge about the domain plus general thinking skills.

    If you have high-level thinking skills, you can become proficient in a new domain just by learning the new domain-specific knowledge; you don’t need to learn the thinking skills all over again.

    Einstein said “An education is what remains after you have forgotten everything you learned in school.” I’m pretty sure his point was that thinking skills remain, even after all the narrow domain-specific factoids have been forgotten (or have become irrelevant).

    Anecdote: Once upon a time, a friend and I were conducting sea trials in a large, brand-new sailboat. The two of us had worked together before, debugging large computer programs. As you can imagine, debugging a computer program requires a detailed understanding of the computer language ... whereas debugging a boat requires considerable knowledge about how boats work, which is quite a different body of knowledge. However, both of us were struck by the fact that we used essentially the same process in both cases. We checked the typical case, then we checked the edges of the envelope, then we checked the corners of the envelope. When we observed small anomalies, we made a note of them, and then did whatever was necessary to make them reproducible. And so forth. We both knew what had to be done, and we each knew what the other guy was thinking, which helped us to work quickly and efficiently.

*   Contents

1  Thinking is Important, Learnable, and Portable
2  Game-Show Tests
3  Some Basic Advice: Learning and Problem-Solving
4  Devious Methods of Solution
5  Check Your Work
5.1  Design the Work to be Checkable
5.2  Verify the Solutions
6  Practice Thinking about Puzzles and Riddles
7  Ill-Posed Problems
8  Things to Beware Of
8.1  Symbol versus Thing Symbolized
8.2  Tradeoff versus Multi-Dimensional Optimization
8.3  Ambiguities and False Equivalences
8.4  Hypotheses Omitted from Consideration
9  A Few Basic Problem-Solving Skills
10  Learning, Memorization, Recall, and Thinking
10.1  Connections Make Knowledge Useful
10.2  Memorizing and/or Deriving
10.3  Thinking Includes Algorithms, Mnemonics, Memory, and Feeling
10.3.1  Presence of One Thing vs. Absence of Another
10.3.2  Mnemonics plus Understanding
10.3.3  Some Examples
10.3.4  Tools versus Crutches
10.4  Creative Thinking Also Requires Methods and Memories
11  Scientific Methods
12  Groups versus One-on-One Teaching
13  Difficulty versus Importance
14  Some Favorite Puzzles
14.1  Inverse Mapping
14.2  Nine Dots
14.3  The Barometer Story
14.4  Mississippi Flow
14.5  Wolf / Goat / Cabbage
14.6  South / East / North
14.7  Bongard Problems
14.8  Grand Tour
15  Creativity, Originality, Artistry, and Imagination
15.1  Open-Ended Tasks
15.2  Building Block Approach
15.3  Indirect Approaches
15.4  Concepts and Applications
15.5  Open-Ended Tasks, Again
16  Teaching Critical Thinking
16.1  Building a Richly-Connected Memory
16.2  Cultivate Originality and Style
16.3  Double Negative : Getting Rid of Kadavergehorsamkeit
16.4  Double Negative : Getting Rid of Nonsense
16.5  Test-Prep versus Life-Prep
17  A Call to Action
17.1  Thinking
17.2  Testing
18  References

2  Game-Show Tests

  1. In recent years, schools have been placing more and more emphasis on a certain type of test. The following properties are typical of such tests:

    I call such things “game-show tests”. They cause some serious problems, as we now discuss. (Additional discussion can be found in reference 1.)

  2. It is distinctly unhelpful to place much weight on game-show tests. You can see at a glance that such tests have several built-in weaknesses.
  3. I find that game-show tests are not good for predicting anything I care about. I would never hire somebody based on a good test score, and I would never reject somebody based on a bad test score. I’ve known too many absolutely brilliant people who didn’t do well on game-show tests.

    I used to say that such tests don’t predict anything at all, but if things keep going the way they are, such tests will begin to predict success in school ... for the simple reason that success in school is being measured, more and more, by such tests. This is circular in a truly ghastly way. It encourages rote learning and discourages thinking.

  4. There is a natural tendency for teachers to teach to the test. That can be either a good thing or a bad thing, depending on how good the test is. We need better tests. We need much, much better tests.

    In particular, we need tests that measure thinking skills.

  5. On the plus side: Most kids enjoy thinking. Most kids enjoy riddles, if properly presented. They entertain each other by telling riddles. Any reasonable-sized bookstore will carry books of riddles – including physics riddles – even though they don’t carry textbooks.

    If you do it right, kids will increase their thinking skills and enjoy it.

  6. On the minus side: In many (but not all) textbooks, the end-of-chapter problems do not require thinking, and indeed train students not to think. Instead, the problems call for rote regurgitation of factoids presented in the chapter.

    After years of a steady diet of such problems, students will be alarmed and recalcitrant if you suddenly assign them homework that requires nontrivial thinking. You will have to explain that your course is different from other courses, past and present. Then you will have to patiently teach them the required thinking skills. Then you can assign problems that require thinking, with gradually increasing complexity.

3  Some Basic Advice: Learning and Problem-Solving

Some parts of critical-thinking task are hard, but some of them are easy. Indeed you should have learned some of them in third grade, such as item 2, item 3, and item 9 in the following list.

Here are a few ideas that contribute to critical thinking, and can be applied to almost any field:

1.
Learn how to learn. Mull over each new idea, to see now it is connected to everything else you know. Make this a habit, a reflex. Do this in class. Do this all day every day.

This is hugely important. A memory is not useful, and hardly even counts as a memory, if you cannot recall it when needed. Thinking about the connections increases the usefulness of each memory, by increasing the number of ways in which it can be recalled. See section 10.1 and reference 3.

2.
Check your work. Any problem worth doing is worth doing in two different ways, as a check. Some additional discussion can be found in section 5. Some examples that benefit from (or utterly depend on) finding multiple solutions can be found in section 14.

3.
Don’t judge a book by its cover. Don’t judge a steak by its sizzle. Beware of ambiguities, as discussed in section 8.3.

4.
Build up good math skills.

For example:

More generally:

With the math, you can learn a relatively small number of powerful, versatile principles. You can understand where the rules come from and how they fit together.   Without the math, you are reduced to learning a huge number of unprincipled inflexible rules that apply in special situations. These rules have to be learned by rote, because there is no way to see where they come from or how they fit together.

The principles are easy to remember, because they get used again and again. Every time a memory is used, it gets stronger.   Special situations arise so rarely that by the time the situation arises, you are likely to have forgotten the special-purpose rule.

5.
Become proficient at estimation and qualitative reasoning. This includes expanding things to lowest order, scaling, dimensions, units, et cetera. See reference 4 and reference 5.

6.
Be on the lookout for conservation laws and symmetry principles.

7.
Draw the diagram. If it’s electronics, draw the circuit diagram. If it’s relativity, draw the spacetime diagram. Don’t try to solve the problem in your head if you don’t have to. Beginners often try to get away without drawing the diagram ... even in situations where experts would immediately draw the diagram.

8.
Be methodical. Don’t hurry so much that you make dumb mistakes. There is no advantage to getting the wrong answer quickly.

9.
Neatness helps. See reference 6 for more on this.

10.
Paper is cheap. If you need more paper, get more paper. Again, see reference 6 for more on this.

11.
If you’re playing a game, know the rules of the game. If you’re taking a test, know the rules of the test. See reference 7.

12.
When possible, be scientific. Some more-advanced principles and methods for doing science are discussed in reference 8.

13.
Thinking clearly about cause-and-effect relationships is particularly important ... and all too often, this is very badly taught. This cuts right to the core of what science is. As Galileo and others have pointed out, physics must describe what happens, and it may or may not explain how it happens, but the fundamental laws almost never say why it happens. This is discussed in detail in reference 9.

14.
You should apply critical thinking skills to all of your thinking, all day every day. Critical thinking is not some sort of add-on. It is not something you save for special occasions. It has to be baked in, like the oatmeal in oatmeal cookies. You can’t bake the cookies without oatmeal, and then sprinkle oatmeal on them afterwards. See also section 16.

4  Devious Methods of Solution

Sometimes the indirect approach is more convenient than the direct approach. Also, as discussed in section 5.2, a devious solution can serve as a valuable cross-check on the straightforward solution.

Here are some examples, in order of increasing complexity:

a) Add 198 plus 215.  

You can easily do this in your head if you rearrange it as (215 + (200 − 2)) which is 415 − 2 which is 413.   The small point here is that by rearranging things, a lot of carrying can be avoided.

b) Decide which is larger: 12345/12347 or 23457/23459.  

You can easily do this in your head if you rearrange it as (1 - 2/12347) versus (1 - 2/23459). Both are less than 1. The latter is clearly closer to 1.   The small point here is that by rearranging things, and by using quanitative facts about arithmetic, such as the idea that inverse is a monotonic function, long division can be avoided.

Note: You could perhaps consider a shorter problem, perhaps 5/7 versus 7/9.   On the other hand, the longer version makes it clear that this is meant to be a logic puzzle, not a simple division problem. It increases the value of the clever approach over the brute-force approach. The indirect, understanding-based approach is equally easy for both versions. So it’s not clear that the shorter version is doing anybody a favor.

c) In your head, without a calculator or even pencil and paper, decide which is larger: 25/28 or 15/17.  

We can write the question mathematically as

25
28
 ¿>? 
15
17
             (1)

  This looks simpler than the previous question, because the numbers are smaller, but it is more challenging, because the two sides are more nearly equal.

We can rewrite this as

a
a+3
 ¿>? 
b
b+2
             (2)

  We know the value of a and b, so we don’t need to solve for them, but for now let’s leave them as symbolic rather than numeric.

Cross multiply. Throw away the resulting ab term from both sides. This leaves us with

2a ¿>? 3b
             (3)

 

Now plug in the numeric values and do the multiplication.   I can do all of this in my head, in less time than it takes to find a pencil and paper.

More importantly, these examples illustrate several larger points:

For the arithmetical examples considered here, the straightforward approach would have worked; it just would have been less convenient. This stands in contrast to the nine dots puzzle (section 14.2), where the straightforward approach simply doesn’t work, and an imaginative approach is absolutely necessary.

In some quarters, the term “compensation” is applied to situations like this. I don’t particularly like this term, especially given that more expressive, more descriptive terms are available. I prefer to speak in terms of multiple, imaginative, devious, indirect, and/or outside-the-box approaches to the problem.

Example (b) is more challenging that example (a). Both call for an indirect approach, but (b) requires stringing together a somewhat longer chain of ideas. Indirect reasoning and multi-step reasoning are both important skills, but they are not entirely the same skill. (On the other hand, they are not entirely different, because very commonly the indirect approach involves replacing one or two difficult steps by a larger number of easy steps.)

5  Check Your Work

5.1  Design the Work to be Checkable

It’s important to make the work checkable. Some day you will be doing something important, where a mistake could cost millions of dollars and/or put people’s lives at risk. Important work obviously needs to be checked. Therefore you should get in the habit of making your work easy to check. Even if you are doing exercises that aren’t really important, pretend they are important, and check the work. To put it bluntly, if you cannot be trusted to check your work and to make it easily checkable by others, you won’t get hired to do anything important.

The first step toward making the work checkable is to show your work. Also, to the extent that you have checked your own work, show the checks.

Show your work.
Check your work.
Show the checks.
     

The same principle applies to software: Document your code. For more on this, see reference 10.

Here’s a simple example of what it means to show your work in a useful way: When doing a calculation, whenever you write down a number, write an equation or a sentence that explains what the number means. That is to say, do not write down a bunch of unlabeled numbers:

2
3
4
24
                   (4)

but instead write equations:

L = length = m  
W = width = m  
H = height = m  
V = volume = L W H   
  = 24 m3  
             (5)

This applies not only to calculations on paper, but also to computer programs including spreadsheets. If one cell in the spreadsheet contains a number, use the neighboring cells to document what the number means.

There is a double rationale for labeling the numbers: It is partly so that you can go back and check your work ... and partly so that other folks can check your work.

This is one of the reasons why standardized multiple-guess tests are the enemy of critical thinking: Only the final answer is scored, so there is no incentive for making the intermediate steps checkable. (Arguably there is an indirect incentive for double-checking the result, because it improves the odds of getting the right answer ... but one major reason for grades in the first place is that students at the introductory level do not respond well to indirect incentives. They need more direct, immediate feedback.)

Also note that the architecture of a multiple-guess test rewards taking a slap-dash approach to every problem. IMHO we urgently need better tests, and a better educational system in general, something with better rewards for meticulous work. That includes choosing a modest number of important problems, rather than a huge number of trivial problems.

Suggestion to teachers: One small step in the right direction is to arrange the scoring scheme on every quiz so that getting the numerically-correct answer is nowhere near sufficient for getting full credit. Another part of the score is for showing the work leading up to the answer, and yet another part is for double-checking the answer and showing that work as well.

Also: Sometimes it helps to assign the same question twice, perhaps as two items on the same assignment, or perhaps on two separate assignments a week apart. The point is to find a second, independent method of solution. Anybody who used method A the first time should use method B or C the second time. Anybody who used method B the first time should use method A or C the second time. And so forth. It must be emphasized that we are not redoing the problem because the first solution was wrong. The point is that there are multiple perfectly correct solutions, and we want to find them.

5.2  Verify the Solutions

There are many, many ways that a solution can be verified.

Verifying the solution is valuable, but beware that it does not protect you from all possible errors. For example, if you have overlooked some elements of the solution-set, verifying whatever solutions you have found won’t alert you to the existence of other solutions. As another example, in the trigonometric example given above, if you inadvertently reverse the roles of a and b, you will get the wrong answer, but it will still satisfy the four trigonometric constraints itemized above.

6  Practice Thinking about Puzzles and Riddles

Loosely speaking, any problem that requires thinking is called a puzzle or (equivalently) a riddle. Also, most puzzles have the further property that it is much harder to find a solution than it is to verify and understand a solution once it has been found. For example, consider the “eleven words in one” puzzle (reference 11). A given solution can be verified directly ... but a direct attack to find the solution would be thousands of times harder, since it would require searching through all the six-letter words in the English language.

Note: Easy verification is related to what computer scientists call the NP property. (If you don’t know what this means, don’t worry about it.) This is also related to what some puzzle aficionados call this the “Aha!” property, especially if the puzzle hinges on a single point that is obvious in retrospect.

Puzzles can be classified along various axes, as we now discuss.

One axis indicates how much domain knowledge the puzzle requires. Let’s call this the K axis. There are thousands of available puzzles that are near K=0. They are completely self-contained, i.e. the statement of the problem contains all the information necessary to solve it. Good starting places include the “20 questions” game (reference 12) and the “twelve coins” puzzle (reference 13). A collection of classic puzzles by Henry Dudeney can be found in reference 14. Some of them are word puzzles, while others involve (in subtle ways) a fair bit of mathematical sophistication. From the same era and in a similar style we have the notorious Sam Loyd (reference 15). More recently there are whole series of books by the likes of Raymond Smullyan and Martin Gardner. Self-contained puzzles are useful as a starting point, so that students can get accustomed to thinking even before they have much domain knowledge. As it says in reference 16, “Children lack knowledge and experience, but not reasoning ability.”

Moving along the K axis we come to problems that are “almost” self-contained, in the sense that they depend on facts that are unstated but well-known and easy to bring to mind. Farther along this axis are problems that require some amount of domain-specific knowledge. Reference 17 is a well-known source of open-ended questions and puzzles that involve modest amounts of physics knowledge.

At the far end of the K axis we find problems that require deep and broad knowledge of the world. To illustrate the range of the K axis, consider the following contrast:

The “Who Owns the Fish” problem (reference 18) is intricate enough to scare away most people, but it is near K=0, because it is completely self-contained and well-posed. The statement of the problem contains just the information required to solve the problem ... no less, and essentially no more.   The “Mississippi Flow” problem (section 14.4) problem is very far from being self-contained. It requires you to rack your brain searching for information that might help solve the problem. A wide search is necessary, because seemingly very disparate tidbits of information turn out to be helpful. This is characteristic of a wide range of real-world problems.

We can also define a B axis, which indicates to what extent a direct approach suffices, or not. The nine-dots puzzle (section 14.2) is the quintessential example and the source of the expression “outside-the-box thinking”. Other venerable examples where the direct approach fails include the dog-duck-grain problem and the orchard with 10 trees in five straight rows of four trees each.

If an indirect approach is needed, you need to use your imagination to find it, as discussed in section 4.

We can also define a H axis, which indicates how large is the space of hypotheses that must be considered. The number of hypotheses might be small, large, or infinite. Examples include

It is also worth noting that some puzzles (and many real-world problems) have multiple solutions; that is, there are multiple members of the solution set. As an elementary example, suppose the desired answer is x and we know that x2 = 81. If you find an x-value that solves the equation, you may or may not have found the desired answer.

A much more challenging example is to find the complete solution-set to the “south/east/north triangle problem” (section 14.6). Many people find one solution and express absolute certainty that it is the only solution. It’s not.

For some reason that I don’t fully understand, finding one solution creates a tremendous psychological barrier to finding another solution. Perhaps this is just a result of poor training: the students have been trained to expect that every homework problem will have only one solution.

We now turn to a topic that is somewhat related but somewhat different, namely methods of solution. (This topic was introduced in section 4.) For example, there are two completely independent ways of finding how much water flows in the Mississippi. That means we can ask questions at two different metaphysical levels:

a) How much water?
b) What are the methods?

Question (a) has essentially only one answer, but question (b) has a solution-set with at least two members.

Again it seems that finding one answer to question (b) creates a tremendous psychological barrier to finding another answer.

It must be emphasized that being able to solve question (a) in two different ways is a tremendously valuable skill, because it vastly decreases the chance of making an undetected error.

7  Ill-Posed Problems

We now consider problems that are underspecified, overspecified, or otherwise ill-posed. The most troublesome kind of ill-posed problems involve inconsistencies. That is, sometimes the “facts” you’re working with are not entirely true.

To deal with such problems, you need to move beyond black-and-white notions of true-and-false; instead you need to weigh the probabilities. Similarly, you are no longer dealing with facts; instead you are weighing the evidence.

Some of the inconsistencies are exogenous, i.e. they come from what other people have told you. Other inconsistencies are endogenous, i.e. they come from assumptions that you have made on your own.

Some “recreational” puzzles, especially those that involve outside-the-box thinking, are useful for developing a subset of critical thinking skills, because they tempt you to make false assumptions, and force you to question your assumptions.

On the other hand, the overwhelming majority of “recreational” puzzles are well-posed, which means they don’t really exercise the full range of critical thinking skills.

For more discussion of ill-posed problems, see reference 19.

8  Things to Beware Of

8.1  Symbol versus Thing Symbolized

It is a classic student-pilot mistake to flip down the handle that controls the landing gear, and then announce that the landing gear is down. We agree that the handle is down, but there are many ways in which the landing gear itself might fail to come down. (This kind of failure is rare, but it is the pilot’s job to defend against rare-but-important failures.)

We agree that the handle in question is meant to symbolize the landing gear. It is even shaped like a miniature landing gear. However, the handle is just a symbol; the real landing gear is something dramatically different. Wanting the landing gear to be down is not sufficient. It is the pilot’s job to verify that the actual landing gear is down and locked.

Never confuse a symbol with the thing symbolized.
     

A closely-related issue is discussed in section 8.2.

8.2  Tradeoff versus Multi-Dimensional Optimization

On Madison Avenue, there is a saying: “The sizzle sells the steak”.

The problem is that Madison Avenue is infamous for applying sizzle to products that do not deserve it. However, that is a dangerous oversimplification, because there are in fact four possibilities:

    wholesome   unwholesome
    steak   steak
sizzle:   A   B
no sizzle:   C   D

This is related to what we discussed in section 8.1, insofar as the sizzle is only a symbol that is meant to call attention to the steak.

Logical thinking requires considering all four possibilities. In contrast, it is a common mistake to think that there are only two possibilities, namely B and C. Sometimes people get so disgusted by bad products with deceptive advertising that they make a knee-jerk transition to the diagonally-opposite policy, namely a good product with little or no advertising. This is a bonehead move. The correct move is diagonal, but rather horizontal, from B to A. You can perfectly well apply sizzle to a nice wholesome steak.

More generally, the point is: Do not assume two variables are linked. Some variables are linked, but some are not. If they are linked, there may be tradeoffs; you may have to optimize one variable at the expense of the other. However, if they are unlinked, you may be able to optimize both of them simultaneously.

Here is another example, applied to persuasion in general. Category B might be called empty rhetoric ... but keep in mind that not all rhetoric is empty.

    true and   bogus and
    sincere   insincere
persuasive:   A   B
unpersuasive:   C   D

Here is yet another example, applied to politics. Category B might be called dirty politics ... but keep in mind that not all politics is dirty. You want to find a politician who has good policies and enough charisma and political strength to get people to go along with those policies.

    good policy   bad policy
political strength:   A   B
political weakness:   C   D

8.3  Ambiguities and False Equivalences

Very commonly, there are two or more concepts that are not clearly distinguished. This is a major obstacle to correct thinking.

Often the ambiguity is reflected in the names we give things. That is to say, there are multiple different things represented by the same symbol. For example, in physics there are two different things that “gravity” might mean, and at least four different things that “heat” might mean. In math, there are at least three different things that “closed” might mean, and two different things that “primitive” might mean. And so on. Sometimes it is possible to figure out from context which meaning should apply, but this is laborious and error-prone. As a specific example, a well-known cryptography book quoted one definition of “primitive” in a context where the other definition was required. Additional examples of ambiguous and/or misleading terminology can be found in reference 20.

Sometimes you can fix this problem by using different words, for example replacing the word “heat” with TdS, if that’s the intended meaning.

Sometimes you can fix the problem by adding adjectives to the old words, for example talking about “barogenic gravity” versus “framative gravity”.

8.4  Hypotheses Omitted from Consideration

By way of example, suppose you were asked to fit a sine wave to a set of measured points as shown in figure 1. The obvious solution to this problem is shown in figure 2.

vapnik-sin-pts
Figure 1: Some Points to be Fitted
vapnik-sin-1
Figure 2: Sine Wave Fitted to Points

That looks like a good fit. The amplitude, frequency, and phase of the fitted function are determined to high precision, according to the standard formulas.

Even so, some crucial questions remain: How sure are you that this is the right answer? How well does this fitted function predict the position of the next measured point? These are tricky questions, because an unrestricted search for the sine wave that best fits the points is almost certainly not the best way to predict the next point. Figure 3 is the key to understanding why this is so.

vapnik-sin-2
Figure 3: Sine Wave Over-Fitted to the Same Points

It turns out that for almost any set of points, you can always find some sine wave that goes through the points, as closely as you please, if you make the frequency high enough. However, this can be considered an extreme example of overfitting and the over-fitted sine wave will be useless for predicting the next point. Another term that gets used in this connection is bias-variance tradeoff. These facts can be quantified and formalized using the Vapnik-Chernovenkis dimensionality and related ideas. A sine wave has an infinite VC dimensionality.

The sine wave stands in contrast to a polynomial with N adjustable coefficients, for which the VC dimensionality is at most N. That means if you fit the polynomial to a large number of points, large compared to N, the coefficients will be well determined and the polynomial will be a good predictor.

There are some deep ideas here, ideas of proof, disproof, predictive power, et cetera. For more on this, see the machine-learning literature, especially PAC learning. Reference 21 is a good place to start.

This sine-wave example calls attention to the fact that the family of fitting functions we are using (sine waves with adjustable amplitude, frequency, and phase) has an infinite VC dimensionality, even though there are only three adjustable parameters. We see that three data points – or even a couple dozen data points – are nowhere near sufficient to pin down these three parameters. This tells us that VC dimensionality is the important concept, and “number of parameters” is only an approximate concept, sometimes valid but definitely not always.

fit-two-rectangles
Figure 4: Fitting Two Rectangles

Another example of what can go wrong is shown in figure 4. The black curve represents the raw data. We have lots and lots of data points, with very high precision. We know a priori that the area under the black curve is the sum of two rectangles – a red rectangle and a blue rectangle. All we need to do is a simple fit, to determine the height, width, and center of the two rectangles. As you can see from the figure, there are two equally good solutions. There are two equally perfect fits. Alas, this leaves us with very considerable uncertainty about the area, width, and center of the blue rectangle.

Some problems in this category can be solved by introducing some sort of regularizer, as discussed in reference 19.

Additional examples to show how easy it is for people to fool themselves into “knowing” that they have “the” answer (when in fact they have not considered all the possibilities) can be found in reference 22.

9  A Few Basic Problem-Solving Skills

1.
One commonly hears advice along the lines of “don’t give up” or “Be more patient”. I consider that too simplistic, and too open to misinterpretation. It would be better to say that you need to develop judgement about when to give up.

In the words of the famous philosopher Don Schlitz:

You’ve got to know when to hold ’em,
know when to fold ’em,
know when to walk away, know when to run.
     

The school experience – especially the standardized “game-show” testing discussed in section 2 – gives many people the destructive idea that if it takes more then 45 seconds to solve a problem, they should give up. In the real world, you don’t get 40 questions in 30 minutes. That’s off by multiple orders of magnitude. More commonly you get 4 questions in 300 minutes, or something even beyond that. Therefore you must learn not to give up too soon.   At some point you should give up. You don’t want to spend the rest of your life stuck on some problem that you can’t solve. If you don’t want to give up entirely, you can set the problem aside temporarily, and return to it later, after you have acquired more knowledge and skill.

If you give up on the main goal you are admitting defeat. Many people are too quick to give up on the main goal.   Many problems require exploring the possibilities. That involves choosing tentative, hypothetical sub-goals. If such a hypothesis doesn’t work out satisfactorily, you need to backtrack and redo the analysis, choosing the next item from the list of hypotheses. Many people are too slow to give up on an untenable hypothesis (and therefore too slow to begin consideration of alternative hypotheses).

The process of exploring the hypotheses can often be formalized as searching a tree. More formally, this is called combinatorial search. Solving a Sudoku puzzle is a classic example of combinatorial search. Also, many chess problems involve searching a tree. Another example is exploring a maze. Giving up on dead-end sub-goals is absolutely necessary for making progress toward the main goal.

Keep your eyes on the prize.
     

2.
Real-world problems are almost never cut-and-dried, and are almost never multiple choice. Open-ended questions predominate. Story problems predominate. Get used to it.

3.
Real-world questions require you to search through everything you know, sifting your memory for seemingly-disparate facts athat can be brought to bear on the problem. The “Mississippi Flow” problem (section 14.4) is a good illustration of this. (This is very unlike end-of-chapter problems that involve only things you learned in that chapter.)

4.
Check all the plausible hypotheses. Consider all the plausible scenarios. Look at the problem from more than one viewpoint. The more open-ended the problem, the wider the range of hypotheses must be.

5.
Check each hypothesis against all the available evidence, or at least a sufficiently-large, representative sample of the evidence. (This is in some sense the mirror image of item 4.)

You will get into all sorts of trouble if you pay attention only to evidence that supports your favorite hypothesis. (This is called “selecting the data” and it is a Bad Thing.) For each hypothesis, you should check equally diligently for supporting evidence and conflicting evidence.

6.
If a given hypothesis makes more than one prediction, check all the predictions. It is common to find a hypothesis that makes five different predictions, four of which are wrong. If there is no good reason for this, i.e. if the logic behind all five predictions is similar, then in all likelihood the hypothesis is fundamentally wrong, and the one correct prediction is just fortuitous.

Indeed, the most dangerous models are the ones that get the right answer 99% of the time, in “normal” situations – and then betray you in some rare but critical situation.

7.
The problem with item 4 and item 5 is that this advice works better for experts than for students. Students, pretty much by definition, don’t have all the data, and they don’t have a full complement of theories to check against the data.

If you leave non-experts to figure stuff out on their own, they run the risk of latching on to some oversimplified theory that explains some undersized set of data. You can’t fault them for this. At this stage they have no way of knowing that their simple theory will not survive when more data becomes available.

8.
Sometimes it helps to do “warm-up exercises”. That is, if you can’t do the problem as stated, try attacking a simpler similar problem. For example, if you can’t immediately see how to put 8 non-interacting queens on an 8×8 chess board, see if you can put 4 of them on a 4×4 board.

Indeed, sometimes solving a small instance of the problem puts you in a position to solve all larger instances by induction.

9.
There may be more than one element in the solution-set. See section 7 and reference 19.

10.
Don’t panic if you need to know something that doesn’t appear in the statement of the problem. Figure it out, as discussed in section 10. Real-world problems are often overdetermined, underdetermined, or otherwise ill-posed, as discussed in reference 19.

10  Learning, Memorization, Recall, and Thinking

10.1  Connections Make Knowledge Useful

We want knowledge to be useful, which means you must be able to apply the knowledge when needed. We want memory to be useful, which means you must be able to recall the memory when needed.

The key to having a good memory is to put stuff in to your memory in such a way as to facilitate getting it out of your memory when needed.

In the 1890s William James (reference 3) described memory in terms of the associations between ideas:

Each of the associates is a hook to which [the memory] hangs, a means to fish it up when sunk below the surface. Together they form a network of attachments by which it is woven into the entire tissue of our thought. The ’secret of a good memory’ is thus the secret of forming diverse and multiple associations with every fact we care to retain. But this forming of associations with a fact, – what is it but thinking about the fact as much as possible? Briefly, then, of two men with the same outward experiences, the one who thinks over his experiences most, and weaves them into the most systematic relations with each other, will be the one with the best memory.

This (along with lots of other evidence) tells us that thought and memory are inextricably intertwined. Analytical thought requires facts; otherwise there is nothing to analyze. Conversely, the process of memorizing and recalling facts is itself a thought process ... and if done right, it is a rather deep though process.

The cycle time for neurons in the human brain is on the order of 20 milliseconds. That’s 50 cycles per second. In contrast, there are many many thousands of facts that you can recall in less than a second, incluing names, phone numbers, vocabulary words, et cetera. That should make it obvious that recall is not a serial process. You do not run down a sequential list of facts, looking for the one you want. Instead, the recall process is massively parallel. Roughly speaking, it’s like posing a question to a very large audience: If any one person knows the answer, they raise their hand, and you receive the answer very quickly.

In item 10 in section 9, the advice was to “figure it out”. Sometimes that’s exactly the right advice, especially if you are not accustomed to figuring things out, or if you didn’t realize that a given problem was figure-outable.   Once you have begun the process of figuring things out, having somebody tell you “Figure it out” is no longer of much help.

Learning to be smart is a long-term process. It is a lifestyle, in much the same way that the habit of healthy eating and regular exercise is a lifestyle.   By the time you are facing a hard problem with a short deadline, it is too late. You cannot make yourself significantly smarter in the time available.

If you want to improve your ability to recall things, you need to plan ahead. You need to make an extra effort at the time you encounter each new idea.   If you wait until you need to recall the idea, making an extra effort at that time won’t help much. It’s too late.

The idea is to live your life in such a way that you continually get smarter. One of the key ingredients is this: Every time you hear a new idea you should turn it over in your mind, looking for ways in which it is connected to other ideas. Make a note of which old ideas are consistent with – or inconsistent with – the new idea. The process of mulling ideas and looking for connections takes time and effort. Over a period of weeks and months and years, these connections will make you smarter. They will make you more prepared to handle hard problems.

Let’s be clear that we are talking about something very odd: Forming a useful memory requires making a conscious effort to teach your subconscious what to do. The recall process is mostly subconscious ... but by consciously mulling over an idea you can form the connections that will allow the subconscious processes to work properly, later, when they are needed.

Now that we have discussed useful learning, let’s say a few words about useless learning. As a first example, relying on cramming is a bad idea. Anything you learn in a couple of days you will forget a couple of days later. Cramming might improve your grade this semester, but it guarantees you will crash and burn next semester. If a “cram” situation ever arises, it tells you something very important: It means you need to change your lifestyle and change your learning habits so you never get into that situation again.

As another example: Rote learning allows you to recall an idea in one way. As such, it counts as a memory, but it’s not a very useful memory. In contrast, mulling over an idea and establishing connections to other ideas allows you to recall the idea in 100 different ways, which is 100 times more useful.

10.2  Memorizing and/or Deriving

Reference 5 explains how a scaling argument based on figure 5 can be used to figure out the formula for the area of an ellipse.

scale-ellipse
Figure 5: Scaling Y but Not X

This leaves us with multiple ways of figuring out the area of an ellipse: You could just plain remember the formula from high-school geometry, and/or you could look it up, and/or you could easily reconstruct it whenever it is needed.

I know some people who have quite bad memory who are successful physicists. They carefully remember a few fundamental facts, and rederive everything else on an as-needed basis. For example, with a little practice, you can rederive the formula for the area of an ellipse faster than most people can recall it from memory (and with less probability of error).

It may be that some people develop extra-sharp thinking skills as a way of compensating for bad memory ... in analogy to the way that blind persons often develop extra-sharp hearing skills. However, I am not going to recommend bad memory any more than I would recommend blindness. Memory is a valuable skill. Obviously it is best to have a good memory and good thinking skills.

Feynman said that knowledge is like a grand tapestry. A forgotten fact is like a hole in the tapestry. You should be able to repair the hole in several different ways, by reweaving down from the top, or up from the bottom, or in from the sides. Any important fact can be rederived in numerous ways, because our knowledge has numerous interconnections.

Therefore: You should practice rederiving things. Even if it is something that you remember, rederive it anyway. This provides multiple advantages: First, it serves as a cross-check on your memory. Secondly, it builds up your thinking skills. Thirdly, it improves your understanding and recall of facts related to the one you are looking for, by exercising the all-important connections between facts.

Remember that any important formula should be derivable in multiple different ways, so if you derived it one way last time, try to derive it another way next time.

Some things can’t be derived, so you just have to remember them.

Conversely, some things can’t be remembered, so you just have to figure them out. In particular, if/when you visit unexplored territory, it is nice to be able to derive new formulas on the spot. It is a really good feeling to know that even though you are in unexplored territory, you are not lost. Based on your good thinking skills, you can move around more freely than most people do in familiar territory.

In contrast, the guy who tries to get by on memory alone, to the neglect of good thinking skills, will get seriously stuck as soon as he sets foot in unexplored territory, because the facts he needs are nowhere in his memory.

Last but not least: There is no clear-cut distinction between remembering something and figuring it out. If there is any distinction at all, it is of zero importance. Memory is itself a thought process. Sometimes it is a subconscious process, and sometimes it is a recognizably conscious process, but there is no important distinction. As an example, if I need to know the square root of 40, I can never remember the numerical value, but I know at least two ways of figuring it out in my head. I can figure it out to 1% accuracy in less time than it takes to talk about it, and figure it out to 0.1% accuracy almost as quickly. There’s thinking involved, but not much in the way of creative thinking, because I know exactly what procedures to use. You could ask whether this counts as memory, or as thinking, or both ... but the answer doesn’t matter.

10.3  Thinking Includes Algorithms, Mnemonics, Memory, and Feeling

10.3.1  Presence of One Thing vs. Absence of Another

Once upon a time, there was a sophomore who heard that fruits and vegetables are good for you. So he ate nothing but apples and celery for three months. Then he died.

Some members of the community reacted by saying “Apples are corruption! Celery is emblematic of everything that is wrong with society today! We must destroy all fruits and vegetables immediately!”

I beg to differ. I still think fruits and vegetables are good for you. I don’t think the problem was what the guy ate ... the problem was what they guy didn’t eat.

10.3.2  Mnemonics plus Understanding

Let’s turn our attention now to understanding, and now it relates to algorithms, mnemonics, formalism, et cetera.

I’ve heard math teachers tell me, in all seriousness, that long division is evil, because it is an algorithm, and all algorithms are mindless. (If you want to know how I do long division, see reference 6.)   I get really tired of hearing that.

I’ve heard chemistry teachers tell me that students should not be allowed to use Gaussian elimination to balance chemical reaction equations, even when the number of variables is huge, because that would be an algorithm, and all algorithms are rote, and rote is evil.   I get really tired of hearing that, too.

I’ve heard physics teachers, in all seriousness, use the word “algorithms” as an antonym for understanding.   I get really tired of hearing that, too.

My point is that properly-chosen algorithms / mnemonics / equations / procedures / formalisms / methods are good for you. Really they are ... just like fruits and vegetables are good for you, as part of a balanced diet. If a student has some formal tools but lacks a gut feeling for how things work, the problem is not what the student has ... the problem is what the student doesn’t have.

Everyone needs a balanced diet. That is, everyone needs gut feelings and formalism.

  1. Without some gut feeling, you wouldn’t know how to derive the equations. You wouldn’t even know which equations were worth deriving. And you wouldn’t know how to use whatever equations you were given.
  2. Without the formalism, you have no good way of knowing whether your gut feelings are correct. Furthermore, if you’re not methodical, errors will creep into your work.
theory-feeling
Figure 6: Theory and Gut Feeling

Real understanding is represented by point B, in the upper-right corner, where there is a high level of feeling for the subject backed up by a high level of rigor.

As indicated by the red and blue arrows, you don’t get to the goal in one step. You start out with a little bit of feeling and a little bit of formalism. They reinforce each other and provide a foundation for the next step. The red leverages the blue and the blue leverages the red. And so you itsy-bitsy-spider your way up and over toward point B.

Let’s be clear:

  1. There is a problem – day in and day out – with people who do not have enough feeling for the subject. This is a problem, to some extent, everywhere and at all times. Feynman railed about this at every opportunity. For example, he discussed the Brazilian version of the problem in reference 23.

    The problem is not what the students have; the problem is what they don’t have. They don’t have a feeling for the subject.

    This situation is represented by point D in figure 6. It sometimes goes by the name “rigor mortis”, which is a pretty good name for rigor without feeling.

  2. At the opposite extreme, there is also a huge problem when people try to get by on their gut feelings, to the neglect of formalism, algorithms, mnemonics, et cetera.

    This manifests itself in many ways. As an example, sometimes people sling buzzwords around without any real understanding. If they had checked their feelings against the theory, they would have known their feelings were nonsense.

    Many additional examples are classified under the educationalese term “negative transference”. That means your gut feeling based on experience in one domain might give you the wrong answer when applied in another domain.

    I’m not saying that gut feelings are bad. I’m saying that gut feelings have to be checked against the facts.

    Red Queen: “Why, sometimes I’ve believed as many as six impossible things before breakfast.”
         — Lewis Carroll

    Also, I’m saying that sometimes having some sophistication gives you useful information about the limits of validity of your gut feelings.

    Lady Thiang: “This is a man who thinks with his heart, His heart is not always wise.”
         — Oscar Hammerstein

This sheds some light on the so-called “new math” and its relationship to “old math”, which has remained an unsettled issue since the 1960s. (If you’re interested in the history of this, reference 24 is a reasonably informative, non-hysterical, non-polemical news article.) This issue is commonly referred to as the “Math Wars” but I don’t like to use that term. The warlike aspects are a discredit to everyone involved. The sensible approach is to use smart, efficient algorithms2 and to understand the principles involved.

Some people object to algorithms because they can be memorized. Of course algorithms can be memorized, but that’s usually irrelevant, sometimes an advantage, and never a disadvantage. As mentioned in section 10.2, I don’t recommend doing away with memory, for the same reason I don’t recommend blindness. Memory is not the opposite of thought, nor the enemy of thought. Using an algorithm is not necessarily the non-thoughtful approach; usually it is the most thoughtful approach. Algorithms are like tools. When I tighten a bolt, I use a wrench. That does not make me any less skillful than the guy who tries to tighten the bolt with his bare hands. I’m allowed to use the wrench, even though I didn’t invent it or even manufacture it.

Continuing that thought: There have many occasions where I did invent and construct a specialized wrench or other tool to solve a specialized problem. Building custom tools and jigs requires an investment, but often this approach pays off handsomely, leading to overall faster and better results, compared to the brute-force head-on approach.

It is always possible to learn an algorithm in a mindless way, and to apply the algorithm by rote. That’s unsurprising, because any tool can be abused. Similarly equations can be abused by students who plug and chug, without any thought as to what the symbols mean. However:

You should never use “equation” as a synonym for plug-and-chug. You should never use “algorithm” as a synonym for mindless. You should never use “systematic” as a synonym for rote.   If you mean rote, say “rote”. If you mean mindless, say “mindless”. If you mean plug-and-chug, say “plug-and-chug”.

Having a tool does not oblige you to abuse the tool.   You must not blame the presence of one tool for the absence of another.

A tool that is well-suited for “Task A” might be laughably ill-suited for “Task B” – and vice versa. It’s your job to figure out which tool to use for the task at hand. This requires judgement.

10.3.3  Some Examples

  1. If you want a numerical solution to a system of N linear equations in N unknowns, Gaussian elimination is incomparably more appropriate than Cramer’s rule. It is much less laborious and more numerically stable. See reference 25. In order of descending cleverness we have:
  2. Some of the smartest guys I know are algorithms guys. Donald Knuth wrote some books, including
  3. Press, Teukolsky, Vetterling and Flannery wrote a book called Numerical Recipes that you may have heard of. I know these guys; They are famous for their understanding, not lack thereof.
  4. Feynman diagrams are basically an algorithm for organizing a bunch of calculations.
  5. Suppose you are doing numerical integration of the equation of motion for (say) a harmonic oscillator, or the Kepler problem.
    1. You could use a crude algorithm, such as generic first-order Euler ... or
    2. you could use a symplectic Verlet algorithm, which is exactly the same amount of work, but produces a better result, better by many orders of magnitude.

    This is relevant to any discussion of why it pays to be smart and well educated. Any dummy can do a crude numerical integration, but once you start doing it there is a tremendous incentive to use a smarter algorithm. If you don’t know some physics, you have no idea what phase space is or why a symplectic integrator is advantageous. If you don’t know some algebra, you can’t even be part of the conversation about second-order versus first-order.

10.3.4  Tools versus Crutches

An algorithm is like a tool, much like a piano is a tool for making music. Having a powerful tool doesn’t automagically make me a good musician, but it doesn’t make me worse. Having the tool does not preclude me from having a feel for the music, based on artistry as well as understanding.

So it is with physics formalism and mathematical algorithms: Using the tool does not automagically bestow understanding, but neither does it detract. By using the tool wisely, you can get a lot more done.

When I teach a kid to ride a bike I do not first require him to build the bike from scratch. When I teach a kid to play the piano I do not first require him to build the piano from scratch. By the same token, when teaching kids to do physics, I do not require them to reinvent all the formalism from scratch.

I often cook without a recipe, improvising based on what’s available and on the whim of the moment. On the other hand reading a cookbook won’t make me a worse cook, and quite likely will make me safer, make me more efficient, and produce a better end-product.

Algorithms do not prevent understanding, but neither do the bestow it ... or even require it.

Physics formalism is very powerful. Everybody, present company not excepted, uses the formalism without fully understanding it. Feynman said he didn’t really understand quantum mechanics (even though, relatively speaking, he understood it better than anyone else alive). Newton did not pretend to attribute any deep meaning to the law of universal gravitation (“hypotheses non fingo”). I daresay a lot of other folks are in the same boat or worse, yet we are proficient at using the formalism to get good results.

In any case, as the proverb says, it is a poor workman who blames his tools (or his algorithms).

We should also say a few words about crutches:

Sometimes there is a legitimate need for a crutch. That can happen if somebody has a broken leg .... after you have taken direct action to treat the underlying malady and provided you have briefed the user on the correct usage and limitations of the crutch.   On the other hand, crutches can actually cause secondary injuries, especially if overused or abused. For a person with normal abilities, a crutch is worse than useless. It gets in the way, and hinders development of normal performance.

So ... there are upsides and downsides to crutches. We should not over-react to the upsides or the downsides. I’ve seen some algorithms – such as the infamous “density triangle” – that should be categorized as crutches. They may be useful in some rare, temporary situations, but otherwise are worse than useless.

If you see somebody using a crutch that is not really needed, it is a good idea to wean them off the crutch, sooner rather than later.

Last but not least: The right answer depends on the background and developmental level of the student. If a five year old kid asks “how does this flashlight work”, he does not want a lecture on the chemistry of batteries or the physics of LEDs. A more appropriate answer would be something purely operational, such as “you need to twist it, like so.”

If the student actually wants a more detailed answer, he can always ask a more detailed question.

10.4  Creative Thinking Also Requires Methods and Memories

In section 10.3 we argued that memory is part of thought (not the opposite of thought or the enemy of thought). Similarly we argued that algorithms and methods are part of thought (not the opposite of thought or the enemy of thought). These parts reinforce each other in a lattice, as shown in figure 6.

The same applies to creativity. Not all thinking counts as creative thinking, but if you are going to do any creative thinking, it will necessarily be based on a foundation of memories and methods, of gut feelings and algorithms.

Most inventions can be described as pushing forward the frontier of knowledge. In order to do this, you need to know where the frontier is! In almost all cases, usefully original thinking is not wildly original. For example, Beethoven is famous for breaking the rules of classical music theory ... but he did not break all the rules at once. He broke a rule here and a rule there, in crafty and purposeful ways.

11  Scientific Methods

Any discussion of critical thinking must necessarily cover much of the same ground as a discussion of scientific methods. See reference 8.

12  Groups versus One-on-One Teaching

Consider the following scenario: I pose the “Mississippi Flow” problem to two different people who have nominally similar educational backgrounds and experience.

The usual case is that I work with the person for 45 minutes, telling them “don’t give up” and “if you need to know that, figure it out” ... and giving a series of hints. At the end of this time, they have a solution. They realize in retrospect that in principle they could have solved the problem, in the sense that they knew everything necessary to permit a solution. At the same time, they realize that in practice they could never have found the solution on their own, because they would not have been able to organize their thinking in such a way as to call attention to the relevant facts.   In a not-very-small minority of the cases, the person can solve the puzzle very very quickly. They outline the method of solution in about four seconds, and then take another few seconds to carry out the required multiplications.

The fact that proficiency with this sort of problem-solving is so unevenly distributed makes this sort of problem difficult to discuss in a classroom situation. The class as a whole, working as a team, can solve the problem relatively quickly, but that defeats one of the major purposes, namely giving each person experience racking their brain to find and organize the required bits of information. I don’t really know how to solve this problem. It would be ideal to spend 45 minutes with each student one-on-one, going over this puzzle, but that would be prohibitively expensive in a typical school setting.

Similar considerations apply to homework. If the purpose of the exercise is to get experience racking one’s brain, the purpose is defeated if students google the solution, or get the solution from a classmate. This problem cannot be prevented, but it can be fairly well controlled, as follows: You can separate the sheep from the goats by assigning a modified version of the puzzle on a closed-book in-class quiz. Someone who understands the method of solution will be able to solve the modified version instantly, whereas someone who merely copied the solution will not. (I don’t know of any suitable modifications of the “Mississippi Flow” problem, but others such as the “Who Owns the Fish” problem are readily modifiable.)

13  Difficulty versus Importance

Let us return to the question of what is a puzzle. Consider the contrast:

Many puzzles have the unfortunate property that even if you solve the puzzle, it’s still just a puzzle. The reward for solving it is trivial, artificial, or very indirect. Most homework problems are in this category; that is, the teacher already knows the right answer, and is not going to make any life-or-death decisions based on the student’s answer.   In many real-world situations, there is a lot riding on the question. It may truly be a life-and-death decision.

As my friend Larry says: If it’s not worth doing, it’s not worth doing right.   If it’s really worth doing, it’s worth double-checking to make sure you did it right.

Consider someone who is learning to ride a bike. Why are they doing it? They typically are not doing it for the challenge; they are not doing it because the learning process is difficult. They are doing it because being able to ride a bike will empower them to go places and do things they could not do otherwise.

Consider the following four scenarios:

Problem A is hard, and the solution is worth $10.00.   Problem B is hard, and the solution is worth $100.00.

Problem C is easy, and the solution is worth $10.00.   Problem D is easy, and the solution is worth $100.00.

Given the choice, I would prefer problem B over problem A every time. That is, we should not value puzzles because they are hard; instead we should value puzzles if and when the answer is important. Homework problems have indirect value if (and only if) they teach skills that will have direct value later.

It is also true that given the choice, I would prefer problem C over problem A. Easy problems are preferable to hard problems, other things being equal.

Of course problem D is the most preferable of all.

More generally, I need to do a cost/benefit analysis. Given the choice between an easy, low-value problem and a hard, high-value problem, a tradeoff must be made. Making wise tradeoffs requires analysis and judgement.

In any case, we need to maintain a clear understanding of what is primary versus what is secondary, what is directly valuable versus what is only indirectly valuable, and what is real versus what is artificial.

Therefore do not get carried away with doing puzzles for the sake of doing puzzles. Choose puzzles that cultivate some useful general skill. Explicitly discuss what skills are being taught, and why. (See section 9 for some basic thoughts about this.)

The idea is neither to work harder, nor to work less hard. The idea is to get more done, by being clever. Things that formerly seemed difficult become easy once you know how. Above all, you should learn to solve important problems.

For more on this, see reference 26.

14  Some Favorite Puzzles

Some of these are interesting because they have more than one answer, i.e. the solution-set is not a singleton. Others are interesting because even though there is only one final answer, there are multiple methods of solution.

14.1  Inverse Mapping

I have a quantity x such that x2=81. Please tell me the value of x. How do you know? How sure are you?

14.2  Nine Dots

Arrange nine dots in three rows of three:

    
    
    

The task is to draw a path consisting of four straight contiguous line segments, such that the path goes through all of the dots.

14.3  The Barometer Story

Given a barometer, how many different ways can you think of for measuring the height of a building?

This is a classic, although the original story (reference 27) set up the question differently, not quite so directly.

14.4  Mississippi Flow

Please give me an estimate of how much Mississippi River water flows past New Orleans in a year. This is a closed-book question; don’t look anything up; figure it out.

14.5  Wolf / Goat / Cabbage

Once upon a time a farmer went to market and purchased a wolf, a goat, and a cabbage. On his way home, he came to a river crossing where a small boat was available. The boat was so small that the farmer could carry only himself and a single one of his purchases : the wolf, the goat, or the cabbage.

If given a chance, not in the farmer’s presence, the goat would eat the cabbage, and the wolf would eat the goat.

The goal is to keep all of the purchases intact and get them to the far side of the river. How would you do it?

The “cannibals and missionaries” problem is similar. It permits fewer outside-the-box solutions, which is usually a bad thing, but might sometimes help somebody get out of a rut.

There are a number of other “river crossing puzzles”, and more generally a vast number of “transport puzzles”.

14.6  South / East / North

You start out at point A. You travel strictly south for one mile. You then make a right-angle turn and travel strictly east for one mile. You then make another right-angle turn and travel strictly north for one mile. It turns out that you are now back at point A. So, please tell me, where is point A? How do you know? How sure are you?

Note: For present purposes, we approximate the earth as being perfectly spherical. Point A is on the surface, and all travel takes place along the surface.

14.7  Bongard Problems

Bongard problems (reference 28) have the advantage that they don’t require much if any domain- specific knowledge, so you can use them on Day One of the course. Also, there are many dozens of them (unlike the Mississippi Flow problem, which is a one-off). The disadvantage is that Bongard problems are artificial puzzles. The downside to puzzles and games is that even if you win the game, it’s still just a game. The idea here is to use Bongard problems as scaffolding, to allow students to feel what it’s like to deal with open-ended problems. In the long run they won’t need the scaffolding, because there are plenty of real-world problems begging to be solved.

Students generally enjoy Bongard problems. They teach some useful thinking skills, including the necessity of looking at a problem from more than one viewpoint.

14.8  Grand Tour

Here’s a challenge of a different kind: Can you draw a continuous path that crosses each of the line segments in figure 7, crossing each one once and only once?

Crossing at the corners is not allowed.

7-bridges
Figure 7: Tour Challenge

The path shown in red in figure 8 is almost – but not quite – a successful solution; it fails to cross the middle vertical line segment. Can you do better than this?

7-bridges-tour
Figure 8: Tour Challenge : Failed Example

15  Creativity, Originality, Artistry, and Imagination

15.1  Open-Ended Tasks

Let’s talk about creativity, originality, artistry, and imagination. These topics are closely related to the idea of open-ended questions and to the larger idea of critical thinking, in ways that will be discussed shortly.

Whenever (in section 15.2 or elsewhere) I talk about “the building block approach”, please imagine a child building something out of Legos ... something complicated and imaginative. The question of what to build and how to build it is open-ended. There is an astronomical number of right answers.   In contrast, please do not imagine a menial laborer building a brick wall. In this situation, if you ask where the next brick comes from, there is usually only one right answer. Similarly, if you ask where the next brick needs to go, there is usually only one or two right answers.

With the Legos, reducing it to a step-by-step, multiple-choice, checklist-oriented activity would spectacularly bad. It would remove the spontaneity, creativity, and open-endedness.   With menial bricklaying, multiple-choice questions might be perfectly appropriate.

Here’s another analogy: At many universities, the music department offers a “music appreciation for dummies” course. The students listen to music and talk about it. There is little if any open-endedness. This stands in contrast to the course in “composition and orchestration” that is taken by music majors, by real musicians, where originality and artistry are required. There is a great deal of structure, but also a great deal of open-endedness.

I mention this because all too often, the introductory physics course degenerates into “physics appreciation for dummies”. The students are on the outside looking in. The students look at physics and talk about physics, but they don’t actually do any physics. In particular,

  1. They do not engage in any creative or open-ended activities, and
  2. they cannot imagine that there ever could be any creativity or imagination (let alone artistry) in physics.

I insist that it doesn’t have to be this way. Physics, even at the introductory level, does not have to be a mindless, joyless, multiple- guess activity.

Life is not a multiple-guess test. Teaching is certainly not a multiple- guess job. I mean, seriously, when was the last time a student came up to a teacher and said “I’m confused, and here are the four possible ways in which I could be confused. Pick one.”

Anybody who has more than a day of experience in the teaching profession (or any other profession) knows what it’s like to deal with open-ended questions. Everybody on this list knows in their bones how to do it.

There may be some questions about how best to teach this idea, but that is to be expected.

On the other hand, there are some tried-and-true ideas that we can keep in our bag of tricks, and there are some known pitfalls to be avoided. The following sections contain suggestions from the keen-grasp-of-the-obvious department.

15.2  Building Block Approach

When faced with a complex task, consider the classic “building block” approach. It consists of two phases:

The first phase of this approach (learning the individual building-blocks) is not sufficient. You also need the second phase, namely putting the blocks together to complete the overall edifice.

As an aid to the second phase, consider the “Music Minus One” approach as described in reference 29.

That is, rather than asking kids to solve a complex real-world problem ab initio, you hand them an almost-complete solution and let them provide the missing piece(s). I do this routinely with flying students, when they are learning to land the airplane:

  1. while I do N−1 of the tasks, the student takes care of the one remaining task, e.g.
  2. same as the above, but with less kibitzing from me
  3. the student does two of the N things
  4. et cetera, gradually increasing the level of responsibility and complexity.

The building-block approach stands in contrast to the “all-or-nothing” fallacy. Never allow yourself to be put in a position where the only options are:

  1. throwing kids into deep water and letting them either “sink or swim”, or
  2. completely abandoning the idea of teaching them to actually swim.

There are lots of more nuanced approaches, whereby they gradually learn how to swim.

15.3  Indirect Approaches

Suppose you don’t know anything about how to play the piano. Now imagine plopping yourself in front of a grand piano and trying to play the Hammerklavier sonata (reference 30). If you believe in the direct approach, you start by playing just the first note. That’s doable, right? Then you play just the first two notes. That’s doable, right? Then imagine working your way through the whole piece that way. If at first you don’t succeed, try harder. TRY HARDER!

Alas, that is not going to work. Instead you need to back up many many steps and approach the problem indirectly, using ultra-simple level-1 pieces and lots of scales and études. This is indirect because later, after you have mastered the masterpieces, it is OK to forget the level-1 pieces. They are are not wrong and do not need to be unlearned; they just get left behind or integrated and absorbed ... just as a chick leaves behind its shell and absorbs its yolk-sac.

Another term for this is scaffolding: To assemble a huge statue, you need scaffolding ... but afterwards you remove the scaffolding. It’s not needed anymore, and it just gets in the way. You can call this unlearning if you want, but it’s the ultra-simple non-problematic kind of unlearning, because it’s obvious which part is statue and which part is scaffolding. It’s obvious what you need to remove. There is nothing confusing or deceptive.

Remark: Only on the rarest of occasions does it help to tell a student to “TRY HARDER”. Usually the problem is that the kid doesn’t have the requisite foundation. You have to go back and build the foundation.

To say the same thing another way: Putting more effort into an approach that isn’t working is worse than useless. See section 9 for more on this.

Remark: The scales-and-études approach is indirect in another way: There is some sort of Cartesian direct product. Imagine a matrix with N rows and M columns. Studying N different scales and études prepares you to play M different repertory pieces. There is not a one-to-one relationship between a given étude and a given sonata.

15.4  Concepts and Applications

That leads us to two more fallacies, which are mirror images of each other. As usual, both extremes are wrong.

At one extreme is the repertory-only approach, aka learning-by-doing, aka the problem-solving-only approach. Imagine trying to learn a new subject, reading nothing but the Schaum’s outline. This is nuts. You need more explanation than that. You need more theory than that.   At the opposite extreme is the principles-and-concepts-only approach. This corresponds to giving the kid an enormous set of Legos, enough in principle to build almost anything ... but not allowing him to ever actually build anything. This is also nuts. If the principles and concepts are not used for interesting and important applications, there will be zero motivation and zero retention.

As a more positive way of saying the same thing, you need teach both principles and applications, so that applications illustrate and motivate the theory, and the theory enables and explains the applications. At this point I do the itsy-bitsy-spider thing with my hands.

15.5  Open-Ended Tasks, Again

That leads us back to the topic of open-endedness. There are plenty of things in the world that are not open-ended. If you are the home plate umpire, for every pitch you need to call “ball” or “strike”. It’s multiple choice. You can’t split the difference.

On the other hand, there are also lots of open-ended things in the world. Most of real life consists of dealing with open-ended questions. Some examples that can be used in the classroom include:

We need to emphasize open-ended problems in order to restore balance, because in recent years closed-ended checklist-oriented multiple-guess problems have been grotesquely overemphasized.

Suggestion: Whenever selecting or creating quiz questions, include a goodly proportion of open-ended questions. Whenever I see a multiple-guess question, I wince, and then ask myself whether it could be converted into an open-ended question.

If you say multiple-choice questions save a lot of time, because they are easier to grade, then I respond as follows: If we are going to talk about savings, we need to talk about the costs, too. The cost of overemphasizing short-answer multiple-guess tests is that they defeat the purpose of the entire educational system. The cost exceeds your entire salary plus overhead and more than that besides. Ask the kids whether they would rather learn a small number of truly useful things, or be “exposed to” a huge number of useless things that they won’t remember anyway.

Besides, consider the options for a 50-minute test: You could assign 65 multiple-guess questions (45 seconds apiece) or you could assign 4 open-ended questions (12 minutes apiece). You may find that grading the 4-question version does not take much (if any) longer than grading the 65-question version.

16  Teaching Critical Thinking

As mentioned in section 3, the critical thinking must be baked into the thinking process, like the oatmeal in oatmeal cookies, not sprinkled on as an afterthought. So it is with the teaching of critical thinking: It has to be baked into the curriculum. It is not something you can advocate for 15 minutes on Tuesday morning and then ignore – or penalize – the rest of the time.

16.1  Building a Richly-Connected Memory

Combining several of the items mentioned in section 15 and elsewhere, we come back to the idea that 99.999% of reasoning is massively parallel and subconscious. Consider the Mississippi Flow problem. Most people have a very hard time with this problem. Solving it involves racking the brain and sifting the memory, searching for information that is somehow related to the problem. If on Day One of the course you ask students to do this, they can’t do it, and pestering them to “try harder” won’t help ... and the direct approach won’t work either. This problem doesn’t rise to the Hammerklavier level of complexity, but the same idea applies: Rather than attacking it directly, you need to go back and spend a long time working on scales and études so as to build up the skills necessary to attack the problem.

More specifically: You can’t fetch stuff out of your memory in a good way if you didn’t put it into your memory in a good way. This helps explain why the current overemphasis on fully-scripted problem-solving and multiple-guess quizzes is so poisonous. If there is a script for solving every problem that is going to be on the state test, then students naturally get the idea that rote learning is sufficient. The hallmark of rote learning is that each idea can be recalled in exactly one way. Technically that counts as a memory, but it is not a very useful memory. As discussed in section 10.1, the smart approach is to mull over each new idea, checking it against previously-known ideas, looking for connections ... and, conversely, checking for inconsistencies. If you do this, each idea can be recalled in 100 different ways, which makes it 100 times more useful than a rote memory.

You can’t fetch stuff out of your memory in a good way
if you didn’t put it into your memory in a good way.
     

The point is, you need to make it a habit to give every new idea this treatment. Don’t wait until you have cavities to start brushing your teeth. Don’t wait until you are facing an open-ended question to suddenly wish you had a more agile, effective mind. Wishing won’t help. This is one of the many things I like about the Harry Potter stories: the kids didn’t get to be powerful wizards overnight. They didn’t do it by praying or wishing. They worked hard for years to develop their skills, including reasoning and teamwork as well as the more domain-specific skills.

There is a bit of a chicken-and-egg problem here, because until the students learn how to build up richly-connected memories, they won’t be very good at solving open-ended problems ... and conversely, until they have some success at recalling off-the-wall and out-of-the box ideas, they won’t appreciate the value of richly-connected memories, and won’t be motivated to do the work – the years of work – necessary to build such memories.

Here is a bit of an exercise: The association game. Call on students, in order, so that everybody has to participate. The assignment is to come up with some word or idea that is associated with the Mississippi.

Kids who are called on later have the advantage of more time to think, but the disadvantage that the low-hanging fruit has already been picked.

Then we can go back and take those items two at a time, looking for other connections. In particular, what do riverboats and Huckleberry Finn have in common? What does that mean? Could that possibly help solve the Mississippi Flow problem?

Don’t tell me students can’t play this game. They play six degrees of Kevin Bacon for fun. The downside to 6KB is that even if you win, it’s still just a trivia game. What we’re doing here is just as much fun, but it’s better because it’s not just a game. We are building up the skills needed to solve important real-world problems.

More generally, you can play six degrees of physics. The law of universal gravitation is related to Coulomb’s law which is related to conservation of flux lines which is related to conservation of other things (such as the butter gun discussed in Feynman) which is related to continuity of world lines in spacetime which is ............

Students (and parents etc.) get irate if they think you are cheating, if they think you are not playing by the rules.

Therefore it is super-important to make the point to all concerned that you’re not cheating and you’re not merely changing the rules ... you’re changing the game. Tell them:

You don’t show up to play football wearing your baseball uniform and carrying your bat and mitt. It’s a different game, with different rules, different skills, and different equipment.

   
So it is with this class. You’ve spent 12 years learning how to play trivia games, and there’s nothing wrong with that, but in this class we are playing a whole different game. It has different rules, and requires different equipment and different skills. For starters, rather than flirting with a large number of trivial problems, we are going to solve a small number of important problems. There will be lots of open-ended questions, and relatively few multiple- guess questions. There will be few if any questions that can be answered in 45 seconds. Creativity and originality will be encouraged.

   
We do not choose problems because they are hard, or because they are easy. We choose problems that are important.

   
If you can find an easy solution to an important problem, that’s the biggest win of all. That’s what we call low-hanging fruit. I love low-hanging fruit. I go for the watermelons, because they are huge and delicious and very, very low-hanging.

   
In this class, we do not do hard problems. We will sometimes do problems that would have been hard if you didn’t know the tricks. So let’s get started. Let me show you a few good tricks.

You’ll have to give that speech multiple times before anybody believes you ... and then you have to deliver. They’ve heard (most of) that speech before, from people who didn’t mean it and/or didn’t even understand what they were saying.

Also at some point it is necessary to explain what I mean by “good tricks”.

Good tricks help solve lots of important problems. If the trick can do that, it’s a good trick, and I don’t care whether you call it physics or non-physics.

   
Mathematicians think that all of physics is just a bunch of tricks for guessing the right answer without doing a mathematically-rigorous proof. I take it as a compliment, even if it was not intended as such.

   
In contrast, a worthless trick solves only one or two problems, none of which are very important. Learning such a trick is not worth the trouble. The poster child for this involves simplifying the fraction

16
64
 
             (6)

by “canceling” the sixes. Even when it works it’s a ridiculous method.

   
Worthless tricks come in various flavors and colors:

  1. Rote memory counts as a memory, and sometimes it is exactly the right approach, but it comes at a terrible cost. It’s not very scalable. You could make a list of all 500,000 questions that could be on the final, and if you want to memorize and retain all of the questions and answers, Rain Man style, that’s fine with me. However, if your name is not Rain Man, you will find it easier – a whole lot easier – to learn a handful of deep principles and learn how to apply them.
  2. According to legend, when they (separately) faced the Kobayashi Maru scenario, Ensign James T. Kirk and Ensign Montgomery Scott each found ways of «solving» the problem that depended in the fact that it was a simulation, not a real situation. In other words, they converted an important problem into an unimportant problem and quote «solved» it. That is the opposite of good behavior. That is not a survival skill in real life.

    Our goal is to convert hard important problems into easy important problems. It has to remain important or it’s not worth the trouble.

    And by the way, at any decent school, if something goes wrong with the simulation, you do not get credit for a win. We come up with a better scenario and you get to start over. Life is not some made- for-TV courtroom drama where the objective is to win on a technicality.

16.2  Cultivate Originality and Style

Suggestion: On occasion, assign the same exercise more than once, with instructions to find the answer by a different method. This is not easy, but it can undoubtedly be done. For example, more than 250 different proofs of the Pythagorean theorem are known.

Discuss the various solutions. Start by deciding which are correct ... but don’t stop there. It is also appropriate to evaluate the degree of originality ... but don’t stop there, either. As shown in figure 9, we don’t want to cultivate originality just for the sake of originality; we want to cultivate correctness, style, and elegance.

original-style
Figure 9: Correct and Elegant, Not Just Original

Originality without correctness is the domain of kooks and crackpots. Originality without good style is sometimes perverse and sometimes just weird. Good style is subjective, but it is nevertheless real and important. For more about the role of style, elegance, and artistry in science, see reference 31.

16.3  Double Negative : Getting Rid of Kadavergehorsamkeit

Einstein complained that his teachers required Kadavergehorsamkeit – the obedience of a corpse.

Once upon a time, I was helping a kid translate a sentence for his Spanish homework. He thought of two possible translations: the literal translation, and the fluent translation. He immediately wrote down the literal translation. I thought the fluent translation would be better, because in real life, nobody with any sense would use the literal translation. The teacher speaks Spanish; she will know this is better.

However, the kid was adamant:

Everybody else is going to do use the translation in the back of the book. If I do that, it will be marked correct. If I do anything else, it might or might not be marked correct. I could argue the point, and I might even win, but then I get the reputation for being argumentative, which is not good. By the time she gets around to grading the papers, she will have forgotten what the question was, so it doesn’t matter whether she speaks Spanish or not. The most important thing is to get the same answer as everybody else. Just because you think the fluent translation is better doesn’t mean the teacher thinks it is better. Maybe in your world it is better, but in my world there is a downside and no possible upside to the fluent translation.

I told the kid he was right, and apologized for giving bad advice. We briefly discussed the difference between what works in school and what is the right thing in real life.

We discussed the possibility of providing the second translation off to the side, or as an appendix, or asking about it in class. The kid stuck to his position: That school’s motto is “the nail that sticks up gets hammered down” and nothing was going to tempt the kid to stick his head up.

The lesson here is simple: If you want kids to learn to think, you have to reward the fluent translation. Also explicitly mention that it is possible to have more than one correct solution to homework questions. Ask in class if anybody has any questions. Ask if anybody can come up with alternative approaches.

It is not sufficient to merely say you want such things; you have to practice what you preach. Merely tolerating it would be an improvement over what we have now, but actually rewarding it would be even better. If you want kids to not learn by rote, you must not grade by rote.

I am quite aware that this requires the teacher to do more work. It is, however, necessary work.

16.4  Double Negative : Getting Rid of Nonsense

Here’s a constructive suggestion that is simple yet super-important: Start by removing the thousands of little things that reward conformity and rote regurgitation while penalizing creativity and critical thinking. You don’t have to remove all of them at once; just remove them a few at a time, as you come to them, all day every day.

For example, all too often on page 101 of the textbook there will be a definition, and then on page 105 there will be an “exercise” that calls for regurgitation of the definition, word for word. Suggestion: don’t assign that exercise. Come up with some exercise that requires applying the idea rather than memorizing empty words. If you can’t come up with such an exercise, then the idea must not be very important, and you need not assign any exercise whatsoever on that topic.

Here’s an even more obvious and more important suggestion: Don’t require students to learn things that can’t possibly be true. For example:

Another suggestion: Set up a program that rewards students for finding errors in the textbook. The reward should depend on the importance of the error: One point for simple spelling errors, and many points for fundamental misconceptions.

Similarly, assign students to find examples of nonsense in real life. There are plenty of blatant examples, including advertisements for “low calorie energy bars” and so forth. The aisles of a typical drug store contain homeopathic “drugs”, magnetic-therapy bracelets, and many other products that could not possibly work as advertised. Many news articles cannot keep straight the distinction between a millisievert and a millisievert per hour. Politicians promise to reduce taxes and balance the budget without cutting government programs. More subtle forms of nonsense are even more abundant.

Last but not least, get rid of the high-stakes game-show tests, as discussed in section 2.

16.5  Test-Prep versus Life-Prep

One thing that poisons the system is the idea that “practice makes perfect”. That is, all too often, students and teachers alike think that the best preparation for the test is to do things similar to the test over and over again. Alas, this is a terrible approach. First of all, there is heavy time pressure on the SAT and similar tests, so in that context the students are well advised to not take the time to ponder and/or play. Secondly, during the test there is no reward for showing the work, and not enough time to do it. This teaches exactly the wrong lesson about the importance of showing the work.

One thing that helps a little bit is to explain the distinction between knowledge-in and knowledge-out ... that is, between learning and recall. The test measures your ability to pour knowledge out of your head, as quickly as possible. The skills necessary for this task are wildly different from the skills necessary for putting knowledge into your head.

The two ideas are linked, but only very indirectly. They are linked because you can’t get knowledge out efficiently unless you put it in properly. Still, the point remains, the procedures for knowledge-in are very different from the procedures for knowledge-out. The former absolutely requires mulling each new idea, to see how it connects – and how it conflicts – with other ideas. See section 10.1. This takes time and effort. It requires making lots of connections, because you rarely know in advance which connections will pay off.

Another thing that helps a little bit is to explain the difference between test-prep and life-prep ... and to explain about burning the furniture.

Cramming for a test might make sense in an emergency, but it is like burning the furniture. I’m not even sure it’s good test-prep, but I guarantee that it’s not good life-prep. Cramming is a bad practice, and it should be avoided like the plague, except maybe in emergencies. Anything that is quickly learned will be quickly forgotten. Unless it is an absolute emergency, your life will be better if you spend the time properly learning a few things, rather than pseudo-learning a whole bunch of things.

As a teacher, there are lots of little things you can do to gradually convince students you’re serious about mulling, pondering, and playing:

The most important life-prep lesson is this: As a student, the key to doing better on the test does not consist in working harder during the test. Instead, what really matters is how you lead your life in the months and years leading up to the test. If you ponder stuff and play with stuff, that makes you smarter, bit by gradual bit. Then when the test comes around – and when real-world challenges come around – you will have a much easier time.

One last thing: The movie “Stand and Deliver” revolves around an SAT test ... but the movie is far more insightful than you normally expect a movie to be. My all-time favorite line is this. The day before the big test, a student quietly speaks to the teacher:

Claudia: You’re worried that we’ll screw up royally tomorrow, aren’t you?

Jaime Escalante: Tomorrow’s another day. I’m worried you’re gonna screw up the rest of your lives.

17  A Call to Action

17.1  Thinking

Thinking is hard. Teaching and learning are hard. Teaching and learning critical thinking skills is particularly hard.

We need to do these things, even though they are hard. Some specific suggestions and hints on how to do this were presented above. See the table of contents.

17.2  Testing

Let’s be clear:

Those provisos are important! The current reliance on high-stakes multiple-guess tests is not the right approach. Not even close.

Recommendation: Step #1: We simply must stop over-emphasizing high-stakes “standardized” multiple-guess tests. I’m not sure we need to do away with all such tests immediately, but the over-reliance needs to end, immediately and entirely. It is simply outrageous that we would judge students, teachers, and/or schools based on such horrible tests. For more on this, see reference 1.

Recommendation Step #2: We need to establish proper systems for assessment, accountability, and standardization.

We need to take step #1 immediately. Step #2 will take some time and effort. We can begin step #2 in parallel with step #1, but we need to take step #1 immediately, without waiting for completion of step #2. The assessment, accountability, and standardization we have now is worse than nothing, so there is no downside to step #1.

18  References

1.
John Denker,
“State-Mandated High-Stakes Trivia Tests – Or Not”
www.av8n.com/schools/eclbe-testing.htm

2.
John Denker,
“Breadth and Depth of Learning”
www.av8n.com/physics/breadth-depth.htm

3.
William James,
Talks to Teachers On Psychology; and to Students on Some of Life’s Ideals (1899).
http://books.google.com/books?id=XYSsCLlF_mkCprintsec=frontcover
Chapter XII deals specifically with memory.
http://ebooks.adelaide.edu.au/j/james/william/talks/chapter12.html

4.
John Denker,
“Dimensional Analysis”
www.av8n.com/physics/dimensional-analysis.htm

5.
John Denker,
“Scaling Laws”
www.av8n.com/physics/scaling.htm

6.
John Denker,
“Hints on How to Do Math”
www.av8n.com/physics/math-hints.htm

7.
John Denker
“Multiple-Guess Tests”
www.av8n.com/physics/multiple-guess.htm

8.
John Denker,
“Scientific Methods”
www.av8n.com/physics/scientific-methods.htm

9.
John Denker,
“Cause and Effect”
www.av8n.com/physics/causation.htm

10.
John Denker,
“Suggestions for Writing Good Software”
www.av8n.com/computer/htm/good-software.htm

11.
Find a six-letter English word that contains within it eleven other English words, without re-ordering the letters. I know of three different solutions. (This is an elaboration of a problem given in reference 18.)

12.
John Denker,
“Twenty Questions”
www.av8n.com/physics/twenty-questions.htm

13.
John Denker,
“The Twelve-Coins Puzzle”
www.av8n.com/physics/twelve-coins.htm

14.
Henry Ernest Dudeney,
536 Puzzles and Curious Problems
Collected and edited by Martin Gardner
http://jnsilva.ludicum.org/HMR13_14/536.pdf

Beware that in some cases (e.g. problem 5) the answer in the back of the book is not correct.

15.
Sam Loyd’s Cyclopedia of 5000 Puzzles, Tricks and Conundrums – With Answers
Lamb Publishing Corp. (1914)
http://www.mathpuzzle.com/loyd/Thumbnails.html

16.
John D. Bransford, Ann L. Brown, and Rodney R. Cocking, editors
“How People Learn: Brain, Mind, Experience, and School” (National Academy Press, 1999)
http://www.nap.edu/catalog.php?record_id=6160
http://www.csun.edu/~SB4310/How%20People%20Learn.pdf

17.
Jearl Walker,
The Flying Circus of Physics

18.
NIH / NIEHS “Tuff Stuff Riddles”
http://www.niehs.nih.gov/kids/rd7.htm

19.
John Denker,
“How to Deal with Ill-Posed Questions”
www.av8n.com/physics/ill-posed.htm

20.
John Denker,
“Weird Terminology”
www.av8n.com/physics/weird-terminology.htm

21.
David Haussler,
“Overview of the Probably Almost Correct (PAC) Learning Framework”
http://www.cs.iastate.edu/~honavar/pac.pdf

22.
Some examples where it is a challenge to consider all the possibilities:
./ill-posed.htm#sec-snap-string
./ill-posed.htm#sec-yo-yo
./ill-posed.htm#sec-s-e-n

23.
Richard Feynman,
Surely You’re Joking, Mr. Feynman.
The passage about rote versus feeling is quoted at
http://www.jsharrison.com/korea/2004/02/
(search for “in regard to education”).

24.
Jocelyn Noveck,
“Renegade parents teaching ’old math’ on the sly”
Associated Press (20 July 2008).

25.
John Denker,
“Balancing Reaction Equations Systematically”
www.av8n.com/physics/chem-balance.htm

26.
John Denker,
“How to Evaluate Creative Ideas”
www.av8n.com/physics/projectology.htm

27.
Alexander Calandra,
“The Barometer Story”
Current Science, Teacher’s Edition (1964).
http://www.mrao.cam.ac.uk/~steve/astrophysics/webpages/barometer_story.htm

28.
Bongard problems, discussion: http://www.foundalis.com/res/diss_research.html
Index of Bongard problems: http://www.foundalis.com/res/bps/bpidx.htm

29.
Wikipedia article, “Music Minus One”
http://en.wikipedia.org/wiki/Music_Minus_One

30.
Ludwig van Beethoven,
Sonata Op. 106 “Hammerklavier
http://files.sheetmusicarchive.net/compositions_b/BTSN106.PDF
Performed by Valentina Lisitsa:
http://www.youtube.com/watch?v=8TwysjbPmus

31.
Paul Lockhart,
“A Mathematician’s Lament”
http://www.maa.org/external_archive/devlin/devlin_03_08.html http://www.maa.org/external_archive/devlin/LockhartsLament.pdf

32.
John Denker,
“Uncertainty
as Applied to
Measurements and Calculations” www.av8n.com/physics/uncertainty.htm

33.
John Denker,
“Chemical versus Physical Change?”
www.av8n.com/physics/chemical-physical.htm

34.
John Denker,
“Principles of Teaching and Learning”
www.av8n.com/physics/pedagogy.htm

1
The infamous “density triangle” and “Ohm’s law triangle” were invented to serve as crutches for people who have no clue about algebra. If you’ve never heard of these things, that’s good. Don’t worry about it.
2
See reference 6 for ways to make long division and long multiplication more efficient.
[Contents]

Copyright © 2006 jsd