Science and creativity

BY DAVID FABER, FEB 21, 2008

“Numquam ponenda est pluralitas sine necessitate.”
–-William Occam (c. 1288–1347)

Engineers are people with a special set of skills in a domain that enables them to effectively solve problems. If they have the proper skill set, they are hired to satisfy a set of specifications, at which point the problem is solved, and everybody goes home happy.

Scientists can also be engineers, because the only real prerequisite is possession of a certain specialized skill set. However, the methods of a scientist acting as an engineer can differ from those of an engineer. A scientist does not just try to find a solution to a problem, but attempts to postulate and implement the best solution. This is accomplished by rigorously searching for a deeper understanding of the domain and creating layers of abstraction to describe the domain’s elements and their relationships.

The process of searching for an effective abstraction system, or a theory, is supposed to be entirely governed by the law and language of science, rationality. Through rationality, you can prove if a theory is good or bad, and if your abstractions are indeed capturing the domain or not. However, the value of a theory beyond its provable attributes is often measured by Occam’s Razor. Consequently, after all the rigor and reason required by science, it is the vague, amorphous concepts of simplicity and elegance that draw lines between a good solution and a better solution. This is the leap of faith a believer in science often takes.

Roughly translating to “Pluralities shall never be posited unnecessarily,” Occam’s Razor is most often taken to mean that the simplest and most elegant theory, the one with the fewest assumptions and contingent parameters, is the best. Science, supposedly a completely rigid and rational process, is known to use this subjective idea of simplicity and elegance as a metric for evaluating competing theories that otherwise seem to function and perform equally. This is where you find the sometimes obfuscated, but always essential need for creativity in science. It would be easy to simply come up with a sufficient theory to explain some data by just positing each piece of data as a part of the theory, but only a creative solution will be modular, useful and concise.

Let me give an example to make these concepts more concrete. Suppose you measure the radius and area of a circle with a ruler to come up with the set of points

      {(9, 254.469), (4, 50.265),
       (0,   0.000), (2, 12.566),
       (7, 153.938), (3, 28.274)}

which are in the form of (radius measured, area measured). You’d like to find a theory to explain the origin of these data points. One completely sufficient theory would be

      f(r) = 254.469   if r == 9
             50.265    if r == 4
             0         if r == 0
             12.566    if r == 2
             153.938   if r == 7
             28.274    if r == 3

This theory is in the form of a piece-wise function, and is 100% accurate in accounting for the data you have been given. However, the theory is rather undesirable. What if you need to know the value when the radius is 1? Or π? This theory is incapable of handling anything except for the observed values. It assumes that the model generating the original data points only could generate exactly those data points – but there is no real reason to assume that. You could add on to your theory

      f(r) = -666   if r == 12.742

and now it can account for a point outside the measured data set, but what are the chances that the area is actually -666 when the radius equals 12.742? While there’s no evidence to the contrary, there’s also no support for this, and therefore it is only adding unjustified complexity to the model. Another problem is that this model’s representation requires many terms, and if the data set continued to grow, the model would grow linearly with the size of the data. This is no good.

So how does one find a better solution? There’s no way to fully and rationally explain this because it is intuition and creativity that drive the process. However, you know a desirable solution would be simpler, reusable, and stationary as more data arrives (assuming the incoming data remains consistent with the model…accuracy and function always come before Occam’s Razor!) What you’re looking for is some way to encapsulate all the information you have by means of a more abstract representation of the data than the data itself. To find this, maybe you decided to sort the points in order of increasing r values. Then, it suddenly becomes more clear that the value of ƒ(r) is increasing, so maybe you decided to graph the points in order to visualize this increase. Then you could clearly see that its rate of growth was also increasing, so maybe some sort of higher order polynomial could model it. You know that the equation for the area of a square, which was discovered long ago, is a polynomial of degree 2, so maybe your creative thinking process will suggest that on an abstract level, the area of a circle is related to the area of a square, and you’ll decide to try to find a degree two polynomial. Maybe you’re Archimedes and you derived this equation entirely in the abstract without cutting out a lot of measured squares of paper. Maybe it took a few ideas and tests to get to that point. But after creatively inventing hypotheses and testing them, you eventually performed some sort of polynomial interpolation, and ended up with the much more elegant theory of

      f(x) = Π * x^2

You are much more satisfied with this solution because it is more abstract, simple, and elegant.

I will pick it up here at another time, because there is still a lot more to say. For example, what does this have to do with making websites and Keller & Faber, or what other domains can this apply to? Why are you still talking? These issues and more will be covered so keep coming back! Until then, enjoy the new website and have a lovely day.

Posted under science.