Introduction to Rationality I: Experiments

You know what? I've been throwing around the word 'rationality' since we've started here, and I've never actually bothered to explain it, so from time to time I'm going to explain a few key points. As a general summary of what might be expected, let's take it as read that I find rationality to be deeply rooted in two complimentary disciplines: science (by which is meant the ability to confirm hypotheses using imperical data), and logic (by which is meant the ability to resolve conflicting but unmeasurable statements using analogues).

Today, though, I want to talk about a particular, and very central, concept: The Experiment. Fundamentally, there are only ever two kinds of experiments: Thought Experiments, which are a discipline of Philosophy, and the unrelated Material Experiments, that are a discipline (and, in fact, the foundation) of science.

Material Experimentation is often called the scientific method, and it is the only reliable way of gathering information about the physical world.* A material experiment uses the manipulation of a single variable in order to test a hypothesis, which is an educated guess based on observation of the material world. This imposes a number of pros and cons on material experimentation.

*There is also statistical analysis, which is useful (within degrees) for predicting the behavior of human systems, in fields like urban planing, political science, marketing, economics, and the like. We won't discuss that much.

Pros:

• Reliability: A properly-performed experiment offers reliable, repeatable, and verifiable evidence in favour of a hypothesis, or a final and definitive refutation of the same.
• Direct Translation: An experiment can (and should) be readily understood by the person(s) performing it. In fact, one of the strongest advantages of an experiment is that it directly translates into knowledge. In this way, experimentation is the direct connection between the material and the mental.
• Quantification: An experiment will always include quantitative data, which is data that can have a numerical value attached to it, a key component of measurement and the basis of the idea of empirical evidence.
• Objectivity: A properly performed experiment is objective. The manner of the test should be so controlled as to make it impossible to introduce a bias into the system, save for whatever bias exists in the variable you are attempting to measure.

Cons:

• Labour Intensive: Setting up a proper experiment takes time, materials, effort, and thought. Often, experimentation is beyond the scope of the average person for any combination of those factors. Fortunately, there are research scientists who make a living performing experiments of all sorts and their results are published and generally available from your public library, if nothing else. Never the less, some experiments are cheap and simple enough that even an amateur scientist could take part. A list of such ideas might be a topic for another time.
• Increasing Complexity: The more rigorous the experiment or the more precise the idea, the more work is involved, and, proportionately, the more complex the experiment becomes. An experiment to determine the most effective plant food for your tomatos might be very easy to do. An experiment to determine the molecular structure of an exotic new enzyme or protein is very complex. Climate modelling is one of few applications for which we still construct computers that fill entire buildings, at least as far as complexity is concerned. In short, the further removed a hypothesis comes from a person's day-to-day life, the greater the expert knowledge they will need in order to decipher the results.
• Verification: Even the most disciplined person can make a mistake, and sometimes a source of bias in an experiment is overlooked. Because of this, scientists frequently perform experiments that other groups have already performed, to either verify or refute their results. Such processes are time-consuming, and for the amateur scientist who has developed his own experiment, it can often be difficult to find anyone to attempt to re-create it.

Conversely, there is the idea of a Thought Experiment. It is exactly what it sounds like: a test of a hypothesis without an actual material experiment. This is often useful in the study of ethics, or in fields where experimentation would be largely impossible, such as the famous  Schrödinger's cat experiment. A thought experiment is not merely thinking over an idea to see if it seems to make sense, however. Thought experiments can and must be backed up with calculations based on the existing science.

For a less abstract example, let me explain a form of thought experiment we use all the time in business... the Business Case. A company wants to know if it should move forward with a new product line, expansion, or just about anything else. Before it does so, it gathers every sort of applicable information and has it analysed statistically. Everything from consumer behavior to traffic patterns to engineering limitations is under consideration. The various options are all considered with respect to a number of dependant variables, and the most favourable option is chosen.