Experimental Design FAQ

Announcements and items of immediate importance.
Post Reply
Frank Sanns
Site Admin
Posts: 2117
Joined: Fri Jun 14, 2002 2:26 pm
Real name: Frank Sanns

Experimental Design FAQ

Post by Frank Sanns »

Getting good experimental results with a Fusor is not alway an easy task to accomplish. It is easy with a conditioned Fusor that is run daily. Straying from there and figuring out what results are due to variables can quickly become difficult. I am not sure this has ever been resolved on this forum or even in laboratories.

One of the reasons for this is the misconception that One Factor at a Time experimentation is the best way to get a result. It seems intuitive to just change one variable and get a result. In reality, this is one of the least reliable ways to get a meaningful result. While this CAN work in some very controlled circumstances, it is not a reliable way to understand the importance of other factors influencing the result. This can confound the results and either make them meaningless or worse yet, misleading.

For some background, I will list myself as such a One Factor at a Time believer in my teens and early 20s. I mean, what could be better than to run all of the permutations of experiments changing just one variable at a time. Lots of experiments and lots of results. Unfortunately, much is left out.

Then, while working for Bayer in the 1980s, I had the fortune to spend some days with Genichi Taguchi. If you do not know who Taguchi is, he is the one very responsible for the Japanese solving quality control problems before they happened. In contrast, the US would run quality control after items or vehicles were actually produced, then trying to figure out what went wrong then tried to fix it. The later was a miserable failure costing a great deal of time and money. So how could the Japanese anticipate a quality issue and resolve it before it ever made it into production. The answer is something called the Taguchi method.

Taguchi built on work of the US engineers like Juran and Deming but he took it much farther. The concept of Taguchi is to look at all of the variables that you think may be important to the result but also design the experiments to take into account factors that you may not have considered. It also includes interactions between these variables.

To study all of this could take a hundred experiments or more to study even the simplest of systems. An example of something simple might be to study the output of solar panels based on dopants. Some of the important variables might be:

1. Concentration of the dopant
2. Purity of the silicon
3. Composition of the dopant
4. Depth of implantation of the dopant

and many more. But what also much be included is the temperature behavior of the cells across a wide range of service temperatures. Type of glass and thickness and finish. Wavelength of the illuminating light and on and on. You can see how a One factor at a Time, full factorial design can quickly get out of control with the number of experiments. Using just 3 different levels for just 4 of those factors gives 81 experiments. But that would not do it since many other factors need to be considered. Hundreds of experiments would be needed to conduct a full factorial design and some things would still be missing.

Taguchi used experimental designs that only needed a tiny fraction of the experiments to be done to yield even more data and better results that doing all of those full factorial designs. It seems counter intuitive but it is so. Not only is it so but it is one of the most reliable ways to get meaningful results for variables that you understand and those that you don't. It also gives a value for noise for each of the results so the impact of the variables can be continuously accessed for significance.

Not just any design works. The design needs to be based on an orthogonal array layout of the experiment. Using and L9 orthogonal array, only 9 experiments are needed to study the 4 variables at 3 different levels. Only 9 vs the 81 for a full factorial design!

The way the design works is to have a balanced mix of the three levels and four parameters so that each change can be summed up for each parameter change. For the solar cell example above, the sums would be as follows:

Sum up the low level of dopant for all of the experiments that also changed the other three variables. Then do the same for the mid level and then the high level. So the effect of dopant level is know for all other changing parameters.

Next would be to look at the purity of the silicon at the H, M, L levels and sum it up while all of the other parameters change. This is done for each series. This gives replicates at each level so not only are the effects of the single variable obvious but the noise is also seen. This is really important for factors that may be changing with time, or other factors that are important but not identified. This is the ANOVA or Analysis of Variance.

Search Taguchi in the literature and ANOVA. You will see what orthogonal arrays look like and how they can benefit doing experiments. This is especially true with a wild beast like a fusor that has so many things going on. Remember that any one experiment has zero confidence limit. Replicates are an important part of assessing variables and noise in a system. Instead of running replicates that are the exact same experiment, run a Taguchi orthogonal array and assess other variables and experimental changes with no more work and end up with more meaningful results.
Achiever's madness; when enough is still not enough. ---FS
We have to stop looking at the world through our physical eyes. The universe is NOT what we see. It is the quantum world that is real. The rest is just an electron illusion. ---FS
User avatar
Richard Hull
Moderator
Posts: 14975
Joined: Fri Jun 15, 2001 9:44 am
Real name: Richard Hull

Re: Experimental Design FAQ

Post by Richard Hull »

Interesting concept, the Taguchi method. One would indeed need to know a lot of minutia and how it might foul a single process of many complex steps. some obvious and some no so obvious but to be determined via controlled experiment related to one single possible avenue or process of the whole.

This sounds like jobs for a team of teams around an end goal. Each team assigned the goal of perfection within their narrow assigned goal which must no affect other team's goal allowing for many best scenarios within each team. Laudable and a proven process. While the end product of great usefulness and reliability might be the result. In the engineering world the process might cost a good deal in man hours and experiment.

Some experts in the many touched upon processes might be needed.

Within our effort, this involves often one lone wolf who, at best, comes armed with 3 or 4 key skills of use, demanding a learning period for the others that must be brought to bear under the umbrella of making an optimized fusion engine or neutron source. This bund, these forums, are open source and truly valuable to the other processes which must be absorbed and turned into working assemblages.

The Taguchi method demands precision near expert teams for each avenue in a larger process working in concert.

Richard Hull
Progress may have been a good thing once, but it just went on too long. - Yogi Berra
Fusion is the energy of the future....and it always will be
The more complex the idea put forward by the poor amateur, the more likely it will never see embodiment
Post Reply

Return to “Announcements and Site Administration Topics”