Experience with Experiments: Factorial Design

    Let's say you're mixing the ultimate smoothie. You're putting in all your favourite fruits: mango, apple, banana, strawberry, blueberry, pineapple, just everything. But you're not just doing this once, no, you've got to make this smoothie perfect; so you try out different amounts of each fruit in different balances. You try out different amounts of milk, maybe you add crushed ice, any factor you can think of that might affect the taste.

    Finally, once you have your smorgasbord of smoothies, you give them out to friends to enjoy, and give them a rating (as you still need to decide which smoothie is truly the ultimate smoothie). Obviously you haven't made every smoothie possible from the ingredients you have, the number of combinations is endless...but how might you decide which combination is the best? Is there an easy way to find out if a combination you haven't tried is better than any of the ones you have made?

     This is one way of designing an experiment! In this case, a good way to try and discover what the ultimate smoothie is would be through a technique called factorial design. In factorial design, we take all the factors that might impact the result of the experiment, and experiment through multiple trials where a factor is either at the highest amount or lowest amount we would use. In this case, we could choose the lowest level as a single cube of a fruit, maybe an inch on each side, and the highest level as 5 of those cubes. We could also have high and low levels for the amount of milk, or the amount of crushed ice.

      The next step would be doing multiple trials of all these experimental smoothies. Now for our ultimate smoothies, we might try out combinations of 10 different fruits, but for simplicity, lets consider an experiment with just mango, strawberry, and crushed ice as factors. We could visualize our experiment as a cube, where moving along an axis (as in the diagram below) represents changing the amount of a factor. 

(Ruttiman, 2015)

We would have: a = amount of mango, b = amount of strawberry, c = amount of crushed ice. The red center-point would be a test where we took the average amount of all 3 levels. Once we’ve created these 9 different smoothies, we could do multiple trials of each type, and have people do a taste test and rate each smoothie from 1-10. This would give us a response variable of the average taste score.

      Once we have this data, we might fill out a chart like below:

# of mango cubes	# of strawberry cubes	Amount of crushed ice (0 = none, 1 = half cup)	Taste Score
1	1	0	6.0
1	1	0	7.5
5	1	0	8.2
5	1	0	9.4
1	5	0	3.2
1	5	0	7.5
1	1	1	4.0
1	1	1	5.1
5	5	0	3.2
5	5	0	4.6
5	1	1	8.1
5	1	1	9.2
1	5	1	4.2
1	5	1	6.0
5	5	1	4.0
5	5	1	4.0
3	3	0.5	8.1
3	3	0.5	7.6
3	3	0.5	6.8
3	3	0.5	7.2

    We could then plug this into modelling software (such as Minitab), which could show us based on the results we got, what the likelihood of different factors being significant to the taste score is. What we would get is an analysis like below. This analysis shows us what the effect of each factor is, and the effect of the interaction terms. Interaction terms are basically the effects of combining multiple flavours; for example, maybe individually increasing mango or strawberry doesn’t affect taste much, but together it does have a strong effect!

    There are a lot of numbers in this analysis and it may seem like nonsense, but for now, the only numbers we are interested in are those in the p-value column. The p-value is a measure of whether each factor is likely to have an impact on our experiment. For simplicity’s sake, we can just say if the p-value is greater than 0.01, the factor probably doesn’t matter. So, we can adjust the model we’ve come up with by deleting factors and interaction terms one at a time and rerunning the software until every p-value is significant, or <0.01. As we remove factors 1 by 1, the p-values will change, as the factors relative significance to each other change.

    Then, we end up with this (comparatively) much simpler analysis:

    You might notice that the Mango p-value is still >0.01, but we keep it because the p-value for the interaction between mango and strawberry is <0.01. It wouldn't make sense to remove mango from our model when we know the combination of the mango and strawberry effects is significant. From our simplification, we’ve found out that the amount of ice made almost no difference to the average taste score of our smoothies! The analysis software can also provide us estimates of how effective each factor is at influencing taste.

This gives us coefficients for a basic formula for our taste score. Basically, using the data we've provided, its told us that if we gave someone a smoothie with 1 mango chunk and 1 strawberry chunk and no ice, they would give it an average score of 5.023, and then gives us what the effect of adding more mango, strawberry or ice will do to the score. This tells us that we can estimate our taste score as:

Taste score = 5.023 + 1.041*Mango chunks + 0.166*Strawberry chunks – 0.272 Mango Chunks*Strawberry Chunks.

 

    From our experiment, we’ve found that while adding more mango and more strawberry makes the smoothie tastier, the combination of flavours actually decreases the taste score more than strawberry increases it! We could then see that to get the highest taste score, we could try putting in as much mango as possible and as little strawberry as possible. If we truly wanted to create the ultimate smoothie, we could redo this process, but instead of with 3 factors, we could have as many as we wanted.

    So there you go, an introduction to experimental design, and a statistician's strategy for making the best smoothie ever (add a lot of mango), all in one.

Image Citation:

Ruttiman, B. G. (2015, January). Design Space of a 3-factor 2-level full Factorial DOE with center point [Digital image]. Retrieved October 19, 2020, from https://www.researchgate.net/figure/Design-space-of-a-3-factor-2-level-full-factorial-DOE-with-center-point_fig1_276166727

Coverly, D. (2014, March). Speed Bump [Cartoon]. Retrieved October 27, 2020, from https://www.thecomicstrips.com/comic-strip/Speed+Bump/2014-03-21/108599

Gore, M. (2020, May). Best Triple Berry Smoothie [Digital Image]. Retrieved October 19, 2020, from https://www.delish.com/cooking/recipe-ideas/a24892347/how-to-make-a-smoothie/

Ann, M. (2016, August). Mango Peach Smoothie [Digital Image]. Retrieved October 19, 2020, from https://thebeachhousekitchen.com/mango-peach-smoothie/