A/B Testing: Simple Table Bread and the Order of Operations

WELL THAT DID NOT WORK

I tried to run a clean A/B bread test. One missing instruction turned it into a rather involved three-loaf experiment. The corrected bloom won, and the mistake I made explains why we believe NoAdsCooking.com has room to exist in the very noisy space of internet food resources.

Raise your hand if this is you.

You decide to make bread. You look up how to make bread. One recipe tells you to bloom the yeast. Another says blooming is unnecessary. Check the comments. BreadMama92 insists the water must be exactly 105°F while GarlicGoblin argues with her about how that doesn't matter because he swaps sugar for honey to get the best yeast activation. Really clear instructions for something so simple shouldn't be this hard, but here we are.

Meanwhile, the page keeps moving because another advertisement has loaded above the instructions. A shameless plug for NoAds|Cooking? Perhaps, but it's not like we're making any of this up.

Baking bread starts out with joy and quickly morphs into FUD even though no one is really trying to manipulate you. The info is just really bad.

There are thousands of ways to make the same loaf of bread, but there does not need to be this much confusion around flour, water, yeast and salt.

Under These Constraints This Happens

The creators of NoAds|Cooking have worked in digital and technology long enough to appreciate discrete variable control.

When something breaks in software, “the page does not load” is not enough information. Does it fail in Safari or Chrome? Every browser? Does it happen on network or off? Is the user signed in? Did it begin after an update? We narrow it down as quickly as possible.

The point is not to impress anyone with a recital of technical observations. It is to say, “Under these constraints, this happens.”

Bread and, frankly, most cooking instruction should work the same way.

Your great-grandmother’s recipe may have been flawless. She was also probably buying the same flour, using the same yeast and making it in the same kitchen every week with far less inconsistency between her and anyone else. Variability 100 years ago was far easier to control for.

Now buying flour can feel like consulting a pocket periodic table. Protein percentages. Gluten development. Hydration ratios. Instant yeast. Active dry yeast. Rapid-rise yeast. Must everything turn into the energy drink aisle/paradox of choice(like finding bread recipes online?!!?)

We just wanted a small loaf of table bread to eat with dinner.

Let’s narrow this down quickly. Bread flour? Check. Salt? Check. Fleischmann’s Active Dry Yeast? Check.

Now come the instructions, along with a heaping helping of analysis paralysis.

Which Yeast Are We Talking About

Recipes often use “dry yeast” as though every packet contains the same product. It does not.

Active dry yeast was traditionally dissolved in warm liquid before mixing. Fleischmann’s still instructs bakers to dissolve its active dry yeast in warm water; in fact, it is still printed on the back of its jar. King Arthur says modern active dry yeast can be added directly to the dry ingredients. Both methods appear in widely used recipes.[1][2]

Instant yeast is designed for direct mixing. Rapid-rise yeast is also direct-mixed but follows a faster schedule. Those products are related. They are not, however, identical instructions in different packaging.

The internet gives home bakers several procedures for a product carrying the same active dry label:

• Dissolve it in warm water and add the salt later.
• Dissolve it in warm water while keeping the salt in the flour.
• Mix the yeast and salt directly into the flour.
• Combine everything and start kneading.

Fleischmann’s own ciabatta recipe keeps salt in the flour while dissolving active dry yeast separately. King Arthur has tested dissolved and undissolved active dry yeast side by side and reports that modern active dry yeast can work either way.[2][3]

That did not tell me which method would be best in my kitchen or yours, or even indicate what differences I may encounter during or after the bake.

Therefore, a test is in order.

The Formula

Two small loaves, same formula:

• 200 grams bread flour
• 140 grams water
• 4 grams kosher salt
• 2 grams Fleischmann’s Active Dry Yeast

Same flour. Same yeast. Same ingredient weights. Same kitchen. Same oven.

In the testing hypothesis world, an A/B test compares two versions. Between versions A and B of the experiment, one meaningful thing changes while everything else remains as stable as possible.

Example:

Hypothesis: NoAds|Cooking helps me make the best food I have ever had

Test: Make something intuitively without NoAds' articulate instructions, then make it again following the advice, best practices and sage wisdom aggregated across recorded time and brought to you here in a completely free, privacy-protected, account-less destination of sheer beauty (stay with me).

Compare: The undisputed winner has to be following along with NoAds|Cooking, especially when you are on a recipe and engage their "Focus Mode" (okay, we are done with the self-promotions, I think).

Back to our plan here.

Loaf A was the all-in method. I mixed the active dry yeast into the flour with the salt, then added the water.

Loaf B was the bloom-first method. I dissolved the active dry yeast in warm water before adding it to the flour.

One variable: yeast handling.

Except... that was not what I actually tested.

The Damn Salt

The procedure gave exact ingredient weights, water temperature, rising time and baking temperature. It did not clearly say where the salt went during the bloom.

So I put the salt into the warm water with the yeast. Why? Because I figured you had to dissolve it and nothing really said I didn't. I have made bread probably 50 times over the last three years and no matter what I do, it never feels the same. Like not being able to remember how to spell the word "definitely (FIRST TRY!)" when I am writing.

I was running a test about controlled variables and had accidentally changed two things at once and almost didn't even realize it. That last part is the problem.

Loaf A used dry yeast distributed through flour, with the salt also in the flour. Loaf B used pre-hydrated yeast sitting in salted water before meeting the flour.

That is not a clean A/B test. It was now what is called a "confounded comparison." If loaf B behaves differently, I cannot tell whether pre-hydrating the yeast caused it, whether the salted water caused it, or whether both acted together.

I was not thrilled.

A/B Tests and Multivariate Tests

An A/B test asks which of two versions performs better. A multivariate or factorial test deliberately changes more than one factor and tests the combinations. The point is not merely to produce more versions. It is to separate the effect of each factor and see whether the factors interact.[5]

Calling the loaves A, B and C does not automatically make this multivariate. If each loaf is simply a complete recipe version, it is a three-arm A/B/C comparison. It becomes multivariate when the individual factors and their combinations are planned.

A proper two-factor bread test could vary yeast handling and salt placement across four loaves:

Testing every combination would make this a small factorial experiment. NIST defines a factorial experiment as one that uses every combination of the selected factor levels.[5]

I did not run that experiment. I accidentally produced runs 1 and 4, then added run 3 later. Run 2 is missing, and if you want to tackle that one and email me at admin@noadsalliance.org to let us know how it went, we would be happy to hear from you! There was also only one loaf in each condition, so ordinary loaf-to-loaf variation could easily look more meaningful than it is. Normally, these tests need to be conducted at least several hundred times to reach a concrete determination, but I'm going with the idea that bread has been made a zillion times (probably more), and we are just looking for an idea of what works amid the abundance of bad information.

What we have now is better referred to as an exploratory three-arm kitchen test with one confounded branch (salt introduction), certainly not proof carved into stone. That does not make our observations useless. It tells us how carefully they need to be described.

Kneading A and B

At this point, I would have chosen the all-in method without hesitation. It was faster. It was easier. It felt better.

The First Rise

Both doughs were given the same 90-minute first rise.

The all-in dough was in the green bowl. The salted-bloom dough was in the yellow bowl.

The apparent winner was obvious.

The all-in dough had been easier to knead and had risen higher. The salted-bloom dough had been difficult from the beginning and was visibly behind.

But this is just one milestone in which we observe the differences; we still have to see the process through.

Shaping

I gently pressed each dough into a small rectangle, rolled it into a short loaf and placed it seam-side down on the same baking sheet.

Shaping removed some of the accumulated gas and tightened the dough. The second rise and the oven gave both loaves another chance to expand.

Then the Result Reversed

Both loaves were baked together at 425°F.

The salted-bloom loaf baked taller and looked better.

This was the dough that had been stiffer, harder to knead and smaller after the first rise. Nearly every earlier observation suggested it was losing.

Then it overtook the all-in loaf.

The easiest dough and largest first rise did not produce the better finished bread.

Cutting and Tearing the Crumb

I cut approximately two-thirds of the way through each loaf, then tore through the remaining third by hand.

The clean cut shows the size and distribution of the air pockets. The torn section shows how the crumb pulls apart. It exposes strands and internal structure that a knife can flatten.

The all-in loaf (left) tasted great.

I also gave Loaf A to someone who did not know which method it represented. Their review was simple: “This bread is amazeballs.” I have the text.

One taster is not a panel. It still matters. Loaf A was not merely acceptable. It earned an uncomplicated, enthusiastic reaction without anyone explaining the experiment first.

If it had been the only loaf on the counter, I would have devoured it and considered the method settled. Nothing about it tasted like failure. It was good homemade bread.

But it was not the only loaf on the counter.

The salted-bloom loaf (Right) tasted about the same, but its texture was better. It was chewier and less dense. The same amount of flour had expanded into a larger structure, giving each bite more surface and pull.

The flavor had not changed much. The experience of eating it had.

Enter Loaf C

The original A/B test could not be repaired. It could be extended.

Loaf C used the corrected bloom method: active dry yeast dissolved in plain warm water, with the salt mixed into the flour.

C immediately felt different. It was the easiest of all three doughs to knead and produced the strongest final proof.

It was also made later and baked as the only loaf on the tray. The temperature in the house had also changed, which means C cannot be treated as a perfect comparison with A and B. Ambient temperature is a noise factor, exactly the kind of variable that makes home-kitchen testing difficult.

C carried its volume through the oven. It held a balanced shape, browned evenly and showed no obvious collapse or seam failure.

The Final Crumb

The exterior had already won. Then I cut and tore it.

Version C won.

It was the easiest dough to knead, produced the strongest final proof, kept its volume in the oven and finished with the most open, developed crumb. The corrected method produced the complete loaf I wanted.

That does not erase the changed room temperature or the fact that each condition was tested only once. It does settle which loaf won this round.

What the Three Loaves Show

• Modern active dry yeast worked when mixed directly into flour.
• Salt in the blooming water did not kill the yeast.
• The largest first rise did not produce the largest finished loaf.
• The easiest dough in the first comparison did not produce the bread I preferred.
• The all-in loaf still earned “This bread is amazeballs” in a blind taste test.
• The corrected-bloom dough was the easiest to knead and produced the strongest finished loaf.
• Equal ingredients and equal clock times did not create equal bread.
• One loaf per condition is enough to observe differences, but not enough to prove their cause.

What did we learn? Salt is not just flavor. It slows yeast fermentation and strengthens gluten, among other effects.[4] That makes its placement and concentration relevant to what happened, but this test does not isolate those effects cleanly enough to assign cause.

The result is not “blooming always wins.” It is more useful than that.

The all-in method was simple and produced bread someone called amazeballs without knowing anything about the test. The salted-bloom method was miserable to knead and still produced the better A/B loaf. The corrected-bloom method was the easiest of all and produced the best complete loaf.

That is what happened under these constraints.

The Small Table Bread This Test Produced

These loaves also settled the size of the recipe.

Two hundred grams of flour makes the table bread I actually want, and possibly the table bread you want as well. It is not a weekly sandwich loaf or an enormous boule that sits on the counter for four days and gets chucked when a mold spot appears. It is a small loaf placed beside dinner for three or four people to slice, tear or pull apart. It should accompany the meal, not fill everyone before the meal begins. There should be little or none left when dinner is over.

Ingredients

• 200 grams bread flour, approximately 1⅔ cups
• 140 grams warm water, approximately ½ cup plus 1 tablespoon
• 2 grams active dry yeast, approximately ⅔ teaspoon
• 4 grams kosher salt

Use the gram measurements for the most consistent result. Kosher salt varies by brand, so a volume conversion is not provided.

Method

1. Warm the water to approximately 105°F to 110°F. Dissolve the active dry yeast in the water and let it hydrate for 5 minutes. Dramatic foaming is not required; it might even look like sediment.
2. Mix the salt thoroughly into the bread flour. Add the yeast water and mix until no dry flour remains.
3. Cover and rest the rough dough for 15 minutes. Do not rush past this step.
4. Knead by hand for approximately 8 to 10 minutes, until smooth, pliable and slightly tacky.
5. Cover and let rise for approximately 60 to 90 minutes, until visibly expanded and aerated.
6. Gently press the dough into a rectangle approximately 6 inches wide and 4 inches deep. Starting with the 6-inch edge closest to you, roll the dough away from you into a compact loaf.
7. Pinch the seam closed, tuck the ends underneath and place seam-side down on a parchment-lined baking sheet.
8. Cover and proof for approximately 30 to 45 minutes, until visibly puffy. A gentle fingertip press should return slowly while leaving a slight indentation.
9. Bake at 425°F for approximately 20 to 25 minutes. Begin checking at 20 minutes. The loaf is ready when evenly browned and approximately 200 to 205°F internally.
10. Cool for at least 30 minutes before slicing or tearing. Serve the loaf whole at the table and finish it with dinner.

Sources and Further Reading

[1] Fleischmann’s Yeast 101. Manufacturer guidance for dissolving active dry yeast in 100°F to 110°F water.

[2] King Arthur Baking Active Dry Yeast. A side-by-side discussion of dissolved and undissolved modern active dry yeast.

[3] Fleischmann’s Ciabatta Bread. A recipe that dissolves active dry yeast separately while keeping salt in the flour mixture.

[4] King Arthur Baking Why Is Salt Important in Yeast Bread. An explanation of salt’s effects on fermentation, gluten, crust color and flavor.

[5] NIST Engineering Statistics Handbook Factorial Experiments. Definition of a factorial experiment and factor interactions.