Simple Methods for Market Growers to Assess Culinary Quality with Consumers

Introduction

Consumers expect superior flavor, texture, and appearance from fresh produce. This is especially true of produce grown organically and purchased from local farms (Bonti-Ankomah and Yiridoe, 2006). Farmers selling direct to local markets value these qualities when choosing varieties, but they may not have a way to systematically evaluate them. This article discusses simple methods for doing culinary quality evaluations, so farmers can make informed decisions about the best varieties for their farms and markets. In this article, we define culinary quality as flavor, texture, appearance, and any additional desirable attributes for cooking or fresh eating, which may vary by crop.

Although many vegetable breeders consider flavor when developing vegetable varieties, end users are not often engaged in the evaluation process. This means that breeders may miss out on particular traits of interest to Community Supported Agriculture (CSA) customers, chefs, or other local food consumers. Gathering information on which varieties customers prefer not only helps farmers choose varieties, it also generates important feedback for plant breeders working in organic systems. More farmers trialing and evaluating the culinary quality of vegetables means more information to guide breeding decisions. The information you gain from a quality evaluation can be shared with other farmers and your customers to help you choose varieties. It can also be shared with your seed sources, particularly if the seed companies you work with are connected to the breeders of those crops. Many of the independent seed companies serving organic systems are either involved in plant breeding or are in close connection to the breeders of the seed they are selling. They are very interested in getting feedback on culinary quality!

Brief Background

The methods in this bulletin were developed by two programs working with end users to evaluate the culinary quality of vegetables and small grains: The Seed to Kitchen Collaborative (SKC) and the Culinary Breeding Network (CBN). Both organizations draw on recent research in rapid sensory evaluation, reviewed in Dawson and Healy (2018) and several years of experience testing and refining methodology with chefs and farmers.

The Seed to Kitchen Collaborative is based in Madison, Wisconsin and is a program of the Urban and Regional Food Systems Group in the Horticulture Department at the University of Wisconsin, Madison. Founded in 2014, SKC conducts vegetable variety trials under organic management on research stations in Wisconsin and works with farmers in the Upper Midwest to evaluate varieties in on-farm trials. As part of the trials, SKC staff evaluate flavor in all varieties and coordinate monthly tasting events with a group of 7 to 10 chefs. Each event focuses on one or two crops, from fresh tomatoes in August, to storage potatoes in March. Criteria for the evaluations are developed collaboratively with the chefs and plant breeders or the seed companies who contributed varieties.. All the produce for the evaluations comes from university-led variety trials conducted at the West Madison Agricultural Research Station. Culinary quality and agronomic information from the trials are distributed to plant breeders, seed companies and local farmers to help farmers choose the best varieties for their farms. Each year in the late summer or early fall, SKC hosts an annual event, called Farm to Flavor, which highlights outstanding varieties from the program. More information on SKC and Farm to Flavor can be found at https://seedtokitchen.horticulture.wisc.edu/.

The Culinary Breeding Network is based in Portland, Oregon and was founded by Lane Selman of Oregon State University in 2010. Selman noticed that while plant breeders often evaluate in-progress varieties for flavor, they do not usually involve end users in the process. After conducting an inaugural culinary quality evaluation of sweet peppers involving Portland-area chefs, farmers and plant breeders, she launched CBN. Her goal is to ensure that breeders benefit from the perspectives of farmers, chefs and eaters, and farmers and culinary professionals benefit from varieties selected with them in mind. CBN hosts an annual event called the Variety Showcase at which chefs, bakers and brewers partner with plant breeders to highlight new and novel varieties in dishes designed for the event, which is open to the public. More information on CBN can be found at https://www.culinarybreedingnetwork.com/.

Raw and pureed samples of winter squash at a Culinary Breeding Network event. Photo credit: Shawn Linehan.

How to Set Up an Evaluation and Who to Involve

There are many ways to evaluate culinary quality, from informal taste tests with your crew in the field, to more formal evaluations with a group of culinary professionals. No matter where your evaluations fall on this spectrum, controlling for certain variables will help ensure reliable results.

Environmental Variables

Many of the same environmental factors that influence the productivity of crop plants can also influence culinary quality. These include temperature, rainfall, soil quality, wind direction and pest or disease pressure. It is important to design culinary quality evaluations to account for these factors. First, try to evaluate crops at peak maturity and during a stretch of average weather. If you evaluate sweet corn that is slightly past the harvest window, or tomatoes after a big rainstorm, the flavor and quality will not be representative of how that crop performs under ideal or typical conditions.

Second, it is important to account for variation within the field. At their research station, SKC uses a randomized complete block design in our trials to account for field variation. If you are interested in doing a variety trial on your farm in addition to a culinary quality evaluation, there is more information in the The Growers’ Guide to Conducting On-Farm Variety Trials (2018). Culinary quality evaluations can be done on varieties you are growing for market that are not part of a variety trial on your farm; however, to ensure that you are evaluating flavor differences due to variety and not due to soil type or management, all the varieties should come from the same field. Pick samples of the same variety from multiple parts of the field and combine them to ensure that no single fruit, root, or leaf is considered singularly representative of the variety being evaluated. Take note of the parts of the field where water tends to collect, where the soil may be more compact, or where pest or disease pressure is particularly bad. Be sure to avoid focusing your evaluation on these spots.

Minimizing Bias

Once samples are collected, the evaluation process should be designed to be as objective as possible. There are several ways to do this. First, it is a good idea to disguise the names of the varieties you are evaluating. Variety names are often catchy, so evaluators may subconsciously favor a variety if they like the name. Labeling varieties with a three-letter code instead of the variety name can prevent name bias. Just be sure not to lose the key linking codes to variety names! It’s best to avoid common three letter acronyms like IRS or OMG. These can also induce unconscious bias. Second, have evaluators try samples in a random order. People often have stronger reactions to the first variety they taste than to subsequent varieties. Having all the evaluators try samples in a different order reduces the influence of tasting order on results. Third, ask evaluators not to express their opinions in any way until the evaluation is over. Hearing how a friend feels about a variety or seeing someone scrunch up their face can easily sway an evaluator’s opinion. Offering earplugs to evaluators is a good way to keep them from talking to each other or getting distracted by ambient noise.

In general, it is a good idea to make the evaluation environment as comfortable as possible, so evaluators can focus on each sample. Evaluators should be in good health on evaluation day; a stuffy nose will greatly affect a person’s ability to taste.

Pepper samples are prepared and presented uniformly to minimize evaluator bias. Photo credit: Shawn Linehan.

Involving the End User

Flavor evaluations are a great way to generate valuable information while also conducting outreach to current and potential customers. Doing a taste test at a farm tour or farmers market can generate enthusiasm among general customers. Engaging culinary professionals in an evaluation can help farmers select particular varieties for high-end markets and potentially build new sales relationships. Who participates in an evaluation will influence how the evaluation is structured. As a general rule, a baseline of reliable information can be obtained either from a large group of untrained evaluators, or a smaller group of professional evaluators (Frost et al., 2015). In a large public group, extreme data points will cancel each other out to reveal general trends. In a small professional group, much more precision can be expected from tasters, so fewer data points are needed to make trends apparent. A large group of professional evaluators is, of course, ideal! The Culinary Breeding Network often involves chefs, plant breeders and experienced produce growers together in evaluations.

Chefs are highly experienced tasters, as well as potential local food customers. Take advantage of chefs’ expertise by evaluating traits important to them. For example, the general public might be interested in how a winter squash tastes roasted, but chefs might also be interested in the taste and texture of a raw, shaved squash for salads. Chefs might also be interested in single serving sized fruits, or more powerful flavors (like bitter greens) and their preferences may influence culinary trends.

‘

Chefs, farmers and plant breeders evaluate tomato and pepper varieties at a Culinary Breeding Network tasting event for the Northern Organic Vegetable Improvement Collaborative. Photo credit: Shawn Linehan.

Farm to table restaurants and chefs interested in supporting local farmers have had a large impact on the local food movement. Understanding the end users’ priorities will help make evaluation criteria maximally applicable to real world scenarios. The chefs involved in SKC and CBN are interested in finding varieties that work for both their restaurants and the local farmers they purchase from. They do not want a variety that has excellent flavor but is difficult or expensive to grow locally. They have been key stakeholders supporting varieties that combine productivity and culinary quality.

Regardless of the audience, it is important to make the evaluation setting clean, streamlined and attractive with as little variation between samples as possible. Using identical bowls or plates (white or colorless/clear is best; avoid colors or patterns) to display samples and cutting samples in to bite-size pieces in the same way for each variety will reduce bias. Providing clean toothpicks for picking up samples, and offering water and unsalted crackers as a palate cleanser will also enhance evaluators’ abilities to focus on the task at hand. See the reference list for helpful guidelines on food safety considerations when preparing samples.

It is important to taste the crop in the manner in which people are accustomed to eating it. For example, sweet corn tastes great in the field raw, but most consumers eat it boiled a few days after harvest. Varieties can taste and perform quite differently in different applications. When tasting red bell peppers, for example, you would ideally taste them raw, sautéed, and roasted, as all three preparations are common. However, consider the amount of labor required to prepare samples and whether your evaluators can handle tasting that many samples. “Palate exhaustion” is a real concern, especially when working with the public rather than culinary professionals. You may want to focus on just one preparation at a tasting. In addition to the potential for palate exhaustion, any type of cooking takes a great deal of care to ensure that all samples are treated uniformly. Slightly different times in boiling or roasting can drastically change the texture and overall qualities of prepared samples. If cooking, it is best to have someone experience do the preparation, or practice ahead of time. A rule of thumb is to evaluate 6 varieties at a time, with no more than 5 questions about each, in 1 to 2 preparations.

Description vs. Preference

At the beginning of an evaluation event, the host will provide each participant with an evaluation sheet that includes questions aimed at obtaining the desired information about the varieties in question. If the evaluation event is a one-off, it may not be necessary to collect evaluators’ names, but it is necessary to be able to link evaluation sheets from each individual if there are separate sheets for different samples. If the same evaluators will be involved in recurring evaluations, it is important to have the names or some form of identification on the evaluation sheets so you can see whether one evaluator consistently differs from the rest of the group. This may indicate that the one evaluator tastes things differently from the general populace or doesn’t fully understand what is being asked. If it becomes clear that one evaluator is dramatically shifting the trends of the group, consider removing this person’s data from the final analysis. If two or three people differ from the group, they may represent a segment of the population that tastes differently or has different preferences. This can be the case if there are different cultural backgrounds or ages of evaluators. In this case it is important to consider the preferences of the different groups rather than just taking an average across all evaluators.

Culinary quality evaluations fall into two general categories: descriptive and preference-based or “hedonic.” Descriptive evaluations ask the evaluator to objectively rate certain qualities present in the sample. Hedonic evaluations ask the evaluator whether, how much, and why they like the sample. In more scientific sensory analysis, descriptive and hedonic evaluations are generally kept separate, since evaluators may be biased in their descriptions based on whether or not they like something. In practice, for culinary quality evaluations, it is possible to combine these two types of evaluation by asking descriptive questions first, and hedonic questions second. This is not ideal, but it is a balance between minimizing bias and getting enough relevant information. Both types of information are often needed for making decisions, and having separate tastings is not practical.

If you are only trying to find out whether your target audience likes a certain variety, hedonic information is sufficient. If you would like to know what types of varieties your customers like and why, then both types of information are useful. For variety selection it is important to know something about the culinary attributes of each variety and whether that combination is desirable. Over time, if it is clear that certain characteristics are always preferred, you may be able to do just one type of evaluation or the other. In general, when working with the general public, preference data is the most reliable, and easiest to get. When working with professionals, both types of evaluation are possible.

If you are asking both types of questions at the same time, is important to separate the descriptive and hedonic questions in evaluator’s minds. For example, an evaluation sheet might begin by asking for a 1 to 5 rating of sweetness, acidity, bitterness and flavor intensity for each variety. After evaluators have rated all varieties, they may be asked to go back and end with a 1-5 rating of overall liking for each variety. This effectively separates descriptive from hedonic evaluation of culinary quality but does require evaluators to taste samples twice and may increase the time commitment or decrease the number of samples you can evaluate. A secondary option is to do a descriptive evaluation with professionals such as chefs and then using the same varieties, do a hedonic evaluation with the general public at a farmers’ market stand or CSA pickup. How much the evaluation focuses on description or preference depends on the goals of the evaluation and the interests of the end users.

Different Methods: Rating, Description and Comparison

There are many ways to design an evaluation questionnaire. The simplest is to use a rating system. The Seed to Kitchen Collaborative uses 1 – 5 or 1- 9 (1 is typically “low”) rating scales to describe culinary quality elements such as sweetness, acidity, bitterness, umami (savory flavor), and flavor intensity (Figure1). The Culinary Breeding Network also uses a 1-9 scale, advising evaluators to use only the odd numbers and reserve the evens for truly intermediate scores. Rating scales are also used to describe hedonic perceptions such as overall liking.

Date:___________________

Crop:___________________

Tester Name:_____________

Instructions

1) For appearance, rate how appealing each variety looks: what is the likelihood you would purchase this variety at a market?

2) For texture, this is a subjective score, 1= poor, 5 = excellent

3) For each flavor category, note the strength of that particular flavor component. Try to be objective, without thinking about whether you like the strength of that component.

4) Umami is a savory flavor component. It is present in foods like mushrooms, soy sauce and tomatoes. Only score for tomatoes.

5) For intensity, 1=flavorless, 5 = very intense. This should relate to the degree of the characteristic vegetable flavor of this crop, not to any off flavors. Try to be objective, without thinking about whether you like the degree of intensity.

6) For overall category, give your global appreciation (1= poor, 5 = excellent) of the flavor of each variety, excluding the appearance of the variety. Do this at the end, once you have filled out the rest of the datasheet.

7) For unusual flavors, note any particularly strong flavors or anything that tastes “off”. Write in what you are tasting, and leave blank if you do not taste anything unusual.

1=low/poor 2=moderately low 3=moderate 4=moderately high 5=high/excellent

Read more here:: Simple Methods for Market Growers to Assess Culinary Quality with Consumers