Farmer-Led Research FAQ

Q. What kind of design can I use to answer my question?

A. There are different types of research design to help you explore your on-farm curiosity. The choice of design and method of examination depends on the specific question, your motivation and capacity.

OBSERVATIONAL STUDY: An observational study involves collecting and analyzing data without changing anything. Aspects to observe depend on your specific curiosity, and may include emergence, growth rate, colour, health, smell, soil quality, harvest ease, grazing preference, etc.

• Observational studies can occur on a single farm over one season, but more confidence comes with observing over multiple seasons or on multiple farms, if possible.
• Observational studies are less likely to be funded through EFAO’s Farmer-Led Research Program unless there is clear justification for why this design is appropriate and can produce impactful results.

SIDE-BY-SIDE DEMONSTRATION: A side-by-side demonstration – sometimes called a “look-see” – compares changes in management (i.e. a treatment) to a “business-as-usual” no-treatment control.

• Demonstrations are not randomized or replicated, and usually involve side-by-side strips of treatment and control. Things to consider when setting up demonstrations include selecting a piece of land that is relatively uniform across the strips. 
• Side-by-side demonstration studies usually occur as a standalone on a single farm over one season. If you repeat a side-by-side across 3+ fields or farms or 3+ more seasons, then you have a replicated trial and can often use statistical models to assess the difference between or among your treatments.
• Standalone demonstration studies are less likely to be funded through EFAO’s Farmer-Led Research Program unless there is clear justification for why this design is appropriate and can produce impactful results.
• Refer to On-Farm Demonstrations Research Guide for more information on how to design a demonstration study.

RANDOMIZED & REPLICATED TRIAL: A randomized and replicated trial compares multiple pairs/sets of treatments or comparisons, and the treatments or comparisons are randomly assigned in order to avoid bias.

• Replicating and randomizing allow you to use statistical models, like t-tests (for 2 treatments, or pairs) and analysis of variance (ANOVA; for 3+ treatments, or blocks) among others, to test whether a difference between or among treatments is by random chance or due to the treatment you are testing.
• Read more about replication here and randomization here.

SCREENING TRIALS: Sometimes variety selection start with a preliminary year of screening, where the farmer can evaluate whether a variety merits consideration for a replicated trial. This involves observing a larger number of varieties than is practical in a replicated trial; checking for trueness-of-type or other seed quality concerns; evaluating the overall uniformity of an adequately large population; identifying potential strengths or weaknesses of a variety; or experimenting with a core question that is less likely to be influenced by field variation (e.g. does this variety sell well at market?).

• Screening trials evaluate a larger number of varieties in only one plot per variety (i.e unreplicated) on one or more farms.
• Screening trials are more likely to be funded with strong justification for the importance of the specific crop.
• Refer to The Grower’s Guide to Conducting On-farm Variety Trials by the Organic Seed Alliance for more information.

VARIETY TRIALS: A variety trial helps farmers manage risk by identifying optimal genetics for a grower’s unique environmental, production system and market conditions. They are appropriate when the farmer has narrowed the varieties to a practical number for replicated evaluation and has practical and informative metrics for evaluation.

• Variety trials evaluate each variety in multiple plots at the same time following a replicated experimental design.
• Variety trials are more likely to be funded when 3+ farms are involved, each with at least two replicates and a check variety. This design is sometimes done with “mother-daughter” sites, where the mother site assesses all varieties and the daughter sites assess a smaller selection of sites.
• Refer to The Grower’s Guide to Conducting On-farm Variety Trials by the Organic Seed Alliance for more information.

PLANT BREEDING TRIAL: A breeding trial selects traits in order to change the genetics of a genetically variable population and produce a new variety with desired characteristics.

• The traits that are chosen and the methods of selection vary according to the breeder. Many farmer-led breeding trials select for a holistic suite of traits including management ease, regional adaptation, storage and flavour. 
• Before committing to a long-term breeding project it is important to evaluate the potential parental material that you’ll start with. This typically takes place by doing a variety trial of varieties that most closely have the traits you are looking for. 
• Refer to Introduction to On-Farm Organic Plant Breeding and Participatory Plant Breeding Toolkit by the Organic Seed Alliance; and Breeding Organic Vegetables: A Step-by-step Guide For Growers by Rowen White and Bryan Connolly for more information.

Q. What is a treatment?

A. A treatment is something that you do or apply to a crop or field in randomized and replicated trials and demonstrations. For example, a treatment could be a cover crop mix, compost rate, grazing management, seeding method, etc.

Q. What is a control?

A. In the context of randomized and replicated trials and demonstrations, a control is your “business-as-usual” management. In the context of breeding and variety trials, a control is the “check” or standard commercial variety. In all cases, the control is used to measure the effect of the treatment/trial variety, compare the growing conditions of the trial season to seasonal averages, and communicate results to others.

Q. What is an experimental unit?

The experimental unit is the entity that you want to make inferences about. With farmer-led research, we rarely want to make inferences about a single head of lettuce or a single heifer, for example. Rather, we want to know how a bed or harvestable unit of lettuce responds, or a group or herd of cattle respond. For this reason, an experimental unit on the farm is usually representative of our crop. This could mean 15 heads of lettuce, a strip of corn, 20 head cattle, or 6 saskatoon bushes, etc. Deciding on the specifics of an experimental unit comes down to your instinct as a farmer and the practicalities of your trial.

Q. What is a replicate and how do I design one?

A. A replicate is a multiple of your pair or set of treatments, including your control treatment. You can replicate “in space” by setting up multiple pairs or sets of plots, beds, strips, fields, paddocks, farms, etc. Or you can replicate “in time” by repeating the comparison at multiple times like in different succession plantings, cohorts, or years.

Replication is important because it gives you a “second opinion”. With only one pair or set of strips/plots/beds/animals/etc. in a trial, your conclusion is based on one observation and you do not know if the results you see are a random fluke or a real treatment effect. By replicating the experiment in multiple places or at multiple times, you can be certain that the effect measured is consistent.

EXAMPLE OF REPLICATING “IN SPACE”: You are comparing tillage (control) to two cover crop treatments in strips and you decide to test this with 5 replicates. This design results in five sets with three strips each. Each strip is randomly assigned to one of the cover crop treatments or tillage, for a total of 15 strips or plots.

EXAMPLE OF REPLICATING “IN TIME”: You are comparing a standard feed ration (control treatment) to a reduced protein treatment for raising chickens and you decide to test this with 4 replicate groups of chickens. This design results in 4 cohorts of chickens, with each cohort divided into two groups that are randomly assigned to one of the feed rations.

For other examples of replication, please see previous farmer-led research reports at: efao.ca/research-library.

Q. How many replicates do I need?

A. For a randomized and replicated trial, you need a bare minimum of 3 replicates. This allows you to run basic statistics and determine if any difference you see is due to a random fluke or a real treatment effect.

That said, we prefer at least 4, with 5 or 6 replicates being ideal. A minimum of four replicates gives you room in case one replicate fails or is damaged. In general, the more replicates you have, the greater your ability to detect a difference between or among treatments. If you are going to the effort of setting up a trial, you might as well give yourself the best chance of getting reliable data!

Q. What is randomization and why is it important?

A. Randomization is assigning treatment(s) and control to each experimental unit by chance. By doing so, you eliminate any bias you have. Randomization also allows you to account for any effects of heterogeneity of the soil or land (i.e. drainage).

Randomizing can be done in several ways. If you have only two treatments (or treatment and control), you can flip a coin to assign them. If you have more than two treatments, you can assign a treatment order by picking out of a hat. In all cases, you can use a random number generator in a spreadsheet program using function: =rand(). 

In general, we try to randomize in pairs or blocks. This means that treatments are not randomly assigned across all experimental units, but are randomly assigned within pairs (in the case of two treatments) or blocks (in the case of more than two treatments).

RANDOMIZING WITH A COIN: You want to assign 2 varieties of spinach to 8 beds, for a total of 4 replicate pairs. First, you denote variety A as heads and variety B as tails.

• Randomized paired design (standard way): Divide the 4 beds into 2 pairs. Flip a coin and assign the corresponding variety to the first bed in the first pair. Assign the other variety to the second bed. You have now randomly assigned the first pair. Do the same for the second pair.
• Completely randomized design (less conventional, but has its place): Flip a coin and assign the corresponding variety to bed 1. Flip the coin again and assign the corresponding variety to bed 2. If both beds are the same variety, assign beds 3 and 4 to the other variety. If beds 1 and 2 have different varieties, then flip the coin again to assign a variety to bed 3. Bed 4 is assigned by elimination.

RANDOMIZING WITH A HAT: You want to assign 6 varieties of spinach to 30-half beds (15 full beds), for a total of 5 replicates. 

• Randomized block design (standard way): Divide the full beds into 5 sets or blocks of 3 beds (6 half beds) each. You want to assign varieties in such a way that each block has every variety. Cut up 4 equal sized pieces of paper, write the name of each variety on a piece, and place the paper pieces in a hat. Shake up the paper, draw from the hat and assign the corresponding variety to the first half bed in the first block. Without replacing the paper, draw a second piece of paper from the hat and assign to the second half bed, and repeat for the other two beds in the first block. Replace all paper into the hat and repeat for the other 4 blocks.
• Completely randomized design (less conventional, but has its place):  Cut up 30 equal sized pieces of paper, write the name of each variety on 5 pieces each, and place all 30 pieces in a hat. Shake up the paper, draw from the hat and assign the corresponding variety to the first half bed. Without replacing the paper, draw a second piece of paper from the hat and assign to the second half bed. Repeat until all pieces are drawn and all half beds are assigned.

RANDOMIZING WITH A RANDOM NUMBER GENERATOR: You want to assign 4 varieties of barley to 20 strips, for a total of 5 replicates. 

• Randomized block design (standard way): Divide your 20 strips into 5 blocks or sets with 4 strips each. You want to assign varieties in such a way that each block has every variety. In Column A of a spreadsheet, enter “V1”, “V2”, “V3” or “V4” into 4 cells. Copy and paste this grouping 4 times down the column, for a total of 20 cells (A1 through A20). In Column B of the spreadsheet, enter function “=rand()” in B1 through B20. Select/highlight columns A and B and copy and paste formula results or paste as values only. For each group or block, assign the variety with the lowest number to the first strip in a block, the variety with the second lowest number to the second strip, etc. for the other two strips in a block. Repeat this process of assigning varieties within a block to the other replicates.
• Completely randomized design (less conventional, but has its place): In Column A of a spreadsheet, enter “V1”, “V2”, “V3” and “V4” in 5 cells each, for a total of 20 cells (A1 through A20). In Column B of the spreadsheet, enter function “=rand()” in B1 through B20. Select/highlight columns A and B and copy and paste formula results or paste as values only. Keeping the columns selected/highlighted, sort by Column B. Use this order to assign varieties to your half beds.

Q. I want to test whether an amendment like a type of fertilizer, microbial inoculant, or soil booster works. How should I proceed?

A. It is often difficult to detect the effectiveness of an amendment. For this reason, we recommend a two-step process for testing the efficacy of an amendment.

STEP 1: Compare the effects of the amendment in a side-by-side or observational study. This could look something like adding the amendment to a strip or plot in your beds or field beside a strip without the amendment. Even better – try multiple pairs of strips! Keep in mind that paired strips should be as similar to each other as possible in all other regards, including similar soil type, drainage, shade, crop, crop history, etc.  

• Record notes on the details of your set-up like the factors mentioned above, amendment rate, application date(s), management; and any observations that are relevant and practical to your interest in the amendment. This may be things like crop health, pest control, yield, etc. Also remember to take lots of photos of your trial.

STEP 2: If you observe a difference and still want more information, you have good preliminary data to apply to EFAO’s Farmer-Led Research Program. Follow the instructions in the application, using the observational data and information you recorded in your side-by-side plots.