By Charles Copley, PhD
Director of Data Science
Ancera
Every year, companies serving the poultry industry pool their expertise to develop new vaccines, pharmaceuticals, litter treatments, antimicrobials and feed additives for poultry and other food-animal sectors — all of which will be scrutinized during trials and sales cycles.
As poultry companies continue to increase their oversight of products in response to tightening margins, suppliers are under greater pressure to provide third-party data demonstrating product efficacy.
Pen trials have traditionally served as the gateway to customer adoption; however, they often fail to represent the total population and environmental conditions of the real world.
Many integrators and suppliers such as Ancera are therefore adopting a new approach — one that leverages data science and simulations to improve trial outcomes, increase sales of efficacious products and reduce the cost of servicing customers.
Thousands of simulations, one trial
Data science and probabilistic modeling — a statistical technique that factors in the impact of random events or actions in predicting the potential occurrence of future outcomes — are powerful tools for simulating hundreds of outcomes in preparation for a high-risk mission with a low tolerance for failure.
For example, when planning space missions, NASA uses computer-generated simulations as a low-cost way to examine the probability of success for orbit trajectories, landings, environmental conditions, crisis scenarios and other factors that have an unbearably high cost in the real world.
Core to the simulated trial design is a technique called Bayesian modeling, a method of statistical inference used to update the probability of an outcome with additional data. It’s particularly well-suited toward smaller data sets, such as understanding an entire population based on representative samples. In fact, in recent years, Bayesian adaptive clinical trials have become increasingly popular in human medicine due to their flexibility and cost efficiency. They can be used for evaluating poultry products, too.
Aligning on success criteria in advance
To provide computer-simulated trials and estimate their probabilities of success, technology partners serving the poultry industry can set their success criteria and trial design before absorbing the cost in the field.
Identifying the goals of the trial is the first step toward simulating quantitative results. While field-trial simulations can be used on a range of products — vaccines, probiotics and management tools, to name a few — for the purposes of this example, we will examine a feed additive used to reduce Salmonella loads in live production.
Using new technology, companies that offer products and services to the poultry industry can work with Ancera to identify Salmonella serotypes in their samples and set specific Salmonella-reduction goals. Goals are tailored for success, and Ancera works with allied companies to understand objectives and expectations. For example, a very modest goal might have a high probability of success, but it may be attributed to “noise” in the data rather than product efficacy. Detecting small differences in a trial may have a lower likelihood of success, but results are more accurate and can be attributed to product efficacy.
Designing for real-world evidence generation
Trial simulations are designed for the principles of real-world evidence. Pen trials often fail to account for the operational, biosecurity and sampling constraints present in live operations. Ancera’s trial simulation tools allow industry partners to work with integrators to design a field trial based on parameters such as:
- Determining if a producer is able to split feeds
- Monitoring for control and treatment groups
- Requirements to look at the same houses longitudinally
- Studying subsequent flocks
Incorporating prior assumptions
To understand pre- and post-intervention efficacy, a trial design must incorporate critical assumptions to detect sensitive changes from existing production data.
When the focus is Salmonella, Ancera works with customers to create prior datasets to determine pre-intervention Salmonella prevalence (the proportion of houses with positive detections) as well as the pre-intervention load (a Log number, accompanied by an expected standard deviation).
Setting trial-design characteristics
When designing a trial, companies serving the poultry industry should take into account their access to a poultry company’s locations and their operational constraints. For example, testing more houses during both pre- and post-intervention will increase the sample size and help add greater certainty to the study. However, it may also increase the complexity and cost.
With Ancera’s proprietary diagnostics, which use assays compatible with non-invasive fecal samples and boot swabs, vendor companies can increase the scalability and unlock greater sample sizes at a more reasonable budget.
Finally, because the simulations are based on random variables and probability distributions, they can be set to run multiple times, allowing companies to gather a robust dataset to understand their likelihood of success prior to deploying costly resources in the field.
Some important questions to ask when designing trials:
- If the product reduces Salmonella load by 1 Log (90%), what data, samples and diagnostics will be required to substantiate that with some level of confidence?
- If the reduction is less than 1 Log, will it be detectable at functional levels?
- At what level is a detection limit reached? The lower the level, the more important it is to distinguish results.
Analyzing simulated trial results
Once technology partners have established success criteria, constraints, assumptions and trial-design characteristics, the models allow scientists to preview the kind of data and results they are likely to obtain, thus making it possible to make up for cost-benefit estimations objectively. Ancera works with technology partners to demonstrate a change throughout a trial and measure that difference with a degree of confidence based on simulated data.
Keep in mind, the goal of this notional example is to understand the chance of success in detecting a 1-Log reduction in Salmonella load. We are assuming a starting prevalence in 50% of the houses with mean pre-intervention load of 2.51 Logs with a standard deviation of 1.3.
We plan to test for 1 week prior and 1 week after any intervention or product is applied. We are modeling four houses per week with eight samples per house and have asked the model to calculate five trials with 2,000 simulations per trial.
The second of five simulated trials calculates a 91.85% probability of detecting a 1-Log reduction between pre- and post-intervention results. Note the Log 10 MPN reduction between the blue (pre-intervention) and orange (post-intervention) distributions.
Simulation 2
Click to enlarge
The additional simulations provide expected pre- and post-intervention distributions based on random chance that would have been in the real world, but overall, the combined trial results de-noise this, leading to an 86.40% probability of detecting a 1-Log reduction.
Combined trial results
Click to enlarge
Summary takeaways
- It is critical for technology partners to align on quantitative-success criteria and starting assumptions, as they will impact the trial-simulator results. In some cases, if there is not enough data for a pre-trial starting assumption, it may make sense to conduct a small pre-trial investigation for an accurate starting point.
- Try different objectives and trial designs to improve the probability of success while still being able to make cost-benefit decisions. The benefit of the trial simulator is that it provides a low-cost way to model several inputs and outcomes that ultimately determine whether or not your product efficacy is demonstrated in a live customer environment.
- A structured simulation allows technology partners and integrators to align ahead of time on specific, valuable and actionable product-efficacy requirements as well as a feasible trial design. A common framework for the trial can reduce the cost of the trial through more intelligent sampling but also ensure that any labor and operational inputs are used efficiently before anyone goes into the field.
For help with planning your next field trial, contact Ancera to learn more.
Editor’s note: Content on Modern Poultry’s Industry Insights pages is provided and/or commissioned by our sponsors, who assume full responsibility for its accuracy and compliance.
