A perfect day. A perfect meal. A perfect gentleman. A perfect score.
We all know what these terms mean. However, it is highly unlikely that any two of us would define the above four phrases in exactly the same way. Which is to say that perfection is in the eye of the beholder. So what about perfect data? Is it possible to generate perfect data in a clinical trial and, if yes, what would it look like?
According to Drs Cynthia Kleppinger and Leslie Ball of the US FDA, “No clinical trial is perfect and, therefore, no dataset is perfect.”1 This statement acknowledges the fact that, whenever humans collect data, they sometimes make mistakes. Training and experience can improve outcomes.2 Checklists and multiple reviews can catch many of the mistakes. But, eventually, humans get distracted, fatigued, bored, confused, or otherwise focused on something other than the task before them. That errant focus frequently fosters errors. Some suggest that eliminating the human element will enhance results. But, even when humans are replaced by machines or computers, mistakes occur due to design flaws, programming errors, power outages, and breakdowns.
So, should we just throw up our hands and accept the inevitable? I don’t think so. At the risk of appearing quixotic, I believe that the quest for perfection is worthwhile. Like an exponential curve, we may never reach our destination. However, through constant striving, we will move ever closer to our goal. But how will we know that we are making progress?
In an 1883 lecture on the units of electrical measurement, Lord Kelvin (aka William Thomson), the famous physicist, suggested that “… when you can measure what you are speaking about, and express it in numbers, you know something about it. But, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind. It may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the state of science…whatever the matter may be.”3 While Lord Kelvin may have been a shortsighted skeptic when it came to airplanes, radio, and x-rays, he made an important contribution to the management science of his day.
Clinical trial data beg to be measured, but determining which metrics to capture differs from study to study. In Bullseye!, Schiemann and Lingle posit that few activities are measured effectively because there is “…disagreement on what to measure and a failure to update information frequently.”4 In other words, to have a successful clinical trial, every member of the project team must concur on the critical metrics, and then the team must compare actual performance against those metrics regularly throughout the study. Moreover, choosing the right metrics should occur during the study’s planning stage, not defined retroactively at the end of the trial. Whatever metrics we decide to capture should help us determine not only how we performed on the trial being measured, but also how we might do better in the future.
There are many things to measure during a clinical trial, but three essential ones for any trial are:
Quality. Error rates can be tabulated, in both paper and electronic data capture systems. By categorizing data errors, the project team can determine their cause and develop a preventive action plan for other sites and future studies.
Timeline. Thorough vetting of potential investigators, comprehensive study training, and constant focus on enrollment can keep study timelines from slipping. When it comes to clinical trials, time really is money. So maintaining timelines is crucial to success.
Budget. Detailed budgeting during the planning stage, followed by timely accounting of ongoing expenditures and regular budget-to-actual comparisons, will eliminate unforeseen outlays and last minute surprises.
These three metrics will not deliver perfection. They will, though, bring you much closer than you ever expected. Happy counting!
1Building Quality in Clinical trials With Use of a Quality Systems Approach, Clinical Infectious Diseases, 2010; 51(S1): S111.
2Duffey, R.B., and Saull, J.W., Know the Risk, 2003, page 179.
3Thomson, William, lecture on “Electrical Units of Measurement” (3 May 1883), Popular Lectures, Vol. I, p. 73.
4Schiemann, William A. and John H. Lingle, Bullseye! Hitting Your Strategic Targets Through High-Impact Measurement, 1999, page 42.