In the previous article, I wrote about why the phrase \u201ca study showed\u201d should not be the end of the discussion. But we should be equally careful with another sentence: \u201cThere is no evidence for that.\u201d<\/p>\n
It can mean very different things. Perhaps the question has barely been studied at all. Perhaps there are a few small or poor-quality studies that do not allow a firm conclusion. Perhaps studies exist but their findings are contradictory. Or perhaps several sound studies already allow us to say, with reasonable confidence, that a meaningful benefit is unlikely.<\/p>\n
Reducing all these situations to \u201cit has not been proven\u201d turns a gap in knowledge into a black-and-white answer. And in fact, we do not have that answer.<\/p>\n
One of the best-known articles about this problem appeared in the BMJ back in 1995. Statisticians Douglas Altman and Martin Bland drew attention to a very simple point that is still often misunderstood. If a study finds no statistically significant difference between two groups, that does not automatically mean that no difference exists. The study may simply have been too small, or its estimate too imprecise, to detect a real difference with confidence.<\/p>\n
Imagine, for example, that a new joint-support supplement is being studied. One small study finds no statistically significant difference between dogs receiving the supplement and dogs not receiving it. We may conclude that this study found no convincing evidence of an effect. But we cannot automatically conclude that the supplement has no effect.<\/p>\n
Those are two different claims.<\/p>\n
There is an equally important caution in the opposite direction. If a study found no evidence of an effect, we cannot conclude, \u201cso it may actually work extremely well.\u201d It may. But we do not know.<\/p>\n
And sometimes \u201cwe do not know\u201d is the most accurate answer.<\/p>\n
This is where things get a little more interesting. The expression \u201cabsence of evidence is not evidence of absence\u201d is sometimes used almost as a universal lifeline.<\/p>\n
A product showed no effect in a study? \u201cThat does not mean it does not work.\u201d Three studies failed to find an effect? \u201cScience does not know everything yet.\u201d Ten studies showed no clinically meaningful benefit?<\/p>\n
At some point, we cannot keep relying on a knowledge gap indefinitely.<\/p>\n
What matters is not only whether the result was statistically significant, but also how precisely the study was able to estimate a possible effect. Confidence intervals are one of the tools used to assess this.<\/p>\n
In a small study, the confidence interval may be very wide. The results may be compatible with a benefit, harm or essentially no effect. That kind of study leaves the question open.<\/p>\n
But if sufficiently large, well-conducted studies provide precise estimates and rule out, with reasonable confidence, a benefit large enough to matter clinically in real life, the situation is different.<\/p>\n
Then we are no longer dealing merely with an \u201cabsence of evidence\u201d. We begin to have evidence of the absence of an effect, or at least evidence that a meaningful effect is unlikely.<\/p>\n
So the question is not only: \u201cDid the study find a statistically significant result?\u201d<\/p>\n
It is also: \u201cWhich possible outcomes are still compatible with the data we have?\u201d<\/p>\n
There are many gaps in veterinary medicine.<\/p>\n
Many supplements, medicinal plants, nutritional approaches and integrative methods have not been studied in dogs as extensively as, for example, commonly used medicines. And the results from a study on one specific supplement cannot automatically be applied to a different supplement.<\/p>\n
There can be very different reasons for this.<\/p>\n
Research is expensive. Some interventions have no large company willing to fund studies. Some questions are difficult to standardise. With certain diseases or groups of animals, recruiting a sufficiently large sample is hard.<\/p>\n
But here we need to be careful.<\/p>\n
The reason research is lacking tells us nothing by itself about whether an intervention works. \u201cIt has not been studied because nobody can make money from it\u201d does not prove effectiveness. \u201cIt has not been sufficiently studied\u201d does not prove ineffectiveness.<\/p>\n
Both statements primarily describe the state of our knowledge.<\/p>\n
This is where the subject becomes especially important to me.<\/p>\n
If an integrative veterinarian says they have used a method in dozens or hundreds of patients and repeatedly observed good results, I do not believe that experience should simply be thrown away because we do not have a perfect randomised trial. It is information.<\/p>\n
The same applies to an owner’s experience. If a dog had a problem, something was changed and the problem disappeared, that is a real observation.<\/p>\n
But the next question is more difficult: why did the situation improve?<\/p>\n
Was it because of the intervention? Did the diet change at the same time? Was the dog given medication? Would the natural course of the condition have led to improvement anyway? Does the symptom fluctuate over time? Did the owner also begin paying closer attention to the dog’s body weight, exercise or sleep?<\/p>\n
An individual experience cannot always separate these possibilities. That does not make the experience worthless. It simply determines how much evidential weight it carries.<\/p>\n
Here is another consideration that, in my view, receives far too little attention.<\/p>\n
Risk.<\/p>\n
We do not need exactly the same strength of evidence for every decision. If we are considering a low-risk supportive measure that can easily be stopped, while monitoring the dog’s condition, we may be willing to accept greater uncertainty.<\/p>\n
It is quite another situation when a decision might delay effective treatment, lead to stopping medication, cause an important diagnosis to be missed or involve a potentially dangerous substance.<\/p>\n
Here, the threshold for evidence should be higher.<\/p>\n
For example, \u201cthe potential supportive effect of this plant has not been properly studied in dogs\u201d may leave room for further research or discussion with a veterinarian. But it does not justify the conclusion, \u201ctherefore we should always use it instead of a proven treatment.\u201d<\/p>\n
Ignorance does not make risk disappear. Nor do the words \u201cnatural\u201d, \u201ctraditional\u201d or \u201cintegrative\u201d automatically make an intervention low-risk.<\/p>\n
The same principle works in the other direction.<\/p>\n
For conventional medicine, \u201cit is used in medicine\u201d should not be enough either. We should still examine what benefits have been demonstrated, in whom, what the risks are and how well the treatment suits the individual patient.<\/p>\n
The same standard for everyone.<\/p>\n
Our principle is neither \u201cnatural always before medicine\u201d nor \u201conly use something if there is a perfect study\u201d. When the situation allows it, we prefer to begin with a gentler, lower-risk and reversible measure, and observe how the individual animal responds.<\/p>\n
This might mean changing activity, body weight, environment or diet, using an appropriate supportive supplement, a medicinal plant or another integrative approach. However, the specific choice must suit that individual animal: dose, method of use, existing illnesses, medication and potential interactions all matter.<\/p>\n
If it helps, wonderful. We have gained valuable practical information about that animal, even if one dog’s experience does not justify a universal treatment claim. If it does not help, or the effect is insufficient, we move on: further diagnostics, another intervention or medication when needed.<\/p>\n
Introducing medication does not mean that an integrative approach has failed. Sometimes a medicine is the next necessary tool; sometimes it is used alongside supportive measures; and sometimes medication must be started immediately because delaying treatment would be the greater risk.<\/p>\n
Leaving a condition untreated is not a risk-free alternative to treatment. Equally, the statement that \u201con average, the medicine was well tolerated in studies\u201d does not mean that every individual animal will react in precisely the same way.<\/p>\n
No. In real life, decisions have to be made even when perfect evidence does not exist. Evidence-based veterinary medicine is not just about ranking scientific papers.<\/p>\n
RCVS Knowledge describes EBVM as combining the best available scientific evidence, clinical experience and the particular circumstances of the animal and owner.<\/p>\n
And this brings us back to one of FendaF’s most important ideas: the average dog or cat does not live in your home.<\/p>\n
A study tells us what happened in the group studied and what the average effect or risk was. That is a valuable starting point, but the next question is always how well that knowledge fits the individual animal. Age, illnesses, medication, previous reactions, living environment and many other factors can change both the benefits and the risks.<\/p>\n
This is not an argument for ignoring scientific evidence. Quite the opposite: it is a reason to apply evidence rather than merely copy it. RCVS Knowledge describes evidence-based veterinary medicine precisely as integrating clinical evidence, professional experience and the individual circumstances of the patient and owner.<\/p>\n
The word \u201csafe\u201d is not an absolute property either. The sensible question is always: safe for whom, at what dose, for how long, with which other diseases and medicines, and what are we monitoring during use?<\/p>\n
Sometimes we therefore have to decide in the midst of uncertainty. But that uncertainty should be visible in the decision.<\/p>\n
Instead of saying, \u201cThis definitely works,\u201d we can say, \u201cThere are some interesting preliminary findings, but the clinical evidence is currently limited.\u201d<\/p>\n
Instead of saying, \u201cThere is no evidence, so it does not work,\u201d we can say, \u201cAt present, there is not enough high-quality evidence to reach a firm conclusion about its effect.\u201d<\/p>\n
And sometimes: \u201cThe existing studies are sufficiently robust and precise that a meaningful benefit appears unlikely.\u201d<\/p>\n
These sentences may sound less striking than \u201cworks\u201d or \u201cdoes not work\u201d.<\/p>\n
But they are much more informative.<\/p>\n
If you want to understand quickly where a health claim stands, try separating these three situations:<\/p>\n
1. \u201cThere is good evidence of an effect.\u201d<\/p>\n
This means we have enough relevant, high-quality data to support the claim with reasonable confidence.<\/p>\n
2. \u201cThere is currently insufficient evidence of an effect.\u201d<\/p>\n
This means the question remains open. It is not evidence in favour of an effect, nor necessarily evidence against it.<\/p>\n
3. \u201cThe existing evidence speaks against this claim.\u201d<\/p>\n
This means there is already enough information to consider the claim unlikely, or at least too strong in its original form. Confusing these three situations leads to many arguments in which people are actually talking about different things.<\/p>\n
One person says \u201cit has not been proven\u201d and means \u201cwe do not know\u201d. Another hears \u201cit does not work\u201d. A third replies, \u201cbut it worked for me\u201d.<\/p>\n
All three may be speaking perfectly sincerely from their own perspectives and still talking past one another.<\/p>\n
It seems to me that we do not like uncertainty. We want to know whether a food is good or bad. Whether a medicine is safe or dangerous. Whether a supplement works or does not. Whether a medicinal plant helps or is pointless.<\/p>\n
But biology is not particularly concerned with our love of two clear answers.<\/p>\n
Sometimes the evidence is strong. Sometimes it is weak. Sometimes we have only a biologically plausible mechanism. Sometimes there are years of clinical experience that have not yet been properly studied.<\/p>\n
And sometimes there is simply a large blank area on the map.<\/p>\n
Critical thinking does not mean rushing to fill that blank with our favourite answer.<\/p>\n
Rather, it means being able to say, \u201cThis is where our current knowledge ends.\u201d<\/p>\n
That can be where research begins.<\/p>\n
The next time I encounter a claim followed by \u201cthis has not been proven\u201d or \u201cthere are no studies on it\u201d, I want to ask:<\/p>\n
Are there truly no studies, or have the existing studies simply failed to find statistically significant results?<\/p>\n
If studies do exist, how precise are their results?<\/p>\n
Do the available data still leave room for a clinically meaningful effect?<\/p>\n
Do we have a biologically plausible mechanism or clinical experience that offers a reason to investigate further?<\/p>\n
What happens if our assumption turns out to be wrong?<\/p>\n
Is a possible low-risk trial more appropriate for this particular animal than moving immediately to a stronger intervention, or would the cost of waiting be greater?<\/p>\n
That last question matters especially. Evidence does not exist in a vacuum. The value of a decision also depends on the potential benefit, potential harm and available alternatives. This is why evidence with exactly the same degree of uncertainty may lead to completely different decisions in two different situations. To me, that is more accurate than pretending science always supplies a simple \u201cyes\u201d or \u201cno\u201d.<\/p>\n
\u201cNot proven\u201d does not automatically mean \u201cdoes not work\u201d. But just as importantly, it does not mean \u201ctherefore it might work and we can claim that it does\u201d.<\/p>\n
Sometimes it simply means that we do not know enough yet.<\/p>\n
And that, too, is knowledge.<\/p>\n
Altman & Bland: Absence of evidence is not evidence of absence, BMJ 1995<\/a><\/p>\n Phil Alderson: Absence of evidence is not evidence of absence, BMJ 2004<\/a><\/p>\n Veterinary Evidence: Strength of evidence<\/a><\/p>\n Veterinary Evidence: Guidance for writing the clinical bottom line<\/a><\/p>\n Peter Cockcroft: Grading the evidence and writing the clinical bottom line<\/a><\/p>\n RCVS Knowledge: Evidence-based veterinary medicine, an introduction<\/a><\/p>\n