Randomnized Controlled Trials
A mini guide for the reader to interpret the quality of studies
Many studies are published, evidence based therapy is on the move, but...we may not forget to focus on some important issues :
- Is the published work intrinsic correct (e.g. are therapeutic protocols carried out in a correct way ?) and/or relevant (e.g. do the results have a real implication) ?
- Do studies have clinical relevance for the field therapist, or is it merely a demonstration of "lab science"?
- The most important question : does the published work inspire us to adapt and improve our clinical reasoning and therapeutic acting in daily practice ; in other words, are we going to change something in our thinking and acting, resulting in a change in our diagnostic and therapeutic protocol ?
Or...is the published work merely (unfortunately) a demonstration of "research for the sake of research" or, "science for the sake of scientist" ?
The information below will help you to distinguish good studies from less relevant studies.
Did the authors take into account following basic principles :
- Literature review
- Is the hypothesis and the purpose of the study clearly described ?
- Is this purpose clinically relevant and/or useful in daily clinical practice ?
- Are the results of the study valid ; can they be trusted ?
- Do the authors clearly describe which persons are included and excluded in this study ; is there a clear description of inclusion and exclusion criteria ? A clear description of all procedures used ? Are therapeutic procedures carried out in a correct way ; are they easily reproducible ?
- Are the test subjects divided in groups at random ?
- Are the researchers, therapists and test subjects blinded ?
- Are there less than 15% drop-outs ?
- Is there a good statistical interpretation of the data ; did they use a p-value which is set sharp enough ?
- Did the authors calculate a standard deviation value for at least one of the results ?
- Is there a follow up and is the period inbetween long enough ?
- Is it possible to recreate or optimize this study on the basis of all data described in this article ?
- Are the results interpreted with adequate statistical interpretations ? (This is of course very difficult to interpret as a reader ; mostly we just have to rely on the work of the statistical expert ; so, let's be cautious in jumping to conclusions too fast).
- The "probability-question" : are the results due to coincidence or not ? A p value < 0.05 is considered significant, but a value < 0.01 or even < 0.001 is very significant.
- Did the authors mention some chance of errors or points of optimalisation of the study ?
- Do the authors discuss the results of the study in relation to those suggestions ?
- Do the authors reach a conclusion that has clinical consequences for the therapist and that benefits the patients ?
(many thanks to F.M. Koller and C. Beyerlein who organized literature evenings for therapists in Germany and for their contribution to this section ; they inspired me)