Checking the facts
Throughout this website I have tried to be as rigorous as any scientist might be in writing a clinical paper. You will find major assertions, as well as facts and figures justified by reference to reliable texts. Statistics are the language by which scientific information is communicated. Statistics help us distinguish between the unknown, the professional hunch and the known. Do not be put off by the rather dry appearance of a scientific text. The reporting, when it is good, is highly readable. These texts are written to be understood, not for obfuscation.
This is part two of the series Understanding Scientific Papers. Part one is about Understanding Statistics.
What is a scientific paper?
Scientific journals are not immune from science-babble. I often wonder whether such articles are peppered with long and complex words because they aid accuracy, or whether they are there to impress other scientists. In my writing, I prefer to use plain English words wherever possible and when it has been necessary to use a technical term, I have done my best to find a plain English equivalent. Sometimes medical terms such as hypoglycaemia or adipose tissue are used within the text. These words are more precise that more vernacular terms such as, high blood sugars, or body fats. But when such terms are used, I hope that the context and accompanying descriptions, make the text comprehensible to the lay reader. Sometimes I have added a lay term in brackets, meaning that it is a useful interpretation of the medical term, but possibly not quite so precise.
I have cited trusted evidence sources as far as is possible, picking out for the reader research that is as up to date as possible or in some cases seminal (and thus sometimes much older).
Here are a few of the things I look out for:
Peer reviewed work
This is work that is checked for the journal, prior to publication, by a panel of other scientists. Sometimes a pre-review text is published. We saw this a lot during the pandemic, when it became more important to disseminate the results of studies really quickly, even if later some of those studies proved to be less useful than others. Suddenly clinicians were thrown back onto their own professional judgement to make decisions with the benefit of insight, but nothing as good as scientific proof, that might take years to amass.
Published in recognised journals
Journals such as the British Medical Journal (BMJ), the New England Journal of Medicine (NEJM), Journal of the American Medical Association (JAMA) and The Lancet are all well-known. But there are hundreds of less well known, but very well-respected journals out there that publish good work, and often specialist work. In a study of longevity, where we are interested in the intricate workings of metabolism, epigenetics and cell biology, as well as very specialist work relating to ageing, you will see references to specialist journals with interests in gerontology, nutrition, cell-biology, bio-chemistry and metabolism. But no paper, even a peer-reviewed study, led by a venerable professor and published by a well-respected journal, is free from flaws. A single example of research, from a small sample, may not be representative of a wider truth.
Unbiased
Some studies are not as rigorous as they should be, the reporting of statistics can be misleading. There can be typos, especially in data hot-off-the-presses. sometimes the data is not presented in the most transparent format. Some writers are past masters at judicious editing. Look for how subjects are selected, how they are matched with control groups, whether placebos were used and what the protocols of the study might be. Some studies are subsidised by commercial groups, lobby groups or pharmaceutical agencies. That can mean that the studies become tainted by commercialism: I always check the funding of the research I include and its level of independence.
Open access
While open access isn’t a mark of veracity – there is a lot of open-source rubbish out there – it does mean that my readers can see for themselves. Wherever possible I have provided a citation to a study that is freely available online. I urge you to read these studies, they have been selected for their accuracy, their legibility and because they are up to date. Sometimes I refer to an older study which was influential and has stood the test of time. The older studies are sometimes more difficult to find on the internet, but can be found in some public libraries or by visiting (on-line or in person) the Wellcome Medical History Library at London, Euston.
Relevant
I will look at the type of people who were tested or studied, prioritising European or UK studies, with a good mix of gender, ethnicity, a range of ages and relevant lifestyles. For example, a study of adolescent male athletes may not be particularly relevant to longevity of post-menopausal women. Similarly, animal studies may provide some useful insights into mechanisms, but we are very different to mice, or fruit flies; such results prove nothing for humans.
Reproducible
A group of similar studies build-up to a body of evidence that becomes impossible to ignore. Meta-analysis, where results from a group of studies are pooled and recalculated can offer higher statistical exactitude. In scientific journals reviews are often interesting to read because they report on results from recent studies across the globe, offering good insights into what we are learning.
Sense check
Correlation is not causation. By this I mean that just because we can see a strong relationship between the intervention and the outcome, we cannot prove that the latter is caused by the former. A good scientist always considers why results have been returned. I will explain, in as simple terms as possible, the scientific rational behind the statistics. If it doesn’t make sense to me, I don’t use the research.
A note of warning
But beware! Scientific history is littered with the results of studies that looked reasonable at the time, but which later were called into question. Large, well-funded, epidemiological studies are no exception. My best advice is to read the study for yourself. While they can be impenetrable, you’ll soon begin to be able to sift the wheat from the chaff. At the end of each section, I include a reference to publicly available studies that you can read online for yourself.
Sometimes new studies are published open access for a while then go back behind a paywall. However, these days more and more scientists and research funders, such as the Wellcome Trust, are committed to open access publication. If you cannot get hold of the study, refer to the corresponding email address which can usually be found in the publicly available abstract. A short email request made directly to the principal author, will almost always bring you a pdf copy of the study by return.
We are all capable of making up our own minds.
A beta reader is a test reader of an unreleased work (similar to beta testing in software), who gives feedback to the author, from the perspective of an average reader. If you’d like to help me get this book as good as it can be, I’d love to hear from you. You’ve got plenty of time to make up your mind. The chapters will be ready from June onwards. I’m sorry there is no fee for this work, except a free copy of the finished book and an acknowledgement. This is not a commercial exercise. Say Tomato, the publisher of the book and of the website is run as a self-financed social enterprise. I shall be eternally grateful. Any profits from Glorious Summer will be directed into Say Tomato’s work for women over 40.
Contact me directly wendy@wendyshillam.co.uk if you are interested, giving a brief description of yourself and your reading experience. (It doesn’t matter if you haven’t done any beta reading before – avid readers make the best beta readers.) Thank you.
Wendy Shillam
Trained in environmental and bio-sciences, at Bristol Univeristy and University College London, Wendy Shillam is a registered clinical nutritionist, specialising in longevity. She is the founder of the social enterprise Say Tomato! that provides free and trusted weight loss advice for women over forty. This part of the blog is a precursor to publication of her book, Glorious Summer – the secrets of longevity. You can read about it here.</s