When visibility craving compromises research

A recent survey showing that up to 4,000 girls in Machakos County were impregnated between January and May has sparked public debate, with both the Education and Health ministries claiming exaggeration.

This begs the question: when is a research considered credible? Without condemning the survey, I believe it is time to reflect on the entire research system in Kenya.

There are multiple factors that would contribute to false findings in a research, and the probability of the media detecting them is low, especially because reporters have limited information of how the study has been conducted.

Research findings are as good as the input, well known as the GIGO concept. This means Garbage in, garbage out. This simply implies that a researcher has to consider certain scientific merits to ensure a study is of sound design and methodology.

For instance, is the methodology sufficient to prove the conclusions arrived at by the study? Did the study include enough and unbiased respondents to get meaningful results? What are the margins of error? And what are the shortcomings of the research?

In many cases, lack of research integrity has seen bias, incentives and conflicting interest influence the way a study is conducted, thereby contributing to research misconduct.

Even worse, technology has made it cheaper and easy to conduct and publish research, while competition among journals have driven some researchers prioritise visibility over integrity.

On the other hand, the media’s insatiable appetite for new studies, especially those that give them scoops, has meant such articles continue to be published.

It s in this context that Education Cabinet Secretary George Magoha’s sentiments on the teenage pregnancy report ought to be understood.

He said: “I would like to interrogate the rate of the pregnancies because they look obnoxious. I am asking myself as a professor, did these girls go and report that they are pregnant? Who is giving us these figures because they seem to be definitive? Could some NGOs who are keen on pushing sex education be using these exaggerated numbers.”

This particular scenario is an eye opener and provides an opportunity to take a more active role in preventing misconduct throughout the research process.

But who gets to decide whether the experimental protocol is appropriate and ethical in the country? What are the specific guidelines to be followed?

In Kenya, the National Commission for Science, Technology & Innovation (NACOSTI) handles licence application process for researchers, both academic and non-academic. The body may also monitor and evaluate the research.

“We check on compliance only. For academics we expect the school to approve the scientific merits of the research. As for non-academics, some research studies are conducted through accredited institutions such as Kemri and Amref, which have unique guidelines for a credible research,” says Godfrey Kalerwa, the director-gneral of NACOSTI.

“Other private entities conduct their research through institutions such as Marketing & Social Research Association (MSRA), which also have their specific code of research. The gap occurs when a research is not conducted under any guidelines.”

Mr Kalerwa notes that there is need for the country to have a pool of independent experts to perform an in-depth review of proposals before conducting a research. This would ensure that the scientific merits of research are not compromised.

Currently, a private institutions are only required to pay Sh20,000 and submit research proposals, a passport and identity card to conduct research.

“We received 6,623 proposals on different areas of study last year, and it is difficult to go through all the proposals checking if all the merits are fully satisfied. It would be great to have advisory from experts in particular fields,” Mr Kalerwa says.

Variability in the research methods and processes used in different disciplines limits adoption of standard recommendations.

Mr Kalerwa is calling for standard practice in execution of research protocols in order to ensure the various stakeholders are on the same page.

On the political front, an opinion poll is designed to collect opinions on issues such as candidates, parties, and willingness to vote from a particular sample.

However, the difference in results among pollsters has become a “big headache” for voters and citizens to decide which opinion poll to believe.

For example, in the last election, two pollsters Infotrak and Ipsos displayed different findings on who the Kenyans planned to vote for as their president.

Infotrak put opposition Nasa candidate Raila Odinga at 49 per cent and President Uhuru Kenyatta at 48 percent. On the other hand, Ipsos showed Mr Kenyatta would win with 47 per cent against Mr Odinga with 44 per cent.

When it comes to polls, a researcher need to consider a number of things. Besides a sufficient sample size, demographic profile plays a great role. In other words, if there are representative issues with the sample then making it bigger doesn’t necessarily improve it.

In addition, inappropriate or faulty methodology leads forecasting off-target. Conducting a conventional poll requires the pollster finds a representative sample of the population, identifies the respondents that are registered and likely to vote, probes their voting intention, deals with undecided voters, and then hopes people do not change their minds in the polling booth.

Each of these stages can produce systematic bias, over and above the random variation — the margin of error — that we would expect in any sample.

This just shows that Kenya needs a more critical appraisal of the processes used to generate, process, and analyse data, which requires greater transparency through easy discoverability and access to the data behind published findings.