{shortcode-03adbc4cd5c7ebffb4424fb946c1d37b05d6a452}

On June 9, 2020, fourth-year School of Engineering and Applied Sciences Ph.D. candidate Yuntian Deng came across a tweet from former president Donald Trump showing a Fox News report indicating that “Wuhan showed signs of virus last August,” attributed to a Harvard study.

Sensing that this tweet could have a “really big social impact,” Deng immediately looked online for the original paper to better understand the evidence posed to support the reported findings.

What Deng found was a yet-to-be-peer-reviewed study that had been made available on Harvard DASH, an open-access repository for scholarly work by Harvard affiliates.

The research article, titled “Analysis of hospital traffic and search engine data in Wuhan China indicates early disease activity in the Fall of 2019,” supported a hypothesis that the novel coronavirus emerged in Wuhan earlier than the documented start of the pandemic in December 2019.

Conducted by researchers at Harvard Medical School, Boston Children’s Hospital, and Boston University, the study examined satellite images of six hospital parking lots in Wuhan as well as search word data from the Baidu index, a Chinese data sharing platform analyzing online user activity. The paper cited an “upward trend” in hospital traffic and search volume for diarrhea — a Covid-related symptom — beginning in the fall 2019.

Upon closer examination of the research article, Deng and fellow SEAS graduate student Shijian “Martin” Jin found what they believed to be multiple flaws in the presentation and methods of the study.

An ‘Unsatisfying’ Investigation Result

Deng and Jin presented these concerns in an open letter dated June 15, 2020 — signed by over 60 Harvard graduate students and alumni — to the HMS administration and the study’s corresponding author, John Brownstein, an HMS professor and chief innovation officer of Boston Children’s Hospital.

Specifically, the students cited inconsistent usages of language throughout the paper and the possible use of data dredging — a technique in which statistical tests can be run multiple times to eventually reach a desired conclusion.

After receiving no response from HMS for nearly a month, Deng and Jin met with Graduate School of Arts and Sciences Dean for Academic Programs and Diversity Sheila Thomas and sent a follow-up email on July 14. HMS Chief of Staff Jennifer M. Ryan responded to their letter later that day, promising an institutional review of the research article, according to emails reviewed by The Crimson.

After six months of silence, Deng wrote an email on Jan. 25, 2021 asking for investigation updates. He sent another follow-up email on March 18, 2021, nearly nine months after HMS’s last correspondence.

After looping in the Harvard and SEAS diversity offices in a separate email thread on the same March day reiterating the posed concerns, Deng received a response on April 1 from Boston Children’s Hospital vice president of research administration August “Gus” Cervini and HMS Dean for Faculty and Research Integrity Gretchen Brodnicki, which conveyed that a review by Boston Children’s Hospital found “no concerns” with the research article under its institutional policies, according to the emails reviewed by The Crimson.

In response to what he characterized as an “unsatisfying” investigation result, Deng replied asking for more details about the review process regarding decision-making protocols, specific evidence that supported the review’s result, and the diversity of the investigation committee — to which he received no reply.

Spokespeople for HMS and BCH confirmed in a statement an investigation of the article occurred, but did not provide specific details regarding the review.

‘Not a Logical Choice’

Since becoming available to the general public, the article has garnered widespread media attention.

In June, The Crimson reported on the study’s preliminary findings suggesting the early emergence of Covid-19 in Wuhan late August.

ABC News prematurely published a report on the study a few hours prior to the article’s actual release due to a server issue with Harvard DASH on June 8, according to an emailed statement from Brownstein.

The paper was also picked up by Chinese media, after which the article became a widely circulated topic of conversation among Chinese social media users.

“Everyone was talking about it,” said Hao Chen, a Chinese national and post-doctoral researcher at the Institute for Numerical and Applied Mathematics at the University of Göttingen in Germany. “It was very controversial in China.”

Chen, along with four researchers from Harvard, University of Virginia, Zhejiang University, and Huazhong University of Science and Technology, began compiling a comment paper to identify and explicate their concerns over the study. Chen said the purpose behind publishing this paper was to “leave a record in the academic literature.”

The comment paper was also published on Harvard DASH on July 8, 2020 — exactly a month after the original study became available on the same repository.

Their comments added to the voices of many other experts’ on the limitations of the study, including what Deng and Jin outlined in their open letter.

Deng and Jin’s open letter, as well as Chen and his team’s comment paper, specifically pointed to shortcomings in the study’s use of Baidu search engine data.

The original study found that, similar to how elevation of searches for the keyword “cough” coincides with the yearly influenza season, the search trend for the keyword “diarrhea” — which the study defined as “a more COVID-19 specific symptom,” — was elevated since August 2019.

Both native Mandarin speakers themselves, Deng and Jin noted the authors used a Mandarin phrase that translates to “symptoms of diarrhea,” rather than the much more common translation of the word “diarrhea.”

“The search term they used was not really Chinese language. That sounds very strange to Chinese ear,” Chen said.

After reproducing the search engine test using the more common Mandarin translation of “diarrhea,” Deng and Jin said their results did not match those of the authors.

“All search terms were vetted by a native Mandarin speaker on our study team,” Brownstein wrote in an emailed statement to The Crimson. “The search term we chose for “diarrhea” (腹泻的症状) was chosen because it was the best match for confirmed cases of COVID-19 and was suggested as a related search term to coronavirus by the Baidu search engine’s 需求图谱 (Demand Map) tool.”

Donald A. “Don” Goldmann ’65, a pediatrician at Boston Children’s Hospital and professor at the Harvard School of Public Health and HMS, questioned the focus of the study on search trends of “diarrhea” due to its lack of uniqueness to Covid-19.

“Diarrhea is not unique at all. It’s a very common symptom,” Goldmann said. “For all I knew there was a national outbreak of some viral diarrhea at that time — who knows?”

In fact, Chen and his team also attempted to reproduce the original search engine test, but found that the elevated trend in search queries appeared “nationwide,” not only in Wuhan. Chen added that the elevated search trend “doesn’t appear in other search terms” that are more typical symptoms of Covid-19, including shortness of breath, coughing, or fever.

Goldmann also said the original study could have included mention of the other Covid-19 symptoms and search trends they analyzed, even if the analysis didn’t show significant trends.

Responding to this, Brownstein argued that the Baidu index lacked data on the proper keywords for other coronavirus symptoms.

“Many other words for COVID-19 unique symptoms such as loss of taste or smell were not recorded by the Baidu index prior to May 2020,” he wrote.

Deng and Jin, as well as Chen’s team, also suggested the potential that the study misinterpreted the statistical data taken from the Baidu search queries. According to Chen, Brownstein’s team used a statistical method to smooth the data — reducing statistical noise in order to better visualize the trends.

“They are definitely manipulating the data, but manipulating is not falsifying the data,” Chen said. “So the data is true, but the way they analyze the data is problematic.”

Brownstein argued that his team used the smoothing method to “highlight local trends in the data,” and still included each discrete data point on the figures to be “as transparent as possible.”

The study’s usage of satellite data of six Wuhan hospital parking lots also prompted concerns.

Deng and Jin questioned the selection criteria for the six Wuhan hospitals — one of which was a women and children’s hospital — in analyzing the emergence of the virus.

Chen argued this hospital was “not a logical choice” in selecting for patients showing symptoms of Covid-19, given that this hospital does not have a “fever clinic.”

Several experts said in interviews they found the use of hospital images as evidence unconvincing.

“I thought it was an interesting idea to use the satellite imagery to try and come up with inferences about disease from hospital car parks,” said Paul Digard, a professor of virology at the University of Edinburgh. “But when I read it, I thought it was underpowered for the implications.”

“The parking lot data is interesting,” added Goldmann, the BCH pediatrician. “But then you have to ask yourself, are all those people really flocking to the hospital because they have symptoms of Covid?”

Brownstein acknowledged in his emailed statement that having additional control cities would have increased the robustness of the study. He noted, however, that a significant amount of cloudy weather during November 2019 to February 2020 in Wuhan created limitations in high quality satellite images that could be analyzed.

“We presented results for all the hospitals that we had enough data points on, and sensitivity analyses show the overall trends are robust,” he wrote. “Of note, the exclusion of any single hospital -- including Wuhan Women and Children’s -- did not change the results of our findings.”

Social Implications

In addition to its methods, critics pointed to concerns regarding the potential social implications of the study, arguing it may spur hatred against China and Chinese people, and that this possibility was exacerbated by wide media circulation of the study’s conclusions under the Harvard brand.

“The inconclusive or unscientific statement in the title and abstract has been cited as a fact and used to vilify China, provoking hatred against Chinese, especially Chinese students in the US,” Deng and Jin wrote in their open letter last June. “This not only hurts the feeling of Chinese students at Harvard, but also denigrates Harvard’s reputation as an institution that is known for her research integrity.”

“As members of the Harvard community, we were very concerned to see such a misleading paper affiliated with Harvard. We respect scientific investigations and understand that some hard truths might be disturbing, but to the best of our judgment, this article is misleading to the public,” they added.

Deng and Jin noted that while the paper’s title indicated that the findings supported the hypothesis of early disease activity in Wuhan, the conclusion states that the retrospective analysis “cannot verify” the data’s connection to the SARS-CoV-2 virus.

When asked about the potential social implications of the study, Brownstein wrote that he and his research team are “very concerned” about the stigmatization and bias that comes from research that associates a country with a virus.

“Our team has been actively working on early identification of disease outbreaks for two decades. We’ve published similar methods around early insights on H1N1 in Mexico, Zika in South America and foodborne outbreaks in the US,” he wrote. “Our longstanding work to identify the origins of disease emergence is core to our team’s efforts to inform and improve global health surveillance.

“Our work is guided purely by science and nothing else,” he added.

The preliminary data was presented in a seminar at the Harvard Department of Biomedical Informatics, according to Brownstein. But after some of the co-authors were subject to “significant personal attacks on social media,” Brownstein said he and his team decided to “hold back” any further presentations or publications until after the peer-review process.

Brownstein added that the paper has now been peer-reviewed and accepted for publication, accounting for changes based on numerous suggestive comments they received.

Citing HMS and Boston Children’s Hospital’s “lack of transparency” with the review process, Deng and Jin demanded a full disclosure of the investigation protocols involved in the article’s review in their April 1 response to the administrators.

“I am happy to accept that we are wrong in our arguments,” Deng said. “But I think at least they need to refute our concerns in our original email point by point.”

Deng added that he believed HMS failed to address student concerns surrounding ethical considerations and potential social impacts of the paper, particularly during a spike in anti-Asian violence.

“Please please please take this and the two thousand Chinese students/scholars at Harvard seriously,” Deng and Jin wrote in the April 1 email. “Please prove your care for Asian community with action, not words.”

—Staff writer Ariel H. Kim can be reached at ariel.kim@thecrimson.com.

—Staff writer Anjeli R. Macaranas can be reached at anjeli.macaranas@thecrimson.com.