GLOBAL CHALLENGES: EBOLA OUTBREAK THE HUNT FOR EBOLA: BUILDING A DISEASE SURVEILLANCE SYSTEM IN LIBERIA, 2014-2015 SYNOPSIS When the first cases of Ebola virus disease appeared in Liberia at the end of March 2014, a critical first step in preventing an epidemic was to identify those who had contracted the virus. However, Liberia's disease surveillance capacity remained feeble in the wake of a 14-year civil war that had weakened the health system, and citizens' distrust of the government sometimes raised risks for public health teams dispatched to carry out that vital surveillance function. In August, as the number of new infections began to escalate, the government and its international partners shifted to a proactive strategy. Rather than wait for families to call for help, they began to engage local leaders and community health workers in hunting the disease. They also developed data management practices to more effectively track and analyze the evolution of the epidemic. By year-end, most of the new Ebola infections involved Liberians who were already under observation. In another important measure of success, the time between patients' onsets of symptoms and their medical isolations shortened markedly. The ability to hunt down Ebola slowed the spread of the disease and helped bring an end to the epidemic in May 2015. Leon Schreiber and Jennifer Widner drafted this case study based on interviews conducted in Monrovia, Liberia, in April and May 2016 and with international organizations from June to August 2016. Béatrice Godefroy provided initial guidance. Princeton University's Grand Challenges program supported the research and development of this case study, which is part of a series on public management challenges in the West African Ebola Outbreak response. INTRODUCTION In May 2014, physician assistant Alpha Tamba returned to his village in Liberia's Lofa County to fight Ebola virus disease, an often fatal hemorrhagic fever that was creeping into the region. Working in a health team organized by a local nongovernmental organization (NGO) and the International Rescue Committee, he had encountered angry opposition when visiting other villages to convey information and identify those exposed to the disease. But as the son of a local family, he had the credibility to dispel doubts about the danger of Ebola and to persuade villagers to take specific, effective actions to protect themselves and their community.1 Tamba's effort, recounted in an International Rescue Committee report later that year, demonstrated the value of personal relations in communicating credibly during Liberia's Ebola crisis, when the public had little trust in the government and even less confidence in outsiders. The residents of Lofa County were especially hostile to health workers.2 The area had suffered heavy fighting during a 14-year civil war, which had ended only a decade earlier, and Ebola revived a sense of personal insecurity. Residents watched friends and loved ones with malaria-like symptoms go to hospitals or clinics-and sometimes not return. During July, many of those health centers began closing down, overwhelmed by the disease;3 and while new isolation and treatment facilities were under construction, ambulances sometimes transported people to distant locations where family members were unable to find them. Fear, distrust, and rumor had serious implications for the contact tracers and case management teams whose job was to locate people who had been exposed to someone with Ebola and monitor them for 21 days, the disease's incubation period. "The face of the response is the contact tracers," said Mosoka Fallah, who has a doctorate in microbiology and immunology and was head of disease surveillance in Montserrado County, home of the country's capital. "If an ambulance doesn't show up, people blame the contact tracers because they see them every day." Fallah added that contact tracers were sometimes taken hostage by angry community members. In a country with few passable roads and extremely limited internet access, even highly trained teams struggled to locate people. Further, without the cooperation of skeptical Liberians, there was little prospect of success. In early August 2014, when the rate of new infections began to increase rapidly and the president declared a national emergency, the Liberian government and its international partners decided it was time to find a new way to manage disease surveillance: through contact tracing, case management, and data sharing. THE CHALLENGE The standard approach to contact tracing usually entailed, first, identifying those who had been infected and second, tracking down all the people who had been in contact with each infected person since the onset of illness-called the line list. Case managers then conducted follow-up visits with all contacts to monitor them for signs and symptoms of disease during the incubation period. That monitoring information became the basis for forecasting supply needs and critical interventions, such as isolation, treatment, and public education. Out in the field, the names attached to those two roles-contact tracers versus case managers-sometimes flipped, but the functions remained the same. Effectively carrying out this function required speed, accuracy, and strong rapport between health officers and citizens. Seamless coordination, close alignment of interests between field personnel and their supervisors, and respect for citizen concerns-ranging from information needs to information privacy-were all essential. During the early months of the epidemic, Liberia's health ministry struggled to create the parts of that process and assemble them into a functioning system. "We had never done contact tracing before," said epidemiologist Luke Bawo, who later assumed responsibility for data management with the Incident Management System (IMS) the Liberian government set up in mid 2014 to coordinate all elements of the country's fight against the disease. A lack of capacity and a skeptical populace stood in the way of the effort. Even into August, some counties had no people trained to conduct contact tracing or collect lab specimens, and there were few ambulances or trained drivers to transport those who fell ill during the 21-day monitoring period. Both the US Centers for Disease Control and Prevention (CDC) and the World Health Organization (WHO) had sent advisers to help when the first cases appeared at the end of March. They provided software for tracking the disease, set up a data management system, trained staff, introduced standardized forms for collecting information on contacts, and helped all three of the most affected countries-Liberia, Sierra Leone, and Guinea-begin to produce weekly situation reports. At the end of April, with no signs of new cases, they left, only to return in July after the disease exploded again. The initial model was top-down and case centered. Small teams of county health-care workers visited communities when someone reported a sick neighbor or family member or when there was a death possibly attributable to Ebola. The teams then tried to discover who had been exposed, assessed how contact had occurred, provided information about how to limit transmission to others, and set up return visits to monitor for signs of the disease (text box 1). Then they took their reports back to the county office, which relayed the information to the health ministry in Monrovia. Any new approach would have to solve the problems that dogged that initial strategy. There was a shortage of village health clinics and community-level monitors who could serve as Ebola sentinels and alert the authorities to cases of the disease either through conventional reporting channels or on a special hotline set up in early August. The government drew on a pool of unpaid community volunteers, who usually worked on vaccination campaigns, to become contact tracers in villages and neighborhoods. But not enough people were available to help, and none of the volunteers had training or experience with contact tracing. In the midst of the Ebola outbreak, Liberia had only 51 doctors, 978 nurses and midwives, and 269 pharmacists4 for a country population of 4.3 million. Although the standard components of contact tracing were theoretically straightforward and did not require scarce clinical knowledge, Liberia, with an adult literacy rate of only 43%,5 was short on people with the skills needed to fill out the forms that were the basis for the line list. To make matters worse, there was little capacity to supervise those who carried out the work. Communication between the teams responsible for each element of disease surveillance was slow. Fallah said that contact tracers, who reported to district surveillance officers and then to country-level health ministry officials, relied on case management committees to confirm and classify each Ebola diagnosis (figure 1) and to review the list of contacts before they could begin the 21-day follow-up visits. In addition to their responsibility for listing those who had been in contact with someone who was ill with the disease, case management committees-which consisted of doctors and trained nurses as well as international partners-were responsible for tracking tests and test results and for arranging the transportation of infected people to Ebola treatment units. Similar systems existed in Sierra Leone and Guinea (text box 2). "Sometimes it took three or four days before we got the list [of contacts]. By the time we got back to the house [to follow up], the contact was lost," Fallah said. During the time between the identification of contacts (handled by case management) and follow-up, a contact who feared officials might go into hiding or flee to a different part of the country. Stories about people who never returned from isolation or treatment intensified the distrust left over from the civil war. Even if all of the conventional elements were in place, Liberia's system was still reactive rather than proactive-another big challenge, Fallah said. "First of all there had to be an alert"-typically through a community member calling the Ebola hotline, "before we could respond," he said. In the Liberian context of citizen distrust of the government, "we realized that couldn't work." If people lived in remote areas or hid and no one alerted the response, the disease could flourish and spread. Addressing the problem required a procedure to search for cases rather than wait for alerts from a distrustful populace. Information management posed another pressing challenge. A data team, located first in the health ministry and later in the country's special Incident Management System, was responsible for collating the line lists the district and county teams created and for producing a daily national situation report on numbers of new infections, numbers of people exposed, and locations of outbreaks. Epidemiologists needed those situation reports, called sitreps, to forecast numbers of people likely to become affected, to flag vulnerable communities, and to help target resources appropriately. The resources to help carry out that work were limited, however. Bawo, who was responsible for this arm of the response, lamented the inadequacies of the database software the health ministry used. "Prior to Ebola, we did not have an automated system, and Ebola exposed [that] weakness," he said. "We had to physically set up an ad hoc, parallel reporting system." Without reliable Internet access, reports came from the field by whatever means people could find. The information was then entered into a database. As it flowed in, the information itself caused confusion. The county reports drew on information from too many sources: case notification forms completed in the field, as well as similar reports from treatment centers, laboratories, and burial teams. Sometimes there were duplicate entries. And because names were spelled inconsistently, it was difficult to be certain whether two reports were actually talking about the same person or different people. In villages without streets and in cities whose streets had no names, there were no addresses to help sort out identities.6 Nor did people have identity cards that would clarify who was who. Further, those involved in disease surveillance had to deal with the broader challenges of cooperating and coordinating with the other essential parts of the Ebola outbreak response. People who were asked to quarantine themselves needed food, water, chlorine, and help looking after their family members who remained behind, as well as their livestock or crops; without such help, they might hesitate and flee. And if a contact tracer or manager called for an ambulance, the vehicle and medical team had to arrive quickly to prevent further infection. The different elements "didn't work in isolation," said Chea Sanford Wesseh, Liberian assistant minister of health statistics who later headed the contact-tracing operation for all regions outside Montserrado County. "If you have lapses in one component of the response, it affects all the others," he said. Contact tracers and case managers relied on the logisticians for crucial supplies-including stationery, personal protective equipment, and transportation; on laboratories to process Ebola samples; and on social mobilization teams to communicate information to people. Those working on disease surveillance had to be mindful of such interactions and adapt constantly. Without cooperation from families and their communities, however, even successfully addressing those difficult challenges would do little to improve capacity. In early August, distrust boiled over. With more than 80 new Ebola cases registered throughout the country every week, the Liberian government confirmed that the deadly outbreak had reached the densely populated neighborhood of West Point in the capital city, Monrovia. In West Point, the disease encountered a highly impoverished community deeply distrustful of the government's Ebola prevention edicts. Wesseh explained that "the distrust and belief that Ebola was not real and that it was a political conspiracy affected the . . . response." When the disease reached West Point, "we had cases running away and hiding." After the first case was registered in the community on August 6, the government hastily converted a school building in the community into a temporary Ebola isolation center. But anger grew when infected people from outside West Point were brought to the facility, leading to fears that Ebola was being imported into the neighborhood. On August 18, West Point rioting residents looted the Ebola isolation center of its supplies and equipment, including contaminated materials. With no system in place to locate the dozens of people potentially exposed to Ebola during the chaos, government forces cordoned off the entire community. The forced quarantine sparked rage, and in subsequent unrest, the military opened fire, killing a teenage boy and injuring two other people.7 Contact tracers and case managers could not get to work. FRAMING A RESPONSE "West Point was a turning point," Fallah recalled. "Until then, [the Ebola response] was a completely top-down approach," he said. "The most important thing West Point taught us was that the top-down approach was not going to work." In mid-August, as the rate of new infections began to increase rapidly, President Ellen Johnson Sirleaf and key ministers adopted a CDC proposal to reengineer disease surveillance. They took coordination of the Ebola response out of the beleaguered health ministry and accelerated the ministry's efforts to create a dedicated Incident Management System. The president appointed Tolbert Nyenswah, an assistant minister for preventive health care, as head of the IMS. The IMS structure designated a single contact point for each main function and brought together all partners in task teams that operated under its umbrella. Alongside dedicated committees for social mobilization, laboratory services, case management, and logistics, Nyenswah created an IMS epidemiological surveillance committee tasked with managing Ebola-related data and a separate committee for contact tracing focused on field surveillance. Nyenswah appointed Bawo to head data management, and Wesseh chaired contact tracing outside Montserrado County. Fallah remained responsible for Montserrado, including Monrovia. WHO became the co-chair for contact tracing, and the CDC co-led epidemiological surveillance (figure 2). The West Point incident spurred Fallah to propose a new model for surveillance specifically designed for the densely populated urban slums of Montserrado County. The county presented a set of unique challenges for disease surveillance: It was densely populated (West Point alone had 80,000 residents living in less than one-half square kilometer), and its residents frequently traveled to other parts of the country by moped, bus, bicycle, or boat, as well as on foot, in order to tend farms, trade, or visit family members. By contrast with past outbreaks in other parts of Africa, where Ebola was limited to remote rural communities, in Liberia-as in Sierra Leone and Guinea-the disease spread rapidly to capital cities, putting many more people at risk. In an article published later, Fallah and his collaborators noted that the West Point riots revealed how "inadequate consideration of [cultural and organizational] factors hindered the initial top-down Ebola response. . . . Slum and ethnic settlements tend to have cultures characterized by the rituals of prevailing tribal groups and a strong sense of connectedness among persons who have lived together for decades. Externally led interventions, such as quarantine and body collection, that did not engage regarded community leaders led to distrust for and underuse of these interventions."8 Fallah, who had grown up in Monrovia and was in West Point during the quarantine riots, was aware that he had to energize Liberia's passive Ebola surveillance system. Doing so required enlisting active participation by the country's neglected local leaders. "We wanted to do horizontal communication: neighbors to neighbors," he said. "You need guys who know their community to literally go and find the sick." In Montserrado County, which quickly became the epicenter of the Liberian Ebola outbreak, the idea was to shift the model "to running [reactive contact-tracing] surveillance and active surveillance parallel to each other," Fallah explained. There were already examples of that approach. In the community of Soul Clinic in Paynesville, outside Monrovia, Fallah reached out to Jonathan Enders, administrator of a local school serving about 400 students. In July 2014, Enders had recognized the threat posed by the Ebola outbreak. When all schools closed in the face of the epidemic, "I got different leaders together to get organized into a task team," Enders said. Under Enders's leadership and without any support from the government, community volunteers in Soul Clinic began by gathering all of the information on Ebola they could find. They also mapped the area, and using "jerricans from the school and chlorine donated by the community, we set up roadblocks with handwashing buckets on all roads into the area," Enders said. In addition to monitoring newcomers for symptoms of Ebola, Soul Clinic created a system for providing water and food for households with sick people. "Because of the fear of Ebola, the community didn't want them to use the hand pumps," Enders said. "So we mobilized young people to bring them water. Sick people put their buckets outside their houses every morning, and the youths would fill the buckets without touching them. We also collected some money to buy rice." Enders illustrated the power of community involvement by recounting an incident in which a father had died of the disease and the mother and two children had been quarantined in their house. When one of the children started showing Ebola symptoms, Enders got word from one of the family's neighbors that "the family was planning to take the child into hiding." He immediately called one of his task team colleagues and agreed that "we would meet at 5 a.m. the next morning at that house before they got up. By responding quickly, we were able to arrest that situation and prevent them from going into hiding. If we had had to wait for the [IMS to respond], it would have been too late." By the end of August, with the outbreak accelerating at a pace of more than 100 new cases per day and the number of Ebola deaths in Liberia at nearly 700,9 it was clear that in order to implement an effective community surveillance operation, Fallah, who ran disease surveillance in Montserrado County, and Wesseh, who ran disease surveillance in the rest of the country, had to develop ways to recruit, train, compensate, and monitor the work of many more contact tracers. Fallah and Wesseh decided to complement reactive contact tracing by creating a bottom-up surveillance structure like the one Enders had fostered: in the form of teams of community members who proactively searched for Ebola cases in their neighborhoods (text box 3). For Bawo and his colleagues, the goal was to build an information management system that could effectively track new cases and forecast trends in new infections. GETTING DOWN TO WORK Implementing the vision of bottom-up, active contact tracing and of case management while improving data management required simultaneous actions on several fronts. It took time to put the pieces into place. Launching an experiment Fallah's first step was to organize an initial round of town-hall-style meetings with groups in West Point on September 17. He chose West Point because it was both an especially hostile place for health-care workers and a major source of new infections. His agenda for the meetings had several aims. "First, tell them we apologize for the top-down approach. The second thing was to listen to them. Don't say a single word, and write down what they say." The next step was to ask the people assembled at the meetings to suggest solutions. "This gave them a sense of empowerment, of being in control," Fallah said. "Our philosophy became that the community leads and we follow." An important goal was to identify both formal and informal community leaders, "the people whom the community listened to," Fallah said. "Some of them were elected leaders, but others were students and teachers the community looked at to lead them. We had to bring them on board." Based on what he learned during the meetings and drawing on support from NGO Action Against Hunger as well as from the United Nations Development Programme (UNDP), Fallah created what he called the Community-Based Initiative. The idea was to reach out to local traditional and religious leaders as well as medical students and teachers "of all ages" so as to enlist their help. Fallah invited them to become "active case finders" who would go door-to-door to inquire about sick family members; to learn about instances of possible exposure, such as unreported deaths and funerals; to identify problems; and to answer questions. A main part of the focus was on events that might trigger infection. Following a one-day training session on the basics of case finding, at which recruits learned the basics of infection prevention, contact identification, and reporting, Fallah's team mapped the West Point area comprehensively. The team then assigned each active case finder 25 households to visit on a daily basis.10 The case finders reported to coordinators, who reported to district supervisors, who in turn reported to the county. Within 24 hours of the program's launch in West Point on September 17, the team of active case finders had identified and arranged transport for 42 people with Ebola-like symptoms to treatment units. The team also uncovered 34 suspected Ebola deaths and a number of secret burials that had not been reported to the authorities through the existing contact-tracing system.11 Although the Community-Based Initiative was an emergency response and not designed intentionally as a pilot project, its immediate success in West Point offered the IMS a new model for tracing a path out of the crisis: (1) listen to community concerns; (2) involve formal and informal local leaders in the process of designing solutions; and (3) empower and train community members to become actively involved in surveillance efforts. Armed with the model, the IMS and its partners sought to apply the principles throughout the national Ebola disease surveillance effort. Rolling out field surveillance The first step in expanding the Community-Based Initiative beyond West Point was to build in September 2014 a network with leaders in other parts of Montserrado County, the epicenter of most new infections. Fallah explained that "as we drove around [Monrovia], [we] saw a lot of organization going on. . . . Some [communities] were doing [surveillance] on their own." Working with a small group of local contacts and colleagues from Action Against Hunger and the UNDP, Fallah sought out community leaders who had taken their own actions to combat Ebola and drew them into his initiative. The goal was that networks of local leaders would identify and recruit people to become contact tracers and active case finders. Fallah held town-hall-style meetings with affected communities to discuss involvement in the Community-Based Initiative. Once they had accepted to participate, local leaders mapped out their areas to determine the locations and numbers of households as well as their training and resource needs. At a second community meeting, Fallah and his team informed people about the roles they could play and also asked them to nominate peers to serve as supervisors. The meetings created the platform for bottom-up community surveillance across Montserrado County. Wesseh oversaw a similar process, on a smaller scale, outside the capital city area. The chain of command outside Montserrado extended from the team to the district, from the district to the county, and then to the IMS. But Montserrado County, with a quarter of the country's population, was bigger and more complicated than other areas. Fallah's office divided the county into four sectors. Active case finders and contact tracers reported their daily activities to a team of supervisors, who then reported to their sector coordinators. Supervisors used a mobile app developed by students from the Yale School of Public Health's Center for Infectious Disease Modeling and Analysis to speed up the reporting process and to send out contact tracers, ambulance teams, or other support as needed. In addition to the reporting chain, Enders, who was coordinator for one of the four sectors in Montserrado, said there were also independent monitors to help supervise. "They were an independent group that checked whether what the sector group reported was correct," Enders said. Fallah said he expanded the original monitoring team that carried out spot checks in the field "from 2 to 33 people, 11 of whom were mobile supervisors." Partners such as WHO embedded staff members to assist. Fallah and his partners then turned their attention to strengthening the systems for compensating, training, and monitoring the new contact tracers and active case finders, who eventually numbered 5,400.12 Another 1,000 worked in related capacities. The government and NGOs had already started to address compensation, a hot issue in the wake of an August protest by health-care workers over delayed pay. By October, when Fallah's system was ready to roll, the IMS had created a standard policy for paying contact tracers and the new active case finders at the same rate. "Outside Montserrado, the payment was standardized at $50 per month. Within Montserrado, it was $75 to $150," said Wesseh. The reason for the pay difference, he explained, was that "in Montserrado, we had more educated people than in the other counties, including university students and graduates. We thought it was fair to give them a little more than just $50." In a county in which 83.8% of the population lived below the poverty line of $1.25 per day, both amounts constituted significant incentives.13 The UNDP paid the wage bill as well as the training costs. Wesseh said contact tracers were drawn from a pool of community health volunteers who, before the Ebola outbreak, had usually not been paid for their services. In the context of the Ebola emergency, "we had to motivate them to do contact tracing" by paying a wage, he said. At a time when schools and some businesses had closed and food costs were rising, the prospect of employment was attractive, but the payments also served as a token of appreciation for the risks taken by the people who helped. An important distinction in terms of compensation was that contact tracers were "activated" only after cases appeared in their area, whereas active case finders in Montserrado were going door-to-door every day. Wesseh said, "When there is a case, [we] just use the county health team's officers to recruit automatically those people . . . who are community [volunteers] to do contact tracing." Consequently, contact tracers were paid only when their services were required. With technical support from WHO epidemiologists, the IMS disease surveillance committee also focused on mass training for active case finders. In August, the IMS prepared a curriculum that covered how to identify cases, talk about Ebola, and file reports, as well as safety issues. But the procedures taught in the August curriculum did not completely suit the new strategy. Fallah said one of the shortcomings of the early training efforts was the fact that the people called contact tracers were not authorized to compile official lists of the contacts of potential Ebola cases. Instead, the contact tracers had to wait for case managers to do the actual listing before they could begin the 21-day monitoring process. That division of labor slowed response time. As a result, "it became necessary to go back and teach contact tracers to do everything, from contact identification to contact listing and follow-up," Fallah said. From October on, community members who became contact tracers learned to investigate and develop the listing form (exhibit 1). The forms they used categorized those exposed to Ebola according to level of contact. The highest priority was assigned to contacts who had touched the bodily fluids of an infected person; the lowest priority was assigned to contacts who slept, ate, or spent time in the same household or room as an infected person did. Empowering contact tracers to complete the form without having to wait for case managers had the potential to speed up the response. The training provided for active case finders in Montserrado County was less comprehensive. Most of the training sessions were one-day events. Siedoh Freeman, a UNDP field associate and medical student who worked with the Community-Based Initiative, said, "The training wasn't complicated; it was basically infection prevention, some social mobilization, and most important, case finding." During the peak of the epidemic in October, the system enabled the disease surveillance team to significantly increase the size of its corps in the field. According to Fallah, the Community-Based Initiative eventually had a team of 6,500 active case finders in Montserrado County. Wesseh estimated in addition that there were "5,000 to 10,000" contact tracers in the field or on standby throughout the country during that period. Managing data Contact tracing helped district and county officials link families to ambulances, treatment, services, and burial teams, but the information generated was critical to other important functions as well. The availability of accurate epidemiological data was crucial in the fight against Ebola because such data had the potential to enhance Liberia's ability to forecast and respond to the outbreak. Those functions hinged on the IMS's ability to process and analyze the data produced by thousands of people working on the ground. But during the early months of the outbreak, coordination problems and poor data quality had paralyzed the ministry's data management committee. Bawo, who had been appointed by the health ministry in early June 2014 to lead data management, noted that he had had to start from scratch. "We did not have a system for capturing and instant reporting of [Ebola case] alerts," he said. That portion of the health information system had been lacking, and the first task was to create a structure "that would coordinate everyone. We needed to develop a strategic direction for data management," he said. Bawo's staff came mainly from his original unit in the health ministry and from other parts of the ministry. They set up operations in a conference room on the fourth floor of the health ministry's headquarters in Monrovia and then turned their attention to designing "a singular reporting process . . . to defragment all of the multiple streams [data sources]," which included not only county-referred case investigation reports but also information referred from Ebola treatment units, laboratories, and burial teams (text box 3). In May, the CDC had given Liberia's health ministry an adapted version of a data management system called Epi Info to help address the ministry's needs. Epi Info was a suite of software tools that anyone could use at no charge to help track and visualize epidemiological information. The CDC introduced a version of the software designed for viral hemorrhagic fever, referred to as VHF. But the Epi Info system, originally designed for small, localized Ebola outbreaks in Uganda, had not been updated for the new, Liberian context. Dikena Jackson, administrator in the health ministry's nutrition division who volunteered to join Bawo's unit in July 2014, said: "We had so many variables that didn't really apply to Liberia. When I was in Lofa and tried to enter the name of a village, it would give me the name of a community somewhere in East Africa." Although the CDC began to update the software, the changes made the platform unstable or altered the way it worked, and there were no tech people to help users solve those problems. The software presented another issue, too. Jackson explained that Epi Info allowed only "one computer and one person at a time [to] enter data. It was impossible for two or more persons to enter data simultaneously." In contrast with previous outbreaks, which had been limited to single locations, Liberia now had many clusters of cases in different parts of the country. In June and July, before the national Incident Management System was established, the majority of cases had come from Lofa County on the border with Guinea, where Liberia's Ebola epidemic had originated. The data management team in Monrovia initially relied on the county health team in Lofa to relay Ebola case statistics to the team via e-mail-for subsequent input into Epi Info/VHF. Jackson said, "By mid-July, we had so much information that had never been entered!" There were almost a thousand forms from Lofa County still to add. Bawo realized it was critically important to create a centralized management system to improve coordination and make sure that Lofa County's statistics got incorporated into the database. In early August, he asked a three-member team from the data management committee, including Jackson, to travel to Voinjama, the capital of Lofa County, to create a new structure for data management. Jackson said, "It was now our job to go to Lofa, document everything that was there and put it into the system." The ministry arranged a rapid Epi Info training session for Jackson and his colleagues, many of whom had had no previous experience with database management. When they got to Voinjama, "we set up an intranet system so two or more persons could enter the data into Epi Info. But we couldn't share the updated files [with the team in Monrovia] directly through the program, so after we entered it on that side, we had to e-mail the consolidated file back to them." Improving data quality With a rudimentary structure in place for transmitting consolidated information back to the national office in Monrovia, the primary problem Jackson's team confronted was the poor quality of data submitted through the case investigation forms. The use of multiple forms for the same case confounded identification and tracking. Because paper forms might be contaminated with the Ebola virus, such documents could not be conveyed with patients. As a result, case managers, ambulance crews, and clinic staffers all used new forms each time they encountered a new patient. The case management team completed the first while visiting an infected person at home or on the way to the Ebola treatment unit. Then staff members at the treatment unit would send a second for the same infected person, and a few days later, a lab would convey an Excel file containing the person's official test results. Sometimes each new form opened for a case had its own unique ID instead of the patient ID. The result was that "the same Ebola case would end up with three different case IDs," Jackson said. To make matters worse, it was hard to be certain that the forms were for the same person, because volunteers had only limited training and did not always complete the forms correctly. Nicknames sometimes substituted for real names, or names got misspelled. As time went on, contact-tracing information began to arrive not only on paper forms but also electronically-through a variety of cell phone apps. Although intended to speed the flow of information, the proliferation of media sometimes sowed confusion (text box 4). The Lofa team's solution was to work only with information submitted by the laboratory, which the team regarded as the most reliable data source. According to Jackson, "What we did was to go through the laboratory records in Excel and Epi Info one by one to compare the names and make sure they matched." It was a time-consuming exercise. "Because names were misspelled and case IDs were unreliable, we had to go through it line by line," he said. At IMS headquarters in Monrovia, Bawo created a similarly structured work flow for the entire data management team. He assigned Jackson, after Jackson's return from Lofa County, to manage all incoming data from Ebola laboratories. (During the height of the outbreak, Liberia had five laboratories for testing Ebola samples.) Bawo and Jackson created a central e-mail address to which all labs sent t results. "The lab was, and will always be, the most credible source for confirmed cases," Bawo said. "You can have suspect cases, many of which will be negative or indeterminate. But it's the lab source that will definitely be the key to following the trend of the disease" (text box 5). Although that strategy potentially undercounted cases because not everyone received a lab diagnosis and because sometimes people died and were buried before lab confirmation, it was a beginning. With the laboratory data as the backbone of the system, Bawo assigned other team members to process information from case management committees, contact tracers, and Ebola treatment units. "Once we get the lab results, we match them with the data from those other sources," Bawo said, adding that burial teams investigated and submitted information related to the bodies of Ebola victims who were not immediately identifiable. Bawo had the job of synchronizing the different sources. "Eventually, at the end of the workday, everything converged onto me," he said. He then prepared the situation report to send to the leaders of the Incident Management System committees at 5 a.m. the following day. There was also a daily 4 p.m. meeting, when members of the data management unit presented for discussion the information in the situation report. Those sessions were open to technical people and epidemiologists from any organization. Early on, other standardization problems also caused confusion. For example, for a time there was uncertainty about when the week used for counting cases began: on a Sunday, a Saturday, or some other day? "When does an Epiweek start? What definition do you use for a suspected case? These things all changed and were frequently different from location to location and depended on who you were reporting to, particularly early on in the epidemic," said Amanda Tiffany, an epidemiologist with Epicentre, the epidemiology and research group of Médecins Sans Frontières (MSF, or Doctors Without Borders), speaking of her experience across all three of the most-affected countries. Shifting platforms In late August and early September, the data entry problems worsened and the software issues increased. There were so many reports to load into the database that US embassy officials and UN personnel with data entry skills volunteered their time and worked in shifts. But as the rate of new Ebola cases accelerated and the disease spread to all of Liberia's 15 counties, Epi Info/VHF began to fail. On September 10, the system collapsed, forcing workers back to paper-based records. Bawo said: "People began basically just to manage using the paper format. At some point, my team would be calling all around the country, asking how many suspect cases [there were] and trying to write out the names." Bawo said that although the Epi Info platform was useful "because it integrated laboratory data and had built-in modules for contact tracing and case investigation," it was also limited. "It was built for managing an outbreak of 30 cases," he said. "It became ineffectual as the number of cases increased," partly because several users could not enter data at the same time and partly because upgrades had made the system unstable. An additional limitation stemmed from the software, which did not enable epidemiologists or technicians to scrutinize the data and correct mistakes easily. Everything had to be correct when entered into the database, and given that names in Liberia were often spelled differently based on a person's regional or ethnic background, the team realized it was time for a fresh approach that permitted greater flexibility. Bawo initially attempted to shift to another open-source software platform, called District Health Information Software version 2, which had been in use at the health ministry prior to the Ebola epidemic. But the new platform had many of the same problems as the old one did. All information had to be entered correctly the first time. "With all these challenges and with the outbreak not waiting for us, we had to go to other solutions," said Bawo. Eventually, Bawo's committee decided to use Microsoft's Excel program as the primary Ebola data management platform. Excel spreadsheets could be merged and easily corrected-if users handled the files carefully. A person who had Internet access could download files from Dropbox, a free, cloud-based file-sharing system; add new information; and upload the revised spreadsheets for others to use. Disease surveillance teams in Guinea and Sierra Leone had also started to use Excel in lieu of more-sophisticated platforms. Although the Excel system worked better than alternatives, in practice it was far from perfect. Sometimes someone would sort a column according to a specific parameter and forget to select and sort the rest of the spreadsheet at the same time, scrambling the data. Dropbox could also create problems because it allowed simultaneous editing. Sharing data through Dropbox caused some to worry about violating patient privacy protections, too. A single, user-friendly software platform accessible from multiple locations would have been a better option, but it did not exist at the time. Making sense of the data Developing and maintaining the database represented only part of the challenge. The Incident Management System addressed many of the logistical problems that had dogged the disease surveillance operation during the early months of the outbreak. But in October, the high volume of information coupled with delayed data entry, corrections in past reports, and mistakes had also created a crisis in another critical function: identifying patterns and trends. Both assessing impact and targeting activity effectively depended on eliminating the noise, or error, in the data and on getting the numbers right. Fortunately, at a critical moment in mid-October, a new volunteer with the skills to penetrate the numerical murk appeared on the scene. Hans Rosling, a Swedish statistician internationally known for his work on data visualization, flew to Liberia on October 20 at the invitation of Peter Graaff, head of the UN Mission for Ebola Emergency Response. Rosling described the situation as "complete confusion," saying, "We didn't know whether the epidemic curve was rising, whether it was flat, or whether it was going down." But he said it was possible to solve the noise problem. The first step, he said, was to "rapidly create an epidemic curve based only on positive lab results." To achieve that goal, the IMS data team first had to develop an algorithm capable of removing duplicate entries within the results. Most of the laboratory duplications had resulted from the fact that Ebola survivors were usually tested twice: once when they entered a treatment unit and a second time once they had recovered and were ready to leave. "But we didn't have anyone who knew Excel well enough" to code the required algorithm, Rosling said. With clearance from Incident Management System leaders and taking measures to ensure patient confidentiality, Rosling sent the data files to his son, Ola, who had cofounded the Gapminder Foundation with his father and was a former product manager of public data at Google. Gapminder had a global reputation for helping teachers, students, and policy makers visualize development data by using a simple, Excel-based application. Ola created in 18 hours the algorithm Bawo's team needed. "He also created the new indicator we used: 'average number of cases per day during the last 21 days,'" Rosling said. Bawo explained the significance of the indicator: "The window period for infection after contact [with an Ebola-infected person] is 21 days. If we are getting zero confirmed cases as those days go by, we begin to hope we are getting to the end." Averaging helped reduce the noise in the data, said Benjamin Dalziel, an epidemiologist from Oregon State University who was part of the Ebola response in neighboring Sierra Leone. "You want to know who falls ill, on what date, and where. It sounds simple, but the data are always noisy," he explained. "Not every ill person goes to the hospital immediately after noticing symptoms. So, for instance, not all of the people diagnosed on July 1 actually fell ill on July 1. And if you are trying to estimate the rate of infection, you have to account for that. Averaging helps address that problem, but there are different ways to average. The weighting strategies vary." As a check, Bawo and Rosling "got the Liberian members of the team to eyeball the data," Rosling said. "Only Liberians could check those names for accuracy." The result was "a beautiful indicator, because it gave a smooth curve," he said. "And like a snake, it started to go down. On November 1, data managers presented the new indicator at the morning IMS meeting. "We showed that the epidemic had turned around; it's on its way down," Rosling recalled. With a reliable platform and epidemic curve indicators in place by mid November, Rosling was satisfied that the reforms had "turned data processors and number crunchers into detectives." The data management team created a war room at the health ministry so the Montserrado team could analyze and display all Ebola cases in the county. "We filled the walls with graphs and maps [of Montserrado County] and used colored stickers to indicate outbreaks," Rosling said. "If you want people to follow an epidemic, it has to be physical." The team scheduled daily meetings with Fallah and his team of contact tracers and active case finders in the war room where "they could point at the map," Rosling explained. Analytical capacity improved once everyone could see the locations of the new infections and see the patterns. By the time the data systems were running well, the number of new infections per week had fallen into the 40s from 353 during the first week of November. At the time, a total of 6,525 people were infected in Liberia.14 Although confirmation that the epidemic was easing was good news, the finding triggered surprising reactions. Bawo recounted how, on the political level, "some people said: 'You have to be careful [about publishing the data], lest people get a false sense of security.'" Someone with an unreported infection could create a new hotspot, certainly, but capacity to contain a new outbreak was finally in place by this point (text box 6). Rosling said some groups involved in the Ebola fight had other reasons for wanting to keep a lid on the finding. "We had a lot of problems with [attempts to] exaggerate [the number of Ebola cases] in November and December," because some organizations "wanted to generate high numbers [in order to] increase their resources." But "we resisted that," Bawo said. Instead, he noted, responders resolved to "just be honest about the situation: 'We are reporting [the numbers] as they are.'" OVERCOMING OBSTACLES By late October, the creation of a coordinated disease surveillance system that directly engaged members of affected communities, as well as the introduction of an Ebola data management structure, had improved the IMS's ability to respond to new cases. But the sheer number of total reported cases in Liberia, which stood at 6,525 by November 5,15 meant that responders were still unable to track the epidemic in real time. Bawo's philosophy during the early months of the epidemic had been that "even the barest of data can support a response, so [basic data] is better than wasting valuable time trying to be detailed," but responders were aware that getting to zero new cases would require much more detailed information and rapid sharing of that information with Fallah's field team. With the fundamentals in place, Bawo and the data management group worked to improve direct coordination with the surveillance team on the ground. No one knew whether a new outbreak would pop up and spread rapidly, but armed with the knowledge that the epidemic curve was easing, responders intensified their focus on the one remaining Ebola hotspot: Montserrado County. The densely populated neighborhoods and the mobility of Montserrado's urban population meant it was the hardest place to eradicate Ebola from. Nevertheless, in early December, President Sirleaf set a target of "getting to zero" new Ebola infections. In order to achieve that goal, she said, "we all have to intensify our efforts to travel that very difficult last mile. To go from 100 to 90 is hard, but to go from 10 to 0 is even harder."16 In response, the disease surveillance operation "moved from the general [level] to the specific," Jackson said. Nyenswah created a Montserrado Incident Management System to further focus the response in Montserrado County, building on Fallah's pilot program. The county's four sectors were subdivided into zones, a strategy that had been successful during immunization campaigns. The zones were based on informal community identification rather than government administrative boundaries, which enabled a grassroots effort to track Ebola cases and contacts.17 With creation of the Montserrado Incident Management System and the more granular focus on Montserrado County, responders again modified the surveillance system. Their first step was to stop listing contacts for suspected cases in Monrovia and to focus only on following up on laboratory-confirmed cases. Rosling explained that that decision was taken in reaction to a phenomenon he called "Ebola economics." During the early months of the outbreak, skepticism about Ebola meant people were hesitant to work with contact tracers; but with an improved system in place by December 2014, "people were very willing to be listed as contacts," he said. "They would even ask their [sick] family members: 'Can't you put me on that list? Because then I get food" while under quarantine. According to Fallah, such responses were so numerous that "we had huge contact follow-up, when maybe only 40% of them should actually have been followed." Following too many contacts strained manpower and compromised quality. In addition to working only with laboratory-confirmed cases, Bawo created a direct link between data managers and Fallah's field team. Jackson, who was in charge of processing all incoming laboratory data, pointed out that initially, "the lab results were coming from the lab only the next day. So if there was a positive case, no one would know until the next day." Responders addressed that delay by "informing the lab to call me as soon as they have a positive" case, Jackson said. Jackson said he entered each case into the Excel database and "immediately called Fallah to relay the information." Jackson gave Fallah the name, age, sex, and location of the infected person, which further enhanced Fallah's ability to move quickly. "Dikena [Jackson] was able to call me and send the information in 30 seconds," Fallah said. "As soon as the results came and they were negative, we dropped the contacts. . . . If there was a confirmed case, [Jackson] was able to alert me." Fallah explained: "Earlier, we followed everyone, . . . but now we wanted quality, not quantity. After we improved the system in December, we followed fewer suspected cases," and it was easier to target the work effectively. ASSESSING RESULTS In spite of the disease's rapid spread and Liberia's extraordinary resource constraints, the creation of a bottom-up community surveillance system combined with improved coordination and data management, alongside other actions, contributed to the fact that "Liberia's steep epidemic trajectory got reversed several weeks before that of other countries," Fallah concluded.18 The bottom-up system clearly was not the only reason for the reversal. Many communities took it upon themselves to help limit infection in the face of abundant evidence that the disease was highly dangerous. However, the improved system made it possible to target other interventions more effectively and may have helped spread knowledge that communities used in managing the response themselves. Wesseh estimated that during the peak of the outbreak, there were "5,000 to 10,000" contact tracers in the field or on standby. In Montserrado County, an additional 6,500 people worked as active case finders. The bottom-up involvement of community members improved Liberians' cooperation with the response. Based on data from six districts in Montserrado where the Community-Based Initiative was deployed after September 21, 2014, the creation of active case finding increased the number of contacts reported per Ebola case to 9.9 from 7.6, whereas the average amount of time that passed from the onset of an infected person's symptoms until the person got isolated decreased to 4.7 days from 6.5. That was a crucial breakthrough, because the infectiousness of people carrying the virus increased substantially over the course of the disease.19 "Over a period of time, we began to see the number of new cases . . . and the number of contacts listed [going] down," Bawo said. "The cases were no longer sprouting up in new communities; the cases began to localize." At a 2016 meeting with community members who had served as active case finders, Fallah said: "As complex and condensed as West Point is, it was the first community to go to zero. In November, we recorded zero cases of Ebola in West Point, which was confirmed by African Union epidemiologists. It was ordinary people like you who took the risk."20 Bawo said another indicator of the on-the-ground success of disease surveillance arose when "we began to see that all new reported cases already existed on a contact listing." Fallah explained the logic: "If contact tracing is working . . . then no contact should die in the community; the symptoms should be picked up [before the infected person dies]. We had to seal the gap." By late December 2014, "that started happening. . . . That's when we knew we were reaching the end," Bawo said. The fact that data managers correctly concluded on November 1 that Liberia's Ebola epidemic had peaked was another key signal of an improving surveillance system. Amid much skepticism from certain partner organizations and compared with the confusion that had reigned during the early months of the response, the Liberian data management unit was the first group to show that the epidemic was easing. Although Rosling cautioned that disease surveillance was "a dynamic process . . . that only later started working more or less," the team's accurate prediction was a testament to the progress it had made in building a coordinated and effective system for community surveillance. Data quality gradually improved. MSF and other organizations that depended on the information to try to assess where to position people and supplies kept a sharp eye out for inaccuracies. If something didn't look right, they contacted the data management unit. If the WHO situation report offered different information, they also quizzed WHO. Faced with a crippled health-care system that lacked everything from logistics to legitimacy, data management was initially a neglected aspect of Liberia's Ebola response. However, in the end, disease surveillance and data management triumphed over most of the steep challenges that had dogged early efforts to contain the Ebola outbreak. "The importance of data is not easily appreciated by nondata people," said Bawo, the chemist and epidemiologist who led the data management committee of Liberia's Incident Management System. "Everybody looks at the ambulances and the sick people that are being taken out and all of that. But without the analysis and the interpretation of data, it would have been extremely difficult to reach as far as we reached during the epidemic." REFLECTIONS Improvements in contact tracing and data management were fundamental to bringing the Ebola epidemic under control. As the outbreak wound down, the question for all of the partners involved in the response was how to do better the next time an infectious disease struck in a region where health systems were weak and infrastructure was limited. The approach to disease surveillance across countries gradually converged, in part because of learning and sharing between the people directly involved. Countries blended traditional contact tracing with active case finding. District or county health offices assembled and maintained the line lists and local databases, forwarding selected information to the national level.21 The protocols were also similar, based on international standards. What differed were the levels of engagement in training, building teams, and supporting data management. Both domestic leadership and outside involvement mattered. In Liberia, the United States helped build the system through partnerships with the CDC and with nongovernmental groups. In Sierra Leone, the UK government did the same. And in both countries, a central system was created to manage the response. But Guinea's government coordination was more limited, and there was comparatively less outside support. The health ministry was left largely on its own to address the challenge, and the government-highly sensitive to appearances-initially limited the collection and release of data. Responders drew several lessons. One was that acceptance of contact tracers and case managers depended heavily on other arms of the response. Even though members of the Incident Management System were organized into committees that focused on different aspects of the epidemic, disease surveillance started to improve only after responders explicitly focused on bridging the gaps, ensuring that critical supplies like chlorine and food reached families in quarantine and that ambulances arrived quickly to transport the ill. Second, event-based surveillance that focused on ceremonies, for example, turned out to be as important as case-based surveillance.22 Community norms, too, shaped behavior at funerals and weddings, so local knowledge mattered in estimating the numbers of new cases that might follow and the degree of follow-up required. Amanda McClelland, who managed a variety of functions for the International Federation of the Red Cross, said, for example, that for communities in Sierra Leone, a deceased person's status-an infant, a child, an adult, a healer, or local leader-affected whether traditional ways held sway over new rules. As a result, some deaths, such as those for local leaders or healers, caused a spike in infections, whereas others did not. A third recommendation was for all key partners to agree on data management software, and when the next infectious disease outbreak occurred, information was more accurate and flowed faster than it had in 2014. Responders placed a premium on visual cues or other prompts that could increase levels of completeness in reporting. Above all, Liberia's early struggles with creating an Ebola surveillance system revealed the importance of building trust and empowering affected communities. Epidemiologist Mosoka Fallah emphasized that community involvement was the major lesson from his experience in heading disease surveillance in Montserrado County. Responders must "listen to the people," Fallah said. "What happens in an emergency . . . [is that] you want quick answers, but you'll end up losing more time if you don't stop to listen first." He added, "The power of the community is so critical." As a local or international responder, "it's good that you have expertise, but you have to [acknowledge] . . . that communities have the power to change anything." For Fallah, listening to communities and acknowledging their power to effect change from the bottom up were key to overcoming the pervasive distrust that initially characterized the relationship between the Ebola response and many Liberians. "At the end of the day, if you're going to find the [Ebola] contacts, the community has to talk to you. And they can talk to you only if they trust you." ADDENDUM: MODELING AND FORECASTING Hans Rosling joined Liberia's data management effort in October. But outside Liberia, others had also launched their own initiatives to clean the data and model the disease. The information that contact tracers generated was critical for forecasting future trends. It helped fuel estimates of the likely number and location of new infections, which was data essential for deciding how many beds a hospital needed and how many supplies to procure, ship, and stock. No one could wait. The modelers' challenge was to predict new cases based on information drawn from the contact-tracing line lists. Central to that calculation was the basic reproduction number, or R, the average number of secondary cases when a new infection appeared in a previously uninfected population. When R was above one, the epidemic would expand. Several efforts to model the outbreak proceeded in tandem. When the disease was initially confirmed, Peter Piot, who had first discovered the Ebola virus in 1976, in central Africa, exhorted scientists to get to work quickly, and he provided data from an outbreak in East Africa for a team at Imperial College London to use in developing a model. Shortly afterward, the Wellcome Trust offered to pay the salaries for an Imperial College team to divert from its other work and invest in building a model that Médecins Sans Frontières could use to help forecast supply needs. The US Centers for Disease Control and Prevention and the World Health Organization launched their own initiatives to do the same, and the US National Institutes of Health brought other scientists into the effort under the auspices of its Models of Infectious Disease Agent Study. The first problem the scientists encountered involved getting the data from the contact-tracing operations in the field. They needed to know more than the number of new cases-whether suspected, probable, or confirmed. To forecast, it was important also to identify district of residence, district in which the case was reported, age, sex, signs of symptoms, date the symptoms started, date the case was identified, number of contacts per person, form of contact, and whether the person infected was someone already on the list of contacts. WHO acquired some of that information from country databases and merged it into a single database. However, patient privacy concerns and lack of clarity about who owned the information limited WHO's willingness to share. For the public, WHO adapted country-generated summary situation reports, or sitreps, and made a version of them available as pdf files on the Web, as did each affected country's health ministry. But the files contained only the barest information. Moreover, they were not in a form that statistical software could read. To bootstrap around that second shortcoming, Caitlin Rivers, a graduate student at Virginia Tech in the United States took the pdf files of the health ministry sitreps and manually put the information into a form statistical programs could read. Beginning in July 2014, Rivers made her data available to others through GitHub, an open-source software development collaborative. The Imperial College team initially decided to take the pdf files of the sitreps, limited though they were, and clean up the information they contained to the extent possible. Because of their close working relationship with MSF, they were able to secure additional detail to help with forecasting bed needs at the chiefdom or district level. Regardless, it took a lot of effort to reduce the noise in the data, because there were often multiple reports about the same person. The Imperial College teams figured that about 50% of staff time went to that thankless task, which had to be done carefully. The CDC also provided some data, but usually only after a month's delay, which was too long for modelers to provide timely assistance for people on the ground. Personnel rotations at the CDC slowed information exchange and bred frustration. The CDC Epidemic Intelligence Service officers sent to West Africa for month-long assignments had job-related publication requirements and sometimes released the data from their work only after they had written their papers. They were not alone. People from other agencies did the same. But duplication of effort and inability to forecast needs effectively resulted from these practices. Data access was not the only challenge. Epidemic modeling was an imperfect science, only as good as its underlying assumptions and the quality of the information available. Rivers said, "It is hard to quantify behavior change, and that's a huge driver [of infection rates]." Epidemiologist Dalziel agreed. "In retrospect, the other data we would have wanted are data about behavior," he said. "How did people alter their behavior once they became infected? But we knew so little about that...It was clearly important, but models typically don't take account of it directly." Modelers also worried that they lacked good data on R, the average number of secondary cases, scientists often made educated guesses, which could prove wrong. There was a communication problem, too. The models rested on varying corrections for suspected under-counting. Moreover, the generated predictions were always model-based estimates, with confidence intervals or similar statements about likelihood and many assumptions. Each statistical method had its own way to express uncertainty. For nonscientists unused to thinking in those terms, including most journalists, the tendency was to take the upper bound or high estimate on whatever the model spit out as a prediction-and that's usually what journalists and civil servants focused on. Rarely did newspaper coverage spell out either the assumptions or the degree of uncertainty.. The first country-level forecasts sent shockwaves through governments and communities. On September 23, 2014, using a basic model and limited data, the CDC estimated 550,000 cases in Liberia and Sierra Leone by January 20, 2015, if behavior did not change and there were no interventions. The CDC team said that if it corrected for suspected underreporting, the number of cases could go as high as 1.4 million.23 It also generated forecasts that showed what it expected to happen if there were stronger efforts to intervene. If 70% of patients with the Ebola virus went to emergency treatment units or community care centers instead of remaining home, the modeling team said, and if people complied with safe burial practices, "it would 'bend the curve,' causing transmission to drop off substantially." 24 MSF epidemiologist Amanda Tiffany was in Sierra Leone at the time. When she saw the results, she said, "Oh, gosh, this can't be right." It did not line up with what she was seeing, but because everyone was working so hard, often in locations far away from knowledgeable colleagues, it was hard to perceive the big picture. On October 16, a WHO team headed by Chris Dry, offered a different projection: 20,000 probable and confirmed cases across the three most affected countries taken together-about 10,000 in Liberia alone-by early November.25 That more modest projection came closer to the final numbers than did the initial CDC estimates. Rosling's efforts to clean up the data and map new infections so that people could visualize the information helped show that behavior change was slowing or containing the epidemic. The Imperial College team headed by Sebastien Funk and Anton Camacho focused on creating a stochastic model that allowed for random variation in several of its components and generated probability distributions for potential outcomes. It also generated forecasts for chiefdoms or subnational units-a helpful attribute for organizations like MSF, which had to assess the needs for doctors and hospital beds in specific locations. Looking back on the effort, scientists urged the epidemiological community to develop standards and guidance for presenting results and for information sharing. Although some teams posted their codes, it was still hard to reproduce results or to translate results into executable recommendations. Better data, models that could take account of behavior change, and faster and secure release of information to designated teams as well as care in releasing and communicating projections, could have enabled more-efficient and more-effective investment. Innovations for Successful Societies makes its case studies and other publications available to all at no cost, under the guidelines of the Terms of Use listed below. The ISS Web repository is intended to serve as an idea bank, enabling practitioners and scholars to evaluate the pros and cons of different reform strategies and weigh the effects of context. ISS welcomes readers' feedback, including suggestions of additional topics and questions to be considered, corrections, and how case studies are being used: iss@princeton.edu. Terms of Use In downloading or otherwise employing this information, users indicate that: a. They understand that the materials downloaded from the website are protected under United States Copyright Law (Title 17, United States Code). This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/. b. They will use the material only for educational, scholarly, and other noncommercial purposes. c. They will not sell, transfer, assign, license, lease, or otherwise convey any portion of this information to any third party. Republication or display on a third party's website requires the express written permission of the Princeton University Innovations for Successful Societies program or the Princeton University Library. d. They understand that the quotes used in the case study reflect the interviewees' personal points of view. Although all efforts have been made to ensure the accuracy of the information collected, Princeton University does not warrant the accuracy, completeness, timeliness, or other characteristics of any material available online. e. They acknowledge that the content and/or format of the archive and the site may be revised, updated or otherwise modified from time to time. f. They accept that access to and use of the archive are at their own risk. They shall not hold Princeton University liable for any loss or damages resulting from the use of information in the archive. Princeton University assumes no liability for any errors or omissions with respect to the functioning of the archive. g. In all publications, presentations or other communications that incorporate or otherwise rely on information from this archive, they will acknowledge that such information was obtained through the Innovations for Successful Societies website. Our status (and that of any identified contributors) as the authors of material must always be acknowledged and a full credit given as follows: Author(s) or Editor(s) if listed, Full title, Year of publication, Innovations for Successful Societies, Princeton University, http://successfulsocieties.princeton.edu/ References 1 International Rescue Committee, "A Different Kind of Army: A Call to Place Community Leadership at the Center of the Ebola Response," November 2014, p. 9. Accessed August 3, 2016, at https://www.rescue.org/sites/default/files/document/662/ebolacalltoactionlowres.pdf. 2 Saliou Samb and Adam Bailes, "As Ebola stalks West Africa, medics fight mistrust, hostility," Reuters, July 14, 2014. Accessed August 3, 2016, at http://in.reuters.com/article/us-health-ebola-westafrica-idINKBN0FI0P520140714. 3 Jason Beaubien, "Ebola Shuts Down the Oldest Hospital In Liberia," NPR, August 12, 2014. Accessed July 3, 2016, http://www.npr.org/sections/goatsandsoda/2014/08/12/339863749/ebola-shuts-down-the-oldest-hospital-in-liberia. 4 Farouk Chothia, "Ebola drains already weak West African health systems," BBC News, September 24, 2014. Accessed July 3, 2016, http://www.bbc.com/news/world-africa-29324595. 5 "At a glance: Liberia," UNICEF. Accessed July 3, 2016, http://www.unicef.org/infobycountry/liberia_statistics.html. 6 Several sources, including an overview paper: Ashley L. Greiner et al., "Addressing contact tracing challenges-critical to halting Ebola virus disease transmission," International Journal of Infectious Diseases 41(2015):53-55. 7 "Liberian Government's Blunders Pile Up in the Grip of Ebola," Time, September 2, 2014. Accessed January 20, 2016, http://time.com/3247089/liberia-west-point-quarantine-monrovia/. 8 Mosoka Fallah, Bernice Dahn, Tolbert G. Nyenswah, Moses Massaquoi, Laura A. Skrip, Dan Yamin, Martial Ndeffo Mbah, Netty Joe, Siedoh Freeman, Thomas Harris, Zinnah Benson, and Alison P. Galvani, "Interrupting Ebola Transmission in Liberia through Community-Based Initiatives," Annals of Internal Medicine 164(5):367-370. 9 "Ebola virus disease, West Africa-update 28 August 2014," World Health Organization. Accessed June 8, 2016, http://www.afro.who.int/en/clusters-a-programmes/4264-ebola-virus-disease-west-africa-28-august-2014.html. 10 Mosoka Fallah, Bernice Dahn, Tolbert G. Nyenswah, Moses Massaquoi, Laura A. Skrip, Dan Yamin, Martial Ndeffo Mbah, Netty Joe, Siedoh Freeman, Thomas Harris, Zinnah Benson, and Alison P. Galvani. "Interrupting Ebola Transmission in Liberia Through Community-Based Initiatives," Annals of Internal Medicine, 164 (5): 367-370. 11 Mosoka Fallah, Bernice Dahn, Tolbert G. Nyenswah, Moses Massaquoi, Laura A. Skrip, Dan Yamin, Martial Ndeffo Mbah, Netty Joe, Siedoh Freeman, Thomas Harris, Zinnah Benson and Alison P. Galvani. "Interrupting Ebola Transmission in Liberia." 12 "You stood while others fled . . . UNDP staff tells active case finders," July 1, 2016. Accessed August 31, 2016, at http://www.lr.undp.org/content/liberia/en/home/presscenter/articles/2016/07/01/-you-stood-while-others-fled-undp-staff-tells-active-case-finders.html/. 13 At a glance: Liberia, UNICEF. Accessed July 3, 2016, http://www.unicef.org/infobycountry/liberia_statistics.html. 14 Ebola Response Roadmap Situation Report, World Health Organization, November 5, 2014. Accessed June 12, 2016, http://apps.who.int/iris/bitstream/10665/137510/1/roadmapsitrep_5Nov14_eng.pdf?ua=1. 15 Ebola Response Roadmap Situation Report, World Health Organization. 16 "Liberia: Govt Upbeat about Ebola Progress, Wants It Out in 2014," AllAfrica, December 9, 2014. Accessed January 18, 2016, http://allafrica.com/stories/201412100270.html. 17 Thierry Cordier-Lassalle, "Summary of sector response model: A quality management system that led to Liberia's success in fighting Ebola," World Health Organization. Accessed June 29, 2016, http://www.who.int/csr/disease/ebola/sector-approach-liberia.pdf. 18 Mosoka Fallah, Bernice Dahn, Tolbert G. Nyenswah, Moses Massaquoi, Laura A. Skrip, Dan Yamin, Martial Ndeffo Mbah, Netty Joe, Siedoh Freeman, Thomas Harris, Zinnah Benson, and Alison P. Galvani. "Interrupting Ebola Transmission in Liberia." 19 Mosoka Fallah, Bernice Dahn, Tolbert G. Nyenswah, Moses Massaquoi, Laura A. Skrip, Dan Yamin, Martial Ndeffo Mbah, Netty Joe, Siedoh Freeman, Thomas Harris, Zinnah Benson and Alison P. Galvani. "Interrupting Ebola Transmission in Liberia." 20 "You stood while others fled . . . UNDP Staff tells Active Case Finders," July 1, 2016. 21 "Ebola Surveillance-Guinea, Liberia, Sierra Leone," CDC Morbidity and Mortality Weekly Report Supplement, July 8, 2016. Accessed September 2, 2016, at http://www.cdc.gov/mmwr/volumes/65/su/su6503a6.htm. 22 "A Plan for Community Event-Based Surveillance to Reduce Ebola Transmission-Sierra Leone, 2014-2015," CDC Morbidity and Mortality Weekly Report, January 30, 2015. https://www.cdc.gov/mmwr/preview/mmwrhtml/mm6403a7.htm. 23 See overview of the CDC's modeling effort in Martin Meltzer et al., "Modeling in Real Time during the Ebola Response," CDC Morbidity and Mortality Weekly Report, July 8, 2006. Accessed September 2, 2016, at http://www.cdc.gov/mmwr/volumes/65/su/su6503a12.htm. 24 See overview of the CDC's modeling effort in Martin Meltzer et. al. "Modeling in Real Time during the Ebola Response." 25 http://www.nejm.org/doi/full/10.1056/NEJMoa1411100#t=articleBackground. ISS is program of the Woodrow Wilson School of Public and International Affairs: successfulsocieties.princeton.edu. ISS invites readers to share feedback and information on how these cases are being used: iss@princeton.edu. (c) 2017, Trustees of Princeton University. This case study is made available under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Terms of use and citation format appear at the end of this document and at successfulsocieties.princeton.edu/about/terms-conditions.