This blog post is going to be a quick summary of the major things we have discovered about the NTDs and the interventions targetting them, and what we now believe the cost per disability adjusted life year (DALY) achieved by treating NTDs to be.
When we launched in 2009, we estimated the cost-effectiveness of deworming treatments to be around $5 per DALY, which made it the most cost-effective charitable intervention that we could find. However, some of the sources used to arrive at this figure were subsequently shown to contain significant errors. As a result we are now much less confident about this figure. GiveWell has produced an independent cost-effectiveness estimate for deworming of around $170 per DALY, which we believe to be reliable as far as it goes. On the other hand, we think that this fails to fully take into account some factors - including the work the charities do to hand the programs over to governments - which will significantly raise the cost-effectiveness of deworming projects. Taking this into account, we expect deworming charities to be as cost-effective as preventing malaria by providing insecticide-treated nets, and possibly more cost-effective. Nonetheless we are quite uncertain about this, and recognise that more work needs to be done to more accurately estimate the full impact.
The term 'neglected tropical diseases' covers about 15 or so conditions which are particularly prevalent in low income regions in Africa, Asia, and the Americas. A lot of them are parasitic infections, and they are mostly quite cheap to treat. However these treatments are not well funded. We are going to focus on two types of NTDs: Soil-Transmitted Helminths (STHs), and Schistosomiasis (also knows as bilharzia, bilharziosis or snail fever), as these are the ones we believe are particularly cost-effective to treat, and are the target of two of our recommended charities.
STHs - more commonly known as intestinal worms - can cause diarrhoea, abdominal pain, weakness, and anaemia, and may impair learning ability and physical growth. Schistosomiasis is another parasitic worm, whose larvae live in and infect humans from, pools of freshwater. Schistosomiasis can cause progressive damage to the intestinal and urinary systems. Both STHs and schistosomiasis are easy to treat with cheap orally administered drugs that cost around $0.50 per person. Moreover, these treatments have no serious side-effects, so it is possible to give the treatment to whole populations without going through the expensive screening process first.
I'm going to talk about three main sources of information that have come to light since we first made the webpages. The first of these will be the errors we found in the Disease Control Priorities Project (DCP2) estimates, the second is going to be research GiveWell has done to come up with a better cost-effectiveness estimate for deworming treatment, and the third will be the new, updated, Cochrane review of STH-deworming treatments. These aren't the only new sources of information to come out over the last few years, but they are the three most significant.
One of the main sources we first used when recommending deworming as a cost-effective charity was the second round of the Disease Control Priorities Project (DCP2), which aimed to collate cost-effectiveness estimates, in $ per DALY, for a wide range of different interventions. DCP2 gave the cost of treating STHs as $3.41 per DALY saved (i.e. approximately one DALY averted for every seven $0.50 treatment given), and of between $3.36 and $6.92 per DALY for schistosomiasis. These numbers were in the ballpark of several other pieces of data we had, included a prominent paper by Miguel and Kremer (2004), and so these figures were then used to calculate the rough cost-effectiveness of deworming interventions.
However, in 2011 it emerged that in reporting estimate for schistosomiasis, DCP2 had mistakenly inserted a decimal point, and the figure should actually have been $336 - $692 per DALY. Moreover, it transpired that the spreadsheets used to calculate these figures for STHs had other serious mistakes, which led to this estimate also being wrong by a factor of about 100, making the revised figure for treating STHs to be $326. (For more details, see GiveWell's blog post) On further investigation into the DCP2 figure, it emerged that the methods missed some important factors, and used error-prone estimates, making it likely that the corrected estimates were significantly more pessimistic than they ought to be.
Now, in light of this, GiveWell has been attempting to produce a transparent, reliable cost-per-DALY estimate of deworming treatments. They have produced an account of combination deworming, and a spreadsheet to estimate the cost-effectiveness. Without delving into the details, this model seems to be a corrected version of DCP2's methodology, and gives an estimate for combination deworming of around $5000 per life saved. Obviously, this is a slightly different measure to the DALY measure we were using before, and conversion between the two is non-trivial. As a rough, and probably conservative, estimate of this in DALYs, we can consider saving a life to be equivalent to approximately averting 30 DALYs, which makes the figure for combination deworming around $170 per DALY.
In July 2012 there was an updated Cochrane review, which aggregated 42 papers on the effects of STH-deworming treatment. The review came to a couple of main conclusions: that in small, screened trials - where people are first screened for STHs and then treated - treatment of infected individuals was associated with an increase in weight and haemoglobin, which implies a reduction in anaemia. However, it also concluded that there was no significant evidence that unscreened deworming - i.e. in which a whole population is treated without first checking for infections - had a significant effect on weight, nutrition, haemoglobin, or school performance (all of which have been previously thought to be an effect of the treatments).
Now, there being a significant effect in screened trials but not in unscreened trials doesn't make much sense, as it is the same drug in both cases, and the screening process change the bottom line. The most plausible explanations are that either the smaller sample sizes in screened trials make the smaller studies' results spurious, or that this is a result of how the significance of an effect is measured: those not infected 'dilute' the measurable effect the treatment has on the infected, causing the effect to drop out of statistical significance in unscreened populations. In any case, the review seems to indicate that STH-deworming is less effective than we at first thought, and it is clear that more research on the true impact is needed.
Those remain the main new and concrete sources of information on NTDs. However, there are still a couple of things we need to consider and several reasons why we think the cost-effectiveness might still be higher than the GiveWell estimate and the Cochrane review might lead us to believe.
One of the conclusions of the Cochrane report is that there is no good evidence that deworming has a positive effect on school attendance. However, Miguel and Kremer (authors of the influential 2004 paper) together with other experts in the field, have written a response arguing that this finding is misleading, draw erroneous conclusions about school attendance from the data, and maintaining that the educational benefits alone are sufficient to warrant deworming treatments. These arguments seem persuasive, and it appears likely that there are significant educational benefits. We would welcome further research on this subject.
Further, while we believe GiveWell's methodology for estimating the cost-effectiveness of deworming to be generally good, there are a few factors which we feel are not fully taken into account, and which may have a significant impact on the findings.
Perhaps most importantly, both SCI and Deworm the World (the two deworming charities we recommend) follow the guidelines of good developmental charity, and work with the local governments, eventually handing over the program to the government. They have a very good track record of doing this! This means that by going into a country, SCI (or DtW) bring forward the time at which a deworming program is implemented by the government by a number of years. For example: suppose that SCI goes into a country 20 years before a deworming program would otherwise have been started, and run the program for two years before handing it over to the country's government. The 2 years of work from SCI could result in 20 years of more deworming than would have otherwise occurred, meaning that their work was 10 times as cost-effective as if the program had simply stopped after two years. In this hypothetical case, this 'hand-over multiplier' would be 10, which is probably unrealistic. However we do not consider a figure of 5 to be implausible. By not taking into account this multiplier, our estimates implicitly set it to 1, which is unrealistically low. Since this number makes such a difference to the estimate, it needs to be taken into account. A goal of future research should be to find a realistic range.
Also, there is some dispute over the "disability-weight" given to the conditions caused by NTDs. The official figures for Schistosomiasis give a weight of 0.6% lasting for a year. Charles King has argued that this should be 5 to 10 times higher, lasting for the rest of the person's life. Using these figures would reduce the cost-per-life-saved to under $1000. Not being an expert in the subject, I don't feel qualified to judge who is right on this matter, but hopefully the disagreement will spur more discussion to clarify the issue. At this point we can probably expect that the true figure lies somewhere in between, which would make the real cost per life saved somewhere between $1000 and $5000.
Thus, to summarise: the DCP2 estimate - one of the sources we initially used to give a cost-effectiveness estimate - of treating NTDs was incorrectly calculated, and - due to the Cochrane review - we are less certain of our belief that deworming significantly improves school attendance and performance. GiveWell have corrected the DCP2 estimate, and enhanced the methodology to get an estimate of approximately $5000 per life saved (about $170 per DALY).
However, we believe that this estimate does not fully factor in that our recommended charities hand over the running of the program to the local government, and is calculated using the most conservative disability weightings. Moreover, the educational benefits remain unclear, and could be significant. These factors create a significant 'upside potential' as we further look into NTDs. It is possible that SCI and DtW are as cost-effective, or more cost effective, than providing insecticide-treated bednets for protection against malaria, though we do not yet have a good numerical estimate for the actual value. Future work, from us and other, needs to: