The GiveWell Blog

I had a typical reaction to the disaster in Myanmar: wanting to do something. I have spent very little time looking into the area of disaster relief, so after a bit of Googling and discussion with Elie, I gave to Population Services International for two reasons:

• PSI was the winner of our “saving lives” cause for 2007; we are extremely impressed with the organization as a whole, particularly its commitment to thorough self-monitoring. We don’t know much about their relief operations, but I would bet on PSI over any other international relief organization I know of just in terms of the extent to which it “runs a tight ship” with solid monitoring and oversight that allows accountability from the field to the top.
• PSI has a major and long-established presence in Myanmar; I believe (based mostly on this article) that having a pre-existing presence is important, particularly in a situation like this where the idiosyncracies of the area and particularly government seem important. I’m most comfortable with an organization that is used to getting work done in this political and cultural environment.

This is an informal, personal recommendation; it is backed not by an in-depth research project, but by the quick heuristics above.

This also got me thinking, though, about the more general cause of “disaster relief.” We looked into this cause back in 2006 (when we were still a part-time group of volunteers) and found very little. We aren’t aware of any organizations that are exclusively committed to disaster relief; rather, it seems to us that most relief efforts come from large humanitarian organizations, such as PSI, the Red Cross, World Vision, CARE, Direct Relief International, etc. that spend most of their time and money on direct, day-to-day (not disaster-related) aid. This makes sense, since it means emergency aid efforts can be aided by already-on-the-ground presences.

However, it isn’t necessarily the case that the best “day-to-day” relief organization is the best disaster relief organization. The former may be best accomplished through meticulously planned long-term projects that rely on proven techniques to get the maximal dollar-for-dollar impact; by contrast, I would guess that a disaster presents problems that are unusually simple to solve (people who need basic supplies, but who don’t necessarily suffer from a host of interrelated physical, economic, and cultural obstacles), and that speed and efficiency are more important. I’d be very interested in a compiled summary of disaster relief efforts over the last 10 or so years - which organizations were first, and most instrumental, in each relief effort. It seems feasible that such a summary could be created by polling affected governments and citizens, but I’ve never seen one.

I also wonder whether there are cost-effective “disaster preparedness” measures that can aid particularly vulnerable areas in advance. I was shocked at the death toll from this particular disaster, and I wonder whether a similar storm in the U.S. could have been nearly as devastating. It’s possible that disaster preparedness comes mostly from widespread economic prosperity, and that nonprofits are ill-equipped to bring about the kinds of drastic changes that would be needed to improve preparedness (and/or that the areas least equipped for disasters also have other, more important problems). But it also seems possible to me that constructing some extra shelters - or equipping communication infrastructure to provide effective early warnings - could save lives far more effectively than focusing only on after-the-fact interventions.

Looking into these questions, as with just about any area of philanthropy I can think of, would take significant time and resources. I’m not sure whether we’ll get to do it anytime in the near future. But it seems likely to me that the costs of such investigation would be more than justified. When disaster strikes, a lot of people reach straight for their wallets, and give without having time to think about their different options. But the thinking could be done, centrally, in advance - imagine what a difference that would make.

We’ve wondered why scholarship programs don’t have a stronger impact on academic achievement, and have guessed that it’s because disadvantaged children are so far behind by age 5 that they need special schools, with a special approach, if they’re to have any hope of catching up.

The quote below, from an article in the Washington Monthly (h/t Kevin Carey), offers another possibility: the private schools students with vouchers attend may be little better than the public schools they leave. This is a report on the Milwaukee voucher project, not the New York programs we’ve focused on, but it makes me wonder if New York private schools could be as troubled.

In 2005, a team of reporters from the Milwaukee Journal Sentinel visited all but a handful of the private choice schools, and found that “the voucher schools feel, and look, surprisingly like schools in the Milwaukee Public Schools district. Both … are struggling in the same battle to educate low-income, minority students.” The Journal Sentinel also reported that the absence of oversight from the much-derided government bureaucracy had led to a significant waste of public funds, and even outright fraud. At least ten of the 125 private schools in the voucher program “appeared to lack the ability, resources, knowledge, or will to offer children even a mediocre education.” Most of those schools were led by individuals who had negligible experience and had no resources other than state payments.

In response to Elie’s recent series of posts on malnutrition, John J. comments:

“What specific nutrients are they missing…etc etc etc?”

Here’s a question for you: What possible use is this question to Givewell? Do you really need to get into such miniscule depth with regard to poor people who can’t afford enough food and who are malnourished as a result?

One might accuse you of wasting the time of non profit workers with such picky detail. Serously, I’m not just being cranky here–what was the point of these questions? It seems to me that common sense is enough here: people don’t get enough of the rights kinds of foods to eat, and helping them get enough is…helpful. Really, what was your point? And, additionally, what was the point of this post at all?

First of all, we don’t believe that food aid is necessarily helpful: we’ve seen plausible arguments that it can do more damage (by undermining local farmers’ business) than good. (See this critique from Philanthropy Action, co-maintained by Board member Tim Ogden, as well as this story on CARE’s decision to withdraw from the US government’s food aid program.) Broad enough outcomes data could mitigate this concern, as could clear information on the local food market in the region in question; without it, we’d still bet that food aid is a good thing on average, but could easily be wrong.

But the reason we ask such specific questions isn’t primarily to determine whether aid helps; it’s to find the aid that will help as much as possible (in ways that fit our philosophical goals). Like any donor, we choose between literally thousands of charities; a core idea of GiveWell is that under these circumstances, it doesn’t make sense to settle for “some” good accomplished.

Given the variety of different approaches to malnutrition, we expect that different charities vary wildly in both:

• What kind of life change they’re bringing about. For example, Vitamin A deficiency may significantly increase the likelihood of death before age 5, while deficiencies in Iron and Iodine may lead to Anemia and reduced I.Q. These are fundamentally different benefits that can’t be reduced to the same terms - and that different donors will value differently. In order to understand our options, we need the details of what sorts of malnutrition are being addressed, and where.
• How many lives they’re changing (i.e., cost-effectiveness). We find it possible that some charities are simply carrying better-conceived and -executed programs than others – that means more people helped, for the same funds. And even if different malnutrition programs turned out to be roughly comparable to each other, we’d still want to know how they compare to all the other health interventions out there, from hospitals and health centers to condoms and bednets.

These issues don’t matter very much if the only line you draw is between “donation was squandered” and “donation helped people.” But if you want to help people as much as you can, the lack of public answers to our questions is a real problem.

Below is the email I received from Kelli M. Donley, Program Director at Esperanca, in response to the questions I asked in Part I and Part II of this series:

Dear Elie,

Hi there! I am going to try to answer your questions to the best of my
knowledge. I hope this helps:

These questions apply to your malnutrition programs in Mozambique and
Nicaragua (except where noted):

- Is the problem “receiving enough food” or receiving the right types of
food (Mozambique)?

Both. There is not enough food in Mozambique at the moment because of
poverty, the way the population has changed with the civil war, etc. Also,
the food they are able to grow is limited. Where we work, it is
predominantly swap. So, they grow corn, rice and millet, but have little
variety to the fruit and vegetables they are consuming.

In Nicaragua, poverty and environmental concerns have limited the diet and
access to a balanced diet.
- What does Esperanca teach community members? Which foods to plant? How to
cultivate and plant them?

In Mozambique we focus solely on three areas: cholera, malaria and HIV
prevention and treatment. Our partnering agency in this country - Care for
Life - has a small farm where they teach small animal husbandry, improved
seed cultivation, improved environmental techniques to prevent erosion, etc.

In Nicaragua, we have seed banks that promote long-term growing of healthier
crops. We also focus on environmental concerns, as in how to prevent soil
erosion. I like to think our midwifery training program also helps minimize
malnutrition by working with pregnant women to teach them to breastfeed and
why it is important.

- How often do they hold classes and how long each does each last?

This varies by country and by community. These classes are run by the
nonprofits we partner with in the country.
- How well do community members retain and implement what they’re taught?

They regularly attend courses and are putting this knowledge to use
immediately in their communities. One would hope they are retaining a lot of
it.
- What nutrients do the foods contain and how does this match up to the
nutrients community members likely lack?

I do not have this information to share with you. I can tell you that meat
and dairy are extreme luxuries in both areas. Fresh fruit and vegetables are
often limited in scope and supply.
- How effectively does this program reduce malnutrition of specific
nutrients?
- What effect does this have on the outcomes (e.g., mortality rate, general
health, I.Q.) of community members?
- How many people do they help?
- How much does this program cost?

Our programs are not broken down into these sorts of statistics for
analysis. We have general overhead costs for each project. We reach 11,000
people in Mozambique and 30,000 in Nicaragua each year. One would hope we
are improving their quality of life and their life-spans, although we do not
employ any sort of researcher to determine this. It is all through
word-of-mouth and monthly reporting from our field officers that we see
results.

I also have some questions about programs to fight malnutrition in general:

Again, I don’t have the information to answer these specific malnutrition
questions. I hope what I’ve provided will be of some use!

- What specific nutrients are people missing that affect them? Without
knowing this, there’s no way to decide (regardless of the type of
intervention listed above) which nutrients are the most important. Is it
vitamin A? Iron? Iodine?
- What happens because of each type of deficiency (e.g., anemia because of
lack of iron, blindness because of lack of vitamin A, death)? How likely are
each of these outcomes given a particular deficiency? Understanding this
will help me decide which sort of program is most appealing to me - not a
fully scientific/objective decision, since my view of the “good life”
affects which disorders I most want to address.
- In what region do the people most affected live? What age are they? Any
special circumstances (i.e., pregnant women)? Before focusing on helping a
group of people, I think it’s important to know what other obstacles they
face. I want to help people for which malnutrition is the main (or a
significant) obstacle to living a full, happy life. If the people face many
more obstacles (e.g., other diseases, war, etc.) helping them here may not
do as much good.

In Part I, I listed some questions about malnutrition that I think you’d have to answer to have any sense of what you’re accomplishing with your donation, and how it compares to your other options. I then went on a search - similar to what we did part-time before we started GiveWell - to see what sorts of answers, and organizations, I could easily find.

Googling “Charity Malnutrition” doesn’t turn up anything that looks like an obvious first choice, and results from a Charity Navigator search for “malnutrition” provide nine organizations, five of which serve people in the United States.

We’ve already looked closely at Helen Keller International (click for more info), and the mission for the Children’s Shelter of Cebu doesn’t sound like what I’m looking for. This leaves Curamericas Global and Esperanca, two organizations that, based on their mission statements available on Charity Navigator have extremely broad mandates to improve health in the developing world. I chose to look into Esperanca.

Esperanca is a relatively small organization, with expenses, according to its 2006 IRS Form 990 (free registration required at Guidestar) in 2006 of $1.8 million.

Esparanca’s website provides information on their ongoing projects in Bolivia, Mozambique, and Nicaragua. The programs in Mozambique and Nicaragua explicitly address malnutrition:

• Mozambique: “In conjunction with Care for life, our public health programs in Mozambique include:Teaching community members how to make sure their families are receiving enough food”
• Nicaragua: “One of our foremost public health concerns in Nicragua is nutrition. We provide the necessary supplies and training to families and community groups to start neighborhood gardens. These gardens grow quickly and supplement traditional diets with much needed fruits and vegetables. Malnutrition accounts for 54% of all deaths of children under age five, (WHO).”

Esparanca’s Annual Report doesn’t add any information about the programs in these two countries.

The only program related information in Espranca’s 2006 IRS Form 990 is in response to the IRS requirement (in Part III) that “all organizations must describe their exempt purpose achievements in a clear and concise manner.” Esparanca responds, “Provide medical services and public health programs in the poorest areas of the world throughout North America, South America, and Africa, serving 100,000 people.” (Box a)

Given that information, I’m left with a set of unanswered questions. I emailed these questions to Esperanca (they responded yesterday, though we aren’t going to post their response until/unless we get their permission). The questions that follow apply to both programs, except where noted:

• Is the problem “receiving enough food” or receiving the right types of food (Mozambique)?
• What does Esperanca teach community members? Which foods to plant? How to cultivate and plant them?
• How often do they hold classes and how long each does each last?
• How well do community members retain and implement what they’re taught?
• What nutrients do the foods contain and how does this match up to the nutrients community members likely lack?
• How effectively does this program reduce malnutrition of specific nutrients?
• What effect does this have on the outcomes (e.g., mortality rate, general health, I.Q.) of community members?
• How many people do they help?
• How much does this program cost?

Note that these questions aren’t just about the scientific/factual details of malnutrition; they’re about the organization’s competence, effectiveness, and even philosophical priorities. It seems clear that two equally legitimate and well-meaning organizations could produce wildly different answers to them, and thus could be diverging wildly in terms of how what they do fits with my goal of helping people. That’s why I think it’s important to answer them. And I can’t find answers anywhere on the website, Annual Report, or tax forms.

I also checked out CARE’s nutrition page, but it does not help with the questions above. CARE’s website has a searchable database of ongoing projects, but the example project I looked at only provided information on the country in which the project takes place, the goal, and the broad method for implementation (in this case, flour fortification). That’s all somewhat helpful, but in order to donate to this CARE project, I’d have to do a lot of independent research along the lines of the questions I listed above.

My intent here isn’t to criticize the charities above - the information they’re giving may have more to do with what’s being asked for than with what they have. But regardless, the fact remains - if I want to help people with malnutrition, at this point I seem to be stuck. Without more research, my only option is blind faith.

In this post, I’m going to take a fresh look at malnutrition, a cause that interests me a lot. I have two goals in mind here:

• Think through the issue, and what I’d need to know to decide where to donate. This is particularly relevant since there’s a good chance we’ll be studying malnutrition in the coming year.
• Give you a sense of what an individual donor would have to do to make an informed giving decision (and how quickly finding information becomes tricky). This process is roughly what we went through almost two years ago, ultimately resulting in GiveWell’s creation.

The problem

Malnutrition is a major debilitating factor in the developing world:

Sean Stannard-Stockton wants to see more research focused on particular nonprofits, rather than on “techniques” for helping people; his reasoning is that this would be more useful to donors.

I don’t believe it’s possible to evaluate a nonprofit as an organization, completely in isolation from what it does and whether it works. Especially if I’m trying to make a case to individual donors who don’t know me or the people running the nonprofit. (I’ve argued this more fully in the past).

Phil Steinmeyer is more interested in techniques than in nonprofits; his reasoning is that differences in the effectiveness of different techniques are large enough to overwhelm organizational differences. (One example of this that I’d give is the question of fighting diarrhea by building wells/latrines or focusing on promotion of oral rehydration therapy; there is little obvious synergy between the two, and little reason to believe that they’d be similar in terms of effectiveness.)

I believe there is some value in evaluating “techniques” in the abstract, but doing so is not sufficient if you’re trying to figure out where to donate. The devil is in the details: it’s essential to know whether a nonprofit is carrying out a “technique” in a manner and context that match up with the “technique” you’ve read about. I don’t know of any “techniques” that are so simple, and so clearly effective, that I would bet on a charity simply because of formal adherence to such “techniques,” regardless of where, when, and with whom (and how faithfully) it’s adhering to them.

That’s why it’s crucial that we look at specific charities, judging them on what they do and what the evidence is that it works. It’s not the only analysis we do (we also look at independent research), and it has been the most intensive and expensive part of our process, but we see it as necessary for anyone trying to produce truly valuable and actionable information for individual donors.

Sean asks (via email):

What’s your view on whether funders should do research on techniques and then fund organizations that use those techniques or do research on organizations and let them decide on techniques? I was intrigued with your education research post, but was wondering if it might make more sense to find smart dynamic nonprofits who will figure out the best techniques to use and change strategy as more information becomes available.

My literal response is that it depends on the funder’s priorities and techniques - I don’t think there is much to be gained by debating the approach “funders” should take in the abstract. But I want to share how we deal with this question, as naive funders (i.e., not experts in the issues) aiming to serve more naive funders (i.e., individual donors), because we do have a specific philosophy on it and we’d appreciate feedback.

My ideal is to fund at the highest level I can have confidence in, i.e., delegate as many decisions as possible to to someone who I feel confident will make those decisions well.

So, my ideal would be to donate not to a charity, but to another funder. If a major foundation, such as the Gates Foundation, could convince me that they consistently make decisions using (a) a strong process, (b) good reasoning, and (c) subjective/philosophical values that are close to mine, I would give to them and let them do the rest (and get rid of our own, now redundant overhead). This was one of the first things we tried when GiveWell was still a part-time volunteer club. What stopped us was that we couldn’t find a single foundation that publicizes substantive information about how it makes its decisions, why it chooses to do A instead of B, and what evidence there is regarding its past and likely future impact. We couldn’t be confident in the institutions without such information; we couldn’t think of a way to get them to share information, since such institutions generally don’t have incentives that we can affect. So we moved on to trying to find great charities.

Again, the goal was ultimately to find a great organization - one that’s better at what it does than we could ever be, and can make its own compelling, evidence-based case for its effectiveness - and give with no strings attached. In some cases, we found exactly this: for example, the Nurse-Family Partnership’s outcomes evaluation is available via peer-reviewed publications, its basic model is clearly described on its website, and it provided documents to fill in gaps in our understanding. PSI was a similar case: after some independent checks on its estimates, we felt we could trust its process as a whole, even for activities we haven’t researched.

In other causes, the strongest applicants could provide some pieces of the puzzle, but not the full top-down case for why their approach was the best available. That’s where we had to start looking on our own for information about what approaches are likely to work, and pick organizations that fit with what we had found. There’s a spectrum here. KIPP gave us about 60% of what we needed to have confidence in it, and after some independent analysis, we ended up feeling that it was our best bet. By contrast, our Cause 2 (global poverty) applicants gave us so little to go on that we ended up betting on an approach, more than an organization.

Between blind faith and micromanagement is conditional confidence: trusting an organization to make decisions because of an evidence-based case that they can make them well. That’s our ideal; when it isn’t available, some degree of micromanagement (i.e., picking an organization based on its approach) seems preferable to blind faith.

About a year ago, we participated in a Giving Carnival on the topic of metrics in philanthropy, and laid out the metrics we were planning on using - along with caveats. Seeing a continuing interest in this topic (even including a panel called “Metrics Mania”), I’d like to share a bit of my progressed thinking on the matter - specifically that debating metrics in the abstract seems unlikely to go anywhere useful.

As promised, our research tried to divide charities into causes to make “apples to apples” comparisons; but as many predicted (and as we acknowledged would be true to some extent), even after this narrowing it was impossible to truly put different charities in the same terms. Any two given job training programs serve different groups of people; comparing them directly on “people placed in jobs” is a futile endeavor. We looked at health programs’ abilities to “save lives,” but not all lives are the same, and different health programs have wildly different and difficult-to-quantify impacts on non-life-threatening conditions.

This doesn’t mean that metrics are futile or useless. There’s a big difference between being able to demonstrate emotionally relevant results (such as lives saved) and having no evidence other than an unfamiliar development officer’s report of a gut feeling. And there can be enormous differences in the “cost per person” associated with different approaches, enough - when taken in context and accompanied by intuitive judgment calls - to make a big difference in my view of things. For example, it’s typical for employment assistance programs to cost in the neighborhood of $10,000-$20,000 per person served, while we’re ballparking certain developing-world aid programs as $1,000 per life saved - though the story doesn’t end there, that difference is larger than I would have guessed and larger than I can ignore.

Bottom line, both the metrics we used and the ways we used them (particularly the weight of metrics vs. intuition) ended up depending, pretty much entirely, on exactly what decision we were trying to make. We took one approach when all we knew was our general areas of focus; we modified our approach when we had more info about our options; we frankly got nothing out of the completely abstract discussions of whether metrics should be used “in general.” There are as many metrics as there are charities, and there are as many necessary debates about metrics as there are giving decisions.

I agree with those who say metrics are important, and those who say we can’t let them dictate everything (and it seems that nearly everyone who weighs in on “the metrics debate” says both of these things). But I don’t feel we can have the really important conversation at that level of abstraction. Instead, the conversation we need to have is on the specifics. In our case, and that of anyone who’s interested in NYC employment assistance or global health interventions, that means asking: did we use appropriate metrics for the charities we’ve evaluated? Are there other approaches that would have been a better use of our time and resources? Did we compare our applicants as well as we could have with the resources we had?

This is a conversation with a lot of room for important and fruitful disagreement. If you’re interested in the question of metrics, I ask that you consider engaging in it - or, if you don’t find our causes compelling, conducting and publishing your own evaluations - rather than trying to settle the question for the whole nonprofit sector at once.

There’s a big question in my mind about K-12 education. The question is which of the following hypotheses about helping disadvantaged children is the best bet:

1. Disadvantaged children are so far behind by age 5 that there’s nothing substantial to be done for them in the K-12 system.
2. Disadvantaged children are so far behind by age 5 that they need special schools, with a special approach, if they’re to have any hope of catching up.
3. Disadvantaged children generally attend such poor schools that just getting them into “average” schools (for example, parochial schools without the severe behavior and resource problems of bottom-level public schools) would be a huge help.

My view on KIPP vs. the Children’s Scholarship Fund, for example, hinges mostly on my view of #2 vs. #3. Of course, believing #1 would make me want to avoid this cause entirely in the future. We’ve been examining academic and government literature to get better informed on this question, but we’ve noticed a serious disconnect between what we most often want to know and what researchers most often study.

To answer our question, you’d study how students do when they change schools, focusing on school qualities such as class size, available funding, disciplinary records, academic records, and demographics. However, most academic and government studies of voucher/charter programs focus instead on whether a school is designated as “public,” “private,” or “charter.”

Three prominent examples:

• The New York City Voucher Experiment intended to examine the impact of increased choice (via vouchers) on student achievement; the papers on it (Kruger and Zhu 2003; Mayer et al. 2002; Peterson and Howell 2003) conduct a heated debate over who benefited, and how much (if at all), from getting their choice of school, but do not examine or discuss any of the ways in which the schools chosen differed from the ones students would have attended otherwise.
• “Test-Score Effects of School Vouchers in Dayton, Ohio, New York City, and Washington, D. C.: Evidence from Randomized Field Trials” (Howell et al. 2000), a review of several voucher experiments, also discusses the impact of vouchers without reference to school qualities.
• “Apples to Apples: An Evaluation of Charter Schools Serving General Student Populations” (Greene et al. 2003) performs similar analysis with charter schools, looking broadly at whether charter schools outperform traditional public schools without examining how, aside from structurally, the two differ.

To be sure, there are exceptions, such as recent studies of charter schools in New York including Hoxby and Murarka 2007. But in trying to examine the three hypotheses above, I’ve been struck by how often researchers pass over the question of “good schools vs. bad schools” (i.e., the 3 hypotheses outlined above) in favor of the question of “private vs. public vs. hybrid schools.”

When a debate is focused on government policy, it makes sense for it to focus on political questions, such as whether the “free market” is better than the “government.” But when you take the perspective of a donor rather than a politician, this question suddenly seems irrelevant. Some public schools are better than others; some private schools are better than others; and I, for one, would expect any huge differences to be driven more by the people, practices and resources of a school than by the structure of its funding (i.e., whether donors, taxes, parents or a mix are paying).

That’s why we’d like to see more studies targeted at donors, rather than politicians. But for it to happen, donors have demand it.

I think the following comment, from Andrea, is broadly representative of a common criticism we receive.

One thing Givewell is missing, and has been criticized for in the past, is that people are already working on these issues, and that one small organization like Givewell can’t solve this “problem” where all others have failed. For example, Holden’s point that he thinks global health is a cause ripe for funding has obviously already been discovered by none other than Bill Gates, employs many staff who research causes in the way Givewell proposes, but along the well-tested model Sean describes. Again, this points to the naive quality of the entire Givewell enterprise.

We believe that GiveWell has something unique to offer - but this something is not our analytical abilities, or our research process, or our “focus on results.” We believe that many grantmakers, including the Gates Foundation, may be conducting more thorough research and analysis than we can; we have believed this since the very beginning of our project; and to my knowledge, we haven’t implied otherwise at any point. (If you believe there are instances where we - GiveWell, not others writing about GiveWell - have implied otherwise, please point me to them.)

We don’t think we’re inventing the wheel; but we’re reinventing it, out of necessity, because no one else will share their blueprint. Back in August of 2006, when we were first putting serious effort into figuring out where to give, we started by calling foundations and asking them to share their research; as detailed in Appendix D of our business plan from March 2007, the consistent answer we got was that specific information on grantee results - beyond the highly general, selected parts that foundations choose to share - is confidential.

We believe that information about how to help people should never be secret. GiveWell’s uniqueness is not in its ability to conduct thorough research, but in its willingness to share it.

Phil Cubeta’s recent post about payday loans got me thinking about our choice to grant a microfinance organization in our Global Poverty cause.

We decided to grant Opportunity International for two reasons:

• The belief that there is likely a significant shortage of access to credit in the developing world.
• The very fact that someone repays a loan with interest likely demonstrates that the loan is used for something that is likely life-improving.

But, doesn’t the same analysis apply to payday loans? I’d bet that there’s a similar lack of credit for very small loans for borrowers with questionable credit-worthiness. And the very fact that lenders run this business likely indicates that borrowers are consistently paying back their loans, even at exorbitant interest rates (400-1000% annualized, according to the Center for Public Policy Research). The same logic that says microfinance is helping people would seem to imply that payday loans are as well.

On the other hand, it’s also possible that many borrowers are only able to repay their loans by taking out another loan - that what we’re witnessing is not a group of people getting back on their feet, but a group of people getting caught in a cycle of debt. Note that this could be numerically consistent with very high (~95%) repayment rates, the statistics commonly cited by microfinance organizations to illustrate their effectiveness in helping people - someone who borrows to pay off another loan 19 times, before finally defaulting, has a 95% repayment rate.

We’re left with two plausible yet conflicting hypotheses about the way in which the practice of making small loans at relatively high interest rates affects those in need. In one case, those in need access much needed credit (albeit at high interests rate) which allows them to weather a difficult financial period and potentially pull themselves out of poverty. In the other, those in need borrow and ultimately find themselves in a debt trap, borrowing more to repay previous loans.

We’ve generally been very frustrated with how little information we’ve been able to get on microfinance operations - who is borrowing, what they’re using the loans for, what their standard of living is, and what happens to that standard of living over time. Without this kind of information, we’re still only guessing at whether microfinance organizations and payday loan operations are helping people pull themselves out of poverty, or simply helping them get caught in cycles of debt.

I’m going out of town for the next week and will have only sporadic internet access. A few odds and ends:

• At our Board meeting on Tuesday the 4th, we officially named our grantees in Causes 3 (Nurse-Family Partnership) and Cause 4 (KIPP). We think both are excellent options for donors. Our writeups on these causes are drafted, but won’t be public for a few weeks yet, as we still need to clean them up and give all applicants a chance to review them. Minutes and audio from the meeting will also be available in a few weeks.
• As a result of the attention we received in January, both positive and negative, many questions were raised about our model, approach, and finances. Many of these questions are addressed in our existing FAQ, particularly relating to what about our project is unique and uniquely needed and why we feel our reviews have significant value-added, even as we acknowledge their limitations. However, some of the questions raised were not well answered, and we have used the feedback to help improve our content. We’ve added the following FAQs:
  • How can GiveWell simultaneously evaluate charities and raise money for its own grantmaking? Does that create a conflict of interest?
  • How often does your Board of Directors meet?
  • Do you conduct site visits to your applicants?
  • What is your relationship to other organizations with the name GiveWell?
• We have posted our updated budget to our Board records page, with substantially more annotations than before.
• Our focus since Jan. 1 has been completing our research on Causes 3 and 4, in the interest of giving out grants as quickly as possible. We will now be focusing on producing a writeup on our first year - what went well, what went poorly, and what we’ve learned - and our concrete plans for continuing and expanding from here.

A little over a year ago, we ballparked the cost to save a life from malaria at $200. Now we think it’s closer to $1000. Last August, I wrote that K-12 education is my favorite cause. A week ago, I owned up to moving it to the bottom of my list, and far preferring global health (even if it’s 4x as costly as I originally guessed). And that’s not even mentioning our December Board meeting, where I walked in with three suggestions as to how we should grant and was convinced to change my mind on two of them.

If you like a man who knows where he stands, I’m not your type. I’m a newcomer to all of the many areas we’re studying, and I have a lot to learn. You’re going to see me change my mind again and again, sometimes going in a circle to come back to what seemed all along like common sense (though with updated information and reasoning behind it). Then again, the history of most areas of human knowledge can be described the same way. That’s what you can expect when you (a) have a lot to learn; (b) are willing to learn it.

The reason I’ve been flip-flopping so violently, and will probably continue to do so, is the same reason this project is worthwhile: there is not enough information out there, nor are there the right kind of information aggregators, for a donor to become well-informed in a reasonable period of time. Currently, the only option I know of for an individual donor is to, in effect, take wild guesses about whom to help, how to help them, and which organization to work with in order to do so. Because of this, GiveWell doesn’t need to provide infallible analysis in order to provide a valuable service; instead, we’re working to make our guesses better and better informed. We’re sharing our progress in real-time because we want as many donors as possible to be able to see what we’re finding and improve on their own guesses. We don’t guarantee that our recommendations will stay the same over time; but probabilistically speaking, they will be better and better.

I’m aware that I often sound authoritative and 100% confident, even when I’m not (this is an idiosyncrasy I’ve had all my life, and I am working on it). But if you look past the tone, you’ll see a flip-flopper; and given the high-complexity, low-information work we’re doing, you shouldn’t expect any less.

For our first round of grantmaking, we chose to investigate five broad causes, two in the developing world (saving lives and fighting poverty) and three in the developed world (early childhood care; K-12 education; employment assistance).

This decision - doing five causes, instead of narrowing our scope from the outset - had serious costs. Even with all the time we’ve spent - far more than a typical donor can - we still feel that our understanding of each cause is very far from thorough. Trying to deal with all five causes at once has been logistically tough, and we even missed the deadline we set for ourselves for giving out our grants (we awarded only three grants at our December 2007 meeting, and will be awarding the remaining two within the next few weeks). In other words, we absolutely bit off more than we could chew.

Yet as we look to next year, I find myself wanting to bite off even more - to go broader, not narrower. Part of the reason is that doing five causes has allowed us to provide information to a broader set of potential donors; while we didn’t analyze these causes as deeply as I would have liked, I still feel that we have found is more than what existing donor resources offer, and far more than we could find when we were casual donors ourselves.

Doing five causes also had another benefit: while we didn’t become experts in any one cause, we learned enough about each to radically change the way we think about them - and prioritize them. Last August, I wrote that K-12 education was the cause that excited me most; yet having learned more about a donor’s options in each cause - and how much each costs, how reliable each seems, etc. - I’m now most excited about global health, and I’d rank K-12 education last out of our five causes. In a nutshell, though I most want to help Americans take advantage of all the opportunities America offers, I now believe that the developing world’s needs are so much more drastic that I’d rather help them. My new view is still open to revision, but it’s better informed than the one I came in with. And I wonder whether I’d be even more excited about something else, if we’d gotten a chance to look into charities that focus on abandoned children, or on disease research, or on the environment.

It may be more typical for a grantmaker to start its work by picking an area that it can really become an expert in. Donors are often told to start their search by volunteering, so they can see their area of interest up close. But if you care both about how you impact lives and how many lives you impact, I’d argue that your most important decision is the top-level, bird’s-eye one - whom you help, and how. I feel there’s more to be gained by learning a little about all your options than by getting it perfectly right within a predefined scope. Our approach is often frustrating in its lack of depth, but our goal isn’t to become the best at understanding any one area; it’s to give as well as possible.