The GiveWell Blog

We are looking to hire a Research Analyst to help us collect and analyze data on several hundred international charities, and ultimately identify the ones that can best use donations to change lives. If you believe you are a good fit - or know someone who is - please send a resume to info@givewell.net.

About the role

We are currently conducting an in-depth examination of international aid charities. At first, the Analyst will focus on helping us to collect information from charities’ websites and annual reports about what sorts of programs they run; we will match this information with academic literature on which sorts of programs are highly cost-effective, in order to identify charities with the greatest potential to have a real impact. Over time, the Analyst may become a permanent member of our research team, with broader responsibilities.

No particular experience or skills are required. Instead, we are looking for a quick learner and independent thinker, with genuine passion for our mission and interest in our work.

The Analyst will be our third employee. The first two are located in New York City and Boston. The new employee can work from anywhere as long as s/he is accessible via phone and IM. This role is full-time, but will be terminated within a few months if the fit is not good. It is not a good fit for someone whose primary concern is job security. It is an excellent fit for someone who is genuinely passionate about our vision of a world where charities raise money not just by traditional marketing techniques, but by truly demonstrating their ability to change lives.

Philanthropy Action (co-maintained by Board member Tim Ogden) has an interview up with Abhijit Banerjee and Esther Duflo, principals of the Poverty Action Lab (one of our favorite groups). This quote (from Esther Duflo) particularly resonated with me:

There is no evaluation yet of the impact of a microfinance loan – we have the first preliminary results ever of the impact of a plain vanilla, group lending microfinance model. That’s it. It is not as if there have been mixed results before now. The studies don’t exist. And that is microfinance, where there are already a hundred economists studying it.

Microfinance fortunately did not go out of fashion before [randomized evaluation] came into fashion, so we have a chance to have a meeting of minds here. But other things did. Take fertilizer subsidies: at first they were fashionable, now they are unfashionable. In the meantime we have not learned what fertilizer subsidies do. They might be good or bad, at this point I don’t know. They’re coming back in fashion, by the way. All of this without a single piece of evidence about whether a subsidy changes the demand for fertilizer. This is just one example of what to me is the biggest mistake, which is doing the same things over and over and over again without learning from the experience, whether it is fertilizer subsidies or microfinance.

I recommend the whole thing.

This week, The Chronicle of Philanthropy wrote an article about the creation of the Alliance for Effective Social Investing. We wholeheartedly applaud Steve Butz and other members of the Alliance for their efforts, and really hope they succeed. There are too few organizations focusing on the effectiveness of charitable programs, and we’re excited to see their first results.

I’ve briefly looked at the survey they plan to send to nonprofits and here are some quick thoughts. Ultimately, the survey focuses on procedures and processes as opposed to impact and results and therefore has two problems:

1. Charities often say they track outcomes even if they don’t. This happened to us consistently last year. (You can view our Round 1 application we sent to international charities linked on this page along with all the materials we received. In particular, look at the answers to section III on the application. )
2. There are no specifics about what each organization does and what effects it has. All the questions are abstract about whether or not each organization tracks their outcomes. Donors need to know what impact they can expect from their donations not whether or not a charity has a “process in place” to track outcomes.

[As an aside, I’d really appreciate a tool that simply lists all of an organization’s programs. If there’s one type of information I’d like for all charities, it’s a simple specification of what they do and where they work. This is not currently available anywhere. (Guidestar offers very brief summaries of a charity’s programs off its 990, but nowhere can I see the specifics of each of its activities.) We’re currently working on building such a tool for international aid, with the help of some great volunteers.]

Finally, I’m concerned that this survey won’t accomplish its most important goal: distinguishing between effective and ineffective organizations. We’ve analyzed the Children’s Scholarship Fund and the Nurse-Family Partnership. Both organizations collect a large amount of data about their clients, and I believe they would each answer the Alliance’s survey identically. Nevertheless, we believe that NFP is running an effective, impactful program and we strongly recommend them; we think that CSF’s strategy is marginally effective (i.e., not making a substantial difference) at best and ineffective at worst.

Any useful evaluation tool has to distinguish between two programs like CSF and NFP. If it doesn’t, it falls short in the most important way.

All of this criticism is offered in the hope of dialogue and improvement. We’re rooting hard for our competitor, and if they want any help or information from us, they’ll get it.

To me, the most interesting part of the recent discussion of FORGE (see the last several posts on Tactical Philanthropy) is the disclosure that moving to a more effective model directly caused a loss of revenue, because it lowered volunteer involvement.

In a nutshell, FORGE runs programs for refugee communities; it shifted from having volunteers manage the programs to having the refugees themselves manage them. (More here). I’ll take FORGE at its word that the refugees were easier to manage (it’s plausible to me that they were more plugged into their communities and therefore more effective).

But apparently, the lack of work for volunteers translated directly into a loss of funding, because volunteers doubled as fundraisers. Logically, I’d think that if you were volunteering for a cause you were passionate about, and then you were released in order to make the program more effective, you would now be more excited (not to mention having more time) to raise money from your friends. But that isn’t what happened.

This story matches with anecdotes we’ve heard from many people in the nonprofit sector, claiming that volunteers are essentially useless in program terms (i.e., they cost more time to manage than the value they add). I believe that to many charities, using volunteers is a way to get people personally involved with, excited about, and personally invested in the organization so that they’ll donate and fundraise, the real value-added.

I’ve generally found that adding a new person into a work process nearly always costs a lot of time, especially up front, for training and managing. It can be worth it if (a) they’re going to put in enough hours to overcome that cost eventually; (b) the task they’re working on is extremely well-defined, meaning minimal management. As we get more systematic about our research process, we are able to use volunteers more effectively (and in fact have several working well now, with more slots open); but there have been times in the past when we’ve had far more requests for volunteer work than useful things for people to do. (When this has happened we’ve simply turned away the volunteers - our policy is to take volunteers only when we have good work for them.)

Next time you’re thinking of volunteering for a charity, ask yourself if you’re looking to do good or feel good. If the former, take a hard look at whether what you’re doing is really worth as much to the charity as a donation.

(As a side note on FORGE: I applaud FORGE’s honesty about past mistakes in this area. I agree with Sean’s claim that “in a world with limited transparency, we need to celebrate transparency on its own.” And I even think that there’s some argument to be made for promoting and supporting FORGE just for showing unusual honesty. However, I also agree with with Curtis Chang that FORGE hasn’t yet made a good case for its actual impact on people’s lives.)

In addition to our charity-specific investigations, we’re looking to review as much literature as possible on the following questions. Note that these were originally posted to our email list, before it went public.

1. What is the evidence that aid works/has worked at all? That it has caused reductions in infant mortality, economic growth, or anything else?
2. Has aid worked better in some parts of the world than others? Are there any broad patterns in where and when aid works (as opposed to what interventions)?
3. Can we expect health aid to create economic growth? Can we expect economic aid to work in areas where health is poor?
4. Why have some parts of the world emerged from poverty while others haven’t? Is there anything aid can do to make the former more likely? (#2 is about whether aid has accomplished proximate goals like improving health - #3 asks what the biggest success stories are and whether there’s any plausible case that aid *could* accelerate them.)
5. What are the risks of aid causing harm, and what evidence is there for their severity? Possibly ways that aid can cause harm include:
   • Overpopulation due to declining mortality
   • Crowding out government aid; encouraging governments to remain corrupt
   • Talent drain: turning all of Africa’s brightest into health/aid workers
   • Economic distortion: outcompeting private farmers and for-profit aid companies with subsidized prices
6. What is the current allocation of aid across the world? How much of it is going to programs that don’t work or aren’t proven? How much of it is going to programs that appear overfunded?
7. How can one determine whether an intervention is funded to capacity?

Via Joanne Jacobs: San Francisco Chronicle reports that Oklahoma and Georgia have seen no improvement on achievement test scores since implementing universal preschool programs. It also refers to a discouraging-sounding large-scale study of Tennessee’s preschool program, although it doesn’t give a specific citation (and I can’t find one online).

A couple things to keep in mind:

• All of the discouraging results cited here refer to achievement test scores. Possible impacts on mental health, later life outcomes, etc. are not discussed.
• The Tennessee finding is reported as excluding “at-risk kids.” We’ve always thought it very possible that early childhood care is most beneficial to at-risk children, and indeed that the gains for such children may account for the entire observed effects.

Our existing position on large-scale preschool programs is that no strong evidence exists for their effectiveness. The programs discussed here are unusually high-intensity programs, so the findings do call into question whether replicating the encouraging results of model programs is even theoretically possible.

Note that none of this discussion pertains directly to our current top charity in early childhood care, the Nurse-Family Partnership (our review here).

Malaria is one of sub-Saharan Africa’s biggest killers; lymphatic filariasis is one of its most debilitating. Malaria matters reports on new efforts to combat both at once, as they are both mosquito-transmitted. We’re surprised that there isn’t a longer history of such efforts.

Donors interested in microfinance shouldn’t miss the recent posts by Philanthropy Action. Note that Philanthropy Action is co-edited by Timothy Ogden, a member of our Board of Directors.

The coverage discusses papers that we don’t yet see at Innovators for Poverty Action (co-host of the conference and a favorite source of ours). To me the two most interesting posts are:

• A challenge to the conventional wisdom that microfinance should target women. Personally, I don’t find the analysis here very compelling - it’s looking at grants, not loans, and as such the characteristics of clients could be very different from those of a typical microfinance program. I’d be very interested in seeing the same analysis for loans.
• The results of the first true randomized controlled trial on a microcredit program - finding “no impact of access to credit on outcomes for education, health, or women’s empowerment.” This directly contradicts our current, weakly optimistic take, not to mention conventional wisdom. We eagerly anticipate checking this paper out once it becomes available.

As a reminder, our blog is a personal and informal forum, and does not represent the official views of GiveWell. Our About This Blog page states:

While our main site features researched and edited content, aiming to present all our information in a neutral way for you to make your own decisions, our blog is highly personal. Posts do not represent the official views of GiveWell; they represent the unfiltered opinions of project members, who may (and often do) disagree with each other.

We aim to share all of our thoughts relating to the choices faced by individual donors. The Prop 8 issue strikes me as a particularly good opportunity, to the point where I’ve made a personal donation partly on “ROI” grounds (as opposed to purely personal grounds); so I’ve posted my personal decision and reasoning. In general, we have a principle of full disclosing our values and biases, rather than denying or hiding them.

However, GiveWell the organization, as represented by the Board of Directors as well as Elie and me, does not officially endorse or oppose Proposition 8 (and the donation I referred to was a personal one).

I gave to a group fighting Proposition 8 in California because it struck me as a really good opportunity to make a difference with a donation. This is about as far from our normal focus as it gets, and I don’t have nearly the knowledge of political advocacy that I would like to have, so I’m not going to go into this much. But in a nutshell:

• The polling on this issue looks close. Unlike the Presidential election, it may be close enough that some extra advertising could make the big difference.
• I’d also guess that it’s relatively easy to translate dollars into votes for something like this. A lot of people will be turning out primarily to vote for President; a well-done ad or phone call, or just a reminder, could be enough to get them to vote No on Proposition 8.
• As such, I don’t really need to be convinced that the people running this organization are particularly brilliant or have amazing advertising abilities; as long as they’re using money to do things like run ads and make phone calls, it’s probably money well spent. (And there is enough content on the website that I don’t fear fraud.)
• I feel strongly that gay couples should be able to get legal marriage licenses. I think most political issues are more complex/two-sided than advocates recognize, but not this one. If you feel differently, this post doesn’t apply to you, and that’s fine.

I hope that sometime in the future, we’re able to learn more about the conditions under which advocacy money matters and the conditions under which it doesn’t. With what little I know, and even without an ability to remotely quantify it, this seemed like too good of a buy to pass up, so I’m passing it on.

GiveWell has maintained a mailing list since late July that we use to share our notes on research in progress. The emails tend to be more detailed and less theme-driven than blog posts, and the volume tends to be higher (averaging around an email a day, except when we’re heavily focused on non-research aspects of our project).

We are now making the contents of the list (though not the full identities of participants) public. We also invite anyone who is interested in participating to apply via the list’s page; we will approve anyone we feel is applying out of legitimate interest in our research. The list’s page is here:

GiveWell Research Mailing List on Yahoo! Groups

Note that old emails to the list (between its start in July and its opening late last week) are not currently available, due to our recent switch from Google Groups to Yahoo! Groups (the latter offers superior privacy protection for list members).

This book excerpt (via Overcoming Bias) suggests that stressing the health benefits of latrines can be less effective than stressing the pride benefits of “feel[ing] royal, because the royal family had one.”

Also mentions that the Community-Led Total Sanitation program (which we’ve discussed before) stresses “disgust” over safety concerns.

More evidence that health education can work in counterintuitive ways, implying the value of constant experimentation and evaluation rather than merely recording “class attendees.” (Similar ideas discussed previously here).

We received the following email from reader David Micley:

I want to donate $500 to an effective charity. Ive been doing research on your site and it seems like PSI is a great choice in terms of helping global health. I have not yet made my decision as to which charity I will donate to, but as I continue to research, I continue to ask myself a fundamental question. How much of my money actually makes a difference in the effectiveness of a large charity? The amount of money that a huge charity has such as PSI makes me feel as if my $500 will be but a penny in a wishing well. Will my money be more effective donating to smaller charities that are in more need of money? Or is the large charity truly the place to find the most effective charitable work, and even if I feel my impact isn’t so strong relative to the size of the charity, it will be the most efficient way to help other people in need?

Our FAQ recommends that small donors with little information give to larger charities, but doesn’t address this angle - the question of whether $500 has more impact when it’s a larger percentage of the budget.

My view is that the size of a charity is less relevant here than whether it is operating at full capacity.

If a charity is already at an “equilibrium” where its costs are about equal to revenues, and it’s serving everyone its core activities can serve, then it will get questionable value for an extra $500. This is true whether the charity is large or small. It may attempt to expand its activities, start new programs, and serve more people, but a $500 donation seems unlikely to be the key driver behind such an expansion.

If a charity has more clients than it can currently afford to serve, or more worthwhile projects on the table than it can currently fund, a $500 donation can help it serve more people (or serve them better) - regardless of how big the charity is as a whole. To use a for-profit analogy, when you give McDonald’s 99c, your contribution is an extremely tiny percentage of its overall revenue, but it still produces an extra burger. McDonald’s, Inc. had no role in endorsing or funding this analogy.

Figuring out the extent to which a charity is “at capacity,” and what the impact of additional funds will be, is something that we struggle with, and we have no easy or fully reliable way of doing it. However, it’s worth noting that a large charity may be better positioned to handle increases in revenue, and use them to expand projects that are already repeatable/scalable, than a small one. And we feel relatively confident that the large charities we recommend are very far from serving everyone they could serve.

For this reason, the “will I make a difference?” question seems to tilt slightly in favor of giving to large charities when making small donations. $500 might be a small percentage of PSI’s revenues, but if you put credence in our estimate that PSI prevents a death for every ~$1000, that donation can be a huge deal in human terms.

We constantly emphasize the huge amount of money that is given by individuals. However, the figures we usually point to refer to all giving; people often, correctly, point out to us that many individual gifts are made to support churches, alma maters, public goods, etc., rather than to help those in need.

A study by Google.org and the Center for Philanthropy at Indiana University provides some analysis on exactly this point. It uses survey data to examine how much of individuals’ giving is actually intended to help those in need. It estimates both the amount given directly to humanitarian organizations (3rd column) and the amount given “indirectly” to help those in need - for example, donations to churches that use some of their funds for humanitarian programs (4th column). The summary table is below, taken from page 29.

Note that individuals making less than $1 million per year account for ~$66 billion of giving focused on the needs of the poor, $17 billion of which is given directly to relevant organizations. These numbers are smaller than the headline number of $223 billion, but they’re still huge (and still dwarf the giving of Gates and other foundations).

(This study has been around for a while and we cited it in our original business plan, but we’ve never made this point directly on our website.)

The last blog post shares general thoughts on Money Well Spent. Specifically, though, this bit really struck me (page 12):

In our personal lives, we regularly make year-end gifts to organizations for which we have warm feelings. These gifts make us feel good, and doubtless they help good organizations. But this isn’t the way to change the world, and it certainly is not a responsible way to give away someone else’s money.

Why give away your own money in a way that would be “irresponsible” with someone else’s?

Why be smart, disciplined and strategic when giving large grants, and then drop all of these principles for your individual gifts?

Especially when individual gifts collectively dwarf foundations’ grants?