Monday, March 30, 2015

Biomedical funding is broken; crowdfunding is not the fix.

Last week, I wrote Calibrating crowdfunding expectations – a post based on our experience running the protocols.io Kickstarter campaign. I shared that pre-launch, we had wildly incorrect expectations about the average backing amount. We expected $70-$100, but the real mean turned out to be $39. So we had to lean on our close relatives for help, and of the $54K that we raised, $28K came from just four relatives. In this post, I want to focus on the implications of our experience for research crowdfunding.

Crowdfunding platforms can work very well for pre-sales of popular items. Even in the case of our protocols.io, while not great for funding, it was very useful to give visibility to our effort. However, while projects on Kickstarter need marketing, science projects need funding rather than advertising. Moreover, unlike ZappyLab with thousands of existing users that we asked to back our Kickstarter campaign, most scientists do not have an easy network to appeal to for crowdfunding.

Based on our experience, I was concerned that researchers turning to crowdfunding were setting themselves up for a harsh month of crowdfunding education, with very little chance of successfully raising the research dollars that they so desperately need. I know that biomedical science funding is broken, but I don’t see how crowdfunding is fixing it.

Of course, our crowdfunding was aimed at scientists to support protocols.io (a communication platform), which is very different from the research projects posted on experiment.com (the leading site for research crowdfunding) and aimed at a general audience. Therefore, to check if our experience is consistent with that of the scientists running projects on experiment.com, I took a quick look at all biology projects on the site (84 projects funded at $3K or above; spreadsheet here). The average contribution for all of them was $94, very close to our protocols.io average of $107.

The problem is that most of the 50 projects had very few backers, and that means mostly funded by close friends and relatives, which strongly skews the average towards higher numbers. I plotted the average contribution as a function of the number of backers. Figure 1 below shows that the more backers a project has, the lower the average. For the most popular projects with more than 100 backers, the averages hover around $50 – much closer to the $39 that we saw when excluding the relatives.

Figure 1. Average contribution as a function of the number of backers of the project. Numbers are from all 84  biology projects successfully funded on experiment.com at $3,000 or above.

The projects on experiment.com, just as on Kickstarter, are all-or-nothing. That is, if a project does not reach its funding goal, it gets none of the contributions. As I wrote in my previous post, there are good reasons to structure funding campaigns this way, but it also creates an extraordinary pressure on the scientist running the campaign to somehow get it to 100%. I have spoken to a number of people who ran campaigns on experiment.com, and several indicated that they had to use their own credit cards at the end to make the campaign successful. Based on this, I hypothesized that projects raising close to 100% of their total will often be rescued by relatives as was our protocols.io or will be partially self-funded by the scientist. Consistent with this prediction, graphing the average contribution as a function of the percent of the raised funding target shows a strong bias towards inflated averages around 100%-105%.

Figure 2. Average contribution as a function of the percent of funding targetNumbers are from all 84  biology projects successfully funded on experiment.com at $3,000 or above.

I know personally Cindy Wu and Denny Luan, the co-founders of experiment.com. I like and respect both of them and know that they truly have noble goals and intentions. I am publishing this post because I think transparency and correct expectations are important.

------------------
Note 1. There are also philosophical reasons to question crowdfunding for research. The whole point of NIH grant committees reviewing and scoring the proposals, the point of HHMI funding innovators -- is to support good research and researchers. There are many problems with NIH funding and the selections, but as imperfect as these processes are, they aim to select good science. But crowdfunding cannot judge the merit of the proposal or the proposing scientist. Crowdfunding selects for scientists who are good at convincing the public to support them. Crowdfunding selects for projects that resonate with the public. As such, crowdfunding isn't well suited for supporting high quality research.

Note 2. If you are planning to launch a crowdfunding campaign, take a look at the experience of Jacquelyn Gill, which is remarkably similar to our experience.

10 comments:

  1. Shameless plug for our crowdfunding paper, which comes to some of the same conclusions:

    Byrnes JEK, Ranganathan J, Walker BLE, Faulkes Z. 2014. To crowdfund research, scientists must build an audience for their work. PLOS ONE 9(12): e110329. http://dx.doi.org/10.1371/journal.pone.0110329

    We saw even lower rates of individual donations: ~$50 / donor compared to the ~$100 mentioned here. We also saw that the effect of existing networks was very important - which is why it's, you know, the title of the paper. :)

    One of the most encouraging figures was that 44% of projects met their (modest) goals - far better than any federal funding agency, I suspect.

    While I know some folks have made big claims for the prospects for crowdfunding science, I personally have never thought of it as "the fix" for biomedical research funding. I think it's more valuable to think of crowdfunding as one part of a larger funding portfolio: http://neurodojo.blogspot.com/2013/06/crowdfunding-is-wind-power-of-science.html

    ReplyDelete
    Replies
    1. Thank you Zen. But few things to keep in mind:

      1. On experiment.com, 40% of the projects "succeed." But how many of these succeed because the scientist largely self-funds on her own credit card?

      2. The modest sums are $500-$1000. In biomedical research, how many projects must you list to get the $200K you need for the year? The NIH rates are abysmal, but I'd rather work on the 10% chance of a $2m grant than put in a month of work for $1,000 on experiment.com.

      Delete
  2. Last night I was watching the PBS show "Cancer: The Emperor of All Maladies". It's a really interesting look at how cancer research has evolved over the last century. But it has parallels to all the other fields, too. There were a couple of parts relevant to this that struck me.

    1. They noted the hugely successful March of Dimes effort to crowdsource polio vaccine research. Imagine today trying to crowdfund vaccine research, with shrieking anti-vaxxers dogging the researchers and the media. Ugh.

    2. The Jimmy Fund cancer public fundraising was a similar effort that learned from MoD. They did good stuff. But they also highlighted how even really wealthy benefactors like the Laskers (and that amazing force of Mary Lasker) knew that the real funding had to come from the government.

    In general, I really think public whims are not the best method to fund science. I'm glad they exist, but they are not enough and never will be.

    On recent crowdfunding exercises specifically, another thing I note is that they are often funded by other scientists. I see names I recognize in the lists. And I donate to a lot of them. I've kicked in for human genomics research for a woman with a debilitating condition, glowing GMO plants, banana slug genome sequencing, frozen frog genomics, raising the profile of women in science, tomato breeding, insect photography, and more. But I guess what I'm saying is we are often funding our own choir. Is that bad? Not necessarily. But it's not as broad as we might imagine--and that's not detectable in the data, I'll bet.

    ReplyDelete
  3. Needless to say, I also agree that crowdfunding is never going to replace government funding for science (nor should it). It's another tool in the funding mix.

    That said, I think it's useful to think of crowdfunding for scientists from a long-term perspective. Suppose that a new PhD student decides to make an investment in building an online audience for her research from the start of her program. 10 years later, as an assistant professor, she has 3,000 Twitter followers, an email mailing list with 500 subscribers, and a network of science journalist contacts.

    Based on Zen et al.'s estimates, it's not unreasonable to think that she could regularly raise, say $20,000 per year by leveraging that network (and without putting in a huge amount of work).

    http://fundedscience.com/im-a-scientist-whats-my-audience-worth/

    That's not a large amount of funding, but it's enough for a grad student to try out one "out-of-the-box" experiment per year, whose results might help attract funding from traditional funding agencies. It's no panacea and certainly not the emergency cash infusion that most crowdfunders are looking for at the moment, but it's one way in which crowdfunding could really help researchers out over the course of their careers.

    ReplyDelete
  4. Thank you Nick for taking the time to comment. You write, "it's not unreasonable to think that she could regularly raise, say $20,000 (and without putting in a huge amount of work)." I disagree for the following reasons:

    1. There is absolutely no indication that 3,000 Twitter followers can help to raise $20,000. Not based on our experience as a company with huge outreach on social media and thousands of users we could lean on. Not based on the experience of Jacquelyn Gill on experiment.com (linked to above) - she had 6,000 followers and barely was able to get $10K.

    2. There is no indication that it can be done "without putting in a huge amount of work." Again, see Jacquelyn's experience - it was a month of non-stop outreach, at the exclusion of all other work. In our case, it was exactly the same. Much of the ZappyLab startup experience was hard. Nothing was as hard as that month of the Kickstarter campaign. I slept 3-4 hours on average the entire month. Didn't see my kids.

    3. There is no evidence that crowdfunding campaigns can be repeated by people like Jacquelyn with increasing ease.

    4. There is no evidence that an average crowdfunding contribution is $242 as you assume in your link. But as my graphs show, as our Kickstarter shows, as Jacquelyn average shows - the average backing is closer to $40-50.

    In summary, there is no evidence today that research crowdfunding is an effective investment of time for a scientist. But there is plenty of evidence that through a month of non-stop work, harassing relatives and friends and possibly self-funding, you can get $1,000-$3,000 once. If we are upfront with this evidence, how many scientists will invest their time into crowdfunding?

    ReplyDelete
    Replies
    1. "If we are upfront with this evidence, how many scientists will invest their time into crowdfunding?"

      Depends.

      Traditional academics hired by universities have many institutional incentives to pursue large grants. Few of them are going to try crowdfunding.

      But the picture for early career scientists might be different. They are less likely to have strong institutional incentives (or impediments, depending on your point of view) that would impede them from trying crowdfunding. There's a lot of grad student projects for a few hundred or a few thousand dollars can "grease the wheels" and make a big difference to a dissertation. Lots of societies hand out student awards that size.

      "(T)here is no evidence today that research crowdfunding is an effective investment of time for a scientist."

      Not everything everyone does in science has to be efficient. There's some things you do just because. I've written grants, and I've done crowdfunding. And for me, crowdfunding was more fun. YMMV. :)

      Delete
    2. I just realized the $242 average contribution is for university-affiliated crowdfunding campaigns. Perhaps this is due to PR teams from the university engaging alumni donors. If so, and if the university can do all the work for the researcher in promoting the campaign, perhaps it's reasonable. But the independent platforms like experiment.com, there is no evidence that investing time into a campaign on them is wise.

      Delete
    3. All fair points, but I still think your take is overly pessimistic.

      1) Agree that the $242 is likely an overestimate of what independent campaigns are likely to get (an alumni base is probably more motivated to give larger amounts; universities may be selective in which projects get approved; though they definitely don't want to do all the promotion for the scientists - the whole point of crowdfunding for them is to download the fundraising to community-members). That said, I think crowdfunding for science is likely to become more and more integrated into university outreach & fundraising operations, so the Experiment averages may be an under-estimate of the long-term average donation to university-based research projects. Might also make more sense for scientists to not go independent, and try to work with their institution's development folks.

      2) The main point of my post, though, was that scientists who are thinking about crowdfunding need to think about it in the context of a broader career & outreach strategy. As you say, $1,000-$3,000 really isn't enough to justify the work on its own. Along with the broader career benefits of outreach (citations, possibility of recruiting crowdsourcing participants, attracting collaborators and other funders), the key question is what the lifetime value of a donor/supporter acquired through the campaign is. There really isn't enough evidence at the moment to estimate this, but it's probably much greater than the $50 they give today. Granted, researchers would have to have a good strategy for donor retention & stewardship in order to realize that value.

      3) Which brings me back to: It's the audience-building that matters. For crowdfunding to become genuinely useful, scientists need to become fundraising entrepreneurs over the long-term. This means not just Twitter, but also building an email list & media contacts (The PLOS paper shows that those were much more efficient channels than Twitter) and develop partnerships with other organizations that might support their goals (e.g., foundations).

      If scientists commit to doing this over, say, 10 years, then I don't think it's unreasonable that they might be able to raise a small, but not insignificant, amount on an annual basis. If they're not willing to commit to marketing their research over the long-term, then they probably shouldn't try fundraising at all. (And again, none of this is to say that crowdfunding can make up systemically or individually for $$ lost from other sources.)

      Delete
    4. Zen, regarding "fun" in crowdfunding, that also depends on your outreach network and social media following, as you well know. You have 4,000 twitter followers and an actively-read blog. How many students do too? (And getting a following on Twitter takes a VERY long time; I have 1K after two years of actively contributing there.)

      I just want truth in advertising. If everyone knows that running a campaign on experiment.com is a tremendous amount of work, that it's likely to take all of your time for a month, and if you are a professor with 6,000 Twitter followers and very creative about getting visibility for your campaign, the most you can raise is $10K after that month - if this is obvious to everyone, then I have no issues with crowdfunding. But as you can see in the graph above, few people know this.

      Delete
  5. By many measures, the biological and medical sciences are in a golden age. That fact, which we celebrate, makes it all the more difficult to acknowledge that the current system contains systemic flaws that are threatening its future.


    Rim Sanson,
    Fundau

    ReplyDelete