Thursday, March 17, 2016

Where does scientific society money go, and the future of open-access


There's been a bit of a storm on Twitter recently about PLoS, open access, and scientific societies. One underlying issue seems to be what the "return on investment" is from publishing in different venues.


I've held leadership positions in 3 major societies over the past 5 years (Society for the Study of Evolution, Genetics Society of America, and American Genetic Association), so I re-examined the reports I got from those societies to compare/ contrast. I won't present exact numbers here, since that's inappropriate and since I'm looking at a single year's report (2012 or 2014) for each society rather than a multi-year running average.

All three societies were (at least as of those reports) financially quite healthy, each with reserves well over $1 million that produce some investment income. All three, at least in those years, were saving more than spending. Much of society yearly income in all cases was from their flagship journal (and specifically institutional subscriptions of that journal). I should add-- while colleagues are often critical of price gouging by publishers like Wiley-Blackwell, well over half of the proceeds from publication may go to the scientific societies, so we scientists & scientific societies are the gougers. (The publishers are just our hit-men.) The annual conferences produced a very small net profit for two societies (SSE, and GSA, though the latter sometimes loses money in some specific organismal meetings in specific years) but a net loss for the third (AGA, which keeps an outlay for the conference in its yearly budget).

What do they spend their journal income on? Clearly some of it is used to keep publishing cheap for scientists. All societies have some other shared expenses like stipends for some editorial (or occasionally officer) positions, travel for the board/ council members, etc. In the year I examined, SSE spent a lot on student awards (over $100K on students alone) as well as some on other awards, various educational initiatives and annual meeting symposia. More recently, SSE has taken on large expenditures for funding outreach efforts and a workshop for trainees on preparing for diverse careers. GSA maintains multiple staff (some shared with other societies) so most of the expenses went there, but the staff work on their journals, spearhead education & outreach efforts, communications & policy activities, etc. GSA also gave out various awards both to distinguished established researchers as well as trainees, as well as funds for the diverse expenses mentioned above (e.g., board travel). AGA spends a ton on "special event awards" which fund workshops and symposia at other meetings, and also spends on its own annual conference. AGA also more recently started sizable "Ecological, Evolutionary and Conservation Genomics (EECG) Awards" for graduate students and postdocs.

The flagship journal from each of these three societies (Evolution, Genetics, and Journal of Heredity) is not open-access, but an open-access option can be purchased for a particular submission at the time of acceptance. Uptake on that is low-- I think under 20%. Each journal makes or will make their old issues freely available online (Evolution does not currently but will do this for issues >2 years old beginning in 2017). Publication costs are moderate (Genetics) or even waived for short papers in these journals (Evolution for members, J Heredity for everyone).

Here's what these journals and societies do now-- they get large sums of money from institutional journal subscriptions (which cost $500-$1200 annually) and use those funds both to make publication cheap to the scientific community and to provide several services (including grants) for the scientific community. Libraries pay in order to grant access to their communities. What this means, by definition, is that access is not "free", and if a reader does not use one of these libraries (e.g., they do not work for a big university), then they have to wait perhaps 2 years to see this published science or have to pay a one-time use fee.

The open-access movement argues this setup is unacceptable. Federal grant dollars often support the research (let's focus on federally funded USA-based scientists and USA-based readers for simplicity here), so as a citizen who "paid" for the research with their tax dollars does not get instant free access to the results of that research, but has to either wait 6 months to 2 years (depending on journal) or pay again to see sooner. Even that is not exactly true always, given the growth of preprint servers like arXiv or bioRxiv-- the core science may be available immediately and free, but the edits from the peer review process and copy-editing are not.

Could these society journals go full open-access? Sure, but it's non-trivial since the funding currently garnered from libraries would be cut. With such a great drop in income, there would be two choices: raise prices elsewhere (by hefty open-access fees for submissions and/ or making conferences more expensive so they are profit-generating) and/ or cut services (i.e., reducing student grants, education/ outreach activities, and/ or funding workshops for trainees). Some institutional libraries may still also support open-access publishing (e.g., my institution provides such funds), but these funds are often limited and run out (e.g., as of when I'm writing this in March 2016, my institution's open-access funds for fiscal year 2015-2016 are already depleted).

How does PLoS manage? PLoS doesn't provide all the same services to the community that a scientific society does. I say this not as a criticism of PLoS-- they function solely for publication (and do it extremely well) whereas societies now function for facilitating publication while also providing diverse other services. I'm thrilled PLoS is running a profit now-- they are providing a valuable vehicle for publication and have been leaders in the move toward openness in science. They are also a business, so they need to run a profit-- again, our scientific societies are all running profits, too.

Basically, we have set up a situation where any major move to open-access by a profitable society journal bears risk of a major reduction in other services to which the community has grown accustomed. Is this risk worth it? Here, I will go out on a limb and give a personal opinion rather than just an "option." I would argue yes with a caveat. I feel that societies have no greater obligation than to maximize the dissemination of science, even if that means accelerating access by 1-2 years. Giving out student grants or funding workshops & outreach events-- those are all very valuable, but their impact is typically limited to direct participants. The "potential" impact of accelerating access to all the literature (both inside and outside the scientific community) seems greater to me. Making science available immediately is the democratization of knowledge. Further, moving to open-access removes the community's reliance on libraries, which has always been a source of financial uncertainty-- it provides the opportunity to go to what I think may be a more stable model, even if less lucrative. The caveat I would add is that there would have to be support for publication costs for scientists with limited grants... I would hate for science to not be able to be published in X journal because the research team did not have (sufficient) funding. I realize the consequence may be some reduction of other society services (and certainly I would not advocate those going to zero), but I think the positives outweigh the negatives.

I realize not all will agree with my opinion above, and I respect that the other side is quite valid. I also do NOT argue that a move by society journals to open-access be taken hastily-- clearly, we need to assess the potential financial consequences and be prepared to react if there will be some losses. Nonetheless, I hope this stream of consciousness is useful or at least thought-provoking. Please feel free to disagree, but please be nice.

Sunday, February 21, 2016

Dear Reviewer 2...

Dear Reviewer 2:
I thank you for the review of my manuscript, though it was rather brief. I understand you think this "isn't the strength of advance necessary to publish" in the journal to which I submitted, but why did you take 6 weeks to decide this when the advance was described in the abstract, which you saw before agreeing to review? I would have been happier with more feedback on why you think this is "obvious" since I could not find other papers saying it, at least in this context. Could you please point me to some examples? With more detail, I could have discussed this with the grad student first-author, when she sat across the table from me with eyes puffy, red, and watery and tried to think about whether academia is a good fit for her in the long-term after receiving such a snippy review. I guess you were busy.

Dear Reviewer 2:
I thank you for the review of my manuscript. I appreciate that you think my conclusion is not yet rock-solid, and there are more experiments to be done. I agree-- I said this myself in the manuscript, too. On the other hand, this already took 3 years of work by multiple people, and the inference is quite strong, far more likely than the alternatives. I did send this to a discipline-specific journal rather than a broad journal, too. Might there be some value to disseminating this idea and the support garnered so far now (as well as appropriate qualifiers), so others can learn of it and think about it? My grant is up for renewal, my collaborator is up for tenure, and the postdoc who did the work is on the job market. The next experiment that you say is needed requires another 3+ years and hundreds of thousands of dollars, and it may not even work. I guess none of that matters more than only 100% rock-solid interpretations getting published.

Dear Reviewer 2:
I thank you for the review of my grant proposal. I see you have lots of concerns. You say you think my approach won't work, but people use it all the time for studies like this-- I cite many such papers in the Methods. I thought I listed the potential outcomes and interpretations, too, but you didn't comment on that table-- did you see it? You also note that it's unclear how well results in my system will extend to other systems. Is that not true for any experimental study in any system? Isn't that the value of having many studies on any phenomenon, and then doing a meta-analysis later? I wish you had elaborated on these facets before submitting your review--  I could then relay them to the people I'm laying off so they know why they're losing their jobs.

Dear Reviewer 2:
I thank you for the review of my grant proposal. Your comment that the advances from it would be "incremental" stung a little. No one has tested this hypothesis before, ever. What hurt even more was that you described why it's incremental in such vague terms-- did you actually read what I proposed, or did you write your review based on the Aims page? Those things you said are "well known" are indeed well known, but that's not what I proposed to examine-- I was digging much deeper assuming those to be true. You also criticized my proposed data analyses as "simplistic" but did not elaborate what you mean-- are they failing to test the hypothesis merely because they are simple? I worked on this proposal for months, even missing out on a vacation with my wife & kids, but I confess, I feel like you flipped through the proposal in minutes before writing this review. I guess I'll miss the next vacation, too, as I try prepare a new proposal and divine what went wrong with the last one.

Dear Reviewer 2:
I know you feel you are doing your duty to improve science. I know reviewing is a thankless task and takes away from more visible progress, like doing your own science. I know other reviewer 2's have stabbed you from behind their cloaks of anonymity, and perhaps you even think one of them was me (and you may be right). I know you think your science is better than how it's evaluated and better than some science that gets published/ funded. Perhaps you think that your lab's science is better than mine. Maybe. But please know this. Some of my results may seem "obvious" to you, and some may still be incomplete/ in-progress. I will try some risky projects, and some of my results may not extrapolate. I'm merely human, just like you. And I have been reviewer 2. But I'm going to try to remember all of these experiences so as to try to not be "just like you" in the review process. In that regard, I thank you for reminding me of the human face behind scientific products.


(Note: These are all based on experiences over many years, not very recent events.)