[GOAL] Re: Fwd: The Open Access Interviews: Paul Royster
On Wed, Sep 17, 2014 at 10:53 AM, Jean-Claude Guédon jean.claude.gue...@umontreal.ca wrote: Most interesting dialogue. I will focus on two points: 1. *Using the Web of Science collection as a reference*: this generates all kinds of problems, particularly for disciplines that are not dominated and skewed by the impact factor folly. This is true, for example, of most of the social sciences and the humanities, especially when these publications are not in English. The purpose of using WoS (or SCOPUS, or any other standardized index) as a* baseline* for assessing OA repository success is to be able to estimate (and compare) *what percentage of an institution's total annual refereed journal article output has been self-archived. * Raw total or annual deposit counts tell us neither (1) whether the deposits are refereed journal articles nor (2) when the articles were published, nor (most important of all) (3) what proportion of total annual refereed journal article output is deposited. Institutions do not know even know their total annual refereed journal article output. (One of the (many) reasons for mandating self-archiving is in order to get that information.) The WoS (or SCOPUS, or other) standardized database provides the denominator against which the deposits of those articles provide the numerator. Once that ratio is known (for WoS articles, for example), it provides an estimate of the proportion of total institutional article output deposited. Anyone can then correct the ratio for their institution and discipline, if they wish, by simply taking a (large enough) sample of total institutional journal article output for a recent year and seeing what percentage of it is in WoS! (This would obviously have to be done discipline by discipline; and indeed the institutional totals should also be broken down and analyzed by discipline.) So if D/W, the WoS-deposit/total-WoS ratio = R, and w/s, the WoS-indexed-portion/total-output-sample = c, then c can be used to upgrade W to the estimate of total institutional article output, and the WoS deposit ratio R can be compared to the deposit ratio for the non-WoS sample (*which must not, of course, be derived from the repository, but some other way!*) to get a non-WoS ratio of Rc. My own prediction is that R and Rc will be quite similar, but if not, c can also be used to correct R to better reflect both WoS and non-WoS output and their relative sizes. But R is still by far the easiest and fastest way to get an estimate of institutional deposit percentages. (As far as I can see, none of this has much to do with impact factor folly. For non-English-language institutions, however, the non-WoS correction may be more substantial.) Stevan has also and long argued about limiting oneself to journal articles. I have my own difficulties with this limitation because book chapters and monographs are so important in the disciplines that I tend to work in. Also, I regularly write in French as well as English, while reading articles in a variety of languages. Most of the articles that are not in English are not in the Web of Science. A better way to proceed would be to check if the journals not in the WoS, and corresponding to deposited articles, are peer-reviewed. The same could be done with book chapters. Incidentally, if I limited myself to WoS publications for annual performance review, I would look rather bad. I suspect I am not the only one in such a situation, while leading a fairly honourable career in academe. Authors are welcome to deposit as much as they like: articles, chapters, books, data, software. But OA's primary target (and also its primary obstacle) is journal articles. Ditto for OA mandates. All disciplines, including the social sciences and humanities, in all languages, write journal articles. This discussion is about the means of measuring the success of an OA self-archiving mandate. It applies to all journal articles (and refereed conference articles) in all disciplines. There are problems with mandating book deposit, or even book chapter deposit, so that is being left for later. Nothing is being said about performance review except that the way to submit journal articles should be stipulated to be repository deposit. 2. *The issue of rules and regulations.* It is absolutely true that a procedure such as the one adopted at the Université de Liège and which Stevan aptly summarizes as (with a couple of minor modifications): *henceforth the way to submit refereed* *journal article** publications for annual performance review is to deposit them in the [appropriate] IR *. Liège does not mandate the deposit of books. However, obtaining this change of behaviour from an administration is no small task. At the local, institutional, level, it corresponds to a politically charged effort that requires having a number of committed OA advocates working hard to push the idea. Stevan should know this from his own
[GOAL] Re: Fwd: The Open Access Interviews: Paul Royster
A reasonably quick response as I do not want to go into discursive tsunami mode... 1. Stevan admits that his evaluation of compliance is an approximation, easy to get, but not easy to correct. This approximation varies greatly from one institution to another, one circumstance to another. For example, he admits that language plays a role; he should further admit that the greater or smaller proportion of SSH researchers in the research communities of various institutions will also play a role. in short, comparing two institutions by simply using WoS approximations appears rash and unacceptable to me, rather than simply quick and dirty (which I would accept as a first approximation). The impact factor folly was mentioned because, by basing his approximation on the WoS, Stevan reinforces the centrality of a partial and questionable tool that is, at best, a research tool, not a management tool, and which stands behind all the research assessment procedures presently used in universities, laboratories, etc. 2. Stevan and I have long differed about OA's central target. He limits himself to journal articles, as a first step; I do not. I do not because, in the humanities and social sciences, limiting oneself to journal articles would be limiting oneself to the less essential part of the archive we work with, unlike natural scientists. Imagine a universe where a research metric would have been initially designed around SSH disciplines and then extended as is to STM. In such a parallel universe, books would be the currency of choice, and articles would look like secondary, minor, productions, best left for later assessments. Then, one prominent OA advocate named Stenan Harvard might argue that the only way to proceed forward is to focus only on books, that this is OA's sole objective, and that articles and the rest will be treated later... Imagine the reaction of science researchers... 3. Liège does not mandate anything, so far as I know; it only looks into the local repository (Orbi) to see what is in it, and it does so to assess performance or respond to requests for promotions or grant submissions. If books and book chapters are more difficult to treat than articles, then place them in a dark archive with a button. This was the clever solution invented by Stevan and I agree with it. 4. To obtain mandates, you need either faculty to vote a mandate on itself (but few universities have done so), or you need administrators to impose a mandate, but that is often viewed negatively by many of our colleagues. Meanwhile, they are strongly incited to publish in prestigious journals where prestige is measured by impact factors. From an average researcher's perspective, one article in Nature, fully locked behind pay-walls, is what is really valuable. Adding open access may be the cherry on the sundae, but it is not the sundae. The result? OA, as of now, is not perceived to be directly significant for successfully managing a career. On the other hand, the OA citation advantage has been fully recognized and accepted by publishers. That is in part why they are finally embracing OA: with high processing charges and the increased citation potential of OA, they can increase revenues even more and satisfy their stakeholders. This is especially true if funders, universities, libraries, etc., are willing to pay for the APC's. This is the trap the UK fell into. 5. SSH authors are less interested in depositing articles than STM researchers because, for SSH researchers, articles have far less importance than books (see above), and, arguably, book chapters. 6. I am not citing rationales for the status quo, and Stevan knows this well. This must be the first time that I have ever been associated with the status quo... Could it be that criticizing Stevan on one point could be seen by him as fighting for the status? But that would be true only if Stevan were right beyond the slightest doubt. Hmm! I personally think he is right on some points and not so right on others. Also, I am simply trying to think about reasons why OA has been so hard to achieve so far, and, in doing so, I have come to two conclusions: too narrow an objective and too rigid an approach can both be counter-productive. This said, trying to have a method to compare deposit rates in various institutional and mandate circumstances would be very useful. I support Stevan's general objective in this regard; I simply object to the validity of the method he suggests. Alas, I have little to suggest beyond my critique. I also suggest that a better understanding of the sociology of research (not the sociology of knowledge) is crucial to move forward. Finally, I expect that if I saw Stevan self-archive his abundant scientific production, I would be awed by the lightning speed of his keystrokes. But are they everybody's keystrokes? Jean-Claude Guédon -- Jean-Claude Guédon Professeur titulaire Littérature comparée Université de Montréal Le jeudi 18
