School of History and Philosophy of Science RESEARCH SEMINAR [The University of Sydney] [https://d31hzlhk6di2h5.cloudfront.net/20250909/31/8c/47/4b/1bf63ffa65bd451410b1837c_1276x850.jpg] If peer review is broken, what can fix it? Some suggestions from the repliCATS project (Collaborative Assessments for Trustworthy Science). Fiona Fidler (University of Melbourne)
Dates: Monday, 15/9/2025 Start Time: 5:00pm (Note: Seminar start time is at 5:00pm) Venue: F23.501. Michael Spence Building, Level 5, Room 501 How to register: Free, no registration required Website: https://hps-events.sydney.edu.au/<https://t.e2ma.net/click/g68j3x/8zmgipmb/4gpee2e> Abstract: In many scientific fields, post-publication surveys of the literature find that peer reviewers routinely overlook methodological flaws and statistical errors, avoid reporting suspected instances of fraud, and commonly reach a level of agreement barely exceeding what would be expected by chance. Other studies expose the extent of gender bias in peer review, and questionable editorial protocols that lack transparency. Anecdotally, editors also report increasingly difficulty in recruiting reviewers. What can be done about these well-known problems? This talk proposes an alternative model of peer review, drawing from expert elicitation, deliberation and decision-making literature and our experience running the repliCATS project. There is perhaps a limited role for AI in an overhaul of peer review, but that is not the focus of the talk. Now in its seventh year, the repliCATS project has evaluated over 4,000 published social science articles across 8 disciplines, including psychology, economics, and education, as well as hundreds of preprints on PsyArXiv. For each paper, a diverse group of experts discuss and forecast the likely replicability of the research findings and make a variety of other judgements about the credibility of the evidence presented using a structured deliberation protocol. This talk will present our approach to evaluating research, and for cases where we have the outcome of actual replication studies, data about the accuracy of our forecasts. Bio: Fiona Fidler is Professor and Head of the History & Philosophy of Science (HPS) Program at the University of Melbourne. She is broadly interested in how experts, especially scientists, make decisions and change their minds. Her past research has examined how methodological change occurs in different disciplines, including psychology, medicine and ecology. She is also interested in methods for eliciting reliable expert judgements to improve decision making, including peer review decisions. She has been active in establishing the Metascience community in Australia, and was the founding president of the Association for Interdisciplinary Metaresearch and Open Science (AIMOS). She is co-director of the MetaMelb Research Initiative at the University of Melbourne, and lead PI of the repliCATS project (Collaborative Assessments for Trustworthy Science). [https://images.e2ma.net/0/images/templates/spacer.gif] [The University of Sydney] Keep in touch [Facebook]<https://t.e2ma.net/click/g68j3x/8zmgipmb/k9pee2e> [Twitter]<https://t.e2ma.net/click/g68j3x/8zmgipmb/01qee2e> [Instagram]<https://t.e2ma.net/click/g68j3x/8zmgipmb/guree2e> [LinkedIn]<https://t.e2ma.net/click/g68j3x/8zmgipmb/wmsee2e> [YouTube]<https://t.e2ma.net/click/g68j3x/8zmgipmb/cftee2e> Copyright © 2025 The University of Sydney, NSW 2006 Australia Phone +61 2 9351 2222 ABN 15 211 513 464 CRICOS Number: 00026A Please add [email protected] to your address book or senders safe list to make sure you continue to see our emails in the future. Manage<https://app.e2ma.net/app2/audience/signup/1976084/1957350/1439896038/105406278118/?s=7EajSOyfe47wZP56X6bXQdDL_rGXoNtdT5C3NnV_fnw> your preferences | Opt out<https://t.e2ma.net/optout/g68j3x/8zmgipmb?s=bz0Iia9dkunAhx1FFqlWJwzc9sXzwRbRS3ZhrpGhRuI> using TrueRemove® Got this as a forward? Sign up<https://app.e2ma.net/app2/audience/signup/1976084/1957350.1439896038/> to receive our future emails. View this email online<https://t.e2ma.net/message/g68j3x/8zmgipmb>. Disclaimer<https://t.e2ma.net/click/g68j3x/8zmgipmb/s7tee2e> | Privacy statement<https://t.e2ma.net/click/g68j3x/8zmgipmb/8zuee2e> | University of Sydney<https://t.e2ma.net/click/g68j3x/8zmgipmb/osvee2e>
--------- SydPhil mailing list To unsubscribe, change your membership options, find answers to common problems, or visit our online archives, please go to the list information page: https://mailman.sydney.edu.au/mailman/listinfo/sydphil
