Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

Simple item page
dc.contributor.authorBreznau, Nate
dc.contributor.authorRinke, Eike Mark
dc.contributor.authorWuttke, Alexander
dc.contributor.authorNguyen, Hung H. V.
dc.contributor.authorAdem, Muna
dc.contributor.authorAdriaans, Jule
dc.contributor.authorAlvarez-Benjumea, Amalia
dc.contributor.authorAndersen, Henrik K.
dc.contributor.authorAuer, Daniel
dc.contributor.authorAzevedo, Flavio
dc.contributor.authorBahnsen, Oke
dc.contributor.authorBalzer, Dave
dc.contributor.authorBauer, Gerrit
dc.contributor.authorBauer, Paul C.
dc.contributor.authorBaumann, Markus
dc.contributor.authorBaute, Sharon
dc.contributor.authorBenoit, Verena
dc.contributor.authorBernauer, Julian
dc.contributor.authorBerning, Carl
dc.contributor.authorBerthold, Anna
dc.contributor.authorBethke, Felix S.
dc.contributor.authorBiegert, Thomas
dc.contributor.authorBlinzler, Katharina
dc.contributor.authorBlumenberg, Johannes N.
dc.contributor.authorBobzien, Licia
dc.contributor.authorBohman, Andrea
dc.contributor.authorBol, Thijs
dc.contributor.authorBostic, Amie
dc.contributor.authorBrzozowska, Zuzanna
dc.contributor.authorBurgdorf, Katharina
dc.contributor.authorBurger, Kaspar
dc.contributor.authorBusch, Kathrin B.
dc.contributor.authorCarlos-Castillo, Juan
dc.contributor.authorChan, Nathan
dc.contributor.authorChristmann, Pablo
dc.contributor.authorConnelly, Roxanne
dc.contributor.authorCzymara, Christian S.
dc.contributor.authorDamian, Elena
dc.contributor.authorEcker, Alejandro
dc.contributor.authorEdelmann, Achim
dc.contributor.authorEger, Maureen A.
dc.contributor.authorEllerbrock, Simon
dc.contributor.authorForke, Anna
dc.contributor.authorForster, Andrea
dc.contributor.authorGaasendam, Chris
dc.contributor.authorGavras, Konstantin
dc.contributor.authorGayle, Vernon
dc.contributor.authorGessler, Theresa
dc.contributor.authorGnambs, Timo
dc.contributor.authorGodefroidt, Amelie
dc.contributor.authorGroemping, Max
dc.contributor.authorGross, Martin
dc.contributor.authorGruber, Stefan
dc.contributor.authorGummer, Tobias
dc.contributor.authorHadjar, Andreas
dc.contributor.authorHeisig, Jan Paul
dc.contributor.authorHellmeier, Sebastian
dc.contributor.authorHeyne, Stefanie
dc.contributor.authorHirsch, Magdalena
dc.contributor.authorHjerm, Mikael
dc.contributor.authorHochman, Oshrat
dc.contributor.authorHovermann, Andreas
dc.contributor.authorHunger, Sophia
dc.contributor.authorHunkler, Christian
dc.contributor.authorHuth, Nora
dc.contributor.authorIgnacz, Zsofia S.
dc.contributor.authorJacobs, Laura
dc.contributor.authorJacobsen, Jannes
dc.contributor.authorJaeger, Bastian
dc.contributor.authorJungkunz, Sebastian
dc.contributor.authorJungmann, Nils
dc.contributor.authorKauff, Mathias
dc.contributor.authorKleinert, Manuel
dc.contributor.authorKlinger, Julia
dc.contributor.authorKolb, Jan-Philipp
dc.contributor.authorKolczynska, Marta
dc.contributor.authorKuk, John
dc.contributor.authorKunissen, Katharina
dc.contributor.authorSinatra, Dafina Kurti
dc.contributor.authorLangenkamp, Alexander
dc.contributor.authorLersch, Philipp M.
dc.contributor.authorLobel, Lea-Maria
dc.contributor.authorLutscher, Philipp
dc.contributor.authorMader, Matthias
dc.contributor.authorMadia, Joan E.
dc.contributor.authorMalancu, Natalia
dc.contributor.authorMaldonado, Luis
dc.contributor.authorMarahrens, Helge
dc.contributor.authorMartin, Nicole
dc.contributor.authorMartinez, Paul
dc.contributor.authorMayerl, Jochen
dc.contributor.authorMayorga, Oscar J.
dc.contributor.authorMcManus, Patricia
dc.contributor.authorMcWagner, Kyle
dc.contributor.authorMeeusen, Cecil
dc.contributor.authorMeierrieks, Daniel
dc.contributor.authorMellon, Jonathan
dc.contributor.authorMerhout, Friedolin
dc.contributor.authorMerk, Samuel
dc.contributor.authorMeyer, Daniel
dc.contributor.authorMicheli, Leticia
dc.contributor.authorMijs, Jonathan
dc.contributor.authorMoya, Cristobal
dc.contributor.authorNeunhoeffer, Marcel
dc.contributor.authorNust, Daniel
dc.contributor.authorNygard, Olav
dc.contributor.authorOchsenfeld, Fabian
dc.contributor.authorOtte, Gunnar
dc.contributor.authorPechenkina, Anna O.
dc.contributor.authorProsser, Christopher
dc.contributor.authorRaes, Louis
dc.contributor.authorRalston, Kevin
dc.contributor.authorRamos, Miguel R.
dc.contributor.authorRoets, Arne
dc.contributor.authorRogers, Jonathan
dc.contributor.authorRopers, Guido
dc.contributor.authorSamuel, Robin
dc.contributor.authorSand, Gregor
dc.contributor.authorSchachter, Ariela
dc.contributor.authorSchaeffer, Merlin
dc.contributor.authorSchieferdecker, David
dc.contributor.authorSchlueter, Elmar
dc.contributor.authorSchmidt, Regine
dc.contributor.authorSchmidt, Katja M.
dc.contributor.authorSchmidt-Catran, Alexander
dc.contributor.authorSchmiedeberg, Claudia
dc.contributor.authorSchneider, J. Urgen
dc.contributor.authorSchoonvelde, Martijn
dc.contributor.authorSchulte-Cloos, Julia
dc.contributor.authorSchumann, Sandy
dc.contributor.authorSchunck, Reinhard
dc.contributor.authorSchupp, J. Urgen
dc.contributor.authorSeuring, Julian
dc.contributor.authorSilber, Henning
dc.contributor.authorSleegers, Willem
dc.contributor.authorSonntag, Nico
dc.contributor.authorStaudt, Alexander
dc.contributor.authorSteiber, Nadia
dc.contributor.authorSteiner, Nils
dc.contributor.authorSternberg, Sebastian
dc.contributor.authorStiers, Dieter
dc.contributor.authorStojmenovska, Dragana
dc.contributor.authorStorz, Nora
dc.contributor.authorStriessnig, Erich
dc.contributor.authorStroppe, Anne-Kathrin
dc.contributor.authorTeltemann, Janna
dc.contributor.authorTibajev, Andrey
dc.contributor.authorTung, Brian
dc.contributor.authorVagni, Giacomo
dc.contributor.authorVan Assche, Jasper
dc.contributor.authorvan der Linden, Meta
dc.contributor.authorvan der Noll, Jolanda
dc.contributor.authorVan Hootegem, Arno
dc.contributor.authorVogtenhuber, Stefan
dc.contributor.authorVoicu, Bogdan
dc.contributor.authorWagemans, Fieke
dc.contributor.authorWehl, Nadja
dc.contributor.authorWerner, Hannah
dc.contributor.authorWiernik, Brenton M.
dc.contributor.authorWinter, Fabian
dc.contributor.authorWolf, Christof
dc.contributor.authorYamada, Yuki
dc.contributor.authorZhang, Nan
dc.contributor.authorZiller, Conrad
dc.contributor.authorZins, Stefan
dc.contributor.authorZoltak, Tomasz
dc.date.accessioned2025-01-20T21:01:18Z
dc.date.available2025-01-20T21:01:18Z
dc.date.issued2022
dc.description.abstractThis study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.
dc.fuente.origenWOS
dc.identifier.doi10.1073/pnas.2203150119
dc.identifier.eissn1091-6490
dc.identifier.issn0027-8424
dc.identifier.urihttps://doi.org/10.1073/pnas.2203150119
dc.identifier.urihttps://repositorio.uc.cl/handle/11534/92855
dc.identifier.wosidWOS:000892368900018
dc.issue.numero44
dc.language.isoen
dc.revistaProceedings of the national academy of sciences of the united states of america
dc.rightsacceso restringido
dc.subjectmetascience
dc.subjectmany analysts
dc.subjectresearcher degrees of freedom
dc.subjectanalytical flexibility
dc.subjectimmigration and policy preferences
dc.subject.ods03 Good Health and Well-being
dc.subject.odspa03 Salud y bienestar
dc.titleObserving many researchers using the same data and hypothesis reveals a hidden universe of uncertainty
dc.typeartículo
dc.volumen119
sipa.indexWOS
sipa.trazabilidadWOS;2025-01-12
Files
Collections
Artículos de revistas