- Tytuł:
- Same data, different analysts : variation in effect sizes due to analytical decisions in ecology and evolutionary biology
- Autorzy:
-
Thawley, Christopher J.
Ferreira-Arruda, Thalita
Rugemalila, Deusdedith
Taff, Conor Claverie
Hager, Heather A.
Schmoll, Tim
Ke, Alison
Fisher, David N.
Payo-Payo, Ana
Mos, Benjamin
Abbey-Lee, Robin N.
Martin, Charles A.
Manrique-Garzon, Laura Milena
Michelangeli, Marcus
Shapiro, Julie Teresa
Rosenberg, Michael S.
Schroeder, Julia
Cox, Murray P.
Novotny, Jessie Lanterman
Santos, Leticia Pereira
Kačergytė, Ineta
Muraina, Taofeek Olatunbosun
Griffith, Frances J.
Riva, Federico
Paterson, John Harold
García-Cervigón, Ana I.
Camerlenghi, Ettore
Macphie, Kirsty H.
Schmidt, Marcus
Youngflesh, Casey
Palacio, Facundo Xavier
Campbell, Sara E.
Skupien, Fabrício Luiz
Keogan, Katharine
Perry, Kayla I.
Bell, Kristian
Good, Megan Kate
Harrison, Natasha Dean
Paquet, Matthieu
Zilio, Giacomo
Pottier, Patrice
Organ, Chris L.
Döbert, Timm Fabian
Greenler, Skye M.
Marchand, Philippe
Smith, Jeremy A.
Elmore, Jared Alan
Turnell, Biz R.
Jucker, Tommaso
Gandy, Sara L.
Verrelli, Brian C.
Atkins, Jeff W.
Razafindratsima, Onja H.
Vélez, Juliana
Johnsson, Martin
Crouch, Connor Davidson
de Sousa, Alexandra Allison
Martin, Dominic Andreas
Tanentzap, Andrew J.
Walker, Xanthe J.
Wolfson, David William
Bradham, Jennifer
Ross, Jessica
Atkinson, Joe
Barrett, Meghan
Gould, Elliot
Shearer, Caroline L.
Wood, Andrew
Law, Alan
Kaltz, Oliver
Gliksman, Daniel
Drobniak, Szymon
Hagan, James G.
van Oordt, Francis
Hsu, Bin-Yan
Billman, Peter D.
de Lima, Daniela Oliveira
Blake, Shannon
Finch, Elizabeth A.
Kothari, Shan
Griffith, Daniel M.
Herrera-Chaustre, María Laura
Slinn, Heather Lea
Ramananjato, Veronarindra
Zimmer, Cédric
Meiners, Scott J.
Martinig, April Robin
Goslee, Sarah C.
Campos, Leonardo L. F.
Gilles, Marc
Duffy, Alexandra Grace
Baker, Christopher M.
Vergara-Florez, Diana Carolina
Santostefano, Francesca
Tompkins, Emily M.
Dunn, Robert P.
Dobson, Adam J.
Salazar, Stephen M.
Eberhart-Hertel, Luke
Kochan, David P.
Takola, Elina
Géron, Charly
Freund, Cathryn A.
Blake, Charlie K.
Chik, Heung Ying Janet
Telford, Richard J.
Magellan, Kit
Zimmerman, Gregory Mark
Bertram, Michael Grant
Luquet, Martin
Roast, Michael James
Nooten, Sabine S.
Hamilton, Daniel G.
D’Amelio, Pietro B.
Goldspiel, Harrison B.
Mortensen, Jennifer
Vanderwel, Mark C.
Yen, Jian D. L.
Niemelä, Petri
McCallum, Erin S.
Malm, Lisa E.
Trlica, Andrew
Christie, Alec Philip
Mair, Magdalena M.
Weaver, Ryan J.
Salmón, Pablo
Caravaggi, Anthony
Harris, Jonathan Philo
Elsherif, Mahmoud Medhat
Vitali, Valerio
Murphy, Penelope Wrenn
Aloni, Irith
Havird, Justin Chase
Sánchez-Tójar, Alfredo
English, Holly M.
Pedersen, Karen Marie
Clark, Bethany L.
Walker, Jeffrey
Brand, Jack A.
Schtickzelle, Nicolas
Zitomer, Rachel A.
Kusch, Jillian M.
Larson, Courtney L.
Haesen, Stef
Chunco, Amanda J.
Moiron, Maria
Heaton, Andrew J.
Schilling, Hayden T.
Killion, Alexander K.
Lagisz, Malgorzata
Bulla, Martin
Tarjuelo, Rocío
Hasnain, Sarah Syedia
Sollmann, Rahel
Fidler, Fiona
Merkling, Thomas
Grebe, Nicholas M.
Román-Palacios, Cristian
Frank, Graham S.
Gomes, Dylan G. E.
Kelly, Clint D.
Simón, Diego
Olin, Agnes Birgitta
Bradfer-Lawrence, Tom
Manhart, Michael
Whelan, Shannon
Villamil, Nora
Cressman, Kimberly A.
Alcaraz, Carles
Griffioen, Maaike
Fernandez-Juricic, Esteban
Cardoso, Pedro
Russell, Avery L.
Jung, Martin
Girndt, Antje
Fiorenza, Evan A.
Bordes, Camille Nina Marion
McCauley, Mark
Saccone, Patrick
Gratton, Paolo
Tortorelli, Claire Marie
Crotti, Marco
Dobler, Ralph
Fourcade, Yoan
Griffith, Simon C.
Rocha, Felipe Pereira
Nelli, Luca
Sutton, Guy F.
Ensminger, David C.
Beltrán, Iván
Martin, Jake Mitchell
Keppeler, Friedrich Wolfgang
Boyd, Melissa Anna
Gosnell, J. Stephen
Waryszak, Pawel
Brengdahl, Martin I.
Lindsay, Shane
Chuang, Angela
Forstmeier, Wolfgang
Tidau, Svenja
Fieberg, John
Johnson, Douglas H.
Perez, Grégoire
Lauterbur, M. Elise
Kuppler, Jonas
Mäntylä, Elina
Thierry, Hugo
Catanach, Therese A.
Vanderwel, K. Michelle
Bonisoli-Alquati, Andrea
Sadeh, Asaf
Iranzo, Esperanza C.
Aguirre, Luis A.
L’Herpiniere, Kiara
Parker, Timothy H.
Smith, Grania Polly
Whitney, Kaitlin Stack
Lembrechts, Jonas J.
Choy, Emily Sarah
Léandri-Breton, Don-Jean
Bonnet, Timothée
Dutta, Trishna
Dunning, Jamie
Garretson, Alexis C.
Mandeville, Caitlin P.
Iverson, Erik N. K.
Nightingale, Josh
Still, Shannon Michael
Abbott, Jessica K.
Parker, Darren James
Ruuskanen, Suvi
Fontaine, Amélie
Scott, Drew A.
Vanderwolf, Karen J.
Ostevik, Kate L.
Lievens, Eva J. P.
Botterill-James, Thomas
Berauer, Bernd J.
Mammola, Stefano
Geary, William L.
Jimoh, Saheed Olaide
Duncan, Alison B.
Fraser, Hannah S.
White, Rachel Louise
Ge, Xuezhen
Van de Vondel, Stijn
Chen, Xuan
Carroll, Charles J. W.
Rowland, Freya E.
Grames, Eliza M.
Nakagawa, Shinichi
Pascall, David J.
Covernton, Garth A.
Thomson, Jacqueline
Pasquarella, Valerie J.
Swallow, Ben
Bliard, Louis
Stuber, Erica F.
Bello, Suleiman Kehinde
Lauck, Katherine S.
Grossman, Jake J.
Urban, Lara
Proulx, Raphaël
Weller, Daniel L.
Roche, Dominique G.
Larkin, Daniel J.
Boyle, Sarah A.
Korsten, Peter
Contina, Andrea
Proulx, Michael J.
Van Eeckhoven, Jens
Ferguson, Stephen M.
Sharma, Nitika
Marshall, Benjamin Michael
Nolazco, Sergio
Schultz, Nick L.
Kim, Dongmin
Lalla, Kristen Marianne
Ernst, Ulrich Rainer
Rennison, Diana J.
Güncan, Ali
Riyahi, Sepand
Moreira, Bruno
Iannarilli, Fabiola
Wedegärtner, Ronja E. M.
Yanco, Scott W.
Randimbiarison, Finaritra Tolotra
Vollering, Julien
Schneider, Adam C.
Vesk, Peter A.
McNew, Sabrina M.
Arekar, Kunal
Nilsonne, Gustav
Bose, Aneesh P. H.
Walter, Jonathan A.
Gannon, Dustin G.
Howard, Tanner J.
Scroggie, Michael Peter
Doherty, Tim S.
Sitvarin, Michael I.
Vieira, Marcus Vinícius
Altschul, Drew
Fuentes-Lillo, Eduardo
Pruett, Jessica L
Schrock, Allie E.
Sales, Kris
MacLeod, Ross
Jorna, Jesse
Bussière, Luc
van der Wal, Jessica Eva Megan - Opis:
- Although variation in effect sizes and predicted values among studies of similar phenomena is inevitable, such variation far exceeds what might be produced by sampling error alone. One possible explanation for variation among results is differences among researchers in the decisions they make regarding statistical analyses. A growing array of studies has explored this analytical variability in different fields and has found substantial variability among results despite analysts having the same data and research question. Many of these studies have been in the social sciences, but one small “many analyst” study found similar variability in ecology. We expanded the scope of this prior work by implementing a large-scale empirical exploration of the variation in effect sizes and model predictions generated by the analytical decisions of different researchers in ecology and evolutionary biology. We used two unpublished datasets, one from evolutionary ecology (blue tit, Cyanistes caeruleus, to compare sibling number and nestling growth) and one from conservation ecology (Eucalyptus, to compare grass cover and tree seedling recruitment). The project leaders recruited 174 analyst teams, comprising 246 analysts, to investigate the answers to prespecified research questions. Analyses conducted by these teams yielded 141 usable effects (compatible with our meta-analyses and with all necessary information provided) for the blue tit dataset, and 85 usable effects for the Eucalyptus dataset. We found substantial heterogeneity among results for both datasets, although the patterns of variation differed between them. For the blue tit analyses, the average effect was convincingly negative, with less growth for nestlings living with more siblings, but there was near continuous variation in effect size from large negative effects to effects near zero, and even effects crossing the traditional threshold of statistical significance in the opposite direction. In contrast, the average relationship between grass cover and Eucalyptus seedling number was only slightly negative and not convincingly different from zero, and most effects ranged from weakly negative to weakly positive, with about a third of effects crossing the traditional threshold of significance in one direction or the other. However, there were also several striking outliers in the Eucalyptus dataset, with effects far from zero. For both datasets, we found substantial variation in the variable selection and random effects structures among analyses, as well as in the ratings of the analytical methods by peer reviewers, but we found no strong relationship between any of these and deviation from the meta-analytic mean. In other words, analyses with results that were far from the mean were no more or less likely to have dissimilar variable sets, use random effects in their models, or receive poor peer reviews than those analyses that found results that were close to the mean. The existence of substantial variability among analysis outcomes raises important questions about how ecologists and evolutionary biologists should interpret published results, and how they should conduct analyses in the future.
- Dostawca treści:
- Repozytorium Uniwersytetu Jagiellońskiego
Artykuł