.

.

PSYCHOLOGY :

RESEARCH: DESIGN: FLAWS:

STATISTICS :

DATA :

DATABASE SEARCH RESULTS:

New Critique Sees Flaws in Landmark Analysis of Psychology Studies

.

.

New Critique Sees Flaws in Landmark Analysis of Psychology Studies

By BENEDICT CAREY

MARCH 3, 2016

http://www.nytimes.com/2016/03/04/science/ psychology-replication-reproducibility-project.html

.

A shorter URL for the above link:

.

http://tinyurl.com/h3oo3mg

.

.

A landmark 2015 report that cast doubt on the results of dozens of published psychology studies has exposed deep divisions in the field, serving as a reality check for many working researchers but as an affront to others who continue to insist the original research was sound.

.

On Thursday, a group of four researchers publicly challenged the report, arguing that it was statistically flawed and, as a result, wrong.

.

The 2015 report, called the Reproducibility Project, found that less than 40 studies in a sample of 100 psychology papers in leading journals held up when retested by an independent team. The new critique by the four researchers countered that when that teams statistical methodology was adjusted, the rate was closer to 100 percent.

.

Neither the original analysis nor the critique found evidence of fraud or manipulation of data.

.

The critique was published in Science, the journal that published the original report. On Thursday, Science also published a strong rebuttal from the authors of the original replication project.

.

That study got so much press, and the wrong conclusions were drawn from it, said Timothy D. Wilson, a professor of psychology at the University of Virginia and an author of the new critique. Its a mistake to make generalizations from something that was done poorly, and this we think was done poorly.

.

Brian A. Nosek, a colleague of Dr. Wilsons at Virginia who coordinated the original replication project, which took several years, countered that the critique was highly biased: They are making assumptions based on selectively interpreting data and ignoring data thats antagonistic to their point of view.

.

The challenge comes as the field of psychology is facing a generational change, with young researchers beginning to share their data and study designs before publication, to improve transparency. Still, the new critique is likely to feed an already lively debate about how best to conduct and evaluate so-called replication projects of studies.

.

.

The complete article may be read at the URL above.

.

.

Comment on Estimating the reproducibility of psychological science
Daniel T. Gilbert1,*,, Gary King1, Stephen Pettigrew1, Timothy D. Wilson2
+ Author Affiliations
?*Corresponding author. E-mail: gilbert@wjh.harvard.edu
Science  04 Mar 2016:
Vol. 351, Issue 6277, pp. 1037
DOI: 10.1126/science.aad7243

http://science.sciencemag.org/content/351/6277/1037.2

.

Estimating the reproducibility of psychological science
Open Science Collaboration*,
+ Author Affiliations
Corresponding author. E-mail: nosek@virginia.edu
Science  28 Aug 2015:
Vol. 349, Issue 6251, pp.
DOI: 10.1126/science.aac4716
[Full Text]

http://science.sciencemag.org/content/349/6251/ aac4716.full?ijkey=1xgFoCnpLswpk&keytype=ref&siteid=sci

OR

http://tinyurl.com/zz2b2dl

.

CONCLUSION

No single indicator sufficiently describes replication success, and the five indicators examined here are not the only ways to evaluate reproducibility. Nonetheless, collectively these results offer a clear conclusion: A large portion of replications produced weaker evidence for the original findings despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the original effect sizes. Moreover, correlational evidence is consistent with the conclusion that variation in the strength of initial evidence (such as original P value) was more predictive of replication success than variation in the characteristics of the teams conducting the research (such as experience and expertise). The latter factors certainly can influence replication success, but they did not appear to do so here.

.

Reproducibility is not well understood because the incentives for individual scientists prioritize novelty over replication. Innovation is the engine of discovery and is vital for a productive, effective scientific enterprise. However, innovative ideas become old news fast. Journal reviewers and editors may dismiss a new test of a published idea as unoriginal. The claim that we already know this belies the uncertainty of scientific evidence. Innovation points out paths that are possible; replication points out paths that are likely; progress relies on both. Replication can increase certainty when findings are reproduced and promote innovation when they are not. This project provides accumulating evidence for many findings in psychological research and suggests that there is still more work to do to verify whether we know what we think we know.

.

DATA COLADA
THINKING ABOUT EVIDENCE AND VICE VERSA

03.03.16
by Uri Simonsohn
[47] EVALUATING REPLICATIONS: 40% FULL ? 60% EMPTY

http://datacolada.org/2016/03/03/47/

.

Summary

.

1. Replications must be analyzed in ways that allow for results to be inconclusive, not just success/fail

.

2. Design differences between original and replication should be prominently disclosed.

.

.

Database Search Results

Google Domain Limited Web Search (BLOGS)

http://tinyurl.com/j7urc95

.

Google Domain Limited Web Search (NEWS)

http://tinyurl.com/zmeoe98

.

Google Videos

http://tinyurl.com/h9rohlq

.

Google Domain Limited Web Search (VIDEOS)

http://tinyurl.com/ze5sahh

.

Google Images

http://tinyurl.com/goueks9

.

Google Domain Limited Web Search (IMAGES)

http://tinyurl.com/hckfpe6

.

Google Domain Limited Web Search (PUBMED)

http://tinyurl.com/hxyvmcq

.

PUBMED

http://tinyurl.com/gvs5lqe

.

Google Domain Limited Web Search (GOV)

http://tinyurl.com/hd6e95d

.

.

WEBBIB1516

http://tinyurl.com/q8tavoy

.

.

Sincerely,
David Dillard
Temple University
(215) 204 – 4584
jwne@temple.edu
http://workface.com/e/daviddillard

Net-Gold
https://groups.io/g/Net-Gold
http://groups.yahoo.com/group/net-gold
http://listserv.temple.edu/archives/net-gold.html
https://groups.io/org/groupsio/Net-Gold/archives
http://net-gold.3172864.n2.nabble.com/

Temple University and Google Sites Research Guides
AND Discussion Group Directory
http://tinyurl.com/ngda2hk

OR

https://sites.google.com/site/researchguidesonsites/

RESEARCH PAPER WRITING
http://guides.temple.edu/research-papers
EMPLOYMENT
http://guides.temple.edu/employment-guide
INTERNSHIPS
http://guides.temple.edu/employment-internships
HOSPITALITY
http://guides.temple.edu/hospitality-guide
DISABILITIES AND EMPLOYMENT
http://guides.temple.edu/c.php?g=134557
INDOOR GARDENING
https://groups.io/g/indoor-gardening
Educator-Gold
http://groups.yahoo.com/group/Educator-Gold/
K12ADMINLIFE
http://groups.yahoo.com/group/K12AdminLIFE/

PUBLIC HEALTH RESOURCES INCLUDING EBOLA
http://guides.temple.edu/public-health-guide

STATISTICS SOURCES RESEARCH GUIDE
http://guides.temple.edu/statistics-sources

Social Work and Social Issues Discussion Group
https://groups.io/g/social-work

Tourism Discussion Group
https://groups.io/g/Tourism

Digital Scholarship Discussion Group
https://groups.io/g/DigitalScholarship/threads
https://listserv.temple.edu/cgi-bin/wa?A0=DIGITAL-SCHOLARSHIP
https://groups.yahoo.com/neo/groups/digital-scholarship/info
https://digitalscholarshipandscholarlypublication.wordpress.com/

Copyright Research Guide
Copyright, Intellectual Property and Plagiarism Sources
http://guides.temple.edu/copyright-plagiarism
Fair Use
http://guides.temple.edu/fair-use

Blog
https://educatorgold.wordpress.com/

Articles by David Dillard
https://sites.google.com/site/daviddillardsarticles/

Information Literacy (Russell Conwell Guide)
http://tinyurl.com/78a4shn

Nina Dillard’s Photographs on Net-Gold
http://www.flickr.com/photos/neemers/

Twitter: davidpdillard

Temple University Site Map
https://sites.google.com/site/templeunivsitemap/home

Bushell, R. & Sheldon, P. (eds),
Wellness and Tourism: Mind, Body, Spirit,
Place, New York: Cognizant Communication Books.
Wellness Tourism: Bibliographic and Webliographic Essay
David P. Dillard
http://tinyurl.com/o4pn4o9

Rail Transportation
https://groups.io/org/groupsio/RailTransportation

INDOOR GARDENING
Improve Your Chances for Indoor Gardening Success
http://tech.groups.yahoo.com/group/IndoorGardeningUrban/

SPORT-MED
https://www.jiscmail.ac.uk/lists/sport-med.html
http://groups.yahoo.com/group/sports-med/
http://listserv.temple.edu/archives/sport-med.html

HEALTH DIET FITNESS RECREATION SPORTS TOURISM
https://groups.yahoo.com/neo/groups/healthrecsport/info
http://listserv.temple.edu/archives/health-recreation-sports-tourism.html

.

.

Please Ignore All Links to JIGLU
in search results for Net-Gold and related lists.
The Net-Gold relationship with JIGLU has
been terminated by JIGLU and these are dead links.
http://groups.yahoo.com/group/Net-Gold/message/30664
http://health.groups.yahoo.com/group/healthrecsport/message/145
Temple University Listserv Alert :
Years 2009 and 2010 Eliminated from Archives
https://sites.google.com/site/templeuniversitylistservalert/

.

.

.

-=-=-=-=-=-=-=-=-=-=-=-

 

.

 

.

Groups.io Links:

You receive all messages sent to this group.

View This Message (#82): https://groups.io/g/DigitalScholarship/message/82

 

.

 

.

 

 

Advertisements