{"id":537,"date":"2015-05-22T13:26:54","date_gmt":"2015-05-22T18:26:54","guid":{"rendered":"http:\/\/dataabinitio.com\/?p=537"},"modified":"2015-05-22T13:26:54","modified_gmt":"2015-05-22T18:26:54","slug":"data-or-it-didnt-happen","status":"publish","type":"post","link":"https:\/\/dataabinitio.com\/?p=537","title":{"rendered":"Data or It Didn\u2019t Happen"},"content":{"rendered":"

There\u2019s a story in the news this week about the requested retraction of a study on changing people\u2019s minds on same sex marriage<\/a>. I always find it interesting when retraction stories are picked up by major<\/a> news<\/a> outlets<\/a>, especially when the article\u2019s data (or lack thereof) is central to the reasons for the retraction.<\/p>\n

The likely retraction (currently an expression of concern) in question concerns a study published in Science<\/a> last year looking at the effect of canvassing on changing people\u2019s minds. Study participants took pre- and post-canvassing online surveys to judge the effect of canvassing on changing opinions. While the canvassing data appears to be real, it looks like the study\u2019s first author, Michael LaCour, made up data for the online surveys.<\/p>\n

The fact of the faked data is remarkable enough, but what particularly interests me is how it was discovered. Two graduate students at UC-Berkeley, David Broockman and Joshua Kalla, were interested in extending the study but had trouble reproducing the original study\u2019s high response rate<\/a>. Upon contacting the agency who supposedly conducted the surveys, they were told that the agency did not actually run or have knowledge of the pre- and post-tests. Evidence of misconduct mounted when Broockman and Kalla were able to access the original data<\/a> from another researcher who posted it in compliance with a journal\u2019s open data policy. They found anomalies once they started digging into the data.<\/p>\n

In my work, I talk a lot about the Reinhart and Rogoff debacle<\/a> from two years ago where a researcher gaining access to the article\u2019s data led to the fall of one of the central papers supporting economic austerity practices. We\u2019re seeing a similar result here with the LaCour study. But in this case, problems arose due to a common practice in research: using someone else\u2019s study as a starting point for your own study. Building from previous work is a central part of research and bad studies have problematic downstream effects. Unfortunately, such studies aren\u2019t easy to spot without digging into the data, which often isn\u2019t available.<\/p>\n

There\u2019s an expression that goes \u201cpictures or it didn\u2019t happen<\/a>,\u201d suggesting that an event didn\u2019t actually take place unless there is photographic proof. I think this expression needs to be coopted for research to be \u201cdata or it didn\u2019t happen.\u201d \u00a0Unless you can show me the data, how do I know that you actually did the research and did it correctly?<\/p>\n

I\u2019m not saying that all research is bad, just that we need regular access to data if we\u2019re going to be able to do research well. We can\u2019t build a house on a shaky foundation and without examining the foundation (data) in more detail, how will we find the problems or build the house well?<\/p>\n

So next time you publish an article, share the data that support that article. Because remember, data or it didn\u2019t happen.<\/p>\n","protected":false},"excerpt":{"rendered":"

There\u2019s a story in the news this week about the requested retraction of a study on changing people\u2019s minds on same sex marriage. I always find it interesting when retraction stories are picked up by major news outlets, especially when … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,20],"tags":[],"class_list":["post-537","post","type-post","status-publish","format-standard","hentry","category-opendata","category-researchmisconduct"],"_links":{"self":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts\/537","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=537"}],"version-history":[{"count":3,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts\/537\/revisions"}],"predecessor-version":[{"id":540,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts\/537\/revisions\/540"}],"wp:attachment":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=537"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=537"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=537"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}