{"id":454,"date":"2014-08-05T09:31:14","date_gmt":"2014-08-05T14:31:14","guid":{"rendered":"http:\/\/dataabinitio.com\/?p=454"},"modified":"2014-07-31T19:15:19","modified_gmt":"2014-08-01T00:15:19","slug":"data-dictionaries","status":"publish","type":"post","link":"https:\/\/dataabinitio.com\/?p=454","title":{"rendered":"Data Dictionaries"},"content":{"rendered":"<p>Recently, I was reading through Christie Bahlai\u2019s excellent <a title=\"Practical Data Management Takes its Show on the Road\" href=\"http:\/\/practicaldatamanagement.wordpress.com\/2014\/07\/17\/practical-data-management-takes-its-show-on-the-road\/\">roundup of spreadsheet best practices<\/a>\u00a0when I started thinking about documenting spreadsheets. You see, best practices say that spreadsheets should contain only one large data table with short variable names at the top of each column, which doesn\u2019t leave room to describe the formatting and meaning of the spreadsheet\u2019s contents. This information is important, especially if you are trying to use <a title=\"Twitter: #otherpeoplesdata\" href=\"https:\/\/twitter.com\/hashtag\/otherpeoplesdata?src=hash\">#otherpeoplesdata<\/a>, but it honestly doesn\u2019t belong in the spreadsheet.<\/p>\n<p>So how do you give context to a spreadsheet\u2019s contents? The answer is a data dictionary. And seeing as I haven\u2019t found a good post on data dictionaries and data dictionaries are right up there with <a title=\"README.txt\" href=\"https:\/\/dataabinitio.com\/?p=378\">README.txt<\/a>\u2019s\u00a0as a Documentation Structure of Awesomeness<sup>TM<\/sup>, I obviously need to give them a whole post on this blog.<\/p>\n<p>So what is a data dictionary? A data dictionary is something that describes the data in a dataset. Generally, a data dictionary includes an overall description of the data along with more detailed descriptions of each variable, such as:<\/p>\n<ul>\n<li>Variable name<\/li>\n<li>Variable meaning<\/li>\n<li>Variable units<\/li>\n<li>Variable format<\/li>\n<li>Variable coding values and meanings<\/li>\n<li>Known issues with the data (systematic errors, missing values, etc.)<\/li>\n<li>Relationship to other variables<\/li>\n<li>Null value indicator<\/li>\n<li>Anything else someone needs to know to better understand the data<\/li>\n<\/ul>\n<p>This list represents the types of things you would want to know when faced with an unknown dataset. Not only is such information incredibly useful if you\u2019re sharing a dataset, but it\u2019s also useful if you plan to reuse a dataset in the future or you are working with a very large dataset. Basically, if there\u2019s a chance you won\u2019t remember the details or never knew them in the first place, a data dictionary is needed.<\/p>\n<figure id=\"attachment_457\" aria-labelledby=\"figcaption_attachment_457\" class=\"wp-caption alignnone\" style=\"width: 734px\"><a href=\"https:\/\/dataabinitio.com\/wp-content\/uploads\/2014\/08\/20140813_LemurTableSubset.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-457   \" title=\"Lemur Spreadsheet Subset\" alt=\"Lemur Spreadsheet Subset\" src=\"https:\/\/dataabinitio.com\/wp-content\/uploads\/2014\/08\/20140813_LemurTableSubset.jpg\" width=\"724\" height=\"462\" srcset=\"https:\/\/dataabinitio.com\/wp-content\/uploads\/2014\/08\/20140813_LemurTableSubset.jpg 1257w, https:\/\/dataabinitio.com\/wp-content\/uploads\/2014\/08\/20140813_LemurTableSubset-300x191.jpg 300w, https:\/\/dataabinitio.com\/wp-content\/uploads\/2014\/08\/20140813_LemurTableSubset-1024x653.jpg 1024w\" sizes=\"auto, (max-width: 724px) 100vw, 724px\" \/><\/a><figcaption id=\"figcaption_attachment_457\" class=\"wp-caption-text\">Lemur Spreadsheet Subset<\/figcaption><\/figure>\n<p>Let\u2019s look at a real world example from some newly released data from the Duke Lemur Center (<a title=\"Life history profiles for 27 strepsirrhine primate taxa generated using captive data from the Duke Lemur Center\" href=\"http:\/\/dx.doi.org\/10.1038\/sdata.2014.19\">data descriptor<\/a>, <a title=\"Data from: Life history profiles for 27 strepsirrhine primate taxa generated using captive data from the Duke Lemur Center\" href=\"http:\/\/dx.doi.org\/10.5061\/dryad.fj974\">dataset<\/a>). I downloaded the \u201cDataRecord_3_DLC_Weight_File_06Jun14.csv\u201d file from Dryad and found that, while the dataset is very clean (yay!), I can\u2019t interpret all of the data from the spreadsheet alone. For example, what does the variable \u201cAgeAtWt_mo_NoDec\u201d mean or what does the \u201cTaxon\u201d variable code \u201cCMED\u201d stand for? Enter the data dictionary in the form of the README.doc file.<\/p>\n<figure id=\"attachment_461\" aria-labelledby=\"figcaption_attachment_461\" class=\"wp-caption alignnone\" style=\"width: 574px\"><a href=\"https:\/\/dataabinitio.com\/wp-content\/uploads\/2014\/08\/20140813_LemurDataDictionarySubset.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-461  \" title=\"Lemur Data Dictionary Subset\" alt=\"Lemur Data Dictionary Subset\" src=\"https:\/\/dataabinitio.com\/wp-content\/uploads\/2014\/08\/20140813_LemurDataDictionarySubset.jpg\" width=\"564\" height=\"662\" srcset=\"https:\/\/dataabinitio.com\/wp-content\/uploads\/2014\/08\/20140813_LemurDataDictionarySubset.jpg 697w, https:\/\/dataabinitio.com\/wp-content\/uploads\/2014\/08\/20140813_LemurDataDictionarySubset-255x300.jpg 255w\" sizes=\"auto, (max-width: 564px) 100vw, 564px\" \/><\/a><figcaption id=\"figcaption_attachment_461\" class=\"wp-caption-text\">Lemur Data Dictionary Subset<\/figcaption><\/figure>\n<p>The Lemur data dictionary nicely lays out information on each variable in the dataset. For example, it defines the variable \u201cAgeAtWt_mo_NoDec\u201d as<\/p>\n<blockquote><p>Age in months with no decimal:\u00a0 AgeAtWt_mo\u00a0 value rounded down to a whole number for use in computing average individual weights (FLOOR(AgeAtWt_mo))<\/p><\/blockquote>\n<p>It also has a whole separate table listing the various Taxon codes. This is just the type of added context that describes the variables enough to make the data useful. It\u2019s also the type of information that you can\u2019t smoosh into a spreadsheet without ruining the spreadsheet\u2019s order and computability. So this data dictionary is adding a lot of value and context to the data without messing up the data themselves.<\/p>\n<p>The Lemur dataset can be easily understood and reused because it has clean data, well-named variables, and a nice data dictionary. If you are sharing your data publically, or even just with your future self, plan to give your data the same treatment. And if you don\u2019t have time to do all three preparations? Make the data dictionary. You can\u2019t use data you don\u2019t understand.<\/p>\n<p>Now go out and make some data dictionaries!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recently, I was reading through Christie Bahlai\u2019s excellent roundup of spreadsheet best practices\u00a0when I started thinking about documenting spreadsheets. You see, best practices say that spreadsheets should contain only one large data table with short variable names at the top &hellip; <a href=\"https:\/\/dataabinitio.com\/?p=454\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[18],"tags":[],"class_list":["post-454","post","type-post","status-publish","format-standard","hentry","category-documentation"],"_links":{"self":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts\/454","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=454"}],"version-history":[{"count":11,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts\/454\/revisions"}],"predecessor-version":[{"id":467,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts\/454\/revisions\/467"}],"wp:attachment":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=454"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=454"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=454"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}