{"id":410,"date":"2014-04-17T14:40:37","date_gmt":"2014-04-17T19:40:37","guid":{"rendered":"http:\/\/dataabinitio.com\/?p=410"},"modified":"2014-04-17T14:40:37","modified_gmt":"2014-04-17T19:40:37","slug":"zero-v-null","status":"publish","type":"post","link":"https:\/\/dataabinitio.com\/?p=410","title":{"rendered":"Zero v. Null"},"content":{"rendered":"
\"Zero\"<\/a>
“Null Schatten \/ Zero shade” by Winfried, https:\/\/www.flickr.com\/photos\/w-tommerdich\/3630153810 (CC BY-NC-SA)<\/figcaption><\/figure>\n

An important aspect of data management is performing quality control on your data. This means checking your data for errors, ensuring consistent formatting, documenting the meaning of your variables, etc. Also under the umbrella of quality control is how to represent the absence of value in a dataset.<\/p>\n

The absence of value can mean many things in a dataset including a true zero, that the data point is missing, that the data point is not applicable to this entry, etc. Unfortunately, many of these cases end up with the same label (or several different labels for any one case!) in a dataset, either \u201c0\u201d, a blank entry, \u201cNA\u201d or something else. And when it comes to calculating values like averages, there is a big difference between a \u201c0\u201d that is a true zero and one that is a placeholder for missing data. Therefore, we need to establish some best practices around absence of value.<\/p>\n

The first rule is that \u201c0\u201d always represents true zero and nothing else. This means that you\u2019ve made a measurement and that measurement happens to be zero. Only using \u201c0\u201d for this case makes your subsequent calculations accurate.<\/p>\n

The second rule is to pick a good null label. This label will represent a lack of measurement. One of the best null labels is the blank entry, which most programs will interpret as null (so long as you\u2019re careful not use a space instead of a blank). A secondary option is to use the null value preferred by your primary analysis program: \u201cNA\u201d in R, \u201cNULL\u201d in SQL, and \u201cNone\u201d in Python, etc. (see Table 1 of White, et al.<\/a>). However, this option is less ideal as it can result in unexpected problems if you don\u2019t modify the nulls in your dataset before using it in a different program. So it\u2019s best to stick with the blank entry for all of your null data points.<\/p>\n

The third rule is to be consistent. There is no point in standardizing something if you\u2019re not going to be consistent about it and, in this case, consistency makes for accurate calculations. So pick a system and stick with it!<\/p>\n

Finally, you should document anything that isn\u2019t standard. Want to use blanks for missing data and \u201cNA\u201d for not applicable data points? You can do it, so long as you are clear and upfront (and consistent) about the system you use.<\/p>\n

Keeping zero and null straight is not difficult, but it takes a little conscious effort to be sure that everything is accurate. This effort is worth it in the long run, as your datasets are streamlined and your calculations turn out correct.<\/p>\n

 <\/p>\n

References:<\/strong><\/p>\n

This post is about nothing*<\/a>, Practical Data Management for Bug Counters. 30 Jan 2014.<\/p>\n

White, et al. Nine simple ways to make it easier to (re)use your data<\/em>. Ideas in Ecology and Evolution. 6(2). 2013. http:\/\/library.queensu.ca\/ojs\/index.php\/IEE\/article\/view\/4608\/4898<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

An important aspect of data management is performing quality control on your data. This means checking your data for errors, ensuring consistent formatting, documenting the meaning of your variables, etc. Also under the umbrella of quality control is how to … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[16],"tags":[],"class_list":["post-410","post","type-post","status-publish","format-standard","hentry","category-dataanalysis"],"_links":{"self":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts\/410","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=410"}],"version-history":[{"count":8,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts\/410\/revisions"}],"predecessor-version":[{"id":419,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=\/wp\/v2\/posts\/410\/revisions\/419"}],"wp:attachment":[{"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=410"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=410"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataabinitio.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=410"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}