What ethical standards should data analysts be held to?

Is there truth in numbers anymore? At the Digital Analytics Association (DAA) Philadelphia Symposium on Oct. 19, which focused on eethics in analytics, keynote speaker Simon Dumenco, Editor-at-Large at Ad Age, cautioned conference attendees that we can’t trust numbers just because they’re numbers, in this day and age.

Dumenco’s presentation focused on a few salient examples from recent news, and lamented that the media has come to worship landmark numbers, without taking a more critical look at what they really mean. For example, back in June 2017, Mark Zuckerberg announced that the Facebook “community” had officially grown to over 2 billion monthly active users. The media quickly picked up this statistic, reinforcing the idea that 2 billion monthly active users was a meaningful and significant number. Dumenco, however, was more skeptical. Why, he asked, should we trust self-reported statistics (especially from an organization with a history of incorrect reporting)? And do we really know what counts as an “active” user for Facebook? What about automatic logins?

It’s time for an industrywide reality check, says Dumenco.

As data analysts, it’s up to us to start the conversation about what meaningful numbers are (and aren’t). Some of his key takeaways were as follows:

1. Do stop believing… in fake math.

• Don’t trust a number if it’s self-reported. If there aren’t third-party auditors, how can we know that statistics are real? Facebook has claimed it can reach more young people than actually exist in the U.K., U.S. and other countries. Maybe Facebook is not the new holy grail of millennial advertising.

2. Garbage in, garbage out

• Dumenco reported that garbage demographic data is being marketed to advertisers for microtargeting. For example, when Dumenco checked out his Facebook demographics, he discovered that he’s Hispanic (he isn’t) and that he loves baby clothes (he doesn’t have kids). Bad data is often worse (especially when it comes to decision making) than no data at all.

3. Or maybe we’re watching the death of math?

• Important data is being deleted by our government. Donald Trump’s administration deleted data on the Puerto Rico crisis from FEMA’s website on Thursday, Oct. 5, reporting that only 9.2 percent of the island had power and only 54.2 percent of residents had access to drinking water after the hurricane. (This information was later restored.) The information which remained, however, was the increase in number of federal workers on the ground, miles of roadway cleared, and percentage of grocery stores reopened. These limited statistics certainly made things look better than they actually were. But without all the statistics, how can we add up how things are really going?

4. Save us! And sign the Web Analyst’s Code of Ethics

• People manipulate data knowingly and unknowingly in the media, and face very little pushback. It’s up to data analysts to remember that organizations often report numbers for their self-interest, and not because they’re committed to truth. If you can identify ambiguities or flaws, there’s probably information missing. Data often needs a peer review.

As a data analytics professional organization, Dumenco mentioned, perhaps it’s up the DAA to set up or demand data providers to commit to a third party auditing process. The DAA offers a “Web Analyst Code of Ethics,” that analysts can sign. Here, DAA members can acknowledge that they have “a greater accountability to Internet users worldwide to balance data collection and analysis with responsible data stewardship.” Additionally, we can use our voice to prevent others (i.e. the Trump administration) from modifying public records when it suits their political agenda.

Sometimes data analytics is life and death. The repercussions of decisions that are made based on flawed data can spiral out of control.