Remembering the Past
One of the first things most organizations ask about their data assets is how to resolve the various bits and pieces of information that they’ve collected from all over the place into a single, trusted, version of the truth. A “golden record” that will tell them just the most up-to-date information about every customer they have, for example. It’s a worthy and important use of data, to tease out that sort of information, but there is value in keeping track of the past and being able to access it as well.
In retail analytics, one particular use of historical data is the ability to view segment migration; that is, the way in which customers move between different categorizations over time, from (perhaps) a once-in-a-while shopper to a value-driven regular shopper to an inelastically loyal shopper. These categories are derived, typically, from the latest shopping data combined with other metrics, but an historical understanding of where each customer has been can yield important second-level metrics about the health of the organization.
Are you seeing long-time loyal customers suddenly moving to less reliable segments? Is the percentage of your revenue that’s generated by one-time shoppers increasing dramatically? Answers to questions like this can only be gotten by collecting and maintaining historical data, especially by doing so with metrics that are generated through calculations on the data you’ve already got - a process I’ll call looping back.
By looping back the calculation of that segment categorization, and tagging it with a timestamp, you’re introducing a historical record of what you calculated that day - importantly, you’re recording what you reported out that day, since it’s possible that over time your calculation for segments might change. By introducing these timestamped records of what you’ve previously calculated, you’re helping to generate your own brand-new source of data, one that will help you to build out second-level metrics like migration direction, migration velocity, and more.
Such loopbacks are not the only reason to hang on to historical data. Imagine your customer data; over time lots of information about your customers will change; their addresses, phone numbers - even their names. By holding on to every copy of a customer record that you’ve ever received, and marking it with a timestamp, you can start to get a picture of what may have happened to that customer - when did they move? when did they get a new phone number? In some cases, you may be able to determine with reasonable accuracy that they got married, or divorced. That information may be useful from a marketing perspective or simply in adding more demographic color to your knowledge about your customers. And it’s impossible if you don’t maintain those historical data rather than writing over them.
Larger organizations, certainly, are already aware of this. But organizations of all sizes could probably benefit from the reminder that it’s important to ask the question about each piece of data you receive: how might this change in the future? And additionally, what does a change imply? Asking such questions could lead to decisions about data retention, as well as ideas for analytic metrics that could shape the future of the organization.