I had this wonderful rant all queued up about how a major drug store chain has crossed the line into “creepy” with their updated rewards card program. And then I realized that I already predicted this would happen over 20 years ago, and how many privacy advocates are closing the barn door after the cows have already left.

Let’s start with what’s relevant from the rant I had already typed out. Recently, I had the pleasure of having a run-in with a clerk at a national drug store chain. They are getting pretty insistent at signing people up for their “privacy-eliminating marginal discount card”, to the point that I actually got in a small argument with the clerk. The clerk helpfully (?) said something to the extent of “oh, I don’t give them my REAL information…”

As a quick sidebar, I found that personally humorous. Here is a clerk outright telling me that she’s committing an act of fraud against her employer, and all but encouraging me to do the same so I can save a dollar. Wow.

Anyway, it got me thinking about how this particular national chain was going about it all wrong, and how another chain (a national grocery store chain that operates a regional hypermarket.. oh, hell, I’m talking about Fred Meyer) seems to do it right. The differences were all academic, actually, because after thinking about it for a bit I even came to the conclusion that I wasn’t 100% satisfied with how Uncle Freddy does it, either; and there was plenty of privacy implications with the Fred Meyer approach.

So, here’s how the Fred Meyer Rewards program works, and why at first it seems like a much better way than most other loyalty card programs. When you shop at Fred Meyer, regardless of whether or not you have a Rewards card you pay the same price. There’s no “$1 off with our discount card” nonsense* and very few actual gimmicks: at the end of every quarter, they send you a coupon worth a small percentage of that period’s purchases.  They also give you points towards discounted gasoline purchases, and even keep track of purchases at the coffee kiosk for free coffee.

Okay, so that seemed like a better approach.  Until I unearthed a little piece of Commodore 64 code I wrote as a kid on a recent “storage unit spelunking adventure.”

Let’s set the wayback to the 1980’s.  A grocery store chain in Southern California had a novel way of handling checks.  Rather than have the cashiers check a master list for bad checks, or having the (primitive by today’s standards) cash register maintain a list, they had a stand-alone check authorization computer.  It sat in the front of the store, and you had a mag-stripe card that you’d use to get your check “approved” before you went shopping.  This system fascinated me, because it seemed like an elegant hack to an obvious problem.  I envisioned all kinds of wonderfully complicated approaches to granting approval: some probably predicting the fraud detection algorithms used by modern credit card processors to determine “iffy” transactions.

In the end, I simply started writing a small program to do the same thing using the C-64.  I did this mostly as an exercise in creating a simple database system that used CBM’s “REL” files (which were somewhat unreliable and REALLY slow, quite an accomplishment for the slowest disk hardware of any 8-bit micro).  All it did was assign a unique 12-digit number to every “customer”, kept a running tally (in tens of dollars, rounded down) of how much they purchased in a 48-hour period, and had a “bad customer” flag.  And then, I did something back then that today seems… prescient.

The “bad customer” flag was actually one byte in one version, later two.  It started out just being a “if this is present, decline the check” flag.  I then wrote a quick little routine that allowed for four “check customer” states: bad, approve for amount of purchase, approve for cash back, approve for cash only.  Then, somewhere along the line, I got a crazy idea: I added a “customer type” series of bits.  I envisioned initially four customer types: household, commercial, employee, and one I called “geezer”, which in my (then) 14-year-old mind I can interpet to mean “honored citizen” in our modern politically correct vernacular.

The last version of the program I edited took an interesting turn.  I can’t rightly say where I got this idea, but I apparently added a second byte to the field, and added a flag I called in a REM statement “alcoholic”.  In digging through the text file notes on the disk, here was my thinking:

Interesting idea: since the cashier is typing an approval number into the cash register (and we can cross reference the approval number to the check writer), we can probably write something to scan the cash register data at the end of the day and mark a particular customer if they purchase something specific.. say, for example, they buy a beer we can mark a flag that says “this guy buys booze, let’s send this boozehound some coupons for more booze!”  We can write flags for specific department keys or even specific items, and then set the flags at the evening reconcile based upon SKUs purchased or department keys.

I’d be remiss to not point out that this is in 1984.

We already have the makings of a great privacy-violating program right here.  This was on a primitive 8-bit microcomputer with dodgy disk hardware, a very limited BASIC programming language, and an architecture that was great for playing games, not so much for hard-core data processing.

In contrast, now that I have 30 years of computer science under my belt, and much more knowledge of what was available to a regional grocery store chain in 1984; I can see that this would have been trivial to implement using an IBM minicomputer (or, more likely, one of the clones made by NCR and the like) and the COBOL programming language.  It is likely that a minicomputer would have already been driving the cash registers: this is when bar-code scanning at the supermarket was now universal, and many of those point-of-sale systems were driven by some variant of that hardware.  Based upon my memory, I even think the “prototype” that got me thinking about this in 1984 was NCR cash registers.

Recently, a lot of people in the upper echelons of companies like Facebook, Google, and (the former) Sun Microsystems have made statements that all come down to “privacy is irrelevant.”  Looking back at what a kid with a C-64 was able to envision 25 years ago, I now totally understand what those words mean.

Okay, so you can try to live your life without rewards cards, frequent flyer programs, and no Facebook page.  In the end, however, you’re still trackable.

I recently had a conversation with a friend-of-a-friend who works for a regional retailer here in the Pacific Northwest (NOTE: not the aforementioned Fred Meyer).  I won’t mention them by name.  This retailer has a small “frequent shopper” rewards program, and also maintains a pretty impressive customer database and one wicked-cool data warehouse.  In their data warehouse, they can call up any transaction on any day anywhere in the chain in the past 10 years.  If they paid by check, there’s an image of the check.  If they paid by credit card, there’s the signature.  Every part of the transaction was captured.

What I didn’t expect was how much of the data was further mined beyond just what was on the surface.  He then showed me a pilot project that they’ve been working on that is being driven both by the marketing department and the buyers (the people who choose what products the store carries).  What they showed me proves that “privacy is irrelevant.”

They euphemistically call it ‘anonymous capture.’  What ‘anonymous capture’ does is to try to find patterns in non-loyalty transactions that allow them to identify individual customers and their buying habits without having loyalty data.  They claim that as many as 40% of these “anonymous transactions” can actually be identified to individual customers, and by closely analyzing the transactions they can collect the demographics information they are looking for without the loyalty program.

A lot of the way this system actually works is a closely guarded corporate secret.  But it’s all based on the fact that humans are amazingly predictable creatures.

He shows me the purchases of one particular anonymous customer.  He pulls up ten receipts over a two-month period, and explains which items on the receipts probably triggered the algorithm and why.  The algorithm said that these ten purchases are likely the same person: female, married, 30-45 years old, 1 or 2 children, upper-middle class income.  He then pulls a file folder out of his desk drawer of photos from the store surveillance cameras, taken at the time and date of the transactions.

Guess what?  They’re indeed all the same woman.  And in one or two of the stills, you can clearly see her two tweener children, making the age, marital status, and income bracket clearly within what the algorithm predicted.

These were cash transactions.  The system had nothing to go on other than the frequency of the purchases, the items purchased, and the times and dates of the transactions.

Then it got disturbing as he said “let’s go further down the rabbit hole.”  Now, granted, this was a demonstration: this was a repeatable result that my friend knew in advance would work.   But it is still scary.

He starts a process that mines the historical archives, looking for this “profiled customer” to see if he can ever find a name.  Sure enough, at a different store in the chain there was a debit card purchase from this same “customer” (according to the purchase profile), and it was confirmed by looking at store surveillance cameras.  The system predicted a lot more about this person at this point: once you confirmed the link in the software, the system now predicted that she worked near store #2’s location, and that she probably worked in health care.

Friend then showed me a couple of other printouts he had in the file: a Facebook page for the person (likely found by name) that clearly demonstrated these additional facts were true.

One customer, who never filled out a “rewards card application”, but has now been identified just as granularly as if she had handed this chain her Facebook page and said “go nuts.”

“But,” I hear you say, “we had to have a human involved!  Surely, that makes it not practical!”

Nope.  This was just done for this one customer (well, I’d gather, for a statistically relevant subset of customers) to “prove” the system “worked,” or more likely, to get a feel for how frequently the system “didn’t work.”

And that’s where things get a little creepy.  The system works, 100%, for gathering the data they need.

See, all they care about is the fact that this woman’s purchases give them an idea of what a 30-45 year old woman with a moderate income and two kids buys from their store.  To a large degree, even if the woman wasn’t the exact same identifiable woman with a Facebook profile it wouldn’t matter.  They’re looking for the trend, the mean.  The individual doesn’t matter.

And that’s why “creepy drugstore privacy” is a red herring.  Privacy advocates holler about their personal privacy, and they’re thinking that loyalty card programs care about capturing data about the individual. They probably couldn’t care less about you: they only want to know enough about you to figure out what bucket to put you in, and to make sense of your purchase data relevant to that bucket.

At the end of the day, there’s a huge upside to you, the consumer, of all this data mining.  I purchase probably 80% of the things I need at Fred Meyer.  I buy most of my groceries, a lot of my clothing, and all of my medicines.  I purchase the majority of my fuel from Fred Meyer stations now that I live near one.  I buy a small percentage of my media and electronics from there, but enough to give a reasonably clear picture of my entertainment habits.  From this, Uncle Freddy has a pretty clear profile of who I am and what demographics I’m in.  They probably know I’m single and male.  They know from my address I live in a modest apartment complex in a middle-income part of town.  They can probably surmise my income based upon the amount of money I spend in their store, and even what things I buy.  And my fuel purchases (mid-grade unleaded and diesel) gives them some idea that I own two cars, and they could probably accurately determine that one of them is older and/or an import.  They can probably also guess from my purchases in the Euro-food aisle (a unique feature at my Freddy’s that isn’t present at a lot of their stores) that I’m either a gourmand or of central European ancestry (and they’d be wrong there, but how wrong really?).

Point is in how this data is used.  From this data, they have a pretty clear picture of what Fred Meyer needs to do to keep my business.  Or, not keep it, if I’m not a desirable customer.  They can collect all this data from all the customers of their stores, and get a precise laser-guided missile of products to land at the store so that they have what I need at a price I’ll pay, and (perhaps more importantly) nothing I won’t buy.  Shelf space is expensive, demographics are cheap.

There’s a local legend that Fred Meyer (the man) offered to pay parking tickets for anybody who got one while shopping at his downtown store: all they needed to do was turn in the ticket at the Customer Service counter with their sales receipt and they’d be cheerfully refunded.  Meanwhile, he collected all the tickets and discovered exactly WHERE his customers were coming from, and how much they were spending at his store when they came.  Using this data, he opened a store in Portland’s Hollywood district, and became one of the Pacific Northwest’s retail success stories.

In the end, isn’t that a benefit to me?  There are downsides (and that’s a whole different discussion), but in the end, the store is there to serve me the customer.

If Fred Meyer can use the data that I’m diabetic and love chocolate to ensure that they carry more sugar-free chocolate bars, they can have that data.  Mine away, good merchant, mine away.

*: There is a system where you can load coupons onto your Rewards card and you will get the preferred pricing at checkout. At the moment, this feature seems under-marketed: it seems more like a perk for getting you to check the website than a feature of the Rewards card program, but it remains to be seen how aggressively they will market this in the future..