Tesco is quietly building a profile of you, along with every individual in the country - a map of personality, travel habits, shopping preferences and even how charitable and eco-friendly you are. A subsidiary of the supermarket chain has set up a database, called Crucible, that is collating detailed information on every household in the UK, whether they choose to shop at the retailer or not.
The company refuses to reveal the information it holds, yet Tesco is selling access to this database to other big consumer groups, such as Sky, Orange and Gillette. "It contains details of every consumer in the UK at their home address across a range of demographic, socio-economic and lifestyle characteristics," says the marketing blurb of dunnhumby, the Tesco subsidiary in question. It has "added intelligent profiling and targeting" to its data through a software system called Zodiac. This profiling can rank your enthusiasm for promotions, your brand loyalty, whether you are a "creature of habit" and when you prefer to shop. As the blurb puts it: "The list is endless if you know what you are looking for."
This publicity material was, until recently, available on the website of dunnhumby, but now appears less forthcoming. Attempts by a number of Guardian reporters to retrieve their own personal information under the Data Protection Act led to a four month battle; the request was ultimately denied so the Guardian has appealed to the Information Commissioner. Tesco has provided some personal data held by Clubcard, the loyalty scheme that monitors members' shopping and which has been credited with fuelling the supermarket group's astronomical growth in the past decade.
But as far as Crucible is concerned, the company admits it has "put great effort into designing our services" so information is classed in a way that circumvents disclosure provisions in the Data Protection Act. Clues about the content of dunnhumby's database have appeared in the company's marketing literature. Crucible, it says, is a "massive pool" of consumer data. "In the perfect world, we would know everything we need to know about consumers. We would have a complete picture: attitudes, behaviour, lifestyle. In reality, we never know as much as we would like." But Crucible, it suggests, has got much further than rival systems by pooling data from several sources and then using the vast Clubcard data pool to profile customers.
Together, Crucible and Zodiac can generate a map of how an individual thinks, works and, more importantly, shops. The map classifies consumers across 10 categories: wealth, promotions, travel, charities, green, time poor, credit, living style, creature of habit and adventurous.
A "Mrs Pumpkin" is cited: she makes pennies work when she shops, mostly uses cash, has a steady repertoire of products but experiments with the new, shops at various times, spends a little more on eco-friendly items, is involved with charitable giving, is rarely away and likes promotions for things she buys.
How does Tesco get the information? Clubcard is used to target promotions at particular cardholders. But Crucible is separate and Tesco insists that while loyalty scheme data is used by Crucible it does so anonymously rather than a house-by-house, name-by-name basis.
Dunnhumby's chairman, Clive Humby, offers a few more clues. Companies such as Experian, Claritas and Equifax have databases on individuals and Crucible collects from them all. Any questionnaire you may have completed, any reader offers you responded to, are bought to build up a picture of attitudes and habits. Crucible also trawls the electoral roll, collecting names, ages and housing information. It uses data from the Land Registry, Office for National Statistics and other bodies to generate a profile of the area you live in. Zodiac is employed to provide a more detailed profile. The combination is valuable to many consumer goods firms: dunnhumby generated profits of £4m on sales of £28m in the last year for which accounts are available. Some £12m of business was done directly with Tesco.
Mr Humby and Edwina Dunn founded dunnhumby. The two have a reputation as shrewd operators in the marketing industry and still own shares in the firm alongside Tesco's majority stake. How the supermarket group and other customers use the data is less clear. One former employee involved in the company's marketing told the Guardian that it can be used to decide how to target offers to individuals or where to open new stores.
A Tesco spokesman said last night: "All work carried out by dunnhumby is regulated by the Data Protection Act and the Direct Marketing Association Code of Practice." But, as the supermarket unveils yet another set of sparkling half-year figures today, one thing is clear: while past success may have been built on the company knowing its customers, Tesco plans to secure its future by knowing everyone else's customers as well.
Profile of an upmarket C10 deserter
When it comes to my personal information, I'm a natural paranoid. So when signing up for a Tesco Clubcard to get those cashback vouchers and offers, I made a point of providing as little information as the application would allow.
No matter. According to Tesco's disclosures under the Data Protection Act (DPA), in the year my card was in use the supermarket managed to build a substantial - if rather wayward - portrait of this reluctant shopper's habits. A formal DPA request, followed by numerous letters to and fro, a terse telephone conversation and finally, a fax explaining that, yes, this information would be used in a journalistic exercise, finally produced two sides of information.
Apparently, I'm a gal who hankers after "finer foods"- indeed, a "natural chef", though friends tell me this probably has more to do with my tendency to cook with natural ingredients than any signs of being a budding Nigella. I am, Tesco determines, "upmarket" - a reference, I suspect, to my habit of buying organic food (Green & Blacks mint chocolate being a particular favourite).
The database defines me through the past four years, placing me in the mysterious "C10" category for 2003, having been an "H13" a year earlier - whatever that means. My "family type" is "other," though alternative social options are not listed. Most importantly for the supermarket, I just don't spend as much as I could there. Under "share of spend" with Tesco I am deemed to have "potential".
My household carries a "reference number", the date of my last visit, with branches used in the past. It says whether I have used Clubcard vouchers and correctly states I do not want my personal information to be passed to other parts of the "Tesco Group". There is no information as to whether I am diabetic, teetotal or have a special diet.
Five slots describe my "shopping habits", each carries the words "Not shopped in last eight weeks". Clearly, I'm a Tesco deserter and a prime candidate for those £10-off vouchers that have been dropping through the letter box of late.
· To learn how to get your personal information under the Data Protection Act, see www.theguardian.com/foi