Experience Platform · Profile record ingestion
The load that wiped the names
At 02:00 the daily ecommerce batch lands in the customers dataset, which uses the XDM Individual Profile class and is enabled for Profile. It carries one change: Laura has a new email. By morning, her profile has no first name. Nobody deleted anything.
Example assumptions
Customer, datasets, values and volumes are made up.
Customers dataset
Laura's profile, yesterday and today
Attribute by attribute: what was there, whether the 02:00 batch carries it and what is left. In red, what gets wiped.
Journey Optimizer · website
What happens at 09:00 in AJO and on the website
Three moments in the morning. The journey email is personalised with {{profile.person.name.firstName}}.
The whole batch
And it is not just Laura
The same happens to every customer in the batch.
The recommendation
Upsert on profile datasets, one source per dataset, and test it first in a sandbox
Standard ingestion is not a bug: it is the documented behaviour. The problem is not deciding how each source should come in before connecting it.
- 01
Decide how each source comes in
Before connecting it: whether it sends the whole record or only what changed. The dataset set-up depends on that.
- 02
Upsert where partial changes come in
In batch, a dataset enabled for Profile and for upsert. In streaming, Data Prep partial updates. That way each source sends only what changed and does not empty the rest.
- 03
One dataset, one system writing to it
The club tier, in its own dataset, written only by the loyalty programme. Real-Time Customer Profile combines the fragments with the merge policy, and no load overwrites another system's data.
- 04
Test it in a development sandbox
A test batch with a single record, then check the profile in the Profile viewer: which attributes end up null. Ten minutes before connecting the source in production.