sick of seeing architecture diagrams with box innocuously labelled "de-duplicate" or similar. 90% of complexity is prbly in that box.