We had the same issue when importing Contacts and trying to associate them to Accounts. We found that the duplicate detection process was also searching deactivated Accounts, and this is where the duplicate reference came into play.
In talking to our developers, the only way around this is custom code to remove deactivated Accounts from the duplicate detection search, although this may fix one issue and create another when trying to import Accounts.
Hope that helps.