Thank you for your response Muhammad.
I've tried the solution you posted of setting the entity= 'false'. It didn't work for whatever reason. The entity is still visible.
I've also tried using the SiteMap Editor (which should be easy to use), but I can't get it to connect properly to my CRM organization (CRM Online within Office 365).
This is something that should be really easy to change, i'm puzzled as to why it's not.
Microsoft should build a sitemap editor right into the CRM application. It shouldn't require messing with import/export solutions and using third party editors.