This is because the to/from fields on an email are technically different from the customer field on the case.
The to/from fields are known as activity parties - they can handle multiple values (account, contact, other email enabled records) while the customer field on the case can only handle accounts and contacts. The workflow GUI prevents errors by limiting you to only map same type fields. Like you can't map a text field to a whole number field - that just would't be valid.
A custom workflow activity might be in order here to convert the activity part item to an account or contact.