Could you do boots instead of the heels, and maybe a leather jacket? It might make it a bit more casual if you're worried.