Black and brown can go together if you have it balanced properly.

What color shirt are you wearing? That could make the big difference.