Do it! I changed majors between junior and senior years to try to avoid having to go to grad school. Now I'm about to change back because I realized psychology is where I belong. Don't waste time trying out something you know you don't love just because what you do will take effort. Anything worth having is worth working for.