Kind of a sticky situation, how do I be me without being like them?

Just wear what you want and if they happen to notice and say you look a bit different, just give them a laugh and say you weren't sure what was expected so you decided to go a bit more conservative to be on the safe side. Make sure to tell them it's refreshing to see someone who enjoys your taste. They'll like that.