I am a big man and always have been, thus my signature. Whales swim far and fast and dive deep. As an instructor in UWO I weighed 220. I weighed 185 in training however and even that was a disadvantage. Just think of the simple things like the guy next to you having to pull up or push up or carry around 40 pounds less.
Lose the weight, but wisely - perhaps under the guidance of a trainer. You need to maintain your fitness/strength while losing.
Don't kid yourself into believing it doesn't matter.