I saw officers come to SF from nearly all combat arms branches - individual ability, a positive attitude, common sense, and trainability were important qualifiers for overcoming most entry level shortcomings.
IMO - your basic branch should be the one in which you have the greatest aptitude and personal desire to serve as it will provide you the greatest chance to succeed and, if you find the SF option is out of reach, a branch in which you can offer the most for the overall good of the service.
Remember, you're considering entering a higher level of personal commitment and selflessness - your choice - just be honest with yourself and your country.
Richard