You're right, BMI is no more than a rough guide at best. Unless you use it with body fat calculations as well, it's so inaccurate that it's almost unusable as an accurate way of judging a healthy body bulk makeup.
I know of one rugby player (
Sebastian Chabal) who has a BMI of 31 but a body fat percentage of 7%. If you just knew and used his BMI, you'd say he was obese but he's actually a hulking super-fit bruiser of a man.