Well, you have one side of the issue here. And, BTW, I agree that HBU isn't optional. But there are interpretations, that give effectively give more weight to the SOW. You just need to google.
And this gets back to the more fundamental issue. USPAP is POORLY written. It really doesn't know how to deal with these kinds of issues in a consistent and non-contradictory way. They get these different components into the USPAP, which are quite capable of contradicting each other - but don't handle the possible eruption of contradictions. What if? What if the assignment conditions ask for the HBU to be sidestepped with respect to certain issues, e.g. an excess parcel with a different HBU than the main parcel? A simple statement that the SOW can't do this or that would be fine, - provided it is general enough to encompass all situations.