Sounds like you may have an underwriter that thinks an adjustment has to be above a certain percentage...some appraisers around here will not make an adjustment unless it is at least 2% of the sales price of the comp to which it is to be applied.
I don't necessarily blindly buy into that methodology, or it isn't applicable in every case anyway, and have to spell it out sometimes in the report. If you have support for your adjustment, tell the UW it is a supportable adjustment and move on. Otherwise, you may need to supply a comp with a similar set of outbuildings or find a sale that isn't really comparable but has a pole barn and run down shed and attempt to extract their value...
FYI, in our market, pole barns quite often have limited contributory value. Usually, they only really contribute if they are a genuinely useful amenity (horse farm, working ag farm, etc). Otherwise, depending on size, they are often equated by buyers to these cheap aluminum "car covers" that approximate detached carports. I think the double-car carport versions of those are selling brand new around here for $595 installed.