"Point is, the only way to find out is to compare houses that sold with pools to houses that sold without them. That is the right answer and every other answer is wrong."
The above poster was dead on right and that is the answer to your loan officer.
Based on the poster's to this thread it looks to me like the central and upper midwest just does not value pools. Many reasons, but most probably due to limited season of use and expense of upkeep.
Whereas in the Sun Belt states, CA, AZ, NV, NM, TX and FLA due to longer warm seasons pools can be amentities that add value. In my part of CA the San Joaquin Valley, where we have 6 to 8 months a year of warm weather and 3 to 5 months of downright hot then pools have a value, but not as much as the cost to install. Generally over the years I have found that a pool adds roughly $10/sf to the value of the home, in the very hot summer time this value may reach $!5/sf. During the cold foggy months maybe $5/sf. You can see the seasonal market aspects of a pool amenity.
What is really funny to me is a home sold in Jan. cold and foggy month may only have a $5/sf added value, but with a typical 90 day escrow the sale does not close till April right at the beginning of the warm months, good deal. Alternantly, an Aug sale, very hot month, $15/sf for pool, same 90 day escrow move in Nov. cold time the buyer gets no immediate use for an item that may have cost him $20,000 or more, all he gets is the expense of upkeep and electrical bills for 3 to 4 months.