Appraisal Statistics

[George Hatch]

I think a lot depends on what is "meaningful" to the intended users. Credibility is always judged within the context of the intended use. Says so in the book. The book says nothing about "results must be accurate".

If I think "accurate" is $300k but the property is in escrow for $295k or $305k, how meaningful is my accuracy within the context of their intended use? Same holds true for reviews, only moreso. If I think $300k is "accurate" but the original appraisal is for $295k or $305k and I'm an accuracy fundamentalist then I am faced with the dilemma of either saying the appraisal is either accurate (which I don't actually believe) or inaccurate (which I do believe) because there can be only one.

I mean, we round value conclusions for a reason. Nobody who knows anything about appraising takes Zillow seriously in part because they don't round their Zestimates.

Moreover, I already mentioned that buyers and sellers and brokers use valuation models. Inasmuch as they back their value conclusions with their own money and we use those conclusions in our own work the prospect of calling them all wrong kinda creates a feedback loop for us, doesn't it?


Those valuation models don't include a ficticious "adjustment" that we appraisers seem to be depicting as if that's what they're doing. So when there's *user-added* noise in the raw data our attempts to be more "accurate" than the market creates an unreasonable expectation that actually can undermine our credibility. Was that $305k transaction a poor expression of Market Value or not?

Besides, what happens IRL when the combination of adjustments covering those pesky covariables produces a sub-optimal range of adjusted value indicators? What does the appraiser do with that? They have to revert to using more qualitative analysis - like the buyers and sellers use - to reconcile for value. Done properly that qualitative is just as valid as what the market participants are doing, if not moreso as a result of working with a (appraiser-defined) tighter range to begin with. Even if that range could have been tighter if we had refined it more.

IRL, most of the valuation of an SFR comes from qualifying your subject and the comparables sufficiently to identify exactly which of them are more similar and comparable overall. Not when we quibble over whether the GLA adjustment on that 1,300sf comparable should have been $57 instead of $60.

Just because we can put the monkey in a silver suit doesn't make him an astronaut. Just because I say the $305k contract price is inaccurate doesn't make it unreasonable within the context of the intended use of my assignment.

 

[DWiley]

I think a lot depends on what is "meaningful" to the intended users. Credibility is always judged within the context of the intended use. Says so in the book. The book says nothing about "results must be accurate".

If I think "accurate" is $300k but the property is in escrow for $295k or $305k, how meaningful is my accuracy within the context of their intended use? Same holds true for reviews, only moreso. If I think $300k is "accurate" but the original appraisal is for $295k or $305k and I'm an accuracy fundamentalist then I am faced with the dilemma of either saying the appraisal is either accurate (which I don't actually believe) or inaccurate (which I do believe) because there can be only one.

I mean, we round value conclusions for a reason. Nobody who knows anything about appraising takes Zillow seriously in part because they don't round their Zestimates.

Moreover, I already mentioned that buyers and sellers and brokers use valuation models. Inasmuch as they back their value conclusions with their own money and we use those conclusions in our own work the prospect of calling them all wrong kinda creates a feedback loop for us, doesn't it?


Those valuation models don't include a ficticious "adjustment" that we appraisers seem to be depicting as if that's what they're doing. So when there's *user-added* noise in the raw data our attempts to be more "accurate" than the market creates an unreasonable expectation that actually can undermine our credibility. Was that $305k transaction a poor expression of Market Value or not?

Besides, what happens IRL when the combination of adjustments covering those pesky covariables produces a sub-optimal range of adjusted value indicators? What does the appraiser do with that? They have to revert to using more qualitative analysis - like the buyers and sellers use - to reconcile for value. Done properly that qualitative is just as valid as what the market participants are doing, if not moreso as a result of working with a (appraiser-defined) tighter range to begin with. Even if that range could have been tighter if we had refined it more.

IRL, most of the valuation of an SFR comes from qualifying your subject and the comparables sufficiently to identify exactly which of them are more similar and comparable overall. Not when we quibble over whether the GLA adjustment on that 1,300sf comparable should have been $57 instead of $60.

Just because we can put the monkey in a silver suit doesn't make him an astronaut. Just because I say the $305k contract price is inaccurate doesn't make it unreasonable within the context of the intended use of my assignment.

The sooner we eliminate "accurate" from the appraiser vocabulary, the better. What does it even mean to have an "accurate" opinion? Are my opinions always 100% accurate, because they are, with certainty, my opinions?

 

[Pittsburgh Pete]

Always felt my job was to "get close" to the market value. Never thought an exact or accurate answer was possible.

 

[J Grant]

Moreover, I already mentioned that buyers and sellers and brokers use valuation models. Inasmuch as they back their value conclusions with their own money and we use those conclusions in our own work the prospect of calling them all wrong kinda creates a feedback loop for us, doesn't it?

Their valuation methods ( RE agent or CMA based if using an agent) is not so much "wrong" as geared toward their goal...selling at higher price for seller, for borrower leveraging as much financing as they can get, and for a RE agent getting the deal closed to earn a commission. It's good for us to be aware of their reasoning, and see the CMA or "Comps" an agent pulled to show borrower (or show us) to see how it agrees or disagrees with the value and opinions we develop. Calling them all wrong from outset creates a feedback loop but so does calling them right from outset create a feedback loop.

Accurate in an appraisal value opinion is a contradiction and I don't know why it is on the URAR form. But it is. The fact that a point value has to be one precise, specific number is not the same as saying it is an "accurate" number...The accuracy would have to be compared to a benchmark to be accurate or not, and the whole point of a market value opinion is the value is developed from the appraisal, therefore there is no zero tolerance numerical benchmark for it...unless an appraiser uses the SC price for that, but doing so , appraisal made to reaching a predetermined value or outcome in the certs is a no.

The only way the term accurate makes sense is to describe how the appraisal was developed..as accurately as possible, given time and resource realities. There are more multiple meanings for the word accurate besides mathematical precision, among them, truthfully and carefully,

For appraisals, imo word credibly supported rather than reasonable is the better standard, since reasonable is a very loose umbrella and almost anything short of the outlier or inane can be said to be reasonable. The appraiser develops the market value opinion when doing the appraisal through the steps, research and different approaches, and in turn that work itself, whatever it consisted of supports the value opinion.

 

[Terrel L. Shields]

What does it even mean to have an "accurate" opinion?

Fannie Mae requires the appraiser to provide complete and accurate reports
he or she provides all of the information required by the forms and presents the applicable data accurately
This analysis will result in more accurate reporting on market conditions,
he or she is providing in the appraisal report is accurate
appraisers to provide them with thorough, accurate, and objective appraisal reports that result in reliable opinions of market value
lack of knowledge and/or experience can lead to inaccurate property valuations
help to ensure the accurate valuations and appropriate appraisal practices​

It's like ****. Fannie knows it when they see it. Otherwise, they have nary a clue what it means

 

[J Grant]

Always felt my job was to "get close" to the market value. Never thought an exact or accurate answer was possible.

As many agree, accurate and an opinion are not compatible. Imo accurate was incorrectly used as a proxy for a one numerical point value which is the specific number $ amount.

As Terrell posted, the expectation of accurately developing the appraisal is a goal that makes more sense then expecting the point value itself be "accurate."..

Of course a range of probable market value exists around a point value. Only appraisers seem to agonize about this. Since we do have to select and commit to a point value in most assignments, be comfortable you can defend and explain it as your best credibly supported and most probable per the MV definition used.

 

[Deleted member 130081]

Semantics. "Accuracy" could be defined as how close an appraiser's opinions of value match up to subsequent associated sale prices over a number of appraisals. There are different standards of accuracy, that allow someone to say that some value is "accurate" if it is within a certain distance of the target, however you define the target. Some appraisers are happy with getting their adjusted sales comps within 5% of each other using accepted methods of calculating and applying adjustments. Others may target 1%. There is nothing wrong with targeting 1%, even if 5% is considered "reasonable". I think it means something if you can adjust your comps to get them that close using statistics alone - and that means something. You can do this without statistics - but it can be very difficult, especially if you keep adding on more comps. If you have a good MARS model for calculating adjustments, you can throw a hundred comps into your grid and still have a tight range of adjusted values. ...

In many cases, of course, we can't even be sure we are within 10% or worse, because of a lack of good comps, lack of history, erratic market conditions.

We are trying to find some certainty in a sea of uncertainty - and statistics can help a lot in many cases.

Or, maybe diving too deep into statistics results in additional confusion. Don't get me wrong here, I am all for crunching numbers. Heck, I even like doing it. The thing is, and you have made this point for me in your posts, there is a serious reasonable doubt to the credibility of results when it comes to using statistics. My point is therefore, it is dangerous and reckless to promote their use too heavily, without also being up front about their reliability. This last part is truly the problem. You see it, I see it, plenty of appraisers see it but, do our clients see it? Do appraisers do a good job of communicating the credibility of their valuations to clients? I think the ugly answer to that is often not. The result is then, that clients believe these sorts of things are reliable, and then demand they be employed. The sales grid in lending assignments is the greatest example of this, and is even another degree further into the problem. The further problem is that Lending clients designed the forms and subsequent methods employed, not appraisers. Because these are the most widely used formats, most people are familiar with them and accept them as the norm. The result is pretty much everyone thinks adjustments are an accurate thing (dumb appraisers too), because too few of us point out that they are not. Why don't we speak up? LOL - Well, that goes right back to who designed the forms and most significantly, who pays for them to be used! USPAP says we must reconcile the methods and techniques used in the appraisal. It stops short of specifically saying we must communicate the reconciliation to our clients in our reports, however considering the rest of the USPAP, this ought to be a given (and considering appraisers like to split hairs when it comes to the meaning of words and text, it ought to be spelled out, IMO). When was the last time you reviewed a report that had a statement concerning the unreliability of adjustments on a sales grid? LOL - Pretty much never right? Instead, appraisers fluff it up with all sorts of nonsense, including citing the use of statistics. Appraisers like to complain about all sorts of problems from the client side. I think this is a case where appraisers ought to take a look at their own behavior and make some changes.

So, I'll end where I started. Statistics can be useful, but it is not a good idea to act is if they are the solution to the problem of adjustments, when all they really are is an available tool that works once and a while. I think you misrepresent the use of the word "many", especially when you admit there is no way to test the reliability. IMO, the word "few" is more appropriate. But again, I am open to being wrong.

And because I like to write too long I'll add this, we are called Independent Fee Appraisers, not Value Calculators.

 

[George Hatch]

Regardless of how it got there, "accuracy" is still on the Fannie form because they're bankers, not appraisers. Every time they go for a form revision the appraisers bring the accuracy issue up and they still continue to misuse the term.

But lets not kid ourselves what the bankers mean by accurate - THEY don't think that $300k appraisal is accurate in relation to a $305k contract price and that it's the contract that's wrong. And neither does anyone else. Only appraisers indulge in that conceit.

Regardless of the model, data qualification is a signficant issue. Most of us spend way more time/effort qualifying our data than actually going through the mechanics of analyzing it. Just sayin'.

ZAIO used a lot of data to build models, but they at least qualified it. That's what consumed most of the franchisees' time and effort.

 

[J Grant]

The problem with statistics imo is they give a false reading of "accuracy". The strength of math and statistician is also their weakness when applied to human behavior and markets such as RE influenced so greatly by trends, financing interest rates and the variables among properties, participants and transactions. Statistics leaves a number trail that supports its own conclusions . Which is why some think it is superior. Yes it can show data driven "support", but to what end? If the results are "off" regarding what the market would most probably return, who cares how well "supported" they were.

 

[J Grant]

Regardless of how it got there, "accuracy" is still on the Fannie form because they're bankers, not appraisers. Every time they go for a form revision the appraisers bring that up.

But lets not kid ourselves what the bankers mean by accurate - they don't think that $300k appraisal is accurate in relation to a $305k contract price. And neither does anyone else.

Regardless of the model, data qualification is a signficant issue. Most of us spend way more time/effort qualifying our data than actually going through the mechanics of analyzing it. Just sayin'.

ZAIO used a lot of data to build models, but they at least qualified it. That's what consumed most of the franchisees' time and effort.

?? Are bankers and the" neither does anyone else " the people who get to opine the MV opinion, or does the appraiser ? If they decide the MV opinion of 300k is not accurate compared to a 305k SC price, other than the fact that a buyer signed it, then they need to prove it. Most RE agents when the SC price is inflated above MV can not prove it...all they do is send in superior higher priced sales and jump around saying the market is hot.

BTW I have never had problems if I come in a small increment like above relative to a SC , the parties re negotiate down to it or to their own compromise around it. Or of course a buyer can put more cash down.

Clearly whenever a 305k SC price is reasonable, supported an appraiser will opine at 305k (or above, ) The nonsense that 305k was in the adjusted value range of the comps but appraiser played god and opined 300k is BS. I

AMV's , zestimates etc use lots of data but the weakeness is not qualified. Fannie;s own data bank from UAD is qualified, yet the "comps" they send in a CU can contain some good ones, but also a large number of "dumb" ones. Yes those sales mechanically matched a criteria of in a mile and a year back and % of sf to subject, but they can also be "wrong" ...such as they are in a non age restricted community and subject was age restricted, subject had a defect none of the fannie CU comps have etc. If the best, most qualified data bank can still get it wrong what does that say about the limitations of the process.