Appraisal Statistics

[RCA]

All these posts have an element of truth to them...

Buyer's main "valuation" model comes down to their budget range. Most buyers ( except the super wealthy) are shopping within a budget what they can afford./if financing what mortgage amount they qualify for.

Within their budget range, buyers try to get the best property they can , according to which features are meaningful to them. For some buyers that might be a bigger space, others want newer, others want a view or near urban downtown etc. Many buyers of course want a combination of features but still usually have a few that are critical to them . .

Valuation issues arise because within a similar price range in the same area, different buyers pay different amounts for features important to them , while other buyers will accept a defect to get something they want. A buyer might pay 400k for a 1600 sf older house needing work because it's a block from the beach, vs paying the same 400k for a newer, 1800 sf, upgraded property a mile away inland. If both of these houses show up in a statistical search of 1600-1800 sf properties within the one mile radius, they will give false results for the other. .

Partially, yes. But I think too that appraisal has always had this problem: How can anybody verify the accuracy of an appraisal? You think about it. You have all these appraisers running around saying they don't do any regression. They just "know" the value. And people say, "Yeah my wife is alwasy looking at Zillow and all the real estate forums and guessing prices - and she is pretty damn good too. So, why is an appraiser any better -- because ...."

The whole profession gets in an uproar whenever an appraiser questions like this, - in a general non-specific sense. Of course reviewers do this under strict guidelines. And everyone else in the profession complains left and right. - I give you evidence: It is impossible to create a competition for appraisers to see who is best in appraising value for a certain property type in a certain area - because it is virtually impossible. We could do this: (1) A house comes up for sale in a certain neighborhood, (2) We get the owners permission to allow any number of local appraisers to independently appraise his house and give us an estimate of market value with a given marketing time. We then wait for the house to be sold and see who comes closest. But you know the actual sale price, if the house even sells, is to a large extent subject to chance; as well as the time it actually takes to sell the house. This is a big protection for appraisers and we know in the past many people have taken advantage of it to cheat buyers and investors out of large sums of money. ---- So, certain people are looking around for ways to give legitimacy to the whole practice: (1) Education, (2) Tests, (3) Regulations (4) Statistics and generally burying appraisers under so much rubbish they don't have any idea what the real goal is any more. ... If you could only separate out the "good" appraisers. But, you honestly can't. Reviewers are totally biased - like most people - and simply point fingers at anyone who looks suspicious (IMO) or who is politically in the wrong field at the wrong time. Well, I am exaggerating. But I think there is a bit of truth to it all.

 

[residentialguy]

Tell me exactly how someone can verify that you are a good appraiser?

By how many "likes" they have on the AF Forum. Everybody knows that.

 

[J Grant]

Partially, yes. But I think too that appraisal has always had this problem: How can anybody verify the accuracy of an appraisal? You think about it. You have all these appraisers running around saying they don't do any regression. They just "know" the value. And people say, "Yeah my wife is alwasy looking at Zillow and all the real estate forums and guessing prices - and she is pretty damn good too. So, why is an appraiser any better -- because ...."

The whole profession gets in an uproar whenever an appraiser questions like this, - in a general non-specific sense. Of course reviewers do this under strict guidelines. And everyone else in the profession complains left and right. - I give you evidence: It is impossible to create a competition for appraisers to see who is best in appraising value for a certain property type in a certain area - because it is virtually impossible. We could do this: (1) A house comes up for sale in a certain neighborhood, (2) We get the owners permission to allow any number of local appraisers to independently appraise his house and give us an estimate of market value with a given marketing time. We then wait for the house to be sold and see who comes closest. But you know the actual sale price, if the house even sells, is to a large extent subject to chance; as well as the time it actually takes to sell the house. This is a big protection for appraisers and we know in the past many people have taken advantage of it to cheat buyers and investors out of large sums of money. ---- So, certain people are looking around for ways to give legitimacy to the whole practice: (1) Education, (2) Tests, (3) Regulations (4) Statistics and generally burying appraisers under so much rubbish they don't have any idea what the real goal is any more. ... If you could only separate out the "good" appraisers. But, you honestly can't. Reviewers are totally biased - like most people - and simply point fingers at anyone who looks suspicious (IMO) or who is politically in the wrong field at the wrong time. Well, I am exaggerating. But I think there is a bit of truth to it all.

Having a contest with one house unnecessary ...esp since sales price does not always equate to market value, but when it does and a property sells within a short time after appraisal, one would expect a reasonable range of close to sc or actual sales price among the competent...but if the house had a reasonable price contract of sale not much of a test lol...

An appraiser can be seen to be "good" (or superior or lousy, ) over a period of years by the body of their work. Were they "right" in retrospect , at identifying and analyzing trends and conditions of the market cycle they were appraising in ? How often did the properties, sell within reasonable range to their opinion of market value (when the sales prices were reasonable) ? How often did they produce a stinker... (any appraiser can have one but hopefully a minimal amount )

A good appraiser reviewer is not engaging in bias or pointing fingers-- it's possible but imo absurd that as ongoing practice since others can "review" the review! And can hold the reviewer accountable and rebut their findings. A credible review, like a credible appraisal, is self explanatory on read with conclusions that are well supported and explained..

Seems now clients have the best of both world since they have access to data, statistics and AVM's and run them as a "Check" on appraisals...

Neither an appraisal, nor a computer/statistically derived valuation ever be the 100% agreed on as "accurate" or the "one" . There is always somebody who can challenge results, and a property can sell at a different price than the value opinion or estimate,

The more interesting question might be to compare computerized results vs appraiser results in the market and see which are "right" or reliable for use over a period time over different market cycles. If both appraisals and computer results perform reasonably well for their purposes they are best off left as they are, products that can be used to vet each other or augment each other.

The one thing we can say about an AVM or statistical product is that it has no agenda or bias influencing results. But the problem is it also lacks insight or perspective.. Thus results, even though mathematically "accurate" can be "dumb" or "off".

An appraisal (or evaluation) is a market problem to be solved, not a math problem to be solved.

Math is a tool used in solving a market problem but not a stand alone solution, because of the "dumb" data in data out aspect. The issue is people are dazzled by the evidence trail of math since it ends up proving, or "supporting" its own results.. but only afterward, in the real market place or among human activity can the results be experienced as reliable or not.. Such as fake news or photos can appear to be real, and financial statements from a Madoff can look convincingly right ...imo with mass data or platforms that can widespread bad data results such as Zillow,/internet people with market experiences perspective and insight ( appraisers) will be more needed than ever to vet the data and wade through results. Whether clients want to pay/allow time for thatt is another matter.

The scary trend among fannie /lender /AMC interests is not just control over the data but control over the appraiser, making the appraiser nothing more than a marginalized participant producing at warp speed, not even inspecting the property in a hybrid and economically vulnerable to the very same client influence the appraiser was supposed to be in a position for an independent opinion/analysis of...it simply is not possible in an appraiser whether fee or staff under constant pressure for volume and speed production and ranked by scorecards performance to be "independent. With an AMC the algorithm or computerized performance rankings, based on service ,deadlines,fees and revisions along with some kind of quality "Metric" is the gateway to receiving orders. Under this system, an appraiser who hands in a report s six hours later due to further analysis/verification is lower in performance rankings ( even if appraiser let the client know time).

Statistical results can be used well by appraisers who understand them, or as a data dump and sign by appraisers working at speed and volume..

 

[Deleted member 130081]

All these posts have an element of truth to them...

Buyer's main "valuation" model comes down to their budget range. Most buyers ( except the super wealthy) are shopping within a budget what they can afford./if financing what mortgage amount they qualify for.

Within their budget range, buyers try to get the best property they can , according to which features are meaningful to them. For some buyers that might be a bigger space, others want newer, others want a view or near urban downtown etc. Many buyers of course want a combination of features but still usually have a few that are critical to them . .

Valuation issues arise because within a similar price range in the same area, different buyers pay different amounts for features important to them , while other buyers will accept a defect to get something they want. A buyer might pay 400k for a 1600 sf older house needing work because it's a block from the beach, vs paying the same 400k for a newer, 1800 sf, upgraded property a mile away inland. If both of these houses show up in a statistical search of 1600-1800 sf properties within the one mile radius, they will give false results for the other. .

I really like this post. You touch on something appraisers and lenders don't like to think about or admit, which is that each individual values different elements differently. That concept really throws a monkey wrench (further) in the accuracy of dollar adjustments and sheds light on how the mechanical and formula approach to appraisal is questionable from the get-go.

I think appraisal can be like grabbing a handful of sand. The harder you squeeze it, the less you have. IMO, the best appraisals occur when an experienced appraiser simply puts themselves in the shoes of the buyer and market, and approaches the valuation in the way the buyer would, as best they can (like a personal property appraiser would). This sort of thing requires an appraiser to act as a human, which of course is difficult and often impossible to put down on paper. Because of that difficulty, and the requirement to "prove" all we do, appraisers instead work it backwards, starting with the raw data and trying to plug in the magical formula that explains the universe. The problem I see, is the unrealistic expectation of support that is typical with lending clients. That is the driving force behind how we do things, with the unintended consequence being everyone thinking it has to be done that way every time.

I love doing non-lender assignments for "regular" people. People who don't need or even want a 50 page report and every little detail explained or proven. They just want an appraisal that is as good as it can be, for a feasible price. It is amazing when I describe to them what I can do, and what I cant do, and how lending appraisals are silly, that they all agree and understand instantly.

IMO, too many people fail to call a spade a spade.

 

[RCA]

I really like this post. You touch on something appraisers and lenders don't like to think about or admit, which is that each individual values different elements differently. That concept really throws a monkey wrench (further) in the accuracy of dollar adjustments and sheds light on how the mechanical and formula approach to appraisal is questionable from the get-go.

I think appraisal can be like grabbing a handful of sand. The harder you squeeze it, the less you have. IMO, the best appraisals occur when an experienced appraiser simply puts themselves in the shoes of the buyer and market, and approaches the valuation in the way the buyer would, as best they can (like a personal property appraiser would). This sort of thing requires an appraiser to act as a human, which of course is difficult and often impossible to put down on paper. Because of that difficulty, and the requirement to "prove" all we do, appraisers instead work it backwards, starting with the raw data and trying to plug in the magical formula that explains the universe. The problem I see, is the unrealistic expectation of support that is typical with lending clients. That is the driving force behind how we do things, with the unintended consequence being everyone thinking it has to be done that way every time.

I love doing non-lender assignments for "regular" people. People who don't need or even want a 50 page report and every little detail explained or proven. They just want an appraisal that is as good as it can be, for a feasible price. It is amazing when I describe to them what I can do, and what I cant do, and how lending appraisals are silly, that they all agree and understand instantly.

IMO, too many people fail to call a spade a spade.

Well I didn't say anything about the "mechanical and formula approach." If you mean by the latter "statistical approach" then that is primarily for the "Sales Comparison Approach" - which has to be reconciled in varying degrees with the Cost and Income Approach. But for lending - the Sales Comparison Approach is given by far the most value. So, that is why we are giving it special concern.

What I was saying is that if you take 10 appraisers and have them appraise, based on statistical methods, a SINGLE home that is going up for sale, and then compare their opinions of value with the actual sale price, you still won't know which appraisal is more accurate really - because the actual sale price has elements of randomness thrown in, as well as hidden "covariate" and "confounder" variables - variables that affect price but are not available for analysis. Now if, on the contrary you could have them all appraise 20+ homes, the same homes at the same point of sale, and then averaged how far they were each off - then you would have something that is useful. BUT THAT IS (VIRTUALLY) IMPOSSIBLE.

 

[Elliott]

Bert said, "if you take 10 appraisers and have them appraise, based on statistical methods, a SINGLE home that is going up for sale, and then compare their opinions of value with the actual sale price, you still won't know which appraisal is more accurate really - because the actual sale price has elements of randomness thrown in, as well as hidden "covariate" and "confounder" variables - variables that affect price but are not available for analysis."

If you take one appraiser who appraised it in accordance with USPAP, who didn't base it on statistical methods, you'd have a supported opinion of value, and that is what appraisers are expected to do.

 

[George Hatch]

WRT value conclusions I aspire to "reasonable", not "accurate".

 

[RCA]

Bert said, "if you take 10 appraisers and have them appraise, based on statistical methods, a SINGLE home that is going up for sale, and then compare their opinions of value with the actual sale price, you still won't know which appraisal is more accurate really - because the actual sale price has elements of randomness thrown in, as well as hidden "covariate" and "confounder" variables - variables that affect price but are not available for analysis."

If you take one appraiser who appraised it in accordance with USPAP, who didn't base it on statistical methods, you'd have a supported opinion of value, and that is what appraisers are expected to do.

I'm sure many assignments can be handled without statistics. For adjustments I think you can still use matched pairs analysis. However, it falls under the concept of "overfitting" - although if you only have a few good comps, that may be your only choice.

There are many parts of USPAP however, that strongly support the argument of using statistics where it improves the credibility of results. Throughout USPAP we find statements like "... an appraiser must avoid making an unsupported assumption or premise about ...", "... supported by an appropriate analysis of such data...".

In any case, it is just a question of time before companies like HouseCanary work out the kinks and supply you a MARS type model with the exact adjustments you need to "consider". Most appraisers however will not be in a good position to argue with them - unless they are pretty good at statistics. HouseCanary type tools will invariably fail because they don't have good data. That will be a systemic problem.

Some of us are interested in trying to understand everything, some not.

Also refer to the new Advisory Opinion 37: "Statistical tools may be employed to support adjustments. Because of the number of independent variables required for analysis of both residential and non-residential properties, a useful statistical tool is multiple linear regression. ... [ and here it goes IMO a bit off course about sample size ...]".

Notwithstanding, there are many other things that play a role in developing a value conclusion like zoning, erosion, etc..

 

[Eli]

WRT value conclusions I aspire to "reasonable", not "accurate".

Ok, can I inspire you to an opinion of more reliable and credible as a member of society?

See, I know where you go. Intended use and user.

You realize the dichotomy in your argument.

It’s like an excuse GH. If public trust is the goal. Be all we can be.

 

[RCA]

WRT value conclusions I aspire to "reasonable", not "accurate".

Semantics. "Accuracy" could be defined as how close an appraiser's opinions of value match up to subsequent associated sale prices over a number of appraisals. There are different standards of accuracy, that allow someone to say that some value is "accurate" if it is within a certain distance of the target, however you define the target. Some appraisers are happy with getting their adjusted sales comps within 5% of each other using accepted methods of calculating and applying adjustments. Others may target 1%. There is nothing wrong with targeting 1%, even if 5% is considered "reasonable". I think it means something if you can adjust your comps to get them that close using statistics alone - and that means something. You can do this without statistics - but it can be very difficult, especially if you keep adding on more comps. If you have a good MARS model for calculating adjustments, you can throw a hundred comps into your grid and still have a tight range of adjusted values. ...

In many cases, of course, we can't even be sure we are within 10% or worse, because of a lack of good comps, lack of history, erratic market conditions.

We are trying to find some certainty in a sea of uncertainty - and statistics can help a lot in many cases.