Give me a break

[DWiley]

That is excellent you have a MAPE of about 7%.

If you think a MAPE of 7% is good, then how would you characterize a model with a MAPE that is less than half of that?

 

[RCA]

I'm pretty sure the model picked up some significant splines, especially during the 2005 to 2008 range.

I do a number of these models, going back to different periods. Yes, you have a good argument that when you go back in time, sale age becomes a more important factor in the variance of prices, and thus a high R2 may simply be reflecting more how well your model reflects the change in price with respect to time rather than other features such as GLA. That is true, but unavoidable because it is still necessary to get a good resolution on current property characteristic differences. When you go back in time you are doing that for only one reason - to get a greater variety of homes that differ in the composition of bedrooms, bathrooms, GLA, Lot Size and so on. Another way to put this is if I do a regression only going back two years, more of my R2 relates to how well the model does in reconciling the impact of features other than sale age on price than a model that goes back 15 years. Yet when you apply the first model to the last year's comps, you will find it doesn't do as good of a job reconciling differences. More importantly, with fewer samples, you are going to wind up overfitting your model so that it is far weaker in predicting what a similar home would sell for if say a different buyer came along. My experience, is that MARS in any case can do a pretty good job of removing the impact of time on sale prices. IF you believe that there are factors occurring over an extended period of time that impact buyers tastes and market value with respect to home style, GLA, bath count and so on, you can a 2 or 3 way regression that just includes SaleAge vs GLA, SaleAge vs LotSize, SaleAge vs Bathrooms and so on. Then you will get a more complex model. I always run a number of these MARS runs with 2 and 3 way interaction if I suspect something like that is going on.

But, to be truthful, I would never put 3-way interactions in a report, they are too hard to explain. And I would very much try to avoid putting even 2-way interactions in the report. On this latest report, yesterday, I reran without the 2-way interaction and came up with a tighter fit of 8 comps all under 0.20% deviation from the average. I find this hard to believe myself.

To reiterate a previous post, I emphasize that with respect to the comps, I always let the buyer decide what the subjective features are worth. That means, I use that residual to get the value of all features not covered by the first stage regression. I have a special Excel template that shows all the comps. It jumps to a table that has all my CQA rating 0.0, 0.5, 1.0, .... 10.0 next to the adjust amount based on my scoring of the 220+ comps. I put in the scores for the subject in the first column, the CQA generated based on the residuals for all of the comps going into the sales grid, and then go in and factor those out into individual adjustments. The factoring is not important in the final adjusted sale price, its only purpose to to explain to the reader what it is between each comp and the subject that leads to the adjustment. But look, if I think the patio is average looking from looking at the MLS pictures for the comp, but the buyer has paid a lot more for the property than predicted by the model, I may very well have to rate that backyard higher than I otherwise would simply based on my observation of pictures. To reiterate, the buyer has absolute control over the total of all subject value contributions, I only divvy them up based on my analysis. The actual subject feature adjustment is

[Comparable Feature Adjustment] = [Subjective Feature Value Contribution] - [Comparable Feature Value Contribution]

What is absolutely critical to everything here, is my subjective assignment of a CQA score to the subject. But only the buyers control the comparables. That is the difference between the way I do things and the way most other appraisers do things. Most other appraisers will fly by the seat of their pants on the the comparable adjustments - and that is a major source of inaccuracy for them.

 

[RCA]

BTW here is a pic of the interactions matrix, which shows how you can specify that you want a model of specific interactions:



And in MARS you can rotate this 3D around on the X,Y, Z dimensions to see exactly what it is doing, and here you are getting the value contribution of both the bathrooms+saleage where they interact together (I am not putting this in my final report, but it does shed light on that drop in price going from 2.5 to 3 baths that is occuring with more recent sales. You might ask: Why?

 

[djd09]

Refinance Appraisal Waivers Gain Popularity During the Pandemic​

Home values have skyrocketed in the course of the pandemic. In December of 2020, the median price of a home sold was nearly 13% higher than where it stood in December of 2019, according to the National Association of Realtors. As such, many homes are likely to appraise for much higher than they normally would, and so lenders aren't bothering with the formal process.

Refinance Appraisal Waivers Gain Popularity During the Pandemic

Refinancing usually involves a home appraisal. But these days, more and more lenders are waiving that requirement.

www.fool.com

price setting is not difficult

 

[djd09]

do the mortgage brokers sign a certification.

 

[Mike Ault]

No kool-aid involved, just data. You know, that stuff we appraisers are supposed to analyze in support of our opinions. . Do you have any actual data (data, not anecdotes) related To AVM performance?

I would say that given your involvement in the development of the AI Residential Database you thought that AVMs were a great idea well prior (by at least 10 years) to any actual performance data showing the same. They certainly may be great predictors of value but who is running them? And do we trust them with the economic well-being of our nation or are they making profit driven (a.k.a. efficiency) decisions that pose systemic risk again? I don't believe that it is a good idea let someone value there own collateral with no oversight other than some portfolio risk management section (who can be easily replaced if they get uppity) and then sell those securities as investment grade. We know how well the rating agencies did when they were supposed to downgrade risky securities prior to the last crash, like most people in this business they followed the money.

 

[DWiley]

do the mortgage brokers sign a certification.

Assume that you are a lender or investor and you own the mortgage on my home. I want to refi. You already own the loan, regardless of what the current value is or the condition. If you can refi that loan and reduce my payment a few hundred dollars, what, from a risk management point of view would an appraisal add to the equation? I mean, suppose I am underwater with regard to the value. In that case, you are already in a risky position, and lowering my payment only reduces the risk that I default. Or, suppose my home is in poor condition - again, you already have the loan and the risk of the house condition. So, please explain ( and I am asking for an explanation not a reference to some news article) how the lender’s risk is reduced in that scenario by obtaining a fresh appraisal. Thx in advance for the thoughtful reply that I know I can count on.

 

[Mejappz]

If you think a MAPE of 7% is good, then how would you characterize a model with a MAPE that is less than half of that?

On properties in the sales range Bert is working on. I would call that sales chasing.

 

[DWiley]

I would say that given your involvement in the development of the AI Residential Database you thought that AVMs were a great idea well prior (by at least 10 years) to any actual performance data showing the same. They certainly may be great predictors of value but who is running them? And do we trust them with the economic well-being of our nation or are they making profit driven (a.k.a. efficiency) decisions that pose systemic risk again? I don't believe that it is a good idea let someone value there own collateral with no oversight other than some portfolio risk management section (who can be easily replaced if they get uppity) and then sell those securities as investment grade. We know how well the rating agencies did when they were supposed to downgrade risky securities prior to the last crash, like most people in this business they followed the money.

You assume incorrectly. The AIRD concept was for the benefit of appraisers, not AVMs. Back in those days AVMs where not what they are today. Back the some large AVMs had a PC 10 of only about 50% and a MAPE of almost 10%. Now many have PC10s (measured against actual sale prices) that are much higher and MAPEs that are much lower. As for the rating agencies, they can rate on the basis of the performance of loans originated based on AVM use versus the performance of similar loans (not all loans) based on appraisal use.

But, you are right to be concerned about who is running the AVMs. Different models are designed for different purposes. Many commercial AVMs are designed, for example, to provide the maximum hit rate, because that is what drives revenue for the AVM provider

On the other hand, some are designed for actual risk management. Such AVMs produce fewer results, or won’t provide a result at all if there isn’t enough data or the FSD is too large. The development of transparent AVM testing standards, as prescribed in DF, is long past due. There is a reason that the GSEs will not rely on the results of third party AVMs in the waiver process.

 

[Mejappz]

No kool-aid involved, just data. You know, that stuff we appraisers are supposed to analyze in support of our opinions. . Do you have any actual data (data, not anecdotes) related To AVM performance?

Did you read my post earlier on in this thread? I asked you to send me some of Freddie’s data and I would test it for you (free of charge). The offer still stands.