Earth

[RCA]

Progress on getting R Earth to generate models on a level with Salford Systems MARS.

I have a data set that I created for a recent Estate Appraisal in Montara that was difficult. I had to use bootstrapping with MARS to get an GCV-R2 up to 0.85.

The best I could do with earth was an R2 of about 0.55 with 1 interaction and 0.64 with 2 interactions. However, Stephen Milborrow also, in addition to the September 2020 update to earth, a paper also from September called "Variance Models in Earth", that discusses a new parameter to earth called "varmod.method". Using this parameter I was able to generate a very nice model with an GCV-R2 of 0.86.

This means, everyone can do Mars with free software.

And, believe me, I am very sure the method I have proposed for the Sales Comparison Approach will become the standard.

I'm putting together a small paper that I will put up on the webiste tonight. Let you know in a subsequent post.

BTW, one of the members sent me an email a couple of days ago about when I was coming out with a book on the method. And this obstacle was the reason I put it off. I can't think of any appraisers willing to fork over $16K for statistics software. And the old earth in particular, often did not produce good models on difficult data. So after finishing the Montara appraisal Sunday, I decided to dig back into the latest on R earth:

 

[RCA]

If you set earth to do 2-way interaction (3D models) by setting "degree=2", you can push the GCV-R2 up to 0.88, but it's probably not worth dealing with. Two-way interactions can be put on the Sales Grid as say

SaleAge-GLA ....
SaleAge-LotSize ...
GLA-Garage ....

But you may have trouble with your clients.

 

[Mejappz]

But you may have trouble with your clients.

Especially those in the GSEs who think they know what they are talking about and have not the slightest clue. Is it a C3 a C4? An R-squared of 92% with a COV of 4% is laughable to them. They are busy building “game-changing technology.” i.e., a more efficient way for them to steal the data from appraisers. I am looking forward to the white paper and subsequent videos Bert.

 

[Elliott]

"R2 of about 0.55 with 1 interaction and 0.64 with 2 interactions."

Slightly better than flipping a coin? Making point in craps? I guess its good enough for appraising using 'statistical analysis.'

 

[RCA]

"R2 of about 0.55 with 1 interaction and 0.64 with 2 interactions."

Slightly better than flipping a coin? Making point in craps? I guess its good enough for appraising using 'statistical analysis.'

No actually in the sciences, they are often happy to get something like R2= 0.40 - or even much less.. What that can mean, for example, is that they have discovered the reason for 40% of the variance in some variable. For example, some test that if it results in a TRUE indication means the person tested has a 40% chance of having a heart attack during the next week.

We are rather demanding in our application of regression. As a rule of thumb appraisers are expected to be within 5% of the true value of a property. Or in other words (although not quite the same thing) their comparable adjusted values are expected to be within 5% of the mean. Some appraisers try to be far more accurate than that. On the last appraisal, all of my adjusted values were within 0.20% of their mean.

Now as a footnote, the deviation of the adjusted values really says more about how good the model is, than how accurate the value is. The accuracy of the value for the subject depends on how the appraiser rates the subjective features in relation to the comps. So, an appraiser can easily rate the subject much higher or lower in quality than the comps an get a quite different value. All the adjusted values will move up or down in the same uniformity depending on his judgement. But, if you start off with an R2 of 0.80 for the measured variables, that leaves 20% variation to be accounted for. So if the appraiser is off by 10% in his judgement that will only add a 10% x 20% = 2% deviation in the final value (assuming it is based on the average of the adjusted values). Now, varying the quality, condition, view, functional utility, etc., ratings up or down more than 10% should be fairly noticeable to about anyone else. Assuming that is the case, then it is not likely to happen. The only caveat is that if your subject is near the bottom or top of the CQA (Condition/Quality/Appeal) curve, small differences in judgement will make a big difference in value. However, and on the other hand, the differences in Quality/Condition (usually condition for the properties on the bottom and quality for those near the upper end) in these parts of the curve are usually so enormous that you can usually pretty accurately place your subject between the right two comps, which already have a CQA score based on their sale price, --- and so in that way it is easier to grade comps at the far ends of the CQA curve than in the middle; even though percentage errors in those areas have big consequences.

CQA curves from 4 different appraisals:

Montara:



El Granada:


Pacifica (Area A):


Pacifica (Area B):


NOW, isn't it nice to have an objective way to determine market reaction to the condition of fixer-uppers at the low-end and overbuilt homes at the high end?

 

[J Grant]

Fixer uppers appeal to a different buyer than upgrades houses. The fixer upper appeals typically to an investor or flipper/contractor ) and highly improved houses appeal to an owner occupant buyer wanting move in ready upgraded and wiling to pay $ for it ) , THAT is the key to understanding the price differences between the two. The math only shows us that there is a price difference.

 

[DWiley]

No actually in the sciences, they are often happy to get something like R2= 0.40 - or even much less.. What that can mean, for example, is that they have discovered the reason for 40% of the variance in some variable. For example, some test that if it results in a TRUE indication means the person tested has a 40% chance of having a heart attack during the next week.

We are rather demanding in our application of regression. As a rule of thumb appraisers are expected to be within 5% of the true value of a property. Or in other words (although not quite the same thing) their comparable adjusted values are expected to be within 5% of the mean. Some appraisers try to be far more accurate than that. On the last appraisal, all of my adjusted values were within 0.20% of their mean.

Now as a footnote, the deviation of the adjusted values really says more about how good the model is, than how accurate the value is. The accuracy of the value for the subject depends on how the appraiser rates the subjective features in relation to the comps. So, an appraiser can easily rate the subject much higher or lower in quality than the comps an get a quite different value. All the adjusted values will move up or down in the same uniformity depending on his judgement. But, if you start off with an R2 of 0.80 for the measured variables, that leaves 20% variation to be accounted for. So if the appraiser is off by 10% in his judgement that will only add a 10% x 20% = 2% deviation in the final value (assuming it is based on the average of the adjusted values). Now, varying the quality, condition, view, functional utility, etc., ratings up or down more than 10% should be fairly noticeable to about anyone else. Assuming that is the case, then it is not likely to happen. The only caveat is that if your subject is near the bottom or top of the CQA (Condition/Quality/Appeal) curve, small differences in judgement will make a big difference in value. However, and on the other hand, the differences in Quality/Condition (usually condition for the properties on the bottom and quality for those near the upper end) in these parts of the curve are usually so enormous that you can usually pretty accurately place your subject between the right two comps, which already have a CQA score based on their sale price, --- and so in that way it is easier to grade comps at the far ends of the CQA curve than in the middle; even though percentage errors in those areas have big consequences.

CQA curves from 4 different appraisals:

Montara:

View attachment 51499

El Granada:
View attachment 51500

Pacifica (Area A):
View attachment 51501

Pacifica (Area B):
View attachment 51503

NOW, isn't it nice to have an objective way to determine market reaction to the condition of fixer-uppers at the low-end and overbuilt homes at the high end?

Good stuff. I especially like the comment of the model strength vs accuracy. Did you do any PC10 back testing on actual sales?

As far as appraisal variance, data suggests that something in the 10% to 15% range is more realistic than the 5% number that gets thrown around a lot

 

[TerryRohrer]

Good stuff. I especially like the comment of the model strength vs accuracy. Did you do any PC10 back testing on actual sales?

As far as appraisal variance, data suggests that something in the 10% to 15% range is more realistic than the 5% number that gets thrown around a lot

I saw a study where the author carefully selected homes with very few differences and concluded that there was a 10% variation that was typical noise in the market. I'm more suspect when I see reports where all the sales are adjusted to leave almost no explained difference, particularly out here in the wild west where "typical" describes groups of 1.

 

[Meandering]

I saw a study where the author carefully selected homes with very few differences and concluded that there was a 10% variation that was typical noise in the market. I'm more suspect when I see reports where all the sales are adjusted to leave almost no explained difference, particularly out here in the wild west where "typical" describes groups of 1.

But doesn't the AI's book, The Appraisal of Real Estate, state that the point of making any adjustment to the sale price of the comps is to "minimize" the economic difference? And following that, adjustments that leave zero difference have then found that bare minimum to be less than one dollar, or whatever the final opinion is rounded too?

"Typical" buyers and sellers, really don't do this type of analysis. If they did, they would never have been suckered into some of the loans they signed for, not so many years ago.

.

 

[TerryRohrer]

But doesn't the AI's book, The Appraisal of Real Estate, state that the point of making any adjustment to the sale price of the comps is to "minimize" the economic difference? And following that, adjustments that leave zero difference have then found that bare minimum to be less than one dollar, or whatever the final opinion is rounded too?

"Typical" buyers and sellers, really don't do this type of analysis. If they did, they would never have been suckered into some of the loans they signed for, not so many years ago.

.

Not sure where you are reading at, but in my view, adjustments are to account for the measureable impact physical differences have on price, as best we can measure. Every day, I see houses that appear to be similar in most respects, but sell for significantly different prices. Some of those differences in price cannot be reliably explained...there are at least four different parties to each pair of sales, each with their own agenda and circumstances and motivations and bank accounts and available credit and a host of other personal perspectives. None of those differences are contemplated in our definition of market value, but they certainly impact the price of every sale we have ever been aware of. Those differences cannot be reliably explained, so it is a fantasy to expect an appraisal where every sale adjusts to the same value. If that happens, you can be sure that adjustments are being made that are not warranted, and/or adjustments that should be made are not being made.