Appraisal Statistics: Regression - Do Your Really Know How Much Work It Is?

[Artemis Fowl]

Sensitivity Analysis: $FREE
MARS: $15,000
Educating oneself to the point they understand Michael & Bert's posts: $PRICELESS

 

[RCA]

Before you mentioned it, I had never knew that this software existed. After reading up on its capabilities, and trying to grasp the concepts behind each tool, I'm beginning to see how this can be very useful. The big question is, how could an appraiser justify the $15,000 per year price tag? I could see where this would be a great tool for mass appraisal, and county governments might pony up the cash to pay for it.

Well, that is an obstacle. I believe that MARS is so useful, that Minitab/Salford systems could be persuaded to make a special version for appraisers, especially since appraisers can be identified by license number through a national registry. (Those who had the old basic MARS, like me, can continue into the future at $360/year.) Also, I was told a couple of weeks ago by the sales manager at Salford that if you provide a class in MARS, the teachers and students get a free license for SPM (all packages) for one year. I think, if we can get enough appraisers to use it, a special deal may be possible. But you would have to be talking numbers, and you would need the involvement of an organization like the Appraisal Institute.

 

[RCA]

How long does it take to perform its calculations? How powerful of a computer do you need and does it offload some of its calculations to the graphics card?

Salford Systems MARS is extremely fast due to it's proprietary algorithms. It just so happens the technique it employs, allows a high degree of optimization. That is one of the advantages of MARS - it can handle relatively large data sets compared to other statistical methods.

 

[RCA]

I truly hope so, and your point that it is within the grasp of the "average" is encouraging! But just so that I know a little more about where you are coming from, do these apps measure and correct for autocorrelation, heteroskedasticity and multicolinearity? Or is it up to the user? Do you make adjustments for them in your models?

Also to answer the question collinearity: You had best remove collinear variables; otherwise MARS will remove all except one for you and you may not get what you want. So, for example, if you bring in all the data fields from MLSListings.com, you will find that a good number are collinear or wrapped up in other fields; e.g. you have Beds_Max, Beds_Min and Bedrooms. I've never seen a case where they are not one and the same. They are collinear. You probably want "Bedrooms" in your model, so you should not put Beds_Max or Beds_Min in your input.

 

[RCA]

Bert-

Let me ask you a question from a boots-on-the-ground residential appraiser....
Assume I'm valuing a home in a relatively conforming neighborhood. Typical suburban tract development. In my initial search parameters, assume I'm gathering data to these parameters: Size range 1,800sf to 2,350sf; age range 30-45 years; lot size 8,000sf to 13,000sf. Assume over the last year, there are 40 data points and the price range is $700,000 to $825,000; assume there may have been an outlier or two at $600k and one at $1,000,000 and I throw those out of the mix. Assume the market has been increasing (overall) at 6% and based on the price distribution by date, we see homes at the beginning of the period selling for less than homes at the tail (consistent with the expectation of an overall rising market).

From the ground level, it would appear the differences in value are based on (a) condition, (b) size of the home, (c) possibly the specific location, (d) to a degree another amenity (say, a pool), and (e) the motivations of the individual buyer/seller.

Question: It seems to me that I've already filtered the data to a relatively high degree. I don't have a random distribution; I have neighborhood-specific sales, all which more or less compete with one another, share the same/very similar buyer pool, and for the most part, equally share external/economic factors/forces.
At this point, how much more discrete must I get in my filtering system or analysis whereby a simple, linear regression doesn't provide a reasonable starting point to determine a GLA adjustment within that data set such that it cannot be refined in the grid using simple sensitivity analysis after adjusting for other factors (condition, let's say) which I can also support using old-fashion pairing?

Back up, please. Why did you throw those so-called outliers out? Just because of price? I would have left them in and let MARS try to find an explanation for why the values are so high. I'm entering 50+ variables. Who, knows, maybe there is an explanation. You want MARS to look at everything - even with missing variables, so it can extract as much intelligence as possible. That is the whole idea. MARS needs a minimum of about 15 variables to function. So your data set of 40 is just fine. You should just run MARS and then compare the output to your understanding of the neighborhood, then proceed to improve and extend the model (assuming you don't have all the pertinent variables represented in the input). As to deciding how much further you should, when you should stop, is easy: How much of the price variance can you account for? Can you do any better?

 

[RCA]

Sensitivity Analysis: $FREE
MARS: $15,000
Educating oneself to the point they understand Michael & Bert's posts: $PRICELESS

Well, by "sensitivity analysis" I am not sure what you are talking about. As I understand it, it is a way to test a given model's output against variations in the input, for the purpose of prediction, not so much for the purpose of constructing the model in the first place. MARS can and is used for "sensitivity analysis". But I'm guessing you must mean something else. To reiterate, we are talking about constructing value models, i.e. equations, to predict home sale prices based on known inputs such as GLA, date of appraisal, lot size, bath count and so on.

 

[RCA]

How long does it take to perform its calculations? How powerful of a computer do you need and does it offload some of its calculations to the graphics card?

You can run MARS easily on a standard laptop, certainly for typical appraisal size data sets that are under 1000 properties. Although the low-price "Basic" license I have is limited in terms of how much data it can handle, you would have to get into thousands or properties to hit that limit. And at some point, you might want to extend memory or move up to a workhorse desktop with an Extreme I7 or top-notch Xeon chip. MARS is really fast, I wouldn't worry about it. The higher end models mostly just give you more automation, letting MARS run in the background on large data sets automatically improving parameters that you need to do manually with the Basic package. But, I have never had problems with the Basic MARS package. (Again note, Salford-Systems.com no longer sells the Basic MARS package.)

 

[hastalavista]

Back up, please. Why did you throw those so-called outliers out? Just because of price?

For a typical residential property in the neighborhood I described, it wouldn't be just because of price. I would try to decide why they were so outlying (the low price would likely be easier to discover than the high price; but I've seen homes sell for 25% higher than market... as recognized by the buyer... because the home happens to be next door to their children/grandchildren). But clearly they don't fit the bulk of the data and I wouldn't be hesitant to throw it out as an unreliable indicator.

[/quote]As to deciding how much further you should, when you should stop, is easy: How much of the price variance can you account for? Can you do any better?[/QUOTE]
In my sensitivity analysis (and I try to include one outside-the-grid sale), in the neighborhood I describe, I can usually account, on average, for 95-98% of the price variances with the adjustments I apply. I may have one individual comparable that is off by more than 5%, almost never more than 10%, but collectively 95-98% is the result.

But I'll restate my question (and I appreciate the time you are taking to respond): Given the data set I described and for the typical residential appraisal completed by a competent and market-knowledgeable residential appraiser, how much additional confirmation/accuracy/reliability (pick any term you think is appropriate) is gained by the additional and more sophisticated analytical techniques vs. using the more simple techniques (linear regression and pairings for some elements)? That may be an unreasonable question to ask based on the hypothetical I'm proposing.
The significant question (in my mind) is does that difference become meaningful in terms of the intended use of the appraisal (and I'm not asking you that question; I think the answer is the individual appraiser's to make based on his/her understanding of the intended use).

In other words, by using less sophisticated techniques, the sales may adjust to a range between $815k and $835k; the appraiser picks their point within that range based on the rationale described in the reconciliation.
Assuming that range is reasonable and credible, I'm trying to gauge how material it would be to go deeper and use a more sophisticated technique?

If I'm doing a high-end litigation case, I want to use all the techniques I can (or at least more and better techniques than the other side! ).
If I'm doing typical mortgage finance work, I want to use the techniques that are sufficiently reliable and adequate, get me to a credible result within the context of the intended use.
In order to determine how far I should go, I need to know how material the differences are between low-level vs. high-level analysis.

Your answer may be...

"Look, for what you describe, I think MARS is going to be better... period. But the way you phrase the question, I'm not sure it is going to make a material difference. This assumes you've done what you describe correctly!"​

Or

"What can I tell you? MARS is going to be better because your simple analysis does not and cannot consider the factors that MARS can. Whether that's good enough for you is your decision to make."
​

Or your answer may be something quite different.

 

[Artemis Fowl]

...
The significant question (in my mind) is does that difference become meaningful in terms of the intended use of the appraisal (and I'm not asking you that question; I think the answer is the individual appraiser's to make based on his/her understanding of the intended use).
...

This...

 

[RCA]

For a typical residential property in the neighborhood I described, it wouldn't be just because of price. I would try to decide why they were so outlying (the low price would likely be easier to discover than the high price; but I've seen homes sell for 25% higher than market... as recognized by the buyer... because the home happens to be next door to their children/grandchildren). But clearly they don't fit the bulk of the data and I wouldn't be hesitant to throw it out as an unreliable indicator.

As to deciding how much further you should, when you should stop, is easy: How much of the price variance can you account for? Can you do any better?[/QUOTE]
In my sensitivity analysis (and I try to include one outside-the-grid sale), in the neighborhood I describe, I can usually account, on average, for 95-98% of the price variances with the adjustments I apply. I may have one individual comparable that is off by more than 5%, almost never more than 10%, but collectively 95-98% is the result.

But I'll restate my question (and I appreciate the time you are taking to respond): Given the data set I described and for the typical residential appraisal completed by a competent and market-knowledgeable residential appraiser, how much additional confirmation/accuracy/reliability (pick any term you think is appropriate) is gained by the additional and more sophisticated analytical techniques vs. using the more simple techniques (linear regression and pairings for some elements)? That may be an unreasonable question to ask based on the hypothetical I'm proposing.
The significant question (in my mind) is does that difference become meaningful in terms of the intended use of the appraisal (and I'm not asking you that question; I think the answer is the individual appraiser's to make based on his/her understanding of the intended use).

In other words, by using less sophisticated techniques, the sales may adjust to a range between $815k and $835k; the appraiser picks their point within that range based on the rationale described in the reconciliation.
Assuming that range is reasonable and credible, I'm trying to gauge how material it would be to go deeper and use a more sophisticated technique?

If I'm doing a high-end litigation case, I want to use all the techniques I can (or at least more and better techniques than the other side! ).
If I'm doing typical mortgage finance work, I want to use the techniques that are sufficiently reliable and adequate, get me to a credible result within the context of the intended use.
In order to determine how far I should go, I need to know how material the differences are between low-level vs. high-level analysis.

Your answer may be...

"Look, for what you describe, I think MARS is going to be better... period. But the way you phrase the question, I'm not sure it is going to make a material difference. This assumes you've done what you describe correctly!"​

Or

"What can I tell you? MARS is going to be better because your simple analysis does not and cannot consider the factors that MARS can. Whether that's good enough for you is your decision to make."
​

Or your answer may be something quite different. [/QUOTE]

You may not need MARS. But it can save you a lot of time on some properties, just because of all the crazy work it does. You can look at the floor adjustment MARS came up for a condo I did in Case Study 2 of http://docs.salford-systems.com/BertCraytor.pdf

For complex data sets, where there are many variables that affect price, it is not very easy at all to get adjustment correct.


Otherwise, without knowing what you are doing, I can't say much. But imagine a case like this: I had to appraise a duplex in Burlingame years back. Such house sales are rare in Burlingame. But I could go over to other cities and counties and analyze those properties, taking into consideration, location, neighborhood, city and county. MARS is capable of discovering the effect of location (Longitude/Latitude), neighborhood, city and county and throwing it into the model. It could hopefully then provide you with a model that accounts for the effect of GLA and other variables on price. I don't think you would want to try to do this manually, as you would probably be held in contempt of court.