How does AI class a house?

[RCA]

Sorry they I got my C & Q mixed up. I am looking for a non-bias way to judge those borderline Q4 to Q3 houses.

The question is "What does the Market say about the difference between Q3 and Q4?" The Market, I would say, kind of tells you the difference between a newer Q3 and a newer Q4. A beat-up Q2, really beat-up, doesn't have much value at all, does it? So, the market says that the Q2 level Wolf 6-burner range, 5 years old, condition C3, has a marginal value of $3K; a 15-20-year-old, in good condition, maybe $600. [ I am thinking of one of those Wolf range tops. People go out and buy used Wolf range tops if they are in decent condition - and save a lot of money. In fact, the pre-Sub-Zero ones are worth more than the new ones.] When it comes to market value, there is this interplay between C and Q. Let the Market tell you what the combination is worth, then split it up as you see fit. Of course, for an entire house to be Q2 or Q3, we have to judge everything, although we understand you wind up going from room to room to get room averages, then average the room averages, then average exterior components, and in the end compare that to your residual and then come to some kind of global understanding about the quality and condition of the house or even property as a whole.

How about aesthetics? Some houses are "just" adorable - and you can't be quite sure why. The human mind is always looking for a kind of perfect balance in features, a magical combination, and what appeals to one man may not appeal so much to another. But the market is composed of many, and all the values get mixed up into a kind of structured mess, "structured" because hopefully we have ranked the properties, if not exactly, approximately - and good enough for our purposes.

 

[OSU Beavers]

I am not looking at value now, I just am looking for an answer that is not AI marketing speak such as "dynamic machine learning".

 

[RCA]

Actually, I think dynamic machine learning most like violates USPAP. Whatever you do has to be capable of being repeated, - like running regression. If you are brought before a board, they may want you to show them how you came to your conclusion. -> Bring out the program and run it for them on your data!

Me: Can dynamic machine learning be replicated before a board of inquiry in order to prove how you came to your results?

Claude: 'This is one of the most important questions in appraisal methodology — and it strikes at the fundamental tension between predictive power and legal defensibility.

The Core Problem​

Modern ML (gradient boosting, neural nets, random forests, even MARS ensembles) produces results through processes that are:

• Path-dependent — the model fit depends on data order, random seeds, hyperparameter search
• Non-algebraic — no closed-form equation you can write on a whiteboard
• Opaque to cross-examination — "the algorithm decided" is not a defensible answer before a board, AMC reviewer, or court

A board of inquiry or appraisal review will ask:

"How did you arrive at this $47,500 adjustment for condition?"

With dynamic black-box ML (Kalman Filter, OGD, RLS, Bayesian Updating) the honest answer is: "A model optimized a loss function across 847 transactions and produced this number." That answer will not survive scrutiny." [Note there are "static" black-box algorithms that will repeat performance if given the same random seed to begin with.]

---

The Replication Standard​

Legal and regulatory contexts impose a replication requirement that ML generally fails:

• Can an independent reviewer reproduce your result from your stated methodology?
• Can you explain each adjustment in terms a competent appraiser would recognize?
• Is the process documented sufficiently to withstand USPAP Standard 1 scrutiny?
• Could opposing counsel's expert witness deconstruct your method and find it sound?"

... But then again, I am guessing you mean something else actually:

Automated Valuation Models with continuous retraining (CoreLogic, Zillow Zestimate) are the real-world case you likely had in mind:

• Retrain on rolling windows of new sales
• Model state on the effective date of your appraisal may not be recoverable
• The value estimate for a property on March 1 vs March 15 may differ not because the property changed but because 200 new sales were ingested.

 

[RCA]

But come to think of it, I need to allow all of my stat programs to produce their starting random seed in the output and to take a random seed in the input to replicate a previous run. I forgot that!!

 

[OSU Beavers]

Yes, that is a complicated way of saying "I need to understand it before I use and rely on it"

 

[RCA]

Yes, that is a complicated way of saying "I need to understand it before I use and rely on it"

That's what USPAP says.

 

[RCA]

Some people don't think they can learn things, but they do. Some think they can learn anything, but they really don't - although they can pass tests.

In the end, either you can or you can't get things done.

Some people with all the certifications and degrees in the world, can't do much more than twiddle their thumbs.

 

[RCA]

The best way to learn is by doing. Just keep things simple. I started off using basic MARS regression, much the same way I would multi-linear regression. Only I would typically use the defaults for parameters, - for better or worse. And back in those days, around 2004-5, they didn't have so many parameters.

Now? After many years of users and developers tweaking things, they just naturally get more complicated. You know, you are just trying to make it better.

For example, date-time adjustments (e.g. sale_age), are typically so many $/day increase or decrease in price depending on the date. But that is the same for small and large houses. That is not quite right is it? You would want the increase/decrease in price to be based on the size or price of the house. We can do that, it is just another wrinkle.

 

[RCA]

There are some strange things that can be done with earth() (aka MARS), for example, we can alter pricing through an area, much as if we were using kriging. This can be done by creating additional basis functions and using 2nd-degree interactions, while blocking 1st-degree single-variable relationships with the sale price. Although certainly not to the extent of kriging, which is used for exploring oil and mineral deposits. Kind of like taking a high-priced town like Hillsborough or Burlingame and finding a set of streets higher on the hills where the very expensive homes run, only it meanders through the hills and is not easy to delineate. Kriging might be better, but kriging is more work and isn't widely understood in real estate. I know a bit about it because my son-in-law is a high-level expert in the subject.

 

[RCA]

I am not looking at value now, I just am looking for an answer that is not AI marketing speak such as "dynamic machine learning".

Another thought that comes to me is that if you don't have a good strong model, that is to say a weak model with say an R2 below 0.73 or so, then the model will be unstable, - that is, it can jump around on you. You need to try to get the R2 above 0.80 and the CVR2 as high as possible, possibly also over 0.80, but maybe as low as 0.60 (if you just don't have enough data). It just has to be the best you get, and if that is the case, when you rerun Earth, even with a different random seed, you will likely get the same model kicked out again. ...