AI Agents and Hybrid Appraisals

[RCA]

Surprisingly, a search of "AI Agents" on this forum only turns up one thread related to Spyware.

Do you know what an AI Agent is? It is taking an LLM+Reasoning Engine like Claude or ChatGPT or Grok-2 and developing a software system around it that can do the work of a human, - SUCH AS creating a Hybrid Appraisal Report from inspection data. The inspection data comes into the AI Agent, which analyzes the inspection and associated MLS data with usually Random Forests Regression and some other methods (at least in the case of Fannie Mae, Clear Capital and House Canary)

So, the industry is trying to keep you blind to the fact of what AI Agents really are --- but you should know, it is going to be SURVIVAL OF THE FITTEST, - nothing to do with saving appraiser jobs for the sake of saving appraiser jobs. You should know by now, of course, that the latter goal does not count for much.

So, when Appraisal Institute and GSE leaders and others talk about the glory of AI for appraisal, ...., please understand the implications.

My take on this, is that to compete in the long term, - and certainly smart educated appraisers will always have a role to play - you need to understand the coming competition from AI Agents.

Of course Claude already knows about all of this, by the way. No doubt, we have others asking it for advice:

ME: "Is it possible to create an AI agent to do residential appraisals using hybrid inspection data?"

Claude: "Let me help sketch out how an AI system could assist with residential appraisals while incorporating hybrid inspection data.

I've created a framework for an AI residential appraisal agent that could work with hybrid inspection data. Here are the key components and considerations:

1. Hybrid Data Integration:

• The system processes various data sources including virtual tours, photos, owner-provided information, and professional inspection reports
• Location data is incorporated including school ratings, crime rates, and proximity to amenities
• Market conditions and comparable sales are analyzed

1. Key Features:

• Automated property feature extraction
• Condition assessment from hybrid inspection data
• Market analysis and comparable property identification
• Confidence scoring system to validate appraisal accuracy

1. Risk Management:

• The system includes risk factor identification
• Confidence scores are calculated based on data quality and completeness
• Multiple data sources are used to cross-validate findings

1. Machine Learning Approach:

• Uses XGBoost for regression analysis
• Includes feature importance analysis to understand value drivers
• Implements model evaluation metrics (MAE, MAPE, RMSE)"

Here is. the Python it generated for this (attached)
ME: "Do companies already use this kind of system?"

Claude: "Yes, several companies are developing and using AI-assisted appraisal systems, though with varying levels of sophistication and different regulatory compliance approaches:

Fannie Mae's Modern Appraisal Delivery (MAD) initiative includes AI components for data validation and risk assessment. They've been testing hybrid appraisals that combine traditional methods with AI analysis.

Clear Capital has developed ClearCollateral Suite, which uses AI to analyze property data and photos. They're working with lenders on hybrid appraisal products that combine automated valuation with physical inspections.

HouseCanary provides automated valuation models (AVMs) that incorporate AI analysis of property characteristics, market data, and visual data. Their system is used by several major lenders.

However, there are important limitations and regulatory considerations:

1. Most systems are currently used as supporting tools rather than complete replacements for human appraisers
2. Regulatory requirements still often mandate traditional appraisals for certain loan types
3. Many lenders use AI systems primarily for initial screening or as a secondary validation tool
4. The accuracy and reliability of these systems can vary significantly by market and property type

The technology is still evolving, and there's ongoing debate in the industry about the appropriate balance between automated and traditional appraisal methods. Would you like me to elaborate on any particular aspect of how these systems are being implemented in practice?"

 

[RCA]

So, this issue is rather depressing. Really. Jobs are already going down the tube. Rather fast.

You know what happens:

1. AI Systems add more information, more detail and more rules.
2. The work becomes more difficult for humans.
3. Who come to depend on AI Agents, that they need to interface with.
4. Things can get complicated, very complicated.
5. The humans interfacing with AI Agents will need to be pretty smart and well-educated (self-education acceptable).

What is society going to do? Of course, they are going to have to start taxing the AI Agents - and Robots, as if they were workers. (Shhh. Don't tell the AI investors! -- Although, surely, some already know that AI is not going to be as lucrative as many promise.). SO, that is to say, AI corporations will have not only Corporation Tax to worry about, they will also have Robot Labor Tax. And, somehow, they are going to have to pass this tax on to consumers -- many who don't have jobs -- which means people will somehow get paychecks for not doing any work. Will this work? Pun intended.

 

[Fernando]

No AI appraisal is more creative than Fernando.

 

[Surf Cat]

No AI appraisal is more creative than Fernando.

That's an understatement bro...

 

[Surf Cat]

The technology is still evolving, and there's ongoing debate in the industry about the appropriate balance between automated and traditional appraisal methods. Would you like me to elaborate on any particular aspect of how these systems are being implemented in practice?"

Everyone should be acutely aware by now that the traditional appraisal is on the way out. In my opinion, and why so many appraisers are bucking the hybrid appraisal as it currently stands....is backwards.

Even Claude has it wrong. Where the wheels come off the wagon is in the very step.

1. Hybrid Data Integration:

• The system processes various data sources including virtual tours, photos, owner-provided information, and professional inspection reports

In my opinion the licensed, trained, real estate appraiser should be the one conducting the inspection and the initial search for the comparables.

Why? Because people lie and they always will. Especially when there's money & emotions involved. Inspections conducted by human appraisers would cut through the nonsense like linear regression cuts through data.

Location, externalities, quality of construction, the quality of remodeling ( professional or DIY), sagging roofs, sloping floors, non permitted additions. And let's not forget measuring to ANSI standards. All these aforementioned items is where the rubber meets the road.

A virtual inspection can't capture all these intricate items like a trained human can. It can't be conveyed by some Uber driver or Junior realtor quickly going through the property to get to the next one for 50 bucks. Then pass it off to the chained to the desk appraiser to make sense of it all. This is why that whole hybrid thing is flopping.

Once the licensed, insured, professional appraiser is done with the inspection, then this information could be thrown at the AI robot to write the report, considered the comparables that the inspecting appraiser chose initially, "along with" Collateral Underwriter, Freddie's AVM, etc. The AI robot would quickly be able to ascertain which neighborhoods are alternative competitive ones to the subject's immediate neighborhood via median sale price points, and school district ratings. Ascertain adjustments for tennis courts, equestrian properties, panoramic/jetliner views all in an objective, non-subjective manner "in minutes". This is the time consuming part.

Once the AI robot is finished with the report, it is sent back to the inspecting appraiser to review, sign and send in.

That in my opinion would be a win win hybrid solution.

 

[Meandering]

And the Maytag repair man asks, “why do you complain about having to replace your appliances every 5 years? Oh yeah because they are cheaper than buying the ones that used to last 30 years.”

Oh, they aren’t cheap anymore?

 

[RCA]

Learn. something new:

These AI Agents are reportedly using Random Forests or XGBoost to get adjustments. Both are BLACK BOX approaches, i.e. their internal decision process is complex and not easily interpretable.

Random Forests builds regression trees independently and in parallel. It uses a random subset of features and bootstrap sampling.

XGBoost builds regression trees sequentially, using the residuals to improve the subsequent regression runs. In fact, it uses the first and second derivatives in the residuals. So, it is somewhat superior to Random Forests.

=====. And so, the question is: ..... ======

Can the AI Agent with XGBoost, do a better job at lower cost than the appraiser?

1. It certainly costs less.
2. We have its R2 on past data to prove how accurate it is. According to Claude, they are getting R2 values of 0.85 to 0.93, CVR2 values around 0.80 to 0.88. - That is not bad at all. A good traditional appraiser is expected to do better than 0.90 or 90% in accuracy.
3. But, for example, I expect to get better than 0.98 with my RCA method. But, unfortunately, I cost a lot more. If you want that kind of accuracy, come to me. Well, if you are willing to pay $1000-$2000.
4. Why can I bet more accurate? Because I use MARS, which isn't a Black Box approach. I know what I am doing.

Also worth noting, because AI Agents use Black Box approaches, they are, IMO, undoubtedly suffering from overfitting. They can detect overfitting ONLY via the CVR2 value based on accuracy on old data subsets (take an old sale, go back in time and predict its sale price and see how close the get to the actual sale price, over and over). With MARS, you get an understandable model you can read and judge by the terms whether it is overfitting - it can be very obvious.

 

[Digger88]

The AI robot would quickly be able to ascertain which neighborhoods are alternative competitive ones to the subject's immediate neighborhood

They want to do away this altogether

I'm looking forward to AI agents running locally on pc when it comes to a free version. Microsoft is coming out with one next month for paid users of Copilot

 

[OSU Beavers]

A local, experienced and honest appraiser who is given appropriate time and pay will ALWAYS produce a superior product to this hybrid, AMV or AI crap.

 

[RCA]

Los Cabo is way too expensive for Mexico. They say it is a tourist town built solely for Americans with American prices. Esperanza is $1K/night. Dinner menu items $50 or so.

Go to Merida or Oaxaca. Or take some CIA advice and just stay out of Mexico.

Not that I have ever been to Mexico, leastways not much further South than Tijuana or Juarez, - I know people that have going there. Los Cabos should be safe though, my daughter went there for her friend's wedding last year - who's rich father is retired somewhere up in the hills above Los Cabos.