The Race To Automate Appraisal Process

[chad hampton]

Yeah, after I typed it, I realized all county info is public records and counties can't withhold data. Makes sense.

 

[Thomas Holding]

Thomas,

I am curious if you have access to MLS data and if so does it help you?

Assessors here are using Zillow and Realtor.com to verify condition.

We have access to many different types of data including MLS. We also mail out verification questionnaires to property owners when any transaction is noted. Tax offices are great resources for verified sales data if you know what their qualification codes are. I can't speak for assessors in MI, but in the two counties in NC where I have worked, we would never rely on Zillow or Realtor.com for an estimate of value, the Dept. of Revenue and the Property Tax Commission would probably audit us in an instant if they discovered that we were using that as the basis for our valuations and would put a stop to it.

 

[RCA]

I'm of the opinion that 80% - 85% of residential appraisals could in fact be accurately valued (within +/- 2.5%-5% of market value) using computer modeling, however that 80% number comes with a huge caveat. The data used has to be extremely accurate and it has to be complete and timely for the computer modeling to work. The other factor that is just as important is that the models used to analyze a property must be customized down to the neighborhood level in order for them to have a chance at being accurate enough. All of this data takes an incredible amount of work to collect, verify, model, and maintain, and this is the most costly component of the appraisal process. I work in the tax office of a relatively small county (60,000 parcels), and we have a full time staff of 6 appraisers to perform all of these tasks. That's just for data collection. The infrastructure required to support our data collection and retention is very costly as well. Our IT department handles that portion of the equation and I honestly don't know the amount of their resources that are contributed to the appraisal department. I almost forgot to mention the GIS department. They have a staff of 4 people working full time just to maintain our parcel data and maps. We are all salaried, have a fleet of cars, use specialized equipment and software and work very hard to get our data right. All of these resources are required just to appraise properties once every 4 years. It takes our department a full year just to analyze and value all of the 60,000 parcels and that process involves a large amount of time and effort.

There are no shortcuts, and if you take any, the accuracy of the computer valuation will go down. Currently, most AVMs use a combination of tax data and appraiser data. The tax data gets old pretty fast. It is usually only updated once a year to reflect any changes that occurred during the past tax year, assuming that the property owners notified the tax office of any changes or if the local municipality notified the tax office of any new building permits. In North Carolina, there are 100 counties, and each county has their own system. Some counties have more resources, some less. Some have highly trained staff, many don't. Oftentimes their computer systems are antiquated and their data has not been cleaned up in years or even decades which causes errors, many which are never picked up. Counties have different reval years, and they sometimes change their computer systems, which changes the data that which is used in AVM modeling. Other data users must then adjust their models to work correctly. There are just too many variables to mention and they are always changing.

The point I'm trying to make is this, It takes a huge amount of resources just to acquire accurate data to use in an AVM. If any part of the process is lacking, and bad data gets into the system, or If there is no recent data available in certain areas, then the AVM will be less reliable. Artificial Intelligence, computer modeling, or what ever you want to call it can be a useful tool, but they all require huge amounts of data to work reliably. This takes a bunch of money and manpower, and I don't believe that Google, Corelogic, Zillow, or any one of the other multitudes of companies trying to crack the push button appraisal nut will/can actually devote the resources needed to do do the job with the accuracy required that would eliminate the necessity of using a qualified appraiser. These companies might say that their product can do the job but they are not telling the whole truth.

As more and more qualified appraisers leave the profession, the most accurate and reliable source of data will grow smaller and smaller, forcing them to rely more and more on property tax data, which was never meant to be used in the way that they are trying to use it. All it will take is the next financial crash to show the limitations of their modeling. If financial institutions feel that this risk is acceptable, then who knows what is ultimately going to happen to the residential appraisal profession, I can tell you with a fair amount of certainty that the accuracy and reliability of automated appraisals will plateau in a few short years because the amount of accurate data available to feed these systems will plateau.

Oh, and I almost forgot, no computer model has been invented that can accurately and reliably value the 15 to 20% of properties that don't fit the mass model. These odd or complicated properties have to be done the old fashioned way, and that takes time, and above all, skill.

One take off on this is using machine learning to teach computers to judge appeal based on MLS photos. We know appraisers do this - so one could argue it is a sound concept. I would counter that argument with this: Once the photos become established as an important factor of automated valuation, provided AVMs are the sole source of valuation, then staging companies will offer services to stage each room with their controlled lighting, with cleaning and polishing, photo touch-up (to admissible limits), in and out in one day, - to fool the algorithms for judging appeal. Appraisers, who have compared many MLS photos to the real thing are also aware that the photos can be quite misleading in terms of quality. Yes, unless you are willing to possibly sacrifice 10-20% in accuracy, their is no real replacement for an inspection by a knowledgeable professional who can convert what he sees into reliable numbers.

Nonetheless, the stubborn, reckless enthusiasm of the AVM industry and their cohorts (e.g. some of the AMCs) will likely do much harm, before it is corrected.

Planning on selling your house in a couple of years? Do yourself a favor by taking time to clean each room, touch up with paint, add new blinds and curtains and take a photo for when that day comes. Maybe do one room every quarter, as finances permit. - Get someone you know who is good at photography and Photoshop.

 

[Artemis Fowl]

Assessor data is appropriate for use in mass appraisal models...not as sole source for singular property appraisal purposes. Were that the case...this would be called the AssessorsForum.com.

 

[RCA]

Assessor data is appropriate for use in mass appraisal models...not as sole source for singular property appraisal purposes. Were that the case...this would be called the AssessorsForum.com.

At least here in California, assessor data is the data that is collected by the County Assessor's office and is the basis for MLS data - which is what residential appraisers use. That is, the MLS will pull its initial data for a property from the assessor's database and then let agents add to it. Also, records sales transaction data gets fed into the assessor database and eventually comes back out in the numerous services such as NDCData and CoreLogic's DataExperss.

Not all of the assessor data necessarily gets through to the MLS or other data services for one reason or another.

 

[Eli]

At least here in California, assessor data is the data that is collected by the County Assessor's office and is the basis for MLS data - which is what residential appraisers use. That is, the MLS will pull its initial data for a property from the assessor's database and then let agents add to it. Also, records sales transaction data gets fed into the assessor database and eventually comes back out in the numerous services such as NDCData and CoreLogic's DataExperss.

Not all of the assessor data necessarily gets through to the MLS or other data services for one reason or another.

It’s not that way here. Tennessee is a very diverse market so the percentage estimates above on an avm are ludicrous.

The agents are required to list appraiser involved on a sales transaction to other MLS members in West Tenn. I am pretty sure middle and east Tn are similar in some ways. Most appraisers also contribute GLA, which is grossly different from tax data many times.

There are so many other elements of comparison and market forces that are involved in a very heterogeneous market. (Constantly changing) that make an avm like throwing darts.

I rarely do more than observe tax assessment value. Sometimes it is really close in small homogeneous areas. Many times, it is way off.

They know it and know I know it. Lol

 

[George Hatch]

WRT the data, we spend most of our time qualifying it and transposing it into our reports, and very little time analyzing it. We have software that will do the transposing, and that saves appraisers time, but there's really not that much time to be saved with the analyses because most appraisers don't spend that much time building adjustments and reconciling for value.

On the commercial side the ASB has told appraisers to be careful with DCF analyses as slight changes in the modeling can have profound effects. Appraisers need to be mindful to understand what the model is modeling.

I just took a 3-day course in appraising green homes and one of their approaches is to use a discounted cash flow tool to value the NPV of its energy savings. The term "discounted cash flow analysis" was never uttered in that class. Except by me when I brought it up. That's because I was the only commercial appraiser in the room and have taken the income cap courses. As such, I can do almost everything that model can do except predict the amount of energy involved based on the latitude of the location, the calcs for which are available from other sources online. I can do it by hand with a calculator and I can build my own spreadsheet model in Excel - as can any appraiser who works with income streams as part of their day job.

I know for a fact that nobody else - including the course instructor - understood what they were looking at, how it works and what concepts and applications were in play with that model. I could immediately see that at least a couple of the assumptions that are normally considered to be variables were being projected by that model in a fixed manner, and that the net savings was automatically being depicted at THE VALUE of that attribute as opposed to being simply a reduction in the operating expenses in an Income Approach none of these appraisers were going to complete in any case.

But my biggest problem with this solution is that they were telling those appraisers to use a type of analysis they have no previous exposure to, do not understand, and producing results they cannot explain to their users. If some user was to ask what those results do and don't mean these appraisers would be wholly incapable of explaining it because they haven't been trained in it. If "Cost /= Value" is a thing in our business, then "Income /= Value without a Rate" is also a concept that the appraisers who are trained to work with income will understand to apply on exactly the same basis. .

So yeah, I'm generally opposed to appraisers using tools they don't understand to do things they can't explain. Which with the various forms of RA will be most appraisers who are being told to use them. "Trust the magic box" is not an explanation I would be interested in making to one of my readers. "Trust the magic box because lenders will accept it" is even less acceptable. TO ME. Others will probably disagree.

 

[George Hatch]

BTW, most buyers and sellers in the SFR markets who are developing their own opinions of value (as opposed to taking their broker's word for it) employ a qualitative analysis and rank the subject among their comps. Virtually all brokers on the residential side use a qualitative analysis in their valuations. That's the methodology that is actually used in the market by its participants. Whose actions we are attempting to quantify.

 

[gregb]

Nine out of ten times, when it favors their agenda, brokers will put forth a price per square foot analysis in the ROV.

 

[George Hatch]

BTW, I think we're long overdue for the ASB to incorporate some of the material mentioned in SR5/SR6 (development and reporting Mass Appraisals) as they relate to the use of valuation models and statistical analysis. Although mass appraisals are a distinct appraisal discipline, there are some obvious parallels relating to the use of statistical modeling that probably should have been incorporated all along in SR1/SR2.

A mass appraisal includes

1) identifying the properties to be appraised;

2) defining market area of consistent behavior that affect the creation of value in that market area;

3) identifying characteristics (supply and demand) that affect the creation of value in that market area;

4) developing a model structure to determin the contribution of the individual characteristics affecting value;

5) calibrating the model structure to determing the contribution of the individual characteristics affecting value;

6) applying the conclusions reflected in the mode to the characteristics of the property(ies) being appraised; and

7) reviewing the mass appraisal results

Not EXACTLY the same as SR1 appraising, but we perform all of those steps to one degree of another in our SR1 appraisal work, it's the tools we have been using that are different. Well, now that we're starting to use more of the tools, it becomes important to be mindful about the expectations that would normally be associated with the usage of those tools.

As with SR1, SR5 has similar requirements for development, albeit with somewhat different elaborative comments based on what they're doing and how they're doing it

5-1 In developing a mass appraisal and appraiser must

(a) be aware of, understand, and correctly employ those recognized methods and techniques necessary....
(b) no commit a substantial error of omission or commission....
(c) not render a mass appraisal in a careless or negligent manner

Then you get to 5-4

(a) identify the appropriate procedures and market information require to perform the appraisal.....
(b) employ recognized techniques for specifying property valuation models (this is where they get into model specification and the various formats)
(c) employ recognized techniques for calibrating mass appraisal models

Again, we have parallels for all those functions in SR1 appraising even though most of us haven't previously been thinking in these terms. Even though Fannie built their adjustment grids, we still do some model specification, and we most definitely do model calibration.

Obviously, SR6 will have the commensurate parallels with SR2 as it relates to mass appraising and including the commentary related to the use of some of these methods and techniques.


The point I'm making is this - there are minimum expectations that go along with the use of these tools that go far beyond pressing the "calculate" button, and we should be encouraging appraisers to take the college level courses in statistics that it takes to understand the basics of what these modeling tools are doing and how to competently use them. BEFORE using them.

You know how users sometimes ask you how you came up with that adjustment or why you didn't give the disco ball in the living room any value? Same thing applies with the use of these tools.