Avm's, Are They Reliable?

[Tim Hicks (Texas)]

This was the topic of another thread, but somehow it degenerated.


AVM's are widely accepted. I have limited experience with them. This is what I have seen. What I do know is that most data is old and the ones I have seen use "projected" (inflated for the rise in the market?) sales prices and not the actual sales price. Why are they allowed to use sales 2-3 years old and underwriters around here will not accept anything over six months without a lengthy explanation. What does the sales price on a property two years ago have to do with current market conditions? Why would they corrupt the data more by projecting (adding) to that sales price? Garbage in, garbage out. Frankly, AVM's are no better than the programmers allow them to be and the programmers are not appraisers. Appraiser assisted AVM's are worse, because we of all people should know the limits without a physical inspection and the quality of bad data that is presented. Why would we want to help any AVM company? It boggles my mind!


Do AVM's verify their data? Do they understand the concept of competing and superior additions? Do they consider the most relevant and recent data? Do they understand anything? Why are they so highly regarded? What is the only benefits they provide? price and time?


Price: I refuse to lower my fees to combat an inferior product.


Time: I can only hope that I can decrease my turn times to combat this obvious factor that the mortgage industry has a relevant gripe.

 

[Austin]

Tim:
I hate to tell you this, but if you are using a commercial residential forms program, you are using an AVM. The valuation model can be automatic or semi-automatic but it is still a valuation model. It is the algorithm that constitutes the valuation model meaning procedures and methods you are forced to follow or that the programs performs without your input. Most forms programs make the most significant adjustment for size using one of three algorithms none of which are market oriented or data supported. The entire FNMA appraisal form & system coupled with their guidelines and review procedure constitutes an enforced valuation model. Some forms programs are so sophisticated that they warn you if you deviate from the algorithm and make you explain why you deviated. Even the appraisal regulatory system is involved by enforcing FNMA guidelines some times even when the appraisal is not an FNMA assignment.
This is not an AVM fight, it is a fight of whether the algorithm involves an appraiser or does not involve an appraiser. The only legitimate method of appraising is to have a system that extracts the sequence of adjustments from the market and correctly analyzes data and this requires a warm blooded thinking appraiser because each assignment is different. Any method will work in a large cookie cutter subdivision with a lot of sales. I can write you one in about ten minutes that will work perfectly in those areas.

 

[Carnivore]

Tim,

I had an intersting conversation with an underwriter from one of the TOP 4 banks in the nation the other day. I just happened to be appraising her house. We talked about appriasals and how they differed from around the nation. She stated something very interesting that had to do with AVM products. Interestingly she saiid her bank uses them but not like they have in the past. Apparently, things like value were getting in the way of a smooth loan process. Borrowers were complaining that this three or four step process was taking to long. For example, they start out with simple equity to tax value with Credit deciding the risk. If thats marginal then an AVM. If that sucks then a 2075(rarely) or a 2055 exterior or maybe a 2055 interior.

I ask her about a few things as it pertains to accuracy and the confidence level. She was surprised to hear me say CONFIDENCE Level. I ask her has she or the bank ever ask the AVM peddler to explain or verify there confidence level. She did not understand at first. So I said simply, How do you know that the value GIVEN is accurate. If its based on a condfidence level have you questioned them on what that means and how they arrived at it. In other words make them prove it.

She proceeded to tell me that the bankwas not real happy with the results and that they were gradually heading back to FULL URAR or 2055 Interiors w/ sketch. Seems that the AVM's in our market anyway are not working very well at all. Especially in the so called cookie cutter S/D's. They did expect problems in high end mixed use neighborhoods, but were very surprised at the results of cookie cutters. Its gets better, even condo developements are screwed up.

ON SOAP BOX
Essentially, here is whats plaqueing AVM's. It seems mixed use, foreclosures, huge builder incentives, close outs, investor
buys, family transfers, multple purchases, DEED CORRECTIONS(no sale), FLIPS, PUMP Sales by realtors, fire sales by realtors, City and County special incentive programs, ERRORS: square footage, age, baths and room counts, CENSUS tract mixes that throw curve balls at any neighborhood factors. The list goes on and on.

OFF SOAP BOX

Ya'll got to love this

One last thing. An AVM in the hands of a market competent, appraiser are unbeatable.


A question occured to me after I posted the phrase in blue above. Why has there be no solicitiations or offers to sell an AVM to me, an appraiser.

I think I will start a poll on how many of us have been e-mailed, faxed or offered by mail an AVM program.

 

[Austin]

Andrew:

I ask her about a few things as it pertains to accuracy and the confidence level.

Andrew, the present system and methodology of appraising can't answer that question either. Not only that, but I have never seen a correctly adjusted data set of comparable sales from an FNMA form appraisal. The present system can't even define a perfectly adjusted data set much less show a confidence level. The only method of creating a confidence level is to use about 30 sales, have the regression predict the prices, compare the variance of the actual and predicted prices, and calculate the confidence level. You make a good point, but your own appraisal methodology fails your test of meassuring confidence.

 

[Carnivore]

Austin,

I am not questioning my methodology, I am also not questioning the AVM peddlers methodology either. I am pointing out that theere data sets are so corrupt that they are useless and that because of this corruption the confidence level would be extremely low. Given that, how does one arrive at the confidence level when they dont know what data being used is actual sales or not and if they are valid sales is the physical data accurate?

I currently dont have a good AVM for my residential use. I wish I did and will as soon as I can find one for sale.

 

[Ross (CO)]

Tim, ......Everything you stated in your lead-in comments is just what so many of us have known and understood about these computed valuation products. One-by-one the uncomfortable experiences of their use and application will come home to roost and some clients (lending entities who relied on them to issue a loan ) will have sincere second thoughts about using them again,....perhaps. I reported on a home in early March, not far from where I live, and one situated within a classically identified neighborhood as tract development of generally 175-275K pricing, with the repeated design styles every so many doors away. Despite what the data says, and shows, and despite the fact that I used 2 additional pending sales and an active listing (since we were coming out of winter and good to show the current pulse of activity) this individual was dis-pleased with my value opinion. Any support for what he had in mind would have seen his home as the highest-valued home in the neighborhood, period, and (if sold tomorrow) would be about $30K higher than anything sold within 2 sq. miles since July of 2002. His home still only has a 2-car garage and no basement walk-out, and a very minimal and quite boring landscape.

So, what does he do ? He writes a letter to the State officials who oversee my license and practice, and sends them an obvious truck-load of mumbo jumbo raw MLS data from over the entire metro area, and even national "appreciation data" to claim that his house "HAS TO BE" worth at least $ zzz,zzz ! ----Here's the AVM part of the story. His letter to State (copied to me when quickly resolved) claims that he had an "appraisal" done in Jan. 2002 (14 months before I saw the home) by a large, well-recognized national bank which came in at $299,600. It is the ..$600 part that surely speaks to an AVM product. This "appraisal" was not sent in with all the other garabage data numbers this fellow dredged up with the help of a Realtor friend he referred to. This agent supplied a good 7 or 8 LISTINGS, not sales, from a considerably higher-priced and partly-custom development about 3 to 4 miles away from subject. No doubt any AVM generated with the proper setting of data search parameters would have stayed right there in immediate subject subdivision, and found sufficient comp data to stay perhaps +/- $10K from where my value conclusion was.

This individual wrote of woes of hardship upon his family caused by my report. Obviously, my report never got him his new loan (at a new off-the-wall value), and he is forced to confront the company who gave out that $299,600 value for their loan in Jan. '02 ! At best, peak sale prices in subdivision at that time were $265-270K. Thank you, AVM, for serving your customer/borrower well. He's in tough times and distraught, and needs relief. So, how does the AVM-user (lender) now come back to defend themselves ? I am very curious to know how they verbally, and in-writing, defend their use of valuation models when poorly programmed and outcomes are so greatly flawed.

State's reply to h/o was an expression of (my words here) feeling his pain, reminding him that Colorado r.e. history has seen the ups-and-downs of property values before....and the report you shared with us from March of this year...is without any violations of license law, and demonstrates an application of thorough market data analysis which were summarily verified by us, and also offers you additional pending contract and active list data to supplement the sales used.

Al Queida is not completely eradicated....but they are greatly weakened. AVM's are still going to be used for some residential lending.....but they are continuing to show their flaws and un-reliability. Location, location, location, the very creedo we have always heard from traditional realty agents.....easily swept aside by them, and by AVM programmers, when a different goal needs to be achieved.

 

[Doug Smith SRA]

Former Speaker of the House Tip O'Neil is famous for saying, "All politics is local." To paraphrase Speaker O’Neil, there is good claim that “all appraising is local.” The criticism of the AVM product generally centers on missing the particulars within a market and sometimes a neighborhood. There is another issue more important and that is accountabilty.

AVM's, however, have been alive and well right under or nose up until now but they have not engendered much debate. These AVM's are being cranked out by local assessor offices. How come, they aren't branded unreliable etc? The reason is plain. They are tweaked by real persons who go out and photo the subject, in some cases measure them and interview the owners. But the real reason they work is the homeowner has a stake in getting it right and when it is not right, the hapless assessor hears about it.

Any form of valuation will work if there is accountability. Accountability is absent from the AVM industry. Look on your software report offerings. There is no form for a review of an AVM. Until the lender puts in a system of reviews, or some sort of report card, the AVM produces have free reign to produce what ever they can spit out. Fannie and Freddie spent a lot of time and effort to revise the 2000 review form. Now, I would like to have them produce a form for appraisers to review the AVM product. Steve Santora said in another context on another thread about sooner seeing vampires sun bathing, wearing cruxifixes than that will happen.... Garlic anyone???

 

[Tim Hicks (Texas)]

So, if lenders want to rely on them, why are they allowed to use 2-3 year old sales? We aren't.

 

[Mike Simpson]

WE TOLD YOU SO!!!

Years ago many of us were discussing the multitude of reasons why AVM's not only wouldn't work effectively, but also why they were a danger to the economy in their present form.

The reason?--many of those who've touted the use of and sing the praises of AVM's aren't appraisers at all! As Andrew - Charlotte so aptly pointed out; in the hands of a qualified, knowledgeable appraiser AVM's could be a very valuable tool. However, when opportunist possessing little knowledge of appraising or appraisal concepts jump into the business, something is bound to go wrong.

First of all you've got to understand where the AVM's data comes from, and who their data entry operators are (appraisers for the lack of a better word). Some companies mine data from their approved appraisers--and being overwhelmingly greedy--take a large portion of the appraisers fee...25-50% (or more to in some cases). Many of these appraisers (in my experience) are sweat shop entities who send unlicensed people into the field (further cutting the fee) to perform inspections and complete assignments. Naturally, the data's going to be corrupted--not to mention some laws being broken!

Other companies utilize tax assessor's records--don't even get me started there, what's happening with taxes in my region has a lot of folks up hopping mad!!!

Finally, my clients are telling me the same thing that Andrew - Charlotte were saying; they're using these things (AVM's) much less often because they're proving to be just too unreliable. Moreover, I've many collegues who've told me that their clients are giving them the same feedback.

I've no doubt there is a use for AVM's, and that they'll be used with more caution in the future, but far from putting an appraiser out of a job--current trends strongly suggest otherwise.

-Mike

As an afterthought; I've had many people ask me about some of these online house value predictors. I tell them very often you get what you pay for, however, in this instance I don't think you get even that!

 

[Fred]

Posted
"I ask her about a few things as it pertains to accuracy and the confidence level. "

Information is available about "accuracy." The easiest ratio test is the same one assessors use. Run values for properties where they have an appraisal. One of the large providers had been running within 10% of the appraisal 50% of the time. They reported a big jump in accuracy, because assessor, or less reliable data, had been replaced by better appraisal data. The rate went to within 10% of the appraisal 65% of the time. The company expressed doubt that they would be able to hold 65%, if the sales market and refi's slowed, hurting their ability to have good, fresh data.

IMO, within 10% of the appraisal 65% of the time is not to be taken lightly. Once that number gets above 75%, you would have to start asking the key question. What percentage of the time will a second appraisal come in within 10% of the first? There is certainly a point of "close enough" for someone underwriting a large number of properties.

There is an upcoming seminar of "predictive technologies." Countrywide and other valuation players will be showing the results of their latest neural network and artidical intelligence forays.