So Punk, Can You Prove Your Adjustments, Go Ahead Prove Them !

[Deleted member 130081]

For paired sales our office uses a proprietarty program we developed to compare MLS data against eachother over 16 criteria. It compares across those criteria and allows you to specify which item you are trying to measure, and tolerances for the other 15 items. It's a beast and takes about 5-7 minutes to cross-compare about 300 properties. But the needles in the haystack are there, though it requires computing power and algorithims. The total number of calculations is a factorial and runs into the hundreds of thousands. It is excel based but should not be, as excel was not designed as a procedural database. We are playing with the idea of converting it to SQL and making it available to the appraisal public. My week is consumed with appraisal and I never seem to find the time to work on it, so it remains a pet project. Every time I see one of these discussions about paired sales I get a little more motivated, as the technology exists to runs pairs for each appriasal. We do it every day. I am not the smartest guy in the room, just a tinkerer. I guess I'm just not convinced other appraisers would find it valuable as they have concluded paired sales is only an abstraction taught in school.

Very interesting and I am intrigued. Full disclosure, I am of the opinion that dollar adjustments, at least in the formats/forms they exist today, have little-to-no place in appraisal, due to the question if people even purchase in such a way in the first place and secondly if they do, albeit subconsciously, that the data available is not good or plentiful enough to produce a credible result that might be better than what an appraiser could conclude from what I call a simple trends analysis. I've bolded some things I wonder about.

There are 33 potential adjustments on a 1004, not including the spaces open for anything else.

You are able to find 300 relevant properties in the markets you work in?

Is a needle in the haystack a quantity you feel good about?

Do you understand the hundreds of thousands of calculations? On this point I expect you might - it aint rocket science, but garbage in garbage out comes to mind.

You say the technology exists and you use it everyday, but is it credible?

I do believe a program could be developed to extract adjustments. What I question, is the quality of the result, and the correlation between the expense of the method and the quality of result. Doesn't seem like a good use of resources to me, when other solutions to the problems are out there, for the whopping cost of zero.

A bit off here, but lots of appraisers use tablets and such in the field on an inspection. They claim it saves them lots of time and its entirely worth it. I don't think I could argue that it might in fact save them time. For me, a piece of paper and a pencil works pretty good. I am not sluggish on a computer, so transferring/touching the data a second time into the form report isn't a big deal for me - certainly about the last thing that comes to mind when I wonder where all my time went. Paper is nice too, because I wont lose it when my computer crashes. So, when I consider using a tablet, I wonder why I would spend so much money for so little benefit and additional risk? Pencils cost almost nothing, rarely break, mostly work in the rain and don't need to be recharged or upgraded. The point is maybe tech can be created and used in appraisal, but at what expense and what benefit? I think that's a relevant consideration.

Why is it so important to "prove" adjustments? How vital to the process is it really? How much benefit are we getting from so much toil???

 

[Su Long]

Very interesting and I am intrigued. Full disclosure, I am of the opinion that dollar adjustments, at least in the formats/forms they exist today, have little-to-no place in appraisal, due to the question if people even purchase in such a way in the first place and secondly if they do, albeit subconsciously, that the data available is not good or plentiful enough to produce a credible result that might be better than what an appraiser could conclude from what I call a simple trends analysis. I've bolded some things I wonder about.

There are 33 potential adjustments on a 1004, not including the spaces open for anything else.

You are able to find 300 relevant properties in the markets you work in?

Is a needle in the haystack a quantity you feel good about?

Do you understand the hundreds of thousands of calculations? On this point I expect you might - it aint rocket science, but garbage in garbage out comes to mind.

You say the technology exists and you use it everyday, but is it credible?

I do believe a program could be developed to extract adjustments. What I question, is the quality of the result, and the correlation between the expense of the method and the quality of result. Doesn't seem like a good use of resources to me, when other solutions to the problems are out there, for the whopping cost of zero.

A bit off here, but lots of appraisers use tablets and such in the field on an inspection. They claim it saves them lots of time and its entirely worth it. I don't think I could argue that it might in fact save them time. For me, a piece of paper and a pencil works pretty good. I am not sluggish on a computer, so transferring/touching the data a second time into the form report isn't a big deal for me - certainly about the last thing that comes to mind when I wonder where all my time went. Paper is nice too, because I wont lose it when my computer crashes. So, when I consider using a tablet, I wonder why I would spend so much money for so little benefit and additional risk? Pencils cost almost nothing, rarely break, mostly work in the rain and don't need to be recharged or upgraded. The point is maybe tech can be created and used in appraisal, but at what expense and what benefit? I think that's a relevant consideration.

Why is it so important to "prove" adjustments? How vital to the process is it really? How much benefit are we getting from so much toil???

The 300 pieces of data is just the limit excel can run in a reasonably efficient manner, as currently configured. Out of those 300 properties, or corresponding 44,850 pairs, there are probably five to ten pairs for the more conforming criteria (GLA, lot size, pool, baths, etc.). Given all the variables at play (16 in this case), the odds of finding two properties that line up close enough save for a single item (the item for which you run the pair) is small. That's why it's so difficult to find them manually, and that's why we try to use as many properties as possible. Often we only have 100+ properties that even qualify to be compared, in which case the odds are that much smaller.

When I say equations, I am referring to excel equations. I wrote them, so I do understand them, but the actual math behind the program is limited to subtraction...pretty simple. The program just compares one property in the set to every other property and determines if there is another property that is the same except for a particular item, say a pool. It then subtracts the two sales prices. For 16 criteria this results in 717,600 comparisons (44,850 pairs x 16 criteria). It seems like a lot, but all that it is doing is a textbook paired sale analysis considering 16 different possible criteria. Why 16? Because some of the criteria have to be broken down into binary sub-criteria/dummy variables: golfcourse view (1) or no golfcourse view (0); short sale (1) or no short sale (0), etc. By making all that data binary, simple subtraction between properties for any given criteria results in a similarity (0) or a difference (1).

Is it credible? Interesting question. To the extent that hedonic pricing informs appraisal, I believe it is credible. To the extent that markets, particularly real estate markets, are inefficient, different paired sales are not exact. Sometimes they are even curiously different. I attribute this to market inefficiency and information omitted from MLS, and/or info not picked up by the algorithm that scrapes the comment section of the MLS. We also use a different regression program comparing that same data set and the answers are usually quite similar. Over hundreds of appraisals, I feel safe in concluding that the program(s) work sufficiently well and that our market is reasonably efficient to support the use of hedonic pricing/paired sale. I am in a suburban area of southern California, so I imagine it would be less useful in rural Iowa.

Appraisal is a field of applied economics. As such, it is dirty and noise infiltrates every step of the process. I am far from a neo-classicist, but I do believe that market efficiency holds at least in the weak form, and that hedonic pricing as developed in paired sale informs more than it confuses the adjustment...when done competently. The program just makes that pairing feasible, because there is NO way I could even approach the data crunching required to obtain credible results as a human with bills to pay and clients to please . Speaking of....

 

[Deleted member 130081]

The 300 pieces of data is just the limit excel can run in a reasonably efficient manner, as currently configured. Out of those 300 properties, or corresponding 44,850 pairs, there are probably five to ten pairs for the more conforming criteria (GLA, lot size, pool, baths, etc.). Given all the variables at play (16 in this case), the odds of finding two properties that line up close enough save for a single item (the item for which you run the pair) is small. That's why it's so difficult to find them manually, and that's why we try to use as many properties as possible. Often we only have 100+ properties that even qualify to be compared, in which case the odds are that much smaller.

When I say equations, I am referring to excel equations. I wrote them, so I do understand them, but the actual math behind the program is limited to subtraction...pretty simple. The program just compares one property in the set to every other property and determines if there is another property that is the same except for a particular item, say a pool. It then subtracts the two sales prices. For 16 criteria this results in 717,600 comparisons (44,850 pairs x 16 criteria). It seems like a lot, but all that it is doing is a textbook paired sale analysis considering 16 different possible criteria. Why 16? Because some of the criteria have to be broken down into binary sub-criteria/dummy variables: golfcourse view (1) or no golfcourse view (0); short sale (1) or no short sale (0), etc. By making all that data binary, simple subtraction between properties for any given criteria results in a similarity (0) or a difference (1).

Is it credible? Interesting question. To the extent that hedonic pricing informs appraisal, I believe it is credible. To the extent that markets, particularly real estate markets, are inefficient, different paired sales are not exact. Sometimes they are even curiously different. I attribute this to market inefficiency and information omitted from MLS, and/or info not picked up by the algorithm that scrapes the comment section of the MLS. We also use a different regression program comparing that same data set and the answers are usually quite similar. Over hundreds of appraisals, I feel safe in concluding that the program(s) work sufficiently well and that our market is reasonably efficient to support the use of hedonic pricing/paired sale. I am in a suburban area of southern California, so I imagine it would be less useful in rural Iowa.

Appraisal is a field of applied economics. As such, it is dirty and noise infiltrates every step of the process. I am far from a neo-classicist, but I do believe that market efficiency hold at least in the weak form, and that hedonic pricing as developed in paired sale informs more than it confuses the adjustment...when done competently. The program just makes that pairing feasible, because there is NO way I could even approach the data crunching required to obtain credible results by a human with bills to pay and clients to please . Speaking of....

Outstanding. Thank you for your reply. I am far from convinced that methods like this are the best course for us moving forward, nor am I convinced they produce credible results, or at least results more credible than what I can do without all that work, however I am very happy to see you at least understand what it is you are doing (I only had to google one word thank you very much - thought I was reading a post from Terrel for a second there). I would be interested to hear from you again after you have done some self-reflection of your method - for now you seem to be pleased and satisfied with the results, though I would wonder how much digging you have done to test the methods so far, compared to how much time has been spent just setting it up and using it . I am applauding, not knocking you here.

"...an appraiser must reconcile the quantity and quality of the data..."
"...an appraiser must reconcile the methods and techniques used..."
"...an appraiser must not allow the assignment conditions of the client to be so limited, that the credibility of the report is compromised..."

I think about that stuff all the time.

 

[Thomas Holding]

The 300 pieces of data is just the limit excel can run in a reasonably efficient manner, as currently configured. Out of those 300 properties, or corresponding 44,850 pairs, there are probably five to ten pairs for the more conforming criteria (GLA, lot size, pool, baths, etc.). Given all the variables at play (16 in this case), the odds of finding two properties that line up close enough save for a single item (the item for which you run the pair) is small. That's why it's so difficult to find them manually, and that's why we try to use as many properties as possible. Often we only have 100+ properties that even qualify to be compared, in which case the odds are that much smaller.

When I say equations, I am referring to excel equations. I wrote them, so I do understand them, but the actual math behind the program is limited to subtraction...pretty simple. The program just compares one property in the set to every other property and determines if there is another property that is the same except for a particular item, say a pool. It then subtracts the two sales prices. For 16 criteria this results in 717,600 comparisons (44,850 pairs x 16 criteria). It seems like a lot, but all that it is doing is a textbook paired sale analysis considering 16 different possible criteria. Why 16? Because some of the criteria have to be broken down into binary sub-criteria/dummy variables: golfcourse view (1) or no golfcourse view (0); short sale (1) or no short sale (0), etc. By making all that data binary, simple subtraction between properties for any given criteria results in a similarity (0) or a difference (1).

The program just makes that pairing feasible, because there is NO way I could even approach the data crunching required to obtain credible results as a human with bills to pay and clients to please . Speaking of....

Su Long, I would love speak with you sometime regarding your program. I have developed a similar matched pairs tool for use in our tax office that is very similar, however I came up with a slightly different approach to work around the limitations of Excel.

When you are dealing with so many combinations and permutations, Excel tends to bog down due to memory limitations. My latest version can compare bedrooms, full baths, and half baths (0-10 units each), Marshall & Swift quality categories (6), detached / attached garages and carports (0-6 units each), basements in three configurations (unfinished, partially finished, and fully finished), fireplaces (0-3 units) and pools (0-1 units) for a total of 11 categories. The combination matrix on this thing would fill up every cell on a sheet. I have done up to 14 categories, however by culling my data (pre-filtering out some elements) eliminated the need for the extra categories. By comparing just the text inputs (feature codes) of each category attributed to each comp with every other comp and flagging ones that differ by only one category, it relieved Excel of having to perform excessive computations. The only real computation takes place after the paired sales have been found. The actual comparison of the comps is handled by the vba code itself which is much more efficient. The program can compare approximately 1000 properties in a little over a minute however larger samples take more time due to the exponential increase in comparisons. If matched pairs are found, the output tells me that "Parcel# 1234567 with GARAGE-ATTCH-2 is SUPERIOR to Parcel# 765432 with GARAGE ATTCH-1 VALUE = $12500, After finding enough matched pairs for each element (a statistically significant number), I can use the results to build the lookup tables in my mass appraisal model.

In regards to comments by other appraisers on this thread, matched pairs are more common than you might think. Does this technique give you an adjustment for every possible element? Absolutely not! For the elements that you can't find using matched pairs, there are other techniques that you can apply using readily available data that will provide supportable adjustments. There is no SINGLE technique that I know of that reliably provides ALL the adjustments that might be necessary in a typical appraisal, however, by knowing what data is available to you, you can devise methodologies and analysis tools that work with the information you have. BTW, I don't work in a large metropolitan area. Our county is pretty small with about 60,000 parcels total, and less than half of those are improved, and yet I am still able to extract useful data with this technique.

Su Long, I sincerely hope that you continue to work on your program and develop a tool that appraisers can use. This is the kind of development we should all support, because it will ensure that individual appraisers can compete with "Big Data" by offering a superior product. Maybe you could to do a Kickstarter campaign to fund the development of your program. I do understand how much effort it takes to make something like this. It took me over two months to come up with a working version of my program and that was doing nothing else for 8 hours each day,

 

[Terraform]

I think I would rather support the current method, which is based on an artful approach to value. GL.

 

[Terraform]

What you seemingly are trying to develop is an algorithm to feeling based buying decisions. There is no algorithm to this problem.

 

[Dublin ohio]

How about: If you can't amaze them with ability. Baffle them with BS

 

[jay trotta]

Tom H;
"In regards to comments by other appraisers on this thread, matched pairs are more common than you might think. Does this technique give you an adjustment for every possible element? Absolutely not! For the elements that you can't find using matched pairs, there are other techniques that you can apply using readily available data that will provide supportable adjustments. There is no SINGLE technique that I know of that reliably provides ALL the adjustments that might be necessary in a typical appraisal, however, by knowing what data is available to you, you can devise methodologies and analysis tools that work with the information you have. BTW, I don't work in a large metropolitan area. Our county is pretty small with about 60,000 parcels total, and less than half of those are improved, and yet I am still able to extract useful data with this technique.

Su Long, I sincerely hope that you continue to work on your program and develop a tool that appraisers can use. This is the kind of development we should all support, because it will ensure that individual appraisers can compete with "Big Data" by offering a superior product. Maybe you could to do a Kickstarter campaign to fund the development of your program. I do understand how much effort it takes to make something like this. It took me over two months to come up with a working version of my program and that was doing nothing else for 8 hours each day,"

That would appear to be a significant investment of Time & Knowledge and I applaud you for that, it would also appear that residential appraising has become one of the more difficult business's to be involved in lately. It will not translate to a Days Pay for a Days work under it's current environment now or into the future.

 

[Terrel L. Shields]

the legitimate exclusion of outliers (which in theory should be explained in the appraisal report)

Which begs the questions...Is everything within your search parameter "fair game"? Do you reduce your search area to get rid of outliers as possible? At what point is the search area "too small"? Isn't at some point, everything an "outlier" if not the closest, most recent and proximate sales? Does anything in USPAP even require three comps? Secondary market wants extra listings and sales then bangs you over the head for not either weighing them or excluding them. The end question, of course, does this extra vetting amount to a better understanding of the value? Is it mere sop for the reviewers? Is the report more accurate for including mindless piffle rather than ignoring meaningless data? If so, why? How?

The nature of the appraisal today (residential - secondary market) is a contrived format which is a hold over from decades upon decades ago. Perhaps a big picture analysis using this "big data" is long overdue. Throw out the notion that we need "three" comps and need to "adjust" them. A fixed adjustment grid has never been required. Many of us don't use them, rather use a form of ranking analysis that brackets the value you solve for, and is one of nine recognized methods of adjustments. But Fannie only recognizes 1...and does so with gusto, and to what effect? Does it make the value more accurate and I certainly think there is little evidence that it does.

 

[Frederick R. Ruffell]

Very interesting and I am intrigued. Full disclosure, I am of the opinion that dollar adjustments, at least in the formats/forms they exist today, have little-to-no place in appraisal, due to the question if people even purchase in such a way in the first place and secondly if they do, albeit subconsciously, that the data available is not good or plentiful enough to produce a credible result that might be better than what an appraiser could conclude from what I call a simple trends analysis. I've bolded some things I wonder about.

There are 33 potential adjustments on a 1004, not including the spaces open for anything else.

You are able to find 300 relevant properties in the markets you work in?

Is a needle in the haystack a quantity you feel good about?

Do you understand the hundreds of thousands of calculations? On this point I expect you might - it aint rocket science, but garbage in garbage out comes to mind.

You say the technology exists and you use it everyday, but is it credible?

I do believe a program could be developed to extract adjustments. What I question, is the quality of the result, and the correlation between the expense of the method and the quality of result. Doesn't seem like a good use of resources to me, when other solutions to the problems are out there, for the whopping cost of zero.

A bit off here, but lots of appraisers use tablets and such in the field on an inspection. They claim it saves them lots of time and its entirely worth it. I don't think I could argue that it might in fact save them time. For me, a piece of paper and a pencil works pretty good. I am not sluggish on a computer, so transferring/touching the data a second time into the form report isn't a big deal for me - certainly about the last thing that comes to mind when I wonder where all my time went. Paper is nice too, because I wont lose it when my computer crashes. So, when I consider using a tablet, I wonder why I would spend so much money for so little benefit and additional risk? Pencils cost almost nothing, rarely break, mostly work in the rain and don't need to be recharged or upgraded. The point is maybe tech can be created and used in appraisal, but at what expense and what benefit? I think that's a relevant consideration.

Why is it so important to "prove" adjustments? How vital to the process is it really? How much benefit are we getting from so much toil???

I would like to hear your thoughts on Digital Photography and Fax machines, I am still using Polaroids and pasting them onto the paper hard copy and then sending the whole hard copy report to the client via the Pony Express! To hell with technology brother!