"Now I am become Death, the destroyer of worlds" - Robert J. Oppenheimer

[RCA]

Or, rather: "Now I am become Death, the destroyer of traditional appraisal."

So, as already stated in a previous post, today I just added two new apps, glmnetUI and mgcvUI to earthUI.

All three apps will currently kick out an Excel spreadsheet that is a raw intermediate sales grid, with price estimates, residuals, value contributions, adjustments, net and gross total adjustment amounts and percentages, followed by the adjusted sale price.

All done in a flash. Assuming you have the data, you could sit down and go through a ton of old appraisals to see just how far the are off and where appraisers made their critical mistakes.

1. earthUI, is tha app for "earth()" or "MARS". It will usually give the optimal sales grid, unless data is lacking. In that case, we have:
2. glmnetUI, with is probably the most advanced linear regression that gives good R2 and CVR2 values. Its downside is the total gross adjustment percentages, which are often 2-3 times larger than Earth's.
3. Then there is mgcvUI, which provides curvilinear (smoothed) value contribution curves for the variables.
4. If you feed the knots created by earthUI into mgcvUI, you will arguably get the best output. Almost the same R2 and CVR2 as earthUI, the same total gross adjustment percentages, but more realistic smoothed curvilinear variable curves. The downside: The underlying math is based on tensors, at least in the case where you are using interactions.

These apps destroy traditional appraisal. They really do.

I doubt UAD 3.6 will see the light of day, at least not without dramatic changes. Of course, the dummies over at the GSEs - well, they ARE dummies.

I think this can't be anything but good. But it is a change for appraisers; You will not need to do programming - these apps will pretty much do everything, except break down the residuals and review/fine-tune the model.

 

[RCA]

BTW, these are open source programs - available to anyone.

 

[Digger88]

Looks good bro when will you have drag and drop data import like Spark etc my brain hurts reading this but I think you got a fan haha

Transitioning from a traditional appraiser to a "Valuation Engineer" using the Residual Constraint Approach (RCA) sounds intimidating when reading a 41-page technical paper. However, the core philosophy is actually quite practical: let the software handle the math for measurable features, and use your expert intuition solely to rank the unmeasurable features (Condition, Quality, Appeal).

To integrate this into your business with maximum speed and minimal disruption, you must treat this as a software-augmented workflow rather than a mathematical re-education. You do not need to understand the underlying calculus of Multivariate Adaptive Regression Splines (MARS); you just need to know how to operate the machinery.

Here is your blueprint for a fast, low-disruption implementation.

---

1. The Tech Stack & Setup (One-Time Friction)​

To avoid a steep learning curve, do not try to build the algorithms from scratch. Rely on the established open-source tools mentioned in the paper.

• Install R and RStudio: This is free and handles the heavy lifting.
• Install the earth Package: This is the specific R package maintained by Stephen Milborrow that runs the MARS regression.
• Create a Standardized Folder Template: Build a master folder template (as outlined by Craytor) containing subfolders for Data, Code, and Documents. Duplicate this template for every new appraisal.
• Build the Core Excel Workbook: Create your master MLS.xlsx file with the five required sheets (MLSData, Regression Variables, Calculations, Interactions, Aggregations). This becomes your standard control panel.

2. Standardizing Your Data Pipeline​

The RCA model is hungry for data, but feeding it shouldn't take extra time if you standardize your MLS exports.

• Export Volume: Stop pulling 5 to 10 comps. Broaden your MLS search to pull 300 to 600 closed sales over a longer period (or a wider radius for homogeneous neighborhoods).
• The 15-to-1 Rule: Ensure you have roughly 15 closed sales for every variable you intend to measure.
• Format: Ensure your MLS export is RESO Certified (RCF format) and cleanly maps to your MLS.xlsx file.

3. The Day-to-Day Workflow (The Fast Track)​

Once your templates are built, the day-to-day appraisal process shifts. You will spend less time agonizing over grid adjustments and more time looking at a ranked list.

1. Run the Script (Stage 1): Dump your MLS export into the folder, hit "run" on your R script, and let MARS generate the baseline price and measured feature value contributions (GLA, Lot Size, Age, etc.).
2. Sort by Residuals: The software will generate a list of all your comps, sorted by their "Residual/SF" (the leftover value not explained by the math). This creates a 0.0 to 10.0 CQA (Condition-Quality-Appeal) score.
3. The Appraiser's Job (Stage 2): Look at this ranked list. Use your inspection notes and photos to place your subject property into this hierarchy. If it is better than 60% of the comps, give it a CQA of 6.0.
4. Calculate Final Value: The software adds the MARS-estimated price to your subject's residual value. This is your final value conclusion.

4. Client Presentation & Compliance (Zero Disruption)​

The biggest risk to your business is confusing your clients or lenders. The RCA accounts for this by generating three distinct types of Sales Grids.

• Use the GSE Upload Grid: The math guarantees that every comp in your grid will perfectly adjust to match your subject's final value. Use your software to format this data specifically for standard forms (like Fannie Mae) and upload it into your existing form software (like Alamode).
• Keep the Complex Grid in the Workfile: Keep the "RCA Processing Grid" for your own auditing and workfile compliance.
• Leverage Proof II: Because the math dictates that the sum of the residual breakdown adjustments will never change the final adjusted sale price, you can confidently defend your value without having to manually tweak individual grid lines to hit a target number.

 

[RCA]

Looks good bro when will you have drag and drop data import like Spark etc my brain hurts reading this but I think you got a fan haha

Transitioning from a traditional appraiser to a "Valuation Engineer" using the Residual Constraint Approach (RCA) sounds intimidating when reading a 41-page technical paper. However, the core philosophy is actually quite practical: let the software handle the math for measurable features, and use your expert intuition solely to rank the unmeasurable features (Condition, Quality, Appeal).

To integrate this into your business with maximum speed and minimal disruption, you must treat this as a software-augmented workflow rather than a mathematical re-education. You do not need to understand the underlying calculus of Multivariate Adaptive Regression Splines (MARS); you just need to know how to operate the machinery.

Here is your blueprint for a fast, low-disruption implementation.

---

1. The Tech Stack & Setup (One-Time Friction)​

To avoid a steep learning curve, do not try to build the algorithms from scratch. Rely on the established open-source tools mentioned in the paper.

• Install R and RStudio: This is free and handles the heavy lifting.
• Install the earth Package: This is the specific R package maintained by Stephen Milborrow that runs the MARS regression.
• Create a Standardized Folder Template: Build a master folder template (as outlined by Craytor) containing subfolders for Data, Code, and Documents. Duplicate this template for every new appraisal.
• Build the Core Excel Workbook: Create your master MLS.xlsx file with the five required sheets (MLSData, Regression Variables, Calculations, Interactions, Aggregations). This becomes your standard control panel.

2. Standardizing Your Data Pipeline​

The RCA model is hungry for data, but feeding it shouldn't take extra time if you standardize your MLS exports.

• Export Volume: Stop pulling 5 to 10 comps. Broaden your MLS search to pull 300 to 600 closed sales over a longer period (or a wider radius for homogeneous neighborhoods).
• The 15-to-1 Rule: Ensure you have roughly 15 closed sales for every variable you intend to measure.
• Format: Ensure your MLS export is RESO Certified (RCF format) and cleanly maps to your MLS.xlsx file.

3. The Day-to-Day Workflow (The Fast Track)​

Once your templates are built, the day-to-day appraisal process shifts. You will spend less time agonizing over grid adjustments and more time looking at a ranked list.

1. Run the Script (Stage 1): Dump your MLS export into the folder, hit "run" on your R script, and let MARS generate the baseline price and measured feature value contributions (GLA, Lot Size, Age, etc.).
2. Sort by Residuals: The software will generate a list of all your comps, sorted by their "Residual/SF" (the leftover value not explained by the math). This creates a 0.0 to 10.0 CQA (Condition-Quality-Appeal) score.
3. The Appraiser's Job (Stage 2): Look at this ranked list. Use your inspection notes and photos to place your subject property into this hierarchy. If it is better than 60% of the comps, give it a CQA of 6.0.
4. Calculate Final Value: The software adds the MARS-estimated price to your subject's residual value. This is your final value conclusion.

4. Client Presentation & Compliance (Zero Disruption)​

The biggest risk to your business is confusing your clients or lenders. The RCA accounts for this by generating three distinct types of Sales Grids.

• Use the GSE Upload Grid: The math guarantees that every comp in your grid will perfectly adjust to match your subject's final value. Use your software to format this data specifically for standard forms (like Fannie Mae) and upload it into your existing form software (like Alamode).
• Keep the Complex Grid in the Workfile: Keep the "RCA Processing Grid" for your own auditing and workfile compliance.
• Leverage Proof II: Because the math dictates that the sum of the residual breakdown adjustments will never change the final adjusted sale price, you can confidently defend your value without having to manually tweak individual grid lines to hit a target number.

There you go. That is a nice, aggressive attitude. Open-source software is appropriate for appraisal when a large population is concerned about being discriminated against. With it, you have nothing to hide - anyone can run programs, if they want to go to the trouble (not really that much) of getting the data. Anyone who wants to play shenanigans with appraisals has to reckon with the possibility that their work could be reasonably checked by anyone.

But while you don't have to understand the math in detail, especially the more advanced math like Tensors, you will need experience working with these tools to tweak the parameters and determine whether the models make sense. You need to get in the habit of developing a deep understanding of how people make decisions about purchasing or renting property. You need to develop a sense of how feature configurations operate across different neighborhoods and large market areas to shape market value. There are many configurations available, and these are influenced by market value. A lot of this is 101 Appraisal, such as what happens when big employers move in and out of areas, you have old money in certain areas that never seems to budge, location, politics, crime, schools, zoning, and on and on and on and on. And all these features seem to operate based on "quantum relationships". It' is like art in a way: There are magic combinations of ingredients that affect appeal and consequently price like magic. And you need to extract this information from the market in a strictly objective and quantifiable way. With the RCA method, we are fortunate to have vital constraints available that allow us to capture otherwise purely subjective features, such as the impact of condition, quality of construction, aesthetics, functional utility, and the rest of everything that constitutes "appeal."

The future appraiser becomes the manager of this new toolset. Whether he is good at his trade, whether he succeeds in this career path, and to what extent, will depend on surviving the attacks of rather hostile warriors who will show up on his doorstep from time to time.

I would guess, therefore, that future appraisers will need a pretty substantial understanding of the toolset to defend before clients, users, audit boards, and in court. If you can't fully handle the necessary defensive posturing when the going gets rough, you may want to find a proxy to stand in. Your proxy will want to be assured, though, that you used defensible methods in your valuations.

So, yes, conditionally, you don't need to understand the algorithmic and programmatic details to do the work. But if that is the case, you had better make sure you can find a defensive proxy if and when you come under fire. In any case, the more and deeper understanding you have, so much the. better.

Nowadays, you can spend hours discussing subjects with Claud, ChatGPT, or Grok to make sure you understand concepts inside and out. You might never actually feel you can 100% understand neural networks or tensors, to the extent of writing mathematical proofs on a blackboard, but going that far is rarely necessary.

 

[RCA]

BTW, I have about 1.8M sales transactions around the SF Bay Area via automated "Broker Feed", going back 25 years with all photos. That is just for these counties: San Mateo, Santa Clara, Santa Cruz, Monterey, Alameda, Contra Costa. I could get small San Benito I guess. But then I can pull about 2,500 records at a time from other systems such as BAREIS, MetroList and SF MLS. The latter is probably good enough for my remaining use cases. Larger databases incure more costs and more maintenance. So, there has to be a limit.

 

[Tom D]

Is there a manual somewhere that translates your tech talk into people english. Talk about Mars, now we are at Neptune leaving the galaxy.
Digger 88, i like the new term you have used for this profession, 'valuation engineer'. However, fannie can hire monkies to input all that data, or mr AI will imput it. And if it put's out accurate adjustments, then CU takes it and runs, as they say. I think the proper term should be 'valuation computation', human not needed.
Thank you both for that conformation.

 

[Digger88]

I like knowing what I don't know but this is beyond 99.9% of appraisers grasp, maybe less who knows. I joke with my bro on here @Functional Obs and send him grids on difficult appraisals that I label God teir adjustments that are pulled out of my rear lol. After 15,000 of these they're always right (sometimes haha) I have seen everything but sometimes these pesky lenders and lawyers want to see my work, dammit

Looks impressive but
until you develop a drag and drop app how many people do you think are going to implement this and if it's as impressive as you think why wouldn't you make a drag and drop app and cash in?

 

[RCA]

I have just added the Excel "Intermediate Sales Grid" to earthUI. - I am mucking around with the fixed variables right now to get them just right.
1. Of course, the grid, except for the residual breakdowns, is automatically loaded from your MLS spreadsheet and all the MARS calculations (most value contributions and adjustments). In the lower section where we typically put these subjective valuations, there is the Residual amount followed by the usual subjective values: Condition, Quality of Construction, Functions; Utility, View, Design/Style/Aesthetics - and then you have 6 or so blank rows for whatever you want. As you add in values and adjustments, your residual and adjusted residual amounts decrease until 0.

So, what do you want with Spark. Well, sure I have used it. If you have a comp and you need to fill it in - and you can see what other appraisers are using, you could argue that their opinion combined with your view of the comp photos is better than just your view of the comp photos. It would save you some time to pull in Spark.
1. But remember, you will still have to give that feature, not only an "adjustment" now, you have to have a "value contribution" for that feature. That can all be done by using your best judgment as a way of interpreting what is assigned to the residual. However, you break down that residual into subjectively valued components, it will have no impacton the adjusted sale price. That is fixed by the constraints. You are making too much of this work now. See, with RCA, you don't have to lose sleep over those adjustments you have to make - they will have no impact on sales value. The worst is that somebody says, well, they don't agree with your value assessment of the condition. OK. So what? It's not going to change the value conclusion. You will find that this work will go somewhat faster. It depends on how much regard you want others to have of your valuation of "Art Work". Whoops, the value is already set to be the residual. What you are doing is giving an interpretation of why the value is what it is -- and in the worst case, you can say part of the breakdown is that the house sold for more or less than what it's real MV was (in your opinion). Now, one might think that would impact the value conclusion. Yes, the subject has to be $0 for the variable: "Over-or-under-MV", so you will get an adjustment to this variable for the subject. - But remember, you already assigned a residual to the subject, based on your overall ranking of it against the sorted comparable residuals. And you know what? That ranking by residual is the Market deciding on what kinds of features are worth more than others. It is really the market making the real decision here on price, and yes, you are placing the subject in this sorted list, and that is the one point where you do make a subjective decision that matters. An auditor, however, can make sure that the model and the comparables' ranking make sense and that your placement of the subject in that ranking also looks kosher. If you have a fixer ranked with some nice homes, that will be pretty evident.

2. So, this work can go pretty smoothly, I would say, once you have developed some skill in using R/earth or earthUI.

3. The end result is that your value opinion will feel right. It should be right in the middle of the "sweet spot". --- You get this feeling it is perfect. If you have done your job.

====
Now as it stands, if you are using a 1004 or eventually the UAD 3.6 you will have to transcribe my "Intermediate Sales Grid" values to the form. But the last thing I will try to do, near the end of the year, I imagine, is create my own UAD 3.6 and transfer the values into it.

Spark? Well, was that ever a good idea? To rely on other appraisers' value decisions. I do not think you will miss it. I have looked at their most recent incarnations and one thing I can say - they absolutely have very little understanding of statistics. Still with the current status of things, they do save appraiesrs time and help them sleep better at night relyingon someone else judgment --- but remember you are still responsible for the product. Yet of course, shoud you get pulled before the board, you can always say "I pulled this data from Spark!"

 

[RCA]

Here is a zip file of an Excel "Intermediate Sales Grid" that is created almost instantaneously.
Note has the 30 best comps out of 1500 sales. Look at their Gross Adj Percentages. Not bad. And they all have the same adjusted sale price. How long would it take you to make such a sales grid? So, perfect. (well I still have some more tweaking on the grid lines near the top to make it look more like a 1004.

Notice I did break down the residual of the first comp.

 

[RCA]

Here is a zip file of an Excel "Intermediate Sales Grid" that is created almost instantaneously.
Note has the 30 best comps out of 1500 sales. Look at their Gross Adj Percentages. Not bad. And they all have the same adjusted sale price. How long would it take you to make such a sales grid? So, perfect. (well I still have some more tweaking on the grid lines near the top to make it look more like a 1004.

Notice I did break down the residual of the first comp.

There is a mistake in the residual breakdown, --- see if you can find it.