Open Source Program For Classification of Home Room Photos

[RCA]

Software & Documentation & Room Photos (inder images folder)
GitHub: https://github.com/wcraytor/KgNN01
https://github.com/wcraytor/KgNN01/blob/main/notebooks/room_type_identification.ipynb
Kaggle: (Almost completed, this will give you 30 hours/week, free, of running the code in Kaggle)

I just finished a Python and Jupyter program for classifying rooms by one of 10 room types: bathroom, dining, gaming, kitchen, laundry, living, office, terrace, and yard. It operates on a dataset of approximately 12,000, 512x512 pixel room photos. You can select the percentage of photos you want it to use, in case you don't have a fast computer. However, this typically lowers the accuracy of classification.

1. If all photos are used for training (70%), validation (15%), and testing (15%), I can get about 98% accuracy on the training data and 88% accuracy on the test data. This is not perfect, of course, but then if an appraiser were to go through these 12,000 photos, he probably couldn't do much better. Just imagine the photo of a living room with a dining room table in it: Is it really a living room or a dining room? Or how about gaming vs office when both typically have TVs and computers? So, some inaccuracy is to be expected, because that is the nature of things.

2. Here is the "Confusion Matrix" produced by just using 25% of the data - and it gets about 87% accuracy on the validation data. That is not bad. Actually, at 100% of the data, the accuracy only went up to 89%. What you see in the colored cells on the diagonal is the number of correct classifications. The others indicate the number of times the given room was incorrectly classified for another. As you might expect, the most confusion occrurs with living vs dining rooms, With the 100% sampling, you get a more realistic picture of confusiong: gameing rooms vs offices and so on. Quite frankly this will be good enought for valuation purposes.



Here are the results for running with 20 "epochs" (i.e. repeated runs, where the model matrix is consistently modified with partial deriivatives, aka the hill-climging approach.



Next Stage: I will modify the program to create models that use an array of photos for each property to predict the house price residuals - to be used in predicting the residual for the Subject Property. IF that works sufficiently good, then that CLOSES THE LOOP for fully objective and automated appraisal. Bad News vs Good News depending on your perspective.

 

[cashcatz]

This is awesome, thanks for sharing.

 

[sputnam]

Software & Documentation & Room Photos (inder images folder)
GitHub: https://github.com/wcraytor/KgNN01
https://github.com/wcraytor/KgNN01/blob/main/notebooks/room_type_identification.ipynb
Kaggle: (Almost completed, this will give you 30 hours/week, free, of running the code in Kaggle)

I just finished a Python and Jupyter program for classifying rooms by one of 10 room types: bathroom, dining, gaming, kitchen, laundry, living, office, terrace, and yard. It operates on a dataset of approximately 12,000, 512x512 pixel room photos. You can select the percentage of photos you want it to use, in case you don't have a fast computer. However, this typically lowers the accuracy of classification.

1. If all photos are used for training (70%), validation (15%), and testing (15%), I can get about 98% accuracy on the training data and 88% accuracy on the test data. This is not perfect, of course, but then if an appraiser were to go through these 12,000 photos, he probably couldn't do much better. Just imagine the photo of a living room with a dining room table in it: Is it really a living room or a dining room? Or how about gaming vs office when both typically have TVs and computers? So, some inaccuracy is to be expected, because that is the nature of things.

2. Here is the "Confusion Matrix" produced by just using 25% of the data - and it gets about 87% accuracy on the validation data. That is not bad. Actually, at 100% of the data, the accuracy only went up to 89%. What you see in the colored cells on the diagonal is the number of correct classifications. The others indicate the number of times the given room was incorrectly classified for another. As you might expect, the most confusion occrurs with living vs dining rooms, With the 100% sampling, you get a more realistic picture of confusiong: gameing rooms vs offices and so on. Quite frankly this will be good enought for valuation purposes.

View attachment 103516

Here are the results for running with 20 "epochs" (i.e. repeated runs, where the model matrix is consistently modified with partial deriivatives, aka the hill-climging approach.

View attachment 103517

Next Stage: I will modify the program to create models that use an array of photos for each property to predict the house price residuals - to be used in predicting the residual for the Subject Property. IF that works sufficiently good, then that CLOSES THE LOOP for fully objective and automated appraisal. Bad News vs Good News depending on your perspective.

So... you are aiding in the eventual elimination of appraisers? Why would any appraiser need or want software to classify rooms from photos? I don't know about you but, I recognize a kitchen or bedroom or bathroom at the time that I see it.

 

[RCA]

So... you are aiding in the eventual elimination of appraisers?

You are apparently not at all aware of all of the work that is progress on identifying and extracting information from photos over the past several decades. Other well-known appraisal software developers have been doing work on this sort of thing for at least the past decade. I am looking at the state of the art techniques to assess their power and accuracy. This sample is for me just a stepping stone. But it also may be of interest to a very few on this forum, as it begins to demonstrate the power or Neural Networksk and I hope give a sense of how they work.

In any case, I was not clear in saying that neural networks + RCA + MARS will completely automate appraisal. It will, however, with some other tools, such as those for preprocessing photos, be able to, for all practical purposes, completely automate the process of creating the Sales Grid in a fully objective manner. Other work will require manual work or other automated processes. And an appraiser knowledgeable in programming and statistics (the right kind of statistics) will be needed to manage and monitor the automation and validate the final product. So, there will always be appraisers, but far fewer and more highly skilled. And the appraisers will more likely be called "Valuation Engineers" - because they will have qualifications more on par with typical software engineers.

Why would any appraiser need or want software to classify rooms from photos?

Eventually these methods will become the new standard.

I don't know about you, but I recognize a kitchen, a bedroom, or a bathroom at the time that I see it.

But you can't do a good job of ranking, say, all the MLS photos for a market area in terms of their value contribution, in a period of a couple of hours, and then objectively and accurately place the subject photos in said ranking so that residual value contributions can be estimated. Maybe in another year you will see that more clearly.

 

[RCA]

One might argue that the standards can be held in check by appraisers. I doubt that is the case.

1. Either appraisers and their standards will go with the flow and keep up with the technology.

2. Or one day, that standards will simply be sidelined and the old technology switched out for the new - as has happened so many times in the past several hundred years.

3. I am guessing the current appraisal system will be heavily criticized and shown to be highly inferior and more inefficient than the newer technology, new laws and regulations will be enacted .... faster than you think.

4. I also believe that the future will be Independent Valuation Engineers, who will focus on specific market areas, perhaps several adjacent metro areas. or several smaller states.