Automated valuation models (AVMs) are helpful for providing quick property valuations, but their accuracy varies depending on the AVM used.
Different AVMs are built using different types of machine learning algorithms, so they each have their own level of accuracy. One AVM may produce an entirely different result than another, and they are not always the most precise.
For instance, the AVM that Xome® uses to calculate the Xome Value® of properties has a different level of accuracy than other real estate applications. The level of accuracy is determined using a “confidence score,” which represents a measure of the reliability associated with the AVM in use. You will see a confidence score alongside the Xome Value where it is generated for auction properties.
Without having trust in an AVM, what is the point of using it? That’s where confidence scores come in. They provide users with a frame of reference, giving them a look at how much they may be able to trust a particular AVM. Knowing how accurate and reliable the model’s results are, buyers, sellers, and real estate professionals can feel secure about their decisions.
The importance of confidence scores in real estate valuations
AVM confidence scores are important for a variety of reasons. Not only do they measure the accuracy and reliability of the model’s final estimate, but they also may provide accountability of the model itself. This helps to battle regressivity in AVMs and reduce bias.
They are especially helpful for risk assessment. Confidence scores can set risk thresholds by both the model and its users when evaluating AVM estimates.
Developers may design the AVM with specific thresholds to determine the level of confidence it assigns to each valuation. By establishing a minimum confidence score threshold, the model can filter out AVM estimates that do not meet a desired level of confidence. As a result, the AVM may not generate an estimate when it has insufficient data and a confidence score below a certain threshold.
Similarly, the AVM user can also filter out properties from their research if the final estimate’s confidence score does not meet their expectations. This way they can avoid relying on less reliable estimates that could spoil any real estate plans they have.
How do AVM confidence scores work?
Confidence scores are calculated using a metric called the Forecast Standard Deviation (FSD). This metric is the standard deviation of the percentage sales errors and shows the level of variability in the AVM estimate. The percentage sales error is the ratio of sales error divided by selling price.
The lower the FSD percentage is, the higher the probability of accuracy is for the AVM estimated value. But the FSD is different from the confidence score, which has a higher probability of accuracy the higher the confidence score is.
When looking at property listings on Xome, you will notice the Xome Value alongside a range of estimated prices. The high and low values are derived from the FSD value.
The confidence score uses the FSD percentage to subtract it from 100%. If the FSD is 10%, the confidence score is calculated using the following formula:
100% – FSD 10% = 90% Confidence Score
Keep in mind that you cannot compare the confidence scores of one AVM to those of a different AVM provider. Each AVM provider defines and uses their own FSD and confidence scores, and the calculation methods may vary.
A high confidence score vs. a low confidence score
The higher the confidence score is, the lower the level of risk there is associated with the estimated value based on the available property data. On the other hand, a lower confidence score indicates a higher degree of uncertainty, suggesting that the valuation has higher risk and may be less reliable.
However, interpreting high and low confidence scores does not necessarily translate into “good” or “bad” scores. The exact threshold for categorizing as good or bad is subjective. It depends on several factors, such as the purpose of the valuation and the risk tolerance of the user.
A “good” confidence score might be considered above a certain threshold, such as 70%, 80%, or 90%, depending on the specific AVM used. The person using the AVM determines these thresholds.
Some users may consider a confidence score above 70% or 80% as good, while others may have more conservative thresholds of above 90%. The 90% group could consider anything below that as “bad” and would filter those out of their research.
Just because a confidence score might appear good to one person, it may not be considered the same to another.
The evolving role of confidence scores in AVMs
Confidence scores are not consistent across different AVM providers and their interpretation also differs from user to user. Even so, they are still a valuable resource for getting a rough idea of the home’s value.
The more standardized AVMs become, standardization of confidence scores may soon follow — and the easier it will be to compare them.
See our confidence scores in action while searching properties for auction on Xome.com.
