Regulation Archives

AVMetrics Completes Transition to Predictive Testing Methodology

After running our legacy and PTM™ testing processes in parallel for several years, AVMetrics has formally completed its transition to Predictive Testing Methodology (PTM™) as our sole testing platform. The legacy testing process has been retired.

This transition was not abrupt, and it was not done in isolation. On the contrary, this milestone marks the completion of a multi-year effort that involved model-builders and AVM users.

Model vendors participated actively in PTM™ testing, recognizing the importance of delivering results that better reflect how their models perform for end users—lenders, servicers, and investors. Over the past several years, AVMetrics worked in close collaboration with AVM vendors and institutional clients to evaluate both approaches side by side. This parallel period gave our clients, AVM vendors, and our own team the opportunity to validate results side by side. That validation is now conclusive, and continuing to maintain the legacy system no longer serves the needs of the industry.

Why the legacy method is being retired

Traditional AVM testing compares model estimates to sale prices after those prices—and the listing data that precedes them—are already available to the models. As we’ve documented extensively, many AVMs incorporate MLS listing prices into their estimates, a phenomenon known as listing price anchoring, which inflates apparent accuracy in ways that don’t reflect real-world performance. When a model knows the listing price before producing its estimate, the resulting test measures something closer to consistency with known information than true predictive accuracy.

The anchoring effect isn’t theoretical—it’s quantified. Zillow reports a median error of approximately 1.7% for on-market homes but 7.2% for off-market homes. Redfin reports nearly identical figures: roughly 2.0% on-market and 7.6% off-market. That gap—a three- to four-fold difference in accuracy—represents the influence of listing price anchoring on AVM performance. Independent analysis by the AEI Housing Center corroborates this: their 2024 evaluation of five AVM providers found that “springiness”—how much a model’s estimate jumps when listing or sale prices become available—showed the widest variance in scores of any criterion tested, with some providers receiving failing grades. Since the vast majority of AVM use cases involve unlisted properties—refinances, home equity lending, portfolio assessments, loss mitigation—testing that includes listing data paints a misleading picture of how models will actually perform when it counts.

PTM™ solves this by using AVM estimates produced before properties are listed. Our Model Repository Database (MRD™) stores monthly valuations for every residential property in America from every participating AVM. By matching these pre-listing estimates against eventual arm’s-length sale prices, we isolate each model’s genuine predictive capability on a level playing field.

Aligned with regulatory expectations

This transition also reflects longstanding supervisory expectations for how AVMs should be tested, validated, and governed. While regulatory frameworks have evolved toward a more principles-based approach, the core elements of sound model oversight remain consistent: independent validation, outcomes-based testing, and a clear understanding of model limitations. Three authorities continue to frame these expectations:

Interagency Appraisal and Evaluation Guidelines (2010), Appendix B. The federal banking agencies direct that:

To ensure unbiased test results, institutions should compare AVM results to actual sales data in a specified trade area or market prior to the information being available to the model. If more than one AVM is used, each should be validated.
Institutions should evaluate the underlying data used in the model(s), including data sources and types, frequency of updates, quality control performed on the data, and the sources of data in states where public real estate sales data are not disclosed.
Institutions should not rely solely on validation representations provided by an AVM vendor.

Model Risk Management Expectations. Current supervisory guidance reinforces the importance of:

Independent validation and effective challenge
Outcomes analysis comparing model outputs to real-world results
Ongoing monitoring aligned to model use and materiality

AVM Quality Control Standards (2025). The QC Standards establish that AVMs must be accurate, reliable, and nondiscriminatory, with institutions responsible for testing, monitoring, and controlling for bias and performance risk. While the QC Standards do not prescribe a specific methodology, it places responsibility on institutions to demonstrate that their approach is reasonable and well-supported. PTM™ operationalizes these principles at scale: independent, pre-information testing against actual arm’s-length sales, conducted continuously across every participating model.

What this means for the industry

For lenders and AVM users: AVMetrics’ independent AVM testing now reflects how models actually perform in the scenarios that matter most—refinances, HELOCs, portfolio assessments, and other situations where no listing price exists. The results you receive are a more realistic measure of the accuracy you can expect in production.

For AVM vendors: This transition reflects a shared direction. Vendors have participated in and supported PTM™ testing, recognizing the importance of delivering performance transparency that aligns with client expectations. Vendors continue to receive anonymized comparative analysis showing where their models stand relative to the field.

For the industry: This marks a move toward a more consistent, independent, and forward-looking standard for AVM validation—one that supports better model selection, stronger governance, and more credible cascade optimization and compliance decisions.

What hasn’t changed

What hasn’t changed is our role.

AVMetrics continues to provide independent, transparent AVM testing and validation aligned with the expectations set forth in the Interagency Guidelines and AVM Quality Control Standards. This independence remains a key distinction. Many AVM performance reports available in the market are produced by model providers evaluating their own models. While these efforts provide useful monitoring insights, they do not offer the same level of objectivity, comparability, or cross-model evaluation. AVMetrics’ role continues to be providing a consistent, third-party view across all participating models, supporting a more complete and defensible understanding of performance. We rigorously evaluate model performance and deliver comparable, objective results across providers—supporting due diligence, model validation, and ongoing monitoring. We continue to provide Model Preference Tables™, cascade optimization, geographic performance rankings, and all of the analytical products our clients rely on.

Just as importantly, we continue to provide clear guidance on appropriate use:

When an AVM is fit for purpose
When one model is more appropriate than another
And when an AVM should not be relied upon at all

The scope and rigor of our testing have only increased. Our focus remains the same: enabling clients to make informed, defensible decisions about AVM use through independent validation and consistent, evidence-based analysis.

Why AVMetrics’ Fair Housing Methodology Surpasses Vendor Approaches

The Fair Housing analyses published by AVM vendors such as Veros and Clear Capital represent important early efforts to evaluate potential disparate impact in automated valuation models. These studies contribute useful perspective to an evolving area of the industry, but they are inherently constrained by scope, methodology, and—most importantly—objectivity. Their findings are self-assessments rather than independent evaluations: each vendor analyzes only its own model, using its own data and assumptions, and typically concludes that little to no bias exists, which limits their usefulness for broader risk management and supervisory purposes.

Regulated institutions, however, must operate under much more rigorous expectations. The new Interagency AVM Quality Control Standards require lenders to demonstrate that AVMs used in credit decisions are independently validated and fairly applied. This standard cannot be meaningfully satisfied by vendor-authored whitepapers alone.

AVMetrics’ methodology is designed specifically to meet these supervisory needs. Rather than focusing on individual model performance within internally defined samples, AVMetrics conducts standardized, national-level testing across 700,000 to 1 million transactions each quarter. This approach ensures that fairness conclusions reflect real-world market diversity and enables consistent evaluation across models, markets, and time.

AVMetrics independently tests eight different dimensions in which AVMs could potentially disadvantage protected classes, including coverage rates (hit rate), accuracy, precision, and other core performance measures. To support statistically meaningful comparisons, AVMetrics has invested in neighborhood-level demographic data, enabling analysis across comparison neighborhoods- avoiding the masking effects of county-level aggregation while preserving sufficient sample size beyond census-tract granularity.

Further, AVMetrics applies Standardized Mean Difference (SMD)—the same effect-size metric commonly used in fair-lending analytics—providing a clear measure of whether disparities are material, not simply detectable. In contrast, many model-specific analyses typically use raw accuracy differences or simple correlations, which offer no interpretive scale for examiners assessing practical significance. AVMetrics’ approach produces metrics that are grounded in established methodology, interpretable, and defensible.

As the next generation of AVMs incorporates increasingly complex machine learning and generative AI techniques, vendor-driven testing becomes even less transparent. AVMetrics’ methodology is intentionally model-agnostic: we can evaluate the fairness and performance of traditional hedonic models, GBDT-based systems, deep learning models, or hybrid AI architectures with equal rigor. As models become more opaque, the need for a neutral, independent evaluator becomes increasingly essential.

In contrast to analyses intended to provide general assurance around individual models, AVMetrics delivers regulatory-grade evidence. By identifying how model risk and policy risk can interact to generate disproportionate impacts—an expectation embedded in the new regulatory framework—our testing equips lenders with the actionable intelligence needed to inform, calibrate, and justify their risk-policy decisioning.

As regulatory expectations around AVM fairness continue to mature, institutions must move beyond model-specific assurances toward independent, repeatable, and scalable evaluation frameworks. AVMetrics’ fair housing methodology is purpose-built to meet these expectations, providing lenders with nationally consistent, statistically rigorous, and model-agnostic evidence of AVM performance and potential disparate impact. By aligning testing design with supervisory standards and real-world production environments, AVMetrics enables institutions not only to identify and manage fair-lending risk, but also to demonstrate compliance with confidence in increasingly complex valuation ecosystems.

What DOJ’s Disparate-Impact Rollback Doesn’t Change About AVM Fairness

The Department of Justice’s recent move to eliminate disparate-impact liability under its Title VI regulations has raised understandable questions across housing and credit markets. But for lenders, GSE partners, and valuation providers preparing for the AVM Quality Control Standards, one thing is clear:

The obligations around AVM fairness haven’t gone away.

The interagency AVM rule—effective October 1, 2025—explicitly requires institutions to establish policies, practices, procedures, and control systems to ensure AVMs comply with applicable nondiscrimination laws. That requirement remains fully intact. So do the supervisory expectations of prudential regulators, FHFA, and CFPB around managing fair lending and bias risk in automated systems, whether or not DOJ narrows its enforcement tools under Title VI.

Even with political shifts, the industry continues to operate under:

The Fair Housing Act, where disparate-impact liability is still recognized by the Supreme Court.
ECOA/Reg B fair lending expectations, which continue to incorporate statistical evidence of adverse outcomes.
Growing scrutiny of AI and automated valuation, highlighted by recent GAO recommendations urging clearer guidance on emerging technology risks.

In short: Regulatory pendulums swing—but AVM fairness risk remains.

Institutions still need independent, statistically rigorous testing to understand whether their AVMs or cascades produce unjustified disparities, and to document business justification and alternatives when they arise. That’s where AVMetrics’ fifth-factor validation fits the bill. Our analysis is national, extensive, independent, thorough, examiner-ready and tested for significance.

Best Practices for AVM Testing: Why Sale Prices Matter More Than Appraisal Values

The Challenge of AVM Testing in Quality Control
Automated Valuation Models (AVMs) have become a cornerstone of the appraisal quality control process, with many lenders and Appraisal Management Companies (AMCs) implementing what is commonly known as the “15% rule.” Under this framework, if an appraised value falls within 15% of the AVM estimate, the appraisal undergoes only a cursory review. Conversely, variances exceeding 15% trigger more thorough scrutiny or require review by another appraiser.

While this approach offers operational efficiency, it fundamentally misunderstands the distinction between precision and accuracy. This article examines why using appraisals as benchmarks for AVM testing creates a dangerous circular logic and proposes a more robust methodology centered on actual sale transactions.

Understanding Precision Versus Accuracy
Comparing AVM values to appraised values measures consistency—or precision—not accuracy. This distinction is critical yet often overlooked in the industry. While understanding the consistency between different valuation methods has operational value, particularly for ensuring that credit decisions are based on reliable estimates regardless of methodology, it serves a fundamentally different purpose than accuracy testing.

True accuracy in property valuation refers to how closely an estimate reflects the actual market value at a specific point in time. Market value, regardless of the valuation method employed, is universally defined as the most probable price negotiated between a willing buyer and seller in an arm’s length transaction. Therefore, the only meaningful measure of accuracy is the comparison to the actual negotiated sale price—the endpoint of the price negotiation process.

The Circular Logic Problem
The practice of using appraisals as benchmarks for AVM testing raises a fundamental question: Are we testing whether the model provides an accurate value consistent with actual sale prices, or merely a precise value consistent with appraisal estimates? This distinction has significant implications.

If AVMs tested against appraisals are mistakenly labeled as “accurate” rather than simply “consistent with appraisal estimates,” the quality control process becomes streamlined in a potentially dangerous way. More appraisals pass through without complications or delays from further review, creating a self-reinforcing cycle. This circular process—using AVMs in quality control to validate appraisals while simultaneously using appraisals as benchmarks to test AVMs—is fraught with risk and undermines the integrity of both valuation methods.

The Critical Role of Blind Testing
Determining the accuracy of any valuation estimate requires blind testing—the property’s sale price must be unknown when the estimate is produced. While implementing blind testing for appraisals presents significant challenges due to USPAP standards requiring appraisers to review and consider the terms of sale contracts, AVMs can and should be tested blindly with relative ease.

Most AVM vendors already conduct blind testing routinely as part of their quality assurance programs. This process allows modelers to understand model accuracy before a sale price becomes known—a critical capability for calibrating models and improving their predictive accuracy.

The Importance of MLS Data Suppression
The blind testing process must extend to MLS data, particularly listing prices. While listing prices occasionally match final sale prices, they more typically represent the starting point of buyer-seller negotiations. However, the listing price itself is not arbitrary—considerable analysis has already been completed to determine it.

Real estate professionals conduct thorough studies of the property, neighborhood, and relevant market activity—many of the same activities required for property valuation—before establishing a listing price. When AVMs have access to this information, they gain a significant advantage that may not reflect their true predictive capabilities. Therefore, proper testing must ensure that listing prices remain unknown when estimates are produced, maintaining the integrity of the blind testing process.

MLS data suppression also aligns with real-world use cases. In prominent AVM applications such as mortgage refinancing or home equity loans, the properties serving as collateral are rarely listed on MLS. Most lenders actually prohibit refinancing properties that are currently or recently listed. Testing AVMs without MLS data therefore provides a more realistic assessment of model performance in actual lending scenarios.

Regulatory Guidance and Compliance
The breadth and scope of AVM testing—including methodology selection and resource allocation—must be tailored to each institution’s unique circumstances. Regulatory guidance provides flexibility for organizations to demonstrate compliance within the context of their specific risk assessments.

While no explicit prohibition exists against using appraisal values as benchmarks, the Interagency Appraisal and Evaluation Guidance offers clear direction in Appendix B: “To ensure unbiased test results, an institution should compare the results of an AVM to actual sales data in a specified trade area or market prior to the information being available to the model.” This affirmative statement strongly supports the use of sales transactions as the foundation of AVM testing methodology.

Conclusion: Building a Robust Testing Framework
The distinction between measuring precision and accuracy in AVM testing is not merely academic—it has practical implications for lending decisions, risk management, and regulatory compliance. While comparing AVMs to appraisals may offer insights into consistency between valuation methods, it cannot and should not be mistaken for accuracy testing.

A robust AVM testing methodology must incorporate the following core principles:

• Use actual sale transactions as the primary benchmark for accuracy testing

• Implement truly blind testing protocols that exclude sale prices during estimate generation

• Suppress MLS listing price data for the benchmark property only to ensure unbiased testing. All other MLS-derived information—such as property characteristics, comparable sales, and broader market metrics—should remain fully available and not suppressed.

• Recognize the different purposes served by precision and accuracy measurements

As the regulatory guidance makes clear, institutions have the flexibility to design testing programs that suit their specific needs and risk profiles. However, this flexibility should not obscure the fundamental requirement: AVM testing must measure actual predictive accuracy against real market outcomes, not merely consistency with other valuation estimates. Only through rigorous, unbiased testing against actual sale prices can institutions truly understand and rely upon the accuracy of their automated valuation models.

Feds to Lenders: Take AVMs Seriously

Regulators are signaling that they are going to be looking at how AVMs are used and whether lenders have appropriately tested them and continuously monitor them for valuation discrimination. This represents a change in the focus on AVMs and the need for all lenders to focus on AVM validation to avoid unfavorable attention from government regulators.

On Feb 12, the FFIEC issued a statement on examinations from regulators. It specifically stated that it didn’t represent a change in principles, nor a change in guidance, and not even a change in focus. It was just a friendly announcement about the exam process, which will focus on whether institutions can identify and mitigate bias in residential property valuations.

Law firm Husch Blackwell published their interpretation a week later. Their analysis included consideration of the June 2023 FFIEC statement on the proposed AVM quality control rule, which would include bias as a “fifth factor” when evaluating AVMs. They interpret these different announcements as part of a theme, an extended signal to the industry that all valuations, and AVMs in particular, are going to receive additional scrutiny. Whether that is because bias is as important as quality or because being unbiased is an inherent aspect of quality, the subject of bias is drawing attention, but the result will be a thorough examination of all practices around valuation, including AVMs, from oversight to validation, training, auditing, etc.

AVM quality has theoretically been an issue that could be enforced by regulators in some circumstances for over a decade. What we’re seeing is not just an expansion from accuracy into questions of bias. We’re also seeing an expansion from banks into all lenders, including non-bank lenders. And, they are signaling that examinations will focus on bias, which is an expansion from the theoretical requirement to an actual, manifest, serious requirement.

Our Perspective on Brookings’ AVM Whitepaper

As the publisher of the AVMNews, we felt compelled to respond to the Brookings’ very thorough whitepaper on AVMs (Automated Valuation Models) published on October 12, 2023, and share our thoughts on the recommendations and insights presented therein.

First and foremost, I would like to acknowledge the thoroughness and dedication with which Brookings conducted their research. Their whitepaper contains valuable observations, clear explanations and wise recommendations that unsurprisingly align with our own perspective on AVMs.

Here’s our stance on key points from Brookings’ whitepaper:

Expanding Public Transparency: We wholeheartedly support increased transparency in the AVM industry. In fact, Lee’s recent service on the TAF IAC AVM Task Force led to a report recommending greater transparency measures. Transparency not only fosters trust but also enhances the overall reliability of AVMs.
Disclosing More Information to Affected Individuals: We are strong advocates for disclosing AVM accuracy and precision measures to the public. Lee’s second Task Force report also recommended the implementation of a universal AVM confidence score. This kind of information empowers individuals with a clearer understanding of AVM results.
Guaranteeing Evaluations Are Independent: Ensuring the independence of evaluations is paramount. Compliance with this existing requirement should be non-negotiable, and we fully support this recommendation.
Encouraging the Search for Less Discriminatory AVMs: Promoting the development and use of less discriminatory AVMs aligns with our goals. We view this as a straightforward step toward fairer AVM practices.

Regarding Brookings’ additional points 5, 6, and 7, we find them to be aspirational but not necessarily practical in the current landscape. In the case of #6, regulating Zillow, it appears that existing and proposed regulations adequately cover entities like Zillow, provided they use AVMs in lending.

While we appreciate the depth of Brookings’ research, we would like to address a few misconceptions within their paper:

Lender Grade vs. Platform AVMs: We firmly believe that there is a distinction between lender-grade and platform AVMs, as evidenced by our testing and assessments. Variations exist not only between AVM providers but also within the different levels of AVMs offered by a single provider.
“AVM Evaluators… Are Not Demonstrably Informing the Public:” We take exception to this statement. We actively contribute to public knowledge through articles, analyses, newsletters (AVMNews and our State of AVMs), quarterly GIF, a comprehensive Glossary, and participation in industry groups, task forces. We also serve the public by making AVM education available, and we would have been more than willing to collaborate or consult with Brookings during their research.

But, we’re obligated not to just give away our analysis or publish it. Our partners in the industry provide us their value estimates and we provide our analysis back to them. It’s a major way in which they improve, because they’re able to see 1) an independent test of accuracy, and 2) a comparison to other AVMs. They can see where they’re being beaten, which means opportunity for improvement. But, in order to participate, they require some confidentiality to protect their IP and reputation.

We should comment on the concept of independence that Brookings emphasized. Independent evaluation is exceedingly important in our opinion, as the only independent AVM evaluator. Brookings mentioned in passing that Mercury is not independent, but they also mentioned Fitch as an independent evaluator. We agree with Brookings that a vendor who also sells, builds, resells, uses or advocates for certain AVMs may be biased (or may appear to be biased) in auditing them; validation must be able to “effectively challenge” the models being tested.

We do not believe Fitch satisfies ongoing independent testing, validation and documentation of testing which requires resources with the competencies and influences to effectively challenge AVM models. Current guidelines require validation to be performed in real-world conditions, to be ongoing, and to be reported on at least annually. When there are changes to the models, the business environment or the marketplace, the models need to be re-validated.

Fitch’s assessment of AVM providers is focused on each vendor’s model testing results, review of management and staff experience, data sourcing, technology effectiveness and quality control procedures. Fitch’s methodology of relying on analyses obtained from the AVM providers’ model testing results would not categorize them as an “independent AVM evaluator,” as reliance on testing done by the AVM providers themselves does not meet any definition of “independent” per existing regulatory guidance. AVMetrics is in no way beholden to the AVM developers or the resellers in any way; we draw no income from selling, developing, or using AVM products.

For almost two decades, we have continued to test AVMs against hundreds of thousands (sometimes millions) of transactions per quarter and use a variety of techniques to level the playing field between AVMs. We provide detailed and transparent statistical summaries and insights to our newsletter readers, and we publish charts that give insights into the depth and thoroughness of our analysis, whereas we have not observed this from other testing entities. Our research spanning eighteen years shows that even overall good-preforming models are less reliable in certain circumstances, so one of the less obvious risks that we would highlight is reliance on a “good” model that is poor in a specific geography, price level or property type. Models should be tested in each one of these subcategories in order to assess their reliability and risk profile. Identifying “reliable models” isn’t straightforward. Performance varies over time as market conditions change and models are tweaked. Performance also varies between locations, so a model that is extremely reliable overall may not be effective in a specific region. Furthermore, models that are effective overall may not be effective at all price levels, for example: low-priced entry-level homes or high-priced homes. Finally, very effective models will also produce estimates that they admit have lower confidence scores (and higher FSDs), and which should in all prudence be avoided, but without adequate testing and understanding may be inadvertently relied upon. Proper testing and controls can mitigate these problems.

Regarding cascades, the Brookings’ paper leans on cascades as an important part of the solution for less discriminatory AVMs. We agree with Brookings: a cascade is the most sophisticated way to use AVMs. It maximizes accuracy and minimizes forecast error and risk. By subscribing to multiple AVMs, you can rank-order them to choose the highest performing AVM for each situation, which we call using a Model Preference Table™. The best possible AVM selection approach is a cascade, which combines that MPT™ with business logic to define when an AVM’s response is acceptable and when it should be set aside for the next AVM or another form of valuation. The business logic can incorporate the Forecast Standard Deviation provided by the model and the institution’s own risk-tolerance to determine when a value estimate is acceptable.

Mark Sennott (industry insider) recently published a whitepaper describing current issues with cascades, namely that some AVM resellers will give favorable positions to AVMs based on favors, pricing or other factors that do NOT include performance as evaluated by independent firms like AVMetrics. This goes to the additional transparency for which Brookings’ advocates. We’re all in favor.

We actually see a strong parallel between Mark Sennott’s whitepaper and the Brookings’ paper. Brookings makes the case to regulators, whereas Sennott was speaking to the AVM industry, but both of them argue for more transparency and responsible leadership by the industry. Sennott appears to be very prescient, in retrospect.

In order to ensure that adequate testing is done regularly we recommend that a control be implemented to create transparency around how the GSE’s or other originators are performing their testing. This could be done in a variety of ways. One method might require the GSE or lending institution to indicate their last AVM testing date on each appraisal waiver. Regardless of how it’s done, the goal would be to create a mechanism that would increase commitment to appropriate testing. The GSE’s could provide a leadership role by demonstrating how they would like lending institutions to demonstrate their independent AVM testing as required by OCC 2010-42 and 2011-12.

In conclusion, we appreciate Brookings’ dedication to asking questions and providing perspective on the AVM industry. We share their goals for transparency, fairness, and accuracy. We believe that open dialogue and collaboration by all the valuation industry participants are the keys to advancing the responsible use of AVMs.

We look forward to continuing our contributions to the AVM community and working toward a brighter future for this essential technology.

AVM Regulation – Twists and Turns to Get Here

The Era of Full Steam Ahead!

Six months before the pandemic, we published an article on the outlook for regulation related to AVMs. At the time, we identified three trends.

The administration was encouraging more use of AVMs (e.g., via hybrids), and tempering that with calls for close monitoring of AVMs.
The de minimis threshold change foreshadowed an increase in reliance on AVMs in some lower value mortgages.
The Appraisal Subcommittee summit was focused on standardization across agencies and alternative valuation products, namely, AVMs. Conversation focused on quality and risk as well as speed.

We saw those trends pointing to increased AVM use balanced by a focus on risk, quality and efficiency.

Sure enough, the following events unfolded:

The de minimis threshold was indeed raised, right before the pandemic changed everything.
The appraisal business was turned upside down for a period during the pandemic.
Property Inspection Waivers (PIWs) took off in a big way as Fannie and Freddie skipped appraisals on a huge percentages of their originations (up to 40% at times).

Halt! About Face!

And then the new administration changed the focus entirely. No longer were the conversations about speed, efficiency, quality, risk and appraisers being focused on their highest and best use. Instead, conversations focused on bias.

Fannie produced a report on bias in appraisals. CFPB began moving on new AVM guidelines and proposed using the “fifth factor” to measure Fair Lending implications for AVMs. Congress held committee hearings on AVM bias.

New Direction

Then The Appraisal Foundation’s Industry Advisory Council produced an AVM Task Force Report. Two of AVMetrics’ staff participated on the task force and helped present its findings recently in Washington D.C.

The Task Force made specific recommendations, but first it helped educate regulators about the AVM industry.

One specific recommendation was to consider certification for AVMs. Another was to use the same USPAP framework for the oversight of AVMs as is used for the oversight of appraisals. It’s all laid out in the AVM Task Force Report.

Taking It All In

Our assessment three years ago was eerily accurate for the subsequent two years. Even the unexpected pandemic generally moved things in the direction that we were pointing to: increased use of AVMs through hybrids.

What we failed to anticipate back then was a complete change in direction with the new administration, and maybe that’s to be expected. It’s hard to see around the corner to a new administration, with new personnel, priorities and policy objectives.

The Task Force Report provides some very practical direction for regulations. But the recent emphasis on fair lending, which emerged after the Task Force began meeting and forming its recommendations, could influence the direction of things. The end result is a combination of more clarity and, at the same time, new uncertainty.

AVMetrics Responds to FHFA on New Appraisal Practices

FHFA, the oversight agency for Fannie Mae and Freddie Mac, published a Request for Input on December 28, 2020. The RFI covered Appraisal-Related Policies, Practices and Processes. AVMetrics put forth a response including several pages and several exhibits making the case for using AVMs responsibly and effectively in a Model Preference Table®. Here is the Executive Summary:

The lynchpin to many of the appraisal alternatives is an Automated Valuation Model, a subject which AVMetrics has studied assiduously and relentlessly for more than 15 years. We point out that even an excellent AVM can be improved by the use of a Model Preference Table. MPTs enable better accuracy, fewer “no hits” and fewer overvaluations.

We also suggest an escalated focus on AVM testing, and we use our own research and citations of OCC Interagency Guidelines to emphasize the importance of testing to effectively use AVMs. We suggest that an “FSD Analysis” like the one we describe reduces risk by avoiding higher risk circumstances for using an AVM.

We suggest that the implementation of a universal MPT by the Enterprises will improve the collateral tools available and reduce the risk of manipulation by lenders. We also believe that a universal MPT can help redeploy appraisers to their highest and best use: the qualitative aspects of appraisal work. Our suggestion is that the GSEs endeavor to make the increased use of AVMs a benefit to appraisers, increasing their value-added and bringing them along in the transition.

AVMetrics’ full response is available here:

AVMetrics response to FHFA_RFI Final Download

An Interview with Lee Kennedy: Trends, the Future, and Regulation

The AVMNews sat down with our publisher Lee Kennedy to discuss trends in the industry.

AVMNews: Lee, as the Managing Director at AVMetrics, you’re sitting at the center of the Automated Valuation Model (AVM) industry. What changes have you seen recently?

Lee: There’s a lot going on. We see firsthand how the evolution of the technology has affected the sector dramatically. The availability of data and the decline in costs of storage and computing power have opened the doors to new competition. We see new entrants using new techniques and built by fresh faces. We still have a number of large players offering well-established AVMs. But, we also see the larger players retiring some of their older models. The established AVM players have responded in some cases by raising their game, and in other cases, by buying their upstart rivals. So, we’ve seen increased competition and increased consolidation at the same time.

And, it’s true that the tools keep getting better. It’s not evenly distributed, but on average they continue to do a better and better job.

AVMNews: In what ways do AVMs continue to get better?

Lee: AVMetrics has been conducting contemporaneous AVM testing for over a decade now, and we have many quantitative metrics showing how much better AVMs are getting. Specifically, we run statistical analysis around the comparison of AVM estimates to sales prices that are unknown to the models. We have seen increases in model accuracy rates measured by percentage of predicted error (PPE), mean absolute error (MAE) and a host of other metrics. Models are getting better at predicting sale prices and when they miss, they don’t miss by as much as they used to.

AVMNews: What about on the regulatory side?

Lee: There is always a lot going on. The regulatory environment has eased in the last two years reflecting a whole new attitude in Washington, D.C. – one that is more open to input and more interested in streamlining. Take, for instance, the 2018 Treasury report that focuses on advancing technologies (See “A Financial System That Creates Economic Opportunities”).

Last November, I was at a key stakeholder forum for the Appraisal Subcommittee (ASC). One area of focus was harmonizing appraisal requirements across agencies. Another major focus was how to effectively employ new tools in support of the appraisal industry, including the growth of Alternative Valuation Products that utilize AVMs.

AVMNews: I know that you also wrote a letter to the Federal Finance Institutions Examination Council (FFIEC) about raising the de minimis threshold, below which some lending guidelines would NOT require an appraisal. This year in July they elected to change the de minimus threshold from $250,000 to $400,000 for residential housing. What are your thoughts?

Lee: Well, I think that the question everyone is struggling with is “What does the future hold for appraisers and AVMs?” Obviously, the field of appraisers is shrinking, and AVMs are economical, faster and improving. How is this going to play out?

First, my strong feeling is that appraisers are a valuable and limited resource, and we need to employ them at their highest and best use. Trying to be a “manual AVM” is not their highest and best use. Their expertise should be focused on the qualitative aspects of the valuation process such as condition, market and locational influences, not the quantitative (facts) such as bed and bath counts. Models do not capture and analyze the qualitative aspects of a property very well.

Several companies are developing ways of merging the robust data processing capabilities of an AVM with the qualitative assessment skills of appraisers. Today, these products typically use an AVM at their core and then satisfy additional FFIEC evaluation criteria (physical property condition, market and location influences) with an additional service. For example, the lender can wrap a Property Condition Report (PCR) around the AVM and reconcile that data in support of a Home Equity Line of Credit (HELOC) lending decision. This type of hybrid product offering is on the track that we’re headed down. Many AMCs and software developers have already created these types of products for proprietary use or for use on multiple platforms.

AVMNews: AVMs were supposed to take over the world. Can you tell us what happened?

Lee: Well, the Financial Crisis is one thing that happened. Lawsuits ensued, and everyone got a lot more conservative. And, the success of AVMs developed into hype that was obviously unrealistic. But, AVMs are starting to gain traction again. We are answering a lot more calls from lenders who want help implementing AVMs in their origination processes. They typically need our help with policies and procedures to stay on the right side of the Office of the Comptroller of the Currency (OCC) regulations, and so in the last year, we’ve done training at several banks.

Everyone is quick to point out that AVMs are not infallible, but AVMs are pretty incredible tools when you consider their speed, accuracy, cost and scalability. And, they are getting more impressive. Behind the curtain the models are using neural networks and machine learning algorithms. Some use creative techniques to adjust prices conditionally in response to situational or temporary conditions. We test them and talk to their developers, and we can see how that creativity translates into improved performance.

AVMNews: You consult to litigants about the use of AVMs in lawsuits. How do you think legal decisions and risk will affect the use of AVMs?

Lee: This is an area of our business, litigation support, where I am restricted from saying very much. It has been and continues to be an enlightening experience as some of the best minds are involved in all aspects of collateral valuation and the “Experts” are truly that… experts in their fields as econometricians, statisticians, appraisers, modelers, etc.… It is also very interesting with over 50 cases behind us now, to get a look behind the legal system curtain and how all of that works. Therefore, I want to emphasize that my comments for our interview are in the context of contemporaneous AVMs that were tested during the time period shown here and not a retrospective AVM that was looking back to these time periods.

AVMNews: AVMetrics now publishes the AVM News – how did that come about?

Lee: As you and the many subscribers know, Perry Minus of Wells Fargo started that publication as a labor of love over a decade ago. When he retired recently, he asked if I would take over as the publisher. We were honored to be trusted with his creation, and we see it as a way to be good citizens and contribute to the industry as a whole.

AVMNews: I encourage anyone interested in receiving the quarterly newsletter for free to go to http://eepurl.com/cni8Db

The AVMNews is a quarterly newsletter that is a compilation of interesting and noteworthy articles, news items and press releases that are relevant to the AVM industry. Published by AVMetrics, the AVMNews endeavors to educate the industry and share knowledge about Automated Valuation Models for the betterment of everyone involved.

AVM Regulatory Outlook

As always, changes are coming to the valuation industry. These changes have been germinating in government and industry for a long time, but they’ve made progress in the last year, and I believe that they’re likely to emerge sometime this year. I expect that we may see more regulatory changes liberalizing the use of AVMs soon.

I think that you’ll come to that same conclusion, too, if I share a couple milestones that I’ve observed and put them together with some insights I’ve gathered from talking to industry leaders.

The first milestone I will highlight was the July 2018 Financial System report by Secretary Mnuchin, which is consistent with the administration’s new attitude towards regulation. The report is far-reaching, and it includes thoughtful commentary about the uses of AVMs (see, for example, page 103-106). It recommends updating FIRREA appraisal requirements to accommodate increased usage of AVMs and hybrids. It also advocates for increased monitoring of AVMs and the application of rigorous market standards. And, it recommends focusing the use of AVMs and hybrids on loan programs with other mitigating risk factors.

The next milestone I will highlight was the proposed change in the de minimis threshold that was put out for comment in November of last year. The change would raise the threshold below which a residential mortgage could be originated with an evaluation, utilizing an AVM in lieu of a traditional appraisal. It would be raised from $250,000 to $400,000.

To those milestones I would add a third data point. Last November I attended the Appraisal Subcommittee roundtable entitled: “The Evolving Real Estate Valuation Landscape.” As part of the of the Federal Financial Institutions Examination Council, the roundtable brought together industry representatives and government officials (see the table below) to discuss real estate valuation.

The day was split into two sessions; the morning and afternoon sessions each began with a panel of industry experts who addressed a series of prepared questions. In addition, there was a roundtable discussion focused on quotes from the July 2018 Financial System report referenced above.

The topic for the morning discussion was “Harmonizing Real Estate Valuation Requirements Across the Federal Government.” This session focused on identifying various federal appraisal statutory and regulatory requirements and exploring opportunities to harmonize those requirements, e.g., VA, FHA, and FHFA all having differing valuation requirements and standards.

The afternoon panel discussion topic was; “The Evolution of Real Estate Valuation” which focused on evolving valuation needs in commercial and mortgage lending. A key area of this session was focused on Alternative Valuation Products inclusive of AVM’s and their increasing used by lenders and the secondary market.

The roundtable discussion started with quotes about AVMs and hybrid valuation products and focused on standards. The group also contemplated how alternative valuation techniques can impact quality and mitigate risk. Finally, one quote that focused on speeding the adoption of technology was discussed.

As I write this six months later, I see the pieces of the puzzle coming together. Obviously, there is momentum behind the increased usage of AVMs, for their independence, increasing accuracy, speed and efficiency. But there is also an implicit concern to avoid opening the door to more risk. I see this being expressed by talk about “standards,” alternative products, such as “hybrids” and increased monitoring.

As I have written elsewhere, I welcome changes that make better use of our valuable and limited resources, namely the appraisers themselves. As AVM quality improves and the number of appraisers shrinks, we should encourage appraisers to be focused on their highest and best use. Their expertise should be focused on the complex, qualitative aspects of property valuation such as the property condition and market and locational influences. They should also be focused on performing complex valuation assignments in non-homogeneous markets. Trying to be a “manual AVM” is not the highest and best use of a highly qualified appraiser, and I expect that Treasury, the FDIC and legislators are moving in this same direction.

Lee Kennedy

Participants in “The Evolving Real Estate Valuation Landscape” Appraisal Subcommittee, Federal Financial Institutions Examination Council, 2018
Government	Trade Organizations	Industry Participants
The Appraisal Foundation (TAF)	American Bankers Association	AVMetrics, LLC
Association of Appraiser Regulatory Officials (AARO)	American Society of Appraisers	Bank of America
Consumer Financial Protection Bureau (4)	American Society of Farm Managers and Rural Appraisers	Clarocity Valuation Services
Federal Deposit Insurance Corporation(3)	Appraisal Institute	ClearBox
Federal Housing Finance Agency(4)	Homeownership Preservation Foundation	CoreLogic
Federal Reserve Board(5)	Independent Community Bankers of America	Cushman & Wakefield Global Services, Inc.
Freddie Mac	Mortgage Bankers Association	Farm Credit Mid-America
Internal Revenue Service	National Association of Home Builders	First American Mortgage Solutions
National Credit Union Administration	National Association of Realtors	Genworth Financial
Office of the Comptroller of the Currency (4)	Real Estate Valuation Advocacy Association (REVAA)	JPMorgan Chase & Company
Tennessee Real Estate Appraisers Commission	State appraiser coalitions representative	Old Line Bank
Texas Appraiser Licensing and Certification Board		Quicken Loans
U.S. Department of Agriculture		ServiceLink
U.S. Department of Justice (2)
U.S. Department of the Interior (2)
U.S. Department of Veterans Affairs
US. Department of Housing and Urban Development

Tag: Regulation

Why the legacy method is being retired

Aligned with regulatory expectations

What this means for the industry

What hasn’t changed