Financial regulators (e.g, the OCC, FED, NCUA, etc) require that institutions independently test AVMs, as outlined in OCC 2010-42 and 2011-12. Our detailed report describes how our process and results satisfy these requirements in great detail and with extensive documentation.
In about 50 pages, the report describes our methodology in detail, including our test data parameters, sample size, coverage and data sources. Currently our process uses multiple data sources and identifies about 500,000 candidate transactions per quarter in the U.S. It presents our analyses and the many measures of accuracy and precision that we use to evaluate AVMs. Also, it describes our scoring algorithms and how we rank model performance.
As part of the report, we correlate the guidance directly to our testing process to show it satisfies the Agencies’ expectations for selecting, using, validating and monitoring a valuation model or tool, including the following:
- The necessary expertise and the fact that institutions may need to engage a third party
- The importance of performing due diligence on model design and development
- The establishment of minimum performance criteria
- Requirements for assessment of strengths and weaknesses design and performance
- Evaluation of the model developer’s scoring system and its appropriateness as an indicator of reliability by property type and location
- And, perhaps most importantly, model testing, including sample sizes, geographic analyses, testing frequency and standards of performance measures and results