Tag: Independent Testing

It seems like only yesterday we were lamenting the hubris of Volkswagen, loading software into their TDI models to fake out emissions tests on tens of millions of vehicles.  Here we are again, this time with Mitsubishi.  The only real surprise is that these companies don’t learn.

Hyundai in 2012, Ford in 2014, Volkswagen in 2015, and now Mitsubishi, although this is not even their first scandal.  In the early 2000s, Mitsubishi was embarrassed by defects that were covered up.

It’s surprising that these companies cannot identify the root cause is faulty business processes.  Instead, they root out the responsible parties and do a mea culpa, or the CEOs resign in shame for their leadership failures (as in the case of Volkswagen last year).  Why doesn’t anyone realize that if your system is to self-test for emissions and mileage, eventually you are going to have a problem, because that is not a foolproof system?

The faulty business process is their lack of independent testing.  These emissions and mileage results are vital business inputs, and the integrity of those results is mission critical.  Where are their controls?

Our industry is financial services, where federal regulations have long required independent testing in many areas.  Our specific segment of the industry is the Automated Valuation Model (AVM) business, which has a regulatory mandate for independent validation.  Financial institutions use many different kinds of computer models to improve decision making, and AVMs are one kind of model.  They estimate property values, and for banks that makes loans on property, that comes in handy in dozens of ways.

But, if there are systematic problems with AVMs, for example, if they over-valued everything by 20%, it could cause a huge problem for banks and credit unions.  This is where we come in.  We independently test and validate every commercially available residential AVM on a continuous basis, thoroughly, rigorously and impartially.  And, the beneficiaries are everyone.  Banks and credit unions benefit, borrowers benefit, and even the AVM developers benefit because of the feedback we provide to them as well as the broader consumer confidence in their products.

Certainly it is incumbent upon leaders to create a culture of integrity.  One way of doing that is to do more than admonish people to be honest.  Instead, create a system where there is independent testing, and make sure that everyone knows that their results will be tested.  Voila!  When people know they are being checked, integrity soars, and everyone wins.  Don’t just demand integrity; build it into the process!

Lee Kennedy, principal and founder of AVMetrics in 2005, has specialized in collateral valuation, AVM testing and related regulation for over three decades.  Over the years, AVMetrics has guided companies through regulatory challenges, helped them meet their AVM validation requirements, and commented on pending regulations to help bring clarity and sanity to the situation.  Lee is an author, speaker and expert witness on the testing and use of AVMs.  Lee’s conviction is that independent, rigorous validation is the healthiest way to ensure that models serve their business purposes.  Every commercially available AVM vendor trusts AVMetrics to provide feedback to them on their models, facilitating each model’s continuous improvement.