Large Bank Supervision Training, Indianapolis, IN, November 18-20, 2019.

Large Bank Supervision Training

Indianapolis, IN, November 18-20, 2019

Large Bank Supervision Training Indianapolis, IN November 18 - 20, 2019

Attendees Arkansas State Bank Department Jerry Corrothers

501-324-9019 501-324-9019 501-683-1115 501-324-9019

Kent Darr

R.D. Jackson

Jordan Patterson

Georgia Department of Banking & Finance Daisy Mitchell


Indiana Department of Financial Institutions Danny Elick

317-232-3955 317-499-4623 317-232-3955 317-232-3955 765-449-9832 317-232-3855 317-453-2528

Kristy Hubele Mike Nickell Gage Russell

Deron Thompson Robin Upchurch Steve Wachter

Louisiana Office of Financial Institutions Chris Floyd Maryland Office of Financial Regulation Kevin Fitzsimmons


410-230-6106 410-230-6022 410-230-6022 410-230-6106 601-321-6902 601-359-1031 662-417-4675 601-321-6901 601-321-6901 601-321-6907 601-321-6901

Irvin Libao

Emmanuel Mgboji Darren Stevens

Mississippi Department of Banking & Consumer Finance Charlotte Corley

Jeff Cox

Mark Hudson

Paul Lion

Nicky Shelton Roger Sinclair

Erik Smith

Missouri Division of Finance Blake Gulliford


Montana Division of Banking and Financial Institutions Joe Kuntz

406-248-2742 406-841-2918

John Ross

North Carolina Office of Commissioner of Banks Michelle Gresham


Pennsylvania Department of Banking and Securities Chris David

717-903-8508 717-783-4242

Alex Rauschenberger

South Dakota Division of Banking Drew Haack

605-773-3421 605-367-4360 605-367-4527

Brady Schlechter

Mike Zellmer

Tennessee Department of Financial Institutions Grant Casselberry


Texas Department of Banking Jay Voigt

512-475-1300 512-475-1300 512-475-1300

Jacque Willardson

Lea Wilson

Utah Department of Financial Institutions Dean Merrill

801-538-8830 801-538-8836

Darryle Rude

Instructors BancorpSouth Bank Patrick House

Ty Lambert

Consumer Financial Protection Bureau John Schroeder

Darling Consulting Group Mike Guglielmo

Federal Deposit Insurance Corporation Christine Rivera

Mark Sheely Jim Watkins

Federal Reserve Bank of Cleveland Brian Dragoo

Stephen Jenkins

Federal Reserve Board Kathryn Ellis

Financial Institutions Growth, LLC Randy Cameron

Dave Wood

Conference of State Bank Supervisors Staff Daniel Berkland

202-559-1987 202-808-3557 304-549-9584 202-728-5742

Jim Cooper Tom McVey

Daniel Schwartz

2019 CSBS Large Bank Supervision Forum for States Indianapolis, IN

Monday November 18 1:00 PM – 1:30 PM

Welcome & Introductions Jim Cooper Senior Vice President – Regulatory Policy

Dan Schwartz Director, Policy Development

Tom McVey Director, Learning Services Conference of State Bank Supervisors

Large Bank State Examiner Panel Erik Smith - Moderator Supervisory Review Examiner Mississippi Department of Banking & Consumer Finance

1:30 PM – 2:45 PM

Drew Haack South Dakota Division of Banking

Mark Hudson Mississippi Department of Banking & Consumer Finance

Daisy Mitchell Georgia Department of Banking & Finance

Jacque Willardson Texas Department of Banking


2:45 PM – 3:00 PM 3:00 PM – 4:30 PM

Breakout Sessions and Group Discussions

Open Forum

4:30 PM – 5:00 PM

Networking Reception

5:30 PM – 7:00 PM

2019 CSBS Large Bank Supervision Forum for States Indianapolis, IN

Tuesday November 19 7:30 AM – 8:30 AM


Federal Regulator – Federal Deposit Insurance Corporation Christine Rivera Section Chief / Large Bank Supervision

8:30 AM – 10:00 AM

Mark Sheely Shared National Credit (SNC) Coordinator Jim Watkins Senior Deputy Director Federal Deposit Insurance Corporation

10:00 AM – 10:15 AM Break 10:15 AM – 11:30 AM Large Bank Internal Perspectives Ty Lambert EVP & Chief Data Analytics Officer

Patrick House SVP & Director of Profitability BancorpSouth Bank

11:30 AM – 12:45 PM Lunch 12:45 AM – 1:45 PM

Federal Regulator – Consumer Financial Protection Bureau John Schroeder Regional Director Supervision, Enforcement and Fair Lending Consumer Financial Protection Bureau

1:45 PM – 2:00 PM


Federal Regulator – Federal Reserve Bank Stephen Jenkins Vice President - Large Institution Supervision Group

2:00 PM – 3:15 PM

Brian Dragoo Assistant Vice President and LFBO Management Group Coordinator Federal Reserve Bank of Cleveland

Kathryn Ellis Lead Financial Institution and Policy Analyst Federal Reserve Board

2019 CSBS Large Bank Supervision Forum for States Indianapolis, IN Model Risk Management – What does it take for an effective challenge? Michael Guglielmo Managing Director Darling Consulting Group

3:15 PM – 4:45 PM


5:00 PM

Wednesday November 20 7:30 AM – 8:30 AM


Leverage Lending – What are the risks? Randy Cameron Managing Principal Dave Wood Managing Principal Financial Institutions Growth, LLC

8:30 AM – 10:00 AM

10:00 AM – 10:15 AM Break 10:15 AM – 11:45 AM The LIBOR Transition & More Daniel Berkland Director, Supervisory Processes

Conference of State Bank Supervisors

11:45 AM – 12:00 PM Open Forum / Adjourn

Shared National Credit Program

Chris Rivera, FDIC Mark Sheely, FDIC

Restricted - Confidential Supervisory Information


Table of Contents

 Overview of Shared National Credits (SNC) Program

 Leveraged Lending (LL)

 Credit Structure Trends

Restricted - Confidential Supervisory Information



 Dates to 1977  Initial threshold of $20MM increased to $100MM in 2018  Provides the examiners and bankers with uniform risk rating for exposures that shared by three or more regulated companies  Provides uniform assessment of credit risk management practices  Has been very useful in defining systemic risks and responses  Leverage Lending  Oil & Gas

Restricted - Confidential Supervisory Information


Current Trend in SNC Portfolio

Portfolio Size and Growth • Total commitments have grown 8.2% since 3Q18 to $4.8T • Growth centered in Investment Grade Leverage Borrowers Aggregate SNC population credit risk is moderate; however, LL risk remains high • Criticized rate has hovered between 6.5 and 7 percent for past couple of years • Vast majority of problem credits in Non-Investment Grade Leverage Non-Bank Lenders continue to hold majority of adversely rated credit • CLO’s and Mutual Funds largest buyer of riskier leveraged credit • Banks managing risk thru Originate to Distribute model

Restricted - Confidential Supervisory Information


 Global Economy  Speculative M&A activity funded predominately with syndicated debt continues to grow  Leveraged credit rated B- and worse has doubled since last recession  Financial Institutions  Prevalence of Cov Lite and First Lien only loan structures expected to reduce/delay defaults and increase loss rates, respectively  Liquidity will be stressed during next recession  $1 trillion in unfunded leveraged credit  Backfill funding source for Non-Bank lenders – CLO’s, Mutual Funds, etc.  Agencies  Impact on staffing: Current use of 300 examiners twice per year would likely double Leverage Lending – Key Risks

Restricted - Confidential Supervisory Information


LL Characteristics

Bank-Owned (Total $1,494B)

Other Investors-Owned ( Total $831B)

Category Ownership • CLO (51%) • Loan/Debt Fund (34%) • Endowment/Retirement (3%) • Investment Manager (3%) • Insurance Company (2%) • All Other (7%)

Bank-Owned SM+Class Top Industries

Other Investors-Owned SM+Class Top Industries

Restricted - Confidential Supervisory Information and not for public use


New LL Transaction Structure Assessments

• Examiners continue to note a prevalence of marginal or weak transaction structures

Restricted - Confidential Supervisory Information and not for public use


LCQI Scores Show Modest Improvement

• Strengthening covenants reflect investor push back on B rated credits • While Loan Covenant Quality Indicator (LCQI) scores have improved, credit documentation remains loose • Despite the improvement, the average LCQI scores remain squarely in “weak” territory

Note: Institutional Term Debt

Restricted - Confidential Supervisory Information


Covenant protections weaker today than before last recession

- lite issuances reached a new record in

• According to Moody’s, institutional term cov

2018 and now account ~ 80% of the volume • As outlined in the Moody’s study, transaction structure weaknesses include more than just financial covenants

Restricted - Confidential Supervisory Information


LBO Leverage Levels Continue to Increase

Source: LPC Refinitiv  Leverage levels on private equity driven LBOs are above pre-crisis peaks with 77% and 41% of 1Q19 deals over 6x and 7x leverage, respectively  Incremental debt capacity in credit agreement provisions has also risen giving potential for further borrower leveraging and senior debt collateral dilution after origination

Restricted - Confidential Supervisory Information and not for public use


Structural Protection Also Decreasing

Source: Moody’s investor Services

• Debt Loss absorption capacity continues to deteriorate after the credit crisis. First-Lien debt cushion has fallen to a low of 17%. • This will lead to lower estimated recoveries on senior debt positions held by both non-banks and regulated institutions

Restricted - Confidential Supervisory Information and not for public use


Repayment Capacity Stress Test

1) Current Portfolio 2) Annual Revenue Growth Capped at 3%








- YoY revenue growth capped at 3% beginning in 2019





3) 700bps Gradual Rate Increase 4) Severe Shock to Free Cash Flows Year 1 (-100%) Year 5 (-20%) Year 2 (-100%) Year 6 (-10%) Year 3 (-75%) Year 7 (-5%) Year 4 (-50%) Severe shock based on worst periods in 64 year history of FCF for almost all publicly traded companies from CompuStat











Baseline Portfolio 3% Revenue Growth 700 Basis Point Shock to Interest Paid

Severe Shock to FCFs

< 35% 35% to 50% > 50 to 80% > 80%

Data from 206 3Q19 SNC Review LL obligors

Restricted - Confidential Supervisory Information and not for public use


Moody’s Rating Comparison: 2008 vs Current

• Single B loan issuance in the LL market since the last recession has increased reflecting higher risk levels • Within the Single B credit band, there has been a significant rise in the percentage rated B2/B3 indicating a high volume of credits at the edge of Non-Pass • When the 2008 recession hit, 22% of B2 credits and 53% of B3 credits were downgraded within 12 months to Caa or worse

Restricted - Confidential Supervisory Information and not for public use



Restricted - Confidential Supervisory Information


Restricted FR

Katheryn Ellis, Lead Financial and Institution Policy Analyst Federal Reserve Board Division of Supervision and Regulation Federal Reserve Supervision of $10- $100 Billion Banking Organizations

November 19, 2019

Restricted FR


 $10b - $100b firms are supervised as Regional Banking Organizations (RBOs)  The portfolio is geographically diverse with the largest volume of state member banks as compared to other Federal Reserve portfolios  We continue to refine the supervisory program to vary based on size, complexity and type of institution being supervised (i.e. SMBs, Complex BHC and Shell BHCs)  Financial performance mirrors overall sound economic conditions  Recently identified risk management issues are centered in three areas including risk management, IT and operational risk and credit risk  There are several supervisory policy and program initiatives in place to enhance the RBO supervisory program


Restricted FR

 The number of RBOs has increased 130 percent since 2010  Aggregate RBO assets has increased 110 percent since 2010 RBO Rapid Growth due to Consolidation The number and aggregate assets of RBOs continue to increase


Restricted FR

All Reserve Banks Supervise RBOs

RBO Charter Type by District and Asset Size

 RBO portfolio contains more SMB assets than any other portfolio  The number of RBOs supervised by the Federal Reserve, FDIC and OCC is nearly equally split

Reserve Bank



0 4 0 1 9 0 1 1 1 1 3 2 2 2 1 2 8 0 3 3 2 5 3 0 1 4 0 2 4 1 3 5 0 5 10 1 26 56 8 17 37 6 7 17 1


New York


Philadelphia Cleveland Richmond

3 6 5 8 8 5 7 8

Atlanta Chicago St. Louis


Minneapolis Kansas City


San Francisco




$10‐25 billion $25‐50 billion $50‐100 billion

60 25


2 1



Restricted FR

RBO Business Model: Traditional and Specialty Organizations

Traditional RBO Institutions  Four RBO SMBs have commercial real estate loan concentrations  RBO loan quality metrics are favorable, but examiners note increasing risk primarily from changes in underwriting practices Specialty RBO Organizations  Commercial Savings and Loan Holding Companies  Fifty percent or more of total consolidated assets or total revenues derived from commercial activities  Broker-dealer SLHCs and BHCs  Significant broker-dealer activities including retail brokerage, securities underwriting and asset management


Restricted FR

Financial Performance Ratios Median RBO financial performance has generally improved since the crisis


Restricted FR

Frequently Cited Risk Management Issues Exams and inspections conducted in 2018 and 2019

 Most cited issues relate to risk management, credit risk, IT and operational risk


Restricted FR

Calibration of RBO Supervision RBO HC Noncomplex RBO HC Complex


Specialty HC


Full reliance on IDIR and State Colleagues

Partial reliance on IDIR and State Colleagues

FRB is IDIR.. Collaborate with State Colleagues One Roll-up + several targets RFI Workprogram (core & expanded questions); Internal risk ID tools

Partial reliance on IDIR & other regulators

Supervision Requirements

One Roll-up

One Roll-up + 1 target

One Roll-up + potential target(s) RFI Workprogram (core &

Exam Requirements & Tools

RFI Workprogram (core questions); Shell HC job aids and templates

RFI Workprogram (core &

expanded questions)

expanded questions)


Restricted FR

RBO Supervisory Approach for SMBs and Complex HCs

RBO Supervision Places Emphasis on:  Understanding the entire institution  Assessing unique risk profile  Developing a supervisory plan based on identified and anticipated risk  Performing periodic monitoring  Providing clear and concise feedback  Coordinating with state colleagues and other federal regulators

Annual Institutional Overview

Supervisory Communication (Roll-ups, Targets, etc.)

Annual Risk Assessment (RA)

Semi-Annual Continuous Monitoring

Annual Supervisory Plan


Restricted FR

Supervisory Approach Differs From Community and Large Bank Supervision

Supervisory Program Elements

RBO ($10B - $100B)

CBO (< $10B)

LFBO (> $100B)

Targeted Reviews Drive Annual Exam Report (complex) or Annual Exam Report only (noncomplex)

Horizontals and Targeted Reviews Drive Annual Exam Report

Full-Scope Exam (12 – 18 months)

Examination Process

Accelerated Exams Based on Offsite Monitoring

Monitoring Activities



Based on Exam Cycle

Formal Supervisory Plan

Annual Plan

Annual Plan

General Examination Approach

CPC and Risk Specialists

Exam/Inspection Resources

Rotating EICs

Dedicated team


Restricted FR

Supervisory Policy and Program Initiatives

Regional Banking Policy Initiatives  Tailoring for $50-100B  Capital planning clarification  Model risk management Key Supervisory Program Initiatives  Bank Exams Tailored to Risk (BETR)  MIS Hours  BSA/AML Program Review  Outreach with Industry and Regulators  Credit underwriting review


Federal Reserve System Large and Foreign Banking Organizations (LFBO)

Portfolio Overview & Priorities CSBS Large Bank Supervision Forum November 19, 2019

Steve Jenkins, Federal Reserve Bank of Cleveland Brian Dragoo, Federal Reserve Bank of Cleveland


Presentation Goals

Provide an overview of the LFBO portfolio composition.

Discuss conclusions from significant 2019 horizontal assessments.

Share priorities from the 2020 supervisory plans.

Provide an overview of the Large Financial Institution (LFI) Rating System and recently finalized rules tailoring regulations for LFBOs. LFBO Portfolio Composition The LFBO portfolio consists of 17 domestic bank holding companies (BHCs), 8 foreign banking organizations (FBOs) with a U.S. intermediate holding company (IHC), and 11 FBOs without a U.S. IHC.

The portfolio includes 7 state-chartered member banks.

Firm asset size ranges from $51 billion to $482 billion, and the portfolio collectively represents approximately a quarter of total U.S. BHC assets.


Supervisory Approach

The LFBO supervisory approach incorporates teams of examiners and risk specialists at Reserve Banks, and staff from Board S&R and DCCA to conduct a range of supervisory activities.

risks, and knowledge gaps. Plans focus on firm-specific reviews to account for the diversity of firms and risks in the portfolio, continuous monitoring to stay informed of changes and emerging risks, and horizontal reviews to gain a portfolio perspective of higher risk areas. An annual ratings assessment is conducted that aggregates results and provides an overall

When safety and soundness concerns are identified or where firms do not meet supervisory standards, issues are communicated and firms are directed to take corrective action.


Significant 2019 Horizontal Assessments

Horizontal Capital Review The scope included assessing loss estimation methodologies for retail (1-4 family mortgage and HELOC) and wholesale (CRE), governance processes, and MRA remediation. Capital planning practices continue to improve. Review findings reflect generally mature risk identification, loss forecasting, and governance processes. There were a limited number of MRAs cited, primarily from the wholesale work stream. Horizontal Liquidity Review The scope focused on evaluating stress testing, contingency funding plans (CFPs), Regulation WW liquidity buffers, and FR 2052a data and reporting controls. Risk management practices continue to improve. Stress testing practices are generally effective. Firms have robust FR 2052a data and reporting controls; governance and independent review of the CFP is adequate for almost all firms; and most firms fully confirm to Reg. WW requirements. Firm practices were predominately assessed as being generally consistent with supervisory standards. Exceptions were mainly related to stress testing and liquidity buffers.


Significant 2019 Horizontal Assessments

Horizontal Cybersecurity Review The review will assess the resiliency of higher-risk operations at five LFBOs with critical operations. For the remaining firms, the review will focus on endpoint security (security of networks when accessed through remote devices) and include transaction testing. The 2019 cybersecurity supervisory work remains in progress. The horizontal vetting has been scheduled for February 2020. Preliminary results indicate that the increased focus on control testing has resulted in more MRAs and less favorable overall assessments. Shared National Credits Non-pass commitments for the oil & gas industry have continued to contract since 3Q16, driving the reduction in overall non-pass commitments. Bank regulatory risk rating accuracy is acceptable at most institutions. The accuracy of risk ratings have performed reasonably well during the past few reviews. Risk from leveraged lending remains high. Leveraged loans make up a significant portion of the SNC portfolio. These loans present layered risk when weak transaction structures are coupled with high leverage and/or aggressive repayment assumptions.


2019 Annual Ratings Assessment

The LFBO business line recently held vetting discussions for the 2019 annual ratings roll-up using the new LFI ratings framework. Two themes are evident from the vetting discussions: Firm are managing through significant changes, including those related to strategy, leadership, products and services, and delivery channels. A number of teams are considering supervisory messages regarding the importance of maintaining sound operations and risk management through significant business change. To varying degrees, many firms have weaknesses in operational risk areas. These areas include third party risk management, model risk management, cybersecurity, information technology, fraud controls, and risk & control self-assessments.

Governance and Controls Ratings and plan to send supervisory messages accordingly.


2020 Supervisory Priorities

7 Horizontal Capital Review HCR will focus on assessing retail loss forecasting (auto/card), wholesale loss forecasting (C&I), pre- provision net revenue (PPNR), and non-projection methodology. Horizontal Liquidity Review HLR will focus on evaluating material internal stress testing assumptions, data aggregation for FR 2052a reports, and cash flow forecasting processes. Evaluate compliance with Regulation YY intraday liquidity monitoring and management requirements. Horizontal Cybersecurity Review The cyber review will focus on the adequacy of controls for critical internet facing applications, including boundary defense, configuration management, and capacity/resiliency. Examiners will evaluate controls to mitigate threats posed by difference scenarios, and conduct added testing in key areas (access management, data governance, patch management, etc.). Shared National Credits Review Assess asset quality and credit risk management through the SNC program. Coordinated Leveraged Lending Review The leveraged lending review will assess the quality of credit risk management, underwriting and independent cash flow projections, and evaluate the framework. The review will be risk-focused, conducted at a select group of firms with heighted exposure. Coordinated CECL Review Evaluate the material elements of processes to derive the Allowance for Credit Losses at SMBs. Leverage the PFR for CECL work at non-complex firms, and conduct additional loss estimation work at complex firms with material assets outside of IDI (that are subject to CECL). Coordinated LIBOR Transition Review 2019 LIBOR transition work involves assessing the portfolio state of readiness and completing a range of practice assessment. LIBOR transition supervisory work in 2020 will focus on high-priority firms. Coordinated Third Party Risk Management Review The review will assess the adequacy of third party due diligence and risk assessment processes, with a focus on third parties that access or maintain confidential/customer information. 2020 Supervisory Priorities (continued)


Large Financial Institution Rating System

On November 2, 2018, the Federal Reserve finalized a new rating system for large financial institutions (LFIs) .

Applicability: U.S. bank holding companies (BHCs) with total consolidated assets of $100 billion or more; Non-insurance, non-commercial savings and loan holding companies (SLHCs) with total consolidated assets of $100 billion or more; and All U.S. intermediate holding companies (IHCs) of foreign banking organizations (FBOs) (with total consolidated assets of $50 billion or more.

Ratings will be assigned for LFBO firms in early 2020. Initial LFI ratings for each of the three component ratings will be assigned at the same time.

After the initial 2020 ratings assignment, individual component ratings may be assigned at different times of a given year.


Large Financial Institution Rating System

The LFI rating system evaluates whether a firm possesses sufficient financial and operational strength and resilience to maintain safe and sound operations through a range of conditions. The three component ratings assessed are: Capital Planning & Positions (C) Effectiveness of governance and planning processes used to determine capital needs Sufficiency of capital positions Liquidity Risk Management & Positions (L) Effectiveness of governance and risk management processes used to determine liquidity needs Sufficiency of liquidity positions

Governance & Controls (G&C)

Management of core business lines and independent risk management and controls


Large Financial Institution Rating System The LFI ratings system uses a four-category rating scale. Broadly Meets Expectations Conditionally Meets Expectations

Deficient 1 Deficient 2 There is no stand-alone composite rating and there are no subcomponent ratings.

To be considered well managed for financial holding company status (FHC) and other purposes, all three component rating (C, L, and G&C) must be rated Broadly Meets Expectations or Conditionally Meets Expectations. Noncompliance with the well-managed FHC standard for any one component (or with the well- capitalized standard) will result in restrictions on certain activities.

A firm with a Deficient-1 or Deficient-


Finalized Tailoring Rules

On October 10, 2019, the Board approved two final rules that establish a revised framework for applying prudential standards to large U.S. banking organizations and foreign banking organizations based on their risk profile. The first final rule is a Board-only rule that tailors the application of prudential standards to large banking organizations, and applies prudential standards to certain savings and loan holding companies (SLHCs). The second final rule was issued jointly with the OCC and FDIC, tailoring the application of the The final rules establish a revised framework of four risk-based categories for determining the applicability and stringency of prudential standards. Firms will be sorted into categories based on the factors below. Asset Size Cross-Jurisdictional Activity Reliance on Short-Term Wholesale Funding Nonbank Assets Off-Balance Sheet Exposure


Finalized Tailoring Rules

The risk-based categories for determining the applicability and stringency of prudential standards are listed below. Category I: U.S. global systemically important bank holding companies will remain subject to the most stringent standards. Category II: Firms of global scale those with very significant size ($700 billion or more in assets) or cross-jurisdictional activity ($75 billion or more) will be subject to stringent prudential standards, including requirements based on standards developed by the Basel Committee on Banking Supervision. Category III: Firms with $250 billion or more in assets, or firms with $100 billion to $250 billion in assets that meet one or more risk-based thresholds, will be subject to prudential standards that are appropriate for firms with an elevated risk profile, relative to firms subject to Category IV standards. Category IV: Firms with $100 billion to $250 billion in assets that do not meet a risk-based threshold will be subject to reduced capital, liquidity, and risk-management requirements relative to firms subject to Category III standards, reflecting their more limited risk profile.


Large Bank Supervision Forum

Effective Challenge in Model Risk Management November 19, 2019

Michael Guglielmo, Managing Director

(978) 499‐8159

© 2019 Darling Consulting Group, Inc.  •  260 Merrimac Street  •  Newburyport, MA 01950  •  Tel: 978.463.0400  •

Effective Challenge

Page 2

© 2019 Darling Consulting Group, Inc.

Common Reference

“Management then provided  effective challenge.”

Page 3

© 2019 Darling Consulting Group, Inc.

Effective Challenge Defined Guidance on Model Risk Management (OCC 2011‐12/ Fed SR 11‐7 / FDIC FIL‐22‐2017 ) 

 Incentives to provide effective challenge to models are stronger when   Greater separation of that challenge from the model development process  Well‐designed compensation practices   Corporate culture   Competence is a key to effectiveness since technical knowledge and modeling skills are necessary to  conduct appropriate analysis and critique  Challenge may fail to be effective without the influence to ensure that actions are taken to address  model issues  Explicit authority  Stature within the organization   Commitment and support from higher levels of management Effective Challenge :  “critical analysis by objective, informed parties that can identify model limitations and produce appropriate changes. Effective challenge depends on a combination of incentives , competence , and influence .”

Page 4

© 2019 Darling Consulting Group, Inc.

Effective Challenge in Practice Current Industry Observations & Trends

 Sometimes only thought of as a MRM and validation‐related activity  Commonly associated with the actual model validation itself  Inconsistent within organizations, horizontally across the industry  Evidence frequently limited

Common regulatory criticism:  “model validation failed to demonstrate effective challenge”

Page 5

© 2019 Darling Consulting Group, Inc.

Key Takeaways

 Effective Challenge is everyone’s responsibility!  Understanding expectations, inspirational leadership, and fostering a collaborative risk culture are driving forces  Regulatory consistency will help too

Page 6

© 2019 Darling Consulting Group, Inc.


Regulatory & Industry Perspectives Effective Challenge In Practice Evolving & Exemplary Practices Concluding Thoughts / Q&A

Page 7

© 2019 Darling Consulting Group, Inc.

Two Approaches to Regulation



“Tell me the goal”

“Tell me what to do”

Example: 1996 Interest Rate Risk policy statement (FIL-52-96)

Example: 5% Tier 1 leverage is required to be well- capitalized

Page 8

© 2019 Darling Consulting Group, Inc.

MRM Has Been Expanding In Practice

 2011 MRM guidance  ̶  expanded in practice over past 8 years  Model Risk Management (MRM) continuing to evolve  Horizontal review (examiners and validators) resulting in trickle down of  best practices   Broader risk management initiatives adding additional emphasis   Enterprise Risk Management (ERM)  Operational risk management  Data, technology, model use and complexity expanding  Regulatory focus has been on large and medium organizations but  expectations for community banks / credit unions is ramping up FDIC FIL‐22‐2017:  Adoption of Supervisory Guidance on Model Risk Management (6/7/17) Statement of Applicability to Institutions under $1 Billion in Total Assets: It is not expected that this guidance will pertain to FDIC‐supervised institutions  with under $1 billion in total assets unless the institution's model use is significant, complex, or poses elevated risk to the institution.

Page 9

© 2019 Darling Consulting Group, Inc.

Model Governance & Three Lines of Defense

Governance: Board (Vision & Tone from the Top) Align risk appetite, business strategies, and the budget

1st Line Business Lines

2nd Line Model Risk Management

3rd Line Internal Audit

 Manage the business and  model  development  Involved in day‐to‐day risk  management  Follow a risk process  Apply internal controls and risk  responses

 Oversee and challenge business  line risk management  Provide guidance, direction and  a different perspective  Develop risk management  framework

 Review the 1 st and 2 nd lines   Challenge current processes  independently  Objective evaluation and 

assurance on effectiveness of risk  mgmt (design & implementation)

Page 10

© 2019 Darling Consulting Group, Inc.

Vision, Communication, & Talent

• Coach Clear Strategy (Vision) − Worst – top doesn’t care − Best – clear “tone from the top” • Coordination across 3 lines − Worst – excessive confrontation − Best – collaboration & respect • Business Lines begin Defense! − Worst – not my job − Best ‐ effective limit identification

Page 11

© 2019 Darling Consulting Group, Inc.


Regulatory & Industry Perspectives Effective Challenge In Practice Evolving & Exemplary Practices Concluding Thoughts / Q&A

Page 12

© 2019 Darling Consulting Group, Inc.

2nd Line of Defense: Model Risk Management Roles and Responsibilities Relative to Effective Challenge

MRM Framework

Validation •Sets validation  standards •Defines scope •Manages resources  (internal and  external) •Validation  management and  oversight

Communication •Communicates to 1 st

Lifecycle Management •Model update or  upgrade procedures •Benchmarking /  challenger modeling •The annual touch •Retirement or  replacement

•Policies •Procedures •Effective challenge  description/definition  •Establishing authority  & leadership

LOD (validation  documentation,  findings, ideas, 

recommendations)  •Reports to MRMC •Reports to BOD •Examiner interaction •Q&A

Page 13

© 2019 Darling Consulting Group, Inc.

2nd Line of Defense: Model Risk Management Common Challenges or Pitfalls

 Cultural and leadership issues  Lack of authority and stature within organization  Insufficient processes or established expectations  Inadequate policies and procedures   Insufficient education or training  MRM review practices  LOB executive awareness/understanding  Poor validation scoping/vendor (validator) selection  Insufficient documentation, minutes, etc.  Resourcing (financial and personnel)

Page 14

© 2019 Darling Consulting Group, Inc.

1st LOD: The Model Owner, LOBs Effective challenge roles and responsibilities

Ongoing  Monitoring • Back testing/  outcomes  analysis • Benchmarking  • Sensitivity and  stress testing • Other  performance  measurement 

 Lifecycle  Management • Model update or  upgrade  processes • Benchmarking/  challenger  modeling • The annual touch • Retirement and  replacement


Development • Data selection  and testing • Model 

• Vendor 

assessment and   selection • Model selection • Design and  construct

development and  testing

• Documentation

Page 15

© 2019 Darling Consulting Group, Inc.

1st Line of Defense: The Model Owner, LOBs Common Challenges or Pitfalls

 Lack of awareness and/or cultural resistance  Lack of formal processes or established expectations  Lack of proper incentives and motivation    Insufficient education or training  Model development and MRM practices  Documentation standards  LOB/executive awareness/understanding  Knowing right questions to ask and interpreting responses  Under‐documented models  Lack of ongoing monitoring and lifecycle management    Limited MRM direction or leadership  Resourcing

Page 16

© 2019 Darling Consulting Group, Inc.

3rd Line of Defense : Audit, Board of Directors Roles and Responsibilities Relative to Effective Challenge

Audit • Makes sure effective challenge is  taking place • Assesses sufficiency of MRM  policies and procedures  • Confirms 1 st and 2 nd line  following established standards  and procedures • Processes and controls • Select revalidation

Board of Directors • Makes sure effective challenge  is taking place • Establishes risk culture • Sets tone • Defines risk appetite • Delegates authority for  management and oversight • Confirms process working as  intended

Page 17

© 2019 Darling Consulting Group, Inc.

3rd Line of Defense: Audit, Board of Directors Common Challenges or Pitfalls

 Underdeveloped risk culture/leadership  Insufficient awareness or understanding  Lack of requisite knowledge and skills  Quantitative  Business expertise  Lack of well documented expectations (polices and procedures)   Resourcing

Page 18

© 2019 Darling Consulting Group, Inc.


Regulatory & Industry Perspectives Effective Challenge In Practice Evolving & Exemplary Practices Concluding Thoughts / Q&A

Page 19

© 2019 Darling Consulting Group, Inc.


 Compliance oriented perspectives are prevalent  In‐house validation staff often lack beneficial business expertise  (external validators as well)  Cost sensitivity can drive less optimal choices / decisions  Differing validation rigor expectations for large vs. smaller  institutions is challenging  At the end of the day, regulatory emphasis / criticism is the usual  catalyst to inspire more robust effective challenge practices

Page 20

© 2019 Darling Consulting Group, Inc.

The Good, the Bad, and the Ugly Examples of  Exemplary and Not So Exemplary Practices

Policies and procedures Education, cross training and outreach

Validation scope specification MRM leadership and resourcing Cultural evolution

Page 21

© 2019 Darling Consulting Group, Inc.


Regulatory & Industry Perspectives Effective Challenge In Practice Evolving & Exemplary Practices Concluding Thoughts / Q&A

Page 22

© 2019 Darling Consulting Group, Inc.

Keys to Success Clear Policies & Procedures

 MRM needs to establish clear set of expectations   Well defined roles and responsibilities  Model development and management practices  Model documentation standards  Communication protocols  Model lifecycle   Escalation and remediation   Assignment of sufficient authority and stature  Clear delineation of effective challenge process for each LOD

Policies and procedures designed to inform and educate all stakeholders 

Page 23

© 2019 Darling Consulting Group, Inc.

Keys to Success Training & Education

 Provide ongoing training and education  Development or 3rd party model/vendor selection  Model development / testing and implementation practices  Model documentation  Management, executive and BOD education  Statistics for non‐statisticians       Interpreting results  What questions to ask    Two‐way knowledge transfer  Quantitative  Business  

Not a one and done exercise – institutions need to invest in ongoing support

Page 24

© 2019 Darling Consulting Group, Inc.

Keys To Success Ongoing Evolution of the Risk Culture

 Ongoing training and education that promotes healthy,  corporate‐wide risk culture  Reinforcement of authority and stature of MRM and relation to  broader ERM initiatives  Development of  incentives that promote improved model  performance and dissuades “gotcha” mindset (not all about risk)  Fostering of a collaborative spirit

Important: we are all on the same team!

Page 25

© 2019 Darling Consulting Group, Inc.

Benefits of a Successful Effective Challenge Process  The Strategic Side

 Better models!  Reduced implementation and remediation costs  Establishment of a performance‐focused culture  Increased confidence with stakeholders

 Drives more active decision‐making  More favorable regulatory interactions

Page 26

© 2019 Darling Consulting Group, Inc.

MRM Often Has Begun By Dissecting the Guidance There are 190 “Shoulds”

Control Environment


•Access Control

1 4 1

•Change Management

•Version Tracking

Ongoing Monitoring 14

•Policies & Procedures Governance 39


 4  5  2  3


•General •Overrides

•Roles & Responsibilities: 1st Line of Defense   3 •Roles & Responsibilities: 2nd Line of Defense  8 •Roles & Responsibilities: 3rd Line of Defense 14

•Sensitivity & Stability

Risk Assessment & Validation 75

•Development Documentation Inventory & Documentation 21

•Annual Reviews

  2 10 28   3 15   4   6   7


•Conceptual Soundness and Model Design

•Model Inventory

7 3


•Validation Documentation

•Implementation and Process Verification

•Outcomes Analysis •Sensitivity Analysis •Third‐Party Models •Third‐Party Validators

•Dataset Creation Model Development


4 4 2 7 7


•Implementation & Use




•Third‐Party Models

Page 27

© 2019 Darling Consulting Group, Inc.

Risk Management is a Cultural Mindset





Page 28

© 2019 Darling Consulting Group, Inc.

Aligning Risk Management Strategy & Organizational Culture

Risk Management  Strategy Drives:

Organizational Culture  Drives:

Risk Culture Chain







CORRECT Management of  Risk

ACTUAL Management of  Risk

Danger: If Culture is not fully aligned with Risk Strategy, Culture wins (and the organization loses).

Page 29

© 2019 Darling Consulting Group, Inc.

Secrets to MRM Success & Corporate Benefit Lessons from the Mouse  We are all “cast members”   Perform with an experience mindset, not a  task mindset!  Make personal accountability a part of the  organization’s culture  Little wins will add up!

Page 30

© 2019 Darling Consulting Group, Inc.

Key Takeaways

 Effective Challenge requires all three lines of defense to work together  Committed leadership, establishing clear expectations, and fostering a healthy risk culture is paramount  An ounce of inspirational regulatory emphasis will help

Page 31

© 2019 Darling Consulting Group, Inc.


Page 32

© 2019 Darling Consulting Group, Inc.

Large Bank Supervision Forum

Effective Challenge in Model Risk Management November 19, 2019

Michael Guglielmo, Managing Director

(978) 499‐8159

© 2019 Darling Consulting Group, Inc.  •  260 Merrimac Street  •  Newburyport, MA 01950  •  Tel: 978.463.0400  •

Michael R. Guglielmo

With over 30 years of experience in strategic risk management,  Mike Guglielmo has provided technical and strategic consulting to  a diverse group of financial institutions. Mike is also a frequent  author and top‐rated speaker on a variety of balance sheet and  model risk management and operational risk management topics.  He serves as Chairman of the Board for the Financial Managers  Society and is a faculty member for the FMS Institute and ABA  Risk Management School. During his tenure at DCG, Mike has served in various capacities,  including director of financial analytics. In addition, he is a  technical resource for the ongoing development of many of DCG's  quantitative and strategic risk management products and  services. Prior to joining DCG, Mike managed the ALCO and  strategic planning processes for a regional bank in the northeast.  Mike is a graduate of Fairfield University with a degree in  economics.

Managing Director (978) 499‐8159

Page 34

© 2019 Darling Consulting Group, Inc.

DCG’s Vantage Point

Four Decades of Community and Mid‐Size Bank & Credit Union Experience

• Independent Balance Sheet Management Advisory  – Custom outsourced ALCO‐related services (modeling, reporting, strategic advisory)   – IRR, liquidity, capital planning, credit and capital stress testing – Risk mitigation and performance consulting • Behavioral Modeling and Analytics  – Deposits360°™ – Prepayments360°™ • Liquidity Stress Testing Modeling Technology and Consulting  – Liquidty360°® (online liquidity stress‐testing platform) • Stress event and relief strategy simulation  • Early warning monitoring – Liquidity risk and funds management reviews and consulting • Capital Planning and Credit Stress Testing – Robust statistically‐based credit stress testing for community banks and credit unions • Model Risk Management Consulting and Validation – MRM consulting and education – All financial and operational risk models

Page 35

© 2019 Darling Consulting Group, Inc.

About DCG

Independent balance  sheet / model risk  management  consultancy

Serving the industry  for nearly 40 years

Over 115 members of  the firm

National presence  with more than 600  clients served  annually

Thought leaders and  highly regarded  speakers, authors,  and educators

Page 36

© 2019 Darling Consulting Group, Inc.

DCG Services

Consulting and Advisory  Strategic balance sheet advisory services  Model development and management  Effective ALCO processes  Capital planning and stress testing  ALCO performance and process reviews  Liquidity management process reviews  Risk mitigation Model Risk Management and Validation  Expert validation of all financial and operational  risk models including ALM, liquidity, credit, CECL,  and BSA/AML  Large, mid‐sized, and community institutions  Model Risk Management consulting and advisory  Model Risk Management audits, peer reviews,  and performance assessments

Behavioral Studies and Cloud‐Based Analytics  Deposits360°®‐ core deposit study and analytics  Prepayments360°™ – loan prepayment study and  analytics  Liquidty360°® – online liquidity stress testing  solution   Credit Stress Testing & CECL Benchmarking  Statistically based credit loss modeling for  community banks and credit unions  CECL peer results and assumption benchmarking  Education  In‐house seminars and workshops  Webinars  Annual balance sheet management conference  Examiner training

Page 37

© 2019 Darling Consulting Group, Inc.

DCG Resources

White Papers  ALM modeling & ALCO performance  Liquidity risk and funds management  Quantitative assumption development / support  Capital planning and stress testing  CECL  Model Risk Management and validation  E‐Mail Newsletters (Complimentary)

Annual Balance Sheet and MRM Conference  6 tracks covering balance sheet and model risk  management   300+ attendees with up to 13 hours CPE available   Continuously running for over 35 years  National and regional seminars and conferences   Custom education for directors, executives, and  practitioners  Over 80 educational events annually Webinars, Seminars, and In‐House Education  Complimentary webinars

 DCG Insights  DCG Bulletin  MRM Uncovered

How To Contact DCG


Page 38

© 2019 Darling Consulting Group, Inc.

Made with FlippingBook - Online magazine maker