Exam Feedback May 2021 Part 2 Exam Feedback

[LucreziaB]

Is this the model risk question? I reviewed all the options and I thought option A made the most sense. I think A was something along the lines of the CRO should challenge the model developers to document the model and manage its inventory. I mean these are 2 challenges with model risk management, shouldn't the people designing the model should be the ones documenting assumptions and managing inventory but they can be passive about this or disregard this part of the process, so isn't it up to model risk or CRO to call this out?

overall these question highlights the flaw of the exam, poorly structured answered choices that are up for different interpretation. I wouldn't fault anyone who picked A or B in this case. I can't even count how many questions I was torn between two close answers.

I agree. I deal a lot with model risk management in my position. Answer A was more clearly correct to me based on that experience (because really only the modeler can properly document his own model and this is part of owning your model). However, I thought the answer about challenge was very plausible because, even though that's the best way for second line to do its job effectively and "challenge" implies more of an outsider, 1LOD is the First LOD for a reason. Our modelers and users are constantly monitoring and discussing model performance, and users have to approve the use of the model, too. All of that is arguably a "challenge" and in my experience modelers and users are better positioned to vet a model than 2LOD; 2LOD is a control against 1LOD wanting to overlook defects in a model because it benefits business operations/modelers don't want to admit of errors.

 

[Ullaskarkera]

Some of the questions I partially recall:

1. ES at 97.5%
2- basel ii.5 capital requirement (answer was 27.3 - C?)
3- Liquidity stress scenario - think I went for switch from short to long term funding, don't think it's right.
4- question on RAF
5- Asset allocation and security selection
6- Securitization question on correlation and volatility i think; equity and subordinated increases and senior decreases?
7- liquidity asset buffer - answer 450?
8- Asset 1 contribution from total portfolio VAR, something like that.
9- stress loss question - answer is C i think, 1.7
10- question on LTP
11- bond price after 3 years question

All i can remember for now

1. ES at 97.5% --- this was the easy calculator usage bit ...cant remeber the ans
2- basel ii.5 capital requirement (answer was 27.3 - C?) --- this was one the question with too much info ..i picked up to calculate MR and got 27.3 too
3- Liquidity stress scenario - think I went for switch from short to long term funding, don't think it's right. ----selected the same short term to long term option
4- question on RAF ---was the answer to select all possible to risks ?
5- Asset allocation and security selection ---- small number and then big number ? something like 0.8 and 4.X
6- Securitization question on correlation and volatility i think; equity and subordinated increases and senior decreases? --- yeah , selected the same
7- liquidity asset buffer - answer 450? ---- did you add the 12 month operational figures given ? or only the stres inflow outflow and Contigency number
8- Asset 1 contribution from total portfolio VAR, something like that.
9- stress loss question - answer is C i think, 1.7 --- did you calculate using stressed PD and subtracted normal PD values ? ..seems answer was C
10- question on LTP
11- bond price after 3 years question


12 : BCVA question ...ans D ? -5K something ?
13 : no default in 3 yr question , 2 portfolios and correlation ogiven
14 : some predatory trading , model risk , moral hazard question , was A or D the answer ? guess i selected A
15 UL component , correlation was given and total portfolio split as 16K and 24K , guess i selected 131 as answer

 

[mariomansour]

1. ES at 97.5% --- this was the easy calculator usage bit ...cant remeber the ans
2- basel ii.5 capital requirement (answer was 27.3 - C?) --- this was one the question with too much info ..i picked up to calculate MR and got 27.3 too
3- Liquidity stress scenario - think I went for switch from short to long term funding, don't think it's right. ----selected the same short term to long term option
4- question on RAF ---was the answer to select all possible to risks ?
5- Asset allocation and security selection ---- small number and then big number ? something like 0.8 and 4.X
6- Securitization question on correlation and volatility i think; equity and subordinated increases and senior decreases? --- yeah , selected the same
7- liquidity asset buffer - answer 450? ---- did you add the 12 month operational figures given ? or only the stres inflow outflow and Contigency number
8- Asset 1 contribution from total portfolio VAR, something like that.
9- stress loss question - answer is C i think, 1.7 --- did you calculate using stressed PD and subtracted normal PD values ? ..seems answer was C
10- question on LTP
11- bond price after 3 years question


12 : BCVA question ...ans D ? -5K something ?
13 : no default in 3 yr question , 2 portfolios and correlation ogiven
14 : some predatory trading , model risk , moral hazard question , was A or D the answer ? guess i selected A
15 UL component , correlation was given and total portfolio split as 16K and 24K , guess i selected 131 as answer

12 : BCVA question ...ans D ? -5K something ? - - > i subtracted the DVA from CVA as i recall, answer was 1,000 something - not sure if right.

13 : no default in 3 yr question , 2 portfolios and correlation ogiven - - > I think i answered 86.5% on this one? Or maybe that was a different answer

14 : some predatory trading , model risk , moral hazard question , was A or D the answer ? guess i selected A --> the frictions question? Not sure if it was adverse selection

15 UL component , correlation was given and total portfolio split as 16K and 24K , guess i selected 131 as answer

 

[badbunny]

I agree. I deal a lot with model risk management in my position. Answer A was more clearly correct to me based on that experience (because really only the modeler can properly document his own model and this is part of owning your model). However, I thought the answer about challenge was very plausible because, even though that's the best way for second line to do its job effectively and "challenge" implies more of an outsider, 1LOD is the First LOD for a reason. Our modelers and users are constantly monitoring and discussing model performance, and users have to approve the use of the model, too. All of that is arguably a "challenge" and in my experience modelers and users are better positioned to vet a model than 2LOD; 2LOD is a control against 1LOD wanting to overlook defects in a model because it benefits business operations/modelers don't want to admit of errors.

same here, worked in 1 & 2 LoD. In a serious organization 1LoD will never challenge their models, that’s a nonsense. How is the model development team gonna bring up weaknesses on their own models? They will try to cover up if so.

that’s 2LoD job, challenge; it’s even on a the Guide to Internal Models from the European Banking Authority and the Capital Requirements Regulation.

 

[adeex18]

same here, worked in 1 & 2 LoD. In a serious organization 1LoD will never challenge their models, that’s a nonsense. How is the model development team gonna bring up weaknesses on their own models? They will try to cover up if so.

that’s 2LoD job, challenge; it’s even on a the Guide to Internal Models from the European Banking Authority and the Capital Requirements Regulation.

exactly! but the way I interpreted option A was that the CRO would challenge the model guys to document model assumptions and manage inventory rather than the CRO asking them to challenge their own model which I agree is not the right thing to do.

 

[jho012345]

The Covid times ... spot and future oil price fell. Isn't the A also make sense? I was really confused because I clearly read A ( the all assets price dropped including T-Bond) that long term Treasury and investment grade bonds declined unlike 2008 before government policy and I also read B the oil price dropped too. I was really considering one of them for 5 min sigh.

I totally agree! I was so annoyed with this question because it lacks the perspective dates.... oil did crash, treasuries did crater then came roaring back so it was all predicated on if you read the silly basel whitepaper on asset values that is not incredibly out of date relative to where the market is trading at today... I probably spent 6 minutes on this one! lol!

 

[jho012345]

1. ES at 97.5% --- this was the easy calculator usage bit ...cant remeber the ans
2- basel ii.5 capital requirement (answer was 27.3 - C?) --- this was one the question with too much info ..i picked up to calculate MR and got 27.3 too
3- Liquidity stress scenario - think I went for switch from short to long term funding, don't think it's right. ----selected the same short term to long term option
4- question on RAF ---was the answer to select all possible to risks ?
5- Asset allocation and security selection ---- small number and then big number ? something like 0.8 and 4.X
6- Securitization question on correlation and volatility i think; equity and subordinated increases and senior decreases? --- yeah , selected the same
7- liquidity asset buffer - answer 450? ---- did you add the 12 month operational figures given ? or only the stres inflow outflow and Contigency number
8- Asset 1 contribution from total portfolio VAR, something like that.
9- stress loss question - answer is C i think, 1.7 --- did you calculate using stressed PD and subtracted normal PD values ? ..seems answer was C
10- question on LTP
11- bond price after 3 years question


12 : BCVA question ...ans D ? -5K something ?
13 : no default in 3 yr question , 2 portfolios and correlation ogiven
14 : some predatory trading , model risk , moral hazard question , was A or D the answer ? guess i selected A
15 UL component , correlation was given and total portfolio split as 16K and 24K , guess i selected 131 as answer

Regarding the ES at 97.5% -> I remember it was the average to the worst 4 values which was pretty easy to calculate....
Asset Allocation and Security Selection -> So i couldn't back into the exact formulas they were using but I think that it was smaller # for asset allocation and larger number being stock selection since the under/over weights relative to the benchmark were small. (at least this is how i rationalized it)
9 - I calced the stressed EL value using the EAD * Stressed PD * LGD....
10 - What was the question on LTP again?

 

[jho012345]

For libor seems they had 2 questions...1.sofr libor which is more volatilie....sofr is more volatile....but if we use 3m sofr compounding rate vs 3m libor...libor is more volatlie..not sure which ans they would consider,.....2.libor still has more outstanding contracts.




Climate risk...something like mkt oper and liq risk are impacted.

Covid times....spot and future oil price fell.

Cyberrisk....effected..independent of location

UNsupervised learning...clustering

AML TF....more complex and use of ML

small retail bank...loan..3rd party...something like backtesting black hole

I recollect above 8 from current issues

Regarding the AI/ML question.... it was something like... Analyst X at Firm Y wants to use an AI/ML technique without using labeling. What type of AI/ML should they implement: A.) Clustering B.) Lasso C.) SVM D.) Logit and I actually picked SVM.... because it is unsupervised... Was this the question you were thinking of?

 

[adeex18]

Regarding the ES at 97.5% -> I remember it was the average to the worst 4 values which was pretty easy to calculate....
Asset Allocation and Security Selection -> So i couldn't back into the exact formulas they were using but I think that it was smaller # for asset allocation and larger number being stock selection since the under/over weights relative to the benchmark were small. (at least this is how i rationalized it)
9 - I calced the stressed EL value using the EAD * Stressed PD * LGD....
10 - What was the question on LTP again?

I think the LTP question was something about the CFO reviewing the LTP and best risk management practices - I think one of the choices was to undervalue stable sources of funding and liquid assets, I chose that.

 

[LucreziaB]

same here, worked in 1 & 2 LoD. In a serious organization 1LoD will never challenge their models, that’s a nonsense. How is the model development team gonna bring up weaknesses on their own models? They will try to cover up if so.

that’s 2LoD job, challenge; it’s even on a the Guide to Internal Models from the European Banking Authority and the Capital Requirements Regulation.

I never said 1st line was the only or primary challenger. This is exactly why it was a poor question even if there was still a clear best answer...The question asked what would be appropriate for a CRO to ask groups to do, not implying any of these actions were exclusive of other actions. Page 141 of the ops risk text book describes user challenges to models (and their limitations, which you also name). Even if their challenge is limited, they still have a valid perspective - and it would be concerning if they didn’t do any testing/challenge of a model they would have to use regularly for business purposes - and the CRO should have them participate in the model testing process.

 

[jho012345]

I never said 1st line was the only or primary challenger. This is exactly why it was a poor question even if there was still a clear best answer...The question asked what would be appropriate for a CRO to ask groups to do, not implying any of these actions were exclusive of other actions. Page 141 of the ops risk text book describes user challenges to models (and their limitations, which you also name). Even if their challenge is limited, they still have a valid perspective - and it would be concerning if they didn’t do any testing/challenge of a model they would have to use regularly for business purposes - and the CRO should have them participate in the model testing process.

This is a "grey" zone that we constantly battle if you are a "model" owner (don't get me started on SR 11-7) at a Bank. Model Risk starts with the ensuring appropriate development which cannot be performed by the second LoD. it MUST start with the 1LoD challenging their model, finding inherent limitations and assumptions etc.... then model risk layers on their validation/independance. Every shop in practice does this differently and yes, it was a crappy question!

 

[Rhim0001]

Exactly, First Line of Defence doesn't challenge their own models; its 2LoD which does that (Risk Managers / Validation Unit) and after their "Ok" it gets approval.

So that’s why I say this is tricky

Regarding the ES at 97.5% -> I remember it was the average to the worst 4 values which was pretty easy to calculate....
Asset Allocation and Security Selection -> So i couldn't back into the exact formulas they were using but I think that it was smaller # for asset allocation and larger number being stock selection since the under/over weights relative to the benchmark were small. (at least this is how i rationalized it)
9 - I calced the stressed EL value using the EAD * Stressed PD * LGD....
10 - What was the question on LTP again?

For q9 that’s the only way i could get the answers which match the options, i recall they did not specify the correlation between each, which may also require calculation but I’m not bothered to do so

 

[Rhim0001]

Regarding the AI/ML question.... it was something like... Analyst X at Firm Y wants to use an AI/ML technique without using labeling. What type of AI/ML should they implement: A.) Clustering B.) Lasso C.) SVM D.) Logit and I actually picked SVM.... because it is unsupervised... Was this the question you were thinking of?

I’ve done one practice question regarding this, pretty sure it’s clustering since it’s not labelled

 

[mariomansour]

I’ve done one practice question regarding this, pretty sure it’s clustering since it’s not labelled

Yup went for clustering, most probably the right answer

 

[Rhim0001]

This is a "grey" zone that we constantly battle if you are a "model" owner (don't get me started on SR 11-7) at a Bank. Model Risk starts with the ensuring appropriate development which cannot be performed by the second LoD. it MUST start with the 1LoD challenging their model, finding inherent limitations and assumptions etc.... then model risk layers on their validation/independance. Every shop in practice does this differently and yes, it was a crappy question!

Agree, I think both options are correct, it’s reasonable for CRO to do both jobs.In normal practice business managers do review their models before submitting for approval. But since this is more likely to be required by CIO/COO/CFO, which are all first line of defence, so I picked option B.

but anyway, I think garp should not have given these kind of controversial questions. I do believe both are correct

 

[jho012345]

Yup went for clustering, most probably the right answer

Did the question specifically mention unsurpervised learning? I just remembered the "without data labeling" aspect...not sure if this is right but I picked SVM.... "One-class SVMs are a special case of support vector machine. First, data is modelled and the algorithm is trained. Then when new data are encountered their position relative to the “normal” data (or inliers) from training can be used to determine whether it is “out of class” or not — in other words, whether it is unusual or not. Because they can be trained with unlabelled data they are an example of unsupervised machine learning."
Source: https://medium.com/@jamesstradling/...ne-class-support-vector-machines-129579a49d1d

 

[ratboy]

I agree with all the above comments in that it was predominately qualitative....

There was one question that took up half the page and was lots of blah blah between investments in several different products. It then went on to say that the investor made two additional transactions, a correlation swap and something else I believe.

I spent so much time trying to understand the question that I left it till the end. Then time was short and I just read the answers and one correct one that stood out was that if the realised correlation was greater than the fixed the investor would gain. I ticked that one. TBH not sure if it was the right one but at least the payoff was correct.

 

[Rhim0001]

I agree with all the above comments on that it was predominately qualitative....

There was one question that took up half the page and was lots of blah blah between investments in several different products. It then went on to say that the investor made two additional transactions, a correlation swap and something else I believe.

I spent so much time trying to understand the question that I left it till the end. Then time was short and I just read the answers and one correct one that stood out was that if the realised correlation was greater than the fixed the investor would gain. I ticked that one. TBH not sure if it was the right one but at least the payoff was correct.

Dave

This was purely experience on tackling questions
I usually start by reading the last sentence of the question to determine if I had to read the whole question or not. This would definitely save a lot of time

 

[badbunny]

So that’s why I say this is tricky

For q9 that’s the only way i could get the answers which match the options, i recall they did not specify the correlation between each, which may also require calculation but I’m not bothered to do so

Q9 I performed as you, we could get the correlation among them since we have the EL under normal conditions but if taken that correlation assumption to get Stress Loss the answers did not match any of the given ones. So the Stress Loss is just EL - ELs. ELs just taking PDs.

Yup went for clustering, most probably the right answer

yeah it was clustering, AML.

 

[Rhim0001]

Q9 I performed as you, we could get the correlation among them since we have the EL under normal conditions but if taken that correlation assumption to get Stress Loss the answers did not match any of the given ones. So the Stress Loss is just EL - ELs. ELs just taking PDs.

yeah it was clustering, AML.

Oh i have just calculated stress ELs for the answer
Which is simply Ead*stressed PD*LGD, didn’t subtract this from normal EL
But i have no idea how to do this question anyway, I just used the above method since I could not think of another one