Part 1 Feedback (November 2016 FRM Exam)
Most of the feedback was posted in Nicole’s thread at https://forum.bionicturtle.com/threads/november-2016-part-1-exam-feedback.10028/. It may be hard to believe, but with respect to exam difficulty, the comments are approximately in line with history. There has never been an FRM exam that engenders widespread confidence. This year was no different, many candidates perceived the exam to be difficult or very difficult; e.g., “Way to hard, many unexpected questions and too tight on time.” … “Harumph! so yes a couple of curve balls. I blew up on my time management.” We did notice another perennial feedback point: GARP’s practice exams are vastly insufficient, by themselves, to prepare for the exam experience. I’ve been making this point, from a mathematical perspective for a long time: any sample set of practice exams, each containing 100 or 80 (or 50 or 40) cannot by definition cover the syllabus, which is multiples wider in breadth. If you take five GARP practice exams, you will still encounter questions you have not seen before. For two reasons: first, GARP's own practice questions tend to cluster around a few concepts and recycle. Second, as they age they become less relevant (an analogy to HS VaR: you want lots of data/sample, but you need to extend the window to get more data) because the exam itself aspires to be current and forward-looking.
Some selected comments (no reason for my selection except I found them interesting):
to hear this because I really do prefer to write FX questions that utilize the real-world quote syntax, which is base/quote. When you read EUR/USD by anybody who actually works in FX, they are quoting dollars per base 1.0 Euro.
In terms of concepts mentioned, I collected the following (included but hardly limited to):
Huge feedback thread at https://forum.bionicturtle.com/threads/november-2016-part-2-exam-feedback.10027/unread. Similarly tough experiences; e.g., ”All in all a very tough exam. I am hoping for about 50% correct answers.” Selected comments:
Most of the feedback was posted in Nicole’s thread at https://forum.bionicturtle.com/threads/november-2016-part-1-exam-feedback.10028/. It may be hard to believe, but with respect to exam difficulty, the comments are approximately in line with history. There has never been an FRM exam that engenders widespread confidence. This year was no different, many candidates perceived the exam to be difficult or very difficult; e.g., “Way to hard, many unexpected questions and too tight on time.” … “Harumph! so yes a couple of curve balls. I blew up on my time management.” We did notice another perennial feedback point: GARP’s practice exams are vastly insufficient, by themselves, to prepare for the exam experience. I’ve been making this point, from a mathematical perspective for a long time: any sample set of practice exams, each containing 100 or 80 (or 50 or 40) cannot by definition cover the syllabus, which is multiples wider in breadth. If you take five GARP practice exams, you will still encounter questions you have not seen before. For two reasons: first, GARP's own practice questions tend to cluster around a few concepts and recycle. Second, as they age they become less relevant (an analogy to HS VaR: you want lots of data/sample, but you need to extend the window to get more data) because the exam itself aspires to be current and forward-looking.
Some selected comments (no reason for my selection except I found them interesting):
- “The exam was half easy, half hard, not tough, not standard. There were questions given, relatively easy when solved on BionicTurtle.com, but leaving blank faces staring to the page when in action.” .. “I found the GARP and BT practice tests to be most helpful.” https://forum.bionicturtle.com/threads/good-luck-on-the-exam-tomorrow.10024/#post-46522
- "With regards to the exam content, a lot of EL and mortgage related questions. Very little of the topic material from GARP's practice papers for 2013, 2014, and 2016 were covered in this exam - I think the practice papers did not serve as a guideline as they were supposed to. For example, in the GARP practice papers, there were no questions on EL, one mortgage related question, Poisson distribution, SWAPs, Akike Info Criterion." https://forum.bionicturtle.com/threads/november-2016-part-1-exam-feedback.10028/page-3#post-46645
- “I think that Schweser questions were do-able and didn't really match the difficulty of the real exam, meanwhile most BT questions were more difficult than the real exam. So, although the difficulty of the exam is between Schweser and BT practice questions, the key factor is time! 100 questions in 4 hours, translating it into 2.4 minutes per question, I genuinely feel that most questions during the exam needed more than that! Like some people said, the theory questions were just horrible and needed more than 2.4 minutes to think through. And I remember they made a mistake in their question and the corresponding answer where they asked about confidence LEVEL and time period but in the answer, it talked about confidence INTERVAL and time period. They didnt make the quantitative questions any easier either, I must say GARP did really well in terms of challenging the exam takers, haha.” https://forum.bionicturtle.com/threads/november-2016-part-1-exam-feedback.10028/page-3#post-46670
- This sounds like good advice to me: “In retrospect I wish I had read the entire paper before starting the exam” https://forum.bionicturtle.com/threads/november-2016-part-1-exam-feedback.10028/page-4#post-46758
to hear this because I really do prefer to write FX questions that utilize the real-world quote syntax, which is base/quote. When you read EUR/USD by anybody who actually works in FX, they are quoting dollars per base 1.0 Euro.In terms of concepts mentioned, I collected the following (included but hardly limited to):
- Unexpected loss, including unexpected loss for two-loan portfolio; e.g., “A lot more FX [oddball currencies] and UL questions. Was speaking to another candidate on his second exam afterwards who was stumped by the UL question and not knowing how to calculate variance on the PD. Wow. Poor guy. David drums that one into all of us.” Thank you. He’s referring to variance(EDF) = EDF*(1-EDF). And, I will feel responsible if you forget this! Quickly, what’s the variance of a fair coin?
- Akaike information criterion (AIC) and Schwarz informatin criterion (SIC) https://en.wikipedia.org/wiki/Akaike_information_criterion
- Greek neutralization, include delta hedge; and gamma and vega neturalization
- Poisson distribution
- Autocorrelation
- MBS and convexity
- Covariance time series
- Volume and open interest
- American call option value with binomial tree
- Bullet vs. Barbell portfolio
- Lots of bond questions with added complexity
- strip hedge vs stack hedge when forward rate curve is upward sloping
- A commodity forward with convenience yield
Huge feedback thread at https://forum.bionicturtle.com/threads/november-2016-part-2-exam-feedback.10027/unread. Similarly tough experiences; e.g., ”All in all a very tough exam. I am hoping for about 50% correct answers.” Selected comments:
- “Was a tough nut to crack.. good mix of quantitative and qualitative questions.. Quantitative was still manageable, theory questions were just bouncers..” I don’t know what a bouncer is, sorry?! “A lot of op risk questions, hazard rates, backtesting, model risk. a lot more qualitative questions than expected”
- “You can look upon course providers for questions only. The core reading material must be read once or twice. No one comes to the level of BT in framing questions. Just solving BT and reading reference books would make lot of difference. GARP throughout remained assymetric in their difficulty level. In few exams they asked questions of average difficulty and some they were very tough. So better not judge GARP that how hard they can be next time.” https://forum.bionicturtle.com/threads/november-2016-part-2-exam-feedback.10027/page-10#post-46763
- “Holy cow, the Part 2 exam is the toughest exam ever, even though I passed level 3 CFA, this exam is way harder than than even the CFA level 2 and level 3. A lot of questions are item set with VERY long reading but only a small part of that is required to answer the questions. Many concepts was checking again and again but it is not ever seen in any previous exams or practice exam (CoCo was tested 3 times). The VaR question was very tricky, not like anything was seen and the liquidity adjusted VaR question was using the number of day to dispose of assets! (by the way anyone know how to do that question?). The most annoying thing is they changed the format of the exam into more like CFA level 3 with long descriptions using for multiple questions, it is so hard to keep going back and forth and they put a lot of redundant things in the questions that make it super confusing. The concepts question was supposed to be walk in the park in previous year but this year it is like the most tricky ones ever, lots of questions about the CCP and correlations with very hard concepts and answers that are so close it is very difficult to chose. Overall, I would say I got 88% for practice exam but for this one I am not even sure I can get 50%!!! Anyone know what is the usual minimum cut off for a pass?” https://forum.bionicturtle.com/threads/november-2016-part-2-exam-feedback.10027/#post-46529
- “My feeling is GARP simply provides the topics and reading materials. They do not want to 'spoon feed' the candidates. It is up to the candidates to master the material on their own. I think that is the appropriate approach. Like what I mentioned, if a 'professional exam' simply recycles test questions from a test bank, it becomes too easy and the certification loses value. All the candidate has to do is spend enormous amount of time memorizing, regurgitating, etc all available past examples. It becomes mechanical. It doesn't test problem solving skills, intelligence and real quantitative and critical thinking skills. But if a test contains new, tricky, multi-level and multi disciplinary questions, it raises the bar further and tests candidates real problem solving skills. I think that's what differentiates this exam from other exams. Personally, I hope GARP continues in this path. This guarantees that FRM holders have mastery of the subject matter PLUS excellent problem-solving skills.” https://forum.bionicturtle.com/threads/november-2016-part-2-exam-feedback.10027/page-15#post-46881
- “This time GARP crossed the threshold of BT. In FRM language, the questions were at Block Maxima (GARP) level given the BT the Mu, μ (the threshold). Earlier BT use to stay above the GARP level but now things have changed.” https://forum.bionicturtle.com/threads/november-2016-part-2-exam-feedback.10027/page-7#post-46689
- “You have 1000 P&L values. 99% VaR is USD 1.484 (it is given) which is the 10th item. And it is included in the ES calculation. Why this 10th item, which is the VaR, is included in the ES calculation?” https://forum.bionicturtle.com/threads/november-2016-part-2-exam-feedback.10027/page-12#post-46810 From me (David): @raya Understandable interpretation, but ES does not depend on (has no relationship to) VaR. The ES is a weighted average conditional on a probability threshold, but VaR is a quantile. So the 99.0% ES is the average of the 1.0% tail, which requires no reference to VaR. If there are 1,000 equally-weighted observations, the 99.0% ES is the average of the worst ten. Notice how that requires no reference to the VaR, which is good because VaR has multiple valid answers in this discrete case.”
- "Here are the list of things tested as I remember: Liquidity Duration, CCP (Heavily tested), ES, , Hedge Fund strategies (very hard question, list the disadvantage of strategies), CoCo (3 questions), Performance Attribution (very tricky one with asset allocation alpha), Illiquid asset return concepts, Backtesting (Confidence level vs Var confidence level), Operational Risk (heavily tested on concepts questions), Model Risk, Technology Risk, Securitization process, Correlation, Derive Hazard rate from multiple methods, Cummulative PD, Credit Spread (derived from CDS), Marginal CVA, RAROC, Risk Plan, Many more things that I cant remember now" https://forum.bionicturtle.com/threads/november-2016-part-2-exam-feedback.10027/#post-46530
- “So exhausted after taking the exam. I would say the exam was fair to tough. Quantitative questions are quite manageable. Only those simple formulas are tested: Three-tier securitization, LVaR, CVaR, Ho-Lee Model, PSA/CPR/SMM, Hazard rate model, Spread and RR, Correlation Swap, RAROC, Tier I Capital Ratio, Incremental VaR. Most difficult questions come from those vignette questions and ops risk questions. Some questions are very very tricky in a way that I can eliminate 3 choices, but the only one left seems incorrect as well" https://forum.bionicturtle.com/threads/november-2016-part-2-exam-feedback.10027/#post-46535
- “There was also a question on the shape of the implied volatility curve for options on a stock which is in an M&A process and is therefore expected to either go up by 25% or go down by 20%. This puzzled me for a while.” https://forum.bionicturtle.com/threads/november-2016-part-2-exam-feedback.10027/page-2#post-46565
- http://trtl.bz/nov16-p2-epic-fb-1
- http://trtl.bz/nov16-p2-epic-fb-2
- http://trtl.bz/nov16-p2-epic-fb-3
- Backest VaR
- Portfolio VaR, including MVaR & iVaR
- PSA & SMM
- RAROC
- CVA
- Counterparty risk, including netting
- Hazard rate
- CoCo (more than once?!)
- Tracking error
- Central Counterparties
- Asset/liability pension
- Cybersecurity (two questions)
- Ho Lee model (interest rate model)
- CCP
- LVaR, CVaR
- Correlation swap
- Basket CDS
- Securitization overcollateralization
- MBS
- Credit spread
- Question asking for CLN, CDS and TR Swap which provides most protection to buyer?
- Forging Best practices in Risk Management (CI-1)
- Year in Risk 2016 http://www.rmmagazine.com/2016/12/01/year-in-risk-2016/
- US corporate bonds: The weight of debt https://www.ft.com/content/41213b02-b87e-11e6-ba85-95d1533d9a62 “Credit ratings have been dragged lower by the boom in debt issuance. Two decades ago, more than 66 per cent of the companies rated by S&P held an investment grade rating; today it is 45 per cent. The percentage of groups with an A or higher rating has roughly halved over that period. Now only two US companies — Microsoft and Johnson & Johnson — carry the highest triple-A opinion.”
- Financial Technology Start-Ups to Get a License to Bank http://www.nytimes.com/2016/12/02/b...ology-start-ups-to-get-a-license-to-bank.html “The licenses from the Office of the Comptroller of the Currency, which oversees many national banks, will be available to companies like Square and Lending Club that accept deposits, facilitate electronic payments or lend money … The introduction of the licenses, called special purpose national bank charters, underscores how quickly so-called fintech firms are growing and how significant they are considered to the future of the financial industry.”
- Enron Still Matters, 15 Years After Its Collapse http://www.nytimes.com/2016/12/01/b...till-matters-15-years-after-its-collapse.html “Yet the Enron bankruptcy remains vitally relevant to today’s corporate board members — not because it was the largest American bankruptcy at the time, but because it gave birth to the fiduciary guidelines and best practices that form the contours of modern corporate governance. It is, in essence, where it all began in terms of corporate responsibility. The board policies, procedures and codes of ethics to which today’s corporate directors are subject are the direct descendants of the lessons learned from the staggering oversight failures of the Enron board.”
- Contagion in the CDS Market https://www.financialresearch.gov/working-papers/2016/12/01/contagion-in-the-cds-market/
- Three kinds of avoidable statistics mistakes that I notice in published papers http://www.cs.cornell.edu/~asampson/blog/statsmistakes.html “Failure to Reject = Confirmation: It’s tempting to think, when p < α, that you’ve found the opposite thing from the p < α case: that you get to conclude that there is no statistically significant difference between the two averages. Don’t do that! Simple statistical tests like the t-test only tell you when averages are different; they can’t tell you when they’re the same.”
- When Genius Failed (nice review!) https://investingcaffeine.com/2016/11/21/when-genius-failed/
- The Man Who Invented the World's Most Important Number (LIBOR) https://www.bloomberg.com/news/features/2016-11-29/the-man-who-invented-libor-iw3fpmed
- Finalising Basel III: Coherence, calibration and complexity, Keynote speech by Stefan Ingves http://www.bis.org/speeches/sp161202.pdf
Last edited:
)
I will look forward to your passing, don't let me down! 
Thanks
