Exam Feedback November 15, 2008

[Yun chiang Tai]

Hello David:
This year is totally different. I guess GARP has pushed itself to the higher ground!! The exam is designed
by finance professors for sure. Therefore, it is academic approach instead of .... I think it is a transition moment
since It is biased to Corporate risk management I think. The answer is not as tough as others thought in the
first place since there are many ones are designed for not to be calculated in detail for the time constraint.
I still think the time is limited, especially the clock here is a little lousy and imprecise, we lost more than five
minutes in each session in local time. That is they should feel sorry for... Especially,I already spend more than four
full months without rest. They should give the best working guy some favors. But the quasi-past exam questions
are not easy to find in each session. I think it is a little pity for the hard-working guys. But those questions are
tough and intricate designed by finance professors. We should give them some plaudit.... But this year, those
questions are too long, many ones take more than two and half minutes to read over once, how can we answer
them in time?? Maybe GARP should consider to extend the test time after this one...
The exam is great compared to the PhD qualification, especially in the finance field...Therefore, people are not
familiar with corporate finance and financial theory may be a little disappointed. Those guys who are familiar
with the CFA exams will feel great disappoint for this exam since the question is too long and the answer is too
short. Of course, it is carefully designed and no past information can be used for this one.
Most important, I really read those materials several times, but it needs a good memory for all of them. This time
they have all been covered in the exam. Especially those new material. I guess this exam is a piecework by the
academic professors, especially the US finance academic society....
All in all, I am grateful to D.H. as his "due diligence" for the material preparation. It does help me a lot, especially
for a foreign not use English as the native language. The exam shows a grate challenge....
sincerely...
chris
(from a vanishing nation by OTHERS!!)

 

[Ened]

The exam was not so difficult, but we had way too few time to solve every question. I ended up guessing more than 10 questions in session 1 because there was less than 5 minutes left. Some questions took me more than 5 minutes to solve, others even more than 10 minutes. To me this exam was easier than the 2008 practice exam. I am sure I could have scored 90% if we had one more hour. Now I guess I scored around 75% (I picked "good" in the survey). Time management is so important!
I was disappointed to see so many detail questions on Basel. Many were not covered in the AIMs. GARP announced that we didn't need to know the details on Basel. That was a wrong advise...
Also surprised to see difficult question on Qualifying SPE's.
Amaranth question was in fact not about the case study on Amaranth but just an application of things we should know. The question on Binomial distribution could easily be solved using a binomial tree, so we didn't need the Binomial distribution. The question about Poisson distribution I didn't even try to solve because of time limitations. It seemed out of place.

Looking back, it was very good to have Bionic Turtle to have as study tool. It made me start on time, which is important as there is so much material to cover. For me, the screencasts and the flash questions were really helpful. Great work, David! The forum and excel applications to a lesser extent. It went often too much into detail, and the time spend on there does not pay off at the exam in my opinion. Most important to pass is to start early, get the intuition and to practice a lot.

 

[Mudaltiru]

Thanks David for all the support.
Exam was as tough as expected. But not beyond readings or our capacities.
May be the strategy of testing by lenghthiness and distractions may be minimized/avoided as FRM not purely a competitive/elimination exam.
QSPE was there but Amaranth, per se, was not there though mentioned in a lengthy descriptive question.
One must/should have absolute clarity on Greeks and Basels. I mean including all intricacies. Many go with overall idea and broad understanding only.
Repeated practice in Quant and application of imp. formulas will save time for other majority of subjective questions. Overall less calci was used.
I faced a peculiar problem(shall I name it 'duality dilemma'). In most of the subjective questions say for about 40% of 140, I could easily eliminate two choices but was then struck with dilemma of choosing one from remaining two.

One question I remember on TROR.
A co. issues bonds of 300m - 6.5%. Enters into TROR on LIBOR+50bp. Payment semi-annually. At the end of a period there was 2% deterioration in assets and LIBOR was 4%. What is net flow?
a)net inflow 6m
b)net inflow 12m
c)net outflow 6m
d)net outflow 12m
What is wrong with my understanding because I felt it is 0.
Can somebody add/clarify the question or what I am missing.

Thanks to all.
Hope not to return to the forum next year.
But, will be happy, if had to return.

mudaltiru

 

[ankit singhania]

One question I remember on TROR.
A co. issues bonds of 300m - 6.5%. Enters into TROR on LIBOR+50bp. Payment semi-annually. At the end of a period there was 2% deterioration in assets and LIBOR was 4%. What is net flow?
a)net inflow 6m
b)net inflow 12m
c)net outflow 6m
d)net outflow 12m
What is wrong with my understanding because I felt it is 0.
Can somebody add/clarify the question or what I am missing.??

Well Ur given Options are wrong.As far as i remember It was 9 million instead of 6M

And the answer is net outflow 9 million outflow...Here Inflow : 300*2.25(4+.5/2) = 6.75 million
Outflow : 300*.02(decreased in margin) + 300*.0325 = 15.25
Therefore 6.75 - 15.25 = -6 million .Here the trick was that they said " decreased in margin of loan"(not asset deterioration) . here they used the margin term which needs to pay back to the Fixed reciever..I think...But the answer was : C

 

[kolzhas2008]

i did fail ...will be with bionicturtle for 2009 exam(

 

[vkaul1]

Were some questions changes in different papers? I thought that not everybody got the same exam they were jumbled up

 

[chih22]

I too thought this exam was too long and focused too narrowly as I felt a lot of the great stuff I learned while preparing for this exam through BT wasn't tested or tested appropriately.

I've worked in risk management at a bank in the interest rate risk group overseeing hedging and portfolio management of our fixed income and loan portfolios for over 3 years and have my CFA charter and I found this exam to be more difficult than any of the CFA exams I've taken.

Perhaps it is just age getting to me, as I can no longer focus and force myself to focus through 2 entire 2.5 hour exam sessions. Even though CFA exams were longer, the questions in the CFA exams, I thought were generally more theoretical, and those involving calculations were at least long case studies had at least had 2 to 3 questions that followed so that one didn't spend 2 minutes reading the case and then have to reread again and spend 5 minutes just to end up guessing on one question because there is too much data that I couldn't figure out where to start. I'm sure if I had more time, the question wouldn't have been as difficult, but there was no consistency in the exam format as some questions took over 5 minutes and some took 20 seconds so it was hard to manage time and not panic and stay focused during the last hour because if the last 20 questions on the exam are all long questions then there was no wasy one could finish, so I found myself not reading as carefully and skipping around...

Thus, I believe GARP, successfully, has made this exam biased towards those that are better at test taking. It is just my opinion that test taking is a skill and a strategy that is learned from being in school or being in perpetual studies. In my opinion, a recent PHD or MS in finance student, with no experience can do better on this exam than someone who has more work or practical work experience or studied with BT and knew the concepts well but may not have been as prepared mentally and strategically for this type of exam...I thought the keys to passing was maintaining focus, avoiding distractions, time management...At times, I felt I was taking an IQ test and I was exhausted.

Overall, I enjoyed my experience here on BT, but if I did not pass, then that is it for me as I will not retake this exam in 2009. I very much enjoyed learning and reading everything here in risk management, but I was very disapointed with the exam given by FRM. I would have liked to see more questions applying VaR rather than those that required one to scale it from annual to daily or to 10-day and all in the meanwhile, remain composed that I've already spent 3-4 minutes on one question punching numbers into my calculator and still don't have an answer yet.

Regarding the survey for this post, I voted my experience was "bad." As far as how well I think I performed on the exam, I felt I was 70-75% in the morning session and perhaps 55-60% in the afternoon session.

Regards,
Chih

 

[David Harper CFA FRM]

Thank you everyone for providing such *great* feedback about the exam. I have learned so much from these comments. I hope you all get to relax a bit now. Try not to judge only by your initial feeling: the exam is difficult. Every year, it is hard to find somebody who is very confident they passed.

I really want to improve BT for 2009. I think we need to do a better job in regard to practice questions. I am tempted to make that the first priority. One challenge, frankly, however is motivating people to start early preparation. I started very early in the year (e.g., with the screencast episode Questions) but, to tell the truth, for whatever reason, those early practice questions did not get much traffic. So, it is a two-fold challenge, I think: (i) We have to give more, better, and relevant practice questions but also (ii) we have to motivate candidates to start a bit earlier.

Do you agree? I am thinking of, for 2009, perhaps sending weekly/bi-weekly practice questions by email. This year I did send a bi-weekly email newsletter but it was a long thing with tips and then questions at the end. I got the feeling that few people found those helpful? Maybe I should make the newsletter focus about practice questions. Your feedback would be appreciated....

(I am hiring help to take all the historical sample questions, minus the bad ones, to publish them up. So we will hundreds more sample questions next year)

@Anil: You said, May be preparation material could have been more exam oriented! Agreed, we will focus on this.

@Rajababu: Thanks your for all the detail

@John (JOp): You have given awesome detail. I have already used some of this to give feedback to GARP. The transition/migration matrix is a good example. Currently (2008) it is implicit, we covered it, but you are correct, it was not explicit in the AIMs. I am confident GARP is receptive to further improvements in the Study Guide/AIMs so that testable ideas are comprehensively linked to assignments. And you saidy, "I think new application based practice questions is the key." This seems to be the key message to BT, so I appreciate that.

@Ened: Given our previous email conversations, I must give you special thanks for your feedback. Your appreciation is maybe more than I deserve. What you said is especially helpful, I will be thinking hard about this: "For me, the screencasts and the flash questions were really helpful....The forum and excel applications to a lesser extent. It went often too much into detail, and the time spend on there does not pay off at the exam in my opinion" This is good to know...as you know, I think BT should have spent more time on questions.

@Vivek: no, same questions for everybody. I don't know about sequence but everyone got the same exam.

Thanks again, David

 

[vkaul1]

I think Chih your exam was good and you will pass . You are getting overall 65-70% which is easy passing.
Vivek

 

[David Harper CFA FRM]

Chih22 - I agree with Vivek. I bet you will pass. It may be hard to believe, but that is nearly as much confidence as you will find coming out of the exam. Please come back and tell us....David

 

[Mudaltiru]

Thanks Ankit. Answer must be C.

"And the answer is net outflow 9 million outflow...Here Inflow : 300*2.25(4+.5/2) = 6.75 million
Outflow : 300*.02(decreased in margin) + 300*.0325 = 15.25
Therefore 6.75 - 15.25 = -6 million .Here the trick was that they said “ decreased in margin of loan"(not asset deterioration) . here they used the margin term which needs to pay back to the Fixed reciever..I think...But the answer was : C "

It is outflow of 6.00+9.75=15.75 less inflow 6.75= correct answer net outflow of 9.00 million.

I was caught in the trick. Can u or David throw some light on 'decrease in margin of loan' and how it is equivalent to appreciation of loan(I mean why the co. should pay 6m for decrease in margin of loan).
Thanks
mudaltiru

 

[ankit singhania]

I have voted as bad...I m expecting 75-80% in 1st half..and in 2nd half it may be 50-60%...I don't think i am going to clear it..

 

[john.ophof]

HI David,

Fine that you see our feedback remarks as useful. The early birds were indeed useful (I discovered an answer of one of the questions Riskmetrics and Garch(1,1) difference for example but I watched it to long ago on the exam). The problem with the questions and the spreadsheets and the PDF is the limited time available. If you want to do all you must choose what to do. Reading all the material as you suppose to do takes a piece of your time. Reading and listening to your movies seeing some spreadsheets.
To have it more practical and exam based. You can made it more case based. Provide generate data with a randomizer and let student do all the testing based on a case. And let the student make a regression in stead of following the aims one for one make the case and see you cover all the aims. I know this takes time too to make but 2009 will be especially around statistics quit the same..
A lot of things can be tested with this (Skewness, Jackbarra, t-test sample definitions, inference testing, confidence testing, historical VAR, VAR). A lot of the material can be used in risk management. Operation risk can also be done with a small LDA case reforging statistical practice.

Let the case do the work. One other thing what can be done is a flash exam per segment and make it lengthly and get a result. First make your questions and result per mail with time pressure in it.

Another thing what might be useful make couples of study-bodies to make exercises and sparring with fellow students possible (out of the box thinking). And make some kind of competition between the teams, competition is always helpful.

This is focussed on passing the exam but I think it is also more productive. I know things are not as easy as putting them in a fast reply but maybe you can roll your mind about it.

I hope Garp don't remove the Risk matrix questions from grading since I was correct on that one

John

 

[Yun chiang Tai]

Hello David:
the exam is totally different compared with past ones. I think they try to renew all of those ideas. If they try to
do that. I think they have to provide some materials for the testing preparation instead of just the papers and
chapters in books. But I know it is really hard for me. Because I have no good memory for all of those materials.
The FRM exam this year is tough I think and is suitable for those ones have been lectured by those professors
I guess.(Of course, they have to be newbies not those ones with them several years ago!) The test this year
needs a lot to test skill and I think that nobody can finish it on time without guessing. Therefore, pass or
not pass may be due to LUCK. That makes me feel a little disappointed. But we have to face the music as
I am not a native American. Therefore the language gap does exist between me and tests. But after the exam,
I really feel the test is more like a phD qualification instead of a test for those ones working in the company.
The past experience is a little void for this exam. But I know the examer try so hard to have some questions
that can tell the different of testers. I still think it is not about the early preparation since after the morning
session, I heard a lot of people talking about the preparation seems to be useless. I know a lot of them go
to the CRAM school but it seems a little.... I already take the past exam of 2000 to 2003 FRM exams which is
provided in the Jorion's book and take those 31 chapters at least two times. Of course, I also take the practice
exams of 2006 to 2008 provided by D.H. website. But as others mentioned, it likes some IQ tests. The question
is long but the answer is short. I believe they are a lot of traps there, and many ones have not to be calculated
by calculator but by hands and simple mind.
Sincerely
chris

 

[chamoloco]

It seems like everyone else has covered much of the exam. I, as others, was tired by the time I covered the lengthy questions on the second part. I found myself speed reading them and not really paying attention. Recommendation, sleep well the days before!

David, your site covered very well the items tested. Thank you very much!! I will most likely have to take the test again next year since I was not able to dedicate the time to studying that I should have, unless they become very generous with the curve

As such, you can add me to your forecasted revenues (hopefully at a discounted price) for next year! I really learned a lot from your presentations. The way you display the material is an advantage, specially the way you combine the tutorials with the study guides, which complement each other.

 

[vkaul1]

What is this ankit if your score is like you expect you will sail through. Why do you guys list the exam performance and bad and not good, I don't understand. After expecting 65% in a tough exam what more do you want.

 

[chih22]

I did view all of the early bird sessions and I looked at most of the spreadsheets and I found them helpful for learning the concepts. I chose BT over Scheweser, eventhough I used Schweser for most of my CFA exam preparation because I liked Dave's early bird screencasts so much and I liked the fact there were spreadsheet examples and I thought BT was more than worth the money for everything I've learned. So, my decision was to go with the actual GARP readings and BT rather than spend $500 on Schweser alone. I did read some of the GARP readings, but there is just too much information there and there are no questions at the end of the chapter like what CFAI would do for their assinged readings. It was just impossible to remember what is relevant based on GARP's readings alone.

So, in hindsight, I do agree that more case studies that were long and lengthy with a lot of distracting data may have better prepared us for the exam, but it is difficult to come up with realistic sample exam questions. I recall from my CFA preparation that I didn't think Scheweser did a good job with sample exam questions as well and the best source for sample exam questions that used the same sort of traps and language were from CFAI themselves and those practice exams were $50 each and I spent $250 on just practice level 3 CFA exams in 2007, but they were extremely helpful and I passed. Thus, I think David did a great job this year, despite not having enough long lengthy questions for us...but my point is that making long case study questions that will be similar to GARP is a very very difficult task and it probably wouldn't be cheap unless we expect Dave to not sleep at night or perhaps sell it as an additional add-on package towards the end of the cirriculum. Personally, I would have liked to see GARP follow CFAI by putting out more (better quality) sample exam questions...one's that weren't just taken from past exams but written in the same year as the actual exam is given. I would have paid at least $100 for 150 good sample exam questions that were more like the questions on the actual 2008 exam...at least, that would have prepared us for all the different flavors of scaling VaR to different time scales and the long 1/2 page and 3/4 page cases. But, in all fairness, they should provide it for free because the readings and the exam fee isn't cheap and there shouldn't be an advantage for those who can afford to purchase the additional questions.

 

[simeonfishman]

I think fundamentally the problem with this year's exam was emphasis focused on deciphering what the question was asking of you, rather than testing knowledge of concepts. Triple negative elimination questions, multiple distracting text and data may serve GARP in making the exam harder to pass, but perhaps not harder in the way they would really prefer...determining high quality risk professional candidates.

I, like another member who posted here, haven't taken an exam since my very distant university days, and felt the exam was more about how good you are at taking exams rather than how well you understood the material. I feel I was pretty diligent in my studies and grasped the concepts well- I started with david's early bird session, I actually read nearly all the core materials, followed through all david's screencasts and most spreadsheets. I still felt I was at a disadvantage in the exam and most likely have failed due to question deciphering/answering strategy.

David's course has been invaluable. The screencasts, study notes and spreadsheets were of most use to me. I did end up also purchasing the Schweser notes. For reasons that follow... this isn't meant to be a critique of Bionic Turtle as I think it is the superior learning product. More in the vain of constructive feedback to help make BT even better:-

BT pros
- The full hour screencasts were great. The study notes were also really good. David excels at breaking down complex ideas into visual concepts that really capture the essence of what you need to know. The Basel II diagrams and the Merton model screencast are just two of the many examples that I think helped tremendously.
- I found the spreadsheets were very helpful in breaking down every step of the calculations. For me, this is how you really understand an approach. Its often those tiny steps that are omitted in the core readings that trip you up, and the spreadsheets help clarify a lot. There were maybe a few too many peripheral sheets where David could save some time and focus on some other areas. There was some areas in the Schweser notes where the math just didn't make sense, David's spreadsheets were spot on. If anyone has both you should look at the Surplus at Risk spreadsheet... it's very clear what's going on here. In Schweser its just plain wrong and misleading.

Room to grow
- Where Schweser helped me was in two places-
1) They were concise in explaining dense textual, non-quantitative material such that you can grasp what you need to know. This was especially the case for Op risk where there was so much regulatory stuff. In my opinion, although BT is great at summarizing more quantitative concepts, for the qualitative areas, David's notes were a little too sparse with not enough background or context to know how to use in the exam.
2) Questions- One thing I did like about Schweser was that each section then summarizes key concepts and then has concept checking questions to get you into the question answering mind-set. This is an approach that I think would benefit BT as it helps tie materials all together and bring us more into a exam ready frame of mind.

I should emphasize again, Bionic Turtle was definitely my primary source of study, and I only purchased Schweser a little under 2 months before the exam. I'm very impressed with David Harper and the quality and sheer hard work he puts into the BT program. I'd love to see it continue to improve (especially seeing as there's a good chance I will be taking the FRM again next year).

Thanks

Simeon

 

[David Harper CFA FRM]

@ John:
In regard to, "The problem with the questions and the spreadsheets and the PDF is the limited time available....To have it more practical and exam based. You can made it more case based. "
- Yes, I agree, I agree, thank you.
"And make some kind of competition between the teams" - Great idea, I think maybe a contest where winner(s) gets refund and/or prize to stimulate participation. Awesome idea!

@Chris:
In regard to "Therefore the language gap does exist between me and tests. But after the exam, I really feel the test is more like a phD qualification instead of a test for those ones working in the company. "
I mentioned this to GARP leadership yesterday; i.e., that the exam needs to take care to avoid a bias against non-native American/English speakers (most of my customers are outside the U.S., some are not native English-speaking and many do not bring recent academic experience). But, since I spend all year giving preparation, my deepest concern is that the preparation be relevant to the exam. BT needs to improve, of course, but the exam also needs to better reflect the assignments and the AIMS which constitute the course provider roadmap (or, what i am doing all year?). And the sample questions needs to get better. Including, incorrect/distracting sample questions need to be weeded out. I am summarizing all feedback in this thread for GARP (they read this thread anyway, so your feedback here is already seen by GARP)....the good news is: I believe they are quite committed to improvements in the exam. I truly believe the FRM is gold standard for risk and will only get better.

@Rowsey:
"hopefully at a discounted price:" of course, absolutely. My goal has always been to bring down the price for the exam prep.

To be candid about pricing, the #1 issue this year, for me, was piracy: we made everything downloadable because many customers requested download (e.g., to view on laptop/riding the train). The "price" of this is they are easier to pirate. This year, acute pain for us because, for example, one pirate sold those (illegal) copies from a major selling platform. He sold many copies before we shut him down (actually, I referred it to Kaplan as I did not have the resources/time to pursue and they were able to shut him down). I even have two visitors to the forum who admit this plainly; culturally, it seems, not paying for digital assets has become almost normative. The problem is, I need to get paid, of course, to provide the service and (just as important) to invest in improvements. Next year, I want to address the generous, wonderful feedback on this forum so I'd like to hire help specifically for (i) more, better questions and (ii) better notes. At the same time, i want to maintain convenience for customers so I want to keep the downloadable aspect. Finally, I want to be the low cost leader (I would argue our super low prices, over the last three years, has driven down prices among the training providers generally). But with low price like ours, it is critical that all users pay. It is not something you asked about , but i thought i'd share this dilemma. It's not a long run problem: in the long run, my analysis of the business model is that it will actually tend toward free (I believe) and sponsors will pay for attention. As the value of customer attention will soon exceed the cost of developing the product.

@chih22:
Great points. About BT: "making long case study questions that will be similar to GARP is a very very difficult task and it probably wouldn’t be cheap unless we expect Dave to not sleep at night or perhaps sell it as an additional add-on package towards the end of the curriculum." You are so correct. The absolutely hardest thing to do, as a provider, is write good quality questions. Easy to write lame questions; very timing consuming to write relevant questions anchored in assignments. And new candidates don't necessarily bring this as a buying criteria. Some providers sell "we have 1,000s of questions" but irrelevant questions can do a disservice. I will be thinking about an add-on as you suggest. This year, on this forum, we discovered that even with GARP's own samples (which are quite imperfect), there are very few high-quality questions in circulation generally.

"[GARP] should provide it for free because the readings and the exam fee isn’t cheap and there shouldn’t be an advantage for those who can afford to purchase the additional questions." I am giving GARP this same feedback; included with registration *should* be a better set of relevant sample questions, at least a starter set.

@Simeon:
Fantastic feedback, thank you for (arftully) highlighting both pros and cons and for taking the time to offer constructive feedback. I sincerely agree with both sides, including Room to Grow. (and, totally agree, we've gone about far enough with *number* of spreadsheets. I think the relevant XLS can be improved but we don't need to add too many more. Need to shift emphasis to the other areas.) I've printed your post and it becomes part of my planning for next year.

Thanks you!
David

 

[Ganesh]

Thank you David for your detailed webcasts...made preprations a bit easier