Clarifying Understanding for Ch 4 - Backtesting VAR

[frm_prep]

Hi,

I have the following understanding - Does this make sense or am I missing something here?

We may choose to accept a 99% VAR model with 95% or 99% (or any other) level of confidence. Hence, using Jorian's example from book, assuming we use a 99% VAR (i.e. p=.01), over 250 days (i.e. T=250). Expected value of exceptions = pT = 2.5. Assume we encountered 8 exceptions. Using Binomial, and as mentioned in the video tutorial by David, the probability of (x=8) will only be 0.3% and cummulative probability of x>8 is only 0.4%. Hence, we can reject this model with 99.6% confidence.

However, on the same example, using normal approximation to binomial (I know we shouldn't use it coz T is only 250 here, and pT = 2.5 which is much lesser than 10, but just for understanding sake).

calculated z score = x-pt/root(p(1-p)T) = 2.23. This is greater than 2 tailed critical z value of 1.96 (for 95% confidence) but smaller than 2 tailed critical z value of 2.6 (for 99% confidence). So, we reject the model at 95% confidence, but we cannot reject with 99% confidence.

Is this correct?

Note: I haven't yet attempted practice questions - so in case the answer is already there, please pardon.

 

[David Harper CFA FRM]

Hi @frm_prep Yes, I agree with one difference: isn't your Z of 2.23 using the binomial variance rather than binomial standard deviation? Using the normal approximation (for the accurate binomial), I get (8-2.5)/sqrt(99%*1%*250) = 3.50, and would therefore reject also at the 2T 99.0% confidence. Put another way, I get a 2T p-value of (1-NORM.S.DIST(D27,TRUE))*2 = 0.047%. Otherwise, your logic looks to be spot-on. Below is a table I just added to the upcoming revision to Jorion backtest notes. The point is simply to show that the two cases that do happen to pass the guideline test, indeed to have binomial models with much less heavy tails. Thanks!

 

[frm_prep]

Ah got it ! Thanks David. I am gonna make that mistake of forgetting the sq root a few more times - in the practice questions. But eventually will remember, hopefully before the exam