Backtesting

[delalma]

I post for the first time on Bionic Turtle. I never tried to pass the FRM exam but I have passed the CFA successfully. I hope to find good advice from experienced market risk analyst here. Thank you in advance.

My current issue is to process a backtest on the delta normal VaR. I know that the longer the test period the better it is. However, every day of test need the company portfolio positions. If i want to test the model for 1 year I need 1 year of daily portfolio positions. The issue is that It is not possible in my case as there are some data overwriting itself. This means that it is not possible to get past data without storing those data every day.

Actually, I calculate directly a company wide VaR based on the inventory positions. Calculating the inventory positions can be done even on historical data. But when I modify the data to reflect the real risk the company is taking (eg. the net CFD position is considered to be excluded from the risk attributable to the company), those overwriting items prevent to get historical positions.

So the question is if I can validate the model on the inventory positions, but when the model will be implemented, use the risk position.
It is probably not totally clear for the reader, but I can answer any question, if somebody want to help me.

Best,
delalma

 

[Nicole Seaman]

I post for the first time on Bionic Turtle. I never tried to pass the FRM exam but I have passed the CFA successfully. I hope to find good advice from experienced market risk analyst here. Thank you in advance.

My current issue is to process a backtest on the delta normal VaR. I know that the longer the test period the better it is. However, every day of test need the company portfolio positions. If i want to test the model for 1 year I need 1 year of daily portfolio positions. The issue is that It is not possible in my case as there are some data overwriting itself. This means that it is not possible to get past data without storing those data every day.

Actually, I calculate directly a company wide VaR based on the inventory positions. Calculating the inventory positions can be done even on historical data. But when I modify the data to reflect the real risk the company is taking (eg. the net CFD position is considered to be excluded from the risk attributable to the company), those overwriting items prevent to get historical positions.

So the question is if I can validate the model on the inventory positions, but when the model will be implemented, use the risk position.
It is probably not totally clear for the reader, but I can answer any question, if somebody want to help me.

Best,
delalma

Hello @delalma

I just wanted to point out that we have a search function in the forum that will likely be helpful in answering your questions. The search function is here: https://forum.bionicturtle.com/search/?type=post. If you type in backtest, there will be many threads that come up with a great deal of discussion about backtesting VaR. Specifically, here is what comes up when I search: https://forum.bionicturtle.com/search/105895/?q=backtest&t=post&o=relevance .

Bionic Turtle has over a decade of forum discussions so it is very likely that your questions have been answered somewhere in the forum. Using the search function can prevent you from having to wait for someone to answer your question, especially now right before the exam when the forum is extremely busy and it becomes impossible to answer all of the questions that are asked. I hope this is helpful!

Nicole

 

[ami44]

Hi delalma,

if I understand you correctly you need the backtest to validate your risk model and not for regulatory reporting, right?

I think the answer depends on the modifications that you make to get from inventory position to the risk position.

If these modifications are not part of your model that you want to validate and the risk position is not significantly different in it‘s risk than the inventory position, than you are ok. You are supposed to backtest your model on a realistic portfolio and thats what you are doing. In my opinion, that portfolio does not need to be exactly your historical risk position.

A backtest compares the VaR with P&L. Do you take out the CFDs only from VaR, or also from the P&L? In the first case your modifications might be part of your model, in the second case you are just slightly changing the portfolio for which you calculate the VaR.

But this is all speculation by me. A lot depends on the exact circumstances.

Was that helpful?

 

[delalma]

Hi delalma,

if I understand you correctly you need the backtest to validate your risk model and not for regulatory reporting, right?

I think the answer depends on the modifications that you make to get from inventory position to the risk position.

If these modifications are not part of your model that you want to validate and the risk position is not significantly different in it‘s risk than the inventory position, than you are ok. You are supposed to backtest your model on a realistic portfolio and thats what you are doing. In my opinion, that portfolio does not need to be exactly your historical risk position.

A backtest compares the VaR with P&L. Do you take out the CFDs only from VaR, or also from the P&L? In the first case your modifications might be part of your model, in the second case you are just slightly changing the portfolio for which you calculate the VaR.

But this is all speculation by me. A lot depends on the exact circumstances.

Was that helpful?

Hi ami44

I need to thank you for your answer. It is really challenging to work when data are not there. Actually, I was hoping to validate the model both for validating my model and regulatory purposes. How do you think the model should differ depending on the target?

 

[delalma]

Hello @delalma

I just wanted to point out that we have a search function in the forum that will likely be helpful in answering your questions. The search function is here: https://forum.bionicturtle.com/search/?type=post. If you type in backtest, there will be many threads that come up with a great deal of discussion about backtesting VaR. Specifically, here is what comes up when I search: https://forum.bionicturtle.com/search/105895/?q=backtest&t=post&o=relevance .

Bionic Turtle has over a decade of forum discussions so it is very likely that your questions have been answered somewhere in the forum. Using the search function can prevent you from having to wait for someone to answer your question, especially now right before the exam when the forum is extremely busy and it becomes impossible to answer all of the questions that are asked. I hope this is helpful!

Nicole

Hi Nicole

Thank you for your answer. You are right I did not look through the other threads. I will. Thanks.

 

[ami44]

Actually, I was hoping to validate the model both for validating my model and regulatory purposes.

For regulatory purposes you have to backtest on your actually portfolio. You have to show, that your modifications do not make a difference, which is probably not possible.
But that is just a problem for the start, right? In production you can backtest your model every day and store the result, i.e. you can compare the P&L from yesterday to today with the VaR from yesterday and determine if there was a breach. No need to look back further into the past.

How do you think the model should differ depending on the target?

I’m not sure what you mean by that?

 

[delalma]

For regulatory purposes you have to backtest on your actually portfolio. You have to show, that your modifications do not make a difference, which is probably not possible.
But that is just a problem for the start, right? In production you can backtest your model every day and store the result, i.e. you can compare the P&L from yesterday to today with the VaR from yesterday and determine if there was a breach. No need to look back further into the past.


I’m not sure what you mean by that?

Thank you. That was the answer I was expecting.