Incorrect calibration & Data problems
- Thread starter ajsa
- Start date
HI asja, Dowd connotes calibration with parameters. So, in regard to GARCH(1,1) for est volatility, a calibration mistake would be fitting the wrong alpha/beta/omega weights (parameters) to arrive at the model...then having the model, data problems would be mistakes in giving the model a flawed input (mistakenly calculating prior variance)...or, another examlpe, in Basel II standardized OpRisk, each business line's gross income is multiplied by a beta factor (12%, 15%, 18%) ....pretty rough way to measure (calibrate) oprisk, so likely calibration error, then data problems are related to our inputs into that model.
David
David
Sure ajsa ... In Dowd, i think he parses it differently: incorrect model spec would refer to the choice of model. e.g.,
for estimating volatilty:
* model spec: I pick GARCH to estimate volatility (model spec) versus AGARCH or someother GARCH
*calibration: I use MLE to get GARCH parameters
* data: I use a historical dataset to input into my calibrated GARCH model
or for estimating a tail loss:
* model spec: I select an POT EVT model to estimate VaR and ES
* calibration: I get the two parameters in POT (scale and tail)
* data mistake: I feed it bogus inputs (% threshold)
hope that helps...David
for estimating volatilty:
* model spec: I pick GARCH to estimate volatility (model spec) versus AGARCH or someother GARCH
*calibration: I use MLE to get GARCH parameters
* data: I use a historical dataset to input into my calibrated GARCH model
or for estimating a tail loss:
* model spec: I select an POT EVT model to estimate VaR and ES
* calibration: I get the two parameters in POT (scale and tail)
* data mistake: I feed it bogus inputs (% threshold)
hope that helps...David
...or (these are interesting distictions, i think), if i try it on Jorion's VaR mapping:
1. Model spec: have i used a good mapping model, principal, duration or cash flow? ... here in VaR mapping, the easy "model spec" mistake would be to omit a key risk factor (e.g., liqudity) from the model (since mapping is about reducing to primatives)
2. Calibration: what VaR confidence to use (I think this would qualify?) or maybe even my choices in regard to stressing the the risk factors
3. data: inputs
David
1. Model spec: have i used a good mapping model, principal, duration or cash flow? ... here in VaR mapping, the easy "model spec" mistake would be to omit a key risk factor (e.g., liqudity) from the model (since mapping is about reducing to primatives)
2. Calibration: what VaR confidence to use (I think this would qualify?) or maybe even my choices in regard to stressing the the risk factors
3. data: inputs
David
Hi David,
"* model spec: I select an POT EVT model to estimate VaR and ES "
I wonder if this kind of mistake should belong to the "incorrect model application" category ("wrong model")?
BTW I am also confused in that one example of "incorrect model application" is "insufficient # of trial in Monte Carlo simulation" while one example of "incorrect calibration" is "inappropriate sample period". Isn't the distinction too subtle?
Thanks.
"* model spec: I select an POT EVT model to estimate VaR and ES "
I wonder if this kind of mistake should belong to the "incorrect model application" category ("wrong model")?
BTW I am also confused in that one example of "incorrect model application" is "insufficient # of trial in Monte Carlo simulation" while one example of "incorrect calibration" is "inappropriate sample period". Isn't the distinction too subtle?
Thanks.
Hi asja,
You may be right, I may be mistaken about that. I struggle with the Dowd's typology and especially the distinction between specification and application. Just intuitively, based on my own usage, a "hierarchy" feels something like:
* Select a type of model
* Specify the model (e.g., variables and parameters)
* Calibrate the model
...Dowd's "hierarchy" implies to me, perhaps mistakenly, that specification (16.2.1) preceeds application (.2); i.e., "select the model" then "specify" the model (his application = my specification)...in his 16.2.1, the comparison of VaR methods implies selection... but I am confused by "wrong model" you quote because he starts with "a good model is incorrectly applied" ... so my own experience is not this nuanced. 1. you have lots of models to choose from, then 2. you operate (specify) on the model; but that frankly leads to the lack of a crisp distinction between application/implementation.
...it occurs to me the simple order drives my confusion; if you are right, his order isn't natural.
...and then, i agree with you the distinctions may be too fine; e.g., POT is a sub-class of EVT, so is selecting POT a choice of models or an implementation/specification within EVT?
(re Monte Carlo, as the literature on MCS is very well-developed, I bet there is a decisive answer to the categorization of "insufficient trials" vis a vis models. I just don't know it but if i find it i will copy back)
Sorry i am not more help, I appreciate your deep dive on this area (which after all, is suddenly a hot topic), I look forward to learning more about this myself
Thanks, David
You may be right, I may be mistaken about that. I struggle with the Dowd's typology and especially the distinction between specification and application. Just intuitively, based on my own usage, a "hierarchy" feels something like:
* Select a type of model
* Specify the model (e.g., variables and parameters)
* Calibrate the model
...Dowd's "hierarchy" implies to me, perhaps mistakenly, that specification (16.2.1) preceeds application (.2); i.e., "select the model" then "specify" the model (his application = my specification)...in his 16.2.1, the comparison of VaR methods implies selection... but I am confused by "wrong model" you quote because he starts with "a good model is incorrectly applied" ... so my own experience is not this nuanced. 1. you have lots of models to choose from, then 2. you operate (specify) on the model; but that frankly leads to the lack of a crisp distinction between application/implementation.
...it occurs to me the simple order drives my confusion; if you are right, his order isn't natural.
...and then, i agree with you the distinctions may be too fine; e.g., POT is a sub-class of EVT, so is selecting POT a choice of models or an implementation/specification within EVT?
(re Monte Carlo, as the literature on MCS is very well-developed, I bet there is a decisive answer to the categorization of "insufficient trials" vis a vis models. I just don't know it but if i find it i will copy back)
Sorry i am not more help, I appreciate your deep dive on this area (which after all, is suddenly a hot topic), I look forward to learning more about this myself
Thanks, David
...i was just looking at Gujarati because I was thinking that econometrics really has something of a head start on model risk, to understate matters
of course, it's grounded in regressions, but he sort of has everything under the "specification" umbrella
e.g., specification as big bucket
(note Dowd's 16.2.3 Implementation even includes "specification" which makes Dowd's implementation arguably overlapping with specification)...David
of course, it's grounded in regressions, but he sort of has everything under the "specification" umbrella
e.g., specification as big bucket
(note Dowd's 16.2.3 Implementation even includes "specification" which makes Dowd's implementation arguably overlapping with specification)...David
Similar threads
