How to compare models?

[Julianlapoutre]

Hello,
I have a question, i would like to find wich model is the best to price options (imagine BS and heston for example).
I was thinking do it like that :
First use monte carlo simulation to estimate an option price
Then use differents model to price option with same parameters
Then find wich is the best using monte carlo price as benchmark

Thanks for help

 

[aaronhotchner]

It doesn't make sense to use the same parameters in two different models. Black-Scholes provides arbitrage-free prices, so if your parameters describe reality then the Black-Scholes price is guaranteed to be correct. Using the same conditions in the Heston model is either going to give you the same price or a wrong price.

More importantly, the better model is the one that matches market prices better. Using Monte Carlo does not achieve this.

 

[Tianyu Li]

I think simply calculate "MAPE" between market price and each model price is a more standard way.

 

[Julianlapoutre]

I think simply calculate "MAPE" between market price and each model price is a more standard way.

Thanks a lot for your help, what is MAPE ? please ?

It doesn't make sense to use the same parameters in two different models. Black-Scholes provides arbitrage-free prices, so if your parameters describe reality then the Black-Scholes price is guaranteed to be correct. Using the same conditions in the Heston model is either going to give you the same price or a wrong price.

More importantly, the better model is the one that matches market prices better. Using Monte Carlo does not achieve this.

In fact, i don't want to use the same parameters. I mean for BS i'll use historic volatility wherase for HN i'll use GARCH parameters.

So you think that i price an option using BS model (for exemple SP500 option with given maturity and strike, using historical volatility) and then i price the same options using Heston Nandi option using GARCH parameters, then i'll compare with the real price given by the market ?

thanks for help guys and sorry for my english

 

[Tianyu Li]

Thanks a lot for your help, what is MAPE ? please ?



In fact, i don't want to use the same parameters. I mean for BS i'll use historic volatility wherase for HN i'll use GARCH parameters.

So you think that i price an option using BS model (for exemple SP500 option with given maturity and strike, using historical volatility) and then i price the same options using Heston Nandi option using GARCH parameters, then i'll compare with the real price given by the market ?

thanks for help guys and sorry for my english

MAPE stands for mean absolute percentage error.

I believe someone has done the similar thing before, such as
http://web.hku.hk/~jinzhang/finance/Zhang_Shu.pdf

 

[aaronhotchner]

The problem is more complicated than you probably want it to be. Models are not good or bad by themselves. I can take a "great" model and make it really terrible by feeding it inaccurate assumptions. Likewise, I can take an "awful" model and make it incredible by feeding it whatever combination of inputs leads me to the exact market price. If you can't get Black-Scholes to predict future market prices, it can mean one of two things: Black-Scholes isn't good, or you gave it bad parameters. You can't tell which is the truth.

 

[Sanket Patel]

Hello,
I have a question, i would like to find wich model is the best to price options (imagine BS and heston for example).
I was thinking do it like that :
First use monte carlo simulation to estimate an option price
Then use differents model to price option with same parameters
Then find wich is the best using monte carlo price as benchmark

Thanks for help

Look at the theory underlying the models.

Taking BS, for example, calls and puts can be replicated through a combination of bond and stock. Create a portfolio which is priced with the 'model' and a second 'replicating' portfolio. You can test the performance of the model by tracking the difference between the model-priced portfolio and the replicating portfolio.

This is just the beginning. You then have to put the model through a whole range of scenarios - boundary conditions, low vol, high vol, zero vol, upward sloping yield curve, inverted yield curve, negative rates, etc.

 

[Julianlapoutre]

MAPE stands for mean absolute percentage error.

I believe someone has done the similar thing before, such as
http://web.hku.hk/~jinzhang/finance/Zhang_Shu.pdf

Thanks i'll look this paper

The problem is more complicated than you probably want it to be. Models are not good or bad by themselves. I can take a "great" model and make it really terrible by feeding it inaccurate assumptions. Likewise, I can take an "awful" model and make it incredible by feeding it whatever combination of inputs leads me to the exact market price. If you can't get Black-Scholes to predict future market prices, it can mean one of two things: Black-Scholes isn't good, or you gave it bad parameters. You can't tell which is the truth.

Look at the theory underlying the models.

Taking BS, for example, calls and puts can be replicated through a combination of bond and stock. Create a portfolio which is priced with the 'model' and a second 'replicating' portfolio. You can test the performance of the model by tracking the difference between the model-priced portfolio and the replicating portfolio.

This is just the beginning. You then have to put the model through a whole range of scenarios - boundary conditions, low vol, high vol, zero vol, upward sloping yield curve, inverted yield curve, negative rates, etc.

Yes i understand that i have to calibrate the model, but BS model only need variance wich is not direcly observable on market isn't it ?
So if i use Heston-nandi Garch (1,1) model i'll find 3 Parameters (to calibrate the Garch Model) then i'll will find an option price, and this price will be the "fair" price for this model.
I understand that their is a problem for BS as i can't really find the variance parameter but i can find implicit variance from the market data, and then try to see wich of BS or HN model will have the lower tracking error ?

For example, i wanna work with SP500 option price Maturity 3 month and strike 1700 :
1 Step) I'll looking for this options price on market
2 Step) I use Heston-Nandi model on SP500 spot to evaluate GARCH parameter and then i can price the option using HN model
3 Step) I calculate implicit volatility using 1 Step and i "put it" in BS model to estimate BS price
4 Step) With the HN-Price and BS-Price i can determinate wich is closer with Market price

I'm sure this is not the good way ... so if you could help me

Another question : i have a doubt about BS model assumption : Does asset return should be uncorrelated ? And does this return are independent ?

thanks for help guys

 

[Julianlapoutre]

HI guys

So i have collected data for my option : 3 strike and 10 maturity for each

And i have priced my HN-option using :
Option Pricing Models and Volatility Using Excel-VBA (Wiley Finance)
So i have tried to find the BS implied vol of market data and i see a smile then i have tried to find the BS implied vol of simulated call price (simulated using HN model) and i have found that IV looks like to be almost constant.

So i think that mean that the HN model is good ? but the problem is that the level of the IV is different for my two model. i mean that for market data the IV starts at 0,19 whereas for simulated date IV starts (and almost stay) at 0.24. Is there a problem with my HN parameter ?

Now i would like to find IV on market data using BS model and HN model and compare the Smile, so i hope to compare which is the best model
what do you think about thanks.

 

[aaronhotchner]

The implied vol of simulated options will always be constant if you simulate with constant vol...

It never should be if you simulate with jump-diffusion, for example.

 

[Julianlapoutre]

I have estimate with a Garch model so i havent estimated with a constant vol ...

 

[aaronhotchner]

If your Black-Scholes implied vols are near-constant, you're either not looking across a large enough range (too close to at the money) or your simulations have near-constant vol. Black-Scholes doesn't make mistakes, and it should generally not exhibit constant vol for market prices.

 

[Julianlapoutre]

I'm working on very deep/out money and ATM call.
Once told me that it's near-constant because HN model capture the stochastic volatility and that's why when i use BS on HN-simulated price i found a near-constant variance. He told me that is because smile is a problem of BS model and that HN model allow to avoid smile. So i you can tell why it's false cause i'm lost thanks for help

 

[aaronhotchner]

An assumption of Black-Scholes is constant vol, so if you are pricing options for underlying securities that have stochastic vol, you should see the volatility smile. If you're not seeing it, then your simulation doesn't have (enough) stochastic vol. Take some sample paths of the simulation and check out the vol for them, and see if they're the same.

 

[Julianlapoutre]

Ok thanks a lot for yout help i have understood
so their is a problem with my model
i have used "Option Pricing Models and Volatility Using Excel-VBA" to estimate parameter of Heston Nandi model.

So in this book authors use Nelder-Mead algorithm to maximise log likelihood function.
They work on SP500 and use as starting value those of table 1 in HN's paper. But i'm working on Cac40 data and i have found that estimated parameters are too dependent of starting value. I think that the algorithme find local maximum. So i don't know how to deal with it, if someone have an idea. (i don't know how to avoid maximum local and i'm not practionner with algorithm)

Thanks a lot for ur help