Fast Brute-Force Optimization Software

sideburn

New Member
Hello Quants!

Which backtesting software produces fast brute-force optimization (run all possible parameter combinations)?

Thanks
 

daleholborow

Active Member
You don't say what you are trying to do. You don't say why you are trying to do it. You don't say anything of use, or of interest. You should really correct that, if you want to get any advice, of any use, at all.

That said, my initial and immediate advice would be "don't rely on brute force". Seriously. Even a casual google search for "typical" optimization problems should convince you that applying even a rudimentary level of smarts (i.e. studied, targeted, relevant techniques) will VASTLY improve your chances of success - where success is defined as "arrive at a practical workable solution".

As a case study, I suggest to you problems involving timetabling, such as might be used for a school to schedule teachers with subjects with groups of students in rooms. A brute force quickly becomes one of those "more paths than there are grains of sand in the universe" situations... a studied, calculated, insightful attempt might make use of the many talented, clever, interested researchers who have looked at this type of problem, and you might then go "oh great.. with a decent practically designed algorithm, and some applied effort, I can solve this in a few minutes/hours".

Don't be lazy. Understand your problem and make use of research. And at least try to make your questions clear to the people you are asking for help.
 

sideburn

New Member
Sorry, I'm new to quant world...I'll try to explain more details:
Sounds like your concern is to avoid brute-force due to over-fitting issue, right? I plan to do series steps of the process. It's called walk-forward optimization or walk-forward analysis as described Walk forward optimization - Wikipedia. First step is to do brute-force optimization NOT with whole data set but let's say 10,000 ticks out of 10 millions ticks (one year worth ticks). Then I would take a note of top 10 to 20 best results and their parameter settings. Second step is to backtest those 10-20 settings on next 2,000 ticks. It will eliminate some "bad ones". Third step is to repeat the cycle but moving forward 2,000 ticks. Until let's say 5 to 10 cycles is complete. Last step is to backtest with whole data set to see if performance still stands. My issue is doing brute force optimization with 10,000 ticks every cycle. Let's say the trading logic is:
Fast moving average with period range from 5 to 100 step 5
Slow moving average with period range from 100 to 200 step 5
RSI with period range from 5 to 100 step 5
That's 7220 possible combinations along with 10,000 ticks which is over 72 million iterations just doing step 1 per cycle.
I tried backtrader engine which uses Python. The backtest (not optimization) is real slow. I tried to find a software that does fast brute force optimization before I can continue the walk forward process. Hope this helps what I'm looking for?
 

daleholborow

Active Member
Ok. So you want a hill climber or gradient descent algo probably, at least to start with. Think about applying jumps to avoid local Maxima, so simulated annealing might be applicable as well.

Now... I start with my trusty friend Google, and type "walk forward parameter optimization" and voila:

Parameter Optimization in Algorithmic Finance Part I: Walk forward optimization - Quantopian Blog

In that the author immediately mentions a couple of those techniques as applying to your problem, and also that he works in python, so your chances of finding libraries you can use are good.

Which brings me to two further tips for People posting to forums: always try searching around yourself first, and also mention what you've found our what difficulties you've had... Because if I can find that in a few seconds, I can't help but think you could have managed the same thing.
 

Daniel Duffy

C++ author, trainer
Think about applying jumps to avoid local Maxima, so simulated annealing might be applicable as well.

Differential Evolution is very robust. Gives global minimum.
 
Last edited:

sideburn

New Member
Thanks for the info...now I’m wondering what do you as a quant normally do to optimize parameters while avoiding curve-fitting issue? Isn’t it a walk-forward optimization? Or a completely different method?
 

TehRaio

Active Member
Think about applying jumps to avoid local Maxima, so simulated annealing might be applicable as well.

Differential Evolution is very robust. Gives global minimum.
yes this.

@sideburn use a genetic algorithm, it's not totally brute force: let's say it's gradient free stochastic optimization. start there Examples — DEAP 1.2.2 documentation

if what you just need is to do grid search over a certain range of input, there is also this but I haven't implemented it myself: GitHub - rsteca/sklearn-deap: Use evolutionary algorithms instead of gridsearch in scikit-learn
 

Daniel Duffy

C++ author, trainer
I think DE is more intuitive than GA.
Did some investigation to apply DE to computing implied volatility f(sig;T,K,r) = 0 by writing as a least squares min f(sig)^2. DE was the most robust of all the methods I tried.

And it is gradient-free.
 
Top