So, for 6 months I was working very hard to create an algo. And then something happened that made me quit...
I began my journey by applying a simple machine learning technique. It gave me great returns. So I go excited!
Later I found out that there was a thing called bid ask. And with it the algo would get shitty results.
Then I had a very interesting and creative idea. I worked hard... I searched for the average bid ask and just to be safe, assumed that all my trades had double that value + some commissions.
I achieved a yearly gain of 1000%! And sometimes even more, consistently. The data was from 2010-2016, so not updated. But that got me really excited. I I was sure I would become a millionaire! I found the secret.
Then I went for more recent data. And downloaded companies from sp500 and other big ones. This time, however, the gain wasn’t so Amazing. Not only that, but I would end up losing money with this algo at some years.
So why suddenly my 10x yearly return machine wasn’t working anymore?
Well, the difference was on the dataset. The 1st dataset had 5k companies! While the other around 1k.
I found out that my algo would select companies with a very low volume. I then found out that the bid ask for those was companies was crazy high, many times above 5%.
I didn’t give up!
I rewrote another huge algo, but this time only sp500 companies! And they must belong to sp500 at that specific time!
More than that, I gathered data from 1995.
I tested my new algo, and now something amazing was happening, I was having crazy gains again!!! Not so crazy as before but around 100-200% yearly. I made the program run from 1995.
And the algo would use all its previous data from that day. And train the machine learning algo for each day. It took a long time...
Anyway, I let it run, feeling confident. But then, when it reach the year 2013, I started just losing money. And it just got worse...
So I thought. Maybe using data from 1995 to train a model in 2013 won’t make sense. Better to just consider that last few days.
This in fact improved the results. I realized that the stock market is not like physics. There are no universal formulas, it is always changing.
So my idea of learning from the previous x days seemed genius. I would always adapt. and it is in fact a good idea that worked better.
Then I tried it in the present times and it didn’t go very well.
But why did it work for the year 200 and not for 2020?
Then it came to me: because the stock market is a competition! And even an algo competition. Back in 2000 the ml techniques were way less advanced. So I was competing with the AI from 20 years ago! That’s not fair. Also, back in the day they didn’t have this amount of data. The market wasn’t as efficient.
I also found out that my algo was kinda good with smallish companies, but bad with huge ones such as Microsoft. The reason: there is more competition. So the market is much more efficient. It is easier to find patterns in smaller companies.
However the bid ask will usually be bigger. So you are kinda fucked. It is very hard to find the edge.
I built another algo. Simpler, no AI this time. It was able to work the best. Yearly gains 60-150% yearly. What was the problem then? Well too have these gains I would have to invest 100% of my money.
I tried with 50% or sharing between 2 stocks, and it was still great. But with 33% it stopped being great. I ran with slight altered parameters and it chose a stock that lost 70% in one day (stamps). And it wasn’t such a small company.
So here I become aware of the low probability risks. And how investing 100% is a very dangerous idea. You just lose everything you had gained for years.
I have to admit that this strategy is actually kinda good. The best I created so far. And could have a bit potential. But would need some refinement.
...
So far I gave many reasons why I would give up. But here’s the one that made me quit: -what works today may become obsolete tomorrow.
It’s a risk you are taking. In the real world not only it may get worse. But you find out that you didn’t account enough for the slippage.
Why would I risk, when I can invest normally and still have 8% gains. While if I do algo trading you won’t get a big difference from the market (probably). The diference is that the algo is probably riskier.
My other problem is how I can compete? There are literally companies that have teams of PhDs doing this stuff. How can I compete? And they have access to data I don’t.
It’s an unfair game. And the risk is too high for me. I prefer the classical way now. Less stress and probably better results.
PS: but if you believe you have a nice strategy do not give up! What didn’t work with me may work with you. This is just my xp.
Also my strategy would be short term no long term.