By Dr. Lipa Roitman Co-founder and Chief Scientist I Know First  and Yaron Golgher CEO & Co-Founder  I Know First

The majority of people are incredulous about the prospect of using AI to correctly forecast the stock market. They believe the market is too random and therefore, unpredictable. However, the mistake these people make is that they fail to understand that the market is not simply a series of indiscriminate/ arbitrary actions, but rather the combination of all of these individual events form a chaotic system.

So what exactly is a chaotic system?

Chaotic systems are dynamical systems that are highly sensitive to initial conditions and arise from a multitude of reasons. One of the primary ways that market acts chaotically is because (despite one of the most basic economic assumptions) people do NOT always act rationally because of psychological concepts like risk aversion in which losses have a greater emotional impact than gains, trend lovers who follow the herd, news bias, and overreaction. Moreover,  chaos is the result of the complexity of the market itself due to a myriad of different valuation models being constantly used, conflict between the short and long term, and more.

Chaotic systems are classified by a few distinct features. First, persistent trends can be identified thanks to the stability and memory of systems. This means that happens in the future is at least partly dependent on what has already happened. Additionally, paradigm changes or ‘phase transitions’ are prevalent in chaotic systems. This means that the accumulation of small random imbalance result eventually in a large and seemingly unprecedented change in the market as a whole. Finally, there is a relatively high likelihood of a black swan event, or wide swing outlier; a fat-tail distribution is a more appropriate way to statistically represent the stock market than a normal Gaussian distribution.

So the market behaves chaotically, but how can we quantify and account for this chaos?

The market constantly has positive and negative feedbacks. These opposing feedbacks keep the market in balance and can create irregular cycles known as bubbles. For example, an optimism about innovation or some other positive event can lead to an energy and a positive bubble. Alternatively, if pessimism about the market prevails, a negative bubble forms. Most of the time the damping effect of feedback pops the bubble and brings the market back to consensus.

In order to analyze these bubbles we use a combination of classical Fourier analysis for regular cycles and wavelet analysis for non-stationary chaotic market cycles. In wavelet analysis we find  three types of noise or volatility: random white, brownian, and frequency dependent noise. Random white noise has no correlation in time, Brownian noise has no correlation between increments, and frequency dependent (fractal) noise is the result of random shocks to the system. Using the Hurst exponent H it then becomes possible to find the trends by distinguishing between predictable fractal noise and random unpredictable time series.

H is extremely significant in identifying where in the cycle the chaotic system is.

  • If H< ½, then there is a negative feedback with high noise.
  • If H=½ , then it is an unpredictable Brownian motion.
  • If ½ <H< 1, then there is a positive feedback with less noise that is frequency dependent.

Thus it becomes possible to identify the current trends in the stock market. Artificial intelligence can use all of this information as well as other inputs such as asset prices, trading volumes, market indices, exchange rates, interest rates, and more in order to create a holistic view of the market and the ways in which assets interact with one another.

I Know First uses these methods in order to create daily stock forecasts that not only include the predicted direction of the asset, but also a predictability indicator. Moreover, I Know First provides predictions across a multitude of time horizons from 1 day to 1 year in order to resolve the conflict between the short term and long term outlooks. The company uses machine learning strategies to find order in the chaos and build the relationships between assets. Then, I Know First uses this information to identify the best investment opportunities that could have otherwise gone unnoticed. Additionally, I Know First can also organize the chaotic system of the stock market into categories and offers its subscribers different stock packages that include predictions for industries, types of companies, countries, and more. Implementing I Know First predictions into a portfolio as a trading support tool leads to consistent outperformance of the market as a whole.