I first started with sentiment data back in 2005, and I believe their initial adoption to be back in 2000. At Dow Jones we teamed up with Ravenpack to mine the DJ news feed, at about the same time that Reuters was launching a similar product having purchased Corpora.

What we were interested in was changing news into useful data that could be crunched by our data thirsty clients. The Ravenpack systems would study each and every story from Dow Jones ascertaining which asset classes, people, entities were mentioned in a story, and measure the sentiment in the story towards those targets. I am simplifying of course, as Ravenpack did a lot more than just that.

Join the iFX EXPO Asia and discover your gateway to the Asian Markets

Natural Language Processing

We used a technology called Natural Language Processing, which is used by many companies including Google to understand language. For many years we worked with Hedge Funds and prop desks around the world. They would consume the data, building trading and risk models and integrating their ideas with other data sets. Out would come a signal.

Fast forward some years and there is still a large core group of hedge funds that are using this data in ever more complex ways. However, usage is expanding into different areas; we are finding regulators using data to uncover market abuse and every day traders incorporating news signals into their trading, especially in the USA where there are plenty of subscription websites emerging. Of greater interest was when Reuters earlier this year stopped selling directly to Hedge Funds, and allowed their sentiment data to be freely accessed over their Reuters Eikon platform, expanding the usage overnight to 300,000 plus traders.

Area of Expansion

In Europe we are seeing this trend emerge in a different model. Where users expect their brokers to offer services free of charge, brokers are subscribing to sentiment data to offer to their clients. I believe this area will expand quickly as brokers find ever more individual ways of serving up the data to their clients. Here are a few models that I have heard recently:

1. Curating twitter to find interesting tweets on the subject a trader is looking, and measuring the sentiment in that tweet. Essentially creating a curated news feed from Twitter.
2. Dashboards for sentiment, brokers are offering visualisations of the mood of the market

Soon I believe we will see the first Expert advisors built from sentiment data, and we may well see some brokers build sentiment into their Risk model; the possibilities are gradually being discovered.

So what we are seeing, is a once niche data set being democratised, and creating a launch pad for further data sets to be introduced into the trading world. The day of Big Data trading is upon us, now.