News Sentiment Data Approaching A Key Inflection Point?
- Natural Language Processing is used by many companies including Google to understand language.
I first started with sentiment data back in 2005, and I believe their initial adoption to be back in 2000. At Dow Jones we teamed up with Ravenpack to mine the DJ news feed, at about the same time that Reuters was launching a similar product having purchased Corpora.
What we were interested in was changing news into useful data that could be crunched by our data thirsty clients. The Ravenpack systems would study each and every story from Dow Jones ascertaining which asset classes, people, entities were mentioned in a story, and measure the sentiment in the story towards those targets. I am simplifying of course, as Ravenpack did a lot more than just that.
Natural Language Processing
We used a technology called Natural Language Processing, which is used by many companies including Google to understand language. For many years we worked with Hedge Funds and prop desks around the world. They would consume the data, building trading and risk models and integrating their ideas with other data sets. Out would come a signal.
Fast forward some years and there is still a large core group of hedge funds that are using this data in ever more complex ways. However, usage is expanding into different areas; we are finding regulators using data to uncover market abuse and every day traders incorporating news signals into their trading, especially in the USA where there are plenty of subscription websites emerging. Of greater interest was when Reuters earlier this year stopped selling directly to Hedge Funds, and allowed their sentiment data to be freely accessed over their Reuters Eikon platform, expanding the usage overnight to 300,000 plus traders.
Area of Expansion
In Europe we are seeing this trend emerge in a different model. Where users expect their brokers to offer services free of charge, brokers are subscribing to sentiment data to offer to their clients. I believe this area will expand quickly as brokers find ever more individual ways of serving up the data to their clients. Here are a few models that I have heard recently:
- Curating twitter to find interesting tweets on the subject a trader is looking, and measuring the sentiment in that tweet. Essentially creating a curated news feed from Twitter.
- Dashboards for sentiment, brokers are offering visualisations of the mood of the market
Soon I believe we will see the first Expert advisors built from sentiment data, and we may well see some brokers build sentiment into their Risk model; the possibilities are gradually being discovered.
So what we are seeing, is a once niche data set being democratised, and creating a launch pad for further data sets to be introduced into the trading world. The day of Big Data Big Data Big data refers to the collection of data that is too complex and too large for processing by standard database tools. There is no specific quantity of data, which is set as a minimum level to be considered Big data. Image the data collected on global credit card transactions. Many governments used Big data analysis to study the recent pandemic spread. The term Big data was first introduced in 1980 by Charles Tilly.The term Big data was primarily used in computer science, statistics, and econome Big data refers to the collection of data that is too complex and too large for processing by standard database tools. There is no specific quantity of data, which is set as a minimum level to be considered Big data. Image the data collected on global credit card transactions. Many governments used Big data analysis to study the recent pandemic spread. The term Big data was first introduced in 1980 by Charles Tilly.The term Big data was primarily used in computer science, statistics, and econome Read this Term trading is upon us, now.
I first started with sentiment data back in 2005, and I believe their initial adoption to be back in 2000. At Dow Jones we teamed up with Ravenpack to mine the DJ news feed, at about the same time that Reuters was launching a similar product having purchased Corpora.
What we were interested in was changing news into useful data that could be crunched by our data thirsty clients. The Ravenpack systems would study each and every story from Dow Jones ascertaining which asset classes, people, entities were mentioned in a story, and measure the sentiment in the story towards those targets. I am simplifying of course, as Ravenpack did a lot more than just that.
Natural Language Processing
We used a technology called Natural Language Processing, which is used by many companies including Google to understand language. For many years we worked with Hedge Funds and prop desks around the world. They would consume the data, building trading and risk models and integrating their ideas with other data sets. Out would come a signal.
Fast forward some years and there is still a large core group of hedge funds that are using this data in ever more complex ways. However, usage is expanding into different areas; we are finding regulators using data to uncover market abuse and every day traders incorporating news signals into their trading, especially in the USA where there are plenty of subscription websites emerging. Of greater interest was when Reuters earlier this year stopped selling directly to Hedge Funds, and allowed their sentiment data to be freely accessed over their Reuters Eikon platform, expanding the usage overnight to 300,000 plus traders.
Area of Expansion
In Europe we are seeing this trend emerge in a different model. Where users expect their brokers to offer services free of charge, brokers are subscribing to sentiment data to offer to their clients. I believe this area will expand quickly as brokers find ever more individual ways of serving up the data to their clients. Here are a few models that I have heard recently:
- Curating twitter to find interesting tweets on the subject a trader is looking, and measuring the sentiment in that tweet. Essentially creating a curated news feed from Twitter.
- Dashboards for sentiment, brokers are offering visualisations of the mood of the market
Soon I believe we will see the first Expert advisors built from sentiment data, and we may well see some brokers build sentiment into their Risk model; the possibilities are gradually being discovered.
So what we are seeing, is a once niche data set being democratised, and creating a launch pad for further data sets to be introduced into the trading world. The day of Big Data Big Data Big data refers to the collection of data that is too complex and too large for processing by standard database tools. There is no specific quantity of data, which is set as a minimum level to be considered Big data. Image the data collected on global credit card transactions. Many governments used Big data analysis to study the recent pandemic spread. The term Big data was first introduced in 1980 by Charles Tilly.The term Big data was primarily used in computer science, statistics, and econome Big data refers to the collection of data that is too complex and too large for processing by standard database tools. There is no specific quantity of data, which is set as a minimum level to be considered Big data. Image the data collected on global credit card transactions. Many governments used Big data analysis to study the recent pandemic spread. The term Big data was first introduced in 1980 by Charles Tilly.The term Big data was primarily used in computer science, statistics, and econome Read this Term trading is upon us, now.