How to Use Twitter to Guide Your Investments

@MikeSchultz: Hot tip – invest in ZRA

@WholeWorld: RT @MikeSchultz Hot tip – invest in ZRA

OK, it’s not quite that easy (and anyway, you don’t need a tweet to know that Zirra’s worth investing in). But more and more investors are looking to Twitter as a crucial source of information, or even as the primary driver for their investment strategy.

Things started getting excited for Twitter-based investing when Professor Johan Bollen, of Indiana University, found that you could look at the whole set of tweets, on any and all subjects, use sentiment analysis to find out the worldwide mood, and predict what would happen to the Dow Jones Industrial Average 3-4 days later, with 87% accuracy. More specifically, whether or not the worldwide mood, as expressed in its tweets, was “calm,” predicted the direction of the DJIA 87% of the time, which is very impressive.

(Now, to be fair, he didn’t really look at the whole world of Twitter, just the English language tweets. But technology has improved – for example, since 2013 Lexalytics offers sentiment analysis in Mandarin – so that problem can be resolved.)

Additional studies confirmed his findings, even using simpler versions of sentiment analysis, also adding “happiness” as an additional mood that predicted the direction of the stock market, and confirming that it doesn’t work for same-day predictions, only with a lag of a few days.

Buoyed by the good news that Twitter could make you rich, Derwent Capital Markets launched its “Absolute Return” fund, a hedge fund entirely driven by Bollen’s suggested approach. This would’ve been a really exciting experiment to watch, and it beat the market in its first month. But then sadly, the “Twitter Fund,” as it was known, abruptly closed, because it had almost no investors. It turned out that almost no investors were comfortable trusting the mood of the masses. It doesn’t mean it was a bad strategy, it just means that this kind of thinking, finding – and trusting – investment information gleaned at a far remove from “the experts” wasn’t something investors were comfortable with.

Times change, though, and within a few years the number of companies trading based on Twitter data, though still small, is growing. The quality of sentiment analysis continues to improve, and more importantly, investor conservatism is giving way to growing acknowledgement that the world as a whole knows more than the experts, and Twitter is a good way of accessing that collective sense.

There are a variety of approaches out there. One idea is to use data analysis or sentiment analysis in order to gain key information about impending changes. If you know from Twitter that something’s about to change, even if you just know it a couple minutes before the rest of the world, that can be huge. You can look just at the tweets of financial insiders, now included in Bloomberg’s delivery service, to gain insight into how things are shifting in a particular market a few minutes before it actually happens. Or you can look at the tweets of the entire world to identify on-the-ground breaking news that, so long as you correctly interpret the implications, tells you to buy or sell, right now. That’s the service offered by companies like Dataminr – breaking news that clearly will affect the value of specific companies or specific markets, you just need to act on it fast.

Another approach is to look just at tweets that are tech or stock related. One study found that without even looking which stocks are being talked about, you can know whether the market will be going up 90 minutes later. Turns out there are certain keywords that bode well for the market. Wait until those key words are buzzing on Twitter and you know it’s time to buy.

Finally, how about using Twitter sentiment analysis in such a simple, obvious way it almost feels old-fashioned? Analyze all the Twitter feeds referencing stocks, and see which stocks are being tweeted about most favorably.

That’s what Market Prophit is now doing. Since May of this year, they operate a Social Media Sentiment Index. Looking at the S&P 500, they choose 25 companies to include in the index, based on Twitter sentiment analysis as well as the sheer volume of tweets to see who’s being talked about most.

In a certain sense, what they’re doing, to aggregate the crowd’s thinking about these stocks, seems very similar to what the stock market itself does. But maybe the difference is that in the market itself, buyers’ and sellers’ choices are dependent on what they think other people think of those stocks – i.e., how high will others buy it from them, or how low will others sell it back to them. Whereas on Twitter, you’re closer to getting people’s actual feelings about the stocks.

Weirdly, at the same time as offering their index, so clearly based on the wisdom of crowds, Market Prophit also has a service trying to identify those financial bloggers who are the best of the best, the so-called Market Prophits. The theory of collective wisdom shows that the crowd, as a whole, consistently does better than the so-called experts, who do well for a while and then don’t do so well for a while. People’s comfort level with trusting the collective wisdom is growing, but there’s still a lot of ambivalence and a lot of holding onto the old “find the expert” way of thinking, and here you can apparently see that even within the same company.

No need to find the prophet – we’re all prophets, together.

Crowd Mediation on the Local and Geopolitical Scale

How far can you really keep going with this business of crowd wisdom? In this series of posts on crowd-based dispute resolution (in crowdfunding; adjudicating civil disputes; and in virtual worlds), the least likely area for the crowd to be able to contribute is in mediation. After all, the mediator’s job includes being a great listener to what’s really bothering each side, and what they actually care about most, and that way he can come up with some creative suggestions to resolve their dispute. And active, empathetic listening isn’t exactly the behavior that comes to mind when describing the crowd.

Just to get our background straight: what is mediation? Two parties are having an argument, any kind of argument, and they want to avoid having to ratchet up their fight, like by going to court or going to war. They agree to first try to resolve their differences, but the gap between their positions, or the personal antipathy, is so great that they can’t do it themselves. So they bring in a third party, the mediator, whose job isn’t to say who’s right or what the penalty should be but rather to help them negotiate. In effect, to help them talk to each other and hear each other.

So what possible role can there be for the crowd?

For one thing, creativity. A key part in the job of the mediator is to help suggest new solutions, based on what he’s heard, that the parties involved might not have thought of but that they can live with. Diverse ways of thinking is one of the greatest benefits of the crowd, so if you can find a way of canvassing a crowd to get its suggestions (or if the issue is already being talked about on social media, then you just need to analyze what’s being said), you could bring out a wide range of possibilities a single mediator never would have thought of.

Let’s make this concrete. In e-commerce disputes, companies like Youstice, Modria, and until recently Square Trade can provide automated mediation, for example when online buyers and sellers have a dispute. One side registers its dispute, and selects which options for resolution would satisfy them. The other side receives the same list, and if both sides select the same option, problem solved. Online dispute resolution is already big and only getting bigger (including an upcoming European Union-wide platform).

The better the range of options suggested by the mediator, whether a human being on or offline or a piece of software, the better the chances of a happy resolution. Most e-commerce disputes are fairly straightforward (I paid for X and got Y; I paid for X and got it a month late), so you don’t need a range of possible solutions. But for trickier disputes, a key piece of mediation is creatively thinking of new options that neither side had even considered, yet that address the heart of their concern. The crowd, with its wildly diverse ways of thinking, thinks of the creative ideas; the human third party, or the parties to the dispute themselves, see those ideas and discover a new satisfactory resolution.

Another role for the mediator can be helping the parties be realistic. If you feel you can trust the objectivity of the mediator, you might be more likely to listen when he says your demand is way too optimistic. A paper by van den Herik and Dimov suggests that here, too, the crowd can come in. Before going to court, you want to know what outcome you can realistically hope for. The crowd, diverse and neutral, is much wiser than the upset, self-interested party to the dispute, and can accurately predict the BATNA: the Best Alternative to a Negotiated Dispute. If you know that what you’re asking for is far off from what the crowd thinks you can hope to get in the best scenario, you’re likely to reassess your position and become a lot more willing to compromise.

Finally, what about crowd mediation for the geopolitical scene? Two countries have a dispute, big or small. The people who are ultimately affected are the citizens, and their government is meant to be representing them in the negotiation. In this case, it may be worth thinking about crowd wisdom in a different sense. The point of mediation is to find a solution that both sides can live with. Since the sides are actually made up of citizens, you need to actually know what the citizenry can live with, and not just what the politicians think they need to do in order to get reelected. How can we find that out? Polls are the obvious solution, but they’re limited to the options given in the question and affected by the way the question is framed.

But with political sentiment analysis, we can look at the whole enormous data set of informal political discourse on social media to get a much better picture of how people are feeling about an issue. And maybe even more importantly, how strongly they feel about it. Within the group of people who are upset about a given issue, you can see which potential solutions leave the more or less upset. Where the sentiment regarding a potential settlement is less negative, you’ve got a solutionthat the citizenry is more able to live with. And then the government knows it can bring that to the negotiating table. Better sentiment analysis = better identifying what the people can live with = government that’s more confident about taking the risk and agreeing to make a deal.

What, is sentiment analysis going to bring about world peace? Not too likely, but it’d be worth seeing how people actually feel about potential compromises for resolving even a smaller dispute involving their country.