I think there are wider problems that extend to non-money based systems: as soon as people start making decisions based on prediction markets, people will start deliberately distorting them to achieve desired goals. Like if you want AI safety funding to increase, why not deliberately put an overoptimistic forecast on AGI arriving on metacalculus? The question is weighted the same as way easier ones when it comes to your forecasting questions, so it won't even affect your top forecaster rating by much.
Yes, though these can be mitigated somewhat by more sophisticated scoring systems. The simplest example is if you also measure how "controversial" your prediction is, e.g. if people largely predicted something wouldn't happen, while you successfully predicted it would, you'd get more points (some platforms already do a version of this). This can be extended with a tagging system that records in which *domains* you have a good track record. This would prevent people from building up a "good track record" by e.g. successfully predicting every day that the sun will rise.
Although we might still expect some group bias here, so perhaps we also need something akin to the "birdsong" algorithm.
What I'm not arguing for is "futarchy" (that governments solely rely on prediction markets/prediction platforms), I envision the future more pluralistic, using a combination of; academic surveys, prediction platforms, hired researchers etc etc. They will all have their biases but by taking those into account and comparing them they will collectively be less biased.
In any case, prediction platforms will be better than prediction markets since there's less chance people will spread misinformation, act on harmful incentives, or gamble away their life savings.
Yeah, I'd be happy with a system where prediction platforms are used as an additional tool among many to evaluate claims, as long as they aren't used as a replacement for those other methods.
Part of doing this sucessfuly would be a clear-headed understanding of their limitations, so I'm glad you wrote this up!
I worked for Metaculus briefly and disastrously. They were quite obsessed with this idea of allocating decisions and funding based on Metaculus questions and simply would not understand the obvious point you are making here. I think it ultimately stemmed from Anthony lacking time to devote to "his" project while being extremely badly ignorant of how to properly hand it off to someone. So we ended up in a situation where the out-of-touch founder would have these ideas he was so sure about, and all the crazy then-CEO cared about was placating the founder and similar outside stakeholders so there was no clear process for reversing these stupid decisions.
I have to tell you that prior to working at Metaculus, I was non-EA but for the most part favorably disposed to them, or to who I thought they were. After seeing what it was actually like, I have come around to the conviction that the EA community is a massive disgrace and all of their supposed beliefs are hypocritical half-beliefs in the manner of a religion. When you give money to an EA organization it is likely to go to sending people to fake conferences, or hiring their buddy's project for services they would never otherwise have hired, and so on. The whole thing is functionally just a competition for places, most of which will never be obtained, pretending to be some kind of philosophical social movement. This is actually pretty obvious as soon as you get any kind of serious handle on these places, and the only reason EAs don't know it is that to be an EA is almost definitionally to be an inexperienced and impractical kid.
"hiring their buddy's project for services they would never otherwise have hired" is a direct reference btw to how Metaculus paid money to Verity, a worse version of Ground News, for useless services. Verity is the project of Max Tegmark, who is Anthony's longtime friend. Now that the Future of Life Institute has all that meme coin money, Tegmark has been able to hire his wife to supposedly run it. It's likely legal but it's all so blatantly and disgustingly corrupt.
Though I will say that the "global development" and to a lesser extend the "animal welfare" charities that EAs promote are still pretty good. It is, in my opinion, mostly the "meta-EA" and "longtermism" charities that people should be wary of.
Right, GiveWell was always my model of what EA was before I knew what EA really was, and back when I liked EA. In my view it's basically an indulgence-buying project to try and rub off some perceived virtue on the weirdos who want to fill the lightcone with shrimp or whatever it is that they're doing.
Fascinating post. I didn't know how often outcomes are influenced deliberately to profit from these markets. Still, it seems like these problems are solvable. Betting markets themselves would profit from banning people from participating if they can directly influence the outcome, or at least forcing those people to make all of their trades without anonymity.
Betting platforms (and let’s not give unearned valor to these by calling them prediction markets) are an u equivocally bad idea.
They have caused immense harm and no societal gain in the sports betting sector, and will cause even more harm in the political sector.
You rightly point out issues with addiction and perverse incentives, yet you barely mention the mammoth risk of malign agents.
The Russians, Saudis, and high-wealth individuals would find it a trivial task to wildly affect the market. If Trump is trailing, a series of bets involving a trivial amount of money could dramatically pervert prediction.
This could be stopped by creating finite pools, but that would be easily hacked as well.
The entire exercise of these markets, while perhaps academic in the past, is profit and speculation, and the potential harm is immense. There should be no conversation about promoting or legalizing.
Prediction markets or speculative markets are useful in any economy with a sound basis. These markets act as signaling when the requirements are properly are met. The difficulty is that the requirements haven't been met for quite a long time now, and they haven't been met because of various forms of money-printing, or betting on leverage which indirectly ties to money-printing (in its infinitely conceivable forms).
When you can bet more than you have, you utilize the full distorting power of money-printing when that bet goes bad. Someone has to pay it, if its not the person making the bet, its everyone holding the currency, and this is what most people misunderstand. The problems we see today aren't problems in small niche areas, they are chasms that have been carved out underneath everyone's feet for decades with most unaware of the danger, seeing small crevasse here and there as merely the small ups and downs of nature.
I'm sorry, I don't see the link between acting on market incentives and money printing. Are you saying that what e.g. Jontay Porter did is a kind of money printing?
Great post!
I think there are wider problems that extend to non-money based systems: as soon as people start making decisions based on prediction markets, people will start deliberately distorting them to achieve desired goals. Like if you want AI safety funding to increase, why not deliberately put an overoptimistic forecast on AGI arriving on metacalculus? The question is weighted the same as way easier ones when it comes to your forecasting questions, so it won't even affect your top forecaster rating by much.
Yes, though these can be mitigated somewhat by more sophisticated scoring systems. The simplest example is if you also measure how "controversial" your prediction is, e.g. if people largely predicted something wouldn't happen, while you successfully predicted it would, you'd get more points (some platforms already do a version of this). This can be extended with a tagging system that records in which *domains* you have a good track record. This would prevent people from building up a "good track record" by e.g. successfully predicting every day that the sun will rise.
Although we might still expect some group bias here, so perhaps we also need something akin to the "birdsong" algorithm.
What I'm not arguing for is "futarchy" (that governments solely rely on prediction markets/prediction platforms), I envision the future more pluralistic, using a combination of; academic surveys, prediction platforms, hired researchers etc etc. They will all have their biases but by taking those into account and comparing them they will collectively be less biased.
In any case, prediction platforms will be better than prediction markets since there's less chance people will spread misinformation, act on harmful incentives, or gamble away their life savings.
Yeah, I'd be happy with a system where prediction platforms are used as an additional tool among many to evaluate claims, as long as they aren't used as a replacement for those other methods.
Part of doing this sucessfuly would be a clear-headed understanding of their limitations, so I'm glad you wrote this up!
I worked for Metaculus briefly and disastrously. They were quite obsessed with this idea of allocating decisions and funding based on Metaculus questions and simply would not understand the obvious point you are making here. I think it ultimately stemmed from Anthony lacking time to devote to "his" project while being extremely badly ignorant of how to properly hand it off to someone. So we ended up in a situation where the out-of-touch founder would have these ideas he was so sure about, and all the crazy then-CEO cared about was placating the founder and similar outside stakeholders so there was no clear process for reversing these stupid decisions.
I have to tell you that prior to working at Metaculus, I was non-EA but for the most part favorably disposed to them, or to who I thought they were. After seeing what it was actually like, I have come around to the conviction that the EA community is a massive disgrace and all of their supposed beliefs are hypocritical half-beliefs in the manner of a religion. When you give money to an EA organization it is likely to go to sending people to fake conferences, or hiring their buddy's project for services they would never otherwise have hired, and so on. The whole thing is functionally just a competition for places, most of which will never be obtained, pretending to be some kind of philosophical social movement. This is actually pretty obvious as soon as you get any kind of serious handle on these places, and the only reason EAs don't know it is that to be an EA is almost definitionally to be an inexperienced and impractical kid.
"hiring their buddy's project for services they would never otherwise have hired" is a direct reference btw to how Metaculus paid money to Verity, a worse version of Ground News, for useless services. Verity is the project of Max Tegmark, who is Anthony's longtime friend. Now that the Future of Life Institute has all that meme coin money, Tegmark has been able to hire his wife to supposedly run it. It's likely legal but it's all so blatantly and disgustingly corrupt.
Thanks for the insights! Always interesting to hear from an insider. I agree that EA tends to be very inner-circle-y, in fact I made a whole post about it: https://bobjacobs.substack.com/p/how-democratic-is-effective-altruism
Though I will say that the "global development" and to a lesser extend the "animal welfare" charities that EAs promote are still pretty good. It is, in my opinion, mostly the "meta-EA" and "longtermism" charities that people should be wary of.
Right, GiveWell was always my model of what EA was before I knew what EA really was, and back when I liked EA. In my view it's basically an indulgence-buying project to try and rub off some perceived virtue on the weirdos who want to fill the lightcone with shrimp or whatever it is that they're doing.
Fascinating post. I didn't know how often outcomes are influenced deliberately to profit from these markets. Still, it seems like these problems are solvable. Betting markets themselves would profit from banning people from participating if they can directly influence the outcome, or at least forcing those people to make all of their trades without anonymity.
Betting platforms (and let’s not give unearned valor to these by calling them prediction markets) are an u equivocally bad idea.
They have caused immense harm and no societal gain in the sports betting sector, and will cause even more harm in the political sector.
You rightly point out issues with addiction and perverse incentives, yet you barely mention the mammoth risk of malign agents.
The Russians, Saudis, and high-wealth individuals would find it a trivial task to wildly affect the market. If Trump is trailing, a series of bets involving a trivial amount of money could dramatically pervert prediction.
This could be stopped by creating finite pools, but that would be easily hacked as well.
The entire exercise of these markets, while perhaps academic in the past, is profit and speculation, and the potential harm is immense. There should be no conversation about promoting or legalizing.
> and high-wealth individuals would find it a trivial task to wildly affect the market
I actually talked about this in a previous post: https://substack.com/@bobjacobs/p-137844099
> This could be stopped by creating finite pools, but that would be easily hacked as well.
Agreed, though maybe "no pool" "markets" (something akin to metaculus) could still work?
The challenge is that if it is just “chits” you lose the pricing function, it if it is cash-driven it is gamed (or irrelevant).
I would be remiss if I failed to mention that AI will make all of these markets fraud machines.
perhaps there is a world where there is strict, traceable identity, but I am not sure that is realistic.
Prediction markets or speculative markets are useful in any economy with a sound basis. These markets act as signaling when the requirements are properly are met. The difficulty is that the requirements haven't been met for quite a long time now, and they haven't been met because of various forms of money-printing, or betting on leverage which indirectly ties to money-printing (in its infinitely conceivable forms).
When you can bet more than you have, you utilize the full distorting power of money-printing when that bet goes bad. Someone has to pay it, if its not the person making the bet, its everyone holding the currency, and this is what most people misunderstand. The problems we see today aren't problems in small niche areas, they are chasms that have been carved out underneath everyone's feet for decades with most unaware of the danger, seeing small crevasse here and there as merely the small ups and downs of nature.
I'm sorry, I don't see the link between acting on market incentives and money printing. Are you saying that what e.g. Jontay Porter did is a kind of money printing?