Some thoughts. If it is relevant, I have maths background, am male, have south asian ethnicity:
- I believe there is a strong selection effect here. The type of people who like or are comfortable with the reasoning that is core to EA and/or rationality are also far more likely to do subjects like econ / comp sci / philosophy. Types of reasoning include decoupling, or, quantifying costs/benefits.
- I would be curious to know what the stats are like in EA student groups. Which kind of people are initially attracted during a student fair or who make first engagements? If it skews one way, what are the reasons for this?
- Have you tried reading / recruiting from non-standard backgrounds? If yes, would be curious to read about your experiences!
Thanks for adding your demographics, but for future commenters, it's not necessary. Having data on demographics is important for large groups since it informs dynamics and biases, but that information is much much less useful when trying to draw conclusions about individual people.
I study "moral science" myself, which is 50% philosophy and 50% various social sciences including economics, psychology, sociology, etc. I used to run EA Ghent until, today actually. I recruited mostly philosophy and moral science students, since I knew them the best. But using 'quant people are more interested in EA, therefore it's more relevant for EA' as an argument is a bit strange since it's assuming the conclusion (not that you were making that argument, that's just one interpretation). Perhaps the methodology baked into EA is suboptimal, and EA has mistakenly become unattractive to people who would be high impact.
One problem I notice between these different research areas/different researchers, is the use of different methodologies. I used to also be pretty much die-hard pro-quantifying costs/benefits, but have become less so as I learn more about other disciplines. For example, my latest blog post is about how we managed to enshrine animal welfare in the Belgian constitution. How do you quantify that? Well, you don't. The effects are so broad, multifaceted and indirect that they're impossible to quantify. With things like medical interventions, we can run an RCT (which the EA framework loves), but the same cannot be done with constitutional changes since we don't have a "control Belgium". RCTs are great, but they also have drawbacks. They are expensive and measure narrow, direct, continuous effects, while they're unpractical for broad, indirect, or discontinuous effects. See, e.g. "the problem of marginalism" https://eprints.gla.ac.uk/289530/1/289530.pdf
But obviously, changing the constitution has a big impact; just because we can't quantify it doesn't mean it doesn't exist. And that's where other methodologies come in. E.g. the historical method will tell you that yes, changing a constitution has indeed had big impacts on the history of countries. It can tell us about big trends, while RCTs can tell us about tiny, discrete trends (which, combined with the cost, does mean that RCT-based interventions privilege the status quo more). Focusing so much on quantification can lead us towards the McNamara fallacy: https://en.wikipedia.org/wiki/McNamara_fallacy
Interesting post. I've skimmed over some sections, but I think I'll be reading it again soon. Thanks for sharing. However, I did notice it mostly doesn't apply to what I wrote. Let's assume his premises and methodology are all correct (I haven't checked, it's not peer reviewed, and he's not an academic, so that's a bit bold, but why not). He draws the conclusion that any given economics, sociology, education, and "other" are good at replicating, and social psychology, political science, criminology, psychology, evo psychology, cognitive psychology, management and marketing are bad at replicating.
Now let's look at the disciplines I mentioned: economics, sociology, anthropology, gender studies, geography, political science, and history. As you can see, I didn't mention psychology, partly because I'm well aware of the problems in that field (and in fact, my post shows that psychology is worse at citing other sciences than economics), and partly because I only consider it a semi-social science. I didn't mention marketing because I know the problems it has, and I didn't mention management because I didn't even think of it (but wouldn't have included it if I had thought of it). That leaves us with economics and sociology, which he says are good, and political science, which he says is bad. Okay, what about the others I mentioned: anthropology, gender studies, geography, and history? They're not part of this paper, or if they are, they're part of "Other". If it's "Other," it's mostly good, and if it isn't, then it mostly doesn't seem to apply to what I wrote. And importantly, these are disciplines that are much better at addressing his seventh point; they do create more theory.
Also, this is not how almost anyone actually reads about other disciplines. Mostly, we read books about a topic that only show a couple of the best conclusions the field has reached in the past decades, aka mostly stuff that has been replicated. A given criminology paper has a low chance of being replicated. So what? Focus on reading the papers that were replicated; that's what I would recommend anyway.
- I agree with you that fantasticanachronism post does not challenge the underlying methodologies, tools or reasoning styles, or the strongest parts of the fields.
- "Perhaps the methodology baked into EA is suboptimal, and EA has mistakenly become unattractive to people who would be high impact." Always interesting to consider what the blindspots are of EA! Do you already know styles of reasoning that are missing in EA?
- "How do you quantify that? Well, you don't.". "E.g. the historical method says changing a constitution has had big impacts". I use 'quantify' in a broad sense to include things like "what percentage of historical constitutional changes had big impact" (which is impossible to answer literally as stated, but is at least the kind of thing I'd like to get a sense of), rather than 'quantify = rct'.
- I dont think constitutional changes are 'obviously good'. In your case, my instinct is it is good (I am between vegetarian and vegan so yay for animal welfare progress!). But, policy/legal changes could have unexpected negative consequences - e.g. imagine this makes it even harder to do any building or construction because there is extra bureaucracy around the affect on insects living in the ground being constructed on.
- Contrary to the tone in your comments, EA is open to many kinds reasoning / quantification other than RCTs. One example is there are no RCTs to measure impact of x-risk work. Second example is that if you listen to EA (adjacent) podcasts (like 80k or ClearerThinking), you will be exposed to large range of ideas. E.g. big one I recently listened to on 80k is with Mushtaq Khan - highly recommended if you're interested in policy!
- I re-read your OP and my comments have already veered off track (but hopefully in a way that is enjoyable!). Tackling your OP directly, it is not clear exactly what your critique of EA is and precisely which problems from economics you think EA has inherited. E.g. you have stats about how economists undervalue multidisciplinary research, but I dont think EA suffers from that problem. My reading is you have two main claims. First, EA undervalues certain fields of social science - I think the best arguments for this would be to find examples of reasoning/methodology from those that are being missed by EAs. Second, EA has bad recruitment strategies which leads to under-representation of those fields and of various demographic groups. I think this is likely, but for me what would be interesting to know is how EA could improve their recruitment.
> Always interesting to consider what the blindspots are of EA! Do you already know styles of reasoning that are missing in EA?
Well, part of the problem is that I've been steeped in EA culture too, so I'm still in the process of figuring this out myself. The main thing I would say is to not use one methodology for everything; switch methodologies depending on what is appropriate for the subject. E.g. when trying to understand whether films have become more inclusive, we could grab EA's trusty regression analysis, and this will tell you something, but why not grab narrative analysis, since it's specifically designed for that kind of stuff.
> I use 'quantify' in a broad sense to include things like "what percentage of historical constitutional changes had big impact" (which is impossible to answer literally as stated, but is at least the kind of thing I'd like to get a sense of), rather than 'quantify = rct'.
Yes, an RCT is the most extreme example, but the broader point is that a lot of historical analysis is not quantitative; it's qualitative.
> Contrary to the tone in your comments, EA is open to many kinds reasoning / quantification other than RCTs. One example is there are no RCTs to measure impact of x-risk work. Second example is that if you listen to EA (adjacent) podcasts (like 80k or ClearerThinking), you will be exposed to large range of ideas. E.g. big one I recently listened to on 80k is with Mushtaq Khan - highly recommended if you're interested in policy!
The bulk relies on certain methods/disciplines. The fact you said quantifications is already pointing in that direction. What about qualitative research, like what most historians are doing? If you like black humor, read the origins of the term 'McNamara fallacy' for the types of things that can go wrong with overrelying on quantification.
> I dont think constitutional changes are 'obviously good'.
I don't know where that quotation comes from, but it's not from my comment. I said it obviously has a big impact, not which direction that impact will be.
>" My reading is you have two main claims. First, EA undervalues certain fields of social science - I think the best arguments for this would be to find examples of reasoning/methodology from those that are being missed by EAs.
Is the beginning of my post not evidence for it?:
>" When you scroll through the Effective Altruism Forum you’ll find lots of economic analyses.
There is a tag for Economics with 161 posts, as well as one for Economic growth (130 posts), Economic inequality (10 posts), Economics of artificial intelligence (28 posts), Welfare economics (17 posts), Tax Policy (21 posts), Markets for altruism (33 posts) and many other subcategories.
Now I really like economics and I love economic analyses, but I think it’s very surprising how little presence the other social sciences have. The Social science tag only has 27 posts and there is no tag for sociology, anthropology, gender studies, geography, political science, or many other social sciences.
When you look at the People page of the EA forum, you’ll find lots of economists (and entrepreneurs) but almost no other social scientists. You will also notice that the vast majority of the people on this page are white men. This has been this way for a long time; this is what the page looked like a year ago: "<
> "a lot of historical analysis is not quantitative; it's qualitative"
Can you give an example of a qualitative research tool that EAs are under-using? Making it concrete will make it easier for me to understand.
> "Is the beginning of my post not evidence for [EA undervaluing certain fields of social science]?"
It is some evidence, but not strong evidence. The same reasoning could be used to show that EA undervalues every possible field of study, which is not what you are arguing for (e.g. you mentioned previously you would not include management in your list). My question is why you believe that EA is undervaluing "sociology, anthropology,..." in particular, and a good way to do this is to point to specific reasoning or tools in those fields and how EAs could use it to further their goals.
To emphasise, I am not saying they don't exist, but it would be useful to have concrete examples. I'm also fine if you do not know a good example off the top of your head, but I'd also like to know if that is the case.
> "I said it obviously has a big impact, not which direction that impact will be."
First, apologies for misunderstanding. In many contexts 'impact' implicitly means 'positive impact' (e.g. 'I would like to have an impactful career'), so it was simple misunderstanding on my part. Second, why did you say that historical analysis can show constitutional change can have large impacts (but not the direction of impact)? Why is this useful?
> Can you give an example of a qualitative research tool that EAs are under-using? Making it concrete will make it easier for me to understand.
Sure! I already mentioned the historical method and narrative analysis, which are two big ones, but a specific one I'm currently looking at is action research, since it would allow us to do (prospective) pro-social actions while we research. (And again, I'm not advocating that we replace everything with them, just that we add them to our toolbelt)
> why did you say that historical analysis can show constitutional change can have large impacts but not the direction of impact?
I was merely careful in my post not to prematurely claim that the constitutional change we implemented will be a positive one. The historical method can show us the direction of impact, but only when paired with a moral framework. So for example, the historical method can shows us which policies lead to better lives for animals, but whether that's a good direction depends on whether you use a moral framework that considers animals moral patients.
> My question is why you believe that EA is undervaluing "sociology, anthropology,..." in particular, and a good way to do this is to point to a specific reasoning or tools in those fields and how EAs could use it to further their goals. To emphasise, I am not saying they don't exist, but it would be useful to have concrete examples. I'm also fine if you do not know a good example off the top of your head, but I'd also like to know if that is the case.
I'll do you one better, I'll give you "explorable explanations" on social science topics I think you will find interesting. These are brief, fun, interactive, explanations of the topic:
This is great! Thanks very much. Don't have time now to look into details but appreciate the back and forth. I hope someday you'll be comfortable identifying as EA again. Seems like you have a valuable perspective to add.
I don't think we should update too hard on this. He states his viewpoint that EA does a lot of qualitative research when relevant, and then links one case study by an EA as an example to back it up. Which in this context is totally fine and normal; it's a comment, not a post or study... but it's still only one example.
Hmmmm as the person who created the "Tax Policy" tag - it might just be a tagging problem rather than the posts not existing. I probably had to put in an hour or so finding relevant posts and writing up the wiki.
It sounds like you might be a good person to do the same for the social science tags you suggested?
Also increasing the number of women in EA, I ran created the London EA Women and NBs group chat and ran monthly meet ups for about a year. I don't think direct "recruitment" of women would work, it's more about encouraging people (especially woman) to lead and build the EA they want to see.
I didn't suggest "direct recruitment of women" (nor did I say "EA should"), I suggested "reading and recruiting more from other disciplines too."
I've had incredibly bad experiences with the tag system. When I removed Elon Musk and SBF from the EA people page with a comment explaining why (*after* the FTX scandal), it got downvoted, and I had to ask a friend to upvote it. When I added a tag for "abuse" and wrote a whole wiki-page with psychology literature and citations, it got downvoted and removed with the excuse that EA didn't deal with that topic. I had listed numerous articles that the tag could be added to, and even now when you search "abuse" on the EA forum you'll get 544 post results. I personally suspect the real reason it got deleted was that among the posts I wanted to add the tag to was the post "An Exploration of Sexual Violence Reduction for Effective Altruism Potential" by Kathy Forth, an EA member they'd prefer people forget about (because of the assault and subsequent suicide).
I also shouldn't be the one to make the tag, since I left EA after the manifest scandal made it clear the EA community would continue to deplatform leftists while platforming fraudsters and race "scientists" and the like. I really should have left after my years of unpaid and unacknowledged labor for EA got rewarded with a high profile EA publicly, indirectly (and falsely) accusing me of doxxing people, without him facing any repercussions, but I kept on going until the day the manifest people said they would invite some of those "scientific racists" to speak again next year, which was the straw that broke the camel's back.
Hmmm I don't think this is entirely fair. I haven't met anyone in EA (other than young students) who don't read outside of EA and idk I think you are suggesting that more funding / volunteer time should go to speculatively finding people in other fields.
Tbh I would agree with their editorial decision on a tag called "Abuse", from looking at the 544 articles it's too wide ranging and could include Animal Welfare, Domestic Violence, Corruption, Political suppression of women in misogynistic countries, Criticisms of EA culture, Reducing risks from malicious/abusive actors etc. I doubt they'd have an issue with a Sociology tag based on their Wiki FAQ https://forum.effectivealtruism.org/topics/ea-wiki-faq
I'm not sure I'd agree that Manifest is EA - maybe EA-adjacent.
Who are you expecting recognition from? Idk but I'd say being able to donate to effective charities and work directly on pressing problems is a massive privilege and there's a responsibility for privileged people to use it to improve the world.
I think the EA forum is a useful resource for working out the best ways to do that, so I want to contribute as I found it helpful, but honestly doesn't owe me more than that :/
Look at what I wrote: "reading and recruiting more from other disciplines too.", emphasis on the "more" aka I didn't claim that they don't read outside of EA (in fact it wasn't even about EA, it was about economics).
> I would agree with their editorial decision on a tag called "Abuse"
You are agreeing with them without even knowing anything about what I had written. I made it very clear in the article what form of abuse it was for. I had cordoned off all those different versions of abuse, e.g. on the wiki article I wrote that for animal abuse people should use the animal welfare tag, for abuse in EA use the criticism of EA tag etc etc. (I didn't say those 544 posts all would've qualified, I merely pointed to it to show their excuse didn't hold any water)
> I'm not sure I'd agree that Manifest is EA
One of the Core Topics on the EA forum is Forecasting and when you click on it manifold right there on the sidebar and it's also on the wiki article itself. Also manifest was promoted on the forum, manifund has been given funding by the EA community, and prominent EAs were there at manifest.
> Who are you expecting recognition from?
Read it again, I didn't expect recognition, I expected that "years of unpaid and unacknowledged labor for EA" not get "rewarded with a high profile EA publicly, indirectly (and falsely) accusing me of doxxing people, without him facing any repercussions". That's a very different thing.
> Idk but I'd say being able to donate to effective charities and work directly on pressing problems is a massive privilege and there's a responsibility for privileged people to use it to improve the world.
Donating to effective charities and working on pressing problems is a great thing, and just because I'm no longer in EA doesn't mean I won't continue to do so, that's a weird implication (also working on pressing problems is more of a privilege if you get any money or help doing it, otherwise the privilege is in the prerequisite socio-economic conditions and not the work itself). I would say people should do that, but that EA is not the best *way* to do it.
I was perfectly fine working and donating all those years to EA, until I learned that it was going to chateaus in england and czechia, and used to promote crypto-scammers and "race scientists" (not just with the manifest thing, but also the (reaction to) the Bostrom email, FLI offering funding to a far-right foundation that, among other things, promoted holocaust denial, and the list goes on). An EA scamming me on the forum without any repercussions (a labor scam, not a crypto scam), and then after that the doxxing thing also didn't help my enthusiasm. Not to mention the EA forum soft-censoring things: https://substack.com/@bobjacobs/note/c-68603515
EDIT: I do believe in the things I wrote here, but the tone was overly harsh. Sorry Gem!
I'm feeling a bit frustrated by your reply here as I don't think it is a charitable reading of what I said and doesn't seem to reflect on the trade off of any community so I'm going to disengage.
Do you think the reason that EA has those issues is because it’s an offshoot of economics (so just gains its biases? I’m a little confused on the story here I guess) or that there is a hidden third thing correlated with both (perhaps the type of analytical mindset tho it could be different or a bunch of things) that both economics and EA shares?
1) It started from charity analysis, where economists/economic tools are a natural default. Due to demographic lag and the founder effect this remained the default, even after EA started to become interested in things like bio-security, nuclear security, policy etc
2) The large overlap with the rationalists community, who are more hostile to the other disciplines because they are more "woke/skeptical of capitalism/lefty".
Demographic networks and the founder effect are problems every movement/community has to deal with, but there are some structural features which worsen it for EA (e.g. the groupthink inducing karma system: https://substack.com/@bobjacobs/note/c-68603515 ) Though I do expect that, as with other movements, it will slowly decrease over time.
The minimum wage example here isn’t a strong point in favor of your main argument since the “it doesn’t dramatically increase unemployment” results, both theoretically and empirically, came from within economics, not outside of it. In fact it kind of shows the opposite - Econ still doesn’t need non-Econ to question itself (and frankly I’m not sure what existing non-Econ could contribute to this debate constructively, which is different than what non-Econ could possibly contribute if it itself wasn’t so hopelessly muddled)
No, it came from both within economics and outside of it (e.g. sociologists like Rubery)
But even if you don't like this example I can give you others:
Psychological research showed that the premises of neoclassical models, e.g. that of the homo economicus, were empirically wrong. See for example the work of Kahneman and Tversky
Economic history has contributed massively to economics, and you can thank those historians for a lot of those nice datasets economists rely on. Often the historic data contradicts economic models. See for example the work of Robert Allen.
Anthropologists have also contributed to disproving economic models. Such as James Scott showing peasant economies are not driven by profit maximization but by subsistence security.
Political science has contributed to economics by showing that political institutions, not just market forces, drive economic development. (e.g. How Nations Fail)
Similarly, sociology has contributed with concepts like 'social capital' (e.g. Bourdieu) that e.g. show how education impacts economic outcomes beyond what economic theories could predict.
And this is only how other social science have contributed to economic *models*, that's not even talking about how other social sciences are necessary to discover and document the harms of markets (and thereby are necessary for analyzing the long-term trajectory of markets). The most classic example is greenhouse gasses, where we need to listen to other social sciences (like climate policy experts) to find fixes for the economy.
> what non-Econ could possibly contribute if it itself wasn’t so hopelessly muddled
I hope this is a joke, or perhaps an accidentally misphrased sentence, because I would lose a lot of faith in humanity if people actually think economics is the only social science that has contributed to our knowledge.
Why is Economics so insular? Is it a feature of the historical development of the field, for instance the forefathers of the field moved towards treating it like a pure discipline rather than a social science?
It could also have to do with the study itself. Economics seems focused on creating mathematical models for how a society arranges resources. This is a daunting task, which is made easier by reducing people into rationalist agents, and any attempt to increase the complexity of these agents likely increases the computational complexity. Therefore a lot of the research from psychology or sociology likely create more accurate models, but with greater complexity. Compare behavioural economics to traditional economics.
Another question surmises in terms of the demographics. I want to see if the demographics are an artifact of the history of Effective Altruism, a secondary effect of its strong ties to Economics, or if different demographics have cultural influences that reject the main philosophy of EA. For instance, in cultures that are more status-oriented, like South-Asia, I wonder if EA can't gain any traction because of its disregard for those status games.
I think it's mostly the factors you mention, which are then exasperated by the hyper-competitiveness of getting job security in the field, plus a comparatively small amount of "top" scientific journals for economics (which all favor model-heavy approaches). I remember reading an economics book that at one point applied economic insights to the field of economics itself and concluded that the reason math and math-heavy models became so prominent is not because it increases the accuracy, but rather because it's a way to signal intelligence. I can't for the life of me remember which book it was though.
> For instance, in cultures that are more status-oriented, like South-Asia, I wonder if EA can't gain any traction because of its disregard for those status games.
Yeah maybe, though it probably also doesn't help that EA is very reliant on funding from only a handful of funders from the anglo-american world. Those funders only read English-language things, run in anglo-american social circles, and, like every human being, has biases towards their ingroup. If we look at the startups EA funds it is *heavily* skewed towards ones from the anglo-american world, so it might also be that the South-Asian EAs just have more trouble securing funding.
I'm interested, what would be the alternative to a math-heavy model? Computational?
I don't know if the math-heavy models are more prominent because people try to signal intelligence, or if people in other fields which are considered intelligence featuer math, and that bleeds over to economics. For instance, iirc physics has some overlap to economics, and I think people would generally assume physicists are smarter than, say, a sociologist, so a physicists' approach to economics likely has more credence than a sociologists. Maybe it's not as malicious, but it leads to the same outcome.
> Yeah maybe, though it probably also doesn't help that EA is very reliant on funding from only a handful of funders from the anglo-american world.
Yeah I think that's where I'm leaning towards personally. I don't know if EA would push towards more diverse funding metrics though. In my (biased) view, EAists tend to disagree with DEI, so the idea of allocating more funding to EAs in other regions likely won't ever get traction, compounding the issue of making funding more difficult to secure, creating a vicious cycle.
Thank you for this article though, you gave me a lot to reflect on.
> I'm interested, what would be the alternative to a math-heavy model?
Empiricism. There's nothing intrinsically wrong with math-heavy models, but you only get out what you put in. So if you make complex models of the homo-economicus you can generate many papers describing the results, but... well... so what? We're not the homo economicus so it's not really useful. It might be better to just focus on empiricism, at least for a while. I think we should step out of the armchair and go out to collect data: observe what's happening in the real world, and see if patterns fall out of that data, instead of describing the result of math formulas based on premises we know can't be applied to the real world.
To be fair this shift is already happening a bit, as the graphs I presented seem to indicate. Economics seems to especially start paying attention to the demographic-data sociologists are collecting, so I expect economics to become a more integrated discipline in a couple of decades.
> I don't know if the math-heavy models are more prominent because people try to signal intelligence, or if people in other fields which are considered intelligence featuer math, and that bleeds over to economics. For instance, iirc physics [...] Maybe it's not as malicious
It's not malicious (well except for that one economist I quoted), it's a race to the bottom. This phenomenon of wanting to be like physicists actually has a name: it's called physics-envy: https://en.wikipedia.org/wiki/Physics_envy
> EAists tend to disagree with DEI, so the idea of allocating more funding to EAs in other regions likely won't ever get traction, compounding the issue of making funding more difficult to secure, creating a vicious cycle. Thank you for this article though, you gave me a lot to reflect on.
You're very welcome, glad you liked it!
I think it's not so much a principled opposition to DEI as a desire to get funding. Everyone wants themselves and their contacts to be doing well so everyone is willing to take a little bit of the pie to ensure that. I think most EAs would be fine if a lot of the rest of the pie goes to people outside of their circle, but they'd very much like a slice of the pie for their circle first. Of course, if everyone does that not much remains.
I am new to the Substack community and therefore astonishingly late to this conversation, but I wanted to engage re: 'the insularity of economics'.
For context, I am currently in the middle of my PhD in Economics. The simple reason I observe significant disdain for other social sciences within Economics is that their methods of analysis are either not empirical or not causal empirical analysis, if they are empirical at all. This critique is magnified by the fact that most economists truly are interested in positive rather than normative evaluation, and where positive questions are posed, solid, empirical answers are critical.
In contrast, other social sciences are in the throes of serious replicability crises because many of their empirical results were never identifying real empirical phenomena (instead, strong correlations and associations) and even many experimental results were so shoddily designed (or fake) that they don't stand up to replication. Then, we have the social sciences that simply posit theories and armchair reason from the top of the ivory tower whose findings are largely irrelevant to the average person and policymakers.
To be clear, Economics suffers from some of these critiques too. Many Economics papers require assumptions that barely apply to superhuman AI minds, let alone a normal person thinking two seconds about which pen to buy at a supermarket.
However, Economics has also produced real, meaningful results within its more methodological fields like Econometrics (a cutting edge field that has practically revolutionized the quality of empirical evidence across multiple social sciences), its theoretical fields like Market Design (overhauling the medical residency matching market, internet auctions, and kidney exchange), and its applied fields like Behavioral Economics (nudge theory and the economics of charity).
So a large part of the appeal of Economics and why we doubt other fields is about the quality of evidence and results.
Now, there are other huge problems within Economics, and I don't really care to defend those. I would just be wary that as you invite other fields and consider their insights that you apply the same rigor of evidence to them as one does in Economics: rigorous RCTs in experiments, causal analysis in empirics, and validated predictions in theory.
Thanks for the comment. I think I do disagree on some points though:
First of all, I don't think that "empirical" = exclusively quantitative. Qualitative analysis answers questions that quantitative work simply can’t touch, at the very least not right now. It's important to have e.g. a boots on the ground ethnographer, or an historian going through the archives if you want to know about the “how” and “why” of mechanisms, meanings, and institutional evolution, since experiments or regressions cannot answer that on their own. I think this "ivory tower" accusation can also be lobbed at the economists, maybe actually moreso than some other social sciences like ethnography. In any case, triangulating across methods usually produces more reliable knowledge than insisting on a single gold standard.
Secondly, while you acknowledge some of the problems in social science are present in economics, you seem to imply the "replication crisis" isn't one of them. I think this isn't true: coding errors, hidden degrees of freedom, and garden-variety p-hacking are very much present in flagship economics journals. More importantly, the successes of economics often come from other disciplines, like mathematics/game theory (in the case of matching theory) and cognitive psychology (in the case of behavioral economics). I don't say that as a dunk against economics, this *should* be how science is done, I'm saying this to point out how interdisciplinary approaches have led to some of economics biggest successes and we should thus probably be doing more of it.
Now as an ethicists I would like to make a minor point that normativity is inescapable, both in a "foundations of science/knowledge" kinda way, and in a "understanding society" kind of way. I don't believe economists are *actually* uninterested in normative evaluation, and I don't *want* them to be uninterested in it since the normative aspect is so important. But setting that aside, I think other social science are at least as interested in sticking to to the descriptive over the normative as economics is. I mean, it's even the central creed of linguistics, so at least there, I would contend, they've been much moreso than the economists.
Lastly, other social science disciplines have actually made *huge* contributions to social science. I already mentioned cognitive psychology, but there are many more:
I'm a voting theorist, which may or may not be a branch of philosophy, but at least the overarching field of election science is a social science, and it made huge contributions to electoral design. Is that a subdiscipline of political science? Probably? The edges are blurry, but what is definitely a subdisciplines of political science that made huge contributions is public opinion research, where modern methods like MRP and policy-tracking in general are now the cornerstone of a lot of other social science research.
Same for sociology: I already mentioned computational sociology in another comment, but there are other sub-disciplines too. Demography for one is hugely important for all the other social sciences, and I consider ignoring demography for so long one of economics' biggest blunders. But also things like 'social network analysis', which has given us a whole new toolset to study so many things (from job hunting, to friendship) that ordinary regression ignored. It has also helped in the subfield of social epidemiology, which has proved so vital during covid and beyond.
I could go on and on; climate economics needs geography/earth-system science, labor economics needs sociology, education economics needs pedagogy, behavioral economics needs psychology etc etc... Ignoring adjacent fields can slow recognition of key mechanisms (like what happened with social capital and institutional path dependence) that matter for understanding how to increase global welfare.
And tbh, I think I might actually secretly like economics the most of all the social sciences, which is why it's especially heartbreaking to see it shoot itself in the foot so much.
Yes! I agree on the value of interdisciplinary approaches. I am a matching theorist myself and realize that what makes Economics great is that despite its elitist tendencies, when it does incorporate other fields, it does so in a way that typically improves and adds knowledge that benefits the domain of Economics and the domain of the intersection. Publications in top journals usually reflect this: they often require strong applications that might not fit the mold of Economics while also contributing general insights to some economic phenomena.
I will disagree with and steelman your critique about interdisciplinarity at the same time. I think Economics has very strong engagement with some fields. You named a few: political science (political economy), network science (network economics), and graph theory (matching theory). Heck, my own research touches social choice theory as well. If I decided tomorrow that I wanted to be a voting theorist, my advisors might look at me a little funny, then say, "Whatever, go for it."
However, you are right that there are some empirical methods that economists won't touch with a ten foot pole. Ethnography is one. There is probably room for some econometrician to figure out how to formalize that as a standard of evidence, to be honest, and that would be fascinating and valuable.
I don't want to get into the weeds too much, so I will just note that I'm generally sympathetic to your points about normative analysis and contributions of other fields.
My biggest real disagreement is about the replication crisis point.
Yes, there are papers that have been demonstrably manipulated in some form or another. Unfortunately, as it's not easy to identify statistical gaming in research, I can only speculate as to its commonality across fields. My point wasn't that Economics deals with less of the replication crisis, but that it's, at the very least, harder to publish papers that might contain invalid conclusions even with honest researchers. For example, it is nigh impossible to publish an experiment that is not an RCT or an applied paper that has only correlational rather than causal analysis in any top Economics journal. In contrast, both of these would fly fine in most sociology and psychology journals, then a plurality of correlational studies get compiled into a meta-analysis that is somehow supposed to amplify the rigor of the evidence. Unfortunately, aggregations of correlations are still just correlations. I would not have a problem with this if it were widely acknowledged, but oftentimes social sciences grade themselves on a curve and take the 'best available evidence' as 'good evidence'. The two are not equivalent.
It doesn't mean that we shouldn't do these types of studies or that other social sciences are not valuable. It just means that what we can learn from some papers is limited, and, as you rightly state, a plurality of methods is necessary to learn the truth.
> I think Economics has very strong engagement with some fields. You named a few: political science
I disagree for reasons cited in my post. As Fourcade et al found, economists were the only group that thought interdisciplinary research was worse than research from a singular field, and economists cited political science way less than the other way around (17.5% vs 0.8%, that's a huge difference!). Now this is from a while back so there might be a paper out there I haven't seen that shows things have improved. In fact, I would expect it to have improved given that Angrist et al showed a trend of it improving. However, since it started off at such an incredible lowpoint I'd still expect it to be low overall.
> For example, it is nigh impossible to publish an experiment that is not an RCT or an applied paper that has only correlational rather than causal analysis in any top Economics journal. In contrast, both of these would fly fine in most sociology and psychology journals
That may be true for sociology, but I'm not sure it's true for psychology. I think contemporary top psychology journals are also wary of purely correlational studies. But take this with a grain of salt since psychology is not my field so it might be that some top journals are better than others and I just so happen to read the good ones. The only reason I think this is true for sociology is that it doesn't seem to *have* "top journals" in the way economics does. This has the drawbacks already mentioned, but it also has some upsides. Suicides seems to be very common in economics, even for some of the most accomplished economists (e.g. Weitzman, Krueger, Farhi). It seems like because economics is so much more hierarchical than sociology, it's also a lot more stressful/cut-throat. The infamous "top 5" can be a career maker or breaker, regardless of the quality of the papers you write. Now if the "top 5" really were an objective measure of who was the best in the field then arguments could be made that this might be worth it. But that doesn't appear to be the case: https://www.aeaweb.org/articles?id=10.1257/jel.20191574
As so often when people represent their strict hierarchy as natural and necessary, it appears this hierarchy also reflects ingroup bias and what we in dutch call "vriendjespolitiek" (aka helping your contacts and underlings and punishing people who speak up against you): https://showmethemath.org/2022/01/heckman-judd-war-ii/
This is of course very bad for people outside of the ingroup who want to enter it, which is what the whole second section of my post (on the strong bias against women and minorities in the discipline) was about.
Interesting thoughts - perhaps my institution is unique, but we have a strong bend towards interdisciplinary approaches so I have not any anecdotal experience with economists that frown on them. I will reserve judgment on that till more updated studies are published.
The tyranny of the top five is indeed a huge problem, but it is in some ways subordinate to the "lust of the top ten". The article you cite writes, "The observed pattern of publication behavior suggests that the number of T5 publications required for tenure decreases with department ranking." Economics can emphasize the worst in high-performing stardom seekers which I do think has serious ethical problems. No argument from me there. Nevertheless, the high research standards are not limited to just the top five. They are prevalent across every journal that the cited article presents as influential.
I do have an axe to grind against correlational studies, to be fair, and not just in social science. I think that researchers should be a lot more careful about how studies get used in public policy recommendation. My biggest grievance is early recommendations to avoid peanut exposure in infants---following which a great number of peanut allergies emerged in the U.S.---and subsequent RCT findings that, actually, early exposure is crucial to prevent allergies.
Thanks for the comments! This was a good read and engaging discussion.
Some thoughts. If it is relevant, I have maths background, am male, have south asian ethnicity:
- I believe there is a strong selection effect here. The type of people who like or are comfortable with the reasoning that is core to EA and/or rationality are also far more likely to do subjects like econ / comp sci / philosophy. Types of reasoning include decoupling, or, quantifying costs/benefits.
- It is feasible that there is not easy insights to be gained from the social sciences. This is a pretty damning perspective on social sciences by person who reviewed 2000+ papers for Replication Study: https://fantasticanachronism.com/2020/09/11/whats-wrong-with-social-science-and-how-to-fix-it/#.
- I would be curious to know what the stats are like in EA student groups. Which kind of people are initially attracted during a student fair or who make first engagements? If it skews one way, what are the reasons for this?
- Have you tried reading / recruiting from non-standard backgrounds? If yes, would be curious to read about your experiences!
Thanks Lovkush, I shall read that paper.
Thanks for adding your demographics, but for future commenters, it's not necessary. Having data on demographics is important for large groups since it informs dynamics and biases, but that information is much much less useful when trying to draw conclusions about individual people.
I study "moral science" myself, which is 50% philosophy and 50% various social sciences including economics, psychology, sociology, etc. I used to run EA Ghent until, today actually. I recruited mostly philosophy and moral science students, since I knew them the best. But using 'quant people are more interested in EA, therefore it's more relevant for EA' as an argument is a bit strange since it's assuming the conclusion (not that you were making that argument, that's just one interpretation). Perhaps the methodology baked into EA is suboptimal, and EA has mistakenly become unattractive to people who would be high impact.
One problem I notice between these different research areas/different researchers, is the use of different methodologies. I used to also be pretty much die-hard pro-quantifying costs/benefits, but have become less so as I learn more about other disciplines. For example, my latest blog post is about how we managed to enshrine animal welfare in the Belgian constitution. How do you quantify that? Well, you don't. The effects are so broad, multifaceted and indirect that they're impossible to quantify. With things like medical interventions, we can run an RCT (which the EA framework loves), but the same cannot be done with constitutional changes since we don't have a "control Belgium". RCTs are great, but they also have drawbacks. They are expensive and measure narrow, direct, continuous effects, while they're unpractical for broad, indirect, or discontinuous effects. See, e.g. "the problem of marginalism" https://eprints.gla.ac.uk/289530/1/289530.pdf
But obviously, changing the constitution has a big impact; just because we can't quantify it doesn't mean it doesn't exist. And that's where other methodologies come in. E.g. the historical method will tell you that yes, changing a constitution has indeed had big impacts on the history of countries. It can tell us about big trends, while RCTs can tell us about tiny, discrete trends (which, combined with the cost, does mean that RCT-based interventions privilege the status quo more). Focusing so much on quantification can lead us towards the McNamara fallacy: https://en.wikipedia.org/wiki/McNamara_fallacy
Interesting post. I've skimmed over some sections, but I think I'll be reading it again soon. Thanks for sharing. However, I did notice it mostly doesn't apply to what I wrote. Let's assume his premises and methodology are all correct (I haven't checked, it's not peer reviewed, and he's not an academic, so that's a bit bold, but why not). He draws the conclusion that any given economics, sociology, education, and "other" are good at replicating, and social psychology, political science, criminology, psychology, evo psychology, cognitive psychology, management and marketing are bad at replicating.
Now let's look at the disciplines I mentioned: economics, sociology, anthropology, gender studies, geography, political science, and history. As you can see, I didn't mention psychology, partly because I'm well aware of the problems in that field (and in fact, my post shows that psychology is worse at citing other sciences than economics), and partly because I only consider it a semi-social science. I didn't mention marketing because I know the problems it has, and I didn't mention management because I didn't even think of it (but wouldn't have included it if I had thought of it). That leaves us with economics and sociology, which he says are good, and political science, which he says is bad. Okay, what about the others I mentioned: anthropology, gender studies, geography, and history? They're not part of this paper, or if they are, they're part of "Other". If it's "Other," it's mostly good, and if it isn't, then it mostly doesn't seem to apply to what I wrote. And importantly, these are disciplines that are much better at addressing his seventh point; they do create more theory.
Also, this is not how almost anyone actually reads about other disciplines. Mostly, we read books about a topic that only show a couple of the best conclusions the field has reached in the past decades, aka mostly stuff that has been replicated. A given criminology paper has a low chance of being replicated. So what? Focus on reading the papers that were replicated; that's what I would recommend anyway.
Thanks for detailed reponses, Bob!
- I agree with you that fantasticanachronism post does not challenge the underlying methodologies, tools or reasoning styles, or the strongest parts of the fields.
- "Perhaps the methodology baked into EA is suboptimal, and EA has mistakenly become unattractive to people who would be high impact." Always interesting to consider what the blindspots are of EA! Do you already know styles of reasoning that are missing in EA?
- "How do you quantify that? Well, you don't.". "E.g. the historical method says changing a constitution has had big impacts". I use 'quantify' in a broad sense to include things like "what percentage of historical constitutional changes had big impact" (which is impossible to answer literally as stated, but is at least the kind of thing I'd like to get a sense of), rather than 'quantify = rct'.
- I dont think constitutional changes are 'obviously good'. In your case, my instinct is it is good (I am between vegetarian and vegan so yay for animal welfare progress!). But, policy/legal changes could have unexpected negative consequences - e.g. imagine this makes it even harder to do any building or construction because there is extra bureaucracy around the affect on insects living in the ground being constructed on.
- Contrary to the tone in your comments, EA is open to many kinds reasoning / quantification other than RCTs. One example is there are no RCTs to measure impact of x-risk work. Second example is that if you listen to EA (adjacent) podcasts (like 80k or ClearerThinking), you will be exposed to large range of ideas. E.g. big one I recently listened to on 80k is with Mushtaq Khan - highly recommended if you're interested in policy!
- I re-read your OP and my comments have already veered off track (but hopefully in a way that is enjoyable!). Tackling your OP directly, it is not clear exactly what your critique of EA is and precisely which problems from economics you think EA has inherited. E.g. you have stats about how economists undervalue multidisciplinary research, but I dont think EA suffers from that problem. My reading is you have two main claims. First, EA undervalues certain fields of social science - I think the best arguments for this would be to find examples of reasoning/methodology from those that are being missed by EAs. Second, EA has bad recruitment strategies which leads to under-representation of those fields and of various demographic groups. I think this is likely, but for me what would be interesting to know is how EA could improve their recruitment.
> Always interesting to consider what the blindspots are of EA! Do you already know styles of reasoning that are missing in EA?
Well, part of the problem is that I've been steeped in EA culture too, so I'm still in the process of figuring this out myself. The main thing I would say is to not use one methodology for everything; switch methodologies depending on what is appropriate for the subject. E.g. when trying to understand whether films have become more inclusive, we could grab EA's trusty regression analysis, and this will tell you something, but why not grab narrative analysis, since it's specifically designed for that kind of stuff.
> I use 'quantify' in a broad sense to include things like "what percentage of historical constitutional changes had big impact" (which is impossible to answer literally as stated, but is at least the kind of thing I'd like to get a sense of), rather than 'quantify = rct'.
Yes, an RCT is the most extreme example, but the broader point is that a lot of historical analysis is not quantitative; it's qualitative.
> Contrary to the tone in your comments, EA is open to many kinds reasoning / quantification other than RCTs. One example is there are no RCTs to measure impact of x-risk work. Second example is that if you listen to EA (adjacent) podcasts (like 80k or ClearerThinking), you will be exposed to large range of ideas. E.g. big one I recently listened to on 80k is with Mushtaq Khan - highly recommended if you're interested in policy!
The bulk relies on certain methods/disciplines. The fact you said quantifications is already pointing in that direction. What about qualitative research, like what most historians are doing? If you like black humor, read the origins of the term 'McNamara fallacy' for the types of things that can go wrong with overrelying on quantification.
> I dont think constitutional changes are 'obviously good'.
I don't know where that quotation comes from, but it's not from my comment. I said it obviously has a big impact, not which direction that impact will be.
>" My reading is you have two main claims. First, EA undervalues certain fields of social science - I think the best arguments for this would be to find examples of reasoning/methodology from those that are being missed by EAs.
Is the beginning of my post not evidence for it?:
>" When you scroll through the Effective Altruism Forum you’ll find lots of economic analyses.
There is a tag for Economics with 161 posts, as well as one for Economic growth (130 posts), Economic inequality (10 posts), Economics of artificial intelligence (28 posts), Welfare economics (17 posts), Tax Policy (21 posts), Markets for altruism (33 posts) and many other subcategories.
Now I really like economics and I love economic analyses, but I think it’s very surprising how little presence the other social sciences have. The Social science tag only has 27 posts and there is no tag for sociology, anthropology, gender studies, geography, political science, or many other social sciences.
When you look at the People page of the EA forum, you’ll find lots of economists (and entrepreneurs) but almost no other social scientists. You will also notice that the vast majority of the people on this page are white men. This has been this way for a long time; this is what the page looked like a year ago: "<
> "a lot of historical analysis is not quantitative; it's qualitative"
Can you give an example of a qualitative research tool that EAs are under-using? Making it concrete will make it easier for me to understand.
> "Is the beginning of my post not evidence for [EA undervaluing certain fields of social science]?"
It is some evidence, but not strong evidence. The same reasoning could be used to show that EA undervalues every possible field of study, which is not what you are arguing for (e.g. you mentioned previously you would not include management in your list). My question is why you believe that EA is undervaluing "sociology, anthropology,..." in particular, and a good way to do this is to point to specific reasoning or tools in those fields and how EAs could use it to further their goals.
To emphasise, I am not saying they don't exist, but it would be useful to have concrete examples. I'm also fine if you do not know a good example off the top of your head, but I'd also like to know if that is the case.
> "I said it obviously has a big impact, not which direction that impact will be."
First, apologies for misunderstanding. In many contexts 'impact' implicitly means 'positive impact' (e.g. 'I would like to have an impactful career'), so it was simple misunderstanding on my part. Second, why did you say that historical analysis can show constitutional change can have large impacts (but not the direction of impact)? Why is this useful?
> Can you give an example of a qualitative research tool that EAs are under-using? Making it concrete will make it easier for me to understand.
Sure! I already mentioned the historical method and narrative analysis, which are two big ones, but a specific one I'm currently looking at is action research, since it would allow us to do (prospective) pro-social actions while we research. (And again, I'm not advocating that we replace everything with them, just that we add them to our toolbelt)
> why did you say that historical analysis can show constitutional change can have large impacts but not the direction of impact?
I was merely careful in my post not to prematurely claim that the constitutional change we implemented will be a positive one. The historical method can show us the direction of impact, but only when paired with a moral framework. So for example, the historical method can shows us which policies lead to better lives for animals, but whether that's a good direction depends on whether you use a moral framework that considers animals moral patients.
> My question is why you believe that EA is undervaluing "sociology, anthropology,..." in particular, and a good way to do this is to point to a specific reasoning or tools in those fields and how EAs could use it to further their goals. To emphasise, I am not saying they don't exist, but it would be useful to have concrete examples. I'm also fine if you do not know a good example off the top of your head, but I'd also like to know if that is the case.
I'll do you one better, I'll give you "explorable explanations" on social science topics I think you will find interesting. These are brief, fun, interactive, explanations of the topic:
Here's one on a topic in the field of computational sociology: https://ncase.me/polygons/
Here's two on topics in the field of election science: https://ncase.me/ballot/ http://polytrope.com/district/
Here's one on a topic in the field of network science: https://ncase.me/crowds/
This is great! Thanks very much. Don't have time now to look into details but appreciate the back and forth. I hope someday you'll be comfortable identifying as EA again. Seems like you have a valuable perspective to add.
Independent of our discussion, I found this comment from Will Macaskill, which is relevant. https://forum.effectivealtruism.org/posts/jSPGFxLmzJTYSZTK3/reality-is-often-underpowered?commentId=KvYM2XPG7mD7aFvv6
I don't think we should update too hard on this. He states his viewpoint that EA does a lot of qualitative research when relevant, and then links one case study by an EA as an example to back it up. Which in this context is totally fine and normal; it's a comment, not a post or study... but it's still only one example.
Hmmmm as the person who created the "Tax Policy" tag - it might just be a tagging problem rather than the posts not existing. I probably had to put in an hour or so finding relevant posts and writing up the wiki.
It sounds like you might be a good person to do the same for the social science tags you suggested?
Also increasing the number of women in EA, I ran created the London EA Women and NBs group chat and ran monthly meet ups for about a year. I don't think direct "recruitment" of women would work, it's more about encouraging people (especially woman) to lead and build the EA they want to see.
Related: https://forum.effectivealtruism.org/posts/Pz7RdMRouZ5N5w5eE/ea-should-taboo-ea-should
I didn't suggest "direct recruitment of women" (nor did I say "EA should"), I suggested "reading and recruiting more from other disciplines too."
I've had incredibly bad experiences with the tag system. When I removed Elon Musk and SBF from the EA people page with a comment explaining why (*after* the FTX scandal), it got downvoted, and I had to ask a friend to upvote it. When I added a tag for "abuse" and wrote a whole wiki-page with psychology literature and citations, it got downvoted and removed with the excuse that EA didn't deal with that topic. I had listed numerous articles that the tag could be added to, and even now when you search "abuse" on the EA forum you'll get 544 post results. I personally suspect the real reason it got deleted was that among the posts I wanted to add the tag to was the post "An Exploration of Sexual Violence Reduction for Effective Altruism Potential" by Kathy Forth, an EA member they'd prefer people forget about (because of the assault and subsequent suicide).
I also shouldn't be the one to make the tag, since I left EA after the manifest scandal made it clear the EA community would continue to deplatform leftists while platforming fraudsters and race "scientists" and the like. I really should have left after my years of unpaid and unacknowledged labor for EA got rewarded with a high profile EA publicly, indirectly (and falsely) accusing me of doxxing people, without him facing any repercussions, but I kept on going until the day the manifest people said they would invite some of those "scientific racists" to speak again next year, which was the straw that broke the camel's back.
Hmmm I don't think this is entirely fair. I haven't met anyone in EA (other than young students) who don't read outside of EA and idk I think you are suggesting that more funding / volunteer time should go to speculatively finding people in other fields.
Tbh I would agree with their editorial decision on a tag called "Abuse", from looking at the 544 articles it's too wide ranging and could include Animal Welfare, Domestic Violence, Corruption, Political suppression of women in misogynistic countries, Criticisms of EA culture, Reducing risks from malicious/abusive actors etc. I doubt they'd have an issue with a Sociology tag based on their Wiki FAQ https://forum.effectivealtruism.org/topics/ea-wiki-faq
I'm not sure I'd agree that Manifest is EA - maybe EA-adjacent.
Who are you expecting recognition from? Idk but I'd say being able to donate to effective charities and work directly on pressing problems is a massive privilege and there's a responsibility for privileged people to use it to improve the world.
I think the EA forum is a useful resource for working out the best ways to do that, so I want to contribute as I found it helpful, but honestly doesn't owe me more than that :/
> who don't read outside of EA
Look at what I wrote: "reading and recruiting more from other disciplines too.", emphasis on the "more" aka I didn't claim that they don't read outside of EA (in fact it wasn't even about EA, it was about economics).
> I would agree with their editorial decision on a tag called "Abuse"
You are agreeing with them without even knowing anything about what I had written. I made it very clear in the article what form of abuse it was for. I had cordoned off all those different versions of abuse, e.g. on the wiki article I wrote that for animal abuse people should use the animal welfare tag, for abuse in EA use the criticism of EA tag etc etc. (I didn't say those 544 posts all would've qualified, I merely pointed to it to show their excuse didn't hold any water)
> I'm not sure I'd agree that Manifest is EA
One of the Core Topics on the EA forum is Forecasting and when you click on it manifold right there on the sidebar and it's also on the wiki article itself. Also manifest was promoted on the forum, manifund has been given funding by the EA community, and prominent EAs were there at manifest.
> Who are you expecting recognition from?
Read it again, I didn't expect recognition, I expected that "years of unpaid and unacknowledged labor for EA" not get "rewarded with a high profile EA publicly, indirectly (and falsely) accusing me of doxxing people, without him facing any repercussions". That's a very different thing.
> Idk but I'd say being able to donate to effective charities and work directly on pressing problems is a massive privilege and there's a responsibility for privileged people to use it to improve the world.
Donating to effective charities and working on pressing problems is a great thing, and just because I'm no longer in EA doesn't mean I won't continue to do so, that's a weird implication (also working on pressing problems is more of a privilege if you get any money or help doing it, otherwise the privilege is in the prerequisite socio-economic conditions and not the work itself). I would say people should do that, but that EA is not the best *way* to do it.
I was perfectly fine working and donating all those years to EA, until I learned that it was going to chateaus in england and czechia, and used to promote crypto-scammers and "race scientists" (not just with the manifest thing, but also the (reaction to) the Bostrom email, FLI offering funding to a far-right foundation that, among other things, promoted holocaust denial, and the list goes on). An EA scamming me on the forum without any repercussions (a labor scam, not a crypto scam), and then after that the doxxing thing also didn't help my enthusiasm. Not to mention the EA forum soft-censoring things: https://substack.com/@bobjacobs/note/c-68603515
EDIT: I do believe in the things I wrote here, but the tone was overly harsh. Sorry Gem!
I'm feeling a bit frustrated by your reply here as I don't think it is a charitable reading of what I said and doesn't seem to reflect on the trade off of any community so I'm going to disengage.
Let's agree to disagree.
Sorry for making you feel frustrated, that wasn't my intention.
Do you think the reason that EA has those issues is because it’s an offshoot of economics (so just gains its biases? I’m a little confused on the story here I guess) or that there is a hidden third thing correlated with both (perhaps the type of analytical mindset tho it could be different or a bunch of things) that both economics and EA shares?
I think it's a combination of two things:
1) It started from charity analysis, where economists/economic tools are a natural default. Due to demographic lag and the founder effect this remained the default, even after EA started to become interested in things like bio-security, nuclear security, policy etc
2) The large overlap with the rationalists community, who are more hostile to the other disciplines because they are more "woke/skeptical of capitalism/lefty".
Demographic networks and the founder effect are problems every movement/community has to deal with, but there are some structural features which worsen it for EA (e.g. the groupthink inducing karma system: https://substack.com/@bobjacobs/note/c-68603515 ) Though I do expect that, as with other movements, it will slowly decrease over time.
The minimum wage example here isn’t a strong point in favor of your main argument since the “it doesn’t dramatically increase unemployment” results, both theoretically and empirically, came from within economics, not outside of it. In fact it kind of shows the opposite - Econ still doesn’t need non-Econ to question itself (and frankly I’m not sure what existing non-Econ could contribute to this debate constructively, which is different than what non-Econ could possibly contribute if it itself wasn’t so hopelessly muddled)
> came from within economics, not outside of it
No, it came from both within economics and outside of it (e.g. sociologists like Rubery)
But even if you don't like this example I can give you others:
Psychological research showed that the premises of neoclassical models, e.g. that of the homo economicus, were empirically wrong. See for example the work of Kahneman and Tversky
Economic history has contributed massively to economics, and you can thank those historians for a lot of those nice datasets economists rely on. Often the historic data contradicts economic models. See for example the work of Robert Allen.
Anthropologists have also contributed to disproving economic models. Such as James Scott showing peasant economies are not driven by profit maximization but by subsistence security.
Political science has contributed to economics by showing that political institutions, not just market forces, drive economic development. (e.g. How Nations Fail)
Similarly, sociology has contributed with concepts like 'social capital' (e.g. Bourdieu) that e.g. show how education impacts economic outcomes beyond what economic theories could predict.
And this is only how other social science have contributed to economic *models*, that's not even talking about how other social sciences are necessary to discover and document the harms of markets (and thereby are necessary for analyzing the long-term trajectory of markets). The most classic example is greenhouse gasses, where we need to listen to other social sciences (like climate policy experts) to find fixes for the economy.
> what non-Econ could possibly contribute if it itself wasn’t so hopelessly muddled
I hope this is a joke, or perhaps an accidentally misphrased sentence, because I would lose a lot of faith in humanity if people actually think economics is the only social science that has contributed to our knowledge.
Why is Economics so insular? Is it a feature of the historical development of the field, for instance the forefathers of the field moved towards treating it like a pure discipline rather than a social science?
It could also have to do with the study itself. Economics seems focused on creating mathematical models for how a society arranges resources. This is a daunting task, which is made easier by reducing people into rationalist agents, and any attempt to increase the complexity of these agents likely increases the computational complexity. Therefore a lot of the research from psychology or sociology likely create more accurate models, but with greater complexity. Compare behavioural economics to traditional economics.
Another question surmises in terms of the demographics. I want to see if the demographics are an artifact of the history of Effective Altruism, a secondary effect of its strong ties to Economics, or if different demographics have cultural influences that reject the main philosophy of EA. For instance, in cultures that are more status-oriented, like South-Asia, I wonder if EA can't gain any traction because of its disregard for those status games.
> Why is Economics so insular?
I think it's mostly the factors you mention, which are then exasperated by the hyper-competitiveness of getting job security in the field, plus a comparatively small amount of "top" scientific journals for economics (which all favor model-heavy approaches). I remember reading an economics book that at one point applied economic insights to the field of economics itself and concluded that the reason math and math-heavy models became so prominent is not because it increases the accuracy, but rather because it's a way to signal intelligence. I can't for the life of me remember which book it was though.
> For instance, in cultures that are more status-oriented, like South-Asia, I wonder if EA can't gain any traction because of its disregard for those status games.
Yeah maybe, though it probably also doesn't help that EA is very reliant on funding from only a handful of funders from the anglo-american world. Those funders only read English-language things, run in anglo-american social circles, and, like every human being, has biases towards their ingroup. If we look at the startups EA funds it is *heavily* skewed towards ones from the anglo-american world, so it might also be that the South-Asian EAs just have more trouble securing funding.
I'm interested, what would be the alternative to a math-heavy model? Computational?
I don't know if the math-heavy models are more prominent because people try to signal intelligence, or if people in other fields which are considered intelligence featuer math, and that bleeds over to economics. For instance, iirc physics has some overlap to economics, and I think people would generally assume physicists are smarter than, say, a sociologist, so a physicists' approach to economics likely has more credence than a sociologists. Maybe it's not as malicious, but it leads to the same outcome.
> Yeah maybe, though it probably also doesn't help that EA is very reliant on funding from only a handful of funders from the anglo-american world.
Yeah I think that's where I'm leaning towards personally. I don't know if EA would push towards more diverse funding metrics though. In my (biased) view, EAists tend to disagree with DEI, so the idea of allocating more funding to EAs in other regions likely won't ever get traction, compounding the issue of making funding more difficult to secure, creating a vicious cycle.
Thank you for this article though, you gave me a lot to reflect on.
> I'm interested, what would be the alternative to a math-heavy model?
Empiricism. There's nothing intrinsically wrong with math-heavy models, but you only get out what you put in. So if you make complex models of the homo-economicus you can generate many papers describing the results, but... well... so what? We're not the homo economicus so it's not really useful. It might be better to just focus on empiricism, at least for a while. I think we should step out of the armchair and go out to collect data: observe what's happening in the real world, and see if patterns fall out of that data, instead of describing the result of math formulas based on premises we know can't be applied to the real world.
To be fair this shift is already happening a bit, as the graphs I presented seem to indicate. Economics seems to especially start paying attention to the demographic-data sociologists are collecting, so I expect economics to become a more integrated discipline in a couple of decades.
> I don't know if the math-heavy models are more prominent because people try to signal intelligence, or if people in other fields which are considered intelligence featuer math, and that bleeds over to economics. For instance, iirc physics [...] Maybe it's not as malicious
It's not malicious (well except for that one economist I quoted), it's a race to the bottom. This phenomenon of wanting to be like physicists actually has a name: it's called physics-envy: https://en.wikipedia.org/wiki/Physics_envy
> EAists tend to disagree with DEI, so the idea of allocating more funding to EAs in other regions likely won't ever get traction, compounding the issue of making funding more difficult to secure, creating a vicious cycle. Thank you for this article though, you gave me a lot to reflect on.
You're very welcome, glad you liked it!
I think it's not so much a principled opposition to DEI as a desire to get funding. Everyone wants themselves and their contacts to be doing well so everyone is willing to take a little bit of the pie to ensure that. I think most EAs would be fine if a lot of the rest of the pie goes to people outside of their circle, but they'd very much like a slice of the pie for their circle first. Of course, if everyone does that not much remains.
I am new to the Substack community and therefore astonishingly late to this conversation, but I wanted to engage re: 'the insularity of economics'.
For context, I am currently in the middle of my PhD in Economics. The simple reason I observe significant disdain for other social sciences within Economics is that their methods of analysis are either not empirical or not causal empirical analysis, if they are empirical at all. This critique is magnified by the fact that most economists truly are interested in positive rather than normative evaluation, and where positive questions are posed, solid, empirical answers are critical.
In contrast, other social sciences are in the throes of serious replicability crises because many of their empirical results were never identifying real empirical phenomena (instead, strong correlations and associations) and even many experimental results were so shoddily designed (or fake) that they don't stand up to replication. Then, we have the social sciences that simply posit theories and armchair reason from the top of the ivory tower whose findings are largely irrelevant to the average person and policymakers.
To be clear, Economics suffers from some of these critiques too. Many Economics papers require assumptions that barely apply to superhuman AI minds, let alone a normal person thinking two seconds about which pen to buy at a supermarket.
However, Economics has also produced real, meaningful results within its more methodological fields like Econometrics (a cutting edge field that has practically revolutionized the quality of empirical evidence across multiple social sciences), its theoretical fields like Market Design (overhauling the medical residency matching market, internet auctions, and kidney exchange), and its applied fields like Behavioral Economics (nudge theory and the economics of charity).
So a large part of the appeal of Economics and why we doubt other fields is about the quality of evidence and results.
Now, there are other huge problems within Economics, and I don't really care to defend those. I would just be wary that as you invite other fields and consider their insights that you apply the same rigor of evidence to them as one does in Economics: rigorous RCTs in experiments, causal analysis in empirics, and validated predictions in theory.
Thanks for the comment. I think I do disagree on some points though:
First of all, I don't think that "empirical" = exclusively quantitative. Qualitative analysis answers questions that quantitative work simply can’t touch, at the very least not right now. It's important to have e.g. a boots on the ground ethnographer, or an historian going through the archives if you want to know about the “how” and “why” of mechanisms, meanings, and institutional evolution, since experiments or regressions cannot answer that on their own. I think this "ivory tower" accusation can also be lobbed at the economists, maybe actually moreso than some other social sciences like ethnography. In any case, triangulating across methods usually produces more reliable knowledge than insisting on a single gold standard.
Secondly, while you acknowledge some of the problems in social science are present in economics, you seem to imply the "replication crisis" isn't one of them. I think this isn't true: coding errors, hidden degrees of freedom, and garden-variety p-hacking are very much present in flagship economics journals. More importantly, the successes of economics often come from other disciplines, like mathematics/game theory (in the case of matching theory) and cognitive psychology (in the case of behavioral economics). I don't say that as a dunk against economics, this *should* be how science is done, I'm saying this to point out how interdisciplinary approaches have led to some of economics biggest successes and we should thus probably be doing more of it.
Now as an ethicists I would like to make a minor point that normativity is inescapable, both in a "foundations of science/knowledge" kinda way, and in a "understanding society" kind of way. I don't believe economists are *actually* uninterested in normative evaluation, and I don't *want* them to be uninterested in it since the normative aspect is so important. But setting that aside, I think other social science are at least as interested in sticking to to the descriptive over the normative as economics is. I mean, it's even the central creed of linguistics, so at least there, I would contend, they've been much moreso than the economists.
Lastly, other social science disciplines have actually made *huge* contributions to social science. I already mentioned cognitive psychology, but there are many more:
I'm a voting theorist, which may or may not be a branch of philosophy, but at least the overarching field of election science is a social science, and it made huge contributions to electoral design. Is that a subdiscipline of political science? Probably? The edges are blurry, but what is definitely a subdisciplines of political science that made huge contributions is public opinion research, where modern methods like MRP and policy-tracking in general are now the cornerstone of a lot of other social science research.
Same for sociology: I already mentioned computational sociology in another comment, but there are other sub-disciplines too. Demography for one is hugely important for all the other social sciences, and I consider ignoring demography for so long one of economics' biggest blunders. But also things like 'social network analysis', which has given us a whole new toolset to study so many things (from job hunting, to friendship) that ordinary regression ignored. It has also helped in the subfield of social epidemiology, which has proved so vital during covid and beyond.
I could go on and on; climate economics needs geography/earth-system science, labor economics needs sociology, education economics needs pedagogy, behavioral economics needs psychology etc etc... Ignoring adjacent fields can slow recognition of key mechanisms (like what happened with social capital and institutional path dependence) that matter for understanding how to increase global welfare.
And tbh, I think I might actually secretly like economics the most of all the social sciences, which is why it's especially heartbreaking to see it shoot itself in the foot so much.
Yes! I agree on the value of interdisciplinary approaches. I am a matching theorist myself and realize that what makes Economics great is that despite its elitist tendencies, when it does incorporate other fields, it does so in a way that typically improves and adds knowledge that benefits the domain of Economics and the domain of the intersection. Publications in top journals usually reflect this: they often require strong applications that might not fit the mold of Economics while also contributing general insights to some economic phenomena.
I will disagree with and steelman your critique about interdisciplinarity at the same time. I think Economics has very strong engagement with some fields. You named a few: political science (political economy), network science (network economics), and graph theory (matching theory). Heck, my own research touches social choice theory as well. If I decided tomorrow that I wanted to be a voting theorist, my advisors might look at me a little funny, then say, "Whatever, go for it."
However, you are right that there are some empirical methods that economists won't touch with a ten foot pole. Ethnography is one. There is probably room for some econometrician to figure out how to formalize that as a standard of evidence, to be honest, and that would be fascinating and valuable.
I don't want to get into the weeds too much, so I will just note that I'm generally sympathetic to your points about normative analysis and contributions of other fields.
My biggest real disagreement is about the replication crisis point.
Yes, there are papers that have been demonstrably manipulated in some form or another. Unfortunately, as it's not easy to identify statistical gaming in research, I can only speculate as to its commonality across fields. My point wasn't that Economics deals with less of the replication crisis, but that it's, at the very least, harder to publish papers that might contain invalid conclusions even with honest researchers. For example, it is nigh impossible to publish an experiment that is not an RCT or an applied paper that has only correlational rather than causal analysis in any top Economics journal. In contrast, both of these would fly fine in most sociology and psychology journals, then a plurality of correlational studies get compiled into a meta-analysis that is somehow supposed to amplify the rigor of the evidence. Unfortunately, aggregations of correlations are still just correlations. I would not have a problem with this if it were widely acknowledged, but oftentimes social sciences grade themselves on a curve and take the 'best available evidence' as 'good evidence'. The two are not equivalent.
It doesn't mean that we shouldn't do these types of studies or that other social sciences are not valuable. It just means that what we can learn from some papers is limited, and, as you rightly state, a plurality of methods is necessary to learn the truth.
> I think Economics has very strong engagement with some fields. You named a few: political science
I disagree for reasons cited in my post. As Fourcade et al found, economists were the only group that thought interdisciplinary research was worse than research from a singular field, and economists cited political science way less than the other way around (17.5% vs 0.8%, that's a huge difference!). Now this is from a while back so there might be a paper out there I haven't seen that shows things have improved. In fact, I would expect it to have improved given that Angrist et al showed a trend of it improving. However, since it started off at such an incredible lowpoint I'd still expect it to be low overall.
> For example, it is nigh impossible to publish an experiment that is not an RCT or an applied paper that has only correlational rather than causal analysis in any top Economics journal. In contrast, both of these would fly fine in most sociology and psychology journals
That may be true for sociology, but I'm not sure it's true for psychology. I think contemporary top psychology journals are also wary of purely correlational studies. But take this with a grain of salt since psychology is not my field so it might be that some top journals are better than others and I just so happen to read the good ones. The only reason I think this is true for sociology is that it doesn't seem to *have* "top journals" in the way economics does. This has the drawbacks already mentioned, but it also has some upsides. Suicides seems to be very common in economics, even for some of the most accomplished economists (e.g. Weitzman, Krueger, Farhi). It seems like because economics is so much more hierarchical than sociology, it's also a lot more stressful/cut-throat. The infamous "top 5" can be a career maker or breaker, regardless of the quality of the papers you write. Now if the "top 5" really were an objective measure of who was the best in the field then arguments could be made that this might be worth it. But that doesn't appear to be the case: https://www.aeaweb.org/articles?id=10.1257/jel.20191574
As so often when people represent their strict hierarchy as natural and necessary, it appears this hierarchy also reflects ingroup bias and what we in dutch call "vriendjespolitiek" (aka helping your contacts and underlings and punishing people who speak up against you): https://showmethemath.org/2022/01/heckman-judd-war-ii/
And it's not just nepotism within journals we have to worry about, the same happens within universities: https://edworkingpapers.com/sites/default/files/ai20-324.pdf
This is of course very bad for people outside of the ingroup who want to enter it, which is what the whole second section of my post (on the strong bias against women and minorities in the discipline) was about.
Interesting thoughts - perhaps my institution is unique, but we have a strong bend towards interdisciplinary approaches so I have not any anecdotal experience with economists that frown on them. I will reserve judgment on that till more updated studies are published.
The tyranny of the top five is indeed a huge problem, but it is in some ways subordinate to the "lust of the top ten". The article you cite writes, "The observed pattern of publication behavior suggests that the number of T5 publications required for tenure decreases with department ranking." Economics can emphasize the worst in high-performing stardom seekers which I do think has serious ethical problems. No argument from me there. Nevertheless, the high research standards are not limited to just the top five. They are prevalent across every journal that the cited article presents as influential.
I do have an axe to grind against correlational studies, to be fair, and not just in social science. I think that researchers should be a lot more careful about how studies get used in public policy recommendation. My biggest grievance is early recommendations to avoid peanut exposure in infants---following which a great number of peanut allergies emerged in the U.S.---and subsequent RCT findings that, actually, early exposure is crucial to prevent allergies.
Thanks for the comments! This was a good read and engaging discussion.
You too, always love to get comments from insiders.