Skepticism About MIT’s Gender Balance Win in MechE

If it really is true, as MIT suggests, that the gender ratio in their department is convincing a substantial percentage of women to enter MechE who otherwise would have avoided a STEM field its a big deal. However, upon reflection there are some aspects that are troubling.

First, as the article suggests, they engage in fairly extensive recruitment and some degree of affirmative action for female students in STEM fields. This calls into question the existence of any such effect as for all we know MIT is just recruiting women interested in MechE away from other schools. Indeed, even just considering the benefit MIT is suggesting (women are more attracted to programs with a reasonable gender balance) one should expect MIT’s efforts here to be worsening the gender balance at other schools like caltech

But if you really believe that gender imbalance both makes life worse for female students and repeals them from the field it seems downright irresponsible to attrach female MechE majors from other schools (without a better understanding of how these effects work). If, as seems quite plausible, the discomfort (and willingness to drop out/not major) is most extreme when the percentage of women is the least (e.g. superlinear as percent goes to 0), then this could be a substantial net harm as the gains from greater gender equality at MIT are more than offset by the decreased gender equality at other schools. It all depends on the specific numbers but its concerning that people seem convinced this is a good thing without even having an intuition about the size and direction of this cross school interaction.

Before anyone applauds these results we really need some good studies checking that MIT’s efforts really are bringing more women into MechE. I hope they are but I fear that they may be doing the exact opposite. If I had to guess I’d bet that any positive effect of gender balance is offset by the fact that MIT is harder/more competitive than the other schools who would otherwise get many of the women MIT recruits and I expect the harder/more competitive a science class the more likely people (of either gender) are to drop out to a less quantitative subject (but that’s just speculation).

Also, I’d like to know what people whether MITs affirmative action efforts create a situation in which men tend to noticeably outperform women. As much as I hated the huge gender ratio at caltech I very much appreciated the fact that they were obviously equals. Now, like everything else, what I appreciated isn’t what matters but it does seem like we should at the very least have a pretty firm grip on what kind of effects on subsequent attitudes affirmative action has before we praise the policy. Even if, this effect doesn’t appear at MIT right now (e.g. they most just steal girls from caltech and cmu) it might if more schools try to implement such a policy.

I find it pretty crazy when MIT is congratulating itself when they don’t seem to have any grip (or at least are hiding it) on what they are trying to achieve or whether their policies achieve it.

Now, of course, most social programs will depend greatly on priors and I’d be happy with a short little explanation about why they think the net benefit of achieving gender balance in their departments is worth the effect it has on other schools. Are they suggesting their policy would and could universalize and benefits would be seen from that? Some words about why would be nice. Also some words about why they have the intuition any blowback is worth the cost. As it is it kinda makes one feel like you are being scammed with a meaningless advertising statistic.

I think its quite possible MIT’s policy is net beneficial but I’ve yet to see any cogent account of why I should think that so if you have one I’d love to hear it.

As an aside I’d add that while I don’t think there is any inherint moral value in making sure men and women are equally represented in every discipline, only in making sure they are equally welcome and have equal access, but I do think there would be substantial societal gains to increasing the number of women in STEM fields. Not only would this make scientists happier (and less socially isolated and less likely to accidentally harass) but merely making it clear that quantitative, systematic thing oriented reasoning isn’t anti-female.

Evaluating Gender Bias Claims In Academia Part 1

Does The Data Support The Interpretation

For a number of reasons I think it’s vital that we have a good empirical grip on the reasons why different genders are over/under represented in various disciplines and at various levels of acclaim in those disciplines. There is the obvious reason, namely that, it is only through such an understanding that we can usefully discuss claims of unfairness and evaluate schemes to address those claims. If we get the reasons for under/over representation in various areas wrong we not only risk failing to correct real instances of unfair based treatment but also undermining the credibility of attempts to address unfair treatment more generally. This isn’t only about avoiding gender based biases but, more broadly, identifying ways in which anyone might face unjust hardship in pursuing their chosen career and succeeding at it1.

Also, even putting questions of fairness and discrimination to the side there are important social and cultural reasons to care about these outcomes. For instance, the imbalance of men and women in STEM fields both imposes personal hardships on both genders in those fields but also creates an excuse for dismissing the style of thinking developed by STEM disciplines. As such, identifying simple changes that could substantially increase female participation in STEM subjects is desirable in and of itself and similar cultural considerations beyond mere fairness extend to other fields. However, I worry that incorrect interpretation of the empirical data could lead us to overlook such changes especially when they don’t fit nicely into the default cultural narrative2.

Point is that I genuinely want to accurately identify the causes of gender differences in educational attainment and academic outcomes. One could be forgiven for thinking that we’ve already nailed down these causes. After all every couple of months one sees a new study being touted in the mainstream media claiming to show sexism playing a role in some educational or professional evaluation. Unfortunately, closer examination of the actual studies conducted often reveals that they don’t actually support the interpretation provided and everyone suffers from a misleading interpretation of the empirical data.

So, in an attempt to get a better picture of what the evidence tells us, every time I see a new study claiming to document gender bias or otherwise explain gender differentials in outcomes I’m going to dive into the results and see if they support the claims made by the article. While I can’t claim that I’m choosing studies to examine in a representative fashion I do hope that comparing the stated claims to what the data supports will help uncover the truth.

Gender and Publishing in Political Science

I ran across this claim that there is gender bias against female authors in political science in the wall street journal blog monkey cage. For once, the mainstream media deserves credit because they accurately conveyed the claims made by the study.

The study claims to show gender bias in political science publication based on an analysis of published papers in political science. By coding the authors of published papers the study gives us the following information about the rate of female publication.

Line A represents the share of women in the ladder faculty at the largest 20 PhD-granting departments in the discipline (27%). Line B represents the share of women among all APSA members (31%). Line C represents the share of women among all newly minted PhDs as reported in the NSF’s survey of earned doctorates.

The paper deserves credit for recognizing that this may reflect some degree of sorting by subfield and recognizing that sorting into subfield might falsely create the impression of bias even when none was present. However, any credit granted should be immediately revoked on account of the following argument.

However, gendered sorting into subfields would not explain is the pattern we observe for the four “generalist” journals in our sample (AJPS, APSR, JOP and POP). These four journals—official journals either of the national association or one of its regional affiliates—are all “generalist” outlets, in that their websites indicate that they are open to submissions across all subfields. Yet, as figure 3 shows, women are underrepresented, against all three benchmarks, in three of those four “generalist” journals.

The mere fact that these are generalist journals in no way means that they are not more likely to publish some kinds of analysis rather than others. As the study goes on to observer women are substantially underrepresented in quantitative and statistical work while overrepresented (at least as compared to their representation at prestigious institutions) in qualitative work. Despite the suggestion by the study authors to the contrary choosing, for valid intellectual (or even invalid gender unrelated) reasons, to value quantitative work more highly and publish it more readily doesn’t constitute gender bias in journal publication in the sense that their conclusions and ethical interpretations assume.

Line A represents the share of women in the ladder faculty at the top 20 PhD-granting departments in the discipline (27%). Line B represents the share of women among all APSA members (31%). Line C represents the share of women among all newly minted PhDs, as reported in the NSF’s survey of earned doctorates (40%).

Ideally, the authors would have provided some more quantitative evaluation of what part of the observed effect was explained by choice of subfield and mode of analysis. However, I think it’s fair to say based on the graph above that women aren’t so overrepresented in publications in qualitative areas for subfield preferences to explain everything so lets put the concern about subfield/analysis type based sorting to one side and return to the primary issue

This paper also deserves praise for recognizing that merely comparing the percentage of women in the field with the percentage of prestigious female publications will merely reflect the fact that past discrimination means the oldest, and most influential, segment of the discipline is disproportionately male. In other words, even assuming that all discrimination and bias magically vanished in the year 2000 one would still expect to find men being published and cited at a greater rate than women for the simple reason that eliminating barriers to female participation biases female representation to the less experienced parts of the discipline. By breaking down authors by their professorial rank the study is able to minimize the extent to which this issue affects their conclusions.

Percentage female authorship by professorial rank

Importantly, in the discussion section (and throughout the paper) the study makes it clear that it takes this result to be evidence of bias. The WSJ post was quite right in understanding the paper to be alleging gender bias in publication. Yes, the study doesn’t claim to decide whether this bias is a result of female authors being rejected more frequently or female authors being less likely to publish in the most prestigious journals but in either case it assumes that the ultimate explanation is pernicious gender bias.

The paper also explores the issue of gender based coauthorship and the relative prevalence of papers with all male authors, mixed gender etc.. etc.. These patterns are used to motivate various speculations about the fears women may face in choosing to coauthor but the complete lack of any attempt to determine to what extent these patterns are simply the result of subfield and analysis type preferences, e.g., quantitative and statistical analysis might lend themselves more frequently to coauthorship, and the relevant percentages of women in those fields undermines any attempt to use this data to support such speculations. While I believe that female scholars do face real concerns about being insufficiently credited as co-authors the ways such concerns could play our are so varied that I don’t think we can use this data to draw the conclusion the study authors do: women aren’t benefiting equally from trends toward coauthorship. However, I’m going to set this issue aside.

Political Science Hiring Biased Toward Women?

At this point one might be inclined to think this paper should get pretty good marks. Sure, I’ve identified a few concerns that aren’t fully addressed but surely it makes a pretty good case for the claim of gender bias in political science? Unfortunately, that’s simply a mirage created by thinking about the data in exactly one way. Notice that one could equally well use the same data and analysis to draw the conclusion: Women Hired in Political Science Despite Fewer Publications. After all the way one gets professorial jobs is by publishing papers and this data suggest that women at the same professional level have less publications than their male colleagues.

Now I think there are multiple plausible ways of resisting the conclusion that this data shows a bias in favor of women in hiring. For one, if past discrimination means that men and women at the same professional level haven’t had the same amount of time to right papers (e.g. women are more likely to have just got the job) then the conclusion is suspect. For another, one might point out that not all the jobs given the same professorial rank in the study are really equivalent. There are further reasons to doubt these conclusions, but each and every reason equally well undermines any support this data provides for claims of gender bias.

Ultimately, I think it’s safe to say that while this study shows that women publish in influential journals at a rate lower than their representation in the political science profession would suggest it does little to identify a cause. If you came into this with the prior that said: the reason women are underrepresented in political science is because they face bias and other obstacles you’ll explain this effect in terms of bias and obstacles. In contrast, if you came in with the prior that said: the reason women are still underrepresented in political science is because of gender related differences in ability/interest (which need not be negative it could as well be a greater affinity for some rival career option) then the data are perfectly compatible with women gravitating towards more qualitative less rigorous aspects of the profession and putting greater focus on teaching and other aspects of the profession that don’t result in publications.

Frankly, I don’t know enough about political science to have much opinion on this point one way or another. However, I do think we can safely mark this study down as misleading at least insofar as it is cited as further evidence of gender bias against women. Don’t get me wrong, I think that is a very plausible interpretation of the data but I’m just sharing the bias I came in with rather than being persuaded by evidence.


  1. For instance, differences in male/female performance might justify studying ways in which standard pedagogy favors/disfavors particular learning styles. Accurate empirical data on this subject, regardless of what it shows about gender, lets us correct the ways in which the current system may be unfair to those with particular learning styles, e.g., consider the recent evidence about how mandated attendance can actually hurt performance by those who don’t find the lecture component of a course useful. 
  2. For instance, I worry that it is precisely women’s better performance in high school mathematics and generally greater willingness to approach subjects as desired by their high school (or early college) teachers rather than going their own way which is responsible for some of the observed disaffinity towards studying higher mathematics among women. Ideally, one would teach mathematics by merely communicating the underlying ideas and allowing students to use their conceptual understanding to solve problems. However, few students have the interest and ability to, say, use their conceptual understanding of the derivative to find the maximum value of a given function and the educational system is unwilling to abandon the idea that almost all college freshmen should be able to solve such problems. As a result lower level mathematics is forced to adopt a formulaic approach the favors rote memorization of algorithms meaning that gaining real insight and experiencing mathematics as an enjoyable puzzle often requires rejecting the approach seen in the classroom and working things out on one’s own. I worry that we lose many women who might otherwise be interested in mathematics simply because they are more devoted to working within the framework they are given but because this is largely seen as a positive value it gets neglected as a potential problem. If true, it might be that simple interventions like explicitly encouraging students to deviate from the rote rules being taught if they understand enough to do so could make a big difference.