In order to understand the extent to which different genres are unequally represented among books that are reviewed and written by men and women, we conducted an initial study using data from the New York Times Book Review. We used Goodreads to match the books reviewed to a set of genres, which we used to create this analysis. You may download the full report on this data analysis here.      

Theme	Women (#)	Women(%)	Men(#)	Men(%)	P-Value	Skew
Gender	166	64	92	36	< 0.001	F
Romance	266	47	304	53	< 0.001	F
Government	50	16	254	84	< 0.001	M
Family	79	59	54	41	< 0.001	F
Economics	15	13	102	87	< 0.001	M
Science	168	25	504	75	< 0.001	M
Humanities	23	16	120	84	< 0.001	M
Youth	236	42	325	58	< 0.001	F
Magical Realism	33	57	25	43	< 0.001	F
Spirituality	36	21	138	79	< 0.001	M
Sports	9	13	58	87	< 0.001	M
War	47	22	166	78	< 0.001	M
Adventure	58	23	192	77	< 0.001	M
Business	18	18	81	82	0.001	M
Sexuality	37	47	41	53	0.011	F
History	289	29	693	71	0.013	M
Crime/Mystery	132	29	328	71	0.042	M
Arts	88	29	224	72	0.071	M
Fantasy	135	37	227	67	0.094	F
Comedy	53	29	132	71	0.212	M
Environment	57	38	93	62	0.225	F
American	36	39	57	61	0.271	F
Social Science	62	29	149	71	0.273	M
Military History	8	24	26	76	0.277	M
Health/Body	19	27	52	73	0.313	M
Education	10	38	16	62	0.54	F
Leisure	48	31	108	69	0.553	M
Self-Help	13	37	22	63	0.595	F
Race	8	28	21	72	0.694	M
Asian Literature	9	28	23	72	0.708	M
European Literature	63	33	129	67	1	M
Dark / Death	41	33	84	67	1	M
Social Issues	8	32	17	68	1	M

In order to understand how topic bias may be manifesting in book reviews, our research team felt it was important to first understand how publications select the books they review. Our hope was that having a better sense of this process would help us identify where topic bias may be creeping in and who might have the power to address it. As such, we designed a survey that would give us insight into the book selection process. We sent the survey out to 97 publications and received 29 responses. Here’s some of the key takeaways from the response data:

1. Publications do care about diversity. Two thirds of publications value having a diverse range of authors represented in their reviews section and one quarter of publications actively look for diverse authors when selecting books to review. This means that at least a quarter of publications should be open to combatting topic bias.

2. Section editors have the power to make change. Section editors (i.e. arts editors, books editors in newspapers/magazines), followed by reviewers, have the most influence over which books get reviewed. Advocacy around topic bias should then focus on these two targets.

3. Publications already have useful book review data. While most publications do not have standardized selection processes – meaning biases can easily creep in – most publications (55.2%) do track the books they review, and some publications (38.3%) also track all the books they receive for review.

Publications have the motivation, the means, and some of the data they need to make change. What they’re missing is actual knowledge of their own topic bias. This is where we come in: Just Review has developed a self-assessment tool that publications can use on the track data they already have from tracking their reviews to assess the level of topic bias in their publication. If section editors can determine where topic bias is occurring, they can take measures to stop it. For the full survey results, see here: Survey Data Summaries and Conclusions.

To gain further traction on the ways in which the content of books chosen to be reviewed differs between male and female authors, we also decided to examine the language within the reviews. To test if there were biases in the language of reviews towards men and women, we decided to train a classifier and extract words it found were most deterministic when deciding when a reviews was written about a female author or a male author's book. For those unfamiliar with the term classifier in machine learning, it is an algorithm whose task is to attach a label, in this case author gender, to an input, in this case the review, by determining to which previously trained on labelled group the review is most similar. In this case, our classifier was trained on 6738 (50/50 - male/female) randomly selected reviews from the New York Times book reviews from 2004-2014, and then tested for an accuracy score on 100 reviews.

We tried two different sets of word features, one containing the 1500 most common words and 2000 most common words having removed stop words (pronouns, conjuncts, determiners...). In the first case, the classifier reached an accuracy rate just shy of 70% and in the second 60%. Below are listed the top 25 most deterministic words for the first model:

The list suggests yet again that the problem may lie not in the reviewers’ bias, but in the themes of the books selected for review, with female authored books concentrating on family (boyfriend, grandmother, mothers, daughter) and the "near" (nearing) , while male authored books on society (entities, congress, commander, leadership) and the "far" (farthest).

Contains the word	Ratio
immunity	male : female = 4.2 : 1.0
entities	male : female = 3.8 : 1.0
congress	male : female = 3.6 : 1.0
gist	male : female = 3.6 : 1.0
wires	male : female = 3.3 : 1.0
fabricated	male : female = 3.2 : 1.0
farthest	male : female = 3.0 : 1.0
geological	male : female = 3.0 : 1.0
forgery	male : female = 3.0 : 1.0
commander	male : female = 2.9 : 1.0
leadership	male : female = 2.8 : 1.0
herself	female : male = 5.3 : 1.0
cremated	female : male = 4.4 : 1.0
boyfriend	female : male = 4.3 : 1.0
halftruths	female : male = 3.7 : 1.0
discriminatory	female : male = 3.4 : 1.0
nearing	female : male = 3.4 : 1.0
grandmother	female : male = 3.3 : 1.0
northwest	female : male = 3.3 : 1.0
rumored	female : male = 3.2 : 1.0
boarded	female : male = 3.2 : 1.0
connectedness	female : male = 3.0 : 1.0
rentcontrolled	female : male = 3.0 : 1.0
mothers	female : male = 3.0 : 1.0
she	female : male = 2.9 : 1.0
daughter	female : male = 2.8 : 1.0