AccessLex
What the 2020 July Bar Exam Results Tell Us—and What They Don’t
This is the first in a pair of blog posts exploring what we have learned from states’ published July 2020 bar exam results. We share these pieces to explore the potential benefits that might flow from the release of disaggregated bar exam data, inform and promote data-driven decision making in matters relevant to bar examinations more broadly, and identify research questions related to the July 2020 bar and future examinations.
Please visit our supplementary data dashboards for jurisdiction-level July 2020 bar results and detailed historical bar results.
The July 2020 bar exam presented a number of variations on what had been more typical bar exam formats and testing conditions prior to the pandemic. With those changes came opportunities for analysis and evaluation, albeit limited by a dearth of available data.
In this article and another to follow, we aim to identify relevant research questions related to the July 2020 bar administration and results – and those of future examinations – that the legal education community could begin to answer with more and better data. For starters, we think it is important to know who took the exam, who did not take the exam and why, and how this compares with previous exam administrations. Additionally, it would be helpful to better understand the relationships between bar passage and various examination conditions and formats, particularly for takers who have been historically marginalized.
Within the 28 jurisdictions that reported July 2020 counts of bar takers and passers as of March 2021, first-time and repeat pass rates rose 6 and 15 percentage points, respectively, while the number of takers plummeted 20 percent compared to July 2019. (We have created a dashboard to visualize
different facets of the exam at the jurisdiction level; isolating the data by administration, first-time or repeater group, and outcome variable of interest.) Although American Bar Association (ABA) reporting puts the 2020 first-time bar passage increase at a more modest 3 percentage points, this figure refers to February and July results combined and does not include repeater takers. These ABA reports also show that the national percentage of law graduates who did not take the bar rose 8 percentage points in 2020 compared to 2019.
This information only tells us part of the story of the disruption to the July 2020 bar exam; we currently lack July 2020 counts of bar takers and passers from 23 U.S. jurisdictions, and public data from reporting jurisdictions do not explain the rise in pass rates or the fall in bar examinees. To attempt to understand these occurrences, we need more detailed, anonymized data regarding subgroups of examinees–ideally disaggregated by race, ethnicity, gender, and socioeconomic status.
While it is not yet common practice for jurisdictions to collect and disseminate such data, California and New York have had success collecting, studying, and publishing this data in the past, and have used this information to make scoring policy and testing format changes.
Without more data, the outcomes and association implications of the July 2020 bar exam remain difficult to study, both in how they might have been incomparable to prior administrations and in considering similarities. In order to responsibly study future trends, we must have systems for collecting data at the subgroup level and full participation among all states. As jurisdictions have shifted to longer remote bar exams for 2021 administrations; as the pursuit of diversity, equity, and inclusion continues in the legal academy and profession; and with more permanent bar reform on the horizon; we need an improved data sharing system in place so that, when bar passage rates fluctuate, we can find out why.
In a following post, we will describe the doors that disaggregated bar exam data could open in greater detail.