

Table of Contents of All Academic and Bar Success Literature Reviews by Subject
Table of Contents of All Academic and Bar Success Literature Reviews by Subject
In-School Bar Exam Success Interventions
How do MBE-focused in-school interventions influence bar exam success?
In-School Bar Exam Success Interventions
How do MBE-focused in-school interventions influence bar exam success?
Several studies examined the effectiveness of in-school interventions focused on the multiple-choice Multistate Bar Examination (MBE) in improving bar success. Most of these interventions taught multiple-choice exam-taking strategies, reviewed areas of substantive law, and included extensive practice answering multiple choice questions, often under timed conditions. There was some variation in the timing of these interventions: one school had a second-year intervention, five had third-year interventions, and one intervention was available to both second- and third-year students. The studies also varied in the intended recipient of the intervention, with some targeting students at the bottom of the class and some open to all students. Only three of the interventions were fully focused on the MBE, while the others included preparation for the essay sections of the bar exam as well.
Each study asserts that these MBE-focused in-school interventions are effective in improving diagnostic exam scores, bar exam scores, or bar passage rates.
Alphran et al.
Alphran et al.
The University of the District of Columbia Clarke School of Law’s 3L Bar Skills Preparation Workshops, introduced in 2003 and ended in 2006,[57] were weekly workshops led by academic support faculty that focused on essay and multiple-choice strategies and practice.[58] Students could also participate in a workshop specifically focused on multiple-choice questions presented by an outside vendor.[59] In 2005, the Bar Skills Workshops also incorporated an essay writing workshop and videotaped substantive law review lectures provided by a commercial bar preparation company.[60] The course was replaced by the PTEX Essay Writing Program after the school’s Bar Passage Task Force recommended more focus on essay writing practice and the ABA began allowing for-credit bar preparation courses.
Alphran examined bar passage data from 291 UDC students who graduated in 2003-2008 to see if there were any relationships with the interventions offered. For the entire sample, first-time bar passage rates for students who participated in either the Bar Skills Workshop or took the PTEX course were higher than those who did not (62.5% versus 47.8%).[61] The difference was even greater for the bottom half of the class (46.6% versus 21.6%).[62] The article does not define “participation” in the Bar Skills Workshop.
Alphran then performed a chi-square test to see if the interventions were statistically significant factors in bar passage. For the entire sample period, he found that the interventions were statistically significant.[63]
[57] Derek Alphran et al., Yes We Can, Pass the Bar. University of the District of Columbia, David A. Clarke School of Law Bar Passage Initiatives and Bar Pass Rates—From the Titanic to the Queen Mary!, 14 UDC/DCSL L. Rev. 9, 33 (2011), https://heinonline.org/HOL/P?h=hein.journals/udclr14&i=11.
[58] Id. at 19.
[59] Id.
Florida International University College of Law
Florida International University College of Law
Florida International University College of Law’s Academic Excellence Program incorporates several interventions: 1) Introduction to the Study of Law course in the first semester, 2) Legal Reasoning course in the second semester, 3) Legal Analysis course in the third semester, 4) Advanced Legal Analysis course in the fifth semester, 5) U.S. Law and Procedure course in the final semester, and 6) a mentorship program during the post-graduation bar preparation period.
Advanced Legal Analysis is an elective fifth semester course.[64] The course is required for the bottom 20% of the class.[65] The course is focused on teaching bar exam skills, with students writing bar exam-style essays and completing multiple-choice questions. The course also reviews three subjects that are tested on both the MBE and Florida essays: Torts, Contracts, and Real Property.
U.S. Law and Procedure is a final semester elective open to all graduating 3Ls, and almost all the class takes it.[66] It is required for the bottom 20% of the class.[67] The course teaches bar exam skills, splitting time between the Multistate Bar Exam and the Florida essays. It reviews all MBE-tested subjects and seven of the Florida-tested subjects.[68]
FIU Law began implementing these interventions in Spring 2014. The 2015 graduating class had access to the full program except for the first semester course, while classes graduating in 2016 and beyond had access to the full program. Schulze reports that FIU Law became the Florida law school with the highest bar passage rate beginning in July 2015 and credits the program.[69]
Ruiz performed a regression analysis for the relationship between the two 3L interventions and first-time bar passage. The sample was 665 FIU Law students who graduated from May 2015 to 2019, and took the bar exam (in any jurisdiction) between July 2015 and 2019.[70] Of those students, 97% took U.S. Law and Procedure, while 11.1% took Advanced Legal Analysis.[71]
Predictors used in the logistic regressions were: LSAT score; undergraduate GPA; final law school GPA; number of elective doctrinal courses taken (that are tested on the Florida bar); minority status; gender; part-time or full-time enrollment status; commercial bar preparation program used; and whether they enrolled in Advanced Legal Analysis or U.S. Law and Procedure.[72] Ruiz found that both third-year courses were statistically significant and positive predictors of bar passage. U.S. Law and Procedure had a larger odds ratio than Advanced Legal Analysis (4.733 versus 2.128).[73] For a student with a 2.65 final LGPA who took both 3L courses, their probability of bar passage was 48 percentage points higher than a student with the same LGPA who took neither.[74]
[64] Louis N. Schulze, Jr., Using Science to Build Better Learners: One School’s Successful Efforts to Raise Its Bar Passage Rates in an Era of Decline, 68 J. Legal Educ. 230, 235 (2019), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2960192.
[65] Raul Ruiz, Leveraging Noncognitive Skills to Foster Bar Exam Success: An Analysis of the Efficacy of the Bar Passage Program at FIU Law, 99 Neb. L. Rev. 141, 177 (2020), https://heinonline.org/HOL/P?h=hein.journals/nebklr99&i=146.
[66] Raul Ruiz, Leveraging Noncognitive Skills to Foster Bar Exam Success: An Analysis of the Efficacy of the Bar Passage Program at FIU Law, 99 Neb. L. Rev. 141, 179 (2020), https://heinonline.org/HOL/P?h=hein.journals/nebklr99&i=146; Louis N. Schulze, Jr., Using Science to Build Better Learners: One School’s Successful Efforts to Raise Its Bar Passage Rates in an Era of Decline, 68 J. Legal Educ. 230, 236 (2019), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2960192.
[67] Raul Ruiz, Leveraging Noncognitive Skills to Foster Bar Exam Success: An Analysis of the Efficacy of the Bar Passage Program at FIU Law, 99 Neb. L. Rev. 141, 179 (2020), https://heinonline.org/HOL/P?h=hein.journals/nebklr99&i=146.
[68] Id.
[69] Louis N. Schulze, Jr., Using Science to Build Better Learners: One School’s Successful Efforts to Raise Its Bar Passage Rates in an Era of Decline, 68 J. Legal Educ. 230, 236–37 (2019), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2960192.
[70] Raul Ruiz, Leveraging Noncognitive Skills to Foster Bar Exam Success: An Analysis of the Efficacy of the Bar Passage Program at FIU Law, 99 Neb. L. Rev. 141, 192 (2020), https://heinonline.org/HOL/P?h=hein.journals/nebklr99&i=146.
[71] Id. at 207.
[72] Id. at 195–96.
[73] Id. at 197.
[74] Id. at 200–01.
Jellum and Reeves
Jellum and Reeves
University of Richmond School of Law began offering a Supplemental Bar Preparation course to 3L students in both semesters in Spring 2001. The course teaches exam-taking skills, reviews substantive law in six bar-tested subjects (Constitutional Law, Contracts, Sales, Real Property, Criminal Law, and Criminal Procedure), and provides exam practice. Two hours of each class is dedicated to subject review, with another hour for students to answer timed multiple-choice and essay questions. At the time Jellum and Reeves wrote about the program, it was not for credit.[75]
Richmond also provides weekly individual tutoring sessions to graduates during the bar preparation period. Graduates prepare answers to bar exam-style questions and then receive feedback from their tutor, or they answer questions under exam-style conditions.[76]
Jellum and Reeves analyzed Virginia bar passage data from July 1997 through July 2004 (eight administrations before the program was implemented in Spring 2001, and seven administrations after).[77] They performed proportions tests comparing the bar passage rate of students before and after the program was implemented for the overall classes, as well as for the upper and bottom halves of the classes. They found that bar passage rates improved after the program’s implementation, with especially large improvements in the bottom half of the class.[78]
Jellum and Reeves also found that after the program was implemented, graduates in the bottom half of the class who participated in the program had higher bar passage rates than those who did not (71.6% for participants versus 55.7% for non-participants).[79] “Participation” was defined as attending at least half of the Supplemental Bar Preparation course classes or at least one tutoring session.[80]
Jellum and Reeves note that the University of Richmond enacted an LSAT admission floor of 150 beginning with the class of 2000, but this was unlikely to be the cause of the bar passage rate increase because very few students had been admitted below that floor in the past.[81] There was also little change in median LSAT or UGPA over the period.[82]
[75] Linda Jellum & Emmeline Paulette Reeves, Cool Data on a Hot Issue: Empirical Evidence That a Law School Bar Support Program Enhances Bar Performance, 5 Nev. L.J. 646, 662 (2005), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1098473.
[76] Id. at 663.
[77] Id. at 669.
[78] Id. at 671.
[79] Id.
[80] Id. at 669–70.
[81] Id. at 673–75.
[82] Id. at 676.
Johns
Johns
At University of Denver Sturm College of Law, Legal Analysis Strategies is a final semester course that is required for students who had a second-year GPA of 2.6 or lower.[83] Other students may take the course as an elective. The course is focused on teaching exam strategies (both essays and multiple-choice) and review of substantive law.
Johns analyzed bar passage data for 642 Denver students who graduated in May 2008-2010 and immediately took the Colorado bar exam that following July.[84] In this sample, 47.2% took Legal Analysis Strategies.[85]
Graduates with a 1L GPA at or below 2.9 who took Legal Analysis Strategies passed the bar exam at a higher rate than those in that 1L GPA range who did not take the course.[86]
Johns also utilized linear regression analysis to test the relationship between these programs, LSAT score, law school GPAs, and bar exam score. Variables tested were: age; sex; underrepresented minority status; enrollment status; LSAT; 1L GPA; final GPA; and participation in the three interventions.[87] Johns found that participation in Legal Analysis Strategies was a statistically significant and positive predictor of bar exam score (albeit less predictive than either final law school GPA or LSAT score).[88] Taking Legal Analysis Strategies translates to a 4.794 point increase in bar exam score.[89]
[83] Scott Johns, Empirical Reflections: A Statistical Evaluation of Bar Exam Program Interventions, 54 U. Louisville L. Rev. 35, 37 (2016), https://heinonline.org/HOL/P?h=hein.journals/branlaj54&i=41.
University of Hawaii Richardson School of Law
University of Hawaii Richardson School of Law
University of Hawaii Richardson School of Law has two academic and bar success interventions: 1) Advanced Legal Analysis, and 2) Multistate Bar Exam Intensive Course.[90]
The Multistate Bar Exam Intensive Course was a weeklong course for 2L and 3L students taught over spring break of 2020 by a commercial bar preparation company.[91] The course was intended to introduce the bar exam format, present bar preparation and test-taking strategies, and review some areas of 1L substantive law. The course was intended to be in-person, but was forced to become an asynchronous online class by the COVID pandemic.[92]
For the Multistate Bar Exam Intensive Course, 55 students participated in at least one component, 43 of whom were 3Ls. Fifteen students completed all components of the course (all review classes, the pre-course test, and the post-course test), 11 of whom were 3Ls. For the 3Ls who completed the entire course, the average score increase from pre-course to post-course test was 12.2% (8.5% for 2Ls).[93]
University of Maine School of Law
University of Maine School of Law
University of Maine School of Law administered an abbreviated MBE-style diagnostic exam from a commercial bar preparation company to students after their 1L year,[94] with the purpose of identifying students needing academic support and areas of the 1L curriculum that needed improvement.[95] The diagnostic was a four-hour, 120 multiple choice question test covering Torts, Real Property, Contracts, Civil Procedure, and Criminal Law.[96] Students could take the test at any time during a four day period.[97] Diagnostic scores were divided by subject area, as well as whether the question was on knowledge of a legal rule, application of the rule, or combined.[98] Exam results were shared with the students soon after taking the exam, and sent to them again soon after graduation.[99]
The researchers performed a Pearson’s correlation analysis between several variables (scores on the diagnostic, LSAT, UGPA, law school grades) and bar passage.[100] For the class of 2022, 57 students who took the diagnostic in September 2020, they found that grades in the civil procedure and the first semester legal research and writing courses were most predictive of bar passage (correlation coefficients of 0.64 and 0.56, respectively), as well as 1L GPA (0.54).[101]On the diagnostic test, scores from Real Property Application (0.3), Analysis Combined Score (0.27), and Criminal Law Combined (.24) were most predictive of bar passage.[102] This analysis was not done on the class of 2023 and 2024, presumably because their bar results were still pending at the time.
The school also looked at trends on diagnostic exam outcomes across the three cohorts. The average composite score for 2022 and 2023 was 57%, but it increased to 61% for 2024, driven mostly by increases in Real Property and Civil Procedure scores.[103]
University of San Diego School of Law
University of San Diego School of Law
University of San Diego School of Law implemented a dedicated Black bar taker study group in 2021.[104]
The Black bar taker study group was focused on the 3L spring MBE course offered at USD. The study group met weekly over Zoom during 3L spring of 2021 and continued to meet after graduation to focus on essay and MPT practice.[105] Participants also received access to an MBE-focused commercial bar preparation program.[106] The study group had nine participants, with seven students on average attending the weekly meetings.[107]
The school reported that the 2021 study group had an 87.5% bar passage rate, compared to 40% in 2019 and 27.2% in 2018.[108] The report notes that the California bar exam cut score was lowered during that timeframe.[109]
The school is expanding the study group initiative to graduates in the bottom third of the graduating class and students from other underrepresented demographic groups.[110]
The school also analyzed data from 2018 and 2019 bar exam results and found that the following other bar interventions had a positive effect on bar passage: completing the bar exam simulation, submitting practice bar exam essays for grading, attending all the bar essay workshops, using certain commercial bar preparation courses, and scoring over 50% on a mock MBE.[111]
[104] USD Directed Grant Program 2020 Final Report 3-4 (2022) (on file with AccessLex Inst.).
[105] Id.
[106] Id.
[107] Id.
[108] Id. at 4.
[109] Id.
[110] Id.
[111] Id. at 5–6, 14–15.
How do essay-focused in-school interventions influence bar exam success?
How do essay-focused in-school interventions influence bar exam success?
Several studies examined the effects of essay-focused in-school interventions on bar exam success. All of the programs studied took place in a student’s third year of law school. All the interventions included extensive practice answering bar exam essay questions, often under timed conditions. They also frequently included the teaching of essay strategies, in-class discussion of practice essays, and review of substantive law. Four interventions were fully focused on the essay sections of the bar exam, while the others included preparation for the multiple-choice section. All programs were open to all students, but half were required for students at the bottom of the class.
Most of these studies assert that these essay-focused, in-school interventions are effective in boosting bar exam scores and passage rates. One study found that the intervention had no statistically significant effect.
Alphran et al.
Alphran et al.
The University of the District of Columbia Clarke School of Law’s 3L Bar Skills Preparation Workshops, introduced in 2003 and ended in 2006,[112] were weekly workshops led by academic support faculty that focused on essay and multiple-choice strategies and practice.[113] Students could also participate in a workshop specifically focused on multiple-choice questions presented by an outside vendor.[114] In 2005, the Bar Skills Workshops also incorporated an essay writing workshop and videotaped substantive law review lectures provided by a commercial bar preparation company.[115] The course was replaced by the PTEX Essay Writing Program after the school’s Bar Passage Task Force recommended more focus on essay writing practice and the ABA began allowing for-credit bar preparation courses.
The PTEX Essay Writing Program, started in 2007, is a 14-week writing skills class focused on preparation for the bar exam essays and MPT, specifically subjects tested on the Maryland bar exam.[116] PTEX Company was an outside vendor that the law school contracted with to provide the program.[117] Over the course, students write several in-class and take-home essays and MPTs, ending with a final exam that incorporates both types of questions.[118] UDC continued to offer the multiple-choice skills workshop provided by the outside vendor.[119]
Alphran examined bar passage data from 291 UDC students who graduated in 2003-2008 to see if there were any relationships with the interventions offered. For the entire sample, first-time bar passage rates for students who participated in either the Bar Skills Workshop or took the PTEX course were higher than those who did not (62.5% versus 47.8%).[120] The difference was even greater for the bottom half of the class. The article did not define “participation” in the Bar Skills Workshop.
Looking specifically at 2007-2008 (the PTEX years), for students in the bottom half of the class, course-takers had a higher bar passage rate. However, for the entire class, the bar passage rate was lower for course-takers than those who did not take the course.[121]
Alphran then performed a chi-square test to see if the interventions were statistically significant factors in bar passage. For the entire sample period, he found that the interventions were statistically significant. Looking at only the 2007-2008 period, he found that the PTEX course was not statistically significant.[122]
[112] Derek Alphran et al., Yes We Can, Pass the Bar. University of the District of Columbia, David A. Clarke School of Law Bar Passage Initiatives and Bar Pass Rates—From the Titanic to the Queen Mary!, 14 UDC/DCSL L. Rev. 9, 33 (2011), https://heinonline.org/HOL/P?h=hein.journals/udclr14&i=11.
[113] Id. at 19.
[114] Id.
[115] Id. at 21. 1
[116] Id. at 27.
[117] Id. at 25.
[118] Id. at 30.
[119] Id. at 25.
[120] Id. at 36.
[121] Id.
[122] Id. at 37–38.
Bratman
Bratman
Beginning in 2006, University of Pittsburgh School of Law offers an Analytical Writing course in spring of 3L year. The course focuses on the legal writing and analysis skills necessary to perform well on the essay sections of the bar exam.[123] Classes usually involve either taking timed practice questions or discussing those questions.[124] Feedback comes in the form of individualized feedback from the professor, self-evaluation, and peer evaluation.[125]
While the course is open to all students, students with low GPAs are invited to enroll and given priority registration (the specific GPA threshold is based on the previous year’s bar exam statistics, but it is usually the bottom 30% of the class).[126] The course has an enrollment cap of 48 students, so not all students in the bottom 30% of the 250 student class could enroll if they chose to.[127]
Writing was chosen as a focus because 1) recent graduates who had failed the bar exam did better on the multiple choice sections; 2) at the time (pre-UBE) jurisdictions where graduates were taking the bar exam were giving greater weight to the written components; and 3) the feeling that commercial bar preparation companies would do a better job of teaching multiple-choice testing skills than law school faculty.[128]
Bratman compared the bar passage rate for students in the bottom 30% of the class who took the course against those in the bottom 30% who did not for two graduating classes, 2006 and 2007.[129] In 2006, students who took the course had Pennsylvania bar exam passage rate of 71% compared to 53% for those who did not take the course. For all jurisdiction bar exam passage, course-takers had a 68% pass rate compared to 59% for non-takers.[130] In 2007, students who took the course had Pennsylvania bar exam passage rate of 71.4% compared to 68.8% for those who did not take the course. For all jurisdiction bar exam passage, course-takers had a 73.5% pass rate compared to 57.1% for non-takers.[131]
[123] Ben Bratman, For-Credit Bar Exam Preparation: A Legal Writing Model, Bar Exam’r, Nov. 2007, at 28, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1285456.
[124] Id. at 30.
[125] Id. at 31.
[126] Id. at 29.
[127] Id.
[128] Id. at 28.
[129] Id. at 31.
[130] Id.
[131] Id. at 31-32.
Florida International University College of Law
Florida International University College of Law
Florida International University College of Law’s Academic Excellence Program incorporates several interventions: 1) Introduction to the Study of Law course in the first semester, 2) Legal Reasoning course in the second semester, 3) Legal Analysis course in the third semester, 4) Advanced Legal Analysis course in the fifth semester, 5) U.S. Law and Procedure course in the final semester, and 6) a mentorship program during the post-graduation bar preparation period.
Advanced Legal Analysis is an elective fifth semester course.[132] The course is required for the bottom 20% of the class.[133] The course is focused on teaching bar exam skills, with students writing bar exam-style essays and completing multiple-choice questions. The course also reviews three subjects that are tested on both the MBE and Florida essays: Torts, Contracts, and Real Property.
U.S. Law and Procedure is a final semester elective open to all graduating 3Ls, and almost all of the class takes it.[134] It is required for the bottom 20% of the class.[135] The course teaches bar exam skills, splitting time between the Multistate Bar Exam and the Florida essays. It reviews all MBE-tested subjects and seven of the Florida-tested subjects.[136]
FIU Law began implementing these interventions in Spring 2014. The 2015 graduating class had access to the full program except for the first semester course, while classes graduating in 2016 and beyond had access to the full program. Schulze reports that FIU Law became the Florida law school with the highest bar passage rate beginning in July 2015 and credits the program.[137]
Ruiz performed a regression analysis for the relationship between the two 3L interventions and first-time bar passage. The sample was 665 FIU Law students who graduated from May 2015 to 2019, and took the bar exam (in any jurisdiction) between July 2015 and 2019.[138] Of those students, 97% took U.S. Law and Procedure, while 11.1% took Advanced Legal Analysis.[139]
Predictors used in the logistic regressions were: LSAT, UGPA, LGPA, number of elective doctrinal courses taken (that are tested on the Florida bar), minority status, gender, part-time or full-time enrollment status, commercial bar preparation program used, and whether they enrolled in Advanced Legal Analysis or U.S. Law and Procedure.[140] Ruiz found that both 3L courses were statistically significant and positive predictors of bar passage. U.S. Law and Procedure had a larger odds ratio than Advanced Legal Analysis (4.733 versus 2.128).[141] For a student with a 2.65 final LGPA who took both 3L courses, their probability of bar passage was 48 percentage points higher than a student with the same LGPA who took neither.[142]
[132] Louis N. Schulze, Jr., Using Science to Build Better Learners: One School’s Successful Efforts to Raise Its Bar Passage Rates in an Era of Decline, 68 J. Legal Educ. 230, 235 (2019), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2960192.
[133] Raul Ruiz, Leveraging Noncognitive Skills to Foster Bar Exam Success: An Analysis of the Efficacy of the Bar Passage Program at FIU Law, 99 Neb. L. Rev. 141, 177 (2020), https://heinonline.org/HOL/P?h=hein.journals/nebklr99&i=146.
[134] Id. at 179.
[135] Id.
[136] Id.
[137] Louis N. Schulze, Jr., Using Science to Build Better Learners: One School’s Successful Efforts to Raise Its Bar Passage Rates in an Era of Decline, 68 J. Legal Educ. 230, 236–37 (2019), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2960192.
[138] Raul Ruiz, Leveraging Noncognitive Skills to Foster Bar Exam Success: An Analysis of the Efficacy of the Bar Passage Program at FIU Law, 99 Neb. L. Rev. 141, 192 (2020), https://heinonline.org/HOL/P?h=hein.journals/nebklr99&i=146.
[139] Id. at 207.
[140] Id. at 195–96.
[141] Id. at 196–97.
[142] Id. at 201.
Jellum and Reeves
Jellum and Reeves
University of Richmond School of Law began offering a Supplemental Bar Preparation course to 3L students in both semesters in Spring 2001. The course teaches exam-taking skills, reviews substantive law in six bar-tested subjects (Constitutional Law, Contracts, Sales, Real Property, Criminal Law, and Criminal Procedure), and provides exam practice. Two hours of each class is dedicated to subject review, with another hour for students to answer timed multiple-choice and essay questions. At the time Jellum and Reeves wrote about the program, it was not for credit.[143]
Richmond also provides weekly individual tutoring sessions to graduates during the bar preparation period. Graduates prepare answers to bar exam-style questions and then receive feedback from their tutor, or they answer questions under exam-style conditions.[144]
Jellum and Reeves analyzed Virginia bar passage data from July 1997 through July 2004 (eight administrations before the program was implemented in Spring 2001, and seven administrations after).[145] They performed proportions tests comparing the bar passage rate of students before and after the program was implemented for the overall classes, as well as for the upper and bottom halves of the classes. They found that bar passage rates improved after the program’s implementation, with especially large improvements in the bottom half of the class.[146]
Jellum and Reeves also found that after the program was implemented, graduates in the bottom half of the class who participated in the program had higher bar passage rates than those who did not (71.6% for participants versus 55.7% for non-participants).[147] “Participation” was defined as attending at least half of the Supplemental Bar Preparation course classes or at least one tutoring session.[148]
Jellum and Reeves note that the University of Richmond enacted an LSAT admission floor of 150 beginning with the class of 2000, but this was unlikely to be the cause of the bar passage rate increase because very students had been admitted below that floor in the past.[149] There was also little change in median LSAT or UGPA over the period.[150]
[143] Linda Jellum & Emmeline Paulette Reeves, Cool Data on a Hot Issue: Empirical Evidence That a Law School Bar Support Program Enhances Bar Performance, 5 Nev. L.J. 646, 662 (2005), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1098473.
[144] Id. at 663.
[145] Id. at 669.
[146] Id. at 671.
[147] Id.
[148] Id. at 669–70.
[149] Id. at 673–75.
[150] Id. at 676.
Johns
Johns
At University of Denver Sturm College of Law, Legal Analysis Strategies is a final semester course that is required for students who had a 2L GPA of 2.6 or lower.[151] Other students may take the course as an elective. The course is focused on teaching exam strategies (both essays and multiple-choice) and review of substantive law.
Johns analyzed bar passage data for 642 Denver students who graduated in May 2008-2010 and immediately took the Colorado bar exam that following July.[152] 47.2% of the sample took Legal Analysis Strategies.
Graduates with a 1L GPA at or below 2.9 who took Legal Analysis Strategies passed the bar exam at a higher rate than those in that 1L GPA range who did not participate in any of these interventions.[153]
Johns also utilized linear regression analysis to test the relationship between these programs, LSAT score, law school GPAs, and bar exam score. Variables tested were: age, sex, underrepresented minority status, enrollment status, LSAT, 1L GPA, final GPA, and participation in the three interventions.[154] Johns found that participation in Legal Analysis Strategies was a statistically significant and positive predictor of bar exam score (albeit less predictive than either final LGPA or LSAT).[155]Taking Legal Analysis Strategies translates to a 4.794-point increase in bar exam score.[156]
Mainero
Mainero
Chapman University Fowler School of Law offers two for-credit bar preparation classes for 3L students. Both are required for students in the bottom quartile of the class based on their 2L GPA but elective for other students.[157]
Legal Analysis Workshop is focused on the California bar exam’s performance test. Students write weekly performance test essays and take two midterms and a final in the form of three-hour performance tests. About half of all students take the course.[158]
Select Topics in American Law is focused on preparing for the California bar exam’s subject essay questions, along with a review of substantive law. Students write weekly essays at home, which are reviewed in class. They then write another essay in class each week. The midterm and final are both three-hour essay exams modeled on the California bar exam. More than 90% of students take Select Topics.[159]
Chapman University also has a post-graduation bar preparation program that is available to all graduates. It is offered both right after graduation and in mid-December for repeat-takers.[160] The program includes three mock bar exams, followed by six sessions after each bar (one to cover each subject tested on the mock exam), then a final mock bar exam.[161] Graduates can also submit up to 30 essays for feedback from faculty.[162]
Legal Writing Skills was first offered in 2007-2008, Legal Analysis Workshop in 2010-2011, Select Topics in 2009-2010, and the post-graduation program in 2007. From 2009 to 2014, Chapman University had a July bar passage rate higher than the California ABA law school average (except in 2010).[163]
Mainero analyzed bar passage data for 142 Chapman University students who graduated in 2009 to see if there was a relationship between taking Selected Topics and bar passage. That year, 59 students took the course and 83 did not.[164] Mainero performed a chi-square analysis and found there was a positive and statistically significant relationship between taking the course and passing the California bar exam.[165]
Mainero also performed a chi-square analysis to examine the relationship between the number of practice essays written during the post-graduation period and bar passage, looking at data from 2011-2013. He found that writing more essays had a positive and statistically significant relationship with bar passage.[166] There was an especially large jump in pass rate between the students who wrote 0-9 essays and those who wrote 10-14.
[157] Mario W. Mainero, We Should Not Rely on Commercial Bar Reviews to Do Our Job: Why Labor-Intensive Comprehensive Bar Examination Preparation Can and Should Be a Part of the Law School Mission 37 (Chap. U. Dale E. Fowler Sch. of L. Legal Stud. Rsch. Paper Series, Paper No. 15-01, 2014), https://dx.doi.org/10.2139/ssrn.2546001.
Santa Clara University School of Law
Santa Clara University School of Law
Santa Clara University School of Law has three academic and bar success interventions: 1) the UP Point Policy, 2) Advanced Legal Writing, and 3) Bar Exam Planning Workshop. [167]
The UP Point Policy, implemented in 2015, is a requirement that all students take four upper-level doctrinal bar course (Constitutional Law I and II, Evidence, and Professional Responsibility) and earn a C+ or better.[168] Students who do not earn a C+ or better in those courses must continue taking doctrinal bar courses from an approved list until they successful meet the four-course requirement.[169] Students who do not meet that requirement by their final semester are required to participate in the other two interventions in their final semester.[170]
Advanced Legal Writing is a final semester course that reviews substantive Torts, Contracts, and Criminal law and teaches test-taking strategies for the bar exam. The course has weekly written practice exams to prepare students for the bar essays and MPT and includes a 6.5-hour mock bar exam. Enrollment in the course is required for students who did not complete the UP Point requirements by their final semester and open to anyone in their final semester.[171]
Santa Clara University analyzed bar passage data to evaluate these programs.[172] The report does not state what methods were used.
For Advanced Legal Writing, they reported that taking the course had a small positive impact on the likelihood of bar passage.[173]
Zeigler et al.
Zeigler et al.
New York Law School implemented its Comprehensive Curriculum Program in 2003. It includes a few different interventions: 1) a guided curriculum, 2) a second semester Principles of Legal Analysis course, 3) a fifth semester New York Law in National Perspectives course, and 4) a final semester Consolidated Legal Analysis course.
New York Law School’s guided curriculum is required for all students in the bottom 25% of the class after their first year.[174] Students on this path are required to take the three courses noted above, plus Corporations; Wills, Trusts, and Future Interests; Individual Income Tax, and New York Practice. Additionally, they must take a minimum number of doctrinal and skills-based electives from an approved list. Students in the bottom 10% of the class are also required to add an extra semester to their law school studies.
New York Law in National Perspectives is a fifth semester elective, but it is required for the bottom 25% of the class based on their 1L grades.[175] The course focuses on writing bar-exam style essays, along with review of legal rules from earlier courses.
Consolidated Legal Analysis is a final semester elective, but it is required for the bottom 25% of the class based on their 1L grades.[176] The class focuses on legal analysis skills and writing bar-exam style essays.
New York Law School also implemented several other policy changes intended to improve bar passage rates. All law school exams were required to be closed book, to better mirror the New York bar exam, and professors were encouraged to include multiple-choice questions in their exams.[177] The school also changed its attrition policies in 2003. Before the change, students with a first-semester GPA under 1.5 or a first-year GPA under 2.0 were placed on academic probation, with a committee making the final decision whether to dismiss them. After the change, students with a first-semester GPA under 1.75 or a first-year GPA under 1.9 were automatically dismissed. Students with a first-year GPA between 1.9 and 1.99 are presumptively dismissed, with a committee making the final decision.[178]
New York Law School saw first-time bar passage rates increase in 2006 (when the first class to experience the entire intervention graduated), 2007, and 2008.[179] Most of the increase in bar passage was in the bottom half of the class.
How do metacognitive in-school interventions influence bar exam success?
How do metacognitive in-school interventions influence bar exam success?
A couple of studies examined the effects of incorporating a metacognitive intervention into a third-year bar preparation course on bar exam success. Metacognitive interventions are designed to teach students how to be better learners, providing them with knowledge of study skills and strategies and coaching them on how best to use them.
Neither study found evidence that these metacognitive interventions improved bar exam success.
Gundlach and Santangelo
Gundlach and Santangelo
Gundlach and Santangelo examined the effects of metacognitive skills on success in law school and bar passage.[180] Their sample was 196 3L students at Hofstra University Deane School of Law, all of whom were enrolled in a required 3L Perspectives in Legal Writing and Analysis bar preparation course in 2021 and 2022.[181] 174 of the students had participated in a 1L version of this study, and 58 of them received a 1L metacognition intervention at that time.[182]
The researchers administered a survey to students on the first day of the course asking questions about which of 15 learning strategies they intended to use. This was followed by a lecture on learning strategies and metacognition.[183] Students were surveyed three more times over the course of the semester after receiving feedback on graded assignments, asking whether the learning strategies were helpful and whether they would make changes to their learning strategies moving forward.[184] Survey questions were coded into two groups: those showing knowledge of metacognition and those showing regulation (making appropriate decisions about learning strategies).[185]
The authors used Fisher’s Exact Test or Chi-Square depending on sample sizes.[186]
They found that at the beginning of the 3L course, 76% of participants demonstrated knowledge of metacognition and 81% demonstrated regulation. These increased to 94% and 92% respectively at the end of the semester.[187] They found no evidence that participation in the 1L intervention impacted their 3L metacognition knowledge. Students who took the 1L intervention were no more likely to be aware of learning strategies at the beginning of the 3L course.[188] However, students who took the 1L intervention were more likely to show regulation and be using active learning strategies at the beginning of the 3L course.[189]
The authors additionally found that metacognitive knowledge at the end of the course was associated with success in the course, while knowledge at the beginning of the course was not.[190] Metacognitive regulation was associated with success in the course at all points.[191]
Metacognitive knowledge at the end of 3L year was not associated with final LGPA, but demonstration of metacognitive regulation was. The authors propose that knowing about metacognitive strategies is not enough — knowing how and when to put them into practice is more important.[192]
Finally, the authors found no relationship between metacognitive knowledge or regulation with bar passage. The authors propose that because bar passage is related to law school performance, metacognitive regulation may feed into that relationship.[193]
[180] Jennifer A. Gundlach & Jessica R. Santangelo, The Influence of Metacognitive Skills on Bar Passage: An Empirical Study 21 (2023), https://arc.accesslex.org/grantee/84/.
[181] Id. at 21, 31.
[182] Id.
[183] Id. at 22–23.
[184] Id. at 24–26.
[185] Id. at 29–30.
[186] Id. at 32.
[187] Id. at 34.
[188] Id. at 37.
[189] Id.
[190] Id. at 39–40.
[191] Id. 40–41.
[192] Id. 42–43.
[193] Id. at 44.
Santa Clara University School of Law
Santa Clara University School of Law
Santa Clara University School of Law has three academic and bar success interventions: 1) the UP Point Policy, 2) Advanced Legal Writing, and 3) Bar Exam Planning Workshop.[194]
The UP Point Policy, implemented in 2015, is a requirement that all students take four upper-level doctrinal bar course (Constitutional Law I and II, Evidence, and Professional Responsibility) and earn a C+ or better.[195] Students who do not earn a C+ or better in those courses must continue taking doctrinal bar courses from an approved list until they successful meet the four-course requirement.[196] Students who do not meet that requirement by their final semester are required to participate in the other two interventions in their final semester.[197]
The Bar Exam Planning Workshop is a required course final semester course for students who did not complete the UP Point requirements by their final semester.[198] The course focuses on study strategies for the bar exam and self-management skills, and students create a personal bar preparation plan.
Santa Clara University analyzed bar passage data to evaluate these programs.[199] The report does not state what methods were used.
The researchers reported that the Bar Exam Planning Workshop had no meaningful effect on bar passage.[200]
How do doctrinal coursework policies influence bar exam success?
How do doctrinal coursework policies influence bar exam success?
Several studies examined the effects of taking doctrinal (i.e., bar-tested subject) coursework on bar exam success. Five examined the relationship with the number of doctrinal courses taken, one examined the relationship with completing specific doctrinal courses, and three examined the relationship with doctrinal course performance.
The evidence that the number of doctrinal courses taken influences bar exam success is mixed. Two studies found that there was a relationship, two studies found a relationship only for students in lower final law school GPA quartiles, and one found no relationship at all.
All studies examining completion of or performance in specific doctrinal courses found relationships with bar exam success, but there was significant variation in which subjects were predictive.
Austin et al.
Austin et al.
Austin et al. examined the relationship between specific course performance on overall bar score and subcomponent percentile performance, as well as the relationship between skills courses and bar score.[201] They analyzed data from 1,572 Texas Tech University School of Law graduates who took the Texas bar exam between February 2008 and July 2014.[202]
The authors conducted a series of regression analyses.[203] They found that 1L Civil Procedure performance was a strong predictor of bar exam score, even though it was not tested on the Texas bar exam during the study period. The authors attribute that result to the course teaching similar skills and methods as those used on the bar exam.[204] They similarly found that 1L Legal Research and Writing course performance was a strong predictor of bar exam score.[205]
For graduates who took the MBE for the first time in 2008-2013 (a period when NCBE was reporting subject-specific performance), the researchers examined the relationship between performance in specific doctrinal subjects and performance on that subject on the MBE. They found that:
- Criminal Law and Criminal Procedure course performance accounted for 15% of Criminal Law MBE performance.
- Property course performance accounted for 13.3% of Real Property MBE performance.
- Evidence course performance accounted for 12.8% of Evidence MBE performance.
- Contracts course performance accounted for 10.9% of Contracts MBE performance.
- Constitutional Law course performance accounted for 8.1% of Constitutional Law MBE performance.
- Torts course performance accounted for 6.9% of Torts MBE performance.
None of the above doctrinal courses contributed to performance for repeat MBE-takers.[206] For first-time MBE-takers in 2014 (when NCBE stopped reporting subject-specific performance), the researchers found that performance in the above doctrinal courses accounted for 35.6% of MBE performance, but that Evidence and Contracts were the only significant contributors.[207]
For first-time Texas essay takers:[208]
- Contracts and Commercial Law courses accounted for 8.4% of performance on UCC essays.
- Business Entities course performance accounted for 4.8% of performance on Business Associations essays.
- Consumer Law accounted for 2.6% of performance on Consumer Law essay.
- Family Law and Wills and Trusts had no practical effect on performance on those essays.
- Property and Oil and Gas courses had no relationship with Real Property essays performance.
- Wills and Trusts and Guardianship courses had no effect on Trust or Guardianship essay performance.
For Texas Procedure and Evidence Exams:[209]
- Texas Criminal Procedure and Evidence courses performance predicted performance on the Criminal subcomponent, but general Criminal Procedure did not.
- Texas Trial and Appellate Procedure performance predicted performance on the Civil subcomponent.
[201] Katherine A. Austin et al., Will I Pass the Bar Exam?: Predicting Student Success Using LSAT Scores and Law School Performance 102–03 (2017), https://dx.doi.org/10.2139/ssrn.2832835.
[202] Id. at 109.
[203] Id. at 114.
[204] Id. at 117.
[205] Id. at 118.
[206] Id. 121–23.
[207] Id. 121–23.
[208] Id. at 123–25.
[209] Id. at 125–26.
Bolus
Bolus
Bolus examined data from graduates of 11 ABA-approved California law schools and their performance on the July 2013, 2016, and 2017 California bar exams.[210] Data was available for 7,563 examinees. The sample was not reflective of the total examinee population because graduates of non-ABA approved law schools were not included, but they were “reasonably representative” of the ABA-approved school population.[211] Nine schools provided information on courses taken by students, five provided data on bar preparation courses taken, eight provided data on clinical courses taken, and eight provided data on externships taken.[212]
The author conducted a correlational analysis to examine the relationship between performance in doctrinal courses and bar exam performance. Only students who graduated in three years and immediately took the bar exam were included in the analysis.[213]
Bolus found that doctrinal course performance had a moderate positive correlation with both essay and MBE performance on questions related to that subject. But he also found that the correlation between course performance and test performance on questions unrelated to that subject was essentially the same.[214] He thus concludes that these relationships are likely due to “general legal knowledge,” and not knowledge specifically gained in any one doctrinal course.[215]
The author then looked at whether there was a relationship between doctrinal and non-doctrinal GPA and bar exam score. He found that the correlation between doctrinal GPA and bar score was larger than between non-doctrinal GPA and bar score, with an especially large difference on MBE score.[216]
[210] Roger Bolus, Performance Changes on the California Bar Examination: Part 2, at 7-9 (2018), https://www.calbar.ca.gov/Portals/0/documents/admissions/Examinations/Bar-Exam-Report-Final.pdf.
[211] Id. at 14.
[212] Id. at 48, 53.
[213] Id. at 49.
[214] Id. at 49.
[215] Id. at 50.
[216] Id. at 51.
Farley et al.
Farley et al.
Farley et al. studied data from five cohorts of University of Cincinnati College of Law students to explore factors that predict bar passage. The data included 404 students who graduated in 2012-2016 and took the Ohio bar exam.[217] Administrative data on these students was obtained from the law school, and bar passage and score data from the state supreme court. [218] Of the students, 334 passed the bar exam, 49 failed, and 19 were unknown.[219]
The researchers used binary logistic regression modeling to examine factors contributing to first-time Ohio bar passage. Three different models were used with variables included based on chronology: pre-law, post-1L, and post-3L.[220] The variables for the pre-law model were LSAT score, undergraduate GPA, undergraduate selectivity, whether they were a humanities/social science/policy major, gender, age, and underrepresented minority status. The post-1L model added first-semester and second-semester law school GPA. The post-3L model added final law school GPA and upper-level doctrinal bar course count.[221]
In the post-3L model, the number of upper-level elective doctrinal courses taken was a statistically significant and positive predictor of bar passage. Each additional course taken was associated with a 2.1 times greater odds of bar passage.[222]
[217] Amy N. Farley et al., Law Student Success and Supports: Examining Bar Passage and Factors that Contribute to Student Performance 9–10 (2018), https://dx.doi.org/10.2139/ssrn.3237546.
[218] Id. at 10.
[219] Id. at 13.
Kuehn and Moss
Kuehn and Moss
Kuehn and Moss examined ten years of data from graduates of Washington University School of Law and Wayne State University Law School to study the effects of taking experiential and doctrinal coursework on first-time bar passage.
The sample included 2,401 Washington University and 1,490 Wayne State students who graduated from law school in 2006-2015.[223] First-time bar passage data was obtained from school records or the state bar. For Wayne State graduates, only those taking the Michigan bar exam (85% of the class) were included in the study.[224] The researchers reviewed student transcripts to determine whether experiential or doctrinal courses were taken. They only recorded whether the course was taken and the number of credits, not the student’s grade (noting that most experiential courses are pass-fail).[225]
The authors performed a correlation analysis between the number of doctrinal course credits taken and first-time bar passage for the same groups of graduates.[226] They found a small positive statistically significant correlation. Looking at students by LGPA quartile, the correlation was only statistically significant for the fourth quartile and bottom 10% of students at Washington University, and the bottom half of students at Wayne State (but not the bottom 10%).[227]
They performed a logistic regression analysis of the fourth-quartile students at Washington University and the bottom half at Wayne State to examine the effects of taking more or less doctrinal courses than the average at the school.[228] They found that at Washington University, fourth-quartile graduates who took fewer than four doctrinal courses had reduced odds of passing the bar than those who took four (the school average), but there was no statistically significant increase when taking more than four.[229] At Wayne State, bottom-half graduates who took fewer than seven doctrinal courses (school average was six) had reduced odds of passing the bar than those who took seven (school average was six), but there was no statistically significant increase when taking more than seven.[230]
[223] Robert R. Kuehn & David R. Moss, A Study of the Relationship Between Law School Coursework and Bar Exam Outcomes, 68 J. Legal Educ. 624, 634 (2019), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3446111.
[224] Id. at 632.
[225] Id. at 633.
[226] Id. at 644.
[227] Id.
[228] Id. at 645.
[229] Id.
[230] Id. at 645–46.
New York Board of Law Examiners and AccessLex Institute
New York Board of Law Examiners and AccessLex Institute
The New York Board of Law Examiners and AccessLex Institute studied what behaviors and characteristics of bar-takers were most associated with success on the New York bar exam. AccessLex administered an online survey to New York bar candidates after they took the July 2016-February 2018 bar exams (but before their scores were released).[231] The survey asked candidates about demographic and household information, their law school experience, how they had prepared for the exam, and about their employment and financial situation.[232] The survey also asked if they had taken law school courses on UCC Article 9, Business Associations (Agency and Partnership), Conflict of Laws, Constitutional Law, Corporations, Evidence, Family Law, or Trusts and Estates.[233] The survey was modified after its first administration in July 2016.[234]
5,495 first-time takers and 484 second-time takers responded to the survey (about 40% of all first-time takers, and 24.5% of repeat-takers).[235] The first-time taker responders had similar bar passage rates to the total first-time taker group, while second-time taker responders had slightly lower bar passage rates.[236]
The authors used multivariate logistic regression modeling to examine which variables had an effect on bar passage, with separate models being used for first-time and second-time bar takers.[237]
The study found that taking a Corporations or Evidence course in law school had a statistically significant and positive effect on first-time bar passage.[238] The Evidence course also had a statistically significant positive effect on second-time bar passage, as did Conflict of Laws.[239] The number of doctrinal courses taken in law school was not statistically related to bar passage.[240] Finally, completing a law school bar preparation course was also not statistically related to bar passage.[241]
Rush and Matsuo
Rush and Matsuo
Rush and Matsuo examined the relationship between the number of doctrinal courses taken and bar passage. Their sample included 827 St. Louis University School of Law students who graduated between 2001 and 2005, and took the Missouri bar exam for the first time.[242]
The authors ran sample t-tests to determine whether there was statistical significance in the means of the tested variables, and then conducted binary logistic regression tests by class rank quartile.[243] All students in the first quartile passed the bar exam, even though they took different numbers of doctrinal courses, so no statistical tests were run on that group.[244]
For the second quartile, there was no statistically significant difference in the mean number of doctrinal courses taken between those who passed and failed the bar exam, and binary logistic regression found it was not a predictor of bar passage.[245]
For the third quartile, there was a small to moderate statistically significant difference in the number of doctrinal courses taken, and it was a predictor of bar passage. The authors say it may not be practically significant, however, because the difference in mean number of doctrinal courses taken was less than one course.[246]
For the fourth quartile, there was a small to moderate statistically significant difference in the number of doctrinal courses taken, but binary logistic regression testing did not find it was a predictor of bar passage.[247] For the bottom 10% of the class, there was no statistical significance in the number of doctrinal courses taken.[248]
Santa Clara University School of Law
Santa Clara University School of Law
Santa Clara University School of Law has three academic and bar success interventions: 1) the UP Point Policy, 2) Advanced Legal Writing, and 3) Bar Exam Planning Workshop.[249]
The UP Point Policy, implemented in 2015, is a requirement that all students take four upper-level doctrinal bar course (Constitutional Law I and II, Evidence, and Professional Responsibility) and earn a C+ or better.[250] Students who do not earn a C+ or better in those courses must continue taking doctrinal bar courses from an approved list until they successful meet the four-course requirement.[251] Students who do not meet that requirement by their final semester are required to participate in the other two interventions in their final semester.[252]
Santa Clara University analyzed bar passage data to evaluate these programs.[253] The report did not state what methods were used.
For the UP Point Policy, they examined whether the doctrinal course number was appropriate and found that taking and doing well in six to seven upper-level doctrinal courses gave students the best chance of bar passage.[254]
University of Maine School of Law
University of Maine School of Law
University of Maine School of Law administered an abbreviated MBE-style diagnostic exam from a commercial bar preparation company to students after their 1L year,[255] with the purpose of identifying students needing academic support and areas of the 1L curriculum that needed improvement.[256] The diagnostic was a four-hour, 120 multiple choice question test covering Torts, Real Property, Contracts, Civil Procedure, and Criminal Law.[257] Students could take the test at any time during a four day period.[258] Diagnostic scores were divided by subject area, as well as whether the question was on knowledge of a legal rule, application of the rule, or combined.[259] Exam results were shared with the students soon after taking the exam, and sent to them again soon after graduation.[260]
A Pearson’s correlation analysis was performed between several variables (scores on the diagnostic, LSAT, UGPA, law school grades) and bar passage.[261] For the class of 2022, 57 students took the diagnostic in September 2020, and they found that grades in the civil procedure and the first semester legal research and writing courses were most predictive of bar passage (correlation coefficients of 0.64 and 0.56, respectively), as well as 1L GPA (0.54).[262] On the diagnostic test, scores from Real Property Application (0.3), Analysis Combined Score (0.27), and Criminal Law Combined (.24) were most predictive.[263] This analysis was not done on the class of 2023 and 2024, presumably because their bar results were still pending at the time.
The school also looked at trends on diagnostic exam outcomes across the three cohorts. The average composite score for 2022 and 2023 was 57%, but it increased to 61% for 2024, driven mostly by increases in Real Property and Civil Procedure scores.[264]
Zeigler et al.
Zeigler et al.
New York Law School implemented its Comprehensive Curriculum Program in 2003. It includes a few different interventions: 1) a guided curriculum, 2) a second semester Principles of Legal Analysis course, 3) a fifth semester New York Law in National Perspectives course, and 4) a final semester Consolidated Legal Analysis course.
New York Law School’s guided curriculum is required for all students in the bottom 25% of the class after their first year.[265] Students on this path are required to take the three courses noted above, plus Corporations; Wills, Trusts, and Future Interests; Individual Income Tax, and New York Practice. Additionally, they must take a minimum number of doctrinal and skills-based electives from an approved list. Students in the bottom 10% of the class are also required to add an extra semester to their law school studies.
New York Law School also implemented several other policy changes intended to improve bar passage rates. All law school exams were required to be closed book, to better mirror the New York bar exam, and professors were encouraged to include multiple-choice questions in their exams.[266] The school also changed its attrition policies in 2003. Before the change, students with a first-semester GPA under 1.5 or a first-year GPA under 2.0 were placed on academic probation, with a committee making the final decision whether to dismiss them. After the change, students with a first-semester GPA under 1.75 or a first-year GPA under 1.9 were automatically dismissed. Students with a first-year GPA between 1.9 and 1.99 are presumptively dismissed, with a committee making the final decision.[267]
New York Law School saw first-time bar passage rates increase in 2006 (when the first class to experience the entire intervention graduated), 2007, and 2008.[268] Most of the increase in bar passage was in the bottom half of the class.
How do experiential coursework policies influence bar exam success?
How do experiential coursework policies influence bar exam success?
A few studies examined the effect of taking experiential courses on bar exam success. Experiential coursework includes clinics, externships, and simulation courses.
Most of the studies found no relationship between experiential coursework and bar exam success. One found evidence that clinic participation, specifically, had a negative effect on bar exam score.
Austin et al.
Austin et al.
Austin et al. examined the relationship between specific course performance on overall bar score and subcomponent percentile performance, as well as the relationship between skills courses and bar score.[269] They analyzed data from 1,572 Texas Tech University School of Law graduates who took the Texas bar exam between February 2008 and July 2014.[270]
The researchers did a t-test of specific student activity participation and mean final LGPA and bar exam score. They found that students participating in journal had higher mean final GPAs and bar exam scores;[271] students participating in clinics had slightly higher mean final GPAs, but slightly lower bar exam scores;[272] and students participating in Board of Barristers had higher mean final GPAs and bar exam scores.[273]
Bolus
Bolus
Bolus examined data from graduates of 11 ABA-approved California law schools and their performance on the July 2013, 2016, and 2017 California bar exams.[274] Data was available for 7,563 examinees. The sample was not reflective of the total examinee population because graduates of non-ABA approved law schools were not included, but they were “reasonably representative” of the ABA-approved school population.[275] Nine schools provided information on courses taken by students, five provided data on bar preparation courses taken, eight provided data on clinical courses taken, and eight provided data on externships taken.[276]
Bolus also examined whether taking law school bar preparation courses, clinical credits, or externships impacted bar exam score. He found a negative relationship between bar preparation courses and bar exam score, but that the relationship disappeared after controlling for LGPA (since they were often required for low GPA students).[277] He found no relationship between clinical credits and bar score.[278] He did find a weak positive relationship between externship participation and bar exam score, but the relationship disappeared after controlling for LGPA.[279]
Johns
Johns
Johns studied three years of data from graduates of University of Denver Sturm College of Law in 2008-2010 to explore the relationship between externship participation and first-time Colorado bar exam score.[280] The sample was 637 graduates with a mean bar exam score of 302.58 (276 is a passing score) with a standard deviation of 22.445.[281] 62% of the sample participated in at least one externship, with the majority (55%) doing one or two.[282] Across LGPA quartiles, students who did and did not participate in externships had similar first-time bar passage rates (with externship participants slightly higher), except in the bottom quartile where externship participants had a pass rate 16 points higher than non-participants.[283]
Johns used linear regression analysis to determine the effect of externship participation on bar exam score. He found that externship participation did not have a statistically significant effect on bar exam score, and neither did the number of externships completed.[284]
Johns proposes that externship participation may have no effect on bar exam score because the bar exam may not be a valid test of legal practice skills and competency.[285]
[280] Scott Johns, A Statistical Exploration: Analyzing the Relationship (If Any) Between Externship Participation and Bar Exam Scores, 42 Okla. City U. L. Rev. 281, 285–86 (2018), https://dx.doi.org/10.2139/ssrn.3099761.
[281] Id. at 290–91
[282] Id. at 293.
Kuehn and Moss
Kuehn and Moss
Kuehn and Moss examined 10 years of data from graduates of Washington University School of Law and Wayne State University Law School to study the effects of taking experiential and doctrinal coursework on first-time bar passage.
The sample included 2,401 Washington University and 1,490 Wayne State students who graduated from law school in 2006-2015.[286] First-time bar passage data was obtained from school records or the state bar. For Wayne State graduates, only those taking the Michigan bar exam (85% of the class) were included in the study.[287] The researchers reviewed student transcripts to determine whether experiential or doctrinal courses were taken. They only recorded whether the course was taken and the number of credits, not the student’s grade (noting that most experiential courses are pass-fail).[288]
The authors performed a correlation analysis between the number of experiential course credits taken and first-time bar passage for the full sample at each school, as well as for different law school GPA quartiles.[289] The correlation between number of experiential credits and bar passage was very small and negative, and only statistically significant for the full Wayne State sample.[290] Students who passed and students who failed the bar exam took nearly the same number of experiential credits.[291]
They also performed a logistic regression analysis to control for law school grades and found no relationship between experiential credits and bar passage.[292]
[286] Robert R. Kuehn & David R. Moss, A Study of the Relationship Between Law School Coursework and Bar Exam Outcomes, 68 J. Legal Educ. 624, 634 (2019), https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3446111.
[287] Id. at 632.
[288] Id. at 633.
[289] Id. at 638.
[290] Id.
[291] Id.
[292] Id. at 639–40.