Earlier this fall, I published an op-ed raising concerns about discrepancies in NASBA’s publicly reported 2024 CPA Exam performance data in “The NASBA Report: Candidate Performance on the Uniform CPA Examination – 2024 Edition.” NASBA has been issuing these reports for 45 years, but the latest version contains material inaccuracies.
Processing Content
NASBA reports it ” … began gathering data on CPA Examination candidates in 1982 and has published reports on performance and selected characteristics since 1985, demonstrating NASBA’s role as a trusted resource to Boards of Accountancy, the academic community and the accounting profession. Due to the necessary and important focus on bringing CPA Evolution to fruition, NASBA had paused publication until fall of 2024, while significant changes to reflect the new exam and licensure structure were underway to the National Candidate Database, plus other NASBA systems and platforms.”
Unfortunately, the new reporting system designed for the new exam structure has produced data in the 2024 book and associated rankings that are largely inaccurate, yet the book remains publicly available, and as of this week, I still see universities reporting the inaccurate rankings.
After several months of detailed candidate-level review between the NASBA data team and our team in the accounting department at the University of Northern Iowa, we now have a clearer understanding of what occurred and why it happened. We worked together to reconcile CPA exam results for the University of Northern Iowa’s 2024 CPA exam candidates. What we found was a material distortion of program-level outcomes reported by NASBA. Of the 61 UNI CPA candidates active during the period, 28 candidates (nearly 46%) were missing from our university results. Where did they go? After a candidate-by-candidate review, NASBA confirmed the following errors:
- 19 candidates were listed under a community college.
- Six candidates appeared as having no degree.
- Three candidates were attributed to a different institution.
The root cause
The root cause of the discrepancies is a result of how education data is collected at the time a candidate applies to sit for the CPA exam, and the fact that degree information is not updated after graduation. This distinction matters. As the University of Northern Iowa has done for decades, an increasing number of accounting programs now intentionally encourage students to sit for CPA exam sections before graduation, often embedding CPA review into the final undergraduate or graduate semester. Doing so improves pass rates and aligns with the profession’s goal of increasing the CPA pipeline. Ironically, programs that do this are now systematically penalized in public data under NASBA’s current reporting practices and those programs’ students’ CPA performance is reported erroneously.
Near-term improvements
NASBA has acknowledged the limitations in the current process and recently shared forthcoming improvements in a national webcast. The most significant improvements are:
- NASBA will now refrain from all institutional rankings. Publishing comparative rankings when data errors exist risks misleading stakeholders. It also has real consequences for programs and the people within them. Furthermore, outputs may be, in part, influenced by student aptitude, not program quality. Without controlling for confounding factors, it is impossible to determine program quality merely from exam results, and doing so could lead students to make poor choices.
- NASBA is returning to offering custom reports. My colleagues and I at the University of Northern Iowa have been fortunate to work with NASBA to receive the first new custom report, and we are still tweaking it with their data team to develop a format that will be useful to not just UNI, but other universities as well. The expanded option includes useful data like demographics, which can help universities glean a lot more from the results, and perhaps identify populations who could use more targeted preparation strategies. Hopefully a relatively standard format will allow NASBA to produce the custom institutional level reports quickly and for a relatively low cost.
There is work left to do
Still, the root cause of inaccurate results will still persist in the 2025 Candidate Performance Book, according to NASBA, reducing the value of the data. This mainly applies to institutions that have community college transfer students and mechanisms that help students take the CPA exam before graduation, both are growing trends nationally. One indicator of the distortion is that a community college appears to be a top-ranked program nationally despite bachelor’s degree requirements for CPA licensure in Iowa.
Looking ahead, NASBA still needs to correct its data collection problems and results should reflect a candidate’s correct university affiliation. Perhaps collecting all the relevant data on the front end or allowing for updating degree information after graduation would significantly improve accuracy.
This investigation brought clarity to problems with NASBA’s current data-reporting structure. After two years of the new CPA Evolution exam, we still do not have reliable public data to evaluate how candidates did at the program level. But hats-off to NASBAs new leadership for acknowledging the problems and for their transparency. You can’t make improvements if you don’t acknowledge the deficiencies and NASBA has done that and appears to be working on getting it right. We are the “accounting profession”. Getting it right shouldn’t be optional.
Credit: Source link
