Race-Based Poll Taxes And 20th-Century Discrimination

Professor Jeremy Bearer-Friend of the George Washington University Law School discusses poll taxes in the 20th century and how they were used to target taxpayers based on race and ethnicity.

This transcript has been edited for length and clarity.

David D. Stewart: Welcome to the podcast. I’m David Stewart, editor in chief of Tax Notes Today International. This week: malicious taxation.

In the U.S. taxes have been used throughout history to restrict who gets to vote. The most recognizable tax policy of this kind was the poll tax. Our guest today takes a look at poll taxes and other race-based tax weapons and their effect on the 20th century.

This week’s episode is part of a series we’ve been doing examining how tax rules affect marginalized groups. We’ll include links in the show notes to our previous episodes on the intersection of tax and racial inequality, LGBTQ rights, feminism, diversity and international tax policy, tribal taxation and wealth, and inequality.

Here to talk more about this is Tax Notes contributing editor and tax historian Joseph Thorndike. Joe, welcome back to the podcast.

Joseph J. Thorndike: Hey Dave, it is great to be back on my favorite podcast.

David D. Stewart: Well, I understand you recently did an interview about this subject. Could you tell us about your guest?

Joseph J. Thorndike: So I had a chance to talk with Jeremy Bearer-Friend. He’s an associate professor of law at George Washington University, and he’s an expert on the civic nature of taxpaying. The way that the act of paying taxes shapes a person’s relationship to government.

David D. Stewart: And what sort of issues did you get into?

Joseph J. Thorndike: Well, we talked about his new paper, “Race-Based Tax Weapons.” In this paper he tries to look at and answer some of those questions about the links between race and ethnicity and rights and privileges of citizenship.

But as you said, he does this by looking specifically at poll taxes both in the U.S. and in other places. I think it’s a really insightful way to examine this.

David D. Stewart: All right, let’s go to that interview.

Joseph J. Thorndike: Jeremy, welcome to the podcast.

Jeremy Bearer-Friend: Thanks so much for having me, Joe.

Joseph J. Thorndike: Well, it is always a pleasure seriously. So let’s get started here. You’ve written this paper about racially targeted tax weapons. Let’s start with something pretty basic. What’s a tax weapon?

Jeremy Bearer-Friend: I wanted to look at taxes that try to harm specific groups, and I focused on taxes that target harm based on race, ethnicity, or ancestry.

Joseph J. Thorndike: I mean, I get the sense that this is more than just a title. It’s sort of a concept that you’ve developed here. So why is this concept important? This notion of a weapon, of a tax-based weapon.

Jeremy Bearer-Friend: So those of us who support a robust fiscal system that has a broad safety net and lots of federal revenue generally think of taxes as benevolent, right? That we want there to be revenue, it’s the price of civilization. But that doesn’t mean that there aren’t some forms of taxation that should concern us.

And it’s a separate question to look at tax policy and ask, “Is this tax targeting harm on a specific group based on their race, ethnicity, or ancestry?” I think that’s an independent question of some of the other considerations that are also important. We should also be thinking about measurability, thinking about efficiency. But we can also ask this question: Does this tax target harm at a specific group based on the race, ethnicity, or ancestry?

Joseph J. Thorndike: Does it matter if it does that deliberately or not? I mean, I can imagine that some taxes might inflict some targeted harm without anyone meaning to do it exactly. Does it matter?

Jeremy Bearer-Friend: If we care about the harm, it really doesn’t, right? If we care about certain groups being harmed relative to similarly situated taxpayers who otherwise have the same profile, then I think we should just care about repairing the harm.

It’s also, I think, a messy process to try to attribute intentionality to a legislative body. That’s not just one person’s mind anyway, that’s the whole Congress’s mind. So that would lead I think to less clean results.

Joseph J. Thorndike: Right, doesn’t stop people from doing it, of course. We make generalizations about that kind of stuff all the time, but you’re right, I think it is a little problematic.

All right, so for a historian like me, especially a U.S. historian like me, poll taxes are all about Jim Crow. It’s all about the southeastern United States for the most part, but they are more than that as you made clear in the paper.

And you include some case studies of poll taxes in other places, other times. Can you tell us a little bit about each of them just to sort of get us oriented about what the subject matter, the nuts and bolts of what your paper is about?

Jeremy Bearer-Friend: I’d love to, and I think a good place to start with that is even just defining poll taxes, right? So we’ve looked at race-based tax weapons, and that’s the title of the paper, quite provocative purposefully. But that paper exclusively is looking at poll taxes.

And poll tax is derived from the middle English word for head. It’s the top of the head that grows hair, and a poll tax is a tax on personhood. If you are a person, you have to pay tax and usually a certain type of person: if you’re male, if you’re of a certain age.

In the U.S., in the southeastern states that Joe mentioned, there was a period of associating the tax with voting rights. That is, nonpayment of the tax would then prevent one from being able to vote. But that’s an added feature you can put on a poll tax or not.

The core component is that the taxable base is personhood, the number of people in a jurisdiction. And so in this paper what I wanted to do is look at two poll taxes that are explicit in the law that they are based on the race, ethnicity, or ancestry of the taxpayer.

So you could look at a photograph, and if you’re able to identify race or ethnicity that way, you could see who is paying tax and who isn’t. Rather than an income tax, if you looked at a picture, you wouldn’t know who was liable.

And then I looked at two poll taxes from the 20th century that make no mention of race, ethnicity, or ancestry at all in the statute. I thought there would be some utility to comparing those two things. Let’s look at poll taxes that mention race and ethnicity and poll taxes that don’t and see if there are different outcomes.

Joseph J. Thorndike: So what four places are you looking at?

Jeremy Bearer-Friend: One of the poll taxes is one of these Jim Crow taxes adopted after Reconstruction to limit the right of Black citizens to exercise their voting rights. And that was in Texas and adopted in 1902. And there is a tax on all men of a certain age, a few exemptions for disability and a few narrow categories. And that one is tied to the right to vote. So nonpayment means inability to vote.

Then I looked at a poll tax, a tax on heads, in California. This one was the titled Alien Poll Tax. And this was based on whether someone was native-born in the U.S. or not. That in the time period was also entwined with racial identity. And this tax was from 1921 in California. It was actually quite common to use poll taxes as a tool for managing immigration in other jurisdictions. And California is just one example of that.

I then looked at a poll tax that was imposed by the British colony and protectorate of Kenya. So here a colonial government imposing a poll tax that is also based on race, ethnicity, or ancestry in the statute itself. That if you are an individual of European or Asiatic extraction, you’re exempt, but if you are native African, you are liable for this tax. It’s also limited to men of a certain age.

And then lastly, I looked at the poll tax from the Thatcher government in the United Kingdom adopted in the late 1980s that tried to replace property tax system with a head tax system, a poll tax system.

Joseph J. Thorndike: You mentioned something earlier about how some of these poll taxes involve an element of visibility, like you can tell who’s going to be paying just by what they look like. But that’s going to be problematic with some of these taxes. You can’t necessarily recognize an immigrant when you see them. And honestly, this is one of the key stories in the history of the American South, you can’t always recognize an African American by looking at them in the South. So that’s an added wrinkle it seems like to me.

Jeremy Bearer-Friend: Right, and I appreciate you raising that. It’s reductive to think that race, ethnicity, or ancestry is reducible to appearance. Those categories are more complex than that.

Joseph J. Thorndike: And we can come back to this, but it just occurred to me that the way — and we’ll talk about this — but the way some of these poll taxes are framed, they’re not mentioning specific groups or ethnicities, and yet they’re finding proxies for it. And the proxies might be actually a lot better at identifying the groups they’re trying to target than the actual racial category which is invisible sometimes.

Jeremy Bearer-Friend: Right, the tax collector or the tax authority might not have any question related to race or ethnicity on the form, let alone any language about race or ethnicity in the statute, and yet be quite effective at targeting specific groups.

Joseph J. Thorndike: That’s really interesting. So before we get any further into what you actually wrote about, let me ask you that law review editor’s question. So why do I care about this? You’ve got these case studies. I mean, you can tell me why poll taxes are important, but what’s good about doing this comparison? What’s the value here?

Jeremy Bearer-Friend: So there’s a long tradition of scholarship in critical race theory that looks to disparate racial impact of laws that make no mention of race or ethnicity. That is well understood in other areas of commercial law like bankruptcy law or in criminal law. And I wanted to take on some of that work here in tax. And to do that, I wanted to compare taxes that are explicit about race or ethnicity to taxes that aren’t. Sort of a unique comparison opportunity. It’s not like we have an income tax that was explicitly based on race or ethnicity.

So there’s important work that’s been done, including by Dorothy Brown and Beverly Moran, about disparate racial impact of various tax policies that make no mention of race or ethnicity. And what I’m adding to the conversation is looking at taxes that actually do mention race or ethnicity and comparing those with taxes that don’t.

Joseph J. Thorndike: So you’re writing this very comparative history, and I just want to push you a little bit on that. And it’s partly because I personally have always been afraid of writing comparative history. So I admire your courage to do it. I always feel a little out of my depth when I do it. Because you can’t be a specialist in everything in every place in time, but your subjects really do vary across both place and time.

Are such comparisons really workable, or can they really tell us that much, given that the specificities of each case study seem like they might overweigh whatever commonalities exist?

Jeremy Bearer-Friend: It really is an audacious project, certainly from a historian standpoint. And when I was listing the jurisdictions I looked at, I’m sure the listeners thought so too, hopping from Texas, California, Scotland, and Kenya. So different continents and different decades. I’m looking throughout the 20th century. So there is a real downside to that.

It does mean not having the rich historical context for each of those taxes and understanding the societies that surround those taxes. On the other hand, in all four of those jurisdictions, I’m reading the statute, I’m actually lifting out the law itself and comparing those laws to each other. And that has a bit more consistency over time.

In the legal profession, we are expected not only to study what the law is, but then to make arguments about what the law should be. And comparing these statutes across different jurisdictions is a very helpful tool for assessing what might be attractive or unappealing. But I think in the end, my findings should be evaluated with that caveat. That this is achieved by someone who does not purport to be an expert in colonial Kenya, right?

I am an expert in tax, and I was looking at the tax statutes from that jurisdiction, but I cannot say I’m an expert in that region.

Joseph J. Thorndike: I mean it seems completely reasonable to me. My friend and colleague, Dennis Ventry, once said to me that legal scholars always want their history to do something for them. They don’t want to just write history for history’s sake.

So just before we go any further, I want to ask you one more question, sort of organizing question here. Which is, you frame the article among other ways as a contribution to the law and political economy movement. And I think it might be interesting for our listeners to know a little bit about that. Because if you go to the Law and Society meeting every year and other meetings too, but you know a lot about the law and political economy movement because there are all sorts of sessions on it. But for most people I think it’s still relatively unknown even in the scholarly tax world. I’m not sure that that many people know. And so could you tell us a little bit about it, what it’s trying to do? Are they all tax lawyers among other things?

Jeremy Bearer-Friend: Right. So mostly they’re not tax lawyers, though they are law professors, and I won’t claim to speak for them, and in some ways I’ll get in trouble for summarizing their work. But one thrust of it — this isn’t comprehensive — but one key component of it is decentering the economic approach to studying law and the economic approach to making recommendations about law. So in order to look at tax policy from a perspective that isn’t just within the public finance literature. And I think I do that here by being rooted in the statute and with my concerns about tax history and also racial bias.

But it connects even with what you were mentioning earlier about disciplinary norms and historians versus economists. In economics, it is quite typical to make a recommendation about what the law should be or what policy should be. It’s not stigmatized. And in economics it’s also quite typical to make projections about the future to say, and if you do the X, Y will happen. I don’t see historians generally being comfortable with that, saying, “If you did this policy, you would get this outcome and I recommend you choose this.”

But the consequence of that disciplinary norm is that economists have then had far more influence on policymaking than historians have, which doesn’t necessarily lead to the best outcomes. So I think law and political economy is trying to recenter, rebalance that power, bring in other voices.

Joseph J. Thorndike: It seems like it recenters certain issues in tax policymaking that also have gotten ignored a lot for a long time. And I’m thinking specifically of questions about fairness or even justice to the extent that we can distinguish those two.

I remember asking an economist why the economists of the 1940s used to write about those topics all the time and now they really never write about them and are almost dismissive of them. And his answer was that the economists decided they had no comparative advantage in talking about fairness and justice compared to the philosophers or even the historians or anyone else. So they just seeded the field. But since economists are in so much control over the tax policy of this world, it had the effect of pushing those topics to the side for a long time in the formulation of tax policy.

And to some extent I think that affected the legal academy too. Law professors used to write more about those amorphous topics and then moved more toward the economic interpretation.

Jeremy Bearer-Friend: And just to elaborate on that with one example, I think a lot of those outside of the academy would be surprised to hear that it’s quite common in public finance to point to poll taxes as an ideal tax. I mean they are regularly used as a hypothetical tax that has the least inefficiencies associated with it, and we compare it as a baseline against income taxes or property taxes or consumption taxes.

And so I wanted to look at poll taxes myself, not as an economist, and see as a lawyer what those laws actually are. And I think it is quite worrying that they would be used as a baseline for other taxes. It doesn’t seem appealing to me to do that.

Joseph J. Thorndike: I mean often I feel like those things are like — there’s also an implicit acknowledgement and caveat that comes when they’ll say, “Well, and of course these might be unfair, but they’re super efficient,” and then they’re often going on efficiency. A lot of the time even the well-meaning practitioners of that school will offer acknowledgment about what’s being sidelined here, but then completely blow past it.

Jeremy Bearer-Friend: But then not actually change the practice. I’ve seen that done quite effectively. I’m sure they would all nod their heads and agree with me that I had the right critique and then just continue on their way doing the same thing.

Joseph J. Thorndike: A lot of those questions about power and fairness and justice are not susceptible to measurement very well. So they really make economists uncomfortable, and they often make certain kinds of legal scholars uncomfortable. Whereas for a historian, we live in the gray areas. So it’s totally fine for us.

Although my sense is that historians play some role in this law and political economy movement, they’re not necessarily at the vanguard of it, they’re not really the vanguard of anything anymore these days. But I think that it’s a historicized approach. One that most historians I think are pretty comfortable with, certainly compared to the alternatives.

Jeremy Bearer-Friend: And you’ll see, I don’t think it would make sense in this paper if I had tried to calculate the exact effect size of a certain wording included in the statute or not. The fact that the cases are so diverse that wouldn’t really be a meaningful result. But that doesn’t mean there aren’t other findings, but they are qualitative.

Joseph J. Thorndike: All right, so we’ve talked a little bit about what poll taxes are and in their purest form, pretty neutral. They apply to every single human being in a jurisdiction that might seem neutral, but in fact they aren’t really neutral, and they’ve pretty notoriously targeted certain groups of people, perhaps inadvertently, sometimes deliberately. But what are the ways in which you can target an ostensibly neutral tax, one that’s just on everybody?

Jeremy Bearer-Friend: Thank you for this question. It’s really at the heart of the paper. And actually if there’s one takeaway I hope the listeners hold onto, it’s that the most famous example of poll taxes that Americans are familiar with, this use of tax policy to prevent Black citizens from voting in the South.

If you go to that statute and you read the law, you will see no mention of tax liability based on the race, ethnicity, or ancestry of that taxpayer. That was not included in the law that you only owed the tax if you were Black. And in fact, in Texas there had been poll taxes prior to Reconstruction that were limited to white men. So race and ethnicity have actually already been in the law before, here it’s being removed and used as this tool to oppress a specific group.

Joseph J. Thorndike: That seems like it gives them like a, what an ironic, rhetorical advantage. They can claim to be making the tax less targeted, more neutral, and in fact be making it the opposite of that.

Jeremy Bearer-Friend: Right. So that’s an example then of not only how to do targeting within a poll tax, but also why the absence of language about race or ethnicity might make targeting even more effective because it actually conceals the agenda.

So part of how that worked in the Southern context was through strategic administrative dysfunction. So the tax nominally is on everyone, and that’s something that’s also quite interesting about using poll taxes to look at targeting taxes nominally on all persons within a certain category — here all men over 18 — and yet you can achieve targeting then by only enforcing the law against the groups that you are trying to keep in this case from voting.

So there’s very minimal investment in any type of tax collection for these poll taxes. It’s totally diffused in the region. It’s not seen as something that will be enforced except for against certain groups on election day.

There’s also in the law a huge lag between when the tax is due and when election day is as well, so they don’t end up being paired. And that also adds to the dysfunction of the administration.

Joseph J. Thorndike: So you see that in the poll tax in Texas really clearly. And I’m sure you would find that in most if not all poll taxes in the U.S. South for instance. But do you see that in your other case studies too? That sort of strategic dysfunction that comes up everywhere.

Jeremy Bearer-Friend: In the U.K. context, there was enormous unrest related to this switch from property taxes to poll taxes. But again, there was quite a bit of discretion amongst local councils of where they were going to target enforcement resources. And typically enforcement was most aggressive in cities, urban areas and urban areas also correlating with race, ethnicity, or ancestry in that country.

Joseph J. Thorndike: So oppression in that case really best carried out at the local level. They’re best equipped to do the oppressing when they’re closest to the taxpayer. Is that the idea?

Jeremy Bearer-Friend: I think it’s not that it could only be done through local government, but that has been a strategy for racial targeting in the past, also even in the administration of many New Deal programs. To, at the federal level, try to have some neutrality but then delegate so much to the state and assume in the states there will be a different kind of sorting.

Joseph J. Thorndike: So I’ve got, drawn from the paper here, your three ways of targeting taxes being racial proxies, finding something that generally correlates with race or the ethnicity that you’re trying to control. Tax localization, which we’re just talking about moving enforcement closer to the local level. And then the strategic dysfunction about making a tax hard to comply with so you have an opportunity to come after people.

So when you’re using these proxies and the nefarious purposes behind a poll tax are sort of hidden, is that better or worse? It just seems, out of the gate, it seems like explicit targeting would be the worst, but I get the feeling like that’s not your answer.

You write at the start, poll taxes also demonstrate how the absence of explicit targeting can better enable targeting. So could you just expand on that a little bit? You already have touched on it.

Jeremy Bearer-Friend: Right. And this also gets to why I needed history to do work for me. Because in our current constitutional order, we just wouldn’t be able to pass laws that have this type of targeting in the tax policy. So I had to look historically in order to find these points of comparison.

But in the case of the Alien Poll Tax in California, you have successful litigation within a year of implementation. So that poll tax only lasts for a single year because it’s found to be unconstitutional. And that tax was explicit in its statute of being based on this ancestry component.

So when including that language in a tax, you obviously make it more vulnerable to legal challenge. And so that does seem less effective. In the end, it doesn’t stay on the books as long as the poll tax in the southeastern United States, the Texas example.

That doesn’t even get removed until the ’80s. We have a constitutional amendment that bars implementation of it before then, but it stays on the books for a really long time. And I think part of that longevity is because of the way that targeting was hidden within it.

Joseph J. Thorndike: That makes sense. All right, the last section of this paper is disarming the weapons. This is really the so what section here. This is your policy prescription. So what are the things that you’ve taught us here about poll taxes and these other jurisdictions, what do they tell us about crafting policy right now?

Jeremy Bearer-Friend: More than anything, it points to the importance of data. It points to why it’s so necessary to disaggregate tax data by race and ethnicity in order to identify whether targeting is happening and then try to intervene in it. We need more transparency about what our government is up to. And I think taxpayers are entitled to know some basic descriptive statistics on race and ethnicity.

So if we think that it is wrong for tax policy to target certain groups based on race or ethnicity, if we think it’s wrong that similarly situated taxpayers would get different outcomes based on their race or ethnicity, then we’d want to do something about it, and we can’t do something about it until we have more data. I mean we need to know that it’s happening.

Joseph J. Thorndike: Right. So make sure I understand you here. What you’re saying is that the ostensible neutrality that comes through in not talking about race in any fashion, not counting things in racial categories because that seems to inject race into a place where it shouldn’t be, right? That’s in fact a problem. That’s perpetuating the problem under the guise of preventing the problem. Am I right?

Jeremy Bearer-Friend: Right. It’s wholly insufficient to ensure nondiscrimination. Not asking about race and not mentioning race or not including race and data doesn’t prevent policy and enforcement from being discriminatory. And another area where I think history was doing work for me is that I chose historical policies rather than 21st century policies because I wanted people to listen.

I think there might be more of a knee-jerk reaction if I was finger pointing at the 21st century, but people are perhaps more comfortable finger pointing at the 20th century. So I can say, “Well, we all agree that these poll taxes were bad, so we’re all on the same team there.” But then when I elaborate on the findings, “Look, those taxes didn’t mention race or ethnicity, but they were clearly targeted.”

We in the 21st century do have taxes that also don’t mention race or ethnicity. Are they targeted? And to answer that question, we need better data.

Joseph J. Thorndike: I think to me that has been one of the great contributions of the scholarship in the last few years, like Dorothy Brown’s book. To really bring to the surface the way all of these ostensibly neutral, and maybe even neutral in intent sometimes, policies have ended up being anything but neutral in operation, but that there’s a norm about not talking about these things. That if you talk about race, that’s a problem.

And what you’re doing, it seems to me is trying to sort of take the stigma away from talking about it, from putting the subject on the table and then doing the math, doing the counting. Because you can’t make the problem of racial oppression go away by pretending that it doesn’t exist. Is that right?

Jeremy Bearer-Friend: Yes, exactly true. And we are seeing some progress in the administration when there was wide reporting about disparate audit rates by race that did lead to congressional hearings and that’s led to new levels of review by the Internal Revenue Service. And I think they understand that that is not tenable. It needs to change. And again, in order for them to demonstrate that they’ve changed, they’re going to have to document. How did they verify that they’re not doing that?

Joseph J. Thorndike: All right, so near the end of the paper you make a suggestion that I thought was really intriguing, which is that there’s an awful lot of talk, especially among conservatives, about weaponizing the IRS. The way we could turn the IRS against people, that by implication we should view the IRS as the agent of harm and oppression. And that by looking at poll taxes and the way that they have been enforced or not enforced, it kind of changes the dialogue around weaponization.

And yet pick up on that dialogue at the same time because it seems like a pretty resonant dialogue, right? That Americans — and I think that there’s reason for that — I think Americans are justified in being suspicious of the tax collector. I mean it just makes like every other society for millennia. But it seems to me like you’re suggesting a way to change that conversation and turn it in a different direction.

Jeremy Bearer-Friend: So I hope that I am providing a more objective definition of weapons so that we can then accurately test whether something is or isn’t. And I lean on what is now a somewhat disfavored principle of horizontal equity to make that definition. I say, as you all recall from your own federal income tax classes, horizontal equity is that similarly situated taxpayers should be taxed similarly.

And so I describe these weapons as when a similarly situated taxpayer is taxed differently, and it is targeted based on their race, ethnicity, or ancestry. We could quite easily then, using that standard, ask, “Is it true that those with conservative political beliefs are being treated differently by our IRS?” I think that’s a testable question. I’m pretty confident the answer of that question is no, that that targeting is not happening. So it’s an inaccurate statement empirically.

So it’s helpful to have the definition to then measure the outcome, but I do think it’s a fair question to ask. Is tax policy harming targeted groups? Let’s ask that question, and let’s ask that along [with] other questions. It’s not the only standard to evaluate tax policy.

Joseph J. Thorndike: You do have the very interesting section in the paper where you talk about the disfavor that has fallen upon horizontal equity, which has always seemed so curious to me. Because like a historian and a person who writes about contemporary politics, I’ve always been struck that horizontal equity is more resonant with the voting population than most other standards of fair tax policy.

So it’s one of those great moments where the academy really falls out of step with the general population. I think people believe in horizontal equity. That’s my sense of it from every poll I’ve ever seen. It is the one standard that everyone can agree on. People can fight about what progressivity means and how progressive something should be, but people don’t fight about the idea that we should all pay taxes equally.

And so the extent that you can engage that set of ideals with the voting population, I’m thinking strategically here, basically. It seems to me that that’s helpful. You’re likely to find a willing audience for that. I don’t know, was very impressed by that point you made and the direction that your paper takes it there.

Jeremy Bearer-Friend: Thank you. I hope so. I hope we see progress on this data front and on the racial targeting and auditing.

Joseph J. Thorndike: Well, Jeremy, it’s been great to have you here on the podcast, and I’m sure we’ve really missed huge swaths of the paper, but it really was enormously helpful for me. And I am a card-carrying historian. I was very happy with the way you were getting your history to work for you in this.

So I think it’s doing what you need it to do. I recommend that anybody who wants to learn more about this, this is a great paper.

Jeremy Bearer-Friend: And it’s available on the Social Science Research Network, SSRN. You can download it for free. Titled “Race-Based Tax Weapons” by professor Jeremy Bearer-Friend.

Joseph J. Thorndike: Thanks again, Jeremy. Great to have you.

Jeremy Bearer-Friend: Thank you. Huge fan of Tax Notes. Bye-Bye.