A Technical Note: Comparison Over Time and Editions of RLS

As RLS grows—aiming to measure a fuller potential for safeguarding the free exercise of religion and to provide a richer set of data to interested citizens, legislators, and scholars alike—it is important for users to note the scope of what the index is combining in a given year.

Consider any individual state. Its score on RLS 2023, relative to 2022, will reflect two (potential) sources of change: what the state did to its laws from year one to year two (especially on the original eleven safeguards from RLS 2022) and what RLS added to its scope in the form of new safeguards for 2023. Many users of RLS, understandably, will want to celebrate any increase in their state’s score, which would indicate that the state in fact maintains a greater proportion of potential safeguards than previously reported. It is worth noting, though, that only a substantive change in a state’s laws between years has practical implications for its residents. If, alternatively, a state’s RLS score changes only because of the composition of what RLS measures, the new observations do not necessarily have any felt impact on residents in that state over time.

Though different in their implications, both sources of change to a state’s RLS score are meaningful. The differences, however, must be carefully understood when making comparisons within a state over years and when comparing those improvements (or declines) relative to other states over the same time period.

Figure 2: Breaking Down the Overall Change in a State’s RLS Score

Figure 2 from the executive summary breaks down the overall change in five states’ scores over the two years of RLS: 2023’s number 1 and number 50 states, as well as the three states that made changes to their statutory laws. The horizontal axis labels “2022 Score” and “2023 Score” indicate that the vertical measures above them are the official, reported RLS scores in that year. The label “2023′” on that same axis indicates that the simulated scores plotted directly above are what the scores in 2023 would be had we not expanded the set of safeguards measured.

Connecting all the points from left to right, then, forms something like trajectories where upward-sloping lines can be interpreted as improvement and downward-sloping lines as decline. However, only the first line segment for each of the five states illustrated above is an “apple-to-apple” comparison (since the safeguards are held constant). Still, RLS 2023 is a meaningful “orange” in that it provides more information on the circumstances of religious liberty in each state and across states.

Notice that Illinois’ score increased, on net, by 4 percentage points from 2022 to 2023. That change combines the fact that Illinois made no improvements on the original eleven safeguards from RLS 2022 (indicated by a horizontal line segment at the start of the blue trajectory) but scored well on the three new safeguard areas. In fact, Illinois has in place in 2023 all five new component items that comprise RLS 2023’s three new safeguards. An Illinoisan might understandably be proud to be number 1 on the aggregate RLS 2023 score, and certainly should be glad Illinois’ score did not decrease, which would indicate a lack of the newly measured safeguards, but we should also be clear that none of the change in the overall score for Illinois was due to statutory changes.

Similarly, West Virginia made no changes to its existing laws in time for RLS 2023’s most recent measurement (represented by a horizontal line segment at the start of the red trajectory). But its score moves opposite of Illinois’ with the addition of the new safeguards in RLS 2023. West Virginia, in fact, has in place none of the five items comprising the three new statutory safeguards measured in RLS 2023.

We can contrast these simpler cases with the more complicated cases of the three states that made changes in 2022 that affected their RLS 2023 score. South Carolina (in green) and Rhode Island (in yellow) made statutory changes that improved their degree of safeguarding on the original eleven safeguards. In fact, South Carolina adopted a general conscience provision for health-care providers that improved its score greatly (notice the steep slope of its first line segment) while also seeing improvements in its score due to the new items added to RLS in 2023 (the second line segment). It’s worth noting that implementing statutes protecting conscience for health-care providers of a broad array of health-care services is a quick way for a state to improve its RLS score. Not only will it satisfy RLS’s requirements for that particular safeguard, but often these laws have practical spillovers, improving state scores on abortion, sterilization, and contraception refusal safeguards or even health insurance contraceptive mandates.

Between RLS 2022 and 2023, Rhode Island no longer required excuses for absentee voting, newly offering an alternative to voting in the polling place for religious reasons (e.g., holidays) or no particular reason at all. This improved its score on the original eleven safeguards, but that improvement was moderated by the addition of new items in RLS 2023 (as indicated by the initial upward-sloping line segment followed by the downward slope of the second line segment in yellow). In the end, Rhode Island still finishes with a higher score on RLS 2023 than RLS 2022.

In the case of Connecticut (in purple), which eliminated the religious exemption for childhood immunizations, its score initially decreases (shown as a downward-sloping portion at the start of its trajectory), but its performance on the five new items comprising three new safeguards moderates its descent (note the subsequent uptick in the final line segment), recovering some of its loss for the aggregate score reported for RLS 2023.

Figure 3: Weighting Safeguards

Finally, in making comparisons of a state’s scores across time, one should note that, with an expanding scope, each of the original eleven safeguards now represents a smaller portion of the whole. All the safeguards are weighted equally, and RLS is measured on a scale of 0 to 100 percent so as to reflect the percent of feasible safeguards a state has in place. But now with fourteen safeguards in 2023, a single safeguard is one-fourteenth (approximately 7 percent) of the whole (potential) score rather than one-eleventh (about 9 percent) of the whole.

Figure 3 features two pie charts, one for each RLS year, showing the weight of safeguards (represented each by a slice of the pie) and the resulting influence of six or seven topically related groups of safeguards (indicated by the different colors). The size of any piece of the pie indicates what percentage of the overall RLS score in 2022 or 2023 a safeguard comprises, again noting that safeguards are weighted equally for simplicity and transparency of our measure. The magnitude of a color in the pie shows the implications of that equal weighting on the influence of particular topics or issues within RLS analyses.

Comparing across pie charts in Figure 3, starting at noon and working clockwise, one can see that absentee voting, health-care provision safeguards (of which there are four), the exemption for religious employers from contraceptive mandates, marriage and wedding safeguards (of which there are three), and RFRA all carry less weight in the overall total of RLS 2023 than in 2022.

The only topical group that now carries more weight in RLS 2023 than 2022 is the one associated with school-aged children, which expanded in scope from only religious exemptions from the childhood immunization requirement to also include the requirement of public schools to grant excused absences for religious reasons. That is, the group is now comprised of two safeguards. Put differently, while childhood immunizations carry less weight in RLS 2023, more of RLS 2023 has to do with school-aged children than did 2022.

The other new safeguards in RLS 2023 coalesce around religious ceremonies or sacraments. Since these safeguards were not present in RLS 2022, they had no weight. This topical group is comprised of two safeguards: (1) the exception clergy may receive from mandatory reporting laws when hearing confidential or penitential communication and (2) exemptions for minors’ use of alcohol in religious ceremonies. Thus this group now comprises one-seventh of the aggregate score, since it is made up of two of fourteen total safeguards.

Interested in learning more about how RLS 2023 works? See our methodology page or download the executive summary for more information.