Publications

2024

1. Springer

   On Prediction-Modelers and Decision-Makers: Why Fairness Requires More Than a Fair Prediction Model

   Scantamburlo, Teresa, Baumann, Joachim, and Heitz, Christoph

   (2024)

   Abs arXiv

   An implicit ambiguity in the field of prediction-based decision-making regards the relation between the concepts of prediction and decision. Much of the literature in the field tends to blur the boundaries between the two concepts and often simply speaks of ’fair prediction.’ In this paper, we point out that a differentiation of these concepts is helpful when implementing algorithmic fairness. Even if fairness properties are related to the features of the used prediction model, what is more properly called ’fair’ or ’unfair’ is a decision system, not a prediction model. This is because fairness is about the consequences on human lives, created by a decision, not by a prediction. We clarify the distinction between the concepts of prediction and decision and show the different ways in which these two elements influence the final fairness properties of a prediction-based decision system. In addition to exploring this relationship conceptually and practically, we propose a framework that enables a better understanding and reasoning of the conceptual logic of creating fairness in prediction-based decision-making. In our framework, we specify different roles, namely the ’prediction-modeler’ and the ’decision-maker,’ and the information required from each of them for being able to implement fairness of the system. Our framework allows for deriving distinct responsibilities for both roles and discussing some insights related to ethical and legal requirements. Our contribution is twofold. First, we shift the focus from abstract algorithmic fairness to context-dependent decision-making, recognizing diverse actors with unique objectives and independent actions. Second, we provide a conceptual framework that can help structure prediction-based decision problems with respect to fairness issues, identify responsibilities, and implement fairness governance mechanisms in real-world scenarios.

2. CLSR

   Discrimination for the Sake of Fairness: Fairness by Design and Its Legal Framework

   Hoch, Holly, Hertweck, Corinna, Loi, Michele, and Tamò, Aurelia

   Computer Law & Security Review (2024)

   Abs SSRN

   As algorithms are increasingly enlisted to make critical determinations about human actors, the more frequently we see these algorithms appear in sensational headlines crying foul on discrimination. There is broad consensus among computer scientists working on this issue that such discrimination can be reduced by intentionally collecting and consciously using sensitive information about demographic features like sex, gender, race, religion etc. Companies implementing such algorithms might, however, be wary of allowing algorithms access to such data as they fear legal repercussions, as the promoted standard has been to omit protected attributes, otherwise dubbed “fairness through unawareness”. This paper asks whether such wariness is justified in light of EU data protection and anti-discrimination laws. In order to answer this question, we introduce a specific case and analyze how EU law might apply when an algorithm accesses sensitive information to make fairer predictions. We review whether such measures constitute discrimination, and for who, arriving at different conclusions based on how we define the harm of discrimination and the groups we compare. Finding that several legal claims could arise regarding the use of sensitive information, we ultimately conclude that the proffered fairness measures would be considered a positive (or affirmative) action under EU law. As such, the appropriate use of sensitive information in order to increase the fairness of an algorithm is a positive action, and not per se prohibited by EU law.

2023

1. Springer

   Fairness and Risk: An Ethical Argument for a Group Fairness Definition Insurers Can Use

   Baumann, Joachim, and Loi, Michele

   Philosophy & Technology (2023)

   Abs

   Algorithmic predictions are promising for insurance companies to develop personalized risk models for determining premiums. In this context, issues of fairness, discrimination, and social injustice might arise: Algorithms for estimating the risk based on personal data may be biased towards specific social groups, leading to systematic disadvantages for those groups. Personalized premiums may thus lead to discrimination and social injustice. It is well known from many application fields that such biases occur frequently and naturally when prediction models are applied to people unless special efforts are made to avoid them. Insurance is no exception. In this paper, we provide a thorough analysis of algorithmic fairness in the case of insurance premiums. We ask what “fairness” might mean in this context and how the fairness of a premium system can be measured. For this, we apply the established fairness frameworks of the fair machine learning literature to the case of insurance premiums and show which of the existing fairness criteria can be applied to assess the fairness of insurance premiums. We argue that two of the often-discussed group fairness criteria, independence (also called statistical parity or demographic parity) and separation (also known as equalized odds), are not normatively appropriate for insurance premiums. Instead, we propose the sufficiency criterion (also known as well-calibration) as a morally defensible alternative that allows us to test for systematic biases in premiums towards certain groups based on the risk they bring to the pool. In addition, we clarify the connection between group fairness and different degrees of personalization. Our findings enable insurers to assess the fairness properties of their risk models, helping them avoid reputation damage resulting from potentially unfair and discriminatory premium systems.

2. EWAF’23

   FairnessLab: A Consequence-Sensitive Bias Audit and Mitigation Toolkit

   Hertweck, Corinna, Baumann, Joachim, Loi, Michele, and Heitz, Christoph

   Proceedings of the 2nd European Workshop on Algorithmic Fairness (EWAF’23) (2023)
3. EWAF’23

   Unification, Extension, and Interpretation of Group Fairness Metrics for ML-Based Decision-Making

   Baumann, Joachim, Hertweck, Corinna, Loi, Michele, and Heitz, Christoph

   Proceedings of the 2nd European Workshop on Algorithmic Fairness (EWAF’23) (2023)
4. EWAF’23

   An Open-Source Toolkit to Generate Biased Datasets

   Baumann, Joachim, Castelnovo, Alessandro, Crupi, Riccardo, Inverardi, Nicole, and Regoli, Daniele

   Proceedings of the 2nd European Workshop on Algorithmic Fairness (EWAF’23) (2023)
5. EWAF’23

   Fair Machine Learning Through Post-processing: The Case of Predictive Parity

   Baumann, Joachim, Hannák, Anikó, and Heitz, Christoph

   Proceedings of the 2nd European Workshop on Algorithmic Fairness (EWAF’23) (2023)
6. ACM

   IJCAI’23 DEMO

   Bias on Demand: A Modelling Framework That Generates Synthetic Data With Bias

   Baumann, Joachim, Castelnovo, Alessandro, Crupi, Riccardo, Inverardi, Nicole, and Regoli, Daniele

   In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (2023)

   Abs

   Nowadays, Machine Learning (ML) systems are widely used in various businesses and are increasingly being adopted to make decisions that can significantly impact people’s lives. However, these decision-making systems rely on data-driven learning, which poses a risk of propagating the bias embedded in the data. Despite various attempts by the algorithmic fairness community to outline different types of bias in data and algorithms, there is still a limited understanding of how these biases relate to the fairness of ML-based decision-making systems. In addition, efforts to mitigate bias and unfairness are often agnostic to the specific type(s) of bias present in the data. This paper explores the nature of fundamental types of bias, discussing their relationship to moral and technical frameworks. To prevent harmful consequences, it is essential to comprehend how and where bias is introduced throughout the entire modelling pipeline and possibly how to mitigate it. Our primary contribution is a framework for generating synthetic datasets with different forms of biases. We use our proposed synthetic data generator to perform experiments on different scenarios to showcase the interconnection between biases and their effect on performance and fairness evaluations. Furthermore, we provide initial insights into mitigating specific types of bias through post-processing techniques. The implementation of the synthetic data generator and experiments can be found at https://github.com/rcrupiISP/BiasOnDemand.

7. PLOS

   I win it’s fair, you win it’s not. Selective heeding of merit in ambiguous settings

   Kandul, Serhiy, and Nikolaychuk, Olexandr

   PLOS ONE (2023)

2022

1. Distributive Justice as the Foundational Premise of Fair ML: Unification, Extension, and Interpretation of Group Fairness Metrics

   Baumann, Joachim, Hertweck, Corinna, Loi, Michele, and Heitz, Christoph

   (2022)

   Abs arXiv Poster

   Group fairness metrics are an established way of assessing the fairness of prediction-based decision-making systems. However, these metrics are still insufficiently linked to philosophical theories, and their moral meaning is often unclear. We propose a general framework for analyzing the fairness of decision systems based on theories of distributive justice, encompassing different established "patterns of justice" that correspond to different normative positions. We show that the most popular group fairness metrics can be interpreted as special cases of our approach. Thus, we provide a unifying and interpretative framework for group fairness metrics that reveals the normative choices associated with each of them and that allows understanding their moral substance. At the same time, we provide an extension of the space of possible fairness metrics beyond the ones currently discussed in the fair ML literature. Our framework also allows overcoming several limitations of group fairness metrics that have been criticized in the literature, most notably (1) that they are parity-based, i.e., that they demand some form of equality between groups, which may sometimes be harmful to marginalized groups, (2) that they only compare decisions across groups, but not the resulting consequences for these groups, and (3) that the full breadth of the distributive justice literature is not sufficiently represented.

2. A Justice-Based Framework for the Analysis of Algorithmic Fairness-Utility Trade-Offs

   Hertweck, Corinna, Baumann, Joachim, Loi, Michele, Viganò, Eleonora, and Heitz, Christoph

   (2022)

   Abs arXiv

   In prediction-based decision-making systems, different perspectives can be at odds: The short-term business goals of the decision makers are often in conflict with the decision subjects’ wish to be treated fairly. Balancing these two perspectives is a question of values. We provide a framework to make these value-laden choices clearly visible. For this, we assume that we are given a trained model and want to find decision rules that balance the perspective of the decision maker and of the decision subjects. We provide an approach to formalize both perspectives, i.e., to assess the utility of the decision maker and the fairness towards the decision subjects. In both cases, the idea is to elicit values from decision makers and decision subjects that are then turned into something measurable. For the fairness evaluation, we build on the literature on welfare-based fairness and ask what a fair distribution of utility (or welfare) looks like. In this step, we build on well-known theories of distributive justice. This allows us to derive a fairness score that we then compare to the decision maker’s utility for many different decision rules. This way, we provide an approach for balancing the utility of the decision maker and the fairness towards the decision subjects for a prediction-based decision-making system.

3. Perceptions of Efficiency vs. Fairness Tradeoffs in Algorithm-based HR Selection: Insights from Two Online Experiments

   Kandul, Serhiy, and Leicht-Deobald, Ulrich

   (2022)

   Abs

   Organizations increasingly rely on algorithms to increase the efficiency of their HR processes. However, research shows that higher accuracy of an algorithm often conflicts with group fairness metrics. With two online experiments on Mechanical Turk (Study 1: N= 283; Study 2: N = 277) we address people’s perception of efficiency vs. fairness tradeoff in HR context, where participant weigh the utilities of the decision maker against the utilities of the job candidates. In Study 1, we x the degree of efficiency gain and vary the degree of violation of a fairness metric (with 4/5 rule satisfied or not) compared to a nearly perfectly fair benchmark. We find that the higher the degree of the fairness violation (for the same gain in efficiency), the more likely people are to choose a fair algorithm. If a disparity produced by a more efficient algorithm is low, within the 4/5 rule, signicantly more people prefer an efficient algorithm over a fair one. Furthermore, we find that people’s preferences over algorithms are driven by their fairness perceptions; this effect holds for both statistical parity and equality of opportunity. Interestingly, participants who believe that females are under-represented in the industry tolerate disparities that favour female job candidates. In Study 2, we find participants are weakly more sensitive to eciency vs. fairness tradeoffs when the baseline efficiency of the algorithm is low.

4. ACM

   Is calibration a fairness requirement? An argument from the point of view of moral philosophy and decision theory.

   Loi, Michele, and Heitz, Christoph

   In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (2022)

   Abs

   In this paper, we provide a moral analysis of two criteria of statistical fairness debated in the machine learning literature: 1) calibration between groups and 2) equality of false positive and false negative rates between groups. In our paper, we focus on moral arguments in support of either measure. The conflict between group calibration vs. false positive and false negative rate equality is one of the core issues in the debate about group fairness definitions among practitioners. For any thorough moral analysis, the meaning of the term “fairness” has to be made explicit and defined properly. For our paper, we equate fairness with (non-)discrimination, which is a legitimate understanding in the discussion about group fairness. More specifically, we equate it with “prima facie wrongful discrimination” in the sense this is used in Prof. Lippert-Rasmussen’s treatment of this definition. In this paper, we argue that a violation of group calibration may be unfair in some cases, but not unfair in others. Our argument analyzes in great detail two specific hypothetical examples of usage of predictions in decision making. The most important practical implication is that between-group calibration is defensible as a bias standard in some cases but not others; we show this by referring to examples in which the violation of between-group calibration is discriminatory, and others in which it is not. This is in line with claims already advanced in the literature, that algorithmic fairness should be defined in a way that is sensitive to context. The most important practical implication is that arguments based on examples in which fairness requires between-group calibration, or equality in the false-positive/false-negative rates, do no generalize. For it may be that group calibration is a fairness requirement in one case, but not in another.

5. ACM

   Enforcing Group Fairness in Algorithmic Decision Making: Utility Maximization Under Sufficiency

   Baumann, Joachim, Hannák, Anikó, and Heitz, Christoph

   In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (2022)

   Abs arXiv

   Binary decision making classifiers are not fair by default. Fairness requirements are an additional element to the decision making rationale, which is typically driven by maximizing some utility function. In that sense, algorithmic fairness can be formulated as a constrained optimization problem. This paper contributes to the discussion on how to implement fairness, focusing on the fairness concepts of positive predictive value (PPV) parity, false omission rate (FOR) parity, and sufficiency (which combines the former two). We show that group-specific threshold rules are optimal for PPV parity and FOR parity, similar to well-known results for other group fairness criteria. However, depending on the underlying population distributions and the utility function, we find that sometimes an upper-bound threshold rule for one group is optimal: utility maximization under PPV parity (or FOR parity) might thus lead to selecting the individuals with the smallest utility for one group, instead of selecting the most promising individuals. This result is counter-intuitive and in contrast to the analogous solutions for statistical parity and equality of opportunity. We also provide a solution for the optimal decision rules satisfying the fairness constraint sufficiency. We show that more complex decision rules are required and that this leads to within-group unfairness for all but one of the groups. We illustrate our findings based on simulated and real data.

6. ACM

   People Are Not Coins: Morally Distinct Types of Predictions Necessitate Different Fairness Constraints

   Viganò, Eleonora, Hertweck, Corinna, Heitz, Christoph, and Loi, Michele

   In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (2022)

   Abs SSRN

   A recent paper (Hedden 2021) has argued that most of the group fairness constraints discussed in the machine learning literature are not necessary conditions for the fairness of predictions, and hence that there are no genuine fairness metrics. This is proven by discussing a special case of a fair prediction. In our paper, we show that Hedden’s argument does not hold for the most common kind of predictions used in data science, which are about people and based on data from similar people; we call these “human-group-based practices”. We argue that there is a morally salient distinction between human-group-based practices and those that are based on data of only one person, which we call “human-individual-based practices”. Thus, what may be a necessary condition for the fairness of human-group-based practices may not be a necessary condition for the fairness of human-individual-based practices, on which Hedden’s argument is based. Accordingly, the group fairness metrics discussed in the machine learning literature may still be relevant for most applications of prediction-based decision making.

7. IEEE

   Group Fairness in Prediction-Based Decision Making: From Moral Assessment to Implementation

   Baumann, Joachim, and Heitz, Christoph

   In 2022 9th Swiss Conference on Data Science (SDS) (2022)

   Abs arXiv

   Ensuring fairness of prediction-based decision making is based on statistical group fairness criteria. Which one of these criteria is the morally most appropriate one depends on the context, and its choice requires an ethical analysis. In this paper, we present a step-by-step procedure integrating three elements: (a) a framework for the moral assessment of what fairness means in a given context, based on the recently proposed general principle of "Fair equality of chances" (FEC) (b) a mapping of the assessment’s results to established statistical group fairness criteria, and (c) a method for integrating the thus-defined fairness into optimal decision making. As a second contribution, we show new applications of the FEC principle and show that, with this extension, the FEC framework covers all types of group fairness criteria: independence, separation, and sufficiency. Third, we introduce an extended version of the FEC principle, which additionally allows accounting for morally irrelevant elements of the fairness assessment and links to well-known relaxations of the fairness criteria. This paper presents a framework to develop fair decision systems in a conceptually sound way, combining the moral and the computational elements of fair prediction-based decision-making in an integrated approach. Data and code to reproduce our results are available at https://github.com/joebaumann/fair-prediction-based-decision-making.

8. AAAI

   Gradual (In)Compatibility of Fairness Criteria

   Hertweck, Corinna, and Räz, Tim

   In Proceedings of the 36th AAAI Conference on Artificial Intelligence (2022)

   Abs arXiv Poster

   Impossibility results show that important fairness measures (independence, separation, sufficiency) cannot be satisfied at the same time under reasonable assumptions. This paper explores whether we can satisfy and/or improve these fairness measures simultaneously to a certain degree. We introduce information-theoretic formulations of the fairness measures and define degrees of fairness based on these formulations. The information-theoretic formulations suggest unexplored theoretical relations between the three fairness measures. In the experimental part, we use the information-theoretic expressions as regularizers to obtain fairness-regularized predictors for three standard datasets. Our experiments show that a) fairness regularization directly increases fairness measures, in line with existing work, and b) some fairness regularizations indirectly increase other fairness measures, as suggested by our theoretical findings. This establishes that it is possible to increase the degree to which some fairness measures are satisfied at the same time – some fairness measures are gradually compatible.

2021

1. IEEE

   A Systematic Approach to Group Fairness in Automated Decision Making

   Hertweck, Corinna, and Heitz, Christoph

   In 2021 8th Swiss Conference on Data Science (SDS) (2021)

   ELSI award

   Abs arXiv

   While the field of algorithmic fairness has brought forth many ways to measure and improve the fairness of machine learning models, these findings are still not widely used in practice. We suspect that one reason for this is that the field of algorithmic fairness came up with a lot of definitions of fairness, which are difficult to navigate. The goal of this paper is to provide data scientists with an accessible introduction to group fairness metrics and to give some insight into the philosophical reasoning for caring about these metrics. We will do this by considering in which sense socio-demographic groups are compared for making a statement on fairness.

2. ACM

   Group Fairness: Independence Revisited

   Räz, Tim

   In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (2021)

   Abs arXiv

   This paper critically examines arguments against independence, a measure of group fairness also known as statistical parity and as demographic parity. In recent discussions of fairness in computer science, some have maintained that independence is not a suitable measure of group fairness. This position is at least partially based on two influential papers (Dwork et al., 2012, Hardt et al., 2016) that provide arguments against independence. We revisit these arguments, and we find that the case against independence is rather weak. We also give arguments in favor of independence, showing that it plays a distinctive role in considerations of fairness. Finally, we discuss how to balance different fairness considerations.

3. ACM

   On the Moral Justification of Statistical Parity

   Hertweck, Corinna, Heitz, Christoph, and Loi, Michele

   In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (2021)

   Best Student Paper

   Abs arXiv Poster

   A crucial but often neglected aspect of algorithmic fairness is the question of how we justify enforcing a certain fairness metric from a moral perspective. When fairness metrics are proposed, they are typically argued for by highlighting their mathematical properties. Rarely are the moral assumptions beneath the metric explained. Our aim in this paper is to consider the moral aspects associated with the statistical fairness criterion of independence (statistical parity). To this end, we consider previous work, which discusses the two worldviews "What You See Is What You Get" (WYSIWYG) and "We’re All Equal" (WAE) and by doing so provides some guidance for clarifying the possible assumptions in the design of algorithms. We present an extension of this work, which centers on morality. The most natural moral extension is that independence needs to be fulfilled if and only if differences in predictive features (e.g. high school grades and standardized test scores are predictive of performance at university) between socio-demographic groups are caused by unjust social disparities or measurement errors. Through two counterexamples, we demonstrate that this extension is not universally true. This means that the question of whether independence should be used or not cannot be satisfactorily answered by only considering the justness of differences in the predictive features.