External Evaluation of Discrimination Mitigation Efforts in Meta’s Ad Delivery

Imana, Basileal and Shen, Zeyu and Korolova, Aleksandra and Heidemann, John

citation

Basileal Imana, Zeyu Shen, Aleksandra Korolova and John Heidemann 2025. External Evaluation of Discrimination Mitigation Efforts in Meta’s Ad Delivery. Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT) (Athens, Greece, Jun. 2025), to appear. [DOI] [PDF] [Dataset]

abstract

The 2022 settlement between Meta and the U.S. Department of Justice to resolve allegations of discriminatory advertising resulted is a first-of-its-kind change to Meta’s ad delivery system aimed to address algorithmic discrimination in its housing ad delivery. In this work, we explore direct and indirect effects of both the settlement’s choice of terms and the Variance Reduction System (VRS) implemented by Meta on the actual reduction in discrimination. \newline We first show that the settlement terms allow for an implementation that does not meaningfully improve access to opportunities for individuals. The settlement measures impact of ad delivery in terms of impressions, instead of unique individuals reached by an ad; it allows the platform to level down access, reducing disparities by decreasing the overall access to opportunities; and it allows the platform to selectively apply VRS to only small advertisers. \newline We then conduct experiments to evaluate VRS with real-world ads, and show that while VRS does reduce variance, it also raises advertiser costs (measured per-individuals-reached), therefore decreasing user exposure to opportunity ads for a given ad budget. VRS thus passes the cost of decreasing variance to advertisers. \newline Finally, we explore an alternative approach to achieve the settlement goals, that is significantly more intuitive and transparent than VRS. We show our approach outperforms VRS by both increasing ad exposure for users from all groups and reducing cost to advertisers, thus demonstrating that the increase in cost to advertisers when implementing the settlement is not inevitable. \newline Our methodologies use a black-box approach that relies on capabilities available to any regular advertiser, rather than on privileged access to data, allowing others to reproduce or extend our work.

reference

@inproceedings{Imana25b,
  author = {Imana, Basileal and Shen, Zeyu and Korolova, Aleksandra and Heidemann, John},
  title = {External Evaluation of Discrimination
          Mitigation Efforts in {Meta's} Ad Delivery},
  booktitle = {Proceedings of the  ACM Conference on Fairness, Accountability, and Transparency (FAccT)},
  year = {2025},
  sortdate = {2025-06-26},
  project = {ant, pimawat},
  jsubject = {algorithmic_fairness},
  pages = {to appear},
  month = jun,
  address = {Athens, Greece},
  publisher = {ACM},
  jlocation = {johnh: pafile},
  keywords = {algorithmic fairness, VRS, Meta, cost},
  doi = {tba},
  url = {https://ant.isi.edu/%7ejohnh/PAPERS/Imana25b.html},
  xotherurl = {https://arxiv.org/abs/2410.xxx},
  pdfurl = {https://ant.isi.edu/%7ejohnh/PAPERS/Imana25b.pdf},
  blogurl = {https://ant.isi.edu/blog/?p=2181},
  dataseturl = {https://ant.isi.edu/datasets/addelivery-vfs}
}