Abstract

‘Scale the model, scale the data, scale the compute’ is the reigning sentiment in the world of generative AI today. While the impact of model scaling has been extensively studied, we are only beginning to scratch the surface of data scaling and its consequences. This is especially of critical importance in the context of visio-linguistic datasets such as LAION, which are continually growing in size and built based on large-scale Internet dumps such as the Common Crawl, which is known to have numerous drawbacks ranging from its quality, legality, and contents. These datasets then serve as the backbone for large generative models, contributing to the operationalization and perpetuation of harmful societal and historical biases and stereotypes. In this paper, we investigate the effect of scaling datasets on hateful content through a comparative audit of two datasets: LAION-400M and LAION-2B. Our results show that hate content increases with dataset scale, measured both qualitatively and quantitatively using a metric that we term as Hate Content Rate (HCR) and that filtering dataset contents based on Not Safe For Work (NSFW) values calculated based on images alone does not exclude all the harmful content in the alt-text accompanying images. We end with a discussion of future work and the significance of our results for dataset curation and usage in the AI community. Content warning: This paper contains content that some readers may find disturbing, distressing, and/or offensive.