Thinking with Archival Theory in Algorithmic Design

A photograph of “Ashley’s Sack,” as discussed by Tiya Miles in All That She Carried (Random House, 2021). The sack was embroidered by Ruth Middleton in 1921. Photo courtesy of Courtesy Middleton Place / Wikimedia Commons.

Nearly a decade into the Fourth Industrial Revolution—an era characterized by the convergence of several emerging technologies such as cloud computing, blockchain, generative AI, and other algorithmic systems—the issue of algorithmic bias has entered the mainstream. Algorithmic bias generally refers to the uneven effects of algorithmic systems driving the automation and digitization of industries, often reflecting and reproducing social hierarchies marked by race, class, disability, sexuality, gender, and other categories of difference. These systems include facial recognition algorithms that have higher error rates for individuals with darker skin tones[1] online advertising platforms that perpetuate discriminatory practices in housing and job advertisements[2] credit scoring algorithms that reanimate the exclusion of racialized groups from accessing credit, loans, and other financial services[3] and predictive healthcare technologies that result in misdiagnosis, inadequate treatment, and uneven access to care[4].

This range of concerns grows in large part from advocacy efforts of groups like the Algorithmic Justice League, investigative reporting by organizations like ProPublica, academic research by scholars such as Safiya Noble, Ruha Benjamin, and Meredith Broussard, among others, and newly enacted regulation by the European Union. Even the IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems has weighed in, developing a set of design guidelines that emphasize the need for transparency, accountability, and a focus on human well-being. Along with this work, which has highlighted the individual and social harms of discriminatory algorithms if left unchecked, has come a range of proposals from within the field of computer science for mitigating algorithmic bias and its pernicious effects. These include proposals for documenting the composition of datasets (e.g. «Datasheets for Datasets»[5] and the performance of models (e.g. «Model Cards for Model Reporting»[6]), heightened standards for data sharing and research replicability, and mechanisms for enhanced interpretability and explainability of the algorithms themselves (e.g. «Honest Students»[7]).

Each of these projects represents an admirable—and necessary—intervention into the issue of algorithmic bias. Their practical proposals complement work from the fields of design research, critical data studies, STS, and HCI, among others, which have proposed a range of analytical methods for examining, challenging, and attempting to change the unequal structural power at the root of algorithmic bias. But more work is required if we are to be able to intervene into the issue of bias at its source. As these scholars inform us, the source of algorithmic bias is not any particular dataset or model, but instead the structural power differentials that produce social inequality—legacies of capital and colonial power already baked into practices of employment and hiring, healthcare, credit scoring, advertising and more. It is this social inequality that, in turn, produces biased datasets; and that prohibits any single approach to algorithmic «fairness» from achieving its intended effect.

This observation leads to this essay’s main contribution: rather than theorizing algorithmic systems as closed problems to solve, we offer a reframing of the idea of algorithmic bias itself. More specifically, we propose the concept of absence, as theorized through archival studies, as one that might more securely anchor investigations into the causes of algorithmic bias. We believe that an emphasis on absence, rather than bias, can prompt more generative, more capacious, and more creative interventions into the various sources at the roots of bias. In what follows, we describe the concept of absence as it has been theorized in the field of archival studies. We then illustrate how absence takes shape across an unexpected site of computation—the quilt—and how archival theory enabled those who encountered absences in their work to move beyond them. We close by reflecting on embodied knowledge and the possibility of building that sensorial perception into algorithmic design. By providing a way of thinking through the biases in the data that cannot ever be fixed, we offer a path for moving forward with technical work that allows both the data and its biases to be taken into account.

A final note before we proceed: it is not without ambivalence that we present this shift in thinking. We believe that this shift toward absence matters for conversations on data bias. However, we also recognize that our contribution cannot definitively «solve» the issue of bias—much less redress the harms brought about by generations of trauma. It is equally important to acknowledge that this theoretical tradition cannot seamlessly be «imported» into data practices in its entirety. Rather, we offer our intervention as an invitation—a call for us, in our role as data analysts, to care for the absence of something without first determining what it is or what we are projecting onto it. By approaching absence within our data practices with care and attentiveness, we come to recognize data as the mere surface or symptom that obscures the people and relationships behind it. In this way, absence helps us interrogate how our methods reflect and sustain deeper, long-standing forces of capital and colonial power, challenging us to reimagine their role within and beyond the worlds that data seeks to represent and contain.

Absence as Power

To explore the generative potential of absence,we first turn to the work of Haitian American anthropologist Michel-Rolph Trouillot, who offers a formulation of four «moments» (or ways in which) «silence» shapes the archival record: The moment of fact creation (the making of sources); the moment of fact assembly (the making of archives); the moment of fact retrieval (the making of narratives); and the moment of retrospective significance (the making of history) in the final instance.[8]

While Trouillot presents these moments as discrete, he later explains how each is returned to again and again as scholars attempt to make meaning from the documents that the archive contains (or omits, as the case may be). Trouillot takes as his focus the Haitian Revolution, so the process of «the making of sources» is bound up in colonial power relations between French citizens and Haitian subjects, which are of course also racial power relations because of the dominant identities of each group. Colonial governments possess the institutional, educational, and physical apparatuses to produce documents about colonial subjects, who largely lack the means to create their own official records . This moment in which power enters and shapes the contents of the archive occurs even before the archive is constituted. By the same token, we might recognize how datasets are shaped by unequal power even before they are created. In other words, the imbalance of power—and the resulting relative presence and absence of sources—carries through to the moment of archive (or dataset) creation. The point at which the French government decided which documents to preserve in their official records and which documents to discard further consolidated the colonial perspective on the events that transpired and why. Returning to the archive even centuries later, as Trouillot explains, one is still subject to those original forces of power and more. A researcher may choose how to tell a story, but when confronted with the one-sidedness of the colonial archive—and its intentional presences and absences—recovering accounts outside the colonial frame becomes profoundly difficult.

Keeping in mind Trouillot’s articulation of the ways that power shapes the construction of archives, we can begin to see how a similar (if less overtly colonial) dynamic impacts the process of dataset creation. One might think not only of datasets that derive directly from digitized colonial archives—the Bibliothèque nationale de France, for example–but also of datasets like the Colossal Clean Common Crawl corpus, or C4, which explicitly documents its decision to include certain «high quality» web documents while excluding others. An immediate concern is the representation of world languages in datasets drawn from the Internet, which remains predominantly Anglophone, with users centered in the Global North. In addition, Gururangan, et al.[9] have shown how the quality filters imposed on the original Common Crawl dataset exhibit a preference for language associated with wealthier, more educated, urban ZIP codes, layering socio-economic bias atop the dataset’s global linguistic asymmetries. This, too, is the power in the making of sources and archives.

As for the making of narratives and the attribution of retrospective significance, these layers of power only compound. It becomes increasingly difficult to draw upon data that represents non-dominant groups, just as developing narratives that reflect their realities requires an even greater effort. Trouillot’s atomization of the moments by which power enters the archive thus becomes a lens through which to refract the power that similarly overdetermines the construction of datasets, better attuning us to the contents of the archive and what narratives they make possible, as well as to what is absent. These absences, in turn, direct us back to questions of power, fostering alertness to what stories the archive enables and what stories it precludes.

This, too, is a lesson that might be mapped onto the data. Just as silences enter the dataset upon its creation, so too do second-order silences enter at the moment of narrative making. Consider, as Bender et al.[10] do in «Stochastic Parrots,» how the filtering of text relating to queerness and queer people in the C4 dataset results in a model lacking examples of queer language to draw upon when users (or researchers) prompt a large language model like GPT4 to generate liberatory examples of queer life; or how even datasets that are constructed with care and intention, such as the Colored Conventions Corpus, which documents the meeting minutes of the aforementioned Colored Conventions–Black organizing meetings that took place during the nineteenth century, cannot be used to recover the voices or stories of the women organizers involved, since their contributions were not documented in their own time. Once again, we are confronted with what in archival theory is described as archival silence–the absences in the archive, both unintentional and by design, that limit what the archive can disclose.

Absence as Presence

A second strand of theories of the archive takes up this issue of archival silence, urging us to reconceive archival absence not as a void but as a commanding presence in its own right. While such silences cannot reveal what historical figures said in everyday conversation, where their everyday activities took them, or how they truly lived their lives, the very absence of this information—and our own awareness that it should exist—can open a space for what we do not and, in some cases, cannot ever fully know.

Two concepts from the field of critical data studies are helpful here. The first is the notion of the null value, as articulated by Jacob Gaboury[11] and extended by Jessica Marie Johnson[12]. Drawing from database management systems, where yet to be filled database records are initialized with a null value, allocating space for data that may or may not be entered in the future, Gaboury theorizes structural queerness as a conceptual null value: a position that claims space without conforming to binary definitions like 0 or 1. Johnson builds on Gaboury’s concept and applies it to the archive, particularly to the incomplete records in the ledgers of the enslaved. These gaps, frequently surrounding women and girls, function as space-holding absences. Though they lack identifying details, they assert the existence and importance of the lives that went systematically unrecorded. In both their historical context and our present-day interpretations, these null values demand recognition.

Here, Johnson also draws on the work of Saidiya Hartman[13], whose importance in this discussion is impossible to overstate. Hartman’s early work on the archive foregrounds the compounding effects of what she calls the «double bind»[14]— the paradox by which ostensibly emancipatory interventions or corrections can, in fact, reinscribe the very systems of power they seek to dismantle, perpetuating the logics of chattel slavery and anti-Black violence. Hartman argues that one aspect of this accumulation of suffering stems from that any retelling of scenes of subjection fails to escape or exceed their original violences. She asks: «How does one represent the various modes of practice without reducing them to conditions of domination or romanticizing them as pure forces of resistance?» It is in grappling with this challenge—how to retell and represent without reenacting harm—that Hartman confronts the problem of historical absence. Even as the revival of stories that escape documentation remains an impossibility, the absence of those stories plays an active part in the writing of history (or «counter-history»); it becomes a presence.

This conditioning of an absent presence shares a cadence with José Muñoz’s concept of queerness, which he characterizes as an orientation to absence. In his now landmark text, Queering Utopia: The Then and There of Queer Futurity, he proposes that queerness is «that thing that lets us feel that this world is not enough, that indeed something is missing»[15]. This «something» that is missing marks absence as an affective site, a locus of longing, and the beginnings of a path towards knowing that makes use of the ephemeral and the fleeting as meaningful markers of history which evade the devices of neat capture. For Muñoz, ephemera counts as, «all of those things that remain after a performance, a kind of evidence of what has transpired but certainly not the thing itself»[16]. Perhaps, then, absence is able to gesture toward desire.

Desire of course is not unfamiliar to those working with information, be it in archives or datasets, though it is seldom remarked upon. In the realm of information theory, from the work of Claude Shannon and Warren Weaver up through today, absence has been equated with that which is unpatterned, entropic, or without meaning culling forth desires for clarity, legibility, and completion. While reflecting our desires back to us, absence also makes clear the normative structuring and residual legacies which come to define and organize not only what types of information are meaningful but what counts as information to begin with.

Conversely, Muñoz’s embrace of desire as a mode of knowing might suggest a different response to absence–one that approaches it not as a void to be filled, but with a reverence for alternative forms of transmitting knowledge beyond the confines of the index. It is a tacit acknowledgement of the index’s limitations in accounting for other epistemologies, those that dwell not in fixed arrival, but in continual yearning, stitched together through contingent, embodied acts. For Muñoz, the performance, the dance floor, the exhaustion of bodies in motion cannot be reconstructed through the materials left in their wake. Instead, the event itself becomes the medium through which information is shared—a repertoire of collective gestures. His insistence on the importance of these visceral, embodied moments—what he calls ephemera—as evidence of the event is insightful in that it marks a critical distinction between the event and its translation into information. In this framing, absence takes on a unique quality as it renders our desires explicit and unavoidable.

Absence as Productive

Lastly, we consider how the shift from bias to absence offers productive opportunities for the design of algorithmic systems. By labeling this move «productive,» we do not imply that the previous frameworks are not; rather, we emphasize the creativity and imagination this reading brings to our analytic work. «Productive» refers to a form of engagement that remains with the difficulty of speaking to nearby absence, working with absence not as a qualifier of truth or fact, but as a provocation to cull forth a different kind of halting problem. This approach involves engaging in the slower work of attending to an absence, inviting reference.

We begin by looking to Saidiya Hartman’s development of «critical fabulation» as an archival method that engages absence in a generative way. Hartman describes the nature of her process as «a history written with and against the archive»[17]. This entanglement with the archive reveals how its power dynamics can be troubled when its absences become presences: places to question and imagine things differently, i.e. to speculate. This form of speculation is based on careful research and engagement with archival fragments, yet it pushes against the limits and the narratives that have been constructed from those fragments. In approaching absence as an opportunity to illuminate past and present, narration becomes a way to offer an imagining that is critical, personal, and situated. It allows us to imagine new spaces within the absence while also accepting «the ongoing, unfinished and provisional character of this effort»[18]. In doing so, we are able to construct meaning beyond the limits of what the archive captures, while maintaining the absent presence as a null value that cannot be «solved» or «corrected» fully.

Tiya Miles’s[19] deployment of critical fabulation in All that She Carried:The Journey of Ashley’s Sack, a Black Family Keepsake. demonstrates the strategy’s potential to engage meaningfully with absence and incompleteness. The archival record tells us this about Ashley’s sack: in 1921 Ruth Middleton embroidered a few lines documenting the story of her great-grandmother, Rose, on cotton sack, documenting how Rose gave the sack to her daughter Ashley (Ruth’s grandmother), when she was sold away from a plantation at the age of nine. Viewed in the context of a discussion of archives and power, there are two important points to consider. First, Ruth’s embroidery on the sack is a moment of fact creation. It is a personal act that marks the sack as a material connection between Rose, Ashley, and Ruth and the broader experience of Black women of their respective times. If not for Ruth’s embroidery, would we know about the sack? Would we understand its relationship between this family of women and the realities and aftermath of slavery? Ruth’s embroidery makes the sack identifiable to archival systems, which has ultimately led to it being included in the exhibitions at Middleton Place and the Smithsonian National Museum of African American History & Culture (NMAAHC) in Washington, DC. But it also makes other «emergency packs» and «story cloths» more visible within the broader story of Black women that are missing or undertold in the archives. Ten lines of embroidered text offer a glimpse into the lives of the three women: Rose, Ashley and Ruth. Along with an inventory of the items with which Rose filled the bag, Ruth’s act of embroidering their story on the sack captures not only the personal experiences of this family but also introduces a record of practices of love and care that connect Black women in a particular time and place.

In All that She Carried, Miles attempts to trace the lives of Rose, Ashley, and Ruth through Ashley’s sack, treating it not only as an artifact, but also as its own archive. This shift enables a reorientation to the three women involved and to other Black women engaging their collective experience, including Miles. Miles alternates between narrator, researcher, subject, and sometimes even daughter and granddaughter. These various «moments of fact assembly» extend Hartman’s critical fabulation as a method for productively imagining what could have been (and what could be). Consistent with Hartman’s formulation, Miles does not give voice to Rose, Ashley, and Ruth, but occupies the space of their absence—highlighting its presence and shifting our focus, as readers, by anchoring the story of these three women as part of a broader archival story.

This story offers an important addition to the traditional archive. The woven sack filled with contents gives insight into the intersection of love and survival lived by Rose, Ashley, and Ruth,  as well as other enslaved women and their descendants, through their actions, words, and the labor of their bodies. The treatment of this personal artifact and story extends a Black Feminist practice of exploring the connections between the personal and the collective. By telling these personal stories of individual Black women we can move beyond just cataloging their names or understanding them as data reducing them to property or victims of violence. By linking Hartman’s writing practice of critical fabulation with Marisa Fuentes’ archival practice of «reading along the bias grain» and Trouillot’s call to engage with the actual material things that «enslaved people touched, made, used, and carried in order to understand the past,» Miles reconfigures our understanding of what constitutes the archival, revealing the possibilities of imagining within absence.

Absence as/in Design Speculation

We now turn to a set of design speculations that materialize and complicate one or more of the above three dimensions of absence (power, presence, and productive). By design speculations we refer to a specific tradition of speculating what might have happened in the past as a means of addressing bias in the present and future. Within this framework, speculation works in exploratory and experimental ways—situated within, and responsive to the specific archive or dataset it engages. This distinguishes our approach from other speculative practices, such as speculative design and design fiction, which tend to use design as a futuring activity and typically position the designer or author as the driver of that activity. By treating existing works as speculative artifacts, we adopt an interpretive stance, examining the work as a means of unsettling the authority of archival and data-driven systems. From the perspective of technical research, this interpretive stance allows us to attend to the spaces of indeterminacy within datasets and data systems, as well as to what is irrecoverably absent. In so doing, it opens up questions of both social responsibility and personal commitment: what responsibility do we bear as researchers working with particular datasets, and what are our intentions going forward with the pursuit of knowledge within those spaces?

We approach these questions through an inquiry into speculative design with quilts, beginning with what have come to be called «Freedom Quilts»—textiles that covertly encoded escape routes for enslaved Black people in the United States before emancipation and that, according to archival logics, may or may not have actually existed as such[20]. This contested history of the Freedom Quilts challenges the distinctions between absence and legibility, turning our attention to the use of protocols that, rather than relying on transparency, must circulate through opacity and the embedded trust of interdependence. Put another way, the Freedom Quilts thrived as a clandestine network of information sharing precisely because they were rendered invisible within the logic of a plantation system obsessed with endless pursuits of measurement, hierarchy, logistics, and analytics. If their specific patterns and symbols guided those escaping slavery to safe houses or offered instructions and warnings about the journey on the Underground Railroad, they also suggest to us today a different understanding of the encoding of data and the protocols of trust building. They blur the traditional lines between the computational and the social and relational. In this context, the acts of computing—calculating trajectories and distances, encoding geographic information, utilizing pattern recognition—are inseparable from the social relations that undergird their function. Instead, they are contingent upon recurring acts of interpretation, negotiation, trust building, and encoding.

In this way, the nature of code is being written through the creolization of symbols, meanings, context, and codes that change and shift across different sites and ensembles. This complicates our conventional understanding of programming as temporally discrete code, as executable function, as cause and effect. For those whose safety depended on strategic opacity, the idea of code as something transparently readable and executable outside of context was not just inadequate—it was dangerous. Instead, the shifting nature of the code required a great deal of maintenance which further emboldened and fortified the social infrastructures on which the quilts depended. When fugitives encountered these quilts in public, they had to interpret the code both symbolically and contextually, reading it in relation to the geography in which it appeared. In this way, the executability of code as a declarative axiomatic language imagined within syntax is halted. Code is not an absolute instruction but is contingent and contextual, read alongside landscape and relationships.

The non-totalizing concept of computation from the Freedom Quilts opens important lines of interpretation for the contemporary work of architect Curry Hackett. His series of quilted patterns rendered with generative AI—specifically, by prompting the text-to-image model Midjourney—are inspired by another set of quilts designed by Black artisans from a small Alabama town known as Gee’s Bend. Their artworks, though problematically incorporated into the model (as are all artworks used as training data without consent) nevertheless allowed him to conjure otherwise impossible cities. In one such image, two Gee’s Bend quilters appear on a platform suspended above a skyscraper in a dense urban setting. One wears a floral print dress, the other a loose-fitting yellow jumpsuit. But instead of using the scaffolding to clean windows, apply paint, or repair exterior decay, the women appear to be installing one of their quilts. A colorful patchwork made of interspersed rectangular scraps, some patterned and others solid, covers the visible length of the city high rise. The woman in the floral dress looks as if she’s about to climb higher, swinging herself onto the next set of scaffolding to continue the installation.

Using the quilt to rethink architectural space, Hackett’s imagery complicates what counts as ordered and spatially secure. Seeing a soft tapestry atop a towering skyscraper feels both exciting and off kilter. In contrast to the stereotypical output for a genAI prompt for «city of the future»—a minimalist landscape awash with technological chrome and white pedestrians—Hackett’s image re-centers Black cultural production. It elevates the Gee’s Bend quilters both literally and figuratively, recognizing their pattern work as part of a missing dataset in the visualization of urban spatial innovation. As Hackett himself notes: I see this work as an open source repository for free ideas for how to imagine a world that centers Blackness. You know, Black culture, Black aesthetics, Black modes of living.[21]

But like the Freedom Quilts, the quilted skyscraper does more than expose the absence as power or even presence. It also helps broaden the normative aesthetics of pattern. Like the maintenance of the Freedom Quilts, the Gee’s Bend aesthetics in Hackett’s renderings evoke a powerful call to improvisation and contingency. In their aesthetic ordering, contingency—the interdependence of elements—becomes the recursive practice for computing. This practice prefigures contemporary contingent turns in computer science and media studies. It offers a challenge to the narrow and often dangerous view of pattern as a systematic ordering that makes architecture legible and useful. Here we find a more expansive and expressive reading of architectural order that approaches the absence as productive. The quilted skyscraper, in line with the Freedom Quilt, is an object whose troubled existence opens up a way to radically reimagine computational bias as the beginning of possibility.

Absence as Ab-Sense

If we are to pursue absence as a meaningful concept to extend beyond the limitations of bias, we must reject its conventional framing as a negative space that needs to be filled or corrected. Instead, we must ask how our desires for what is not present can reorient our methods of working with data. Put another way: How might the historical—the then and there—proffer a different here and now?

As we have shown across the fields of historiography, gender studies, Black studies, and queer theory, absence doesn’t simply register as a lack. It doesn’t always act as a space for the unknown, but also as a vital signification towards knowing. In these contexts, absence is not silence, not a missing or non-existent sensing. Rather it establishes contact and connection at a level that requires effortful engagement. If absence has historically worked as a site for fabulation (Hartman) or desire (Muñoz), then absence may be also able to forge a different relationship to data—not as the inverse of presence, but as a way to mark space for what is to come. The aspirations for producing knowledge from the archive or the dataset often hinge on a sense of completion, equating quantity with truth. We solve for what can be shown, and proof becomes self-evident. Within these equivalences is a latent voice that speaks of a desire to know. But if desire is not absent from the work produced by the archive or dataset, then what is to be done with it?

Throughout this essay, we have seen how power, presence, and productive(ness) operate as de facto sites of meaning-making. The quilting examples offer spaces to reread dataset silences as invitations suggesting alternative modes of engagement: attunements to formats for pattern decoding and realignment, and resurrections of connections to the inner self. It is within the null value that we find new and different meaning, allowing something unforeseen and generous to emerge.

Yet these examples are not exceptional; they continue a longer lineage of quilts as informational interfaces that store and transmit data. In this way, they act as refractions, altering and resituating the centrality of pattern canonized in the works of Claude Shannon, Warren Weaver, and Norbert Wiener. In their framing, pattern was singular and universal, privileging legibility, uniformity, and repeatability. This separation of signal from noise ensured that the aesthetics of pattern remained absent. The quilts call this assumed and normative aesthetic ordering into question. For aesthetics are not simply a question of style or appearance but indicative of the foundational protocols that define and determine taxonomies of order. This aesthetic order helps to determine what counts as information and what falls beyond its bounds, conceivable only as noise or absence. Yet as we have argued here, absence is not simply the lack of presence, but can also hold within it the noisy spaces for possibility, resounding with the din of improvised relations.

It is in the context of this improvisation that we look to absence as ab-sense. Breaking apart the term foregrounds the prefix «ab» (meaning away or from) alongside the «sense» of embodied perception. To work from embodied perception is to recognize the always implicated and implicating position of the viewer or analyst and to confront the presumed objectivity and enforced distance in data work. In this intimate relation, absence invites an attunement to bodies in motion, to encounters beginning from the sensory, and a particular bodily knowing. It reveals experiences encoded in archives, databases, or technologies as always tied to ideologically shaped lives.

More concretely, this shift to absence enables us to see the practical limits of approaches to bias that frame their interventions as «fixes» for the flaws of such datasets and data systems. Such fixes, which often focus on shifting visible outputs, have the effect of naturalizing the relationship between bias and corrective, rather than retaining focus on the source of the bias in the first place. Reconceptualizing bias as absence points us both to the social, historical, and political conditions that have shaped the datasets we make use of in our work, and to future possibilities for working with data that exceed the bounds of what our existing datasets appear to permit. 

Acknowledgments
We thank our colleagues in the UW TAT Lab and UW Simpson Center for the Humanities «Reimagining Datafication» research cluster as well as funding support offered by the UW Simpson Center and NSF grants 2222242, 2310515, and 2210497. We also thank Afroditi Psarra, Gabrielle Benabdallah, Chari Glogovac-Smith, Lacey Jacoby, and Bettina Judd. A longer version of this essay appears in Proceedings of the 2024 ACM Designing Information Systems Conference.

Works Cited