Header Banner
Human Adaptation Studies
Human Experience of AI

Human Adaptation Studies

Human Adaptation Studies: How Technological Change Reshapes Human Thought, Behavior, Relationships, Institutions, and Our Culture Itself



We spend enormous effort studying technology. We study how systems work, how to build them, how to regulate them, and how to deploy them. Far less attention is given to a different question.


How do humans adapt when technology changes the conditions of everyday life?


Artificial intelligence has made that question difficult to ignore, but it did not begin with AI. Every major technological shift, from the printing press and the telephone to the internet and smartphones, has required people to change how they work, communicate, learn, trust, and relate to one another. Technology changes. People adapt.


I use the term Human Adaptation Studies (HAS) to describe an emerging field of inquiry that examines how technological change reshapes human thought, behavior, relationships, institutions, and culture. 



Human Adaptation Studies begins with a simple assumption: every significant technological shift is also a human one.


Biology studies how organisms adapt to changing environments. Human Adaptation Studies asks what happens when technology changes the human environment.


That assumption changes the questions we ask. Instead of asking whether a system is technically successful, we ask why people choose to trust it or not. Instead of asking whether AI can perform a task, we ask how working alongside AI changes human judgment and expertise. Instead of asking whether social robots are "real" companions, we ask why humans predictably form meaningful relationships with artificial systems in the first place.


Those questions have followed me throughout my career. In telecommunications, I watched organizations purchase sophisticated systems that employees never fully embraced. As an operations leader, I saw technically successful implementations fail because they fit organizational requirements better than human ones. Later, teaching communication, ethics, and leadership, I encountered the same pattern again. Students were not simply learning new technologies. They were adapting to new expectations, new relationships, and new ways of understanding themselves.


The rapid integration of artificial intelligence into everyday life has made these questions impossible to ignore. Educational institutions are reconsidering academic honesty, technological equity, and workforce preparation. Organizations are redesigning workflows, redefining expertise, and experimenting with new forms of collaboration. Individuals are deciding when to trust AI, when to challenge it, what tasks to delegate, and even what kinds of relationships to form with increasingly responsive systems.


These are not simply technical decisions. They involve judgment, trust, communication, identity, responsibility, and changing expectations about work and everyday life. As intelligent systems move beyond performing calculations to participating in cognitive, creative, and emotional activities, understanding the technology alone is no longer sufficient. We must also understand how people adapt to living alongside it.


My own research examines these questions through several connected areas of inquiry, including emotional attachment to social robots, trust and adoption in AI systems, human-AI collaboration, organizational change, and the ethical consequences of increasingly intelligent technologies. Frameworks such as the Personified Affective Transference Model (PATM), the Trust and Affect Systems Model (TASM), and the TRACE Framework each examine different expressions of the same underlying phenomenon.


At first glance, those topics appear unrelated. However, each asks the same question from a different angle: How do humans adapt when technology changes the conditions of everyday life?


Artificial intelligence will not be the last technology to reshape society. Before AI came the internet. Before the internet came television, automobiles, telephones, electricity, and the printing press. The technologies will continue to evolve. The underlying human questions will remain.


Artificial intelligence is one chapter in a much larger story. Human Adaptation Studies (HAS) is not ultimately the study of any particular technology. It is the study of how people respond when technology reshapes the world around them.


Technology evolves. Humans adapt. Understanding that process is the work of Human Adaptation Studies.

Read more
From Tool to System: How AI Reshapes Learning in Higher Education
AI in Education Human Experience of AI

From Tool to System: How AI Reshapes Learning in Higher Education

Artificial Intelligence and Student Life: 

What Is Actually Changing?


Walk into almost any college classroom right now and ask a simple question: how many students are using artificial intelligence to support their learning? Most hands go up, although not always immediately. Some students hesitate. Some look around first. But eventually, the majority of them acknowledge what has already become normal. That moment, more than any institutional policy or faculty debate, reveals how artificial intelligence is already re-shaping student learning in higher education.

What Students Are Actually Doing

It is tempting to start this discussion with definitions: artificial intelligence (AI), generative systems (GAI), machine learning (ML), and large language models (LLM). These explanations have their place, but they don’t reflect how students actually encounter this technology. Students experience AI during moments of difficulty. They face it when they don’t know how to start a paper, when a reading doesn’t make sense, or when they are trying to see if their interpretation is even close to correct. They ask a question, and something responds. The reply isn’t always accurate, but it’s immediate, structured, and confident.

In this sense, AI is not simply a shortcut. For many students, it is functioning as an entry point into the work itself. It provides a way to begin, to test an idea, or to translate confusion into something more manageable. In some cases, it resembles the type of academic support that is unevenly available across student populations.

For many students, AI is not replacing the work. It is making the work possible to begin.



Is This Moment Actually New?

At this point, it is worth asking whether this moment is truly new or whether it is another iteration of a familiar pattern. Higher education has already experienced several technological shifts that prompted similar concerns. Calculators raised questions about the loss of mathematical skill. Search engines reshaped how research was conducted. Word processors changed the writing process itself. Socrates warned that the act of writing things down threatened thinking. In each case, there were warnings about overreliance and diminished ability. Those warnings were not entirely wrong, but they also did not fully capture what was happening. The tool changed, but more importantly, the relationship to the task changed.

What changes with new tools is rarely just the task. It is the relationship to the task.

What Changes When the Tool Responds?

Artificial intelligence appears to follow this pattern, but it also introduces a complication that earlier technologies did not. A calculator does not respond to uncertainty. A search engine does not refine your question in real time. Generative AI does both. Students do not simply input a query and receive an answer. They revise their prompts, adjust their language, and ask again. There is a back and forth that begins to shape how they approach the task.

This raises a question that is not yet fully addressed in most higher education conversations. At what point does repeated interaction begin to alter how a student experiences the learning process? My research on human-AI companionship examines this more directly by asking what kind of relationship a system invites: helper, assistant, guide, or something else entirely. Students may not articulate this in theoretical terms, but they respond to it in practice. They return to the same systems, develop preferences, and begin to rely on them in moments of uncertainty. The system does not possess intention or awareness, but the interaction still produces a relational effect.

If interaction is beginning to shape how students engage with their work, then it also begins to shape how we should think about evaluating that work.

The system may not be relational, but the experience of using it often is.

What Are We Measuring Now?

This becomes more complicated when we consider what is actually being evaluated in higher education. If a student can generate a coherent essay, construct an argument, and produce writing that appears polished, then we have to ask what those outputs were measuring in the first place.

It is easy to frame this as a problem of academic integrity, but that framing may be too narrow. The issue is not only that students can outsource work. That possibility did not begin with AI. It is that the system has long relied on products as proxies for learning. Artificial intelligence makes that reliance more visible.

At the same time, students are not the only ones adjusting their practices. Faculty are also incorporating AI into their workflows. They use it to draft feedback, organize ideas, and manage the increasing demands on their time. This creates a situation in which both sides of the educational exchange are interacting with the same underlying systems. Students produce work with AI. Faculty respond with the assistance of AI.


Where is learning situated when both students and faculty are using AI?

Who Benefits and Who Does Not?

The issue of equity further complicates this landscape. Artificial intelligence has the potential to expand access to cognitive support, particularly for students who may not have consistent access to tutoring or other academic resources. At the same time, access to AI tools, as well as the ability to use them effectively, is uneven. Institutional policies vary, levels of AI literacy differ, and some students are positioned to benefit more quickly than others. AI can both extend opportunity and reproduce existing inequalities depending on how it is implemented and who is able to take advantage of it.

Student perspectives on AI often reflect this complexity. Some students use AI to deepen their engagement with material by testing interpretations and refining their thinking. Others use it to complete tasks efficiently and meet external expectations. Both uses exist simultaneously, and neither can be treated as the singular story of AI in education.

AI does not level the playing field. It rearranges it.

Where Does AI Belong?


Efforts to locate AI within a culture of academic integrity emphasize responsibility, transparency, and ethical engagement. These are important principles, but they rely on an assumption that the boundaries of AI use can be clearly understood and consistently applied. In practice, those boundaries are often blurred. As AI becomes more integrated into the learning process, distinguishing between independent work and assisted work becomes increasingly difficult. This shifts the conversation away from detection and toward questions of trust and accountability.

When we step back from individual cases and institutional responses, a broader pattern begins to emerge. Artificial intelligence is not simply adding a new layer to student life. It is redistributing aspects of cognition, altering how students approach tasks, and introducing new forms of interaction into the learning process. At the same time, it is exposing tensions that have long existed within higher education, particularly the reliance on outputs as indicators of understanding.

The problem is not simply how students are using AI. It is what AI reveals about how learning has been defined.

Why the Humanities Push Back

One place where this tension becomes especially visible is in the humanities. Faculty resistance to AI is often framed as reluctance to adopt a new tool, but that framing misses what is actually at stake. Humanities disciplines are built on a particular understanding of how knowledge is produced through close reading, sustained argument, and the development of an individual voice over time. When AI can replicate parts of that process, even imperfectly, it raises questions that extend well beyond academic integrity. It touches the foundations of disciplinary identity and what counts as meaningful intellectual work. What is being contested is not just method, but the definition of knowledge itself.

I explore this dynamic in more detail in a companion video, where I argue that what looks like resistance is often a rational response to a deeper epistemological shift. If you are interested in how faculty are experiencing this moment from the inside, you can watch that discussion here.

What Comes Next

This does not mean that AI should be rejected. There is clear value in its ability to reduce barriers, support idea generation, and make complex material more accessible. The challenge is not whether to use AI, but how to integrate it in ways that support meaningful engagement rather than replace it. This requires attention to how assignments are designed, how learning is assessed, and how students are taught to work with AI systems responsibly.

The more difficult question is not technical but philosophical. If students can rely on external systems to assist with thinking, writing, and problem-solving, then the location of intelligence becomes less stable. It is no longer confined to the individual. Instead, it is distributed across systems, tools, and interactions.

What follows from that shift is still unfolding. Higher education has not yet determined what should remain individual, what can be distributed, or how those boundaries should be drawn. For now, students are navigating that terrain in practice, often ahead of the institutions designed to guide them.

We are not at the end of this shift. We are in the middle of learning how to live with it.

Read more
TRACE Framework: Five Questions for Using AI Wisely
AI and Work AI in Education Ethics Human Experience of AI

TRACE Framework: Five Questions for Using AI Wisely

TRACE™️: Five Questions for Using AI Wisely


Artificial intelligence has quietly changed the kinds of decisions people make each day. That change is easy to miss because many of the visible activities remain familiar. People still write emails, summarize meetings, search for information, brainstorm ideas, and solve problems. On the surface, the work often looks the same. What has changed is the relationship between the person doing the work and the system participating in it.

Consider how easily an ordinary question becomes a negotiation. A student asks an AI system to explain a difficult concept. A manager requests a summary of a lengthy report. A physician asks for possible differential diagnoses before meeting with a patient. None of these examples removes the human from the process, yet each redistributes part of the cognitive work that was previously performed without technological assistance. The interaction is no longer simply one of using a tool. It becomes a negotiation over evidence, judgment, interpretation, and responsibility.

Much of the public conversation about AI continues to focus on capability. Can the system write well enough? Can it reason? Can it replace particular kinds of work? Those are understandable questions because new technologies are often evaluated according to what they can accomplish. They tell us relatively little, however, about what happens after those capabilities become ordinary. Once AI participates in everyday work, the more interesting question may not be what the technology can do but how people begin adapting around it.

That shift in perspective is where TRACE™️ begins. Rather than evaluating AI as a technology, TRACE examines the conditions created when intelligent systems become participants in human decision-making. The framework is less concerned with whether AI should be used than with understanding the kinds of questions that become increasingly important once its use becomes commonplace. Those questions recur across workplaces, classrooms, hospitals, governments, and homes because they arise from the structure of the interaction itself rather than from any particular application.

TRACE organizes those recurring questions into five domains: trust, reality, agency, cognition, and exposure. Together, they provide a way of examining not only individual decisions but also the broader human adaptations taking place as AI becomes part of everyday life.

Each domain represents a different way AI participates in human decision-making. Although they can be considered individually, they are most useful when viewed together. A trustworthy response may still influence how we understand a problem. An efficient workflow may reduce opportunities for independent thinking. A helpful recommendation may leave important questions about responsibility unanswered. TRACE is therefore less a checklist than a set of perspectives that reveal different aspects of the same interaction.

The Five Domains of TRACE

Trust

How much confidence should I place in this response?

Trust is often treated as though it were a property of technology. We ask whether an AI system is trustworthy in much the same way we ask whether a bridge is structurally sound or whether a calculator produces the correct answer. Yet trust has never worked that way. It is not a characteristic that exists independently of either a person or a machine. It is a relationship that develops between them, shaped by repeated interactions, previous experience, institutional reputation, and the consequences of being wrong.

AI complicates that relationship because it produces something humans are particularly susceptible to mistaking for evidence: fluency. A response that is coherent, confident, and immediate often feels more reliable than one that is hesitant or incomplete. That impression persists even when the underlying information contains errors, omits important context, or presents speculation alongside fact without clearly distinguishing between them. The challenge is rarely that people trust AI absolutely. More often, trust becomes miscalibrated. Confidence gradually becomes attached to presentation rather than evidence.

The purpose of the Trust domain is therefore not to encourage skepticism for its own sake. Distrusting accurate information is no more useful than believing inaccurate information. Instead, Trust asks whether our confidence is proportional to the quality of the evidence supporting the response. The central question is not whether AI can be trusted, but how trust should be calibrated as these systems become routine participants in everyday thinking.

Reality

How might this shape my understanding?

We often imagine that information changes our minds by adding new facts. Just as often, however, it changes our understanding by changing what receives our attention. Every explanation foregrounds certain relationships while leaving others in the background. Every summary decides what deserves emphasis and what can be safely omitted.

This matters because most people encounter AI not as a database but as a conversational partner. The system does not simply retrieve information. It selects examples, establishes tone, chooses analogies, and determines which perspectives appear relevant to the question being asked. Those decisions influence what becomes visible and what remains outside the frame. Over time, repeated interactions can narrow the range of possibilities a person routinely considers without that narrowing ever becoming obvious. Reality, therefore, asks readers to pay attention not only to accuracy but also to framing. A technically correct response may still guide attention toward one interpretation while obscuring others. The question is less "Is this true?" than "How is this response shaping the way I understand the situation before me?" Recognizing that distinction makes visible one of AI's quieter forms of influence.

Agency

Who remains responsible?

Responsibility has always been easier to assign when the boundaries between people and tools are clear. A spreadsheet performs calculations without creating uncertainty about who made the financial decision. A search engine retrieves information without claiming ownership of the conclusions drawn from it. AI occupies a more ambiguous position because it increasingly contributes to judgments that have traditionally been understood as human work.

That ambiguity creates conditions in which responsibility can become surprisingly difficult to locate. Individuals may defer to algorithmic recommendations because the system appears objective. Organizations may attribute undesirable outcomes to the technology rather than to the choices surrounding its design, deployment, or oversight. As decision-making becomes distributed across people and intelligent systems, accountability risks becoming similarly diffuse.

Agency asks readers to resist that diffusion. The question is not whether AI participated in the decision. Increasingly, it will. The more important question is whether human responsibility remains visible once that participation occurs. Technology may redistribute work, but responsibility does not disappear simply because it has become easier to overlook.

Cognition

What thinking am I outsourcing?

Every technology redistributes cognitive work. Writing externalized memory. Calculators reduced the need for mental arithmetic. Search engines transformed information retrieval. AI represents another step in that longer history, but with a notable difference. Rather than supporting isolated cognitive tasks, it increasingly participates across many forms of thinking simultaneously. It summarizes, synthesizes, critiques, brainstorms, drafts, plans, and evaluates. Activities that once required sustained intellectual effort can now be completed collaboratively with a conversational system.

The question, however, is not whether these changes are good or bad. Humans have always developed technologies that extend cognitive capacity. The more interesting question concerns what happens when delegation becomes habitual. Human abilities develop through repeated use. They also decline through repeated delegation. The question is not whether AI changes cognition. It clearly does. The more important question is which cognitive capacities we gradually stop practicing because the technology performs them well enough.

The Cognition domain is concerned with the relationship between efficiency and dependence. Delegating routine work may create opportunities for deeper thinking, but only if the capacities being preserved remain intentional. Otherwise, convenience quietly becomes substitution, and the distinction between assistance and replacement becomes harder to recognize.

Exposure

What information am I sharing?

Privacy is often mistaken for secrecy, as though it matters only when someone has something to hide. Everyday life suggests otherwise. People routinely decide what to reveal, to whom, and under what circumstances. Those decisions make social life possible because they preserve the ability to manage the boundaries between different relationships.

AI systems expand those opportunities for disclosure because they invite interaction in unusually conversational ways. Users describe problems in detail, upload documents, explain workplace processes, and reveal uncertainties that might never appear in a traditional search engine query. Even when individual exchanges seem insignificant, repeated interactions contribute to larger patterns of data collection and inference. What appears to be a series of isolated prompts may also become part of a broader informational portrait.

Exposure encourages readers to think beyond the narrow question of whether a particular prompt contains confidential information. Instead, it asks how participation in AI systems gradually reshapes the boundaries between what remains private, what becomes observable, and what is ultimately available for analysis. Privacy, in this sense, is not merely the protection of information. It is the ongoing ability to decide what parts of ourselves, our organizations, and our thinking remain under our own control.

Why TRACE Matters

Frameworks become useful when they provide language for changes that are otherwise difficult to notice while those changes are still unfolding. AI is introducing new habits of working, learning, deciding, and communicating so gradually that many of its most significant effects appear ordinary by the time we recognize them. TRACE does not attempt to predict every consequence of artificial intelligence. Instead, it offers a way of observing those consequences while they are still emerging.

The five domains are not independent categories, nor are they questions to answer once and set aside. Trust influences how people construct reality. Reality shapes perceptions of agency. Agency affects the kinds of cognition people delegate to intelligent systems. Each interaction also produces new forms of exposure. The domains overlap because they describe different dimensions of the same process: the ongoing adaptation of human thinking, judgment, and social life to increasingly capable AI systems.

For that reason, TRACE is best understood as a practice of attention rather than a checklist. Returning to these questions over time makes it easier to notice how AI is changing not only the work we do, but the conditions under which that work takes place. The technology will continue to evolve. Human adaptation will continue alongside it. The value of TRACE lies in giving us a vocabulary for observing those changes before they become so familiar that we stop seeing them.




Tags: 

AI Ethics, TRACE Framework, Human-Centered AI, AI Literacy, Human-AI Interaction, Trust in AI, Human Adaptation





 

Read more
Presentation Podcast Appearance (BG Ideas) Human Relationships with Robots
Human Experience of AI Presentation

Podcast Appearance (BG Ideas) Human Relationships with Robots

We keep asking if robots can feel. The more interesting question is why we already act like they do.

 Podcast Appearance: BG Ideas

Human Relationships with Robots



When we talk about artificial intelligence, the conversation usually gravitates toward risk: job displacement, automation, and the unsettling possibility that machines may one day outperform us in domains we once considered distinctly human. These concerns are legitimate. However, they tend to crowd out a quieter and, frankly, more interesting transformation that is already underway. 

People are forming meaningful relationships with robots. 

In this episode of BG Ideas, I talk about my research on human-robot interaction: how people interpret and respond to machines designed not just to compete in tasks, but to engage socially, through gestures, expressions, and behaviors that read as relational. We keep asking whether robots can feel. The more interesting (and possibly upsetting) question is why we already act like they do. 

That's where my research lives. As technological systems become more responsive and socially expressive, they begin to occupy territory once reserved for human connection. What we feel in those moments and why we feel it tells us something revealing about who we are becoming alongside our machines. 

It's a stranger, richer story than the standard AI headlines suggest. I hope you will listen in. 

Event:

BG Ideas Podcast
Recorded: October 17, 2025
Bowling Green State University
Bowling Green, OH

Tags: 

Human-Robot Interaction, Affective Technology, Technology and Culture, Presentation,
Read more
Presentation Who Programmed This? Trust, Betrayal, and the Design of Intimacy in They Cloned Tyrone
Ethics Presentation

Who Programmed This? Trust, Betrayal, and the Design of Intimacy in They Cloned Tyrone

 The horror in They Cloned Tyrone is not cloning or artificiality; it is the concealment of design and the ethical danger of identities engineered without consent.


Overview

This talk examined They Cloned Tyrone as a meditation on authorship, identity, and invisible technological infrastructure. Beginning with Fontaine’s moment of recognition of "That’s not me!” when seeing his body on a morgue table, the analysis reframes the film not as a conspiracy narrative, but as a structural argument about what happens when familiarity itself is programmed.

The Glen operates as an engineered environment in which stereotype becomes destabilized shorthand, repetition becomes calibration, and intimacy becomes a design tool. Trust here is not an organic emotional state but a feature of infrastructure, one that lowers resistance and smooths compliance. The clones succeed precisely because they are seamless; nothing feels wrong, so nothing is questioned.

Trust here is not an organic emotional state, but rather a feature of infrastructure

Drawing on empirical research into emotional relationships between humans and AI systems, the talk distinguished between negotiated intimacy (where the artificiality of a relationship is known to both parties) and concealed authorship (where design remains hidden). The ethical violation in the film is not simulation. It is undisclosed shaping. When improvement is framed as benevolence and optimization masquerades as care, identity becomes a product of encoded vision rather than a site of self-determination.

The central argument is ultimately this: the critical question for contemporary AI ethics is not whether systems shape who we become; they do and always have, but whether we can see the shaping, name the designer, and contest the design before we find ourselves looking at a reflection that no longer feels like our own. 

Event

Black Pop Culture Virtual Symposium
February 21, 2026
Virtual Symposium

Tags

Technology and Culture, AI Ethics, Human-Computer Interaction, Presentation
Read more
Trust & Affect System Model (TASM)
AI and Work

Trust & Affect System Model (TASM)

 

Why AI Quietly Fails Before the Metrics Notice

Dr. Kristine Ketel

The Quiet Failure Problem

Most AI-enabled systems do not fail because they are inaccurate. They fail because people quietly stop using them. This pattern is visible across industries but systematically misdiagnosed.

Long before performance metrics show a decline, users hesitate. They overcheck outputs. They build informal workarounds. They restrict use to low-stakes situations or disengage altogether. From the outside, the system appears functional. Inside the system, trust has already begun to erode.

These failures are rarely dramatic. There is no single breaking moment, no obvious error event, no clear line where confidence suddenly collapses. Instead, adoption thins over time. Reliance becomes conditional. What looks like "successful deployment" on a dashboard increasingly resembles quiet avoidance in practice.

When leaders notice low uptake or inconsistent use, the explanation often turns toward users. They lack training. They resist change. They do not understand the system's value. At other times, attention shifts to the technology itself, arguing that models need refinement, interfaces need improvement, and accuracy needs to increase.

Yet these explanations struggle to account for a familiar contradiction: why technically sound systems remain underutilized even after training, iteration, and demonstrated capability.

What is missing from these accounts is time.

Quiet failure unfolds gradually, through everyday interaction. It is not triggered solely by major errors or obvious harms. It emerges through small moments of friction: a recommendation that feels opaque, an automated judgment that feels unfair, a system decision that removes agency without explanation. Each moment on its own appears trivial. Together, they reshape how people relate to the system and what they expect from it.

This is what makes quiet failure difficult to detect and even harder to reverse. Traditional success metrics (accuracy, speed, task completion) often remain stable long after the underlying relationship between people and the system has degraded. By the time performance declines or ethical concerns surface, legitimacy has already been lost in practice.

Importantly, this pattern is not unprecedented.

Organizations have long struggled with systems that appear effective on paper but fail in lived experience. Disengagement precedes turnover. Resistance takes the form of compliance without commitment. Ethical breakdown rarely begins with overt misconduct; it begins with loss of trust, erosion of meaning, and rational withdrawal.

AI-enabled systems are now reproducing these same dynamics. This happens not because users are irrational or fearful, and not because the technology is inherently flawed, but because these systems increasingly shape behavior, authority, and accountability in ways that reflect organizational structures.

This pattern calls for a new analytical model, one that treats AI-enabled systems not as tools but as organizational actors exercising authority. The more relevant question is not whether people trust AI-enabled systems, but why trust becomes difficult, or even irrational, within the systems we design. 

AI as an Organizational Actor

To understand why quiet failure occurs, we need to reconsider what modern AI-enabled systems actually do in real-world contexts.

Most discussions still frame AI as a tool, as an instrument that supports human decision-making, improves efficiency, or automates discrete tasks. That framing no longer captures how AI-enabled systems function in practice, particularly in environments where they shape decisions repeatedly and at scale.

Contemporary AI-enabled systems increasingly perform roles once occupied by organizational structures and human authorities. They evaluate performance, rank priorities, recommend actions, control access to resources, and coordinate workflows across teams and time. In many contexts, they do not merely assist decisions; they shape which decisions are available, visible, or actionable in the first place.

These functions are not neutral.

In organizational settings, systems that evaluate, allocate authority, and constrain action are understood as exercising power even when that power is indirect, procedural, or distributed. Policies, procedures, and managerial hierarchies do not require intentions or emotions to influence behavior. They do so through structure, consistency, and enforcement.

AI-enabled systems now occupy a similar position. Authority is mediated not through human judgment, but through technical design.

This is not a metaphor. AI-enabled systems do not merely resemble organizations symbolically; they increasingly function as organizational actors by shaping incentives, defining norms, and redistributing responsibility. When an algorithm determines whose work is flagged, which cases are escalated, or which recommendations are prioritized, it performs the same coordinating role long assigned to managers, committees, or institutional rules.

Despite this, AI-enabled systems are often deployed without reference to the extensive body of knowledge addressing how authority structures affect trust, motivation, and ethical behavior. The relational consequences of system design are treated as secondary concerns, as user attitudes to be managed after deployment rather than structural outcomes to be anticipated.

This mismatch produces a predictable failure mode.

When AI-enabled systems are designed solely around task performance, they inherit organizational power without organizational safeguards. They influence behavior without providing sensemaking, enforce decisions without accountability, and constrain agency without mechanisms for repair. From the user's perspective, disengagement becomes a rational response to a system that feels unclear, unresponsive, or unjust.

Once AI-enabled systems are recognized as organizational actors, the quiet failure problem becomes legible. In fact, what appears to be a problem of user trust is a problem of system governance.

Why Trust and Affect Are System Variables

Once AI-enabled systems are understood as organizational actors, the limitations of existing trust frameworks come into focus.

Much of the current discussion treats trust as an internal psychological state, something users possess, grant, or withhold based on attitudes toward a system. Affect is often framed as an emotional response users bring into the interaction. These perspectives capture important aspects of experience, but they operate at the wrong level of analysis for understanding quiet failure.

In organizational contexts, trust is not primarily a feeling. It is an infrastructural condition that emerges from how systems distribute power, enforce rules, and respond to breakdown. People do not decide whether to trust an organization in the abstract. Trust becomes rational or irrational through repeated interaction with processes that feel predictable or arbitrary, fair or biased, responsive or indifferent.

The same logic applies to AI-enabled systems that now mediate organizational authority.

In this sense, trust is a property of the system. It is produced, or undermined, by design features such as transparency, consistency, accountability, and repair capacity. When these features are present, trust becomes a reasonable stance. When they are absent, skepticism and disengagement are not failures of attitude; they are logical responses to structural conditions.

Affect operates in a similarly systemic way.

This framework understands affect as elicited by the system itself, rather than treating emotion as a subjective overlay on neutral interaction. AI-enabled systems generate affective conditions through how they constrain choice, allocate agency, and communicate (or fail to communicate) the rationale behind their actions. Confidence, anxiety, resentment, and relief are not incidental reactions. They are signals produced through ongoing interaction with authority embedded in technical form.

These affective conditions shape behavior long before explicit judgments of trust are articulated. Anxiety leads to overchecking. Resentment leads to workarounds. Loss of agency leads to withdrawal. None of these behaviors require a conscious decision to distrust the system; they emerge naturally when the system's design makes trust difficult to sustain.

Trust and affect also interact. Affective strain accelerates trust erosion, while degraded trust amplifies negative affect. This feedback loop unfolds gradually and often invisibly, particularly in systems that continue to meet formal performance criteria.

From a governance perspective, trust and affect function as leading indicators, signals that appear well before accuracy or efficiency metrics register decline.

Introducing the Trust & Affect Systems Model (TASM)

If AI-enabled systems function as organizational actors, and trust is infrastructural rather than psychological, then existing frameworks treating trust as user sentiment are operating at the wrong level of analysis. A systems model is needed that maps how design choices propagate through affect, behavior, and ethical risk over time.

To make these dynamics visible, I propose the Trust & Affect Systems Model (TASM), a systems-level framework for understanding how AI-enabled system design and governance shape trust, affect, behavior, and ethical risk over time.

TASM does not model individual attitudes toward AI, nor does it attempt to measure trust as a psychological state. Instead, it treats trust and affect as emergent properties of sociotechnical systems that increasingly function as organizational actors.

TASM describes a recursive process:

1. System Role & Design: AI-enabled systems are deployed into organizational roles such as evaluator, advisor, gatekeeper, or coordinator. These roles determine how authority is exercised and the amount of discretion users retain.

2. Affective Conditions Produced: Systems generate affective conditions through their role and design. These conditions include confidence or anxiety, clarity or resentment, agency or powerlessness.

3. Trust Infrastructure: Over time, affective conditions contribute to the emergence or erosion of trust infrastructure. Trust infrastructure is shaped by predictability, transparency, consistency, and the ability to make repairs.

4. Behavioral Response: Trust infrastructure shapes use in practice: reliance, overchecking, workarounds, conditional use, or avoidance.

5. Ethical & Organizational Risk: Behavioral adaptations generate ethical and organizational risk, including automation bias, moral disengagement, accountability diffusion, and a decline in legitimacy.

6. Metrics & Oversight (Lagging Indicators): These risks surface, if at all, through conventional performance metrics that detect problems only after trust and affect have already deteriorated.

Figure 1: The Trust & Affect Systems Model diagram

This process is recursive, not linear. Crucially, metrics and oversight feedback into system redesign, often without addressing the trust and affect conditions that produced failure in the first place. Organizations may repeatedly refine technical performance while leaving the underlying causes of disengagement unaddressed.

Consider a performance evaluation system that provides scores without explanation. Employees feel anxiety (affect), begin over-documenting to protect themselves (behavior), which degrades collaboration (organizational risk), while engagement scores remain acceptable (lagging metrics). Each 'improvement' iteration focuses on scoring accuracy while never addressing the opacity that triggered withdrawal in the first place.

This is how quiet failure becomes entrenched.

What This Changes

If trust and affect are treated as system variables rather than user dispositions, the implications for AI design, governance, and ethics are substantial.

For Designers

TASM reframes trust as a design outcome rather than a user goal. Designing for trust doesn’t mean encouraging reliance. It means creating systems with legible authority, explainable decisions, and repairable failures. Opacity and rigidity aren’t just usability issues; they are structural conditions that produce disengagement over time.

For Leaders

The model shifts responsibility away from users and toward governance. Adoption failure doesn't mean people 'don't trust AI,' but that trust has not been made rational within the system. This redirects attention upstream from training and change management to role definition, accountability, and oversight.

For Ethics and Compliance Teams

TASM challenges event-based approaches that focus on harm after the fact. Ethical risk does not originate solely in discrete failures; it accumulates through everyday interaction with systems that obscure agency or diffuse responsibility. From this perspective, ethical breakdown is not an anomaly. It is an emergent property of system design.

Conclusion: Reframing Failure

AI-enabled systems rarely fail all at once. They fail quietly.

They fail through gradual disengagement, conditional reliance, and loss of legitimacy that unfolds long before performance metrics register concern. These failures are often misattributed to user resistance or insufficient training, when they more accurately reflect structural conditions embedded in system design and governance.

The Trust & Affect Systems Model does not argue for greater trust in AI-enabled systems, but for systems that are deserving of trust.

TASM shifts responsibility away from users and toward the structures that shape interaction, authority, and accountability by foregrounding trust and affect as leading indicators of system health. It offers a way of naming a class of failures that have remained difficult to see, even as they recur across domains.

As AI-enabled systems continue to assume organizational roles, the question is no longer if trust matters.

It is whether we are willing to design, govern, and evaluate systems in ways that make trust reasonable.

The next article in this series examines how TASM can be operationalized to assess trust, affect, and ethical risk in AI-enabled systems before failures become visible in performance data.

Tags:

Human-Machine Interaction, AI and Work, Trust and AI, TASM
 

Read more
What is AI Ethics?
Ethics

What is AI Ethics?

Why the most important questions about artificial intelligence are  people-centered.

Have you noticed how quickly using AI begins to feel ordinary?


The first few times, the interaction may still feel novel. You ask for help drafting an email, summarizing a report, explaining a difficult concept, or thinking through a decision that has resisted your own attempts to organize it. The task gets easier, which is useful enough. But after repeated use, the more interesting change may not be the quality of the output. It may be the way the task itself begins to feel different.


People often describe that difference in similar terms. They say they brainstorm differently when AI is available. They check AI before searching the web. They remember fewer details because retrieval is always close at hand. Some feel more confident because they have a responsive system that can help them test an idea before showing it to another person. Others become less certain about where their own judgment ends and the machine’s suggestion begins. Any one of these shifts can be explained as an individual habit, but the pattern becomes harder to dismiss when similar accounts appear across classrooms, workplaces, families, and professions.


That is usually the point at which my attention sharpens. When many people independently report similar changes, I become less interested in treating those changes as personal quirks. The more useful question is what conditions are producing them. AI does not simply add a new tool to an unchanged environment. It changes what is available, what is easy, what is rewarded, what can be avoided, and what begins to feel normal. People adapt to those conditions, often before they have language for what has changed.


This is where my interest in AI ethics begins. Most discussions of AI ethics examine the systems themselves. They ask whether models are biased, whether data were collected responsibly, whether regulations are sufficient, or whether developers have built appropriate safeguards. Those are necessary questions because they help us evaluate the technologies we are creating and the institutions that govern them. They do not, by themselves, explain how people change once those systems become ordinary parts of everyday thinking, learning, work, and decision-making.


The questions that interest me begin with that adaptation. What happens when intelligent systems become ordinary companions in cognitive work? How does living alongside AI reshape judgment, trust, memory, relationships, and responsibility? What changes when technologies that once supported human thinking begin participating in the process of thinking itself? These questions point toward a broader understanding of AI ethics, one concerned not only with whether systems fail but with what happens when they become useful, familiar, and difficult to remove from daily life.


The ethical significance of AI does not begin when a system breaks. It begins when people begin reorganizing their habits around its presence. If we want to understand the long-term consequences of artificial intelligence, we need to study not only the systems we are building but also the conditions those systems create and the patterns of human adaptation that follow.


AI Ethics Is More Than Governance

Ask someone what "AI ethics" means and the conversation usually moves in a familiar direction. Before long someone mentions bias, privacy, regulation, copyright, government oversight, or the need for "responsible AI." That response makes sense. These are the ethical questions that receive the most public attention because they produce legislation, corporate policies, lawsuits, and headlines. They are also questions with reasonably clear objects of analysis. We can examine a model for bias. We can audit how data were collected. We can ask whether an organization complied with a particular regulation or whether appropriate safeguards were put in place.


There is nothing wrong with approaching AI ethics this way. These questions matter because they establish the conditions under which intelligent systems are designed, deployed, and governed. If those conditions are neglected, the consequences can be significant. It would be difficult to argue that governance is anything other than an essential part of AI ethics.


What interests me is what happens after those questions have been asked.


Imagine an AI system that has been carefully designed, responsibly deployed, and legally compliant. The governance questions have largely been answered. The system enters a workplace, a classroom, or a home and gradually becomes part of everyday life. At that point something begins happening that governance alone cannot fully explain. People start reorganizing their behavior around the new possibilities the system creates. Students study differently. Professionals approach writing differently. Teams begin solving problems differently. Over time these adjustments become ordinary enough that people often stop noticing they are adapting at all.


This strikes me as a different kind of ethical question. The system may be functioning exactly as intended, yet its presence is still changing the conditions under which people think, learn, trust, and make decisions. The explanation for those changes cannot be found entirely in the model or entirely in the individual. It emerges through the relationship between the two.


Once the question becomes one of adaptation rather than deployment, the scope of AI ethics begins to expand. We are no longer asking only whether a system complies with policy or satisfies a regulatory framework. We also begin asking how trust develops, what habits become easier to form, how judgment changes when thinking becomes collaborative, and under what conditions appropriate reliance gradually becomes dependence. These questions are not prompted by technological failure. They arise because the technology has become useful enough to become ordinary.


From that perspective, governance remains an indispensable part of AI ethics, but it is no longer the whole field. Understanding AI ethically requires understanding not only how intelligent systems should behave, but also how people and institutions change once those systems become woven into everyday life. The object of study is no longer the technology alone. It is the changing relationship between people, intelligent systems, and the environments they now inhabit together.


AI Changes More Than Work

Much of the conversation about artificial intelligence revolves around work. Will it make us more productive? Which jobs will change? Which professions will disappear? These are reasonable questions because new technologies have always altered the way work is organized. Looking only at work, however, has a way of making every technological change appear primarily economic. The more interesting changes often happen somewhere else.


Consider the technologies that gradually became ordinary over the past few decades. GPS made navigation easier, but it also changed what people needed to remember. Search engines made information easier to retrieve, but they also changed what it meant to know something. Spell check did more than catch typographical errors. It subtly changed the relationship between writers and the act of writing itself. Recommendation systems made discovery more efficient, yet they also began selecting what deserved our attention before we had decided to look for it. None of these technologies simply reduced effort. Each altered the conditions under which a particular cognitive task took place.


Taken individually, these changes seem modest. Together they suggest a different pattern. The technologies people tend to remember for their efficiency often leave a more lasting mark on habits of thought. They redistribute small pieces of cognitive work from the individual to the surrounding environment. Navigation becomes something we consult rather than remember. Information becomes something we retrieve rather than retain. Discovery becomes something we receive rather than pursue. The technology does not eliminate the human activity. It changes the conditions under which that activity occurs.


Generative AI appears to belong to this same history, but it also seems to extend it in an important way. Earlier technologies generally supported thinking from the outside. They helped people calculate, search, navigate, or correct. Increasingly, AI participates in the thinking itself. It suggests interpretations, develops arguments, identifies patterns, critiques ideas, proposes alternatives, and helps organize reasoning before a conclusion has been reached. The distinction is subtle, but it changes the relationship between person and technology in ways that earlier information systems did not.


That observation helps explain why conversations about AI ethics feel different from conversations about earlier technologies. The central question is no longer whether a tool performs its intended function. It is whether repeated interaction with that tool gradually reshapes judgment, memory, learning, trust, and decision-making. Those changes do not announce themselves all at once. They emerge slowly through ordinary use, becoming visible only after enough individual experiences accumulate into a recognizable pattern.


Seen this way, the ethical significance of AI lies not only in what the technology produces but also in what it reorganizes. If intelligent systems increasingly participate in activities that have long been understood as distinctly human forms of cognition, then AI ethics must concern itself with more than technical performance or workplace efficiency. It must also ask how those new conditions influence the ways people come to think, remember, learn, and understand the world.


Ethical Responsibility Is Distributed

Public conversations about AI failures tend to follow a familiar pattern. A story appears about an inaccurate recommendation, a biased hiring tool, a student using AI inappropriately, or a company deploying a system that produces unexpected consequences. The discussion quickly shifts away from the event itself and toward responsibility. Someone argues that the user should have exercised better judgment. Someone else blames the company that built the system. Others insist that regulators should have acted sooner. Although the conclusions differ, the conversation usually shares one assumption: somewhere there is a single place where responsibility properly belongs.


That assumption becomes more difficult to sustain the longer AI remains part of everyday life. Most interactions with intelligent systems are not isolated decisions. They are shaped by habits that develop over time, by systems designed long before any individual begins using them, and by institutions that establish what counts as acceptable, expected, or even possible. Looking for a single point of responsibility often tells us less about the problem than about our preference for simple explanations.


Take the individual using AI. A student decides whether to submit AI-generated work. A physician decides how much confidence to place in a clinical recommendation. A manager decides whether an AI-generated performance review deserves further scrutiny. These are clearly ethical decisions because they involve judgment, responsibility, and trust. At this level the question is not simply whether people use AI. It is how they decide when its recommendations deserve confidence and when independent judgment should prevail.


Those decisions, however, never begin with the individual. They begin much earlier, in the systems people encounter. Every model reflects thousands of design decisions concerning training data, optimization goals, interfaces, safety mechanisms, and acceptable trade-offs. Bias, explainability, privacy, and transparency do not emerge from users. They are properties of the systems users inherit. If similar patterns of confusion, overreliance, or misuse appear across many different people, it becomes difficult to explain those patterns solely through individual choice. The conditions created by the technology itself begin to matter.


Those conditions are, in turn, shaped by institutions. Universities establish policies governing acceptable AI use. Employers decide when AI should assist or replace human judgment. Governments determine regulatory frameworks. Technology companies decide which products are developed, what data are collected, and which values become embedded in commercial systems. These decisions create the environment within which both people and technologies interact. Institutions rarely determine individual behavior directly, but they shape the conditions under which certain behaviors become more likely than others.

What begins to emerge is not a hierarchy but a system. Institutions shape the systems people encounter. Those systems influence habits, expectations, and patterns of judgment. As those patterns become widespread, they reshape the institutions that produced them through new policies, cultural expectations, market pressures, and political demands. Responsibility does not sit permanently at any one point within that system because each part continually influences the others.


This is one reason I think AI ethics is often reduced too quickly to questions of governance or individual responsibility. Both matter. Neither is sufficient on its own. The ethical consequences of AI emerge through the interaction among people, systems, and institutions, making the relationship, not any single actor, the more appropriate object of study.


The Questions Matter More Than the Answers

People sometimes ask what my research has discovered. It is a reasonable question, but I have found myself answering it differently over the years. Rather than beginning with conclusions, I usually begin with the questions. That is not because answers are unimportant. It is because answers belong to a particular moment. Technologies change. Institutions adapt. New practices emerge. A satisfying answer today may become incomplete tomorrow. Well-constructed questions tend to survive those changes because they continue directing our attention toward the underlying mechanisms rather than the temporary forms they happen to take.


Those questions rarely emerge from theory alone. They usually begin with observations that seem too common to dismiss and too consistent to ignore. A teacher notices students approaching writing differently. A manager realizes that AI has quietly become part of everyday decision-making. Someone describes trusting an AI assistant more than they expected. Someone else realizes they no longer remember information because retrieving it has become easier than retaining it. None of these experiences proves anything on its own. The interesting question is why similar observations continue appearing among people who have never met, work in different professions, and encounter AI under very different circumstances.


Once those experiences begin repeating across classrooms, workplaces, healthcare, and public life, they become difficult to explain as isolated decisions. They begin suggesting that the surrounding conditions are changing. My research questions usually emerge at that point. Rather than asking whether AI is good or bad, I find myself asking what these recurring observations reveal about the relationship people are developing with increasingly intelligent systems.


One pattern, for example, appears whenever people describe AI as helping them think rather than simply helping them find information. Earlier technologies generally supported decisions by making information easier to access or calculations easier to perform. Increasingly, AI contributes ideas, organizes reasoning, proposes alternatives, and participates in the process that leads to a decision. That observation raises a different question. What happens when technologies that once supported judgment begin participating in it?


A similar pattern appears whenever people describe changes in how they understand the world around them. Recommendation systems influence what is encountered. AI assistants influence how problems are framed, how evidence is evaluated, and which possibilities seem worth pursuing. The individual experiences vary, but together they suggest that AI is mediating more than information. It is gradually becoming part of the environment through which people construct knowledge, trust expertise, and interpret reality.


The same shift appears when attention moves from systems to experience. Technical measures tell us whether a model performs accurately, efficiently, or reliably. They tell us much less about what it feels like to live with that model over months or years. Trust, uncertainty, confidence, frustration, dependence, and habit often appear in conversations about AI long before they appear in technical evaluations. That difference suggests another question. How do people experience intelligent systems, not simply use them?


The pattern appears again when institutions enter the picture. Organizations typically evaluate AI through governance, compliance, risk, and measurable performance. Individuals encounter the same systems through fairness, understanding, trust, and agency. Neither perspective is mistaken. They are simply observing the system from different locations. That difference, too, becomes a question worth following rather than resolving too quickly.


I do not expect these questions to produce permanent answers. Their value lies elsewhere. They help identify changes while those changes are still becoming ordinary. They provide language for experiences that many people recognize but have not yet connected to one another. Once those connections become visible, the object of inquiry begins to change. The question is no longer simply what AI is doing. It becomes what kinds of social conditions are emerging as people, institutions, and intelligent systems learn to live together.


Following the Pattern

Looking back, I don't think my work began with artificial intelligence. It began much earlier with a recurring observation that appeared in very different settings. Organizations often assumed that better technology would naturally produce better outcomes. Sometimes it did. Just as often, it did not. Systems that appeared technically successful failed to gain adoption. Other technologies became indispensable for reasons their designers had never anticipated. The technical explanation was often sound, yet it seemed to leave something important unexplained.


Over time, those experiences began to resemble one another. They suggested that technical performance and human response were related but not identical. Two systems with similar capabilities could produce very different patterns of use. People trusted one but ignored another. They incorporated one into everyday routines while resisting the other. If those differences appeared repeatedly across organizations, technologies, and contexts, then the explanation seemed unlikely to reside in the technology alone. Something about the relationship between people and systems deserved closer attention.


That line of inquiry eventually led me to artificial intelligence, although AI itself was never the destination. It became another environment in which the same questions appeared in new forms. People described AI as a coworker, a tutor, a brainstorming partner, or, in some cases, as a companion. Others rejected the technology despite acknowledging its capabilities. Neither response seemed especially surprising on its own. What drew my attention was that these patterns emerged repeatedly among people with different backgrounds, different professions, and different reasons for using AI.

Following those patterns gradually led my research in several directions. Trust became one area of study because confidence in a system often proved to be a better predictor of adoption than technical capability alone. Relational AI became another because people consistently formed social responses to systems they knew were artificial. Affect emerged as a third because feelings such as confidence, uncertainty, familiarity, or frustration were not simply reactions to technology. They became part of the conditions under which people decided whether to rely on a system, resist it, or incorporate it into everyday life.


The same process eventually widened again. Individuals never encounter AI in isolation. Their experiences are shaped by schools, workplaces, healthcare systems, governments, and technology companies that establish the conditions under which intelligent systems are introduced, encouraged, restricted, or normalized. More recently, similar questions have appeared in work on AI coworkers and AI companions. These may seem like distinct topics, yet they continue asking variations of the same underlying question: how do people adapt when increasingly intelligent systems become ordinary participants in everyday life?


Seen from a distance, these research areas may appear unrelated. To me, they have always felt like different observations of the same phenomenon. Each begins with an experience that many people recognize. Each becomes more interesting when that experience appears repeatedly across different settings. Each eventually leads away from individual behavior and toward the systems, institutions, and conditions that make those patterns possible. Artificial intelligence happens to be the environment I study today. The larger question has remained remarkably consistent: how do people adapt when the conditions of everyday life begin to change?


Conclusion

Following AI ethics through these questions gradually changes the object of inquiry. The conversation often begins with models, algorithms, governance, or regulation because those are the most visible expressions of the technology. They remain essential subjects of study. Systems should be safe. Data should be collected responsibly. Institutions should establish meaningful forms of accountability. None of those concerns disappear. They simply become part of a larger picture.


That larger picture comes into view once intelligent systems become ordinary enough that people begin reorganizing their lives around them. The ethical questions no longer concern only the technology itself. They concern the conditions the technology creates. People learn differently when AI becomes part of learning. They remember differently when retrieval becomes easier than recall. They make decisions differently when reasoning becomes collaborative rather than solitary. These changes rarely occur because anyone intended them. They emerge gradually as people adapt to a changing environment.


Seen from that perspective, AI ethics begins to look less like the study of intelligent machines than the study of increasingly AI-mediated societies. The technology remains important, but it is no longer the only subject. The relationships among people, intelligent systems, and institutions become equally important because those relationships shape how the technology is understood, adopted, resisted, and ultimately incorporated into everyday life. The ethical questions therefore move beyond what a system is capable of doing and toward what kinds of habits, expectations, and forms of social life that capability gradually makes possible.


I suspect this is why the field continues to expand. Earlier generations often asked how technology should be used. Increasingly, we find ourselves asking what happens when technology participates in activities that have traditionally belonged to human judgment, communication, creativity, and decision-making. Those questions are unlikely to disappear as AI becomes more capable because they are no longer only questions about technology. They are questions about the societies that emerge as people adapt to living alongside increasingly intelligent systems.


Perhaps that is where AI ethics has been heading all along. Not away from governance or technical design, but beyond them. Understanding intelligent systems will continue to matter. Understanding the changing conditions under which people live with those systems may matter just as much.

Read more