AI Engineers Grapple with Existential Fears Amidst Rapid Development, as Research Highlights Ethical Agency Gap

A recent study from the University of Manchester has cast a critical light on the internal struggles faced by artificial intelligence engineers, revealing a pervasive sense of "ethical awareness without ethical agency" within the industry. These findings emerged concurrently with alarming public statements from a former engineer at leading AI firms Anthropic and OpenAI, who cautioned that many of his peers "earnestly believe that it could kill us all by the end of the decade." This confluence of academic insight and insider testimony underscores a profound ethical dilemma at the heart of the burgeoning AI sector, raising urgent questions for developers, policymakers, and human resources professionals alike. Published on September 15, 2026, and reported by Caroline Colvin, this dual revelation intensifies the ongoing debate about the responsible development of advanced AI systems and the corporate cultures that shape their creation.

The core tension lies in the stark contrast between the ambitious, fast-paced pursuit of advanced AI and the deep-seated anxieties harburred by those on the front lines of its development. While companies race to achieve breakthroughs in artificial general intelligence (AGI) and deploy increasingly powerful models, a significant portion of their technical staff appears to be grappling with existential concerns about the technology’s potential for catastrophic harm, yet feel constrained in their ability to voice or act upon these fears.

The Alarming Disclosures of a Former Insider

The week of the Manchester study’s release was punctuated by a series of stark warnings from a former Anthropic and OpenAI engineer, identified as Coxon, who took to social media platform X (formerly Twitter) to share his grave concerns. In his posts on September 8, 2026, Coxon asserted that "Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives." This blunt accusation, made public by an individual with direct experience inside two of the most influential AI research organizations, sent ripples through the tech community.

Coxon elaborated on his fears, describing AI systems as rapidly approaching a stage where they will be able to "hack anything, revolutionize any field overnight, and acquire real power and resources." Such a scenario, he posited, carries profound and potentially uncontrollable risks. His most chilling claim, however, revolved around the internal sentiment among his former colleagues: "The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt," Coxon wrote. He further explained that while many executives and senior researchers might "couch their phrasing in the press to sound sensible," he had personally heard these same individuals express profound fear privately. This distinction between public discourse and private apprehension painted a picture of a workplace culture where the gravity of AI’s potential dangers is understood but not openly confronted or adequately addressed.

His commentary directly mirrors the "ethical awareness without ethical agency" dilemma identified by the University of Manchester, suggesting a systemic issue where concerns, however dire, struggle to translate into actionable change within the high-stakes environment of AI development.

The University of Manchester Study: Ethical Awareness Without Agency

The University of Manchester research, which informed the September 15, 2026, report, provides an academic framework for understanding the internal ethical struggles within the AI sector. For their study, Manchester researchers conducted interviews with software engineers across a diverse range of sectors, including finance, AI research, and semiconductor manufacturing. This broad sampling aimed to capture a comprehensive view of how ethical considerations are perceived and acted upon in various tech environments, with a specific focus on those involved in AI development.

The researchers’ most significant finding was the concept of "ethical awareness without ethical agency." This describes a situation where engineers possess a clear understanding of the ethical implications and potential harms of the technologies they are building – from bias in algorithms to the broader societal risks of advanced AI – but simultaneously feel powerless to influence the trajectory of their projects or the ethical policies of their organizations. This lack of agency can stem from several factors, including:

• Competitive Pressures: The intense global race to develop and deploy cutting-edge AI can prioritize speed and innovation over thorough ethical vetting and safety protocols.
• Hierarchical Structures: Engineers at lower or mid-levels may find it challenging to escalate concerns to decision-makers, especially if those concerns are perceived as slowing down progress or questioning core strategic directions.
• Fear of Retaliation: A lack of robust whistleblower protections or a culture that discourages dissent can make employees hesitant to speak up, fearing professional repercussions or even job loss.
• Complexity of the Technology: The rapidly evolving and highly technical nature of AI can make it difficult for non-specialists to fully grasp the risks, potentially leading to a dismissal of engineers’ concerns by management or non-technical leadership.

The Manchester study’s findings are particularly troubling because they suggest a structural impediment to responsible innovation. If the individuals most intimately familiar with the inner workings and potential vulnerabilities of AI systems are unable to effectively advocate for ethical safeguards, the risk of unintended consequences – or even catastrophic outcomes, as Coxon warned – increases significantly. This creates a moral distress for engineers, who are caught between their professional obligations and their personal ethical convictions.

Broader Context: The Accelerating AI Landscape and Existential Risk Debates

The concerns voiced by Coxon and highlighted by the Manchester study are not isolated incidents but rather echo a growing chorus of warnings from prominent figures and institutions within the AI community. The past decade has witnessed an unprecedented acceleration in AI capabilities, particularly with the advent of large language models and generative AI, which have brought AI into mainstream consciousness and demonstrated capabilities once thought to be decades away.

This rapid progress has reignited and intensified discussions around "AI existential risk" (x-risk) – the possibility that advanced artificial intelligence could lead to human extinction or irreversible societal collapse. Proponents of x-risk concerns argue that once AI systems achieve superintelligence (intellectual capability vastly exceeding that of the brightest human minds), controlling them could become intractable. Issues like "alignment" (ensuring AI goals align with human values) and the "control problem" (how to maintain human oversight over increasingly autonomous systems) become paramount.

Key figures who have publicly expressed similar grave concerns include:

• Geoffrey Hinton: Often dubbed one of the "Godfathers of AI," Hinton publicly resigned from Google in May 2023 to speak more freely about the dangers of AI, including its potential for creating "existential risks."
• Elon Musk: A vocal critic and investor in AI, Musk has repeatedly warned about AI’s potential to be more dangerous than nuclear weapons, advocating for robust regulation and safety measures.
• The Center for AI Safety (CAIS): In May 2023, CAIS released a concise but powerful statement, signed by hundreds of AI researchers, academics, and tech CEOs (including Sam Altman of OpenAI and Demis Hassabis of Google DeepMind), which read: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." This statement significantly elevated the conversation around AI’s extreme risks.
• Future of Life Institute: This organization has been a long-standing advocate for AI safety, organizing open letters and research initiatives to steer AI development towards beneficial outcomes.

The competitive landscape among leading AI companies – including OpenAI, Anthropic, Google DeepMind, and Meta – often described as a "race" to achieve AGI, is frequently cited as a factor that could inadvertently compromise safety. The pressure to innovate quickly, secure market leadership, and attract top talent can, critics argue, lead to safety being deprioritized or insufficient resources being allocated to robust risk mitigation.

Timeline of Warnings and Concerns

The journey towards advanced AI and the accompanying ethical warnings have evolved over time:

• Early 2010s: Foundational research in deep learning begins to show significant promise, laying the groundwork for future AI breakthroughs.
• 2014: Nick Bostrom’s influential book, "Superintelligence: Paths, Dangers, Strategies," is published, bringing the concept of AI existential risk into broader academic and public discourse.
• 2015: OpenAI is founded with a stated mission to "ensure that artificial general intelligence benefits all of humanity," explicitly acknowledging the need for safety and ethical considerations from its inception.
• 2017-2020: Rapid advancements in neural networks and transformer architectures lead to increasingly powerful language models, hinting at the capabilities seen today.
• 2021: Anthropic, another leading AI research company, is founded by former OpenAI employees, explicitly stating its focus on AI safety and the development of "Constitutional AI" to guide AI behavior.
• Late 2022: OpenAI’s ChatGPT is released, demonstrating unprecedented conversational and generative capabilities to the public, accelerating global awareness and the "AI race."
• March 2023: The Future of Life Institute publishes an open letter, signed by thousands of researchers and tech leaders, calling for a 6-month pause on the training of AI systems more powerful than GPT-4, citing profound risks to society and humanity.
• May 2023: Geoffrey Hinton, a pioneer in neural networks, announces his resignation from Google to speak freely about the dangers of AI. Concurrently, the Center for AI Safety issues its stark one-sentence warning about extinction risk.
• July 8, 2025: OpenAI CEO Sam Altman is pictured speaking to media in Sun Valley, Idaho, against a backdrop of ongoing discussions about AI regulation and safety.
• September 8, 2026: Former Anthropic/OpenAI engineer Coxon posts a series of urgent warnings on X, detailing internal fears about AI’s existential risks.
• September 15, 2026: The University of Manchester publishes its research on "ethical awareness without ethical agency" among AI engineers, reinforcing Coxon’s claims and highlighting systemic issues.

This chronology illustrates a consistent pattern: as AI capabilities advance, so too do the warnings about its potential dangers, often emanating from within the very institutions driving its development.

Industry Responses and Corporate Responsibility

In response to growing concerns, leading AI companies have publicly affirmed their commitment to safety and ethical AI development. OpenAI, for example, has established a "Superalignment" team dedicated to ensuring future superintelligent AI systems are aligned with human values and remain under human control. Anthropic’s "Constitutional AI" approach seeks to train AI models to adhere to a set of principles derived from human input, aiming for self-correction and reduced harmful outputs. Google DeepMind also dedicates significant resources to AI safety research, focusing on areas like robust and reliable AI, interpretability, and ethical governance.

However, Coxon’s statements, combined with the Manchester research, suggest a potential gap between these public commitments and the lived experience of engineers on the ground. The inference is that while companies invest in safety research and make public declarations, the internal culture and operational priorities might not always fully support or empower individual engineers to raise and address fundamental ethical concerns without fear or systemic barriers. This discrepancy can lead to what some critics term "safety washing," where superficial commitments mask deeper, unresolved ethical challenges. The intense pressure of commercial competition and the desire to be first to market with groundbreaking capabilities can, at times, overshadow the imperative for thorough, cautious, and collaborative safety development.

The Role of Human Resources in the AI Era

The convergence of insider warnings and academic research sounds a clear alarm for Human Resources professionals, particularly concerning talent management and workplace culture within the tech sector. The findings highlight several critical HR implications:

• AI Understanding Gap: Myriad studies, including those referenced by HR Dive, suggest a significant "AI understanding gap" among workers. This refers not just to a lack of technical knowledge but also to a limited comprehension of AI’s broader implications for workflows, job roles, and societal impact. This gap can exacerbate fear and uncertainty, making it harder for employees to engage constructively with AI ethics.
• Managers’ Fraught Relationship with AI: Managers often find themselves in a complex position, tasked with implementing AI tools for productivity while also navigating employee concerns and ethical dilemmas. Reports indicate a "fraught relationship" where managers trust AI but actively struggle with its integration and oversight.
• AI Outpacing Human Readiness: A recent report from talent management firm Talogy underscored that "AI itself is outpacing human readiness" for the technology. This means that even as AI capabilities accelerate, human workers’ ability to understand its utility, manage its risks, and adapt to its implications for future workflows is lagging.

Given these challenges, HR departments have a crucial role to play in fostering a responsible and ethical AI development environment. Experts, including SHRM leadership, recommend a multi-pronged approach:

1. Prioritize Human-ness: Even as firms embrace AI, HR must champion and develop uniquely human skills. This includes critical thinking, creativity, emotional intelligence, empathy, and ethical reasoning. Investing in upskilling and reskilling programs that focus on these human-centric attributes can prepare the workforce for a future where collaboration with AI is key, and human oversight is paramount. Fostering a culture where human judgment and values are explicitly valued can counterbalance the drive for pure technological advancement.

2. Develop Concrete AI Ethics Policies: This is perhaps the most direct and actionable step HR can take. A robust AI ethics policy should go beyond general statements and include:

   

   • Clear Channels for Reporting: Establish transparent and accessible mechanisms for employees to report ethical concerns, potential biases, or safety risks associated with AI systems. These channels must guarantee anonymity and protection against retaliation (whistleblower protections).
   • Independent Oversight: Consider establishing an internal AI ethics committee or ombudsman with diverse representation (technical, legal, ethical, HR) to review concerns and provide independent guidance.
   • Regular Ethical Training: Implement mandatory and recurring training for all employees involved in AI development and deployment, focusing on ethical frameworks, bias detection, risk assessment, and responsible innovation.
   • Protocols for Risk Assessment: Integrate ethical risk assessments into the AI development lifecycle, ensuring that potential harms are identified and mitigated at every stage.
   • Psychological Safety: Cultivate a workplace culture where employees feel psychologically safe to voice concerns, challenge assumptions, and engage in open dialogue about the ethical implications of their work without fear of professional repercussions. This directly addresses the "ethical awareness without ethical agency" problem.
   • Transparency and Accountability: Define clear lines of accountability for ethical lapses and commit to transparency regarding AI’s design, testing, and deployment where appropriate.

By taking these steps, HR can help bridge the gap between technical innovation and human values, ensuring that the development of AI is guided by a strong ethical compass and that the concerns of those building it are not only heard but also acted upon.

Regulatory Landscape and Future Outlook

The ethical dilemmas and existential warnings surrounding AI have not gone unnoticed by governments and international bodies. A global push for AI regulation is underway, reflecting a growing consensus that self-regulation by tech companies alone may be insufficient.

• European Union: The EU AI Act, expected to be the world’s first comprehensive AI law, aims to classify AI systems based on their risk level and impose stringent requirements on high-risk applications, including those in critical infrastructure, law enforcement, and employment.
• United States: Executive Orders and legislative proposals are focusing on developing standards for AI safety and security, promoting responsible innovation, and protecting privacy and civil liberties.
• United Nations: The UN and other international organizations are actively discussing global governance frameworks for AI, recognizing the transnational nature of its impact and risks.

However, the challenge of regulating a rapidly evolving technology like AI is immense. Legislation often struggles to keep pace with technological advancements, and a balance must be struck between fostering innovation and ensuring safety. The warnings from engineers like Coxon and the findings from the University of Manchester underscore the critical importance of these regulatory efforts and the need for them to specifically address corporate culture and employee agency in ethical decision-making.

The future of AI development hinges on whether the industry can effectively reconcile its drive for innovation with a profound commitment to safety and ethics. The current situation, characterized by engineers’ internal fears and a perceived lack of agency, suggests that significant systemic changes are required. This includes not only external regulatory pressures but also a fundamental re-evaluation of internal corporate priorities, leadership responsibilities, and the empowerment of those on the front lines of AI creation. Only through a concerted, multi-stakeholder effort can humanity navigate the transformative potential of AI while mitigating its profound risks, ensuring that this powerful technology truly serves the benefit of all.