November 17, 2026
the-human-element-in-ai-interviewing-navigating-nuance-and-trust

The pursuit of efficient and equitable hiring practices has led to the development of sophisticated AI tools designed to streamline the recruitment process. At the forefront of this innovation is Eightfold.ai’s AI Interviewer, a platform aiming to extract meaningful insights from candidate interactions. However, the creation of AI that can discern genuine capability from polished presentation, and do so without introducing its own biases, is a complex challenge. This article delves into the intricacies of developing such technology, highlighting the ongoing efforts of Deepali Saxena, Senior Director of Product, and Hussain Kazmi, Principal Solutions Consultant at Eightfold, as they navigate the delicate balance between artificial intelligence and human judgment.

The fundamental challenge in AI-driven interviewing lies in distinguishing between a candidate’s ability to articulate their skills and their actual demonstrated competence. A typical 45-minute interview can yield a wealth of information, but not all of it is directly relevant to a candidate’s potential to succeed in a specific role. Saxena and Kazmi are at the heart of developing AI that can meticulously sift through this data, identifying salient signals without making premature conclusions about their meaning. Their work underscores a critical design philosophy: the AI’s limitations are as crucial as its capabilities.

Beyond Eloquence: Identifying True Capability

The AI Interviewer is not designed to identify candidates who excel at performing in an interview setting. Instead, its core objective is to surface concrete evidence of a candidate’s potential to thrive in the role. "That means looking for meaningful signals across things like role knowledge, problem-solving, communication, judgment, and technical ability," explains Saxena. She elaborates that this evidence is not confined to verbal responses; it can manifest in code, case studies, problem-solving walkthroughs, or written submissions.

The inherent difficulty, Saxena notes, is that strong evidence is often not immediately apparent. "Some candidates are polished but give you very little evidence. Others show strong capability in less obvious ways," she states. This disparity necessitates an AI that can differentiate between impressive rhetoric and tangible proof of required skills. Context is paramount; an answer that is exceptional for one role might be insufficient for another, depending on the job’s demands and seniority.

A pivotal moment for the development team occurred when they observed a candidate deliver a detailed, articulate response that ultimately failed to demonstrate the targeted skill. This scenario prompted a crucial debate: should the AI infer potential capabilities, or should it strictly evaluate what has been explicitly demonstrated? The team opted for the latter, a decision that established "indeterminate" as a valid outcome. When sufficient evidence is lacking for a particular skill, the system now explicitly states this, rather than resorting to guesswork. This principle, "Don’t confuse a good-sounding answer with demonstrated capability, and never invent signals where there isn’t any," continues to guide their product development decisions.

Humans Behind Our Innovations: Teaching AI When to Say “I Don’t Know”.

The Art of Observation Without Interpretation

A core tenet of the AI Interviewer’s design is the distinction between detecting evidence and interpreting its meaning. While an AI could technically register nuances like nervousness, vocal tone, facial expressions, accent, or changes in speaking pace, Saxena’s team made a deliberate choice to exclude these factors from the evaluation process. "The AI can observe the interaction, but it should only make recommendations based on job-relevant evidence in the candidate’s responses," Saxena emphasizes. "It shouldn’t try to determine someone’s personality, emotions, intent, or potential from things that aren’t relevant to the actual evidence being evaluated."

This commitment to objectivity extends to the user experience. Across all Eightfold products, AI-powered features are clearly labeled with a recognizable icon, visible to recruiters, hiring managers, and candidates globally. This transparency ensures that users understand when and how AI is being utilized.

The evaluation framework is built upon the most critical requirements for success in a given role. These requirements are ranked, and evaluation criteria are subsequently developed. If a hiring team’s feedback form omits questions designed to assess a crucial skill, the AI Interviewer can suggest relevant inquiries, ensuring that all candidates have a consistent opportunity to showcase that particular competency. This proactive approach aims to mitigate potential oversights in the human-led design of interview processes.

The Unseen Labor of AI Development

For Saxena, the most significant challenge in perfecting the AI Interviewer lies in establishing appropriate "guardrails" – defining what the AI should not do. "Customers rarely see how much work goes into deciding what the AI should not do," she remarks. For every potential behavior the AI is programmed to exhibit, the team engages in rigorous debate, posing fundamental questions: "Should the AI try to connect the dots and infer what the candidate might be capable of, or should it evaluate only what the candidate has actually demonstrated?" This iterative process of questioning and refinement is crucial for building a trustworthy and effective AI tool.

Kazmi echoes this sentiment from the customer’s perspective. He notes that external perceptions often assume the goal is to maximize data collection. "In reality, it’s about asking the right questions in the right way. The experience has to be useful for the employer, but also human and approachable for the candidate," he explains. The emphasis is on quality and relevance over sheer quantity of interaction.

A Customer Query That Redefined the Conversation

Kazmi points to customer feedback as a catalyst for significant product evolution. He recalls a particular exchange where a client questioned the potential for AI interviewers to introduce bias. "If we use an AI interviewer, how do we make sure we’re not introducing bias by letting hiring teams react to how a candidate looks or how they engage with an AI agent?" the client asked. This question probed the risk of human reviewers allowing personal biases—whether conscious or unconscious—to influence their perception of a candidate based on visual cues from interview recordings, rather than solely on the candidate’s qualifications and responses.

Humans Behind Our Innovations: Teaching AI When to Say “I Don’t Know”.

Kazmi’s initial response focused on the product’s design: AI Interviewer is engineered to assess candidates based on role-relevant responses, skills, and experience, explicitly excluding appearance or personal characteristics. The platform also provides hiring teams with structured feedback and a clear rationale for its assessments. However, the customer’s question lingered, highlighting that the challenge extended beyond the AI model itself to the human element of its application. Consequently, Kazmi began advising hiring teams to consider reviewing AI interview results with video turned off when appropriate, thereby reinforcing a focus on the substance of the candidate’s dialogue.

"It was a good reminder that responsible AI isn’t only about how the model works," Kazmi reflects. "It’s also about how customers design the human process around it." This realization underscores the collaborative nature of responsible AI implementation, where technological capabilities must be integrated thoughtfully into existing human workflows.

Building Trust Through Transparency and Human Oversight

When introducing AI Interviewer to potential clients, Kazmi observes a consistent desire for two key elements before trust can be established. The first is a comprehensive understanding of what the AI is evaluating. "A vague assessment is not enough," he states. Clients need to see the transcript, the summary, the evaluation criteria, and any flagged moments to comprehend why the AI has highlighted certain aspects and to be able to internally articulate that decision. This transparency is vital for fostering confidence in the AI’s outputs.

The second crucial element is reassurance that human judgment remains paramount. While AI Interviewer assists in structuring interviews, identifying patterns, and flagging potential concerns, the ultimate decision-making authority rests with the recruiter or hiring team. They review the context, can override AI recommendations when necessary, and make the final hiring decision. "The value isn’t replacing human judgment," Kazmi asserts. "It’s giving hiring teams better, more consistent information so they can make more confident decisions." This symbiotic relationship ensures that AI serves as a powerful augmentation tool, rather than a replacement for human expertise.

Customer feedback has also been instrumental in refining the product’s capabilities. Several clients inquired about the AI’s ability to update a candidate’s profile with new information discovered during an interview, such as a candidate mentioning experience with a technology not listed on their resume. This request shifted the product’s perceived value from mere interview automation to enhanced candidate intelligence. Currently, interview summaries, feedback, and transcripts are integrated into candidate profiles, providing recruiters with a more holistic view and empowering them to manage this information effectively.

A Synergistic Relationship Fuels Innovation

The close collaboration between Saxena and Kazmi has significantly influenced their respective approaches to their work. For Saxena, customer inquiries have driven the team to enhance the product’s explainability, making it more accessible and understandable. "Customers want to understand how the AI works, not just rely on an interview score," she notes.

Humans Behind Our Innovations: Teaching AI When to Say “I Don’t Know”.

Kazmi, benefiting from his proximity to product development, has learned to prioritize practical impact over purely technological advancements. "I personally get excited about the latest technology, but customers are thinking about the practical impact. Will this help my team move faster? Will it improve candidate experience? Will it help us make better decisions?" he asks. His focus has shifted to understanding customer challenges and articulating how the technology addresses those specific needs.

Both Saxena and Kazmi agree on the critical role of human judgment in defining the boundaries of AI. "In deciding where AI should have boundaries, not just where it’s technically capable," Saxena states. Kazmi concurs, emphasizing that "People need to define how it should behave, what it should and shouldn’t do, and how its recommendations are explained." This shared understanding highlights the ethical imperative to guide AI development with human values and principles.

Charting the Future of AI in Recruitment

Looking ahead, Saxena and Kazmi acknowledge that the work is far from complete. Saxena is focused on enhancing the naturalness of interviews, aiming to create a comfortable experience for candidates while continuously refining the scoring mechanisms against evaluation criteria. A persistent question remains: "How do you make the conversation more natural and more human without bringing in human bias?"

Kazmi is exploring related avenues, recognizing that the AI Interviewer often represents a candidate’s initial interaction with an organization. He believes the experience should not feel generic but should reflect the company’s brand. "The question I don’t think anyone has fully solved is how to give an AI agent enough personality to feel like your brand, while keeping it consistent, responsible, and fair," he muses.

For Saxena, the complexity of these challenges is precisely what makes the endeavor worthwhile. "It sits at the intersection of AI, human judgment, and something that has a real impact on people’s careers," she concludes. "By continuing to improve, we can help recruiters evaluate more candidates thoughtfully, while giving every candidate an opportunity to show what they can do." This commitment to enhancing both efficiency and fairness in the hiring process underscores Eightfold.ai’s vision of "Talent intelligence, human-led." The ongoing development of AI Interviewer exemplifies a forward-thinking approach to leveraging technology for a more equitable and insightful recruitment landscape.

Eightfold.ai is actively seeking individuals passionate about driving such innovations. Prospective candidates interested in contributing to the future of talent intelligence are encouraged to explore the company’s open roles.