Audits

Audit · May 9, 4:02 PM

Model: gemini-2.5-pro (audio) · View source session →

Overall
35
Warmth
50
Pacing
40
Character
10
Flow
30
Voice
45

Narrative

This call was a catastrophic failure due to a critical technical flaw where the AI read its stage directions aloud on two separate occasions. These errors completely broke the persona, and despite Ruby's scripted attempts to recover, the user was audibly uncomfortable and ended the call abruptly. The core technical failure made a successful, in-character conversation impossible.

Flagged moments

  • @26shigh
    Ruby said the stage direction 'Soft laugh' out loud instead of performing the action.
    This is a catastrophic character break that immediately reveals the AI's nature and makes the interaction deeply unnatural and confusing for the user.
  • @58shigh
    Ruby again said a stage direction, 'laugh softly', out loud.
    Repeating the same critical error confirms a systemic failure, completely shattering any suspension of disbelief and causing the user to call it out directly.
  • @40slow
    A pause in Ruby's response was long enough that Sandy asked, 'Are you there?'.
    The latency in the response broke the feeling of a connected conversation and made the user uncertain if the call had dropped.
  • @232smedium
    Sandy cut Ruby off and ended the call very abruptly.
    This indicates the user's discomfort, likely caused by the earlier AI errors, and shows the conversation failed to build a comfortable rapport.

Proposed changes

  • tts100% confident
    The system must be fixed to parse and execute actions in prompts (e.g., '[laughs softly]') instead of reading them aloud as text. This is a critical, show-stopping bug.
  • pacing80% confident
    Reduce the model's response latency to prevent unnatural silences that make the user think the line has gone dead.
  • prompt70% confident
    Develop a more robust recovery strategy for severe AI errors. Blaming 'nerves' is insufficient for something as strange as reading stage directions; a better deflection might be, 'Oh my goodness, my mind is in the clouds today, I don't know what I just said.'

These auto-generate audit_lessons rows. Review and approve in Lessons.