Nonconformance

Purpose

This procedure establishes the process for identifying, documenting, investigating, and addressing nonconformities within Rygen’s AI Management System (AIMS) to ensure that we systematically identify, analyze, and resolve issues within our AI systems.

Scope

This procedure applies to all nonconformities related to:

• AI systems within AIMS scope (X1, Corsair, internal AI tools)
• AIMS processes and procedures
• AI governance and compliance requirements
• AI system performance, security, or ethical issues

Definitions

Nonconformity: Failure to fulfill a requirement of the AIMS, including:

• AI system failures or performance degradation
• Policy violations or process deviations
• Compliance gaps or regulatory issues
• Data governance or security breaches
• Stakeholder complaints related to AI systems

Corrective Action: Action taken to eliminate the cause of a nonconformity and prevent recurrence.

Nonconformity Identification and Immediate Response

When any system developed by Rygen is identified to be non-conformant, it must be identified, contained, and documented.

Identification Sources

Nonconformities may be identified through:

• AI system monitoring and alerts
• Internal audits and reviews
• Stakeholder feedback or complaints
• Risk assessment activities
• Incident reports
• Regular AIMS performance evaluation

Immediate Response

When a nonconformity occurs, the person identifying it shall:

1. Take immediate action to control and correct:

   • Stop or contain the nonconforming activity if safe to do so
   • Implement immediate safeguards to prevent further impact
   • Document the immediate actions taken

2. Deal with consequences:

   • Assess and mitigate any impacts on stakeholders
   • Communicate with affected parties as required
   • Implement temporary measures if needed

3. Create Jira Issue:

   • Create a Bug ticket in Jira with appropriate priority:
     • Highest: Critical AI system failure, security breach, or regulatory violation
     • High: Significant impact on AI system performance or stakeholder trust
     • Medium: Moderate impact requiring prompt attention
     • Low: Minor issues with limited impact
     • Lowest: Documentation or process improvements
   • Include detailed description of the nonconformity
   • Assign to appropriate team lead or Principal AI Engineer

Root Cause Analysis and Corrective Action Planning

After a non-conformance is identified and contained, it must be analyzed and corrected.

Evaluation of Need for Action

The Principal AI Engineer (or designee) shall evaluate whether corrective action is needed by:

1. Reviewing the nonconformity:

   • Analyze the nature and scope of the issue
   • Assess potential for recurrence
   • Consider regulatory or compliance implications

2. Determining root causes:

   • Conduct root cause analysis using Five Whys methodology
   • Consider systemic factors and process gaps
   • Identify contributing factors

3. Assessing similar nonconformities:

   • Search Jira for related issues
   • Review patterns or trends
   • Identify systemic issues

Root Cause Analysis Methods

Use appropriate methods based on nonconformity complexity:

• Five Whys Analysis: For straightforward issues
• Fishbone Diagram: For complex multi-factor issues
• Timeline Analysis: For incident-based nonconformities

Implementation and Follow-up

Corrective actions must be implemented and evaluated to ensure that they successfully addressed and corrected the root cause of the nonconformity

Implement Corrective Actions

Based on the analysis, implement appropriate actions:

• Process improvements or changes
• Training or competence enhancement
• System modifications or upgrades
• Policy or procedure updates
• Additional controls or monitoring

Review Effectiveness

Monitor and review the effectiveness of corrective actions through:

• Follow-up audits or assessments
• Performance monitoring
• Stakeholder feedback
• Trend analysis of similar issues

AIMS Updates

Make necessary changes to the AIMS including:

• Risk register updates
• Process procedure revisions
• Training material updates
• Policy modifications

Documentation Requirements

Non-Conformance Report (Jira)

All nonconformities shall be tracked in Jira with:

• Clear description of the nonconformity
• Priority level assignment
• Root cause analysis results
• Corrective actions planned and implemented
• Effectiveness review results
• Status updates and closure documentation

Corrective Action Report

For significant nonconformities (High or Highest priority), create a Corrective Action Report (CAR-xxx) in the aims-governance repository. This document serves as the single comprehensive record of the nonconformity, consolidating all documentation of the issue, investigation, and resolution. The CAR should contain:

• Executive Summary: Brief overview of the nonconformity
• Postmortem Report: Detailed analysis including:
  • Leadup: Sequence of events leading to nonconformity
  • Fault: How the failure occurred
  • Impact: Effect on stakeholders and operations
  • Detection: How and when discovered
  • Response: Actions taken to address the issue
  • Recovery: Resolution steps and timeline
• Timeline: Chronological sequence of events
• Five Whys Root Cause Analysis: Systematic root cause identification
• Blameless Root Cause: Final root cause without blame assignment
• Lessons Learned: Key insights and improvements identified
• Follow-up Corrective Actions: Jira issues created to prevent recurrence

Roles and Responsibilities

All Personnel

• Report nonconformities promptly
• Take immediate corrective action within their authority
• Cooperate with investigations and root cause analysis

Team Leads

• Ensure team members understand nonconformity reporting process
• Support immediate response and containment actions
• Participate in root cause analysis for their domain

Principal AI Engineer

• Oversee nonconformity evaluation and response
• Approve corrective action plans
• Ensure documentation completeness
• Report significant nonconformities to CTO and AI Governance Committee

AI Governance Committee

• Review significant nonconformities (High/Highest priority)
• Approve major AIMS changes resulting from corrective actions
• Monitor trends and systemic issues

Integration with AIMS Processes

Risk Management

• Update risk assessments based on nonconformity findings
• Add new risks identified through root cause analysis
• Modify risk treatments as needed

Monitoring and Measurement

• Include nonconformity metrics in AIMS performance reporting
• Track corrective action effectiveness
• Monitor trends for continual improvement

Management Review

• Report nonconformity trends and corrective actions to CTO
• Include in quarterly governance committee reviews
• Use findings to inform AIMS improvement priorities

Records and Evidence

Required Documentation

Maintain documented evidence of:

• Nature of nonconformities and subsequent actions taken:
  • Jira issue descriptions and updates
  • Post-mortem reports for significant issues
• Results of corrective actions:
  • Effectiveness reviews documented in Jira
  • Follow-up verification records

Record Retention

• Jira issues: Maintain for 3 years after closure
• Corrective Action Reports: Maintain permanently in aims-governance repository
• Supporting documentation: 3 years minimum

Revision History