Skip to Content
ReferenceEvaluationsEvent Types Reference

Event Types Reference

ARK generates Kubernetes events throughout the AI agent execution lifecycle. These events provide visibility into system behavior and enable detailed evaluation of performance and reliability.

Event Naming Convention

Events follow a systematic naming pattern:

  • Start Events: {Operation}Start - Marks beginning of operation
  • Complete Events: {Operation}Complete - Marks successful completion
  • Error Events: {Operation}Error - Marks operation failure

Query Resolution Events

ResolveStart

Emitted when query resolution begins.

Metadata:

  • queryId: Unique query identifier
  • sessionId: Session identifier
  • queryName: Name of the query resource
  • timestamp: Event timestamp

Example:

{
  "reason": "ResolveStart",
  "message": {
    "Metadata": {
      "queryId": "query-123",
      "sessionId": "session-456",
      "queryName": "example-query"
    }
  }
}

ResolveComplete

Emitted when query resolution completes successfully.

Metadata:

  • queryId: Unique query identifier
  • sessionId: Session identifier
  • duration: Total resolution time in seconds
  • targetsResolved: Number of targets resolved

ResolveError

Emitted when query resolution fails.

Metadata:

  • queryId: Unique query identifier
  • sessionId: Session identifier
  • error: Error message
  • errorCode: Error classification

Agent Execution Events

AgentExecutionStart

Emitted when an agent begins execution.

Metadata:

  • agentName: Name of the agent
  • agentId: Unique agent instance ID
  • queryId: Parent query ID
  • sessionId: Session identifier
  • modelName: LLM model being used

AgentExecutionComplete

Emitted when agent execution completes successfully.

Metadata:

  • agentName: Name of the agent
  • agentId: Agent instance ID
  • duration: Execution time in seconds
  • toolsUsed: Number of tools called
  • llmCalls: Number of LLM interactions

AgentExecutionError

Emitted when agent execution fails.

Metadata:

  • agentName: Name of the agent
  • error: Error message
  • errorType: Error classification
  • retryCount: Number of retry attempts

Tool Call Events

ToolCallStart

Emitted when a tool is invoked.

Metadata:

  • toolName: Name of the tool
  • toolId: Unique tool call ID
  • agentName: Calling agent
  • parameters: Tool parameters (sanitized)
  • timestamp: Call timestamp

ToolCallComplete

Emitted when tool execution completes successfully.

Metadata:

  • toolName: Name of the tool
  • toolId: Tool call ID
  • duration: Execution time in seconds
  • resultSize: Size of result data
  • success: Boolean success indicator

ToolCallError

Emitted when tool execution fails.

Metadata:

  • toolName: Name of the tool
  • error: Error message
  • errorType: Error classification
  • retryable: Whether retry is possible

Team Coordination Events

TeamExecutionStart

Emitted when team execution begins.

Metadata:

  • teamName: Name of the team
  • teamId: Unique team instance ID
  • memberCount: Number of team members
  • executionMode: ‘sequential’ or ‘parallel’

TeamExecutionComplete

Emitted when team execution completes.

Metadata:

  • teamName: Name of the team
  • duration: Total execution time
  • membersExecuted: Number of members that executed
  • successRate: Team success rate

TeamExecutionError

Emitted when team execution fails.

Metadata:

  • teamName: Name of the team
  • error: Error message
  • failedMembers: List of failed member agents

TeamMemberStart

Emitted when a team member begins execution.

Metadata:

  • teamName: Parent team name
  • memberName: Member agent name
  • memberRole: Role in team
  • executionOrder: Order in sequence

TeamMemberComplete

Emitted when team member completes execution.

Metadata:

  • teamName: Parent team name
  • memberName: Member agent name
  • duration: Member execution time
  • handoffData: Data passed to next member

LLM Interaction Events

LLMCallStart

Emitted when LLM API call begins.

Metadata:

  • modelName: LLM model name
  • callId: Unique call ID
  • agentName: Calling agent
  • promptTokens: Number of prompt tokens

LLMCallComplete

Emitted when LLM call completes.

Metadata:

  • modelName: LLM model name
  • callId: Call ID
  • duration: Call duration in seconds
  • completionTokens: Number of completion tokens
  • totalTokens: Total tokens used

LLMCallError

Emitted when LLM call fails.

Metadata:

  • modelName: LLM model name
  • error: Error message
  • errorType: ‘rate_limit’, ‘timeout’, ‘api_error’
  • retryAfter: Seconds before retry (if applicable)

Memory Events

MemoryStoreStart

Emitted when memory storage begins.

Metadata:

  • memoryType: Type of memory (short_term, long_term)
  • agentName: Agent storing memory
  • dataSize: Size of data being stored

MemoryStoreComplete

Emitted when memory storage completes.

Metadata:

  • memoryId: Stored memory ID
  • duration: Storage duration
  • indexed: Whether memory was indexed

MemoryRetrieveStart

Emitted when memory retrieval begins.

Metadata:

  • queryVector: Query vector dimension
  • topK: Number of results requested
  • agentName: Requesting agent

MemoryRetrieveComplete

Emitted when memory retrieval completes.

Metadata:

  • resultsFound: Number of results
  • duration: Retrieval duration
  • relevanceScore: Average relevance score

Special Events

A2ACallStart

Emitted when agent-to-agent communication begins.

Metadata:

  • callerAgent: Calling agent name
  • targetAgent: Target agent name
  • messageType: Type of communication
  • priority: Call priority

A2ACallComplete

Emitted when agent-to-agent communication completes.

Metadata:

  • callerAgent: Calling agent name
  • targetAgent: Target agent name
  • duration: Communication duration
  • responseSize: Size of response

ValidationError

Emitted when input validation fails.

Metadata:

  • component: Component that failed validation
  • validationType: Type of validation
  • errors: List of validation errors

RateLimitExceeded

Emitted when rate limits are exceeded.

Metadata:

  • component: Rate-limited component
  • limit: Rate limit value
  • resetTime: When limit resets
  • queueDepth: Current queue depth

Event Metadata Structure

All events include standard Kubernetes event fields plus custom metadata:

interface EventMetadata {
  // Identity
  queryId?: string;
  sessionId?: string;
  agentName?: string;
  teamName?: string;
  toolName?: string;
  modelName?: string;
  
  // Timing
  timestamp?: string;
  duration?: number;
  startTime?: string;
  endTime?: string;
  
  // Status
  success?: boolean;
  error?: string;
  errorType?: string;
  retryCount?: number;
  
  // Metrics
  tokenCount?: number;
  resultSize?: number;
  memoryUsage?: number;
  
  // Relationships
  parentId?: string;
  correlationId?: string;
  causationId?: string;
}

Using Events in Evaluations

Events can be analyzed using semantic expressions:

# Check for specific event
expression: "events.exists(e, e.reason == 'ToolCallComplete')"
 
# Or using semantic helpers
expression: "tool.was_called()"
 
# Count events
expression: "events.filter(e, e.reason == 'AgentExecutionStart').size() >= 2"
 
# Or using semantic helpers
expression: "agent.get_execution_count() >= 2"

Event Filtering

Events can be filtered by:

  • Query ID: Events for specific query
  • Session ID: Events within a session
  • Time Range: Events within time window
  • Component: Events from specific component
  • Status: Success/failure events

Best Practices

  1. Use semantic helpers instead of raw event matching when possible
  2. Check event metadata for detailed context
  3. Consider event ordering for sequence validation
  4. Monitor error events for failure patterns
  5. Aggregate metrics from complete events
Last updated on
Event Semantic ExpressionsQuery Execution Flow