System Interactions

Provenance creates interactions automatically when alerts fire or recover, and when notifications are delivered. These system interactions use the same schema as user-created interactions and appear in traces, activity search, and dashboards — giving you full observability over automated processes.

System entities

These entities are seeded into every tenant schema and should not be deleted.

Origins

Mnemonic	Title	Purpose
`METRIC_EVAL`	Metric Evaluator	Alert evaluation engine
`NOTIFY`	Notification Engine	Notification delivery tracking

Actions

Mnemonic	Title	Purpose
`fired`	Fired	Alert threshold exceeded (user-configured per alert)
`recovered`	Recovered	Alert threshold no longer exceeded (user-configured per alert)
`notified`	Notified	Notification successfully delivered
`notify-err`	Notify Failed	Notification delivery failed

Resource types

Mnemonic	Title	Purpose
`ALERT`	Alert	Default resource type for alert interactions

Alert interactions

When a metric alert transitions between OK and FIRING states, an interaction is created automatically.

Interaction structure

Field	Value
`resourceId`	The alert's UUID (`alertId`)
`resourceTypeId`	Alert's configured resource type, or system `ALERT` type
`actionId`	`alertActionId` when firing, `recoveryActionId` when recovering
`originId`	System `METRIC_EVAL` origin
`externalReference`	`alert-{alertId}-firing-{timestamp}` or `alert-{alertId}-recovery-{timestamp}`
`uowId`	`null` (alert evaluations are standalone)

Interaction payload (firing)

{
  "alertId": "f3842099-f76a-4ed8-8a27-bce9b6d87d21",
  "alertTitle": "High Repo Push Frequency",
  "metricId": "891bb697-e13c-4083-9b99-85a1fd585dd8",
  "metricName": "Testing Metric Alerts",
  "metricType": "COUNT",
  "threshold": 2,
  "operator": "gte",
  "status": "exceeded",
  "value": 16,
  "dimensions": {},
  "evaluatedAt": "2026-04-09T10:03:35.108Z"
}

Interaction payload (recovery)

{
  "alertId": "f3842099-f76a-4ed8-8a27-bce9b6d87d21",
  "alertTitle": "High Repo Push Frequency",
  "metricId": "891bb697-e13c-4083-9b99-85a1fd585dd8",
  "metricName": "Testing Metric Alerts",
  "metricType": "COUNT",
  "threshold": 2,
  "operator": "gte",
  "status": "recovered",
  "value": 0,
  "dimensions": {},
  "evaluatedAt": "2026-04-09T10:05:06.995Z"
}

Payload field reference

Field	Type	Description
`alertId`	UUID	The alert that fired or recovered
`alertTitle`	string	Human-readable alert name
`metricId`	UUID	The metric being monitored
`metricName`	string	Human-readable metric name
`metricType`	string	Metric aggregation type (`COUNT`, `SUM`, `AVG`, etc.)
`threshold`	number	The configured threshold value
`operator`	string	Comparison operator (`gt`, `gte`, `lt`, `lte`, `eq`, `neq`)
`status`	string	`"exceeded"` when firing, `"recovered"` when back to normal
`value`	number	The actual metric value at evaluation time
`dimensions`	object	Dimension values if the metric uses `groupBy` (e.g. `{"country": "ZA"}`)
`evaluatedAt`	ISO date	When the evaluation occurred

Subscribing to alert interactions

You can create subscriptions that trigger when alerts fire — for example, to send a Slack message or email:

POST /api/subscriptions
{
  "resourceTypeId": "alert-sec-uuid",
  "actionId": "fired-action-uuid",
  "subscriberId": "slack-webhook-uuid",
  "config": {
    "text": "🚨 {{interaction.alertTitle}} — {{interaction.metricName}} is {{interaction.value}} (threshold: {{interaction.threshold}})"
  }
}

Notification interactions

When the notification queue processes a delivery, an interaction is created to track the result. This interaction carries the same uowId as the original interaction that triggered the notification, making the entire flow traceable.

Interaction structure

Field	Value
`resourceId`	Same as the original interaction's `resourceId`
`resourceTypeId`	Same as the original interaction's `resourceTypeId`
`actionId`	System `notified` action (success) or `notify-err` action (failure)
`originId`	System `NOTIFY` origin
`uowId`	Same as the original interaction — links delivery to the triggering event

Interaction payload (success)

{
  "subscriberName": "SendGrid Email",
  "subscriberId": "a1b2c3d4-...",
  "originalInteractionId": "e5f6g7h8-...",
  "adapter": "adapter:email",
  "status": "delivered",
  "statusCode": 202,
  "error": null
}

Interaction payload (failure)

{
  "subscriberName": "Slack Webhook",
  "subscriberId": "a1b2c3d4-...",
  "originalInteractionId": "e5f6g7h8-...",
  "adapter": "adapter:webhook",
  "status": "failed",
  "statusCode": 500,
  "error": "Connection timeout"
}

Payload field reference

Field	Type	Description
`subscriberName`	string	Name of the subscriber that received the notification
`subscriberId`	UUID	Subscriber UUID
`originalInteractionId`	UUID	The interaction that triggered this notification
`adapter`	string	Adapter used for delivery (e.g. `adapter:email`, `adapter:webhook`)
`status`	string	`"delivered"` on success, `"failed"` on error
`statusCode`	number	HTTP status code from the adapter
`error`	string \| null	Error message if delivery failed

Trace example

With notification tracking, a single user action produces a complete trace:

UOW: 550e8400-e29b-41d4-a716-446655440000

1. [web-bff]  order / created    → order-456 created
2. [NOTIFY]   order / notified   → email sent to customer
3. [NOTIFY]   order / notified   → Slack notification to #orders channel

All three interactions share the same uowId, so querying the trace for order-456 shows the full lifecycle — from creation through notification delivery.

Querying system interactions

By origin

GET /api/interactions?origin=METRIC_EVAL
GET /api/interactions?origin=NOTIFY

In activity search

POST /api/activity/search
{
  "filters": {
    "origins": ["METRIC_EVAL", "NOTIFY"],
    "startDate": "2026-04-01"
  }
}

In analytics

System interactions are included in analytics queries. You can build dashboards tracking:

Alert fire/recovery frequency over time
Notification delivery success rate
Average notification latency
Most active alert rules

On this page