Steered User Message Swallowed When Heartbeat Run Produces HEARTBEAT_OK
In steer queue mode, user messages injected into heartbeat runs are lost when the agent produces HEARTBEAT_OK, due to run-level response suppression logic treating the entire run as a heartbeat no-op.
🔍 Symptoms
Primary Manifestation
When a user sends a message on Telegram during an active heartbeat run in steer queue mode, the user receives no response despite the agent generating a correct reply.
Exact Reproduction Sequence
- Configure queue mode as
steer:# config.yaml messages: queue: mode: "steer" heartbeat: interval: 3s - Wait for a heartbeat run to fire (visible in logs as
HEARTBEAT_RUN) - Send a user message within the heartbeat window
- Observe the following in application logs:
[heartbeat] HEARTBEAT_RUN triggered at 2024-01-15T10:30:01.234Z [steer] Injecting user message "What's the weather?" into heartbeat run [agent] Processing run #42 (heartbeat + steered message) [agent] Run #42 produced responses: - HEARTBEAT_OK (heartbeat ack) - TEXT: "The weather is sunny, 72°F" [outbound] Discarding run #42 responses (HEARTBEAT_OK detected) [channel] Delivered: HEARTBEAT_OK ack only (no user message) - User receives nothing—the TEXT response is never sent to Telegram
Diagnostic Evidence
The user-facing response exists in the session transcript with correct generation timestamps but never reaches the channel layer:
$ openclaw session show --id session-abc123
Session Transcript:
[10:30:01.234] ← HEARTBEAT (scheduled)
[10:30:02.456] ← USER: "What's the weather?" (steered into heartbeat run)
[10:30:02.789] → HEARTBEAT_OK
[10:30:03.012] → TEXT: "The weather is sunny, 72°F" ← EXISTS IN TRANSCRIPT
[10:30:03.100] ← Delivered: HEARTBEAT_OK only ← MISSING TEXT
Workaround Verification
Switching to collect mode avoids the issue:
# config.yaml
messages:
queue:
mode: "collect" # ← Workaround: queues as separate turn
In collect mode, the user message queues independently and receives its own run with full delivery.
🧠 Root Cause
Architectural Analysis: Run-Level Response Suppression
The issue stems from a fundamental design assumption in the heartbeat handling logic: a run containing HEARTBEAT_OK is classified as a "heartbeat no-op" and the entire run's outbound delivery is suppressed.
Failure Sequence
- Steer Mode Injection: In
steermode, incoming user messages arriving during an active heartbeat run are injected into that run rather than creating a new one. This is intentional—steerprioritizes latency over isolation. - Multi-Response Run Generation: The agent processes the combined heartbeat + user message context and produces multiple responses in sequence:
Response 1: HEARTBEAT_OK // Agent acknowledges heartbeat with no action Response 2: TEXT // Agent responds to user's actual question - Run Classification Logic: The run classification logic scans responses for any
HEARTBEAT_*marker:// Simplified classification pseudocode function classifyRun(responses): for response in responses: if response.type.startsWith("HEARTBEAT_"): return RUN_TYPE.HEARTBEAT_NOOP // ← Triggers suppression return RUN_TYPE.NORMAL - Premature Suppression: Because
HEARTBEAT_OKis present, the entire run is marked asHEARTBEAT_NOOP, triggering discard logic in the outbound delivery layer:// Outbound delivery pseudocode function deliverResponses(run): if run.classification === RUN_TYPE.HEARTBEAT_NOOP: return // ← Entire delivery skipped, including user response deliverToChannel(run.responses) - User Response Lost: The
TEXTresponse (which is valid user-facing content) is discarded because it shares a run withHEARTBEAT_OK.
Code Location Reference
| Component | File | Problem |
|---|---|---|
| Steer Injection | src/queue/steer.ts | Injects user messages into active heartbeat runs |
| Run Classification | src/runs/classifier.ts | Classifies entire run based on presence of HEARTBEAT_* |
| Outbound Delivery | src/outbound/delivery.ts | Skips delivery for HEARTBEAT_NOOP runs |
Why Collect Mode Works
In collect mode, the user message is queued as a separate followup turn. It receives its own independent run containing only user-facing responses—no HEARTBEAT_* markers—so the classification logic correctly identifies it as NORMAL and delivers it.
🛠️ Step-by-Step Fix
Option 1: Filter Responses Before Classification (Recommended)
Modify the run classification logic to ignore HEARTBEAT_OK responses when determining run type, allowing user-facing responses in the same run to be delivered.
Before
// src/runs/classifier.ts
function classifyRun(responses: Response[]): RunType {
for (const response of responses) {
if (response.type.startsWith("HEARTBEAT_")) {
return RUN_TYPE.HEARTBEAT_NOOP;
}
}
return RUN_TYPE.NORMAL;
}
After
// src/runs/classifier.ts
function classifyRun(responses: Response[]): RunType {
const hasHeartbeatAction = responses.some(
r => r.type.startsWith("HEARTBEAT_") && r.type !== "HEARTBEAT_OK"
);
const hasUserFacingResponse = responses.some(
r => !r.type.startsWith("HEARTBEAT_") && r.type !== "CONTROL"
);
if (hasHeartbeatAction && !hasUserFacingResponse) {
return RUN_TYPE.HEARTBEAT_NOOP;
}
return RUN_TYPE.NORMAL;
}
Option 2: Modify Outbound Delivery to Extract User Responses
If classification cannot be changed, modify the outbound delivery layer to extract and deliver non-heartbeat responses even from HEARTBEAT_NOOP runs.
// src/outbound/delivery.ts
function deliverResponses(run: Run): void {
if (run.classification !== RUN_TYPE.HEARTBEAT_NOOP) {
deliverToChannel(run.responses);
return;
}
// Extract user-facing responses from heartbeat no-op runs
const userResponses = run.responses.filter(
r => !r.type.startsWith("HEARTBEAT_") && r.type !== "CONTROL"
);
if (userResponses.length > 0) {
deliverToChannel(userResponses);
}
}
Option 3: Configuration-Based Workaround
If code changes are not immediately possible, configure the system to avoid the condition:
# config.yaml
messages:
queue:
mode: "collect" # Avoids steered injection into heartbeat runs
heartbeat:
interval: 60s # Reduces chance of user message arriving during heartbeat
# Or disable heartbeat during active conversations:
pause_on_active: true
Deployment Steps
- Backup Configuration
cp config.yaml config.yaml.backup - Apply Fix
# If using Option 1 or 2: vim src/runs/classifier.ts # or src/outbound/delivery.ts npm run build - Restart Service
docker-compose down && docker-compose up -d # Or for systemd: sudo systemctl restart openclaw - Verify Configuration
openclaw config show | grep -A5 "queue:" openclaw status
🧪 Verification
Test Case 1: Steered Message Delivery
Purpose: Verify that user messages injected into heartbeat runs are delivered.
# 1. Configure steer mode with short heartbeat
openclaw config set messages.queue.mode steer
openclaw config set heartbeat.interval 3s
openclaw restart
# 2. Monitor logs in one terminal
openclaw logs --follow | grep -E "(HEARTBEAT|steer|deliver)"
# 3. Send user message during heartbeat window
# Wait for log line: [heartbeat] HEARTBEAT_RUN triggered
# Then immediately send: "Testing steer delivery"
# 4. Verify delivery
# Expected: User receives response on Telegram
# Expected log: [outbound] Delivered: TEXT response to user
Success Criteria:
- User receives the response on Telegram
- Log shows
Delivered: TEXT(notDiscarded) - Session transcript shows both
HEARTBEAT_OKandTEXTresponses
Test Case 2: Pure Heartbeat Still Suppressed
Purpose: Ensure that genuine HEARTBEAT_NOOP runs (without user messages) are still suppressed.
# 1. Wait for heartbeat to fire with NO user interaction
# Monitor logs for a clean heartbeat run
# 2. Expected log behavior
[heartbeat] HEARTBEAT_RUN triggered
[heartbeat] HEARTBEAT_OK generated (no user message)
[outbound] Discarding run (HEARTBEAT_NOOP)
# 3. Verify: User should NOT receive any notification from this run
Success Criteria:
- Heartbeat-only runs produce no user notification
- Log shows
Discardingfor pure heartbeat runs
Test Case 3: Session Transcript Validation
# Get session ID from a recent conversation
openclaw session list --limit 5
Show detailed transcript
openclaw session show –id <SESSION_ID> –verbose
Verify structure contains both response types
Expected output should show:
[timestamp] → HEARTBEAT_OK
[timestamp] → TEXT: “response content”
[timestamp] ← Delivered: HEARTBEAT_OK, TEXT ← Both delivered
Regression Tests
# Run existing test suite
npm test -- --grep "heartbeat"
Run steer mode specific tests
npm test – –grep “steer”
Expected: All tests pass including:
- steer mode message injection
- heartbeat response classification
- outbound delivery filtering
Exit Code Verification
# After fix deployment
openclaw health check; echo "Exit code: $?"
# Expected: 0 (healthy)
Check service logs
docker-compose logs openclaw 2>&1 | tail -20
Expected: No ERROR level entries related to delivery
⚠️ Common Pitfalls
Environment-Specific Traps
| Environment | Pitfall | Mitigation |
|---|---|---|
| Docker | Container clock drift can cause heartbeat timing to become unreliable, exacerbating the race condition between heartbeat runs and user message injection | Ensure NTP synchronization: docker run --cap-add=SYS_TIME openclaw:latest |
| VPS/Cloud | Network latency between Telegram API and server can mask the issue's timing sensitivity | Test with local bot webhook instead of long-polling to reduce variables |
| macOS (dev) | Heartbeat timers may fire less reliably due to system sleep/hibernate states | Disable system sleep during testing: caffeinate -s |
| Windows (dev) | Line ending differences (\r\n vs \n) in config files can cause parsing issues | Use Unix line endings: set FILE_OPTS=-o nowrap in editor |
Timing Sensitivity
The bug is highly timing-dependent. The race window exists between:
- Heartbeat run starts (
HEARTBEAT_RUNlogged) - Heartbeat ack generated (
HEARTBEAT_OKlogged) - User message arrives and is injected
- User response generated
- Run classification occurs
Recommendation: Use a heartbeat interval of 3s or 5s for reliable reproduction testing. Intervals below 1s may make the race condition too tight to reliably trigger.
Configuration Mistakes
- Confusing
steerwithforce:# Wrong - force mode ignores queue entirely messages.queue.mode: "force"Correct - steer mode uses intelligent injection
messages.queue.mode: “steer”
- Heartbeat interval too long masking the issue:
# This interval may never overlap with user messages heartbeat.interval: 300s # 5 minutes - unlikely to catch user messagesBetter for testing
heartbeat.interval: 3s
- Missing channel-specific queue settings:
# Some channels may override global queue mode telegram: queue_mode: "collect" # ← May override steer settingUse channel-agnostic config
messages.queue.mode: “steer”
Misdiagnosis: Confusing Symptoms
These issues may appear similar but have different root causes:
- Missing response due to rate limiting: User receives nothing, but log shows
Rate limitedinstead ofDiscarded - Missing response due to agent silence: Agent never generates a response, log shows no
TEXTin responses array - Missing response due to Telegram delivery failure: Log shows
Deliveredbut user doesn't receive; this is a Telegram/API issue
Key differentiator: This bug shows Discarding run in logs with both HEARTBEAT_OK and TEXT present in the transcript.
Partial Fix Pitfalls
If applying Option 2 (outbound extraction), ensure that:
- Control responses (e.g.,
HANDOVER,TRANSFER) are also filtered appropriately - Analytics/tracking calls still receive the full response array
- Webhook payloads reflect the actual delivered content, not the original run
🔗 Related Errors
HEARTBEAT_TIMEOUT— Heartbeat run exceeded maximum duration; may trigger session cleanup if consecutive timeouts exceed threshold[heartbeat] Run exceeded 30s timeout, forcing HEARTBEAT_TIMEOUTHEARTBEAT_SKIP— Heartbeat run skipped entirely due to active conversation; configurationheartbeat.pause_on_active: true[heartbeat] Skipping HEARTBEAT_RUN - active conversation detectedSTEER_INJECT_FAILED— Steer mode failed to inject message into active run; falls back to queuing[steer] Failed to inject message: no active heartbeat run in progress [steer] Falling back to queue for message "..."DELIVERY_FILTERED— Response was intentionally filtered by channel policy (spam detection, content filtering)[outbound] DELIVERY_FILTERED: response contains blocked keywordRATE_LIMIT_EXCEEDED— Telegram API rate limit hit; responses queued for retry[telegram] Rate limit exceeded (30/30), queuing response for retry in 60sQUEUE_MODE_CONFLICT— Conflicting queue mode settings between global and channel-specific config[config] Warning: telegram.queue_mode conflicts with messages.queue.mode
Historical Context
This issue represents a regression introduced when the run classification system was unified for efficiency. Previously, each response type had independent delivery logic, but the consolidation into run-level classification introduced the suppression behavior for multi-response runs containing HEARTBEAT_OK.
See Also
- OpenClaw Queue Modes Documentation — Detailed explanation of
steervscollectvsforce - Heartbeat System Architecture — Technical deep-dive on heartbeat scheduling and response handling
- Telegram Channel Adapter — Channel-specific delivery considerations