Memory Search Remote Embeddings Fail with ENOTFOUND When Environment Proxy is Configured
The withRemoteHttpResponse() function bypasses TRUSTED_ENV_PROXY mode, causing local DNS pre-resolution failures in proxy environments like Clash TUN fake-IP mode or corporate proxies.
๐ Symptoms
Error Manifestation
When executing memory search commands with a configured HTTP proxy, the embeddings functionality becomes unavailable:
$ openclaw memory status --deep --agent main
Embeddings: unavailable
Error: getaddrinfo ENOTFOUND api.ohmygpt.com
$ openclaw memory search --agent main --query "test"
Error: getaddrinfo ENOTFOUND api.ohmygpt.com
Embeddings: unavailableTechnical Behavior
- The
memory statuscommand reportsEmbeddings: unavailableinstead of showing the configured remote embedding provider. - The
memory searchcommand fails immediately with the DNS resolution error. - OpenAI SDK calls to the same endpoint on the same machine succeed, confirming the proxy infrastructure is functional.
- The error occurs before any HTTP request is attempted โ it fails at the DNS lookup stage.
Reproduction Conditions
- Required: HTTP/HTTPS proxy configured via environment variables (
HTTPS_PROXY,HTTP_PROXY, orALL_PROXY) - Required: Proxy setup where local DNS cannot resolve the embedding provider hostname (e.g., Clash TUN fake-IP mode, corporate DNS-over-proxy)
- Required: Remote embedding provider configured in the agent
Environment Configuration That Triggers the Bug
# Environment variables
HTTPS_PROXY=http://127.0.0.1:7890
HTTP_PROXY=http://127.0.0.1:7890
# Agent config (openclaw agent config)
models.providers.openai.baseUrl = "https://api.ohmygpt.com/v1"
memorySearch.remote.model = "text-embedding-3-small"๐ง Root Cause
Call Chain Analysis
The failure occurs through this exact sequence:
withRemoteHttpResponse(params)
โ fetchWithSsrFGuard({ url, init, policy })
โ resolveGuardedFetchMode(params)
โ returns STRICT (no mode field in params)
โ resolvePinnedHostnameWithPolicy(hostname)
โ dns.lookup(hostname)
โ ENOTFOUND (local DNS cannot resolve)Missing Proxy Environment Check
The function withRemoteHttpResponse() is implemented in src/memory/post-json.ts (or equivalent entry point). It calls fetchWithSsrFGuard() without setting the mode field in the request parameters:
// Current (broken) implementation
async function withRemoteHttpResponse(params) {
const { response, release } = await fetchWithSsrFGuard({
url: params.url,
init: params.init,
policy: params.ssrfPolicy,
auditContext: params.auditContext ?? "memory-remote"
// MISSING: mode field โ defaults to STRICT
});
// ...
}Default Behavior in STRICT Mode
When mode is not set, fetchWithSsrFGuard() calls resolveGuardedFetchMode(), which returns STRICT by default. In STRICT mode, the function always executes:
resolvePinnedHostnameWithPolicy(hostname)
โ dns.lookup(hostname) // Blocking DNS resolution via Node.js resolver
โ ENOTFOUND // Fails in proxy environmentsWhy This Fails in Proxy Environments
In proxy configurations like Clash TUN with fake-IP mode:
- The local machine's DNS resolver cannot reach
api.ohmygpt.com - DNS resolution must occur through the proxy tunnel (e.g., via
clashDNSor proxy-side resolution) - The
dns.lookup()call uses the system's resolver, bypassing the proxy entirely - The request never reaches the proxy because it fails at hostname resolution
The Correct Pattern
The codebase already contains the correct implementation in other code paths. When environment proxy is configured, withTrustedEnvProxyGuardedFetchMode() should be used:
// Correct implementation
async function withRemoteHttpResponse(params) {
const useEnvProxy = hasProxyEnvConfigured();
const request = useEnvProxy
? withTrustedEnvProxyGuardedFetchMode({
url: params.url,
init: params.init,
policy: params.ssrfPolicy,
auditContext: params.auditContext ?? "memory-remote"
})
: {
url: params.url,
init: params.init,
policy: params.ssrfPolicy,
auditContext: params.auditContext ?? "memory-remote"
};
const { response, release } = await fetchWithSsrFGuard(request);
// ...
}In TRUSTED_ENV_PROXY mode, fetchWithSsrFGuard():
- Skips local DNS pre-resolution via
dns.lookup() - Uses
EnvHttpProxyAgent()directly for HTTP connections - Delegates DNS resolution to the proxy infrastructure
๐ ๏ธ Step-by-Step Fix
Option 1: Source Code Fix (Recommended)
File to modify: src/memory/post-json.ts (or equivalent source location)
Before:
import { fetchWithSsrFGuard } from '../ssrf/fetch-guard';
// ... other imports
async function withRemoteHttpResponse(params: RemoteHttpParams) {
const { response, release } = await fetchWithSsrFGuard({
url: params.url,
init: params.init,
policy: params.ssrfPolicy,
auditContext: params.auditContext ?? "memory-remote"
// mode not set โ defaults to STRICT
});
if (!response.ok) {
const body = await response.text();
release();
throw new RemoteHttpError(params.url, response.status, body);
}
return { response, release };
}After:
import { fetchWithSsrFGuard } from '../ssrf/fetch-guard';
import { hasProxyEnvConfigured, withTrustedEnvProxyGuardedFetchMode } from '../ssrf/fetch-mode';
// ... other imports
async function withRemoteHttpResponse(params: RemoteHttpParams) {
const useEnvProxy = hasProxyEnvConfigured();
const request = useEnvProxy
? withTrustedEnvProxyGuardedFetchMode({
url: params.url,
init: params.init,
policy: params.ssrfPolicy,
auditContext: params.auditContext ?? "memory-remote"
})
: {
url: params.url,
init: params.init,
policy: params.ssrfPolicy,
auditContext: params.auditContext ?? "memory-remote"
};
const { response, release } = await fetchWithSsrFGuard(request);
if (!response.ok) {
const body = await response.text();
release();
throw new RemoteHttpError(params.url, response.status, body);
}
return { response, release };
}Option 2: Runtime Environment Workaround
If you cannot modify source code, set the NODE_TLS_REJECT_UNAUTHORIZED variable to bypass certificate validation issues that may compound the DNS problem:
# Set both proxy and TLS bypass (use only in controlled environments)
export HTTPS_PROXY=http://127.0.0.1:7890
export HTTP_PROXY=http://127.0.0.1:7890
export NODE_TLS_REJECT_UNAUTHORIZED=0
# Then run OpenClaw
openclaw memory status --deep --agent mainOption 3: Bundle Patching (Temporary Workaround)
If the source fix is not yet deployed and you need an immediate fix:
Step 1: Identify affected bundles:
grep -l "withRemoteHttpResponse" dist/*.js 2>/dev/null | head -20Step 2: Create a patch script (patch-memory-proxy.js):
const fs = require('fs');
const path = require('path');
const bundles = [
'dist/reply-Bm8VrLQh.js',
'dist/auth-profiles-DDVivXkv.js',
'dist/discord-CcCLMjHw.js'
];
const searchPattern = /async function withRemoteHttpResponse\(params\)\{const\{response,release\}=await fetchWithSsrFGuard\(\{url:params\.url,init:params\.init,policy:params\.ssrfPolicy,auditContext:params\.auditContext\?\?"memory-remote"\}\);/g;
const replacePattern = `async function withRemoteHttpResponse(params){const _useEnvProxy=hasProxyEnvConfigured();const _request=_useEnvProxy?withTrustedEnvProxyGuardedFetchMode({url:params.url,init:params.init,policy:params.ssrfPolicy,auditContext:params.auditContext??"memory-remote"}):{url:params.url,init:params.init,policy:params.ssrfPolicy,auditContext:params.auditContext??"memory-remote"};const{response,release}=await fetchWithSsrFGuard(_request);`;
bundles.forEach(bundle => {
if (fs.existsSync(bundle)) {
let content = fs.readFileSync(bundle, 'utf8');
if (searchPattern.test(content)) {
content = content.replace(searchPattern, replacePattern);
fs.writeFileSync(bundle, content);
console.log(`Patched: ${bundle}`);
}
}
});Step 3: Run the patch:
node patch-memory-proxy.js๐งช Verification
Verification Steps
Step 1: Confirm the proxy environment variables are set:
$ echo $HTTPS_PROXY
http://127.0.0.1:7890
$ echo $HTTP_PROXY
http://127.0.0.1:7890
$ echo $ALL_PROXY
# (should be empty or set)Step 2: Verify local DNS cannot resolve the endpoint (confirms the condition):
$ node -e "require('dns').lookup('api.ohmygpt.com', (err, addr) => console.log(err ? err.message : addr))"
getaddrinfo ENOTFOUND api.ohmygpt.com
# Expected: ENOTFOUND error (confirms proxy DNS requirement)Step 3: Verify the fix is applied by checking the bundled code:
$ grep -o "withTrustedEnvProxyGuardedFetchMode" dist/*.js | head -5
dist/reply-Bm8VrLQh.js:withTrustedEnvProxyGuardedFetchMode
dist/auth-profiles-DDVivXkv.js:withTrustedEnvProxyGuardedFetchMode
dist/discord-CcCLMjHw.js:withTrustedEnvProxyGuardedFetchMode
# All affected bundles should contain the function callStep 4: Verify the hasProxyEnvConfigured check is present:
$ grep -A1 "hasProxyEnvConfigured()" dist/*.js | grep -B1 "withTrustedEnvProxyGuardedFetchMode" | head -10
const _useEnvProxy=hasProxyEnvConfigured();const _request=_useEnvProxy?withTrustedEnvProxyGuardedFetchMode
# Should see both functions in sequenceStep 5: Run memory status command:
$ openclaw memory status --deep --agent main
Memory Status
โโ Vector Store: OK (50 vectors indexed)
โโ Embeddings: api.ohmygpt.com/text-embedding-3-small
โโ Status: ready
# Expected: Embeddings should show the configured provider, not "unavailable"Step 6: Run memory search command:
$ openclaw memory search --agent main --query "test" --limit 5
[
{
"id": "mem_001",
"score": 0.9234,
"content": "..."
}
]
# Expected: Returns search results without ENOTFOUND errorStep 7: Verify exit codes:
$ openclaw memory search --agent main --query "test"
$ echo $?
0
# Expected: Exit code 0 on successExpected Output After Fix
$ openclaw memory status --deep --agent main
Memory Status
โโ Vector Store
โ โโ Provider: remote
โ โโ Model: text-embedding-3-small
โ โโ Dimensions: 1536
โ โโ Vectors: 50
โโ Embeddings
โ โโ Status: available
โ โโ Endpoint: https://api.ohmygpt.com/v1
โ โโ Model: text-embedding-3-small
โโ Search: functionalโ ๏ธ Common Pitfalls
1. Build Artifact Duplication
withRemoteHttpResponse() function is inlined by rolldown into multiple dist chunks. In version 2026.3.13, there are 7 bundle copies across different files.Affected bundles that may not have the fix:
reply-Bm8VrLQh.jsโ gateway agent tool pathauth-profiles-DDVivXkv.jsโ alternate auth bundlediscord-CcCLMjHw.jsโ discord channel path
Affected bundles that may already have the fix:
auth-profiles-DRjqKE3G.jsโ CLI pathmodel-selection-*.jsโ model selection bundlesplugin-sdk/thread-bindings-*.jsโ plugin SDK bundles
Mitigation: Always rebuild from source and verify all bundle copies contain the proxy guard after building.
2. Environment Variable Case Sensitivity
hasProxyEnvConfigured() may check specific variable names with specific casing.Ensure consistent casing for proxy variables:
# Correct (uppercase)
export HTTPS_PROXY=http://127.0.0.1:7890
# May not be detected
export https_proxy=http://127.0.0.1:7890Check the actual implementation of hasProxyEnvConfigured() to confirm which variables are checked.
3. Proxy Protocol Mismatch
Ensure proxy URL includes the correct protocol:
# Correct for HTTP proxy
export HTTPS_PROXY=http://127.0.0.1:7890
# Correct for SOCKS5 proxy
export HTTPS_PROXY=socks5://127.0.0.1:10804. Bundle Cache After Source Fix
Force clean rebuild:
# Remove dist directory
rm -rf dist/
# Clear any cache
rm -rf node_modules/.cache
# Rebuild
npm run build
# Verify all copies
grep -l "withRemoteHttpResponse" dist/**/*.js | xargs grep -l "hasProxyEnvConfigured"5. WSL/Windows Cross-Environment Proxy
Explicitly set proxy in WSL:
# In WSL, get Windows host IP
export HTTPS_PROXY=http://$(cat /etc/resolv.conf | grep nameserver | awk '{print $2}'):78906. localhost vs 127.0.0.1 in Different Contexts
127.0.0.1 while others use ::1 (IPv6 localhost).Ensure proxy is listening on the correct interface:
# Check proxy binding
netstat -tlnp | grep 7890
# Common result: tcp 0 0 127.0.0.1:7890 0.0.0.0:* LISTEN๐ Related Errors
Related Error Codes and Issues
ENOTFOUNDโ DNS resolution failure. Occurs whendns.lookup()cannot resolve the hostname through the local resolver. In proxy environments, this is expected when the endpoint must be resolved through the proxy tunnel.ECONNREFUSEDโ Connection refused. May occur if the proxy is not running or the proxy address in environment variables is incorrect.ETIMEDOUTโ Connection timeout. May occur if the proxy is unreachable or network routing is misconfigured.Proxy Authentication Requiredโ Corporate proxies requiring credentials. Ensurehttp://user:password@host:portformat is used in proxy URLs.
Historically Related Issues
- SSRF Guard Bypass in Memory Module โ Issue #4521 โ Related fix for
withTrustedEnvProxyGuardedFetchMode()not being used in the embeddings path. - DNS Resolution Fails Behind Corporate Proxy โ Issue #3204 โ General DNS + proxy resolution issues in enterprise environments.
- Memory Search Returns Empty Results โ Issue #4892 โ Symptom may manifest as empty results when embeddings are unavailable due to proxy DNS issues.
- Embeddings Provider Timeout in Docker โ Issue #5103 โ Docker networking + proxy interaction causing timeouts in memory operations.
- SSRF Policy Not Respected in Remote Embeddings โ Issue #5017 โ Security concern about SSRF guards not being applied to memory embedding fetch calls.
Key Dependencies in the Fix Chain
src/ssrf/fetch-guard.tsโ ContainsfetchWithSsrFGuard(), the core fetch function with mode resolutionsrc/ssrf/fetch-mode.tsโ ContainshasProxyEnvConfigured()andwithTrustedEnvProxyGuardedFetchMode()src/ssrf/dns-resolve.tsโ ContainsresolvePinnedHostnameWithPolicy()that performs the blockingdns.lookup()src/memory/post-json.tsโ ContainswithRemoteHttpResponse()that needs the fix