你的系统使用了 systemd-resolved
网络由 NetworkManager 或 Netplan 接管
/etc/resolv.conf 为动态生成的符号链接
注意：不同发行版默认配置可能略有差异（如 nsswitch.conf 或 NetworkManager 设置）。
在默认使用 systemd-resolved + NetworkManager / Netplan 的现代 Linux 发行版中，/etc/resolv.conf 往往不是配置源，而是一个动态生成的解析接口。手动修改的内容通常无法跨越下一次网络重连或系统重启，因为经理人们随时准备根据最新的网络环境背刺你的修改。
1. DNS 解析的三层结构（核心心智模型）
需要明确的是，systemd-resolved 只是 glibc NSS 解析链条中的一个可选后端。在现代 Linux 系统中，一次 DNS 查询是一个严密的分层架构：
A. 应用层：应用程序（ping、curl）并不直接查询 DNS 服务器，而是调用 glibc 提供的解析函数（如 getaddrinfo()）。（启用了内置 DoH 的浏览器（如 Chrome、Firefox）是例外，它们会完全绕过本地解析栈，直接与远端 DoH 服务器通信。）
B. glibc 解析器（Stub Resolver）：它的行为取决于 /etc/nsswitch.conf 的配置链：
- 现代 nss-resolve 路径：glibc 通过 IPC 直接与 systemd-resolved 通信，完全不经过 libc 默认的 UDP stub 路径。
- 传统 nss-dns 路径：它读取 /etc/resolv.conf 中指定的 nameserver（通常是 127.0.0.53）并发送 UDP 请求。
- 注意：非 glibc 程序（如 Go 的纯 Go 解析器）仍可能绕过此逻辑直接读取 /etc/resolv.conf。
C. 本地解析服务（systemd-resolved）：接收到请求后，根据 per-link DNS 配置和路由策略选择上游 DNS。
D. 执行层（上游 DNS）：最终将请求发往真正的 ISP、公共 DNS 或内网 DNS。
2. 为什么 resolv.conf 设计成“瞬态报告”？
这不是反人类，而是为了移动化适配。在 Wi-Fi、热点、VPN 频繁切换的今天，系统需要一个“调解员”实时处理链路冲突。如果 resolv.conf 是死文件，当你从公司 VPN 切换到手机热点时，由于 DNS 无法自动更新，你的网络会立刻瘫痪。
3. 权力架构：谁才是真正的老板？
在采用 systemd 体系的发行版中，DNS 解析通常由 systemd-resolved 负责聚合和管理。其生成的解析配置（位于 /run/systemd/resolve/ 下），通常由安装脚本或管理员手动通过符号链接映射到 /etc/resolv.conf。
核心契约：当该链接指向 systemd-resolved 时，网络管理器不再直接写入该文件，而是通过 D-Bus 将配置提交给它。
4. 核心痛点：DHCP “全家桶”污染
当你连上公共 Wi-Fi，DHCP 协议会强推一个 “全家桶”给你：包含 IP、网关和运营商提供的默认 DNS。更阴险的是 IPv6 后门：很多人只堵住了 IPv4，结果系统又通过 DHCPv6 或是 IPv6 的路由器广播（RA）偷偷塞进一个 IPv6 DNS 地址，导致解析再次被覆盖。
5. 三大对策、各自缺点及 VPN 影响
正统派：经理人覆盖 (Overrides)
Ubuntu Server（使用 systemd-networkd 后端）可以在 Netplan YAML 中将 dhcp4-overrides 和 dhcp6-overrides 的 use-dns 设为 false。
⚠️ 生产环境警告：如果你使用的是 NetworkManager 后端，Netplan 的配置不会生效。你必须通过 nmcli 显式关闭自动 DNS 获取。如果你通过 SSH 远程操作，建议使用链式执行以缩短操作窗口，减少因连接中断导致的配置不一致风险：
# 链式执行：修改并立即激活。注意：&& 不能防御错误的配置，如果配置本身有误，你仍会失联。
nmcli con modify <connection-name> ipv4.ignore-auto-dns yes ipv6.ignore-auto-dns yes && nmcli con up <connection-name>
专家建议：对于极其关键的远程服务器，最稳妥的做法是在执行前预设一个 5 分钟后的自动回滚任务（如 echo "nmcli con up <old-connection-name>" | at now + 5 min），确保即使配置错误也能自动恢复连接。
挂钩派：resolvconf 前置挂钩 (Prepending Hook)
通过修改 /etc/resolvconf/resolv.conf.d/head 文件，将配置强行拼接到生成的 resolv.conf 最顶部。
实施前提：在使用 systemd-resolved 的系统上，通常需要将 /etc/resolv.conf 的控制权平滑移交给 resolvconf（如 openresolv 替代包），必要时需调整服务状态以避免接管冲突。
“咖啡店陷阱”：在需要网页认证的公共 Wi-Fi 下，除了传统的 DNS 劫持，部分现代网络已开始通过 DHCP/RA 下发认证 API（RFC 8910/8952）。强行固定 DNS 会破坏网络上下文感知与 Portal 发现流程，极易导致认证页面无法弹出。
VPN 干扰者：极易干扰依赖 resolvconf 或 systemd-resolved 动态注入 DNS 的 VPN（如 WireGuard 的 wg-quick 或 Tailscale）。写在 head 里的死配置会固化解析路径，极易压制或破坏 VPN 的 Split-DNS（分流解析）机制。（注：在 NSS 解析链中，收到 NXDOMAIN 会直接阻断查询；若遇到超时或 SERVFAIL，则可能触发不可预期的回退行为。）
破局之法：如果你的挂钩目标是 127.0.0.1 (dnscrypt-proxy)，这并非死胡同，而是一个适合实现精细化分流的起点。不要解除 head 的锁死，而是将 OS 层的“配置冲突”转化为代理层的“智能分流”：
在 dnscrypt-proxy.toml 中启用：forwarding_rules = 'forwarding-rules.txt'。
在该 .txt 文件中按 <domain> <server address> 格式写入规则：corp 10.8.0.1 或 corp $DHCP。
(架构师防坑警告：官方声明 $DHCP 转发为实验性功能。面对 WireGuard/Tailscale 等非 DHCP 协议下发的 tun 隧道时，它未必能可靠覆盖此类 VPN 场景。针对动态分配的内网 DNS，在服务端固定网关 IP，或在客户端编写 NetworkManager dispatcher.d 钩子脚本进行实时同步，是实战中更为稳妥的工程解法。)
暴力派：chattr 焊死 (The Nuclear Option)
删掉软链接改实体文件，写死 8.8.8.8 后执行 chattr +i /etc/resolv.conf。
6. 那个消失的 127.0.0.53
之所以称之为“消失”，是因为在现代 Linux 配置下，你能在 /etc/resolv.conf 中直视这个地址，却无法在任何物理网络接口上捕获它的流量。虽然 netstat 显示 127.0.0.53 处于监听状态，但由于 nss-resolve 优先走内存级 IPC 通信，绝大多数解析流量根本不会经过 lo 回环接口。
流量路径：
前半程（程序 → Resolver 解析器）：遵循 NSS 解析链的程序（如 curl、ping）若命中 nss-resolve 路径，走 IPC，lo 接口上抓不到包。刻意绕过 NSS、直接发 UDP 裸请求的诊断工具（如 dig），走 stub listener，你在 lo 上能抓到包。
后半程 (Resolver → 上游)：一旦本地缓存不命中，systemd-resolved 必须向外求助。此时报文会通过路由表从物理网卡（如 wlo1）发出。
实际上，systemd 官方强烈建议程序通过 nss-resolve 直接通信。而 127.0.0.53 仅仅是为那些直接发送 UDP 裸请求的程序提供的兼容性 Stub Listener（官方从 v252 开始还在 127.0.0.54 提供了一个跳过本地合成逻辑、偏向纯转发语义的 Proxy 模式）。
7. 真正的入口：nsswitch.conf
在 Linux 中，解析顺序取决于 /etc/nsswitch.conf 的 hosts: 行。现代发行版常见的配置如下：
hosts: files resolve [!UNAVAIL=return] dns
顺序决定论：glibc 严格从左到右按序调用模块。尽管官方建议将 resolve 排在 files 前以利用缓存，但多数发行版仍优先保障 /etc/hosts 的权威。
断路器机制：[!UNAVAIL=return] 意味着只有当 systemd-resolved 进程彻底死亡或不可达时，才会向右回退。如果只是单纯的域名不存在（NXDOMAIN），解析会直接终止，不会触发回退。
8. 短路逻辑与敏感泄露
在许多被旧版部署工具、自动化脚本或人工干预过的系统中，/etc/nsswitch.conf 往往会变成一个充满陷阱的乱序版本。问题并不在于解析模块本身有毒，而在于 NSS 的链式顺序与断路器语法被错误地拼接成了如下模样：
hosts: files mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns mymachines
第一道封锁线：.local 与断路器引发的解析车祸
nss-mdns 模块的职责本就是通过 mDNS 解析 .local，这无可厚非。但如果你公司的 AD 域为 corp.contoso.local，在上述带有断路器的顺序下将演变成灾难。mdns4_minimal 优先在物理局域网发送组播，找不到后，紧随其后的 [NOTFOUND=return] 指令会瞬间截断后续的查询链。这意味着你的 VPN DNS（即使配置在了 resolve 或 dns 层面）根本收不到请求，内网身份验证直接熔断。这是典型的配置顺序陷阱。
队尾的刺客：乱序 mymachines 的隐私泄露
nss-mymachines 用于解析注册到本地 systemd-machined 的容器名。系统官方标准建议将其放置在 resolve 和 dns 之前，以保障本地容器名优先命中，这在正常配置的系统下是完全安全的。然而，在被篡改或错误配置的乱序系统中（如上述配置中它掉到了 dns 之后），它就成了“队尾的刺客”。如果你 ping 一个本地容器（如 test-db），系统会先让 dns 模块去查一圈公网报错后，才轮到它。这意味着这种配置级的 Bug，会将你的本地容器命名架构明文泄露给配置的上游 DNS 服务器（即便上游最终返回 NXDOMAIN，带有你容器名的查询包也已经出网被记录）。
9. Netplan 的秩序 vs. 恶意的破坏: Netplan 的逻辑秩序与版本敏感性
许多人试图在 YAML 中配置 dhcp4-overrides: use-dns: false 来一劳永逸地拒绝 DHCP 污染。这确实确立了一种逻辑秩序，但请务必注意后端与版本的敏感性：Netplan 官方文档指出，use-dns 设定主要针对 networkd 后端生效。
尽管现代 Netplan 确实会尝试将 dhcp4-overrides 翻译并应用到 NetworkManager 的 DHCP 流程中（转化为 NM 的 ipv4.ignore-auto-dns=yes 属性），但这种跨后端的映射高度依赖版本，在诸如 Ubuntu 20.04 LTS 等仍被广泛使用的旧版系统中极易失效。因此，在极度关键的生产场景中，切勿盲目依赖 YAML 的抽象翻译，更稳妥的做法是直接核实 nmcli 的实际执行状态或底层生成的 keyfile。
即使你完美锁定了底层配置，如果有流氓 VPN 客户端直接以 root 权限执行 write() 强覆写 /etc/resolv.conf，或者恶意进程强占了与 systemd-resolved 冲突的 53 端口（例如试图绑定 0.0.0.0:53 而非避开 127.0.0.53:53），你的 nslookup 依然会发生泄露或引发服务死锁。秩序约束的是遵循系统调用的合法组件；要防备真正的破坏者，你依然需要求助于 AppArmor 等强制访问控制或底层的包过滤机制。
10. 隐私防护：DoT、DoH 与 DNSSEC
纯靠锁死 8.8.8.8 无法防止基于明文 DNS（UDP/TCP 53）的链路劫持或透明代理。systemd-resolved 原生支持 DNS-over-TLS (DoT)（用于传输链路加密）以及 DNSSEC（用于记录完整性校验）。通过修改 /etc/systemd/resolved.conf 启用 DNSOverTLS=yes，在上游解析器支持 DoT 的前提下，让解析请求在 TLS 加密通道中传输。这才是比单纯修改文件更深层的防护机制（可防止链路监听与篡改，但不改变对上游解析器的信任假设）。（注：yes 为严格模式，上游不支持 DoT 时查询将直接失败而非回退明文。如需兼顾可用性，可先用 opportunistic 验证上游支持情况。）
TLS 的 853 端口特征明显，易遭识别阻断。dnscrypt-proxy 支持 DNS-over-HTTPS (DoH, Port 443)。DoH 流量与标准 HTTPS 融合，规避针对特定端口的 DPI 拦截。此场景下，systemd-resolved 仅作为调度存根（Stub），加密与混淆由代理层执行。
11. DNS 代理的乱入与内核死锁陷阱
当你试图通过 /head 引入本地代理时，必须小心 53 端口的启动死锁。默认情况下，systemd-resolved 牢牢绑定在 127.0.0.53:53。虽然这与 127.0.0.1 是独立的 IP，但部分传统的第三方缓存工具（如 dnsmasq）在出厂配置下会默认尝试全局监听（绑定 0.0.0.0:53）。在 Linux 内核机制下，当具体 IP 被占用时，后续的全局通配符绑定（Wildcard Bind）会被内核直接以 Address already in use 拒绝。这就是为什么许多极客配置了本地代理却莫名启动失败——除非你懂得去深入修改代理工具的底层配置（如在 dnsmasq 中强制开启 bind-interfaces），否则最终往往不得不退回硬编码 8.8.8.8。
架构师的解法 (The Socket Bypass)： 像 dnscrypt-proxy 这类强制显式声明监听地址的工具，本身就能避开 0.0.0.0 通配符死锁。而更进阶的系统级解法，是直接利用 systemd 的 Socket Activation（套接字激活）。
无需解除 head 的锁死，配置 .socket 单元，将 127.0.0.1:53 的监听权交由 PID 1 接管。这种基于精确匹配的端点控制能确保代理程序优先拿到端口，彻底消除启动时序的竞争。
(注：此架构生效的底层前提是代理工具必须实现了 sd_listen_fds() 接口来接收系统传递的描述符。dnscrypt-proxy 原生具备此能力，从而完美达成了与 systemd-resolved 的无缝共存。)
12. 软链接的“狸猫换太子”
在 Ubuntu 中，/etc/resolv.conf 的指向决定了你的“抗干扰能力”：
13. 观测与旧时代的遗产：没 nscd 导致的缓存缺失
古代 Linux 依靠 nscd 在 libc 层面拦截并提供独立缓存。现代发行版已默认不再预装该组件，并在很大程度上将其边缘化（尽管部分 LDAP/NIS 企业环境仍在苟延残喘）。因此，只要你的程序（如纯 Go 解析器）或挂钩策略绕过了 systemd-resolved 的 IPC 接口，你的查询就会走传统的 nss-dns 路径。由于该路径默认不带缓存，你将跌落回“零缓存”状态，每次查询都会触发真实的物理发包。
14. 服务器环境下的生死法则
15. Docker/K8s 的云原生结界
为什么运行在系统上的容器不会被宿主机的 127.0.0.53 搞崩溃？因为容器引擎拥有一套独立且强悍的旁路法则：
Docker 的条件触发回退（Conditional Fallback）：在默认的 bridge 网络中，Docker 会参考并继承宿主机的 /etc/resolv.conf（并经过引擎的过滤与重写）。法证级细节：如果 Docker（底层 libnetwork 源码）发现宿主机的配置中包含回环地址（如 127.0.0.53），为了防止容器内发生不可逾越的路由死循环，会强制将其剔除。更暗黑的是，如果剔除后没有留下任何有效的上游 DNS，在部分实现与运行环境中，可能会触发兜底机制，注入预设的公共 DNS 服务器（如 8.8.8.8 和 8.8.4.4）。而在自定义网络（User-defined networks）中，Docker 通常会启用内嵌的 DNS 引擎（固定为 127.0.0.11）（除非被显式 DNS 参数或特殊网络模式如 host 覆盖），负责容器内的服务发现与上游转发。
K8s 的集群级接管：在 Kubernetes 世界里，DNS 解析被彻底拔高到了集群控制面。kubelet 会根据 dnsPolicy 策略接管并重写容器内的 /etc/resolv.conf 的内容。在默认的 ClusterFirst 策略族（及相关变体）下，这意味着容器的解析路径不再依赖宿主机的 systemd-resolved 守护进程或其宿主级 NSS 解析链，而是通过容器内部的 libc Resolver 指向集群 DNS Service（通常为 CoreDNS 的 ClusterIP，底层经由 kube-proxy 或 IPVS 完成 VIP 流量转发），将解析路径控制权完全收敛至集群控制面（对于集群外部域名，解析请求最终仍可能由 CoreDNS 根据配置转发至外部上游）。（注意：如果你手贱设置了 dnsPolicy: Default，Pod 将直接继承宿主机的解析配置，从而可能再次暴露于宿主机解析机制引发的异常，例如回环地址不可达）。
16. Android 与系统的分化: 移动端的终极形态（DnsResolver APEX）
在 Android 系统中，/etc/resolv.conf 早就被抛弃。在 Android 9 及以前，解析从 Bionic libc 走向 netd 的 Socket；而从 Android 10 开始，为了追求更独立的模块化升级，DNS Resolver 被独立为 com.android.resolv APEX 模块。现代 Android 架构通过底层专有 API 与这个独立的系统模块进行高性能 IPC 通信。这印证了现代操作系统在网络切换频繁的场景下，“废弃静态文件，拥抱模块化进程通信”是不可逆的演进方向。
17. 浏览器的“黑箱解析栈”：OS 解析权力的越狱与反制
现代浏览器（Chrome、Firefox）自带独立的网络栈。一旦启用内置 DoH，它们将完全无视系统的 /etc/nsswitch.conf 链条与 getaddrinfo() 调用，直接在用户态通过 443 端口与远端建立 TLS 通道。此时 tcpdump port 53 抓包将是一片死寂。
静默升级陷阱 (Automatic Upgrade)
Chrome 的 Secure DNS 默认处于 Automatic 模式。若当前 DNS 服务器可用且支持 DoH，Chrome 会优先尝试 DoH；若失败或不可用，则回退到未加密 DNS。
这种 OS 与浏览器分治的架构，是产生排障幻觉的根源：
假阳性（系统瘫痪，浏览器幸存）：若将 resolv.conf 锁死为无效地址，终端里的 curl 与 ping 会全线崩溃，但浏览器却能靠内置的 DoH 通道正常打开网页。
假阴性（内网通畅，浏览器报非）：在企业 VPN 中，系统路由表明确知道 git.corp 应走内网网关，浏览器却直接将其打包发给公网的 Cloudflare 导致 NXDOMAIN。最终表现为终端里能 ping 通内网，网页却打不开。
状态机动作与回退机制 (Fallback Logic)
要解决脑裂，必须掌握浏览器在解析失败时的内部处理逻辑：
Firefox (TRR 机制)：在默认的 TRR-first 模式下，DoH 查询失败时允许回退到 OS 的原生解析栈。但若设置为 TRR-only，则形成硬死锁，绝不回退明文。此外，当 Firefox 探测到系统层面的家长控制，或匹配到策略中的排除列表时，会主动跳过 TRR 走 OS 路径。（注：若用户手动开启 DoH，该网络探针信号会被忽略）。
Chrome (Secure DNS)：在默认的 Automatic 模式下，允许探测并回退到 OS。但只要用户显式指定了自定义 DoH 供应商，即刻禁止回退。另外，当 Chrome 检测到自身运行在企业受管环境（Enterprise Policy）中时，会自动降级或彻底关闭 DoH。
夺回控制权：架构师的企业级反制策略 绝不能依赖 Netplan 等 OS 层网络配置来约束浏览器，必须在应用策略层彻底将其收编：
探针截杀 (Canary Domain)：Firefox 遵循标准的探针协议。只要系统级 DNS（如 dnscrypt-proxy）将 use-application-dns.net 拦截并返回 NXDOMAIN，Firefox 会自动关闭内部 DoH。（注：此探针仅对默认开启 DoH 的用户有效；若用户手动强制开启 DoH，该信号将被无视，必须改用策略硬编码）。
策略硬编码 (Policy Enforcement)：生产环境中应通过 /etc/firefox/policies/policies.json 配置 DNSOverHTTPS 节点。设置 "Locked": true 可彻底锁定设置面板，利用 "ExcludedDomains": ["*.corp.local"] 实现精准的内网分流。为确保极致安全，追加 "Fallback": false 可强制禁止其在 DoH 失败后向系统明文解析器投降。
唯一能在系统层单向剥夺 Chrome 解析特权的方式，是向系统策略目录写入 JSON 文件，强制注入 "DnsOverHttpsMode": "off"。注意路径的严格区分：
彻底放弃外部抓包，直接切入浏览器的内部诊断接口以验证策略是否生效：
18. SNI 与 ECH 机制核心精简
结构性缺陷：加密链路下的明文背刺
ESNI：演进中的历史草稿
ECH (Encrypted Client Hello)：现行标准 (RFC 9849)
双层架构：将握手劈成两层。
安全依赖（工程视角）：RFC 9849 协议并不强制要求 DNS 本身必须加密，明文 DNS 也能跑通 ECH 握手。但在隐私保护的工程实战中，如果你用 UDP 53 明文去查询 ECH 公钥，目标域名在握手发生前就已经被中间人嗅探或篡改。脱离了 DoH/DoT 的护航，ECH 的隐私防护将沦为空中楼阁。
生态兼容性断层：库支持 ≠ 工具支持
OpenSSL 4.0 已于 2026 年 4 月引入 ECH 底层 API，但整个生态的落地呈现极度割裂的现实：
当前落地最完整的平台：Web 浏览器
Chrome 与 Firefox 是 ECH 的主战场。它们利用自带的独立 TLS 栈（BoringSSL/NSS）越过系统限制，自主实现闭环。但需注意，这并非瞬间的“完美支持”，Chromium 等浏览器受限于 Finch 灰度机制，支持是分批次、依策略逐步释放的。
工具链适配阵痛区：CLI 与原生运行时 这是最容易踩坑的盲点：底层库支持 API，不代表上层工具已经调用。Ubuntu 26.10（预计 2026 年 10 月）是 OpenSSL 4.0 进入系统的预期节点，但库就位只是起点，各工具完成应用层适配才是终点。Python ssl 模块的 ECH 支持目前仍在跟进中（追踪 issue #89730），合并时间表尚无官方定论。至于其他工具，虽然 curl 借由 BoringSSL/wolfSSL 后端已有实验性支持。对于绝大多数依赖系统 OpenSSL 且尚未完成源码级适配的命令行工具而言，SNI 泄露依然是当前的默认行为。
架构解法：收敛出口
在基础密码库完成彻底升级（如未来的主流 LTS 发行版全面拥抱 OpenSSL 4.0+），且庞大的 CLI 工具链彻底完成应用层 ECH 适配之前，试图在操作系统侧逐个手动重编译打补丁是不切实际的。
当前最具可操作性的工程解法：
放弃在滞后的应用栈本身寻找隐私开关。在本地建立支持 ECH 的透明代理或 TUN 隧道（如 cloudflared、sing-box、Xray），强制接管所有应用流量。由代理内核统一封装 ECH 握手，将隐私防护从底层碎片化的工具链中彻底剥离出来。
注：实战中必须确认所选代理工具及配置在出站连接上原生执行了 ECH 握手。例如 Xray/sing-box 需正确配置 TLS/UTLS 行为，而非仅仅作为普通的 HTTPS 转发。
19. 观察普通程序如何发起 DNS 查询
trace_process_dns_route.sh 从真实程序开始观察 DNS 查询。它运行指定命令，并同时记录该命令及其子进程的 socket 调用、systemd-resolved 的活动、resolvectl monitor 输出和可见的明文 DNS packet。这样可以依据实际证据判断程序通过本机 IPC、兼容的 DNS stub，还是直接联系外部 DNS server，而不必先假定它采用哪一条解析路径。
脚本能够识别的常见路径包括：
程序 → glibc NSS → nss-resolve → 本机 Varlink/D-Bus IPC → systemd-resolved
程序 → 127.0.0.53:53 或 127.0.0.54:53 → systemd-resolved DNS stub listener
程序 → 其他 IP 地址的 53 端口 → 直接使用明文 DNS，绕过 systemd-resolved stub
脚本也会报告前往 853 端口的连接，因为这与 DNS-over-TLS 相符。对于 443 端口，它只指出存在 HTTPS 连接，不会仅凭端口把普通 HTTPS 判定为 DNS-over-HTTPS。
当查询确实进入 systemd-resolved 时，脚本会临时启用 debug 日志。Positive cache hit 或 ... from cache 表示记录由缓存返回；Cache miss 表示本地缓存没有可用答案；Using DNS server ... 和 scope dns on ... 分别指出所选上游 DNS server 与网络接口；Using feature level ... 显示采用 UDP、TCP 或 TLS 等传输能力；... from network 则表明该 transaction 最终由网络响应完成。
与此同时，resolvectl monitor 会显示进入 systemd-resolved 的查询名称、记录类型和响应。若使用传统明文 DNS，tcpdump 还可显示查询名称、目标 DNS server 和返回记录。DoT 与 DoH 的 DNS 内容已加密，因此这些连接即使可见，也不能从普通抓包中直接读取查询名称和答案。
#!/usr/bin/env bash
set -Eeuo pipefail
# Trace how one command performs DNS resolution.
#
# The script runs the command under strace, follows its child processes, watches
# local queries handled by systemd-resolved, and temporarily enables debug logs
# so cache hits, cache misses, selected DNS servers, interfaces and network
# transactions can be shown when systemd-resolved is involved.
#
# It can distinguish these application-side routes when evidence is available:
# - glibc/NSS through nss-resolve and systemd-resolved local IPC
# - DNS through systemd-resolved's 127.0.0.53/127.0.0.54 stub listener
# - direct plaintext DNS to another address on port 53
# - direct DNS-over-TLS connection on port 853
# - possible application-managed encrypted DNS on port 443 (not provable from
# the port alone)
#
# Important boundary: systemd-resolved's debug log is system-wide. The script
# narrows the observation window and correlates traced PIDs where possible, but
# unrelated resolver traffic during the same window may still appear.
usage() {
printf '%s\n' \
'Usage:' \
' ./trace_process_dns_route.sh [options] -- command [arguments ...]' \
'' \
'Options:' \
' --flush-cache Clear the entire systemd-resolved cache before running.' \
' --keep-logs Keep the raw strace, monitor and journal logs.' \
' -h, --help Show this help.' \
'' \
'Examples:' \
' ./trace_process_dns_route.sh -- getent ahosts example.com' \
' ./trace_process_dns_route.sh --flush-cache -- getent ahosts example.com' \
' ./trace_process_dns_route.sh -- curl -I https://example.com/' \
'' \
'Run this script as your normal user. It requests sudo only for temporary' \
'systemd-resolved debug logging and journal access. If the whole script is' \
'started with sudo, it attempts to run the traced command as SUDO_USER.'
}
FLUSH_CACHE=0
KEEP_LOGS=0
while (( $# > 0 )); do
case "$1" in
--flush-cache)
FLUSH_CACHE=1
shift
;;
--keep-logs)
KEEP_LOGS=1
shift
;;
-h|--help)
usage
exit 0
;;
--)
shift
break
;;
*)
printf 'Error: unknown option: %s\n\n' "$1" >&2
usage >&2
exit 2
;;
esac
done
if (( $# == 0 )); then
echo 'Error: no command was supplied after --.' >&2
echo >&2
usage >&2
exit 2
fi
COMMAND=("$@")
for cmd in strace resolvectl journalctl systemctl awk grep sed sort paste mktemp date sleep rm kill; do
command -v "$cmd" >/dev/null 2>&1 || {
echo "Error: required command not found: $cmd" >&2
exit 1
}
done
if (( EUID != 0 )); then
command -v sudo >/dev/null 2>&1 || {
echo 'Error: sudo is required for systemd-resolved debug logging.' >&2
exit 1
}
fi
as_root() {
if (( EUID == 0 )); then
"$@"
else
sudo -- "$@"
fi
}
# If the script itself was started through sudo, do not silently run the target
# application as root. Re-run only the traced command as the original user.
TARGET_PREFIX=()
if (( EUID == 0 )) && [[ -n "${SUDO_USER:-}" && "${SUDO_USER}" != root ]]; then
TARGET_PREFIX=(sudo -u "$SUDO_USER" --)
fi
TMPDIR_RUN=$(mktemp -d /tmp/trace_process_dns_route.XXXXXX)
TRACE_PREFIX="$TMPDIR_RUN/strace"
TRACE_COMBINED="$TMPDIR_RUN/strace.combined.log"
MONITOR_LOG="$TMPDIR_RUN/resolvectl-monitor.log"
JOURNAL_LOG="$TMPDIR_RUN/systemd-resolved-journal.log"
TCPDUMP_LOG="$TMPDIR_RUN/plaintext-dns-packets.log"
MONITOR_PID=""
TCPDUMP_PID=""
OLD_LOG_LEVEL=""
LOG_LEVEL_CHANGED=0
RESOLVED_ACTIVE=0
PRIVILEGED_OBSERVATION=0
FINISHED=0
stop_background_process() {
local pid=$1
[[ -n "$pid" ]] || return 0
kill -INT "$pid" 2>/dev/null || true
for ((i = 0; i < 20; i++)); do
kill -0 "$pid" 2>/dev/null || break
sleep 0.05
done
if kill -0 "$pid" 2>/dev/null; then
kill -TERM "$pid" 2>/dev/null || true
fi
wait "$pid" 2>/dev/null || true
}
stop_packet_capture() {
if [[ -n "$TCPDUMP_PID" ]]; then
stop_background_process "$TCPDUMP_PID"
TCPDUMP_PID=""
fi
}
stop_monitor() {
if [[ -n "$MONITOR_PID" ]]; then
stop_background_process "$MONITOR_PID"
MONITOR_PID=""
fi
}
restore_log_level() {
if (( LOG_LEVEL_CHANGED == 1 )) && [[ -n "$OLD_LOG_LEVEL" ]]; then
as_root resolvectl log-level "$OLD_LOG_LEVEL" >/dev/null 2>&1 || true
LOG_LEVEL_CHANGED=0
fi
}
cleanup() {
stop_packet_capture
stop_monitor
restore_log_level
if (( FINISHED == 1 )); then
if (( KEEP_LOGS == 0 )); then
rm -rf "$TMPDIR_RUN"
fi
elif [[ -d "$TMPDIR_RUN" ]]; then
printf '\nThe run ended before completion. Raw evidence was kept in: %s\n' "$TMPDIR_RUN" >&2
fi
}
on_signal() {
exit 130
}
trap cleanup EXIT
trap on_signal INT TERM
if systemctl is-active --quiet systemd-resolved; then
RESOLVED_ACTIVE=1
fi
if (( RESOLVED_ACTIVE == 1 )); then
# Authenticate before starting the observation window. Failure does not stop
# strace; it only removes resolver debug/journal evidence from this run.
if as_root true 2>/dev/null; then
PRIVILEGED_OBSERVATION=1
OLD_LOG_LEVEL=$(as_root resolvectl log-level 2>/dev/null | tail -n 1 || true)
if [[ -n "$OLD_LOG_LEVEL" ]] && as_root resolvectl log-level debug >/dev/null 2>&1; then
LOG_LEVEL_CHANGED=1
else
echo 'Warning: could not enable temporary systemd-resolved debug logging.' >&2
fi
if (( FLUSH_CACHE == 1 )); then
echo 'Warning: clearing the entire systemd-resolved DNS cache before the command.' >&2
as_root resolvectl flush-caches
fi
else
echo 'Warning: sudo authentication failed; resolver debug and journal evidence will be unavailable.' >&2
fi
else
echo 'Note: systemd-resolved is not active. The script will still inspect the application-side sockets.' >&2
fi
START_EPOCH=$(date +%s)
START_CURSOR=""
if (( RESOLVED_ACTIVE == 1 && PRIVILEGED_OBSERVATION == 1 )); then
as_root journalctl --sync >/dev/null 2>&1 || true
START_CURSOR=$(as_root journalctl -u systemd-resolved -n 0 --show-cursor --no-pager 2>/dev/null \
| sed -n 's/^-- cursor: //p' | tail -n 1 || true)
fi
if (( PRIVILEGED_OBSERVATION == 1 )) && command -v tcpdump >/dev/null 2>&1; then
: >"$TCPDUMP_LOG"
if (( EUID == 0 )); then
tcpdump -n -l -vvv -s 0 -i any 'udp port 53 or tcp port 53' >"$TCPDUMP_LOG" 2>&1 &
else
sudo -- tcpdump -n -l -vvv -s 0 -i any 'udp port 53 or tcp port 53' >"$TCPDUMP_LOG" 2>&1 &
fi
TCPDUMP_PID=$!
fi
if (( RESOLVED_ACTIVE == 1 )); then
: >"$MONITOR_LOG"
if command -v stdbuf >/dev/null 2>&1; then
LC_ALL=C stdbuf -oL -eL resolvectl --no-pager monitor >"$MONITOR_LOG" 2>&1 &
else
LC_ALL=C resolvectl --no-pager monitor >"$MONITOR_LOG" 2>&1 &
fi
MONITOR_PID=$!
sleep 0.25
fi
printf 'Tracing command:'
printf ' %q' "${COMMAND[@]}"
printf '\n'
if [[ ${#TARGET_PREFIX[@]} -gt 0 ]]; then
printf 'The command will run as user: %s\n' "$SUDO_USER"
fi
set +e
LC_ALL=C strace \
-ff \
-ttt \
-T \
-yy \
-s 2048 \
-e trace=connect,sendto,sendmsg,sendmmsg,recvfrom,recvmsg,openat,openat2 \
-o "$TRACE_PREFIX" \
-- "${TARGET_PREFIX[@]}" "${COMMAND[@]}"
COMMAND_STATUS=$?
set -e
# Give systemd-resolved enough time to emit completion/debug lines before the
# monitor is stopped and the journal is collected.
sleep 0.40
stop_packet_capture
stop_monitor
restore_log_level
if (( RESOLVED_ACTIVE == 1 && PRIVILEGED_OBSERVATION == 1 )); then
as_root journalctl --sync >/dev/null 2>&1 || true
if [[ -n "$START_CURSOR" ]]; then
as_root journalctl -u systemd-resolved --after-cursor="$START_CURSOR" \
-o cat --no-pager >"$JOURNAL_LOG" 2>&1 || true
else
as_root journalctl -u systemd-resolved --since="@$START_EPOCH" \
-o cat --no-pager >"$JOURNAL_LOG" 2>&1 || true
fi
else
: >"$JOURNAL_LOG"
fi
: >"$TRACE_COMBINED"
shopt -s nullglob
TRACE_FILES=("$TRACE_PREFIX".*)
shopt -u nullglob
TRACED_PIDS=()
for file in "${TRACE_FILES[@]}"; do
pid=${file##*.}
[[ "$pid" =~ ^[0-9]+$ ]] || continue
TRACED_PIDS+=("$pid")
sed "s/^/[pid $pid] /" "$file" >>"$TRACE_COMBINED"
done
contains_trace() {
grep -Eq -- "$1" "$TRACE_COMBINED" 2>/dev/null
}
print_trace_evidence() {
local title=$1 pattern=$2
echo "$title"
grep -E -- "$pattern" "$TRACE_COMBINED" 2>/dev/null | sed -n '1,12p' | sed 's/^/ /' || true
}
print_unique_journal_matches() {
local title=$1 pattern=$2
local matches
matches=$(grep -E -- "$pattern" "$JOURNAL_LOG" 2>/dev/null | sed 's/^[[:space:]]*//' | sort -u || true)
if [[ -n "$matches" ]]; then
echo "$title"
while IFS= read -r line; do
printf ' %s\n' "$line"
done <<<"$matches"
fi
}
# The relevant endpoint patterns are intentionally based on decoded sockaddr
# output produced by strace -yy. They cover both connect() and sendto/sendmsg()
# style clients.
PORT53_PATTERN='(sin_port=htons\(53\)|sin6_port=htons\(53\))'
PORT853_PATTERN='(sin_port=htons\(853\)|sin6_port=htons\(853\))'
PORT443_PATTERN='(sin_port=htons\(443\)|sin6_port=htons\(443\))'
STUB_PATTERN='127\.0\.0\.(53|54)'
VARLINK_PATTERN='/run/systemd/resolve/io\.systemd\.Resolve'
DBUS_SOCKET_PATTERN='/run/dbus/system_bus_socket'
HAS_NSS_RESOLVE=0
HAS_NSS_DNS=0
HAS_VARLINK=0
HAS_DBUS_SOCKET=0
HAS_STUB=0
HAS_ANY_53=0
HAS_DIRECT_53=0
HAS_853=0
HAS_443=0
HAS_RESOLV_CONF=0
HAS_NSSWITCH=0
HAS_HOSTS=0
contains_trace 'libnss_resolve\.so' && HAS_NSS_RESOLVE=1
contains_trace 'libnss_dns\.so' && HAS_NSS_DNS=1
contains_trace "$VARLINK_PATTERN" && HAS_VARLINK=1
contains_trace "$DBUS_SOCKET_PATTERN" && HAS_DBUS_SOCKET=1
contains_trace "$PORT53_PATTERN" && HAS_ANY_53=1
contains_trace "$PORT853_PATTERN" && HAS_853=1
contains_trace "$PORT443_PATTERN" && HAS_443=1
contains_trace '"/etc/resolv\.conf"' && HAS_RESOLV_CONF=1
contains_trace '"/etc/nsswitch\.conf"' && HAS_NSSWITCH=1
contains_trace '"/etc/hosts"' && HAS_HOSTS=1
if (( HAS_ANY_53 == 1 )); then
if grep -E -- "$PORT53_PATTERN" "$TRACE_COMBINED" | grep -Eq -- "$STUB_PATTERN"; then
HAS_STUB=1
fi
if grep -E -- "$PORT53_PATTERN" "$TRACE_COMBINED" | grep -Ev -- "$STUB_PATTERN" | grep -q .; then
HAS_DIRECT_53=1
fi
fi
PID_REGEX=""
if (( ${#TRACED_PIDS[@]} > 0 )); then
PID_REGEX=$(printf '%s\n' "${TRACED_PIDS[@]}" | sort -nu | paste -sd'|' -)
fi
HAS_MATCHED_DBUS_PID=0
if [[ -n "$PID_REGEX" ]] && grep -Eq "D-Bus .*resolution request from client PID ($PID_REGEX)([^0-9]|$)" "$JOURNAL_LOG" 2>/dev/null; then
HAS_MATCHED_DBUS_PID=1
fi
HAS_RESOLVED_QUERY=0
if grep -Eq 'Looking up RR for |New incoming message: .*io\.systemd\.Resolve\.(ResolveHostname|ResolveAddress|ResolveService|ResolveRecord)' "$JOURNAL_LOG" 2>/dev/null \
|| grep -Eq '(^|[[:space:]])(IN|CLASS[0-9]+)[[:space:]]+(A|AAAA|MX|CNAME|TXT|NS|SOA|SRV|PTR|TYPE[0-9]+)([[:space:]]|$)' "$MONITOR_LOG" 2>/dev/null; then
HAS_RESOLVED_QUERY=1
fi
HAS_CACHE_HIT=0
HAS_CACHE_MISS=0
HAS_NETWORK_COMPLETE=0
grep -Eq '(Positive|Negative|NODATA|RCODE [A-Z]+) cache hit for |now complete with <[^>]+> from cache' "$JOURNAL_LOG" 2>/dev/null && HAS_CACHE_HIT=1
grep -Eq 'Cache miss for ' "$JOURNAL_LOG" 2>/dev/null && HAS_CACHE_MISS=1
grep -Eq 'now complete with <[^>]+> from network|Using DNS server .* for transaction|Sending query packet with id|Sending query via TCP' "$JOURNAL_LOG" 2>/dev/null && HAS_NETWORK_COMPLETE=1
echo
echo '================================================================'
echo 'Application-side DNS route'
echo '================================================================'
ROUTE_COUNT=0
if (( HAS_VARLINK == 1 )); then
echo 'Observed route: local Varlink IPC to systemd-resolved.'
echo 'The traced process connected to systemd-resolved through its local io.systemd.Resolve socket.'
((ROUTE_COUNT+=1))
fi
if (( HAS_MATCHED_DBUS_PID == 1 )); then
echo 'Observed route: D-Bus request to systemd-resolved, matched to a traced process PID.'
((ROUTE_COUNT+=1))
elif (( HAS_DBUS_SOCKET == 1 && HAS_RESOLVED_QUERY == 1 )); then
echo 'Possible route: D-Bus to systemd-resolved.'
echo 'The process used the system bus and systemd-resolved handled a query in the same window, but the PID could not be matched conclusively.'
((ROUTE_COUNT+=1))
fi
if (( HAS_STUB == 1 )); then
echo 'Observed route: DNS compatibility stub at 127.0.0.53 or 127.0.0.54 on port 53.'
echo 'The application sent DNS traffic to systemd-resolved through its local stub listener.'
((ROUTE_COUNT+=1))
fi
if (( HAS_DIRECT_53 == 1 )); then
echo 'Observed route: direct plaintext DNS to port 53 outside the systemd-resolved stub addresses.'
echo 'This process or one of its traced children bypassed the local systemd-resolved stub for that request.'
((ROUTE_COUNT+=1))
fi
if (( HAS_853 == 1 )); then
echo 'Observed route: direct connection to port 853, consistent with DNS over TLS.'
((ROUTE_COUNT+=1))
fi
if (( ROUTE_COUNT == 0 )); then
if (( HAS_RESOLVED_QUERY == 1 )); then
echo 'systemd-resolved handled one or more queries during the command, but the selected application-side syscalls did not expose a conclusive entry route.'
else
echo 'No new DNS-resolution route was observed from the traced command or its child processes.'
echo 'Possible explanations include an application cache, /etc/hosts, a reused connection, a fixed IP address, or resolution performed by an already-running process outside the traced process tree.'
fi
fi
if (( HAS_NSS_RESOLVE == 1 )); then
echo 'Additional evidence: the process loaded libnss_resolve, the glibc NSS module for systemd-resolved.'
fi
if (( HAS_NSS_DNS == 1 )); then
echo 'Additional evidence: the process loaded libnss_dns, the traditional glibc DNS NSS module.'
fi
if (( HAS_NSSWITCH == 1 )); then
echo 'Additional evidence: the process read /etc/nsswitch.conf.'
fi
if (( HAS_RESOLV_CONF == 1 )); then
echo 'Additional evidence: the process read /etc/resolv.conf.'
fi
if (( HAS_HOSTS == 1 )); then
echo 'Additional evidence: the process read /etc/hosts.'
fi
if (( HAS_443 == 1 )); then
echo 'HTTPS connections to port 443 were observed. Port 443 alone cannot distinguish DNS over HTTPS from ordinary HTTPS application traffic.'
fi
if (( HAS_VARLINK == 1 )); then
print_trace_evidence 'Varlink IPC evidence:' "$VARLINK_PATTERN"
fi
if (( HAS_STUB == 1 )); then
print_trace_evidence 'systemd-resolved stub evidence:' "$PORT53_PATTERN.*$STUB_PATTERN|$STUB_PATTERN.*$PORT53_PATTERN"
fi
if (( HAS_DIRECT_53 == 1 )); then
echo 'Direct port-53 evidence:'
grep -E -- "$PORT53_PATTERN" "$TRACE_COMBINED" | grep -Ev -- "$STUB_PATTERN" | sed -n '1,12p' | sed 's/^/ /' || true
fi
if (( HAS_853 == 1 )); then
print_trace_evidence 'Port-853 evidence:' "$PORT853_PATTERN"
fi
echo
echo '================================================================'
echo 'Queries seen by systemd-resolved'
echo '================================================================'
if [[ -s "$MONITOR_LOG" ]]; then
echo 'resolvectl monitor output:'
sed -n '1,120p' "$MONITOR_LOG" | sed 's/^/ /'
else
echo 'No query was printed by resolvectl monitor during this command.'
fi
if (( HAS_MATCHED_DBUS_PID == 1 )); then
print_unique_journal_matches 'Resolver requests matched to traced PIDs:' \
"D-Bus .*resolution request from client PID ($PID_REGEX)([^0-9]|$)"
fi
print_unique_journal_matches 'DNS questions recorded by systemd-resolved:' \
'Looking up RR for |New incoming message: .*io\.systemd\.Resolve\.(ResolveHostname|ResolveAddress|ResolveService|ResolveRecord)'
echo
echo '================================================================'
echo 'How systemd-resolved answered'
echo '================================================================'
if [[ ! -s "$JOURNAL_LOG" ]]; then
echo 'Resolver debug evidence is unavailable for this run.'
else
if (( HAS_CACHE_HIT == 1 )); then
print_unique_journal_matches 'Cache-hit evidence:' \
'(Positive|Negative|NODATA|RCODE [A-Z]+) cache hit for |now complete with <[^>]+> from cache'
fi
if (( HAS_CACHE_MISS == 1 )); then
print_unique_journal_matches 'Cache-miss evidence:' 'Cache miss for '
fi
if (( HAS_NETWORK_COMPLETE == 1 )); then
print_unique_journal_matches 'Network-route evidence:' \
'Firing .*transaction|Regular transaction|Using DNS server .* for transaction|Using feature level .* for transaction|Sending query packet with id|Sending query via TCP|now complete with <[^>]+> from network'
fi
print_unique_journal_matches 'Cache-write evidence:' \
'Added (positive|negative|NODATA|RCODE).*cache entry|Added .* cache entry'
if (( HAS_CACHE_HIT == 0 && HAS_CACHE_MISS == 0 && HAS_NETWORK_COMPLETE == 0 )); then
echo 'No explicit cache-hit, cache-miss or upstream-network decision was found in the captured debug log.'
echo 'The result may have come from /etc/hosts, a synthesized/local record, an application cache, or a resolver path whose messages differ on this systemd version.'
fi
fi
echo
echo '================================================================'
echo 'Plaintext DNS packet content'
echo '================================================================'
if [[ -s "$TCPDUMP_LOG" ]]; then
if grep -Eq '([0-9]+\+? (A|AAAA|MX|CNAME|TXT|NS|SOA|SRV|PTR)\?|Flags \[|A [0-9]|AAAA [0-9a-fA-F:])' "$TCPDUMP_LOG" 2>/dev/null; then
echo 'tcpdump observed DNS traffic on port 53 during the command window:'
sed -n '1,160p' "$TCPDUMP_LOG" | sed 's/^/ /'
echo 'For plaintext DNS, this can expose the queried name, record type, DNS server and returned records.'
else
echo 'tcpdump ran, but no plaintext DNS packet was decoded during the command window.'
echo 'tcpdump output:'
sed -n '1,20p' "$TCPDUMP_LOG" | sed 's/^/ /'
fi
else
if command -v tcpdump >/dev/null 2>&1; then
echo 'No plaintext DNS packet was captured during the command window.'
else
echo 'tcpdump is not installed, so packet-level DNS content was not captured.'
fi
fi
echo 'DNS over TLS and DNS over HTTPS encrypt the DNS message, so packet capture cannot display their query names or answers without decryption.'
echo
echo '================================================================'
echo 'Interpretation'
echo '================================================================'
if (( HAS_CACHE_HIT == 1 && HAS_NETWORK_COMPLETE == 0 )); then
echo 'At least one systemd-resolved query was answered from cache, and no upstream transaction was identified in the captured window.'
elif (( HAS_CACHE_MISS == 1 && HAS_NETWORK_COMPLETE == 1 )); then
echo 'At least one systemd-resolved query missed the cache and proceeded through an upstream network transaction.'
elif (( HAS_CACHE_HIT == 1 && HAS_NETWORK_COMPLETE == 1 )); then
echo 'Both cache and network activity occurred. This is normal when a command requests several names or record types, follows aliases, or triggers auxiliary lookups.'
elif (( HAS_RESOLVED_QUERY == 1 )); then
echo 'systemd-resolved handled a query, but the captured log did not provide enough evidence to classify it as cache or upstream network.'
elif (( HAS_DIRECT_53 == 1 || HAS_853 == 1 )); then
echo 'The application used a DNS path outside systemd-resolved, so systemd-resolved cannot report whether that separate resolver used its own cache.'
else
echo 'No new DNS query was conclusively observed.'
fi
echo
printf 'Command exit status: %d\n' "$COMMAND_STATUS"
if (( KEEP_LOGS == 1 )); then
printf 'Raw evidence directory: %s\n' "$TMPDIR_RUN"
fi
FINISHED=1
exit "$COMMAND_STATUS"
运行方法
先添加执行权限：
chmod +x trace_process_dns_route.sh
观察 getent 的完整 DNS 入口路径：
./trace_process_dns_route.sh -- getent ahosts example.com
这里的 -- 分隔脚本选项与需要追踪的命令，后面的内容会原样交给 strace 运行。若希望先清空 systemd-resolved 缓存，使本次运行更容易触发上游查询，可使用：
./trace_process_dns_route.sh --flush-cache -- getent ahosts example.com
--flush-cache 会清空整个 systemd-resolved DNS cache，而不只清除当前域名；它不会影响浏览器或其他程序自行维护的缓存，也不应在不允许扰动系统 resolver cache 的生产环境中随意使用。
追踪 curl 时，命令形式相同：
./trace_process_dns_route.sh -- curl -I https://example.com/
需要保留完整证据时使用：
./trace_process_dns_route.sh --keep-logs -- getent ahosts example.com
脚本会保留 strace、resolvectl monitor、systemd-resolved journal 和 tcpdump 输出，并显示临时目录路径。脚本应以普通用户身份启动；读取 journal、临时调整 resolver 日志级别、启动 tcpdump 或执行可选的 cache flush 时，会按需要请求 sudo。结束或被中断后，脚本会恢复原来的 resolver 日志级别。
如何读取结果
若输出包含：
Observed route: local Varlink IPC to systemd-resolved.
表示目标进程连接了 /run/systemd/resolve/io.systemd.Resolve。这是程序通过本机 IPC 进入 systemd-resolved 的直接证据，而不是根据“没有看到 UDP 53”作出的推断。若同时出现：
Additional evidence: the process loaded libnss_resolve
则该进程还加载了 glibc 的 nss-resolve 模块。模块加载与 Varlink socket 连接互相吻合时，可以确认程序采用 NSS 到 systemd-resolved 的本机解析路径。
若输出包含：
Observed route: DNS compatibility stub at 127.0.0.53 or 127.0.0.54 on port 53.
表示程序没有使用直接 IPC API，而是把 DNS packet 发送给 systemd-resolved 的本地兼容 stub。若输出改为：
Observed route: direct plaintext DNS to port 53 outside the systemd-resolved stub addresses.
则目标进程或其子进程直接联系了其他 53 端口 endpoint，没有经过 127.0.0.53 或 127.0.0.54。脚本会打印相应的 socket syscall，并在 tcpdump 可用时显示明文 DNS 内容。
在 How systemd-resolved answered 部分：
Positive cache hit for example.com IN A
表示 systemd-resolved 已有 example.com 的 IPv4 记录，并从缓存返回。相反，以下日志表示缓存中没有该记录，resolver 随后选择上游 DNS server，并由网络响应完成查询：
Cache miss for example.com IN A
Using DNS server 192.0.2.53 for transaction ...
... now complete with <success> from network
同一个命令可能同时产生 cache 与 network 证据。例如，程序可能同时请求 A 和 AAAA，其中一种记录已缓存，另一种仍需访问网络；CNAME 跳转和 DNSSEC 验证也可能产生额外查询。因此，脚本保留每条 resolver 证据，而不会把整次命令压缩成唯一的 CACHE 或 NETWORK 标签。
观测边界
解释结果时必须保留进程与时间窗口的边界。程序可能复用已经建立的连接，因此本次运行不再查询 DNS；浏览器命令也可能把请求交给既有进程，而该进程不属于当前追踪的进程树。程序自己的 DNS cache 不会出现在 systemd-resolved 日志中，443 端口连接也不能单独证明 DoH。resolver debug 日志和 tcpdump 都是系统范围的，短暂窗口内可能混入其他进程的查询；对于 DoT 与 DoH，普通抓包无法读取加密后的域名和答案。
因此，本节首先回答的是“这个具体程序实际沿哪条路径发起 DNS 查询”。应用侧 strace 说明程序连接了哪个 resolver endpoint，resolvectl monitor 与 resolver journal 说明 systemd-resolved 收到了什么及如何处理，tcpdump 则补充证明明文 DNS packet 是否实际出现在接口上。下一节会把程序入口的差异排除，只观察 systemd-resolved 对连续两次相同查询如何选择网络与缓存。
20. 观察 systemd-resolved 自然选择网络还是缓存
上一节从真实程序出发，重建查询到达 resolver 的路径。本节把观察范围缩小到 systemd-resolved 本身：先清空一次缓存，再连续执行两次完全相同的普通查询，观察第一次取得的网络答案是否会被紧接着的第二次查询直接复用。
以查询 example.com 的 A record 为例，每次执行：
resolvectl --legend=yes --type=A query example.com.
脚本在每个 DNS record type 开始前只执行一次：
resolvectl flush-caches
随后立即运行两次相同查询：
清空 systemd-resolved cache
→ 第一次查询
→ 不再清空 cache
→ 第二次完全相同的查询
预期流程是第一次查询发生 cache miss，systemd-resolved 选择上游 DNS server，从网络取得答案并写入缓存；第二次查询则命中 positive cache，不再建立对应的上游 transaction，并返回相同记录。
配套脚本：trace_resolved_natural_cache_flow.sh
#!/usr/bin/env bash
set -Eeuo pipefail
# Observe systemd-resolved's natural cache behaviour without forcing the query
# source. The same ordinary resolvectl query is run twice:
#
# 1. Immediately after the resolver cache is flushed.
# 2. Immediately again, without another flush.
#
# Neither query uses --cache=no nor --network=no. The script therefore observes
# what systemd-resolved naturally chooses. It compares the returned records and
# correlates resolvectl's source report with resolver debug logs, monitor output
# and optional packet capture.
#
# Important boundary: systemd-resolved logging and packet capture are system-
# wide. The observation windows are short and evidence is filtered by hostname,
# record type and resolver transaction ID, but unrelated traffic can still be
# present. A concurrent lookup can also repopulate the tested cache entry after
# the flush.
usage() {
printf '%s\n' \
'Usage:' \
' sudo ./trace_resolved_natural_cache_flow.sh [options] [hostname] [record_type ...]' \
'' \
'Defaults:' \
' hostname: icanhazip.com' \
' record types: A AAAA' \
'' \
'Options:' \
' --keep-logs Keep resolver, monitor, query and packet-capture logs.' \
' --no-packets Do not start tcpdump.' \
' -h, --help Show this help.' \
'' \
'Examples:' \
' sudo ./trace_resolved_natural_cache_flow.sh' \
' sudo ./trace_resolved_natural_cache_flow.sh example.com A' \
' sudo ./trace_resolved_natural_cache_flow.sh --keep-logs example.com A AAAA' \
'' \
'Warning:' \
' The script clears the entire systemd-resolved cache before testing' \
' each record type. Do not use it where that disturbance is unacceptable.'
}
ORIGINAL_ARGS=("$@")
KEEP_LOGS=0
CAPTURE_PACKETS=1
while (( $# > 0 )); do
case "$1" in
--keep-logs)
KEEP_LOGS=1
shift
;;
--no-packets)
CAPTURE_PACKETS=0
shift
;;
-h|--help)
usage
exit 0
;;
--)
shift
break
;;
-*)
printf 'Error: unknown option: %s\n\n' "$1" >&2
usage >&2
exit 2
;;
*)
break
;;
esac
done
TARGET=${1:-icanhazip.com}
if (( $# > 0 )); then
shift
fi
if [[ -z "$TARGET" ]]; then
echo 'Error: hostname must not be empty.' >&2
exit 2
fi
if (( $# > 0 )); then
RR_TYPES=("$@")
else
RR_TYPES=(A AAAA)
fi
if (( EUID != 0 )); then
command -v sudo >/dev/null 2>&1 || {
echo 'Error: this script requires root privileges, but sudo was not found.' >&2
exit 1
}
exec sudo -- "$0" "${ORIGINAL_ARGS[@]}"
fi
for cmd in resolvectl systemctl journalctl awk grep sed sort paste mktemp tail sleep rm kill date; do
command -v "$cmd" >/dev/null 2>&1 || {
echo "Error: required command not found: $cmd" >&2
exit 1
}
done
systemctl is-active --quiet systemd-resolved || {
echo 'Error: systemd-resolved is not running.' >&2
exit 1
}
RESOLVECTL_HELP=$(LC_ALL=C resolvectl --help 2>&1 || true)
for required_option in '--type=' '--legend='; do
grep -q -- "$required_option" <<<"$RESOLVECTL_HELP" || {
echo "Error: this resolvectl version does not support $required_option." >&2
exit 1
}
done
for rrtype in "${RR_TYPES[@]}"; do
[[ "$rrtype" =~ ^[A-Za-z][A-Za-z0-9-]*$ ]] || {
echo "Error: invalid DNS record type: $rrtype" >&2
exit 2
}
done
FQDN=${TARGET%.}.
LOG_NAME=${FQDN%.}
TMPDIR_RUN=$(mktemp -d /tmp/trace_resolved_natural_cache_flow.XXXXXX)
OLD_LOG_LEVEL=""
LOG_LEVEL_CHANGED=0
ACTIVE_MONITOR_PID=""
ACTIVE_TCPDUMP_PID=""
FINISHED=0
QUERY_STATUS=0
stop_background_process() {
local pid=$1
[[ -n "$pid" ]] || return 0
kill -INT "$pid" 2>/dev/null || true
for ((i = 0; i < 20; i++)); do
kill -0 "$pid" 2>/dev/null || break
sleep 0.05
done
if kill -0 "$pid" 2>/dev/null; then
kill -TERM "$pid" 2>/dev/null || true
fi
wait "$pid" 2>/dev/null || true
}
stop_observers() {
if [[ -n "$ACTIVE_MONITOR_PID" ]]; then
stop_background_process "$ACTIVE_MONITOR_PID"
ACTIVE_MONITOR_PID=""
fi
if [[ -n "$ACTIVE_TCPDUMP_PID" ]]; then
stop_background_process "$ACTIVE_TCPDUMP_PID"
ACTIVE_TCPDUMP_PID=""
fi
}
restore_log_level() {
if (( LOG_LEVEL_CHANGED == 1 )) && [[ -n "$OLD_LOG_LEVEL" ]]; then
resolvectl log-level "$OLD_LOG_LEVEL" >/dev/null 2>&1 || true
LOG_LEVEL_CHANGED=0
fi
}
cleanup() {
stop_observers
restore_log_level
if (( FINISHED == 1 )); then
if (( KEEP_LOGS == 0 )); then
rm -rf "$TMPDIR_RUN"
fi
elif [[ -d "$TMPDIR_RUN" ]]; then
printf '\nThe run ended before completion. Raw evidence was kept in: %s\n' "$TMPDIR_RUN" >&2
fi
}
on_signal() {
exit 130
}
trap cleanup EXIT
trap on_signal INT TERM
OLD_LOG_LEVEL=$(resolvectl log-level 2>/dev/null | tail -n 1 || true)
if [[ -n "$OLD_LOG_LEVEL" ]] && resolvectl log-level debug >/dev/null 2>&1; then
LOG_LEVEL_CHANGED=1
# Give systemd-resolved a moment to apply the runtime log-level change
# before the first query observation window begins.
sleep 0.20
else
echo 'Warning: could not enable temporary systemd-resolved debug logging.' >&2
fi
query_source() {
sed -n 's/^-- Data from:[[:space:]]*//p' "$1" | tail -n 1
}
query_answers() {
# Canonicalize answer rows so harmless ordering differences do not fail the
# comparison. Per-row "-- link: ..." annotations are not record content.
awk '
/^-- / { next }
/^[[:space:]]*$/ { next }
/: resolve call failed:/ { next }
{
sub(/[[:space:]]+-- link:.*/, "")
print
}
' "$1" | LC_ALL=C sort -u
}
contains_word() {
local text=$1 word=$2
[[ " $text " == *" $word "* ]]
}
print_answers() {
local heading=$1 answers=$2
echo "$heading"
if [[ -n "$answers" ]]; then
while IFS= read -r line; do
printf ' %s\n' "$line"
done <<<"$answers"
else
echo ' <none>'
fi
}
regex_escape() {
sed 's/[][(){}.^$*+?|\\/]/\\&/g' <<<"$1"
}
start_monitor() {
local logfile=$1
: >"$logfile"
if command -v stdbuf >/dev/null 2>&1; then
LC_ALL=C stdbuf -oL -eL resolvectl --no-pager monitor >"$logfile" 2>&1 &
else
LC_ALL=C resolvectl --no-pager monitor >"$logfile" 2>&1 &
fi
ACTIVE_MONITOR_PID=$!
}
start_packet_capture() {
local logfile=$1
: >"$logfile"
if (( CAPTURE_PACKETS == 0 )) || ! command -v tcpdump >/dev/null 2>&1; then
return 0
fi
tcpdump -n -l -vvv -s 0 -i any \
'(udp port 53 or tcp port 53 or tcp port 853)' >"$logfile" 2>&1 &
ACTIVE_TCPDUMP_PID=$!
}
run_observed_query() {
local rrtype=$1 query_log=$2 monitor_log=$3 journal_log=$4 packet_log=$5
local journal_cursor="" journal_start_epoch
stop_observers
: >"$journal_log"
# Mark the journal position before starting the query. Collecting the
# completed journal range afterward is more reliable than killing a live
# journalctl follower while very short cache-hit messages are still being
# delivered or buffered.
journalctl --sync >/dev/null 2>&1 || true
journal_start_epoch=$(date +%s)
journal_cursor=$(journalctl -u systemd-resolved -n 0 --show-cursor \
--no-pager 2>/dev/null | sed -n 's/^-- cursor: //p' | tail -n 1 || true)
start_monitor "$monitor_log"
start_packet_capture "$packet_log"
sleep 0.30
: >"$query_log"
set +e
LC_ALL=C resolvectl --legend=yes --type="$rrtype" query "$FQDN" \
>"$query_log" 2>&1
QUERY_STATUS=$?
set -e
# Allow resolver completion messages and packet output to settle, then stop
# the live observers before taking the completed journal snapshot.
sleep 0.50
stop_observers
journalctl --sync >/dev/null 2>&1 || true
if [[ -n "$journal_cursor" ]]; then
journalctl -u systemd-resolved --after-cursor="$journal_cursor" \
-o cat --no-pager >"$journal_log" 2>&1 || true
else
journalctl -u systemd-resolved --since="@$journal_start_epoch" \
-o cat --no-pager >"$journal_log" 2>&1 || true
fi
}
transaction_ids_for_record() {
local journal_log=$1 rrtype=$2
local name_re
name_re=$(regex_escape "$LOG_NAME")
grep -E "transaction [0-9]+ for <${name_re} IN ${rrtype}>" "$journal_log" 2>/dev/null \
| sed -E 's/.*transaction ([0-9]+).*/\1/' \
| LC_ALL=C sort -nu
}
print_resolver_evidence() {
local journal_log=$1 rrtype=$2
local name_re ids id_re record_re matches
name_re=$(regex_escape "$LOG_NAME")
record_re="${name_re} IN ${rrtype}"
ids=$(transaction_ids_for_record "$journal_log" "$rrtype" | paste -sd'|' - || true)
matches=$(grep -E \
"Looking up RR for ${record_re}|Cache miss for ${record_re}|cache hit for ${record_re}|transaction [0-9]+ for <${record_re}>|Added .*cache entry for ${record_re}" \
"$journal_log" 2>/dev/null || true)
if [[ -n "$ids" ]]; then
id_re="(${ids})"
matches+=$'\n'
matches+=$(grep -E \
"transaction ${id_re}([^0-9]|$)|id ${id_re}([^0-9]|$)" \
"$journal_log" 2>/dev/null || true)
fi
matches=$(sed '/^[[:space:]]*$/d; s/^[[:space:]]*//' <<<"$matches" | LC_ALL=C sort -u)
if [[ -n "$matches" ]]; then
while IFS= read -r line; do
printf ' %s\n' "$line"
done <<<"$matches"
else
echo ' No matching resolver debug lines were emitted or captured for this query.'
echo " This does not override resolvectl's direct per-query source report."
fi
}
print_monitor_evidence() {
local monitor_log=$1 rrtype=$2
local name_re
name_re=$(regex_escape "$LOG_NAME")
if grep -Eq "${name_re} IN ${rrtype}" "$monitor_log" 2>/dev/null; then
grep -E -B1 -A5 "${name_re} IN ${rrtype}" "$monitor_log" 2>/dev/null \
| sed -n '1,80p' | sed 's/^/ /'
elif [[ -s "$monitor_log" ]]; then
echo ' The monitor ran, but it did not print the tested record.'
sed -n '1,12p' "$monitor_log" | sed 's/^/ /'
else
echo ' No monitor output was captured.'
fi
}
print_packet_evidence() {
local packet_log=$1 phase=$2
local target_with_dot
target_with_dot="${LOG_NAME}."
if (( CAPTURE_PACKETS == 0 )); then
echo ' Packet capture was disabled by --no-packets.'
return 0
fi
if ! command -v tcpdump >/dev/null 2>&1; then
echo ' tcpdump is not installed.'
return 0
fi
if [[ ! -s "$packet_log" ]]; then
echo ' No packet-capture output was produced.'
return 0
fi
if grep -Fq "$target_with_dot" "$packet_log" 2>/dev/null; then
echo " Plaintext DNS packets containing $target_with_dot were observed:"
grep -F -B1 -A1 "$target_with_dot" "$packet_log" 2>/dev/null \
| sed -n '1,80p' | sed 's/^/ /'
elif grep -Eq '\.853([[:space:]>:]|$)' "$packet_log" 2>/dev/null; then
echo ' Port-853 traffic was observed. This is consistent with DNS over TLS,'
echo ' but the encrypted packet does not expose the tested hostname.'
grep -E -B1 -A1 '\.853([[:space:]>:]|$)' "$packet_log" 2>/dev/null \
| sed -n '1,40p' | sed 's/^/ /'
else
if [[ "$phase" == second ]]; then
echo ' No packet containing the tested plaintext DNS name was observed during the second query.'
echo ' This supports, but does not by itself prove, a cache hit.'
else
echo ' No packet containing the tested plaintext DNS name was decoded.'
echo ' The resolver debug/source evidence remains authoritative; DNS over TLS hides the name.'
fi
fi
}
packet_log_has_target_name() {
local packet_log=$1
grep -Fq "${LOG_NAME}." "$packet_log" 2>/dev/null
}
journal_record_completed_from() {
local journal_log=$1 rrtype=$2 source=$3
local name_re
name_re=$(regex_escape "$LOG_NAME")
grep -Eq \
"transaction [0-9]+ for <${name_re} IN ${rrtype}>.*now complete with <[^>]+> from ${source}" \
"$journal_log" 2>/dev/null
}
journal_record_has_cache_hit() {
local journal_log=$1 rrtype=$2
local name_re
name_re=$(regex_escape "$LOG_NAME")
grep -Eq \
"(Positive|Negative|NODATA|RCODE [A-Z]+) cache hit for ${name_re} IN ${rrtype}|transaction [0-9]+ for <${name_re} IN ${rrtype}>.*from cache" \
"$journal_log" 2>/dev/null
}
printf 'Natural systemd-resolved cache-flow test\n'
printf 'Target hostname: %s\n' "$FQDN"
printf 'Record types: %s\n' "${RR_TYPES[*]}"
echo 'Both lookups are ordinary resolvectl queries.'
echo 'Neither lookup uses --cache=no nor --network=no.'
echo 'The cache is flushed once before the first lookup of each record type; the second lookup is then run immediately without another flush.'
echo 'Warning: each flush clears the entire systemd-resolved DNS cache.'
ALL_OK=1
INDEX=0
for rrtype_raw in "${RR_TYPES[@]}"; do
INDEX=$((INDEX + 1))
rrtype=${rrtype_raw^^}
PREFIX="$TMPDIR_RUN/$INDEX.$rrtype"
FIRST_QUERY_LOG="$PREFIX.first.query.log"
FIRST_MONITOR_LOG="$PREFIX.first.monitor.log"
FIRST_JOURNAL_LOG="$PREFIX.first.journal.log"
FIRST_PACKET_LOG="$PREFIX.first.packets.log"
SECOND_QUERY_LOG="$PREFIX.second.query.log"
SECOND_MONITOR_LOG="$PREFIX.second.monitor.log"
SECOND_JOURNAL_LOG="$PREFIX.second.journal.log"
SECOND_PACKET_LOG="$PREFIX.second.packets.log"
echo
echo '================================================================'
printf 'Testing natural behaviour: %s IN %s\n' "$FQDN" "$rrtype"
echo '================================================================'
echo
echo '[Preparation] Clear systemd-resolved cache once'
resolvectl flush-caches
sleep 0.10
echo 'Cache flush completed.'
echo
echo '[1/2] First ordinary query after the flush'
printf 'Command: resolvectl --legend=yes --type=%s query %s\n' "$rrtype" "$FQDN"
run_observed_query "$rrtype" \
"$FIRST_QUERY_LOG" "$FIRST_MONITOR_LOG" "$FIRST_JOURNAL_LOG" "$FIRST_PACKET_LOG"
FIRST_STATUS=$QUERY_STATUS
FIRST_SOURCE=$(query_source "$FIRST_QUERY_LOG")
FIRST_ANSWERS=$(query_answers "$FIRST_QUERY_LOG")
FIRST_DEBUG_NETWORK=0
FIRST_DEBUG_CACHE=0
journal_record_completed_from "$FIRST_JOURNAL_LOG" "$rrtype" network && FIRST_DEBUG_NETWORK=1
journal_record_has_cache_hit "$FIRST_JOURNAL_LOG" "$rrtype" && FIRST_DEBUG_CACHE=1
if (( FIRST_STATUS == 0 )); then
echo 'Query result: succeeded'
else
printf 'Query result: failed (exit status %d)\n' "$FIRST_STATUS"
fi
printf 'Reported data source: %s\n' "${FIRST_SOURCE:-not reported}"
print_answers 'Records returned:' "$FIRST_ANSWERS"
echo 'Matching systemd-resolved debug evidence:'
print_resolver_evidence "$FIRST_JOURNAL_LOG" "$rrtype"
echo 'Matching resolvectl monitor evidence:'
print_monitor_evidence "$FIRST_MONITOR_LOG" "$rrtype"
echo 'Packet evidence:'
print_packet_evidence "$FIRST_PACKET_LOG" first
if (( FIRST_STATUS != 0 )) || [[ -z "$FIRST_ANSWERS" ]]; then
echo 'First-query result: INCONCLUSIVE — the ordinary lookup did not return a positive record set.'
ALL_OK=0
continue
fi
if ! contains_word "$FIRST_SOURCE" network || contains_word "$FIRST_SOURCE" cache; then
echo 'First-query result: INCONCLUSIVE'
echo 'The first ordinary query was not reported solely as coming from network.'
echo 'Another process may have repopulated the entry after the cache flush, or the name may be local/synthetic.'
ALL_OK=0
continue
fi
if (( FIRST_DEBUG_CACHE == 1 && FIRST_DEBUG_NETWORK == 0 )); then
echo 'First-query result: INCONCLUSIVE'
echo 'resolvectl reported network, but the matching resolver debug evidence indicated only cache.'
echo 'The system-wide observation window may have been contaminated or this systemd version may report the transaction differently.'
ALL_OK=0
continue
fi
echo 'First-query result: NATURAL NETWORK RETRIEVAL OBSERVED'
echo
echo '[2/2] Repeat the exact same ordinary query without another flush'
printf 'Command: resolvectl --legend=yes --type=%s query %s\n' "$rrtype" "$FQDN"
run_observed_query "$rrtype" \
"$SECOND_QUERY_LOG" "$SECOND_MONITOR_LOG" "$SECOND_JOURNAL_LOG" "$SECOND_PACKET_LOG"
SECOND_STATUS=$QUERY_STATUS
SECOND_SOURCE=$(query_source "$SECOND_QUERY_LOG")
SECOND_ANSWERS=$(query_answers "$SECOND_QUERY_LOG")
SECOND_DEBUG_NETWORK=0
SECOND_DEBUG_CACHE=0
SECOND_PACKET_TARGET=0
journal_record_completed_from "$SECOND_JOURNAL_LOG" "$rrtype" network && SECOND_DEBUG_NETWORK=1
journal_record_has_cache_hit "$SECOND_JOURNAL_LOG" "$rrtype" && SECOND_DEBUG_CACHE=1
packet_log_has_target_name "$SECOND_PACKET_LOG" && SECOND_PACKET_TARGET=1
if (( SECOND_STATUS == 0 )); then
echo 'Query result: succeeded'
else
printf 'Query result: failed (exit status %d)\n' "$SECOND_STATUS"
fi
printf 'Reported data source: %s\n' "${SECOND_SOURCE:-not reported}"
print_answers 'Records returned:' "$SECOND_ANSWERS"
echo 'Matching systemd-resolved debug evidence:'
print_resolver_evidence "$SECOND_JOURNAL_LOG" "$rrtype"
echo 'Matching resolvectl monitor evidence:'
print_monitor_evidence "$SECOND_MONITOR_LOG" "$rrtype"
echo 'Packet evidence:'
print_packet_evidence "$SECOND_PACKET_LOG" second
SECOND_EVIDENCE_CONFLICT=0
if (( SECOND_DEBUG_NETWORK == 1 )); then
SECOND_EVIDENCE_CONFLICT=1
fi
if (( SECOND_PACKET_TARGET == 1 )); then
# tcpdump is system-wide. A matching packet may belong to another
# process, but it means this short window is contaminated and cannot be
# treated as a clean cache-only observation.
SECOND_EVIDENCE_CONFLICT=1
fi
if (( SECOND_STATUS == 0 )) \
&& contains_word "$SECOND_SOURCE" cache \
&& ! contains_word "$SECOND_SOURCE" network \
&& (( SECOND_EVIDENCE_CONFLICT == 0 )) \
&& [[ -n "$SECOND_ANSWERS" ]] \
&& [[ "$SECOND_ANSWERS" == "$FIRST_ANSWERS" ]]; then
echo 'Comparison: the second query returned the identical canonical record set.'
echo 'Test result: PASS'
echo 'With no source-forcing options, systemd-resolved naturally used the network after the flush and naturally used its cache for the immediate repeat.'
else
echo 'Test result: INCONCLUSIVE'
if [[ "$SECOND_ANSWERS" != "$FIRST_ANSWERS" ]]; then
print_answers 'First-query records:' "$FIRST_ANSWERS"
print_answers 'Second-query records:' "$SECOND_ANSWERS"
fi
if (( SECOND_DEBUG_NETWORK == 1 )); then
echo 'Matching resolver debug evidence showed an upstream network completion during the second-query window.'
fi
if (( SECOND_PACKET_TARGET == 1 )); then
echo 'A plaintext DNS packet containing the tested name appeared during the second-query window.'
echo 'Because packet capture is system-wide, this may be concurrent traffic, but the window is not clean enough for PASS.'
fi
if (( SECOND_DEBUG_CACHE == 0 )); then
echo 'No matching cache-hit debug line was emitted or captured; this alone is not considered a failure.'
fi
echo 'The immediate repeated query was not established as an identical natural cache result without contradictory network evidence.'
echo 'Possible causes include a very short TTL, changing answers, concurrent resolver activity, or a non-cache source.'
ALL_OK=0
fi
done
echo
if (( ALL_OK == 1 )); then
echo 'Overall result: PASS'
echo 'Every requested record type naturally went to network after a flush and was then returned unchanged from cache by the immediate ordinary repeat query.'
else
echo 'Overall result: INCONCLUSIVE'
echo 'At least one record type did not exhibit the expected natural network-then-cache sequence.'
fi
if (( KEEP_LOGS == 1 )); then
printf 'Raw evidence directory: %s\n' "$TMPDIR_RUN"
fi
FINISHED=1
if (( ALL_OK == 1 )); then
exit 0
fi
exit 1
运行方法
添加执行权限：
chmod +x trace_resolved_natural_cache_flow.sh
直接运行时，脚本默认查询 icanhazip.com 的 A 和 AAAA records：
sudo ./trace_resolved_natural_cache_flow.sh
指定 hostname 与 record type：
sudo ./trace_resolved_natural_cache_flow.sh example.com A
sudo ./trace_resolved_natural_cache_flow.sh example.com A AAAA MX
需要保留全部原始日志或停用 packet capture 时，分别使用：
sudo ./trace_resolved_natural_cache_flow.sh --keep-logs example.com A
sudo ./trace_resolved_natural_cache_flow.sh --no-packets example.com A
脚本需要 root 权限，因为它会清空整个 systemd-resolved cache、临时把 resolver 日志级别改为 debug、读取 resolver journal，并在可用时通过 tcpdump 观察 DNS port 53 与 DoT port 853。结束或被中断后，脚本会恢复原来的日志级别。
第一次查询如何确认来自网络
cache flush 只建立了一个较干净的起点，不能单独证明第一次查询必然访问网络。其他进程可能在极短时间内重新填充同一条记录，目标名称也可能来自本地或 synthetic source。
因此，第一次普通查询必须成功返回非空 records，并由 resolvectl 报告：
-- Data from: network
同时不能报告 cache。脚本随后从 systemd-resolved debug journal 中寻找与目标 hostname、record type 和 transaction ID 对应的证据，例如：
Cache miss for example.com IN A
Using DNS server 8.8.8.8 for transaction 12345
Sending query packet with id 12345
Regular transaction 12345 for <example.com IN A> ... from network
Added positive ... cache entry for example.com IN A ...
对于传统明文 DNS，tcpdump
还可能显示本机向上游 DNS server 发送查询并收到响应。packet capture 属于系统范围的辅助证据，可能混入其他进程的
DNS traffic；若 resolver 使用 DNS over TLS，hostname 也不会出现在明文 packet
中。因此，来源判断仍以逐查询的 resolvectl source report 和对应 resolver transaction 为主。
第二次查询如何确认来自缓存
第二次查询与第一次完全相同，两次之间不再 flush cache。它必须成功返回非空 records，由 resolvectl 报告：
-- Data from: cache
同时不能报告 network。脚本还会规范化两次返回的 records，移除与 DNS record 本身无关的 link annotation，并排序去重；只有 canonical record set 完全一致，第二次结果才符合预期。
resolver journal 可能出现：
Positive cache hit for example.com IN A
Regular transaction ... for <example.com IN A> ... from cache
不同 systemd-resolved 版本或 client API 路径不一定都会输出这些 cache-hit debug lines，因此缺少这类文本本身不构成失败。第二次查询的直接证据仍是 resolvectl 报告 cache，辅以相同的 canonical record set，并且没有发现明确对应的 network completion 或包含目标 hostname 的上游明文 DNS packet。
证据范围与最终判定
脚本在每次查询前记录 systemd-resolved journal cursor，并在查询结束后读取该 cursor 之后的完整 journal range，以减少遗漏短暂 debug message 的机会。证据优先级依次是逐查询的 resolvectl source report、两次 canonical record set 的比较、明确的 resolver network/cache debug evidence，以及系统范围 packet capture 的辅助验证。
当第一次查询由 network 返回正面答案、第二次相同查询由 cache 返回、两次 canonical record set 一致，而且没有证据表明第二次仍发生对应的上游 transaction 时，该 record type 判定为 PASS。这表示 systemd-resolved 在 cache flush 后通过网络取得记录，并在紧接着的重复查询中从缓存返回同一组记录。
若另一个进程在 flush 后抢先查询、记录 TTL 在第二次查询前失效、上游返回动态答案、名称来自 /etc/hosts、synthetic
record、LLMNR 或 mDNS，或者系统范围的 journal 与 packet window 混入其他查询，脚本可能无法得到清楚的
network-then-cache 顺序。negative、NODATA、错误响应，以及 source report 与 resolver
debug evidence 明确冲突，也会使结果变成 INCONCLUSIVE，而不是直接判定缓存机制失效。
