# 现象
2026-03-16: 早上客户发现我司 Web 系统登录不上, reboot 重启服务器恢复业务
reboot 之后, 一线收集了巡检信息返回到研发这边, 需要研发分析出现异常的原因.
2026-03-16: 根据收集回来的信息, 发现没有 messages 日志, pg_log 目录下的日志显示.
ounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(line2026-03-16 00:05:44.200 UTC PostgreSQL JDBC Driver postgres 192.168.100.201(48404) imos 3452719:LOG: duration: 509419.414 ms parse <unnamed>: SELECT 12026-03-16 00:05:44.201 UTC PostgreSQL JDBC Driver postgres 192.168.100.201(34684) stk 3722720:LOG: duration: 553438.912 ms parse <unnamed>: SELECT 12026-03-16 00:05:44.202 UTC 2106:LOG: checkpoint starting: time2026-03-16 00:05:44.203 UTC PostgreSQL JDBC Driver postgres 192.168.100.201(57516) stk 3992715:LOG: duration: 310416.988 ms parse <unnamed>: SELECT 12026-03-16 00:05:45.495 UTC [unknown] postgres 192.168.100.201(56266) stk 4025385:FATAL: canceling authentication due to timeout2026-03-16 00:05:49.456 UTC [unknown] postgres 127.0.0.1(45068) live 4026021:LOG: could not receive data from client: Connection reset by peer2026-03-16 00:05:49.472 UTC [unknown] postgres 127.0.0.1(45026) live 4026018:LOG: could not receive data from client: Connection reset by peer2026-03-16 00:05:49.476 UTC [unknown] postgres 127.0.0.1(45024) live 4026017:LOG: could not receive data from client: Connection reset by peer2026-03-16 00:05:49.770 UTC [unknown] postgres 127.0.0.1(40222) live 819690:LOG: could not receive data from client: Connection reset by peer2026-03-16 00:05:49.826 UTC [unknown] postgres 127.0.0.1(57088) live 2095915:LOG: could not receive data from client: Connection reset by peer2026-03-16 00:05:49.845 UTC [unknown] postgres 127.0.0.1(51630) live 2297111:LOG: could not receive data from client: Connection reset by peer2026-03-16 00:05:49.861 UTC [unknown] postgres 127.0.0.1(51618) live 2297110:LOG: could not receive data from client: Connection reset by peer2026-03-16 00:05:49.875 UTC [unknown] postgres 127.0.0.1(54510) live 3401842:LOG: could not receive data from client: Connection reset by peer2026-03-16 00:06:17.071 UTC 2106:LOG: checkpoint complete: wrote 276 buffers (0.2%); 1 WAL file(s) added, 0 removed, 0 recycled; write=28.691 s, sync=0.678 s, total=32.870 s; sync files=140, longest=0.031 s, average=0.005 s; distance=2313 kB, estimate=4175 kB
可以看到 select 1 就耗时了310416.988 ms 显然不正常
再往前面查看
ounter(lineounter(lineounter(lineounter(line2026-03-15 23:39:38.731 UTC 2106:LOG: checkpoint starting: time2026-03-15 23:40:12.231 UTC 2106:LOG: checkpoint complete: wrote 270 buffers (0.2%); 0 WAL file(s) added, 0 removed, 0 recycled; write=29.756 s, sync=3.235 s, total=33.501 s; sync files=168, longest=0.287 s, average=0.020 s; distance=3710 kB, estimate=4382 kB2026-03-15 23:45:55.167 UTC [unknown] postgres 192.168.100.201(58750) cds 8087:LOG: duration: 6265.354 ms statement: SELECT sys_account_id, dir_code, dir_name, total_cap, spare_cap, pool_id, sys_account_type, full_policy, dir_attr, from_dir_code FROM tbl_cds_usr_dir_info ORDER BY dir_attr ASC , dir_code asc limit 100000 offset 0;2026-03-15 23:46:02.823 UTC [unknown] postgres 192.168.100.201(58746) cds 8086:LOG: duration: 6534.802 ms statement: SELECT COUNT(1) FROM tbl_bm_group_info ;
可疑的 SQL 如下
ounter(lineSELECT sys_account_id, dir_code, dir_name, total_cap, spare_cap, pool_id, sys_account_type, full_policy, dir_attr, from_dir_code FROM tbl_cds_usr_dir_info ORDER BY dir_attr ASC , dir_code asc limit 100000 offset 0;
耗时 6s , 也不太正常. 但是没看是否是这条 SQL 导致的. 没有其他明显信息
没有 sar 信息, 没有 messages 信息
2026-03-17: 二线申请了远程, 我作为研发 DBA 远程上去额外收集了下 sar 信息与 messages 信息, 用于分析
sar -f /var/log/sa/sa16 -q

可以看到问题发生的时间点 08:05:47 附近, 这个时候 负载巨高, 从 8.7 直接激增为 1595, 后面知道下午 04:47:49 负载都在一千多
主要是 CPU 导致的? 还是 IO 导致的? 接着分析
sar -f /var/log/sa/sa16 -u

可以看到08:05:47 这个时间点 统计到的 CPU 占用情况是, 系统占用 86.68% , 用户占用 2.1% ,idle 为 6.33%, 说明主要是系统在消耗 CPU, 系统可能有啥问题
sar -f /var/log/sa/sa16 -d

可以看到 tps 明显变大, 读磁盘的速度明显变高, 磁盘使用率 util 明显变高.
说明此时, 磁盘读使用率也挺高.
sar -f /var/log/sa/sa16 -r

查看下 page 页占用情况
sar -f /var/log/sa/sa16 -B

结合 sar -r 与 sar -B 可以看到 08:05:47 这个时间点 pgpgin 高达 108313 个页(每个页 8k) 从硬盘换入到 os page buffer 中, 也说明存在大量的读硬盘, 而是 pgscank 与 pgscand 表明此时内存不足, OS 在做内存回收, 后续几个小时直到下午 03:16:54 也大都处于 内存不足状态(下面三点多, 客户 reboot 服务器了)
分析如下:
| pgpgin/s | |
| pgpgout/s | |
| fault/s | |
| majflt/s | |
| pgfree/s | |
| pgscank/s | |
| pgscand/s | |
| pgsteal/s | |
| %vmeff |
ounter(line08:45:47 AM 108113.13 98.60 5962.20 148.68 34292.10 613502.85 1386382.03 54162.79 2.71
| pgpgin/s | |||
| pgpgout/s | |||
| majflt/s | |||
| pgscank/s | |||
| %vmeff |
pgpgin/s 108,113 → 每秒需要换入 10.8万 页(约 420MB)majflt/s 5,962 → 每秒 5,962 次需要磁盘I/O的缺页%vmeff 2.71% → 只有 2.71% 的扫描是有效的另外, 可以看到 从 07:40 到 08:05 期间, 正常应该有 07:50, 08:00 两个点的统计信息, 但是此环境中没有
说明此时由于系统负载过高, sar 统计进程都没有正常收集信息了
说明 07:40 到 08:05 期间负载很高, 已经存在异常了, CPU 占用很高, 磁盘读也很严重, 并且由于 OS page 中数据不在, 频繁读磁盘
与之对应的就是 如下大量读 10w 条记录的 SQL
ounter(lineounter(line2026-03-15 23:45:55.167 UTC [unknown] postgres 192.168.100.201(58750) cds 8087:LOG: duration: 6265.354 ms statement: SELECT sys_account_id, dir_code, dir_name, total_cap, spare_cap, pool_id, sys_account_type, full_policy, dir_attr, from_dir_code FROM tbl_cds_usr_dir_info ORDER BY dir_attr ASC , dir_code asc limit 100000 offset 0;2026-03-15 23:46:02.823 UTC [unknown] postgres 192.168.100.201(58746) cds 8086:LOG: duration: 6534.802 ms statement: SELECT COUNT(1) FROM tbl_bm_group_info ;
可以看到 此时查询 10w 条, 并且涉及排序, 消耗内存应该不少, 大概率就是这个 SQL 导致的雪崩(压死骆驼最后一根稻草)
该 SQL 对应的业务通过二线与现场用户确认确实有在 07:45 左右操作过, 说得通了.
如果有 oswatcher 可以方便查看到, 但是由于现场环境中没有部署该工具, 无法查看到相关信息

ounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(lineounter(line帮我分析下 系统messages 日志Mar 16 07:40:04 localhost systemd[1]: Finished system activity accounting tool.Mar 16 07:42:01 localhost systemctl[4004001]: [systemctl start iscsi] called by PID 4003987 (/bin/sh /usr/local/sbin/mp_recovdiskspath.sh)Mar 16 07:44:02 localhost systemctl[4015659]: [systemctl start iscsi] called by PID 4015652 (/bin/sh /usr/local/sbin/mp_recovdiskspath.sh)Mar 16 08:05:45 localhost kernel: [20795758.109932] msserver invoked oom-killer: gfp_mask=0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), order=0, oom_score_adj=0Mar 16 08:05:45 localhost kernel: [20795758.109956] CPU: 3 PID: 14057 Comm: msserver Kdump: loaded Tainted: G OE 6.1.46 #4Mar 16 08:05:45 localhost kernel: [20795758.109961] Hardware name: Intel Corporation Alder Lake Client Platform/AlderLake-N Ddr5 CRB, BIOS VS01AMX80102 11/11/2024Mar 16 08:05:45 localhost kernel: [20795758.109963] Call Trace:Mar 16 08:05:45 localhost kernel: [20795758.109965] <TASK>Mar 16 08:05:45 localhost kernel: [20795758.109970] dump_stack_lvl+0x6c/0x9fMar 16 08:05:45 localhost kernel: [20795758.109980] dump_header+0x4a/0x1f3Mar 16 08:05:45 localhost kernel: [20795758.109986] oom_kill_process.cold+0xb/0x10Mar 16 08:05:45 localhost kernel: [20795758.109992] out_of_memory+0xed/0x3c0Mar 16 08:05:45 localhost kernel: [20795758.110001] __alloc_pages_slowpath.constprop.0+0x790/0xb30Mar 16 08:05:45 localhost kernel: [20795758.110019] __alloc_pages+0x32d/0x360Mar 16 08:05:45 localhost kernel: [20795758.110029] folio_alloc+0x17/0x50Mar 16 08:05:45 localhost kernel: [20795758.110034] __filemap_get_folio+0x17c/0x400Mar 16 08:05:45 localhost kernel: [20795758.110037] ? filemap_fault+0x472/0x8d0Mar 16 08:05:45 localhost kernel: [20795758.110048] filemap_fault+0x48e/0x8d0Mar 16 08:05:45 localhost kernel: [20795758.110053] ? filemap_map_pages+0x375/0x4e0Mar 16 08:05:45 localhost kernel: [20795758.110064] __do_fault+0x31/0x110Mar 16 08:05:45 localhost kernel: [20795758.110069] do_read_fault+0xea/0x150Mar 16 08:05:45 localhost kernel: [20795758.110073] do_fault+0x9a/0x2f0Mar 16 08:05:45 localhost kernel: [20795758.110078] __handle_mm_fault+0x3ae/0x690Mar 16 08:05:45 localhost kernel: [20795758.110095] handle_mm_fault+0x113/0x3e0Mar 16 08:05:45 localhost kernel: [20795758.110102] do_user_addr_fault+0x1ad/0x5c0Mar 16 08:05:45 localhost kernel: [20795758.110111] exc_page_fault+0x6e/0x1d0Mar 16 08:05:45 localhost kernel: [20795758.110117] asm_exc_page_fault+0x22/0x30Mar 16 08:05:45 localhost kernel: [20795758.110120] RIP: 0033:0x7f0c89a15431Mar 16 08:05:45 localhost kernel: [20795758.110130] Code: Unable to access opcode bytes at 0x7f0c89a15407.Mar 16 08:05:45 localhost kernel: [20795758.110131] RSP: 002b:00007f0bd37f57a8 EFLAGS: 00010202Mar 16 08:05:45 localhost kernel: [20795758.110135] RAX: 00007f0bc6f1ac50 RBX: 00007f0bd37f5980 RCX: 00007f0c818227d1Mar 16 08:05:45 localhost kernel: [20795758.110137] RDX: 0000000000000006 RSI: 00007f0c818227d1 RDI: 00007f0bc6f1ac50Mar 16 08:05:45 localhost kernel: [20795758.110139] RBP: 00007f0bd37f57d0 R08: 0000000000000000 R09: 0000000000000000Mar 16 08:05:45 localhost kernel: [20795758.110140] R10: 0000000000000000 R11: 00007f0c7a777480 R12: 0000000000000000Mar 16 08:05:45 localhost kernel: [20795758.110142] R13: 00000000013cc2f0 R14: 00007fff87a14e60 R15: 00007f0bd2ff6000Mar 16 08:05:45 localhost kernel: [20795758.110156] </TASK>Mar 16 08:05:45 localhost kernel: [20795758.110157] Mem-Info:Mar 16 08:05:45 localhost kernel: [20795758.110161] active_anon:398181 inactive_anon:3173671 isolated_anon:0#012 active_file:61 inactive_file:6569 isolated_file:6536#012 unevictable:2610 dirty:0 writeback:0#012 slab_reclaimable:51761 slab_unreclaimable:48533#012 mapped:278904 shmem:402652 pagetables:38092#012 sec_pagetables:0 bounce:0#012 kernel_misc_reclaimable:0#012 free:40032 free_pcp:15 free_cma:0Mar 16 08:05:45 localhost kernel: [20795758.110166] Node 0 active_anon:1592724kB inactive_anon:12694684kB active_file:244kB inactive_file:26276kB unevictable:10440kB isolated(anon):0kB isolated(file):26144kB mapped:1115616kB dirty:0kB writeback:0kB shmem:1610608kB shmem_thp: 8192kB shmem_pmdmapped: 0kB anon_thp: 9371648kB writeback_tmp:0kB kernel_stack:101280kB pagetables:152368kB sec_pagetables:0kB all_unreclaimable? noMar 16 08:05:45 localhost kernel: [20795758.110170] Node 0 DMA free:13320kB boost:0kB min:64kB low:80kB high:96kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15368kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kBMar 16 08:05:45 localhost kernel: [20795758.110176] lowmem_reserve[]: 0 693 14926 14926 14926Mar 16 08:05:45 localhost kernel: [20795758.110185] Node 0 DMA32 free:59992kB boost:0kB min:3136kB low:3920kB high:4704kB reserved_highatomic:0KB active_anon:33184kB inactive_anon:636228kB active_file:0kB inactive_file:116kB unevictable:0kB writepending:0kB present:1073968kB managed:746100kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kBMar 16 08:05:45 localhost kernel: [20795758.110190] lowmem_reserve[]: 0 0 14232 14232 14232Mar 16 08:05:45 localhost kernel: [20795758.110199] Node 0 Normal free:86816kB boost:0kB min:64376kB low:80468kB high:96560kB reserved_highatomic:22528KB active_anon:1559540kB inactive_anon:12058456kB active_file:0kB inactive_file:26668kB unevictable:10440kB writepending:0kB present:15462400kB managed:14583948kB mlocked:0kB bounce:0kB free_pcp:60kB local_pcp:0kB free_cma:0kBMar 16 08:05:45 localhost kernel: [20795758.110204] lowmem_reserve[]: 0 0 0 0 0Mar 16 08:05:45 localhost kernel: [20795758.110212] Node 0 DMA: 0*4kB 1*8kB (U) 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 2*2048kB (UM) 2*4096kB (M) = 13320kBMar 16 08:05:45 localhost kernel: [20795758.110270] Node 0 DMA32: 1212*4kB (UME) 715*8kB (UME) 437*16kB (UME) 382*32kB (UME) 150*64kB (UME) 49*128kB (UME) 22*256kB (UME) 5*512kB (UME) 4*1024kB (UME) 1*2048kB (M) 0*4096kB = 59992kBMar 16 08:05:45 localhost kernel: [20795758.110300] Node 0 Normal: 10318*4kB (UME) 3830*8kB (UME) 571*16kB (UME) 125*32kB (ME) 26*64kB (UM) 1*128kB (U) 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 86840kBMar 16 08:05:45 localhost kernel: [20795758.110325] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kBMar 16 08:05:45 localhost kernel: [20795758.110328] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kBMar 16 08:05:45 localhost kernel: [20795758.110330] 415878 total pagecache pagesMar 16 08:05:45 localhost kernel: [20795758.110332] 0 pages in swap cacheMar 16 08:05:45 localhost kernel: [20795758.110333] Free swap = 0kBMar 16 08:05:45 localhost kernel: [20795758.110335] Total swap = 0kBMar 16 08:05:45 localhost kernel: [20795758.110336] 4138090 pages RAMMar 16 08:05:45 localhost kernel: [20795758.110338] 0 pages HighMem/MovableOnlyMar 16 08:05:45 localhost kernel: [20795758.110339] 301736 pages reservedMar 16 08:05:45 localhost kernel: [20795758.110341] 0 pages hwpoisonedMar 16 08:05:45 localhost kernel: [20795758.110342] Tasks state (memory values in pages):Mar 16 08:05:45 localhost kernel: [20795758.110343] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj nameMar 16 08:05:45 localhost kernel: [20795758.110365] [ 611] 0 611 12234 4804 118784 0 -250 systemd-journalMar 16 08:05:45 localhost kernel: [20795758.110395] [ 641] 0 641 8160 727 86016 0 -1000 systemd-udevdMar 16 08:05:45 localhost kernel: [20795758.110402] [ 857] 32 857 3149 221 69632 0 0 rpcbindMar 16 08:05:45 localhost kernel: [20795758.110407] [ 867] 0 867 23285 739 65536 0 -1000 auditdMar 16 08:05:45 localhost kernel: [20795758.110411] [ 869] 0 869 1771 83 53248 0 0 sedispatchMar 16 08:05:45 localhost kernel: [20795758.110416] [ 889] 81 889 2299 252 61440 0 -900 dbus-daemonMar 16 08:05:45 localhost kernel: [20795758.110421] [ 897] 997 897 732 44 40960 0 0 lsmdMar 16 08:05:45 localhost kernel: [20795758.110426] [ 904] 990 904 77519 621 98304 0 0 polkitdMar 16 08:05:45 localhost kernel: [20795758.110430] [ 907] 0 907 116356 1032 139264 0 0 rngdMar 16 08:05:45 localhost kernel: [20795758.110435] [ 908] 0 908 2870 371 65536 0 0 smartdMar 16 08:05:45 localhost kernel: [20795758.110440] [ 913] 0 913 20018 620 61440 0 -500 irqbalanceMar 16 08:05:45 localhost kernel: [20795758.110445] [ 914] 0 914 12215 769 94208 0 0 systemd-logindMar 16 08:05:45 localhost kernel: [20795758.110449] [ 1013] 0 1013 3944 312 73728 0 -1000 sshdMar 16 08:05:45 localhost kernel: [20795758.110453] [ 1016] 0 1016 90404 5171 180224 0 0 tunedMar 16 08:05:45 localhost kernel: [20795758.110458] [ 1018] 0 1018 392486 6153 319488 0 -999 containerdMar 16 08:05:45 localhost kernel: [20795758.110463] [ 1048] 0 1048 18735 257 102400 0 0 gssproxyMar 16 08:05:45 localhost kernel: [20795758.110468] [ 1386] 0 1386 704 29 40960 0 0 fan_adjustMar 16 08:05:45 localhost kernel: [20795758.110473] [ 1762] 0 1762 347641 9165 376832 0 -500 dockerdMar 16 08:05:45 localhost kernel: [20795758.110477] [ 1765] 0 1765 138348 6232 536576 0 0 rsyslogdMar 16 08:05:45 localhost kernel: [20795758.110481] [ 1791] 0 1791 1002 49 45056 0 0 atdMar 16 08:05:45 localhost kernel: [20795758.110486] [ 1794] 0 1794 5906 213 73728 0 0 crondMar 16 08:05:45 localhost kernel: [20795758.110490] [ 1950] 1001 1950 294408 17646 270336 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110495] [ 1977] 1001 1977 9054 193 81920 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110499] [ 2106] 1001 2106 294533 255901 2220032 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110503] [ 2107] 1001 2107 294503 7848 196608 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110508] [ 2285] 0 2285 178112 2664 106496 0 -998 containerd-shimMar 16 08:05:45 localhost kernel: [20795758.110513] [ 2306] 0 2306 2357 704 49152 0 0 docker_sw_startMar 16 08:05:45 localhost kernel: [20795758.110517] [ 2505] 1001 2505 294408 4455 139264 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110522] [ 2507] 1001 2507 295985 979 159744 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110526] [ 2508] 1001 2508 294956 514 126976 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110531] [ 4591] 0 4591 1730 58 57344 0 0 bashMar 16 08:05:45 localhost kernel: [20795758.110536] [ 4833] 0 4833 86993 1887 159744 0 0 NetworkManagerMar 16 08:05:45 localhost kernel: [20795758.110540] [ 7180] 1000 7180 2487 555 61440 0 0 mosquittoMar 16 08:05:45 localhost kernel: [20795758.110545] [ 7332] 0 7332 44903 4682 184320 0 0 redis-serverMar 16 08:05:45 localhost kernel: [20795758.110549] [ 7353] 0 7353 1536 44 57344 0 0 UProxyMar 16 08:05:45 localhost kernel: [20795758.110554] [ 7364] 0 7364 5761 73 69632 0 0 shMar 16 08:05:45 localhost kernel: [20795758.110559] [ 7369] 0 7369 5794 93 69632 0 0 shMar 16 08:05:45 localhost kernel: [20795758.110563] [ 7395] 0 7395 895162 41531 700416 0 0 javaMar 16 08:05:45 localhost kernel: [20795758.110568] [ 7422] 0 7422 5761 75 65536 0 0 shMar 16 08:05:45 localhost kernel: [20795758.110573] [ 7427] 0 7427 5794 93 65536 0 0 shMar 16 08:05:45 localhost kernel: [20795758.110577] [ 7456] 0 7456 1816446 106963 3424256 0 0 javaMar 16 08:05:45 localhost kernel: [20795758.110581] [ 7528] 0 7528 1297830 143195 1572864 0 0 javaMar 16 08:05:45 localhost kernel: [20795758.110586] [ 7537] 0 7537 1227966 121635 1306624 0 0 javaMar 16 08:05:45 localhost kernel: [20795758.110590] [ 7595] 0 7595 117281 329 135168 0 0 diskmanagerMar 16 08:05:45 localhost kernel: [20795758.110596] [ 7666] 0 7666 317381 2095 225280 0 0 imgMar 16 08:05:45 localhost kernel: [20795758.110600] [ 7729] 0 7729 409338 4020 323584 0 0 midserverMar 16 08:05:45 localhost kernel: [20795758.110604] [ 7890] 0 7890 7417 1634 69632 0 0 openrestyMar 16 08:05:45 localhost kernel: [20795758.110609] [ 7909] 0 7909 8915 3010 81920 0 0 openrestyMar 16 08:05:45 localhost kernel: [20795758.110613] [ 7910] 0 7910 8617 2685 77824 0 0 openrestyMar 16 08:05:45 localhost kernel: [20795758.110618] [ 7912] 0 7912 8554 2684 77824 0 0 openrestyMar 16 08:05:45 localhost kernel: [20795758.110622] [ 7913] 0 7913 8841 2844 81920 0 0 openrestyMar 16 08:05:45 localhost kernel: [20795758.110626] [ 7914] 0 7914 8585 2715 77824 0 0 openrestyMar 16 08:05:45 localhost kernel: [20795758.110630] [ 7915] 0 7915 8617 2685 77824 0 0 openrestyMar 16 08:05:45 localhost kernel: [20795758.110634] [ 7916] 0 7916 8617 2686 77824 0 0 openrestyMar 16 08:05:45 localhost kernel: [20795758.110638] [ 7917] 0 7917 8554 2684 77824 0 0 openrestyMar 16 08:05:45 localhost kernel: [20795758.110642] [ 7918] 0 7918 8554 2684 77824 0 0 openrestyMar 16 08:05:45 localhost kernel: [20795758.110646] [ 7920] 0 7920 8841 2845 81920 0 0 openrestyMar 16 08:05:45 localhost kernel: [20795758.110650] [ 7989] 1001 7989 295231 2641 176128 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110655] [ 8075] 1001 8075 295109 1853 172032 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110659] [ 8076] 1001 8076 295109 1838 172032 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110663] [ 8077] 1001 8077 296374 143879 2220032 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110667] [ 8078] 1001 8078 296037 17909 524288 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110671] [ 8079] 1001 8079 295933 24207 1679360 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110675] [ 8080] 1001 8080 295910 24409 1679360 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110679] [ 8081] 1001 8081 295909 24221 1675264 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110683] [ 8082] 1001 8082 295936 24043 1675264 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110688] [ 8083] 1001 8083 296595 25024 1675264 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110692] [ 8084] 1001 8084 295890 24322 1679360 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110696] [ 8085] 1001 8085 296260 24535 1679360 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110700] [ 8086] 1001 8086 295903 24256 1683456 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110704] [ 8087] 1001 8087 295939 24210 1679360 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110708] [ 8088] 1001 8088 296514 25275 1675264 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110712] [ 8151] 0 8151 65183 1610 819200 0 0 httpdMar 16 08:05:45 localhost kernel: [20795758.110716] [ 8152] 0 8152 2165 41 61440 0 0 rotatelogsMar 16 08:05:45 localhost kernel: [20795758.110720] [ 8153] 0 8153 2165 40 65536 0 0 rotatelogsMar 16 08:05:45 localhost kernel: [20795758.110724] [ 8235] 0 8235 537091 4428 970752 0 0 dmcserverMar 16 08:05:45 localhost kernel: [20795758.110729] [ 8284] 0 8284 473827 12887 1187840 0 0 bmcserverMar 16 08:05:45 localhost kernel: [20795758.110733] [ 8369] 0 8369 700 26 45056 0 0 DiskReadOnlyCheMar 16 08:05:45 localhost kernel: [20795758.110737] [ 8437] 0 8437 271579 2464 311296 0 0 cds_adapterMar 16 08:05:45 localhost kernel: [20795758.110741] [ 8594] 0 8594 629602 19495 1323008 0 0 cdmserverMar 16 08:05:45 localhost kernel: [20795758.110745] [ 8865] 0 8865 652309 54355 1183744 0 0 civetwebMar 16 08:05:45 localhost kernel: [20795758.110749] [ 8921] 0 8921 701 30 49152 0 0 cdmdaemonMar 16 08:05:45 localhost kernel: [20795758.110753] [ 9315] 0 9315 177202 2544 458752 0 0 ndserverMar 16 08:05:45 localhost kernel: [20795758.110757] [ 10401] 0 10401 701 29 45056 0 0 bmdaemonMar 16 08:05:45 localhost kernel: [20795758.110762] [ 10598] 0 10598 125068 23306 1114112 0 0 cluserverMar 16 08:05:45 localhost kernel: [20795758.110766] [ 10825] 0 10825 129149 29441 1257472 0 0 cdvserverMar 16 08:05:45 localhost kernel: [20795758.110770] [ 11263] 0 11263 579468 36247 1228800 0 0 cds_ndserverMar 16 08:05:45 localhost kernel: [20795758.110774] [ 11973] 0 11973 235425 53433 1933312 0 0 dpserverMar 16 08:05:45 localhost kernel: [20795758.110778] [ 12775] 0 12775 681596 25763 1282048 0 0 vodserverMar 16 08:05:45 localhost kernel: [20795758.110783] [ 12789] 1001 12789 295109 1869 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110787] [ 12801] 1001 12801 295109 1869 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110791] [ 12812] 1001 12812 295109 2110 176128 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110795] [ 13052] 0 13052 701 28 45056 0 0 voddaemonMar 16 08:05:45 localhost kernel: [20795758.110799] [ 13938] 0 13938 754396 44391 1687552 0 0 msserverMar 16 08:05:45 localhost kernel: [20795758.110804] [ 14157] 0 14157 180046 2630 98304 0 0 g_ga_msauxiliarMar 16 08:05:45 localhost kernel: [20795758.110808] [ 14959] 0 14959 610051 3289 1118208 0 0 msproxyMar 16 08:05:45 localhost kernel: [20795758.110812] [ 15433] 0 15433 738984 35862 753664 0 0 bus_all_in_oneMar 16 08:05:45 localhost kernel: [20795758.110817] [ 15674] 1001 15674 295284 2980 180224 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110821] [ 15680] 1001 15680 295272 2668 180224 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110825] [ 15709] 1001 15709 295141 2682 180224 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110830] [ 15805] 0 15805 415746 9413 401408 0 0 lcsMar 16 08:05:45 localhost kernel: [20795758.110834] [ 16086] 0 16086 790608 32680 1638400 0 0 bwserverMar 16 08:05:45 localhost kernel: [20795758.110838] [ 16092] 1001 16092 295109 1869 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110842] [ 16094] 1001 16094 295109 1870 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110846] [ 16095] 1001 16095 295314 3421 221184 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110850] [ 16425] 0 16425 781645 21245 1486848 0 0 bmserverMar 16 08:05:45 localhost kernel: [20795758.110854] [ 16468] 1001 16468 295109 1885 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110858] [ 16473] 1001 16473 295268 2947 188416 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110862] [ 16478] 1001 16478 295286 3405 208896 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110866] [ 16953] 0 16953 372858 8413 323584 0 0 distproxyMar 16 08:05:45 localhost kernel: [20795758.110871] [ 17664] 0 17664 355158 7819 303104 0 0 mdmMar 16 08:05:45 localhost kernel: [20795758.110875] [ 17950] 1001 17950 295765 3279 204800 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110879] [ 18248] 0 18248 481970 4873 253952 0 0 serviceManagerMar 16 08:05:45 localhost kernel: [20795758.110883] [ 18710] 0 18710 1554235 608524 6041600 0 0 javaMar 16 08:05:45 localhost kernel: [20795758.110887] [ 18726] 0 18726 1160116 231414 2928640 0 0 javaMar 16 08:05:45 localhost kernel: [20795758.110891] [ 18802] 0 18802 5794 110 73728 0 0 daemonMar 16 08:05:45 localhost kernel: [20795758.110896] [ 19632] 0 19632 701 29 45056 0 0 msdaemonMar 16 08:05:45 localhost kernel: [20795758.110900] [ 19798] 0 19798 701 29 49152 0 0 cdvdaemonMar 16 08:05:45 localhost kernel: [20795758.110904] [ 21015] 0 21015 225502 13453 1638400 0 0 swmqserverMar 16 08:05:45 localhost kernel: [20795758.110909] [ 21144] 0 21144 1188705 381284 3477504 0 0 javaMar 16 08:05:45 localhost kernel: [20795758.110913] [ 21265] 0 21265 499879 204843 1724416 0 0 g_ga_mapserviceMar 16 08:05:45 localhost kernel: [20795758.110918] [ 21371] 1001 21371 295086 1984 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110922] [ 21410] 1001 21410 295086 1904 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110926] [ 21449] 0 21449 179657 3852 135168 0 0 optlogserverMar 16 08:05:45 localhost kernel: [20795758.110930] [ 21470] 1001 21470 296441 10898 438272 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110934] [ 21569] 0 21569 179181 3031 122880 0 0 publicgoserverMar 16 08:05:45 localhost kernel: [20795758.110938] [ 21673] 0 21673 1024603 122329 1310720 0 0 javaMar 16 08:05:45 localhost kernel: [20795758.110942] [ 21897] 0 21897 499866 204597 1728512 0 0 g_ga_exbusinessMar 16 08:05:45 localhost kernel: [20795758.110946] [ 21926] 1001 21926 295598 2533 184320 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110950] [ 21931] 1001 21931 295086 1904 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110955] [ 21965] 1001 21965 295086 1903 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110959] [ 22046] 0 22046 309961 4690 131072 0 0 mapmserverMar 16 08:05:45 localhost kernel: [20795758.110963] [ 22059] 1001 22059 295086 1887 163840 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110967] [ 22077] 1001 22077 295086 1887 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110971] [ 22084] 1001 22084 295086 1904 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110975] [ 22181] 0 22181 978858 105713 1921024 0 0 FuseVMMar 16 08:05:45 localhost kernel: [20795758.110980] [ 22224] 1001 22224 297580 19897 1298432 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110984] [ 22225] 1001 22225 296104 10529 290816 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110988] [ 22240] 1001 22240 295232 3048 196608 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110992] [ 22241] 1001 22241 295270 3557 221184 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.110996] [ 22244] 1001 22244 300995 38297 1634304 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111000] [ 22246] 0 22246 637 30 45056 0 0 swdaemonMar 16 08:05:45 localhost kernel: [20795758.111004] [ 22253] 1001 22253 295077 2036 176128 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111008] [ 22255] 1001 22255 295077 2036 176128 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111012] [ 22259] 1001 22259 302652 148513 2269184 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111016] [ 22265] 1001 22265 295320 4315 233472 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111020] [ 22277] 1001 22277 298426 25850 1380352 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111024] [ 22301] 0 22301 5486 24 49152 0 0 agettyMar 16 08:05:45 localhost kernel: [20795758.111028] [ 22304] 0 22304 5391 24 69632 0 0 agettyMar 16 08:05:45 localhost kernel: [20795758.111033] [ 22326] 1001 22326 302585 147409 2269184 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111037] [ 22327] 1001 22327 302613 148283 2273280 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111041] [ 22328] 1001 22328 302652 147711 2269184 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111045] [ 22371] 1001 22371 297588 19948 1302528 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111049] [ 23414] 1001 23414 295137 2898 176128 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111053] [ 25842] 0 25842 633913 18989 946176 0 0 kbserverMar 16 08:05:45 localhost kernel: [20795758.111057] [ 25924] 1001 25924 295109 1906 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111061] [ 25931] 1001 25931 295287 2996 184320 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111065] [ 31510] 1001 31510 295305 2912 184320 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111070] [ 37077] 0 37077 172679 1706 192512 0 0 imosproxyMar 16 08:05:45 localhost kernel: [20795758.111074] [ 130893] 1001 130893 347615 274339 2637824 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111078] [ 131801] 1001 131801 298414 25843 1384448 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111082] [ 195120] 1001 195120 298428 26223 1380352 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111086] [ 867389] 1001 867389 295779 3458 212992 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111090] [1701237] 1001 1701237 301287 37850 1605632 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111095] [1746158] 1001 1746158 295822 4113 229376 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111099] [1780926] 1001 1780926 295824 4225 229376 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111103] [1780927] 1001 1780927 295826 4223 229376 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111108] [2107176] 1001 2107176 298425 26472 1380352 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111112] [1349864] 1001 1349864 296183 8463 368640 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111116] [1358557] 1001 1358557 296440 11377 471040 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111120] [1358558] 1001 1358558 296475 14077 610304 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111124] [2662623] 1001 2662623 295829 4313 233472 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111129] [1896897] 1001 1896897 296490 14041 602112 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111133] [2851395] 1001 2851395 302535 142378 2265088 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111137] [3920394] 0 3920394 99206 5100 806912 0 0 httpdMar 16 08:05:45 localhost kernel: [20795758.111142] [4056644] 1001 4056644 295864 3219 208896 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111146] [1528477] 0 1528477 99206 5025 806912 0 0 httpdMar 16 08:05:45 localhost kernel: [20795758.111151] [1951152] 0 1951152 99206 5089 811008 0 0 httpdMar 16 08:05:45 localhost kernel: [20795758.111155] [2156339] 1001 2156339 295999 12381 303104 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111159] [2801882] 0 2801882 99206 5024 806912 0 0 httpdMar 16 08:05:45 localhost kernel: [20795758.111163] [2801884] 0 2801884 99208 5064 806912 0 0 httpdMar 16 08:05:45 localhost kernel: [20795758.111167] [3962717] 0 3962717 98861 4799 802816 0 0 httpdMar 16 08:05:45 localhost kernel: [20795758.111171] [3966426] 0 3966426 98948 4893 802816 0 0 httpdMar 16 08:05:45 localhost kernel: [20795758.111175] [4073206] 0 4073206 98951 4885 802816 0 0 httpdMar 16 08:05:45 localhost kernel: [20795758.111179] [4077955] 0 4077955 98950 4894 802816 0 0 httpdMar 16 08:05:45 localhost kernel: [20795758.111183] [4080255] 0 4080255 172679 1728 188416 0 0 imosproxyMar 16 08:05:45 localhost kernel: [20795758.111188] [4080313] 0 4080313 98886 4830 802816 0 0 httpdMar 16 08:05:45 localhost kernel: [20795758.111192] [1563378] 1001 1563378 300117 29892 1413120 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111196] [ 117657] 1001 117657 295803 3503 221184 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111200] [ 118665] 1001 118665 295924 9664 376832 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111205] [ 118703] 1001 118703 295923 10021 401408 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111209] [ 118709] 1001 118709 295925 9454 376832 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111213] [ 759692] 1001 759692 320730 178855 2420736 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111218] [ 761071] 1001 761071 297013 17185 921600 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111222] [ 762477] 1001 762477 320711 174852 2424832 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111226] [ 762479] 1001 762479 320848 177501 2433024 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111230] [ 762481] 1001 762481 320875 176896 2428928 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111235] [ 819690] 1001 819690 306136 61075 2117632 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111239] [1691135] 1001 1691135 295816 3521 221184 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111258] [2297110] 1001 2297110 303952 59286 2097152 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111262] [2297111] 1001 2297111 304757 60801 2105344 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111266] [2090661] 1001 2090661 295786 8607 274432 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111271] [2090697] 1001 2090697 295793 8667 274432 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111275] [2095915] 1001 2095915 304494 59123 2101248 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111279] [2103683] 1001 2103683 296973 14091 757760 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111283] [1258007] 1001 1258007 297166 6839 311296 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111288] [1507253] 1001 1507253 297142 7796 327680 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111292] [3671410] 986 3671410 2878 153 57344 0 0 chronydMar 16 08:05:45 localhost kernel: [20795758.111297] [3401842] 1001 3401842 301072 50944 2064384 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111301] [ 493890] 1001 493890 299284 21830 1593344 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111305] [1389678] 1001 1389678 296310 3005 192512 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111309] [1992166] 1001 1992166 296387 9763 356352 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111313] [1817736] 1001 1817736 296271 3055 200704 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111317] [2481381] 1001 2481381 296165 2976 188416 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111321] [2814438] 1001 2814438 296388 8759 348160 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111325] [1582756] 0 1582756 480393 2075 249856 0 0 fw_baseMar 16 08:05:45 localhost kernel: [20795758.111330] [3030152] 1001 3030152 296330 4351 262144 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111334] [3452719] 1001 3452719 296353 3391 204800 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111338] [3722720] 1001 3722720 296362 3306 188416 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111343] [3902570] 1001 3902570 296323 3354 204800 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111348] [3983564] 1001 3983564 296339 5364 311296 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111352] [3983599] 1001 3983599 296286 3075 200704 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111356] [3992712] 1001 3992712 296298 3334 204800 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111361] [3992715] 1001 3992715 296295 3170 188416 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111365] [4025312] 0 4025312 996 61 40960 0 0 shMar 16 08:05:45 localhost kernel: [20795758.111369] [4025315] 0 4025315 5761 71 69632 0 0 shMar 16 08:05:45 localhost kernel: [20795758.111374] [4025316] 0 4025316 5761 72 65536 0 0 shMar 16 08:05:45 localhost kernel: [20795758.111378] [4025318] 0 4025318 5761 71 65536 0 0 shMar 16 08:05:45 localhost kernel: [20795758.111382] [4025320] 0 4025320 5761 72 69632 0 0 shMar 16 08:05:45 localhost kernel: [20795758.111386] [4025321] 0 4025321 5761 72 65536 0 0 shMar 16 08:05:45 localhost kernel: [20795758.111390] [4025324] 0 4025324 5761 70 73728 0 0 shMar 16 08:05:45 localhost kernel: [20795758.111395] [4025325] 0 4025325 6633 411 77824 0 0 psMar 16 08:05:45 localhost kernel: [20795758.111399] [4025326] 0 4025326 5623 32 73728 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111403] [4025327] 0 4025327 5590 32 69632 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111407] [4025328] 0 4025328 5590 33 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111411] [4025329] 0 4025329 5590 31 69632 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111415] [4025330] 0 4025330 5590 31 69632 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111419] [4025331] 0 4025331 5590 32 69632 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111424] [4025332] 0 4025332 5590 32 61440 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111428] [4025333] 0 4025333 5590 32 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111432] [4025334] 0 4025334 6246 61 69632 0 0 awkMar 16 08:05:45 localhost kernel: [20795758.111436] [4025335] 0 4025335 5453 11 69632 0 0 xargsMar 16 08:05:45 localhost kernel: [20795758.111440] [4025336] 0 4025336 6673 474 69632 0 0 psMar 16 08:05:45 localhost kernel: [20795758.111444] [4025337] 0 4025337 6673 475 65536 0 0 psMar 16 08:05:45 localhost kernel: [20795758.111448] [4025338] 0 4025338 5623 32 61440 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111452] [4025339] 0 4025339 6673 475 77824 0 0 psMar 16 08:05:45 localhost kernel: [20795758.111456] [4025340] 0 4025340 5623 32 69632 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111460] [4025341] 0 4025341 5590 33 69632 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111463] [4025342] 0 4025342 5590 32 73728 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111468] [4025343] 0 4025343 5590 32 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111472] [4025344] 0 4025344 6673 474 69632 0 0 psMar 16 08:05:45 localhost kernel: [20795758.111476] [4025345] 0 4025345 5590 31 61440 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111480] [4025346] 0 4025346 5590 33 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111485] [4025347] 0 4025347 5590 33 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111489] [4025348] 0 4025348 5590 32 61440 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111492] [4025349] 0 4025349 5590 33 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111496] [4025350] 0 4025350 5590 33 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111501] [4025351] 0 4025351 5590 32 73728 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111505] [4025352] 0 4025352 5590 33 69632 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111509] [4025353] 0 4025353 5590 31 61440 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111513] [4025354] 0 4025354 6246 62 73728 0 0 awkMar 16 08:05:45 localhost kernel: [20795758.111517] [4025355] 0 4025355 5590 32 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111521] [4025356] 0 4025356 5453 23 61440 0 0 xargsMar 16 08:05:45 localhost kernel: [20795758.111525] [4025357] 0 4025357 5590 33 61440 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111529] [4025358] 0 4025358 5590 32 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111533] [4025359] 0 4025359 5590 32 61440 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111538] [4025360] 0 4025360 5590 33 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111542] [4025361] 0 4025361 5590 32 61440 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111546] [4025362] 0 4025362 6246 61 69632 0 0 awkMar 16 08:05:45 localhost kernel: [20795758.111550] [4025363] 0 4025363 5590 33 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111554] [4025364] 0 4025364 5590 33 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111558] [4025365] 0 4025365 5453 24 65536 0 0 xargsMar 16 08:05:45 localhost kernel: [20795758.111562] [4025366] 0 4025366 5590 32 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111566] [4025367] 0 4025367 5590 32 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111570] [4025368] 0 4025368 5590 33 69632 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111574] [4025369] 0 4025369 6246 61 69632 0 0 awkMar 16 08:05:45 localhost kernel: [20795758.111578] [4025370] 0 4025370 5590 32 69632 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111582] [4025371] 0 4025371 5453 23 65536 0 0 xargsMar 16 08:05:45 localhost kernel: [20795758.111587] [4025372] 0 4025372 5590 31 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111591] [4025373] 0 4025373 5590 33 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111595] [4025374] 0 4025374 5590 32 69632 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111600] [4025375] 0 4025375 5590 32 69632 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111604] [4025376] 0 4025376 5590 33 73728 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111608] [4025377] 0 4025377 5590 33 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111612] [4025378] 0 4025378 6246 62 69632 0 0 awkMar 16 08:05:45 localhost kernel: [20795758.111617] [4025379] 0 4025379 5453 23 69632 0 0 xargsMar 16 08:05:45 localhost kernel: [20795758.111621] [4025381] 0 4025381 674 19 40960 0 0 catMar 16 08:05:45 localhost kernel: [20795758.111625] [4025385] 1001 4025385 295481 851 155648 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111629] [4025386] 1001 4025386 296111 2080 176128 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111633] [4025387] 0 4025387 6222 223 65536 0 0 crondMar 16 08:05:45 localhost kernel: [20795758.111638] [4025388] 0 4025388 1385 44 45056 0 0 psMar 16 08:05:45 localhost kernel: [20795758.111642] [4025389] 0 4025389 826 26 49152 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111646] [4025390] 0 4025390 826 27 49152 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111650] [4025391] 0 4025391 826 26 49152 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111654] [4025392] 0 4025392 826 25 49152 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111658] [4025393] 0 4025393 826 26 45056 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111662] [4025394] 0 4025394 1957 37 45056 0 0 awkMar 16 08:05:45 localhost kernel: [20795758.111666] [4025395] 0 4025395 714 20 45056 0 0 xargsMar 16 08:05:45 localhost kernel: [20795758.111671] [4025396] 1001 4025396 295998 914 167936 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111675] [4025397] 0 4025397 5626 39 69632 0 0 lsMar 16 08:05:45 localhost kernel: [20795758.111679] [4025398] 0 4025398 5583 28 65536 0 0 grepMar 16 08:05:45 localhost kernel: [20795758.111683] [4025399] 0 4025399 6246 61 65536 0 0 awkMar 16 08:05:45 localhost kernel: [20795758.111687] [4025409] 0 4025409 5748 64 69632 0 0 bashMar 16 08:05:45 localhost kernel: [20795758.111691] [4025410] 1001 4025410 295421 463 106496 0 0 postgresMar 16 08:05:45 localhost kernel: [20795758.111695] [4025411] 0 4025411 934 19 40960 0 0 pgrepMar 16 08:05:45 localhost kernel: [20795758.111699] [4025414] 0 4025414 1053 27 49152 0 0 mp_recovdiskspaMar 16 08:05:45 localhost kernel: [20795758.111704] [4025415] 0 4025415 412 10 32768 0 0 shMar 16 08:05:45 localhost kernel: [20795758.111708] [4025417] 0 4025417 5906 213 61440 0 0 crondMar 16 08:05:45 localhost kernel: [20795758.111711] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/rc-local.service,task=java,pid=18710,uid=0Mar 16 08:05:45 localhost kernel: [20795758.117181] Out of memory: Killed process 18710 (java) total-vm:6216940kB, anon-rss:2434136kB, file-rss:0kB, shmem-rss:96kB, UID:0 pgtables:5900kB oom_score_adj:0Mar 16 08:05:45 localhost kernel: [20795760.247701] oom_reaper: reaped process 18710 (java), now anon-rss:0kB, file-rss:0kB, shmem-rss:32kBMar 16 08:05:45 localhost chronyd[3671410]: Forward time jump detected!Mar 16 08:05:46 localhost chronyd[3671410]: Forward time jump detected!Mar 16 08:05:46 localhost chronyd[3671410]: Forward time jump detected!Mar 16 08:05:46 localhost chronyd[3671410]: Forward time jump detected!Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Face-Change-751687893218492419 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Device-Change-751687901808427014 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-CallSignal-Vis-751687901808427008 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Device-Change-751687897513459721 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Face-Change-751687897513459717 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Person-Change-751687893218492417 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-PeopleCount-Subscribe-751687893218492416 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-PeopleCount-DataPush-751687888923525120 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Person-Change-751687884628557825 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Person-Change-751687880333590529 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-PersonDept-Change-751687884628557824 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Holiday-Change-751687880333590528 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Device-Change-751687876038623233 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Device-Change-751687876038623232 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-cloudEdge-permission-record-update-751687871743655938 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Cloud-Device-Push-751687871743655937 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Cloud-Org-Push-751687871743655936 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Person-Change-751687867448688642 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-PassRecord-Push-751687867448688641 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Device-Service-Offline-751687867448688640 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Client-Unsubscribe-751687863153721346 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Device-KeepAlive-751687863153721345 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-AcsConfig-Update-751687863153721344 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Holiday-Change-751687858858754050 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-cloudEdge-permission-update-751687858858754049 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client System-Epidemic-Cfg-751687858858754048 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-CloudEdge-Org-Syn-751687854563786754 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-PassRecord-Sync-751687854563786753 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Device-Capture-751687854563786752 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Device-Change-751687850268819457 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Device-Change-751687850268819456 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-CallSignal-Change-751687845973852160 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Device-Change-751687828793982977 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Vehicle-Match-751687828793982976 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Relation-Change-751687824499015682 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Channel-Change-Alarm-751687824499015681 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Resource-Channel-Change-751687824499015680 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Producer-256401864393 closed its connection.Mar 16 08:05:46 localhost mosquitto[7180]: 2026-03-16 08:05:44: Client Client_17736179128861 closed its connection.Mar 16 08:05:47 localhost systemd[1]: Starting system activity accounting tool...Mar 16 08:05:47 localhost systemd[1]: Starting dnf makecache...Mar 16 08:05:47 localhost systemd[1]: sysstat-collect.service: Deactivated successfully.Mar 16 08:05:47 localhost systemd[1]: Finished system activity accounting tool.分析下 08:05:45 附近发生了什么事情?
可以看到 08:00:45 这个时间后生了 oom-killer, 并且将最消耗内存的 Java 进程给 kill 掉了.
07:40 附近内存已经比较少了, 然后 OS 发生 scank 与 scand 进行 page 页面扫描以回收内存, 应用被阻塞了 ,系统 hang 住了.
回收内存之后发现内存还是比较紧张, 然后继续扫描, 并达到 oom killer 阈值 , 进而触发 oom killer.
从之前的 sar -B 结果可以看到
我需要解释%vmeff的含义以及2.71%这个数值的意义。%vmeff是内存扫描效率,表示每秒扫描的页数与实际回收页数的比率。通常这个数值应该在合理的范围内,2.71%确实很低,说明内存回收效率很差。
让我详细解释这个概念。
%vmeff=2.71%确实表明内存扫描效率极低,这解释了为什么页面扫描(pgsweep/pgscan)无法有效缓解内存压力。
%vmeff(内存扫描效率)表示:
这个低效率解释了整个事件链:
ounter(line内存压力 → 页面扫描(低效) → 回收少量内存 → 压力持续 → OOM触发