如何使用 jq 在连续的 json 记录流上调用操作

发布于04月17日

因此，我有一个每秒向stdout写入一条JSON记录的进程，如何使用JQ在写入一定数量(=x)的记录后触发操作，并且仍然每秒输出一次？

The process is, to allow easy replay, replaced by a bash shell read loop with a sleep:
( read line ; while [[ -n "${line}" ]] ; do echo "${line}"; sleep 1 ; read line ; done ) < json.data

X个记录之后的操作是计算记录数，并忽略具有"NULL"值的行.

当x为1时，所以处理每一行，解决方案很简单或直接.

( read line ; while [[ -n "${line}" ]] ; do echo "${line}"; sleep 1 ; read line ; done ) < json.data | \
jq -c '. , ([.] | del( .[] | select(to_entries[].value == null)) | length)'
{"date":230415072207,"kwL1Tl":0,"kwL1":0.022,"kwL2Tl":0,"kwL2":0.075,"kwL3Tl":0,"kwL3":0}
1
{"date":230415072311,"kwL1Tl":0,"kwL1":0.023,"kwL2Tl":0,"kwL2":0.08,"kwL3Tl":0,"kwL3":0}  
1
{"date":230415072312,"kwL1Tl":null,"kwL1":null,"kwL2Tl":null,"kwL2":null,"kwL3Tl":null,"kwL3":null}  
0
{"date":230415072415,"kwL1Tl":0,"kwL1":0.023,"kwL2Tl":0,"kwL2":0.08,"kwL3Tl":0,"kwL3":0}  
1
{"date":230415072416,"kwL1Tl":0,"kwL1":0.022,"kwL2Tl":0,"kwL2":0.08,"kwL3Tl":0,"kwL3":0}  
1
...

但是，当我想对每5或100个记录执行此操作时，该怎么办？

X=5的预期输出:

$ ( read line ; while [[ -n "${line}" ]] ; do echo "${line}"; sleep 1 ; read line ; done ) < json.data | \
alert
{"date":230415072207,"kwL1Tl":0,"kwL1":0.022,"kwL2Tl":0,"kwL2":0.075,"kwL3Tl":0,"kwL3":0}
{"date":230415072311,"kwL1Tl":0,"kwL1":0.023,"kwL2Tl":0,"kwL2":0.08,"kwL3Tl":0,"kwL3":0}
{"date":230415072312,"kwL1Tl":null,"kwL1":null,"kwL2Tl":null,"kwL2":null,"kwL3Tl":null,"kwL3":null}
{"date":230415072415,"kwL1Tl":0,"kwL1":0.023,"kwL2Tl":0,"kwL2":0.08,"kwL3Tl":0,"kwL3":0}
{"date":230415072416,"kwL1Tl":0,"kwL1":0.022,"kwL2Tl":0,"kwL2":0.08,"kwL3Tl":0,"kwL3":0}
4
{"date":230415072519,"kwL1Tl":0,"kwL1":0.021,"kwL2Tl":0,"kwL2":0.08,"kwL3Tl":0,"kwL3":0}
{"date":230415072520,"kwL1Tl":0,"kwL1":0.021,"kwL2Tl":0,"kwL2":0.08,"kwL3Tl":0,"kwL3":0}
{"date":230415072623,"kwL1Tl":0,"kwL1":0.022,"kwL2Tl":0,"kwL2":0.075,"kwL3Tl":0,"kwL3":0}
{"date":230415072624,"kwL1Tl":0,"kwL1":0.022,"kwL2Tl":0,"kwL2":0.075,"kwL3Tl":0,"kwL3":0}
{"date":230415072727,"kwL1Tl":0,"kwL1":0.022,"kwL2Tl":0,"kwL2":0.076,"kwL3Tl":0,"kwL3":0}
5
{"date":230415072728,"kwL1Tl":0,"kwL1":0.023,"kwL2Tl":0,"kwL2":0.076,"kwL3Tl":0,"kwL3":0}
{"date":230415072831,"kwL1Tl":0,"kwL1":0.023,"kwL2Tl":0,"kwL2":0.075,"kwL3Tl":0,"kwL3":0}
...

Bash函数"ALERT"应该包含一个正在运行的JQ命令.

alert(){ # # send alerts if needed jq -nc --argjson m "5" ' def alerts(r): r | del( .[] | select(to_entries[].value == null)) | length ; foreach inputs as $in ( {"recs":[]}; if (.recs|length) < $m then .recs += [$in] else .recs=[$in] end; .recs[-1], if (.recs|length) == $m then alerts(.recs) else empty end) ' }

如何使用 jq 在连续的 json 记录流上调用操作

推荐答案

Json相关问答推荐

Jolt转换，如果任何字段为空，则将对象值设置为空

如何使用表键名称GROUP_BY

写入JSON文件的流

使用PowerShell解析文件并获取特定行的值

在 python 中循环 JSON 数组

如何使用Powershell查找所有包含特定键值对的JSON对象并迭代所有对象？

Rust实现：高效解析任意大小的JSON数组

如何在 jq 中按 IP 地址排序？

Jolt 规范将父对象中的所有键应用于数组中的所有对象

使用 Groovy 将 XML 转换为 JSON

如何加入或合并列表元素列表(未知长度)

jq：用列表包装所有第一级原子值

在 Perl Mojolicious 中呈现 JSON 时防止转义字符

PowerShell - 如何迭代 PSCustomObject 嵌套对象？

可以通过 POST 使用 EventSource 传递参数的服务器发送事件 (SSE)

如何使用 Jackson 注释从 HttpResponse 反序列化 JSON 对象？

在浏览器中查看 JSON 文件

使用 JSON.Net 的 C# 到 JSON 序列化

如何对 jq 中的 map 数组中的值求和？

FastAPI：如何将正文读取为任何有效的 json？