添加Linux crontab任务执行Shell脚本生成txt文件并使用splunkforwarder监控

Shell脚本保存路径

Shell脚本保存路径为：/home/SHunicom/prometheus/kubernetes/yaml/prometheus/new

[SHunicom@haier-test-01 new]$ ll *.sh
-rwxrwxr-- 1 SHunicom SHunicom 381 Nov 20 09:18 data_namespaces.sh
-rwxrwxr-- 1 SHunicom SHunicom 112 Nov 20 09:21 data_nodes.sh
-rwxrwxr-- 1 SHunicom SHunicom 131 Nov 20 09:22 data_service.sh

Shell 脚本内容

[SHunicom@haier-test-01 new]$ cat data_namespaces.sh
#!/bin/bash
cd /home/SHunicom/prometheus/kubernetes/yaml/prometheus/new
for namespace in `kubectl get namespace|tail -n +2|awk '{print $1}'`;do
    for name in `kubectl get quota --namespace=$namespace|tail -n +2|awk '{print $1}'`;do
kubectl describe quota $name --namespace=$namespace >> /home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/monitor_namespaces01.txt
done
done

[SHunicom@haier-test-01 new]$ cat data_nodes.sh 
#!/bin/bash
#
kubectl get nodes >> /home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/monitor_nodes01.txt

[SHunicom@haier-test-01 new]$ cat data_service.sh 
#!/bin/bash
kubectl get service --all-namespaces >> /home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/monitor_service01.txt

添加执行权限

cd /home/SHunicom/prometheus/kubernetes/yaml/prometheus/new
chmode +x *.sh

添加Crontab定时计划

[SHunicom@haier-test-01 new]$ crontab -e 
# 添加定时计划
30 3 * * *  /home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/data_namespaces.sh
30 3 * * *  /home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/data_nodes.sh
30 3 * * *  /home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/data_service.sh
# 每天凌晨3点30分执行一次

[SHunicom@haier-test-01 new]$ crontab -l  
# 查看定时计划
30 3 * * *  /home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/data_namespaces.sh
30 3 * * *  /home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/data_nodes.sh
30 3 * * *  /home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/data_service.sh

Splunk 配置

添加索引、管理服务器

$SPLUNK_HOME/bin/splunk add forward-server 192.168.10.12:9997  
# 添加目标索引服务器
$SPLUNK_HOME/bin/splunk set deploy-poll 192.168.10.9:8089  
# 添加管理服务器

修改inputs.conf配置文件

路径为：$SPLUNK_HOME/ets/system/local/inputs.conf

inputs.conf文件内容如下

[SHunicom@haier-test-01 local]$ cat inputs.conf 
[default]
host = haier-test-01

[monitor:///home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/monitor_namespaces01.txt]
index = namespace_resourcequota
sourcetype = namespace_resourcequota
disabled = false

[monitor:///home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/monitor_nodes01.txt]
index = prometheus_monitor
sourcetype = monitoring_nodes
disabled = false

[monitor:///home/SHunicom/prometheus/kubernetes/yaml/prometheus/new/monitor_service01.txt]
index = prometheus_monitor
sourcetype = monitor_service
disabled = false

添加完成后需要重启splunkforwarder服务

[SHunicom@haier-test-01 local]$ $SPLUNK_HOME/bin/splunk restart