如下图所示
在Graylog集群环境上,日志源将日志发送到负载均衡,由负载均衡来分配日志发送到GrayLog具体哪个节点上,实现各节点接入能力和处理能力平衡均分
下面采用nginx作为GrayLog 日志接入的负载均衡
1、配置nginx 的yum源
参考nginx官方文档
http://nginx.org/en/linux_packages.html#RHEL
具体命令如下
vim /etc/yum.repos.d/nginx.repo
添加如下行
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true
[nginx-mainline]
name=nginx mainline repo
baseurl=http://nginx.org/packages/mainline/centos/$releasever/$basearch/
gpgcheck=0
enabled=0
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true
安装nginx
yum install nginx --enablerepo=nginx-stable --disablerepo=epel
2、配置nginx下stream配置文件
vim /etc/nginx/nginx.conf
将http字段中的include /etc/nginx/conf.d/*.conf;移到上方
然后
cd /etc/nginx/conf.d/
mv default.conf /opt
vim graylog_stream.conf
添加如下行
stream {
log_format streamlog_json escape=json
'{"timestamp":"$time_iso8601",'
'"remote_addr":"$remote_addr",'
'"protocol":"$protocol",'
'"status":"$status",'
'"bytes_sent":"$bytes_sent",'
'"bytes_received":"$bytes_received",'
'"session_time":"$session_time",'
'"upstream_addr":"$upstream_addr",'
'"upstream_bytes_sent":"$upstream_bytes_sent",'
'"upstream_bytes_received":"$upstream_bytes_received",'
'"upstream_connect_time":"$upstream_connect_time"'
'}';
access_log /var/log/nginx/stream-access.log streamlog_json ;
access_log syslog:server=10.10.253.101:2514 streamlog_json ;
open_log_file_cache off;
upstream graylog_servers{
server 10.10.253.101:9000;
server 10.10.253.102:9000 backup;
server 10.10.253.103:9000 backup;
}
server {
listen 9000;
proxy_pass graylog_servers;
}
upstream server_input1_switchlog {
server 10.10.253.101:1515;
server 10.10.253.102:1515;
server 10.10.253.103:1515;
}
server {
listen 1515 udp;
proxy_pass server_input1_switchlog;
}
upstream server_input2_firewall-log {
server 10.10.253.101:1514;
server 10.10.253.102:1514;
server 10.10.253.103:1514;
}
server {
listen 1514 udp;
proxy_pass server_input2_firewall-log;
}
}
3、启动nginx服务
systemctl enable nginx
systemctl start nginx
firewall-cmd --add-port=1514/udp --permanent
firewall-cmd --add-port=1515/udp --permanent
firewall-cmd --add-port=9000/tcp --permanent
firewall-cmd --reload
这样就可以用nginx来实现接入日志时的负载均衡效果