51工具盒子

依楼听风雨
笑看云卷云舒,淡观潮起潮落

使用Nginx作为GrayLog日志接入的负载均衡

如下图所示

在Graylog集群环境上,日志源将日志发送到负载均衡,由负载均衡来分配日志发送到GrayLog具体哪个节点上,实现各节点接入能力和处理能力平衡均分

下面采用nginx作为GrayLog 日志接入的负载均衡

1、配置nginx 的yum源

参考nginx官方文档

http://nginx.org/en/linux_packages.html#RHEL

具体命令如下

vim /etc/yum.repos.d/nginx.repo
添加如下行
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true

[nginx-mainline]
name=nginx mainline repo
baseurl=http://nginx.org/packages/mainline/centos/$releasever/$basearch/
gpgcheck=0
enabled=0
gpgkey=https://nginx.org/keys/nginx_signing.key
module_hotfixes=true

安装nginx

yum install nginx --enablerepo=nginx-stable --disablerepo=epel

2、配置nginx下stream配置文件

vim /etc/nginx/nginx.conf 
将http字段中的include /etc/nginx/conf.d/*.conf;移到上方

然后

cd /etc/nginx/conf.d/
mv default.conf /opt
vim  graylog_stream.conf 

添加如下行

stream {
      log_format streamlog_json escape=json 
                     '{"timestamp":"$time_iso8601",'
                     '"remote_addr":"$remote_addr",' 
                     '"protocol":"$protocol",' 
                     '"status":"$status",' 
                     '"bytes_sent":"$bytes_sent",' 
                     '"bytes_received":"$bytes_received",'
                     '"session_time":"$session_time",' 
                     '"upstream_addr":"$upstream_addr",'
                     '"upstream_bytes_sent":"$upstream_bytes_sent",' 
                     '"upstream_bytes_received":"$upstream_bytes_received",'
                     '"upstream_connect_time":"$upstream_connect_time"'
                     '}';
      access_log /var/log/nginx/stream-access.log  streamlog_json ;
      access_log  syslog:server=10.10.253.101:2514 streamlog_json ;
      open_log_file_cache off;
    
      upstream graylog_servers{
        server 10.10.253.101:9000;
        server 10.10.253.102:9000 backup;
        server 10.10.253.103:9000 backup;
      }
      
      server {
        listen       9000;
        proxy_pass   graylog_servers;
      }

      upstream server_input1_switchlog {
          server 10.10.253.101:1515;
          server 10.10.253.102:1515;
          server 10.10.253.103:1515;
      }
      server {
         listen 1515 udp;
         proxy_pass server_input1_switchlog;
      }

      upstream server_input2_firewall-log {
        server 10.10.253.101:1514;
        server 10.10.253.102:1514;
        server 10.10.253.103:1514;
      }
 
      server {
            listen 1514 udp;
            proxy_pass server_input2_firewall-log;
      }
}

3、启动nginx服务

systemctl enable nginx
systemctl start nginx

firewall-cmd --add-port=1514/udp --permanent 
firewall-cmd --add-port=1515/udp --permanent 
firewall-cmd --add-port=9000/tcp --permanent 
firewall-cmd --reload

这样就可以用nginx来实现接入日志时的负载均衡效果

赞(0)
未经允许不得转载:工具盒子 » 使用Nginx作为GrayLog日志接入的负载均衡