文章目录运行环境数据库配置rsyslog配置loganalyzer安装防火墙配置《中华人民共和国网络安全法》第二十一条第三项明确规定网络运营者必须采取监测、记录网络运行状态和网络安全事件的技术措施并按照规定留存相关的网络日志不少于六个月。目前市面上日志类产品接近10W左右功能对于一些小公司来说有些过剩。在这里分享一个我自己基于开源组件搭建的日志服务器PC电脑即可满足性能要求对于法律中的存储要求绰绰有余。不过loganalyzer的展示、搜索功能满足要求分析功能差强人意大家可按需搭建。我自己也针对rsyslog的数据库接口写了一个API数据接口服务后续将放在仓库里共享可配合数据分析脚本使用。运行环境操作系统CentOS 7数据库mysql5.6 yum 默认安装rsyslogd8.24.0 系统自带中间件httpd (Apache) 2.4.6 (yum 默认安装)运行环境php、php-mysql、gd、gd-devel、php-gd (yum 默认安装)loganalyzer4.1.6 (官网下载)其他运行环境安装、配置类似如有差异自行豆包数据库配置1、修改配置文件: /etc/my.cnfskip_name_resolveONinnodb_file_per_tableONinteractive_timeout28800000wait_timeout28800000#防止中文乱码character-set-serverutf8init_connectSET NAMES utf82、安装rsyslog连接至mysql的驱动模块yuminstallrsyslog-mysql安装完成后查看安装结果[rootlocalhost rsyslog-8.24.0]$rpm-qlrsyslog-mysql /usr/lib64/rsyslog/ommysql.so /usr/share/doc/rsyslog-8.24.0/mysql-createDB.sqlommysql.so -- rsyslog输出时用到的模块,需要在/etc/rsyslog.conf配置文件里进行配置mysql-createDB.sql -- 帮我们生成rsyslog所依赖的数据(表,字段,表结构等等)的sql脚本3、导入rsyslog表结构mysql-uUSER-hHOST-pPASSWORD/usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql查看执行结果mysqlshowdatabases;--------------------|Database|--------------------|information_schema||Syslog||mysql||performance_schema|--------------------4rowsinset(0.00sec)mysqluseSyslog Readingtableinformationforcompletionoftableandcolumnnames You can turnoffthis featuretoget a quicker startupwith-ADatabasechanged mysqlshowtables;------------------------|Tables_in_Syslog|------------------------|SystemEvents||SystemEventsProperties|------------------------2rowsinset(0.00sec)4、建立syslog专用的用户账号(数据库名叫Syslog用户名和密码自己改)GRANTALLONSyslog.*TOrsyslog127.0.0.1IDENTIFIEDBYabcd1234;GRANTALLONSyslog.*TOrsysloglocalhostIDENTIFIEDBYabcd1234;flushprivileges;5、修改表结构可以不操作rsyslogd数据默认不会存储客户端IP信息为保存客户端IP地址修改数据库表结构添加FromIP字段mysqluseSyslog;Readingtableinformationforcompletionoftableandcolumnnames You can turnoffthis featuretoget a quicker startupwith-ADatabasechanged mysqlaltertableSystemEventsaddFromIPvarchar(60)defaultnullafterFromHost;Query OK,0rowsaffected(0.03sec)Records:0Duplicates:0Warnings:0若执行本步操作后rsyslog.conf中template insertpl语句为(自行对比差异)$template insertpl,insert into SystemEvents (Message, Facility, FromHost, FromIP, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (%msg%, %syslogfacility%, %HOSTNAME%, %fromhost-ip%, %syslogpriority%, %timereported:::date-mysql%, %timegenerated:::date-mysql%, %iut%, %syslogtag%),SQLrsyslog配置1、编辑配置文件: /etc/rsyslog.conf在此只介绍修改后的配置项其余项默认即可$template insertpl,insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (%msg%, %syslogfacility%, %fromhost-ip%, %syslogpriority%, %timereported:::date-mysql%, %timegenerated:::date-mysql%, %iut%, %syslogtag%),SQL $template Remote,/var/log/%fromhost-ip%/%fromhost-ip%_%$YEAR%-%$MONTH%-%$DAY%.log :fromhost-ip, !isequal, 127.0.0.1 ?Remote :fromhost-ip, !isequal, localhost ?Remote if $fromhost-ip ! 127.0.0.1 and $fromhost-ip ! localhost then { $ModLoad ommysql *.info,*.notice,*.warning;*.err;*.crit;*.alert;*.emerg :ommysql:127.0.0.1,Syslog,rsyslog,abcd1234;insertpl } # The imjournal module bellow is now used as a message source instead of imuxsock. $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $ModLoad imjournal # provides access to the systemd journal $ModLoad imklog # reads kernel messages (the same are read from journald) $ModLoad immark # provides --MARK-- message capability # Provides UDP syslog reception $ModLoad imudp $UDPServerRun 514 # Provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514FACILITY定义服务类型功能、描述符号syslog序列号kern内核信息LOG_KERN0user用户进程LOG_USER1mail邮件LOG_MAIL2daemon后台进程LOG_DAEM3authpriv授权信息LOG_AUTH4syslog系统日志LOG_SYSLOG5lpr打印信息LOG_LPR6news新闻组信息LOG_NEWS7uucpUUCP程序LOG_UUCP8cron计划和任务信息LOG_CRON9认证消息LOG_AUTHPRIV10FTP daemonLOG_LOGTP11NTP subsystemLOG_NTP12log auditLOG_AUDIT13log alertLOG_ALERT14clock daemonLOG_CLOCK15local0-local7本地用户LOG_LOCALn16-23PRIORITY定义优先级含义符号syslog序列号debug调试级——信息量最多LOG_DEBUG7info通知性消息LOG_INFO6notice普通但重要的消息LOG_NOTICE5warning警告消息LOG_WARNING4err出错消息LOG_ERR3crit重要消息LOG_CRIT2alert紧急消息LOG_ALERT1emerg最紧急的消息LOG_EMERG0none不保存信息2、重启rsyslogservicersyslog restartloganalyzer安装loganalyzer安装比较简单按步骤慢慢来就行。参考网站http://www.ifzhai.com/article.php?id9Tips:安装时先关闭selinux否则会因无法写入文件而报错坑啊测试时被坑了一天。。。修改文件 /etc/selinux/configSELINUXdisabled这一步骤还可以使用命令直接修改sed-is#SELINUXenforcing#SELINUXdisabled#g/etc/selinux/configCentOS 6.9及以下好像没这个问题没测试过防火墙配置CentOS 7开始使用firewalld默认的防火墙代替iptables(相比iptablesfirewalld真难用)#开放80端口(网段自己加)firewall-cmd--permanent--add-rich-rulerule familyipv4 source addressxxx.xxx.xxx.0/24 port procotoltcp port80 accept #开放514端口(网段自己加)firewall-cmd--permanent--add-rich-rulerule familyipv4 source addressxxx.xxx.xxx.0/24 port procotoludp port514 accept #对调试机开放3306firewall-cmd--permanent--add-rich-rulerule familyipv4 source addressxxx.xxx.xxx.xx port procotoltcp port3306 accept #限制ssh登陆firewall-cmd--permanent--remove-servicessh firewall-cmd--permanent--add-rich-rulerule familyipv4 source addressxxx.xxx.xxx.xx port procotoltcp port22 accept #加载防火墙策略firewall-cmd--reload#查看防火墙策略firewall-cmd --list-all
【懒人运维】rsyslog+mysql+loganalyzer 日志服务器搭建
文章目录运行环境数据库配置rsyslog配置loganalyzer安装防火墙配置《中华人民共和国网络安全法》第二十一条第三项明确规定网络运营者必须采取监测、记录网络运行状态和网络安全事件的技术措施并按照规定留存相关的网络日志不少于六个月。目前市面上日志类产品接近10W左右功能对于一些小公司来说有些过剩。在这里分享一个我自己基于开源组件搭建的日志服务器PC电脑即可满足性能要求对于法律中的存储要求绰绰有余。不过loganalyzer的展示、搜索功能满足要求分析功能差强人意大家可按需搭建。我自己也针对rsyslog的数据库接口写了一个API数据接口服务后续将放在仓库里共享可配合数据分析脚本使用。运行环境操作系统CentOS 7数据库mysql5.6 yum 默认安装rsyslogd8.24.0 系统自带中间件httpd (Apache) 2.4.6 (yum 默认安装)运行环境php、php-mysql、gd、gd-devel、php-gd (yum 默认安装)loganalyzer4.1.6 (官网下载)其他运行环境安装、配置类似如有差异自行豆包数据库配置1、修改配置文件: /etc/my.cnfskip_name_resolveONinnodb_file_per_tableONinteractive_timeout28800000wait_timeout28800000#防止中文乱码character-set-serverutf8init_connectSET NAMES utf82、安装rsyslog连接至mysql的驱动模块yuminstallrsyslog-mysql安装完成后查看安装结果[rootlocalhost rsyslog-8.24.0]$rpm-qlrsyslog-mysql /usr/lib64/rsyslog/ommysql.so /usr/share/doc/rsyslog-8.24.0/mysql-createDB.sqlommysql.so -- rsyslog输出时用到的模块,需要在/etc/rsyslog.conf配置文件里进行配置mysql-createDB.sql -- 帮我们生成rsyslog所依赖的数据(表,字段,表结构等等)的sql脚本3、导入rsyslog表结构mysql-uUSER-hHOST-pPASSWORD/usr/share/doc/rsyslog-8.24.0/mysql-createDB.sql查看执行结果mysqlshowdatabases;--------------------|Database|--------------------|information_schema||Syslog||mysql||performance_schema|--------------------4rowsinset(0.00sec)mysqluseSyslog Readingtableinformationforcompletionoftableandcolumnnames You can turnoffthis featuretoget a quicker startupwith-ADatabasechanged mysqlshowtables;------------------------|Tables_in_Syslog|------------------------|SystemEvents||SystemEventsProperties|------------------------2rowsinset(0.00sec)4、建立syslog专用的用户账号(数据库名叫Syslog用户名和密码自己改)GRANTALLONSyslog.*TOrsyslog127.0.0.1IDENTIFIEDBYabcd1234;GRANTALLONSyslog.*TOrsysloglocalhostIDENTIFIEDBYabcd1234;flushprivileges;5、修改表结构可以不操作rsyslogd数据默认不会存储客户端IP信息为保存客户端IP地址修改数据库表结构添加FromIP字段mysqluseSyslog;Readingtableinformationforcompletionoftableandcolumnnames You can turnoffthis featuretoget a quicker startupwith-ADatabasechanged mysqlaltertableSystemEventsaddFromIPvarchar(60)defaultnullafterFromHost;Query OK,0rowsaffected(0.03sec)Records:0Duplicates:0Warnings:0若执行本步操作后rsyslog.conf中template insertpl语句为(自行对比差异)$template insertpl,insert into SystemEvents (Message, Facility, FromHost, FromIP, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (%msg%, %syslogfacility%, %HOSTNAME%, %fromhost-ip%, %syslogpriority%, %timereported:::date-mysql%, %timegenerated:::date-mysql%, %iut%, %syslogtag%),SQLrsyslog配置1、编辑配置文件: /etc/rsyslog.conf在此只介绍修改后的配置项其余项默认即可$template insertpl,insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (%msg%, %syslogfacility%, %fromhost-ip%, %syslogpriority%, %timereported:::date-mysql%, %timegenerated:::date-mysql%, %iut%, %syslogtag%),SQL $template Remote,/var/log/%fromhost-ip%/%fromhost-ip%_%$YEAR%-%$MONTH%-%$DAY%.log :fromhost-ip, !isequal, 127.0.0.1 ?Remote :fromhost-ip, !isequal, localhost ?Remote if $fromhost-ip ! 127.0.0.1 and $fromhost-ip ! localhost then { $ModLoad ommysql *.info,*.notice,*.warning;*.err;*.crit;*.alert;*.emerg :ommysql:127.0.0.1,Syslog,rsyslog,abcd1234;insertpl } # The imjournal module bellow is now used as a message source instead of imuxsock. $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $ModLoad imjournal # provides access to the systemd journal $ModLoad imklog # reads kernel messages (the same are read from journald) $ModLoad immark # provides --MARK-- message capability # Provides UDP syslog reception $ModLoad imudp $UDPServerRun 514 # Provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514FACILITY定义服务类型功能、描述符号syslog序列号kern内核信息LOG_KERN0user用户进程LOG_USER1mail邮件LOG_MAIL2daemon后台进程LOG_DAEM3authpriv授权信息LOG_AUTH4syslog系统日志LOG_SYSLOG5lpr打印信息LOG_LPR6news新闻组信息LOG_NEWS7uucpUUCP程序LOG_UUCP8cron计划和任务信息LOG_CRON9认证消息LOG_AUTHPRIV10FTP daemonLOG_LOGTP11NTP subsystemLOG_NTP12log auditLOG_AUDIT13log alertLOG_ALERT14clock daemonLOG_CLOCK15local0-local7本地用户LOG_LOCALn16-23PRIORITY定义优先级含义符号syslog序列号debug调试级——信息量最多LOG_DEBUG7info通知性消息LOG_INFO6notice普通但重要的消息LOG_NOTICE5warning警告消息LOG_WARNING4err出错消息LOG_ERR3crit重要消息LOG_CRIT2alert紧急消息LOG_ALERT1emerg最紧急的消息LOG_EMERG0none不保存信息2、重启rsyslogservicersyslog restartloganalyzer安装loganalyzer安装比较简单按步骤慢慢来就行。参考网站http://www.ifzhai.com/article.php?id9Tips:安装时先关闭selinux否则会因无法写入文件而报错坑啊测试时被坑了一天。。。修改文件 /etc/selinux/configSELINUXdisabled这一步骤还可以使用命令直接修改sed-is#SELINUXenforcing#SELINUXdisabled#g/etc/selinux/configCentOS 6.9及以下好像没这个问题没测试过防火墙配置CentOS 7开始使用firewalld默认的防火墙代替iptables(相比iptablesfirewalld真难用)#开放80端口(网段自己加)firewall-cmd--permanent--add-rich-rulerule familyipv4 source addressxxx.xxx.xxx.0/24 port procotoltcp port80 accept #开放514端口(网段自己加)firewall-cmd--permanent--add-rich-rulerule familyipv4 source addressxxx.xxx.xxx.0/24 port procotoludp port514 accept #对调试机开放3306firewall-cmd--permanent--add-rich-rulerule familyipv4 source addressxxx.xxx.xxx.xx port procotoltcp port3306 accept #限制ssh登陆firewall-cmd--permanent--remove-servicessh firewall-cmd--permanent--add-rich-rulerule familyipv4 source addressxxx.xxx.xxx.xx port procotoltcp port22 accept #加载防火墙策略firewall-cmd--reload#查看防火墙策略firewall-cmd --list-all
