https://www.cnblogs.com/chenqionghe/p/12503181.html
https://blog.csdn.net/alex_xfboy/article/details/88298165
https://www.cnblogs.com/FengGeBlog/p/10300682.html
https://www.jianshu.com/p/d3042a08eb5e
Tomcat日志详解
https://www.cnblogs.com/winner-0715/p/7074280.html
相关服务器安装jdk和redis请参考
http://blog.csdn.net/u013619834/article/details/38894649
http://blog.csdn.net/u013619834/article/details/38899405
负责发送日志的所有服务器安装logstash
1.下载安装包
wget https://download.elastic.co/logstash/logstash/logstash-2.4.1.tar.gz
2.安装
tar zxvf logstash-2.4.1.tar.gz
mv logstash-2.4.1 /usr/local/logstash
mkdir -p /usr/local/logstash/etc
3.修改配置文件
vim /usr/local/logstash/etc/logstash.cnf
添加
input {
file {
type => "tomcat-catalina"
path => "/u02/8080-tomcat/logs/catalina.out"
codec => multiline {
pattern => "^\s"
what => "previous"
}
}
}
output {
redis {
host => "172.17.17.15"
port => 6379
data_type => "list"
key => "logstash-tomcat-catalina"
}
}
4.如果系统中没设置环境变量,需要添加JAVA_HOME环境变量
vim /usr/local/logstash/bin/logstash.lib.sh
添加
JAVA_HOME=/usr/java/jdk1.7.0_79
5.启动
mkdir -p /usr/local/logstash/logs
nohup /usr/local/logstash/bin/logstash -f /usr/local/logstash/etc/logstash.cnf >> /usr/local/logstash/logs/nohup.out 2>&1 &
6.到redis查看
redis-cli
LPOP "logstash-tomcat-catalina"
负责接收数据的logstash服务器安需要安装logstash,并使用以下配置文件
vim /usr/local/logstash/etc/logstash.cnf
添加
input {
redis {
host => "172.17.17.15"
port => 6379
data_type => "list"
key => "logstash-tomcat-catalina"
}
}
filter {
ruby {
code => "event['filedatetag'] = event.timestamp.time.localtime.strftime('%Y-%m-%d')"
}
}
output {
file {
path => "/data/log/tomcat/%{host}/catalina-%{filedatetag}.log"
message_format=>"%{host}----%{message}"
}
#stdout{
# codec=>rubydebug
#}
if [message] =~ "Exception" {
file {
path => "/data/log/tomcat/exception/exception-%{filedatetag}.log"
}
}
}
收集nginx日志的配置文件
客户端上的配置
vim /usr/local/logstash/etc/nginx_log.cnf
input {
file {
type => "nginx_access_log"
path => "/data/logs/nginx/access_log.log"
}
file {
type => "nginx_access"
path => "/data/logs/nginx/access.log"
}
file {
type => "nginx_access_check"
path => "/data/logs/nginx/access_check.log"
}
}
output {
redis {
host => "172.17.17.15"
port => 6379
data_type => "list"
key => "logstash-nginx-log"
}
}
日志服务器上的配置
vim /usr/local/logstash/etc/nginx_log.cnf
input {
redis {
host => "172.17.17.15"
port => 6379
data_type => "list"
key => "logstash-nginx-log"
}
}
filter {
ruby {
code => "event['filedatetag'] = event.timestamp.time.localtime.strftime('%Y-%m-%d')"
}
}
output {
file {
path => "/data/log/nginx/%{host}/%{type}/%{type}-%{filedatetag}.log"
message_format=>"%{message}"
}
}
input {
redis {
host => "172.17.17.15"
port => 6379
data_type => "list"
key => "logstash-nginx-log"
}
}
filter {
ruby {
code => "event['filedatetag'] = event.timestamp.time.localtime.strftime('%Y-%m-%d')"
}
json {
source => "message"
target => "jsoncontent"
}
}
output {
file {
path => "/data/log/nginx/all/%{type}/%{type}-%{filedatetag}.log"
message_format=>"%{message}"
#message_format=>"%{host}----%{type}----%{message}"
}
if [message] =~ "code=514" {
file {
path => "/data/log/nginx/zabbix_monitor/yunxin-code514.log"
message_format=>"%{message}"
}
}
exec {
command => "/usr/local/redis/bin/redis-cli -h 127.0.0.1 incr zabbix_nginx_log_count_%{type}"
}
exec {
command => "/usr/local/redis/bin/redis-cli -h 127.0.0.1 incr zabbix_nginx_log_count_%{type}_%{[jsoncontent][status]}"
}
#stdout{
# codec=>rubydebug
#}
}
本站文章如无特殊说明,均为本站原创,如若转载,请注明出处:使用logstash配置nginx和tomcat日志统一收集到一台日志服务器 - Python技术站