1. 云图网首页
  2. 技术专区
  3. 智能运维

nginx访问日志

• 智能运维


创建nginx访问日志

Nginx日志功能需要在nginx.conf中打开相关指令log_format，设置日志格式，以及设置日志的存储位置access_log，指定日志的格式，路径，缓存大小。

日志格式字段解释

nginx.conf中有关访客日志定义如下
 #a
log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

 access_log  logs/access.log  main;

参数解释 
$remote_addr ：记录访问网站的客户端IP地址
$remote_user ：记录远程客户端用户名称
$time_local ：记录访问时间与时区
$request ：记录用户的 http 请求起始行信息（请求方法，http协议）
$status ：记录 http 状态码，即请求返回的状态，例如 200 、404 、502 等
$body_bytes_sent ：记录服务器发送给客户端的响应 body 字节数
$http_referer ：记录此次请求是从哪个链接访问过来的，可以根据 referer 进行防盗链设置
$http_user_agent ：记录客户端访问信息，如浏览器、手机客户端等
$http_x_forwarded_for ：当前端有代理服务器时，设置 Web 节点记录客户端地址的配置，此参数生效的前提是代理服务器上也进行了相关的 x_forwarded_for 设置

备注
$remote_addr 可能拿到的是反向代理IP地址
$http_x_forwarded_for 可以获取客户端真实IP地址

日志格式参考

nginx访问日志

生成日志

nginx访问日志

生成请求
[root@web-7 /etc/nginx/conf.d]#for i in {1..10};do curl 10.0.0.7/99.png;done

检查日志
[root@web-7 /usr/share/nginx/html]#tail -f /var/log/nginx/access.log

关闭日志

关于日志的2个参数

log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';

access_log  /var/log/nginx/access.log  main;
日志指令语法
access_log  path  [format  buffer=size  | off]

path代表日志存放路径
off是关闭日志
关闭日志记录
#access_log  /var/log/nginx/access.log  main;
access_log off;

此时就不会记录访客日志了

多虚拟主机的日志

1.全局配置

全局定义好日志格式，子页面配置中定义日志路径即可。

[root@web-7 /etc/nginx/conf.d]#cat /etc/nginx/nginx.conf 

user  www;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';


    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

www.linux0224.cc的日志

[root@web-7 /etc/nginx/conf.d]#cat www.linux0224.conf 
server {
    listen       80;
    server_name www.linux0224.cc;
    charset utf-8;
    access_log /var/log/nginx/www.linux0224.log;
    location / {
        root   /usr/share/nginx/html/;
        index  index.html index.htm;
    }
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

nginx访问日志

nginx访问日志

错误日志

Nginx能够将自身运行故障的信息也写入到指定的日志文件中。对于错误信息的调试，是维护Nginx的重要手段，指令是error_log，可以放在http{}全局中，也可以单独为虚拟主机记录。

语法：
error_log file  level;

日志级别在乎debug|info|notice|warn|error|crit|alert|emerg
级别越高，日志记录越少，生产常用模式是warn|error|crit级别
日志的记录，会给服务器增加额外大量的IO消耗，按需修改
自动生成配置文件模板
https://www.digitalocean.com/community/tools/nginx?domains.0.php.wordPressRules=true&domains.0.logging.accessLog=true&domains.0.logging.errorLog=true&global.app.lang=zhCN

去掉主配置的error_log

[root@web-7 /etc/nginx]#cat nginx.conf 

user  www;
worker_processes  auto;

pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';


    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}
记录www页面的错误日志
[root@web-7 /etc/nginx/conf.d]#cat www.linux0224.conf 
server {
    listen       80;
    server_name www.linux0224.cc;
    charset utf-8;
    access_log /var/log/nginx/www.linux0224.log;
    error_log /var/log/nginx/error.www.linux0224.log;
    location / {
        root   /usr/share/nginx/html/;
        index  index.html index.htm;
    }
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}

重启
[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx
[root@web-7 /etc/nginx/conf.d]#ls /var/log/nginx/
access.log  error.log  error.www.linux0224.log  www.linux0224.log

nginx访问日志

www404页面优化

nginx指令error_page的作用是当发生错误的时候能够显示一个预定义的uri；

语法1
error_page 404 /404.html;

语法2
error_page 404 https://error.taobao.com/app/tbhome/common/error.html;

[root@web-7 /etc/nginx/conf.d]#cat www.linux0224.conf 
server {
    listen       80;
    server_name www.linux0224.cc;
    charset utf-8;
    access_log /var/log/nginx/www.linux0224.log;
    error_log /var/log/nginx/error.www.linux0224.log;
    error_page 404 /404.html;
    location / {
        root   /usr/share/nginx/html/;
        index  index.html index.htm;
    }
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}


[root@web-7 /usr/share/nginx/html]#cat 404.html 
<h1 style='red'>您访问的地址有误，请正确查找 </h1>

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

nginx访问日志

基于IP多虚拟主机

10.0.0.77 /www/77/index.html
10.0.0.78 /www/78/index.html

77

[root@web-7 /etc/nginx/conf.d]#cat /etc/nginx/nginx.conf 

user  www;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';


    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}
77的日志
[root@web-7 /etc/nginx/conf.d]#ip addr add 10.0.0.77/24 dev eth0


[root@web-7 /etc/nginx/conf.d]#cat 77.conf 
server {
    listen 10.0.0.77:80;
    server_name _;
    charset utf-8;
    access_log /var/log/nginx/77.log;
    location / {
        root   /www/77/;
        index  index.html index.htm;
    }    
}
[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

nginx访问日志

记录ip77页面的错误日志

全局

[root@web-7 /etc/nginx]#cat nginx.conf 

user  www;
worker_processes  auto;

pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';


    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

77配置文件

[root@web-7 /etc/nginx/conf.d]#cat 77.conf 
server {
    listen 10.0.0.77:80;
    server_name _;
    charset utf-8;
    access_log /var/log/nginx/77.log;
    error_log /var/log/nginx/error.77.log;
    location / {
        root   /www/77/;
        index  index.html index.htm;
    }    
}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

[root@web-7 ~]#touch /var/log/nginx/error.78.log

nginx访问日志

77的404优化
[root@web-7 /etc/nginx/conf.d]#cat 77.conf 
server {
    listen 10.0.0.77:80;
    server_name _;
    charset utf-8;
    access_log /var/log/nginx/77.log;
    error_log /var/log/nginx/error.77.log;
    error_page 404 /404.html;
    location / {
        root   /www/77/;
        index  index.html index.htm;
    }    
}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx


数据
[root@web-7 ~]#cat >> /www/77/404.html <<EOF
> <h1 style='red'>您访问的地址有误，请正确查找 </h1>
> EOF

nginx访问日志

78

全局配置

[root@web-7 /etc/nginx/conf.d]#cat /etc/nginx/nginx.conf 

user  www;
worker_processes  auto;

error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';


    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}
78的日志
[root@web-7 /etc/nginx/conf.d]#ip addr add 10.0.0.78/24 dev eth0


[root@web-7 /etc/nginx/conf.d]#cat 78.conf 
server {

listen 10.0.0.78:80;
server_name _;
access_log /var/log/nginx/78.log;
location /  {
	root  /www/78/;
	index  index.html;
}

}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

nginx访问日志

记录ip78页面的错误日志

全局

[root@web-7 /etc/nginx]#cat nginx.conf 

user  www;
worker_processes  auto;

pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';


    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

78的配置文件

[root@web-7 /etc/nginx/conf.d]#cat 78.conf 
server {

listen 10.0.0.78:80;
server_name _;
access_log /var/log/nginx/78.log;
error_log /var/log/nginx/error.78.log;
location /  {
	root  /www/78/;
	index  index.html;
}

}
[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

[root@web-7 ~]#touch /var/log/nginx/error.78.log

nginx访问日志

78的404优化
[root@web-7 /etc/nginx/conf.d]#cat 78.conf 
server {

listen 10.0.0.78:80;
server_name _;
charset utf-8;
access_log /var/log/nginx/78.log;
error_log /var/log/nginx/error.78.log;
error_page 404 /404.html;
location /  {
	root  /www/78/;
	index  index.html;
}

}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

数据
[root@web-7 ~]#cat >> /www/78/404.html <<EOF
<h1 style='red'>您访问的地址有误，请正确查找 </h1>
EOF

nginx访问日志

部署多域名虚拟主机

blog.linux0224.cc /www/blog/index.html
movie.linux0224.cc /www/movie/index.html

全局
[root@web-7 /etc/nginx]#cat nginx.conf 

user  www;
worker_processes  auto;

pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';


    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}
blog.linux0224.cc
[root@web-7 /etc/nginx/conf.d]#cat blog.linux0224.conf 
server {

    listen 80;
    server_name blog.linux0224.cc;
    charset utf-8;
    access_log /var/log/nginx/blog.linux0224.log;
    location /  {
        root  /www/blog/;
        index  index.html;
    }

}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

[root@web-7 /var/log/nginx]#touch blog.linux0224.log

nginx访问日志

错误日志
[root@web-7 /etc/nginx/conf.d]#cat blog.linux0224.conf 
server {

    listen 80;
    server_name blog.linux0224.cc;
    charset utf-8;
    access_log /var/log/nginx/blog.linux0224.log;
    error_log /var/log/nginx/error.blog.linux0224.log;
    location /  {
        root  /www/blog/;
        index  index.html;
    }

}
[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

nginx访问日志

优化404
[root@web-7 /etc/nginx/conf.d]#cat blog.linux0224.conf 
server {

    listen 80;
    server_name blog.linux0224.cc;
    charset utf-8;
    access_log /var/log/nginx/blog.linux0224.log;
    error_log /var/log/nginx/error.blog.linux0224.log;
    error_page 404 /404.html;
    location /  {
        root  /www/blog/;
        index  index.html;
    }

}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

[root@web-7 /var/log/nginx]#cat >> /www/blog/404.html <<EOF
> <h1 style='red'>您访问的地址有误，请正确查找 </h1>
> EOF

nginx访问日志

movie.linux0224.cc
[root@web-7 /etc/nginx/conf.d]#cat movie.linux0224.conf 
server {

    listen 80;
    server_name movie.linux0224.cc;
    charset utf-8;
    access_log /var/log/nginx/movie.linux0224.log;
    location /  {
        root  /www/movie/;
        index  index.html;
    }

}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx
[root@web-7 /var/log/nginx]#touch movie.linux0224.log

nginx访问日志

错误日志
[root@web-7 /etc/nginx/conf.d]#cat movie.linux0224.conf 
server {

    listen 80;
    server_name movie.linux0224.cc;
    charset utf-8;
    access_log /var/log/nginx/movie.linux0224.log;
    error_log /var/log/nginx/error.movie.linux0224.log;
    location /  {
        root  /www/movie/;
        index  index.html;
    }

}
[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

nginx访问日志

优化404

[root@web-7 /etc/nginx/conf.d]#cat movie.linux0224.conf 
server {

    listen 80;
    server_name movie.linux0224.cc;
    charset utf-8;
    access_log /var/log/nginx/movie.linux0224.log;
    error_log /var/log/nginx/error.movie.linux0224.log;
    error_page 404 /404.html;
    location /  {
        root  /www/movie/;
        index  index.html;
    }

}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

[root@web-7 /var/log/nginx]#cat >> /www/movie/404.html <<EOF
> <h1 style='red'>您访问的地址有误，请正确查找 </h1>
> EOF

nginx访问日志

部署多port虚拟主机

​ 10.0.0.8:81 /www/81/index.html
​ 10.0.0.8:82 /www/82/index.html

全局

[root@web-7 /etc/nginx]#cat nginx.conf 

user  www;
worker_processes  auto;

pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';


    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

子配置文件

[root@web-7 /etc/nginx/conf.d]#cat 81.82.conf 
server {

    listen 10.0.0.7:81;
    access_log /var/log/nginx/81.log;
    server_name _;
	charset utf-8;
    location /  {
        root  /www/81/;
        index  index.html;
    }

}

server {

    listen 10.0.0.7:82;
    server_name _;
    charset utf-8;
    access_log /var/log/nginx/82.log;
    location /  {
        root  /www/82/;
        index  index.html;
    }

}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

nginx访问日志

nginx访问日志

错误日志

[root@web-7 /etc/nginx/conf.d]#cat 81.82.conf 
server {

    listen 10.0.0.7:81;
    access_log /var/log/nginx/81.log; 
    error_log /var/log/nginx/error.81.log;
    server_name _;
	charset utf-8;
    location /  {
        root  /www/81/;
        index  index.html;
    }

}

server {

    listen 10.0.0.7:82;
    server_name _;
    charset utf-8;
    access_log /var/log/nginx/82.log;
    error_log /var/log/nginx/error.82.log;
    location /  {
        root  /www/82/;
        index  index.html;
    }

}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx

nginx访问日志

nginx访问日志

优化404

[root@web-7 /etc/nginx/conf.d]#cat 81.82.conf 
server {

    listen 10.0.0.7:81;
    access_log /var/log/nginx/81.log; 
    error_log /var/log/nginx/error.81.log;
    error_page 404 /404.html;
    server_name _;
	charset utf-8;
    location /  {
        root  /www/81/;
        index  index.html;
    }

}

server {

    listen 10.0.0.7:82;
    server_name _;
    charset utf-8;
    access_log /var/log/nginx/82.log;
    error_log /var/log/nginx/error.82.log;
    error_page 404 /404.html;
    location /  {
        root  /www/82/;
        index  index.html;
    }

}

[root@web-7 /etc/nginx/conf.d]#systemctl restart nginx


[root@web-7 /var/log/nginx]#cat >> /www/81/404.html <<EOF
<h1 style='red'>您访问的地址有误，请正确查找 </h1>
EOF

[root@web-7 /var/log/nginx]#cat >> /www/82/404.html <<EOF
<h1 style='red'>您访问的地址有误，请正确查找 </h1>
EOF

nginx访问日志

nginx访问日志

生产nginx日志提取（awk）练习

1. 统计独立IP数量

生成数据
[root@web-7 ~]#for i in {1..100};do curl 10.0.0.7:81;done

[root@web-7 /var/log/nginx]#awk '{print $1}' 81.log |wc -l
105

2.查看一段时间的IP访问次数，最高的前20个，以及对应次数。 （2022年4月13下午4点到5点）

格式 13/Apr/2022:20:11:04 +0800]

[root@web-7 /var/log/nginx]#awk -v FS=' '  '/[20/May/2022:00:37:28 +0800]|[20/May/2022:00:37:28 +0800]/{print $1}' 81.log |sort -n|uniq -c|sort -rn
    300 10.0.0.7
      5 10.0.0.1

3.查看访问次数最频繁的前10个IP

[root@web-7 /var/log/nginx]#cat 81.log |awk -v FS='-' '{print $1}'|sort -n|uniq -c|sort -nr
    300 10.0.0.7 
      5 10.0.0.1

4.查看请求次数超过1000次的IP地址以及次数。

[root@web-7 ~]#for i in {1..1000};do curl 10.0.0.7:81;done

[root@web-7 /var/log/nginx]#cat 81.log |awk -v FS='-' '{print $1}'|sort -n|uniq -c|sort -nr
   1300 10.0.0.7 
      5 10.0.0.1

5.查看最近10000条记录，访问量次数最多的url是什么（前10）

[root@web-7 /var/log/nginx]#head -n 10000 81.log |awk -v FS='"' '{print $(NF-1)}' |sort -n|uniq -c |sort -nr|nl
     1	   1300 curl/7.29.0
     2	      5 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36 Edg/101.0.1210.47

6. 统计出每天几点，公司业务量是压力最大的？（前10）

[root@web-7 /var/log/nginx]#cat -n  81.log  |awk -v FS='[' '{print $2}' |awk -v FS=']' '{print $1}' |sort -n|uniq -c|sort -nr|nl
     1	    298 20/May/2022:00:43:17 +0800
     2	    293 20/May/2022:00:43:18 +0800
     3	    225 20/May/2022:00:43:16 +0800
     4	    184 20/May/2022:00:43:19 +0800
     5	    100 20/May/2022:00:27:20 +0800
     6	     71 20/May/2022:16:23:27 +0800
     7	     50 20/May/2022:00:37:28 +0800
     8	     50 20/May/2022:00:37:27 +0800
     9	     29 20/May/2022:16:23:26 +0800
    10	      1 20/May/2022:00:19:29 +0800
    11	      1 20/May/2022:00:19:05 +0800
    12	      1 20/May/2022:00:16:08 +0800
    13	      1 20/May/2022:00:12:45 +0800
    14	      1 20/May/2022:00:12:44 +0800

原创文章，作者：jamestackk，如若转载，请注明出处：https://blog.ytso.com/274443.html

赞 (0)
0
上一篇 2022年7月15日
下一篇 2022年7月15日

相关推荐

发表回复

登录后才能评论