Docker 部署,基于 Supervisor 的 crontab (bash sleep) 的实现,以降低内存占用

1、命令行脚本的运行基于 Supervisor 提供支持,持续运行期间,占用内存过高的问题仍然未得到根本解决,查看网址:http://www.shuijingwanwq.com/2019/07/24/3376/

2、查看网址:https://stackoverflow.com/questions/27341846/using-supervisor-as-cron ,通过调用 bash sleep 命令,实现间隔一定时间执行命令行脚本,避免持续不间断的运行,以降低内存占用

3、开发环境,Docker 容器的 CPU:0.04%,内存:168MB,如图1

开发环境,Docker 容器的 CPU:0.04%,内存:168MB

图1

4、查看 supervisord 运行状态:supervisorctl status

[root@09a3838784ce /]# ps aux|grep superviosrd
root      1379  0.0  0.0  10696   996 pts/2    S+   15:09   0:00 grep --color=auto superviosrd
[root@09a3838784ce /]# supervisorctl status
cronolog                         RUNNING   pid 446, uptime 0:30:20
nginx                            RUNNING   pid 441, uptime 0:30:20
php-fpm                          RUNNING   pid 440, uptime 0:30:20
report_client                    RUNNING   pid 444, uptime 0:30:20
yii-cmc-console-user-sync        RUNNING   pid 1341, uptime 0:00:58
yii-config-column-user-sync      RUNNING   pid 1382, uptime 0:00:02
yii-log-delete                   RUNNING   pid 1377, uptime 0:00:16

5、编辑 /etc/supervisord.d/yii-cmc-console-user-sync.ini、/etc/supervisord.d/yii-config-column-user-sync.ini、/etc/supervisord.d/yii-log-delete.ini,这将 每60秒、每60秒、每300秒 运行一次命令

[program:yii-cmc-console-user-sync]
command = bash -c 'sleep 60 && exec php /mcloud/www/pcs-api/yii cmc-console-user/sync'
autorestart = true
startsecs = 0
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-cmc-console-user-sync-stderr.log
stdout_logfile = /data/logs/yii-cmc-console-user-sync-stdout.log
[program:yii-config-column-user-sync]
command = bash -c 'sleep 60 && exec php /mcloud/www/pcs-api/yii config-column-user/sync'
autorestart = true
startsecs = 0
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-config-column-user-sync-stderr.log
stdout_logfile = /data/logs/yii-config-column-user-sync-stdout.log
[program:yii-log-delete]
command = bash -c 'sleep 300 && exec php /mcloud/www/pcs-api/yii log/delete'
autorestart = true
startsecs = 0
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-log-delete-stderr.log
stdout_logfile = /data/logs/yii-log-delete-stdout.log

6、在 3 个命令行的初始位置分别写入对应的文本文件,以检测命令行运行的间隔时间

file_put_contents('/mcloud/www/pcs-api/console/runtime/logs/cmc-console-user-sync-memory-get-usage-' . date('Y-m-d H:i:s', time()) . '.txt', memory_get_usage() / 1024 / 1024 . 'MB');
return ExitCode::OK;

file_put_contents('/mcloud/www/pcs-api/console/runtime/logs/config-column-user-sync-memory-get-usage-' . date('Y-m-d H:i:s', time()) . '.txt', memory_get_usage() / 1024 / 1024 . 'MB');
return ExitCode::OK;

file_put_contents('/mcloud/www/pcs-api/console/runtime/logs/log-delete-memory-get-usage-' . date('Y-m-d H:i:s', time()) . '.txt', memory_get_usage() / 1024 / 1024 . 'MB');
return ExitCode::OK;

7、查看写入的文本文件,以检测命令行运行的间隔时间,符合预期,如图2

查看写入的文本文件,以检测命令行运行的间隔时间,符合预期

图2

[root@38b6559d8629 /]# cd /mcloud/www/pcs-api/console/runtime/logs/
[root@38b6559d8629 logs]# ls -l
total 124
-rw-r--r-- 1 root root 17084 Oct  9 16:39 app.log
-rw-r--r-- 1 root root    17 Oct  9 16:40 cmc-console-user-sync-memory-get-usage-2019-10-09 16:40:56.txt
-rw-r--r-- 1 root root    17 Oct  9 16:41 cmc-console-user-sync-memory-get-usage-2019-10-09 16:41:57.txt
-rw-r--r-- 1 root root    17 Oct  9 16:42 cmc-console-user-sync-memory-get-usage-2019-10-09 16:42:58.txt
-rw-r--r-- 1 root root    17 Oct  9 16:43 cmc-console-user-sync-memory-get-usage-2019-10-09 16:43:59.txt
-rw-r--r-- 1 root root    17 Oct  9 16:45 cmc-console-user-sync-memory-get-usage-2019-10-09 16:45:00.txt
-rw-r--r-- 1 root root    17 Oct  9 16:46 cmc-console-user-sync-memory-get-usage-2019-10-09 16:46:02.txt
-rw-r--r-- 1 root root    17 Oct  9 16:47 cmc-console-user-sync-memory-get-usage-2019-10-09 16:47:03.txt
-rw-r--r-- 1 root root    17 Oct  9 16:48 cmc-console-user-sync-memory-get-usage-2019-10-09 16:48:04.txt
-rw-r--r-- 1 root root    17 Oct  9 16:49 cmc-console-user-sync-memory-get-usage-2019-10-09 16:49:05.txt
-rw-r--r-- 1 root root    17 Oct  9 16:50 cmc-console-user-sync-memory-get-usage-2019-10-09 16:50:06.txt
-rw-r--r-- 1 root root    17 Oct  9 16:51 cmc-console-user-sync-memory-get-usage-2019-10-09 16:51:07.txt
-rw-r--r-- 1 root root    17 Oct  9 16:52 cmc-console-user-sync-memory-get-usage-2019-10-09 16:52:08.txt
-rw-r--r-- 1 root root    17 Oct  9 16:40 config-column-user-sync-memory-get-usage-2019-10-09 16:40:57.txt
-rw-r--r-- 1 root root    17 Oct  9 16:41 config-column-user-sync-memory-get-usage-2019-10-09 16:41:58.txt
-rw-r--r-- 1 root root    17 Oct  9 16:42 config-column-user-sync-memory-get-usage-2019-10-09 16:42:59.txt
-rw-r--r-- 1 root root    17 Oct  9 16:44 config-column-user-sync-memory-get-usage-2019-10-09 16:44:00.txt
-rw-r--r-- 1 root root    17 Oct  9 16:45 config-column-user-sync-memory-get-usage-2019-10-09 16:45:01.txt
-rw-r--r-- 1 root root    17 Oct  9 16:46 config-column-user-sync-memory-get-usage-2019-10-09 16:46:03.txt
-rw-r--r-- 1 root root    17 Oct  9 16:47 config-column-user-sync-memory-get-usage-2019-10-09 16:47:04.txt
-rw-r--r-- 1 root root    17 Oct  9 16:48 config-column-user-sync-memory-get-usage-2019-10-09 16:48:05.txt
-rw-r--r-- 1 root root    17 Oct  9 16:49 config-column-user-sync-memory-get-usage-2019-10-09 16:49:06.txt
-rw-r--r-- 1 root root    17 Oct  9 16:50 config-column-user-sync-memory-get-usage-2019-10-09 16:50:07.txt
-rw-r--r-- 1 root root    17 Oct  9 16:51 config-column-user-sync-memory-get-usage-2019-10-09 16:51:08.txt
-rw-r--r-- 1 root root    17 Oct  9 16:52 config-column-user-sync-memory-get-usage-2019-10-09 16:52:09.txt
-rw-r--r-- 1 root root    17 Oct  9 16:43 log-delete-memory-get-usage-2019-10-09 16:43:56.txt
-rw-r--r-- 1 root root    17 Oct  9 16:48 log-delete-memory-get-usage-2019-10-09 16:48:57.txt

8、在 3 个命令行的初始位置分别写入对应的文本文件,删除成功退出命令,以检测命令行运行的间隔加运行时间(包含命令行自身运行的时间长度)

file_put_contents('/mcloud/www/pcs-api/console/runtime/logs/cmc-console-user-sync-memory-get-usage-' . date('Y-m-d H:i:s', time()) . '.txt', memory_get_usage() / 1024 / 1024 . 'MB');

file_put_contents('/mcloud/www/pcs-api/console/runtime/logs/config-column-user-sync-memory-get-usage-' . date('Y-m-d H:i:s', time()) . '.txt', memory_get_usage() / 1024 / 1024 . 'MB');

file_put_contents('/mcloud/www/pcs-api/console/runtime/logs/log-delete-memory-get-usage-' . date('Y-m-d H:i:s', time()) . '.txt', memory_get_usage() / 1024 / 1024 . 'MB');

9、查看写入的文本文件,以检测命令行运行的间隔加运行时间(包含命令行自身运行的时间长度),符合预期,如图3

查看写入的文本文件,以检测命令行运行的间隔加运行时间(包含命令行自身运行的时间长度),符合预期

图3

[root@adcf56a0b62b /]# cd /mcloud/www/pcs-api/console/runtime/logs/
[root@adcf56a0b62b logs]# ls -l
total 112
-rw-r--r-- 1 root root 17084 Oct  9 16:13 app.log
-rw-r--r-- 1 root root    17 Oct  9 16:15 cmc-console-user-sync-memory-get-usage-2019-10-09 16:15:00.txt
-rw-r--r-- 1 root root    17 Oct  9 16:14 config-column-user-sync-memory-get-usage-2019-10-09 16:14:59.txt
-rw-r--r-- 1 root root    17 Oct  9 16:16 config-column-user-sync-memory-get-usage-2019-10-09 16:16:01.txt
-rw-r--r-- 1 root root    17 Oct  9 16:17 config-column-user-sync-memory-get-usage-2019-10-09 16:17:02.txt
-rw-r--r-- 1 root root    17 Oct  9 16:18 config-column-user-sync-memory-get-usage-2019-10-09 16:18:04.txt
-rw-r--r-- 1 root root    17 Oct  9 16:19 config-column-user-sync-memory-get-usage-2019-10-09 16:19:05.txt
-rw-r--r-- 1 root root    17 Oct  9 16:20 config-column-user-sync-memory-get-usage-2019-10-09 16:20:07.txt
-rw-r--r-- 1 root root    17 Oct  9 16:21 config-column-user-sync-memory-get-usage-2019-10-09 16:21:08.txt
-rw-r--r-- 1 root root    17 Oct  9 16:22 config-column-user-sync-memory-get-usage-2019-10-09 16:22:10.txt
-rw-r--r-- 1 root root    17 Oct  9 16:23 config-column-user-sync-memory-get-usage-2019-10-09 16:23:11.txt
-rw-r--r-- 1 root root    17 Oct  9 16:24 config-column-user-sync-memory-get-usage-2019-10-09 16:24:13.txt
-rw-r--r-- 1 root root    17 Oct  9 16:25 config-column-user-sync-memory-get-usage-2019-10-09 16:25:14.txt
-rw-r--r-- 1 root root    17 Oct  9 16:26 config-column-user-sync-memory-get-usage-2019-10-09 16:26:16.txt
-rw-r--r-- 1 root root    17 Oct  9 16:27 config-column-user-sync-memory-get-usage-2019-10-09 16:27:17.txt
-rw-r--r-- 1 root root    17 Oct  9 16:28 config-column-user-sync-memory-get-usage-2019-10-09 16:28:19.txt
-rw-r--r-- 1 root root    17 Oct  9 16:29 config-column-user-sync-memory-get-usage-2019-10-09 16:29:20.txt
-rw-r--r-- 1 root root    17 Oct  9 16:30 config-column-user-sync-memory-get-usage-2019-10-09 16:30:22.txt
-rw-r--r-- 1 root root    17 Oct  9 16:31 config-column-user-sync-memory-get-usage-2019-10-09 16:31:23.txt
-rw-r--r-- 1 root root    17 Oct  9 16:32 config-column-user-sync-memory-get-usage-2019-10-09 16:32:25.txt
-rw-r--r-- 1 root root    17 Oct  9 16:33 config-column-user-sync-memory-get-usage-2019-10-09 16:33:27.txt
-rw-r--r-- 1 root root    17 Oct  9 16:34 config-column-user-sync-memory-get-usage-2019-10-09 16:34:28.txt
-rw-r--r-- 1 root root    17 Oct  9 16:17 log-delete-memory-get-usage-2019-10-09 16:17:59.txt
-rw-r--r-- 1 root root    17 Oct  9 16:33 log-delete-memory-get-usage-2019-10-09 16:33:00.txt

10、在命令行中抛出异常,以检测 Supervisor 是否持续启动命令行脚本,日志表中持续写入异常日志,仍然持续启动命令行脚本,如图4

在命令行中抛出异常,以检测 Supervisor 是否持续启动命令行脚本,日志表中持续写入异常日志,仍然持续启动命令行脚本

图4

[root@2e54908c8679 /]# ps aux|grep superviosrd
root       858  0.0  0.0  10696   992 pts/2    R+   17:10   0:00 grep --color=auto superviosrd
[root@2e54908c8679 /]# supervisorctl status
cronolog                         RUNNING   pid 446, uptime 0:10:28
nginx                            RUNNING   pid 441, uptime 0:10:28
php-fpm                          RUNNING   pid 440, uptime 0:10:29
report_client                    RUNNING   pid 444, uptime 0:10:28
yii-cmc-console-user-sync        RUNNING   pid 846, uptime 0:00:18
yii-config-column-user-sync      RUNNING   pid 848, uptime 0:00:17
yii-log-delete                   RUNNING   pid 443, uptime 0:10:28
[root@2e54908c8679 /]# cd /mcloud/www/pcs-api/console/runtime/logs/
[root@2e54908c8679 logs]# ls -l
total 256
-rw-r--r-- 1 root root 176844 Oct  9 17:09 app.log
-rw-r--r-- 1 root root     17 Oct  9 17:01 cmc-console-user-sync-memory-get-usage-2019-10-09 17:01:43.txt
-rw-r--r-- 1 root root     17 Oct  9 17:02 cmc-console-user-sync-memory-get-usage-2019-10-09 17:02:44.txt
-rw-r--r-- 1 root root     17 Oct  9 17:03 cmc-console-user-sync-memory-get-usage-2019-10-09 17:03:45.txt
-rw-r--r-- 1 root root     17 Oct  9 17:04 cmc-console-user-sync-memory-get-usage-2019-10-09 17:04:46.txt
-rw-r--r-- 1 root root     17 Oct  9 17:05 cmc-console-user-sync-memory-get-usage-2019-10-09 17:05:47.txt
-rw-r--r-- 1 root root     17 Oct  9 17:06 cmc-console-user-sync-memory-get-usage-2019-10-09 17:06:49.txt
-rw-r--r-- 1 root root     17 Oct  9 17:07 cmc-console-user-sync-memory-get-usage-2019-10-09 17:07:50.txt
-rw-r--r-- 1 root root     17 Oct  9 17:08 cmc-console-user-sync-memory-get-usage-2019-10-09 17:08:51.txt
-rw-r--r-- 1 root root     17 Oct  9 17:09 cmc-console-user-sync-memory-get-usage-2019-10-09 17:09:52.txt
-rw-r--r-- 1 root root     17 Oct  9 17:01 config-column-user-sync-memory-get-usage-2019-10-09 17:01:44.txt
-rw-r--r-- 1 root root     17 Oct  9 17:02 config-column-user-sync-memory-get-usage-2019-10-09 17:02:45.txt
-rw-r--r-- 1 root root     17 Oct  9 17:03 config-column-user-sync-memory-get-usage-2019-10-09 17:03:46.txt
-rw-r--r-- 1 root root     17 Oct  9 17:04 config-column-user-sync-memory-get-usage-2019-10-09 17:04:47.txt
-rw-r--r-- 1 root root     17 Oct  9 17:05 config-column-user-sync-memory-get-usage-2019-10-09 17:05:48.txt
-rw-r--r-- 1 root root     17 Oct  9 17:06 config-column-user-sync-memory-get-usage-2019-10-09 17:06:50.txt
-rw-r--r-- 1 root root     17 Oct  9 17:07 config-column-user-sync-memory-get-usage-2019-10-09 17:07:51.txt
-rw-r--r-- 1 root root     17 Oct  9 17:08 config-column-user-sync-memory-get-usage-2019-10-09 17:08:52.txt
-rw-r--r-- 1 root root     17 Oct  9 17:09 config-column-user-sync-memory-get-usage-2019-10-09 17:09:53.txt
-rw-r--r-- 1 root root     17 Oct  9 17:04 log-delete-memory-get-usage-2019-10-09 17:04:43.txt

11、还原第 8 步骤所做的修改,升级至开发环境,Docker 容器的 CPU:0.04%,内存:335MB,内存增加了 (335MB – 168MB) = 167MB 左右,如图5

还原第 8 步骤所做的修改,升级至开发环境,Docker 容器的 CPU:0.04%,内存:335MB,内存增加了 (335MB - 168MB) = 167MB 左右

图5

12、查看 supervisord 运行状态:supervisorctl status,无甚变化

[root@c37ccca41d34 /]# ps aux|grep superviosrd
root       796  0.0  0.0  10696   992 pts/2    S+   17:48   0:00 grep --color=auto superviosrd
[root@c37ccca41d34 /]# supervisorctl status
cronolog                         RUNNING   pid 446, uptime 0:10:37
nginx                            RUNNING   pid 441, uptime 0:10:37
php-fpm                          RUNNING   pid 440, uptime 0:10:37
report_client                    RUNNING   pid 444, uptime 0:10:37
yii-cmc-console-user-sync        RUNNING   pid 798, uptime 0:00:02
yii-config-column-user-sync      RUNNING   pid 766, uptime 0:00:21
yii-log-delete                   RUNNING   pid 443, uptime 0:10:37

13、总结:基于 Supervisor 的 crontab 的实现,符合预期,命令行每一次的运行结束后,将 每60秒、每60秒、每300秒 后再次开始运行。内存占用的明显上升,根源在于:bash sleep 的持续运行 (占用大量内存) , Yii 2 sleep 的持续运行 (框架在运行期间加载了大量的数据至内存中,内存占用有所减少,但被 bash sleep 的持续运行 所抵消)

14、Supervisor 的配置文件的 program 配置项,将 startsecs 的值调整为 bash sleep 的值,需要考虑进程启动成功的时间, 当 running 状态超过该值时,表明启动成功。注:现在可以确保每次皆是启动成功的

[program:yii-cmc-console-user-sync]
command = bash -c 'sleep 60 && exec php /mcloud/www/pcs-api/yii cmc-console-user/sync'
autorestart = true
startsecs = 60
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-cmc-console-user-sync-stderr.log
stdout_logfile = /data/logs/yii-cmc-console-user-sync-stdout.log
[program:yii-config-column-user-sync]
command = bash -c 'sleep 60 && exec php /mcloud/www/pcs-api/yii config-column-user/sync'
autorestart = true
startsecs = 60
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-config-column-user-sync-stderr.log
stdout_logfile = /data/logs/yii-config-column-user-sync-stdout.log
[program:yii-log-delete]
command = bash -c 'sleep 300 && exec php /mcloud/www/pcs-api/yii log/delete'
autorestart = true
startsecs = 300
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-log-delete-stderr.log
stdout_logfile = /data/logs/yii-log-delete-stdout.log

15、升级至开发环境,Docker 容器的 CPU:0.04%,内存:270MB,内存减少了 (335MB – 270MB) = 65MB 左右,但是,比之最初始的内存占用,内存增加了 (270MB – 168MB) = 102MB 左右,未达到内存占用减少的预期,如图6

升级至开发环境,Docker 容器的 CPU:0.04%,内存:270MB,内存减少了 (335MB - 270MB) = 65MB 左右,但是,比之最初始的内存占用,内存增加了 (270MB - 168MB) = 102MB 左右,未达到内存占用减少的预期

图6

16、查看 supervisord 运行状态:supervisorctl status,第1列是服务名,第2列是运行状态,RUNNING 表示运行中,FATAL 表示运行失败,STARTING 表示正在启动,STOPED 表示任务已停止,第3/4列是进程号,最后是任务已经运行的时间

[root@8db3aeab0ebb /]# ps aux|grep superviosrd
root      2150  0.0  0.0  10696   996 pts/2    R+   10:40   0:00 grep --color=auto superviosrd
[root@8db3aeab0ebb /]# supervisorctl status
cronolog                         RUNNING   pid 446, uptime 20:14:10
nginx                            RUNNING   pid 441, uptime 20:14:10
php-fpm                          RUNNING   pid 440, uptime 20:14:10
report_client                    RUNNING   pid 444, uptime 20:14:10
yii-cmc-console-user-sync        STARTING
yii-config-column-user-sync      STARTING
yii-log-delete                   RUNNING   pid 1906, uptime 0:06:42
[root@8db3aeab0ebb /]# supervisorctl status
cronolog                         RUNNING   pid 446, uptime 20:20:44
nginx                            RUNNING   pid 441, uptime 20:20:44
php-fpm                          RUNNING   pid 440, uptime 20:20:44
report_client                    RUNNING   pid 444, uptime 20:20:44
yii-cmc-console-user-sync        STARTING
yii-config-column-user-sync      RUNNING   pid 2342, uptime 0:01:00
yii-log-delete                   RUNNING   pid 1906, uptime 0:13:16

17、在 3 个命令行的初始位置分别写入对应的文本文件,以检测命令行运行的间隔时间

file_put_contents('/mcloud/www/pcs-api/console/runtime/logs/cmc-console-user-sync-memory-get-usage-' . date('Y-m-d H:i:s', time()) . '.txt', memory_get_usage() / 1024 / 1024 . 'MB');
return ExitCode::OK;

file_put_contents('/mcloud/www/pcs-api/console/runtime/logs/config-column-user-sync-memory-get-usage-' . date('Y-m-d H:i:s', time()) . '.txt', memory_get_usage() / 1024 / 1024 . 'MB');
return ExitCode::OK;

file_put_contents('/mcloud/www/pcs-api/console/runtime/logs/log-delete-memory-get-usage-' . date('Y-m-d H:i:s', time()) . '.txt', memory_get_usage() / 1024 / 1024 . 'MB');
return ExitCode::OK;

18、查看写入的文本文件,以检测命令行运行的间隔时间,每60秒、每60秒、每300秒 运行一次命令,符合预期

[root@b93baf1cb6ee /]# cd /mcloud/www/pcs-api/console/runtime/logs/
[root@b93baf1cb6ee logs]# ls -l
total 104
-rw-r--r-- 1 root root 8549 Oct 11 11:09 app.log
-rw-r--r-- 1 root root   17 Oct 11 11:09 cmc-console-user-sync-memory-get-usage-2019-10-11 11:09:21.txt
-rw-r--r-- 1 root root   17 Oct 11 11:10 cmc-console-user-sync-memory-get-usage-2019-10-11 11:10:22.txt
-rw-r--r-- 1 root root   17 Oct 11 11:11 cmc-console-user-sync-memory-get-usage-2019-10-11 11:11:23.txt
-rw-r--r-- 1 root root   17 Oct 11 11:12 cmc-console-user-sync-memory-get-usage-2019-10-11 11:12:25.txt
-rw-r--r-- 1 root root   17 Oct 11 11:13 cmc-console-user-sync-memory-get-usage-2019-10-11 11:13:26.txt
-rw-r--r-- 1 root root   17 Oct 11 11:14 cmc-console-user-sync-memory-get-usage-2019-10-11 11:14:27.txt
-rw-r--r-- 1 root root   17 Oct 11 11:15 cmc-console-user-sync-memory-get-usage-2019-10-11 11:15:28.txt
-rw-r--r-- 1 root root   17 Oct 11 11:16 cmc-console-user-sync-memory-get-usage-2019-10-11 11:16:29.txt
-rw-r--r-- 1 root root   17 Oct 11 11:17 cmc-console-user-sync-memory-get-usage-2019-10-11 11:17:30.txt
-rw-r--r-- 1 root root   17 Oct 11 11:18 cmc-console-user-sync-memory-get-usage-2019-10-11 11:18:31.txt
-rw-r--r-- 1 root root   17 Oct 11 11:19 cmc-console-user-sync-memory-get-usage-2019-10-11 11:19:32.txt
-rw-r--r-- 1 root root   17 Oct 11 11:10 config-column-user-sync-memory-get-usage-2019-10-11 11:10:21.txt
-rw-r--r-- 1 root root   17 Oct 11 11:11 config-column-user-sync-memory-get-usage-2019-10-11 11:11:22.txt
-rw-r--r-- 1 root root   17 Oct 11 11:12 config-column-user-sync-memory-get-usage-2019-10-11 11:12:24.txt
-rw-r--r-- 1 root root   17 Oct 11 11:13 config-column-user-sync-memory-get-usage-2019-10-11 11:13:25.txt
-rw-r--r-- 1 root root   17 Oct 11 11:14 config-column-user-sync-memory-get-usage-2019-10-11 11:14:26.txt
-rw-r--r-- 1 root root   17 Oct 11 11:15 config-column-user-sync-memory-get-usage-2019-10-11 11:15:27.txt
-rw-r--r-- 1 root root   17 Oct 11 11:16 config-column-user-sync-memory-get-usage-2019-10-11 11:16:28.txt
-rw-r--r-- 1 root root   17 Oct 11 11:17 config-column-user-sync-memory-get-usage-2019-10-11 11:17:29.txt
-rw-r--r-- 1 root root   17 Oct 11 11:18 config-column-user-sync-memory-get-usage-2019-10-11 11:18:30.txt
-rw-r--r-- 1 root root   17 Oct 11 11:19 config-column-user-sync-memory-get-usage-2019-10-11 11:19:31.txt
-rw-r--r-- 1 root root   17 Oct 11 11:13 log-delete-memory-get-usage-2019-10-11 11:13:21.txt
-rw-r--r-- 1 root root   17 Oct 11 11:18 log-delete-memory-get-usage-2019-10-11 11:18:22.txt

19、决定注释 Yii 2 命令行中的 sleep ,以降低命令行单次运行的时间长度。Yii 2 sleep 的持续运行 (框架在运行期间加载了大量的数据至内存中)。以便更进一步的降低内存占用(预计)。

            /* 判断 $httpCmcApiGroupIds 是否为空,如果为空,则成功退出 */
            if (empty($httpCmcApiGroupIds)) {
                // 延缓执行 60 * 60 秒
                // sleep(Yii::$app->params['cmcConsoleUser']['isEmptyYesSleepTime']);

                return ExitCode::OK;
            }

            // 延缓执行 60 秒
            // sleep(Yii::$app->params['cmcConsoleUser']['isEmptyNoSleepTime']);
            return ExitCode::OK;

            /* 判断 $cmcApiGroupIds 是否为空,如果为空,则成功退出 */
            if (empty($cmcApiGroupIds)) {
                // 延缓执行 60 * 60 秒
                // sleep(Yii::$app->params['configColumnUser']['isEmptyYesSleepTime']);

                return ExitCode::OK;
            }

            // 延缓执行 60 秒
            // sleep(Yii::$app->params['configColumnUser']['isEmptyNoSleepTime']);
            return ExitCode::OK;

            // 延缓执行 10 * 60 秒
            // sleep(static::SLEEP_TIME);

            return ExitCode::OK;

20、升级至开发环境,Docker 容器的 CPU:0.04%,内存:292MB,内存增加了 (292MB – 270MB) = 20MB 左右,未符合预期,如图7

升级至开发环境,Docker 容器的 CPU:0.04%,内存:292MB,内存增加了 (292MB - 270MB) = 20MB 左右,未符合预期

图7

21、调整间隔时间,每5秒、每5秒、每60秒 运行一次命令,Docker 容器的 CPU:0.04%,内存:109MB,内存减少了 (292MB – 109MB) = 183MB 左右,并且,比之最初始的内存占用,内存减少了 (168MB – 109MB) = 59MB 左右,已经达到内存占用减少的预期,如图8

调整间隔时间,每5秒、每5秒、每60秒 运行一次命令,Docker 容器的 CPU:0.04%,内存:109MB,内存减少了 (292MB - 109MB) = 183MB 左右,并且,比之最初始的内存占用,内存减少了 (168MB - 109MB) = 59MB 左右,已经达到内存占用减少的预期

图8

[program:yii-cmc-console-user-sync]
command = bash -c 'sleep 5 && exec php /mcloud/www/pcs-api/yii cmc-console-user/sync'
autorestart = true
startsecs = 5
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-cmc-console-user-sync-stderr.log
stdout_logfile = /data/logs/yii-cmc-console-user-sync-stdout.log
[program:yii-config-column-user-sync]
command = bash -c 'sleep 5 && exec php /mcloud/www/pcs-api/yii config-column-user/sync'
autorestart = true
startsecs = 5
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-config-column-user-sync-stderr.log
stdout_logfile = /data/logs/yii-config-column-user-sync-stdout.log
[program:yii-log-delete]
command = bash -c 'sleep 60 && exec php /mcloud/www/pcs-api/yii log/delete'
autorestart = true
startsecs = 60
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-log-delete-stderr.log
stdout_logfile = /data/logs/yii-log-delete-stdout.log

22、查看另一个产品:渠道发布的开发环境,Docker 容器的 CPU:0.04%,内存:289MB,如图9

查看另一个产品:渠道发布的开发环境,Docker 容器的 CPU:0.04%,内存:289MB

图9

23、在产品:渠道发布中,基于同样的流程,基于 Supervisor 的 crontab (bash sleep) 的实现,注释 Yii 2 命令行中的 sleep,每3秒、每3秒、每60秒 运行一次命令,升级至开发环境,Docker 容器的 CPU:0.04%,内存:182MB,内存减少了 (289MB – 182MB) = 107MB 左右,符合预期,如图10

在产品:渠道发布中,基于同样的流程,基于 Supervisor 的 crontab (bash sleep) 的实现,注释 Yii 2 命令行中的 sleep,每3秒、每3秒、每60秒 运行一次命令,升级至开发环境,Docker 容器的 CPU:0.04%,内存:182MB,内存减少了 (289MB - 182MB) = 107MB 左右,符合预期

图10

[program:yii-qq-cw-transaction-video-sync]
command = bash -c 'sleep 3 && exec php /mcloud/www/channel-pub-api/yii qq-cw-transaction-video/sync'
autorestart = true
startsecs = 3
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-qq-cw-transaction-video-sync-stderr.log
stdout_logfile = /data/logs/yii-qq-cw-transaction-video-sync-stdout.log
[program:yii-qq-cw-transaction-article-sync]
command = bash -c 'sleep 3 && exec php /mcloud/www/channel-pub-api/yii qq-cw-transaction-article/sync'
autorestart = true
startsecs = 3
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-qq-cw-transaction-article-sync-stderr.log
stdout_logfile = /data/logs/yii-qq-cw-transaction-article-sync-stdout.log
[program:yii-log-delete]
command = bash -c 'sleep 60 && exec php /mcloud/www/channel-pub-api/yii log/delete'
autorestart = true
startsecs = 60
stopwaitsecs = 10
stderr_logfile = /data/logs/yii-log-delete-stderr.log
stdout_logfile = /data/logs/yii-log-delete-stdout.log

24、总结:
(1)基于 Supervisor 的 crontab (bash sleep) 的实现,确定可以降低内存占用,crontab (bash sleep) 的值尽量控制在 60 之内,更大的话,内存占用会提升得比较厉害
(2)将 startsecs 的值调整为 bash sleep 的值,可有效降低内存占用
(3)Supervisor 的一个进程的运行时间,尽量控制得更短一些。不论是 bash sleep,还是 php sleep ,一旦其值过大,进行的运行时间过长,就会导致内存的过大占用
(4)之前 php sleep 的时候,为何内存占用得厉害,原因可能在于 Yii 2 sleep 的持续运行 (框架在运行期间加载了大量的数据至内存中)。因此,php 命令行本身对于内存的占用也是原因之一,尽量降低 php 命令行本身的运行时间,注释 php sleep
(5)在部署期间,crontab (bash sleep) 的值,建议在多次调整中,综合考虑,取最佳值

 

原创文章,作者:ItWorker,如若转载,请注明出处:https://blog.ytso.com/250484.html

(0)
上一篇 2022年4月29日
下一篇 2022年4月29日

相关推荐

发表回复

登录后才能评论