1. 云图网首页
  2. 技术专区
  3. 大前端

在 Yii2 高级模板中基于 Yii2 队列扩展实现异步执行任务

• 大前端, 技术专区, 数据库, 架构设计, 研发管理, 移动开发, 编程笔记

1、安装 Yii2 队列扩展，执行以下命令

composer require --prefer-dist yiisoft/yii2-queue

2、查看 composer.json，已经安装成功

    "require": {
        "yiisoft/yii2-queue": "^2.1"
    },

3、把多个队列组件注册到控制台，编辑 /console/config/main.php

    "require": {
        "yiisoft/yii2-queue": "^2.1"
    },

4、多个队列组件的配置，编辑 /environments/dev/common/config/main-local.php、/environments/prod/common/config/main-local.php

        'copyAssetQueue' => [ // 复制资源文件队列
            'class' => 'yii/queue/redis/Queue',
            'redis' => 'redis', // Redis 连接组件或它的配置
            'channel' => 'cpa:queue:copy:asset', // 队列键前缀
            'ttr' => 10 * 60, // 作业处理的最长时间，单位(秒)
            'as log' => 'yii/queue/LogBehavior',
        ],
        'qqCwVideoMultipartUploadQueue' => [ // 企鹅号的内容网站应用的视频文件分片上传队列
            'class' => 'yii/queue/redis/Queue',
            'redis' => 'redis', // Redis 连接组件或它的配置
            'channel' => 'cpa:queue:qq:cw:video:multipart:upload', // 队列键前缀
            'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
            'as log' => 'yii/queue/LogBehavior',
        ],
        'pubArticleQueue' => [ // 发布文章队列
            'class' => 'yii/queue/redis/Queue',
            'redis' => 'redis', // Redis 连接组件或它的配置
            'channel' => 'cpa:queue:pub:article', // 队列键前缀
            'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
            'as log' => 'yii/queue/LogBehavior',
        ],
        'sourceCallbackQueue' => [ // 来源回调队列
            'class' => 'yii/queue/redis/Queue',
            'redis' => 'redis', // Redis 连接组件或它的配置
            'channel' => 'cpa:queue:source:callback', // 队列键前缀
            'ttr' => 5 * 60, // 作业处理的最长时间，单位(秒)
            'as log' => 'yii/queue/LogBehavior',
        ],

5、定义每个被发送到队列的任务的基础类，新建 /common/jobs/Job.php

<?php
/**
 * Created by PhpStorm.
 * User: terryhong
 * Date: 2018/9/7
 * Time: 下午4:54
 */

namespace common/jobs;

use yii/base/BaseObject;
use yii/queue/JobInterface;

/**
 * 队列的任务的基础类
 *
 * @author Qiang Wang <shuijingwanwq@163.com>
 * @since 1.0
 */
class Job extends BaseObject implements JobInterface
{
    public function execute($queue){

    }
}

6、每个被发送到队列的任务应该被定义为一个单独的类。基于官方示例，如果您需要下载并保存一个文件，该类可能看起来如下

<?php
/**
 * Created by PhpStorm.
 * User: Qiang Wang
 * Date: 2018/10/18
 * Time: 10:39
 */

namespace common/jobs;

/**
 * 下载并保存一个文件
 *
 * @author Qiang Wang <shuijingwanwq@163.com>
 * @since 1.0
 */
class DownloadJob extends Job
{
    public $url;
    public $file;

    public function execute($queue)
    {
        file_put_contents($this->file, file_get_contents($this->url));
    }
}

7、在 qq 应用中定义对应的下载类，新建 /qq/jobs/DownloadJob.php

<?php
/**
 * Created by PhpStorm.
 * User: Qiang Wang
 * Date: 2018/10/19
 * Time: 10:28
 */

namespace qq/jobs;

/**
 * 下载并保存一个文件
 *
 * @author Qiang Wang <shuijingwanwq@163.com>
 * @since 1.0
 */
class DownloadJob extends /common/jobs/DownloadJob
{

}

8、作业状态的测试，编辑 /qq/rests/qq_cw_app/IndexAction.php

<?php
/**
 * @link http://www.yiiframework.com/
 * @copyright Copyright (c) 2008 Yii Software LLC
 * @license http://www.yiiframework.com/license/
 */

namespace qq/rests/qq_cw_app;

use Yii;
use qq/jobs/DownloadJob;
use yii/data/ActiveDataProvider;
use yii/base/InvalidConfigException;

/**
 *
 * For more details and usage information on IndexAction, see the [guide article on rest controllers](guide:rest-controllers).
 *
 * @author Qiang Wang <shuijingwanwq@163.com>
 * @since 1.0
 */
class IndexAction extends /yii/rest/IndexAction
{

    /**
     * Prepares the data provider that should return the requested collection of the models.
     * @return ActiveDataProvider
     * @throws InvalidConfigException if the configuration is invalid.
     */
    protected function prepareDataProvider()
    {
        // 将作业推送到队列并获得其ID
        $id = Yii::$app->copyAssetQueue->push(new DownloadJob([
            'url' => 'http://wjdev2.chinamcloud.com:8661/robots.txt',
            'file' => './qq/runtime/robots.txt',
        ]));
        // 这个作业等待执行
        var_dump(Yii::$app->copyAssetQueue->isWaiting($id));

        // Worker 从队列获取作业，并执行它
        var_dump(Yii::$app->copyAssetQueue->isReserved($id));

        // Worker 作业执行完成
        var_dump(Yii::$app->copyAssetQueue->isDone($id));
        echo $id;
        exit;

    }
}

9、打开网址：http://api.channel-pub-api.localhost/qq/v1/qq-cw-apps?group_id=spider ，打印结果，发现任务并未执行

bool(true)
bool(false)
bool(false)
1

10、查看 Redis ，队列已经存入，如图1

查看 Redis ，队列已经存入
图1
600;O:19:"qq/jobs/DownloadJob":2:{s:3:"url";s:45:"http://wjdev2.chinamcloud.com:8661/robots.txt";s:4:"file";s:23:"./qq/runtime/robots.txt";}

11、info 命令打印关于队列状态的信息，一个任务状态为等待

./yii copy-asset-queue/info

Jobs
- waiting: 1
- delayed: 0
- reserved: 0
- done: 0

12、run 命令获取并执行循环中的任务，直到队列为空，如图2

run 命令获取并执行循环中的任务，直到队列为空
图2
./yii copy-asset-queue/run --verbose=1 --isolate=1 --color=0
./yii copy-asset-queue/info

2018-10-19 11:07:30 [pid: 36016] - Worker is started
2018-10-19 11:07:30 [2] qq/jobs/DownloadJob (attempt: 1, pid: 36016) - Started
2018-10-19 11:07:30 [2] qq/jobs/DownloadJob (attempt: 1, pid: 36016) - Done (0.255 s)
2018-10-19 11:07:30 [pid: 36016] - Worker is stopped (0:00:00)

Jobs
- waiting: 0
- delayed: 0
- reserved: 0
- done: 1

13、查看 Redis ，键：cpa:queue:copy:asset.messages、cpa:queue:copy:asset.waiting 均已经被删除，如图3

查看 Redis ，键：cpa:queue:copy:asset.messages、cpa:queue:copy:asset.waiting 均已经被删除
图3

14、文件 /qq/runtime/robots.txt 已经被保存，如图4

文件 /qq/runtime/robots.txt 已经被保存
图4

15、为了使开发过程更加友好，您可以向 Yii2 调试模块添加一个面板。面板显示 计数器和队列任务列表。编辑 /environments/dev/api/config/main-local.php、/environments/dev/backend/config/main-local.php、/environments/dev/frontend/config/main-local.php、/environments/dev/qq/config/main-local.php、/environments/dev/rpc/config/main-local.php、/environments/dev/wx/config/main-local.php

if (!YII_ENV_TEST) {
    // configuration adjustments for 'dev' environment
    $config['bootstrap'][] = 'debug';
    $config['modules']['debug'] = [
        'class' => 'yii/debug/Module',
        'panels' => [
            'queue' => 'yii/queue/debug/Panel',
        ],
    ];

    $config['bootstrap'][] = 'gii';
    $config['modules']['gii'] = [
        'class' => 'yii/gii/Module',
    ];
}

16、查看 Yii2 调试模块，已经增加 queue 面板，如图5

查看 Yii2 调试模块，已经增加 queue 面板
图5

17、本地环境是 Windows 10，准备基于 Supervisor 部署至开发、测试、演示、生产环境上，后四者皆是 Centos 7，Supervisor 是Linux的进程监视器。它会自动启动您的控制台进程。

/build/c_files/etc/supervisord.d/yii-copy-asset-queue-worker.ini

[program:yii-copy-asset-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-copy-asset-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-copy-asset-queue-worker-stdout.log

/build/c_files/etc/supervisord.d/yii-qq-cw-video-multipart-upload-queue-worker.ini

[program:yii-qq-cw-video-multipart-upload-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii qq-cw-video-multipart-upload-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stdout.log

/build/c_files/etc/supervisord.d/yii-pub-article-queue-worker.ini

[program:yii-pub-article-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii pub-article-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-pub-article-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-pub-article-queue-worker-stdout.log

/build/c_files/etc/supervisord.d/yii-source-callback-queue-worker.ini

[program:yii-source-callback-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii source-callback-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 1
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-source-callback-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-source-callback-queue-worker-stdout.log

18、容器升级完毕后，执行命令，如图6

容器升级完毕后，执行命令
图6
php /sobey/www/channel-pub-api/init --env=Development --overwrite=All
php /sobey/www/channel-pub-api/yii migrate --interactive=0

19、打开网址：http://wjdev2.chinamcloud.com:8661/qq/v1/qq-cw-apps?group_id=spider ，打印结果，发现任务并未执行

bool(true)
bool(false)
bool(false)
1

20、info 命令打印关于队列状态的信息，一个任务状态为等待

php /sobey/www/channel-pub-api/yii copy-asset-queue/info

Jobs
- waiting: 1
- delayed: 0
- reserved: 0
- done: 0

21、手动运行 queue/listen 命令监听和处理队列任务

php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0

2018-10-19 15:56:04 [pid: 1151] - Worker is started
2018-10-19 15:56:05 [5] qq/jobs/DownloadJob (attempt: 1, pid: 1151) - Started
2018-10-19 15:56:05 [5] qq/jobs/DownloadJob (attempt: 1, pid: 1151) - Error (0.031 s)
> yii/base/ErrorException: file_put_contents(./qq/runtime/robots.txt): failed to open stream: No such file or directory
^C2018-10-19 15:56:13 [pid: 1151] - Worker is stopped (0:00:09)

22、作业状态的测试，编辑 /qq/rests/qq_cw_app/IndexAction.php，设置 file 为绝对路径

        // 将作业推送到队列并获得其ID
        $id = Yii::$app->copyAssetQueue->push(new DownloadJob([
            'url' => 'http://wjdev2.chinamcloud.com:8661/robots.txt',
            'file' => '/sobey/www/channel-pub-api/qq/runtime/robots.txt',
        ]));

23、升级至开发环境后，手动运行 queue/listen 命令监听和处理队列任务，队列任务执行成功，但是 Supervisor 配置未生效，未自动处理队列中的任务，一直处于等待中

php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0

2018-10-19 17:04:57 [pid: 1717] - Worker is started
2018-10-19 17:04:57 [10] qq/jobs/DownloadJob (attempt: 1, pid: 1717) - Started
2018-10-19 17:04:57 [10] qq/jobs/DownloadJob (attempt: 1, pid: 1717) - Done (0.021 s)
^C2018-10-19 17:05:01 [pid: 1717] - Worker is stopped (0:00:04)

24、本地环境是 Windows 10，准备基于 Supervisor 部署至开发、测试、演示、生产环境上，后四者皆是 Centos 7，Supervisor 是Linux的进程监视器。它会自动启动您的控制台进程。startsecs 的值调整为 0

/build/c_files/etc/supervisord.d/yii-copy-asset-queue-worker.ini

[program:yii-copy-asset-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii copy-asset-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-copy-asset-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-copy-asset-queue-worker-stdout.log

/build/c_files/etc/supervisord.d/yii-qq-cw-video-multipart-upload-queue-worker.ini

[program:yii-qq-cw-video-multipart-upload-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii qq-cw-video-multipart-upload-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-qq-cw-video-multipart-upload-queue-worker-stdout.log

/build/c_files/etc/supervisord.d/yii-pub-article-queue-worker.ini

[program:yii-pub-article-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii pub-article-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-pub-article-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-pub-article-queue-worker-stdout.log

/build/c_files/etc/supervisord.d/yii-source-callback-queue-worker.ini

[program:yii-source-callback-queue-worker]
process_name = %(program_name)s_%(process_num)02d
command = php /sobey/www/channel-pub-api/yii source-callback-queue/listen --verbose=1 --color=0
autostart = true
autorestart = true
startsecs = 0
stopwaitsecs = 10
numprocs = 4
redirect_stderr = true
stderr_logfile = /data/logs/yii-source-callback-queue-worker-stderr.log
stdout_logfile = /data/logs/yii-source-callback-queue-worker-stdout.log

25、Supervisor 配置已生效，会自动处理队列中的任务，/sobey/www/channel-pub-api/qq/runtime/robots.txt 文件的生成时间时刻在更新中，如图7

Supervisor 配置已生效，会自动处理队列中的任务，/sobey/www/channel-pub-api/qq/runtime/robots.txt 文件的生成时间时刻在更新中
图7

原创文章，作者：3628473679，如若转载，请注明出处：https://blog.ytso.com/181252.html

赞 (0)
0
上一篇 2021年10月31日
下一篇 2021年10月31日

相关推荐

发表回复

登录后才能评论