並行加速脚本任務編寫總結(協程)

滄浪水 2021-09-18 09:54:26 阅读数:312

加速 脚本 本任

1、環境准備

php7/8+
swoole4.6+

2、示例代碼


Coroutine\run(function () use ($db_work, $db_process) {
$connPool = new PDOPool((new PDOConfig)
->withHost($db_work['host'])
->withPort(3306)
// ->withUnixSocket('/tmp/mysql.sock')
->withDbName($db_work['database'])
->withCharset('utf8mb4')
->withUsername($db_work['user'])
->withPassword($db_work['password'])
);
$processPool = new PDOPool((new PDOConfig)
->withHost($db_process['host'])
->withPort(3306)
// ->withUnixSocket('/tmp/mysql.sock')
->withDbName($db_process['database'])
->withCharset('utf8mb4')
->withUsername($db_process['user'])
->withPassword($db_process['password'])
);
$channel = new Channel(60);
$conn = $connPool->get();
$data1 = $conn->query('SELECT count(*) FROM `goods_images` WHERE image_url!=\'\'')->fetchAll(PDO::FETCH_NUM);
$data2 = $conn->query('SELECT count(*) FROM `goods_sku` WHERE original_img!=\'\' AND is_delete=0')->fetchAll(PDO::FETCH_NUM);
$data3 = $conn->query('SELECT count(*) FROM `goods_spu` WHERE original_img!=\'\' AND is_delete=0')->fetchAll(PDO::FETCH_NUM);
$connPool->put($conn);
$goods_images_total = $data1[0][0];
$goods_sku_total = $data2[0][0];
$goods_spu_total = $data3[0][0];
$todo = [
'goods_images' => $goods_images_total,
'goods_sku' => $goods_sku_total,
'goods_spu' => $goods_spu_total,
];
print_r($todo);
Coroutine::create(function () use ($channel, $todo) {
foreach ($todo as $table => $total) {
$page = 0; //第一頁開始
while ($page++ <= $total) {
$channel->push(['page' => $page, 'table' => $table]);
}
}
});
Coroutine::create(function () use ($channel, $connPool, $processPool) {
while (1) {
$data = $channel->pop(10);
if ($data) {
$page = $data['page'];
$table = $data['table'];
// todo
echo($page . ' <== task執行完畢 ok' . PHP_EOL);
} else {
if ($channel->isEmpty()) {
echo 'images $channel:empty' . PHP_EOL;
break;
}
}
}
});
});

a.分析建立可以獨立運行的最小單比特cell

中間异步瓶頸mysql也使用連接池,連接池容量應該大於協程任務池容量;使用mysql連接池注意“有借有還”,$conn = $connPool->get();$connPool->put($conn);

b.隊列化需求參數,使用channel.push發送參數

加速執行如秒殺處理,應當使用隊列。

c.使用channel.pop接收參數,供給cell執行

協程池添加-取用保持一致

版权声明:本文为[滄浪水]所创,转载请带上原文链接,感谢。 https://gsmany.com/2021/09/20210918095425726J.html