Queue Worker Setup

BotMerze dispatches background work onto five named queues. Order matters in --queue= lists — always put ai_chat_processing first so user-facing chat replies are picked up before slower bulk imports.

Queue	Priority	Purpose
ai_chat_processing	Highest — real-time	Chatbot replies, RAG retrieval, OpenAI calls, proactive product recommendations. End-user latency depends entirely on this queue.
default	High	Embeddings, emails and general
woocommerce	Medium	WooCommerce store sync — products, variations, categories, orders. Heavy/long-running.
shopify	Medium	Shopify store sync — product import chain. Heavy/long-running.
generic-api	Medium	Generic API integration sync — products & orders for custom commerce backends.

ℹ️ Module queue names live in each module’s config and can be renamed:

• core/Modules/WooCommerce/config/woocommerce.php → 'queue_name' => 'woocommerce'
• core/Modules/Shopify/config/shopify.php → 'queue_name' => 'shopify'
• core/Modules/GenericApi/config/genericapi.php → 'queue_name' => 'generic-api'

If you change any of these, update the Supervisor --queue= flags below to match.

⚙️ Default driver: QUEUE_CONNECTION=database (set in core/.env). For higher throughput switch to Redis (QUEUE_CONNECTION=redis) — no app code changes are required.

Manual run (testing only)

cd /home/<domain>/public_html/core
php artisan queue:work --queue=ai_chat_processing,default,woocommerce,shopify,generic-api --tries=3 --timeout=300

Recommended production commands

Single worker (small sites) — one process drains all five queues in priority order:

php /home/<domain>/public_html/core/artisan queue:work \
    --queue=ai_chat_processing,default,woocommerce,shopify,generic-api \
    --sleep=3 \
    --tries=3 \
    --max-time=3600 \
    --timeout=300

Dedicated worker pools (recommended at scale) — separate pools so a multi-hour Shopify import never blocks a live chat reply:

# Pool A — chat traffic (run 2–4 of these)
php /home/<domain>/public_html/core/artisan queue:work \
    --queue=ai_chat_processing \
    --sleep=1 --tries=3 --max-time=3600 --timeout=180

# Pool B — default background (embeddings, mail, webhooks)
php /home/<domain>/public_html/core/artisan queue:work \
    --queue=default \
    --sleep=3 --tries=3 --max-time=3600 --timeout=600

# Pool C — commerce sync (Woo + Shopify + Generic API)
php /home/<domain>/public_html/core/artisan queue:work \
    --queue=woocommerce,shopify,generic-api \
    --sleep=3 --tries=3 --max-time=3600 --timeout=1800

⚠️ Don’t rely on queue:work --once from cron alone for high-volume sites — use Supervisor (below) for true persistent workers.

💡 After every deployment run php artisan queue:restart (or supervisorctl restart botmerze-worker:*) so workers reload your new code.

---

Supervisor Configuration

BotMerze recommends five Supervisor programs: chat queue, default queue, commerce-sync queue (Woo + Shopify + Generic API), Reverb, and an optional dedicated commerce-sync split. Save them all under /etc/supervisor/conf.d/.

1. AI chat queue (high-priority)

/etc/supervisor/conf.d/botmerze-chat-worker.conf:

[program:botmerze-chat-worker]
process_name=%(program_name)s_%(process_num)02d
command=/usr/bin/php /home/<domain>/public_html/core/artisan queue:work --queue=ai_chat_processing --sleep=1 --tries=3 --max-time=3600 --timeout=180
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=3
redirect_stderr=true
stdout_logfile=/home/<domain>/public_html/core/storage/logs/worker-chat.log
stopwaitsecs=3600

2. Default/background queue

/etc/supervisor/conf.d/botmerze-default-worker.conf:

[program:botmerze-default-worker]
process_name=%(program_name)s_%(process_num)02d
command=/usr/bin/php /home/<domain>/public_html/core/artisan queue:work --queue=default --sleep=3 --tries=3 --max-time=3600 --timeout=600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=2
redirect_stderr=true
stdout_logfile=/home/<domain>/public_html/core/storage/logs/worker-default.log
stopwaitsecs=3600

3. Commerce sync queue (WooCommerce + Shopify + Generic API)

/etc/supervisor/conf.d/botmerze-commerce-worker.conf:

[program:botmerze-commerce-worker]
process_name=%(program_name)s_%(process_num)02d
command=/usr/bin/php /home/<domain>/public_html/core/artisan queue:work --queue=woocommerce,shopify,generic-api --sleep=3 --tries=3 --max-time=3600 --timeout=1800
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=2
redirect_stderr=true
stdout_logfile=/home/<domain>/public_html/core/storage/logs/worker-commerce.log
stopwaitsecs=3600

💡 Why one combined commerce pool? Each integration only runs sync jobs when a store is connected and an admin or schedule triggers it. One pool of 2 workers covers all three platforms efficiently. Split it into per-platform programs (below) only if you actively use multiple integrations and they compete for capacity.

Optional: per-platform commerce workers

Use these instead of (not in addition to) the combined commerce program above when you need isolation:

[program:botmerze-woo-worker]
process_name=%(program_name)s_%(process_num)02d
command=/usr/bin/php /home/<domain>/public_html/core/artisan queue:work --queue=woocommerce --sleep=3 --tries=3 --max-time=3600 --timeout=1800
user=www-data
numprocs=1
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/home/<domain>/public_html/core/storage/logs/worker-woo.log
stopwaitsecs=3600

[program:botmerze-shopify-worker]
process_name=%(program_name)s_%(process_num)02d
command=/usr/bin/php /home/<domain>/public_html/core/artisan queue:work --queue=shopify --sleep=3 --tries=3 --max-time=3600 --timeout=1800
user=www-data
numprocs=1
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/home/<domain>/public_html/core/storage/logs/worker-shopify.log
stopwaitsecs=3600

[program:botmerze-genericapi-worker]
process_name=%(program_name)s_%(process_num)02d
command=/usr/bin/php /home/<domain>/public_html/core/artisan queue:work --queue=generic-api --sleep=3 --tries=3 --max-time=3600 --timeout=1800
user=www-data
numprocs=1
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/home/<domain>/public_html/core/storage/logs/worker-genericapi.log
stopwaitsecs=3600

4. Laravel Reverb WebSocket server

/etc/supervisor/conf.d/botmerze-reverb.conf:

[program:botmerze-reverb]
process_name=%(program_name)s
command=/usr/bin/php /home/<domain>/public_html/core/artisan reverb:start --host=0.0.0.0 --port=8080
autostart=true
autorestart=true
user=www-data
redirect_stderr=true
stdout_logfile=/home/<domain>/public_html/core/storage/logs/reverb.log
stopwaitsecs=10

Combined alternative (single program)

If you’d rather run one program that processes both queues with priority:

[program:botmerze-worker]
process_name=%(program_name)s_%(process_num)02d
command=/usr/bin/php /home/<domain>/public_html/core/artisan queue:work --queue=ai_chat_processing,default,woocommerce,shopify,generic-api --sleep=3 --tries=3 --max-time=3600 --timeout=600
user=www-data
numprocs=3
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
redirect_stderr=true
stdout_logfile=/home/<domain>/public_html/core/storage/logs/worker.log
stopwaitsecs=3600

⚠️ The combined form is simpler but a slow Shopify/Woo import on woocommerce/shopify/generic-api can stall chat replies. For production use the dedicated programs above.

Apply

sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl start botmerze-chat-worker:* \
                        botmerze-default-worker:* \
                        botmerze-commerce-worker:* \
                        botmerze-reverb
sudo supervisorctl status

Expected status (with the recommended program set):

botmerze-chat-worker:botmerze-chat-worker_00         RUNNING
botmerze-chat-worker:botmerze-chat-worker_01         RUNNING
botmerze-chat-worker:botmerze-chat-worker_02         RUNNING
botmerze-default-worker:botmerze-default-worker_00   RUNNING
botmerze-default-worker:botmerze-default-worker_01   RUNNING
botmerze-commerce-worker:botmerze-commerce-worker_00 RUNNING
botmerze-commerce-worker:botmerze-commerce-worker_01 RUNNING
botmerze-reverb                                      RUNNING

💡 After every deployment restart all workers so they reload your new code:

sudo supervisorctl restart botmerze-chat-worker:* \
                           botmerze-default-worker:* \
                           botmerze-commerce-worker:*