For arlanix/notification 1.0.0

Queue, cron and command line

The queue

A queue message only points at a row of arlanix_notification_delivery; the row is the truth. A delivery is claimed before it is sent (status from queued to sending, with a claim token), so a message delivered twice, or taken by two consumers, is sent once. Only that claim records what became of it: a consumer still sending after 15 minutes, whose delivery the cron took back, writes over nothing done since, and the log warns that the delivery may go twice.

Deliveries published inside a database transaction go on the queue once it commits: a RabbitMQ consumer would otherwise take a delivery id before its row is written.

Running the consumer

The cron's consumer runner (cron_consumers_runner) starts it every minute when messages wait. Each run takes up to 1000 messages, in batches of 100, and ends.

To run it by hand or under supervisor:

bin/magento queue:consumers:start arlanix_notification_deliver
bin/magento queue:consumers:start arlanix_notification_deliver --max-messages=100 --batch-size=50

Without --max-messages it runs until it is stopped, taking a batch every 10 seconds, and ends when the configuration is saved (Magento's poison pill), so supervisor starts it again with the new settings.

Several at once

A slow or unreachable server holds the process sending to it for up to its timeout. With several consumers the others go on:

// app/etc/env.php
'cron_consumers_runner' => [
    'cron_run' => true,
    'max_messages' => 1000,
    'multiple_processes' => [
        'arlanix_notification_deliver' => 2,
    ],
],

Each process merges only its own batch, so a burst may go as two or three digests instead of one, and deliveries may arrive out of order.

Digests

Arlanix\Notification\Model\Queue\DeliveryMerger, registered for the consumer in MergerFactory, hands a batch's deliveries to DeliveryHandler as one message, and the dispatcher groups them by subscription: each group goes to its protocol as one Digest, the same message repeated as one entry with its count, or as several of up to 25 entries (DigestBuilder::MAX_ENTRIES). When one of them fails, it and the ones after it wait for their retry; those sent before stay sent. One message per batch, not one per subscription: Magento's MergedMessageProcessor acknowledges the wrong queue messages when a merger returns more than one, and RabbitMQ then closes the consumer's channel. What a batch holds is what waited when it was read: a minute's worth under the cron runner, 10 seconds' worth for a consumer that runs on.

Cron jobs

The group runs in a process of its own (use_separate_process).

Delivery statuses

Command line

bin/magento arlanix:notification:publish <topic> <subject> [--body=…] [--severity=info|warning|error] [--attribute=label=value]… [--source=…]

<topic> is a topic's id or its name. It publishes as a module would, through the queue, and says how many deliveries it queued:

bin/magento arlanix:notification:publish "Feed alerts" "Disk almost full" --severity=warning --attribute="Free=2 GB" --source=monitoring
Published to "Feed alerts": 1 delivery(ies) queued.

A topic that is switched off, or an unknown one, is refused with a non-zero exit, and so is a message that could not be recorded, with what went wrong. A queue that refuses the deliveries is no failure of the command: the cron puts them on the queue again.

bin/magento arlanix:notification:monitoring:check

With monitoring, runs its check now, as the cron job does and in its crontab area: what changed goes to the rules that answer it. A check that cannot look is logged to var/log/arlanix.log, and the others still run.