For arlanix/product-feed 1.0.0

Channels

This page is for the PHP developer who needs feeds to go somewhere the module does not deliver to: an FTP server, a cloud bucket, a directory another system collects from, or a platform's API. A channel is that destination. The store chooses it in the Channel field at the top of a feed's Channel section. The module registers three:

A channel is one of two kinds:

• A file channel writes the feed as a file at the feed's schedule, or when the store runs it from the grid, then hands the file to a delivery adapter. The module does the mapping, writing, scheduling and retries; you write the adapter. Most destinations are this kind.
• A push channel sends products one by one, whenever one changes, and records what the platform did with each. Choose it only for an API that takes products individually.

Add a file channel

Write the delivery adapter

The following adapter copies a feed's file into a directory under var/export, where another system collects it. The feed's setting channel/export_directory names the directory:

<?php
declare(strict_types=1);

namespace Vendor\Module\Model\Delivery;

use Arlanix\Feed\Model\Channel\ChannelException;
use Arlanix\Feed\Model\Channel\DeliveryResult;
use Arlanix\Feed\Model\Delivery\DeliveryAdapterInterface;
use Arlanix\Feed\Model\Feed;
use Arlanix\Feed\Model\Output\Artifact\ArtifactStorageInterface;
use Arlanix\Feed\Model\Output\Artifact\StoredArtifact;
use Magento\Framework\App\Filesystem\DirectoryList;
use Magento\Framework\Filesystem;
use Magento\Framework\Phrase;

class ExportDirectoryAdapter implements DeliveryAdapterInterface
{
    public const DIRECTORY = 'channel/export_directory';

    public function __construct(
        private readonly ArtifactStorageInterface $storage,
        private readonly Filesystem $filesystem
    ) {
    }

    public function deliver(Feed $feed, StoredArtifact $artifact): DeliveryResult
    {
        $directory = trim((string)$feed->getSetting(self::DIRECTORY), '/');
        if ($directory === '' || str_contains($directory, '..')) {
            throw new ChannelException(new Phrase('Choose the export directory of this feed.'));
        }
        $path = $directory . '/' . $feed->getIdentifier() . '.' . $artifact->getExtension();

        $source = $this->storage->open($artifact);
        $target = $this->filesystem->getDirectoryWrite(DirectoryList::VAR_EXPORT)->openFile($path, 'w');
        try {
            while (!$source->eof()) {
                $target->write($source->read(1048576));
            }
        } finally {
            $source->close();
            $target->close();
        }

        return new DeliveryResult(1, [], ['destination' => 'var/export/' . $path]);
    }
}

deliver() is called once per run, with the file already written:

• $artifact is the file, in the module's temporary storage. $this->storage->open($artifact) gives a Magento\Framework\Filesystem\File\ReadInterface on it; close it when you are done. getExtension(), getContentType(), getSize() and getChecksum() describe it. The module deletes the file after deliver() returns, so copy or upload it within the call.
• $feed->getSetting('channel/…') reads the feed's settings; see Add the channel's settings.
• To fail the delivery, throw Arlanix\Feed\Model\Channel\ChannelException. The run fails with its message, and the feed is tried again after 15 minutes, then after longer and longer pauses; see the error line of the log table.
• Return a DeliveryResult with 1 and [], and in the third argument what the run should record. Under destination or url, the run's message names where the file went: Delivered 1779 row(s) to var/export/marketplace/google-shopping.csv. Prefer url for an address the platform fetches, and destination for a path on another server.

Register the channel

In your module's etc/di.xml, make the channel a FileChannel with your adapter, and add it to ChannelPool under its type:

<virtualType name="Vendor\Module\Model\Delivery\ExportDirectoryChannel"
             type="Arlanix\Feed\Model\DeliveryV2\FileChannel">
    <arguments>
        <argument name="deliveryAdapter"
                  xsi:type="object">Vendor\Module\Model\Delivery\ExportDirectoryAdapter</argument>
    </arguments>
</virtualType>

<type name="Arlanix\Feed\Model\Channel\ChannelPool">
    <arguments>
        <argument name="channels" xsi:type="array">
            <item name="export_directory" xsi:type="array">
                <item name="label" xsi:type="string" translate="true">Export directory</item>
                <item name="channel"
                      xsi:type="object">Vendor\Module\Model\Delivery\ExportDirectoryChannel</item>
                <item name="on_demand" xsi:type="array">
                    <item name="label" xsi:type="string" translate="true">Export now</item>
                    <item name="queued" xsi:type="string" translate="true">Feed "%1" is queued. Its file is exported by the feed cron, usually within a minute.</item>
                </item>
                <item name="deleted" xsi:type="string" translate="true">Files already exported were left in the export directory.</item>
            </item>
        </argument>
    </arguments>
</type>

The item's name is the channel's type, which a feed stores. Its keys:

A feed on a file channel gets the Format field and the Schedule section from the module, with no form XML of yours; see Writers for the formats.

Add the channel's settings

A channel's settings go in the feed form's Channel section, are stored with the feed under channel/, and are read with $feed->getSetting(). Each one takes three pieces in your module.

1. Register the setting in etc/di.xml, with the normaliser that cleans what the form sends. getSetting() throws for a path that is not registered.

   <virtualType name="Arlanix\Feed\Model\Feed\SettingPool">
       <arguments>
           <argument name="settings" xsi:type="array">
               <item name="channel/export_directory" xsi:type="array">
                   <item name="normalise"
                         xsi:type="object">Arlanix\Core\Model\Settings\Normaliser\StringNormaliser</item>
               </item>
           </argument>
       </arguments>
   </virtualType>
   

   For a number, use Arlanix\Core\Model\Settings\Normaliser\IntNormaliser.

2. Add the field to the Channel section in view/adminhtml/ui_component/arlanix_feed_form.xml. variants lists the channel types the field belongs to; the form shows it only while Channel is one of them.

   <form xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Ui:etc/ui_configuration.xsd">
       <fieldset name="channel">
           <field name="export_directory" formElement="input" sortOrder="100">
               <argument name="data" xsi:type="array">
                   <item name="config" xsi:type="array">
                       <item name="variants" xsi:type="array">
                           <item name="0" xsi:type="string">export_directory</item>
                       </item>
                   </item>
               </argument>
               <settings>
                   <label translate="true">Export directory</label>
                   <notice translate="true">A directory under var/export, such as marketplace.</notice>
                   <dataType>text</dataType>
                   <validation>
                       <rule name="required-entry" xsi:type="boolean">true</rule>
                   </validation>
               </settings>
           </field>
       </fieldset>
   </form>
   

3. Optional: check the settings when the feed is saved. A validator that adds messages refuses the save and shows them. Check only feeds of your channel, and only when the save carries the settings:

   <?php
   declare(strict_types=1);
   
   namespace Vendor\Module\Model\Feed;
   
   use Arlanix\Feed\Model\Feed;
   use Magento\Framework\Phrase;
   use Magento\Framework\Validator\AbstractValidator;
   use Vendor\Module\Model\Delivery\ExportDirectoryAdapter;
   
   class ExportDirectoryValidator extends AbstractValidator
   {
       public function isValid($value): bool
       {
           $this->_clearMessages();
           if (!$value instanceof Feed || $value->getChannelType() !== 'export_directory'
               || !$value->hasSection(Feed::SETTINGS)) {
               return true;
           }
           $directory = (string)$value->getSetting(ExportDirectoryAdapter::DIRECTORY);
           if (str_contains($directory, '..')) {
               $this->_addMessages([(string)new Phrase('The export directory must stay under var/export.')]);
           }
   
           return !$this->hasMessages();
       }
   }
   

   Register it on the feed in etc/di.xml:

   <type name="Arlanix\Feed\Model\Feed">
       <arguments>
           <argument name="validators" xsi:type="array">
               <item name="export_directory" xsi:type="object">Vendor\Module\Model\Feed\ExportDirectoryValidator</item>
           </argument>
       </arguments>
   </type>
   

Then run bin/magento setup:upgrade and bin/magento cache:clean config. Open a feed, set Channel to Export directory, fill in Export directory, save, and choose Export now in the row's Select menu.

Add a push channel

A push channel is for a platform whose API takes products one at a time. The module keeps a record per product row: what was sent, what the platform answered, and what has changed since. Each run sends the rows that changed, retries refused rows, and takes down rows the feed no longer has.

Implement Arlanix\Feed\Model\Delivery\Push\PushChannelInterface, then register it in etc/di.xml as a PushChannel:

<virtualType name="Vendor\Module\Model\Channel\MarketplaceChannel"
             type="Arlanix\Feed\Model\DeliveryV2\PushChannel">
    <arguments>
        <argument name="pushChannel" xsi:type="object">Vendor\Module\Model\Channel\MarketplaceApi</argument>
    </arguments>
</virtualType>

<type name="Arlanix\Feed\Model\Channel\ChannelPool">
    <arguments>
        <argument name="channels" xsi:type="array">
            <item name="marketplace_api" xsi:type="array">
                <item name="label" xsi:type="string" translate="true">Marketplace (API)</item>
                <item name="channel" xsi:type="object">Vendor\Module\Model\Channel\MarketplaceChannel</item>
                <item name="deleted" xsi:type="string" translate="true">The products of feed "%1" stay on the marketplace until you remove them there.</item>
            </item>
        </argument>
    </arguments>
</type>

A push channel has no on_demand and no publishes_file: its feeds run whenever their products change, and the form hides Format and Schedule for them. Its settings are added as for a file channel; see Add the channel's settings.

What the interface asks

A run calls start(), then remove() for the rows to take down and accept() for the rows to send, 25 rows a call, then finish(). A run that fails calls discard() instead of finish(). A run with nothing to send or take down calls none of them.

accept() and remove() answer for each row in the result's outcomes, keyed by the row's key:

return new DeliveryResult(
    $accepted,                                    // how many the platform took
    $errors,                                      // row key => message, for each row refused
    [],
    [
        '42' => ['status' => DeliveryResult::OUTCOME_SYNCED, 'external_id' => 'offer/42'],
        '43' => ['status' => DeliveryResult::OUTCOME_FAILED,
                 'error_code' => 'invalid_gtin', 'error_message' => 'The GTIN is not valid.'],
    ]
);

• OUTCOME_SYNCED marks a row as sent. Its external_id, the platform's name for it, is given back to remove() when the row is taken down.
• OUTCOME_FAILED marks a row as refused: the platform said no to this product. It is offered again up to four times, 15 minutes after the first refusal, 30 after the second and so on, then not until its row changes. The store sees the message in Why?.
• From remove(), OUTCOME_DELETED marks a row as taken down, or already gone.
• A row with no outcome counts as refused.

When the platform fails as a whole, rather than for one product, stop instead of refusing every row: return what was done so far with a reason in the fifth argument, stoppedBecause. The rows already answered keep their outcome, the rows not reached keep their place for the next run, and the run fails with Stopped part way: and the reason. The feed then waits 5 minutes, doubling up to an hour, before its next run; saving the feed ends the wait.

A push run has about a minute. The module starts no call to accept() or remove() once it is up, and sends the rest on the next run. A channel that retries a call to the platform reads $context->getDeadline() in start(), so that it stops waiting when the run's minute ends.

Arlanix_FeedGoogleMerchant is a complete push channel: Arlanix\FeedGoogleMerchant\Model\Channel\MerchantApiChannel in vendor/arlanix/module-feed-google-merchant.