Actions and Metrics

Exports

Export the rows of a search to a CSV file in the background, in the columns the client picks, with a queued, batched action.

The package has no export endpoint: a queued, batched action covers it. The client sends the columns it wants and the search of its list. Each job writes one chunk of rows to a CSV part, and the batch joins the parts into one file that the client downloads.

A small export can stream in the request instead, with lazy(). See Searching outside HTTP.

The action

<?php

namespace App\Rest\Actions;

use App\Models\Ticket;
use Illuminate\Bus\PendingBatch;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Support\Collection;
use Illuminate\Support\Facades\Storage;
use Illuminate\Support\Str;
use Illuminate\Validation\Rule;
use Lomkit\Rest\Actions\Action;
use Lomkit\Rest\Actions\ActionFields;
use Lomkit\Rest\Actions\ActionResponse;
use Lomkit\Rest\Contracts\BatchableAction;
use Lomkit\Rest\Fields\Json;

class ExportTickets extends Action implements BatchableAction, ShouldQueue
{
    public const array COLUMNS = ['id', 'title', 'status'];

    public int $chunkCount = 500;

    public ?int $maxResources = 50000;

    private string $export = '';

    #[\Override]
    public function fields(): array
    {
        return [Json::make('columns')->rules(['required', 'list', Rule::in(self::COLUMNS)])];
    }

    #[\Override]
    public function withBatch(ActionFields $fields, PendingBatch $batch): void
    {
        $export = $this->export = (string) Str::uuid();
        $columns = $fields->columns;

        $batch->withOption('export', $export)->then(static function () use ($export, $columns): void {
            $disk = Storage::disk('exports');
            $csv = fopen('php://temp', 'w+');
            fputcsv($csv, $columns, escape: '');
            $parts = $disk->files($export);
            sort($parts, SORT_NATURAL);

            foreach ($parts as $part) {
                stream_copy_to_stream($disk->readStream($part), $csv);
            }

            rewind($csv);
            $disk->writeStream("{$export}.csv", $csv);
            $disk->deleteDirectory($export);
        });
    }

    /** @param Collection<int, Ticket> $models */
    #[\Override]
    public function handle(ActionFields $fields, Collection $models): void
    {
        $csv = fopen('php://temp', 'w+');

        foreach ($models as $ticket) {
            fputcsv($csv, $ticket->only($fields->columns), escape: '');
        }

        rewind($csv);
        Storage::disk('exports')->writeStream($this->batch()?->options['export'].'/'.$models->first()?->getKey().'.csv', $csv);
    }

    #[\Override]
    public function respond(ActionFields $fields, int $impacted): ActionResponse
    {
        if ($impacted === 0) {
            return ActionResponse::message('No rows to export.');
        }

        return ActionResponse::download(Storage::disk('exports')->temporaryUrl("{$this->export}.csv", now()->addDay()), 'tickets.csv');
    }
}

Declare it in the actions() of the resource, and add the progress route:

public function actions(): array
{
    return [ExportTickets::make()];
}

Rest::resource('tickets', TicketController::class)->withBatchProgress();

exports is a disk of config/filesystems.php. Every worker must reach it, and it must build temporary URLs: s3, or local with 'serve' => true.

Running an export

POST /api/tickets/actions/export-tickets

{
  "fields": [{"name": "columns", "value": ["title", "id"]}],
  "search": {"filters": [{"field": "status", "value": "open"}]}
}

{"data": {"impacted": 5, "batch": "9f0c4d2e-5a1b-4c3d-8e7f-0a1b2c3d4e5f", "result": {"type": "download", "url": "https://files.example.com/3e1f6a2b-7c4d-4e5f-9a0b-1c2d3e4f5a6b.csv?expires=1796205600&signature=…", "name": "tickets.csv"}}}

respond() runs once the jobs are pushed, before they run. The client follows the batch, then downloads the file once finished is true:

GET /api/tickets/actions/export-tickets/batches/9f0c4d2e-5a1b-4c3d-8e7f-0a1b2c3d4e5f

title,id
"Printer jam, floor 2",8
"Password reset",9
"Monitor flicker",12
"Badge reader",13
"Desk phone",14

A search that matches no row pushes no job and writes no file, so the action answers a message: {"type": "message", "message": "No rows to export."}.

How it works

Rows. handle() receives each chunk of the rows that the search matches inside the perimeter of the resource. The worker loads them again by key, as the user who sent the request. A chunk holds $chunkCount rows, or more when the rows need more than rest.limits.sizes.jobs jobs (100). $maxResources caps the rows of one export: more is a 422.
Columns. columns is a Json field. list and Rule::in() accept only a list of the allowed names; anything else is a 422 under fields.columns. only() returns the values in the order the client gave.
Parts. The jobs of a batch can run at the same time, so each job writes its own part, named after the first key of its chunk. Once every job succeeded, the then() callback of the batch joins the parts in that order under a header row. Without sorts, the chunks follow the key, so the file lists the rows by key.
Labels. To name the columns from the labels of the resource's fields, build the header in withBatch() with $this->resource->definition()->fields()->get($column)->getLabel() ?? $column for each column.
XLSX. Keep the CSV parts and write the joined file in then() with any spreadsheet writer library. The rest stays the same.

Metrics

Compute counts, totals, breakdowns, trends and top lists over the rows of a resource, with ranges, time zones and a result cache.

Policies

How Laravel policies guard each endpoint, each resource a request reaches and each row it writes, before validation and before any query.