# Engine extensions

> Restrict every search engine query, declare server-only index attributes and lists of strings, and provide the profile of a Scout engine.

Three kinds act on the searches that run on a search engine through `ScoutEngine`: `RestrictsEngineQueries`, `DeclaresIndexAttributes` and `ProvidesEngineProfiles`. They throw `Lomkit\Rest\Exceptions\InvalidEngineExtensionException`, which names the extension by its name and its class. See [Search engines](/search/search-engines) for the engine searches themselves.

## Restricting engine searches

`RestrictsEngineQueries::restrictEngine(EngineRestrictionContext $context)` restricts each engine query of a resource. Here, the tenant extension of [Query extensions](/extensions/queries#restricting-queries) also restricts the engine, on an attribute that it declares:

```php [Vendor/Tenancy/TenantExtension.php]
use Lomkit\Rest\Extensions\Contexts\EngineRestrictionContext;
use Lomkit\Rest\Extensions\Contexts\IndexAttributesContext;
use Lomkit\Rest\Extensions\Contexts\RestrictionContext;
use Lomkit\Rest\Extensions\Extension;
use Lomkit\Rest\Extensions\Kinds\DeclaresIndexAttributes;
use Lomkit\Rest\Extensions\Kinds\RestrictsEngineQueries;
use Lomkit\Rest\Extensions\Kinds\RestrictsQueries;
use Lomkit\Rest\Fields\Integer;

final class TenantExtension extends Extension implements DeclaresIndexAttributes, RestrictsEngineQueries, RestrictsQueries
{
    public function __construct(private readonly CurrentTenant $tenant) {}

    #[\Override]
    public static function name(): string
    {
        return 'tenant';
    }

    #[\Override]
    public function restrict(RestrictionContext $context): void
    {
        // the SQL restriction, see Query extensions
    }

    #[\Override]
    public function indexAttributes(IndexAttributesContext $context): array
    {
        return [Integer::make('tenant_id')];
    }

    #[\Override]
    public function restrictEngine(EngineRestrictionContext $context): void
    {
        if ($context->rowsRestrictedInSql) {
            return;
        }

        if ($context->user === null) {
            $context->query->matchNothing();

            return;
        }

        $context->query->where('tenant_id', '=', $this->tenant->id());
    }
}
```

- It runs once per engine query, after `ScoutEngine::perimeter()` and before the filters, sorts and instructions of the client: searches with a text, engine lists sent with `"source": "engine"`, `Rest::search()` and actions on their results, also behind `withoutPerimeter()`.
- Its conditions are AND-ed with the perimeter, the client and the other extensions, in any extension order. On a single-value attribute, `whereIn()` lists intersect and `whereNotIn()` lists add up, so a client filter cannot bring back a value it excludes.
- `matchNothing()` gives an empty page without calling the engine, and an action then runs on no row. Every method of `$context->query` returns it, `matchNothing()` included.
- `$context->user` is the user that the policies run for, also under `Rest::search()->as($user)`, `null` for a guest.
- `perimeter()` or `withoutPerimeter()` is still required: an extension does not stand for it.

### The context

<table>
<thead>
  <tr>
    <th>
      Property
    </th>
    
    <th>
      Value
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        $context->query
      </code>
    </td>
    
    <td>
      The restriction: <code>
        where()
      </code>
      
      , <code>
        whereIn()
      </code>
      
      , <code>
        whereNotIn()
      </code>
      
      , <code>
        whereAny()
      </code>
      
       and <code>
        matchNothing()
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        $context->resource
      </code>
      
      , <code>
        $context->user
      </code>
    </td>
    
    <td>
      The searched resource and the user
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        $context->rowsRestrictedInSql
      </code>
    </td>
    
    <td>
      <code>
        true
      </code>
      
       when the <code>
        RestrictsQueries
      </code>
      
       restriction of the same extension already applies in the SQL of the engine query: the Scout <code>
        database
      </code>
      
      , <code>
        collection
      </code>
      
       and <code>
        null
      </code>
      
       drivers, and any profile that implements <code>
        AppliesRowRestrictions
      </code>
      
      . <code>
        false
      </code>
      
       for an extension that does not implement <code>
        RestrictsQueries
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        $context->comparesColumns
      </code>
    </td>
    
    <td>
      <code>
        true
      </code>
      
       when Scout compares each condition with a column of the model table: name only columns there
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        $context->matchesStringLists
      </code>
    </td>
    
    <td>
      Whether the engine matches lists of strings
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        $context->filtersWithGroups
      </code>
    </td>
    
    <td>
      Whether the engine accepts <code>
        whereAny()
      </code>
      
       groups
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        $context->engine
      </code>
    </td>
    
    <td>
      The Scout engine instance that runs the query: test it with <code>
        instanceof
      </code>
      
       to write conditions for one engine
    </td>
  </tr>
</tbody>
</table>

### Groups

`whereAny()` OR-s a group of conditions:

```php
use Lomkit\Rest\Text\Engine\EngineConditionGroup;

$context->query->whereAny(fn (EngineConditionGroup $group) => $group
    ->where('owner_id', '=', $id)
    ->whereAll(fn (EngineConditionGroup $all) => $all->where('public', '=', true)->whereIn('team_id', $teams)));
```

- `EngineConditionGroup` offers `where()`, `whereIn()`, `whereNotIn()`, `whereAll()` for an AND group and `whereAny()` for an OR group.
- The whole group is one condition of the extension, AND-ed with the perimeter, the other restrictions, the instructions, the lists of strings and the client filters: it widens only within itself.
- A group without any condition matches nothing. `whereIn()` with an empty list is false and `whereNotIn()` with an empty list is true, so a true member of a `whereAny()` group removes the whole condition of the extension: refuse or replace an empty list read from the database before you group it.
- A condition added to a group after its closure returned throws `InvalidEngineQueryException`.
- Check `$context->filtersWithGroups` first. The built-in Typesense, Meilisearch, Turbopuffer, `database`, `collection` and `null` profiles accept groups. On Algolia, and on a profile that an extension provides, `whereAny()` throws `InvalidEngineExtensionException`, which names the extension, its class, the resource and the engine.

### What it refuses

- Anything but `where()`, `whereIn()`, `whereNotIn()`, `whereAny()` and `matchNothing()`: no client-style `or`, no sort, no engine option.
- An attribute that is not in the `filterable()`, `indexAttributes()` or `perimeter()` list of the resource, or declared by an extension, and any condition that the engine query refuses. `InvalidEngineExtensionException` quotes the refusal and keeps it as its previous exception.

### Pages, actions and the index

- The rows of an engine page load by key without the `RestrictsQueries` restrictions and without `perimeterQuery()`: the engine is the proof, so a row that a stale index still matches is shown until the index is updated. Make both restrictions say the same, and [keep the index in sync](/search/search-engines#keeping-the-index-in-sync).
- A condition that `restrictEngine()` reads live, such as a `whereIn()` on keys read from the database, applies at once. A change carried by an indexed attribute applies once the row is indexed again.
- Actions on search results, queued or not, read their keys through the `RestrictsQueries` restrictions: a row that the index still matches but the restriction now excludes is not handed to the action.
- A `RestrictsQueries` extension without this kind makes the text searches of the resource throw `InvalidTextSearchException` on an engine that does not apply SQL restrictions. The message offers to leave the resource out with `appliesTo()`, or to add the class to `rest.disabled_extensions`, and `rest:check` reports it under `rows`.
- The Scout `null` driver returns no row, and a `RestrictsQueries` extension is never refused on it. Like `collection` and `database`, it does not compare lists of strings: a `whereIn()` or `whereNotIn()` on a list throws there.

## Declaring index attributes

`DeclaresIndexAttributes::indexAttributes(IndexAttributesContext $context)` declares index attributes that the engine restrictions, `ScoutEngine::perimeter()` and the engine instructions compare. Here, a list of the keys that may read each row:

```php [Vendor/Sharing/ReadersExtension.php]
use Lomkit\Rest\Extensions\Contexts\EngineRestrictionContext;
use Lomkit\Rest\Extensions\Contexts\IndexAttributesContext;
use Lomkit\Rest\Extensions\Extension;
use Lomkit\Rest\Extensions\Kinds\DeclaresIndexAttributes;
use Lomkit\Rest\Extensions\Kinds\RestrictsEngineQueries;
use Lomkit\Rest\Text\Engine\StringListAttribute;

final class ReadersExtension extends Extension implements DeclaresIndexAttributes, RestrictsEngineQueries
{
    #[\Override]
    public static function name(): string
    {
        return 'readers';
    }

    #[\Override]
    public function indexAttributes(IndexAttributesContext $context): array
    {
        return $context->matchesStringLists ? [StringListAttribute::make('readers')] : [];
    }

    #[\Override]
    public function restrictEngine(EngineRestrictionContext $context): void
    {
        $reader = $context->user?->getAuthIdentifier();

        if (! $context->matchesStringLists || $reader === null) {
            $context->query->matchNothing();

            return;
        }

        $context->query->whereIn('readers', [(string) $reader]);
    }
}
```

The model indexes the list: `toSearchableArray()` returns `'readers' => ['12', 'group-3']`. On an engine without lists, this extension shows no row.

- It receives `$context->resource`, a class name, and `$context->matchesStringLists`, no user. It runs once per request and resource.
- Return fields (`ID`, `Integer`, `Number`, `Boolean`, `Text` or `Enum`) or lists of strings with `StringListAttribute::make('readers')`. The attributes keep their real index name, never prefixed, and follow the rules of `filterable()`.
- Clients never use them: they stay out of the client grammar (a filter on one is the `422` of an unknown name), `/details` and the OpenAPI document.
- The package cannot check that `toSearchableArray()` and the index settings hold these attributes: index each one yourself, or check it with [`ChecksResources`](/extensions/responses-and-checks#checking-resources). A resource that alone needs an attribute declares it with [`ScoutEngine::indexAttributes()`](/search/search-engines#index-attributes).

### Owners

- A name that `filterable()` declares throws `InvalidEngineExtensionException`, and so does a name that `ScoutEngine::indexAttributes()`, `perimeter()` or another extension declares with another field type or list marker. The message names the list of the resource as `the indexAttributes() list` or `the perimeter() list`.
- The same declaration is accepted: the one of the resource is used, else the one of the first extension by name.
- `rest:check` reports such a conflict as a `rows` problem.

### Lists of strings

- `whereIn('readers', [...])` keeps the rows that share at least one value with the list, and `whereNotIn()` the rows that share none. `where()` is refused on a list.
- Two or more `whereIn()` lists on one list attribute keep the rows that share at least one value with each list, whichever source sets them: `perimeter()`, the restrictions or the instructions. No list widens another.
- When one list holds the other, only the narrower one is sent: the same rows, with a shorter filter. Otherwise each further list joins the native filter as its own condition, AND-ed at the top level, on the built-in Typesense, Meilisearch and Turbopuffer profiles.
- On Algolia, and on a profile that an extension provides, two lists that neither holds throw `Lomkit\Rest\Exceptions\ConflictingListsException`, which names the sources, the attribute and the engine. Set by an instruction, such a list answers `422` on `search.instructions.N`: `The search.instructions.0 field restricts a list with values that neither hold nor are held by the list that the search already keeps, and the search engine cannot require both lists.` On such an engine, an instruction must not take client values for a list that the perimeter or a restriction also restricts.
- A list can be declared on every engine. Comparing it on an engine whose profile does not implement `MatchesStringLists`, such as the Scout `database`, `collection` and `null` drivers, throws `InvalidEngineExtensionException`: check `$context->matchesStringLists` first.

The index holds each list as:

<table>
<thead>
  <tr>
    <th>
      Engine
    </th>
    
    <th>
      Index schema
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      Algolia
    </td>
    
    <td>
      <code>
        filterOnly(readers)
      </code>
      
       in <code>
        attributesForFaceting
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      Meilisearch
    </td>
    
    <td>
      <code>
        readers
      </code>
      
       in <code>
        filterableAttributes
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      Typesense
    </td>
    
    <td>
      a collection field <code>
        ['name' => 'readers', 'type' => 'string[]']
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      Turbopuffer
    </td>
    
    <td>
      a <code>
        []string
      </code>
      
       attribute, matched with <code>
        ContainsAny
      </code>
      
       and excluded with <code>
        NotContainsAny
      </code>
    </td>
  </tr>
</tbody>
</table>

## Providing engine profiles

A profile tells the package what a Scout engine runs: its operators, sorts, page sizes, owned options and how it pages. `ProvidesEngineProfiles::engineProfile(EngineProfileContext $context)` returns the `EngineProfile` of an engine, or `null`:

```php [Vendor/OpenSearch/OpenSearchExtension.php]
use Lomkit\Rest\Extensions\Contexts\EngineProfileContext;
use Lomkit\Rest\Extensions\Extension;
use Lomkit\Rest\Extensions\Kinds\ProvidesEngineProfiles;
use Lomkit\Rest\Text\Engine\Profiles\EngineProfile;

final class OpenSearchExtension extends Extension implements ProvidesEngineProfiles
{
    #[\Override]
    public static function name(): string
    {
        return 'opensearch';
    }

    #[\Override]
    public function engineProfile(EngineProfileContext $context): ?EngineProfile
    {
        return $context->engine instanceof OpenSearchEngine ? new OpenSearchProfile($context->engine) : null;
    }
}
```

- It receives `$context->resource` and `$context->engine` only, no user. Answer without side effects, and `null` for an engine that the extension does not know.
- The package asks the extensions before its own profiles, so an extension can also replace the profile of a built-in engine. A provided profile goes through the same checks as a built-in one.
- Two extensions that provide a profile for one engine throw `InvalidEngineExtensionException`, which names both and the engine class: return `null` from one of them, or disable one.

### Writing a profile

A profile implements `Lomkit\Rest\Text\Engine\Profiles\EngineProfile`:

<table>
<thead>
  <tr>
    <th>
      Method
    </th>
    
    <th>
      Returns
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        name()
      </code>
    </td>
    
    <td>
      The name of the engine in messages
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        operators(Field $attribute)
      </code>
    </td>
    
    <td>
      The <code>
        FilterOperator
      </code>
      
       cases that the engine compares on the attribute. <code>
        EngineOperators::for($attribute)
      </code>
      
       gives the usual set
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        sortKeys()
      </code>
    </td>
    
    <td>
      The most sorts of a search, <code>
        0
      </code>
      
       to refuse <code>
        sortable()
      </code>
      
      , <code>
        PHP_INT_MAX
      </code>
      
       for no bound
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        ownedOptions()
      </code>
    </td>
    
    <td>
      The Scout options that the profile sets itself, which <code>
        options()
      </code>
      
       cannot set
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        exactTotal()
      </code>
    </td>
    
    <td>
      Whether the total of a page is exact
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        maxPageSize()
      </code>
      
      , <code>
        window()
      </code>
    </td>
    
    <td>
      The largest page and the deepest position the engine pages to, <code>
        null
      </code>
      
       when unbounded
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        ensureUsableBy(TextTarget $target)
      </code>
    </td>
    
    <td>
      Nothing: throws when the model cannot be searched there
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        page(Builder $builder, ScoutModel $model, int $perPage, int $page)
      </code>
    </td>
    
    <td>
      An <code>
        EnginePage
      </code>
      
      : the keys of the page in engine order and the total, <code>
        EnginePage::empty()
      </code>
      
       for none
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        reachable(ScoutModel $model)
      </code>
    </td>
    
    <td>
      The number of hits that an action may read, <code>
        null
      </code>
      
       when unknown
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        literal()
      </code>
    </td>
    
    <td>
      The <code>
        EngineLiteral
      </code>
      
       that writes a value into an engine filter
    </td>
  </tr>
</tbody>
</table>

- Every key that `page()` returns must match the filter that the builder carries: pages trust the engine. Turn off the features that add hits outside the filter, such as pinned, promoted or curated hits, in `page()`, and list their options in `ownedOptions()`.
- A profile written for 3.5 still works.

Optional interfaces of `Lomkit\Rest\Text\Engine\Profiles` add capabilities, checked with `instanceof`:

<table>
<thead>
  <tr>
    <th>
      Interface
    </th>
    
    <th>
      Capability
    </th>
  </tr>
</thead>

<tbody>
  <tr>
    <td>
      <code>
        MatchesStringLists
      </code>
    </td>
    
    <td>
      The engine matches lists of strings. A provided profile gets its lists through Scout's <code>
        whereIn()
      </code>
      
       and <code>
        whereNotIn()
      </code>
      
      : a profile that replaces the built-in Turbopuffer one sends them as <code>
        In
      </code>
      
       and <code>
        NotIn
      </code>
      
      , no longer as <code>
        ContainsAny
      </code>
      
       and <code>
        NotContainsAny
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        ComparesColumns
      </code>
    </td>
    
    <td>
      Scout compares each condition with a column of the model table
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        AppliesRowRestrictions
      </code>
    </td>
    
    <td>
      The profile runs SQL: <code>
        restrictedPage(Builder $builder, ScoutModel $model, int $perPage, int $page, RowRestriction $rows)
      </code>
      
       applies the <code>
        RestrictsQueries
      </code>
      
       restrictions, so they work without <code>
        RestrictsEngineQueries
      </code>
    </td>
  </tr>
  
  <tr>
    <td>
      <code>
        ListsWithoutText
      </code>
    </td>
    
    <td>
      The engine serves <a href="/search/search-engines#engine-lists">
        engine lists
      </a>
      
      : <code>
        matchAll()
      </code>
      
       returns the query that matches every document, <code>
        listSortKeys()
      </code>
      
       the most sorts of a search without text
    </td>
  </tr>
</tbody>
</table>

- `restrictedPage()` is called only when a `RestrictsQueries` restriction applies, and `page()` serves every other search. It must call `$rows->apply()` on the query it runs: one that returns rows without calling it makes the search throw `InvalidEngineQueryException`. A page takes the keys that it returns as they are.
- `servesLists()` on an engine whose profile does not implement `ListsWithoutText` throws `InvalidTextSearchException` at the first search, and `rest:check` reports it.
- The interface that writes `or` and `nested` filters is internal to the built-in profiles: a provided profile answers `422` for them, as Algolia does, and refuses `whereAny()`.
- A provided profile computes no [totals](/search/totals): `search.totals` answers `The search.totals field is refused: the search engine of this resource computes no totals.`
