Engine extensions
Three kinds act on the searches that run on a search engine through ScoutEngine: RestrictsEngineQueries, DeclaresIndexAttributes and ProvidesEngineProfiles. They throw Lomkit\Rest\Exceptions\InvalidEngineExtensionException, which names the extension by its name and its class. See Search engines for the engine searches themselves.
Restricting engine searches
RestrictsEngineQueries::restrictEngine(EngineRestrictionContext $context) restricts each engine query of a resource. Here, the tenant extension of Query extensions also restricts the engine, on an attribute that it declares:
use Lomkit\Rest\Extensions\Contexts\EngineRestrictionContext;
use Lomkit\Rest\Extensions\Contexts\IndexAttributesContext;
use Lomkit\Rest\Extensions\Contexts\RestrictionContext;
use Lomkit\Rest\Extensions\Extension;
use Lomkit\Rest\Extensions\Kinds\DeclaresIndexAttributes;
use Lomkit\Rest\Extensions\Kinds\RestrictsEngineQueries;
use Lomkit\Rest\Extensions\Kinds\RestrictsQueries;
use Lomkit\Rest\Fields\Integer;
final class TenantExtension extends Extension implements DeclaresIndexAttributes, RestrictsEngineQueries, RestrictsQueries
{
public function __construct(private readonly CurrentTenant $tenant) {}
#[\Override]
public static function name(): string
{
return 'tenant';
}
#[\Override]
public function restrict(RestrictionContext $context): void
{
// the SQL restriction, see Query extensions
}
#[\Override]
public function indexAttributes(IndexAttributesContext $context): array
{
return [Integer::make('tenant_id')];
}
#[\Override]
public function restrictEngine(EngineRestrictionContext $context): void
{
if ($context->rowsRestrictedInSql) {
return;
}
if ($context->user === null) {
$context->query->matchNothing();
return;
}
$context->query->where('tenant_id', '=', $this->tenant->id());
}
}
- It runs once per engine query, after
ScoutEngine::perimeter()and before the filters, sorts and instructions of the client: searches with a text, engine lists sent with"source": "engine",Rest::search()and actions on their results, also behindwithoutPerimeter(). - Its conditions are AND-ed with the perimeter, the client and the other extensions, in any extension order. On a single-value attribute,
whereIn()lists intersect andwhereNotIn()lists add up, so a client filter cannot bring back a value it excludes. matchNothing()gives an empty page without calling the engine, and an action then runs on no row. Every method of$context->queryreturns it,matchNothing()included.$context->useris the user that the policies run for, also underRest::search()->as($user),nullfor a guest.perimeter()orwithoutPerimeter()is still required: an extension does not stand for it.
The context
| Property | Value |
|---|---|
$context->query | The restriction: where(), whereIn(), whereNotIn(), whereAny() and matchNothing() |
$context->resource, $context->user | The searched resource and the user |
$context->rowsRestrictedInSql | true when the RestrictsQueries restriction of the same extension already applies in the SQL of the engine query: the Scout database, collection and null drivers, and any profile that implements AppliesRowRestrictions. false for an extension that does not implement RestrictsQueries |
$context->comparesColumns | true when Scout compares each condition with a column of the model table: name only columns there |
$context->matchesStringLists | Whether the engine matches lists of strings |
$context->filtersWithGroups | Whether the engine accepts whereAny() groups |
$context->engine | The Scout engine instance that runs the query: test it with instanceof to write conditions for one engine |
Groups
whereAny() OR-s a group of conditions:
use Lomkit\Rest\Text\Engine\EngineConditionGroup;
$context->query->whereAny(fn (EngineConditionGroup $group) => $group
->where('owner_id', '=', $id)
->whereAll(fn (EngineConditionGroup $all) => $all->where('public', '=', true)->whereIn('team_id', $teams)));
EngineConditionGroupofferswhere(),whereIn(),whereNotIn(),whereAll()for an AND group andwhereAny()for an OR group.- The whole group is one condition of the extension, AND-ed with the perimeter, the other restrictions, the instructions, the lists of strings and the client filters: it widens only within itself.
- A group without any condition matches nothing.
whereIn()with an empty list is false andwhereNotIn()with an empty list is true, so a true member of awhereAny()group removes the whole condition of the extension: refuse or replace an empty list read from the database before you group it. - A condition added to a group after its closure returned throws
InvalidEngineQueryException. - Check
$context->filtersWithGroupsfirst. The built-in Typesense, Meilisearch, Turbopuffer,database,collectionandnullprofiles accept groups. On Algolia, and on a profile that an extension provides,whereAny()throwsInvalidEngineExtensionException, which names the extension, its class, the resource and the engine.
What it refuses
- Anything but
where(),whereIn(),whereNotIn(),whereAny()andmatchNothing(): no client-styleor, no sort, no engine option. - An attribute that is not in the
filterable(),indexAttributes()orperimeter()list of the resource, or declared by an extension, and any condition that the engine query refuses.InvalidEngineExtensionExceptionquotes the refusal and keeps it as its previous exception.
Pages, actions and the index
- The rows of an engine page load by key without the
RestrictsQueriesrestrictions and withoutperimeterQuery(): the engine is the proof, so a row that a stale index still matches is shown until the index is updated. Make both restrictions say the same, and keep the index in sync. - A condition that
restrictEngine()reads live, such as awhereIn()on keys read from the database, applies at once. A change carried by an indexed attribute applies once the row is indexed again. - Actions on search results, queued or not, read their keys through the
RestrictsQueriesrestrictions: a row that the index still matches but the restriction now excludes is not handed to the action. - A
RestrictsQueriesextension without this kind makes the text searches of the resource throwInvalidTextSearchExceptionon an engine that does not apply SQL restrictions. The message offers to leave the resource out withappliesTo(), or to add the class torest.disabled_extensions, andrest:checkreports it underrows. - The Scout
nulldriver returns no row, and aRestrictsQueriesextension is never refused on it. Likecollectionanddatabase, it does not compare lists of strings: awhereIn()orwhereNotIn()on a list throws there.
Declaring index attributes
DeclaresIndexAttributes::indexAttributes(IndexAttributesContext $context) declares index attributes that the engine restrictions, ScoutEngine::perimeter() and the engine instructions compare. Here, a list of the keys that may read each row:
use Lomkit\Rest\Extensions\Contexts\EngineRestrictionContext;
use Lomkit\Rest\Extensions\Contexts\IndexAttributesContext;
use Lomkit\Rest\Extensions\Extension;
use Lomkit\Rest\Extensions\Kinds\DeclaresIndexAttributes;
use Lomkit\Rest\Extensions\Kinds\RestrictsEngineQueries;
use Lomkit\Rest\Text\Engine\StringListAttribute;
final class ReadersExtension extends Extension implements DeclaresIndexAttributes, RestrictsEngineQueries
{
#[\Override]
public static function name(): string
{
return 'readers';
}
#[\Override]
public function indexAttributes(IndexAttributesContext $context): array
{
return $context->matchesStringLists ? [StringListAttribute::make('readers')] : [];
}
#[\Override]
public function restrictEngine(EngineRestrictionContext $context): void
{
$reader = $context->user?->getAuthIdentifier();
if (! $context->matchesStringLists || $reader === null) {
$context->query->matchNothing();
return;
}
$context->query->whereIn('readers', [(string) $reader]);
}
}
The model indexes the list: toSearchableArray() returns 'readers' => ['12', 'group-3']. On an engine without lists, this extension shows no row.
- It receives
$context->resource, a class name, and$context->matchesStringLists, no user. It runs once per request and resource. - Return fields (
ID,Integer,Number,Boolean,TextorEnum) or lists of strings withStringListAttribute::make('readers'). The attributes keep their real index name, never prefixed, and follow the rules offilterable(). - Clients never use them: they stay out of the client grammar (a filter on one is the
422of an unknown name),/detailsand the OpenAPI document. - The package cannot check that
toSearchableArray()and the index settings hold these attributes: index each one yourself, or check it withChecksResources. A resource that alone needs an attribute declares it withScoutEngine::indexAttributes().
Owners
- A name that
filterable()declares throwsInvalidEngineExtensionException, and so does a name thatScoutEngine::indexAttributes(),perimeter()or another extension declares with another field type or list marker. The message names the list of the resource asthe indexAttributes() listorthe perimeter() list. - The same declaration is accepted: the one of the resource is used, else the one of the first extension by name.
rest:checkreports such a conflict as arowsproblem.
Lists of strings
whereIn('readers', [...])keeps the rows that share at least one value with the list, andwhereNotIn()the rows that share none.where()is refused on a list.- Two or more
whereIn()lists on one list attribute keep the rows that share at least one value with each list, whichever source sets them:perimeter(), the restrictions or the instructions. No list widens another. - When one list holds the other, only the narrower one is sent: the same rows, with a shorter filter. Otherwise each further list joins the native filter as its own condition, AND-ed at the top level, on the built-in Typesense, Meilisearch and Turbopuffer profiles.
- On Algolia, and on a profile that an extension provides, two lists that neither holds throw
Lomkit\Rest\Exceptions\ConflictingListsException, which names the sources, the attribute and the engine. Set by an instruction, such a list answers422onsearch.instructions.N:The search.instructions.0 field restricts a list with values that neither hold nor are held by the list that the search already keeps, and the search engine cannot require both lists.On such an engine, an instruction must not take client values for a list that the perimeter or a restriction also restricts. - A list can be declared on every engine. Comparing it on an engine whose profile does not implement
MatchesStringLists, such as the Scoutdatabase,collectionandnulldrivers, throwsInvalidEngineExtensionException: check$context->matchesStringListsfirst.
The index holds each list as:
| Engine | Index schema |
|---|---|
| Algolia | filterOnly(readers) in attributesForFaceting |
| Meilisearch | readers in filterableAttributes |
| Typesense | a collection field ['name' => 'readers', 'type' => 'string[]'] |
| Turbopuffer | a []string attribute, matched with ContainsAny and excluded with NotContainsAny |
Providing engine profiles
A profile tells the package what a Scout engine runs: its operators, sorts, page sizes, owned options and how it pages. ProvidesEngineProfiles::engineProfile(EngineProfileContext $context) returns the EngineProfile of an engine, or null:
use Lomkit\Rest\Extensions\Contexts\EngineProfileContext;
use Lomkit\Rest\Extensions\Extension;
use Lomkit\Rest\Extensions\Kinds\ProvidesEngineProfiles;
use Lomkit\Rest\Text\Engine\Profiles\EngineProfile;
final class OpenSearchExtension extends Extension implements ProvidesEngineProfiles
{
#[\Override]
public static function name(): string
{
return 'opensearch';
}
#[\Override]
public function engineProfile(EngineProfileContext $context): ?EngineProfile
{
return $context->engine instanceof OpenSearchEngine ? new OpenSearchProfile($context->engine) : null;
}
}
- It receives
$context->resourceand$context->engineonly, no user. Answer without side effects, andnullfor an engine that the extension does not know. - The package asks the extensions before its own profiles, so an extension can also replace the profile of a built-in engine. A provided profile goes through the same checks as a built-in one.
- Two extensions that provide a profile for one engine throw
InvalidEngineExtensionException, which names both and the engine class: returnnullfrom one of them, or disable one.
Writing a profile
A profile implements Lomkit\Rest\Text\Engine\Profiles\EngineProfile:
| Method | Returns |
|---|---|
name() | The name of the engine in messages |
operators(Field $attribute) | The FilterOperator cases that the engine compares on the attribute. EngineOperators::for($attribute) gives the usual set |
sortKeys() | The most sorts of a search, 0 to refuse sortable(), PHP_INT_MAX for no bound |
ownedOptions() | The Scout options that the profile sets itself, which options() cannot set |
exactTotal() | Whether the total of a page is exact |
maxPageSize(), window() | The largest page and the deepest position the engine pages to, null when unbounded |
ensureUsableBy(TextTarget $target) | Nothing: throws when the model cannot be searched there |
page(Builder $builder, ScoutModel $model, int $perPage, int $page) | An EnginePage: the keys of the page in engine order and the total, EnginePage::empty() for none |
reachable(ScoutModel $model) | The number of hits that an action may read, null when unknown |
literal() | The EngineLiteral that writes a value into an engine filter |
- Every key that
page()returns must match the filter that the builder carries: pages trust the engine. Turn off the features that add hits outside the filter, such as pinned, promoted or curated hits, inpage(), and list their options inownedOptions(). - A profile written for 3.5 still works.
Optional interfaces of Lomkit\Rest\Text\Engine\Profiles add capabilities, checked with instanceof:
| Interface | Capability |
|---|---|
MatchesStringLists | The engine matches lists of strings. A provided profile gets its lists through Scout's whereIn() and whereNotIn(): a profile that replaces the built-in Turbopuffer one sends them as In and NotIn, no longer as ContainsAny and NotContainsAny |
ComparesColumns | Scout compares each condition with a column of the model table |
AppliesRowRestrictions | The profile runs SQL: restrictedPage(Builder $builder, ScoutModel $model, int $perPage, int $page, RowRestriction $rows) applies the RestrictsQueries restrictions, so they work without RestrictsEngineQueries |
ListsWithoutText | The engine serves engine lists: matchAll() returns the query that matches every document, listSortKeys() the most sorts of a search without text |
restrictedPage()is called only when aRestrictsQueriesrestriction applies, andpage()serves every other search. It must call$rows->apply()on the query it runs: one that returns rows without calling it makes the search throwInvalidEngineQueryException. A page takes the keys that it returns as they are.servesLists()on an engine whose profile does not implementListsWithoutTextthrowsInvalidTextSearchExceptionat the first search, andrest:checkreports it.- The interface that writes
orandnestedfilters is internal to the built-in profiles: a provided profile answers422for them, as Algolia does, and refuseswhereAny(). - A provided profile computes no totals:
search.totalsanswersThe search.totals field is refused: the search engine of this resource computes no totals.
Query extensions
Restrict the perimeter of every resource, add columns for the policies, narrow page sizes per user and add search instructions from an extension.
Authorization extensions
Prepare and version row checks, add to the keys of kept decisions, declare gates and narrow the fields of a resource for each user from an extension.