Text search

A resource declares its text search with text(). With the Contains and FullText strategies, searchable() marks the fields to search, and the text becomes one more condition of the SQL search: filters, scopes, instructions, includes, aggregates, sorts, pagination, perimeterQuery() and actions work with it as they do without it.

use Lomkit\Rest\Fields\ID;
use Lomkit\Rest\Fields\Integer;
use Lomkit\Rest\Fields\Text;
use Lomkit\Rest\Text\Contains;
use Lomkit\Rest\Text\FullText;
use Lomkit\Rest\Text\TextSearch;

public function fields(): array
{
    return [
        ID::make('id'),
        Text::make('name')->searchable(),
        Text::make('description')->searchable(),
        Integer::make('price'),
    ];
}

public function text(): ?TextSearch
{
    return FullText::make()
        ->index('name', 'description')
        ->otherwise(Contains::make());
}

{
  "search": {
    "text": {"value": "desk lamp"},
    "filters": [{"field": "price", "operator": "<", "value": 100}]
  }
}

The response is a normal search page, with the matching rows only. total counts the matching rows.

Strategies

ScoutEngine has its own page: Search engines.

Contains

• Each word of the text must appear in one of the searchable fields, as a part of its value, in any order.
• The words after the tenth are ignored.
• The %, _ and \ characters of the text match as they are typed.
• Case: ILIKE ignores case on PostgreSQL. On MySQL and MariaDB, LIKE follows the collation of the column, and the default _ci collations ignore case. On SQLite, LIKE ignores the case of ASCII letters only.

FullText

• index('name', 'description') declares a group of columns. Call it once per FULLTEXT index: a row matches when one of the groups matches. Without index(), the searchable fields form one group.
• language('english') names the PostgreSQL text search configuration, english by default: letters, digits and _ only. MySQL and MariaDB ignore it.
• otherwise(Contains::make()) runs another strategy on the other databases, such as SQLite and SQL Server. Without it, a search of the resource there throws InvalidTextSearchException, which names the resource and the driver. otherwise() takes a strategy that adds a condition to the SQL query: a ScoutEngine there is a TypeError.
• On MySQL and MariaDB, each group needs a FULLTEXT index on exactly its columns. A group without one throws InvalidTextSearchException, which names the table and the groups.
• On PostgreSQL, the search runs without index, and the same migration, with ->language(...) when the configuration is not english, creates the index that the search uses.

Schema::table('products', function (Blueprint $table) {
    $table->fullText(['name', 'description']);
});

Declaration rules

• text() returns null by default: the resource has no text search, and search.text gives The search.text field is prohibited.
• Only Text fields, and app types that extend Text, can be searchable(). searchable() throws InvalidFieldException on another type, and on a virtual() or writeOnly() field.
• A resource whose text() returns Contains or FullText without any searchable field throws InvalidTextSearchException when it is searched, and so does an index() column that is not a searchable field.
• A model that uses Laravel\Scout\Searchable does not give its resource a text search: declare text().

The request

{"text": {"value": "desk lamp", "mode": "phrase"}}

• text is an object. value is a string of at most 255 characters: The search.text.value field must not be greater than 255 characters.
• A blank, null or missing value, or a value without a letter or a digit, runs the search without text and returns the normal list.
• mode is words by default, a case of Lomkit\Rest\Text\TextMode. A mode that the strategy does not support gives The selected search.text.mode is invalid.
• A key other than value and mode gives a 422 on that key, such as The search.text.fuzzy field is prohibited.
• text in an include gives The search.includes.0.text field is prohibited.

Modes

• FullText keeps the letters and digits of the text. The other characters, such as +, *, :, &, | and !, separate words and have no meaning, except the quotes, - and or of the websearch mode.
• On MySQL and MariaDB, an _ between two letters or digits stays in the word: snake_case is one word there, and two words on PostgreSQL.

What the text narrows

• The text narrows the searched resource only. It does not narrow includes, aggregates or relation filters.
• Includes and aggregates are validated against the fields and operators of their own resource, as without text.
• viewAny is checked before the search runs, and perimeterQuery() applies as in any search.

Relevance

With a strategy that orders by relevance (FullText on MySQL, MariaDB and PostgreSQL), a search with text and without sorts returns the most relevant rows first, then follows defaultOrderBy().

The sort field @relevance places the relevance among the other sorts, desc when direction is omitted:

{
  "search": {
    "text": {"value": "lamp"},
    "sorts": [
      {"field": "price"},
      {"field": "@relevance"}
    ]
  }
}

@relevance gives The selected search.sorts.0.field is invalid. without text, in an include, and with a strategy that does not order by relevance, such as Contains or the otherwise() strategy of FullText.

Trashed rows

search.trashed reads the soft-deleted rows of a model that uses SoftDeletes, with or without text:

{
  "search": {
    "trashed": "with"
  }
}

• with adds the soft-deleted rows, and only returns them alone. Another value gives The selected search.trashed is invalid.
• On a model without SoftDeletes, trashed gives The search.trashed field is prohibited. It gives a 422 in an include too.
• search.text.trashed gives The search.text.trashed field is prohibited.

Actions

An action of a resource that declares text() accepts search.text and acts on the matching rows only. A blank text acts on every row that the rest of the search resolves. search.trashed applies to actions too. The action of a resource without text() refuses search.text with The search.text field is prohibited. See Actions.

Database differences

FullText on MySQL and MariaDB:

• Words are not stemmed: lamps does not find lamp.
• Words shorter than innodb_ft_min_token_size (3 by default) and the InnoDB stopwords, such as the or with, are not indexed. The words mode ignores them and the phrase mode skips them, but a text made of such words only returns no row.
• In the prefix mode, each word of the text must start an indexed word of the row: l and la find lamp, but the lam misses The lamp, whose the is not indexed.
• In the websearch mode each word is required, so a stopword or a short word finds no row. Lower innodb_ft_min_token_size or change the stopword list, then rebuild the indexes.
• A websearch text made of excluded words only returns no row.
• InnoDB only finds committed rows: in tests that run each test in a transaction, such as tests that use RefreshDatabase, a FullText search finds no row that the test wrote.

FullText on PostgreSQL:

• language() sets how words are read. With english, words are stemmed (lamps finds lamp) and English stopwords are ignored: a text made of stopwords only returns no row. With simple, words are only lowercased. Give the index the same configuration, $table->fullText([...])->language('simple'), otherwise PostgreSQL does not use it.
• A row whose value is null in one column of a group does not match through that group. Declare the columns of a group NOT NULL, or give a nullable column its own index() group.
• A websearch text made of excluded words only returns the rows without these words.

Errors