Bridging Laravel Scout to Eloquent

Laravel Scout provides an interface to external search engines such as Algolia and Meilisearch. These are typically remote indexes of the same data that you’re storing locally, but with far faster and more powerful searching and ranking capabilities. However, they have some limitations on how you can express the search itself (allowing only a single string value), and what you can do with the search result (because it comes back as a Laravel Collection and not a Query Builder). This makes it difficult to do things like perform a complex search remotely, and then filter the result further via Eloquent operations, perhaps involving different parts of your database. A typical Scout search might be:

use App\Models\Order;
$orders = Order::search('Star Trek')->get();Code language: PHP (php)

This gives us a bunch of Order instances, but we had no opportunity to ask it to do things like load relations or filter the results while it was doing it.

Fortunately it’s not difficult to bridge these two worlds. The resulting Collection contains model instances, so we can extract their IDs and use them to construct an Eloquent search that selects the same records, but locally this time (Scout already did the heavy lifting of figuring out which ones we wanted):

$builder = Order::query()->whereIn('id', $orders->keys());Code language: PHP (php)

This isn’t ideal (because it will fetch those records a second time), but it will be reasonably efficient because the IDs it searches on are exact matches for primary keys in the database.

We now have a builder that will select the same records as the Scout search did, but we can continue adding to it before requesting the final results.

$result = $builder->where('orders.name', 'like', 'a%')
    ->with(['orderItems', 'customer'])
    ->get();Code language: PHP (php)

So that’s how we can get to use Eloquent features on top of a Scout search.


After finding out that this bridging wasn’t built-in, I submitted a PR to add it, but sadly it was rejected. With the PR code in place, the syntax would have looked like this:

use App\Models\Order;
$orders = Order::search('Star Trek')
    ->toEloquent()
    ->where('orders.name', 'like', 'a%')
    ->with(['orderItems', 'customer'])
    ->get();Code language: PHP (php)

To be fair, this isn’t much of a saving in the external syntax, but it is more efficient because it can get the record IDs directly from Scout without having to load the models from the database. I don’t think that efficiency gain can be obtained from outside Scout’s own code.

I hope that helps someone!

Postman pre-request script for Laravel registration

A common way to register in a Laravel API is to send a POST request to /users containing a username, password, any other info, and also an HMAC signature using a server-side secret. In this example, it’s validated on the server by this class:

    public const HASH_ALGORITHM = 'sha256';

    protected const REQUEST_KEYS = [
        'email',
        'name',
    ];

    private $secret;

    public function __construct(string $secret = null)
    {
        if (! $secret) {
            throw new \InvalidArgumentException('The registration secret must be provided');
        }

        $this->secret = $secret;
    }

    public function verify(Request $request): bool
    {
        return rescue(
            function () use ($request) {
                $hash = hash_hmac(
                    self::HASH_ALGORITHM,
                    json_encode($request->only(self::REQUEST_KEYS)),
                    $this->secret
                );

                return hash_equals(
                    base64_encode($hash),
                    $request->get('signature', '')
                );
            },
            false
        );
Code language: PHP (php)

So we can see that it’s expecting a Base64-encoded HMAC-SHA256 signature of a JSON array containing the email and name properties.

If you’re trying to make this request in Postman, you obviously need to calculate this same signature or it won’t work. Fortunately Postman has pre-request scripts that can inspect bits of your request and environment and generate new elements before your request is sent, and that’s what we need to use.

We don’t want the secret to be saved in our request collection, so we keep it in an environment, and pull it out dynamically when the request is made. Postman includes the Crypto-js package, which includes the necessary signature and encoding functions we need. Coming from PHP, the syntax for these operations feels very convoluted, but it goes like this:

const signature_string = '{"email":"' + request.data.email + '","name":"' + request.data.name + '"}';
const hmac = CryptoJS.HmacSHA256(signature_string, pm.environment.get('REGISTRATION_SECRET'));
const b64 = CryptoJS.enc.Base64.stringify(CryptoJS.enc.Utf8.parse(hmac));
pm.environment.set("REGISTRATION_SIGNATURE", b64);Code language: JavaScript (javascript)

This builds the string to sign from the request elements we need, calculates the HMAC of it using our secret, and then base64-encodes it before saving it in the Postman environment.

You can then add the signature into your request by adding the REGISTRATION_SIGNATURE variable to the body:

Hope that helps someone!