Laravel Scout provides an interface to external search engines such as Algolia and Meilisearch. These are typically remote indexes of the same data that you’re storing locally, but with far faster and more powerful searching and ranking capabilities. However, they have some limitations on how you can express the search itself (allowing only a single string value), and what you can do with the search result (because it comes back as a Laravel Collection and not a Query Builder). This makes it difficult to do things like perform a complex search remotely, and then filter the result further via Eloquent operations, perhaps involving different parts of your database. A typical Scout search might be:
This gives us a bunch of Order instances, but we had no opportunity to ask it to do things like load relations or filter the results while it was doing it.
Fortunately it’s not difficult to bridge these two worlds. The resulting Collection contains model instances, so we can extract their IDs and use them to construct an Eloquent search that selects the same records, but locally this time (Scout already did the heavy lifting of figuring out which ones we wanted):
This isn’t ideal (because it will fetch those records a second time), but it will be reasonably efficient because the IDs it searches on are exact matches for primary keys in the database.
We now have a builder that will select the same records as the Scout search did, but we can continue adding to it before requesting the final results.
So that’s how we can get to use Eloquent features on top of a Scout search.
After finding out that this bridging wasn’t built-in, I submitted a PR to add it, but sadly it was rejected. With the PR code in place, the syntax would have looked like this:
To be fair, this isn’t much of a saving in the external syntax, but it is more efficient because it can get the record IDs directly from Scout without having to load the models from the database. I don’t think that efficiency gain can be obtained from outside Scout’s own code.
A common way to register in a Laravel API is to send a POST request to /users containing a username, password, any other info, and also an HMAC signature using a server-side secret. In this example, it’s validated on the server by this class:
So we can see that it’s expecting a Base64-encoded HMAC-SHA256 signature of a JSON array containing the email and name properties.
If you’re trying to make this request in Postman, you obviously need to calculate this same signature or it won’t work. Fortunately Postman has pre-request scripts that can inspect bits of your request and environment and generate new elements before your request is sent, and that’s what we need to use.
We don’t want the secret to be saved in our request collection, so we keep it in an environment, and pull it out dynamically when the request is made. Postman includes the Crypto-js package, which includes the necessary signature and encoding functions we need. Coming from PHP, the syntax for these operations feels very convoluted, but it goes like this:
This builds the string to sign from the request elements we need, calculates the HMAC of it using our secret, and then base64-encodes it before saving it in the Postman environment.
You can then add the signature into your request by adding the REGISTRATION_SIGNATURE variable to the body:
HELO works very nicely if you’re sending via SMTP using PHPMailer, SwiftMailer, etc. – but lots of apps and scripts rely on PHP’s clunky old mail() function, which isn’t nearly as easy to deal with, and harder to point at HELO.
You can configure “proper” mail servers like postfix to work as a local relay, but it’s horribly complicated and confusing to set up. Fortunately there are simpler alternatives that are much easier. Searching for local relay tools (what this is) will usually point you at ssmtp, however, that doesn’t work on macOS. A better option for macOS is msmtp which is present in homebrew and works perfectly. I usually run HELO on localhost port 2500, and I configure msmtp with a config file (stored in ~/.msmtprc; you should also chmod 600 this file as it may contain secrets) like this, which enables the authentication that HELO requires, and I’ve also enabled logging so you can see what it’s up to:
defaults
host localhost
port 2500
tls off
undisclosed_recipients off
account default
auth plain
user test
password password
logfile ~/logs/msmtp.log
syslog offCode language:Bash(bash)
If you’re using homebrew’s PHP package on macOS, I recommend putting config changes like this in a separate .ini file so that it remains update-safe; I put mine in /usr/local/etc/php/8.0/conf.d/marcus.ini.
With those two things in place, PHP’s mail function will submit to the local msmtp binary, which will then relay the message to HELO over SMTP.
Sending via this this route using PHPMailer is very simple because mail() is the default mailer, so you don’t need to configure anything:
<?phpusePHPMailer\PHPMailer\PHPMailer;
require'vendor/autoload.php';
$mail = new PHPMailer();
$mail->setFrom('from@example.com', 'First Last');
$mail->addAddress('whoto@example.com', 'John Doe');
$mail->Subject = 'Say Hello to HELO';
$mail->Body = '<h1>Hi!</h1><p>This is my HTML body</p>'
$mail->AltBody = 'This is a plain-text message body';
if (!$mail->send()) {
echo'Mailer Error: ' . $mail->ErrorInfo;
} else {
echo'Message sent!';
}Code language:PHP(php)
I enjoyed Riggraz’s observation of “The Stack Overflow Antipattern”, and it made me think of another very similar pattern that I see a lot on Stack Overflow, but it occurs after the pattern that Riccardo describes, and I thought I’d outline that here.
I answer a lot of questions on Stack Overflow. I ask very few, but I’ve still fallen into this trap myself.
Once you’ve been through Riccardo’s antipattern (ignoring the other antipattern of those that don’t even make it to step 1), you are here:
You’ve searched and found some random results
You’ve read some SO questions that were in those results
You’ve still not found a solution
If you’ve got this far, the breadth of the question you want answering has probably been narrowed a little (which helps in its own right: searching is a mild form of rubber ducking), and probably contains the basis of a worthwhile Stack Overflow question.
So you focus on the problem, write it up, and (assuming this rubber-duck exercise didn’t lead you to a solution) post the question, and answers and comments appear reasonably quickly (hey, Stack Overflow rocks!). But often these responses are bogus, half-answers, or raise further questions. It’s at this point that we see the same “not taking the time to think” that Riccardo observed. You are so focused on the original question, you become incapable of solving a far simpler follow-on question.
Here’s a small example I often see:
“I’ve seen docs and code referring to autoload.php, but I can’t find that file”
Searching for autoload.php will find a zillion irrelevant results, because pretty much every PHP project in existence has one. So the question is posted on SO. There is a simple answer to this question, which is
“install composer, run `composer install`, and it will create the autoload.php file for you”
This inevitably leads to the follow-on question:
“how do I install composer?”.
This is a new sub-problem, but one that is instantly solvable by searching because it’s far less ambiguous. However, this is where the abdication of thinking kicks in, and rather than actually doing that search, you ask in the SO question comments, and sit around waiting for an answer from Someone Who Knows™ that posted an answer to the original question. This is frustrating for the answerer, who knows that the asker could find the answer to this question far faster by searching for themselves, but they choose not to because their thinking is turned off. There’s also an element of panic – the asker has obtained the attention of someone capable of understanding their problem, and doesn’t want them to escape before they have addressed the full recursive stack of sub-problems. This has led to the existence of passive-aggressive responses like LMGTFY, which are demeaning and condescending, but reflect the frustrations of those who answer questions.
What’s weirder is that I have observed myself doing this very thing, and I’ve sometimes had to stop myself posting trivial follow-up questions without thinking. Avoiding having to think is evidently a compelling driver.
I emailed Riccardo about his article with some of the thoughts that led me to write this, and he came back with another interesting observation: This loss of confidence that leads one to post trivial follow-up questions is very much like imposter syndrome. Having had to ask a question in the first place can provoke feelings of embarrassment or inadequacy, and anyone that responds in a positive way will appear to be in some way superior, which is fully expected, but at the same time provokes feelings of “we’re not worthy”, further reducing one’s confidence to be able to deal with even simple problems.
We’re not worthy
I know that Stack Overflow (and GitHub issues) can sometimes be harsh on new users, and old hands (like me) can forget what it’s like to be a beginner. It can be very frustrating to answer questions that have been answered many times before (“my PHP script just gives a white screen”), and I’ve occasionally found myself editing my initial reaction to avoid unkindness. In those situations I usually try to overcompensate by offering more general advice about how to avoid getting stuck in dead-end situations like that, rather than just answering the precise question asked.
In PHPMailer I have tried to head off support questions before they arise by adding links to documentation in error messages, but it doesn’t stop people posting questions like this:
I have this error:
> 2020-05-16 07:28:11 SMTP connect() failed.
> https://github.com/PHPMailer/PHPMailer/wiki/Troubleshooting
I have been stuck on this for 2 weeks, and I searched the entire internet three times
Where can I find out how to fix it? You must help me urgently!
I don’t really know what to do when faced with this. Posting a substantive answer is probably pointless – if they have not read what’s right in front of them despite their evident frustration, chances are they will not read any answer you post either, especially since it will only contain exactly what’s in the link provided anyway. Sometimes the best thing to do is vote to close the question, usually as an inevitable duplicate. I see very similar things happening in GitHub issue templates – askers delete the boilerplate text, removing something which would usually help them solve their exact problem in a few seconds, but they go out of their way to make the process take longer and involve others unnecessarily, because apparently not having to think is a more attractive proposition.
I’ve also considered pushing in the other direction, such as by adding “delete this line from the debug output before posting questions about it, or your question will be ignored” as a way of enforcing reading the error messages, but that’s unkind.
I’m not sure how to address the abdication of thinking issue though. Perhaps offer up search results derived from comments or answers, much in the same way that Stack Overflow does when you first post a question? There are probably extensive psychological studies on this pattern of behaviour, and it may well have a name, but that’s a question for another Stack Exchange site.
