Real Full-Text Search for Mastodon (Step-By-Step)

Put some flame-proof pants on, because today we're going to mod Mastodon to enable real full-text searching. This feature is already present in Lemmy, Pleroma, Akkoma, Friendica, etc... and this isn't a crawler, it just allows you to search through all the posts your server already knows about.

This patch is adapted from VyrCossont's code, under the AGPLv3 license, and is designed to work with vanilla Mastodon v4.0.2.

Oh, and of course, you need to have already enabled normal full-text search.

SSH into your server.
Login into the Mastodon account: su - mastodon
Change your directory to whatever your Mastodon installation is. By default: cd /home/mastodon/live
Type git apply. You have git, right?
Copy-paste this:

From fc31e8420b6795c31e8dedf81c3c2e8193cc552c Mon Sep 17 00:00:00 2001
From: Anonymous <>
Date: Fri, 10 Jan 2023 16:52:54 +0000
Subject: [PATCH] Add real full-text search

Does not add a config option; if you're applying this mod, I assume you
already want full-text search.

To use this, run:
systemctl restart mastodon-sidekiq.service mastodon-web.service
RAILS_ENV=production PROGRESS=true bin/rake 'chewy:reset[statuses]'

Issues with this:
- Does not order the posts properly
- May search unlisted posts, unknown
---
 app/chewy/statuses_index.rb    | 1 +
 app/services/search_service.rb | 4 +++-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/app/chewy/statuses_index.rb b/app/chewy/statuses_index.rb
index 6dd4fb18b..298493169 100644
--- a/app/chewy/statuses_index.rb
+++ b/app/chewy/statuses_index.rb
@@ -71,5 +71,6 @@ class StatusesIndex < Chewy::Index
     end
 
     field :searchable_by, type: 'long', value: ->(status, crutches) { status.searchable_by(crutches) }
+    field :searchable_by_anyone, type: 'boolean', value: ->(status) { status.public_visibility? }
   end
 end
diff --git a/app/services/search_service.rb b/app/services/search_service.rb
index 1a76cbb38..0a2e66b53 100644
--- a/app/services/search_service.rb
+++ b/app/services/search_service.rb
@@ -35,7 +35,9 @@ class SearchService < BaseService
   end
 
   def perform_statuses_search!
-    definition = parsed_query.apply(StatusesIndex.filter(term: { searchable_by: @account.id }))
+    statuses_index = StatusesIndex.filter(term: { searchable_by: @account.id })
+    statuses_index = statuses_index.filter.or(term: { searchable_by_anyone: true })
+    definition = parsed_query.apply(statuses_index)
 
     if @options[:account_id].present?
       definition = definition.filter(term: { account_id: @options[:account_id] })
-- 
2.30.2

Press Ctrl-D.
Hopefully it works! This next step will take a while. Run:

sudo systemctl restart mastodon-sidekiq.service mastodon-web.service
RAILS_ENV=production PROGRESS=true bin/rake 'chewy:reset[statuses]'

Once this is done, try searching for any text on your server. It should give you posts that you haven't interacted with before.

That's about it. A cool feature that this enables is that you can even search pictures, since this also searches for alt-text!

=====

CW: Kinda inflammatory comments below. Vent, rant, etc.

. . . . . . . . . . . . . . . . . . . . . . . . Seriously, don't look. . . . . . . . . . . . . . . . . . . . . . . . .

=====

I dislike how “traditional” Mastodon-ians like to speak for the entire Fediverse, as if their choices are agreed-upon by all and have a consensus, while silencing or bullying those that disagree. The whole point of the Fediverse (and free software in general) is that it is decentralised and everyone can use it however they like with different needs and norms, but there's a really exclusionary culture (“go back to Twitter!”) on there currently.

I also dislike how there's a culture of...

Solving technical problems by social solutions: De-federating and shaming crawlers and “ActivityPub trolls” instead of patching up security issues, adding encryption, adding better privacy features or adding better reply controls (which will arguably solve most of the “replyfolk” issue that people usually bring up when talking about searching).
Solving social problems by technical solutions: Intentionally leaving out features such as quote toots, algorithms, or full-text searching. It's a very “We know what's best for you” vibe and arguably Defective By Design.

Speaking of which, if you're using this patch, it might be best not to announce it so you don't end up on fediblock.

Thankfully, this is still free software, and so we can add these features ourselves. This is also an open protocol, so we can use other softwares too that add the “forbidden features”.

=====

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

You went too far!