A third solution would be to add a search feature to Org-mode, that could be used from Org-mode and exported in HTML.
I think it could be done in 3 steps.
# Step 1: Search feature in Org-mode only
This step would add the feature of searching files and items in Org-mode
files.
This search could be done by a user in 2 ways: With an Emacs interactive
function or with a dynamic bloc.
The dynamic block could look like this :
#+BEGIN: search :query "my query" :in-directory "~/Org" :targets files+items
#+END:
Of course other parameters would be available.
If "files" is set as targets, each file in the searched directory would
be treated as an item. And their buffer settings (like #+date or
#+author) as item properties. The buffer setting "#+title" would be
treated as an item title. So, searching for a file would be like
searching for an item. It would be important for the step 2.
At this step, any export of the bloc to HTML would only export its
content.
# Step 2: Add HTML export of the search feature
At this step, a new parameter is added to the search dynamic bloc:
:export
It could have 2 values:
- result
- search
"result" is the default value and when the Org-mode file is exported,
the content of the dynamic bloc is exported. As in step 1.
But with the "search" value, the HTML exporter would ignore the content
of the bloc. Instead, it will:
1) Parse the targets defined with :targets and who are inside directory
defined with :in-directory
2) Save the parsed data in a json file, named like the org-mode file but
with the suffix "-index"
3) In the exported HTML file, add a search field in the place where the
dynamic bloc is and also add the needed JS and CSS in the header
In the HTML exported file, we have now a search field. That could get
its data from the generated json file.
# Step 3: An option in org-publish to auto-generate a search page
In this step, a new option is added to org-publish projects:
:auto-search
When set to "t", before the publication, a new org-mode file is created.
Named "search.org" it contain an auto-generated search dynamic bloc.
This bloc have, as parameters:
- :target set to files
- :in-directory set to the publish project parameter :base-directory
# Usages
The search feature of step 1 could be used independently of HTML export.
For example, users can search in their notes, or in their personal wiki
with this search feature.
The step 2 could also be used to have search features limited to some
part of a website build with Org-mode.
And the step 3 let user add a search feature to their website with only
1 parameter in their org-publish project.
# Implementation
The search feature could be entirely new, or based on org-ql. It would
be nice to be based on org-ql, as I could help to make it upstream. It
would require to add files as target for search.
For the search in the web page, I could write the JS script, re-use an
existing library or not. All the HTML part could be customized with
parameters specific to the HTML exporter.
It's a lot of work and more ambitious than solution 1 or 2 in my
previous e-mail. But it would add the search feature not only in
HTML export but in Org-mode too. And it could help to make org-ql
upstream.
Sébastien Gendre <s...@k-7.ch> writes:
> [[PGP Signed Part:Good signature from B586F7C77239E29E Sébastien Gendre
> <s...@k-7.ch> (trust ultimate) created at 2025-04-01T23:27:30+0200 using RSA]]
> About how to be independent of the search engine. I think of 2
> solutions.
>
>
> # Solution 1
>
> After looking at Pagefind, I was thinking of taking inspiration from
> "org-export-define-backend".
>
> Creating a function named like "org-html-define-search-engine" and that
> will have as parameters:
>
> - A search engine name
>
> - A HTML template for the search field (can be nil)
>
> - A HTML template for the header (to load JS and CSS, can be nil)
>
> - A function to build the index (can be nil for no function)
>
> - A function to install JS and CSS on the output folder (can be nil for
> no function)
>
> - Maybe a function to download/install external command line tool (like
> the pagefind tool used to index content and install JS+CSS)
>
> The new defined engine is stored in a variable and its name is the name
> given to "org-html-define-search-engine".
>
> Then, for a user who want to add a search field in its org-publish
> project, he or she just add an option ":html-search-engine 'pagefind".
> Or any defined search engine.
>
>
> # Solution 2
>
> I was looking into tools suggested for the Hugo static site
> generator:
> https://gohugo.io/tools/search/
>
> And this was interesting:
> https://gist.github.com/eddiewebb/735feb48f50f0ddd65ae5606a1cb41ae
>
> It use Hugo to directly generate an Index: It is configured to export
> the home page into JSON in addition to HTML. Then, it use Fuse.js to add
> a search field in the generated HTML.
>
> It could be possible take this path with Org-mode. In this case:
>
> - It need an official json export in Org-mode to build the index
>
> - The search field is just 2 templates: 1 for the field itself, 1 for the
> header
>
> - We still need to include JS + CSS in the output folder in a way or another
>
> - We can generate the index automatically for each published files or
> let the user do it in another org-publish project
>
>
> This second solution could ask more steps from the user, may be less
> flexible than the solution 1 and I don't know if the performance with
> multiple json files is good. Maybe we need to generate only 1 JSON files
> with an entry for each published files.
>
> But: A JSON org-export could be useful for other case. After all, an
> org-mode file containing headings with properties is a form of database.
> Other persons could find an utility to have a JSON exporter.
>
> And we maybe do an hybrid solution between solution 1 and 2:
>
> - We can have a function to define new search engines plus a simple
> option in org-publish project to choose the used search engine
>
> - And having an official JSON org exporter can be used for some search
> engines when we prefer to do the index with Org-mode/Emacs
>
> - And the official JSON exporter can also be used manually for people
> who want or need it
>
>
> What do you think ?
>
>
>
>
>
> Ihor Radchenko <yanta...@posteo.net> writes:
>
>> Sébastien Gendre <s...@k-7.ch> writes:
>>
>>> Ok, I have some free time for the next weeks.
>>>
>>> To summary, and to refresh my memory:
>>> ...
>>>
>>> Did I miss something ?
>>
>> Everything looks right.
>
> [[End of PGP Signed Part]]
signature.asc
Description: PGP signature
