Thanks for your comments, John, that is very interesting. I'll have to
check out your db code.
I'll drop a penny in the bucket with this:
I spent a few hours trying an alternative approach that uses
org-element-parse-buffer to parse each file, then operates on the parse
tree it produces. I was hoping it might have a chance to be an
improvement, but unfortunately it's not: org-element-parse-buffer is
very slow in comparison to the existing agenda code, and it eats up
almost all the time spent building the (very primitive) agenda views.
So, I think all we need is for someone to write a version of
org-element-parse-buffer in C or CL or whatever their favorite language
that compiles to a fast binary may be. Then we can read that into Emacs
and operate on the resulting tree quickly. Yep, that's ALL we
Seriously, though, it would be amazing if the new library support in
Emacs could be used to interface with a fast Org parser that provided
the same output as org-element-parse-buffer. Maybe it could even be
added to Emacs core, as the Org file format doesn't change much.
All that's way over my head, I'm afraid. Would some kind of bounty work
for a project like this? I'd be willing to contribute a few dollars and
some testing. :)
I also gave a cursory look at profiling org-element-parse-buffer, but I
don't think Nicolas left any low-hanging fruit there. But I'm not
experienced with profiling code like that, so don't take my word for