Re: keeping subtree heading on export

[Matt Price]

thanks for the response, Ihor -- reply inline below:

On Fri, Apr 26, 2024 at 9:45 AM Ihor Radchenko  wrote:

> Matt Price  writes:
>
> > ... I need to export each individual page
> > of the "book" to its own markdown page.  However, jupyter-book expects to
> > find the title of the page in the initial first-level heading.  So I'd
> like
> > to retain the subtree "title" as a first-level heading, and demote the
> > remaining headings to their original state within the larger org
> document.
> >
> > Does anyone know of an existing exporter that already does this, from
> which
> > I can steal? Or if not, how would you suggest I go about doing this?
>
> May you provide an example demonstrating initial Org mode document and
> how the exported md documents should look like?
>
>
I don't think my request was very clear.  Let's say I'm writing a "book"
(really a documentation set of some kind) with several "chapters" (really,
each chapter is an indiviual html page, though more complex nesting is
permitted by jupyter-book).

I write in org-mode:

-
* Chapter 1
text
** Chapter 1.1
text
** Chapter 1.2
* Chapter 2
text
** Chapter 2.1
text
-

And I want to produce two markdown files:

chapter-1.md:
-
# Chapter 1
text
##  Chapter 1.1
text
##  Chapter 1.2
--

chapter-2.md

# Chapter 2
text
## Chapter 2.1
text
-

I tried to learn a little more on my own after posting.  I can set
`org-md-toplevel-hlevel` to `2`, and then in the template function add the
title property "manually" by extracting it from the info communication
channel:

--
(defun org-myst-inner-template (contents info)
  "Return body of document after converting it to Markdown syntax.
CONTENTS is the transcoded contents string.  INFO is a plist
holding export options."
  (let* ((depth (plist-get info :with-toc))
 (headlines (and depth (org-export-collect-headlines info depth)))
 (toc-string (or (mapconcat (lambda (headline)
  (org-myst-format-toc headline info))
headlines "\n")
 ""))
 (toc-tail (if headlines "\n\n" ""))
 (front-matter (org-myst-front-matter))
 (title (org-export-data (plist-get info :title) info)))
(org-trim (concat front-matter toc-string toc-tail "\n" "# " title
"\n\n"  contents (org-myst-footnote-section info)
--

This works ok!  But the problem ocmes because I would like to be able to
*also* sometimes export a whole file (rather than just a subtree) using the
same exporter.That's because I have inherited osme projects where files
take hte form:

chapter-1.org
--
* Chapter 1
text
** Chapter 1.1
text
** Chapter 1.2
--
etc.

The real problem I have is that, when exporting a subtree, I want to set
org-md-toplevel-hlevel to "2" and add the title; when exporting a whole
file, I want to set org-md-toplevel-hlevel to "1" and ignore the title.

I don't think this is how org exporters are supposed to work, but I'm
trying to interface to an established system that has chosen osme
un-org-like conventions.

Is this clearer, and do you see a way to do this?  I am just looking at the
source code and I wonder if I could add some (let) bindings inside
org-myst-export-to-markdown and -as-markdown before calling
org-export-to-[buffer|file].

hmm. Just tried it and it seems to work.  Something seems wrong about this
kind of code, though, in which I let-bind a variable in one function solely
so that I can use it in another.  Is there a better way?



 Un that case ,I would want to set


> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at <https://orgmode.org/>.
> Support Org development at <https://liberapay.com/org-mode>,
> or support my work at <https://liberapay.com/yantar92>
>

keeping subtree heading on export

[Matt Price]

I'm writing a primitive exporter for jupyter-book
, which is a neat way to
maintain static documentation and executable jupyter notebooks from the
same source files.  The source files are written in Myst flavored markdown
, so I started with a simpleminded fork of the gfm
exporter . What I'd really like,
though is something more like ox-hugo
, which maintains a whole blog in a
single org file.  For this to work, I need to export each individual page
of the "book" to its own markdown page.  However, jupyter-book expects to
find the title of the page in the initial first-level heading.  So I'd like
to retain the subtree "title" as a first-level heading, and demote the
remaining headings to their original state within the larger org document.

Does anyone know of an existing exporter that already does this, from which
I can steal? Or if not, how would you suggest I go about doing this?

Thanks,

Matt

Re: [BUG] Org-reveal and setupfile conflict [9.6.1 (9.6.1-gaa48c8)]

[Matt Price]

On Sun, Feb 26, 2023 at 4:04 PM Joshua Lambert 
wrote:

> Remember to cover the basics, that is, what you expected to happen and
> what in fact did happen.  You don't know how to make a good report?  See
>
>  https://orgmode.org/manual/Feedback.html#Feedback
>
> Your bug report will be posted to the Org mailing list.
> 
> When Org files use both Org Reveal and a ~#+setupfile:~ link for
> exported web pages, the CSS of exported Org Reveal slides inherits some
> of the CSS from the setupfile. So, I have to comment out the setupfile
> line each time I export slides. See the sample file below and steps to
> reproduce.
>
> 1. With the following Org file open, it should export the Org Reveal
> slides fine.
> 2. After you add the following line to that Org file after the title,
> try exporting and viewing the Org Reveals slides again. This time, their
> text is too big in the unnumbered lists. I think the CSS from the two
> export
> methods seems to be clashing.
>
> #+setupfile:
> https://fniessen.github.io/org-html-themes/org/theme-readtheorg.setup
>
> 3. Now comment out that setupfile line or delete it and the export works
> fine again.
>


I’m pretty sure this is the expected behaviour. Reveal is very sensitive to
css changes and it’s not recommended to use a setup file in this way with
the reveal exporter

There may be a way for the exporter to strip out extraneous css files, but
I think such a feature may be difficult to implement. You could perhaps ask
Jens, who maintains org-re-reveal.

>
> m.
>
>  End sample Org file --
>
>  Thank you to all who have contributed to the development of Org Mode and
> Emacs.
> J. Lambert
>
>
> Emacs  : GNU Emacs 28.2 (build 2, x86_64-w64-mingw32)
>  of 2022-09-13
> Package: Org mode version 9.6.1 (9.6.1-gaa48c8 @ )
>
> current state:
> ==
> (setq
>  org-link-elisp-confirm-function 'yes-or-no-p
>
>  org-bibtex-headline-format-function #[257 "\300 \236A\207" [:title] 3
> "\n\n(fn ENTRY)"]
>  org-download-file-format-function 'org-download-file-format-default
>  org-log-done 'time
>  org-fontify-done-headline nil
>  org-persist-after-read-hook '(org-element--cache-persist-after-read)
>  org-export-before-parsing-hook '(org-attach-expand-links)
>  org-cycle-tab-first-hook '(org-babel-hide-result-toggle-maybe
> org-babel-header-arg-expand)
>  org-download-method 'attach
>  org-archive-hook '(org-attach-archive-delete-maybe)
>  org-odt-format-inlinetask-function
> 'org-odt-format-inlinetask-default-function
>  org-ascii-format-drawer-function #[771 " \207" [] 4 "\n\n(fn NAME
> CONTENTS WIDTH)"]
>  org-cycle-hook '(org-cycle-hide-archived-subtrees
> org-cycle-show-empty-lines
> org-cycle-optimize-window-after-visibility-change
>   org-cycle-display-inline-images)
>  org-persist-before-read-hook '(org-element--cache-persist-before-read)
>  org-modules '(ol-bbdb ol-bibtex ol-docview ol-doi ol-eww ol-gnus
> ol-info ol-irc ol-mhe ol-rmail ol-w3m org-collector)
>  org-image-actual-width nil
>  org-mode-hook '(#[0 "\301\211 \207" [imenu-create-index-function
> org-imenu-get-tree] 2] t
> #[0 "\300\301\302\303\304$\207" [add-hook change-major-mode-hook
> org-fold-show-all append local] 5]
> #[0 "\300\301\302\303\304$\207" [add-hook change-major-mode-hook
> org-babel-show-result-all append local] 5] org-babel-result-hide-spec
> org-babel-hide-all-hashes)
>  org-babel-load-languages '((emacs-lisp . t) (python . t) (sql . t)
> (sqlite . t) (shell . t))
>  org-latex-format-drawer-function #[514 "\207" [] 3 "\n\n(fn _ CONTENTS)"]
>  org-download-abbreviate-filename-function 'file-relative-name
>  org-latex-format-headline-function
> 'org-latex-format-headline-default-function
>  org-confirm-shell-link-function 'yes-or-no-p
>  org-download-annotate-function 'org-download-annotate-default
>  org-html-format-drawer-function #[514 "\207" [] 3 "\n\n(fn NAME
> CONTENTS)"]
>  outline-isearch-open-invisible-function 'outline-isearch-open-invisible
>  org-startup-indented t
>  org-fold-catch-invisible-edits t
>  org-latex-classes '(("beamer" "\\documentclass[presentation]{beamer}"
> ("\\section{%s}" . "\\section*{%s}") ("\\subsection{%s}" .
> "\\subsection*{%s}")
>   ("\\subsubsection{%s}" . "\\subsubsection*{%s}"))
>  ("article" "\\documentclass[11pt]{article}" ("\\section{%s}" .
> "\\section*{%s}") ("\\subsection{%s}" . "\\subsection*{%s}")
>   ("\\subsubsection{%s}" . "\\subsubsection*{%s}")
> ("\\paragraph{%s}" . "\\paragraph*{%s}")
>   ("\\subparagraph{%s}" . "\\subparagraph*{%s}"))
>  ("report" "\\documentclass[11pt]{report}" ("\\part{%s}" .
> "\\part*{%s}") ("\\chapter{%s}" . "\\chapter*{%s}")
>   ("\\section{%s}" . "\\section*{%s}") ("\\subsection{%s}" .
> "\\subsection*{%s}") ("\\subsubsection{%s}" . "\\subsubsection*{%s}"))
>  ("book" "\\documentclass[11pt]{book}" ("\\part{%s}" .
> "\\part*{%s}") ("\\chapter{%s}" . "\\chapter*{%s}")
>   

Re: citar--library-file-action

[Matt Price]

Bruce will know better than me!!!

On Tue, Oct 25, 2022 at 4:46 PM Matt Price  wrote:

> I’d look at the current function definition to be sure (C-h f
> citar—library-file-action), but open is the standard macOS command for
> opening files. The Linux equivalent is xdg-open. My guess is that somehow
> you’ve told citar or  org or emacs to use open for these links. Or it’s
> possible that the citar devs accidentally hard coded open into the
> function.
>
> On Tue, Oct 25, 2022 at 4:27 PM Henrik Frisk  wrote:
>
>> Hi,
>>
>> After updating a bunch of packages I'm getting this error message when I
>> try to open an associated file via a reference:
>>
>> [cite:@key]
>>
>> Before it was working fine via emark either by clicking on the reference
>> or by running embark-act on it. Now I get this instead:
>>
>> citar--library-file-action: Symbol’s function definition is void: open
>>
>> Any suggestions?
>>
>> /Henrik
>>
>>

Re: citar--library-file-action

[Matt Price]

I’d look at the current function definition to be sure (C-h f
citar—library-file-action), but open is the standard macOS command for
opening files. The Linux equivalent is xdg-open. My guess is that somehow
you’ve told citar or  org or emacs to use open for these links. Or it’s
possible that the citar devs accidentally hard coded open into the
function.

On Tue, Oct 25, 2022 at 4:27 PM Henrik Frisk  wrote:

> Hi,
>
> After updating a bunch of packages I'm getting this error message when I
> try to open an associated file via a reference:
>
> [cite:@key]
>
> Before it was working fine via emark either by clicking on the reference
> or by running embark-act on it. Now I get this instead:
>
> citar--library-file-action: Symbol’s function definition is void: open
>
> Any suggestions?
>
> /Henrik
>
>

unfolding is delayed in relatively small file

[Matt Price]

Hi everyone,

I just updated to recent main branch of org-mode with the new org folding
engine. My emacs is from git, but somewhat out of date (2022-05-08).

WIth a relatively small test file, I find that folding and unfolding a
subtree with children does not cause a redisplay unless I alt-tab away from
my Emacs window and then alt-tab back.  "FOLDED" or "SUBTREE NO CHILDREN"
will echo in the message area, but the state of hte buffer won't change
until I switch focus.

I haven't done any extensive testing yet, and have many third party
packages installed. I've also just now noticed that this behaviour is
somewhat intermittent, and somehow I managed to make it stop briefly in
my   MWE, but I don't seem to be able to trigger the shift back to normal
behaviour in a "real" org file.

I'm wondering if anyone else has seen this and what you've done to fix it!

Thanks,
Matt


MWE:


* fold me

some text
** some more text
let's see what's happening.
** test again

Re: [PATCH] org-attach-attach: fix symlinks

[Matt Price]

sorry I've been MIA -- I have been swamped at work and stopped following
the list!

Thank you for this, but more generally for the huge amount of work you put
in to improving org for everyone. The last year or so has seen such huge
improvements for me -- I am immensely grateful to everyone.

On Mon, Jul 4, 2022 at 9:01 AM Ihor Radchenko  wrote:

> Ihor Radchenko  writes:
>
> > Attaching an alternative patch. It makes use of make-symbolic-link
> > arguments.
>
> Applied onto main via 13a3dbcc9.
>
> Best,
> Ihor
>

Re: [RFC] DOCT: Declarative Org Capture Templates (easier template syntax)

[Matt Price]

On Sat, Mar 26, 2022 at 4:31 AM Ihor Radchenko  wrote:

> No Wayman  writes:
>
> > I'm open to bringing doct's features into Org mode, but I'd prefer
> > it not to be spread out over another two years.
>
> Proper integration of doct into org-capture.el would indeed be ideal. I
> am happy that you are ok with such option.
>
> Practically, we can do it step-by-step in separate patches:
> 1. Introduce alist format for org-capture-templates. No new
>functionality yet, just change the existing supported template
>options to :keyword value pairs. Also, make sure that we keep
>backwards compatibility.
> 2. Extend the alist with new options, like ${keyword} expansion, hooks,
>contexts, children, etc Maybe in a sequence of patches.
> 3. Provide validation of capture templates.
>
> WDYT?
>
> Best,
> Ihor
>

I'll just say that I would be an enthusiastic adopter of the new syntax
(just one data point). I agree wholeheartedly that named properties in
plists are much easier to remember and adjust than positional arguments,
especially when there are lots of the latter.   So, I hope this move
sforward.

Re: #2 Org mode profiling meetup on Sat, Mar 26 (was: Org mode profiling meetup on Sat, Feb 26 (was: profiling latency in large org-mode buffers (under both main & org-fold feature)))

[Matt Price]

On Thu, Mar 24, 2022 at 7:28 AM Bruce D'Arcus  wrote:

> On Wed, Mar 23, 2022 at 7:10 AM Ihor Radchenko  wrote:
> >
> > Dear all,
> >
> > There were several people who came to the last meetup looking for
> > information about debugging Org mode. The last meetup was rather
> > unhelpful in this regard since we dove into a specific use-case.
> >
> > I plan to try once more providing a more general introduction to Org
> > (and Emacs) debugging. Tentatively, I plan to talk about:
> > 1. Running Emacs with clean configuration + latest version of Org
> > 2. Bisecting config to find configuration-related issues
> > 3. Using Emacs profiler and sharing profiler results
> > 4. Answer any questions on the first three topics
>
> This is a great idea, Ihor. Have you considered recording this part
> and sharing it?
>
> I was just going to ask the same thing! I missed the last one too and
would like to benefit from your efforts this time.

folding day headings in org-agenda-mode?

[Matt Price]

I am very psyched to have started using Adam Porter's org-super-agenda, and
I really enjoy using origami-mode to fold section headings now.

However, I do really wish I could use the same keystroke to fold individual
days in the org agenda. I tried just invoking `origami-toggle-node from a
day header, and the results are unintuitive. in general, if I place the
cursor in a day header ("Friday 18 March 2022"), and invoke
`origami-toggle-node`, the header will disappear entirely, which is not the
behaviour I am looking for.

I imagine that, in order for folding to work, something has to be done to
make the folding mechanism aware of the syntactical units.

1) has anyone figured out a way to do this?
2) if not, does anyone have an idea of where I might look in the code for a
place to hack?
3) does this seem like something it would be worth having in org itself? It
feels like it would be a common use case.

thanks as always,

Matt

Re: profiling latency in large org-mode buffers (under both main & org-fold feature)

[Matt Price]

On Wed, Feb 23, 2022 at 12:22 AM Ihor Radchenko  wrote:

> Matt Price  writes:
>
> >>20128  80% - redisplay_internal (C function)
> >> 7142  28%  - assq
> >>  908   3%   - org-context
>
> Note that org-context is an obsolete function. Do you directly call it
> in your config? Or do you use a third-party package calling org-context?
>

Hmm.  I don't see it anywhere in my ~.emacs.d/elpa~ directory or in my
config file. I also went through ORG-NEWS and while it mentions that
org-context-p has been removed, I can't find a deprecation notice about
org-context.  I'm not quite sure what's going on. Will investigate further!

>
> Best,
> Ihor
>

Re: profiling latency in large org-mode buffers (under both main & org-fold feature)

[Matt Price]

sorry everyone, I accidentally sent this to Kaushal this morning,  and then
took quite a while to get back to a computer after he let me know my
mistake!

On Tue, Feb 22, 2022 at 10:12 AM Matt Price  wrote:

>
> On Tue, Feb 22, 2022 at 12:45 AM Kaushal Modi 
> wrote:
>
>>
>>
>> On Tue, Feb 22, 2022, 12:34 AM Ihor Radchenko  wrote:
>>
>>>
>>> I am wondering if many people in the list experience latency issues.
>>> Maybe we can organise an online meeting (jitsi or BBB) and collect the
>>> common causes/ do online interactive debugging?
>>>
>>
>> +1
>>
>> I have seen few people see this issue on the ox-hugo issue tracker:
>> https://github.com/kaushalmodi/ox-hugo/discussions/551#discussioncomment-2104352
>>
>
>
> I htink it's a great idea, Ihor!
>
> Meanwhile, I have a profile report. I had a little trouble getting the
> slowness to return (of course) but, subjectively, it seemed to get worse
> (subjectively slower, and the laptop fan started up b/c of high cpu usage)
> when I created and entered a src block. Apologies for the long paste:
>
>45707  70% - redisplay_internal (C function)
> 8468  13%  - substitute-command-keys
> 6111   9%   - #
>  943   1%- kill-buffer
>  708   1% - replace-buffer-in-windows
>  614   0%  - unrecord-window-buffer
>  515   0%   - assq-delete-all
>  142   0%  assoc-delete-all
>3   0% delete-char
> 8060  12%  - assq
> 2598   4%   - org-context
>   15   0%  org-inside-LaTeX-fragment-p
>   12   0%- org-in-src-block-p
>   12   0% - org-element-at-point
>9   0%  - org-element--cache-verify-element
>9   0% org-element--parse-to
>3   0%org-element--parse-to
>8   0%- org-at-timestamp-p
>8   0%   org-in-regexp
>  642   0%  + tab-bar-make-keymap
>  309   0%  + and
>  270   0%  + org-in-subtree-not-table-p
>  196   0%  + not
>  163   0%  + jit-lock-function
>  115   0%  + org-entry-get
>   96   0%keymap-canonicalize
>   56   0%org-at-table-p
>   52   0%  + #
>   48   0%  + #
>   43   0%table--row-column-insertion-point-p
>   29   0%org-inside-LaTeX-fragment-p
>   27   0%  + menu-bar-positive-p
>   26   0%  + eval
>   24   0%file-readable-p
>   21   0%  + funcall
>   16   0%  + imenu-update-menubar
>   14   0%  + vc-menu-map-filter
>   13   0%  + table--probe-cell
>   12   0%  + or
>   11   0%  + let
>   11   0%  + org-at-timestamp-p
>   10   0%  + flycheck-overlays-at
>7   0%undo-tree-update-menu-bar
>6   0%  + require
>6   0%  +
> emojify-update-visible-emojis-background-after-window-scroll
>6   0%kill-this-buffer-enabled-p
>4   0%mode-line-default-help-echo
>3   0%  + null
> 9192  14% - ...
> 9172  14%Automatic GC
>   20   0%  - kill-visual-line
>   20   0%   - kill-region
>   20   0%- filter-buffer-substring
>   20   0% - org-fold-core--buffer-substring-filter
>   20   0%  - buffer-substring--filter
>   20   0%   - #
>   20   0%- apply
>   20   0% - # F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_18>
>   20   0%  - #
>   20   0%   - apply
>   20   0%- #
>   20   0% - #
>   20   0%  - #
>   20   0%   - apply
>   20   0%- #
>   20   0% + delete-and-extract-region
> 7847  12% - command-execute
> 5749   8%  - funcall-interactively
> 2963   4%   + org-self-insert-command
> 2186   3%   + org-cycle
>  148   0%   + corfu-insert
>  146   0%   + execute-extended-command
>  121   0%   + org-return
>   32   0%   + #
>   26   0%   + #
>   24   0%   + mwim-beginning
>   19   0%   + org-delete-backward-char
>   19   0%   + org-kill-line
>9   0%   + #
>6   0%   + file-notify-handle-event
> 2095   3%  + byte-code
> 1359   2% + timer-event-handler
>  375   0% + org-appear--post-cmd
>  160   0% + corfu--post-command
>   61   0% + org-fragtog--post-c

Re: profiling latency in large org-mode buffers (under both main & org-fold feature)

[Matt Price]

Yes, it definitely seems to be related tofile size, which makes me think
that some kind of buffer parsing is the cause of the problem. I'll replay
in more detail to Ihor, down below!

On Mon, Feb 21, 2022 at 5:22 PM Samuel Wales  wrote:

> i have been dealing with latency also, often in undo-tree.  this might
> be a dumb suggestion, but is it related to org file size?  my files
> have not really grown /that/ much but maybe you could bisect one.  as
> opposed to config.
>
> i am not saying that your org files are too big.  just that maybe it
> could lead to insights.
>
>
> On 2/21/22, Matt Price  wrote:
> > I'm trying to figure out what causes high latency while typing in large
> > org-mode files.  The issue is very clearly a result of my large config
> > file, but I'm not sure how to track it down with any precision.
> >
> > My main literate config file is ~/.emacs.d/emacs-init.org, currently
> 15000
> > lines, 260 src blocks.
> > If I create a ~minimal.el~ config like this:
> >
> > (let* ((all-paths
> >   '("/home/matt/src/org-mode/emacs/site-lisp/org")))
> > (dolist (p all-paths)
> >   (add-to-list 'load-path p)))
> >
> >   (require 'org)
> >   (find-file "~/.emacs.d/emacs-init.org")
> >
> > then I do not notice any latency while typing.  If I run the profiler
> while
> > using the minimal config, the profile looks about like this at a high
> > level:
> >
> > 1397  71% - command-execute
> >  740  37%  - funcall-interactively
> >  718  36%   - org-self-insert-command
> >  686  34%+ org-element--cache-after-change
> >   10   0%+ org-fold-core--fix-folded-region
> >3   0%+ blink-paren-post-self-insert-function
> >2   0%+ jit-lock-after-change
> >1   0%
> > org-fold-check-before-invisible-edit--text-properties
> >9   0%   + previous-line
> >6   0%   + minibuffer-complete
> >3   0%   + org-return
> >3   0%   + execute-extended-command
> >  657  33%  - byte-code
> >  657  33%   - read-extended-command
> >   64   3%- completing-read-default
> >   14   0% + redisplay_internal (C function)
> >1   0% + timer-event-handler
> >  371  18% - redisplay_internal (C function)
> >  251  12%  + jit-lock-function
> >   90   4%  + assq
> >7   0%  + substitute-command-keys
> >3   0%  + eval
> >  125   6% + timer-event-handler
> >   69   3% + ...
> >
> > --
> > However, if I instead use my fairly extensive main config, latency is
> high
> > enough that there's a noticeable delay while typing ordinary words. I see
> > this  regardless of whether I build from main or from Ihor's org-fold
> > feature branch on github. The profiler overview here is pretty different
> --
> > redisplay_internal takes a much higher percentage of the CPU requirement:
> >
> >  3170  56% - redisplay_internal (C function)
> >  693  12%  - substitute-command-keys
> >  417   7%   + #
> >   59   1%  + assq
> >   49   0%  + org-in-subtree-not-table-p
> >   36   0%  + tab-bar-make-keymap
> >   35   0%and
> >   24   0%  + not
> >   16   0%org-at-table-p
> >   13   0%  + jit-lock-function
> >8   0%keymap-canonicalize
> >7   0%  + #
> >4   0%  + funcall
> >4   0%display-graphic-p
> >3   0%  + #
> >3   0%file-readable-p
> >3   0%  + table--probe-cell
> >3   0%table--row-column-insertion-point-p
> > 1486  26% - command-execute
> > 1200  21%  - byte-code
> > 1200  21%   - read-extended-command
> > 1200  21%- completing-read-default
> > 1200  21% - apply
> > 1200  21%  - vertico--advice
> >  475   8%   + #
> >
> > --
> > I've almost never used the profiler and am not quite sure how I should
> > proceed to debug this.  I realize I can comment out parts of the config
> one
> > at a time, but that is not so easy for me to do in my current setup, and
> I
> > suppose there are likely to be multiple contributing causes, which I may
> > not really notice except in the aggregate.
> >
> > If anyone has suggestions, I would love to hear them!
> >
> > Thanks,
> >
> > Matt
> >
>
>
> --
> The Kafka Pandemic
>
> A blog about science, health, human rights, and misopathy:
> https://thekafkapandemic.blogspot.com
>

profiling latency in large org-mode buffers (under both main & org-fold feature)

[Matt Price]

I'm trying to figure out what causes high latency while typing in large
org-mode files.  The issue is very clearly a result of my large config
file, but I'm not sure how to track it down with any precision.

My main literate config file is ~/.emacs.d/emacs-init.org, currently 15000
lines, 260 src blocks.
If I create a ~minimal.el~ config like this:

(let* ((all-paths
  '("/home/matt/src/org-mode/emacs/site-lisp/org")))
(dolist (p all-paths)
  (add-to-list 'load-path p)))

  (require 'org)
  (find-file "~/.emacs.d/emacs-init.org")

then I do not notice any latency while typing.  If I run the profiler while
using the minimal config, the profile looks about like this at a high
level:

1397  71% - command-execute
 740  37%  - funcall-interactively
 718  36%   - org-self-insert-command
 686  34%+ org-element--cache-after-change
  10   0%+ org-fold-core--fix-folded-region
   3   0%+ blink-paren-post-self-insert-function
   2   0%+ jit-lock-after-change
   1   0%  org-fold-check-before-invisible-edit--text-properties
   9   0%   + previous-line
   6   0%   + minibuffer-complete
   3   0%   + org-return
   3   0%   + execute-extended-command
 657  33%  - byte-code
 657  33%   - read-extended-command
  64   3%- completing-read-default
  14   0% + redisplay_internal (C function)
   1   0% + timer-event-handler
 371  18% - redisplay_internal (C function)
 251  12%  + jit-lock-function
  90   4%  + assq
   7   0%  + substitute-command-keys
   3   0%  + eval
 125   6% + timer-event-handler
  69   3% + ...

--
However, if I instead use my fairly extensive main config, latency is high
enough that there's a noticeable delay while typing ordinary words. I see
this  regardless of whether I build from main or from Ihor's org-fold
feature branch on github. The profiler overview here is pretty different --
redisplay_internal takes a much higher percentage of the CPU requirement:

 3170  56% - redisplay_internal (C function)
 693  12%  - substitute-command-keys
 417   7%   + #
  59   1%  + assq
  49   0%  + org-in-subtree-not-table-p
  36   0%  + tab-bar-make-keymap
  35   0%and
  24   0%  + not
  16   0%org-at-table-p
  13   0%  + jit-lock-function
   8   0%keymap-canonicalize
   7   0%  + #
   4   0%  + funcall
   4   0%display-graphic-p
   3   0%  + #
   3   0%file-readable-p
   3   0%  + table--probe-cell
   3   0%table--row-column-insertion-point-p
1486  26% - command-execute
1200  21%  - byte-code
1200  21%   - read-extended-command
1200  21%- completing-read-default
1200  21% - apply
1200  21%  - vertico--advice
 475   8%   + #

--
I've almost never used the profiler and am not quite sure how I should
proceed to debug this.  I realize I can comment out parts of the config one
at a time, but that is not so easy for me to do in my current setup, and I
suppose there are likely to be multiple contributing causes, which I may
not really notice except in the aggregate.

If anyone has suggestions, I would love to hear them!

Thanks,

Matt

Re: handling as special special block in derived export

[Matt Price]

On Thu, Jan 20, 2022 at 11:12 AM John Kitchin 
wrote:

> It depends on what the src things look like. You might be able to just
> call ffap or some variant of it. Here is an example of the follow part that
> works for a file and url for me.
>
> #+BEGIN_SRC emacs-lisp
> (org-link-set-parameters
>  "rstack"
>  :follow (lambda (path)
>   (ffap (or (ffap-url-at-point)
> (ffap-file-at-point)
> #+END_SRC
>
> #+RESULTS:
> | :follow | (lambda (path) (ffap (or (ffap-url-at-point)
> (ffap-file-at-point |
>
>
> [[rstack:(("./screenshots/date-20-01-2022-time-10-36-54.png" . fade-out)
> (src2 . fade-in) ("https://google.com; . fade-out))]]
>
>
> This is what I have so far. I'm pretty happy with it; it's quite concise,
and  will fill in default values for the fade in and out instructions.

When I asked my question, I left out an optional second parameter that
these stacking images can take -- a "fragment-index" number. I'm trying to
figure out a way to add that additional info into the link without making
it (a) unnecessarily long, esp for the majority of cases where I won't use
that index; or (b) impossible to read. I guess I could make each list
element a plist instead? Like ((:s ~/image.png :i 4 :f fade-in-then-out))?

Anyway, here's my code:

(defun r-stack-follow (path)
  (let* ((srcs (read path ))
 (completions (-map (lambda (src) (car src)) srcs)  )
 (chosen  (completing-read "Follow link to: " completions)))
(if (ffap-url-p chosen) (browse-url chosen) (find-file-ace-window
chosen
(defun r-stack-export (path desc backend)
  (cond
   ((eq 'html backend)
 (let ((srcs (read path)))
   (format "\n%s\n"
   (string-join
(cl-loop for index from 1
 for (src . data-fragment) in srcs
 collect
 (format "  "
 (or data-fragment
 (cond
  ((eql 1 index) "fade-out")
  ((eql  (length srcs) index)
 "fade-in")
  (t "fade-in-then-out")))
 src))
"\n"))


(org-link-set-parameters
 "r-stack"
 :follow 'r-stack-follow
 :export 'r-stack-export
 :face '(:foreground "red")
 :help-echo "Click me for a message.")




> John
>
> ---
> Professor John Kitchin (he/him/his)
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
>
>
>
> On Thu, Jan 20, 2022 at 10:21 AM Matt Price  wrote:
>
>>
>>
>> On Wed, Jan 19, 2022 at 12:18 PM John Kitchin 
>> wrote:
>>
>>> I am not sure this is quite what you are looking for. You could use a
>>> macro like this.
>>>
>>>
>>>
>>> {{{r-stack(((src1 . fade-out) (src2 . fade-in) (src3 . fade-out)))}}}
>>>
>>> * code :noexport:
>>>
>>> #+macro: r-stack (eval (r-stack $1))
>>>
>>> #+BEGIN_SRC emacs-lisp
>>> (defun r-stack (src-alist)
>>>   "SRC-alist will be a string containing a list of (src . data-fragment)
>>> src is a url or filename
>>> data-fragment
>>>
>>> Returns a string for export."
>>>   (let ((src (read src-alist)))
>>> (format "#+BEGIN_EXPORT html
>>> 
>>> %s
>>> 
>>> ,#+END_EXPORT"
>>>(string-join
>>> (cl-loop for (src . data-fragment) in src
>>>  collect
>>>  (format "  "
>>>  data-fragment src))
>>> "\n"
>>>
>>>
>>> (r-stack "((src1 . fade-out) (src2 . fade-in) (src3 . fade-out))")
>>> #+END_SRC
>>>
>>> #+RESULTS:
>>> : #+BEGIN_EXPORT html
>>> : 
>>> :   
>>> :   
>>> :   
>>> : 
>>> : #+END_EXPORT
>>>
>>> You could also make a link do that.
>>>
>>
>> Huh.  This wasn't at all what I was thinking but it may be a much better
>> way than I'd htought of.
>>
>> A link seems like it would be a great solution, especially if I could
>> figure out how to get the :follow function to open an individual ~src~
>> path. Would you use "looking-at" to get the right candidate

Re: handling as special special block in derived export

[Matt Price]

On Wed, Jan 19, 2022 at 12:18 PM John Kitchin 
wrote:

> I am not sure this is quite what you are looking for. You could use a
> macro like this.
>
>
>
> {{{r-stack(((src1 . fade-out) (src2 . fade-in) (src3 . fade-out)))}}}
>
> * code :noexport:
>
> #+macro: r-stack (eval (r-stack $1))
>
> #+BEGIN_SRC emacs-lisp
> (defun r-stack (src-alist)
>   "SRC-alist will be a string containing a list of (src . data-fragment)
> src is a url or filename
> data-fragment
>
> Returns a string for export."
>   (let ((src (read src-alist)))
> (format "#+BEGIN_EXPORT html
> 
> %s
> 
> ,#+END_EXPORT"
>(string-join
> (cl-loop for (src . data-fragment) in src
>  collect
>  (format "  "
>  data-fragment src))
> "\n"
>
>
> (r-stack "((src1 . fade-out) (src2 . fade-in) (src3 . fade-out))")
> #+END_SRC
>
> #+RESULTS:
> : #+BEGIN_EXPORT html
> : 
> :   
> :   
> :   
> : 
> : #+END_EXPORT
>
> You could also make a link do that.
>

Huh.  This wasn't at all what I was thinking but it may be a much better
way than I'd htought of.

A link seems like it would be a great solution, especially if I could
figure out how to get the :follow function to open an individual ~src~
path. Would you use "looking-at" to get the right candidate, or can you
think of a way to get completion candidates for a function that opens the
file? I'm looking at your eamples in
https://kitchingroup.cheme.cmu.edu/blog/2016/11/04/New-link-features-in-org-9
but can't quite follow the code.

Thank so mjch for this really interesting solution.

>
> John
>
> ---
> Professor John Kitchin (he/him/his)
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
>
>
>
> On Wed, Jan 19, 2022 at 9:42 AM Matt Price  wrote:
>
>> I am trying ot figure out if I can create a simplified syntax for a
>> particular special block in a derived HTML exporter.
>>
>> I'm trying to produce HTML like this:
>>
>>   > data-fragment="fade-in" src="..."/>
>>
>> The derived backend (org-re-reveal) already has an
>>
>> #+ATTR_REVEAL that an make the data-fragment attributes, so it's not hard to 
>> produce
>>  the desired outpu:
>>
>> #+begin_r-stack
>> #+ATTR_REVEAL: :frag appear[[imglink1]]
>> #+ATTR_REVEAL: :frag appear[[imglink2]]#+end_r-stack
>>
>> However, I'd really like to add a less verbose syntax, like this:
>>
>> #+begin_r-stack :frag (appear appear)[[imglink1]][[imglink2]]#+end_r-stack
>>
>> My question is: will the exporter preserve information from these 
>> header-like arguments, and is
>> there a mechanism I can use in a custom ~special-block-function~ to make use 
>> of htem?
>>
>> Thanks for your help as always!
>>
>> Matt
>>
>>

Re: handling as special special block in derived export

[Matt Price]

On Wed, Jan 19, 2022 at 3:56 PM Berry, Charles 
wrote:

> Matt,
>
> > On Jan 19, 2022, at 5:52 AM, Matt Price  wrote:
> >
> > However, I'd really like to add a less verbose syntax, like this:
> >
> > #+begin_r-stack :frag (appear appear)
> > [[imglink1]]
> > [[imglink2]]
> > #+end_r-stack
> >
> > My question is: will the exporter preserve information from these
> header-like arguments, and is
> > there a mechanism I can use in a custom ~special-block-function~ to make
> use of htem?
> >
>
> Not as you have it there, but
>
> #+header: :frag (appear appear)
> #+begin_r-stack
> [[imglink1]]
> [[imglink2]]
> #+end_r-stack
>
>
> parses as
>
> (special-block
>  (:type "r-stack" :begin 727 :end 815 :contents-begin 775 :contents-end
> 801 :post-blank 0 :post-affiliated 759 :header
> (":frag (appear appear)")
> :mode nil :granularity element :parent nil))
>
> if that helps.
>

Ah, thank you. I had never used #+header: before.

THis helps a little, but I guess the parser works from the inside out. That
is, the special block contents are already parsed before the block itself.
As a result, I don't see how to add classes to the contents on the basis of
the block headers.  Also, it doesn't seem trivial to identify the "direct
children" of the block, the way one might in javaScript with, say,
~document.querySelectorAll()~.or just ~elem.children()~.   So I'm not
convinced my initial impulse to solve the problem this way will be
fruitful.

>
> Chuck
>

handling as special special block in derived export

[Matt Price]

I am trying ot figure out if I can create a simplified syntax for a
particular special block in a derived HTML exporter.

I'm trying to produce HTML like this:

  

The derived backend (org-re-reveal) already has an

#+ATTR_REVEAL that an make the data-fragment attributes, so it's not
hard to produce
 the desired outpu:

#+begin_r-stack
#+ATTR_REVEAL: :frag appear[[imglink1]]
#+ATTR_REVEAL: :frag appear[[imglink2]]#+end_r-stack

However, I'd really like to add a less verbose syntax, like this:

#+begin_r-stack :frag (appear appear)[[imglink1]][[imglink2]]#+end_r-stack

My question is: will the exporter preserve information from these
header-like arguments, and is
there a mechanism I can use in a custom ~special-block-function~ to
make use of htem?

Thanks for your help as always!

Matt

Re: "Orgdown", the new name for the syntax of Org-mode

[Matt Price]

I don't have very thoughtful comments but I'll just say that I really do
also like the idea of a formal syntax; that a staged standard seems to make
sense to me, though I'm ignorant about how syntaxes are normally defined
and managed;  and am generally not super enthusiastic about the particular
name that's been chosen.

It seems like there are lots of people thinking in similar ways about
related issues -- that is, whenever it's possible, treating org files as
syntax trees rather than linear text trees.. The org-ml project comes to
mind; so does the recent work on tree-sitter in emacs; and Ihor's recent
thread on changing fontification.  I guess it would be nice if the
smarter-than-me people involved in all these projects are tracking each
other somewhat to make sure efforts converge as much as possible.

For me it would be really great to have better support for an org syntax in
VSCode & in Node; I'm sure other people have their own priority areas.  A
syntax definition would surely help?


On Mon, Nov 29, 2021 at 8:22 AM Karl Voit  wrote:

> Hi Tim,
>
> * Tim Cross  wrote:
> >
> > Hi Karl,
> >
> > while I can appreciate the point you are making, I'm doubtful your
> > suggestion will gain the traction necessary to work.
>
> You might be right. Only time will tell. ;-)
>
> > To me, it feels a little like the frequent posts from RMS in the
> > emacs-devel list where he gets upset when people refer to Linux
> > instead of GNU Linux.
>
> I disagree here.
>
> Linux vs. GNU/Linux are two different names for the same thing.
> Org-mode is an Elisp implementation and Orgdown is just a syntax
> definition. So they are completely different things.
>
> > To some extent, the distinction will be too subtle for many and
> > often, it isn't clear whether an issue is a syntax definition
> > (orgdown) or an implementation bug or just simply user
> > misunderstanding.
>
> It seems to be the case that the name "Orgdown" is the reason why
> the Org-mode community does not support the idea of an
> implementation-agnostic definition of the syntax. Which is ... kinda
> funny if you think about it.
>
> Well if the project is not working out, at least I made my point and
> we continue to have all those misunderstandings and lack of Orgdown
> support in 3rd party tools (because Org-mode is way too big).
>
> > Perhaps we just need a name for the markup syntax which doesn't actually
> > reference 'org' at all - it simply is the markup syntax which org
> > happens to use. A completely separate name might avoid confusion and
> > would make it very clear that the markup syntax is not org mode. Problem
> > is, naming is terribly difficult and I have no suggestions on what would
> > be a good name.
>
> Oh, there is a very large danger here of getting something that is
> not compatible with Org-mode any more. I don't think that this would
> be a good thing. At least the different flavors killed the fun of
> Markdown for me.
>
> > I have not yet viewed your video, but will certainly be doing so. Again,
> > agree with the sentiment of what your trying to do, just not convinced
> > it is compatible with basic human nature.
>
> Maybe we need to differ between the Org-mode community with
> potential bias and the main target group of people who did use
> Markdown in the past and never have heard of Org-mode before?
>
> --
> get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode:
>> get Memacs from https://github.com/novoid/Memacs <
> Personal Information Management > http://Karl-Voit.at/tags/pim/
> Emacs-related > http://Karl-Voit.at/tags/emacs/
>
>
>

[PATCH] org-attach-attach: fix symlinks

[Matt Price]

I'm having trouble with org-attach-attach if my current buffer is visiting
a filepath starting with "~/". This trivial patch fixes the problem for me
by running (expand-file-name) on the file before attaching. I always use
the 'lns method, so I don't know whether it might be better to
expand-file-name before running any of the methods.

Alternatively, it might actually be better to use a *relative* file path
where possible (e.g. to ensure that project directories are fully
portable), but I'm  not sure how best to do that.

This should be a tinychange, but also I have signed FSF paperwork.
From f618fb512e62ccb2e700242a7678dacbc2b025e2 Mon Sep 17 00:00:00 2001
From: Matt Price 
Date: Mon, 15 Nov 2021 08:14:37 -0500
Subject: [PATCH] org-attach-attach: expand file names before linking

when using symbolic links for attachments, ensure links are fully
expanded to avoid filesystem errors.
---
 lisp/org-attach.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-attach.el b/lisp/org-attach.el
index 75db69c9c..abdce3e7a 100644
--- a/lisp/org-attach.el
+++ b/lisp/org-attach.el
@@ -523,7 +523,7 @@ METHOD may be `cp', `mv', `ln', `lns' or `url' default taken from
((eq method 'mv) (rename-file file attach-file))
((eq method 'cp) (copy-file file attach-file))
((eq method 'ln) (add-name-to-file file attach-file))
-   ((eq method 'lns) (make-symbolic-link file attach-file))
+   ((eq method 'lns) (make-symbolic-link (expand-file-name file) attach-file))
((eq method 'url) (url-copy-file file attach-file)))
   (run-hook-with-args 'org-attach-after-change-hook attach-dir)
   (org-attach-tag)
-- 
2.33.1

reference a remote named block in #+CALL: line

[Matt Price]

I am getting used to calling library-of-babel functions with local data
structures as input variables, e.g. in this line:

#+CALL: list2table(data=common-issues-list, order="rows") :results  table
 raw

where `common-issues-list` is a named org-mode list in the current file,
e.g.

#+NAME: common-issues-list
- some
- things
- here

In this particular case, I'd like to maintain a single data structure and
use it in a bunch of places (it's a document I re-use in several courses,
all of which are published online with different metadata stored in
headings). Is there a syntax for referring to #+NAME'd elements in remote
files?

Sorry if I'm missing something that is explained in the docs -- I am not
finding this in what I think of as the obvious places.

thanks as always,

Mat

Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers

[Matt Price]

On Tue, Sep 21, 2021 at 9:36 AM Timothy  wrote:

> I’m suspect it too short notice for such a large change to make its way
> into Org
> 9.5, but Bastien’s release email is certainly a good prompt to bump this.
>
> Bastien  writes:
>
> > Thank you *very much* for this work and sorry for the slow reply.
> >
> > I urge everyone to test this change, as I’d like to include it in
> > Org 9.5 if it’s ready.
> >
> > I will test this myself this week and report.
>
> All the best,
> Timothy
>

Is this code in main now, and do I have to do anything special to test it
out?

Re: New HTML exporter (was a feature request)

[Matt Price]

On Mon., Oct. 4, 2021, 2:21 a.m. Timothy,  wrote:

> Hi Tim,
>
> > Probably what is needed is a new HTML exporter which is capable of doing
> > what slimhtml does as the default, but includes the additional
> > functionality already in the exporter which many people are using.
>
> I think there may be some promise in the idea of rewriting (large parts
> of) the
> current HTML exporter. The way that the web has changed over the last
> decade I
> think we can now make some assumptions that can simplify things — for
> example
> I think we could safely go HTML5-only
>

> It seems to me that if the html exporter is being rewritten, we might want
to think about ways to make it easier to export some elements as custom
components. As their use becomes more and more common, I would reckon this
will be a desirable feature.


>
>

Re: add a heading w/ #+print_bibliography to parse tree on export

[Matt Price]

thanks John,

On Fri, Jul 30, 2021 at 9:01 PM John Kitchin 
wrote:

> I think the way to do this is to use a function in
> org-export-before-processing-hook to add it if you don't find it.
>
> you can still use org-ml to generate the string that gets inserted. I
> guess you would just go to the end of the buffer and insert it.
>
>
this is what I came up with. I don't know if it might be useful to others,
or if maybe a modified version might be worth putthing into some kind of
package.

(use-package org-cite
  :config
  (defcustom org-export-ensure-biblio
t
"Whether or not to create bibliography slide when missing from document"
:group 'org-export
:type 'string)

  (defcustom org-export-biblio-heading-id
"bibliography"
"custom id for bibliography slide heading in org-re-reveal"
:group 'org-export
:type 'string
:package-version '(org-re-reveal . 1.5))

  (defcustom org-export-biblio-heading-text
"Sources"
"Default Title to display above bibliography in org-re-reveal"
:group 'org-export
:type 'string
)

  (defun mwp-add-biblio-export-hook ( backend)
"If there's a citation object, search for a bibliography keyword;
if that's missing, create a new headline with the keyword. Intended for use
with
org-export-before-parsing-hook"
(save-excursion
  (when (and org-export-ensure-biblio
 (not (re-search-forward "^[ ]?+#\\+print_bibliography:"
nil t))
 (progn
   (goto-char (point-min))
   (re-search-forward org-element-citation-key-re
(point-max) t)))
(let* ((title (or org-export-biblio-heading-text "Sources"))
   (id (or org-export-biblio-heading-id "bibliography")))
  (goto-char (point-max))
  (insert
   (org-ml-to-trimmed-string
(org-ml-build-headline
 :title `(,title)
 (org-ml-build-section
  (org-ml-build-property-drawer
   (org-ml-build-node-property "CUSTOM_ID" id))
  (org-ml-build-keyword "print_bibliography" "")

  (add-hook org-export-before-parsing-hook 'mwp-add-biblio-export-hook))

Matt


> John
>
> ---
> Professor John Kitchin (he/him/his)
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
>
>
>
> On Fri, Jul 30, 2021 at 8:18 PM Matt Price  wrote:
>
>> Goal:
>> conditionally add a headline with content to parse tree using either
>> org-export-before-parsing-hook or org-export-filter-parse-tree-functions
>>
>> Explanation:
>> I use org-re-reveal to make lecture slides from subtrees. Often I forget
>> to add a heading with #+print_bibliography: and as a result org-cite does
>> not generate the bibliography, and my citations are not wrapped in links.
>> I'd like to search *either* the parse tree  *or* the temporary export
>> buffer for the #+print_bibliography keyword, and if it's not found, add a
>> headline to the end of the subtree/parse-tree with the appropriate
>> information.
>>
>> I can easily generate the proper lisp object with the org-ml package:
>>
>> (org-ml-build-headline :title '("Sources")
>>(org-ml-build-section
>> (org-ml-build-property-drawer
>>  (org-ml-build-node-property "CUSTOM_ID"
>> "bibliography"))
>> (org-ml-build-keyword "print_bibliography" "" )
>> ))
>>
>> And if I want I can turn that into a string by wrapping it in
>> (org-ml-to-trimmed-string ...)
>>
>> However, I'm not sure how best to add it to the exported data.  I believe
>> I can add a function to either org-export-before-parsing-hook or
>> org-export-filter-parse-tree-functions, but beyond that I'm a bit lost.
>> It's not immediately obvious how to find the last headline and append
>> another headline after that headline's section. Looking at the nesting
>> confuses me somewhat and I'm afraid I'll break the data structure.
>>
>> Has anyone done something like this before and do you have ideas?
>>
>> Thank you!
>>
>

add a heading w/ #+print_bibliography to parse tree on export

[Matt Price]

Goal:
conditionally add a headline with content to parse tree using either
org-export-before-parsing-hook or org-export-filter-parse-tree-functions

Explanation:
I use org-re-reveal to make lecture slides from subtrees. Often I forget to
add a heading with #+print_bibliography: and as a result org-cite does not
generate the bibliography, and my citations are not wrapped in links. I'd
like to search *either* the parse tree  *or* the temporary export buffer
for the #+print_bibliography keyword, and if it's not found, add a headline
to the end of the subtree/parse-tree with the appropriate information.

I can easily generate the proper lisp object with the org-ml package:

(org-ml-build-headline :title '("Sources")
   (org-ml-build-section
(org-ml-build-property-drawer
 (org-ml-build-node-property "CUSTOM_ID"
"bibliography"))
(org-ml-build-keyword "print_bibliography" "" )
))

And if I want I can turn that into a string by wrapping it in
(org-ml-to-trimmed-string ...)

However, I'm not sure how best to add it to the exported data.  I believe I
can add a function to either org-export-before-parsing-hook or
org-export-filter-parse-tree-functions, but beyond that I'm a bit lost.
It's not immediately obvious how to find the last headline and append
another headline after that headline's section. Looking at the nesting
confuses me somewhat and I'm afraid I'll break the data structure.

Has anyone done something like this before and do you have ideas?

Thank you!

Re: [BUG] Citations in tables are not exported

[Matt Price]

On Thu., Jul. 29, 2021, 8:30 a.m. Nicolas Goaziou, 
wrote:

> Hello,
>
> Matt Price  writes:
>
> > No doubt this is a bit too broad, though I am not entirely clear on why
> > there are any restrictions at all on inserting cites.
>
> There are locations which can break the structure of the document.
> Obviously, we can let users shoot themselves in the foot, but, OTOH,
> a decent interface should prevent the most obvious issues.
>
> WDYT?
>

What I think is, I am far too ignorant to have an opinion. I defer entirely
to yours. Thanks so much for the caption fix, at present it's one of my
main uses for citations so the convenience win is substantial.


> Regards,
> --
> Nicolas Goaziou
>

Re: [BUG] Citations in tables are not exported

[Matt Price]

re: captions, a quick hack is to change line 1373 of oc.el to

((memq type '(nil paragraph keyword)))

This makes my life quite a bit easier.

No doubt this is a bit too broad, though I am not entirely clear on why
there are any restrictions at all on inserting cites.

On Sun, Jul 18, 2021 at 7:29 PM John Kitchin 
wrote:

> I also see this behavior. I think this should be considered a bug, it is
> pretty common to see citations in a table cell.
>
> Additionally, I cannot insert a citation in a caption that is above a
> table (or a figure), I get
> "user-error: Cannot insert a citation here". That should be considered a
> bug in org-cite--allowed-p  I think. It fails because in the caption the
> context is a table, and it has :post-affiliated item which causes
> org-cite--allowed-p  to return nil.
>
> If I put a citation in the caption anyway, it does seem to get exported
> correctly.
>
> I am not sure why the cites don't export inside the table cell though.
>
> Thanks,
>
> Timothy  writes:
>
> > Hi,
> >
> > I've just started playing with citations, but it seems that I've
> > stumbled across a bug where if the citation is in a table cell it will
> > be left "raw". See the following example
> >
> > #+begin_org
> > ,#+begin_src bibtex :tangle activedoc.bib
> > @article{SchulteActiveOrg,
> >  author={Schulte, Eric and Davison, Dan},
> >  journal={Computing in Science Engineering},
> >  title={Active Documents with Org-Mode},
> >  year={2011},
> >  volume={13},
> >  number={3},
> >  pages={66-73},
> >  doi={10.1109/MCSE.2011.41}}
> > ,#+end_src
> >
> > ,#+bibliography: activedoc.bib
> > ,#+print_bibliography:
> >
> > [cite/a:@SchulteActiveOrg]
> >
> > | [cite/a:@SchulteActiveOrg] |
> > #+end_org
> >
> > When exporting to ASCII, I see this:
> >
> > #+begin_example
> > Schulte, Eric, and Dan Davison. 2011. “Active Documents with Org-Mode.”
> > /Computing in Science Engineering/ 13 (3):66–73.
> >
> > (Schulte and Davison 2011)
> >
> > 
> >  [cite/a:@SchulteActiveOrg]
> > 
> > #+end_example
> >
> > I hope that's enough to go off.
>
>
> --
> Professor John Kitchin
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
> Pronouns: he/him/his
>
>

Re: Headings and Headlines

[Matt Price]

I don't really have a strong preference for either but I would love to
remove the cognitive load of wondering whether the name is heading or
header!

On Sat., Jul. 24, 2021, 12:04 a.m. Tom Gillespie,  wrote:

> I enthusiastically support changing the documentation to use heading.
> I use heading in my formal grammar because I have found there are more
> ways that it can be modified and remain grammatically correct when
> used in english sentences. The internal implementation in elisp still
> refers to headlines, but changing the docs would be a good first step.
> Best!
> Tom
>
>

Re: [org-cite] issues with org-cite-make-insert-processor select-style

[Matt Price]

Bruce, are you loading this code with use-package? If so, and if I'm
reading this right, you can perhaps add the missing functions to the

:commands

directive for org-mode? IIUC that should ensure that they are available to
your package, as long as you have an
:after (org oc)
line in the package's use-package directive.

On Thu, Jul 22, 2021 at 6:31 AM Bruce D'Arcus  wrote:

> The problem was load order I guess; putting this of the file fixes it.
>
> So when org-citemake-insert-processor is first loaded, it looks for
> the two functions, which haven't been loaded yet.
>
> I still think a) the error message could say that (that the functions
> aren't found or some such), and b) that it shouldn't break starting
> Emacs.
>
> On Thu, Jul 22, 2021 at 4:27 AM Bruce D'Arcus  wrote:
>
> > If I comment out those lines and use the oc-basic style selector
> > instead to start emacs, and from there reactivate this function and
> > compile and reload the code from the buffer, THEN it works without
> > error.
>
>

Re: setting export options in headline properties

[Matt Price]

arg.  thx, Chuck.

On Wed, Jul 21, 2021 at 4:25 PM Berry, Charles 
wrote:

> Matt,
>
> Check (info "(org) Export Settings")
>
> and especially, the para near bottom:
>
> When exporting sub-trees, special node properties can override the
> above keywords.  These properties have an ‘EXPORT_’ prefix.  For
> example, ‘DATE’ becomes, ‘EXPORT_DATE’ when used for a specific
> sub-tree.  Except for ‘SETUPFILE’, all other keywords listed above have
> an ‘EXPORT_’ equivalent.
>
> HTH,
>
> Chuck
>
> > On Jul 21, 2021, at 11:52 AM, Matt Price  wrote:
> >
> > Hi,
> >
> > I'm not sure if I'm reading the documentation properly, but my
> understnading is that I ought to be able to set export options as subtree
> properties, and that if I do so, they should be picked up by export engines
> when exporting subtrees.  However, that doesn't see to be happening for me,
> and from what I can tell, `org-export-get-environment` is not overriding
> global values when passed the `subtreep` parameter.
> >
> > I tried the following with emacs -Q , which seems to confirm my issue.
> Is this the expected behaviour, and if so, is there some other way for me
> to set subtree-level export options?
> >
> > --
> > ** subtree
> > :PROPERTIES:
> > :SUBTITLE: testing
> > :REVEAL_TITLE_SLIDE: nil
> > :HTML_CONTAINER: section
> > :END:
> >
> >
> > #+begin_src emacs-lisp
> > (let ((info (org-export-get-environment 'html t)))
> >   (string-join `(,(plist-get info :subtitle) ,(plist-get info
> :html-container)) "\n")
> >   )
> > #+end_src
> >
> > #+RESULTS:
> > :
> > : div
> > --
> >
> > thanks as always!
>
>

Re: org-cite: how to include a cite with no key?

[Matt Price]

On Tue, Jul 20, 2021 at 3:32 AM Emmanuel Charpentier <
emm.charpent...@free.fr> wrote:

> Le mardi 20 juillet 2021 à 02:55 -0400, Matt Price a écrit :
>
> Certainly citation of personal communications is common in the fields of
> history and philosophy of science, where it represents an effort to
> responsibly represent the source of ideas owed to other persons.
>
>
> I don't dispute that.
>
>   It's not really a question of whether you personally would do it; it's a
> question of whether org will support a practice that exists in some
> real-world contexts.
>
>
> What I dispute is that such "non-citations" are dangling pointers, better
> replaced by incises or footnotes ... or proper citations.
>

just wanted to say sorry if I was gouchy there.  Shouldn't write emails so
late at night!

>
> --
> Emmanuel Charpentier
>
>

setting export options in headline properties

[Matt Price]

Hi,

I'm not sure if I'm reading the documentation properly, but my
understnading is that I ought to be able to set export options as subtree
properties, and that if I do so, they should be picked up by export engines
when exporting subtrees.  However, that doesn't see to be happening for me,
and from what I can tell, `org-export-get-environment` is not overriding
global values when passed the `subtreep` parameter.

I tried the following with emacs -Q , which seems to confirm my issue. Is
this the expected behaviour, and if so, is there some other way for me to
set subtree-level export options?

--
** subtree
:PROPERTIES:
:SUBTITLE: testing
:REVEAL_TITLE_SLIDE: nil
:HTML_CONTAINER: section
:END:


#+begin_src emacs-lisp
(let ((info (org-export-get-environment 'html t)))
  (string-join `(,(plist-get info :subtitle) ,(plist-get info
:html-container)) "\n")
  )
#+end_src

#+RESULTS:
:
: div
--

thanks as always!

Re: convert subtree or nested list to table

[Matt Price]

On Tue, Jul 20, 2021 at 12:06 PM Nick Dokos  wrote:

> Matt Price  writes:
>
> > Christian et al,
> >
> > I seem to have broken something while fiddling around and I can't quite
> make out what.  Would someone be willing to check for me whether this does
> or does not produce the desired full table? Right now I am again getting a
> truncated result and I'm not
> > fully sure what I might be oding wrong.  Here is what I have:
> >
> > #+NAME: rubric-one
> > - Grade
> >   a. A
> >   b. B
> >   c. C
> >   d. D
> >   e. F
> > - Style
> >   a. Excellent
> >   b. Good
> >   c. Adequate
> >   d. Lousy
> >   e. Failing
> >
> These seem to have zero-width spaces and maybe that breaks the structure.
> To make sure that the structure is recognized properly,
> maybe try a very simple code block:
>
>
> --8<---cut here---start->8---
> #+begin_src elisp :var data=rubric-one :results drawer
> data
> #+end_src
>
> #+RESULTS:
> :results:
> ((Grade (ordered (A) (B) (C) (D) (F))) (Style (ordered (Excellent) (Good)
> (Adequate) (Lousy) (Failing
> :end:
> --8<---cut here---end--->8---
>
> If that does not work, then something is fishy with `rubric-one', so start
> there.
>

Yes, I think that using alphabetical lists broke the data structure because
I didn't have ~org-list-allow-alphabetical~ set to ~t~ in my init file.
User error! sorry. and thank you!!

Re: convert subtree or nested list to table

[Matt Price]

Christian et al,

I seem to have broken something while fiddling around and I can't quite
make out what.  Would someone be willing to check for me whether this does
or does not produce the desired full table? Right now I am again getting a
truncated result and I'm not fully sure what I might be oding wrong.  Here
is what I have:

#+NAME: rubric-one
- Grade
  a. A
  b. B
  c. C
  d. D
  e. F
- Style
  a. Excellent
  b. Good
  c. Adequate
  d. Lousy
  e. Failing

#+NAME: list2table
#+BEGIN_SRC emacs-lisp :var order="columns" data=rubric-one :results table
value raw replace
 (let (longest)
(setq data (map 'list '-flatten data))
(setq data (map 'list (lambda (x) (seq-difference x '(unordered
ordered))) data))
;; Pad out lists to equal length
(setq longest (seq-max (map 'list 'length data)))
(setq data
  (map 'list
   (lambda (l)
 (append l (make-list (- longest (length l)) "")))
 data))
;; Order by columns or rows
(if (string= order "columns")
(apply #'mapcar* #'list data) ; transpose
  data))
#+END_SRC

#+RESULTS: list2table
| Grade |



thanks as always for the help!

Re: org-cite: how to include a cite with no key?

[Matt Price]

Certainly citation of personal communications is common in the fields of
history and philosophy of science, where it represents an effort to
responsibly represent the source of ideas owed to other persons.  It's not
really a question of whether you personally would do it; it's a question of
whether org will support a practice that exists in some real-world
contexts.

On Mon., Jul. 19, 2021, 8:48 p.m. John Kitchin, 
wrote:

> On Mon, Jul 19, 2021 at 7:28 PM Emmanuel Charpentier <
> emm.charpent...@free.fr> wrote:
>
>> > In natbib there is \citetext{priv.\ comm.} which is used to add a
>> > textual citation to the bibliography that doesn't have a key
>> > associated with it.
>>
>> Hmmm... why should you bother to reference a personal communication ?
>> Such private communications may be mentionned in the text (possibly by
>> a footnote) but can't be properly referenced (since there is nothing to
>> refer to). If you feel that this communication must be referred to, you
>> should give it some (written) support and (properly) reference this
>> support.
>>
>
> Who is to say why someone would bother. It is a command on page two of
> http://tug.ctan.org/macros/latex/contrib/natbib/natnotes.pdf that one can
> use.
>
> It is also possible to use  \nocite{*} as a cite, which includes all
> references from a bibliography, and yet contains no key. Even funnier in a
> way is \nocite{key} which just adds entries to the bibliography, but does
> not cite them in the body of a document.
>
> Footnotes are not always allowed in publications, and for various reasons
> not worth defending, in proposals one might want to put this in the
> references because of space limitations.
>
> I count at least 10 examples of such personal communications in the
> references in my library of ~1800 pdfs, so they aren't very common, but
> certainly they exist in the wild. Whether people should do it or not, they
> do.
>
>
>>
>> ISTR that at least CSL and BibLaTeX have types appropriates for a
>> manuscript or a letter. You may also consider your own notes as
>> documents and reference them (properly).
>>
>> > I don't see a way to get something like that in org-cite, since it
>> > seems that a key is always required.
>>
>> Indeed : the key is, in relational algebra terms, the primary key of
>> the bibliographic relation...
>>
>
> I think of it more like a lambda function, but for a cite reference, where
> you
> define what you want inline. It is pretty common in scientific papers
> and proposals to see that.
>
> It may not make sense to make an @misc bibtex entry for that purpose,
> since it is a one time citation for that document, and is like a lambda
> reference.
>
>
>> > This isn't currently recognized as a cite, but something like this
>> > seems like a reasonable solution to me.
>>
>> > [cite/text:@ private communication]
>>
>> Such special casing is probably a bugs' nest... err.. hive. And
>> pointless, as explained /supra/.
>
>
>> HTH,
>>
>> --
>> Emmanuel Charpentier
>>
>> > John
>>
>>

Re: oc-csl/citeproc: suffixes in cite links

[Matt Price]

Ah, that's so immensely helpful. Thank you, this will be very good to know.

On Fri, Jul 16, 2021 at 5:21 PM András Simonyi 
wrote:

> Dear Matt,
>
> yes, oc-csl has to extract the locator information from the suffix
> because CSL processors like citeproc-el work with structured locator
> data. To help this extraction, ocl-csl (similarly to citeproc-org and
> I think pandoc) defines a list of locator expressions to be used in
> the suffix (see the commentary of oc-csl or the citeproc-org README),
> for chapter one can currently use "chap.",  "chaps." or "chapter" --
> hopefully replacing the citation with
> [cite:@GentilcoreTastetomatoItaly2009 chap. 4] will fix the rendering,
>
> best regards,
> András
>
> On Fri, 16 Jul 2021 at 22:40, Matt Price  wrote:
> >
> > (cc:ing Andras in case this issue maybe comes from citeproc)
> >
> > I'm having some trouble with suffixes in cite: links when the oc-csl
> exporter is enabled, e.g. with something like this:
> >
> > #+cite_export: csl "/home/matt/src/styles/apa.csl"
> >
> > this cite:
> > [cite:@GentilcoreTastetomatoItaly2009 ch4]
> >
> > is rendered as:
> >
> > (ch Gentilcore, 2009, p. 4)
> >
> >
> > on export to HTML (in a document with no print_bibliography directive).
> >
> > BY contrast, ob-basic gives:
> >
> > (Gentilcore, David, 2009 ch4)
> >
> > Is oc-csl doing some extra work to process suffix, or is this likely an
> issue with citeproc, or alternatively am I just not really understanding
> what ought to be happening?
> >
> >
> > Also, are other people seeing this same issue? I don't have a testing
> setup that would allow me to easily run emacs -Q (given how much has to be
> imported) so ... well, so, my apologies for not having done that.
> >
> >
> > THanks,
> >
> > Matt
> >
> >
>

oc-csl/citeproc: suffixes in cite links

[Matt Price]

(cc:ing Andras in case this issue maybe comes from citeproc)

I'm having some trouble with suffixes in cite: links when the oc-csl
exporter is enabled, e.g. with something like this:

#+cite_export: csl "/home/matt/src/styles/apa.csl"

this cite:
[cite:@GentilcoreTastetomatoItaly2009 ch4]

is rendered as:

(ch Gentilcore, 2009, p. 4)


on export to HTML (in a document with no print_bibliography directive).

BY contrast, ob-basic gives:

(Gentilcore, David, 2009 ch4)

Is oc-csl doing some extra work to process suffix, or is this likely an
issue with citeproc, or alternatively am I just not really understanding
what ought to be happening?


Also, are other people seeing this same issue? I don't have a testing setup
that would allow me to easily run emacs -Q (given how much has to be
imported) so ... well, so, my apologies for not having done that.


THanks,

Matt

conditionally add heading to tree on export

[Matt Price]

Hi everyone,

Now that org-cite is working (!!!), I am incrementally updating all my
lecture slides to use proper citations.  This is mostly quite simple, but I
find that in the rush before class I am often forgetting to add a new slide
with *both* the

#+print_bibliography:

line that will generate a bibliography, *and* a :CUSTOM_ID: property set to
~bibliography~, which is required for working links around citation
objects.

I'd like to check to see whether a slide with :CUSTOM_ID of ~bibliography~
exists, and if not, insert a heading at the end of the exported text
(either the file or a subtree), something like

** Sources
:PROPERTIES:
:CUSTOM_ID: bibliography
:END:
#+print_bibliography:


What is the appropriate way for me to do this as an end user who doesn't
want to write his own derived backend? Is the appropriate process to write
a function that inspects and modifies the parse tree, and then add that
function to ~org-export-filter-body-functions~?

In fact, I wonder if I want to do this for all HTML export that contains
citation objects. What do other people think about that idea?

Thanks as always,
Matt

Re: Citations merged!

[Matt Price]

Congratulations What time is the parade?

On Fri, Jul 9, 2021 at 8:03 AM Julian M. Burgos 
wrote:

> Amazing!  Thank you to everyone that contributed.  I am looking forward to
> start playing with this. :)
>
> Nicolas Goaziou writes:
>
> > Hello,
> >
> > It took years, but citations are now full part of Org syntax.
> >
> > Thanks to everyone involved over the time!
> >
> > Now, it needs to be documented, but that will come a bit later.
> >
> > Regards,
>
>
> --
> Julian Mariano Burgos, PhD
> Hafrannsóknastofnun, rannsókna- og ráðgjafarstofnun hafs og vatna/
> Marine and Freshwater Research Institute
> Botnsjávarsviðs / Demersal Division
>   Fornubúðir 5, IS-220 Hafnarfjörður, Iceland
> www.hafogvatn.is
> Sími/Telephone : +354-5752037
> Netfang/Email: julian.bur...@hafogvatn.is
>
>

Re: [wip-cite-new] Merging tomorrow?

[Matt Price]

I cannot believe this is finally happening, and I am so stoked and excited
about it. I've been using ~wip-cite-new~ in my classes this week, and these
new tools are absolutely transformative.

Thank you so much for the immense amount of work you put into this.  And
also to Bruce for championing it, and Andras and Denis and others for their
contributions. Really, I feel like there should be a parade.

Matt

On Wed, Jul 7, 2021 at 10:23 PM Thomas S. Dye  wrote:

> Aloha Nicolas,
>
> Good news! I'm looking forward to using this facility.
>
> Thanks to all the contributors.
>
> All the best,
> Tom
>
> Nicolas Goaziou  writes:
>
> > Hello,
> >
> > I think the "wip-cite-new" branch is in good shape now. As
> > a consequence, I'd like to merge it tomorrow.
> >
> > It is documented, but the documentation is scattered across the
> > various
> > "oc" libraries, and some threads in the mailing list. I'll do a
> > summary
> > here, from a user point of view.
> >
> > --8<---cut
> > here---start->8---
> > Basically, in order to use it, you need to first set-up a
> > bibliography,
> > using one or more "bibliography" keywords.  on such a
> > keyword
> > visits the related file. Out of the box, Org supports JSON-CSL
> > and
> > BibTeX (or biblatex) bibliographies.
> >
> > Then, citations can be inserted with the following syntax:
> >
> >   [cite/style:common prefix ;prefix @key suffix; ... ; common
> >   suffix]
> >
> > Spaces are meaningful except those after the initial colon and
> > before
> > the closing bracket.
> >
> > Every part of the syntax is optional, except the brackets,
> > "cite" and
> > the colon. Also the citation must contain at least a key. So its
> > minimal
> > form is:
> >
> >   [cite:@key]
> >
> > The "style" part is detailed below, in the part related to
> > export.
> >
> > Org can insert or edit citations with  (and delete
> > them with
> > ), follow them with , fontify them, and
> > export
> > them. These four actions (insert, follow, activate, and export)
> > are
> > called capabilities.  Libraries responsible for these
> > capabilities are
> > called citation processors.
> >
> > You can select one citation processor for each capability,
> > independently
> > on the others, through the following variables:
> >
> > - org-cite-activate-processor
> > - org-cite-export-processors
> > - org-cite-follow-processor
> > - org-cite-insert-processor
> >
> > Out of the box, Org provides the "basic" (in "oc-basic.el")
> > processor
> > for all of these tasks. It also boasts processors dedicated for
> > export:
> > "csl", "natbib" and "biblatex".
> >
> > During export, output for citations is controlled by their
> > style, which
> > is an Org label that the export processor may recognize and
> > associate to
> > a specific display, or fall-back to a default style (called
> > "nil"). For
> > example, most processors support "noauthor" and "text" styles.
> >
> > Some styles can accept a variant, with the syntax
> > "style/variant".
> > Again, it's up to the processor to associate it to a specific
> > display.
> > Common variants include "bare", "caps" or "full".  They also
> > accept
> > short-hands, like "b", "c" and "f".  Please refer to the export
> > processors' libraries ("oc-basic.el", "oc-csl.el", …) for more
> > information.
> >
> > It is possible to define a default style for a whole document
> > (with
> > "cite_export"), or for all documents (with
> > `org-cite-export-processors').
> >
> > References are displayed with the "print_bibliography" keyword.
> > It is
> > possible to add parameters to its value, as some export
> > processors could
> > make use of them.
> > --8<---cut
> > here---end--->8---
> >
> > Please let me know if there are any objections to the merge.
> >
> > Regards,
>
>
> --
> Thomas S. Dye
> https://tsdye.online/tsdye
>
>

Re: convert subtree or nested list to table

[Matt Price]

It's totally interesting. It's not quite designed for what I'm looking to
do and is perhaps a bit overpowered for my limited needs but I'm going to
test whe nI have a little mroe time!

On Tue, Jul 6, 2021 at 7:51 AM Uwe Brauer  wrote:

> >>> "MP" == Matt Price  writes:
>
> > I have to write a number of text-heavy documents which need to be
> delivered
> > as tables with wrapped paragraphs in most cells. Working directly in
> table
> > format is pretty arduous and uncomfortable.  Has anyone ever written a
> > function to accept a list or subtree as input and process it into a
> table?
>
> > If anyone has done something similar, I'd love some tips!
>
> There is org-listcruncher, which I started to use some days ago. For my
> needs it works quite nicely.
>

Re: convert subtree or nested list to table

[Matt Price]

thank you Juan Mnauel. I'm testing it out, but I do wonder if I would
really rather work with lists and some CSS!

On Tue, Jul 6, 2021 at 8:56 AM Juan Manuel Macías 
wrote:

> Hi Matt, sorry for the slow reply...
>
> Matt Price writes:
>
> > I'd love to try that, thanks. I think it would be really helpful.
> > Much appreciated!
>
> Some previous caveats:
>
> - *The code is very raw*. I wrote it almost as a "proof of concept" for
>   my personal use, therefore it is quite improvable. In any case, it
>   works (but i don't know if it will help you get what you want...).
>
> - The attached example is optimized to export to LaTeX. Use the tabularx
>   environment, which is ideal for tables with a lot of textual content.
>
> - As for the code, there are two basic functions:
>   `my-org/edit-table-cell' and
>   `my-org/kill-edit-table-cell-buffer-and-save'. To edit an empty cell,
>   you have to write something in that cell first.
>
> - The basic idea is that within each cell, the content is a single line
>   (unfilled). In the edit buffer, the content is filled. There are two
>   macros to indicate a line break and a paragraph end: {{{nl}}} and
>   {{{par}}}. In the edit buffer you can put line breaks and white lines,
>   but all of that is lost inside the cell once saved (all is a single
>   line), so those macros are needed to indicate line or paragraph breaks
>   (in LaTeX).
>
> Best regards,
>
> Juan Manuel
>
>

Re: convert subtree or nested list to table

[Matt Price]

I think this is exactly what I want (with just a little moreprocessing).
Thank you so much for the idea!

I'm having a little bit of trouble getting the same output as you though,
and I'm wondering if there might be a setting that I need to change.

Here is what I tried, and the result. Do you have an idea of what is going
wrong here?

Thank you!



#+NAME:essay-rubric
- Category
  - A
  - B
  - C
  - D
  - F
- Writing
  - great
  - good
  - ok
  - lousy
  - awful

#+begin_src emacs-lisp :var contents=essay-rubric :results table
contents
#+end_src

#+RESULTS:
#+begin_src emacs-lisp
| (("Category" |
#+end_src
-
On Wed, Jul 7, 2021 at 6:29 AM tbanelwebmin  wrote:

> Hi Matt
>
> Le 05/07/2021 à 21:44, Matt Price a écrit :
> > I have to write a number of text-heavy documents which need to be
> > delivered as tables with wrapped paragraphs in most cells. Working
> > directly in table format is pretty arduous and uncomfortable.  Has
> > anyone ever written a function to accept a list or subtree as input
> > and process it into a table?
> >
> > If anyone has done something similar, I'd love some tips!
>
> Maybe you could use builtin Babel
> Hereafter you have a starting point
> - Give a name to your input Org list
> - Process it with Emacs-Lisp (or whatever language you are comfortable
> with) to output it as a table
>
>
>  self contained Org Mode example _
>
> Example of a named list
> #+NAME: BBB
> - abc
>   + 123
>   + 456
> - def
>   + red
>   + blue
> - ghi
>   + big
>   + small
>
> Example of converting the named list into a table with Emacs-Lisp
> #+begin_src elisp :var bbb=BBB :results table
> bbb
> #+end_src
>
> #+RESULTS:
> | abc | (unordered (123) (456))   |
> | def | (unordered (red) (blue))  |
> | ghi | (unordered (big) (small)) |
> ___
>
>
>

Re: allow HTML block to escape from outline-text div? WAS: BUG? unable to surround subtrees with html tag

[Matt Price]

On Thu, Jul 1, 2021 at 12:18 PM Timothy  wrote:

>
> Matt Price  writes:
>
> >> I would like to be able to surround some portion of a subtree with a
> tag,
>
> > Sorry for the noise, I believe this is user error.   I misread the
> exported
> > source code, and firefox's developer tools added the closing tag for me,
> so
> > I kept not seeing my mistake. The error here was not realizing that
> section
> > contents get wrapped in their own div with class ~outline-text-N~, so my
> > ~~ resulted in malformed HTML.
> >
> > I don't know if there is a way to somehow slide my own html in between
> the
> > outine-text element and the outline-container element for a child
> subtree.
> > If someone knows a way to do this, I'd appreciate a pointer, but for now
> I
> > think I have to find another way to accomplish this.
>
> As it so happens, this is something which has come up for me too. I
> think in the future it could be worth adding a property that can be used
> to wrap a section in certain HTML tag(s).
>

I'm dealing with it again tonight, and I do find it a bit teeth-gnashing.
In this case what I'm looking to do is to hide a whole section when the
exported hTML page loads.  But right now the subtee children of the
headline remain visible even when the text is hidden.

Whoa! I just realized I can rewrite this myself, since it's a derived
bakend.  I guess the export happens from the leafs upwards, so that the
contents are already rendered by the time theadline is being processed.  So
in my headline exporter, just had to wrap the contents in an additional
div.  It works fine, I feel great about it!

>
> For now though... the best solution would be what Richard suggests.
>
> --
> Timothy
>

Re: convert subtree or nested list to table

[Matt Price]

Thanks for the thoughts, Tim.

My preference is generally to work in HTML, and in fact if I had a decent
platform to work on I could just use a container class and grid or flex
layouts, but the learning management system at my institution strips out
most styling information when HTML is uploaded, so I will probably need a
real table.

I odn't think I could really handle doing this in latex. I am a terrible
latex user!


On Mon, Jul 5, 2021 at 9:27 PM Tim Cross  wrote:

>
> Matt Price  writes:
>
> > I have to write a number of text-heavy documents which need to be
> delivered as tables with wrapped paragraphs in most cells. Working directly
> in
> > table format is pretty arduous and uncomfortable.  Has anyone ever
> written a function to accept a list or subtree as input and process it into
> a table?
> >
> > If anyone has done something similar, I'd love some tips!
>
> No, have not done that. What formats do you need to export the documents
> in?
>
> I ask because if all you need to produce is Latex derived documents
> (i.e. PDF, ps etc) and you do plan to write a function yourself to do
> this, I would work backwards. Latex tables are not very good for your
> use case, but Latex can support what you want to do. Most of the Latex
> table packages are not terribly good at formatting tables containing
> paragraphs of data. They will typically require lots of hand tweaking to
> get the formatting looking right. Getting the right latex package to
> support what you need to do will make the function you will need to
> write a lot easier. Therefore, I would start with a search of the latex
> package archives to find the right package and then write an elisp
> function that generates a latex block which formats your subtree using
> that package. You probably want something which will format a table with
> minipage or similar environments in the cells.
>
> --
> Tim Cross
>
>

add #+ATTR_SOMETHING to headline?

[Matt Price]

Does anyone know if attributes get attached to headline elements during the
export process? I am writing a derived backend in which I would like to
have a line like

(let (add-toggle (org-export-read-attribute :attr_canvas_html headline
:toggle))
)

But as var as i can tell, the call to org-rexport-read-attribute always
evaluates to nil in a file like this:

#+ATTR_CANVAS_HTML: :toggle t
* My TOggle Headline

Am I missing something obvious, or is htis not possible? I notice that none
of the org-xxx-headline functions in ox-xxx.el seem to check for
attributes, so maybe this is by design?

THanks,

Matt

convert subtree or nested list to table

[Matt Price]

I have to write a number of text-heavy documents which need to be delivered
as tables with wrapped paragraphs in most cells. Working directly in table
format is pretty arduous and uncomfortable.  Has anyone ever written a
function to accept a list or subtree as input and process it into a table?

If anyone has done something similar, I'd love some tips!

Re: modify citation links in a derived HTML backend

[Matt Price]

On Sun, Jul 4, 2021 at 8:56 AM Jens Lechtenboerger <
lech...@wi.uni-muenster.de> wrote:

> On 2021-07-03, Matt Price wrote:
>
> > I've added some comments in the issue you linked to, but in the meantime
> > I've also come up with what seems to be at least a semi-viable hack for
> > adding native CSL citation support to org-re-reveal. It involves creating
> > two new variables and then let-setting `citeproc-fmt--formatters-alist`
> in
> > `org-re-reveal-export-to-html`. Something similar could presumably be
> done
> > in other derived backends.
> >
> > I find it quite hackish and I don't know whether perhaps some more
> general
> > solution could be found, but in any case here is the code, which I have
> > inserted into org-re-reveal.el locally:
>
> Thank you for sharing your code, Matt!
>
> What is the general state of this new citation handling with respect
> to export backends?  Did you create the settings for HTML from
> scratch or did you take inspiration from HTML export functionality?
> I guess that basic support should go into ox-html, while small
> modifications could than take care of specifics for reveal.js
> presentations...
>
> Best wishes
> Jens
>

Thanks Jens!

So, there is a rather complicated chain of dependencies and I'm not sure
how best to procee in the specific case of org-re-reveal.

Here is how I understand the current state of affairs (and I think Bruce,
Nicolas, and Andras are all likely to have corrections to my explanation):

- the new syntax and processors are available in the wip-cite-new branch of
hte org git repo, and will likely be merged into the master branch pretty
soon.
- at present, and likely for some time, the best citation processor for
html export is defined in `oc-csl.el` and relies on the citeproc-el
library.  My code will only have an effect if the org file contains lines
like
--
#+bibliography: food-studies.bib
#+cite_export: csl "/home/matt/src/styles/chicago-author-date-mod.csl"

- oc-csl.el relies on citeproc-el to actually process the citations.
- citeproc-el is a fairly complex package with many moving parts. It
supports processing of citations in html and latex backends, but not in any
others (I think this is right!)
- citation formatting is handled in the file citation-formatters. el (
https://github.com/andras-simonyi/citeproc-el/blob/master/citeproc-formatters.el),
which, importantly, uses lexical binding. That file defines:
  -  a *structure* citeproc-formatter (
https://github.com/andras-simonyi/citeproc-el/blob/0857973409e3ef2ef0238714f2ef7ff724230d1c/citeproc-formatters.el#L35
),
 - a *function* citeproc-formatter-fun-create (
https://github.com/andras-simonyi/citeproc-el/blob/0857973409e3ef2ef0238714f2ef7ff724230d1c/citeproc-formatters.el#L51
)
- a *constant* citeproc-fmt--html-alist (
https://github.com/andras-simonyi/citeproc-el/blob/0857973409e3ef2ef0238714f2ef7ff724230d1c/citeproc-formatters.el#L142
)
- and a *variable* citeproc-fmt--formatters-alist (
https://github.com/andras-simonyi/citeproc-el/blob/0857973409e3ef2ef0238714f2ef7ff724230d1c/citeproc-formatters.el#L232)
that creates a set of formatters using citeproc-formmater-fun-create,
bassing it the value of citeproc-fmt--html-alist as raw material for the
html formatter.

The latter is what we need to change for our export -- we want to change
just one of the lambda functions used by the formatter for the duration of
export, without updating the global value of citeproc-fmt--formatters-alist.

Because all these variables are lexically bound -- making the resultant
closure values pretty confusing for me to inspect -- and because the
process of creating a formatter is a bit complex, I was a little  confused
about how best to modify the code, and I'm still not quite sure how to go
about it.  It's pretty easy to do this using the dash library:

-
(let ((org-re-reveal-citeproc-fmt-alist
   (--map-when (eq  (car it) 'cited-item-no)
  `(cited-item-no . ,(lambda (x y) (concat "" x "")))
  citeproc-fmt--html-alist)))
 [do some exporting stuff])
---
or just

-
(defcustom org-re-reveal-citeproc-fmt-alist
 (--map-when (eq  (car it) 'cited-item-no)
  `(cited-item-no . ,(lambda (x y) (concat "" x "")))
  citeproc-fmt--html-alist)
:group org-re-reveal
:type alist)
--

But I'm not sure how best to do it without that.

In any case, the code I have provided here and in gitlab (
https://gitlab.com/oer/org-re-reveal/-/merge_requests/33) is quite hackish
and doubtless likely to error out in many circumstances. Probably I should,
in org-re-reveal-export-to-html,  at least test if the function
citeproc-formatter-fun-create is bound, so:

--
(let ((citeproc-fmt--formatters-alist
  (and (functionp 'citeproc-formatter-create)
   

Re: modify citation links in a derived HTML backend

[Matt Price]

Thanks Jens!

I've added some comments in the issue you linked to, but in the meantime
I've also come up with what seems to be at least a semi-viable hack for
adding native CSL citation support to org-re-reveal. It involves creating
two new variables and then let-setting `citeproc-fmt--formatters-alist` in
`org-re-reveal-export-to-html`. Something similar could presumably be done
in other derived backends.

I find it quite hackish and I don't know whether perhaps some more general
solution could be found, but in any case here is the code, which I have
inserted into org-re-reveal.el locally:

Something to note here is that `oc-csl.el` hard-codes the output format
options in org-cite-csl--output-formats. I wonder if it would be worth
having an extensible variable "org-cite-csl--output-format-alist" that
backends could modify if they have special needs. For instance, I am
thinking about website exporters like Hugo where authors might want to have
separate bibliography pages rather than endnotes, or add "title" attributes
to citations, or possibly even modal tooltips with formatted text.  It
probably doesn't make sense to build all of that in to citeproc.el, but
assing some additional information seems worthwhile.
-
(defvar org-re-reveal-citeproc-fmt-alist
  `((unformatted . citeproc-fmt--xml-escape)
(cited-item-no . ,(lambda (x y) (concat ""
   x "")))
(bib-item-no . ,(lambda (x y) (concat ""
 x)))
(font-style-italic . ,(lambda (x) (concat "" x "")))
(font-style-oblique . ,(lambda (x)
(concat "")))
(font-variant-small-caps . ,(lambda (x)
 (concat
  "" x "")))
(font-weight-bold . ,(lambda (x) (concat "" x "")))
(text-decoration-underline .
   ,(lambda (x)
 (concat
  "" x
"")))
(rendered-var-url . ,(lambda (x) (concat "" x
"")))
(rendered-var-doi . ,(lambda (x) (concat "" x
"")))
;;(rendered-var-title . ,(lambda (x) (concat "" x
"")))
(vertical-align-sub . ,(lambda (x) (concat "" x "")))
(vertical-align-sup . ,(lambda (x) (concat "" x "")))
(vertical-align-baseline . ,(lambda (x) (concat "" x "")))
(display-left-margin . ,(lambda (x) (concat "\n"
x "")))
(display-right-inline . ,(lambda (x) (concat ""
x "\n  ")))
(display-block . ,(lambda (x) (concat "\n\n"
 x "\n")))
(display-indent . ,(lambda (x) (concat "" x
"\n  ")

(defvar org-re-reveal-formatters-alist
  `((html . ,(citeproc-formatter-create
   :rt (citeproc-formatter-fun-create
org-re-reveal-citeproc-fmt-alist)
   :bib #'citeproc-fmt--html-bib-formatter

(defun org-re-reveal-export-to-html
( async subtreep visible-only body-only ext-plist backend)
  "Export current buffer to a reveal.js HTML file.
Optional ASYNC, SUBTREEP, VISIBLE-ONLY, BODY-ONLY, EXT-PLIST are passed
to `org-export-to-file'.
Optional BACKEND must be `re-reveal' or a backend derived from it."
  (interactive)
  (let* ((backend (or backend 're-reveal))
 (extension (concat "." org-html-extension))
 (client-ext (concat org-re-reveal-multiplex-client-ext extension))
 (file (org-export-output-file-name extension subtreep))
 (clientfile (org-export-output-file-name client-ext subtreep))
 (org-html-container-element "div")
 (citeproc-fmt--formatters-alist org-re-reveal-formatters-alist))

(setq org-re-reveal-client-multiplex nil)
(org-export-to-file backend file
  async subtreep visible-only body-only ext-plist)

;; Export the client HTML file if org-re-reveal-client-multiplex is set
true
;; by previous call to org-export-to-file
(if org-re-reveal-client-multiplex
(org-export-to-file backend clientfile
  async subtreep visible-only body-only ext-plist))
file))


On Sat, Jul 3, 2021 at 3:53 AM Jens Lechtenboerger <
lech...@wi.uni-muenster.de> wrote:

> On 2021-07-02, Matt Price wrote:
>
> > Hi,
> >
> > (cc:ing Jens L. in case this is relevant for his dev work on
> org-re-reveal).
>
> Hi Matt,
>
> just a quick reply: Yes, that is certainly relevant for me, but I do
> not have the time to investigate this at the moment.  Note that I
> use references with org-ref and org-re-reveal-ref (for which Bruce
> opened an issue for org-cite support [1]).  Currently, I configure
> org-ref-ref-html (although its doc string suggests otherwise).
>
> Best wishes
> Jens
>
> [1] https://gitlab.com/oer/org-re-reveal-ref/-/issues/2
>

modify citation links in a derived HTML backend

[Matt Price]

Hi,

(cc:ing Jens L. in case this is relevant for his dev work on org-re-reveal).

I'm experimenting with the new citation syntax in slideshows generated with
org-re-reveal.  Mostly it works fine, but cite-links don't function
properly in the slideshow because in reveal, internal links only work when
they refer to a slide (which will generally be generated from a headline in
org-re-reveal export). Otherwise, the framework will reject the location
change.  At least, that's how I read the code here:
https://github.com/hakimel/reveal.js/blob/ade234576e8ddd683cf16d0d8bb0236f37cf1a99/js/controllers/location.js#L32.


Ideally, there would be some way to override this behaviour, but at present
I don't see how to do that from within Reveal.

I also know it is possible to just eliminate the links using
org-cite-csl-no-citelink-backends, but that is a significant loss of
function.

Instead, I would like to modify the links that surround the individual
citations. At present, the generated code will look something like this:

Zubaida, 2009

and I'd like to change it to something like this:

Zubaida, 2009

>From what I can tell so far, the current behaviour of oc-csl is dictated by
a lambda defined in the citeproc package, and set via defconst in an alist
`citeproc-fmt--html-alist` defined in citeproc-formatters.el.

My question: what's the best way to override the output of citation objects
for a specific derived backend? Can I use an export filter? should I
instead use, I don't know, cl-letf to temporarily override some internal
function belonging to citeproc-formatters and return a modified link?

thanks as always for the help!

Matt

Re: [wip-cite-new] Exporting to pandoc md (and from there to zotero odt)

[Matt Price]

Anders, I just want to say that I tested this out and it works great. It's
quite shocking, actually, to have such an easy path from org to odt with
live citations.  I'd love to see this stuff in MELPA or otherwise made more
generally accessible, as i think it fills a substantial need.

On Tue, Jun 29, 2021 at 2:29 PM Matt Price  wrote:

> Anders, just to be clear:  with your new exporters I can, as of today,
> insert the new [cite:@citekey] links using a better-bibtex-generated
> bibliography file, export to odt via pandoc, and then manage my citations
> with zotero in the resultant odt?
>
> it so... cool.
>
> On Wed, Jun 23, 2021 at 4:04 PM Bruce D'Arcus  wrote:
>
>> Cool!
>>
>> We should also get pandoc to support updated org-cite syntax.
>>
>> https://github.com/jgm/pandoc/issues/7329
>>
>>
>> On Wed, Jun 23, 2021, 12:57 PM Anders Johansson 
>> wrote:
>>
>>> Hi all,
>>> I appreciate the work done on the cite functionality!
>>> I use zotero for managing my library and have previously used zotxt
>>> and org-zotxt (along with some additions in my zotxt-extra library)
>>> for being able to cite in org-mode and export correctly to various
>>> formats.
>>> https://gitlab.com/egh/zotxt
>>> https://gitlab.com/egh/zotxt-emacs
>>> https://gitlab.com/andersjohansson/emacs-zotxt-extra
>>>
>>> Zotxt has used a custom link format for citations (similar to org-ref)
>>> which links to zotero but optionally also contains a citekey (defined
>>> via the zotero-better-bibtex extension to zotero). I actually wrote my
>>> PhD thesis with this setup, but the connection to Zotero has always
>>> felt a little wobbly and Zotero is also quite slow for my large
>>> library.
>>>
>>> Now seems to be a great time to convert to a workflow with the new
>>> cite format. To enable a conversion from org mode to word processors
>>> with Zotero ”active” citations I created two libraries to manage this
>>> via pandoc and the pandoc filter created for the zotero-better-bibtex
>>> extension.
>>> (see: https://retorque.re/zotero-better-bibtex/exporting/pandoc/)
>>>
>>> This allows me a workflow like:
>>> 1. keep an updated  .bib-file of my library in zotero (with the help
>>> of zotero-better-bibtex)
>>> 2. cite in org-mode (for me with completion via helm-bibtex and
>>> https://github.com/bdarcus/bibtex-actions/pull/113)
>>> 3. Export to an odt-file with active zotero citations (with oc-pandoc
>>> and ox-md-pandoc-zotero)
>>>
>>> I attach them here in case anyone else finds some part of this workflow
>>> usable.
>>> https://gitlab.com/andersjohansson/org-cite-pandoc
>>> https://gitlab.com/andersjohansson/ox-md-pandoc-zotero
>>>
>>> I couldn’t find any previous definitions for exporting to the pandoc
>>> cite format so I quickly hacked up org-cite-pandoc. I suppose it could
>>> be be widely useful and included in org. Feel free to use it as
>>> suitable (I have FSF copyright assignment for emacs).
>>>
>>> Best,
>>> Anders Johansson
>>>
>>

Re: [wip-cite-new] Exporting to pandoc md (and from there to zotero odt)

[Matt Price]

Anders, just to be clear:  with your new exporters I can, as of today,
insert the new [cite:@citekey] links using a better-bibtex-generated
bibliography file, export to odt via pandoc, and then manage my citations
with zotero in the resultant odt?

it so... cool.

On Wed, Jun 23, 2021 at 4:04 PM Bruce D'Arcus  wrote:

> Cool!
>
> We should also get pandoc to support updated org-cite syntax.
>
> https://github.com/jgm/pandoc/issues/7329
>
>
> On Wed, Jun 23, 2021, 12:57 PM Anders Johansson 
> wrote:
>
>> Hi all,
>> I appreciate the work done on the cite functionality!
>> I use zotero for managing my library and have previously used zotxt
>> and org-zotxt (along with some additions in my zotxt-extra library)
>> for being able to cite in org-mode and export correctly to various
>> formats.
>> https://gitlab.com/egh/zotxt
>> https://gitlab.com/egh/zotxt-emacs
>> https://gitlab.com/andersjohansson/emacs-zotxt-extra
>>
>> Zotxt has used a custom link format for citations (similar to org-ref)
>> which links to zotero but optionally also contains a citekey (defined
>> via the zotero-better-bibtex extension to zotero). I actually wrote my
>> PhD thesis with this setup, but the connection to Zotero has always
>> felt a little wobbly and Zotero is also quite slow for my large
>> library.
>>
>> Now seems to be a great time to convert to a workflow with the new
>> cite format. To enable a conversion from org mode to word processors
>> with Zotero ”active” citations I created two libraries to manage this
>> via pandoc and the pandoc filter created for the zotero-better-bibtex
>> extension.
>> (see: https://retorque.re/zotero-better-bibtex/exporting/pandoc/)
>>
>> This allows me a workflow like:
>> 1. keep an updated  .bib-file of my library in zotero (with the help
>> of zotero-better-bibtex)
>> 2. cite in org-mode (for me with completion via helm-bibtex and
>> https://github.com/bdarcus/bibtex-actions/pull/113)
>> 3. Export to an odt-file with active zotero citations (with oc-pandoc
>> and ox-md-pandoc-zotero)
>>
>> I attach them here in case anyone else finds some part of this workflow
>> usable.
>> https://gitlab.com/andersjohansson/org-cite-pandoc
>> https://gitlab.com/andersjohansson/ox-md-pandoc-zotero
>>
>> I couldn’t find any previous definitions for exporting to the pandoc
>> cite format so I quickly hacked up org-cite-pandoc. I suppose it could
>> be be widely useful and included in org. Feel free to use it as
>> suitable (I have FSF copyright assignment for emacs).
>>
>> Best,
>> Anders Johansson
>>
>

Re: allow HTML block to escape from outline-text div? WAS: BUG? unable to surround subtrees with html tag

[Matt Price]

On Tue., Jun. 22, 2021, 4:30 a.m. Richard Lawrence, <
richard.lawre...@uni-tuebingen.de> wrote:

> Hi Matt,
>
> Matt Price  writes:
>
> >> I would like to be able to surround some portion of a subtree with a
> tag,
> >> e.g.:
> >>
> >> * parent
> >>   some text
> >> #+HTML: 
> >> ** child 2
> >>   some boxed content
> >> ** child 2
> >>more boxed content
> >>  #+HTML:
> >> ** child 3
> >>   unboxed content
>
> > I don't know if there is a way to somehow slide my own html in between
> the
> > outine-text element and the outline-container element for a child
> subtree.
> > If someone knows a way to do this, I'd appreciate a pointer, but for now
> I
> > think I have to find another way to accomplish this.
>
> Is it important that your two headlines in the boxed content export as
> ?
>
> If not, you could just use a structure like
>
> * Parent
> ** Box
> *** Boxed Child 1
> *** Boxed Child 2
> ** Unboxed Child
>
> and use something like the ignoreheading filter (see Worg's "Org Hacks"
> page) to prevent "Box" from producing a separate header, and maybe
> various properties (e.g. UNNUMBERED?) to keep the boxed children from
> appearing as part of the main outline.
>
> Otherwise, #+INCLUDE comes to mind as a possible solution.
>
> Would one of those options work for you?
>

I think the :ignore: hack will not work quite right, but the others I had
not thought of at all, and I will try them out! Thank you,

Matt

>
> --
> Best,
> Richard
>

html export backend with inlined css?

[Matt Price]

I regularly have to upload HTML to a CMS that strips out , 

allow HTML block to escape from outline-text div? WAS: BUG? unable to surround subtrees with html tag

[Matt Price]

On Mon, Jun 21, 2021 at 3:07 PM Matt Price  wrote:

> I'm using the most recent wip-cite-new branch of org, with a recent emacs
> git (28.0.50).
>
> I would like to be able to surround some portion of a subtree with a tag,
> e.g.:
>
> * parent
>   some text
> #+HTML: 
> ** child 2
>   some boxed content
> ** child 2
>more boxed content
>  #+HTML:
> ** child 3
>   unboxed content
>
> However, org seems to close the div for me before the ~** child 2~
> headline.
>
> Is this the intended behaviour? And if so is there any way for me to do
> this?
>

Sorry for the noise, I believe this is user error.   I misread the exported
source code, and firefox's developer tools added the closing tag for me, so
I kept not seeing my mistake. The error here was not realizing that section
contents get wrapped in their own div with class ~outline-text-N~, so my
~~ resulted in malformed HTML.

I don't know if there is a way to somehow slide my own html in between the
outine-text element and the outline-container element for a child subtree.
If someone knows a way to do this, I'd appreciate a pointer, but for now I
think I have to find another way to accomplish this.

thanks & apologies,

Matt

BUG? unable to surround subtrees with html tag

[Matt Price]

I'm using the most recent wip-cite-new branch of org, with a recent emacs
git (28.0.50).

I would like to be able to surround some portion of a subtree with a tag,
e.g.:

* parent
  some text
#+HTML: 
** child 2
  some boxed content
** child 2
   more boxed content
 #+HTML:
** child 3
  unboxed content

However, org seems to close the div for me before the ~** child 2~
headline.

Is this the intended behaviour? And if so is there any way for me to do
this?

Re: [org-cite, oc-basic] configurable open-at-point, font-locking when json?

[Matt Price]

On Tue., Jun. 8, 2021, 5:29 a.m. Nicolas Goaziou, 
wrote:

> Hello,
>
> Matt Price  writes:
>
> > IIUC, the current architecture assigns responsibility for both *citation
> > export* and *in-buffer actions* to a processor shosen by the user (right
> > now, oc-cite, oc-natbib, or oc-csl).
>
> That's incorrect. The current architecture provides three entry points:
> `activate', `follow' and `export'. A processor can handle any, or all of
> these capabilities. For example, `natbib' processor doesn't provide any
> interface for `follow' or `activate'. OTOH `basic' provides the three of
> them.
>

Ah. Well then

>
>
> Users can select a different processor for any of the capabilities
> listed above, independently on the others. So you don't have to change,
> for example, the processor responsible for fontification if you are
> swapping processor used for export.
>

Ah got it.

>
>
>
>
> > I guess this will become complicated once there are processors supporting
> > more exotic backends (e.g. Zotero via zotxt), but package managers could
> > handle that in readme's or perhaps with a single defcustom like, maybe,
> > ~org-zotxt-do-cite-setup~, or similar.
>
> I'm not sure to understand this, as I don't know what zotxt does
> exactly.
>

Well, I'm not sure it's relevant anymore, but zotxt provides a direct API
to the zotero database, and also an emacs library with functions for
querying and processing that API. All I really meant here is that of org
starts to support this kind of nonstandard backend (in which the
bibliography file isn't actually a file anymore), then the :activate and
:follow functions really will have to be different in buffers using that
backend.

>
> > I also think this will reduce code repetition across the 3 processors,
> and
> > generally simplify life for most users (though initial setup may be more
> > frequent.
> >
> > Have I understood correctly, and if so what do you think of this idea?
>
> Unless I'm mistaken, your idea is already implemented, isn't it?
>

Yup. Thanks for the very clear explanation and sorry for the noise.

Matt

Re: [org-cite, oc-basic] configurable open-at-point, font-locking when json?

[Matt Price]

Ni Nicolas and Bruce,

I'm having trouble keeping up with these emails, let alone testing all
these new features!  But this most recent response of yours, Nicolas, makes
me wonder if it's worth raising a concern.

On Mon, Jun 7, 2021 at 5:15 PM Nicolas Goaziou 
wrote:

> Hello,
>
> "Bruce D'Arcus"  writes:
>
> > Similar to my idea to have a configurable capf, could that function be
> > configurable?
> >
> > A couple of people noted, for example, that the preferred choice would
> > be that the actual document be opened. I actually got a PR for exactly
> > this default behavior this past weekend, and bibtex-completion has a
> > function that takes keys and does this. So would be great do something
> > like:
> >
> > (setq org-cite-basic-open bibtex-completion-open-pdf)
>
> If you want to use a different "follow" capability, you need to provide
> a different processor instead of configuring this one.
>

IIUC, the current architecture assigns responsibility for both *citation
export* and *in-buffer actions* to a processor shosen by the user (right
now, oc-cite, oc-natbib, or oc-csl).

In addition, it is more or less expected that some users will frequently
switch back and forth between processors depending on whether they are
exporting to latex or HTML (or something else, like ODT, I guess).

But these two features (in-buffer actions and eport) are functionally and
logically distinct. The current architecture (if I've understood it right)
means that in a non-infrequent use case, the in-buffer actions (and also
fontification & overlays) will likely change back and forth rather
quickly.But surely this is not what most users would want. Instead, they
would likely want to fine-tune the in-buffer appearance and actions
themselves.  Wouldn't it make sense to have a series of defcustoms, like,
maybe, org-cite-open-function, org-cite-font-lock-function, and maybe
others, which could be set by users on their own? Org-cite could supply
some sensible defaults and advanced users could customize.

I guess this will become complicated once there are processors supporting
more exotic backends (e.g. Zotero via zotxt), but package managers could
handle that in readme's or perhaps with a single defcustom like, maybe,
~org-zotxt-do-cite-setup~, or similar.

I also think this will reduce code repetition across the 3 processors, and
generally simplify life for most users (though initial setup may be more
frequent.

Have I understood correctly, and if so what do you think of this idea?

>
> > Second, I completely lose font-locking when using json files. I know
> > you have plans, Nicolas, to add json support, but even without that,
> > shouldn't the citation be highlighted?
>
> Could you provide an example?
>
> Regards,
> --
> Nicolas Goaziou
>
>

wip-cite-new minimal working example?

[Matt Price]

I may have missed something somewhere, but does anyone have a MWE for the
new citation syntax, including a minimal bibliography file, style files for
the various backends, & an org file with included citations? I am trying to
test out oc-csl.el but am pretty sure I am not dong it right.

Re: [wip-cite-new] Initial implementation of `csl' citation processor

[Matt Price]

I'm trying to run this on a fairly recent emacs-master w/ native-comp
enable, using  ~emacs -Q ~/test-cites.org~, where test-cites.org is this:

#+cite_export: csl
"/home/matt/Zotero/styles/chicago-manual-of-style-16th-edition-inline-notes-for-syllabi.csl"
#+bibliography: /home/matt/.emacs.d/zotero-refs.bib
#+begin_src emacs-lisp
  (package-initialize)
  (add-to-list 'package-archives
  '("melpa" . "https://melpa.org/packages/;))
  (add-to-list 'load-path "~/src/org-mode/emacs/site-lisp/org")

  (org-reload)
  (add-to-list 'load-path "~/src/org-mode/lisp")
  (require 'oc)
  (require 'citeproc)
  (require 'oc-csl)
#+end_src

[cite:@zittrain_future_2008]

trying to execute the src block gives me a segfault on org-reload, with
different messages each time. Any idea how I should diagnose, or am I
perhaps doing something wrong?

On Thu, May 27, 2021 at 12:18 PM Bruce D'Arcus  wrote:

> On Thu, May 27, 2021 at 11:02 AM Bruce D'Arcus  wrote:
>
> > One other thing I tested just now not yet accounted for: a locator
> > whose values is a list; for example, that renders as (Doe 2019, pp.
> > 23, 25).
> >
> > Is there an easy way to handle that?
>
> Nevermind; this works as expected.
>
> [cite:@latexcompanion p.23, 24]
>
> Bruce
>
>

Re: there are always ways to go - Re: Sad tweet

[Matt Price]

I am not an emacs contributor, but I think Jonas is saying, "I love emacs
and I am the primary author of one of the two most important emacs
packages. If contributing to emacs is frustrating even for me, think how
many people will give up. We need to change our culture." I would be very
careful about describing him as "attention-seeking." He is a very diligent
and responsive package maintainer who works *constantly* to improve an
important piece of emacs infrastructure.

On Tue., May 25, 2021, 2:53 a.m. Jean Louis,  wrote:

> * Ypo  [2021-05-25 06:09]:
> > I think he is the creator of the magit package.
>
> OK, it's not Org related and is not a direct post... It is just
> an attention seeking... Tweet.
>
> I am following emacs-devel mailing list, who wish to contribute
> package to Emacs is very welcome to do so, and it is very
> straightforward.
>
> Best way to start improvement is in my opinion with the package
> creation.
>
> Those who wish to make changes in the core have to discuss it with
> Emacs core developers, that is how it is, as newly introduced ideas
> are not necessarily thought well for millions of users.
>
> --
> Jean
>
> Take action in Free Software Foundation campaigns:
> https://www.fsf.org/campaigns
>
> Sign an open letter in support of Richard M. Stallman
> https://stallmansupport.org/
>
>

Re: Multiple calc commands with orgbabel

[Matt Price]

Can you explain how you get calc embedded mode working in org? I have never
used it and it sounds interesting, but I don't understand what hte
delimiters are.

On Wed, May 5, 2021 at 2:35 AM Eric S Fraga  wrote:

> On Wednesday,  5 May 2021 at 07:46, pie...@caramail.com wrote:
> > Have been trying to execute multiple calc commands, but when I
> > evaluate the calc expressions, I get just one result.
>
> ob-calc returns the top element of the stack when finished and this will
> be the result of the last operation in the src block.  I don't think
> there's any way around this.
>
> I use embedded Calc for this reason.  You could rewrite your equations
> as simple lines (separated by empty lines from the surroundings) and
> evaluate each in turn with "C-x * u":
>
> fsolve(x 2 + x = 4, x) => x = 1.333
>
> fsolve([x + y = a, x - y = b], [x, y]) => [x = a + (b - a) / 2, y = (a -
> b) / 2]
>
> I added the "=>" at the end of each expression so that the result is
> shown to the right instead of replacing the expression itself (default
> embedded Calc behaviour).
>
> --
> : Eric S Fraga via Emacs 28.0.50, Org release_9.4.5-395-g82fbdd
>
>

async export and then do something with file

[Matt Price]

Hi everyone,

I am wondering if I can perform and export asynchronously and then, without
tying up emacs, wait around for the export to finish and then perform some
other task, like upload the file to a server.  Has anyone tried this
before? I think perhaps the easiest thing to do would be to use async.el or
some similar third-party, but I can't tell what the preferred method is. I
would love osme guidance!

Thanks,

Matt

Re: Using backticks for the inline code delimeter?

[Matt Price]

On Tue, Apr 20, 2021 at 4:24 PM John Kitchin 
wrote:

> I have used an approach like the one here
> https://endlessparentheses.com/define-context-aware-keys-in-emacs.html
>
> to make context aware key-bindings.
>
> THanks John. That post was very helpful -- really all I was looking for
was (org-in-src-block-p), but the larger infrastructure is cool and I'll
keep it around for a while to see if I end up reusing it.

Re: wip-cite status question and feedback

[Matt Price]

As a user, is there any way I can participate at this point? I'm not in a
position to contribute code tight now but really do want to have this
feature as soon as possible. It's going to improve my students' lives quite
a bit.


>
>
>

Re: Using backticks for the inline code delimeter?

[Matt Price]

On Wed., Mar. 31, 2021, 3:22 p.m. Timothy,  wrote:

>
> autofrettage  writes:
>
> > Quick and Dirty: Bind key '`' to ~ in Emacs?
> >
> > (I guess it is clear I haven't thought about the consequences.)
>
> You can add that just to the Org-mode map. That wouldn't be too bad,
> there's always C-q.
>

Is it possible to bind a key in org-mode but bind it back to another
character if you're in a special environment, eg a code block? That would
probably be my preference. So "`" inserts "~" when you're writing text but
"`" in an elisp or markdown SRC block, for instance.

I guess just write a function that checks context? Presumably all the
overloaded keybindings do this already but I guess I don't really know how
they do so.

I do in general wish it were easier to switch between writing markdown and
writing org, since I often have to write markdown for work.

>
> --
> Timothy
>
>

Re: eldoc recursion error

[Matt Price]

On Tue, Sep 8, 2020 at 11:27 AM Matt Price  wrote:

>
>
> On Tue, Sep 8, 2020 at 10:53 AM Bastien  wrote:
>
>> Hi Matt,
>>
>> Matt Price  writes:
>>
>> > In a new org file, add these lines:
>> >
>> > #+begin_src python
>> > print
>> > #+end_src
>> >
>> > position cursor inside block and the error message occurs.
>>
>> I can't reproduce the bug.  What version of Emacs are you using?
>>
>> Can you give a recipe starting with emacs -q?
>>
>
> I htink I have it now.
>
> $ emacs -q
>
I forgot to say, this is emacs git (2.8.0.50 from yesterday) and org-mode
master (current)

>
> (require 'org-eldoc) ;;(or navigate to lisp/org-eldoc.el and eval-buffer)
> create new org file, add this content:
>
> #+begin_src python
>  print
>  #+end_src
>
> ensure eldoc-mode is turned on. place cursor inside block.
>
> If for some reason that doesn' t work try manually running
> `org-eldoc-load` and repositioning cursor.
>
>
>> Thanks!
>>
>> --
>>  Bastien
>>
>

Re: eldoc recursion error

[Matt Price]

On Tue, Sep 8, 2020 at 10:53 AM Bastien  wrote:

> Hi Matt,
>
> Matt Price  writes:
>
> > In a new org file, add these lines:
> >
> > #+begin_src python
> > print
> > #+end_src
> >
> > position cursor inside block and the error message occurs.
>
> I can't reproduce the bug.  What version of Emacs are you using?
>
> Can you give a recipe starting with emacs -q?
>

I htink I have it now.

$ emacs -q

(require 'org-eldoc) ;;(or navigate to lisp/org-eldoc.el and eval-buffer)
create new org file, add this content:

#+begin_src python
 print
 #+end_src

ensure eldoc-mode is turned on. place cursor inside block.

If for some reason that doesn' twork try manually running `org-eldoc-load`
and repositioning cursor.


> Thanks!
>
> --
>  Bastien
>

Re: eldoc recursion error

[Matt Price]

On Tue, Sep 8, 2020 at 10:25 AM Bastien  wrote:

> Hi Matt,
>
> can you provide a recipe to reproduce the problem?
>
> oops, sorry, that was stupid.

In a new org file, add these lines:

#+begin_src python
print
#+end_src

position cursor inside block and the error message occurs.

eldoc recursion error

[Matt Price]

I know there have been a couple of updates to org-eldoc lately.  After
updating to current master, I get this error in source blocks if eldoc mode
is turned on:

eldoc error: (error Lisp nesting exceeds ‘max-lisp-eval-depth’)

Is there an easy fix for this? is it a generic eldoc problem or specific to
org?

Thanks!

Matt

Re: Improving Org Mode for VSCode - Thinking Aloud

[Matt Price]

There are a number of interesting topics here. Like others, I would be very
keen to share my org-based projects with non-Emacs users. I also only have
very limited time & skills to contribute.  There is an existing extension
for org-mode on vscode here:
https://github.com/vscode-org-mode/vscode-org-mode.  That might be the best
place to help out. One piece of low-hanging fruit would be simply to add an
HTML renderer to that plugin (https://github.com/orgapp/orgajs is the
obvious choice). A full-on literate programming engine seems like a very
tough thing to engineer, but perhaps it's possible to imagine a new
architecture in which there are, as you suggest, various code
interpretation engines. It's an exciting idea; I wonder if it's possible to
explore the possibilities in a more concrete and structured way?

On Sun, May 24, 2020 at 4:20 PM rey-coyrehourcq <
sebastien.rey-coyrehou...@univ-rouen.fr> wrote:

> Hi,
>
> Really cool discussion here.
>
> My two cents, Jupyter & Kernel on various langage, which have a very
> large community, could be an interesting backend for org-babel on
> VSCode or anyeditoryouwantusehere.
>
> Lot of things start to appear to collaborate online around
> online/scientific cnotebook/literate programing this last year, and that
> continue, for example i discover recently :
> Stencila : https://github.com/stencila=
>
> org-babel / org-mode outside of emacs has a great potential to lead an
> alternative to other markdown (RStudio / Jupyter) solution.
>
> Best regards
>
> Le dimanche 24 mai 2020 à 06:06 -0700, Jack Kamm a écrit :
> > It would be very good indeed for org-babel if it could be ported to
> > other editors. One of the biggest drawbacks of org-babel notebooks is
> > that I can't collaborate with my colleagues on them, since I can't
> > expect them to use Emacs.
> >
> > Aside from VSCode, I think RStudio would be an excellent target for a
> > few reasons:
> >
> > - Literate programming is already popular among R users (see also:
> >   knitr, sweave, Rmarkdown)
> > - There is a strong ob-R community here
> > - There are some prominent Emacs users among the Rstudio developers
> >   (e.g. Lionel Henry, who I think is both an Rstudio and ESS
> > developer)
> >
> > However, this would be a massive undertaking, and ultimately would
> > need
> > a volunteer to step up to the plate. I don't have any bandwidth to do
> > this in the foreseeable future but dream of working on it one day.
> > The
> > biggest downside -- it would require spending considerable time
> > outside
> > Emacs!
> >
> --
>
>
> Sébastien Rey-Coyrehourcq
> Research Engineer UMR IDEES
> 02.35.14.69.30
>
> {Stronger security for your email, follow EFF tutorial :
> https://ssd.eff.org/}
>
>
>

Re: R session and plotting in x11 window

[Matt Price]

On Sun, Apr 5, 2020 at 1:19 PM Berry, Charles 
wrote:

>
>
> > On Apr 4, 2020, at 4:27 PM, Matt Price  wrote:
> >
> > Does anyone know much about the difference between an R session opened
> by typing M-x R, and the R session opened by org-babel?
>
>
> Short answer: almost none.
>
> Long answer: what `org-babel-R-initite-session' and friends do.
>
:-) thanks, I should have been looking for that

>
> >
> > I'm just learning R and my usual method for learning a language is to
> keep a kind of notebook in org with code snippets they I can execute and
> iterate on rapidly as I learn. This works great in R when I'm just doing
> math.  When I am working on plots, it would be nice to have them open up
> quickly either in emacs or in the standard x11 window that R session opened
> switch M-x R opens up.
> >
> > I know I can set the src block headers to produ e a file, but when I'm
> just iterating rapidly I often switch back and forth between a data output
> and a graphical output, and typing/erasing those headers is clunky and
> slow. It would be easier to just paste the plot command into the console
> and have it pop open the window... But that doesn't seem to work. Anyone
> know if I can tweak something to make that possible?
> >
>
>
> I sam really puzzled by this. Do you have an ECM that illustrates this?
>
> Working interactively on my Mac (Quartz - X11 is the device), I routinely
> do what you describe - usually working from the src edit buffer - and the
> plots are displayed (and older plots are available via clover-left or some
> such).
>
> If I had to guess, I'd say that you are opening an R session, but not
> using it. If you execute a src block, but it does not have a `:session'
> header, a new instance of R will create a plot file and then exit. If you
> look in the default directory, you would see `Rplots.pdf' or some such.
>
> The only other thing that comes to mind is that you opened a device that
> is holding on to all your plots. Try `dev.cur()' in R immediately before
> and after you create a plot and see what the result is.
>
> This was the problem. I don't see that I'm calling dev.set() anywhere but
when the session initiates dev.cur() returns

null
 1

calling dev.set(1) or dev.set(2) launches an R_x11 window and future plots
are displayed there.  As I say, I'm just learning R, and I don't really
understand how the device is set up. I also don't understand why it would
be set to X11 in a plain-old R session, but not in an org-babel R session.
Most references to "device" in ~ob-R.el~ seem to be managing file outputs,
and "X11". For now I don't think I'll explore  a proper solution as I'm
already pretty far down a rabit hole just learning R at all!  But thanks
very much for this workaround.

Matt

> HTH,
>
> Chuck
>
>
>

R session and plotting in x11 window

[Matt Price]

Does anyone know much about the difference between an R session opened by
typing M-x R, and the R session opened by org-babel?

I'm just learning R and my usual method for learning a language is to keep
a kind of notebook in org with code snippets they I can execute and iterate
on rapidly as I learn. This works great in R when I'm just doing math.
When I am working on plots, it would be nice to have them open up quickly
either in emacs or in the standard x11 window that R session opened switch
M-x R opens up.

I know I can set the src block headers to produ e a file, but when I'm just
iterating rapidly I often switch back and forth between a data output and a
graphical output, and typing/erasing those headers is clunky and slow. It
would be easier to just paste the plot command into the console and have it
pop open the window... But that doesn't seem to work. Anyone know if I can
tweak something to make that possible?

Thank you!

Matt

Re: rmarkdown-like production of multiple plots in org

[Matt Price]

Thanks Chuck and all other respondents.  Huh. I'll keep experimenting. I
like Chuck's idea of helper function that pipes theoutput as described but
for now, it might actually beeasier to stay with Rmarkdown (!). thanks, Matt

On Thu, Apr 2, 2020 at 1:39 PM Berry, Charles 
wrote:

>
>
> > On Mar 31, 2020, at 12:23 PM, Matt Price  wrote:
> >
> > I'm completely new to R.
> >
> > I've started working with a project that creates plots using the ggplot
> package -- so by default it creates grid objects, rather than writing to
> files.
> >
> > In rmarkdown/rstudio, I can write something like this in a SOMEFILE.Rmd
> :
> >
> > ```
> > install_github('eeholmes/CoV19')
> > library(CoV19)
> > getdata();
> > plot4(world, 'Ontario Canada')
> > plot2(world, 'Italy')
> > plot4(states, "WA")
> > ```
> >
> > I sort of love how the rmarkdown package will just create all 3 of those
> plots, save them to auto-named files, and render to HTML.
>
> Actually, this is knitr (which rmarkdown Imports) at work.  There are
> options as to how knitr will handle multiple plots in a chunk as described
> in
>
> https://yihui.org/knitr/options/#plots
>
> (which include `fig.show="animate"' to create an animation based on
> multiple plots!)
>
> So this applies to various filetypes in addition to *.Rmd (*.Rnw, for
> one).
>
>
> >  In RStudio, running just that block will also create all three blocks
> and display them in the editor.
> >
> > By contrast, creating a series of many plots in org is fairly tedious.
> I have to name the plot individually & put each function call in its own
> src block. Is there any way to mimic the behaviour of rmarkdown instead? I
> odn't understand babel or R enough to really even see how something like
> that could be implemented, but I'd appreciate some pointers.  Thank you!
>
> Getting babel to handle this seamlessly would be a significant effort.
>
> You can use ox-ravel (https://github.com/chasberry/orgmode-accessories.git)
> to export to *.Rmd and then render the result.  However, that does not have
> the interactivity of `org-babel-execute-src-block' and does not insert the
> graphics into the *.org file.
>
> I suppose that a function could be created to narrow to the src block,
> export it as *.Rmd to a buffer, run that buffer as the `text' arg of
> knitr::knit, then add links for the png's back to the *.org file. I haven't
> thought much about this - getting this to work in a simple case would not
> be too hard, but there may be a can of worms that this approach opens.
>
> HTH,
>
> Chuck
>
>
>

rmarkdown-like production of multiple plots in org

[Matt Price]

I'm completely new to R.

I've started working with a project that creates plots using the ggplot
package -- so by default it creates grid objects, rather than writing to
files.

In rmarkdown/rstudio, I can write something like this in a SOMEFILE.Rmd :

```
install_github('eeholmes/CoV19')
library(CoV19)
getdata();
plot4(world, 'Ontario Canada')
plot2(world, 'Italy')
plot4(states, "WA")
```

I sort of love how the rmarkdown package will just create all 3 of those
plots, save them to auto-named files, and render to HTML.  In RStudio,
running just that block will also create all three blocks and display them
in the editor.

By contrast, creating a series of many plots in org is fairly tedious.  I
have to name the plot individually & put each function call in its own src
block. Is there any way to mimic the behaviour of rmarkdown instead? I
odn't understand babel or R enough to really even see how something like
that could be implemented, but I'd appreciate some pointers.  Thank you!

get subtree contents as string

[Matt Price]

I have the following convenience function that scans contents of a subtree
and then sets a property, adds an overlay, and changes the TODO status:
(defun org-lms-set-grade ()
  "set grade property for all headings on basis of \"- Grade :: \" line
  Use with caution."
  (interactive)
  (save-restriction
(org-narrow-to-subtree)
  (save-excursion
  (org-back-to-heading)
  (while (re-search-forward ol-grade-regex nil t )
(org-set-property "GRADE" (or (match-string 2) 0))
(org-todo "READY"
  (org-lms-overlay-headings)  )

It works, but there's a short (<1sec) non-responsive period when it runs.
I'm wondering if it has to do with ~org-narrow-to-subtree~, so I thought
I'd just copy the contents of the subtree as a string and replace
re-search-forward with string-match. However, I'm not finding a
super-obvious way to copy the subtree as a string. Am I missing something?
And am I likely to be right that that's the source of the delay (as I write
this, I'mwondering if hte overlay drawing might be the real problem? The
whole project this is part of (https://github.com/titaniumbones/org-lms)
suffers from delays while the buffer redraws, so I'm keen to figure out
Better Ways To Do Things.

Thanks as always,
Matt

Re: [PATCH] restore window configuration after org-edit-src-exit

[Matt Price]

Hi, sorry I missed this thread in the peri-holiday rush, just want to say
thank you all for ironing out the difficulties arsing from that change
earlier.

On Tue, Jan 7, 2020 at 4:06 AM Nicolas Goaziou 
wrote:

> Hello,
>
> Jack Kamm  writes:
>
> > Attached is a patch to restore the window configuration after exiting an
> > org source buffer, when org-src-window-setup is one of reorganize-frame,
> > split-window-below, or split-window-right.
>
> Thank you for the patch! Some comments follow.
>
> > @@ -156,8 +156,7 @@ split-window-right Show edit buffer to the right of
> the current window,
> > keeping all other windows.
> >  other-window   Use `switch-to-buffer-other-window' to display edit
> buffer.
> >  reorganize-frame   Show only two windows on the current frame, the
> current
> > -   window and the edit buffer.  When exiting the edit
> buffer,
> > -   return to one window.
> > +   window and the edit buffer.
> >  other-frameUse `switch-to-buffer-other-frame' to display edit
> buffer.
> > Also, when exiting the edit buffer, kill that
> > frame."
>
> It would be useful to also explain what happens when the value is
> `reorganize-frame', `split-window-right', or `split-window-right', e.g.,
> that previous window configuration is restored.
>
> > +(when org-src--saved-temp-window-config
> > +  (set-window-configuration org-src--saved-temp-window-config)
> > +  (setq org-src--saved-temp-window-config nil
>
> It may be useful to wrap `set-window-configuration' within
> `unwind-protect' so we can still reset the
> `org-src--saved-temp-window-config' variable even though something went
> wrong.
>
> WDYT?
>
> Regards,
>
> --
> Nicolas Goaziou
>
>

Re: Small change to link syntax in maint

[Matt Price]

Just an apology for not getting back on this, I haven't managed to text
this properly yet (apologies) but I am pretty sure it will fix my use case,
for which I'm grateful. Thanks Nicolas.

On Sun., Dec. 22, 2019, 9:09 a.m. Nicolas Goaziou, 
wrote:

> Hello,
>
> Matt Price reported an issue[1] with new link syntax in Org 9.3. As
> a consequence, I had to modify it slightly.
>
> Backslash-escaping was required for some closing bracket characters.
> Now, /every square bracket/ within the URI part of a bracket link must
> be escaped.
>
> I updated the manual and the ORG-NEWS file accordingly. Please let me
> know if there's something suspicious.
>
> Sorry for the inconvenience.
>
> Regards,
>
> Footnotes:
> [1]  <
> https://lists.gnu.org/archive/html/emacs-orgmode/2019-12/msg00312.html>
>
> --
> Nicolas Goaziou
>
>

Re: managing aggressive link activation within lines?

[Matt Price]

On Thu, Dec 19, 2019 at 2:55 AM Nicolas Goaziou 
wrote:

> Hello,
>
> Matt Price  writes:
>
> > MWE:
> > test.org:
> > --
> > This is a paragraph with  [[https://google.com][a link]]
> > ---
> > steps to reproduce:
> >
> > put cursor at beginning of file and type ~[[~.  The paragraph text will
> > collapse into the link definition, and (point) will move to the ~a~ in ~a
> > link~, so manual typing of a new link will fail.  Is there anything I can
> > do to stop this from happening?
>
> IIUC, this is related to the link syntax change introduced in Org 9.3,
> which allows, among other things, "[[" within a link. In this situation,
> once you write
>
>   [[This is a paragraph with  [[https://google.com][a link]]
> ^^
>
> the underlined part becomes the link.
>
> I agree this is not optimal. Yet, I'm not sure about how to deal with
> this.
>
> Ah, interesting. That makes a lot of sense.  Would it be unworkably
destructive for me to revert the link syntax locally ? I have no particular
need  for the new functionality and the current situation has been a
significant cognitive burden, not least because I make so many typos that I
am constantly re-editing existing text.

>
>

managing aggressive link activation within lines?

[Matt Price]

Hi everyone,

Not sure if this has always been an issue but with a pretty recent org git
and emacs-git, I'm running into a usability issue with org-activiate-links.

MWE:
test.org:
--
This is a paragraph with  [[https://google.com][a link]]
---
steps to reproduce:

put cursor at beginning of file and type ~[[~.  The paragraph text will
collapse into the link definition, and (point) will move to the ~a~ in ~a
link~, so manual typing of a new link will fail.  Is there anything I can
do to stop this from happening?

>From what i can tell, ~org-activate-links~ was last modified in
1fd07c1eeb0f29de9ce058fac995031413201d4d in late August, but I'm not sure
if that's when this behaviour was introduced, or if it has been around for
a while and I'm only noticing it now. I know many people don't type links
manually, but I do so almost exclusively, so this can be frustrating when
I'm editing existing text.

Thanks as always!

org-custom-id-goto?

[Matt Price]

Is there a quasi-equivalent of ~org-id-goto~  or
~org-babel-goto-named-src-block~ which will jump to a header in the current
buffer? I' like to be able to quickly navigate to a buffer by name; usually
I give custom_id properties to headers that I am likely to link to, so
using that property would make sense. I'd also be happy with  a helm
interface to the actual headline text

Just want ot check if I'm missing something that already exists before
trying to code my own!

Re: [O] org-fast-todo-selection window behaviour?

[Matt Price]

Ah well. I find the new way jarring, but it doesn't seem to bother anyone
else, and as it's a one-line (2 character!) change for me I think I can
carry the diff in my init file for now. In any case it's a small issue I
think.An honour to find myself in disagreement with the org-founder!

On Mon, Oct 21, 2019 at 5:47 AM Carsten Dominik  wrote:

> Hi Matt,
>
> I made this change, because I found the previous way jarring.  The window
> with the selection information showed up in different places depending on
> what the current window setup is. With the new implementation, the info
> window is always in the same predictable place.  After the selection is
> done, the old window setup is restored to exactly what it was
>
> Carsten
>
> On Sun, Oct 20, 2019 at 8:46 PM Matt Price  wrote:
>
>> I've recently noticed a slightly frustrating behavour on the part of
>> org-todo that I think is new and maybe was introduced in mid-August with
>>
>> f1c030bed54737319aeb1d592e3340d6a48cea3a
>>
>> In a split frame,calling org-todo with org-use-fast-todo-selection
>> enabled, ~C-c C-t~ now calls ~delete-other-windows~ before popping up the
>> org-todo keywords window.  Is this necessary? I find this behaviour
>> visually confusing and distracting, and a slowdown to my workflow.  Would
>> it make sense to introduce some kind of defcustom for this? For now I'm
>> just commenting out line 10614 of org.el, but if others want to be able to
>> customize the behaviour I will submit a patch.
>>
>> Maybe there's a reason delete-other-window is necessary, but i don't see
>> it in the commit message nor immediately in the other parts of this
>> otherwise very well-documented commit
>>
>> Thanks!
>>
>>

[O] org-fast-todo-selection window behaviour?

[Matt Price]

I've recently noticed a slightly frustrating behavour on the part of
org-todo that I think is new and maybe was introduced in mid-August with

f1c030bed54737319aeb1d592e3340d6a48cea3a

In a split frame,calling org-todo with org-use-fast-todo-selection enabled,
~C-c C-t~ now calls ~delete-other-windows~ before popping up the org-todo
keywords window.  Is this necessary? I find this behaviour visually
confusing and distracting, and a slowdown to my workflow.  Would it make
sense to introduce some kind of defcustom for this? For now I'm just
commenting out line 10614 of org.el, but if others want to be able to
customize the behaviour I will submit a patch.

Maybe there's a reason delete-other-window is necessary, but i don't see it
in the commit message nor immediately in the other parts of this otherwise
very well-documented commit

Thanks!

[O] org-ref: format full bibliography?

[Matt Price]

I'm successfully using org-ref to create bibliographic entries and
citations in my lectures and syllabi, using ox-hugo and org-re-reveal, and
of late also citeproc-orgref. It's great. Now I'd like to add a
"bibliography' page to my course websites, which provides a long list of
works that students might be interested in. Does org-ref have a command to
just list all the entries in a .bib file? Or even better, to filter then by
some property?   A quick look through the manual didn't turn one up but it
seems like it might be a common need so I thought I'd ask.

Thanks!

Matt

[O] managing repetitive code in export & tangling

[Matt Price]

In my lectures i often have simple examples that build progressively over
several slides. In order to use klipse properly, but also for clarity, I
gneerally repeat variable declarations from slide to slide, so for
instance:

** Making Lists (Arrays)

+#NAME: js-array-historians
#+BEGIN_SRC js
let historians= ["Edward Gibbon", "Leopold von Ranke", "Edward Said", "Joan
Scott"];
#+END_SRC

** Repetition: While Loops
#+NAME: js-while-hist
#+begin_src js
<>
let i = 0;

while (i < historians.length) {
console.log(historians[i] + " was a historian.");
i+=1;
  }
#+end_src

There might then be a sequence of another 5 or 6 slides in which the same
"historians" array is bound to the same value and used as an example.

This works fine on its own. However, I would also like to tangle all this
code to a single file per lecture so students can download a git repo and
play with it directly in a real text editor. Unfortunately, javascript will
error out if a ~let~ bound variable is  redeclared in the same scope.  I'm
wondering if there's any way to specify that a noweb reference only be
included one time in a tangled file. Or if there are cleverer workarounds
that folks can suggest!

Thanks much,

Matt

Re: [O] [RFC] Document level property drawer

[Matt Price]

On Sat., Oct. 5, 2019, 6:10 p.m. Adam Porter,  wrote:

> Marco Wahl  writes:
>
> > Just I got the idea that for a good part this discussion is about
> > personal preferences.
>
> Personal preferences are relevant to this issue in that Org is flexible
> and allows users to configure it accordingly.  But that is not the only
> consideration at stake.  Consistency, compatibility, and longevity are
> even more important.
>
> > What I really find irritating is that "Org ... allows #+KEYWORD: lines
> > to appear anywhere in a file" (This sentence is from you) with the
> > meaning that the settings apply to the whole file.  I think this
> > interpretation of #+KEYWORD: lines is unnecessary and confusing.
>
> Regardless, that is the way Org works, and how it has for many years.
> We can't break that.
>

I'd like to just  quickly chime in in support of Adam's caution on this
issue. I can absolutely see advantages to document level properties, I have
written many code fragments that rely on the use of keywords and expect org
filensyntax to be consistent with what actually exists. I use these code
fragments to hold together a somewhat fragile workflow that allows me to
use org in a work environment that is not especially receptive to simple
text documents. I have invested a lot of time in making those systems run
and sometimes even I don't entirely remember what I did to make them
possible.

It would really, really suck to have those systems break. It would take me
a lot of time to track down the causes and change what I needed to. VMs
that currently pull in Emacs andnorg and my code would stop working. Old
versions of my files would no longer render properly. My efforts to make my
courses and other writings effectively reproducible by others would be
significantly set back. Etc. I think these are the kinds of difficulties
Adam means to describe.

>
>

Re: [O] [PATCH] ox-html: add option to restore old src block behaviour?

[Matt Price]

So I'm pretty interested in this: what to others thi k?

On Sat., Sep. 21, 2019, 8:16 a.m. Jens Lechtenboerger, <
jens.lechtenboer...@fsfe.org> wrote:

> On 2019-09-19, Matt Price wrote:
>
> > Over the summer, commit ded3d27b1468b878197e5fe55a70c5e13350ea27
> > by Nik Clayton was merged to master. It's a one-line change that
> > adds new ~~ tags around each lin of code in html export of
> > source blocks.  It's useful because it allows individual lines to
> > be addressed directly by CSS.
> >
> > However, at least one very common syntax highlighter,
> > https://highlinghtjs.org, expects just a single  tag, as do
> > other common CSS frameworks.
> > [...]
>
> The attached patch adds a new variable org-html-wrap-src-lines to
> control whether code tags should be added or not.
>
> I’m not sure whether :package-version 9.3 is correct.  Also, I set
> the value to t, which does not change the current functionality.
> However, for backwards compatibility (up to version 9.2.6), a value
> of nil would be preferable.  Any thoughts?
>
> Best wishes
> Jens
>
>

Re: [O] How many other people want an org-mode client to the Canvas LMS API?

[Matt Price]

Wow, great, I hope it ends up being useful for you. And that those months
give me time to clean up the mess.

Am working with TAs this semester for the first time since I wrote this, so
I have some incentive to improve the 2 way flow between org and canvas. So
hopefully by the time you look at it, it will be in better shape.

On Mon., Sep. 30, 2019, 8:48 p.m. John Kitchin, 
wrote:

> I am pretty interested in this. I can't use it this semester, but next
> semester I am teaching a new class where it might be useful. I don't
> have much time this fall to look into it, but I will have time in the
> spring before my class starts.
>
>
> Matt Price  writes:
>
> > Hi everyone,
> >
> > I've been working away slowly on my primitive API client for the Canvas
> > Learning Management System, which my university now uses (
> > https://github.com/titaniumbones/org-lms, note the documentation is
> > terrible and may be misleading in places). The code is at my usual very
> low
> > level of quality. I'm interested in improving it, but if I'm going to be
> > the only person who ever uses it, my motivation to do so is somewhat more
> > limited. So I just wanted to ask here if anyone else would really like to
> > have this tool (I've asked before, but it's been a while). For me it's
> been
> > kind of great, and along with ox-hugo has made it possible for me to keep
> > all my course materials in one place. But it might not be worthwhile for
> > others!
> >
> > Matt
>
>
> --
> Professor John Kitchin
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
>
>

[O] How many other people want an org-mode client to the Canvas LMS API?

[Matt Price]

Hi everyone,

I've been working away slowly on my primitive API client for the Canvas
Learning Management System, which my university now uses (
https://github.com/titaniumbones/org-lms, note the documentation is
terrible and may be misleading in places). The code is at my usual very low
level of quality. I'm interested in improving it, but if I'm going to be
the only person who ever uses it, my motivation to do so is somewhat more
limited. So I just wanted to ask here if anyone else would really like to
have this tool (I've asked before, but it's been a while). For me it's been
kind of great, and along with ox-hugo has made it possible for me to keep
all my course materials in one place. But it might not be worthwhile for
others!

Matt

Re: [O] parsing time strings from properties

[Matt Price]

many thanks to both of you. Yours was very interesting to read, Thomas, but
ts makes it quite a bit easier to write:

(defun o-l-date-to-timestamp (date)
  "use ts.el date parse functions return an ISO-compatible
timestamp for transmission to Canvas via API. DATE is a string,
usually of the form `2019-09-26`, but optionally including a full time."

  (ts-format "%Y-%m-%dT%H:%M:%S%:z" (ts-parse-fill 'end date )))

I'm quite looking forward to using dash, s, ts, kv, etc to simplify my
often very obtuse legacy code.


On Sun, Sep 22, 2019 at 6:10 AM Thomas Plass  wrote:

> Hi,
>
> Matt Price wrote at 16:27 on September 21, 2019:
> :
> : :DUE_AT: 2019-09-26
> :
> : ...
> :
> : I'm wondering though how hard
> : it would be to get the current time zone -- or the time zone that the
> course is taught in -- from
> : emacs, and construct the string from that value.
>
> This'll return the offset suffix (if that's what you want) when
> executed in your local time zone (presumably "-04:00"):
>
> (defun Price/local-time-offset-from-iso-date (y-m-d)
>   (let* ((ymd (mapcar (lambda (s) (string-to-number s)) (split-string
> y-m-d "-")))
>  (offsecs (nth 8
>(decode-time
> (apply #'encode-time
>(list 59 59 23 (nth 2 ymd) (nth 1 ymd) (nth
> 0 ymd)))
> (format "%s%02d:%02d"
> (if (> offsecs 0) "+" "-")
> (/ offsecs 3600)
> (% offsecs 3600
>
> On Unix, this'll always work.  On Windows, it works most of the time,
> but may fail in the weeks around switches from and to daylight saving.
>
> Thomas
>

[O] parsing time strings from properties

[Matt Price]

I use a very simple property in my assignment descriptions to set the
assignment due date:

:DUE_AT: 2019-09-26

I then pass this to an API which expects a date parameter like
`2019-09-26T23:59:59-04:00`. Since I only work in one time zone, I  can
just concat the property value with the additional text, though actually I
have to change the `-04:00` string twice a year.  I'm wondering though how
hard it would be to get the current time zone -- or the time zone that the
course is taught in -- from emacs, and construct the string from that
value.

Basically, I want a simple date representation to be interpreted as "the
last possible moment on this date i nthe appropriate time zone". I have not
tried to use timestamps here, in part because I'm more comfortable dealing
with text than with the horrors of time representations in either lisp or
javascript. Alsoi, I find it very very fast to insert a  text string, and
just a little bit slower and more of an interruption to add a timestamp.  I
just wanted to ask how other people manage this kind of operation; maybe I
should make the effort to start using DEADLINE timestamps. I don't really
use them in my own time management so I'm not well-versed in how to manage
them.

In nay case, I as always appreciate all of your help.

Matt

[O] ox-html: add option to restore old src block behaviour?

[Matt Price]

reiterating a question I posted to an old thread that may have gotten lost.

Over the summer, commit ded3d27b1468b878197e5fe55a70c5e13350ea27 by Nik
Clayton was merged to master. It's a one-line change that adds new ~~
tags around each lin of code in html export of source blocks.  It's useful
because it allows individual lines to be addressed directly by CSS.

However, at least one very common syntax highlighter,
https://highlinghtjs.org, expects just a single  tag, as do other
common CSS frameworks. These often leave odd borders or background color
disruptions which somewhat distort the view of the code. There's also a
funny conflict with `org-re-reveal`, which expects the old behaviour (see
https://gitlab.com/oer/org-re-reveal/issues/27).  It would in principal be
possible to fix these issues directly in CSS, but it might be much more
practical to have an option -- a defvar or a file/headline-settable
property -- that restores the old behaviour when desired.  If this would be
welcome, I could do it. I know org already has a bewildering number of
options,though,and the code change would be oddly a number of times as
large as the substantive change of that commit, os thought I'd check first.
Thanks!

Matt

Re: [O] org-map-entries but with arguments?

[Matt Price]

On Wed., Sep. 18, 2019, 9:42 p.m. John Kitchin, 
wrote:

> You can get an alist of all the properties in an entry with
> org-entry-properties, and then you can let-alist these, or do something
> else. Here is an example that might be related.
>
> * test
>   :PROPERTIES:
>   :some-random-property: True
>   :END:
>
> #+BEGIN_SRC emacs-lisp :results code
> (org-entry-properties)
> #+END_SRC
>
> #+RESULTS:
> #+begin_src emacs-lisp
> (("CATEGORY" . "2019-09-18 21:33")
>  ("SOME-RANDOM-PROPERTY" . "True")
>  ("BLOCKED" . "")
>  ("FILE" . "/Users/jkitchin/Box
> Sync/kitchingroup/jkitchin/journal/2019/09/18 21:33/2019-09-18 21:33.org")
>  ("PRIORITY" . "B")
>  ("ITEM" . "test"))
> #+end_src
>
> #+BEGIN_SRC emacs-lisp
> (let-alist (cl-loop for (key . value) in (org-entry-properties)
>collect (cons (intern key) value))
>  .SOME-RANDOM-PROPERTY)
> #+END_SRC
>
> #+RESULTS:
> : True
>
> John
>

Jeez, thanks John. For some reason when I first tried org-entry-properties
I thought it was only reporting back the standard properties, like ITEM.
And in any case I had forgotten how to convert strings to symbols. This is
enormously helpful.

>
> ---
> Professor John Kitchin
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
>
>
>
> On Wed, Sep 18, 2019 at 9:11 PM Matt Price  wrote:
>
>>
>> Sorry, replied to Adam directly by accident.
>>
>> On Wed, Sep 18, 2019 at 8:32 PM Matt Price  wrote:
>>
>>>
>>>
>>> On Wed, Sep 18, 2019 at 5:31 PM Adam Porter  wrote:
>>>
>>>> Matt Price  writes:
>>>>
>>>> > Is there a lisp trick for adding arguments to the function called by
>>>> > `org-map-entries`?
>>>> >
>>>> > I have the following function:
>>>> >
>>>> > (cl-defun org-lms-return-all-assignments ( (send-all nil)
>>>> (also-mail nil) (post-to-lms t) )
>>>> >   "By default mail all subtrees 'READY' to student recipients, unless
>>>> SEND-ALL is non-nil.
>>>> > In that case, send all marked 'READY' or 'TODO'."
>>>> >   (interactive)
>>>> >   (message "Mailing all READY subtrees to students")
>>>> >   (let ((send-condition
>>>> >  (if send-all
>>>> >  `(or (string= (org-element-property :todo-keyword item)
>>>> "READY")
>>>> >   (string= (org-element-property :todo-keyword item)
>>>> "TODO") )
>>>> >`(string= (org-element-property :todo-keyword item)
>>>> "READY")
>>>> >)))
>>>> > (org-map-entries
>>>> >  #'ol-send-just-one))
>>>> >   (org-cycle-hide-drawers 'all))
>>>> >
>>>> > I'd like to relay some of hte functions arguments to the one called
>>>> > internally to do the work.  ~(ol-send-just-one~ takes an ~also-mail~
>>>> > and a ~post-to-lms~ parameter,just like
>>>> > ~org-lms-return-all-assignments~, but I'm not sure how to trick
>>>> > org-map-entries into passing those arguments on. Any hints?  Thank
>>>> > you!
>>>>
>>>> Hi Matt,
>>>>
>>>> If I may, I think org-ql can help you here.  It should also work much
>>>> faster than org-map-entries, because it can skip to entries with the
>>>> desired to-do keywords (although you could also use the MATCH argument
>>>> to org-map-entries to improve its speed).  Try this function (untested):
>>>>
>>>> #+BEGIN_SRC elisp
>>>> (cl-defun org-lms-return-all-assignments-ql ( (send-all nil)
>>>> (also-mail nil) (post-to-lms t))
>>>>   "By default mail all subtrees 'READY' to student recipients, unless
>>>> SEND-ALL is non-nil.
>>>> In that case, send all marked 'READY' or 'TODO'."
>>>>   (interactive)
>>>>   (message "Mailing all READY subtrees to students")
>>>>   (let ((todo-keywords (if send-all
>>>>'("READY" "TODO")
>>>>  '("READY"
>>>> (org-ql-select (current-buffer)
>>&g

Re: [O] org-map-entries but with arguments?

[Matt Price]

Sorry, replied to Adam directly by accident.

On Wed, Sep 18, 2019 at 8:32 PM Matt Price  wrote:

>
>
> On Wed, Sep 18, 2019 at 5:31 PM Adam Porter  wrote:
>
>> Matt Price  writes:
>>
>> > Is there a lisp trick for adding arguments to the function called by
>> > `org-map-entries`?
>> >
>> > I have the following function:
>> >
>> > (cl-defun org-lms-return-all-assignments ( (send-all nil)
>> (also-mail nil) (post-to-lms t) )
>> >   "By default mail all subtrees 'READY' to student recipients, unless
>> SEND-ALL is non-nil.
>> > In that case, send all marked 'READY' or 'TODO'."
>> >   (interactive)
>> >   (message "Mailing all READY subtrees to students")
>> >   (let ((send-condition
>> >  (if send-all
>> >  `(or (string= (org-element-property :todo-keyword item)
>> "READY")
>> >   (string= (org-element-property :todo-keyword item)
>> "TODO") )
>> >`(string= (org-element-property :todo-keyword item) "READY")
>> >)))
>> > (org-map-entries
>> >  #'ol-send-just-one))
>> >   (org-cycle-hide-drawers 'all))
>> >
>> > I'd like to relay some of hte functions arguments to the one called
>> > internally to do the work.  ~(ol-send-just-one~ takes an ~also-mail~
>> > and a ~post-to-lms~ parameter,just like
>> > ~org-lms-return-all-assignments~, but I'm not sure how to trick
>> > org-map-entries into passing those arguments on. Any hints?  Thank
>> > you!
>>
>> Hi Matt,
>>
>> If I may, I think org-ql can help you here.  It should also work much
>> faster than org-map-entries, because it can skip to entries with the
>> desired to-do keywords (although you could also use the MATCH argument
>> to org-map-entries to improve its speed).  Try this function (untested):
>>
>> #+BEGIN_SRC elisp
>> (cl-defun org-lms-return-all-assignments-ql ( (send-all nil)
>> (also-mail nil) (post-to-lms t))
>>   "By default mail all subtrees 'READY' to student recipients, unless
>> SEND-ALL is non-nil.
>> In that case, send all marked 'READY' or 'TODO'."
>>   (interactive)
>>   (message "Mailing all READY subtrees to students")
>>   (let ((todo-keywords (if send-all
>>'("READY" "TODO")
>>  '("READY"
>> (org-ql-select (current-buffer)
>>   `(todo ,@todo-keywords)
>>   :action `(ol-send-just-one ,also-mail ,post-to-lms
>> #+END_SRC
>>
>> OK, this is pretty cool, thank you.  I took John's excellent suggestion
> of using a headline property to store the appropriate actions, but it makes
> sense to switch to org-ql if I can master the syntax (which seems awfully
> powerful).  One questions: does org-ql-select respect buffer narrowing?
> That would be important for me.
>
> Man, hard to hold all this stuff in my head.  ANd very hard to navigate my
> own code now that I see how ugly it is.
>

Another question.  In place of a function or sexp,  the :action key accepts
the keyword "element" as a value, and will return a parsed headline. Is it
possible to then pass that value on to a function that will be evaluated?
I'm asking because I have a bunch of functions with very long `let`
sections in which information is extracted from a headline with
(org-entry-get). It would be nice to use John's plist trick (from the other
thread we're on) to, essentially, let-plist all the properties of the
headline. It would declutter my code significantly.

Re: [O] lisp: scoping vars in repetitive defuns

[Matt Price]

This is fun, thanks John. I really like the plist version put would also
like to loop through the variables in a let statement somehow.

I think what I'm missing is the equivalent of a javascript implicit
destructuring construct:

let { } = object;

which will define new variables prop1, prop2... forever enumerable property
of the object.  Is thre away to do that kind of destructuring bind -- which
binds *everything* in the plist, without knowing the symbol names in
advance? that would be really great.



On Wed, Sep 18, 2019 at 8:48 AM John Kitchin 
wrote:

> You don't really need a macro for this I think. I see it leads to pretty
> clean looking code, but as you noted at the expense of edebuggable
> functions. I don't think you need the lexical-let in your macro though.
> With empty arguments I am not sure it does anything.
>
> Here are some other approaches to do what you want. I wasn't
> able to test these, but I think they are close to correct and would
> provide edebuggable functions for you. The plist is more flexible and
> future proof, you can add anything you want to it, and it won't mess up
> existing functions. The first approach you can add what you want to the
> end of the list, but you can't change the order without (probably)
> messing existing functions up.
>
> #+BEGIN_SRC emacs-lisp
> (require 'f)
>
> (defun dh-variables ()
>   "Return a list of variables for the problem at point."
>   (let* ((gh (org-entry-get (point) "GITHUB"))
>  (base (org-entry-get (point) "ORG_LMS_ASSIGNMENT_DIRECTORY"))
>  (findFiles (list
>  (format "Reflection/%s.md" gh)
>  (format "students/%s.md" gh)))
>  (browseFiles '("index.html"))
>  (testOutput "TestResults/testresults.html")
>  (testCommand "MARKING=instructor npm test"))
> (list gh base findFiles browseFiles testOutput testCommand)))
>
> ;; Here we assign all the values from the function above to variable names.
> (defun dh-view ()
>   "Open viewable files in browser"
>   (destructuring-bind
>   (gh base findFiles browseFiles testOutput testCommand)
>   (dh-variables)
> (loop for f in browseFiles
>   do
>   (browse-url-of-file (f-join base f)
>
>
> ;; alternative with a plist
> (defun dh-variables ()
>   "Return a plist of variables for the problem at point."
>   (let* ((gh (org-entry-get (point) "GITHUB"))
>  (base (org-entry-get (point) "ORG_LMS_ASSIGNMENT_DIRECTORY"))
>  (findFiles (list
>  (format "Reflection/%s.md" gh)
>  (format "students/%s.md" gh)))
>  (browseFiles '("index.html"))
>  (testOutput "TestResults/testresults.html")
>  (testCommand "MARKING=instructor npm test"))
> (list :gh gh
>   :base base
>   :findFiles findFiles
>   :browseFiles browseFiles
>   :testOutput testOutput
>   :testCommand testCommand)))
>
>
> (defun dh-view ()
>   "Open viewable files in browser"
>   (loop for f in (plist-get (dh-variables) :browseFiles)
> do
> (browse-url-of-file (f-join base f
>
>
> ;; Or with the plist as an argument
> (defun dh-view (var-plist)
>   "Open viewable files in browser"
>   (loop for f in (plist-get var-plist :browseFiles)
> do
> (browse-url-of-file (f-join base f
>
>
> ;; called like this (dh-view (dh-variables))
> #+END_SRC
>
> Matt Price  writes:
>
> > On Tue, Sep 17, 2019 at 8:46 AM John Kitchin 
> > wrote:
> >
> >> I don't totally understand what you are trying to do here.
> >>
> >
> > I think the explanation was a little unclear!
> >
> >
> >> If this were Python, it sounds like you want some kind of class that
> >> stores a variable and reuses it several different functions? Something
> kind
> >> of similar to that in elisp is a closure, which is like what you
> described.
> >> For example, here, we define a variable a, and then define two functions
> >> that use it persistently.
> >>
> >>
> >> I think you can wrap this in a macro to make new functions, e.g.
> >>
> >> #+BEGIN_SRC emacs-lisp
> >> (defmacro f-maker (a)
> >>   `(lexical-let ((a ,a))
> >>  (defun f1 (x)
> >>(* a x))
> >>
> >>  (defun f2 (x)
> >>(+ a x
> >>
> >> (f-maker 3)
> >>
> >> (list (f1

[O] org-map-entries but with arguments?

[Matt Price]

Is there a lisp trick for adding arguments to the function called by
`org-map-entries`?

I have the following function:

(cl-defun org-lms-return-all-assignments ( (send-all nil)
(also-mail nil) (post-to-lms t) )
  "By default mail all subtrees 'READY' to student recipients, unless
SEND-ALL is non-nil.
In that case, send all marked 'READY' or 'TODO'."
  (interactive)
  (message "Mailing all READY subtrees to students")
  (let ((send-condition
 (if send-all
 `(or (string= (org-element-property :todo-keyword item)
"READY")
  (string= (org-element-property :todo-keyword item)
"TODO") )
   `(string= (org-element-property :todo-keyword item) "READY")
   )))
(org-map-entries
 #'ol-send-just-one))
  (org-cycle-hide-drawers 'all))

I'd like to relay some of hte functions arguments to the one called
internally to do the work.  ~(ol-send-just-one~ takes an ~also-mail~ and a
~post-to-lms~ parameter,just like ~org-lms-return-all-assignments~, but I'm
not sure how to trick org-map-entries into passing those arguments on. Any
hints?  Thank you!

Re: [O] lisp: scoping vars in repetitive defuns

[Matt Price]

On Tue, Sep 17, 2019 at 8:46 AM John Kitchin 
wrote:

> I don't totally understand what you are trying to do here.
>

I think the explanation was a little unclear!


> If this were Python, it sounds like you want some kind of class that
> stores a variable and reuses it several different functions? Something kind
> of similar to that in elisp is a closure, which is like what you described.
> For example, here, we define a variable a, and then define two functions
> that use it persistently.
>
>
> I think you can wrap this in a macro to make new functions, e.g.
>
> #+BEGIN_SRC emacs-lisp
> (defmacro f-maker (a)
>   `(lexical-let ((a ,a))
>  (defun f1 (x)
>(* a x))
>
>  (defun f2 (x)
>(+ a x
>
> (f-maker 3)
>
> (list (f1 2) (f2 2))
> #+END_SRC
>
> #+RESULTS:
> | 6 | 5 |
>
> This is basically what I want, except it turned out to be easier to just
wrap the body forms in a let *within*the function.  THis is what I came up
with:

#+BEGIN_SRC emacs-lisp

(defmacro dh-factory (name body  docstring)
"A helper macro that sets up the environment to simplify defining multiple
functions with the same environment variables. NAME will bcome the functin
name, BODY is a list containing the lisp forms specific to the function,
and DOCSTRING is an optional ... docstring.  NAME wil lbe wrapped in a
`let` statement setting all the remelvant variables."
  `(lexical-let (())
 (defun ,name ,()
   ,docstring
   (interactive)
   (let* ((gh (org-entry-get (point) "GITHUB"))
 (base (org-entry-get (point) "ORG_LMS_ASSIGNMENT_DIRECTORY"))
 (findFiles `( ,(concat "Reflection/" gh ".md") ,(concat
"students/" gh ".json")))
 (browseFiles `( "index.html" ))
 (testOutput "TestResults/testresults.html")
 (testCommand "MARKING=instructor npm test"))
 ,@body

(dh-factory dh-find-files
((dolist (f findFiles)
   (message "%s" (concat base "/" f))
   (if (file-exists-p (concat base "/" f))
   (find-file-other-window (concat base "/" f) )
 (message "File %s does not exist, not opening." f)))  )
"Open gradable files in Emacs windows")

(dh-factory dh-view
((loop for f in browseFiles
do
(browse-url-of-file (concat base "/" f 
"open viewable files in browser")

;; this one should really pass a variable to allow different branches! oh
well.
(dh-factory dh-status
((magit-status base)
 (let ((currentBranch (shell-command-to-string
   (format "cd %s && git rev-parse
--abbrev-ref HEAD" base) ))
   (currentCommit (shell-command-to-string
   (format "cd %s && git rev-parse HEAD"
base
   (magit-stash-worktree
(format "Stashed random files from %s after commit %s."
currentBranch currentCommit)))
 (magit-checkout (concat gh "-master")))
"Open a magit-status buffer and check out this student's branch
in the repo")


(dh-factory dh-tests
((let ((output))
(with-temp-buffer
   (message (concat "cd " base " ; npm test"))
   (setq output (shell-command-to-string (concat "cd " base " ;
npm test"
(message "WELL, sorta made it through: %s" output)
 (browse-url-of-file (concat base testOutput
"Run tests directly from macs and view output")


#+END_SRC

I quite like it, though it's a bit unwieldy not to have the the source code
available for edebug etc. It feels a little like having a namespace; I'm
not afraid of  polluting the global namespace, but I can use these symbol
names throughout the code I'm processing via the factory.

I'm not sure it's *all* that much more efficient though...

>
> There is also a class system you can probably use like this called eieio,
> Here is one approach to doing this.
>
> #+BEGIN_SRC emacs-lisp
> (require 'eieio)
>
> (defclass Grade ()
>   ((:a :initarg :a)))
>
> (cl-defmethod gf1 ((g Grade) x)
>   (* (oref g :a) x))
>
> (cl-defmethod gf2 ((g Grade) x)
>   (+ (oref g :a) x))
>
> (let ((G (Grade :a 3)))
>   (list (gf1 G 2) (gf2 G 2)))
> #+END_SRC
>
> #+RESULTS:
> | 6 | 5 |
>
> I would love to learn to use eieio but it feels like a bit of a jump.

>
> Finally, maybe the simplest thing to do is pass this information in as an
> argument, e.g. a plist or alist, 

[O] lisp: scoping vars in repetitive defuns

[Matt Price]

I have a number of convenience functions define to help me with grading
assignments. As I go through the semester, i update all of these functions
modestly so that they'rehelpful for grading the current assignment.

I big chunk of these simple functions is taken up just declaring variables
with (let (())) forms.  Each function uses some ofhte same variables, e.g:

(defun dh-find-files ()
  (interactive)
  (let* ((base (org-entry-get (point) "ORG_LMS_ASSIGNMENT_DIRECTORY"))
 (gh (org-entry-get (point) "GITHUB"))
(f2o `( ,(concat "Reflection/" gh ".md") ,(concat "students/" gh
".json"  "01/index.html" "02/index.html" "03/style.css"
"04/style.css"
(message "%s" f2o)
;; make more flexible for resubmits
(shell-command (concat "cd " base " && git checkout " gh "-master"))
(dolist (x f2o)
  (if (file-exists-p (concat base "/" x))
  (find-file-other-window (concat base "/" x) )
(message "File %s does not exist, not opening." x)

(defun dh-tests ()
  (interactive)
  (let* ((base (org-entry-get (point) "ORG_LMS_ASSIGNMENT_DIRECTORY" ))
 (gh (org-entry-get (point) "GITHUB")))
(with-temp-buffer (shell-command (concat "cd " base " && npm test") t))
;; the "t" lets us suppress buffer
(browse-url-of-file (concat base "/TestResults/testresults.html"))
;; (dh-mocha-run)

))

--

This semester I changed some elements of my workflow and I had to update
all the (org-entry-get) calls to new values.  It makes me think the code is
less maintainable than it could be.  I would like to do something like this:

(lexical-let ((base `(org-entry-get (point) "ORG_LMS_ASSIGNMENT_DIRECTORY")
   (gh `(org-entry-get (point) "GITHUB")) )
(defun dh-find-files ()
(with-temp-buffer (shell-command (concat "cd " base " && npm test") t)) ;;
the "t" lets us suppress buffer
(browse-url-of-file (concat base "/TestResults/testresults.html")


Obviously it doesn't work this way. But is there any way to set macros like
this to be expanded later inside a function definition? I feel certain
there must be...

Thanks,

Matt

Re: [O] (no subject)

[Matt Price]

I'm replying a year late to this thread, post-merge, because I've
discovered it breaks interaction with the highlight.js library(
https://highlightjs.org/) and therefore my workflow, which depends on
highlightjs via reveal.js.

My difficulty is with the current version of ~org-html-do-format-code~,
which currently surrounds each loc with a ~~ tag and potentially some
classes:

(format "%s"
  (if num-start
  (format " data-ox-html-linenr=\"%s\"" line-num)
"")
  loc)



On Mon, Oct 15, 2018 at 4:07 AM Nik Clayton  wrote:

> Hi,
>
> I'd like to propose a couple of changes / enhancements to how org-export
> exports some data in to HTML files to make it slightly easier to style
> those files.
>
> The first is re line-numbers.
>
> At the moment those get exported as content in the HTML, although they're
> really additional metadata. Amongst other things, this means that if you
> copy/paste from the output you get the line numbers included in the text
> that's copied.
>
> CSS supports arbitrary counters that can be associated with content,
> starting from an arbitrary value. My current hack that sort of works is the
> following CSS:
>  .
> and a change to org-html-do-format-code to wrap each line in its own
> ... element:
>
> ...
>   ;; Transcoded src line.
>   (format "%s" loc)
> ...
>
> a) Does that sound reasonable?
> b) Should this replace the current approach, or be an option that can be
> toggled by a customisation?
>
> I agree that thisseems to make sense for many applications. But
highlight.js is a very widely-used highlighting library that expects
exactly one ~~ tag inside a given ~~ block. The additional
~~ tags  that org now generates on html export break highlight in
ways I don't believe can easily be fixed. I wonder if we could revisit the
idea of adding a user option that would allow this feature to be switched
off?  If this would be of general use I can prepare a patch.

Thanks,
Matt

Re: [O] superfluous tags in html src block output

[Matt Price]

On Sat, Sep 14, 2019 at 1:53 PM Berry, Charles  wrote:

> This is newer:
>
> ===
> commit ded3d27b1468b878197e5fe55a70c5e13350ea27
> Author: Nik Clayton 
> Date:   Tue Jun 4 11:57:40 2019 +0200
>
> ox-html: Wrap each line of a source block in a code element
>
> * lisp/ox-html.el (org-html-do-format-code): Wrap each line of a
> source block
> in a code element.
>
> This makes it straightforward to add custom decorations to each line
> using CSS :before and :after properties.
>
> ===
>
> ah, man, thank you, that's very helpful.   Clearly this is the issue. I'll
have to figure out the best way to make this work with the `highlight.js`
library that reveal.js depends on for code highlighting.

>
>
> HTH,
>
> Chuck
>
>
> > On Sep 14, 2019, at 8:52 AM, Matt Price  wrote:
> >
> > I'm seeing something I hadn't noticed before in src block html exports.
> Instead of producing structures like:
> >
> > 
> > 
> > ...
> > ...
> > ...
> > 
> > 
> >
> > each individual like is wrapped in its own  tag.  In regular HTML
> exports this doesn't really affect display, but in exports to reveal using
> the highlight.js plugin, code display gets messed up.
> >
> > From what I can tell these code tags are generated in
> org-html-do-format-code, in this section which starts on line 22459 of my
> pretty recent org:
> >
> > (org-export-format-code
> >  code
> >  (lambda (loc line-num ref)
> >(setq loc
> > (concat
> >  ;; Add line number, if needed.
> >  (when num-start
> > (format "%s"
> > (format num-fmt line-num)))
> >  ;; Transcoded src line.
> >  (format "%s"
> >   (if num-start
> >   (format " data-ox-html-linenr=\"%s\"" line-num)
> > "")
> >   loc)
> >  ;; Add label, if needed.
> >  (when (and ref retain-labels) (format " (%s)" ref
> >;; Mark transcoded line as an anchor, if needed.
> >(if (not ref) loc
> > (format "%s"
> > ref loc)))
> >  num-start refs)
> >
> > This code seems to have been around for a while so I don't know whether
> this is new behaviour, but I don't think I've seen line-level  tags
> before.  Can anyone confirm?
> >
> > See also a MWE in this bug report, which is probably erroneously filed
> in the org-re-reveal repo:
> >
> > https://gitlab.com/oer/org-re-reveal/issues/27
> >
> > I'd love to know whether this is expected behaviour, or if I've gone
> wrong somewhere!
> > THanks,
> > Matt
> >
>
>
>