Re: bash code block runs slow on org 9.6

[Ihor Radchenko]

"Majzoub, Eric"  writes:

> I've got a code block that looks like the following.
>
> #+name: read_csv
> #+begin_src bash :results replace :exports none
>   cat ./file.csv | gawk -f ./awkfile.awk | tee .tmp1.dat;
> #+end_src
>
> This runs essentially instantaneously in org 9.5.5, but it takes much
> longer, about 2 seconds or more on org 9.6, using the first release
> version. Pre-release versions of 9.6 were even slower, so it seems it's
> improved. But still, it's a lot slower than 9.5.5. What can I do to
> diagnose the slowdown?

I partially solved the slowdown in
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=76a725317

There is still some slowdown originated from table alignment.  We now
use a more accurate approach to calculate string width, which gives
performance penalty. If needed, table alignment might also be optimized,
but it will not be trivial.

For diagnosing slowdowns, you may refer to
https://list.orgmode.org/orgmode/87ee2nlslt.fsf@localhost/

Let me know if the latest bugfix branch gives you acceptable
performance.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: bash code block runs slow on org 9.6

[Max Nikulin]

On 10/12/2022 01:34, Majzoub, Eric wrote:

This runs essentially instantaneously in org 9.5.5, but it takes much
longer, about 2 seconds or more on org 9.6


An example with no external dependencies:

: (require 'ob-shell)

#+begin_src bash :results replace :exports none
  seq 1 200 | tee /dev/null
#+end_src

"Code block evaluation complete (took 0.1s)."

However I see noticeable pause before results are refreshed (less than 2 
seconds). Org-9.3.1 works faster. Emacs-26.3.


Adding "| wc" makes evaluation instant, so it is parsing of output.

Re: Org functions in source blocks

[Max Nikulin]

On 10/12/2022 03:41, William Denton wrote:


#+begin_src emacs-lisp :results raw
(org-lookup-all "x" '(remote(test_table, @I$A..@II$A)) nil)
#+end_src


There is nothing wrong with `org-lookup-all'. It just does not receive 
arguments you expect. Try


#+begin_src emacs-lisp :results raw
(list "x" '(remote(test_table, @I$A..@II$A)) nil)
#+end_src

#+RESULTS:
(x (remote (test_table (, @I$A..@II$A))) nil)

@I$A and the "remote" function are available in table formulas only. 
Moreover you quoted remote, so this s-expression is not evaluated. In 
addition, unlike in table formulas, in elisp function arguments are 
separated by space, not by comma.


Use :var to pass test_table to a source block

#+begin_src emacs-lisp :results raw :var data=test_table
data
#+end_src

#+RESULTS:
(( A) (! A) ( x) ( ) ( x) ( x) (# 3))

Unfortunately there are no convenient shortcuts for range selectors like 
"@I$A..@II$A" (at least ready to use). You should use some functions 
instead.

Org functions in source blocks

[William Denton]

I ran org-lookup-all in a source block, but it didn't give any output.  I think 
I don't know something basic about Org or Lisp (likely both) and I hope someone 
can point out what's going on.


For example, I have this table, and I can count how many times x appears:

#+name: test_table
|   | A |
|---+---|
| ! | A |
|   | x |
|   |   |
|   | x |
|   | x |
|---+---|
| # | 3 |
#+TBLFM: $2='(length(org-lookup-all "x" '(@I..@II) nil))

I can do the count from another table, referring to a remote table:

#+name: summary
| A |
|---|
| 3 |
#+TBLFM: @2$1='(length(org-lookup-all "x" '(remote(test_table, @I$A..@II$A)) 
nil))

But running org-lookup-all on its own like this gives result nil, not something 
with some xs in it.  Why not?


#+begin_src emacs-lisp :results raw
(org-lookup-all "x" '(remote(test_table, @I$A..@II$A)) nil)
#+end_src


Thanks,

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

Re: bug#59882: Multiple versions of Org in load-path problem

[Eli Zaretskii]

> From: David Masterson 
> Cc: Eli Zaretskii ,  "mic...@schi.nz" ,
>   "emacs-orgmode@gnu.org" ,  "59...@debbugs.gnu.org"
>  <59...@debbugs.gnu.org>
> Date: Fri, 09 Dec 2022 11:51:29 -0800
> 
> > https://www.gnu.org/software/emacs/manual/html_node/elisp/Autoload-by-Prefix.html
> >
> > What you observed will NOT happen if you first run:
> >
> > (clrhash  definition-prefixes)
> 
> Ah! Thank you.  In many ways, I still think of Emacs from what I learned
> about it in the early 80s (yes, pre-Elisp).  I didn't expect this and,
> so, didn't investigate the possibility that it's planned (the Info
> documentation has gotten huge).  Apologies to Eli.

No apologies needed, we all learn something new about Emacs every day.

Re: bug#59882: Multiple versions of Org in load-path problem

[David Masterson]

"Cook, Malcolm"  writes:

>> Eli Zaretskii  writes:
>> 
>> >> Cc: emacs-orgmode@gnu.org, 59...@debbugs.gnu.org
>> >> From: David Masterson 
>> >> Date: Thu, 08 Dec 2022 13:56:03 -0800
>> >>
>> >> In my testing, I found a strange case where, in *scratch*, I get:
>> >>
>> >> (message "%s" org-version)
>> >> ;; Error undefined
>> >> ;; Do 'C-h v org-version'
>> >> (message "%s" org-version)
>> >> 9.3
>> >>
>> >> So, 'describe-variable' on org-version causes Org to be loaded?!?
>> >
>> > Why is it so surprising?  Is this the first time you see that a Help
>> > command loads something to do its job?
>> 
>> Yes, it is surprising.  How did Emacs know to load Org to find org-version? 
>
> This is provided for via emacs' autoload by prefix mechanism.
>
> See:
>
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Autoload-by-Prefix.html
>
> What you observed will NOT happen if you first run:
>
>   (clrhash  definition-prefixes)

Ah! Thank you.  In many ways, I still think of Emacs from what I learned
about it in the early 80s (yes, pre-Elisp).  I didn't expect this and,
so, didn't investigate the possibility that it's planned (the Info
documentation has gotten huge).  Apologies to Eli.

-- 
David Masterson

RE: Multiple versions of Org in load-path problem

[Cook, Malcolm]

> Adding this to bug #59882
> 
> "Michel Schinz"  writes:
> 
> > Just for the record, I also ran into problems when installing Org 9.6
> > using Emacs' package system on top of an older version that came with
> > Emacs. If I tried to install it as usual (M-x list-packages, then
> > install the package from there), I had errors during compilation
> > related to `org-assert-version`, and then if I restarted Emacs, I
> > would get a fatal error in an unrelated package.
> >
> > I managed to solve that problem by:
> > 1. uninstalling Org 9.6 and exiting Emacs, 2. starting Emacs with -q,
> > 3. installing Org 9.6 from there (using M-x list-packages as usual),
> > 4. restarting Emacs.
> 
> Interesting!  I tried this (essentially) and it worked for my case.  In my 
> case, I
> had a built-in Org-9.3 and I was trying to use list-packages to install 
> Org-9.6. I
> checked that using -q still added Org-9.3 to the load-path, but, since Org
> wasn't loaded, the install via list-packages worked.
> 
> The question is what's the proper way of doing this without '-q'?
> 
> > I'm not sure this is related to your problem, or whether that helps
> > (but I hope it does)...
> 
> I think it does.
> 
> Side note:
> 
> In my testing, I found a strange case where, in *scratch*, I get:
> 
> (message "%s" org-version)
> ;; Error undefined
> ;; Do 'C-h v org-version'
> (message "%s" org-version)
> 9.3
> 
> So, 'describe-variable' on org-version causes Org to be loaded?!?  [Cook, 
> Malcolm] 

Yes.  See 
https://www.gnu.org/software/emacs/manual/html_node/elisp/Autoload-by-Prefix.html

But it will NOT be autoloaded if you first:

(clrhash  definition-prefixes)

HTH & YMMV,

Malcolm

> Why do I
> have a feeling this is related to this bug?

Yeah, I'm not sure what you consider the bug to be now.  Remind me?

> 
> --
> David Masterson

[BUG] org-ctrl-c-ctrl-c on keyword makes entire buffer visible

[Matt Lundin]

After upgrading to org 9.6, I find that refreshing the org buffer with
org-ctrl-c-ctrl-c causes the entire buffer (including drawers and other
folded items) to become visible.

I've tested this with a minimal config and the following org file:

--8<---cut here---start->8---
#+title: A buffer

* Test
:LOGBOOK:
- Note taken on [2022-12-09 Fri 11:50] \\
  A note
:END:
--8<---cut here---end--->8---

What I expect to see after typing C-c C-c on the #+title keyword:

--8<---cut here---start->8---
#+title: A buffer

* Test...
--8<---cut here---end--->8---

Instead, what I see is the entire buffer including the drawers.

In a large buffer with many drawers, this causes problems because the
drawers remain open even after cycling visibility locally or globally.

Git bisect suggests that the problem first appeared with
commit b8a0ddf52f4d5b5a3293affb45d1e8fd0426a7f0.

Thanks,

Matt

RE: bug#59882: Multiple versions of Org in load-path problem

[Cook, Malcolm]

> Eli Zaretskii  writes:
> 
> >> Cc: emacs-orgmode@gnu.org, 59...@debbugs.gnu.org
> >> From: David Masterson 
> >> Date: Thu, 08 Dec 2022 13:56:03 -0800
> >>
> >> In my testing, I found a strange case where, in *scratch*, I get:
> >>
> >> (message "%s" org-version)
> >> ;; Error undefined
> >> ;; Do 'C-h v org-version'
> >> (message "%s" org-version)
> >> 9.3
> >>
> >> So, 'describe-variable' on org-version causes Org to be loaded?!?
> >
> > Why is it so surprising?  Is this the first time you see that a Help
> > command loads something to do its job?
> 
> Yes, it is surprising.  How did Emacs know to load Org to find org-version? 

This is provided for via emacs' autoload by prefix mechanism.

See:

https://www.gnu.org/software/emacs/manual/html_node/elisp/Autoload-by-Prefix.html

What you observed will NOT happen if you first run:

(clrhash  definition-prefixes)

> didn't request it to by this action.  Or did it do a package-activate-all (or
> similar) to ensure that everything was loaded?
> That would be overkill.  Sometimes the best answer is 'nil'.
> 
> --
> David Masterson

bash code block runs slow on org 9.6

[Majzoub, Eric]

I've got a code block that looks like the following.

#+name: read_csv
#+begin_src bash :results replace :exports none
  cat ./file.csv | gawk -f ./awkfile.awk | tee .tmp1.dat;
#+end_src

This runs essentially instantaneously in org 9.5.5, but it takes much
longer, about 2 seconds or more on org 9.6, using the first release
version. Pre-release versions of 9.6 were even slower, so it seems it's
improved. But still, it's a lot slower than 9.5.5. What can I do to
diagnose the slowdown?

BTW, I'm using emacs 29.0.60, but it's the same result in emacs
30.0.50. My CSV file is pretty small, maybe 100 lines at most, and the
awk script runs essentially instantaneously on the file in an xterm.

Eric

Re: [BUG] Header-args :results output of code block don't take precedence above headline header-args [9.6 (release_9.6-32-gc59d6d @ /tmp/org/org-mode/lisp/)]

[Berry, Charles]

Rens,

This is not a bug. Seee below.

> On Dec 7, 2022, at 9:47 AM, Rens Oliemans  wrote:
> 
> Hi all,
> 
> A code block with ":results output" as header argument still has its
> output silenced if it's inside a headline with ":results none". MWE:
> 
> 1. Create an .org file with the following content:
> 
> * Sub-tree with =:results none=
> :PROPERTIES:
> :header-args: :results none
> :END:
> 
> #+BEGIN_SRC bash :results output
>  echo "is silent"
> #+END_SRC


This is the correct handling.

See (info "(org) Results of Evaluation").

`output' is a `Collection' option. `none' is a `Handling' option. 

If you want to override a `Handling' option you need another `Handling' option. 
e.g.

#+BEGIN_SRC bash :results output replace
  echo "is silent"
#+END_SRC

HTH,

Chuck

Keys to Category Filter Agenda

[Colin Baxter]

In agenda, you can filter by category by pressing '<' when point on that
particular category. To remove filtering, press '<' again. All this is
explained in the org-mode info. However the initial agenda buffer,
viewed by 'C-c a', gives the keys '<' for filtering but '>' to remove
the restriction. This surely is wrong since '>' is the key to call the
org-agenda-date-prompt, or am I missing something very basic here?

Nest wishes,

Colin Baxter.

Re: Multiple versions of Org in load-path problem

[Sharon Kimble]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Tim Cross  writes:

> David Masterson  writes:
>
>> Tim Cross  writes:
>>
>>> David Masterson  writes:
>>>
 "Michel Schinz"  writes:

> Just for the record, I also ran into problems when installing Org 9.6
> using Emacs' package system on top of an older version that came with
> Emacs. If I tried to install it as usual (M-x list-packages, then
> install the package from there), I had errors during compilation related
> to `org-assert-version`, and then if I restarted Emacs, I would get a
> fatal error in an unrelated package.
>
> I managed to solve that problem by:
> 1. uninstalling Org 9.6 and exiting Emacs,
> 2. starting Emacs with -q,
> 3. installing Org 9.6 from there (using M-x list-packages as usual),
> 4. restarting Emacs.

 Interesting!  I tried this (essentially) and it worked for my case.  In
 my case, I had a built-in Org-9.3 and I was trying to use list-packages
 to install Org-9.6. I checked that using -q still added Org-9.3 to
 the
 load-path, but, since Org wasn't loaded, the install via list-packages
 worked.

 The question is what's the proper way of doing this without '-q'?

This is what is in my 'init.el' file -

=
;;; init.el --- sharon's config -*- eval: (read-only-mode 1) -*-
;; Make sure that Git version of Org mode is being loaded instead of the 
built-in version.
;;; (add-to-list 'load-path (expand-file-name 
"/home/boudiccas/.emacs.d/elpa/org-9.5.5"))
(add-to-list 'load-path (expand-file-name 
"/home/boudiccas/.emacs.d/elpa/org-9.6"))
(add-to-list 'load-path (expand-file-name 
"/home/boudiccas/git/org-contrib/lisp"))

(require 'package)
(setq package-enable-at-startup nil)
(package-initialize)

(require 'ob-tangle)
(org-babel-load-file "/home/boudiccas/.emacs.d/config22-2.org")
=

And the first thing that is loaded is org-9.6 *before* anything else! And you 
could equally have org-mode loading from a git repository on your hard drive, 
just by changing the filepath.

Super-easy without having to faff about with 'emacs -q' etc.

And as you can see I'm using org-9.6 from elpa, and I didn't move over to it 
until this morning, so far so good.  

Thanks
  Sharon.  
  
>>
>> [...]
>>
>>> I don't think there is any safe way to install an updated version of
>>> org-mode other than
>>>
>>> 1. Use the -q approach outlined above
>>
>> Thinking about it, this only works if Org is in elpa as melpa (etc.) are
>> not added to package-archives.  You'd have to do some handwritten elisp
>> out of *scratch* to setup package-archives if Org-9.6 was still coming
>> out of melpa. That's why this can only be labeled as a hack and not a
>> solution.
>>
>
> Well, yes, if your going to use this technique to load a package which
> is not in the default package archives you would need to add that
> archive first. People who use this technique often just have a
> 'update.el' file which they load/evaluate when starting Emacs with -q. 
>
>
>>> 2. Craft your init.el file such that org functionality is only loaded
>>> when explicitly requested and always update as the first action after
>>> starting emacs.
>>
>> In this case, something happened in package-install when trying to
>> install Org-9.6 with a built-in Org-9.3.  During the compilation check
>> (.el -> .elc) many files failed because the new 'org-assert-version'
>> macro was not defined.  Sort of like, after package-install started
>> working on Org-9.6, org-macs.el (where org-assert-version should be) got
>> loaded *before* the new load-path had been set causing it to load the
>> old one from 9.3.  Thereafter, everything went awry. 
>>
>
> You must have some custom code in your init.el or are using something
> like use-package as package.el doesn't try to install or upgrade
> packages during init by default.
>
> Where people can come undone is when they are using use-package and set
> the :ensure t option. In this case, use-package will not know about the
> bundled version and will attempt to install org from ELPA. If use
> package runs after org has already been loaded (possibly because some
> other package has been loaded which depends on/requires org mode and has
> loaded the bundled version) then things will break because you will end
> up with a mixed version install. This is why I always ensure org is the very
> first use-package in my init.el and it comes before any other code which
> loads or initialises anything.  
>
>>> The first approach is actually the easiest. The second is hard to get
>>> right and very fragile because packages like use-package and more
>>> specifically, other packages with leverage off org functionality, make it
>>> impossible to reliably know exactly when org is loaded.
>>
>> Using ':after" in use-package is supposed to help that, but I'm not sure
>> it is reliable.  Packages are often incomplete about what other packages
>> it depends on.
>>
>
> You cannot rely on 

Re: bug#59882: Multiple versions of Org in load-path problem

[Eli Zaretskii]

> From: David Masterson 
> Cc: mic...@schi.nz,  emacs-orgmode@gnu.org,  59...@debbugs.gnu.org
> Date: Thu, 08 Dec 2022 23:43:05 -0800
> 
> Eli Zaretskii  writes:
> 
> >> So, 'describe-variable' on org-version causes Org to be loaded?!?
> >
> > Why is it so surprising?  Is this the first time you see that a Help
> > command loads something to do its job?
> 
> Yes, it is surprising.  How did Emacs know to load Org to find
> org-version?  I didn't request it to by this action.  Or did it do a
> package-activate-all (or similar) to ensure that everything was loaded?
> That would be overkill.  Sometimes the best answer is 'nil'.

I don't think Emacs does package-activate-all.  But Help commands
frequently load packages to provide the documentation of the symbols
you ask about.  If this is surprising, then I suggest to get used to
it.

30.0.50; Failing to match org-agenda properties

[Brent Westbrook]

I sent this to the bug-gnu-emacs list first (bug#59899), but it was
suggested to send it here instead:

In emacs -Q, with a simple org agenda file like:

* TODO try out org mode
SCHEDULED: <2022-12-07 Wed 23:00>

in /tmp/test.org, setting

(setq org-agenda-files '("/tmp/test.org"))

and running M-x org-agenda m and typing the queries SCHEDULED>=""
or SCHEDULED="", I get no results, but if I do
SCHEDULED<="" or SCHEDULED<"", it does work. This is
pretty minor in this example, but for potentially large agenda files,
this could cause a lot of work. For example, in the org-alert package I
help maintain, we use `org-map-entries` filtered with a MATCH argument
like

"SCHEDULED>=\"\"+SCHEDULED<\"\"|DEADLINE>=\"\"+DEADLINE<\"\""

to select only upcoming events. I noticed that I didn't get
notifications for events on my agenda today, and this query now failing
seems to be the root cause. Even simplifying it to the first part
"SCHEDULED>=\"\"", as in the example above, fails to return any
results. I've also tried SCHEDULED<"" alone as a query, but it
yields scheduled items from today and tomorrow.

I've checked this page
(https://orgmode.org/manual/Matching-tags-and-properties.html) of the
org manual and the breaking changes section of the 9.6 release notes but
can't see what I'm doing wrong. I'd be happy to update my queries if
that's the issue, but at least right now I think there might be a bug in
this tag-matching code. I might poke around and try to submit a patch
tomorrow if I can find anything.

I just rebuilt my emacs from the master branch to make sure this hasn't
been fixed in the past couple of days and am still able to reproduce it
on commit 5d68cc99177569a661da0ac75bb7df1ec4098fda. I think it was
working before I updated to the version with org 9.6, but I'm not
entirely sure.

Thanks,

Brent Westbrook

[BUG] Header-args :results output of code block don't take precedence above headline header-args [9.6 (release_9.6-32-gc59d6d @ /tmp/org/org-mode/lisp/)]

[Rens Oliemans]

Hi all,

A code block with ":results output" as header argument still has its
output silenced if it's inside a headline with ":results none". MWE:

1. Create an .org file with the following content:

* Sub-tree with =:results none=
:PROPERTIES:
:header-args: :results none
:END:

#+BEGIN_SRC bash :results output
  echo "is silent"
#+END_SRC

2. Use a minimal config, attached is the one I used. Bug is present on
at least Org 9.6 and 9.5

3. Evaluate the code block(s) and see that their output is silenced.

Attached is a file much like this, but with an additional code block
with a #+HEADER style argument. Also included is the minimal-org.el I
used to reproduce this.

I tried bisecting this, but I can't find a good commit that's recent
enough to compile on my machine without changing Emacs versions. This
might be a clue that this is not a bug but a configuration issue on my
side, but I did my best to rule that out. This is the first time posting
in a (and by extension, this) mailing list, please highlight any errors
I made/things I can do better.

All the best,
Rens Oliemans

Emacs : GNU Emacs 28.1 (build 1, x86_64-redhat-linux-gnu, GTK+ Version
 3.24.34, cairo version 1.17.6) of 2022-07-21
Package: Org mode version 9.6 (release_9.6-32-gc59d6d @
 /tmp/org/org-mode/lisp/)

* Sub-tree with =:results none=
:PROPERTIES:
:header-args: :results none
:END:

#+BEGIN_SRC bash :results output
  echo "is silent"
#+END_SRC

#+HEADER: :results output
#+BEGIN_SRC bash
  echo "is also silent"
#+END_SRC
;; Add latest Org mode to load path.
(add-to-list 'load-path (expand-file-name "/tmp/org/org-mode/lisp"))

;; Relevant settings for bug repro
(org-babel-do-load-languages
 'org-babel-load-languages
 '((shell . t)))

Re: Macro: exporting roman numerals formatted as small-caps

[Christian Moe]

Max Nikulin writes:

> On 08/12/2022 19:38, Carlos Martínez wrote:
>> #+MACRO: sc (eval (if (org-export-derived-backend-p
>> org-export-current-backend 'latex) (concat "@@latex:\\textsc{@@" $1
>> "@@latex:}@@") (concat "@@odt:> text:style-name=\"T1\">@@"$1"@@odt:@@")))
>
>^  ^
> Your missed spaces around $1, but it is unlikely a problem.
>
> Disclaimer: I know almost nothing about odt. Have you customized ODT
> styles? I do not see "T1" in etc/styles/OrgOdtStyles.xml
>
> info "(org) Advanced topics in ODT export"
> https://orgmode.org/manual/Advanced-topics-in-ODT-export.html

Indeed. Carlos, I'm guessing that you formatted a Roman numeral in ODT
as you wanted it to look, saved the document, opened content.xml and
used the resulting XML.

However, what happens then is that the formatting gets saved as an
automatically numbered text style "T1", which isn't going to mean the
same thing the next time you export a document.

See if this works:

- In the document you are using as your ODT_STYLES_FILE, type a Roman
  numeral and format it the way you like.

- Then make it a named character style. In LibreOffice, open the "Styles
  and formatting" sidebar and select the "Character styles" tab. Select
  the formatted Roman numeral in the text. From the "Styles actions"
  dropdown (top right in "Styles and formatting"), choose "New style
  from selection". Give it a name, e.g. "Roman numeral".

- Save the styles file.

- In your macro: replace "T1" with "Roman numeral".

- In all your Org documents where you want this to work: make sure that
  (1) the macro is defined and (2) that ODT_STYLES_FILE points to the
  file you have saved with the style definition for "Roman numeral".

There isn't any simpler way to do it, due to the way LibreOffice stores
all local formatting as generated styles.

Yours,
Christian