[Bug] isearch errors when org-fold-core-style is 'overlays

[Matt Lundin]

I'm finding that isearch fails to unfold the correct region or to search
in the correct region when there are folded regions in a buffer and
`org-fold-core-style` is set to 'overlays.

Here is a minimal recipe for reproducing the bug:

Use a minimal emacs startup file:

--8<---cut here---start->8---
(add-to-list 'load-path "~/org-mode/lisp/")
(setq org-startup-folded t)
(setq org-fold-core-style 'overlays)
--8<---cut here---end--->8---

Open the following file:

--8<---cut here---start->8---
* One
word
* Two
word
--8<---cut here---end--->8---

Issue #1:

Go to the beginning of headline "Two" when the trees are folded.

Type "M-x isearch word [RET]".

Expected behavior: isearch should reveal the entry under headline two
and shift the highlighted region dynamically as the characters typed
begin to match.

What happens: the beginning of the highlighted region remains stuck at
the beginning of the headline and the entry does not unfold until after
the return key is pressed.

Issue #2:

Cycle headlines to folded state. Go to the beginning of headline "Two".

Type "M-x isearch word [RET]".

Note that during this second search dynamically highlighted region
expands to include the folded headline "One". Once return is pressed,
isearch reveals the entry under headline "One" and moves the point to
the "word" there. 

After running git bisect, I traced the issue to commit
6cd7c6fb1cf6363f1057086760bed9875cdd97c7

Thanks,

Matt

Re: [BUG] org-mouse.el breaks org-open-at-point

[Matt Lundin]

Ihor Radchenko  writes:

> Matt Lundin  writes:

>> - What currently happens if org-mouse has been loaded:
>>
>>   - calling "C-c C-o" with the keyboard cycles the visibility of the
>> entry

> Patches welcome!
>
> I think we need to modify `org-open-at-mouse' directly instead of
> using advice.

I can give this a try.

The advice in org-mouse.el currently does the following:

 - cycles when on headline
 - toggles checkbox when on checkbox
 - cycles plain lists when on a list item

Would it make any sense to make this part of the default behavior or
org-open-at-mouse? 

If not, I think I can make it dependent on the value of
org-mouse-features, thus giving user the option of turning off the extra
mouse click behavior via that variable (as a workaround to avoid
undesired side effects of calling interactive help). This would be a
temporary workaround until the org-mouse is converted to a global minor
mode.

Matt

Re: Autoloading side effects (was: Re: [BUG] org-mouse is activated without explicit require)

[Matt Lundin]

Max Nikulin  writes:

> I am afraid, requirement that loading should not cause side effects is 
> too strict and it will make usage inconvenient. Examples:
> - ol-info calls `org-link-set-parameters'.
> - org-protocol installs an advice for `server-visit-files'. (It should 
> not be necessary, but currently Emacs does not provide a better option.)
>
> However I agree that it is undesired when attempt to call interactive 
> help causes side effects. I would say that in such cases loading of 
> packages should be sandboxed and should not propagate to global scope.

For reference, another module that causes global side effects when
calling interactive help is org-ctags.el. It advises `visit-tags-table`
and `xref-find-definitions` and adds a hook to `org-mode-hook`.

Matt

[BUG] org-mouse.el breaks org-open-at-point

[Matt Lundin]

The org-mouse feature advises org-open-at-point in a way that overrides
the desired behavior of calling "M-x org-open-at-point" on a headline.

See the advice function org--mouse-open-at-point. It seems the intended
behavior of this advice is to allow the user to click on headline stars
and cycle the visibility of an entry. However, there is no test to see
if the user is indeed clicking with a mouse.

As a result, this advice also overrides org-open-at-point even when one
is typing.

- Expected behavior 

  - calling org-open-at-point with the keyboard "C-c C-o" should produce
a list of links in the entry as advertised in the docstring ("When
point is on a headline, display a list of every link in the entry,
so it is possible to pick one, or all, of them.")

- What currently happens if org-mouse has been loaded:

  - calling "C-c C-o" with the keyboard cycles the visibility of the
entry

Thanks,

Matt

[BUG] org-mouse is activated without explicit require

[Matt Lundin]

I'm not too familiar with how emacs handles autoloads. However, I find
org-mouse is automatically loaded if I call describe-function and then
tab complete after typing "", "org-", "org-m", etc. This seems enough to
load the entirely of the org-mouse.el file.

The problem is that this adds a lambda function to org-mode-hook that
activates all org-mouse functionality at the next call of org-mode or
org-mode-restart. This has two unexpected effects:

1. It changes the behavior of mouse clicks without the user's explicit
   request.
2. It advise org-open-at-point, with the result that it is no longer
   possible to get a list of all links in an entry when calling
   org-open-at-point on a headline. (See separate bug report on this.)

I am able to reproduce this with a minimal .emacs config and the most
recent git version (after running "make clean && make").

Steps to reproduce:

1. /usr/bin/emacs -Q -l ~/minimal.el

where ~/minimal.el is...

--8<---cut here---start->8---
(add-to-list 'load-path "~/org-mode/lisp/")
--8<---cut here---end--->8---

2. C-h f org-m [tab]

Note that "org-mouse-" functions appear in the completion list. Even if
one aborts describe-function at this point, org-mouse has been loaded
and org-mode-hook now contains a lambda function to activate the
functionality of org-mouse (org-defkey, advice-add, etc.).

3. Open an org buffer for the first time or call "M-x org-mode-restart"
   in an already opened buffer

"C-h v org-open-at-point" now reports the following advice:

--8<---cut here---start->8---
This function has :around advice: ‘org--mouse-open-at-point’.
--8<---cut here---end--->8---

Also notice that org-mouse features are now active in org buffers.

Thanks,

Matt

[BUG] org-ctrl-c-ctrl-c on keyword makes entire buffer visible

[Matt Lundin]

After upgrading to org 9.6, I find that refreshing the org buffer with
org-ctrl-c-ctrl-c causes the entire buffer (including drawers and other
folded items) to become visible.

I've tested this with a minimal config and the following org file:

--8<---cut here---start->8---
#+title: A buffer

* Test
:LOGBOOK:
- Note taken on [2022-12-09 Fri 11:50] \\
  A note
:END:
--8<---cut here---end--->8---

What I expect to see after typing C-c C-c on the #+title keyword:

--8<---cut here---start->8---
#+title: A buffer

* Test...
--8<---cut here---end--->8---

Instead, what I see is the entire buffer including the drawers.

In a large buffer with many drawers, this causes problems because the
drawers remain open even after cycling visibility locally or globally.

Git bisect suggests that the problem first appeared with
commit b8a0ddf52f4d5b5a3293affb45d1e8fd0426a7f0.

Thanks,

Matt

Re: [PATCH] manual: fix keybinding for org-force-cycle-archived

[Matt Lundin]

I just noticed also that the name has been changed to
org-cycle-force-archived and that the key index reference was incorrect.
I've attached an additional patch to fix these in the manual.

Best,

Matt

>From 8412f476e98c3271fac88a170e26ad033501c3d6 Mon Sep 17 00:00:00 2001
From: Matt Lundin 
Date: Sun, 22 May 2022 15:13:09 -0500
Subject: [PATCH 2/2] manual: Fix function name and key index reference

* doc/org-manual.org: Change name to `org-cycle-force-archived` to
avoid obsolete name and fix key index reference.
---
 doc/org-manual.org | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 686412d84..2f5f48120 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -7421,9 +7421,9 @@ The following commands help manage the =ARCHIVE= tag:
   child.  If point is /not/ on a headline when this command is
   invoked, check the level 1 trees.
 
-- {{{kbd(C-c C-TAB)}}} (~org-force-cycle-archived~) ::
+- {{{kbd(C-c C-TAB)}}} (~org-cycle-force-archived~) ::
 
-  #+kindex: C-TAB
+  #+kindex: C-c C-TAB
   Cycle a tree even if it is tagged with =ARCHIVE=.
 
 - {{{kbd(C-c C-x A)}}} (~org-archive-to-archive-sibling~) ::
-- 
2.36.1

[PATCH] manual: fix keybinding for org-force-cycle-archived

[Matt Lundin]

* doc/org-manual.org: Fix a keybinding in Internal archiving
---
 doc/org-manual.org | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 7cba5f18d..686412d84 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -7373,7 +7373,7 @@ its location in the outline tree, but behaves in the 
following way:
   #+vindex: org-cycle-open-archived-trees
   It does not open when you attempt to do so with a visibility cycling
   command (see [[*Visibility Cycling]]).  You can force cycling archived
-  subtrees with {{{kbd(C-TAB)}}}, or by setting the option
+  subtrees with {{{kbd(C-c C-TAB)}}}, or by setting the option
   ~org-cycle-open-archived-trees~.  Also normal outline commands, like
   ~org-show-all~, open archived subtrees.
 
-- 

Re: A test

[Matt Lundin]

I'm sorry for this spam. I was experimenting with org-msg mode. Hence
the nonsense prose in the email.

In particular, I was attempting to ensure that mail to mailing list
addresses would automatically switch to plain text instead of html
upon address completion. But I forgot to remove the Org Mode mailing
list address before sending.

My sincerest apologies!

Matt

Matt Lundin  writes:

> This is not cool. Really
>
> Matt Lundin  writes:
>
>> Dear so and so,
>>
>> Not again. What's with all this craziness?
>>
>> Now there.
>>
>> Goodbye.
>>
>> OK - now we are talking - we're going to make this into a good day.
>>
>> Donec vitae dolor.
>>
>>  Item   Amount 
>> ---
>>  Shoes  80 
>>  Socks  10 
>> ---
>> 90 

Re: A test

[Matt Lundin]

This is not cool. Really

Matt Lundin  writes:

> Dear so and so,
>
> Not again. What's with all this craziness?
>
> Now there.
>
> Goodbye.
>
> OK - now we are talking - we're going to make this into a good day.
>
> Donec vitae dolor.
>
>  Item   Amount 
> ---
>  Shoes  80 
>  Socks  10 
> ---
> 90 

Re: [BUG] incorrect (and slow) indentation of logbook entries

[Matt Lundin]

Matt Lundin  writes:

> Commit e3b79ad2bf7ab7b91c0ad2b8383d639bfe154ce7 from Feb. 9, 2020 (Allow
> a new value for `org-adapt-indentation') introduced a bug that causes
> logbooks to be incorrectly indented when promoting an entry. As a
> result, there are also significant slowdowns when promoting and demoting
> an entry with hundreds of logbook entries.

I did some more investigation and think the bug may lie deeper. The
commit above changed the functions used in org-fixup-indentation. Prior
to the commit org-fixup-indentation called org-indent-line on each line
within a LOGBOOK. Now it calls org-indent-region on the entire LOGBOOK.
So the problems seem to lie in org-indent-region.

To test this, I manually called org-indent-region on a region containing
a LOGBOOK. I saw the same indentation problems and slowness as I
reported in the previous email. Here are the profiling results from
calling org-indent-region on a LOGBOOK containing 303 lines (mix of
notes, clock drawers, state changes):

org-indent-region 303 
20.374798015  0.0672435578
org-element-at-point  655 
6.7059473859  0.0102380876
org-element--parse-to 654 
6.6907969839  0.0102305764
org-element--current-element  36791   
6.3040008640  0.0001713462
org-element--list-struct  3372
3.6317395729  0.0010770283
org--get-expected-indentation 355 
3.004313850.0084628559
org-element-clock-parser  2738
0.7868292469  0.0002873737
org-element-item-parser   29700   
0.7527248339  2.534...e-05
org-element-timestamp-parser  2738
0.7245477349  0.0002646266
org-parse-time-string 5476
0.6267072680  0.0001144461
org-element--collect-affiliated-keywords  4026
0.2572965799  6.390...e-05
org-element-drawer-parser 654 
0.1057650070  0.0001617201
org-element-plain-list-parser 3372
0.0736009140  2.182...e-05
org-get-limited-outline-regexp9063
0.0575772250  6.352...e-06
org-at-heading-p  8083
0.0267149439  3.305...e-06
org-element--cache-put36791   
0.0176797030  4.805...e-07
org-element-paragraph-parser  327 
0.011128276   3.403...e-05

[BUG] incorrect (and slow) indentation of logbook entries

[Matt Lundin]

Commit e3b79ad2bf7ab7b91c0ad2b8383d639bfe154ce7 from Feb. 9, 2020 (Allow
a new value for `org-adapt-indentation') introduced a bug that causes
logbooks to be incorrectly indented when promoting an entry. As a
result, there are also significant slowdowns when promoting and demoting
an entry with hundreds of logbook entries.

For instance, when running org-promote-subtree on an entry with 300+
entries, elp-profiling showed that org-indent-region is called 305 times
for a total of 29.4 seconds.

org-indent-region305 29.465412747  0.0966079106

By contrast, prior to the above commit (e3b79ad) org-promote-subtree
calls org-indent-region only 2 times on the same subtree:

org-indent-region2   0.006902560.00345128

I haven't included the full logbook here for obvious reasons. But to
reproduce the indentation bug, promote the following entry with
org-promote-subtree:

--8<---cut here---start->8---
** TODO Do this
   SCHEDULED: <2020-07-18 Sat .+1d>
   :PROPERTIES:
   :LAST_REPEAT: [2020-07-18 Sat 11:14]
   :END:
   :LOGBOOK:
   - State "DONE"   from "TODO"   [2020-07-17 Fri 23:59]
   - State "DONE"   from "NOW"[2020-07-16 Thu 10:19]
   CLOCK: [2020-07-16 Thu 09:22]--[2020-07-16 Thu 10:19] =>  0:57
   - State "DONE"   from "NOW"[2020-07-13 Mon 15:11]
   CLOCK: [2020-07-13 Mon 14:52]--[2020-07-13 Mon 15:11] =>  0:19
   - State "DONE"   from "NOW"[2020-07-12 Sun 14:44]
   CLOCK: [2020-07-12 Sun 14:09]--[2020-07-12 Sun 14:44] =>  0:35
   - State "DONE"   from "TODO"   [2020-07-10 Fri 16:07]
   - State "DONE"   from "NOW"[2020-07-07 Tue 13:30]
   CLOCK: [2020-07-07 Tue 13:21]--[2020-07-07 Tue 13:30] =>  0:09
   - State "DONE"   from "NOW"[2020-07-06 Mon 09:32]
   :END:
--8<---cut here---end--->8---

After calling org-promote-subtree, notice that the first CLOCK entry and
all subsequent entries are incorrectly indented:

--8<---cut here---start->8---
* TODO Do this
  SCHEDULED: <2020-07-18 Sat .+1d>
  :PROPERTIES:
  :LAST_REPEAT: [2020-07-18 Sat 11:14]
  :END:
  :LOGBOOK:
  - State "DONE"   from "TODO"   [2020-07-17 Fri 23:59]
  - State "DONE"   from "NOW"[2020-07-16 Thu 10:19]
CLOCK: [2020-07-16 Thu 09:22]--[2020-07-16 Thu 10:19] =>  0:57
- State "DONE"   from "NOW"[2020-07-13 Mon 15:11]
CLOCK: [2020-07-13 Mon 14:52]--[2020-07-13 Mon 15:11] =>  0:19
- State "DONE"   from "NOW"[2020-07-12 Sun 14:44]
CLOCK: [2020-07-12 Sun 14:09]--[2020-07-12 Sun 14:44] =>  0:35
- State "DONE"   from "TODO"   [2020-07-10 Fri 16:07]
- State "DONE"   from "NOW"[2020-07-07 Tue 13:30]
CLOCK: [2020-07-07 Tue 13:21]--[2020-07-07 Tue 13:30] =>  0:09
- State "DONE"   from "NOW"[2020-07-06 Mon 09:32]
  :END:
--8<---cut here---end--->8---

Best,

Matt

[BUG] org-fill-paragraph fills paragraph before selected region

[Matt Lundin]

When there is an active region, the function org-fill-paragraph does not
limit itself to the active region but also fills the paragraph before
the active region.

Steps to reproduce. Use the following file. Mark the last two entry
paragraphs, beginning at "Lorem" and ending at the end of the buffer
("erat."). Type 'M-q' (org-fill-paragraph).

--8<---cut here---start->8---
* This is a test

  Aliquam erat volutpat.  Nunc eleifend leo vitae magna.  In id erat non orci 
commodo lobortis.  Proin neque massa, cursus ut, gravida ut, lobortis eget, 
lacus.  Sed diam.  Praesent fermentum tempor tellus.  Nullam tempus.  Mauris ac 
felis vel velit tristique imperdiet.  Donec at pede.  Etiam vel neque nec dui 
dignissim bibendum.  Vivamus id enim.  Phasellus neque orci, porta a, aliquet 
quis, semper a, massa.  Phasellus purus.  Pellentesque tristique imperdiet 
tortor.  Nam euismod tellus id erat.

  Lorem ipsum dolor sit amet, consectetuer adipiscing elit.  Donec hendrerit 
tempor tellus.  Donec pretium posuere tellus.  Proin quam nisl, tincidunt et, 
mattis eget, convallis nec, purus.  Cum sociis natoque penatibus et magnis dis 
parturient montes, nascetur ridiculus mus.  Nulla posuere.  Donec vitae dolor.  
Nullam tristique diam non turpis.  Cras placerat accumsan nulla.  Nullam 
rutrum.  Nam vestibulum accumsan nisl.

  Aliquam erat volutpat.  Nunc eleifend leo vitae magna.  In id erat non orci 
commodo lobortis.  Proin neque massa, cursus ut, gravida ut, lobortis eget, 
lacus.  Sed diam.  Praesent fermentum tempor tellus.  Nullam tempus.  Mauris ac 
felis vel velit tristique imperdiet.  Donec at pede.  Etiam vel neque nec dui 
dignissim bibendum.  Vivamus id enim.  Phasellus neque orci, porta a, aliquet 
quis, semper a, massa.  Phasellus purus.  Pellentesque tristique imperdiet 
tortor.  Nam euismod tellus id erat.
--8<---cut here---end--->8---

This is the result. Note that the first paragraph, which is outside the
selected region, is also filled.

--8<---cut here---start->8---
* This is a test

  Aliquam erat volutpat. Nunc eleifend leo vitae magna. In id erat non
  orci commodo lobortis. Proin neque massa, cursus ut, gravida ut,
  lobortis eget, lacus. Sed diam. Praesent fermentum tempor tellus.
  Nullam tempus. Mauris ac felis vel velit tristique imperdiet. Donec
  at pede. Etiam vel neque nec dui dignissim bibendum. Vivamus id
  enim. Phasellus neque orci, porta a, aliquet quis, semper a, massa.
  Phasellus purus. Pellentesque tristique imperdiet tortor. Nam
  euismod tellus id erat.

  Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Donec
  hendrerit tempor tellus. Donec pretium posuere tellus. Proin quam
  nisl, tincidunt et, mattis eget, convallis nec, purus. Cum sociis
  natoque penatibus et magnis dis parturient montes, nascetur
  ridiculus mus. Nulla posuere. Donec vitae dolor. Nullam tristique
  diam non turpis. Cras placerat accumsan nulla. Nullam rutrum. Nam
  vestibulum accumsan nisl.

  Aliquam erat volutpat. Nunc eleifend leo vitae magna. In id erat non
  orci commodo lobortis. Proin neque massa, cursus ut, gravida ut,
  lobortis eget, lacus. Sed diam. Praesent fermentum tempor tellus.
  Nullam tempus. Mauris ac felis vel velit tristique imperdiet. Donec
  at pede. Etiam vel neque nec dui dignissim bibendum. Vivamus id
  enim. Phasellus neque orci, porta a, aliquet quis, semper a, massa.
  Phasellus purus. Pellentesque tristique imperdiet tortor. Nam
  euismod tellus id erat.
--8<---cut here---end--->8---

Best,

Matt

Re: [BUG] All drawers except property drawers are open at startup

[Matt Lundin]

Matthew Lundin  writes:

> Nicolas Goaziou  writes:
>
>> Matt Lundin  writes:
>>
>>> All drawers default to open at startup except for property drawers. This
>>> includes :LOGBOOK: drawers, which can be quite long and block view of an
>>> entry.
>>
>> I think all drawers are opened at startup, including property drawers.
>>
>>> I ran a git bisect and discovered that the bug was introduced with
>>> commit 8b05c06d427e850d45684f69c5165cd7684e1071 on May 9. 
>>
>> Isn't it related to 88f5ed91c5cf9cb6962c1b441eac7dbab9a4734e instead?
>>
> No. I have the following in my emacs file:
>
> (setq org-startup-folded t)
>

I did some more investigating and it seems my initial git bisect was off
by one commit. (I think I forgot to reload org files on the last
bisect.)

The commit that introduced this bug was
1027e0256903bc2c4ef9edfb1f7279294fa3f195.

In the function org-set-startup-visibility, that commit replaced

(org-cycle-hide-drawers 'all)

with

(org-cycle-hide-property-drawers 'all)

Thus, only property drawers are hidden at startup.

Best,

Matt

[BUG] org-agenda-filter and hyphens in category names

[Matt Lundin]

Hi everyone,

I'm just getting around to exploring the function org-agenda-filter
introduced last year. (Due to busyness, I was running an older version
of Org Mode last year for fear of breaking features I relied on.)

I like the tab completion. However, I'm finding that the completion for
categories often fails because of hyphens in my file and category names.
Let's say I have a file named "my-file.org" with active todos. The
function org-agenda-filter will suggest "my-file" as a possible category
completion. However, if I tab complete "my-file," org-agenda-filter
errors out with the following messages:

  ‘+my’ filter ignored because tag/category is not represented
  ‘-file’ filter ignored because tag/category is not represented

The same thing occurs when I put hyphens in filetags.

Looking at the Org Manual, I see that Org Mode restricts tags to
alphanumeric characters, "@", and "_". And org-set-tags-command will
sanitize tag names when entering them, replacing forbidden characters
with ":".

However, there are no guidelines about forbidden characters in
categories. And filetags at the top of file allow for arbitrary
characters. Both org-agenda-filter-by-category ("<") and
org-agenda-filter-by-tag ("\") work with categories and filetags
containing "-" or "+" in the name.

Would it be possible for org-agenda-filter to escape the set of special
characters [-+<>=/] in category and filetag names when offering them for
completion? 

Best,

Matt

[BUG] All drawers except property drawers are open at startup

[Matt Lundin]

All drawers default to open at startup except for property drawers. This
includes :LOGBOOK: drawers, which can be quite long and block view of an
entry.

I ran a git bisect and discovered that the bug was introduced with
commit 8b05c06d427e850d45684f69c5165cd7684e1071 on May 9. 

Matt

[PATCH] Fix moving cursor in org-set-tags-command

[Matt Lundin]

Commit 44ec473c199262d89b372d8a6cd35bed7672164d from Feb. 23 causes
org-set-tags-command to move the cursor forward 1 char when situated on
headline asterisks.

So if I am on the following level 1 headline with the cursor on the
asterisk as below...

* Headline
^

...and I call org-set-tags command, it moves the cursor forward one space:

* Headline  :tag:
 ^

This is causes problems with org-speed-keys, which requires that the
cursor remain on the headline.

This commit modified a previous change on Feb. 21
(450452de4b790706d187291f9f71a286f8f62004). But that commit also had
problems, since it would move the cursor one asterisk forward on
headlines > 1, thus also interfering with org-speed-keys. In my view
org-set-tags-command should not move the cursor except to fix the very
specific thing that commit 450452de4b was meant to fix: namely the
cursor moving when on a blank headline: i.e., from here...

*** 
^

...to here...

*** :tag:
   ^

I've attached a patch that corrects the problem, but it would be ideal
if we figured out why the cursor is moving in the first place.

Best,

Matt

>From ae5cf0e1110241426e49f573219e9740c25bf8ea Mon Sep 17 00:00:00 2001
From: Matt Lundin 
Date: Thu, 7 May 2020 19:06:08 -0500
Subject: [PATCH 1/1] Fix bug that placed cursor incorrectly when setting tags

* lisp/org.el: (org-set-tags-command) Only fix cursor position in very
specific circumstances (i.e., when cursor is on an empty headline).
---
 lisp/org.el | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index dd017e662..0e4fd7be1 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -11846,8 +11846,9 @@ in Lisp code use `org-set-tags' instead."
 	  (org-set-tags tags)
 ;; `save-excursion' may not replace the point at the right
 ;; position.
-(when (save-excursion (skip-chars-backward "*") (bolp))
-  (forward-char
+(and (looking-at " ")
+	 (string-match "\\*+" (buffer-substring (point-at-bol) (point)))
+	 (forward-char
 
 (defun org-align-tags ( all)
   "Align tags in current entry.
-- 
2.26.2

[Bug] org-store-link should not insert a document level ID property

[Matt Lundin]

The introduction of document-level property drawers (commit
1bdff9f73dc1e7ff625a90e3e61350bdea99f29c from May 2019) introduced
inconsistencies in the behavior of org-id and org-store-link.

If org-id-link-to-org-use-id is set to t or 'create-if-interactive,
calling org-store-link above the first headline in an org file will
insert a PROPERTY drawer and an ID at top of the file, so that the file
(call it "~/test.org") looks like this:

--8<---cut here---start->8---
:PROPERTIES:
:ID:   1f43e860-9e7b-4c8f-82b9-6ed3352e589f
:END:

* First headline
--8<---cut here---end--->8---

However, the link that Org actually stores is "[[file:~/test.org]]", so
the ID is irrelevant.

In addition, a link to a document-level ID does not work. Following
"[[id:1f43e860-9e7b-4c8f-82b9-6ed3352e589f]]" results in this error:

user-error: Before first headline at position 14 in buffer test.org

So either: 

1. org-id and org-store-link/org-open-link should support document level
   ids in a user-configurable way, or
2. org-id-get-create should detect whether it is above the first heading
   and should not create an id

Option #2 would obviously be the easier fix.

Gustav: were IDs within the scope of your initial thinking about
document level properties? Or are these largely irrelevant?

Best,
Matt

[O] [BUG] Duplicate characters in help string when filtering by tag

[Matt Lundin]

I'm finding that when I call org-agenda-filter-by-tag, the string of
character hints is long, with lots of duplicates. E.g.,

Filter by tag [hpcwhpcwhpcwhpcwhpcwhpcwhpcwhpcwhpcwhpcw ]:tag-char, [TAB]:tag, 
[/]:off, [+/-]:filter/exclude, [q]:quit [3 times]

A git bisect reveals that this bug began with commit
2056b60aae528d0ba663d8b3804dc69e98993f1c. After a bit of investigating,
I've found the cause: org--tag-add-to-alist no longer removes duplicate
tags if they belong in a tag group. 

Let me provide a minimal example to illustrate. Take the following
setting:

(setq org-tag-alist
  '((:startgroup)
("home" . ?h)
("phone" . ?p)
("computer" . ?c)
("work" . ?w)
(:endgroup)))

If one has 10 agenda files (none of which has a local #+TAGS
declaration), org-tag-alist-for-agenda will end up containing those tags
multiple times:

--8<---cut here---start->8---
org-tag-alist-for-agenda is a variable defined in ‘org.el’.
Its value is
(#1=(:startgroup)
#2=("home" . 104)
#3=("phone" . 112)
#4=("computer" . 99)
#5=("work" . 119)
#6=(:endgroup)
#1# #2# #3# #4# #5# #6# #1# #2# #3# #4# #5# #6# #1# #2# #3# #4# #5# #6# #1# 
#2# #3# #4# #5# #6# #1# #2# #3# #4# #5# #6# #1# #2# #3# #4# #5# #6# #1# #2# #3# 
#4# #5# #6# #1# #2# #3# #4# #5# #6# #1# #2# #3# #4# #5# #6#)
--8<---cut here---end--->8---

As a result, the help prompt for org-filter-tags-for-agenda looks like
this (notice the duplicate characters):

Filter by tag [hpcwhpcwhpcwhpcwhpcwhpcwhpcwhpcwhpcwhpcw ]:tag-char, [TAB]:tag, 
[/]:off, [+/-]:filter/exclude, [q]:quit [3 times]

Best,
Matt

Re: [O] [PATCH] Ensure org-get-tags includes all local tags

[Matt Lundin]

>> With commit fbe56f89f75a8979e0ba48001a822518df2c66fe, the function
>> org-get-tags incorrectly removes uninherited tags from the list of tags
>> it returns, *even if* they are local tags.
>>
>> Expected behavior: org-get-tags should always return local tags,
>> regardless of whether they are excluded from inheritance. The variable
>> org-tags-exclude-from-inheritance should only apply to tags in parent
>> heading or to file tags.
>>
>> Actual behavior: if a local tag is in org-tags-exclude-from-inheritance,
>> org-get-tags will not return it.
>>
>> I've attached a patch that fixes the issue.
>
> You're right. Thank you.
>
> Could you add a regression test for this and push the change?
>

Hi Nicolas,

Sorry for the delay. I finally had time to improve this fix (my first
fix caused another regression) and add a regression test. At the moment,
I don't have push rights to the Org Mode repo, so I'm attaching the
patch here.

Best,
Matt

>From f3dc39236e8f3940c58ab4e48f629360028131e1 Mon Sep 17 00:00:00 2001
From: Matt Lundin 
Date: Wed, 25 Jul 2018 07:48:10 -0500
Subject: [PATCH] Ensure that org-get-tags returns all local tags

* lisp/org.el: (org-get-tags) Create a clearer separation between
  local and inherited tags in the function, so that
  org-remove-uninherited tags is only run on inherited tags. This is
  important to avoid destroying existing tags when adding new tags.
* testing/lisp/test-org.el: (test-org/get-tags) Add regression test
---
 lisp/org.el  | 15 ---
 testing/lisp/test-org.el |  7 +++
 2 files changed, 15 insertions(+), 7 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 999575d05..94252489c 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -14722,14 +14722,15 @@ Inherited tags have the `inherited' text property."
 (org-with-point-at (or pos (point))
   (unless (org-before-first-heading-p)
 	(org-back-to-heading t)
-	(let ((tags (org--get-local-tags)))
-	  (if (or local (not org-use-tag-inheritance)) tags
+	(let ((ltags (org--get-local-tags)) itags)
+	  (if (or local (not org-use-tag-inheritance)) ltags
+	(setq itags org-file-tags)
 	(while (org-up-heading-safe)
-	  (setq tags (append (mapcar #'org-add-prop-inherited
-	 (org--get-local-tags))
- tags)))
-	(org-remove-uninherited-tags
-	 (delete-dups (append org-file-tags tags)
+	  (setq itags (append (mapcar #'org-add-prop-inherited
+	  (org--get-local-tags))
+  itags)))
+	(delete-dups
+	 (append (org-remove-uninherited-tags itags) ltags
 
 (defun org-get-buffer-tags ()
   "Get a table of all tags used in the buffer, for completion."
diff --git a/testing/lisp/test-org.el b/testing/lisp/test-org.el
index a6b283163..20164beb5 100644
--- a/testing/lisp/test-org.el
+++ b/testing/lisp/test-org.el
@@ -6190,6 +6190,13 @@ Paragraph"
(equal '("foo")
 	  (org-test-with-temp-text "* H1 :foo:\n* H2 :bar:"
 	(org-get-tags 1
+  ;; Make sure tags excluded from inheritance are returned if local
+  (should
+   (equal '("foo")
+	  (org-test-with-temp-text "* Test :foo:"
+(let ((org-use-tag-inheritance t)
+  (org-tags-exclude-from-inheritance '("foo")))
+	  (org-get-tags)
   ;; Pathological case: tagged headline with an empty body.
   (should (org-test-with-temp-text "* :tag:" (org-get-tags
 
-- 
2.18.0

[O] [BUG] org-special-ctrl-k no longer protects tags

[Matt Lundin]

Since commit e87ebca2a7, setting org-special-ctrl-k to t no longer
protects tags if when pressing C-k (org-kill-line) in the middle of a
headline. The expected behavior is described in the docstring for
org-special-ctrl-k: "When in the middle of the headline text, kill the
headline up to the tags."

Take the following example:

* Headline  :tag:

If I hit C-k with the cursor on the letter "l" on the example above, I
expect to get this...

* Head  :tag:

Instead, I get this...

* Head

I think this is because when (looking-at org-tag-line-re) is called on
line 22840, the cursor is in the middle of the line, whereas
org-tag-line-re expects it to be at the beginning of the line.

Best,
Matt

[O] [PATCH] Ensure org-get-tags includes all local tags

[Matt Lundin]

With commit fbe56f89f75a8979e0ba48001a822518df2c66fe, the function
org-get-tags incorrectly removes uninherited tags from the list of tags
it returns, *even if* they are local tags.

Expected behavior: org-get-tags should always return local tags,
regardless of whether they are excluded from inheritance. The variable
org-tags-exclude-from-inheritance should only apply to tags in parent
heading or to file tags.

Actual behavior: if a local tag is in org-tags-exclude-from-inheritance,
org-get-tags will not return it.

This causes problems with functions that call org-get-tags. For
instance, if org-fast-tag-selection-single-key is set,
org-set-tags-command delete all local tags that are in the list
org-tags-exclude-from-inheritance.

I've attached a patch that fixes the issue.

Best,

Matt

>From 407c9c88f7c8629ae99fed5060ef2c428b54ebd8 Mon Sep 17 00:00:00 2001
From: Matt Lundin 
Date: Tue, 5 Jun 2018 12:31:42 -0500
Subject: [PATCH] Ensure org-get-tags returns all local tags

* lisp/org.el: (org-get-tags) Fix a bug that removed local tags if
  they were in the org-tags-exclude-from-inheritance list.
---
 lisp/org.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index d2b4c26ff..8cfb64510 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -14713,8 +14713,8 @@ Inherited tags have the `inherited' text property."
 	  (setq tags (append (mapcar #'org-add-prop-inherited
 	 (org--get-local-tags))
  tags)))
-	(org-remove-uninherited-tags
-	 (delete-dups (append org-file-tags tags)
+	(delete-dups (append (org-remove-uninherited-tags org-file-tags)
+			  tags
 
 (defun org-get-buffer-tags ()
   "Get a table of all tags used in the buffer, for completion."
-- 
2.17.1

[O] [PATCH] Fix git-annex-get call in org-attach

[Matt Lundin]

I've attached a patch that fixes a minor issue in org-attach related to
git annex. Basically, it ensures that a shell command is called from
within the git-annex repository (i.e., the org-attach-directory).
Otherwise, the command results in an error because it is called in the
incorrect directory.

Best,
Matt

>From 6158eee0a36f41d90e78ae59df3cfac03f291157 Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Sun, 25 Mar 2018 22:11:09 -0500
Subject: [PATCH] Fix default directory for git-annex checks

* lisp/org-attach.el: (org-attach-annex-get-maybe) Ensure shell
  command that checks whether a git-annex file exists is called from
  within the git-annex repository. Otherwise, it will result in an
  error.
---
 lisp/org-attach.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/org-attach.el b/lisp/org-attach.el
index fdf652822..192815f4f 100644
--- a/lisp/org-attach.el
+++ b/lisp/org-attach.el
@@ -320,7 +320,8 @@ the ATTACH_DIR property) their own attachment directory."
 (defun org-attach-annex-get-maybe (path)
   "Call git annex get PATH (via shell) if using git annex.
 Signals an error if the file content is not available and it was not retrieved."
-  (let ((path-relative (file-relative-name path)))
+  (let* ((default-directory (expand-file-name org-attach-directory))
+	 (path-relative (file-relative-name path)))
 (when (and (org-attach-use-annex)
 	   (not
 		(string-equal
-- 
2.16.2

Re: [O] [PATCH] org-global-tags-completion-table does not include tags from buffers

[Matt Lundin]

Attached is a patch that fixes the issue.

Best,
Matt

Matt Lundin <m...@imapmail.org> writes:

> If org-tags-alist is customized by the user, the value returned by
> org-global-tags-completion-table does not include any tags from agenda
> buffers and files.
>
> This behavior contradicts the docstring of
> org-global-tags-completion-table, which claims to return the list of all
> tags in all agenda buffer/files.
>
> I believe this bug was introduced with commit
> 4743d43dd8e448b6c440b1e4988bcd353de60cc7 in April 2016. Before that
> commit, Org mode appended tags in org-tags-alist to tags gathered from
> the buffer. After the commit, Org mode no longer gathers buffer tags if
> org-tags-alist is defined.
>
> Line 13731 is the key line:
>
> (or org-current-tag-alist (org-get-buffer-tags)))
>
> AFAICT, org-current-tag-alist only includes tags defined in
> org-tag-alist and org-tag-persistent-alist. So if these are defined, the
> function will never gather the buffer tags.
>
> As an aside, this bug makes filtering the agenda by filetags or tags in
> org buffers impossible due to commit
> 404ac42ee51f0ac0d9cfb8fbefaefbbe96c61017, which requires a match for tag
> completion when hitting "/ [TAB]" in the agenda. Since
> org-global-tags-completion-table does not actually return the tags in
> buffers, it is impossible to filter by them.
>
> I can reproduce this with emacs -Q (emacs 25.3 and Org mode from git).
>
> Best,
> Matt

>From fc09bce2c9efe72a340132a499510658cd03bec2 Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Mon, 19 Mar 2018 09:53:15 -0500
Subject: [PATCH] Include buffer tags in global tags completion 

* lisp/org.el: (org-global-tags-completion-table): Return all tags,
  including tags in the buffer. This fixes a bug that caused buffer
  tags to be excluded if user configured tags either via org-tag-alist
  or the #+TAGS keyword.
---
 lisp/org.el | 9 +
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 73ab32aa0..44b57a60d 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -13724,10 +13724,11 @@ instead of the agenda files."
 		  (mapcar
 		   (lambda (file)
 		 (set-buffer (find-file-noselect file))
-		 (mapcar (lambda (x)
-			   (and (stringp (car-safe x))
-(list (car-safe x
-			 (or org-current-tag-alist (org-get-buffer-tags
+		 (append (org-get-buffer-tags)
+			 (mapcar (lambda (x)
+   (and (stringp (car-safe x))
+	(list (car-safe x
+ org-current-tag-alist)))
 		   (if (car-safe files) files
 		 (org-agenda-files
 
-- 
2.16.2

[O] [Bug] org-global-tags-completion-table does not include tags from buffers

[Matt Lundin]

If org-tags-alist is customized by the user, the value returned by
org-global-tags-completion-table does not include any tags from agenda
buffers and files.

This behavior contradicts the docstring of
org-global-tags-completion-table, which claims to return the list of all
tags in all agenda buffer/files.

I believe this bug was introduced with commit
4743d43dd8e448b6c440b1e4988bcd353de60cc7 in April 2016. Before that
commit, Org mode appended tags in org-tags-alist to tags gathered from
the buffer. After the commit, Org mode no longer gathers buffer tags if
org-tags-alist is defined.

Line 13731 is the key line:

(or org-current-tag-alist (org-get-buffer-tags)))

AFAICT, org-current-tag-alist only includes tags defined in
org-tag-alist and org-tag-persistent-alist. So if these are defined, the
function will never gather the buffer tags.

As an aside, this bug makes filtering the agenda by filetags or tags in
org buffers impossible due to commit
404ac42ee51f0ac0d9cfb8fbefaefbbe96c61017, which requires a match for tag
completion when hitting "/ [TAB]" in the agenda. Since
org-global-tags-completion-table does not actually return the tags in
buffers, it is impossible to filter by them.

I can reproduce this with emacs -Q (emacs 25.3 and Org mode from git).

Best,
Matt

[O] [BUG] org-clock-update-mode-line leaves link markup in mode-line

[Matt Lundin]

Commit 6655429b8d7ee686a8300b61af587599cd656a22 removed code that
stripped link markup from the string displayed in the mode line.

In the past, a heading like the following...

* TODO [[https://en.wikipedia.org/wiki/Org-mode][Org-mode]]

...would be simplified for display in the mode line as...

"[0:04] (Org-Mode)"

Now it displays as...

"[0:04] ([[https://en.wikipedia.org/wiki/Org-mode][Org-mode]])"

Is this change intentional? It is, of course, possible to strip the link
markup with a custom org-clock-heading-function. But it seems to me that
removing link markup from the mode-line display should remain the
default behavior.

Thanks,
Matt

Re: [O] [Bug] M-S Ret keybinding translated to M-Ret

[Matt Lundin]

Allen Li <vianchielfa...@gmail.com> writes:

> On Wed, Feb 7, 2018 at 1:47 PM Matt Lundin <m...@imapmail.org> wrote:
>
>> Matt Lundin <m...@imapmail.org> writes:
>
>> > I discovered that when trying to insert a new item with a checkbox using
>> > M-S Return (org-insert-todo-heading), the keybinding is "translated" to
>> > M-Return (org-meta-return). As a result, all I get is simple list item
>> > without a checkbox.
>> >
>
>> The attached patch fixes this issue.
>
> I'm pretty sure your patch would break those binding in terminal Emacs
> completely, since  cannot be typed in the terminal and Emacs
> does not remap these bindings.

Attached please find a more limited patch that fixes the issue in the
GUI without affecting TTY use. I have only changed bindings for the
M-S-RET or S-RET combinations. These combinations cannot be used in the
TTY anyway and Org Mode already defines alternative TTY bindings for
these combinations.

Best,
Matt

>From 3ec2e2b82ff64d708286cc9f3962acad56771dfb Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Fri, 9 Feb 2018 08:52:41 -0600
Subject: [PATCH] Fix S- and M-S- keybindings for GUI

This prevents these key combinations from being translated to M-RET in
window systems. These bindings already have extra keys defined for
TTY.
---
 lisp/org.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 688e48bcc..e47a7e0ac 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -18859,9 +18859,9 @@ COMMANDS is a list of alternating OLDDEF NEWDEF command names."
 (define-key org-mode-map (kbd "") #'org-shifttab)
 
  RET key with modifiers
-(org-defkey org-mode-map (kbd "S-RET") #'org-table-copy-down)
-(org-defkey org-mode-map (kbd "M-S-RET") #'org-insert-todo-heading)
-(org-defkey org-mode-map (kbd "ESC S-RET") #'org-insert-todo-heading)
+(org-defkey org-mode-map (kbd "S-") #'org-table-copy-down)
+(org-defkey org-mode-map (kbd "M-S-") #'org-insert-todo-heading)
+(org-defkey org-mode-map (kbd "ESC S-") #'org-insert-todo-heading)
 (org-defkey org-mode-map (kbd "M-RET") #'org-meta-return)
 (org-defkey org-mode-map (kbd "ESC RET") #'org-meta-return)
 
-- 
2.16.1

Re: [O] [Bug] M-S Ret keybinding translated to M-Ret

[Matt Lundin]

Allen Li <vianchielfa...@gmail.com> writes:

> On Wed, Feb 7, 2018 at 1:47 PM Matt Lundin <m...@imapmail.org> wrote:
>
>> Matt Lundin <m...@imapmail.org> writes:
>
>> > I discovered that when trying to insert a new item with a checkbox using
>> > M-S Return (org-insert-todo-heading), the keybinding is "translated" to
>> > M-Return (org-meta-return). As a result, all I get is simple list item
>> > without a checkbox.
>> >
>
>> The attached patch fixes this issue.
>
> I'm pretty sure your patch would break those binding in terminal Emacs
> completely, since 
> cannot be typed in the terminal and Emacs does not remap these
> bindings.

Thanks for pointing this out. I hope someone who knows emacs keybindings
better can take the lead here. I was just following the example of the
following commit, which uses "".

,
| commit c3fffcd8d21f248f946e2d39d4bde3ce80dcf6d4
| Author: Nicolas Goaziou <m...@nicolasgoaziou.fr>
| Date:   Mon Jan 15 22:59:40 2018 +0100
| 
| Fix C- binding
`

> Furthermore, I cannot reproduce your issue on Org mode version 9.1.6
> (9.1.6-33-g5b4a71-elpaplus)

That is because the version in elpa not have the problematic commit yet.
The elpa version you are using still uses old way of defining these
keys:

(org-defkey org-mode-map [(meta shift return)] 'org-insert-todo-heading)

The problematic commit (6965098a5357b09575a1119074545eb708cf5944)
changed that line to the following (along with several related changes):

(org-defkey org-mode-map (kbd "M-S-RET") #'org-insert-todo-heading)

Could you evaluate this latter line and see if you can reproduce the
problem.

Thanks,
Matt

Re: [O] [Bug] M-S Ret keybinding translated to M-Ret

[Matt Lundin]

Matt Lundin <m...@imapmail.org> writes:

> I discovered that when trying to insert a new item with a checkbox using
> M-S Return (org-insert-todo-heading), the keybinding is "translated" to
> M-Return (org-meta-return). As a result, all I get is simple list item
> without a checkbox.
>

The attached patch fixes this issue.

>From 240caa630ef053a27536fe9c4725d0480dc4c570 Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Wed, 7 Feb 2018 15:45:45 -0600
Subject: [PATCH] Fix return bindings

---
 lisp/org.el | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 688e48bcc..133bf038a 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -18859,11 +18859,11 @@ COMMANDS is a list of alternating OLDDEF NEWDEF command names."
 (define-key org-mode-map (kbd "") #'org-shifttab)
 
  RET key with modifiers
-(org-defkey org-mode-map (kbd "S-RET") #'org-table-copy-down)
-(org-defkey org-mode-map (kbd "M-S-RET") #'org-insert-todo-heading)
-(org-defkey org-mode-map (kbd "ESC S-RET") #'org-insert-todo-heading)
-(org-defkey org-mode-map (kbd "M-RET") #'org-meta-return)
-(org-defkey org-mode-map (kbd "ESC RET") #'org-meta-return)
+(org-defkey org-mode-map (kbd "S-") #'org-table-copy-down)
+(org-defkey org-mode-map (kbd "M-S-") #'org-insert-todo-heading)
+(org-defkey org-mode-map (kbd "ESC S-") #'org-insert-todo-heading)
+(org-defkey org-mode-map (kbd "M-") #'org-meta-return)
+(org-defkey org-mode-map (kbd "ESC ") #'org-meta-return)
 
  Cursor keys with modifiers
 (org-defkey org-mode-map (kbd "M-") #'org-metaleft)
-- 
2.16.1

[O] [Bug] M-S Ret keybinding translated to M-Ret

[Matt Lundin]

I discovered that when trying to insert a new item with a checkbox using
M-S Return (org-insert-todo-heading), the keybinding is "translated" to
M-Return (org-meta-return). As a result, all I get is simple list item
without a checkbox.

M-x describe-key [Ret] M-S Ret

...shows the following...

--8<---cut here---start->8---
M-RET (translated from ) runs the command org-meta-return
(found in org-mode-map), which is an interactive compiled Lisp
function in ‘org.el’.

It is bound to M-RET, C-c C-x RET.
 
(org-meta-return  ARG)
--8<---cut here---end--->8---

A git bisect traced the source of this problem to the following commit:

--8<---cut here---start->8---
6965098a5357b09575a1119074545eb708cf5944 is the first bad commit
commit 6965098a5357b09575a1119074545eb708cf5944
Author: Nicolas Goaziou 
Date:   Mon Jan 15 16:13:37 2018 +0100

Use a single keybinding representation

* lisp/org.el (org-replace-disputed-keys): Fix docstring.

Use `kbd' macro everywhere.  Also add missing "ESC" equivalents for
"M-" bindings.
--8<---cut here---end--->8---

Thanks,
Matt

Re: [O] What about a space when inserting a src-block?

[Matt Lundin]

Marco Wahl  writes:

> My impression is that one always types a blank after calling
> `org-insert-structure-template' {C-c C-x w s} for creating a src block.
>
> Suggestion: Realize the insert of a space after "#+begin_src" in
> `org-insert-structure-template'.
>
> WDYT?

+1

Also, before this becomes part of an official release, might I also
suggest that we consider changing the default keybinding. I am finding
the "C-c C-x w" binding a bit too close to "C-c C-x C-w," which calls
org-cut-special. I've accidentally lost a couple of subtrees without
noticing.

Matt

Re: [O] [PATCH] Preserve point when calling org-capture in gnus virtual group

[Matt Lundin]

Nicolas Goaziou <m...@nicolasgoaziou.fr> writes:

> Matt Lundin <m...@imapmail.org> writes:
>
>> When calling org-capture in gnus virtual groups (for instance, groups
>> created through searches), I found that the region captured was
>> different than the region I had selected. This is because one of the
>> functions called in org-gnus-store-link moves the point to the end of
>> the buffer. The attached patch fixes the issue.
>
> Applied. Thank you.

Thanks! And here is another patch to fix the same issue in temporary
nnir search groups in gnus.

Best,
Matt

>From 87299d4646b5c3895332e63b5c0419e8a1b7d027 Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Wed, 29 Nov 2017 12:37:07 -0600
Subject: [PATCH] Preserve point when storing link in Gnus nnir groups

* lisp/org-gnus.el: (org-gnus-store-link) This fixes a similar issue
  as commit 84b329516efbe92ea5385f2aa2ad2779ab44b427, but for
  nnir search groups in gnus.
---
 lisp/org-gnus.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/org-gnus.el b/lisp/org-gnus.el
index 27a968f75..f0f5ca2fa 100644
--- a/lisp/org-gnus.el
+++ b/lisp/org-gnus.el
@@ -123,7 +123,8 @@ If `org-store-link' was called with a prefix arg the meaning of
 		(save-excursion
 		  (car (nnvirtual-map-article (gnus-summary-article-number)
 	   (`(nnir . ,_)
-		(nnir-article-group (gnus-summary-article-number)))
+		(save-excursion
+		  (nnir-article-group (gnus-summary-article-number
 	   (_ gnus-newsgroup-name)))
 	(header (with-current-buffer gnus-summary-buffer
 		  (gnus-summary-article-header)))
-- 
2.15.0

[O] [PATCH] Preserve point when calling org-capture in gnus virtual group

[Matt Lundin]

When calling org-capture in gnus virtual groups (for instance, groups
created through searches), I found that the region captured was
different than the region I had selected. This is because one of the
functions called in org-gnus-store-link moves the point to the end of
the buffer. The attached patch fixes the issue.

Thanks,
Matt

>From 7f41fbb98c7cbaa09af8f78b06a57587e4f583a1 Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Wed, 29 Nov 2017 09:17:55 -0600
Subject: [PATCH] Preserve position of point in gnus virtual group

* lisp/org-gnus.el: (org-gnus-store-link) Add a save-excursion to
  protect the position of point. Otherwise, when calling org-capture
  in an article with a virtual group, the point is moved and the
  region captured is incorrect.
---
 lisp/org-gnus.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/org-gnus.el b/lisp/org-gnus.el
index 687bc08b1..27a968f75 100644
--- a/lisp/org-gnus.el
+++ b/lisp/org-gnus.el
@@ -120,7 +120,8 @@ If `org-store-link' was called with a prefix arg the meaning of
  (let* ((group
 	 (pcase (gnus-find-method-for-group gnus-newsgroup-name)
 	   (`(nnvirtual . ,_)
-		(car (nnvirtual-map-article (gnus-summary-article-number
+		(save-excursion
+		  (car (nnvirtual-map-article (gnus-summary-article-number)
 	   (`(nnir . ,_)
 		(nnir-article-group (gnus-summary-article-number)))
 	   (_ gnus-newsgroup-name)))
-- 
2.15.0

[O] [PATCH] Do not save target buffer after archiving subtree

[Matt Lundin]

The attached patch removes the call to save-buffer in
org-archive-subtree. This significantly reduces the time necessary to
archive a subtree, especially when archiving many subtrees in a region.
Without this patch, Org Mode will save the archive buffer after each
subtree is archived. If one were archiving a region of 50 trees, this
would result in 50 writes to disk in quick succession. This patch makes
the behavior of org-archive-subtree match that of org-refile, which does
not save the target buffer after a refile.

Best,
Matt

>From 5c10b578461879d2770a85ae69c8182942074ee7 Mon Sep 17 00:00:00 2001
From: Matt Lundin <mclun...@gmail.com>
Date: Sat, 25 Nov 2017 09:26:15 -0600
Subject: [PATCH] Do not save target buffer after archiving subtree

* lisp/org-archive.el: (org-archive-subtree) Do not save buffer after
  each archive. Saving the archive buffer after archiving each subtree
  results in substantial slowdown and many writes to disk when
  archiving an active region. This brings the behavior of
  org-archive-subtree into line with org-refile, which does not save
  the target buffer after refiling.
---
 lisp/org-archive.el | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/lisp/org-archive.el b/lisp/org-archive.el
index b2e2178dc..37c5d6f62 100644
--- a/lisp/org-archive.el
+++ b/lisp/org-archive.el
@@ -381,10 +381,7 @@ direct children of this heading."
 		 (point)
 		 (concat "ARCHIVE_" (upcase (symbol-name item)))
 		 value
-	  (widen)
-	  ;; Save and kill the buffer, if it is not the same
-	  ;; buffer.
-	  (unless (eq this-buffer buffer) (save-buffer)
+	  (widen
 	;; Here we are back in the original buffer.  Everything seems
 	;; to have worked.  So now run hooks, cut the tree and finish
 	;; up.
-- 
2.15.0

Re: [O] How to use case sensitive org-search-view?

[Matt Lundin]

alain.coch...@unistra.fr writes:

> Hello.  Sorry about this seemingly basic question, but I spent several
> hours (literally) googling and experimenting without any progress...
>
> Suppose I have an org file with
>
> * FOO
> * foo
>
> I would like to use org-search-view (C-c a s) so as to (say) list only
> the first entry.
>
> Among other things I tried using +FOO, +"FOO", +{FOO}, +{\bFOO\b},
> with case-fold-search being t or nil, with several emacs+org versions,
> but each time it lists both headlines.
>
> Is there a way to perform what I want?

Unfortunately, it looks like org-search-view is hard-coded to be
case-insensitive. I think a very hackish workaround would be to advise
org-search-view to create and then remove an "around" advice that wraps
re-search-forward in a "let" declaration setting case-fold-search to
nil. But this would be a fairly horrific hack.

In the long run, I believe the Org Mode code would need to be changed to
make this customizable.

Best,
Matt

Re: [O] function for inserting a block

[Matt Lundin]

Carsten Dominik  writes:

> I have always come down on the side of NOT breaking backward
> compatibility unless we really HAVE TO in order to make progress. The
> reason for this bias is because most Org users are not reading this
> maling list and just want the system to function and to continue to
> function the way they are used to, while it is hopefully improving. It
> will stop them from upgrading if such breakage happens too often.
>
> So I would support reimplement the expansion (including
> org-try-structure-completion for people who use that in custom code),
> if possible of course on the back of one of the built-in expansion
> systems in Emacs, before pushing this change out in a release. I would
> certainly reimplement this in some way for myself, because using these
> abbreviations is already hardcoded in my spine, I think.

I agree. I support removing redundant code behind the scenes, but I also
think we should preserve backwards compatibility in the user interface.
A fair number of people around here have been using Org Mode for more
than a decade, and, for better or for worse, everything about the user
interface is now hardwired in their brains. 

In short, we have a time-saving expansion system that works well for
lots of people. I support re-implementing it on top of another snippet
engine but also leaving it in place until a suitable replacement is
ready.

Matt

Re: [O] org-clock-in only in current buffer

[Matt Lundin]

"Julian M. Burgos"  writes:

> Dear org-moders,
>
> Until a recent update (or an unintended change in my configuration), I
> was able to clock in a task (org-clock-in) *without* org mode trying
> to find open clocks. I cannot figure out what changed, but now
> org-clock-in opens all my agenda buffers searching for open clocks,
> which is not very convenient as I have dozens of agenda files. What
> would be the best way to a) restrict the search of open clocks to the
> current buffer, or b) turn off the search of open clocks?
>

I believe you can set org-clock-auto-clock-resolution to nil to stop
the search for open clocks.

Best,
Matt

Re: [O] Custom tag exportation - Agenda export?

[Matt Lundin]

Pierre-Luc Gauthier  writes:

> So my question is,
> How can I export this tag filtered agenda view to, say a pdf with
> LaTeX customizations and all that nice stuff.
> The goal would be to be able to leave the house with a beautifully
> typeset document containing all @Errands tagged headers *and* content?
>
> If I use org-agenda-write and write a PDF file, I get basically the
> same screen as my view, no content, just the headers and in a rather
> crude layout.
>
> I guess what I need (conceptually) is a way to get all headers having
> said tag (or being a child of) be compiled in a file. Then I know all
> the cool export stuff (e.g.: HTML, LaTeX, etc) will be accessible.
>

The function org-map-entries could help here. For instance...

--8<---cut here---start->8---
(org-map-entries
 '(concat (org-get-heading) "\n" (org-get-entry))
 "@Errands" 'agenda))
--8<---cut here---end--->8---

...will return a list of strings containing the heading and entry text
for all of your entries tagged "@Errands".

Then you could iterate through that list and insert each item as a top
level heading in a new org file. The following, for instance, will
extract all relevant headings and insert them in a temporary buffer
("/tmp/export.org"):

--8<---cut here---start->8---
(with-current-buffer (find-file-noselect "/tmp/export.org")
  (delete-region (point-min) (point-max))
  (insert "* ")
  (insert (mapconcat 'identity
 (org-map-entries
  '(concat (org-get-heading) "\n" (org-get-entry))
  "@Errands" 'agenda)
 "\n* "))
  (org-align-all-tags))
--8<---cut here---end--->8---

This could be further refined. E.g., you could extract specific heading
components with the function org-heading-components and then place them
directly in a LaTeX file.

Best,
Matt

Re: [O] [RFC] org-agenda: Jump directly to line in case of a timestamp

[Matt Lundin]

Marco Wahl  writes:

> Nicolas Goaziou  writes:
>
>> Hello,
>>
>> Marco Wahl  writes:
>>
>>>
>>> Pressing TAB in the agenda would jump to the respective plain list item.
>>>
>>> Find the concrete patch below.
>>>
>>> WDYT?
>>
>> I have the feeling that both behaviours are useful. All things being
>> equal, this is the entry being displayed in the Agenda, not necessarily
>> the line containing the timestamp. Moreover this feature could be less
>> meaningful if `org-agenda-skip-additional-timestamps-same-entry' is
>> non-nil.
>>
>> Maybe an optional argument could allow one to switch from one behaviour
>> to the other. I have no strong opinion about this, though.
>
> Thanks!
>
> I just found function `org-agenda-switch-to' which is similar to
> `org-agenda-goto' but does not the jump to the headline.
>
> I personally configure this function to be on shift-tab.
>
> (org-defkey org-agenda-mode-map [(backtab)] #'org-agenda-switch-to)

And, as a bonus, org-agenda-switch-to is bound to RET by default. I've
happily been hitting return since 2008 to jump directly to timestamps.
:)

The only difference, if I remember correctly, is that
org-agenda-switch-to replaces the agenda buffer with the target org
buffer, whereas org-agenda-goto shows the target org buffer in another
window. 

Best,
Matt

Re: [O] Is there a way to get all agenda TODOs programmatically?

[Matt Lundin]

Marcin Borkowski <mb...@mbork.pl> writes:

> On 2017-10-07, at 16:20, Matt Lundin <m...@imapmail.org> wrote:
>
>> Hi Marcin,
>>
>> Marcin Borkowski <mb...@mbork.pl> writes:
>>
>>> I'd like to get a list of all agenda TODOs, with titles and due dates.
>>> Is there something ready in Org/contrib/blogosphere to help me?
>>
>> This functionality is built into org-mode. The simplest way to get such
>> a list is to use the agenda (specifically M-x org-todo-list). To display
>> scheduling information you can customize the variable
>> org-agenda-prefix-format.
>
> I know (and use) that.  What I want is not /display/, but a /data
> structure/ containing that info.

Ah, I see! A lisp list, not a list on the screen.

>> Finally, if you want simply to gather the todo data programmatically for
>> further processing you can use the function org-map-entries.
>
> Thanks, that seems to be a good pointer.  I'll check it out.

I think something like this would generate a such a list:

(org-map-entries '(cons (nth 4 (org-heading-components))
 (list (org-get-deadline-time nil)))
  "/!TODO" 'agenda)

Each item in the list would look something like this:

("Test" (23000 24400))

You could then feed the list to whatever function you'd like.

Best,
Matt

Re: [O] Is there a way to get all agenda TODOs programmatically?

[Matt Lundin]

Hi Marcin,

Marcin Borkowski  writes:

> I'd like to get a list of all agenda TODOs, with titles and due dates.
> Is there something ready in Org/contrib/blogosphere to help me?

This functionality is built into org-mode. The simplest way to get such
a list is to use the agenda (specifically M-x org-todo-list). To display
scheduling information you can customize the variable
org-agenda-prefix-format.

> The problem I'm trying to solve is to make some kind of a metric of how
> I'm doing with my TODO list, like "how many overdue tasks do I have", or
> even better, such a sum weighted by the time after the deadline, or
> something similar.

If you use the "DEADLINE" keyword (M-x org-deadline, bound to C-c C-d),
then the daily/weekly agenda (M-x org-agenda-list) will display any
overdue tasks.

Finally, if you want simply to gather the todo data programmatically for
further processing you can use the function org-map-entries.

Best,
Matt

Re: [O] Bug: org-capture-templates with %^t

[Matt Lundin]

Matt Lundin <m...@imapmail.org> writes:

> Matt Lundin <m...@imapmail.org> writes:
>
>> After commit 51b431d01365211d4c40b07729d5d11d82b9dfe2,
>> org-capture-templates containing %^t do not work as expected.
>>
>> With this capture template, I am prompted to enter a time via
>> org-read-date. The expected behavior is as follows:
>>
>>  - if I enter a date and time of day, the resulting timestamp should 
>>contain the time of day:
>>  - i.e., entering "Thu 8am" should yield "<2017-10-05 Thu 08:00>"
>>
>> What happens now:
>>
>>  - when I enter a date with a time of day, the timestamp is truncated
>>and contains only a date
>>  - I.e., entering "Thu 8am" now yields "<2017-10-05 Thu>"
>>
>
> There is a further bug here. With the capture template above, if I enter
> a time range - i.e., an end time - the timestamp in the capture buffer
> is incorrect.
>
> Entering... "Thu 8am-10am" yields...
>
>  <2017-10-05 Thu-10:00>
>

Attached please find a patch that fixes these issues. It is a simple
change, but it brings the behavior of %^t and %^u vs. %^T and %^U into
line with the behavior of the interactive function org-time-stamp
without and with a prefix argument.

The escapes %^t and %^u now default to no time of day if the user enters
nothing but allow the optional entry of a time of day via user
interaction.

The escapes %^T and %^U always include a time of day, with or without
user interaction.

Best,
Matt

>From b3e9fc879c8cadd8b634d7f4f3512ba1eaf1b8bf Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Fri, 6 Oct 2017 09:50:05 -0500
Subject: [PATCH] Fix interactive timestamp entry in capture templates

* lisp/org-capture.el: (org-capture-fill-template) Fix interactive
  timestamp entry to match the behavior of org-time-stamp. This fixes
  a bug related to the %^t and %^u template escapes which resulted in
  incorrect timestamps (<2017-10-06 Fri-12:00>). The difference
  between %^t and %^T now corresponds to the difference between
  org-time-stamp called without and with a prefix argument.
---
 lisp/org-capture.el | 9 -
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index 862cdb276..25af674b8 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -1792,11 +1792,10 @@ The template may still contain \"%?\" for cursor positioning."
 		 (let* ((upcase? (equal (upcase key) key))
 			(org-end-time-was-given nil)
 			(time (org-read-date upcase? t nil prompt)))
-		   (let ((org-time-was-given upcase?))
-			 (org-insert-time-stamp
-			  time org-time-was-given
-			  (member key '("u" "U"))
-			  nil nil (list org-end-time-was-given)
+		   (org-insert-time-stamp
+			time (or org-time-was-given upcase?)
+			(member key '("u" "U"))
+			nil nil (list org-end-time-was-given
 		(`nil
 		 (push (org-completing-read
 			(concat (or prompt "Enter string")
-- 
2.14.2

Re: [O] Bug: org-capture-templates with %^t

[Matt Lundin]

Matt Lundin <m...@imapmail.org> writes:

> After commit 51b431d01365211d4c40b07729d5d11d82b9dfe2,
> org-capture-templates containing %^t do not work as expected.
>
> The relevant capture template snippet looks like this:
>
> (setq org-capture-templates
>   '(("a" "Appointment" entry (file "~/inbox.org") "* %^{Appt}\n %^t%?\n 
> %U\n
> %a\n\n %i")))
>
> With this capture template, I am prompted to enter a time via
> org-read-date. The expected behavior is as follows:
>
>  - if I enter a date and time of day, the resulting timestamp should 
>contain the time of day:
>  - i.e., entering "Thu 8am" should yield "<2017-10-05 Thu 08:00>"
>
> What happens now:
>
>  - when I enter a date with a time of day, the timestamp is truncated
>and contains only a date
>  - I.e., entering "Thu 8am" now yields "<2017-10-05 Thu>"
>

There is a further bug here. With the capture template above, if I enter
a time range - i.e., an end time - the timestamp in the capture buffer
is incorrect.

Entering... "Thu 8am-10am" yields...

 <2017-10-05 Thu-10:00>

Best,
Matt

[O] Bug: org-capture-templates with %^t

[Matt Lundin]

After commit 51b431d01365211d4c40b07729d5d11d82b9dfe2,
org-capture-templates containing %^t do not work as expected.

The relevant capture template snippet looks like this:

(setq org-capture-templates
  '(("a" "Appointment" entry (file "~/inbox.org") "* %^{Appt}\n %^t%?\n %U\n
%a\n\n %i")))

With this capture template, I am prompted to enter a time via
org-read-date. The expected behavior is as follows:

 - if I enter a date and time of day, the resulting timestamp should 
   contain the time of day:
 - i.e., entering "Thu 8am" should yield "<2017-10-05 Thu 08:00>"
 - if I enter only a date, the resulting timestamp should contain a date
   with no time of day
 - if I enter nothing at the prompt, the resulting timestamp should
   contain today's date with no time of day

What happens now:

 - when I enter a date with a time of day, the timestamp is truncated
   and contains only a date
 - I.e., entering "Thu 8am" now yields "<2017-10-05 Thu>"

This, I think, is counter to the intent of the difference between %^t
and %^T. The lowercase and uppercase (time of day vs. no time of day)
should apply only to the default entry when nothing is entered. But when
the user enters information at the prompt, the behavior should be the
same as that of org-time-stamp, allowing a flexible decision on the fly
as to whether to enter a time of day.

Best,
Matt

Re: [O] The problem with scaling inline images with imagemagick

[Matt Lundin]

Ярослав Данько  writes:

> Hello everybody,
>
> I have the line (setq org-image-actual-width '(600)) in my init file.
> With all versions of ImageMagick till 6.9.9-13 it worked well, means
> scaling inline images to 600 px in width. 
>
> But with all higher versions (now it is 6.9.9-17) scaling not happens
> with the error `ImageMagick error: no decode delegate for this image
> format `JPEG' @ error/constitute.c/ReadImage/504`.
>
> Is something can be done in the org-mode or it is imagemagick bug?
>
> Thanks in advance.

I have the same problem, and it is not just limited to Org Mode. In
fact, I find that emacs can no longer display jpeg or png images with
the version of imagemagick listed above:

There is an emacs bug report for this:

bug#28626: 25.3; problem displaying images

I'm not sure if there is a corresponding imagemagick report.

Best,
Matt

Re: [O] [ANN] Agenda speed up

[Matt Lundin]

Nicolas Goaziou  writes:

> Some feedback about the new agenda speed would be nice.

One small bug I found with wip speedup branch. When trying to reschedule
in the agenda with org-agenda-do-date-later or
org-agenda-do-date-earlier, org mode gives a message:

"Cannot find time stamp"

Best,
Mat

Re: [O] [ANN] Agenda speed up

[Matt Lundin]

Nicolas Goaziou <m...@nicolasgoaziou.fr> writes:

> Matt Lundin <m...@imapmail.org> writes:
>
>> I am finding that the branch is still much slower than the current
>> master even when no agenda files have changed (i.e., when running
>> org-agenda-redo in an existing agenda buffer without changing
>> anything).
>
> This is interesting. Could you provide a report of the second call to
> the same view? Or better, both reports for the first and subsequent
> view. The second view should definitely be faster.

I've attached the profiling for master and the wip branch. Both were run
with identical agenda files and identical configurations

The first pair of files profiles calling org-agenda-list immediately
after emacs has started up (i.e., before opening agenda files). The
total times are:

master: 5.014304355
wip:6.68215677

The second pair of files profiles calling org-agenda-redo immediately
after generating the agenda with org-agenda-list (i.e., without changing
anything in the agenda files). Note: repeating the redo after that
results in virtually identical times.

master: 0.979825959
wip:1.17870

The third pair of files profiles calling org-agenda-redo after
changing (rescheduling) one item in the agenda.

master: 0.979580617
wip:1.189759099

> I can look into your issue with proper reports as there may be ways to
> improve it, but I'm more interesting in, e.g., halving the 30 s from
> a month view than reducing the 1 s it takes to display a day view.

I agree it is important to reduce the time it takes to display a month
view. Thanks for all your work on this! I'd like to clarify that I am
not asking for further reductions from what the master branch of
org-mode has already achieved. Rather, I am concerned with regressions
from those times. The gap in time above, of course, is not particularly
big. But I'm running into some slower times (e.g., a 1 second
difference) on some of my other custom agenda commands. Take, for
instance, the following command:

--8<---cut here---start->8---
(org-add-agenda-custom-command
 '("q" "Projects"
   ((stuck "")
(agenda ""
((org-agenda-include-deadlines t)
 (org-agenda-entry-types '(:deadline))
 ;; (org-agenda-skip-function
 ;;  '(org-agenda-skip-entry-if 'notregexp ":PROJ:"))
 (org-agenda-include-diary nil)
 (org-agenda-time-grid nil)))
(todo "PROJECT"
  ((org-agenda-todo-ignore-deadlines t)
   (org-agenda-prefix-format " %i %-12:c%l"
   ((org-deadline-warning-days 365
--8<---cut here---end--->8---

The startup times weren't that far apart.

master: 5.728477902
wip:6.68639764

But running org-agenda-redo results in a 1s gap (see the last two
attached files for profiling information):

master:   2.14623279
wip:  3.125571084

Best,
Matt

org-agenda-list   1   
5.014304355   5.014304355
org-agenda-prepare-buffers2   
3.7477554480  1.8738777240
org-agenda-prepare1   
3.303024766   3.303024766
org-get-agenda-file-buffer252 
2.8621154540  0.0113576010
org-mode  63  
2.021190203   0.0320823841
org-agenda-get-day-entries126 
1.043993129   0.0082856597
org-set-startup-visibility63  
0.9549281500  0.0151575896
org-element-at-point  1863
0.8281850030  0.0004445437
org-cycle-hide-drawers82  
0.8039815459  0.009804653
org-element--parse-to 1863
0.7838064640  0.0004207227
org-diary-sexp-entry  30  
0.6159707919  0.0205323597
org-agenda-finalize   1   
0.614693850.61469385
org-agenda-get-sexps  126 
0.604186440.0047951304
org-element--current-element  3412
0.5537156270  0.0001622847
org-agenda-to-appt1   
0.513418731   0.513418731
org-set-regexps-and-options   189 
0.2761626890  0.0014611782
org-refresh-category-properties   126 
0.271197041   0.0021523574
org-install-agenda-files-menu 63  
0.2459687320  0.0039042655
org-refresh-effort-properties 189 
0.2442

Re: [O] [ANN] Agenda speed up

[Matt Lundin]

Matt Lundin <m...@imapmail.org> writes:
>
> Here is a quick comparison of the top elp-results using a couple of commands:

I'm including the full elp results for reference. These were run with my
org agenda files and with customizations that I don't have time to
isolate at the moment. I'll try to provide a report with a minimal
config soon. The attached files contain profiling for the two commands
reported in my previous email.

(org-todo-list "TODO")

(org-agenda-list)

Best,
Matt

org-todo-list 1   
0.954759710.95475971
org-agenda-prepare1   
0.425165363   0.425165363
org-agenda-prepare-buffers1   
0.394949431   0.394949431
org-agenda-get-day-entries63  
0.2680520310  0.0042547941
org-agenda-get-todos  63  
0.262784373   0.0041711805
org-agenda-finalize-entries   1   
0.202006798   0.202006798
org-get-tags-at   454 
0.200070026   0.0004406828
org-agenda-highlight-todo 227 
0.1841641769  0.0008112959
org-back-to-heading   1207
0.1734143220  0.0001436738
org-refresh-category-properties   63  
0.1109007029  0.0017603286
org-refresh-properties126 
0.0968679490  0.0007687932
org-set-regexps-and-options   63  
0.0660081399  0.0010477482
org--setup-collect-keywords   63  
0.061246143   0.000972161
org-refresh-effort-properties 63  
0.055947762   0.0008880597
org-refresh-stats-properties  63  
0.049975411   0.0007932604
org-agenda-finalize   1   
0.048449583   0.048449583
org-element-at-point  126 
0.0278754009  0.0002212333
org-element--parse-to 126 
0.0237195620  0.0001882504
org-get-priority  227 
0.0201254490  8.865...e-05
org-agenda-files  3   
0.0190556840  0.0063518946
org-element--current-element  189 
0.0178631470  9.451...e-05
org-up-heading-safe   398 
0.016063253   4.035...e-05
org-agenda-format-item227 
0.014556003   6.412...e-05
org-entries-lessp 859 
0.0112956840  1.314...e-05
org-agenda-align-tags 1   
0.010916710.01091671
org-agenda-mode   1   
0.009271944   0.009271944
org-get-property-block74  
0.0081775990  0.0001105080
org-outline-level 799 
0.0073207199  9.162...e-06
org-get-todo-state238 
0.005509511   2.314...e-05
org-entry-get 36  
0.0054365299  0.0001510147
org-element-keyword-parser189 
0.0054290560  2.872...e-05
org-at-property-p 38  
0.005193692   0.0001366761
org--property-local-values36  
0.0049136239  0.0001364895
org-agenda-skip   237 
0.0048559309  2.048...e-05
org-add-props 832 
0.0047682290  5.731...e-06
org-agenda-check-for-timestamp-as-reason-to-ignore-todo-item  237 
0.004598511.940...e-05
org-check-agenda-file 126 
0.0042163329  3.346...e-05
org-inlinetask-in-task-p  74  
0.003895827   5.264...e-05
org-get-agenda-file-buffer126 
0.0033892649  2.689...e-05
org-split-string  686 
0.0033632589  4.902...e-06
org-get-limited-outline-regexp641 
0.0033366020  5.205...e-06
org-activate-links9   
0.003008839   0.0003343154
org-remove-uninherited-tags   260 
0.0029396989  1.130...e-05
org-find-base-buffer-visiting 126 
0.0028278890  2.244...e-0

Re: [O] [ANN] Agenda speed up

[Matt Lundin]

Nicolas Goaziou  writes:

> Hello,
>
> Samuel Wales  writes:
>
>> have not beena ble to respons for health reasons.  i have rsposne
>> partly done.  i think result is slightly slower but seems tob e
>> correct if you count recent maint as correct.
>
> It can be slightly slower if you start with a cold cache and never
> re-use it, e.g., when you display only a single day and the most
> important agenda files were modified since last agenda display.

I am finding that the branch is still much slower than the current
master even when no agenda files have changed (i.e., when running
org-agenda-redo in an existing agenda buffer without changing anything).
This is true even when I set org-element-use-cache to t.

> OTOH, displaying, e.g., a whole week, month, year should be a lot
> faster.

Is this an unavoidable trade-off? Since I am constantly refreshing
single day agenda buffers and todo lists, I would much prefer a faster
single day display to a faster week or month display.

Matt

Re: [O] [ANN] Agenda speed up

[Matt Lundin]

Nicolas Goaziou  writes:

> If there is no more feedback nor objection, I'll merge the branch in
> master before the end of the week.
>
> Until then, the changes are still available in wip-agenda-speedup branch
> for review.

Thanks for the heads up. I just had a chance to test the
wip-agenda-speedup branch and find that it significantly slows down the
creation of agenda buffers with my agenda files and custom commands.

I think I have a fairly standard setup (some customizations, additional
features such as habits). I'll do some testing with minimal examples to
see if I can find out why the new branch is so much slower in my case.
In the meantime, I'd like to that the branch *not* be merged until we
are sure that it is actually faster for the majority of use cases.

Here is a quick comparison of the top elp-results using a couple of commands:

(org-todo-list "TODO")

master:
--8<---cut here---start->8---
org-todo-list 1   
0.954759710.95475971
org-agenda-prepare1   
0.425165363   0.425165363
org-agenda-prepare-buffers1   
0.394949431   0.394949431
org-agenda-get-day-entries63  
0.2680520310  0.0042547941
org-agenda-get-todos  63  
0.262784373   0.0041711805
org-agenda-finalize-entries   1   
0.202006798   0.202006798
org-get-tags-at   454 
0.200070026   0.0004406828
org-agenda-highlight-todo 227 
0.1841641769  0.0008112959
org-back-to-heading   1207
0.1734143220  0.0001436738
--8<---cut here---end--->8---

wip-agenda-speedup:
--8<---cut here---start->8---
org-todo-list 1   
1.402434591   1.402434591
org-agenda-day-entries63  
0.4656588689  0.0073914106
org-agenda--entry-from-todo   2217
0.4304873449  0.0001941756
org-agenda-prepare1   
0.387713298   0.387713298
org-agenda-prepare-buffers1   
0.378589420.37858942
org-agenda--file-data 63  
0.2997486200  0.0047579146
org-entry-get 1402
0.2108398869  0.0001503850
org-entry-properties  1366
0.1953800049  0.0001430307
org-agenda-finalize-entries   1   
0.191974038   0.191974038
org-agenda-check-for-timestamp-as-reason-to-ignore-todo-item  237 
0.1819146310  0.0007675722
org-agenda-highlight-todo 227 
0.1735402220  0.0007644943
org-agenda--todo-data 7   
0.1687917040  0.0241131005
org-back-to-heading   2336
0.1648271410  7.055...e-05
--8<---cut here---end--->8---

(org-agenda-list)

master:
--8<---cut here---start->8---
org-agenda-list   1   
1.036426005   1.036426005
org-agenda-prepare1   
0.596309830.59630983
org-agenda-prepare-buffers1   
0.584742966   0.584742966
org-agenda-get-day-entries63  
0.388804281   0.0061714965
org-agenda-get-scheduled  63  
0.287089758   0.0045569802
org-refresh-category-properties   63  
0.280568592   0.0044534697
org-habit-parse-todo  30  
0.178230735   0.0059410245
org-time-string-to-time   219 
0.162822094   0.0007434798
--8<---cut here---end--->8---

wip-agenda-speedup:
--8<---cut here---start->8---
org-agenda-list   1   
1.377235021.37723502
org-agenda-prepare1   
0.594557456   0.594557456
org-agenda-prepare-buffers1   
0.582119253   0.582119253
org-agenda--all-filtered-data 1   
0.307176728   0.307176728
org-agenda--file-data 63  
0.279614084   0.0044383187

Re: [O] How to generate clock report for hours worked each day?

[Matt Lundin]

ConcreteVitamin  writes:

> Thanks, it does show the hours clocked on the current day. Is there
> something on top of this mode that allows me to specify the specific
> day? (Better, I just want to quickly show the  to see the
> trend of my productivity :)).

Yes, you can just move back and forward in the agenda with "b" and "f."
Or you can pick a day on the calendar with "j". And you can view your
aggregated time for the week with "vw".

Best,
Matt

Re: [O] How to generate clock report for hours worked each day?

[Matt Lundin]

ConcreteVitamin  writes:

> org-clock-report is awesome; however, it reports time spent on each
> task. Is there any custom command that you use to generate a clock
> report showing hours worked, across tasks, for each day?
>
> [...]
>
> I just want another command to show how many hours I've worked on
> each day...

You can generate such a report in the daily agenda by activating
org-agenda-clockreport-mode (press R while viewing the agenda). It will
show how much time you clocked during the day, broken down by file,
subtree, etc. You can customize what is shown with the variable
org-agenda-clockreport-parameter-plist.

Best,
Matt

Re: [O] Add ability to force-enable TOC

[Matt Lundin]

Kaushal Modi  writes:

> On Wed, Sep 20, 2017, 6:25 AM Nicolas Goaziou 
> wrote:
>
> I didn't say it was an unusual case. I said it didn't cover all
> the use-cases. Maybe you are really needing a subset of the
> initial feature.
>
> Understood. 
>
> IOW, do mixed numbered/unnumbered headings make sense in TOC?
>
> Correct. That does not make sense to me. That is not my use case. 

I support the use case of unnumbered table of contents.

In addition (although this is probably too much of a corner case to
support officially in Org Mode), it is common in the U.S. to use a mix
of numbered and unnumbered headings in books. One accomplishes this in
LaTeX with the following:

--8<---cut here---start->8---
\tableofcontents

\chapter*{Introduction}
\addcontentsline{toc}{chapter}{Introduction}

\chapter{A Chapter}

\chapter{Another Chapter}

\chapter*{Conclusion}
\addcontentsline{toc}{chapter}{Introduction}
--8<---cut here---end--->8---

The resulting Table of Contents looks like this...

Introduction3
1 A Chapter 5
2 Another Chapter   7
Conclusion  9

Best,
Matt

Re: [O] Confused about the explanation for 'org-cycle'

[Matt Lundin]

I agree that this is confusing. I examined the org repository history,
and it seems that both the documentation and the variable go all the way
back to 2007. I believe the default value of the variable was changed to
nil, but the documentation was not updated accordingly.  

Matt

alain.coch...@unistra.fr writes:

> Hello.
>
> In section "2.3.1 Global and local cycling" of the manual I read:
>
>
>  `' (`org-cycle')
>   _Subtree cycling_: Rotate current subtree among the states
>
>,-> FOLDED -> CHILDREN -> SUBTREE --.
>'---'
>
>   The cursor must be on a headline for this to work(1).  When the
>   cursor is at the beginning of the buffer and the first line is not
>   a headline, then  actually runs global cycling (see
>   below)(2).
>
> with the footnote being
>
>  (2) see the option `org-cycle-global-at-bob'.
>
>
> If I use Org Mode without any customization, then, when using 
> "When the cursor is at the beginning of the buffer and the first line
> is not a headline", absolutely nothing is done.
>
> Sure, this is consistent with the help page of variable
> org-cycle-global-at-bob which says that "When this option is nil,
> don't do anything special at the beginning of the buffer", together
> with the fact that, without any customization, org-cycle-global-at-bob
> is 'nil'.
>
> But I find this very confusing, at least for the beginner, who is
> unlikely to enquire right away about the possible values of
> org-cycle-global-at-bob.
>
> Or perhaps I am missing something at the English level?
>
> Regards,
> alain

Re: [O] (gnus-icalendar-org-setup) not evaluated in .emacs?

[Matt Lundin]

"Loris Bennett"  writes:
>
> Rereading this, I decided to move the settings out from the Customize
> block to before call to gnus-icalendar-org-setup:
>
> (require 'gnus-icalendar)
> (setq gnus-icalendar-org-capture-file "~/git/org.git/gtd.org")
> (setq gnus-icalendar-org-capture-headline (quote ("Exchange")))
> (gnus-icalendar-setup)
> (gnus-icalendar-org-setup)
>
> Now I get all the buttons first time round and didn't have to resort to
> a flaky hack, so thanks, Eric.
>
> But should this kind of ordering dependency happen?  Or should my
> Customize block just be at the beginning of my .emacs rather than at the
> end?

I looked at the gnus-icalendar-code and this ordering dependency is
indeed hard-coded into gnus-icalendar-org-setup. From the code comments:

--8<---cut here---start->8---
;; to enable optional iCalendar->Org sync functionality
;; NOTE: both the capture file and the headline(s) inside must already exist
;; (setq gnus-icalendar-org-capture-file "~/org/notes.org")
;; (setq gnus-icalendar-org-capture-headline '("Calendar"))
;; (gnus-icalendar-org-setup)
--8<---cut here---end--->8---

Matt

Re: [O] changing individual agenda items dates, en-bloc?

[Matt Lundin]

Sharon Kimble  writes:

> How can I change individual items in my agenda so that although they may
> have a date of 2017-09-01 and be repeated daily but haven't been
> accessed in the intervening time period, I can easily change it to miss
> out all the unexpired dates and be rescheduled for a date of my choosing
> please?
>
> I seem to remember that there was some way, either through a add-on
> package or a sequence of commands, but I can't seem to find it now.

You can mark items with "M" and then do a bulk reschedule.

The key sequence for the bulk rescheduling is "B s".

Matt

Re: [O] BUG - strange characters showing in agenda after times displayed

[Matt Lundin]

Sharon Kimble <boudic...@skimble.plus.com> writes:

> Matt Lundin <m...@imapmail.org> writes:
>
>> Sharon Kimble <boudic...@skimble.plus.com> writes:
>>
>>> With the new release of org-mode to 9.1 I am finding with every build of
>>> a new agenda that this is displaying for items with a time as part of
>>> them -
>>>
>>>   organiser:   8:30΄ьԔלڤ  Scheduled: TODO email org-mode list
>>>
>>> This is new and has only appeared with the new agenda which is generated
>>> after the release of org-mode 9.1.
>>>
>>> How do I stop it and get rid of it please?
>>
>> I cannot replicate this. My guess is that it is related to a particular
>> configuration setting on your machine. For instance, what is the value
>> of org-agenda-time-grid? The order of items in this variable changed
>> with the upgrade to 9.1.
>
> ;; Enable display of the time grid so we can see the marker for the current 
> time
> (setq org-agenda-time-grid (quote ((daily today remove-match)
>#("" 0 16 (org-heading t))
>(0900 1100 1300 1500 1700
>
> But time-grid is not enabled in my agenda.

This setting is definitely the problem. I can now reproduce the behavior.

The order of items in org-agenda-time-grid changed recently, which I
think is the root of the problem. The value for this variable should be
a list of four (not three) items. Even if time grid is not enabled, the
3rd item in the setting is used to generate trailing characters that
follow the time string in the agenda. This should be a string, not a
list of numbers.

You can type C-h v org-agenda-time-grid for all the details. Here is the
default value of org-agenda-time-grid for reference. Note how the order
differs from your current setting:

((daily today require-timed)
 (800 1000 1200 1400 1600 1800 2000)
 ".." "")

Hope this helps,
Matt

Re: [O] BUG - strange characters showing in agenda after times displayed

[Matt Lundin]

Sharon Kimble  writes:

> With the new release of org-mode to 9.1 I am finding with every build of
> a new agenda that this is displaying for items with a time as part of
> them -
>
>   organiser:   8:30΄ьԔלڤ  Scheduled: TODO email org-mode list
>
> This is new and has only appeared with the new agenda which is generated
> after the release of org-mode 9.1.
>
> How do I stop it and get rid of it please?

I cannot replicate this. My guess is that it is related to a particular
configuration setting on your machine. For instance, what is the value
of org-agenda-time-grid? The order of items in this variable changed
with the upgrade to 9.1.

Best,
Matt

Re: [O] match by property in agenda view

[Matt Lundin]

Xebar Saram  writes:

> Hi all
>
> i Have this item in a property drawer:
>
> :people: %^{people?|-|allan|bob|joel}
>
> now i have this custom agenda view defined
>
> (add-to-list 'org-agenda-custom-commands
> '("sk" "wtd" 
> tags "people=\"allan\"" 
> ((org-agenda-sorting-strategy '(priority-down effort-down)))
> ))
>
> yet when i launch it it never finds any items which have a :people:
> allan entry in the drawer

I believe you need to use regexp syntax. The above command will find
only find property values that exactly match "allan". The following
command will find property values containing "allan".

(add-to-list 'org-agenda-custom-commands
 '("sk" "wtd" 
   tags "people={allan}"
   ((org-agenda-sorting-strategy '(priority-down effort-down)))
   ))

Best,
Matt

Re: [O] [PATCH] Ensure proper nesting of captured entries

[Matt Lundin]

Matt Lundin <m...@imapmail.org> writes:

> I configure my org-capture templates set to insert items at the end of a
> capture file: i.e., with a simple file target - (file "~/org/inbox.org")
> - and no target headline.

A note of clarification. My post and patch relate to headline entries,
not list items. Apologies for the confusion about terminology.

Best,
Matt

Re: [O] proliferation of agenda tasks, but how to stop it please?

[Matt Lundin]

Sharon Kimble  writes:

> How can I stop my 31-day agenda from showing each and every daily task
> for each and every day, when all I need to see them on is today please?
>
> The change has only occurred since updating to
> 'org-plus-contrib-20170906'.

I believe you can put the following in your agenda file:

(setq org-agenda-show-future-repeats nil)

This has replaced the variable org-agenda-repeating-timestamp-show-all. 

Best,
Matt

[O] [PATCH] Ensure proper nesting of captured items

[Matt Lundin]

I configure my org-capture templates set to insert items at the end of a
capture file: i.e., with a simple file target - (file "~/org/inbox.org")
- and no target headline. In the past, org capture would always insert
these as top level headings, regardless of existing headings in the
target file. Recently, org-capture has begun to nest these entries
according to context. This is undesirable, as it has caused many
important capture items to be buried under unrelated headings.

I did a git bisect and found the problematic commit from June 2, 2017:

57d0a7453d0386f3f1425fc5319b2f42fca16e42

As far as I can tell, that commit took out the line in
org-capture-place-entry that sets level to 1 when there is no target
entry. The attached patch makes a minor change to fix the issue.

Best,
Matt 

>From 20ead737aebc9c46aa291b5a70248c57f3fd64a1 Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Sat, 9 Sep 2017 11:08:06 -0500
Subject: [PATCH] Ensure that top-level capture targets are pasted at level 1

* lisp/org-capture.el: (org-capture-place-entry) Ensure that level is
  set to 1 (i.e., the top level) if there is no headline target
  defined. Otherwise, captured items are improperly nested by context.
---
 lisp/org-capture.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index cd1944d96..2ddb9c505 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -1097,7 +1097,7 @@ may have been stored before."
 (defun org-capture-place-entry ()
   "Place the template as a new Org entry."
   (let ((reversed? (org-capture-get :prepend))
-	level)
+	(level 1))
 (when (org-capture-get :exact-position)
   (goto-char (org-capture-get :exact-position)))
 (cond
-- 
2.14.1

[O] [Bug] org-search-view "bad keyword argument"

[Matt Lundin]

Hi everyone,

Commit 22e916fb369629606d919a0e4f33ec2954c87e04 results in the following
error when calling org-search-view:

cl--delete-duplicates: Bad keyword argument (lambda (a b) (and
(file-exists-p a) (file-exists-p b) (file-equal-p a b)))

Best,
Matt

[O] [PATCH] Don't push url to kill-ring when invoking org-protocol-capture

[Matt Lundin]

From: Matt Lundin <m...@imapmail.org>
To: Org Mode <emacs-orgmode@gnu.org>
Date: Mon, 17 Apr 2017 20:58:30 -0500
Subject: [PATCH] Don't push url to kill-ring when invoking
 org-protocol-capture
--text follows this line--

* lisp/org-protocol.el: (org-protocol-do-capture) Don't push link to
  kill-ring when invoking org-protocol-capture. Otherwise,
  org-protocol-do-capture interferes with capture templates that
  insert the contents of the clipboard or the kill ring.
---
 lisp/org-protocol.el | 1 -
 1 file changed, 1 deletion(-)

diff --git a/lisp/org-protocol.el b/lisp/org-protocol.el
index a59540a30..825435674 100644
--- a/lisp/org-protocol.el
+++ b/lisp/org-protocol.el
@@ -498,7 +498,6 @@ Now template ?b will be used."
 	 (org-capture-link-is-already-stored t)) ;; avoid call to org-store-link
 (setq org-stored-links
 	  (cons (list url title) org-stored-links))
-(kill-new orglink)
 (org-store-link-props :type type
 			  :link url
 			  :description title
-- 
2.12.2

Re: [O] Problems with opening a link

[Matt Lundin]

Hi Carsten,

Carsten Dominik  writes:

> Hi everyone,
>
> I have problems opening a link in org.
>
> The link looks like this:
>
> https://ui.adsabs.harvard.edu/#search/q=author%3A%22Dominik%2CC%22/metrics
>
> I have copied it like this from the address bar in a browser.
>
> If I click on it in Org-mode, the link is modified to
>
> https://ui.adsabs.harvard.edu/%23search/q=author:%22Dominik,C%22/metrics
>
> before being sent to the browser, and the browser cannot resolve it.
> The problem seems to be that # has been turned into %23

I cannot replicate this on Linux, but there do seem to be some
inconsistencies with how Org-mode is escaping/unescaping links:

AFAICT, Org-mode itself is not modifying the "#". The relevant lines of
org-open-at-point are 10747-10748:

  ((functionp (org-link-get-parameter type :follow))
(funcall (org-link-get-parameter type :follow) path))

Just simply copying and pasting the link from the browser, I found that
`path` at the lines above is:

//ui.adsabs.harvard.edu/#search/q=author:"Dominik,C"/metrics

Note: the time one gets to this point, the path has already been
unescaped (see line 10712).

Meanwhile, `type` is...

https

...which results in org-open-at-point calling the following:

(lambda (path) (browse-url (concat "https:" path)))

However, there are inconsistencies when one turns the pasted url above
into an actual org-link (e.g., by calling org-insert-link).

Then the link in org-mode looks like this (note that the escape
percentage characters are themselves escaped, which seems a bit strange
to me):

[[https://ui.adsabs.harvard.edu/#search/q=author%253A%2522Dominik%252CC%2522/metrics]]

Now when you open at point, the `path` at line 10748 is:

//ui.adsabs.harvard.edu/#search/q=author%3A%22Dominik%2CC%22/metrics

Hope this helps in debugging.
Matt

Re: [O] Problems with opening a link

[Matt Lundin]

"Tim O'Callaghan"  writes:

> I've had a similar issue, but with redmine links.
> if i have
>
> #+LINK: rm http://redmine/issues/%s
>
> * link [[rm:3287]] does not work properly any more (with firefox under
> KDE)

The link above is missing ".org". It should look like this:

#+LINK: rm http://redmine.org/issues/%s

Matt

Re: [O] [BUG] org-link-search fails if search string contains new lines

[Matt Lundin]

Nicolas Goaziou  writes:

>> The problem, I think, is the regexp construction in org-link-search.
>> This was introduced back in August of 2015 with commit
>> cfe5bc97f8b18ccbf49d0764746c7563ce8d29da.
>>
>> The problematic line in org.el is 10951:
>>
>> (s-multi-re (mapconcat #'regexp-quote words "[ \t]+\\(?:\n[ \t]*\\)?"))
>>
>> The constructed regexp fails because it assumes a newline will be
>> preceded by whitespace. But often newlines are not preceded by
>> whitespace.
>>
>> Is there a reason the following won't work? 
>>
>> (s-multi-re (mapconcat #'regexp-quote words "[ \t\r\n]+"))
>>
>> This was the method org-link-search used prior to the commit above. Are
>> we trying to avoid matching across blank lines?
>
> Yes, we are.
>
> Fixed, hopefully.

Thanks Nicolas. Yes, it now works with multi-line searches that do not
contain blank lines.

I think there are some instances in which one might want to search
across blank lines, such as when one captures a region that contains one
or more blank lines. E.g., let's say org-context-in-file-links is t (the
default). If one captures a section in org.el containing a blank line,
org-link-search will fail to locate the section.

Here's a sample link to try it out. (It assumes an org-mode repository
at ~/org-mode).

 
[[file:~/org-mode/lisp/org.el::(defsubst%20org-uniquify%20(list)%0A%20"Non-destructively%20remove%20duplicate%20elements%20from%20LIST."%0A%20(let%20((res%20(copy-sequence%20list)))%20(delete-dups%20res)))%0A%0A(defsubst%20org-get-at-bol%20(property)%0A%20"Get%20text%20property%20PROPERTY%20at%20the%20beginning%20of%20line."%0A%20(get-text-property%20(point-at-bol)%20property))%0A]]

Thanks,
Matt

[O] [PATCH] Allow insertion of links with multi-line search strings

[Matt Lundin]

>From 726eba76f31537747a26a7689ee632ec8e9bc01f Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Mon, 27 Mar 2017 09:55:33 -0500
Subject: [PATCH] Allow insertion of links with multi-line search strings

* lisp/org.el: (org-insert-link): Fix regexps to match across
  newlines.
---
 lisp/org.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index e40db18f6..dcfa4fd6f 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -10330,7 +10330,7 @@ Use TAB to complete link prefixes, then RET for type-specific completion support
 ;; option If yes, simplify the link by using only the search
 ;; option.
 (when (and buffer-file-name
-	   (string-match "^file:\\(.+?\\)::\\(.+\\)" link))
+	   (string-match "^file:\\(.+?\\)::\\(\\(.\\|\n\\)+\\)" link))
   (let* ((path (match-string 1 link))
 	 (case-fold-search nil)
 	 (search (match-string 2 link)))
@@ -10340,7 +10340,7 @@ Use TAB to complete link prefixes, then RET for type-specific completion support
 	(setq link search)
 
 ;; Check if we can/should use a relative path.  If yes, simplify the link
-(when (string-match "^\\(file:\\|docview:\\)\\(.*\\)" link)
+(when (string-match "^\\(file:\\|docview:\\)\\(\\(.\\|\n\\)*\\)" link)
   (let* ((type (match-string 1 link))
 	 (path (match-string 2 link))
 	 (origpath path)
-- 
2.12.1

Re: [O] Inconsistent behavior in generating file link search strings

[Matt Lundin]

Matt Lundin <m...@imapmail.org> writes:

> 1. First, org-insert-link truncates the search string. Here are the
> steps to reproduce with emacs -Q:
>
>  - Store a link in a plain text file. The value of org-stored-links is:
>
> org-stored-links is a variable defined in ‘org.el’.
> Its value is
>
> (("file:~/test.txt::Duis aute irure dolor in\nreprehenderit in voluptate 
> velit esse cillum dolore eu fugiat nulla\npariatur." nil))
>
>  - Insert the link in an org buffer using org-insert-link. The resulting
>link looks like this:
>
>[[file:~/test.txt::Duis%20aute%20irure%20dolor%20in]]
>
>  - This seems to run counter to the advertised behavior in
>org-context-in-file-links, which says the entire region will be
>stored by default.
>
>  - The problem is the regex on line 10333 or org.el:
>
>  (string-match "^file:\\(.+?\\)::\\(.+\\)" link))

It turns out that the line I mentioned above is actually relevant only
in some cases, as it applies only when linking to an item in the current
file. 

The regex that cause the problem in most cases is line 10343:

(when (string-match "^\\(file:\\|docview:\\)\\(.*\\)" link)

In this case, insert link will also fail if the filename happens to
contain a new line (a corner-case, I admit!).

Matt

Re: [O] [BUG] org-link-search fails if search string contains new lines

[Matt Lundin]

Matt Lundin <m...@imapmail.org> writes:

>
> Create an active region covering the third sentence and select the
> org-capture "n" template, which creates the following link:
>
> * Test
>  
> [[file:~/test.txt::Duis%20aute%20irure%20dolor%20in%0Areprehenderit%20in%20voluptate%20velit%20esse%20cillum%20dolore%20eu%20fugiat%20nulla%0Apariatur.]]
>
> Try to follow the link. It will open test.txt, but it will also give the
> message and fail to locate the correct position in the file:
>
> "No match for fuzzy expression: Duis aute irure dolor in
> reprehenderit in voluptate velit esse cillum dolore eu fugiat
> nulla pariatur."
>

The problem, I think, is the regexp construction in org-link-search.
This was introduced back in August of 2015 with commit
cfe5bc97f8b18ccbf49d0764746c7563ce8d29da.

The problematic line in org.el is 10951:

(s-multi-re (mapconcat #'regexp-quote words "[ \t]+\\(?:\n[ \t]*\\)?"))

The constructed regexp fails because it assumes a newline will be
preceded by whitespace. But often newlines are not preceded by
whitespace.

Is there a reason the following won't work? 

(s-multi-re (mapconcat #'regexp-quote words "[ \t\r\n]+"))

This was the method org-link-search used prior to the commit above. Are
we trying to avoid matching across blank lines? 

Best,
Matt

[O] [BUG] org-link-search fails if search string contains new lines

[Matt Lundin]

Following links fails to locate the appropriate location in text files
if the search string in the link contains new lines.

Steps to reproduce:

/usr/bin/emacs -Q ~/config/minimal.el

where minimal.el contains...

--8<---cut here---start->8---
(add-to-list 'load-path "~/org-mode/lisp/")
(add-to-list 'load-path "~/org-mode/contrib/lisp/")

(setq org-agenda-files '("~/config/test.org"))

(setq org-capture-templates
  '(("n" "Note" entry
   (file "~/config/test.org")
   "* Test\n %a\n")))
--8<---cut here---end--->8---

Open a file (~/text.txt) containing

--8<---cut here---start->8---
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad
minim veniam, quis nostrud exercitation ullamco laboris nisi ut
aliquip ex ea commodo consequat. Duis aute irure dolor in
reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
pariatur. Excepteur sint occaecat cupidatat non proident, sunt in
culpa qui officia deserunt mollit anim id est laborum.
--8<---cut here---end--->8---

Create an active region covering the third sentence and select the
org-capture "n" template, which creates the following link:

--8<---cut here---start->8---
* Test
 
[[file:~/test.txt::Duis%20aute%20irure%20dolor%20in%0Areprehenderit%20in%20voluptate%20velit%20esse%20cillum%20dolore%20eu%20fugiat%20nulla%0Apariatur.]]
--8<---cut here---end--->8---

Try to follow the link. It will open test.txt, but it will also give the
message and fail to locate the correct position in the file:

"No match for fuzzy expression: Duis aute irure dolor in
reprehenderit in voluptate velit esse cillum dolore eu fugiat
nulla pariatur."

For comparison, evaluate (setq org-context-in-file-links 1) and run
capture again. The captured snippet looks like this:

--8<---cut here---start->8---
* Test
 [[file:~/test.txt::Duis%20aute%20irure%20dolor%20in]]
--8<---cut here---end--->8---

Follow the link. The fuzzy search succeeds and finds the correct
position in the text.

Matt

[O] Inconsistent behavior in generating file link search strings

[Matt Lundin]

I am grabbing a lot of links from plain text files these days and find
that the way in which org generates search strings in file links is
inconsistent. That is, org-capture and org-insert-link behave
differently. 

1. First, org-insert-link truncates the search string. Here are the
steps to reproduce with emacs -Q:

 - Store a link in a plain text file. The value of org-stored-links is:

org-stored-links is a variable defined in ‘org.el’.
Its value is

(("file:~/test.txt::Duis aute irure dolor in\nreprehenderit in voluptate 
velit esse cillum dolore eu fugiat nulla\npariatur." nil))

 - Insert the link in an org buffer using org-insert-link. The resulting
   link looks like this:

   [[file:~/test.txt::Duis%20aute%20irure%20dolor%20in]]

 - This seems to run counter to the advertised behavior in
   org-context-in-file-links, which says the entire region will be
   stored by default.

 - The problem is the regex on line 10333 or org.el:

 (string-match "^file:\\(.+?\\)::\\(.+\\)" link))

2. By contrast, the annotation substitution (%a) in org-capture inserts
the whole search string:

  - Take the following capture template:
   
  (setq org-capture-templates
'(("n" "Note" entry
   (file "~/config/test.org")
   "* Test\n %a\n")))

  - Select the same region in a dummy file as a above and call
org-capture, using the "n" template.

  - The org-capture snippet contains the whole search string, including
new lines:

* Test
 
[[file:~/test.txt::Duis%20aute%20irure%20dolor%20in%0Areprehenderit%20in%20voluptate%20velit%20esse%20cillum%20dolore%20eu%20fugiat%20nulla%0Apariatur.]]
  
I think org-capture (#2) is in keeping with the behavior advertised by
org-context-in-file-links. However, as I will explain in a subsequent
bug report, org-link-search currently fails if the search string
contains new lines, so, in fact, only the one-line search strings
generated by org-insert-link actually work when following links.

Matt

Re: [O] tsia-up sorting strategy sorts agenda by date and ignores time. How can I change that?

[Matt Lundin]

Arkady Grudzinsky  writes:

> Hi,
>
> I'd like to sort my tasks in agenda by inactive creation
> timestamp which includes the time portion.  I have found that
> tsia-up strategy ignores the time portion of the timestamp.  Is
> there a way to take time into account?

Unfortunately, the org sorting relies on org-time-string-to-absolute,
which converts time strings to days only, so the default is just a day
to day comparison.

You could use something like this and then add user-defined-up or
user-defined-down where desired in org-agenda-sorting-strategy:

--8<---cut here---start->8---
(defun my-sort-by-inactive-timestamp-incl-time (a b)
  (let* ((ma (get-text-property 1 'org-marker a))
 (mb (get-text-property 1 'org-marker b))
 (tsa (with-current-buffer (marker-buffer ma)
 (org-entry-get (marker-position ma) "TIMESTAMP_IA")))
 (tsb (with-current-buffer (marker-buffer mb)
 (org-entry-get (marker-position mb) "TIMESTAMP_IA")))
 (seca (if tsa (org-time-string-to-seconds tsa) 0))
 (secb (if tsb (org-time-string-to-seconds tsb) 0)))
(cond ((> seca secb) 1)
  ((> secb seca) -1)
  (t nil

(setq org-agenda-cmp-user-defined 'my-sort-by-inactive-timestamp-incl-time)
--8<---cut here---end--->8---

I imagine there are ways to do this more elegantly (e.g., by iterating
over a and b), but this gets the job done for me.

Best,
Matt

Re: [O] Org export buffer opens at buffer's end

[Matt Lundin]

Hi David,

David Mann  writes:

> When I do C-c C-e to start an org export, the export buffer opens at its
> bottom, with the top cut off in the window.  In past versions the buffer
> opened at the top.

Do you mean the *Org Export Dispatcher* buffer or the final exported
buffer? I cannot reproduce this behavior in either case. Reproducing
with a minimal config would help to track down the problem:

http://orgmode.org/manual/Feedback.html

Matt

Re: [O] Bug in Recent Agenda Mod

[Matt Lundin]

Ian Dunn  writes:

> Commit d262ae53c966c7a745c0fa779149f9eb7486333d "org-agenda: Fix
> agenda standard name when going unsticky" sets org-agenda-buffer-name
> to "*Org Agenda*" if not in sticky.
>
> However, org-agenda-redo sets org-agenda-sticky to nil before redoing the 
> agenda command.
>
> Steps to Reproduce:
>
> 1. Turn on org-agenda-sticky
> 2. Execute any agenda command that only involves org-agenda-list
>- Agenda is in buffer *Org Agenda(CMD)* (ex. *Org Agenda(d:)*)
> 3. Press 'g' (org-agenda-redo)
>- Agenda is now in *Org Agenda*
>
> The included patch fixes this for sticky redo, and will have no effect
> for anything other than a sticky redo.
>
> --
> Ian Dunn
>
> diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
> index b618653d5..04e6685b5 100644
> --- a/lisp/org-agenda.el
> +++ b/lisp/org-agenda.el
> @@ -4076,6 +4076,7 @@ items if they have an hour specification like [h]h:mm."
>(catch 'exit
>  (setq org-agenda-buffer-name
> (or org-agenda-buffer-tmp-name
> +   (and org-agenda-doing-sticky-redo org-agenda-buffer-name)
> (if org-agenda-sticky
> (cond ((and org-keys (stringp org-match))
>(format "*Org Agenda(%s:%s)*" org-keys org-match))

Thanks so much for solving this. I was just trying to figure out why
refreshing my sticky agendas kept creating a new "*Org Agenda* buffer
and am glad I checked here first. I can confirm that this fix works.

Matt

Re: [O] org-stuck-projects confuses a "Next" heading with the NEXT keyword

[Matt Lundin]

Hi Nicholas,

Nicolas Goaziou  writes:

> Hello,
>
> Guy Mayraz  writes:
>
>> I want every project to have a NEXT or WAIT keyword, and have defined
>> org-stuck-projects as follows:
>>
>> (setq org-stuck-projects '("+LEVEL=2/-INACTIVE-DONE" ("NEXT" "WAIT") nil
>> ""))
>>
>> It mostly works, but it does not identify as stuck a project that has a
>> "Next items" subheading and no NEXT keyword.
>
> Fixed. Thank you.

I find that this commit (7c3e0b0fdfdc0cf1a4724cffe17b6bb160278247)
causes Org mode no longer to identify stuck projects correctly.

Steps to reproduce:

I am using emacs version 25.1.1 and the most recent commit in the Org
mode repo.

/usr/bin/emacs -Q -l ~/minimal.el

minimal.el contains...

--8<---cut here---start->8---
(add-to-list 'load-path "~/org-mode/lisp/")
(add-to-list 'load-path "~/org-mode/contrib/lisp/")

(require 'org)

(setq org-agenda-files '("~/config/test.org"))
--8<---cut here---end--->8---

...and test.org contains...

--8<---cut here---start->8---
#+TODO: TODO NEXT | DONE

* Headline
** This project should not be stuck
*** TODO A next action
*** NEXT Another one
--8<---cut here---end--->8---

I am using the default value of org-stuck-projects:

--8<---cut here---start->8---
org-stuck-projects is a variable defined in ‘org-agenda.el’.
Its value is ("+LEVEL=2/-DONE" ("TODO" "NEXT" "NEXTACTION") nil "")
--8<---cut here---end--->8---

M-x org-agenda-list-stuck-projects produces the following:

--8<---cut here---start->8---
List of stuck projects: 
  test:   This project should not be stuck
--8<---cut here---end--->8---

Thanks,
Matt

[O] [BUG] agenda column view summaries missing in block agendas

[Matt Lundin]

I'd like to report another bug in agenda column view summary displays.

If one uses a custom agenda command with multiple blocks, column
summaries in the agenda display do not display for all blocks.

Steps to replicate:

emacs -Q -l ~/minimal.el

Where minimal contains

--8<---cut here---start->8---
(add-to-list 'load-path "~/org-mode/lisp/")
(add-to-list 'load-path "~/org-mode/contrib/lisp/")

(require 'org)

(setq org-agenda-files '("~/test.org"))

(setq org-columns-default-format
  "%40ITEM(Task) %8Effort(Estimate){:} %8CLOCKSUM %20SCHEDULED %20DEADLINE 
%20TIMESTAMP")

(setq org-agenda-custom-commands
  '(("n" "Now"
 ((todo "TODO")
  (agenda "")
--8<---cut here---end--->8---

and ~/test.org contains...

--8<---cut here---start->8---
* TODO Buy groceries
  SCHEDULED: <2017-01-16 Mon>
  :PROPERTIES:
  :Effort:   0:10
  :END:
* TODO Mow lawn
  SCHEDULED: <2017-01-16 Mon>
* TODO Clean house
  SCHEDULED: <2017-01-16 Mon>
--8<---cut here---end--->8---

If I call the custom agenda command above and then org-agenda-columns,
no summary data is displayed for the first block (the todo block).
Summary data does appear at the top of the agenda block.

A git bisect reveals that this bug was introduced in 2015 with the
following commit. Earlier versions of org display independent column
summaries for all agenda blocks.

--8<---cut here---start->8---
f1ee487218da00962eb8b67d1f5ec1f54a5a04db is the first bad commit
commit f1ee487218da00962eb8b67d1f5ec1f54a5a04db

Author: Nicolas Goaziou 
Date:   Tue Jun 2 21:56:50 2015 +0200

org-colview: Handle user-defined inheritance in agenda colview

* lisp/org-colview.el (org-agenda-columns): Use `org-entry-get'
  instead of `org-entry-properties' to handle inheritance for
  user-provided properties.

:04 04 6348c8dd00846a942b50d6495800f5137c99620a 
eedfa94ed29964ee0af9ed22735fcef8158ebea8 M  lisp
--8<---cut here---end--->8---

Best,
Matt

[O] [BUG] No agenda column view summary shown if org-agenda-overriding-header is set

[Matt Lundin]

I find that column view summaries do not display if
org-agenda-overriding-header is set.

Steps to replicate:

emacs -Q -l ~/minimal.el

Where minimal.el contains...

--8<---cut here---start->8---
(add-to-list 'load-path "~/org-mode/lisp/")
(add-to-list 'load-path "~/org-mode/contrib/lisp/")

(require 'org)

(setq org-agenda-files '("~/test.org"))

(setq org-columns-default-format
  "%40ITEM(Task) %8Effort(Estimate){:} %8CLOCKSUM %20SCHEDULED %20DEADLINE 
%20TIMESTAMP")

(setq org-agenda-custom-commands
  '(("o" todo "TODO"
 ((org-agenda-overriding-header "Now")
--8<---cut here---end--->8---

...and ~/test.org contains...

--8<---cut here---start->8---
* TODO Buy groceries
  SCHEDULED: <2017-01-16 Mon>
  :PROPERTIES:
  :Effort:   0:10
  :END:
  [2017-01-16 Mon 15:11]
* TODO Mow lawn
  SCHEDULED: <2017-01-16 Mon>
  [2017-01-16 Mon 15:11]
* TODO Clean house
  SCHEDULED: <2017-01-16 Mon>
  [2017-01-16 Mon 15:11]
--8<---cut here---end--->8---

When calling column view, the overlay summary line with total effort
estimates is missing.

I ran a git bisect and determined that this bug was introduced with
commit 279902ca4da4fe5f0ceff801a3aab51b942b42b7 (Feb. 14, 2016).
Versions of org before this correctly display the summary line.

Best,
Matt

[O] [PATCH] Fix org-clock-load

[Matt Lundin]

Since commit fda64f1ae2110175662b52daa3a5ec0f967f0c0d on November 6,
org-clock-load no longer restores clocks in org-clock-persist-file. The
contents of the file look like this:

--8<---cut here---start->8---
(setq org-clock-stored-history '(("/home/matt/org/inbox.org" . 39479) 
("/home/matt/org/reading.org" . 63478)))
--8<---cut here---end--->8---

The files both exist; the position information is correct; and
org-clock-persist is t. And yet after calling org-clock-load,
org-clock-history, org-clock-loaded, and org-clock-stored-history remain
nil.

The problem, it seems, is that the logic/order of the if statement was
reversed in the commit above. The attached patch should fix the issue.

Best,
Matt

>From 6d649016fbbfaa28c902ee1e71c20ecf332f8a14 Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Mon, 19 Dec 2016 17:24:10 -0600
Subject: [PATCH] org-clock: Fix org-clock-load

* lisp/org-clock.el: (org-clock-load): Fix incorrect order in if
statement that was preventing org-load from loading stored data and
populating org-clock-history.
---
 lisp/org-clock.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lisp/org-clock.el b/lisp/org-clock.el
index 65c13fdf2..6e58ce91a 100644
--- a/lisp/org-clock.el
+++ b/lisp/org-clock.el
@@ -2962,9 +2962,9 @@ The details of what will be saved are regulated by the variable
 (defun org-clock-load ()
   "Load clock-related data from disk, maybe resuming a stored clock."
   (when (and org-clock-persist (not org-clock-loaded))
-(if (file-readable-p org-clock-persist-file)
-	(message "Restoring clock data")
-  (message "Not restoring clock data; %S not found" org-clock-persist-file)
+(if (not (file-readable-p org-clock-persist-file))
+	(message "Not restoring clock data; %S not found" org-clock-persist-file)
+  (message "Restoring clock data")
   ;; Load history.
   (load-file org-clock-persist-file)
   (setq org-clock-loaded t)
-- 
2.11.0

Re: [O] Bug: Org Emphasis Issues (Trunk)

[Matt Lundin]

Ian Dunn  writes:

> I encountered these issues this morning when I updated Org from
> trunk (and reloaded it with C-c C-x !). I believe it's following
> commit "Split `org-emph-re' and `org-verbatim-re'". I've got 2
> issues:

> 1. Nothing longer than one character has emphasis. I find changing
> the 4th element of org-emphasis-regexp-components ("body-regexp")
> to ".*?" fixes this.

I cannot replicate this particular issue. Could you provide a minimal
recipe for replicating?

> 2. Headings of level >= 3 are fontified as if they were
>emphasized. I'm guessing org-do-emphasis-face is considering "***" as
>an acceptable emphasis string.

I can confirm the problems with headlines, though I can only replicate
it with level 2 headings. (Are you perhaps using org-indent-mode?)

With emacs -Q and the latest org from git, if I open the following file,
"Subheading 1" appears in bold. If I delete "Subheading 2," the emphasis
disappears.

* First headline
** Subheading 1
** Subheading 2

Matt

[O] [BUG] Upcoming deadlines sorted in reverse order

[Matt Lundin]

With commit 9299efa3519b3ef3191e8dc06a4466696c720f6c, upcoming deadlines
in the agenda are sorted in reverse order, with more distant deadlines
appearing first and imminent and overdue deadlines appearing last.

Running emacs -Q and using the following org file...

* An overdue deadline 
  DEADLINE: <2016-11-27 Sun 17:00>
* A close deadline 
  DEADLINE: <2016-11-30 Wed 17:00>
* A more distant deadline 
  DEADLINE: <2016-12-06 Tue 17:00>

...I find that the agenda produces this...

Day-agenda (W48):
Tuesday29 November 2016
  test:   In   7 d.:  A more distant deadline
  test:   In   1 d.:  A close deadline
  test:2 d. ago:  An overdue deadline

The expected (and traditional) behavior is this...

Day-agenda (W48):
Tuesday29 November 2016
  test:2 d. ago:  An overdue deadline
  test:   In   1 d.:  A close deadline
  test:   In   7 d.:  A more distant deadline

Thanks,
Matt

[O] [BUG] Scheduled items in past show time of day in agenda

[Matt Lundin]

Hi everyone,

With commit 69ec6258b65a5d317f0dcb275ec2d5a90f72f191 I find that
non-repeating timestamps with a time-of-day designation continue to
appear at a particular time in the agenda even when they are in the
past.

For instance, using emacs -Q, i find thagt the following item scheduled
for yesterday...

* A scheduled item
  SCHEDULED: <2016-11-28 Mon 17:00>

...appears in the agenda for today at 17:00...

Day-agenda (W48):
Tuesday29 November 2016
   8:00.. 
  10:00.. 
  12:00.. 
  13:57.. now - - - - - - - - - - - - - - - - - - - - - - - - -
  14:00.. 
  16:00.. 
  test:   17:00.. Sched. 2x:  A scheduled item
  18:00.. 
  20:00.. 

The expected behavior (i.e., no time of day designation if it is not the
scheduled day) is the following...

Day-agenda (W48):
Tuesday29 November 2016
  test:   Sched. 2x:  A scheduled item

Thanks,
Matt

[O] org-special-ctrl-a/e no longer has the expected effect

[Matt Lundin]

Commit 8d2f0a441174c703ed0ed570e2f0eaf0da5d6aeb broke the expected
behavior org-beginning-of-line.

Steps to reproduce:

(setq org-special-ctrl-a/e '(nil . t)) ;; i.e., I want special
   ;; treament of the end of a
   ;; headline but not the
   ;; beginning 

* A headline
^ cursor initially here

Press C-a

Expected behavior:

* A headline
^ cursor moves here

Actual behavior:

* A headline
  ^ cursor moves here

This behavior is not what one would expect from the docstring of
org-special-ctrl-a/e.

Thanks,
Matt

Re: [O] OReanalyzing archived TO-DOs

[Matt Lundin]

Hi Avery,

Avery Chan  writes:

> One problem is what to do with completed tasks. They clutter up my
> todo list so I usually archive them to another file. My problem is
> that after archiving them, I can't compare the 'effort' with the
> actual 'clock time' easily (i.e. using `C-c C-x C-c
> (org-agenda-columns)` doesn't return the same results in my original
> org file as it does in the archived file.)

I can't replicate this. Could you provide an minimal example that
demonstrates the problem?

> **What's a workflow that will allow me to open up my archived TO-DOs
> to evaluate them?**

I would probably do the same thing you did: i.e., use column view in the
archive file. You could also try including archive files in the agenda
("v A") and then using column view in the agenda. That would have the
effect of merging archived and non-archived tasks into a single view.

Best,
Matt

[O] Request: Move org-eww from contrib to core

[Matt Lundin]

Given that the coming release of emacs 25 ships with the eww browser, I
think it would be worthwhile to make org-eww.el a core org-mode module.
I have been using eww instead of emacs-w3m for some time now with no
problems. Since org ships with support for emacs-w3m, an external
package, it seems to makes sense also to support eww, the built-in
browser.

I noticed that Marco Wahl, the author of org-eww.el, is on the
contributors list, so, if I understand correctly, there shouldn't be
copyright issues.

Matt

[O] [PATCH] Re: links-9.0 v3

[Matt Lundin]

Thanks, John, for improving link functionality in org-mode!

I just discovered that this change breaks the contrib files that use
org-store-link-functions and org-add-link-type.

I'm not familiar enough with the details of org-link-set-parameters to
fix all of the contrib files, but I am including a simple patch to
update a contrib file that I use regularly: org-eww.el.

Thanks,
Matt

>From d473260956493dafd1e37c5f3680813fbf63434e Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Fri, 5 Aug 2016 20:07:40 -0500
Subject: [PATCH] Update org-eww to use new link api

* contrib/lisp/org-eww.el: org-store-link-functions: Remove obsolete
  hook.
  (org-link-set-parameters) Use new api for links.
---
 contrib/lisp/org-eww.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/contrib/lisp/org-eww.el b/contrib/lisp/org-eww.el
index 11ccb68..a443255 100644
--- a/contrib/lisp/org-eww.el
+++ b/contrib/lisp/org-eww.el
@@ -48,7 +48,7 @@
 
 
 ;; Store Org-link in eww-mode buffer
-(add-hook 'org-store-link-functions 'org-eww-store-link)
+(org-link-set-parameters "eww" :store #'org-eww-store-link)
 (defun org-eww-store-link ()
   "Store a link to the url of a eww buffer."
   (when (eq major-mode 'eww-mode)
-- 
2.9.2

[O] [OT] Re: capture with iceweasel/firefox -- nothing works

[Matt Lundin]

Philip Hudson  writes:

> On 4 June 2016 at 02:23, Samuel Wales  wrote:
>> over many years, i have been trying to get org-protocol (or
>> anything similar) to work, and it is not working reliably.
>> that's why i want this feature to work so much.
> ...
>>   *what is the most reliable set of instructions or code to
>>   get iceweasel to work with org?*
>
> Ordinarily my advice would be: Don't. Use Conkeror instead.
>
> However, I see that keyboard-orientation and JavaScript are exactly
> what you don't want, which is a pity. Org-capture from Conkeror is
> bulletproof for me, even with Emacs server running on a different host
> on the same LAN segment.
>
> Conkeror uses the same browser engine as Firefox, so some FF
> extensions work with it, but with an "Emacsified" keyboard-oriented
> front end (M-x, M-:, C-u, C-g, etc) -- all dynamically configurable
> and extensible, though in JavaScript not Elisp. The JavaScripting
> framework uses familiar Elisp idioms wherever possible, like hooks,
> `provide' and `require', for example.

Conkeror is great. I used it for over seven years. However, I think it's
important to note (for anyone thinking of investing time in setting it
up) that conkeror's future is uncertain given Mozilla's planned
deprecation of XUL, its abandonment of xulrunner, and its lack of
official support for using Firefox's browser engine to run third party
apps. This is what a Mozilla developer said in response to a broken -app
flag (which conkeror requires to run) in FF 48 nightly: "This bug isn't
high on the official Mozilla priority list. We don't use -app or have
any testing for it, so it's likely to break."[fn:1] Conkeror is still a
good browser, but if you switch to it, expect to find several broken
features (viewing page source and text files internally, managing
permissions, etc.) and more turbulence in the future.

Matt

[fn:1] https://bugzilla.mozilla.org/show_bug.cgi?id=1271574

Re: [O] default visibility of results drawer

[Matt Lundin]

John Kitchin  writes:

> Is it possible to get the default visibility of results drawers to be
> open?
>
> They are closed when I first open a file, and I have to manually open
> them to see the contents.
>

You can add "#+STARTUP: showeverything" to the top of a file. The
corresponding variable `org-startup-folded' will control this globally.

You can also type C-u C-u C-u Tab to open everything up in the buffer
quickly.

Matt

Re: [O] capture with iceweasel/firefox -- nothing works

[Matt Lundin]

Samuel Wales <samolog...@gmail.com> writes:

> hi matt,
>
> thanks for your reply.
>
> On 6/3/16, Matt Lundin <m...@imapmail.org> wrote:
>> Here is the new style bookmark that works for me:
>>
>> javascript:location.href='org-protocol://capture?url='+encodeURIComponent(location.href)+'='+encodeURIComponent(document.title)+'='+encodeURIComponent(window.getSelection())
>>
>>
>> I'd recommend copying and pasting the javascript line above into the
>> location bar to confirm whether it works. If it does, you can then
>> easily add it to your bookmarks.
>
> will try.  this is supposed to imitate clicking on the unicorn or
> calling the bookmark?

This is the content of the bookmark. Pasting it into the location bar is
just a quick way to test if everything is working prior to saving it
permanently as a bookmark. But it might be easier to go straight to the
bookmark step (see below).

Do you mean clicking on the unicorn icon installed with this extension?

https://addons.mozilla.org/en-US/firefox/addon/org-mode-capture/

If you have installed that extension, I think everything should just
work with minimal configuration. The extension is distinct from the
bookmark method. The difference is that a bookmark is a bit more
universal. Once you get it set up the protocol system-wide, the bookmark
should work in any browser.

> however, i never found any good instructions on what to do to add js
> to bookmarks.

> i have a menu bar and a location bar + icons.  i'd like a button or
> icon on the latter if possible.  my previous thrashing around caused a
>>> symbol which is a drop down menu which contains the bookmarks.
> i'll settle for that if nec.  but that is extra clicking i'd rather
> avoid, and one of them brings up a page with js on it.

To save it as a bookmark in firefox, right click on the bookmark bar,
select "New Bookmark," and paste it into the Location field.

> do i try with selecting and without selecting?

The bookmark should work with and without selections.

> which capture key does it call?

This can be controlled with the emacs variable
`org-protocol-default-template-key'. You can also specify a template
shortcut by changing the bookmark. For instance, to call a template
designated by "x", you could use the following:

javascript:location.href='org-protocol://capture?template=x='+encodeURIComponent(location.href)+'='+encodeURIComponent(document.title)+'='+encodeURIComponent(window.getSelection())

Matt

Re: [O] capture with iceweasel/firefox -- nothing works

[Matt Lundin]

Xebar Saram  writes:

> Hi 
>
> i will add that im exactly in the same situatuon
> i use a window manager (i3) on arch linux and have tried almost every
> solution out there (with FF, chrome etc) with no working method.
> i really wish org-protocol had a much simpler way of capturing
> links,text tec. im not a programmer but to me it seems so complicated.
> it the past before i was using org i had a simple bash script that
> yanked links/text from dwb (a simple web browser) and added that to a
> text file. something like this
>
> #!/bin/bash
> # dwb: xtm
> task add pro:dl +music $DWB_TITLE $DWB_URI 

> where the current title+URI of the webpage was added to a text file
> (task add) command.
>
> would love something simple like that for orgmode

I think you could put something like this in the script:

emacsclient "org-protocol://capture?url=$DWB_URI=$DWB_TITLE"

If you wanted to add the extra project and tag information (e.g.,
"music"), you could create a custom capture template and then designate
the template shortcut (e.g., "m") in the script:

emacsclient "org-protocol://capture?template=m=$DWB_URI=$DWB_TITLE"

Matt

Re: [O] capture with iceweasel/firefox -- nothing works

[Matt Lundin]

Matt Lundin <m...@imapmail.org> writes:

> Samuel Wales <samolog...@gmail.com> writes:
>
>> over many years, i have been trying to get org-protocol (or
>> anything similar) to work, and it is not working reliably.
>>
>> intermittently, it says there is a greedy protocol. the success rate
>> of capturing is around 50% maybe. when it is not working, it never
>> works. i have noticed no correlation with anything that could cause
>> this.

Another thought. Is the emacs server always running (either by starting
emacs as a daemon or by invoking server-start)?

Matt

Re: [O] capture with iceweasel/firefox -- nothing works

[Matt Lundin]

Samuel Wales  writes:

> over many years, i have been trying to get org-protocol (or
> anything similar) to work, and it is not working reliably.
>
> intermittently, it says there is a greedy protocol. the success rate
> of capturing is around 50% maybe. when it is not working, it never
> works. i have noticed no correlation with anything that could cause
> this.
>
> sometimes it captures bogus text when i just want the link, but i
> don't care about that. at least it copied the link.
>
> i've tried:
>
>   1) org-capture firefox extension
>   2) a bookmark like this
>  - 
> javascript:location.href%20=%20'org-protocol://capture-html://w/'%20+%20encodeURIComponent(location.href)%20+%20'/'%20+%20encodeURIComponent(document.title)%20+%20'/'%20+%20encodeURIComponent(function%20()%20{var%20html%20=%20"";%20if%20(typeof%20document.getSelection%20!=%20"undefined")%20{var%20sel%20=%20document.getSelection();%20if%20(sel.rangeCount)%20{var%20container%20=%20document.createElement("div");%20for%20(var%20i%20=%200,%20len%20=%20sel.rangeCount;%20i%20<%20len;%20++i)%20{container.appendChild(sel.getRangeAt(i).cloneContents());}%20html%20=%20container.innerHTML;}}%20else%20if%20(typeof%20document.selection%20!=%20"undefined")%20{if%20(document.selection.type%20==%20"Text")%20{html%20=%20document.selection.createRange().htmlText;}}%20var%20relToAbs%20=%20function%20(href)%20{var%20a%20=%20document.createElement("a");%20a.href%20=%20href;%20var%20abs%20=%20a.protocol%20+%20"//"%20+%20a.host%20+%20a.pathname%20+%20a.search%20+%20a.hash;%20a.remove();%20return%20abs;};%20var%20elementTypes%20=%20[['a',%20'href'],%20['img',%20'src']];%20var%20div%20=%20document.createElement('div');%20div.innerHTML%20=%20html;%20elementTypes.map(function(elementType)%20{var%20elements%20=%20div.getElementsByTagName(elementType[0]);%20for%20(var%20i%20=%200;%20i%20<%20elements.length;%20i++)%20{elements[i].setAttribute(elementType[1],%20relToAbs(elements[i].getAttribute(elementType[1])));}});%20return%20div.innerHTML;}());
>   3) a bookmark like this
>  - 
> javascript:location.href='org-protocol://capture://L/'+encodeURIComponent(location.href)+'/'+encodeURIComponent(document.title)+'/'+encodeURIComponent(window.getSelection())
>   4) i know there's a new one that is supposed to capture
>  format better (so you don't end up with a single
>  paragraph when there were several paragraphs).  that
>  sounds wonderful!  however, i am limited in computer
>  use (4 minutes at a time), so i don't know if it is
>  worth my trying or not.  i have to use those 4 minutes responsibly.

Here is the new style bookmark that works for me:

javascript:location.href='org-protocol://capture?url='+encodeURIComponent(location.href)+'='+encodeURIComponent(document.title)+'='+encodeURIComponent(window.getSelection())

I also placed the following in ~/.local/share/applications/emacsclient.desktop:

--8<---cut here---start->8---
[Desktop Entry]
Name=Emacs Client
Exec=emacsclient %u
Icon=emacs-icon
Type=Application
Terminal=false
MimeType=x-scheme-handler/org-protocol;
--8<---cut here---end--->8---

Since you are not using a desktop environment, you might also need to
make sure that the appropriate xdg tools are installed. On arch linux
the package is xdg-utils.

I'd recommend copying and pasting the javascript line above into the
location bar to confirm whether it works. If it does, you can then
easily add it to your bookmarks.

Note: There is a bug in the implementation of the new org-protocol url
processing that can lead to multiple "greedy protocol" warnings. (I
posted a bug report about this a week or so ago.)

Hope this helps.
Matt

Re: [O] [BUG] error with org-refile and ido-completion-help

[Matt Lundin]

Matt Lundin <m...@imapmail.org> writes:

> Nicolas Goaziou <m...@nicolasgoaziou.fr> writes:
>
>>> A git bisect revealed that the change was introduced in 2015 with the
>>> following commit:
>>>
>>> ,
>>> | commit 50ba0a5ed609f3600f2590f3ba22b8ab3ff3331c
>>> | Author: Nicolas Goaziou <m...@nicolasgoaziou.fr>
>>> | Date:   Sun Jun 7 00:38:58 2015 +0200
>>> | 
>>> | Fix 1a7364177046b8a57ade0aeb9f52bacfc0b8b088
>>> | 
>>> | * lisp/org.el (org-icompleting-read): Let `completing-read' or
>>> |   equivalent sort out type of completion.
>>> | (org-olpath-completing-read): Revert partially
>>> | 1a7364177046b8a57ade0aeb9f52bacfc0b8b088.
>>> `
>>>
>>> It looks like this commit removed some functionality from the now
>>> obsolete org-icompleting-read that made sure that the items in the
>>> completion list passed to the completing read function were strings.
>>
>> Actually, this is a bug in "ido.el", since `ido-completing-read' is not
>> a drop-in replacement for `completing-read'.  The latter accepts lists
>> of strings, but also alist, obarrays and hash tables. The former accepts
>> only list of strings.
>>
>> I suggest to report the bug to "ido.el" maintainers since they probably
>> want to preserve compatibility between the completion functions.
>
> OK. Thanks. I see that a bug report has been on the emacs list since
> 2013: 
>
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=15430
>
> For the time being, it's easy enough to write a custom wrapper around
> ido-completing-read, so I think I'll do that.
>
> It's worth nothing that the author of patch that originally added the
> completing-read-function variable did not anticipate that
> ido-completing-read would be a simple drop-in for completing-read:
>
> http://thread.gmane.org/gmane.emacs.devel/134000
>
> For a long time (since at least 2009), org-mode had built-in support for
> ido-mode completion when refiling. So this does seem to be a deprecation
> of longstanding org-mode functionality.
>
> I'll see if we can add something to the docstrings to alert of the need
> to write a wrapper around ido-completing-read.

I think the simplest thing is just to remove the obsolete info in the
manual. Please see the attached patch.

Best,
Matt

>From d66e08b15ddb107135b5cbaa6582c9461347a3a4 Mon Sep 17 00:00:00 2001
From: Matt Lundin <m...@imapmail.org>
Date: Wed, 1 Jun 2016 12:04:14 -0500
Subject: [PATCH] Remove paragraph about obsolete variables

* doc/org.texi: Remove paragraph documenting now obsolete variables
  org-completion-use-ido and org-completion-use-iswitchbp.
---
 doc/org.texi | 6 --
 1 file changed, 6 deletions(-)

diff --git a/doc/org.texi b/doc/org.texi
index 88c506c..36ab999 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -16931,12 +16931,6 @@ emacs -Q --batch \
 @cindex tag completion
 @cindex link abbreviations, completion of
 
-Emacs would not be Emacs without completion, and Org mode uses it whenever it
-makes sense.  If you prefer an @i{iswitchb}- or @i{ido}-like interface for
-some of the completion prompts, you can specify your preference by setting at
-most one of the variables @code{org-completion-use-iswitchb}
-@code{org-completion-use-ido}.
-
 Org supports in-buffer completion.  This type of completion does
 not make use of the minibuffer.  You simply type a few letters into
 the buffer and use the key to complete text right there.
-- 
2.8.3

Re: [O] [BUG] error with org-refile and ido-completion-help

[Matt Lundin]

Nicolas Goaziou  writes:

>> A git bisect revealed that the change was introduced in 2015 with the
>> following commit:
>>
>> ,
>> | commit 50ba0a5ed609f3600f2590f3ba22b8ab3ff3331c
>> | Author: Nicolas Goaziou 
>> | Date:   Sun Jun 7 00:38:58 2015 +0200
>> | 
>> | Fix 1a7364177046b8a57ade0aeb9f52bacfc0b8b088
>> | 
>> | * lisp/org.el (org-icompleting-read): Let `completing-read' or
>> |   equivalent sort out type of completion.
>> | (org-olpath-completing-read): Revert partially
>> | 1a7364177046b8a57ade0aeb9f52bacfc0b8b088.
>> `
>>
>> It looks like this commit removed some functionality from the now
>> obsolete org-icompleting-read that made sure that the items in the
>> completion list passed to the completing read function were strings.
>
> Actually, this is a bug in "ido.el", since `ido-completing-read' is not
> a drop-in replacement for `completing-read'.  The latter accepts lists
> of strings, but also alist, obarrays and hash tables. The former accepts
> only list of strings.
>
> I suggest to report the bug to "ido.el" maintainers since they probably
> want to preserve compatibility between the completion functions.

OK. Thanks. I see that a bug report has been on the emacs list since
2013: 

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=15430

For the time being, it's easy enough to write a custom wrapper around
ido-completing-read, so I think I'll do that.

It's worth nothing that the author of patch that originally added the
completing-read-function variable did not anticipate that
ido-completing-read would be a simple drop-in for completing-read:

http://thread.gmane.org/gmane.emacs.devel/134000

For a long time (since at least 2009), org-mode had built-in support for
ido-mode completion when refiling. So this does seem to be a deprecation
of longstanding org-mode functionality.

I'll see if we can add something to the docstrings to alert of the need
to write a wrapper around ido-completing-read.

Matt

Re: [O] [BUG] org-mode 8.3.x directly opens URL/ follows Timestamp, if Return is clicked

[Matt Lundin]

Eric Abrahamsen  writes:

>> This bug was fixed on October 31, 2015 with the following commit:
>>
>> a87c34cb8d58e7019c8d081c02bbac9e93cef8ab
>>
>> Unfortunately, that commit must have been applied only to the master
>> branch, as it is not present in the maint branch and thus was not
>> released as part of org-mode 8.3.4. I'm not sure why that commit was
>> never merged into maint. I'm flagging this as a bug to ensure that it is
>> merged.
>>
>> I think the only solution at the moment would be to upgrade to a more
>> recent development version either by checkout org out through git or
>> using elpa:
>>
>> http://orgmode.org/elpa.html
>
> I'm using org-plus-contrib from elpa, and still see this bug...

You are right; I spoke too soon. ELPA packages are based on the maint
branch, so installing from ELPA won't help. Until this bug fix makes its
way to maint, the only solution is to use the org repo or to patch the
ELPA download manually.

Matt

[O] [BUG] error with org-refile and ido-completion-help

[Matt Lundin]

When calling org-refile with org-completion-use-ido set to t, pressing
TAB in ido-mode results in an error. This is because ido-completion-help
expects a list of strings. However, org-refile sends ido-completing-read
a list of lists.

To replicate:

1. /usr/bin/emacs -Q -l ~/.minimal.el

--8<---cut here---start->8---
(add-to-list 'load-path "~/org-mode/lisp")
(add-to-list 'load-path "~/org-mode/contrib/lisp")
(ido-mode)
(setq org-completion-use-ido 1)
--8<---cut here---end--->8---

Go to an org file with headlines:

--8<---cut here---start->8---
* One
* Two
* Three
--8<---cut here---end--->8---

2. Call org-refile on the first headline.

3. Press TAB within the ido minibuffer. (See the backtrace at the bottom
of this post).

A git bisect revealed that the change was introduced in 2015 with the
following commit:

,
| commit 50ba0a5ed609f3600f2590f3ba22b8ab3ff3331c
| Author: Nicolas Goaziou 
| Date:   Sun Jun 7 00:38:58 2015 +0200
| 
| Fix 1a7364177046b8a57ade0aeb9f52bacfc0b8b088
| 
| * lisp/org.el (org-icompleting-read): Let `completing-read' or
|   equivalent sort out type of completion.
| (org-olpath-completing-read): Revert partially
| 1a7364177046b8a57ade0aeb9f52bacfc0b8b088.
`

It looks like this commit removed some functionality from the now
obsolete org-icompleting-read that made sure that the items in the
completion list passed to the completing read function were strings.


Matt

--8<---cut here---start->8---
Debugger entered--Lisp error: (wrong-type-argument arrayp (#("Three" 0 5 (face 
org-level-1 fontified t)) "/home/matt/test.org" "^\\(\\*+\\)\\(?: 
+\\(DONE\\|TODO\\)\\)?\\(?: +\\(\\[#.\\]\\)\\)?\\(?: +\\(?:\\[[0-9%/]+\\] 
*\\)*\\(Three\\)\\(?: *\\[[0-9%/]+\\]\\)*\\)\\(?:[   
]+\\(:[[:alnum:]_@#%:]+:\\)\\)?[]*$" 14))
  ido-no-final-slash((#("Three" 0 5 (face org-level-1 fontified t)) 
"/home/matt/test.org" "^\\(\\*+\\)\\(?: +\\(DONE\\|TODO\\)\\)?\\(?: 
+\\(\\[#.\\]\\)\\)?\\(?: +\\(?:\\[[0-9%/]+\\] *\\)*\\(Three\\)\\(?: 
*\\[[0-9%/]+\\]\\)*\\)\\(?:[]+\\(:[[:alnum:]_@#%:]+:\\)\\)?[
]*$" 14))
  ido-file-lessp((#("Three" 0 5 (face org-level-1 fontified t)) 
"/home/matt/test.org" "^\\(\\*+\\)\\(?: +\\(DONE\\|TODO\\)\\)?\\(?: 
+\\(\\[#.\\]\\)\\)?\\(?: +\\(?:\\[[0-9%/]+\\] *\\)*\\(Three\\)\\(?: 
*\\[[0-9%/]+\\]\\)*\\)\\(?:[]+\\(:[[:alnum:]_@#%:]+:\\)\\)?[]*$" 
14) (#("Two" 0 3 (face org-level-1 fontified t)) "/home/matt/test.org" 
"^\\(\\*+\\)\\(?: +\\(DONE\\|TODO\\)\\)?\\(?: +\\(\\[#.\\]\\)\\)?\\(?: 
+\\(?:\\[[0-9%/]+\\] *\\)*\\(Two\\)\\(?: *\\[[0-9%/]+\\]\\)*\\)\\(?:[
]+\\(:[[:alnum:]_@#%:]+:\\)\\)?[]*$" 8))
  sort(((#("Two" 0 3 (face org-level-1 fontified t)) "/home/matt/test.org" 
"^\\(\\*+\\)\\(?: +\\(DONE\\|TODO\\)\\)?\\(?: +\\(\\[#.\\]\\)\\)?\\(?: 
+\\(?:\\[[0-9%/]+\\] *\\)*\\(Two\\)\\(?: *\\[[0-9%/]+\\]\\)*\\)\\(?:[ 
]+\\(:[[:alnum:]_@#%:]+:\\)\\)?[]*$" 8)) ido-file-lessp)
  ido-completion-help()
  funcall-interactively(ido-completion-help)
  call-interactively(ido-completion-help)
  ido-complete()
  funcall-interactively(ido-complete)
  call-interactively(ido-complete nil nil)
  command-execute(ido-complete)
  read-from-minibuffer("Refile subtree \"One\" to: " nil (keymap keymap (4 . 
ido-magic-delete-char) (6 . ido-magic-forward-char) (2 . 
ido-magic-backward-char) (63 . ido-completion-help) (left . ido-prev-match) 
(right . ido-next-match) (0 . ido-restrict-to-matches) (27 keymap (32 . 
ido-take-first-match)) (67108896 . ido-restrict-to-matches) (26 . 
ido-undo-merge-work-directory) (20 . ido-toggle-regexp) (67108908 . 
ido-prev-match) (67108910 . ido-next-match) (19 . ido-next-match) (18 . 
ido-prev-match) (16 . ido-toggle-prefix) (13 . ido-exit-minibuffer) (10 . 
ido-select-text) (32 . ido-complete-space) (9 . ido-complete) (5 . 
ido-edit-input) (3 . ido-toggle-case) (1 . ido-toggle-ignore) keymap (menu-bar 
keymap (minibuf "Minibuf" keymap (previous menu-item "Previous History Item" 
previous-history-element :help "Put previous minibuffer history element in the 
minibuffer") (next menu-item "Next History Item" next-history-element :help 
"Put next minibuffer history element in the minibuffer") (isearch-backward 
menu-item "Isearch History Backward" isearch-backward :help "Incrementally 
search minibuffer history backward") (isearch-forward menu-item "Isearch 
History Forward" isearch-forward :help "Incrementally search minibuffer history 
forward") (return menu-item "Enter" exit-minibuffer :key-sequence "" :help 
"Terminate input and exit minibuffer") (quit menu-item "Quit" 
abort-recursive-edit :help "Abort input and exit minibuffer") "Minibuf")) (10 . 
exit-minibuffer) (13 . exit-minibuffer) (7 . abort-recursive-edit) (C-tab . 
file-cache-minibuffer-complete) (9 . self-insert-command) 

Re: [O] How to change agenda sorting order temporarily?

[Matt Lundin]

Matt Lundin <m...@imapmail.org> writes:

> "Martin Beck" <elwood...@web.de> writes:
>
>> I'm working with a lot of pre-defined agendas, but I sometimes would
>> like to change the sorting order of one of those to different criteria
>> (by time, alphanumerically, etc.).
>>  
>> I did not find a way to do that in org-mode directly - did I overlook
>> something, or do I really have to change the configuration of the
>> whole agenda and rebuild it?
>>  
>> Related problem:
>> If I change the configuration of the agenda by changing the
>> customizing options, those changes do only have an effect, if I kill
>> the agenda buffer and launch the agenda again.
>> It would be nice, if I just could rebuild the existing agenda with "r"
>> or "g", but that does not seem to be sufficient?
>>  
>
> Here's a quick function that works for me:
>
> (defun my-org-agenda-change-sorting-strategy (strategy)
>   "Change the sorting strategy."
>   (interactive (list
> (completing-read "Choose a strategy: "
>  (mapcar 'cdr (cdr org-sorting-choice))
>  nil t)))
>   ;; adjust the following types as needed - e.g., add 'agenda, etc.
>   (org-agenda-check-type t 'todo 'tags 'search)
>   (let ((org-agenda-sorting-strategy (list (intern strategy
> (org-agenda-redo)))

I should add that with this function the new sort order is temporary.
Pressing "r" for refresh will restore the default sort order for the
buffer. 

Matt

Re: [O] How to change agenda sorting order temporarily?

[Matt Lundin]

"Martin Beck"  writes:

> I'm working with a lot of pre-defined agendas, but I sometimes would
> like to change the sorting order of one of those to different criteria
> (by time, alphanumerically, etc.).
>  
> I did not find a way to do that in org-mode directly - did I overlook
> something, or do I really have to change the configuration of the
> whole agenda and rebuild it?
>  
> Related problem:
> If I change the configuration of the agenda by changing the
> customizing options, those changes do only have an effect, if I kill
> the agenda buffer and launch the agenda again.
> It would be nice, if I just could rebuild the existing agenda with "r"
> or "g", but that does not seem to be sufficient?
>  

Here's a quick function that works for me:

--8<---cut here---start->8---
(defun my-org-agenda-change-sorting-strategy (strategy)
  "Change the sorting strategy."
  (interactive (list
(completing-read "Choose a strategy: "
 (mapcar 'cdr (cdr org-sorting-choice))
 nil t)))
  ;; adjust the following types as needed - e.g., add 'agenda, etc.
  (org-agenda-check-type t 'todo 'tags 'search)
  (let ((org-agenda-sorting-strategy (list (intern strategy
(org-agenda-redo)))
--8<---cut here---end--->8---

Matt

Re: [O] [BUG] org-mode 8.3.x directly opens URL/ follows Timestamp, if Return is clicked

[Matt Lundin]

"Martin Beck"  writes:

> I recently upgraded from org-mode 8.2.x to org-mode 8.3.4 and
> all-in-all, it worked really great without too many side effects.
> Thanks to all of you who contributed and made it such a robust
> upgrade. :-)
>  
> However,  there is one change that is annoying for me every time I
> want to enter a linebreak after a Link, URL or timestamp:
>  
> When I move the cursor after a Link, URL or timestamp and hit return,
> it follows the link and opens the URL or jumps to the date of the
> timestamp in an agenda view.
> I can change that by first entering a space after the link, but I
> often forget that and it's not a really robust workaround for me.
>  
> So I wonder how/when this change happened and if I maybe can get the
> old behavior back?
>  

This bug was fixed on October 31, 2015 with the following commit:

a87c34cb8d58e7019c8d081c02bbac9e93cef8ab

Unfortunately, that commit must have been applied only to the master
branch, as it is not present in the maint branch and thus was not
released as part of org-mode 8.3.4. I'm not sure why that commit was
never merged into maint. I'm flagging this as a bug to ensure that it is
merged.

I think the only solution at the moment would be to upgrade to a more
recent development version either by checkout org out through git or
using elpa:

http://orgmode.org/elpa.html

Matt

Re: [O] Org-agenda-sorting: can I sort by inactive timestamps?

[Matt Lundin]

"Martin Beck"  writes:

> I'm using inactive timestamps to show the creation date of a new
> heading and also to show the modification dates with some short text
> notes about the progress in there:
>  
> Example:
>  
> * Headings Test
> <2016-05-30 Mo 13:18>
> * Heading 1
> [2016-05-30 Mo 13:21]
> * Heading 2
> [2016-05-30 Mo 13:16]
> * Heading 3
> [2016-05-30 Mo 13:16]
> * TODO Find out about sorting by timestamp
> [2016-05-30 Mo 13:12]
> [2016-05-30 Mo 14:22]
> posted question to org-mode users group
>  
>  
> I tried the timestamp-up/timestamp-down sorting criteria in my agenda,
> but they seem to ignore the inactive timestamps:
> ("r" "Refile" tags "+refile+LEVEL=1"
>   ((org-agenda-overriding-header "Refile")
>(org-agenda-sorting-strategy
> (quote
>  (timestamp-up)

> I also tried to use the table-view of the agenda, but in the column
> "timestamp" it only shows the active timestamp of the first heading.
> Questions:
> - is it possible to use inactive timestamps for sorting the agenda?

The timestamp-up here retrieves active timestamps. For inactive
timestamps, you can use tsia-up. See the docstring (C-h v) for
org-agenda-sorting-strategy:

tsia-upSort by inactive timestamp, early first
tsia-down  Sort by inactive timestamp, late first

> - what happens if I have several timestamps (active or inactive) in
> one heading? Will org-mode take the first/last one in the plain text
> "code" or the newest or oldest one by date?

With tsia-up or tsia-down, org will use the first inactive timestamp in
the heading. 

For more details, see...

http://orgmode.org/manual/Special-properties.html

Matt