Re: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-

2022-03-22 Thread Tim Cross


"Tom Davey"  writes:

> Hi Tim, 
>
> Thanks for these thoughtful comments. I agree that the Org developers (to
> whom I, as a mere user, owe enormous thanks) must be wary before making
> changes to how timestamps are handled. 
>
> This argues, I would say, for keeping what I believe was the status quo
> since at least Org version 9.4.4: Agenda views would display entries with
> active timestamps in property drawers. That has been my historical
> experience. 
>
> Tim, has your historical experience been different? In the invoicing example
> you give, were the timestamps in the properties drawer active, or inactive? 
>
> I have just verified with a simple test that Org version 9.4.4, which was
> shipped with Emacs 27.2 I believe, does display entries with an active
> timestamp as the value of a property in the ordinary :PROPERTIES: drawer.
> That's the situation I'm calling the "status quo." I'm wondering if my
> experience coincides with the experience of others. 
>
> Here's the simple entry that will be shown on the Week/Day Agenda view in
> 9.4.4: 
>
> * TODO Test of active timestamps
>:PROPERTIES:
>:Created:  <2022-03-22 Tue 18:30>
>:END:
>
> And note this: adding a second active timestamp to the test entry, e.g., to
> accompany a SCHEDULED: keyword, results in the entry appearing on the Agenda
> twice, as would be expected: 
>
> * TODO Test of active timestamps
>SCHEDULED: <2022-03-22 Tue 18:30>
>:PROPERTIES:
>:Created:  <2022-03-22 Tue 18:30>
>:END:
>
> This second example shows why the variable
> org-agenda-skip-additional-timestamps-same-entry is valuable. I rarely want
> an entry to display twice on the same day. 
>

For my invoicing system, the timestamps are in property draws, but not
under TODO headings, so I guess they wouldn't show up. I have used
timestamps in TODO entry property draws, but don't recall seeing them in
the agenda when I do. However, I also use a couple of custom agenda
views more often than the default agenda view, so I just may not have
noticed.

Personally, I have no use case for TODOs being displayed in the agenda
based on their created date. I do like to log when I created the todo,
but would not want to see that cluttering my agenda where I only want to
see scheduled and deadline tasks. I use a property draw for this type of
information as I consider it metadata about the entry. If I really want
ot see the item in the agenda based on a timestamp, then I explicitly
add that timestamp to the entry (not in  a property draw).

In my invoicing module, property draws are used to track invoicing
metadata i.e. last invoice number, last invoice date, invoice creation
date and invoice period for each invoice, invoice due date, paid datge
etc. Some of the timestamps are 'inactive' (they never change) and some
are active (will get modified). Perhaps that was a mistake on my part
and all of them should be inactive, but at the time, I thought of the
distinction as not being purely about whether they show in the agenda,
but more about whether they were immutable or not. 

The need to have another variable to limit the scope of timestamps when
you do allow timestamps in property draws makes things feel very
'kludgy'. I get very concerned about all these 'enhancements' we keep
adding which then also require additional complexity with multiple
variables interacting at different levels. I find the number of
variables and complexity associated with the agenda compared to when I
first started using org mode to be mind boggling. I often wonder how the
maintainers are able to keep on top of the increasing complexity of org
mode. 

What I definitely would NOT want is for TODO tasks in source blocks
which have a timestamp associated with them showing up in the agenda.
The possible exception might be when the source block is an org source
block. What I want in source blocks is for them to behave like a
microcosm of the original code environment - I want them to behave
exactly as they would when I just edit that code in a code dedicated
buffer with the addition of a minimal set of babel/noweb features to
allow me to work with code spread over multiple source code blocks etc.
I definitely don't want the additional processing overhead of org mode
search through my source blocks looking for things which look like they
might be timestamps or worse, looking for such things and then trying to
do something clever with them (like adding to the agenda).

With regards to timestamps in property draws, I'm not sure that arguing
the status quo is to keep that behaviour. I'm not sure that behaviour
was supposed to work or that it is well defined. It may have worked
previously 'by accident' rather than design (something which tends to
happen more as complexity increases). For example, what is the situation
with timestamps in property draws and inheritance? What is the situation
with timestamps and property draws for items which are not TODOs (i.e.
normal headings with a property 

Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

2022-03-22 Thread Samuel Wales
On 3/22/22, Ihor Radchenko  wrote:
> lax - agenda may match anything date-looking (with or without brackets).
> A specialized regex agenda matcher with regex constructed using current
> agenda date.
>
> Maybe we can introduce a new agenda view? Or maybe a special agenda
> mode, similar to org-agenda-include-inactive-timestamps?

over the years i have seen different opinions on contexts in which
tses [and links, not to make it more complex but to in principle
strive for some type of orthogonality iff discussion goes there]
should show up in agenda and where it should not.

[as a same page type of check, for tses we are talking about
"daily/weekly" which has different semantics from search agenda and
cannot be reproduced with search agenda via parameterization.]

another context [not to create controversy but also for in principle
orthogonality] is line comments, which some think should mean tses and
links therein should not show up in agenda and not fontify and not be
clickable [rule = "remove tses and links from org semantics and
fontification including agenda but not including certain org searchers
like org-occur-in-agend-files"], while others think should mean
"remove from all export" [use case: so you as author can document
exportable stuff inline using comments and still have your tses show
up in daily/weekly if you want that and have clickable links without
having those exposed to the recipient of the exported document, etc.].
links in line comments can be worked around with a bit of code, but
idk about tses.  some will want tses commentable.  some not.

your idea of expanding to other date-like things is an interesting
idea, and so is making an analogy with log mode.

another possibility is to satisfy the preferences users have expressed
[and those with n>1 needs] using a single variable that contains the
contexts that should show in daily/weekly agenda.  org uses multi-item
variables more frequently in recent years [e.g.
org-show-context-detail or visibility for sparse trees] which reduces
vars.  you might have had log mode items in mind, which is similar.

thus in principle it need not be a mode like log mode items, but could
be in agenda custom commnds.

i think a whole view [unlike log mode and tag filtering] might be
confusing to newcomers as [just as with sorting and log mode items and
allow COMMENT and allow ARCHIVE] the regular daily/weekly view is
parameterizable.

i know i got confused by the todo view, wondering if it was something
not creatable with parameterization.

so i would go for your second idea, or the var.  i do think different
preferences exist out there, some strong.

with one or the other, future defaults are trivially changed,
transparent/discoverble by user, and open to user change.

[one problem now being that users can be not merely surprised, but
unaware that stuff is missing from agenda.]

>
> Best,
> Ihor
>



Re: Problem with org-babel and geiser

2022-03-22 Thread Zelphir Kaltstahl

Hello Rudy!

On 3/22/22 21:51, Rudolf Adamkovič wrote:

Zelphir Kaltstahl  writes:

Hello Zelphir!


At some point something changed, I think, […]

In Geiser or in Org?  I use Geiser for work heavily and it works like
magic.  I ask because if the problem resides in Geiser, we could report
a new bug there.

Rudy


Geiser seems to work, if I use /bin/sh with some arguments instead of GNU Bash. 
I am not sure what is going on with that though, because I did not have to do 
that in the past. In org-mode it still does not work. I will have to get into 
the topic again at some point, as I am unsatisfied without having GNU Guile 
source blocks in Org-mode working.

I've successfully used those before in multiple personal or learning projects 
(like 
inhttps://notabug.org/ZelphirKaltstahl/the-little-schemer/src/master/chapter-09-y-combinator/code.org).
 I spent so much time on fixing this problem, that I neglected my actual 
project that I wanted to continue. After 2 or 3 weekends, I left the issue at 
the status in that gitlab issue. I completely blended that one out for now and 
continued by copy pasting into Geiser REPL or using C-c C-r to run regions of 
code. That works at least. But it is less satisfying than doing literate 
programming in org-mode.

If anyone has concrete steps to run, to figure out the problem, I might find the energy to get into the 
matter again. For now I have put it up to "later" / "in the future" / "when some 
update comes" or something like that.

Best regards,
Zelphir

--
repositories:https://notabug.org/ZelphirKaltstahl


Re: citations: org-cite vs org-ref 3.0

2022-03-22 Thread Nicolas Goaziou
Hello,

Max Nikulin  writes:

> Nicolas, I think, a part of problem is that you are not an org-cite
> user. The packages require some *polishing*, but it have to be 
> *user-driven*.

Just to be clear: the development was _user-driven_ from the start.
I read years of discussions about citation, suggested a syntax, waited
for input, amended the syntax… Then I did the same for the whole
library, implementing proof of concepts, waiting for feedback, etc.

> Another point is more serious. Besides citations there are internal
> cross-references. Org supports them but only in a rudimentary form. 
> Actually cross-references are similar to citations in the sense that
> they can have style, prefixes and suffixes, and their appearance
> depends on target properties. Another feature is grouping. However 
> cross-references should not be handled by citation backends, they
> require different handlers. Unfortunately there is no way to define 
> custom "citation" type e.g. "[ref:...]" in addition to "[cite:...]".

There is too little information here for me to understand what
cross-references (or grouping) are, or why they would require different
handlers. In any case, if such thing are deemed necessary in Org Cite,
one can always start a new thread.

The goal of Org Cite is not to do everything, however. Its raison d'être
is to provide building blocks around citations that external packages
can use and extend. Therefore cross-references might belong to an
extension instead.

> To assign additional properties, info "(org) Links in HTML export"
> https://orgmode.org/manual/Links-in-HTML-export.html recommends usage
> of "#+ATTR_HTML", but such technique has several issues:

This is a different issue. Citations are not link, and have a fixed set
of properties.

Regards,
-- 
Nicolas Goaziou



RE: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-

2022-03-22 Thread Tom Davey
Hi Tim, 

Thanks for these thoughtful comments. I agree that the Org developers (to
whom I, as a mere user, owe enormous thanks) must be wary before making
changes to how timestamps are handled. 

This argues, I would say, for keeping what I believe was the status quo
since at least Org version 9.4.4: Agenda views would display entries with
active timestamps in property drawers. That has been my historical
experience. 

Tim, has your historical experience been different? In the invoicing example
you give, were the timestamps in the properties drawer active, or inactive? 

I have just verified with a simple test that Org version 9.4.4, which was
shipped with Emacs 27.2 I believe, does display entries with an active
timestamp as the value of a property in the ordinary :PROPERTIES: drawer.
That's the situation I'm calling the "status quo." I'm wondering if my
experience coincides with the experience of others. 

Here's the simple entry that will be shown on the Week/Day Agenda view in
9.4.4: 

* TODO Test of active timestamps
   :PROPERTIES:
   :Created:  <2022-03-22 Tue 18:30>
   :END:

And note this: adding a second active timestamp to the test entry, e.g., to
accompany a SCHEDULED: keyword, results in the entry appearing on the Agenda
twice, as would be expected: 

* TODO Test of active timestamps
   SCHEDULED: <2022-03-22 Tue 18:30>
   :PROPERTIES:
   :Created:  <2022-03-22 Tue 18:30>
   :END:

This second example shows why the variable
org-agenda-skip-additional-timestamps-same-entry is valuable. I rarely want
an entry to display twice on the same day. 

Tom Davey 

--
Tom Davey
t...@tomdavey.com
New York NY USA

-Original Message-
From: Emacs-orgmode  On
Behalf Of Tim Cross
Sent: Tuesday, March 22, 2022 5:10 PM
To: Ihor Radchenko 
Cc: Ignacio Casso ; emacs-orgmode@gnu.org;
t...@tomdavey.com; Nicolas Goaziou 
Subject: Re: Timestamp parsing inside node properties and other contexts out
of org-element-object-restrictions (was: [BUG] Agenda no longer works for
timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @
/home/ignacio/repos/emacs/lisp/org/)])


Ihor Radchenko  writes:

> Ihor Radchenko  writes:
>
>> After further reading the source code, I figured that agenda is, in 
>> fact, supposed to handle timestamps inside property drawers. Optional 
>> arguments for org-at-timestamp-p imply that, in agenda specifically, 
>> timestamps inside node properties are considered timestamps despite 
>> they are not being parsed as timestamps by org-element.
>
> Even though I fixed the reported issue with agenda not showing 
> headings with matching timestamps inside property drawers, this 
> situation is revealing a big inconsistency in Org mode's handling of
timestamps.
>
> org-at-timestamp-p usage implies that Org syntax for timestamps is not 
> only context-dependent, but also depends on current command!
>
> org-at-timestamp-p is called with non-nil argument in a number of 
> functions in Org:
> - org-clock-timestamps-change
> - org-mouse-delete-timestamp
> - org-mouse-context-menu
> - org-follow-timestamp-link
> - org-get-repeat
> - org-auto-repeat-maybe
> - org-time-stamp
> - ... many more in org.el
>
> So, depending on the current command, Org may on may not treat objects 
> matching org-ts-regexp-both as timestamps.
>
> This situation complicates syntax and makes org-element unreliable 
> when dealing with Org buffers.
>
> Should we just simply allow timestamps to be a part of node property 
> values? Should we _not_ treat timestamp-looking text outside their 
> allowed contexts (like quotes, source blocks, etc) as timestamps?
>

I think we have to be very wary here. I can see any changes here causing
lots of breakage for people. I know for my own use case, I use timestamps a
lot in property draws and various source blocks. I never want any of them
showing up in my agenda.

As an example, I was recently working for a company which required that you
put a timestamp in both a file header and in comments. The format they used
was pretty much the same as an org-mode active timestamp. I use org mode to
tangle the source files I write (as well as manage my client data, such as
todos, invoicing, contacts etc), so these files are searched for agenda
items, but I do not want any of those timestamps causing lines in my agenda
views. These timestamps are most commonly found in source and example
blocks.

I think the only time an org timestamp should be recognised in a source
block is when that source block is an org-mode source block. I don't think
they should ever be 'recognised' in example blocks.  

IN addition, my invoicing solution, which is based on org, uses timestamps
to track invoice periods etc. None of these should ever appear in the
agenda. This information is typically tracked in property draws.

Unfortunately, I think org timestamps is a bit of a mess and we need to be
very careful about further complicating matters. The inability to handle
timezones is a major limitation IMO. The 

Re: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions

2022-03-22 Thread Nicolas Goaziou
Hello,

Ihor Radchenko  writes:

> So, depending on the current command, Org may on may not treat objects
> matching org-ts-regexp-both as timestamps.
>
> This situation complicates syntax and makes org-element unreliable when
> dealing with Org buffers.

This is orthogonal to syntax. I think the docstring of that predicate is
clear: `org-at-timestamp-p' is a convenience function for broader uses
of timestamps, which existed before Element.

> Should we just simply allow timestamps to be a part of node property
> values? Should we _not_ treat timestamp-looking text outside their
> allowed contexts (like quotes, source blocks, etc) as timestamps?

Allowing Org syntax in property values is creating another set of
problems: often the value is really a string that Org shouldn't try to
interpret.

Regards,
-- 
Nicolas Goaziou



Re: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-

2022-03-22 Thread Tim Cross


Ihor Radchenko  writes:

> Ihor Radchenko  writes:
>
>> After further reading the source code, I figured that agenda is, in
>> fact, supposed to handle timestamps inside property drawers. Optional
>> arguments for org-at-timestamp-p imply that, in agenda specifically,
>> timestamps inside node properties are considered timestamps despite they
>> are not being parsed as timestamps by org-element.
>
> Even though I fixed the reported issue with agenda not showing headings
> with matching timestamps inside property drawers, this situation is
> revealing a big inconsistency in Org mode's handling of timestamps.
>
> org-at-timestamp-p usage implies that Org syntax for timestamps is not
> only context-dependent, but also depends on current command!
>
> org-at-timestamp-p is called with non-nil argument in a number of
> functions in Org:
> - org-clock-timestamps-change
> - org-mouse-delete-timestamp
> - org-mouse-context-menu
> - org-follow-timestamp-link
> - org-get-repeat
> - org-auto-repeat-maybe
> - org-time-stamp
> - ... many more in org.el
>
> So, depending on the current command, Org may on may not treat objects
> matching org-ts-regexp-both as timestamps.
>
> This situation complicates syntax and makes org-element unreliable when
> dealing with Org buffers.
>
> Should we just simply allow timestamps to be a part of node property
> values? Should we _not_ treat timestamp-looking text outside their
> allowed contexts (like quotes, source blocks, etc) as timestamps?
>

I think we have to be very wary here. I can see any changes here causing
lots of breakage for people. I know for my own use case, I use
timestamps a lot in property draws and various source blocks. I never
want any of them showing up in my agenda.

As an example, I was recently working for a company which required that
you put a timestamp in both a file header and in comments. The format
they used was pretty much the same as an org-mode active timestamp. I
use org mode to tangle the source files I write (as well as manage my
client data, such as todos, invoicing, contacts etc), so these files are
searched for agenda items, but I do not want any of those timestamps
causing lines in my agenda views. These timestamps are most commonly
found in source and example blocks.

I think the only time an org timestamp should be recognised in a source
block is when that source block is an org-mode source block. I don't
think they should ever be 'recognised' in example blocks.  

IN addition, my invoicing solution, which is based on org, uses
timestamps to track invoice periods etc. None of these should ever
appear in the agenda. This information is typically tracked in property
draws.

Unfortunately, I think org timestamps is a bit of a mess and we need to
be very careful about further complicating matters. The inability to
handle timezones is a major limitation IMO. The inability to handle
daylight savings transitions in a consistent manner (particularly for
calculation of periods, duration, etc) is often a source of errors and
if you are unfortunate enough to regularly travel across different
timezones, forget about using org mode to manage your schedule.

I have done several deep dives into org-mode's timestamp stuff, but
usually come back up gasping for air. Managing data and time data is
often far more complicated than it may appear on the surface. I think we
need to be extremely conservative when considering changes in this area
(it is the main reason I've never managed to find a way to add time zone
data - every solution I could think of was either really high on the
level of breakage and frustration it would create for users or it
adversely impacted the convenience/usability of org timestamps). 




Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

2022-03-22 Thread Tim Cross


Ihor Radchenko  writes:

> Samuel Wales  writes:
>
>> so i guess i am interested in the rationale.  example of ts [and text]
>> search being useful might be an example or ledger block that contains
>> ledger source, or something like that.  i can get why bare ts not
>> being matched inside links might be useful.
>>
>> otoh i can get why text [not ts] and isearch search inside of links
>> can be useful.
>>
>> idk if my intuitions match those of others.  i am partly trying to
>> think of what a newcomer might expect to work [and the simplicity of a
>> rule that he or she would have to remember in order to know what the
>> behavior will be].
>
> I can see your point. Simple timestamp matching by regexp disregarding
> timestamp objects might be useful. However, I do not think that matching
> should be via timestamps in such a case. Rather it should be even more
> lax - agenda may match anything date-looking (with or without brackets).
> A specialized regex agenda matcher with regex constructed using current
> agenda date.
>
> Maybe we can introduce a new agenda view? Or maybe a special agenda
> mode, similar to org-agenda-include-inactive-timestamps?
>

Perhaps I simply don't understand, but I fail to see the need for adding
such functionality. Org files are plain text and you can just use the
build-in or any of the add-on search tools to search for anything,
including timestamps or things which may look like timestamps.

The only time org needs to provide search functionality is when that
search does need to be done within the context of a date/time object.



Re: Re: Problem with org-babel and geiser

2022-03-22 Thread Rudolf Adamkovič
Zelphir Kaltstahl  writes:

Hello Zelphir!

> At some point something changed, I think, […]

In Geiser or in Org?  I use Geiser for work heavily and it works like
magic.  I ask because if the problem resides in Geiser, we could report
a new bug there.

Rudy
-- 
"One can begin to reason only when a clear picture has been formed in
the imagination."
-- Walter Warwick Sawyer, Mathematician's Delight, 1943

Rudolf Adamkovič  [he/him]
Studenohorská 25
84103 Bratislava
Slovakia



Re: [PATCH] Re: [BUG] Hard-coded begin/end in org-insert-structure-template [9.5.2 (release_9.5.2-24-g668205 @ /Users/salutis/src/emacs/nextstep/Emacs.app/Contents/Resources/lisp/org/)]

2022-03-22 Thread Rudolf Adamkovič
Ihor Radchenko  writes:

> We can auto-magically determine whether to use BEGIN_ or begin_
> depending on the case in template type. Tentative patch attached.

Interesting!

> +with \"#+begin_\" and \"#+end_\" added automatically.  If the block
> +type is written using upper case letter, \"#+BEGIN_\" and \"#+END_\"
> +are added instead.

FYI: A typo in "letter" (expected "letters")

(We could also simplify to: "if the block type consists of just
uppercase letter".)

> +  (let* ((case-fold-search t) ; Make sure that matches are case-insnsitive.
> + (region? (use-region-p))

FYI: A typo in "case-insnsitive" (expected "case-insensitive")

Looking forward to seeing this merged!

Rudy
-- 
"I love deadlines.  I love the whooshing noise they make as they go by."
-- Douglas Adams, The Salmon of Doubt, 2002

Rudolf Adamkovič  [he/him]
Studenohorská 25
84103 Bratislava
Slovakia



Re: citations: org-cite vs org-ref 3.0

2022-03-22 Thread Bruce D'Arcus
On Tue, Mar 22, 2022 at 10:42 AM Max Nikulin  wrote:
>
> On 21/03/2022 22:19, Bruce D'Arcus wrote:
> > On Mon, Mar 21, 2022 at 10:41 AM Max Nikulin wrote:
> >
> >> A bit of routine work will alleviate some user issues:
> >> - add missed styles
> >
> > The initial list of style-command mappings was pretty comprehensive,
> > but we left out some of the more obscure biblatex commands because
> > unsure if they were needed, or how best to add them (conceptually
> > there's a mix of different kinds of commands in biblatex, which are
> > hard to fit into a more general style system, for example).
> >
> > Since then:
> >
> > - people have occasionally asked to add new mappings, and Nicolas has added 
> > them
> > - he's also added the styles defcustoms for biblatex, so users can do
> > this themselves
> >
> > In short, I think we're good on this actually.
>
> John Kitchin, this thread, Sun, 20 Mar 2022 20:31:29 -0400.
> https://list.orgmode.org/m2sfrc149c@andrew.cmu.edu:
>
> > I don't know the equivalent of \citenum in CSL.

Right; so John or someone else should send a message to the list
requesting it specifically?

That's how we've done it so far anyway. Not sure what the practical
alternative is.

I will say that much of this discussion is not about org-cite per se,
but rather implementation decisions in the specific bundled
processors. I fear these conversations get hard to track if we aren't
precise about what we're talking about; citation syntax, included
processor functionality or style mappings, documentation, etc.

>  From my point of view it may be a reason to add a new style to
> defaults. It is important whether a tool works out of the box.

But the question here becomes "works for whom?" Citations practices
vary a fair bit by field.

Primarily LaTeX/natbib users like John and other science folks,
biblatex users primarily coming from the humanities as near as I can
tell? Or people coming from markdown and pandoc (which has an
excellent citation system that clearly inspired org-cite) and needing
Word-ready documents for journals etc?

The style system in the bundled processors is designed to work for all
of them, and not privilege one or another. It does this by creating a
new, more general, org-cite style and variant system, which then maps
to different targets.

That approach has obvious advantages: citations are (mostly) decoupled
from particular export processors. One can write a document with these
styles, and get PDF citation and bibliographic output via biblatex,
for example, similar to the HTML or ODT output from the CSL export
processor.

And that also has advantages beyond org. For example, pandoc recently
added support for org-cite, and that includes mapping between the two
style/command systems. So in theory you could convert an org file to
docx, letting pandoc process the citations along the way, and the
result would more than acceptable.

> Custom variables make a document less portable unless they are specified as
> file-local ones.

Indeed!

Or more specifically here, it ties the citations to the export processor.

But maybe for some or many people that's perfectly fine.

One possible idea to consider is to allow two systems in each of
LaTeX-oriented processors: what we might call a default "org-cite"
one, and an optional "literal" one. So if you only use oc-natbib, and
you want the natbib commands directly, you might change a variable to
get that.

I really don't know if that's a good idea or not, but I raise it to
emphasize there's a lot of flexibility with the org-cite design, and
so it's just a question of how we make use of it.

> I think, the goal may be formulated as "John can not
> say the following any more" (at least in respect to citations leaving
> aside cross-references):
>
> > I simply cannot
> > compromise on the capability org-ref provides me, or wait for an
> > alternative complete solution in org-mode.

I don't really see why these are the two choices.

> On the other hand I do not consider the following argument as a strong one
>
> > I do not like the abstraction away from LaTeX cite commands in org-cite.
> > This is an example of a compromise between LaTeX and CSL.
>
> despite I believe that convenience and habits are important. Mapping of
> styles to commands is just a piece of knowledge.

Yes.

> I have no particular opinion if enough efforts should be invested from
> both sides to allow mixing on both citation syntax constructs (org-cite
> and org-ref) in the same document. Bruce, you made a lot for support of
> CSL in org-cite, so I will stressed another direction of feature
> comparison since Bib(La)TeX users should feel themselves first-class
> citizens.

Absolutely! In fact, Denis Meier, who has extensive biblatex
knowledge, helped a lot with the initial mapping there, and some other
biblatex users have contributed feedback along the way.


Bruce

> The choice between org-ref and org-cite, when the former can do its job,
> should be matter of taste and 

Re: [PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library

2022-03-22 Thread psychosis

Hello,
On Di, Mär 22 2022 at 20:27:27 +0800, Ihor Radchenko 
 wrote:



To clarify, org-cite must detect changes in your .bib file and if a
citation record key in the .bib files was changed, trying to insert it
should trigger org-cite to mark the new citation in red. Regardless if
the citation gets inserted at the same of other line.

Or maybe I misunderstand what you did.


Ok, for me it does not seem to work as intended. The old cite-key turns 
immediately red if and if only I insert the new cite-key on the same 
line. For it to turn red if I insert it on other lines, I have to kill 
the buffer and reload it. I used “emacs -Q” with the attached files.



Kind regards

Paul


@book{abelson_et_al_2000,
  title = {Structure and Interpretation of Computer Programs},
  author = {Abelson, Harold and Sussman, Gerald Jay and Sussman, Julie},
  date = {2000},
  publisher = {{MIT P}},
  abstract = {2012}
}


#+bibliography: ~/org/test.bib
[cite:@abelson_et_al_2012]
test
[cite:@abelson_et_al_2000]


Re: Move or rename a file in a link

2022-03-22 Thread João Pedro de Amorim Paula
On 19 March 2022 11:25, Juan Manuel Macías  wrote:

> #+begin_src emacs-lisp
>   (defun my-org-rename-link-file-at-point ()
> (interactive)
> (let* ((curr-dir (if (equal (org-element-property :type 
> (org-element-context)) "attachment")
>(concat (abbreviate-file-name (org-attach-dir)) "/")
>  (abbreviate-file-name default-directory)))
>  (current-path (if (equal (org-element-property :type 
> (org-element-context)) "attachment")
>(concat curr-dir (org-element-property :path 
> (org-element-context)))
>  (org-element-property :path (org-element-context
>  (new-path (read-file-name "Rename file at point to: " current-path)))
>   (rename-file current-path new-path)
>   (message (concat "moved to: " new-path))
>   (if (directory-name-p new-path)
> (setq new-path (concat new-path (file-name-nondirectory 
> current-path)))
>   (setq new-path new-path))
>   (if (equal (org-element-property :type (org-element-context)) 
> "attachment")
> (my-org-replace-link-file (file-name-nondirectory current-path)
>   (replace-regexp-in-string
>curr-dir "" new-path))
>   (my-org-replace-link-file current-path
> (replace-regexp-in-string
>  curr-dir "" new-path)
>
>   (defun my-org-replace-link-file (from to)
> (save-excursion
>   (goto-char (point-min))
>   (while (re-search-forward org-bracket-link-regexp nil t)
>   (cond ((string-match-p (concat "attachment:" from) (match-string 1))
>  (replace-match (concat "[[attachment:" to "]]")))
> ((string-match-p from (match-string 1))
>  (replace-match (concat "[[file:" to "]]")))
> #+end_src

Just a couple of fixes and it seems to be working fine.

diff -u a/org-rename-link.el b/org-rename-link.el

--- a/org-rename-link.el	2022-03-22 12:08:45.790727092 -0300
+++ b/org-rename-link.el	2022-03-22 12:25:31.500218923 -0300
@@ -23,8 +23,11 @@
 (defun my-org-replace-link-file (from to)
   (save-excursion
 (goto-char (point-min))
-(while (re-search-forward org-bracket-link-regexp nil t)
-  (cond ((string-match-p (concat "attachment:" from) (match-string 1))
- (replace-match (concat "[[attachment:" to "]]")))
-((string-match-p from (match-string 1))
- (replace-match (concat "[[file:" to "]]")))
+(while (re-search-forward org-link-bracket-re nil t)
+  (replace-match
+   (cond
+((string-match-p (format "^attachment:%s\\'" from) (match-string 1))
+ (format "attachment:%s" (file-name-nondirectory to)))
+((string-match-p (format "^file.*:%s\\'" from) (match-string 1))
+ (format "file:%s" to)))
+   nil nil nil 1

Diff finished.  Tue Mar 22 12:26:56 2022

All this patch does is to remove the directory name from the
=attachment:file.ext= -- given that the path is calculated by org-attach
--, and use the group 1 of the match string (as well as updating the use
of an obsolete variable), which corresponds to the actual link portion,
leaving the description unchanged.

Cordially,

-- 
João Pedro de A. Paula
IT undergraduate at Universidade Federal do Rio Grande do Norte (UFRN)


Re: citations: org-cite vs org-ref 3.0

2022-03-22 Thread Max Nikulin

On 21/03/2022 22:19, Bruce D'Arcus wrote:

On Mon, Mar 21, 2022 at 10:41 AM Max Nikulin wrote:


A bit of routine work will alleviate some user issues:
- add missed styles


The initial list of style-command mappings was pretty comprehensive,
but we left out some of the more obscure biblatex commands because
unsure if they were needed, or how best to add them (conceptually
there's a mix of different kinds of commands in biblatex, which are
hard to fit into a more general style system, for example).

Since then:

- people have occasionally asked to add new mappings, and Nicolas has added them
- he's also added the styles defcustoms for biblatex, so users can do
this themselves

In short, I think we're good on this actually.


John Kitchin, this thread, Sun, 20 Mar 2022 20:31:29 -0400. 
https://list.orgmode.org/m2sfrc149c@andrew.cmu.edu:



I don't know the equivalent of \citenum in CSL.


From my point of view it may be a reason to add a new style to 
defaults. It is important whether a tool works out of the box. Custom 
variables make a document less portable unless they are specified as 
file-local ones. I think, the goal may be formulated as "John can not 
say the following any more" (at least in respect to citations leaving 
aside cross-references):



I simply cannot
compromise on the capability org-ref provides me, or wait for an
alternative complete solution in org-mode.


On the other hand I do not consider the following argument as a strong one


I do not like the abstraction away from LaTeX cite commands in org-cite.
This is an example of a compromise between LaTeX and CSL.


despite I believe that convenience and habits are important. Mapping of 
styles to commands is just a piece of knowledge.


I have no particular opinion if enough efforts should be invested from 
both sides to allow mixing on both citation syntax constructs (org-cite 
and org-ref) in the same document. Bruce, you made a lot for support of 
CSL in org-cite, so I will stressed another direction of feature 
comparison since Bib(La)TeX users should feel themselves first-class 
citizens.


The choice between org-ref and org-cite, when the former can do its job, 
should be matter of taste and personal preferences unrelated to 
technical limitations.





Re: citations: org-cite vs org-ref 3.0

2022-03-22 Thread indieterminacy
Hi Juh,

juh  writes:

> [[PGP Signed Part:Undecided]]
> Am Sun, Mar 20, 2022 at 08:31:29PM -0400 schrieb John Kitchin:
>> For those who need high fidelity LaTeX export like I do, I think
>> org-ref is still a superior solution. For everyone else, and
>> especially if you do not need sophisticated cross-references and don't
>> want the dependencies of org-ref, org-cite is likely the better
>> solution. 
>
> Thank you very much for the clarification. 
>
> I am new to both and I need something that works together with ConTeXt
> export. Currently I am happy with org-cite. I used org-ref before but
> never got good results in ConTeXt export. So I think that I will stick
> wit org-cite. 
>

I used to use ConTeXt and have a healthy respect for it.
That feedback is good to hear.

> Org-mode is not LaTeX but I wonder if it can be a better replacement for
> my Markdown-->Pandoc-->EPUB/ConTeXt single source publication process.
> Better as I could make more use of org-roam in my writing process.
>

May I recommend the mininimal Markdown format, GemText?
It has a very simple syntax which is easily convertable into Orgmode
syntax.

It could be useful for projects where the flow starts with simple
written prose.

> Though I am a long time emacs user putting all these blocks together is
> a challenge, still.
>
> With your words I now see much much clearer.
>
> juh


Jonathan



[BUG] org-complex-heading-regexp should consider COMMENT keywords [9.5.2 (release_9.5.2-25-gaf6f12 @ /home/ignacio/repos/emacs/lisp/org/)]

2022-03-22 Thread Ignacio Casso
Hello,

I think `org-complex-heading-regexp' and
`org-complex-heading-regexp-format' should consider COMMENT keywords, as
they do with TODO keywords, priorities, and tags.


Firstly, with things as they are, different methods for accessing the
headline text are inconsistent. For example, for the headline below,
(org-entry-get (point) "ITEM"), which uses `org-complex-heading-regexp',
returns "COMMENT My headline", while (org-element-property :title
(org-element-context)), which doesn't, returns "My headline".

*** TODO [#A] COMMENT My headline   :my_tag:


But more importantly for me, this is a problem in particular with
org-capture, where I would like capture targets to be found regardless
of whether they are commented or not. But right now commented headlines
are only found if the COMMENT keyword also appears in the capture
template, and in that case if the headlines are later uncommented they
are no longer found. You can copy the entry below into an org file and
play around with it to reproduce this described behavior.

* COMMENT My Target
  #+begin_src emacs-lisp
(let ((org-capture-templates
   `(("d" "My capture template" entry
  (file+headline ,buffer-file-name "My target")
  "* My content"
  :immediate-finish t
  (org-capture nil "d"))
  #+end_src

The cause of this is the use of `org-complex-heading-regexp-format' in
`org-capture-set-target-location'.

This is the bug I originally wanted to report, but after
exploring the cause I thought that the problem was really in
`org-complex-heading-regexp' and not org-capture's code.

What do you think? Do you know of any other uses of those variables and
how considering COMMENT keywords or not affects them?

If you agree I can send a patch that changes the default value of those
regexp variables. Otherwise, I can send a patch that just modifies the
code of `org-capture-set-target-location' to fix this only for the
org-capture use case.

Regards,

Ignacio


Emacs  : GNU Emacs 29.0.50 (build 13, x86_64-pc-linux-gnu, GTK+ Version 
3.24.20, cairo version 1.16.0)
 of 2022-03-21
Package: Org mode version 9.5.2 (release_9.5.2-25-gaf6f12 @ 
/home/ignacio/repos/emacs/lisp/org/)



Re: [PATCH] Re: [BUG] org-cite: 10 second hang opening a ~4k org file with 10MB bibtex library

2022-03-22 Thread Ihor Radchenko
psychosis  writes:

>> I assume that your bibliography is much larger than 10Mb. Am I 
>> correct?
>
> Actually, it is only 2Mb (7000 entries). Probably there is a problem 
> with the data structure of my .bib-file? I create it from Zotero and 
> the BibLaTeX-export of the BetterBibTeX-Plugin.

> * ELP Profiling Results
>
> org-cite-basic--parse-bibliography  37183   14.256997996  0.0003834278
> org-cite-basic--get-entry   37180   10.932912038  0.0002940535
> org-cite-basic-activate 1   3.796198629   3.796198629
> org-cite-basic--all-keys2   3.655586111.827793055

> I attached the profiler-reports and the ELP-reports for “emacs -Q” 
> with and without the patch. I use the latest Emacs version, built  with 
> “native-compilation” and “pgtk” (GNU Emacs 29.0.50, 
> x86_64-pc-linux-gnu, GTK+ Version 3.24.31, cairo version 1.17.4; Org 
> mode version 9.5.2, release_9.5.2-24-g668205) with an .org-file that 
> only contains one citation and the location of the bibliography. The 
> CPU is an Intel Core i5-3320M @ 2.60GHz × 4.

Judging from ELP data, most of the time is spent fontifying individual
citations, not reading the .bib file. And the fontification gets called
37k times! This is fishy. It is not supposed to happen.

Could you provide your example file and the relevant .bib record?

>> This is bad. Would you mind constructing a reproducer? (using the
>> patched Org version)
>
> I made a mistake, I inserted the changed cite-key on a new line, which 
> does not work. Inserting the new key on the same line as the old one 
> works. I don’t know if this is the intended behavior.

To clarify, org-cite must detect changes in your .bib file and if a
citation record key in the .bib files was changed, trying to insert it
should trigger org-cite to mark the new citation in red. Regardless if
the citation gets inserted at the same of other line.

Or maybe I misunderstand what you did.

>> 2. You may provide profiling information using M-x profiler-start M-x
>> profiler-report and using ELP as I described in the previous messages.
>
> Thank you very much for your help! I should add, I am content with the 
> improvement of the parsing time that your patch provides. So if you 
> assume that the problem is only on my end, please attend to more 
> pressing issues.

It's fine. I plan to work on org-cite in future, so I use such bug
reports as an opportunity to familiarise myself with the source code.

Best,
Ihor




Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

2022-03-22 Thread Ihor Radchenko
Samuel Wales  writes:

> so i guess i am interested in the rationale.  example of ts [and text]
> search being useful might be an example or ledger block that contains
> ledger source, or something like that.  i can get why bare ts not
> being matched inside links might be useful.
>
> otoh i can get why text [not ts] and isearch search inside of links
> can be useful.
>
> idk if my intuitions match those of others.  i am partly trying to
> think of what a newcomer might expect to work [and the simplicity of a
> rule that he or she would have to remember in order to know what the
> behavior will be].

I can see your point. Simple timestamp matching by regexp disregarding
timestamp objects might be useful. However, I do not think that matching
should be via timestamps in such a case. Rather it should be even more
lax - agenda may match anything date-looking (with or without brackets).
A specialized regex agenda matcher with regex constructed using current
agenda date.

Maybe we can introduce a new agenda view? Or maybe a special agenda
mode, similar to org-agenda-include-inactive-timestamps?

Best,
Ihor



Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

2022-03-22 Thread Ihor Radchenko
Samuel Wales  writes:

> - not quite grasp why bare active and inactive tses should not be
> matched by ts agenda almost anywhere including blocks.  perhaps the
> idea is that you might wnt to store lots of data with tses and they
> would be too much clutter?

See
https://orgmode.org/list/20220101122409.ga29...@itccanarias.org

Best,
Ihor



Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g

2022-03-22 Thread Ihor Radchenko
Ihor Radchenko  writes:

> After further reading the source code, I figured that agenda is, in
> fact, supposed to handle timestamps inside property drawers. Optional
> arguments for org-at-timestamp-p imply that, in agenda specifically,
> timestamps inside node properties are considered timestamps despite they
> are not being parsed as timestamps by org-element.

Even though I fixed the reported issue with agenda not showing headings
with matching timestamps inside property drawers, this situation is
revealing a big inconsistency in Org mode's handling of timestamps.

org-at-timestamp-p usage implies that Org syntax for timestamps is not
only context-dependent, but also depends on current command!

org-at-timestamp-p is called with non-nil argument in a number of
functions in Org:
- org-clock-timestamps-change
- org-mouse-delete-timestamp
- org-mouse-context-menu
- org-follow-timestamp-link
- org-get-repeat
- org-auto-repeat-maybe
- org-time-stamp
- ... many more in org.el

So, depending on the current command, Org may on may not treat objects
matching org-ts-regexp-both as timestamps.

This situation complicates syntax and makes org-element unreliable when
dealing with Org buffers.

Should we just simply allow timestamps to be a part of node property
values? Should we _not_ treat timestamp-looking text outside their
allowed contexts (like quotes, source blocks, etc) as timestamps?

Best,
Ihor



Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

2022-03-22 Thread Ihor Radchenko
Ignacio Casso  writes:

>>> What you see in the new Org version is not a bug. Property values are
>>> treated as plain text by Org.
>
> I think that even if timestamps were never intended to be used inside
> property drawers before, the fact that it worked for a long time and
> nothing in the documentation suggested otherwise makes it a de facto
> feature, even if unintended, and should be preserved.

After further reading the source code, I figured that agenda is, in
fact, supposed to handle timestamps inside property drawers. Optional
arguments for org-at-timestamp-p imply that, in agenda specifically,
timestamps inside node properties are considered timestamps despite they
are not being parsed as timestamps by org-element.

> I've located the line in org-agenda.el responsible of the new behavior,
> and the following patch seems to fix it. I suggest it is incorporated
> into the repository, maybe with a variable
> org-agenda-skip-timestamps-in-properties-drawer defaulting to t if not
> everyone agrees.
> -(not (eq 'timestamp (org-element-type 
> (org-element-context)
> +(not (memq (org-element-type (org-element-context)) 
> '(timestamp node-property

I pushed a different version of the patch using org-at-timestamp-p to
bugfix as d9bf64f06.

Best,
Ihor