Re: ob-shell intentions and paperwork (was Bash results broken?)

[Max Nikulin]

On 13/01/2023 22:23, Ihor Radchenko wrote:

Matt writes:


Would you like me to correct how I've incorporated my changes?


No. I was referring to the initial situation with a single commit being
displayed.

I am not sure what Max was trying to point out.


Look at the commit message for
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=4f319088ba5f11d4b6adf808f39f11dfa52c08e4


Merge branch 'ob-shell-cleanup-tests'


It suggests that this is a merge commit for a local branch. Notice that 
the commit has 2 parents:



committer   Matthew Trzcinski 2023-01-11 11:16:53 -0500
parent  07c63df9c7f40b17acb5c517baa0d321098f31da (diff)
parent  9ed2821053cf7b0c2f02fe941a54e1d3a9837aeb (diff)


Second one:

https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=9ed2821053cf7b0c2f02fe941a54e1d3a9837aeb

authorMatt Trzcinski 2022-12-30 13:03:59 -0500
committer Matt Trzcinski 2022-12-31 13:56:27 -0500

test-ob-shell.el: Organize tests

And changes made by this commit are included into diff shown for the 
merge commit 4f319088ba by cgit. E.g. gitk for local repository does not 
show any changes for the merge commit.


So Matt did not squashed commits before committing to the main branch 
and detailed commit messages are preserved. That is why I do not 
consider cgit render as a strong enough reason for reverting.


However I would prefer linear commit history when possible, so I suggest 
to do the following in similar cases (not verified, may have typos)


git fetch  # get latest commits
git checkout ob-shell-cleanup-tests
git rebase origin/main
git checkout main
git pull
git merge --ff-only ob-shell-cleanup-tests

(Omit --ff-only if it is real merge because the fix is committed to the 
bugfix branch.)


It should help to avoid confusion and to make git archeology easier.

Re: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda

[Max Nikulin]

On 14/01/2023 02:06, Jean Louis wrote:

This is good for review as related to PostgreSQL database:


I agree that PostgreSQL is an example of good implementation of 
time-related calculations.



https://wiki.postgresql.org/wiki/Don%27t_Do_This#Don.27t_use_timestamp_.28without_time_zone.29


I do not find this section enlightening in the context of this 
discussion. It is more related to storage formats of DB than to higher 
level aspects suitable for applications.



It would mean that timestamps become time zone aware. Something like this:

<2023-01-15 Sun +1>


Welcome to another set of issues.

First of all, you have just date without specific time point. It may 
mean particular calendar day irrelevant to time zone. ICQ had a bug when 
they decide to store birth dates as timestamps. Significant fraction of 
users received automatic greetings on wrong day, a kind of off by one 
error. So dates must stored as dates omitting time and zone.


Let's assume <2023-01-15 Sun 09:00 +1>

It may be suitable for timestamps in the past, but future is more 
tricky. There is no problem if you are going to watch Lunar eclipse. 
However if your plan is to attend a local event there is a chance that 
you will arrive at wrong time. Sometimes offset of timezones is changed 
and it may happen between the moment when you added a scheduled time and 
the moment of the event.


Notice that Org timestamps already associated with a timezone, the one 
is set in libc (system timezone, or the one set for particular process). 
So daylight saving time and administrative time transitions should be 
taken into account. So Org timestamps may be ambiguous (mostly) 1 hour 
per year around backward time transition, e.g. for timezones having DST. 
This case explicitly specifying time offset helps to avoid uncertainty.

Re: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda

[tomas]

On Sat, Jan 14, 2023 at 12:51:31AM -0500, Tom Gillespie wrote:
> Without wading too far into this at the moment,
> timezones are an extremely tricky problem with
> a whole bunch of design considerations. I am
> reproducing the heading comment from laundry's
> timestamp.rkt in its entirety here. Best!
> Tom
> 
> https://github.com/tgbugs/laundry/blob/master/laundry/grammar/timestamp.rkt

[...]

> ; are two cases, one where the location is clear, "napoleon on [1812-01-01]"
> ; and the other where it is not, the issue is that a single date refers

Where location also includes time, sometimes (think DSTs). IMO, a timezone
is nice for added context, but meaningless without a concrete offset wrt
some agreed upon base (UTC, here on Earth, for example). Unless you use
clearly different shorthands (e.g. CST for Central Standard Time [1]
and CDT for Central Daylight Time).

The nice cherry on top of that is that this context info is sometimes
relevant and sometimes not. A regular appointment ("brush teeth") may
want to be seen relative to the current time zone (so it "moves" when
I travel from Berlin to New York), another ("Jitsi meeting with my
colleagues in Delhi, Dar-Es-Salam and Denpasar") might not.

I warmly recommend the Wikipedia page [2] on that topic.

An interesting problem :)

Cheers

[1] Eh. Nevermind that it also can mean Cuba Standard Time or China
   Standard Time.
[2] https://en.wikipedia.org/wiki/Time_zone

-- 
t


signature.asc
Description: PGP signature

Re: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda

[Daryl Manning]

I agree this is perhaps not a trivial undertaking. It really depends on the
quality of date and time libraries that exist in the lisp ecosystem. For
example, in Rust I found my timezone woes in one of my own apps were
trivial due to the chronos package, but likewise the reliance of Go
depending on its underlying time and date type actually introduced a weird
timezone and DST bug in my own app.

Is/are there lisp packages which handle the complexity involved in time and
date difference calculations and DST changes? That would perhaps be the
first question. It makes things tremendously easier and would reduce much
of the work (perhaps to just the argument of the format and parsing the
datetime stamp and making org-agenda aware of which timezone it's in.).

To Jean Louis' point: using timestamps without timezones is a don't in much
of computing these days and perhaps hearkens back to the simpler age emacs
and org originated in.  =]  (kidding!).

My lips-fu is not adequate to taking this on myself so the issue of who
would want ot work on it over other features is perhaps the bigger
question. Could someone scope out the work and approach at least. I imagine
it is tricky and non-trivial, but perhaps less complex if good libraries
exist as above.

I'd simply make the timezone format a slowly to be evolved to standard over
time with exceptions as noted in my first note. not having tz in timestamps
was a "assume local" assumption and under-specification (or considered
"hidden".).

Daryl.


On Fri, Jan 13, 2023 at 8:34 PM Tim Cross  wrote:

>
> Daryl Manning  writes:
>
> > Following on from thread at
> https://www.reddit.com/r/orgmode/comments/zrppqw/
> >
> > [First off, I just wanted to say thank you to everyone that works on
> org-mode. It is a wonder.]
> >
> > While I realize a few kicks at this can may have been taken, I wanted to
> (re-)propose Timezone support in org-mode. The
> > world is much less local these days and we're all more remote and
> coordinating globally these days.
> >
> > *Background*
> >
> > 1. org-time-stamp-formats TZ currently only affects display and exports
> > 2. org-agenda itself is not TZ aware
> > 3. Several discussions on this have taken place over time
> > 4. Concerns raise included breaking backwards compatibility
> >
> > *Proposal*
> >
> > 1. org-mode sets an optional variable (org-timezone-aware t) which
> enables TZ
> > 2. org-agenda needs a way to determine which timezone it is in
> > 3. Once enabled, any timestamp not exhibiting a TZ in it is considered
> "local time" wherever that is (I do not think UTC
> > would work for this)
> > 4. org-agenda can calc local based on TZ differences
> >
> > I understand this is by no means trivial and quite gnarly with DST and
> such to figure out but I do believe libs exists to
> > deal with that heavy lifting. Currently, it does feel like a hole in
> org-mode as a 21st century organizer (disclaimer:
> > digital nomading so might feel it more keenly). Also, just interested in
> making org-mode a more awesome tool for
> > everybody.
> >
> > I'd love an understanding of the alluded to reservations raised in the
> reddit thread and in the mailing list threads
> > mentioned that the format change might break things (I was unsure if
> that was referring to say, how time ranges were
> > handled, or how say date ranges got dealt with (for example, say a
> flight from Singapore to Vancouver which takes off in
> > one time zone and lands in another - something that is often in my
> cal.).
> >
>
> I agree this would be a great feature to add. However, after having
> looked at it in some detail, I realise that not only is it not a trivial
> task, it is actually a very large and complex task and will require
> extensive work which will almost certainly result in breakage with
> regards to backwards compatibility.
>
> At the risk of over simplifying the matter, I would suggest the big
> challenge here is that there are two somewhat competing (and
> conflicting) use cases. On one hand, you want a high level abstraction
> which makes working with dates and times easy and clear. TO some extent,
> that is what we have now. On the other hand, we need something far more
> complex which is able to handle multiple time zones. This means being
> able to handle both base timezone offsets as well as offset adjustments
> for things like daylight savings time. There is a lot of hidden
> complexity here. You have to handle the fact that daylight saving
> chang-over dates/times are dynamic i.e. not necessarily the same every
> year. This adds the additional complexity of having to somehow track
> historical information regarding such changes, which isn't as readily
> accessible in a consistent manner on all platforms.
>
> Then there is the other 'messy' stuff. For example, when calculating
> time ranges and number of days, hours/ minutes between two dates you
> need to remember to add/remove the hour if the range crosses over a
> daylight savings period. Similarly 

Re: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda

[Tom Gillespie]

Without wading too far into this at the moment,
timezones are an extremely tricky problem with
a whole bunch of design considerations. I am
reproducing the heading comment from laundry's
timestamp.rkt in its entirety here. Best!
Tom

https://github.com/tgbugs/laundry/blob/master/laundry/grammar/timestamp.rkt

; the org spec is currently silent on how org will handle dates beyond
;  and , the relevant standards for expanded representations
; are ISO 8601-1:2019,5.2.2.3 and ISO 8601-2:2019,8.4.3. The problem
; with expanded representations is that they require all parties to
; agree ahead of time how wide the year is and prefix with + and
; leading zeros, I am not sure that this is as robust as we would like
; -01-01 is january first of the year zero, -0001-01-01 is jan
; first of the year 1BCE, and +1-01-01 for jan 1 of 1CE 5
; digits must be prefixed by the plus sign, the standard is not
; entirely clear, but I think that we could do PLUS digits HYPHEN
; or DIGIT-4 HYPHEN and everything would work out

; on a separate but related note, org timestamps are always in local
; time and do not currently support timezones, which is a problem

; further absurdities that are related to the timezone issue: the
; date/time implementation as a text format completely fails if
; authoring on another planet

; given my objective to ensure that org documents can be interpreted
; without having to stick stupid things like #+planet: mars in the
; header or risk your earthling readers getting incorrect dates --- I
; suggest that org switch to storing all dates and times in earth zulu
; time or with an offset as it stands, the timezone setting of the
; computer drafting the document must be known to determine what day
; it was etc. users could still draft using <2020-12-16> but C-c C-c
; would add the correct offset? actually this is tricky, because there
; are two cases, one where the location is clear, "napoleon on [1812-01-01]"
; and the other where it is not, the issue is that a single date refers
; to as many different time intervals as there are timezones and without
; knowing the timezone you don't know the date, similar issue if we were
; to try to convert to sols
; https://www.eecis.udel.edu/~mills/missions.html
; https://www.eecis.udel.edu/~mills/ipin.html
; https://en.wikipedia.org/wiki/Timekeeping_on_Mars
; HOWEVER having applied a bit more brain power to the problem, let's
; just assume that clock synchonization will happen between earth and
; mars so that the unix epoch could be synchronized between the frames
; probably with multiple reference stations in free fall and/or in deep
; space to account for gravitaitonal differences etc. so actually the
; only issue would be a practical notational one, or a UI one, since we
; can use timestamp with timezone or just straight epoc for this to work
; earth vs martian year/day is a much harder issue
; the org spec is currently silent on how org will handle dates beyond

Re: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda

[Samuel Wales]

sounds complex.  but if timestamp syntax is to be changed, my personal
"like a broken record" view is that possibly extensible syntax could
be used, depending on the number of places ts parsing is done
currently.

i.e. on whether current ts parsing is intermingled in core org code
with ts semantics and scattered all over org.  although, i suspect a
lot o that woul dneed changing if you wanted to iplement tz anyway.

org has kept deveoping new syntax; es was designed to stop that being
necessary.  i prefer less proliferation.  es allows a maximal amount
of flexibility using the same code for handling every aspect of it.
it is sexp data.

thus if favorable and tz is still desirable perhaps those who want to
use timezones would use es for tses, and a :tz keyword.  if desired
the org buffer can look like today's tses with whatever tz cookie
syntax you want.  it would not need any parsing.  it would be
analogous to links not showing their brackets/url.  the cookie would
be for presentation only so need only be a line or two of code.

still, adding tz semantics to all of org's features and packages and
personal code for those who want it sounds complex.


On 1/13/23, Daryl Manning  wrote:
> Following on from thread at
> https://www.reddit.com/r/orgmode/comments/zrppqw/
>
> [First off, I just wanted to say thank you to everyone that works on
> org-mode. It is a wonder.]
>
> While I realize a few kicks at this can may have been taken, I wanted to
> (re-)propose Timezone support in org-mode. The world is much less local
> these days and we're all more remote and coordinating globally these days.
>
> *Background*
>
> 1. org-time-stamp-formats TZ currently only affects display and exports
> 2. org-agenda itself is not TZ aware
> 3. Several discussions on this have taken place over time
> 4. Concerns raise included breaking backwards compatibility
>
> *Proposal*
>
> 1. org-mode sets an optional variable (org-timezone-aware t) which enables
> TZ
> 2. org-agenda needs a way to determine which timezone it is in
> 3. Once enabled, any timestamp not exhibiting a TZ in it is considered
> "local time" wherever that is (I do not think UTC would work for this)
> 4. org-agenda can calc local based on TZ differences
>
> I understand this is by no means trivial and quite gnarly with DST and such
> to figure out but I do believe libs exists to deal with that heavy lifting.
> Currently, it does feel like a hole in org-mode as a 21st century organizer
> (disclaimer: digital nomading so might feel it more keenly). Also, just
> interested in making org-mode a more awesome tool for everybody.
>
> I'd love an understanding of the alluded to reservations raised in the
> reddit thread and in the mailing list threads mentioned that the format
> change might break things (I was unsure if that was referring to say, how
> time ranges were handled, or how say date ranges got dealt with (for
> example, say a flight from Singapore to Vancouver which takes off in one
> time zone and lands in another - something that is often in my cal.).
>
> thanks!
> Daryl.
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

Re: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda

[Jean Louis]

* Daryl Manning  [2023-01-13 11:59]:
> *Background*
> 
> 1. org-time-stamp-formats TZ currently only affects display and exports
> 2. org-agenda itself is not TZ aware
> 3. Several discussions on this have taken place over time
> 4. Concerns raise included breaking backwards compatibility
> 
> *Proposal*
> 
> 1. org-mode sets an optional variable (org-timezone-aware t) which enables
> TZ
> 2. org-agenda needs a way to determine which timezone it is in
> 3. Once enabled, any timestamp not exhibiting a TZ in it is considered
> "local time" wherever that is (I do not think UTC would work for this)
> 4. org-agenda can calc local based on TZ differences

Totally understandable. 

In my opinion time functions shall be implemented, if they are not, in
Emacs generally, not only Org specific. 

This is good for review as related to PostgreSQL database:

https://wiki.postgresql.org/wiki/Don%27t_Do_This#Don.27t_use_timestamp_.28without_time_zone.29

Analogously it would be best that Org becomes time zone aware.

It would mean that timestamps become time zone aware. Something like this:

<2023-01-15 Sun +1>

When timestamp becomes time zone aware, then as preliminary option it
would be possible to update all the timestamps in an Org file for
personal use.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Thoughts on this ob language generator

[George Mauer]

I had a need the other day to execute some typescript in an org document.
Now I know that there's an ob-typescript package but that doesn't quite
work the way I want and expects typescript to be installed globally (which
runs into a variety of versioning issues).

There is a better option available with the `npx` program (installed
alongside `npm`) which can install a package along with its dependencies
into a temporary sandbox and run its binaries.

I rewrote the typescript babel plugin to do this and then realized that
there was relatively little in it beyond variable and function names that
was typescript-specific. The exact same process can be used for anything
that has an interpreter up on npm. Coffeescript, mermaidjs, all sorts of
things.

So I made a macro. I'm interested what people here think:
https://github.com/togakangaroo/create-ob-npx

Re: copying text from links makes it with a starting [[

[Samuel Wales]

what i do is the dumb solution.  c-c c-l, kill the url, c-g.

On 1/13/23, Ihor Radchenko  wrote:
> Luca Ferrari  writes:
>
>>> What exactly do you do to copy the link?
>>> See https://orgmode.org/manual/Feedback.html#Feedback
>>>
>>
>> Imagine I've this org fragment:
>>
>> Link: [[https://foo.com]]
>>
>> where the link is displayed without brackets, in org mode.
>> I place the point at the beginning of the first visibile character,
>> hence the 'h', and the C-space M-e M-w and then go to another
>> application and paste the content of the clipboard.
>
> Then, brackets being included is expected behaviour. Org cannot really
> know whether you want to copy link as actual link or as Org markup.
>
> If your link also contains escaped symbols, copying becomes even more
> tricky: [[https:test.asd/\]\]s]] (original link: )
>
> I am not sure what would be the best interface in your situation other
> than creating a dedicated function to copy the link address.
>
> --
> Ihor Radchenko // yantar92,
> Org mode contributor,
> Learn more about Org mode at .
> Support Org development at ,
> or support my work at 
>
>


-- 
The Kafka Pandemic

A blog about science, health, human rights, and misopathy:
https://thekafkapandemic.blogspot.com

Re: [PATCH] lisp/ob-octave.el, was [PATCH] rfc: using ert-deftest with side-effects

[Ihor Radchenko]

Ihor Radchenko  writes:

> So, the test failure is real.

The error buffer contents when the test fails is the following:

warning: using the gnuplot graphics toolkit is discouraged

The gnuplot graphics toolkit is not actively maintained and has a number
of limitations that are unlikely to be fixed.  Communication with gnuplot
uses a one-directional pipe and limited information is passed back to the
Octave interpreter so most changes made interactively in the plot window
will not be reflected in the graphics properties managed by Octave.  For
example, if the plot window is closed with a mouse click, Octave will not
be notified and will not update its internal list of open figure windows.
The qt toolkit is recommended instead.
 line 0: warning: iconv failed to convert degree sign
error: ignoring const execution_exception& while preparing to exit
[ Babel evaluation exited with code 0 ]

Exit code is 0, so octave does finish.

Hence, test assertion that
(should-not (buffer-live-p (get-buffer "*Org-Babel Error Output*")))
does not appear to be accurate.

Leo, should we simply remove the assertion?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: ob-shell intentions and paperwork (was Bash results broken?)

[Ihor Radchenko]

Matt  writes:

>   On Fri, 13 Jan 2023 04:36:40 -0500  Ihor Radchenko  wrote --- 
>  > Cgit displays our bugfix merges with all the required commits.
>  > So, what happened what not ideal either way.
>  
> Would you like me to correct how I've incorporated my changes?

No. I was referring to the initial situation with a single commit being
displayed.

I am not sure what Max was trying to point out.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: ob-shell intentions and paperwork (was Bash results broken?)

[Matt]

  On Fri, 13 Jan 2023 04:36:40 -0500  Ihor Radchenko  wrote --- 
 > Cgit displays our bugfix merges with all the required commits.
 > So, what happened what not ideal either way.
 
Would you like me to correct how I've incorporated my changes?

Re: [FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda

[Tim Cross]

Daryl Manning  writes:

> Following on from thread at https://www.reddit.com/r/orgmode/comments/zrppqw/
>
> [First off, I just wanted to say thank you to everyone that works on 
> org-mode. It is a wonder.]
>
> While I realize a few kicks at this can may have been taken, I wanted to 
> (re-)propose Timezone support in org-mode. The
> world is much less local these days and we're all more remote and 
> coordinating globally these days.
>
> *Background*
>
> 1. org-time-stamp-formats TZ currently only affects display and exports
> 2. org-agenda itself is not TZ aware
> 3. Several discussions on this have taken place over time
> 4. Concerns raise included breaking backwards compatibility
>
> *Proposal*
>
> 1. org-mode sets an optional variable (org-timezone-aware t) which enables TZ
> 2. org-agenda needs a way to determine which timezone it is in
> 3. Once enabled, any timestamp not exhibiting a TZ in it is considered "local 
> time" wherever that is (I do not think UTC
> would work for this)
> 4. org-agenda can calc local based on TZ differences
>
> I understand this is by no means trivial and quite gnarly with DST and such 
> to figure out but I do believe libs exists to
> deal with that heavy lifting. Currently, it does feel like a hole in org-mode 
> as a 21st century organizer (disclaimer:
> digital nomading so might feel it more keenly). Also, just interested in 
> making org-mode a more awesome tool for
> everybody. 
>
> I'd love an understanding of the alluded to reservations raised in the reddit 
> thread and in the mailing list threads
> mentioned that the format change might break things (I was unsure if that was 
> referring to say, how time ranges were
> handled, or how say date ranges got dealt with (for example, say a flight 
> from Singapore to Vancouver which takes off in
> one time zone and lands in another - something that is often in my cal.). 
>

I agree this would be a great feature to add. However, after having
looked at it in some detail, I realise that not only is it not a trivial
task, it is actually a very large and complex task and will require
extensive work which will almost certainly result in breakage with
regards to backwards compatibility.

At the risk of over simplifying the matter, I would suggest the big
challenge here is that there are two somewhat competing (and
conflicting) use cases. On one hand, you want a high level abstraction
which makes working with dates and times easy and clear. TO some extent,
that is what we have now. On the other hand, we need something far more
complex which is able to handle multiple time zones. This means being
able to handle both base timezone offsets as well as offset adjustments
for things like daylight savings time. There is a lot of hidden
complexity here. You have to handle the fact that daylight saving
chang-over dates/times are dynamic i.e. not necessarily the same every
year. This adds the additional complexity of having to somehow track
historical information regarding such changes, which isn't as readily
accessible in a consistent manner on all platforms.

Then there is the other 'messy' stuff. For example, when calculating
time ranges and number of days, hours/ minutes between two dates you
need to remember to add/remove the hour if the range crosses over a
daylight savings period. Similarly you need to ensure you make the
correct adjustment when changing timezones (consider emacs on a laptop
for someone who travels a lot between different time zones). Not only do
you need to take into account the different timezone offset, you also
need to look at the date and then adjust according to the timezone
offset for the current timezone at the time of the timestamp rather than
just considering the current time offset.  

I expect what is needed is an 'internal' UTC based representation which
is used for all calculations and an 'interface' layer, which converts
the UTC representation into the local display representation for
consumption by humans. The problem with this is that it is likely to
break the core feature of org files i.e. that they are in plain text. I
guess we could possibly do somehting clever with overlays such that UTC
date/times are rendered/presented in local time format. However, we
would then also require some type of interface for users to enter
dates/times (to ensure they are converted to the correct UTC internal
format).

However, the real challenge here is that this is a very large piece of
work and it needs someone who is prepared to take it on. I suspect until
someone who needs to scratch this itch sufficiently comes along, this is
a feature which will be unlikely to make it to the top of the todo
list. There are simply far too many existing feature improvements and
additions people will prefer to work on before taking on this
one. Things are made more difficult because it isn't the sort of task
which can be achieved with small increments over time. This is more
likely to be something which needs to be 

Re: [PATCH] org-datetree.el: Allow datetrees with TODO, priority, tags

[Ilya Chernyshov]

Ihor Radchenko  writes:


Ilya Chernyshov  writes:


 (defun org-datetree--find-create
@@ -169,18 +167,19 @@ component.  If INSERT is non-nil and 
there is no match then it is

 inserted into the buffer."
   (when (or month day)
 (org-narrow-to-subtree))
-  (let ((re (format regex-template year month day))
+  (let ((re (format org-complex-heading-regexp-format
+(format regex-template year month day)))


These changes make the docstring for `org-datetree--find-create'
incorrect:

(defun org-datetree--find-create
(regex-template year  month day insert)
  "Find the datetree matched by REGEX-TEMPLATE for YEAR, 
  MONTH, or DAY.
REGEX-TEMPLATE is passed to `format' with YEAR, MONTH, and 
DAY as
arguments.  Match group 1 is compared against the specified 
date
component.  If INSERT is non-nil and there is no match then 
it is

inserted into the buffer."

Please update the docstring to reflect the new behaviour.


Hi!

Since the last patch, I've made the following changes, thanks to 
your

recommendations in Matrix chat:

1. Added a new optional argument MATCH-TITLE to
`org-datetree--find-create' that controls whether to match
REGEX-TEMPLATE against heading title inside complex heading (the 
match

is done via `org-complex-heading-regexp-format') or to match
REGEX-TEMPLATE against the whole heading line(old behavior). That 
way,

we won't break old calls to the function.

2. Added `replace-match' call that renames the first implicitly 
numbered
match group inside REGEX-TEMPLATE to first explicitly numbered 
one. The

replace is performed only if MATCH-TITLE is non-nil(that is,
`org-complex-heading-regexp-format' is used). That way we ensure 
that

even if in the future `org-complex-heading-regexp-format' will be
changed (the number of match groups will be different), we can get
access to the right group matching a date component without 
modifying
the source code of the function. Is there a better way to ensure 
that?


>From 481d8b4049f09cf880b584635a778e976a609f5c Mon Sep 17 00:00:00 2001
From: Ilya Chernyshov 
Date: Thu, 15 Dec 2022 02:08:15 +0600
Subject: [PATCH] lisp/org-datetree.el: Allow datetrees with TODO, priority,
 tags

* org-datetree.el (org-datetree--find-create): Add optional argument
MATCH-TITLE that controls whether to match REGEX-TEMPLATE against
heading title inside complex heading or to match REGEX-TEMPLATE
against the whole heading line.

* org-datetree.el (org-datetree--find-create-group,
org-datetree-find-iso-week-create): Allow finding a datetree with TODO
state, priority, tags, statistics cookies, or COMMENT keyword.

* testing/lisp/test-org-datetree.el
(test-org-datetree/find-date-create,
test-org-datetree/find-iso-week-create): Add tests for a datetree with
tags, TODO or priority keywords.

* etc/ORG-NEWS (Datetree structure headlines can now be complex):
Document the change.

* doc/org-manual.org: Update datetree definition.
---
 doc/org-manual.org|  3 +-
 etc/ORG-NEWS  |  6 
 lisp/org-datetree.el  | 57 ---
 testing/lisp/test-org-datetree.el | 16 +
 4 files changed, 61 insertions(+), 21 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index f3b77ebad..f816f8725 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -22507,7 +22507,8 @@ level.
 ,*** 2022-10-08 Saturday
 #+end_example
 
-Tags are allowed in the tree structure.
+TODO state, priority, tags, statistics cookies, and COMMENT keywords
+are allowed in the tree structure.
 
 [fn:31] This is always the other, not the user.  See the variable
 ~org-link-from-user-regexp~.
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index c5d9bdf6e..0aeb76f2f 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -55,6 +55,12 @@ document header:
 ,#+LATEX_HEADER: \DefineVerbatimEnvironment{lstlisting}{Verbatim}{...whatever...}
 #+END_src
 
+** New features
+*** Datetree structure headlines can now be complex
+
+TODO state, priority, tags, statistics cookies, and COMMENT keywords
+are allowed in the tree structure.
+
 * Version 9.6
 
 ** Important announcements and breaking changes
diff --git a/lisp/org-datetree.el b/lisp/org-datetree.el
index 035ef047a..8a617e90c 100644
--- a/lisp/org-datetree.el
+++ b/lisp/org-datetree.el
@@ -99,16 +99,15 @@ If time-period is month, then group entries by month."
 	  (month (calendar-extract-month d))
 	  (day (calendar-extract-day d)))
   (org-datetree--find-create
-   "^\\*+[ \t]+\\([12][0-9]\\{3\\}\\)\\(\\s-*?\
-\\([ \t]:[[:alnum:]:_@#%%]+:\\)?\\s-*$\\)"
-   year)
+   "\\([12][0-9]\\{3\\}\\)"
+   year nil nil nil t)
   (org-datetree--find-create
-   "^\\*+[ \t]+%d-\\([01][0-9]\\) \\w+$"
-   year month)
+   "%d-\\([01][0-9]\\) \\w+"
+   year month nil nil t)
   (when (eq time-grouping 'day)
 	(org-datetree--find-create
-	 "^\\*+[ \t]+%d-%02d-\\([0123][0-9]\\) \\w+$"
-	 year month day)
+ 

Re: Export Org with Org concept -- Re: Problems with C-c C-e file.org,

[Ihor Radchenko]

Max Nikulin  writes:

> On 07/01/2023 16:04, Ihor Radchenko wrote:
>> 
>> As I said, the requirement to get it into the core is re-creating
>> previous layout and bindings. The layout and bindings may be
>> customizable, but they must be available.
>
> I would say, a layout that is not worse than the current one. Org menus 
> may have more than 1 column and it is advantage for wide windows. A 
> distant goal might be a menu that is more flexible and may adapt for 
> single column suitable for a narrow side window.
>
> Another issue is accessibility. Too long menus are likely terrible for 
> users relying on Emacspeak. I have no idea how to deal with such case, 
> perhaps a better organized hierarchy unveiled level by level. API 
> conventions that allows to change menu implementation may be a rescue as 
> well.
>
> Tim Cross. Re: Proposal: 'executable' org-capture-templaes. Mon, 06 Jun 
> 2022 09:05:29 +1000 https://list.orgmode.org/87sfoi1xde@gmail.com/

I do not insist of extending the functionality. Step one is re-creating
the existing one, including echoing the selected menus, as we do
already.

If we can achieve this, the whole menu library might be moved out of Org
into Emacs core and get further developed there. But it is too early to
talk about this now. I first want to see (with real working patches for
Org) if it is feasible to re-create the existing menus using the
proposed approach.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: ob-shell intentions and paperwork (was Bash results broken?)

[Ihor Radchenko]

Max Nikulin  writes:

>> It looks like you lost all the individual commits and commit messages in
>> the process.
>
> Ihor, it is usual merge commit of a branch with multiple commits. Cgit 
> shows combined changes, but commits was not squashed. The branch started at
> dff9565c2c8bb7fe100c5278136d3de22ce41051 Bastien 2022-12-30 09:17:38 +0100

Cgit displays our bugfix merges with all the required commits.
So, what happened what not ideal either way.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: org-export to icalendar: ignore links?

[Ihor Radchenko]

Timm Lichte  writes:

> Thanks for your quick response! Sorry mine took much longer ;-)
>
> Unfortunately, I cannot confirm that this fixed the problem. I still 
> cannot export org-agenda-files with broken links via 
> org-icalendar-combine-agenda-files under Org v9.6.1, which includes the 
> changes of your commit.

Then, please provide an example of the broken export.
See https://orgmode.org/manual/Feedback.html#Feedback

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: org-export to icalendar: ignore links?

[Timm Lichte]

Thanks for your quick response! Sorry mine took much longer ;-)

Unfortunately, I cannot confirm that this fixed the problem. I still 
cannot export org-agenda-files with broken links via 
org-icalendar-combine-agenda-files under Org v9.6.1, which includes the 
changes of your commit.



Am 11.08.22 um 07:35 schrieb Ihor Radchenko:

Timm Lichte  writes:


I'm trying to export org-agenda-files to a combined ics/icalendar file
via org-icalendar-combine-agenda-files, but this fails due to unresolved
links in the source files. Instead of fixing these links, is there any
way to make org-export more robust and just ignore the links? As far I
know, they are actually not relevant for the icalendar export.

As this is an important step in my daily workflow, any help is more than
welcome.


Thanks for reporting!

Fixed on bugfix via 70a311b00.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=70a311b001153afdf492b3dab5a0816c861e3186

Re: [BUG] org-agenda-span treats 7 differently [9.4.6 (9.4.6-g3ba46c @ ~/.emacs.d/straight/build/org/)]

[Ihor Radchenko]

Eppo Math  writes:

> Many thanks in advance for taking the time to read this!
>
> Issue: In an org agenda block (fed to org-agenda-custom-commands), the
> number 7 passed to org-agenda-span causes the resulting block to begin
> on the first day of the week, ignoring org-agenda-start-day.
>
> Expected Behaviour: org-agenda-start-day and org-agenda-span should be
> options with independent behaviours.
>
> Actual Behaviour: When org-agenda-span is set to 7, the given argument
> org-agenda-start-day is ignored.

This is because of `org-agenda-start-on-weekday' defaulting to 1.
You can bind it to nil in your custom agenda view.

I think we can update the docstrings for `org-agenda-start-day' and
`org-agenda-start-on-weekday' explaining that the latter takes
precedence when agenda span is a week or more.

Will it be enough for you?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] Shift-up/down move the cursor to end of timestamp

[Ihor Radchenko]

Johannes Dahl  writes:

> ... However, the same problem (cursor is positioned
> after closing bracket) occurs with the related commands org-shiftcontrolup and
> org-shiftcontroldown. (For completeness: in 9.6.0 they were even slightly more
> broken in that they shifted the timestamps in different directions.)

Thanks!
This is a bit annoying, because the problem lies within
`org-timestamp-change' logic.
I now fixed the particular problem with `org-shiftcontrolup' on bugfix,
but I am afraid that similar issues may pop up in future.

Fix: https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=97a235cf1

Ideally, we need to patch `org-timestamp-change' logic. See
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=76f9a3c2c

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

[FEATURE REQUEST] Timezone support in org-mode datestamps and org-agenda

[Daryl Manning]

Following on from thread at
https://www.reddit.com/r/orgmode/comments/zrppqw/

[First off, I just wanted to say thank you to everyone that works on
org-mode. It is a wonder.]

While I realize a few kicks at this can may have been taken, I wanted to
(re-)propose Timezone support in org-mode. The world is much less local
these days and we're all more remote and coordinating globally these days.

*Background*

1. org-time-stamp-formats TZ currently only affects display and exports
2. org-agenda itself is not TZ aware
3. Several discussions on this have taken place over time
4. Concerns raise included breaking backwards compatibility

*Proposal*

1. org-mode sets an optional variable (org-timezone-aware t) which enables
TZ
2. org-agenda needs a way to determine which timezone it is in
3. Once enabled, any timestamp not exhibiting a TZ in it is considered
"local time" wherever that is (I do not think UTC would work for this)
4. org-agenda can calc local based on TZ differences

I understand this is by no means trivial and quite gnarly with DST and such
to figure out but I do believe libs exists to deal with that heavy lifting.
Currently, it does feel like a hole in org-mode as a 21st century organizer
(disclaimer: digital nomading so might feel it more keenly). Also, just
interested in making org-mode a more awesome tool for everybody.

I'd love an understanding of the alluded to reservations raised in the
reddit thread and in the mailing list threads mentioned that the format
change might break things (I was unsure if that was referring to say, how
time ranges were handled, or how say date ranges got dealt with (for
example, say a flight from Singapore to Vancouver which takes off in one
time zone and lands in another - something that is often in my cal.).

thanks!
Daryl.

Re: copying text from links makes it with a starting [[

[Ihor Radchenko]

Luca Ferrari  writes:

>> What exactly do you do to copy the link?
>> See https://orgmode.org/manual/Feedback.html#Feedback
>>
>
> Imagine I've this org fragment:
>
> Link: [[https://foo.com]]
>
> where the link is displayed without brackets, in org mode.
> I place the point at the beginning of the first visibile character,
> hence the 'h', and the C-space M-e M-w and then go to another
> application and paste the content of the clipboard.

Then, brackets being included is expected behaviour. Org cannot really
know whether you want to copy link as actual link or as Org markup.

If your link also contains escaped symbols, copying becomes even more
tricky: [[https:test.asd/\]\]s]] (original link: )

I am not sure what would be the best interface in your situation other
than creating a dedicated function to copy the link address.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [BUG] [ox-odt] ODT export Chinese/Han script inserts unexpected spaces in each consecutive line

[Ihor Radchenko]

Cantoraz Chou  writes:

> This commit 
> [[https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=3502ce2dbb29b70cdbb978d144322d48cb00f26d][ox-odt:
> Fix newlines replaced by spaces in Han script]]
> brings in a strange bug.
>
> Exporting ODT produces an unexpected space inserted into each line.
>
> Eg, the simplest code (a consecutive line conposed of two contiguous lines)
>
> #+begin_src org
> 那时，天下人的口音、言语都是一样。
> 那时，天下人的口音、言语都是一样。
> #+end_src
>
> will produce following text:
>
> #+begin_example
> 那时，天下人的口音、言语都是一样。那 时，天下人的口音、言语都是一样。
> #+end_example
>
> Notice the second sentence, the space in `那 时'.

Thanks for reporting!
Fixed, on bugfix.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=cd967ce00

That was rather silly. I forgot that Chinese characters occupy two columns.

P.S. As Chinese user, do you have other gotchas Org might need to
address?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at