Re: org-clock idle time in pgtk Emacs

2022-05-24 Thread Tim Cross


Julien Cubizolles  writes:

> org-clock checks for the 'x window-system in order to use the program
> set up by org-clock-x11idle-program-name. Recent Emacs versions use the
> 'pgtk instead of 'x and as such will default to using
> org-emacs-idle-seconds in org-user-idle-seconds.
>

I"m not sure this is accurate. You should only use the pgtk build of
Emacs if your running wayland. You definitely should not use it if your
running under X.

The big issue is that some important key input facilities don't work
correctly if you run the pgtk build under X (actually, I'm not sure they
work correctly under Wayland either, but then again, Wayland is a
different beast to X and differences can be expected).

The pgtk build is not a replacement for the current xlib+gtk build,
which will remain the correct build when running under X.
Unfortunately, this does mean that if you use both X and Wayland, you
likely will need two builds of Emacs.

There was a fairly long discussion thread about this on emacs-devel
about a month or so ago. The upshot was flagging the need to update the
documentation to clarify that people should not use the pgtk build when
running under X windows.

I suspect this means the below patch will need further refinement. 



> The following patch provides a crude workaround.
>
> I'm using a python program (included below) to report idletime in
> wayland, using the idle-time module. It can be used for
> org-clock-x11idle-program-name.
>
> modified   lisp/org-clock.el
> @@ -1196,7 +1196,7 @@ If `only-dangling-p' is non-nil, only ask to resolve 
> dangling
>  
>  (defvar org-x11idle-exists-p
>;; Check that x11idle exists
> -  (and (eq window-system 'x)
> +  (and (or (eq window-system 'pgtk) (eq window-system 'x))
> (eq 0 (call-process-shell-command
>(format "command -v %s" org-clock-x11idle-program-name)))
> ;; Check that x11idle can retrieve the idle time
> @@ -1213,7 +1213,7 @@ This routine returns a floating point number."
>(cond
> ((eq system-type 'darwin)
>  (org-mac-idle-seconds))
> -   ((and (eq window-system 'x) org-x11idle-exists-p)
> +   ((and (or (eq window-system 'x) (eq window-system 'pgtk)) 
> org-x11idle-exists-p)
>  (org-x11-idle-seconds))
> (t
>  (org-emacs-idle-seconds
>
> #!/usr/bin/env python3
>
> from idle_time import IdleMonitor
>
> monitor = IdleMonitor.get_monitor()
> print(f"{1000*monitor.get_idle_time():.0f}")




Re: About 'inline special blocks'

2022-05-23 Thread Tim Cross


I've not got a lot to add here. I'm not against the idea, but Juan makes
some points below which I think are particularly important and wanted to
add weight to them!

Juan Manuel Macías  writes:

> Hi, Kaushal, thanks for all your interesting comments,
>
> Kaushal Modi writes:
>
>> The challenging part will be deciding the syntax so that there are no
>> false matches.
>>
>> May be reserve "inline_" for inline blocks?
>>
>> e.g. inline_[options]{text}  ?
>
> It seems to me the most consistent option, if we continue in some way
> the syntax of the inline code blocks, which would be the close relatives
> of the inline special blocks. Perhaps (to shorten the syntax a bit)
> 'inline' could be replaced by some reserved symbol. Something like:
>
> &_[options]{text}
>

I agree. Selection of the 'symbol' might be tricky, but the idea is
sound.


> I think a major issue would also be how to properly compact <[options]>
> so as not to result in too overloaded syntax. Maybe something like:
>
> [latex(list of attributes) html(list of attributes)...]
>
> ?
>
> But that is an abuse of direct formatting, which I think should always be
> avoided, especially in a format-agnostic environment like Org, which is
> more of a logician than a typesetter :-)

I think this is a really important point. Whenever we add formatting
specific directives, we always end up in a somewhat uncertain situation
with respect to the other back ends. I also feel that in-line blocks
which support large and complex formatting configuration really defeat
the purpose of an in=line block (which I feel should be kept relatively
simple). I also find complex constructs of this form really degrades the
readability of source documents. 

>
> And, in any case, it is to be expected that the user will not need to
> overload that part, since these hypothetical inline blocks would be
> intended for short segments of text within the paragraph. I think the
> most typical use case would be something like your 'mark' example.
>

Yes, that was my thinking as well. 





Re: Question Regarding Easier Issues To Help With

2022-05-23 Thread Tim Cross


"Samuel Banya"  writes:

> Hey there,
> So I took a look at the following link recently as I finally have had time 
> again over the past couple of months since I've been
> dealing with a lot of personal family stuff, and got some time back again.
>
> Can anyone lead me in the right direction for some beginner tier issues to 
> take a look at, as well as hand holding for any workflow
> on how to actually work on the related issue / source code accordingly:
> https://updates.orgmode.org/#help
>
> I ask because I'm a bit of an Elisp newbie. I'm assuming the first step is to 
> try to reproduce the bug given the user's info, and then
> attempt to look at the codebase to see what might be causing it?
>

Hi Sam,

my recommendation would be to focus on the last part of what you
outlined i.e. recipe to reproduce a bug. 

This is often the most time consuming part for the core maintainers.
Given the size of org mode and the number of add on packages, as well as
the flexible configuration of org, it can be very challenging and time
consuming to get a minimal recipe which can reproduce an issue. 

Once an issue can be reliably reproduced, it typically takes little time
for a fix to be identified. 

Focusing on how to define a minimal reproducible case is also a useful
way to get familiar with org mode and the code structure. It can also
help develop elisp skills. 

Therefore, my suggestion would be to select some reported issues which
have not been confirmed or which don't have a clear, concise and minimal
recipe to reproduce and try to both confirm the issue and provide a
minimal recipe. I believe this would be a huge benefit for the core
maintainers. I also suspect that in many cases, once you do have a
minimal recipe to reproduce an issue, you will also see possible ways to
resolve the issue. When you do, you can report both the recipe and your
suggested solution/patch. The more experienced maintainers will quickly
be able to assess your recipe and proposed solution and provide feedback
for further improvements to your patch or perhaps guidance on how to
narrow things down further. 



Re: I can't set dabbrev to respect the writen case

2022-05-16 Thread Tim Cross


Ypo  writes:

> Thanks, Samuel
>
> I've uninstalled fancy-dabbrev, and it seems to work when I complete using 
> M-/. But, now I don't have candidates to choose the
> completion. Any advice with this?
>
> Best regards,
> Ypo
>

You might want to have a look at the corfu and cape packages. 

https://github.com/minad/corfu
https://github.com/minad/cape

The corfu packages is basically a completions UI package built on-top of
built-in Emacs completion functionality while cape is a set of
completions at point functions which has support for dabbrv (and
others). 

Alternatively, if your a copmany mode user, I believe there are company
dabbrv modes you can install. Company is probably more mature in the
sense it has been around for some time. However, I like the underlying
principals of corfu (i.e. leverage of existing Emacs functionality
rather than re-inventing it). 

I did find with corfu and cape that it took a bit of experimentation and
use to get things 'just right'. However, once you have it working and
are use to it, I find it works well (I'm using Emacs 28 and 29(. 



Re: [BUG] The latest org-fold feature breaks link :activate-func property

2022-04-28 Thread Tim Cross


Ihor Radchenko  writes:

> "Christopher M. Miles"  writes:
>
>> I use extension 
>> [[https://repo.or.cz/org-link-beautify.git][org-link-beautify]] to display 
>> text-property beautify content on link. With
>> latest org-mode source code on commit "1ed9e4223", I got this error which 
>> displayed in the
>> attachment gif.
>
> I do not see any error there, except the fact that Org reveals the links
> contents.
>

the gif attachment is an animated gif, so you need to view it in a
client or other program which will 'run' animated gifs. Even when you
do, it doesn't provide much to go on. 




Re: Bug in orgalist mode's advice on indent-according-to-mode

2022-04-28 Thread Tim Cross


Eric Abrahamsen  writes:

> In Emacs commit f596f0db82c0b1ff3fe8e8f1d8b07d2fe7504ab6, from Nov 2021,
> the function `indent-according-to-mode' was given an optional
> inhibit-widen argument. That argument being passed causes orgalist's
> advice to fail, as the lambda doesn't accept any additional arguments.
> One way to fix it would be like that:
>
>  (unless (advice-member-p 'orgalist-fix-bug:31361 'indent-according-to-mode)
>(advice-add 'indent-according-to-mode
>:around (lambda (old  inhibit-widen)
>  "Workaround bug#31361."
>  (or (orgalist--indent-line)
>  (let ((indent-line-function
> (advice--cd*r indent-line-function)))
>(funcall old inhibit-widen
>'((name . orgalist-fix-bug:31361)
>
> Or I suppose a more future-proof approach might be to use a  and
> then `apply' instead of `funcall'.
>

A better solution would probably be to fix this without using
add-advice. While advice can be a useful escape hatch, it really is best
avoided, especially given that it doesn't always play nice with lexical
binding. I note this one is also calling an undocumented internal
function. 



Re: [BUG] The latest org-fold feature breaks link :activate-func property

2022-04-27 Thread Tim Cross


"Christopher M. Miles"  writes:

> [[PGP Signed Part:Undecided]]
>
> I use extension 
> [[https://repo.or.cz/org-link-beautify.git][org-link-beautify]] to display 
> text-property beautify content on link. With
> latest org-mode source code on commit "1ed9e4223", I got this error which 
> displayed in the
> attachment gif.

That animated gif is not terribly helpful. A backtrace would likely
provide something more concrete to work with. As this is something which is
triggered by an external package not part of org mode, you probably need
to report it to the maintainer of that package. 



Re: [tip] Org speed commands improved

2022-04-26 Thread Tim Cross


Ihor Radchenko  writes:

> Juan Manuel Macías  writes:
>
>> Org speed commands are a major productivity boost and I love them.
>> Lately it has occurred to me to make some modifications with the
>> following configuration, which I share here in case someone wants to try
>> it. The idea is that Org speed commands are activated anywhere in the
>> header (not just at the beginning of the line) *or* when point is at the
>> very beginning of the buffer. This, in my opinion, improves the
>> navigation speed:
>> ...
>> This also serves as a kind of write protection for the header titles. To
>> be able to edit them, we can use this function:
>
> If you are going this far with speed commands, you might as well switch
> to modal editing. What you are describing is basically a modal command
> map with ability to switch to insert map.
>

Funny - as I read Juan's post, as an evil user, that was exactly my
thought. I've never looked at the org speed commands, but as I read the
post, I thought "that looks like what I have with evil mode"




Re: Some feature requests/questions

2022-04-26 Thread Tim Cross


Philip Kaludercic  writes:

> Hi,
>
> I was wondering if someone could help me out to solve these annoyances I
> have been having with Org:
>
> 1. Most things I would like to clock are related to Emacs, but I often
>forget to check my agenda first, clock in, then clock out.  Is there
>some way I can automatically manage clocking when I enter a
>directory/kill the last buffer?

Emacs has various hooks which are run when you enter/leave/kill buffers
or start a mode etc. You could perhaps use them. However, this sort of
automation may not be as useful as you think. You probably jump around
into different buffers more than you realise and often, you will change
buffers even when working on the same task. I also think how you clock
your time is as important as just clocking time. It can be good to
really think about why you want to clock time. What is the real goal. Do
you really want to clock how long you spent in buffer X or is your aim
to clock how long you spent working on task Y, which may have included
multiple buffers, reading books, web pages, blogs, emails, phone
conversations, etc. Even more critical is thinking about how you will
use the data. If you clock at too fine grained a level, you will end up
with hundreds, maybe thousands of clock entries. How will you use that
data and what will that fine grained logging tell you which you didn't
get at a higher more abstract logging level? 

>
> 2. Capturing is configured by describing what keys should do via
>`org-capture-templates'.  Is there some way to use bookmark-like
>strings and completing-read?
>
> 3. Related to the previous point, is there a way to configure capture
>templates /in/ org files?  That would make it easier to dynamically
>add notes, without having to do so centrally, in my configuration
>file.
>

Can you expand on this a bit. What you describe makes me wonder if your
perhaps defining capture templates which are too specific and maybe not
taking full advantage of capture template functionality. 

Typically, your capture templates are defined at a high enough
abstraction level that you only need a very small number which are
defined centrally and rarely need to be modified. This small number of
templates work across projects, tasks etc. Sometimes, the way to enhance
templates is not with the template definition itself, but rather with
the use of other facilities, such as snippets, abbrev tables, macros,
etc. 

I'm not a big fan of scattering org configuration information across
multiple locations. Org is an evolving system with new functionality
being added all the time. While a lot of effort is put into preserving
backwards compatibility, it is often necessary for users to tweak their
own configuration to maintain existing behaviour or turn on some new
feature. When configurations are scattered across multiple files, this
becomes harder to maintain. 

Having said that, the general answer to your question is "Yes, what you
want could be done - this is emacs! However, at this point, there is no
'out of the box' functionality which would do what you want just by
setting a variable or loading an additional file. Basically, answer is
"Yes, but some assembly required". 

> 4. Limiting until when events may repeat, without having to duplicate
>timestamps manually (or via `org-clone-subtree-with-time-shift').
>Better yet being able to do so for an entire subtree/file, and have
>the property inherit itself.

This one is a little more complicated. Working with date/time data is
extremely complicated, particularly as many of the issues are somewhat
subtle and easily overlooked until they come up to bite you. For the
more complex scheduling requirements, some users like to rely on the
capabilities of the Emacs diary sexps, which are supported by org, to a
point. Again, I think most of the building blocks are there, but some
assembly is required. 

>
> All of these issues I have encountered by trying to use Org for the last
> few years at University, managing courses, notes, assignment due dates,
> appointments, etc.  The friction that these issues cause often have me
> become sloppy after a while, which defeats the entire purpose.
>

One of the great benefits of org mode is how well it can be configured
to suit your workflow. This is a fundamental difference between other
solutions which tyhpically require you to modify your workflow to fit
with its view of how things should be done. However, this can also be a
weakness. Sometimes, the way we develop a process 'naturally' is not
always the best or most efficient. We may need to also adjust our
workflows to enable more efficient exploitation of the tools at hand. 

I'm reminded of the old joke about NASA spending millions on developing
a pen which would work in zero gravity while the Russians just decided
to use a pencil. It would be a good idea to spend some time examining
the frictions you encounter and thinking about alternative ways to
eliminate 

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

2022-04-14 Thread Tim Cross


Paul Eggert  writes:

> On 4/14/22 06:19, Max Nikulin wrote:
>
>>  date-time + "America/Los_Angeles" input should not be reduced to timezone 
>> offset
>> in the output.
>
> It depends on the application. For some applications (e.g., generating 
> "Date:" lines
> in email), it is entirely correct to output a timestamp like "14 Apr 2022 
> 15:16:04
> -0700", thus losing the fact that the timestamp was generated with
> TZ="America/Los_Angeles".
>
>> Zone internal object or identifier is important for calculation of other 
>> date-time values based on the origin value.
>
> Again, that depends on the application. It's typically wrong to store an old
> timestamp in a form like "1950-07-01 00:00 Europe/Lisbon", because there is no
> standard for what "Europe/Lisbon" means. If you update your copy of TZDB, or
> interpret such a timestamp on another computer, that can change the 
> interpretation of
> such a timestamp. In this particular case, a change in TZDB release 2021b 
> altered the
> interpretation of this old timestamp because we discovered that DST was 
> observed in
> 1950 in Portugal.
>
> If you want to keep the TZDB identifier for advice about how to interpret 
> dates
> relative to a timestamp, that's fine. But you should keep the UT offset in 
> addition
> to the TZDB identifier, if you want your app to be fully accurate and useful. 
> For
> example, you should store "1950-07-01 00:00:00 + Europe/Lisbon" for a 
> timestamp
> generated by TZDB release 2021a, so that when you interpret the timestamp in 
> release
> 2021b you'll have an idea of what you're dealing with.
>

I think this is a very important point. Timezone data is not static. If
you only record the timezone name, offsets will be calculated using the
current definition, which may not be correct for past timestamps.

A good example of this is the DST values and the date when a TZ
transitions between DST and non-DST periods. That date can change,
either temporarily or permanently. That change can be days or even
weeks. Any date related calculations which only have knowledge about TZ
names and not the specific offset of a timestamp can therefore be out by
a significant amount.



Re: #+latex_header blocks, or, managing lots of LaTeX headers

2022-04-12 Thread Tim Cross


William Denton  writes:

> I have another question related to managing a book I'm doing building for 
> export to LaTeX:
> what do people do for managing all of the headers?
>

I only use the #+latex_headers or #+latext-headers-extra lines for 'one
off' type documents. If I'm going to be writing lots of documents where
those packages are needed, I define a new org-latex-classes entry.

> It's possible to add a new class to the org-latex-classes variable, and in 
> the documentation on
> it I see there are options ([NO-DEFAULT-PACKAGES], [NO-PACKAGES], [NO-EXTRA]) 
> that allow one to
> trim away all the default headers. That makes it easy to take away, but not 
> to add in lines I
> want, short of managing them in my init file.

The org-latex-classes 'placeholder' macros allow you to add or remove
the headers defined in either the default-packages or packages alist or
headers added via latex_headers/latex_headers_extra. However, the point
to note is that this 'value' is just a string, so you can just put in
exactly what you want - you don't have to use the placeholder macros
(though I almost always do - at least the [default-packages] macro, but
usually the [packages] one as well). The value isn't just restricted to
\usepackage lines either - you can put anything there which makes sense
in the latgex header.

>
> What sorts of practices do people have for managing lots of LaTeX headers? 
> Juan Manuel Macías,
> you mentioned something like this---literate programming in Org to export 
> LaTeX source---may I
> ask how you do it?
>

If I find myself using lots of #+latex_headers lines, this tells me
either

1. I need to update/add to the packages alist variable. This is
especially likely if I'm adding the same headers to every document. Just
add it to the packages alist. I tend to leave the default-packages alist
alone.

2. There is a specific 'style' of document I write often and all
documents of this style have the same additional headers. For this, I
create a new entry in org-latex-classes.

I think adding headers via #+latex_headers and #+latext_headers_extra
should be avoided as much as possible. It is a great escape hatch or for
doing 'on-offs', but not the right approach for doing something like
writing a book.

Consider the situation where you have been adding headers to load
language support or a particular code listing package or whatever. You
writing a book and it will have many org files (perhaps one per
chapter). You add these #+latex_header lines to add the packages. Later,
you find the listing package isn't suitable or you need to modify the
language encoding - now you have to edit every org file which makes up
the book.

Alternatively, you define a new org-latex-classes entry call 'my-book'
and use that to add the language encoding and code listing packages
(along with some other 'tweaks' you typically put in the header, maybe
even add a few custom commands etc). Now when you find it necessary to
change the language encoding, code listing package or whatever, you just
have that one definition to change and only need to edit one file.



Re: [BUG] error when saving a new capture [9.5.2 (9.5.2-g971eb6 @ /home/torysa/emacs/.emacs.d/straight/build/org/)]

2022-04-06 Thread Tim Cross


web...@toryanderson.com (Tory S. Anderson) writes:

> Remember to cover the basics, that is, what you expected to happen and
> what in fact did happen.  You don't know how to make a good report?  See
>
>  https://orgmode.org/manual/Feedback.html#Feedback
>
> Your bug report will be posted to the Org mailing list.
> 
>
> I was just saving a capture, no link or anything else. It bound up the thread 
> until I could C-g out of it. 
>
> Emacs  : GNU Emacs 28.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version 
> 3.24.29, cairo version 1.16.0)
>  of 2021-09-01
> Package: Org mode version 9.5.2 (9.5.2-g971eb6 @ 
> /home/torysa/emacs/.emacs.d/straight/build/org/)
>

My first suggestion would be to do an Emacs upgrade. Your running a
pre-release version (Emacs 28.0.50) which for a development version is
extremely old (over 6 months and isn't even one of the release candidate
builds for Emacs 28). Note also that Emacs 28.1 has been released.

The second thing I would do following upgrade is to try and reproduce
the issue using emacs -Q.

Finally, you need to provide more specific details. In order for anyone
to help or debug any issue, the first critical step is to be able to
reproduce it. As it stands now, there is nowhere near sufficient
information to do that. The best thing you can do to help is try to
reproduce the issue in a default configuration (i.e. emacs -Q), which
will help ensure the issue has not been introduced by additional
packages you have installed or any customisation you have done and
providing a minimal recipe to reproduce the issue. Generally, if someone
is able to provide a simple recipe which reliably reproduces the issue,
the issue will be resolved very quickly. If on the other hand, nobody can
reproduce the issue, it probably won't ever be fixed. 



Re: Multiline list entries / Is auto-fill-mode in org(roam) usual?

2022-04-04 Thread Tim Cross


c.bu...@posteo.jp writes:

> Dear Tim
>
> Am 04.04.2022 17:49 schrieb Tim Cross:
>> What is line-fill-mode? It isn't a 'standard' emacs mode, so must be
>> some add-on?
>
> Sorry this was a "typo".
>
>> At any rate, I don't see this problem when using standard
>> auto-fill-mode. With this mode, when lines wrap (because I've exceeded
>> the fill column) or if I use C-q to trigger filling, the wrapped lines
>> are indented correctly so that lists are preserved.
>
> This is not the case with me.
> Please see this screenrecording
> https://debianforum.de/forum/gallery/image/3641/source
>
> In the top window you see the relevant lines from my init.el.
> In the bottom you see a orgroam capture buffer while I am tiping a 
> loreipsum line and exceeding the fill column. It is not intended.
>
> And that is how it looks like in raw org
>
> :PROPERTIES:
> :ID:   7fe439ec-c46d-4da5-902a-3ba40cebb25e
> :END:
>
> #+title: test
> #+date: [2022-04-04 Mo 20:04]
>
> - Lorem ipsum dolro sit amet, consetetur sadipscing elitr, sed diam 
> nonumy
> eirmod tempor invidunt ut
> - second item
>second line in second item
> -
>
> The first list entry got its newline char automatically via 
> auto-fill-mode.
> The second entry got its newline by myself via pressing RET.

I cannot reproduce your issue using a clean Emacs. I suspect this is
either something incorrect in your init.el file or something caused by
org-roam (which is not part of org-mode).

I note from the screenshot you provided that you are explicitly setting
auto-fill-function. This is not the standard way to turn on
auto-fill-mode and it could break mode specific settings for
auto-filling. Instead, you should call auto-fill-mode in the appropriate
mode specific hook (as specified in the emacs manual).

I would suggest you try reproducing your issue in a clean Emacs e.g.

emacs -Q

open an org file

do M-x auto-fill-mode

do M-x org-capture

to start a data capture

do M-x auto-fill-mode to enable auto-filling in the capture buffer

Add list items to the capture buffer, some long, some short and let
auto-fill do the filling

complete the capture

When I do this, the lists are formatted correctly and long lines are
wrapped with the correct item indentation.

This is with

GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30,
cairo version 1.16.0) of 2022-04-05
Org mode version 9.5.2 (release_9.5.2-25-gaf6f12 @ 
/usr/local/share/emacs/29.0.50/lisp/org/)



Re: Multiline list entries / Is auto-fill-mode in org(roam) usual?

2022-04-04 Thread Tim Cross


c.bu...@posteo.jp writes:

> Thanks to the discussion I now can see and describe my problem clearer.
>
> When using line-fill-mode it does not take the current org list indention into
> account. Because of this org lines like this will appear.
>
> - one entry with a very long text ending in the second line
> end
> - second entry
>
> This is recognized (e.g. while html export) as two one-item-lists with a
> paragraph between them.
>
> When I press RET myself the point is correct indented but not when it is done
> automatically via line-fill-mode.
>
> Is there a solution for this?

What is line-fill-mode? It isn't a 'standard' emacs mode, so must be
some add-on?

At any rate, I don't see this problem when using standard
auto-fill-mode. With this mode, when lines wrap (because I've exceeded
the fill column) or if I use C-q to trigger filling, the wrapped lines
are indented correctly so that lists are preserved.




Re: Multiline list entries / Is auto-fill-mode in org(roam) usual?

2022-04-04 Thread Tim Cross


c.bu...@posteo.jp writes:

> Am 04.04.2022 09:16 schrieb c.bu...@posteo.jp:
>> Am 04.04.2022 06:06 schrieb Ihor Radchenko:
>>> A list item continues until
>>> there are _2_ blank lines or until the subsequent line is not indented  >=
>>> current list item indentation:
>> Intereseting. What 2 lines not 1 like in paragraphs. Am I right to say
>> that a paragraph ends with one blank line?
>
> Sorry, this was misunderstanding.
>
> I would like to know why does a list need 2 blank lines instead of 1? For what
> reason exists this rule?


Main reason is because you might want a list item which consists of
multiple paragraphs. A possible alternative would be to require that
lists are indented and flag a list item has ended either by a new list
item marker or by text which is no longer indented. However, not
everyone wants to be forced to indent their lists and like to have the
list item start in the '0' column so we need some other way to mark the
end and 2 blank lines seems a reasonable compromise.
.




Re: [BUG] org-agenda thinks timestamps after 23:00 correspond to the next day [9.5.2 (release_9.5.2-25-gaf6f12 @ /home/ignacio/repos/emacs/lisp/org/)]

2022-03-31 Thread Tim Cross


Max Nikulin  writes:

> New calling convention for `encode-time' exists since emacs-27.1, so it is
> incompatible with yet supported emacs-26. It is unfortunate that sources in 
> Org
> and in Emacs repository diverged, but I am unsure if it reasonable to 
> introduce
> a new compatibility wrapper. Emacs developer may be unhappy if the change will
> be reverted to way deprecated way to call `encode-time'. For almost a year I 
> am
> reading messages here that Emacs-28 is about to be released, so Emacs-26 is
> almost unsupported in Org.
>

I don' think that is correct. The policy is to support the previous 2
major versions, which means when Emacs 28 is released, org will need to
support both 27.x and 26.x. It will likely be necessary to have a
compatibility layer for 26.x given the changes.



Re: [RFC] DOCT: Declarative Org Capture Templates (easier template syntax)

2022-03-23 Thread Tim Cross


Nicolas Goaziou  writes:
>
> Ihor Radchenko  writes:
>
>> Nicolas Goaziou  writes:
>>
>>> I think there should be a direct mapping between Customize interface and
>>> values. Adding this macro as a band-aid to simply configuration is not,
>>> IMO, a solution.
>>
>> I think that current customize interface for org-capture-template is
>> perfectly fine. doct aids users who set org-capture-templates
>> programatically.
>
> I am merely pointing out that the programatically defined value is then
> no longer compatible with Customize interface. It may be important, or
> not.
>

This could be an issue for users who use both customise and
programatically modify their configuration.  



> The fact that we need a tool like "doct" to write templates in a compact
> form may be a sign that the data type is not good enough. Actually it
> sounds like a failure somehow.
>
> Should templates definition be more compact out of the box? Could the
> data type be made more powerful to permit more complex templates without
> relying on doct? IOW, can a tool such a doct be made transparent to the
> user?
>

Yes, I would agree. Such tools are generally a 'code smell'. Underlying
data structures should be transparent and not require helper functions
to assist the user in creating or updating them.

While I can see how doct can help with some use cases, I'm a little
concerned about the mixing of both setting of the org-capture-templatges
variable and adding/manipulating hooks together in one function. It
feels like two separate concerns being meshed into one function/macro
for convenience and attempting to work around limitations in the
existing system.

As you say, doct might be useful as a transition aid, but perhaps the
correct fix is to make it largely unnecessary (which I would assume
would also provide the additional functionality it offers to users of
customise and not only to those who configure their setup
programatically).

>From a personal perspective, I've never needed the additional
functionality offered by doct and while writing a capture template does
require lots of referencing of documentation, I suspect that even with
doct, I would still need to do that as I simply don't write new capture
templates often enough to remember all the options (even if they are
named rather than positional arguments).




Re: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-

2022-03-22 Thread Tim Cross


"Tom Davey"  writes:

> Hi Tim, 
>
> Thanks for these thoughtful comments. I agree that the Org developers (to
> whom I, as a mere user, owe enormous thanks) must be wary before making
> changes to how timestamps are handled. 
>
> This argues, I would say, for keeping what I believe was the status quo
> since at least Org version 9.4.4: Agenda views would display entries with
> active timestamps in property drawers. That has been my historical
> experience. 
>
> Tim, has your historical experience been different? In the invoicing example
> you give, were the timestamps in the properties drawer active, or inactive? 
>
> I have just verified with a simple test that Org version 9.4.4, which was
> shipped with Emacs 27.2 I believe, does display entries with an active
> timestamp as the value of a property in the ordinary :PROPERTIES: drawer.
> That's the situation I'm calling the "status quo." I'm wondering if my
> experience coincides with the experience of others. 
>
> Here's the simple entry that will be shown on the Week/Day Agenda view in
> 9.4.4: 
>
> * TODO Test of active timestamps
>:PROPERTIES:
>:Created:  <2022-03-22 Tue 18:30>
>:END:
>
> And note this: adding a second active timestamp to the test entry, e.g., to
> accompany a SCHEDULED: keyword, results in the entry appearing on the Agenda
> twice, as would be expected: 
>
> * TODO Test of active timestamps
>SCHEDULED: <2022-03-22 Tue 18:30>
>:PROPERTIES:
>:Created:  <2022-03-22 Tue 18:30>
>:END:
>
> This second example shows why the variable
> org-agenda-skip-additional-timestamps-same-entry is valuable. I rarely want
> an entry to display twice on the same day. 
>

For my invoicing system, the timestamps are in property draws, but not
under TODO headings, so I guess they wouldn't show up. I have used
timestamps in TODO entry property draws, but don't recall seeing them in
the agenda when I do. However, I also use a couple of custom agenda
views more often than the default agenda view, so I just may not have
noticed.

Personally, I have no use case for TODOs being displayed in the agenda
based on their created date. I do like to log when I created the todo,
but would not want to see that cluttering my agenda where I only want to
see scheduled and deadline tasks. I use a property draw for this type of
information as I consider it metadata about the entry. If I really want
ot see the item in the agenda based on a timestamp, then I explicitly
add that timestamp to the entry (not in  a property draw).

In my invoicing module, property draws are used to track invoicing
metadata i.e. last invoice number, last invoice date, invoice creation
date and invoice period for each invoice, invoice due date, paid datge
etc. Some of the timestamps are 'inactive' (they never change) and some
are active (will get modified). Perhaps that was a mistake on my part
and all of them should be inactive, but at the time, I thought of the
distinction as not being purely about whether they show in the agenda,
but more about whether they were immutable or not. 

The need to have another variable to limit the scope of timestamps when
you do allow timestamps in property draws makes things feel very
'kludgy'. I get very concerned about all these 'enhancements' we keep
adding which then also require additional complexity with multiple
variables interacting at different levels. I find the number of
variables and complexity associated with the agenda compared to when I
first started using org mode to be mind boggling. I often wonder how the
maintainers are able to keep on top of the increasing complexity of org
mode. 

What I definitely would NOT want is for TODO tasks in source blocks
which have a timestamp associated with them showing up in the agenda.
The possible exception might be when the source block is an org source
block. What I want in source blocks is for them to behave like a
microcosm of the original code environment - I want them to behave
exactly as they would when I just edit that code in a code dedicated
buffer with the addition of a minimal set of babel/noweb features to
allow me to work with code spread over multiple source code blocks etc.
I definitely don't want the additional processing overhead of org mode
search through my source blocks looking for things which look like they
might be timestamps or worse, looking for such things and then trying to
do something clever with them (like adding to the agenda).

With regards to timestamps in property draws, I'm not sure that arguing
the status quo is to keep that behaviour. I'm not sure that behaviour
was supposed to work or that it is well defined. It may have worked
previously 'by accident' rather than design (something which tends to
happen more as complexity increases). For example, what is the situation
with timestamps in property draws and inheritance? What is the situation
with timestamps and property draws for items which are not TODOs (i.e.
normal headings with a property 

Re: Timestamp parsing inside node properties and other contexts out of org-element-object-restrictions (was: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-

2022-03-22 Thread Tim Cross


Ihor Radchenko  writes:

> Ihor Radchenko  writes:
>
>> After further reading the source code, I figured that agenda is, in
>> fact, supposed to handle timestamps inside property drawers. Optional
>> arguments for org-at-timestamp-p imply that, in agenda specifically,
>> timestamps inside node properties are considered timestamps despite they
>> are not being parsed as timestamps by org-element.
>
> Even though I fixed the reported issue with agenda not showing headings
> with matching timestamps inside property drawers, this situation is
> revealing a big inconsistency in Org mode's handling of timestamps.
>
> org-at-timestamp-p usage implies that Org syntax for timestamps is not
> only context-dependent, but also depends on current command!
>
> org-at-timestamp-p is called with non-nil argument in a number of
> functions in Org:
> - org-clock-timestamps-change
> - org-mouse-delete-timestamp
> - org-mouse-context-menu
> - org-follow-timestamp-link
> - org-get-repeat
> - org-auto-repeat-maybe
> - org-time-stamp
> - ... many more in org.el
>
> So, depending on the current command, Org may on may not treat objects
> matching org-ts-regexp-both as timestamps.
>
> This situation complicates syntax and makes org-element unreliable when
> dealing with Org buffers.
>
> Should we just simply allow timestamps to be a part of node property
> values? Should we _not_ treat timestamp-looking text outside their
> allowed contexts (like quotes, source blocks, etc) as timestamps?
>

I think we have to be very wary here. I can see any changes here causing
lots of breakage for people. I know for my own use case, I use
timestamps a lot in property draws and various source blocks. I never
want any of them showing up in my agenda.

As an example, I was recently working for a company which required that
you put a timestamp in both a file header and in comments. The format
they used was pretty much the same as an org-mode active timestamp. I
use org mode to tangle the source files I write (as well as manage my
client data, such as todos, invoicing, contacts etc), so these files are
searched for agenda items, but I do not want any of those timestamps
causing lines in my agenda views. These timestamps are most commonly
found in source and example blocks.

I think the only time an org timestamp should be recognised in a source
block is when that source block is an org-mode source block. I don't
think they should ever be 'recognised' in example blocks.  

IN addition, my invoicing solution, which is based on org, uses
timestamps to track invoice periods etc. None of these should ever
appear in the agenda. This information is typically tracked in property
draws.

Unfortunately, I think org timestamps is a bit of a mess and we need to
be very careful about further complicating matters. The inability to
handle timezones is a major limitation IMO. The inability to handle
daylight savings transitions in a consistent manner (particularly for
calculation of periods, duration, etc) is often a source of errors and
if you are unfortunate enough to regularly travel across different
timezones, forget about using org mode to manage your schedule.

I have done several deep dives into org-mode's timestamp stuff, but
usually come back up gasping for air. Managing data and time data is
often far more complicated than it may appear on the surface. I think we
need to be extremely conservative when considering changes in this area
(it is the main reason I've never managed to find a way to add time zone
data - every solution I could think of was either really high on the
level of breakage and frustration it would create for users or it
adversely impacted the convenience/usability of org timestamps). 




Re: [BUG] Agenda no longer works for timestamps inside properties drawer [9.5.2 (release_9.5.2-24-g668205 @ /home/ignacio/repos/emacs/lisp/org/)]

2022-03-22 Thread Tim Cross


Ihor Radchenko  writes:

> Samuel Wales  writes:
>
>> so i guess i am interested in the rationale.  example of ts [and text]
>> search being useful might be an example or ledger block that contains
>> ledger source, or something like that.  i can get why bare ts not
>> being matched inside links might be useful.
>>
>> otoh i can get why text [not ts] and isearch search inside of links
>> can be useful.
>>
>> idk if my intuitions match those of others.  i am partly trying to
>> think of what a newcomer might expect to work [and the simplicity of a
>> rule that he or she would have to remember in order to know what the
>> behavior will be].
>
> I can see your point. Simple timestamp matching by regexp disregarding
> timestamp objects might be useful. However, I do not think that matching
> should be via timestamps in such a case. Rather it should be even more
> lax - agenda may match anything date-looking (with or without brackets).
> A specialized regex agenda matcher with regex constructed using current
> agenda date.
>
> Maybe we can introduce a new agenda view? Or maybe a special agenda
> mode, similar to org-agenda-include-inactive-timestamps?
>

Perhaps I simply don't understand, but I fail to see the need for adding
such functionality. Org files are plain text and you can just use the
build-in or any of the add-on search tools to search for anything,
including timestamps or things which may look like timestamps.

The only time org needs to provide search functionality is when that
search does need to be done within the context of a date/time object.



Re: How do you manage complex project with Org-mode

2022-03-20 Thread Tim Cross


Ihor Radchenko  writes:

> Tim Cross  writes:
>
>> For example, I
>> would not have a task which says to review my tasks twice a week. Do you
>> really need a task to remind you to do this twice a week? Do you really
>> need to track that you have done this? I would classify such tasks as
>> 'noise' tasks. They really don't perform any real purpose.  It is
>> similar to brain dead project policies which state things like "every
>> function must have a unit test". Many functions don't need a unit test
>> and forcing people to write pointless unit tests does more damage than
>> not having them.
>
> I disagree about this specific example (not with your whole idea).
> Reviewing tasks is important for people who can easily fall into
> continuous routine. Dedicating several hours per week to look into what
> you are doing in a broader perspective is valuable.
>
> Note that I do not mean "review" as looking through _all_ the tasks.
> Rather looking into current projects and deciding what to do, and, more
> importantly, what not to do in the coming week/days/months.
>
> See https://www.benkuhn.net/weekly/
>

My point was not that you don't need to review on a regular basis.
Reviewing your tasks and projects regularly is essential. My point was
that creating a todo task telling you to review your tasks/projects is
an example of a 'noise' task.

A common beginner error I've seen is for people to be so impressed with
org-mode, they decide to create tasks, templates and projects which map
out every aspect of their life. The problem with doing this is that you
then create additional work for yourself in managing these tasks and you
run the risk of being overwhelmed - you have so many tasks that instead
of making your life easier, you now become paralysed by too many task
choices.  

Getting the right balance is challenging and we do need to review
what/how we do things and refine our process in a continuous analyse,
plan, implement, review cycle.  



Re: [BUG] org-capture autoload bug? [9.5.2 (9.5.2-gfbff08 @ /home/ignacio/.emacs.d/elpa/org-9.5.2/)]

2022-03-14 Thread Tim Cross


Ignacio Casso  writes:

>> Regardless, I don't think having the situation where the programmer must
>> know (guess) whether autoload will/could execute during the evaluation
>> of code they write is tenable and am beginning to suspect it may be an
>> Emacs bug OR a subtle bug in org-mode as a result to the transition to
>> lexical binding as the default. My recommendation would be to come up
>> with a non-org specific example which reproduces the behaviour and then
>> ask on emacs-devel, using the example to demonstrate the issue.
>
> I agree. I'm on it.
>

Excellent. I will watch with interest to see what feedback you get!




Re: [BUG] org-capture autoload bug? [9.5.2 (9.5.2-gfbff08 @ /home/ignacio/.emacs.d/elpa/org-9.5.2/)]

2022-03-14 Thread Tim Cross


Max Nikulin  writes:

> On 12/03/2022 02:59, Tim Cross wrote:
>> Ignacio Casso writes:
>
>>>(let ((org-capture-templates
>>>   '(("d" "default" entry
>>>  (file+headline org-default-notes-file "Tasks")
>>>  "* %?"
>>>  (org-capture nil "d")))
>>>
>>> I put an autoload cookie myself and it doesn't fix it, so it's probably
>>> not that. It's the first time I manage autoload cookies though, so I may
>>> have done something wrong. I only tested that the autoload cookie worked
>>> by checking that before loading org-capture, org-capture-templates
>>> appears in the completion list for C-h v, and I can evaluate it.
>> While I don't know if this is a bug, it certainly doesn't seem to be
>> doing the right thing from an 'intuitive' point of view. I would expect
>> when a variable is bound to a value inside a let and a function is then
>> called which uses that variable, the initial let bound value should be
>> used and the result be the same regardless of whether org-capture has or
>> has not been loaded. It means there is a hidden side-effect here, which
>> isn't good and probably needs more analysis. If you had set the value
>> using setq rather than as a let form, it wouldn't be overridden when
>> org-capture is loaded, so why does it when it is a let binding?
>
> For `defcustom' autoload generates no more than
>
> (defvar org-capture-templates nil "...")
>
> It seems, behavior depends on whether `org-capture-templates' has the :set
> attribute. The setter receives nil instead of the let-bound value. I can not 
> say
> I understand the tricks with bypassing lexical binding in `defcustom' and some
> checks in `custom-declare-variable'. Since emacs-26 something has changed:
>
> https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=6309131349
>> -  ;; Use defvar to set the docstring as well as the special-variable-p flag.
>> -  ;; FIXME: We should reproduce more of `defvar's behavior, such as the 
>> warning
>> -  ;; when the var is currently let-bound.
>> -  (if (not (default-boundp symbol))
>
> I am unsure that the setter of `defcustom' should get let-bound value in the
> case of autoloading since it might lead to fragile behavior.
>
> I still think that explicit templates argument for `org-capture' is better 
> than
> relying on autoloading.
>
Yes, it would seem that something has changed and I suspect it may be
related to the introduction of lexical bindings. Explicit template
arguments for org-capture may or may not be better, I don't know.
However, this behaviour seems like it may be the tip of a much bigger
issue outside of org-mode and potential source of bugs for Emacs.

Bottom line is I don't think any function should behave differently
depending on when autoload runs. It could be because the setter used in
the defcustom is incorrect since the change a while back to use
lexical-binding by default or something else. It looks like the setter
on org-capture-template is used to do some form of conversion on the
template specification to force a new format. Not sure how long that has
been there or whether it is actually still required. Could be that this
setter was added to aid in transition to a new template format which
could/should be removed at some point. Personally, I've never liked
custom setters. I typically avoid the custom interface. However, exotic
custom variable setters can mean that using setq won't work correctly
and you need to use custom-set-variable etc. In fact, I'm seeing a lot
more use of custom-set-variable* in init code recently - something which
was almost never seen 20 years ago. 

Regardless, I don't think having the situation where the programmer must
know (guess) whether autoload will/could execute during the evaluation
of code they write is tenable and am beginning to suspect it may be an
Emacs bug OR a subtle bug in org-mode as a result to the transition to
lexical binding as the default. My recommendation would be to come up
with a non-org specific example which reproduces the behaviour and then
ask on emacs-devel, using the example to demonstrate the issue.

> Reading the code I noticed `org-capture-templates-contexts' so I wonder if it
> might help for the original use case.

I thought about that as well. However, from re-reading the OP's posts, I
suspect not. org-capture-templates-contexts seems to be useful when you
want to restrict access to some templates based on context. However,
from reading the OP's posts, I get the impression that what they want is
for templates to be different based on context i.e. they still want the
template associated with 'd', but want its definition to be different.

It is poss

Re: Code blocks and quotes export style

2022-03-13 Thread Tim Cross


"Barton, Mark"  writes:

> I manually check the pdf to see if it fits and use a code formatter to get the
> line widths shorter. I use blacken for python and sqlformat for SQL within
> Emacs. Even then I still hit limits where I recently changed to font size to
> small for the code blocks.
>
> I use minted for syntax highlighting of code blocks.
> #+LaTeX_HEADER: \usepackage{minted}
>
> I set a local variable for org using minted to change the fontsize.
> # Local Variables:
> # org-latex-minted-options: (("fontsize" "\\small"))
> # End:
>

That is basically what I do as well. I use minted and a smaller font for
code listings. I also ensure that when programming, I use a fill column
of 80.

In this day of large monitors, a fill column of 80 for programming
languages can often seem frustrating as it means less code in a
'screen' (window). Increasing fill column to (lets say) 100 can work
well, until it comes time to generate documentation and PDFs etc. Then I
remember why 80 is a better default!

I do use local variables more these days and will set a larger fill
column setting for code when I know I won't be needing to print it or
use it in Latex, but the default is 80 so that whenever I do org blocks,
things wrap at a width which works better for export to PDF etc.



Re: [BUG] org-capture autoload bug? [9.5.2 (9.5.2-gfbff08 @ /home/ignacio/.emacs.d/elpa/org-9.5.2/)]

2022-03-11 Thread Tim Cross


Ignacio Casso  writes:

> Max Nikulin  writes:
>
>> Ignacio, I think, you can add (require 'org-capture) inside your
>> function just before `let' and it would work almost as lazy loading.
>
> Thanks, I was already using (require 'org-capture) in my init file to
> solve this. As I said, it's not really a problem for me, I was just
> reporting it in case it was a bug. I was not sure since I don't really
> know the inner workings of autoloads and custom variables, but since
> this snippet works when org-capture is not yet loaded
>
>   (setq org-capture-templates
>  '(("d" "default" entry
> (file+headline org-default-notes-file "Tasks")
> "* %?")))
>   (org-capture nil "d")
>
> and this snippet also works
>
>   (let ((my-new-var 2))
> (defcustom my-new-var 1 "New variable that did not exist before")
> (message "%s" my-new-var)) ;; This prints 2
>   (message "%s" my-new-var) ;; This prints 1
>
> I thought that this snippet should work too when org-capture is not yet
> loaded, and that the fact that it doesn't could mean that there is a bug
> somewhere
>
>   (let ((org-capture-templates
>  '(("d" "default" entry
> (file+headline org-default-notes-file "Tasks")
> "* %?"
> (org-capture nil "d")))
>
>
>> I have no particular opinion concerning adding autoload cookie to
>> `org-capture-templates'. Emacs has enough number of them, but Org has
>> no such custom variables.
>
> I put an autoload cookie myself and it doesn't fix it, so it's probably
> not that. It's the first time I manage autoload cookies though, so I may
> have done something wrong. I only tested that the autoload cookie worked
> by checking that before loading org-capture, org-capture-templates
> appears in the completion list for C-h v, and I can evaluate it.

While I don't know if this is a bug, it certainly doesn't seem to be
doing the right thing from an 'intuitive' point of view. I would expect
when a variable is bound to a value inside a let and a function is then
called which uses that variable, the initial let bound value should be
used and the result be the same regardless of whether org-capture has or
has not been loaded. It means there is a hidden side-effect here, which
isn't good and probably needs more analysis. If you had set the value
using setq rather than as a let form, it wouldn't be overridden when
org-capture is loaded, so why does it when it is a let binding? 

Might be worth asking a general question on emacs-devel? Stephan or Eli
can probably provide some clarification here (maybe this is somehow
related to the changes associated with the lexical binding stuff?)




Re: How do you manage complex project with Org-mode

2022-03-02 Thread Tim Cross


Sébastien Gendre  writes:

> Hello Tim,
>
> Thanks for your response and advice.
>
> I want to keep Org-mode as simple as possible. As you suggest.
>
> In the past, I ended up several times with a too complex Org-mode
> workflow and stop using it because of that. That because, today, I want
> to keep it simple. Usually, I apply a GTD workflow (or what I think it
> is, I'm not an expert).
>
> As you say, I need to learn skills for project management. But the
> project management methods we learned at school where to rigid. And, at
> work, the method is more "do the job, stop thinking, be professional".
> But it's, or was, the kind of job where you are asked to "not write test
> to save time". I generally have bad experiences at work.

Sadly, the software industry is full of some very poor middle managers
who don't really understand the complexities of the software
life-cycle. Too often, they focus on immediate deadlines and overlook
long-term maintenance. How you deal with such situations is down to
experience, confidence and where your own personal values lie. There has
been more than one job I've left because the way management was running
the project was poor and almost certainly going to lead to eventual
failure. There has been more than one job interview where I've stated
that if they are looking for someone to write code by the pound, I'm not
there man. I will ask leading questions in the interview to evaluate
what 'style' of development/management they use - for example, if they
measure productivity by the number of lines of code you right in a day,
I will thank them for their time and quietly walk away.

>
> To manage school big work, I think of managing them as projects.
>
> I want to apply a simple "Project" workflow:
>
> * Each project is a headline with the status "PROJECT"
> * Each project have the deadline defined by the school work deadline
> * Each project have a complete description with every info needed to work
> * Each project have one or many tasks (as sub headlines with a status)
> * Each task have a importance, time and effort estimation
> * Each task have its own deadline, distributed along the remaining time
> * When I set a task deadline, I look at its estimations and also other 
> projects tasks
> * To create a new project, I use Org-capture with a template
>
> Every time I create a new project, it start with one task: "Planning the
> project". With a deadline at 2 days max. The description of this task is
> a checkbox list of thing to do when planning the project.
>
> And finally, 2 times per week, I got a repetitive task: "Review the
> projects progress". With this, I should be able to adjust spending time
> and effort.
>

All seems like a reasonable starting point. The key is to regularly
review and refine your workflow. I would avoid the tendency to
think you have to put everything into your workflow. For example, I
would not have a task which says to review my tasks twice a week. Do you
really need a task to remind you to do this twice a week? Do you really
need to track that you have done this? I would classify such tasks as
'noise' tasks. They really don't perform any real purpose.  It is
similar to brain dead project policies which state things like "every
function must have a unit test". Many functions don't need a unit test
and forcing people to write pointless unit tests does more damage than
not having them. A unit test needs to be a positive addition and
deciding when it is or is not is part of what being a professional
developer is all about. Likewise with project management. You don't want
tasks for every simple thing you do. You want to record and track the
important tasks. No tool is a replacement for using your brain - these
are just tools and it is down to us to use them in an intelligent and
efficient manner.

Use org mode to make your life easier. If it isn't making it easier,
then you are doing something wrong and need to review and
re-prioritise.



Re: org-modern

2022-03-01 Thread Tim Cross


William Denton  writes:

> I noticed org-modern in the list of new packages available:
>
> https://github.com/minad/org-modern
>
> Worth a look by any Org users, just for the screenshots comparing Org buffers 
> normally and with
> org-modern.  Fine work from Daniel Mendler, who I think is on this list.  I'm 
> trying it out
> right now.
>

I was looking at this package and it looks interesting. However, I do
wonder how much of an impact it has on org performance. I suspect it
will have a significant impact on large org files. Would be interested
in hearing how people find it with larger files.



Re: How do you manage complex project with Org-mode

2022-03-01 Thread Tim Cross


Sébastien Gendre  writes:

> Hello,
>
> I don't know if it's the correct place to ask it. If not, sorry to ask in
> the wrong place.
>

I think it is the correct place. This is a list for general org mode
discussions. Such discussions can be technical or about how to use org mode.

> How do you manage complex project with Org-mode ?
>
> I used Org-mode for several periods of time in recent years. It worked
> very well for short and day to day tasks. When only a few of theme have
> deadlines and when you have plenty of time to do them.
>
> But, as a student, I regularly have big and important projects to do for
> the school. The kind of project who need several days to be done, with
> deadlines too soon, and if you fail one them the consequences can be
> disastrous. And generally, I have to many of these project in the same
> time and not enough time to do all the work. So, I also need to follow
> the progress of each project to choose which is sufficiently advanced to
> be stop for the benefit of another less advanced project.
>
> And I don't know how to manage this kind of projects with Org-mode. How
> to do it, without failing a 6 days project because I spent to much time
> on something else and I have only 3 days left with 3 half-day important
> appointment I cannot cancel. I can't risk failing a single one of these
> project by trying. So, when I am in a period with a lot of these
> projects, I stop using Org-mode and concentrate on doing these project
> as fast as I can. And because I often have this kind of project, I spend
> most of the year without being able to use Org-mode.
>

The first thing I would say is that org mode is NOT going to solve your
problems of too many tasks with competing priorities in too short a time
period or mitigate the impact from missed deadlines etc. In fact, no
project management software can do this.

Org mode is merely a tool which can help you manage tasks, deadlines,
schedules, priorities and project information. It can help you track
your tasks so fewer are overlooked/forgotten, it can help you manage
your deadlines and scheduling of time and it can help you manage
priorities and provide you with an overview of things that can alert you
to issues earlier, allowing for a wider range of mitigation
strategies. However, it is just a tool and how well you use that tool
will come down to experience and self discipline.


> So, if you have any suggestion on how to manage, in Org-mode, projects
> with:
> * Lot of work to do (many days)
> * Short deadline (not enough time)
> * High importance (disastrous consequences in my future in case of fail)
> * Many of them in the same time
> * Progression need to be followed to chose where to sacrifice time to
>   limit the damages
>

That list is almost an exact match for the job criteria on the last job
I applied for, which highlights the point these are individual skills
which need to be learned, primarily through experience and not something
which can be solved by a software tool. Such tools can help ensure
things don't fall through the gaps during busy stressful projects, can
help you assess how much work needs to be done and where corners will
need to be cut or deadlines changed, scope reduced or quality
decreased. At some point, you have to make a call as to what you will
manage in your org-mode files. Too much or too detailed and too much
time will be spent gathering and managing the data. Too little or
insufficient detail and decisions on what has to be sacrificed are
likely to be misguided or wrong. Unfortunately, there is no formula to
calculate this. It will depend on the environment, types of projects and
individual experience and preferences.

The key to using org mode to manage projects is largely about
incremental refinement. You start by defining a plan on how to use org
mode, you then implement that plan and start using it. You then review
how well it is working at some point and take the experience to that
point, both positive and negative, into a new cycle, starting with a new
plan (refined plan), implement, use review and continue this cycle
(probably indefinitely, but likely with longer cycles).

In general, I would recommend the following

- Start simple. Don't try to do everything all at once. Org is extremely
  flexible with a lot of built-in functionality. Trying to use all of it
  all at once is likely going to make it a burden rather than an aid.

- Start with a default org-mode configuration. A mistake I've seen
  people make many times is to immediately start by configuring new and
  complicated TODO states or complicated capture templates or extensive
  tag hierarchies and complex priority levels. Avoid the temptation to
  over engineer your requirements. Use the default configuration for a
  time and then evaluate things and decide where to make some small
  changes.

- Don't let the tail wag the dog. Remember, org mode is there to make
  your life easier. If you find your now spending all your time 

Re: Communication problems and possible problems with the website

2022-02-27 Thread Tim Cross


Bastien Guerry  writes:

>
> Discourse is nice but I'm not favor of installing an instance for Org.
>
> Beginners often ask questions on reddit.com and stackoverflow.com (and
> perhaps elsewhere): perhaps some regular users of these websites could
> serve as "contributors stewards", redirecting interesting bug reports,
> patches or feature requests on this mailing list.
>
> My gut feeling is that we should focus on making the mailing list more
> accessible for beginners, more useful for everyone before considering
> setting up another communication channel.
>

+1. We cannot possibly support every user's preferred platform. Besides,
didn't Discord just do an IPO? This increases the likelihood the
platform will now have a much higher revenue focus and could mean it may
transition over time to either a paid model or add advertising or some
other revenue raising model.

(I notice with interest that Google has started transitioning some of
its services which were previously free to a paid model and I wonder if
the free service gravy train might be coming to an end?)




Re: ox-taskjuggler missing

2022-02-19 Thread Tim Cross


Greg Sullivan  writes:

> Trying to export to taskjuggler (tj3) to get a gantt chart as in this 
> description.
> But I can't find the taskjuggler exporter.
>
> emacs-version
> => GNU Emacs 27.2 (build 1, x86_64-apple-darwin18.7.0, NS appkit-1671.60 
> Version 10.14.6 (Build 18G95)) of 2021-11-18
>
> org-version
> => Org mode version 9.4.4 (release_9.4.4 @ 
> /Applications/Emacs.app/Contents/Resources/lisp/org/)
>
> (require 'ox-taskjuggler)
> => Debugger entered--Lisp error: (file-missing "Cannot open load file" "No 
> such file or directory" "ox-taskjuggler")
>
> There does not seem to be an "ox-taskjuggler" on melpa.
>
> I couldn't find any notes about taskjuggler export explicitly being removed 
> from the standard org distribution.
> Do others with emacs 27.2 and org-mode version 9.4.4 have ox-taskjuggler 
> available?
> If so, I must have messed up my installation.
>
> Thanks for any guidance.
> -- Greg


TaskJuggler is not part of org mode. It is one of the packages in the
org contgrib package, which is now distributed via the nongnu elpa
repository (see https://elpa.nongnu.org/nongnu/).

I haven't used it in a long time and I don't think anyone is maintaining
it, so don't know how well it is working with current org versions.





Re: Reload uncompiled needed

2022-02-17 Thread Tim Cross


"Loris Bennett"  writes:

> Hi,
>
> I have managed to install 9.5 but I now get 
>
>   org-agenda-get-day-entries: Invalid function: (date date)
>
> when I try to generate my agenda.  If I reload Org uncompiled, the error
> disappears.
>
> In *Messages* it seems like the very same version has been reloaded:
>
>   Org mode version 9.5.2 (9.5.2-gfbff08 @ 
> /home/loris/.emacs.d.default/elpa/org-9.5.2/)
>
>   Successfully reloaded Org
>   Org mode version 9.5.2 (9.5.2-gfbff08 @ 
> /home/loris/.emacs.d.default/elpa/org-9.5.2/)
>
> If I try to byte recompile the directory, everything seems
> up-to-date:
>  
>   Checking /home/loris/.emacs.d.default/elpa/org-9.5.2/... [125 times]
>   Checking /home/loris/.emacs.d.default/elpa/org-9.5.2/doc...
>   Checking /home/loris/.emacs.d.default/elpa/org-9.5.2/etc...
>   Checking /home/loris/.emacs.d.default/elpa/org-9.5.2/etc/csl...
>   Checking /home/loris/.emacs.d.default/elpa/org-9.5.2/etc/styles...
>   Done (Total of 0 files compiled, 124 skipped)
>
> The el and elc files seem to have the same date:
>
>   $ ll ~/.emacs.d.default/elpa/org-9.5.2/org.el*
>   -rw-r--r-- 1 loris users 834072 Feb  9 16:45 
> /home/loris/.emacs.d.default/elpa/org-9.5.2/org.el
>   -rw-r--r-- 1 loris users 719363 Feb  9 16:45 
> /home/loris/.emacs.d.default/elpa/org-9.5.2/org.elc
>
> I can't pretend to understand the contents of org.elc, but it does
> contain the following
>
>   (provide \\='org-xyz)\n\nFor export specific modules, see also
>   `org-export-backends'." :group org :set org-set-modules
>   :package-version (Org . "9.5")
>
> The version which comes with Emacs 27 on Debian seems to be 9.3:
>
>   $ zgrep '^;; Version' /usr/share/emacs/27.1/lisp/org/org.el.gz
>   ;; Version: 9.3
>
> I presume I could just delete all the elc-files, but I would like to
> know what is going on.  Can anyone see what is happening?
>

How did you upgrade? (package.el, straight.el, manual git clone etc)?

Did you make sure no org functionality was already loaded into Emacs
before you performed the upgrade?

If you try to upgrade org using a apckage manager, such as package.el,
and you already have some org functionality loaded at the time of the
upgrade, you can get a broken 'mixed' installation. This will often
create the type of errors you are experiencing.

The solution is to uninstall the upgraded version, start a new Emacs
session which has no org functionality loaded (may be necessary to
comment out any org related configuration in your init file to prevent
org being loaded at startup), upgrade org to current version and then
restore your init setup code. Depending on what configuration code you
ahve in your init file and how it is structured, sometimes it is
sufficient to just ensure you do the upgrade immediately after starting
a new Emacs session. Where people run into problems is when they upgrade
org in an Emacs session which has been running for some time (and which
likely has some org functionality already loaded). 



Re: Unable to get current or via use-package

2022-02-10 Thread Tim Cross


Ihor Radchenko  writes:

> Tim Cross  writes:
>
>>> Is that necessary?  Can't I just use the package manager to update Org
>>> along with any other packages?  Or does the issue about not visiting and
>>> .org file before installing via the package manager apply to updates
>>> too?
>>>
>>
>> Even with updates, it is still important that no existing org
>> functionality has been loaded to avoid potential inconsistencies due to
>> mixed version installation. For updates within the same version (i.e.
>> bug fix updates), the risk is low, but may still result in an
>> inconsistent build.
>
> It looks like there was an attempt to fix mixed compilation issue in
> package.el See
> https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=c13baa10d55ec863d3ceaea48c6b2959ece98198
> and https://debbugs.gnu.org/cgi/bugreport.cgi?bug=10125
>
> If I understand that thread correctly, package.el should not have issues
> with mixed compilation. Not so sure about straight.el and certainly it
> will not solve the problem when part of built-in Org is loaded before
> the newer Org version is added to the load-path.
>

Thanks for the info. However, I'm not confident that has addressed the
full issue. I note that patch is from 2014. There has certainly been a
number of people report mixed installation problems since that patch
would have been applied and part of their current version of package.el.
As you point out, we don't know what happens with other package managers
like straight.el and you are likely to get this issue if part of the
built-in version has been loaded before package.el attempts to load the
new version, so that might explain these later reports.

Would seem the best approach and simplest solution is still to ensure no
org functionality has been loaded before attempting an update.





Re: Unable to get current or via use-package

2022-02-10 Thread Tim Cross


Loris Bennett  writes:

> "Cook, Malcolm"  writes:
>
>>
>>  emacs -Q -batch -eval "(progn (require 'package) (package-initialize) 
>> (package-refresh-contents) (package-install 'org))"
>
> Is that necessary?  Can't I just use the package manager to update Org
> along with any other packages?  Or does the issue about not visiting and
> .org file before installing via the package manager apply to updates
> too?
>

Even with updates, it is still important that no existing org
functionality has been loaded to avoid potential inconsistencies due to
mixed version installation. For updates within the same version (i.e.
bug fix updates), the risk is low, but may still result in an
inconsistent build.




Re: the old "zsh:1: command not found: latexmk"

2022-02-08 Thread Tim Cross


bruce robertson  writes:

> I have the directory, /usr/local/texlive/2021/bin/universal-darwin, in .zshrc 
> & .zprofile
> AND
> in exec-path
> Do I need to file a bug to get help?
>
> Thanks,
> Bruce

Have you verified latexmk is in one of those directories?

What does the command

which latexmk

show on your system?





Re: [PATCH] Add support for $…$ latex fragments followed by a dash

2022-02-06 Thread Tim Cross


Rudolf Adamkovič  writes:

> João Pedro de Amorim Paula  writes:
>
>> On 01 February 2022 22:00, Rudolf Adamkovič  wrote:
>>
>>> Me, I cannot use any of these "pretty" features because, simply put,
>>> they break plain text.  For example, they cause misaligned tables and
>>> make the text overflow the fill column, which I keep at 72 columns.
>>
>> […] there are fonts that enforce Unicode (all characters, from what I
>> understand, Unicode included) characters to be exactly 1 unit width,
>> i.e. Unicode characters are the same widths as other characters.  At
>> least I can guarantee that any of the Term (all characters are the
>> same length, but has ligatures) or Fixed (same as Term but no
>> ligatures) for Iosekva [1] have this given property.
>>
>> [1] https://typeof.net/Iosevka/
>
> Thank you for sharing!  I use the amazing "Hack" typeface [1].  Then, I
> fail to understand how the font changes the fact that Org breaks "the
> promise of plain text" in this regard.
>
> For example, Org can hide its emphasis markers.  Later, one opens the
> file in another editor and sees the truth.  The lines go over the fill
> column, the tables have misaligned columns, and so on.
>

The promise of plain text has no inherent promise regarding alignment,
line wrapping etc. The promise of plain text is simply that - a promise
that org files will be just plain text rather than some application
specific format like many other systems (MS Word, LibreOffice, etc).
Provided you can still edit the file with a plain text editor, org has
not broken its promise.

Ironically, it is this plain text basis of org files which makes
on-going support of $..$ (and any extension) so problematic. Without
this restriction, org files could have a format where elements were
'tagged' to remove ambiguity and simplify the parsing process. However,
this would of course mean that either you had to edit the file within
org mode or you would have to be intimately familiar with the internal
structure of org mode files and use an editor which made
editing/updating the internal structure possible. To be very clear, I am
NOT advocating that org should adopt some form of internal tagging or
structure. I'm only attempting to highlight that having a system which
aims to maintain the data source in plain text comes at a cost.

In LaTex et. al. $..$ works well because $ is a special character. If
you want a literal $, you have to escape it. In org, $ is not a special
character. This means we need complex regular expressions in order to
identify when $ is being used for LaTex specific markup and when it is
being used in other contexts (e.g. literal $ sign, variable name). These
regular expressions are difficult to get right, error prone and above
all, incur a significant performance hit. Extending the syntax further
to support $..$- simply makes the situation worse by adding further
complexity.

So far, all the arguments against removal of support for $..$ are based
on inconvenience and reduced readability associated with the
alternatives. Both legitimate concerns. For many, the proposed change
may seem to be just inconvenient change for no real purpose because they
don't deal with the complexities inherent in making $..$ work and for
others, change is something they would always prefer to avoid.

In this instance, I feel we really need to listen to those who put in
such a dedicated effort in maintaining org mode and accept their
position that sustaining $..$ syntax going forward is problematic and
focus on what can be done to mitigate the impact from this change, make
the alternative syntax more convenient and address the readability
issues.




Re: Suggestion: convert dispatchers to use transient

2022-02-03 Thread Tim Cross


Jim Porter  writes:

> On 2/3/2022 2:07 AM, Tim Cross wrote:
>> After 28.0 is released, if transient is a GNU ELPA package, we can
>> probably just make it an org dependency and Emacs 26.x and 27.x should
>> be able to install and run it (would need to be verified).
>
> The transient package is on GNU ELPA already[1], and based on its package
> metadata[2], supports Emacs 25.1+, so I don't think there should be any
> compatibility concerns (at least not with the availability of transient; maybe
> there are other reasons that dispatchers shouldn't change without an option to
> go back).
>
> - Jim
>
> [1] https://elpa.gnu.org/packages/transient.html
> [2]
> https://github.com/magit/transient/blob/440a341831398b825dc2288a10821cf7be1999ca/lisp/transient.el#L9

That is good to know. I knew it was getting added, but wasn't sure how
compatible it would be with older versions.

Not 100% sure about best migration strategy. On one level, providing the
ability for users to choose which interface sounds nice, but on the
other, a main goal is to simplify code and reduce maintenance. Providing
a choice will actually complicate matters and increase maintenance.

Is the transient.el included in Emacs 28 the same as the one on GNU
ELPA? I'm assuming it is, but I have to admit I'm still not 100% clear
on how Emacs handles the situation where you use a library that is both
built-in and available in ELPA. Does Emacs use the latest version
available or does it use the built-in version until you explicitly
select the ELPA versions?




Re: Suggestion: convert dispatchers to use transient

2022-02-03 Thread Tim Cross


Hugo Heagren  writes:

> Org uses various dispatchers, where invoking a command gives the user a
> choice of different sub-commands, chosen by pressing a relevant key,
> from a list displayed on the screen. Some of these dispatchers include
> options which can affect the command chosen. Examples include org-
> capture, org-beamer-select-environment and org-export-dispatch.
>
> These desptachers are idiosyncratic, written for purpose, and each
> behave differently. They have varying levels of customisability, and
> this is reached in different ways for each. Overall, I think the user-
> experience could be more consistent and more easily customisable.
>
> Luckily, recent versions of emacs ship with transient.el, a powerful
> way of building such interfaces in a consistent and easily extensible
> way.
>
> So, I propose to rewrite the current dispatchers as transients. What
> does the community think? I would be happy to work on this unless
> others strongly object, but I don't  know everything about org, so if
> others could help me with a list of other dispatchers which could also
> be converted that would be helpful.
>

This has been discussed previously and I believe the overall consensus
was that moving to transient is the right direction to take.

However, the stumbling point at this time is that org needs to retain
backwards compatibility with at least 2 previous Emacs releases. The
current release is 27.2, so we need to maintain compatibility with 26.x
and 25.x. Emacs 28 is at 'release candidate' status, so we can expect it
will be released within the next 6 months or so. I expect the next org
release will also occur about then. We would then only need
compatibility with 26.x, 27.2 in addition to 28.0

After 28.0 is released, if transient is a GNU ELPA package, we can
probably just make it an org dependency and Emacs 26.x and 27.x should
be able to install and run it (would need to be verified). 



Re: cut and paste not working after xdg-open "org-protocol://store-link?url=URL=TITLE"

2022-01-30 Thread Tim Cross


chris  writes:

> On Monday, 31 January 2022 04:49:59 CET chris wrote:
>> On Monday, 31 January 2022 04:43:34 CET chris wrote:
>> > On Monday, 31 January 2022 04:28:06 CET Ihor Radchenko wrote:
>> > > chris  writes:
>> > > >> What will happen if you try (kill-new "something") in your Emacs
>> > > >> (preferably, emacs -Q)? Will "something" be present in your Wayland
>> > > >> clipboard?
>> > > > 
>> > > > emacs -Q, then `M-: (kill-new "something")`
>> > > > Then `C-y` in `*scratch*`, string "something" is pasted.
>> > > > Then go to terminal (Konsole, specifically), `C-S-v`, string
>> > > > "something"
>> > > > is
>> > > > pasted, too.
>> > > 
>> > > Got it. What about emacsclient -e "(kill-new \"test123\")"?
>> > 
>> > `~/path-to-emacs/lib-src/emacsclient -e "(kill-new \"test123
>> > \")"`
>> > 
>> > Emacs side, `C-y` => "test123"
>> > 
>> > Terminal side, `C-S-v` => no trace of the "test123" string.
>> 
>> My, I missed one pivotal point:
>> Right after that operation, I cannot cut and paste straight away from
>> terminal to emacs.
>> If I copy and paste some string "test567" from terminal to Firefox or
>> whatever, it is working fine, but `C-y` in emacs is still "test123".
>> 
>
> Also, right now, emacs kill-ring is not very big, less than 10 items, so I 
> can 
> see all of them using `M-y`, and the most recent one, after several cut-and-
> paste operations none of them emacs related, is still "test123".
>
> And so now I do the converse operation and that works, and after that copying 
> from terminal to emacs works again.
>
> Bottom line, it seems org-mode independent.
>

This is most likely an Emacs 29 issue. The pgtk stuff has only very
recently been pulled into the mainline development tree and it is very
likely that little testing has been done on wayland using the pure gtk
build (and I'm assuming no wayland X window integration layer).

You probably need to log it as an Emacs bug. It might be worthwhile
asking on the emacs-devel list - it could be a known issue.



Re: [PATCH] Add support for $…$ latex fragments followed by a dash

2022-01-27 Thread Tim Cross


Ihor Radchenko  writes:

> Rudolf Adamkovič  writes:
>
>> Let $r_i$ denote the \(i\)-th rotation of $t$ with a suffix of $\ell|t|$
>> characters deleted, for […]
>>
>> Me, if I could, I would pay money for this feature, for it would allow
>> me to use $$ consistently, focusing on mathematics instead of markup
>> idiosyncrasies of "rotation $i$" versus "\(i\)-th rotation".
>
> Would it improve things for you if we change how \(...\) _looks_ in Org
> buffers?
>
> The problem with parsing is more than just supporting $i$-th and
> similar. For example, AMS style guide explicitly advises against using
> $i$-th in favour of $i$th [1]:
>
> Do not hyphenate “th” expressions: xth, not x-th or xth .
>
> We can theoretically make a change to support "-", but then it will be
> logical to support $i$th as well. (If we don't some users will still be
> confused after trying to write $i$th and then not getting the expected
> results). In this question, it would make sense to implement
> all-or-everything approach. Otherwise, confusion (like raised in this
> thread) will be inevitable.
>
> However, from point of view of Org mode parser, supporting $i$th is a
> nightmare.  Remember that Org mode is _not_ LaTeX and we have to support
> a lot more frivolous syntax (even in LaTeX, runaway $ is often a source
> of cryptic compilation errors). Currently, we _must_ rely on heuristics
> to determine $$-style latex fragments. I do not know any way to support
> $$ syntax without creating deviations from LaTeX. Extending the
> heuristics will not resolve the underlying ambiguity of $$ syntax, just
> hide it within even more obscure cases.
>
> Given the raised concerns, may we solve the issue with too verbose
> \(...\) unambiguous syntax using the following approach:
> 1. Fontify \(...\) replacing the brackets with a single character. For
>example:
>
>   \(...\) -> ⁅...⁆
>
> 2. Provide convenient way to input \(\) brackets through
>electric-pair-mode or trough org-cdlatex-mode.
>
> Best,
> Ihor
>
> [1] https://www.ams.org/publications/authors/AMS-StyleGuide-online.pdf

+1.

Just my $0.02 worth -

I think this is the right approach. Retaining support for $..$ doesn't
seem feasible given all the complexities it brings with it. The main
objections to the alternative appear to centre around readability and
inconvenience of having to type additional characters or dealing with
muscle memory use to $...$. These are essentially interface issues and I
think we can largely address them using existing Emacs facilities. This
will reduce the change impact to that sub-set of org users accustomed to
$...$ while bringing the benefit of a cleaner and potentially more
efficient parser to all org users.

If we do deprecate support for $...$, it might also be a good idea to
see if we can add a utility function which would make it easier for
people to migrate existing documents to the new/alternative syntax. For
the same reason it is hard to reliably parse $...$ syntax, we probably
can't automate that transition, but we should be able to reduce the effort
required to update existing documents. 




Re: Poor org-babel-tangle-file performance with more than 100 trivial noweb-references

2022-01-25 Thread Tim Cross


Ihor Radchenko  writes:

> pareto optimal  writes:
>
>> Using =emacs -Q= to tangle org files with more than over 100 noweb-refs gets 
>> slow fast.
>>
>> Given this org code for N=2:
>> Using Gcc Emacs 28.0.91 (which I typicall use) I get these results:
>>
>> | N blocks | runtime | # of GCs |
>> |--+-+--|
>> |   10 |   0.027 |    0 |
>> |   20 |   0.049 |    0 |
>> |   40 |   0.121 |    0 |
>> |   80 |   0.391 |    0 |
>> |  160 |   1.426 |    0 |
>> |  320 |   6.765 |    0 |
>> |  640 |  23.972 |    0 |
>>
>> so roughly it scales like:
>>
>> #+begin_example
>> 0.8x (10-20)
>> 2.5x  (20-40)
>> 3x  (40-80)
>> 3.5x  (160-320)
>> 4x  (320-640)
>> #+end_example
>>
>> Though I'm not sure how much that can tell us... my guess is noweb-ref's 
>> don't use a dictionary with =O(1)= lookups?
>
> Thanks for the nice benchmark and for providing a test case!
>
> You are using stable version of Org. We did some important performance
> improvements on main. The same benchmark using latest main (26decec):
>
> | N blocks | runtime | # of GCs |
> |--+-+--|
> |   10 |   0.030 |0 |
> |   20 |   0.067 |0 |
> |   40 |   0.065 |0 |
> |   80 |   0.081 |0 |
> |  160 |   0.214 |0 |
> |  320 |   0.597 |0 |
> |  640 |   2.225 |0 |
> | 1280 |   9.221 |0 |
> | 2560 |  36.966 |0 |
>
> And with disabled self-verification code:
> (setq org-element--cache-self-verify nil)
> | N blocks | runtime | # of GCs |
> |--+-+--|
> |   10 |   0.078 |0 |
> |   20 |   0.075 |0 |
> |   40 |   0.063 |0 |
> |   80 |   0.085 |0 |
> |  160 |   0.095 |0 |
> |  320 |   0.178 |0 |
> |  640 |   0.311 |0 |
> | 1280 |   0.819 |0 |
> | 2560 |   2.302 |0 |
> | 5120 |   8.878 |0 |
> |10240 |  32.706 |0 |
>
> Graphical comparison:
>
>
>
>
> There is ~80x improvement.
>
> However, the scaling is still not quite linear:
>
>
>
>
> So, there is still some power-law nonlinearity in the tangle code.
>
> Benchmarking revealed the following:
> 8259  95%   - org-babel-tangle-file
> 8259  95%- org-babel-tangle
> 8170  94% - org-babel-tangle-collect-blocks
> 3620  41%  - org-in-archived-heading-p
> 3600  41% org-before-first-heading-p
>
> Showing that nonlinear scaling comes from regexp search.
>
> Caching org-before-first-heading-p improves the performance further (see
> Org 9.6-dev no self-verify + patch curves):
>
> | N blocks | runtime | # of GCs |
> |--+-+--|
> |   10 |   0.118 |0 |
> |   20 |   0.106 |0 |
> |   40 |   0.136 |0 |
> |   80 |   0.157 |0 |
> |  160 |   0.139 |0 |
> |  320 |   0.212 |0 |
> |  640 |   0.542 |0 |
> | 1280 |   0.797 |0 |
> | 2560 |   1.533 |0 |
> | 5120 |   4.651 |0 |
> |10240 |  16.745 |0 |
>
> The profiling gives:
>
>16817  63%- org-babel-tangle-file
>16280  61% - org-babel-tangle
>16200  61%  - org-babel-tangle-collect-blocks
> 1360   5%   + org-babel-tangle-single-block
> 1210   4%   + org-babel-get-src-block-info
>
> Most of the CPU time is spent in org-babel-tangle-collect-blocks, which
> is basically another regexp search for all the source blocks in buffer.
> The scaling is still slightly non-linear - maybe your source block
> regexp is too complex:
>
> (defvar org-babel-src-block-regexp
>   (concat
>;; (1) indentation (2) lang
>"^\\([ \t]*\\)#\\+begin_src[ \t]+\\([^ \f\t\n\r\v]+\\)[ \t]*"
>;; (3) switches
>"\\([^\":\n]*\"[^\"\n*]*\"[^\":\n]*\\|[^\":\n]*\\)"
>;; (4) header arguments
>"\\([^\n]*\\)\n"
>;; (5) body
>"\\([^\000]*?\n\\)??[ \t]*#\\+end_src")
>   "Regexp used to identify code blocks.")
>
> On the other hand, the test data is already within quite non-realistic
> over 5000 number of blocks. Probably, further performance improvement is
> not very useful (we already have some overheads at smaller N).
>
> Best,
> Ihor
>
> -
> The Org file used to generate plots:
>
> #+name: nocache
> | N blocks | runtime | # of GCs |
>
> |--+-+--|
> |   10 |   0.027 |0 |
> |   20 |   0.049 |0 |
> |   40 |   0.121 |0 |
> |   80 |   0.391 |0 |
> |  160 |   1.426 |0 |
> |  320 |   6.765 |0 |
> |  640 |  23.972 |0 |
>
> #+name: cache
> | N blocks | runtime | # of GCs |
>
> |--+-+--|
> |   10 |   

Re: Extend the existing alternative set of key bindings for terminals (was: Second Ctl in keychord not detected)

2022-01-25 Thread Tim Cross


Ihor Radchenko  writes:

> Tim Cross  writes:
>
>> Ihor Radchenko  writes:
>>
>>> Also, it appears to me that we may keep losing terminal-incompatible
>>> keys in future unless we provide some mechanisms to check terminal
>>> compatibility automatically. Any ideas?
>>>
>>
>> No ideas on this. Problem being I don't think there is anything like a
>> terminfo service which will tell you about what capabilities/bindings a
>> terminal emulator has.
>>
>> Just some thoughts on this -
>>
>> I fear any such attempt is likely to end up in a game of 'wack-a-mole'.
>> While it makes some sense to provide alternative key bindings for Emacs
>> running under the GNU Linux console, especially given the limitations
>> under the console are well defined and constant, I'm not sure
>> we can provide reliable solutions or tests for different terminal
>> emulators (which will often 'reserve' various key bindings for their own
>> use. This is especially true for more advanced terminal emulators like
>> tmux.
>
> We cannot even handle GNU Linux console now. Technically, man 5 terminfo
> describes all the details on how to obtain terminal specs, but I am not
> sure how to extract useful information for key binding purposes. Can we
> do it programatically?
>

I probably wasn't clear enough in what I was trying to explain/suggest.
I'll try to clarify.

I think there are two different issues at play here. Key binding
limitations in the Linux console and key handling in different terminal
emulators. The first is fundamentally a limitation in the low level
kernel terminal driver and not much which can be done except choose
alternative key bindings when running under the linux console. The
second is more about limitations within the specific terminal emulator
program. Some emulators handle this better than others and it will be
near impossible to find key bindings which will work across all
different terminal emulators unless we restrict which key bindings we
use to a much smaller subset, which will inevitably mean many difficult
to use or less convenient bindings. This will just make everyone,
including those using more capable terminal emulators, suffer less
convenient key bindings and key bindings which are significantly
different from those used when running native window/GUI version. It
would make switching between GUI versions and terminal versions of Emacs
even less convenient. 

As an example, on my Ubuntu 21.10 system, when running Emacs under the
Linux console, C-c C-, completely fails. In fact,if you try to do
describe key for that combination, it won't work because Emacs never
sees the second key press. On the other hand, if I run Emacs inside
mate-terminal, Emacs will see C-c , and not C-c C-, (but it does get
both key presses). If I run Emacs under xterm, lxterm or uxterm,
everything actually works just fine. Describe key will report the
correct binding for C-c C-,

I don't believe terminfo will be of any help here. If you look at the
TERM setting for mate-terminal, uxterm, lxterm and xterm, they will all
reference various versions of xterm (often xterm-color or xterm-256color
etc). Looking at the terminfo definitions, I cannot see anything with
would indicate whether C-, for example is supported or not. I know of no
convenient and consistent way to determine if the terminal emulator
being used will support things like C-, or not. For the Linux console, I
think we can use the TERM environment variable to know when we are
running under the Linux console if we want to provide different key
bindings for some commands under the Linux console. There is also the
possibility you could create a keymap for the Linux console which would
configure some of the 'missing' modifier combinations to issue escape
sequences which can be used to emulate the modifier behaviour under GUI
environment - for example, I think you can create a keymap which will
allow C-up/C-dwon/C-left/C-right etc.

The question is, how useful will alternative key bindings actually be
for the Linux console. I imagine the user base for people who only work
under the Linux console is pretty small. For occasional use, the
alternative bindings are unlikely to be that useful as it is too hard to
change finger memory for occasional use and you probably won't remember
the different bindings anyway (I would probably just use M-x command in
these situations).

For the terminal emulator situation, I believe we should do nothing
except provide documentation on how to find or identify an appropriate
terminal emulator which will support the key bindings used by orc, such
as C-c C-,.







Re: Exporting Org file to Html with collapsable headings

2022-01-21 Thread Tim Cross


ZIPING CHEN  writes:

> Hi,
>
> I am working on a large org file with exceptional number of subheadings. For 
> example, I may have org headings inside headings up
> to 20 or 30 layers deep.
>
> I may have many things like this in the middle of the file.
> ** a new heading.
>
> When I export the file to html, I like to export in such a way that the org 
> headings become collapsable headline in html (toggle
> switch). I spend some time search the web, and find a theme called bigblow 
> which does something close to what I want, but not
> exactly. It provide limited layer of toggle switch only. I wish to turn all 
> layers of headings in org file into collapsable headline to
> html. 
>
> Does anybody know a way I can accomplish this?
>
> Thanks a lot.
>
> Ziping


The effect you are after can be achieved using either JS and CSS or with
just CSS. Most of the CSS frameworks out there will support this
functionality. For example

bootstrap - CSS and JS
bulma - CSS only
Tailwind - CSS only

I'm pretty sure all of the above support the ability to create a
minimised version which only includes the features you want. This can
help minimise unwanted interaction between different CSS style
definitions and reduce overall size.

Personally, I prefer bulma or tailwind over bootstrap as it avoids also
having to deal with javascript and having to bundle js code with your
final output. 



Re: Extend the existing alternative set of key bindings for terminals (was: Second Ctl in keychord not detected)

2022-01-21 Thread Tim Cross


Ihor Radchenko  writes:

> Also, it appears to me that we may keep losing terminal-incompatible
> keys in future unless we provide some mechanisms to check terminal
> compatibility automatically. Any ideas?
>

No ideas on this. Problem being I don't think there is anything like a
terminfo service which will tell you about what capabilities/bindings a
terminal emulator has.

Just some thoughts on this -

I fear any such attempt is likely to end up in a game of 'wack-a-mole'.
While it makes some sense to provide alternative key bindings for Emacs
running under the GNU Linux console, especially given the limitations
under the console are well defined and constant, I'm not sure
we can provide reliable solutions or tests for different terminal
emulators (which will often 'reserve' various key bindings for their own
use. This is especially true for more advanced terminal emulators like
tmux.

An alternative which might be worth considering would be to improve
documentation on using different popular terminal emulators, like tmux
which could cover both adapting org key bindings and adapting the key
bindings the emulator uses (a very quick google on this indicates you
can change the tmux bindings, but that detail is apparently not well
documented). Such documentation could include some guidelines on how to
identify the issue, identify at what layer (window manager, terminal
emulator, communication protocol etc) the problematic binding is being
intercepted etc. The interactions at this layer can be complex and
confusing, especially for users who don't have a clear model of how all
the layers interact.  

A more long term strategy which I wonder if we should consider is
whether org would benefit from adopting the use of something like the
hydra package. Org needs a lot of key bindings - many more than most
other modes. Available bindings are in short supply. Perhaps leveraging
off something like hydras would both offer more flexibility and make it
easier to manage. Likewise, could transient mode help in this area?






Re: Depreciating TeX-style LaTeX fragments

2022-01-16 Thread Tim Cross


Colin Baxter   writes:

>> Sébastien Miquel  writes:
>
> > Hi, With respect to readability, I only mean to point out that the
> > $…$ syntax is one less character, and that the \(\) characters are
> > quite overloaded.
>
> Indeed. Compare something like
>
> $g=\lim_{\delta m\to 0}(\delta F/\delta m)$
>
> with
>
> \(g=\lim_{\delta m\to 0}(\delta F/\delta m)\)
>
> Backslash city! I know which one I'd prefer to read.
>
> >> this is a good opportunity to point out that $/$$ are very much
> >> second class citizens in LaTeX now, no matter what you may see in
> >> old documents.
>
> > The posts that you quote are 10 years old. As per [0] (2020),
> > there will be no LaTeX3. Nor is it only old documents that use the
> > $…$ syntax : looking for learning ressources (see [1]), everything
> > that I find uses it. That includes The Not So Short Introduction
> > to LaTeX [2] (2021) and
> > https://en.wikibooks.org/wiki/LaTeX/Mathematics.
>
> Ah, LaTeX3 - whatever happened to that?
>
> > Although I have no evidence of this, my expectation is that the
> > majority of tex users use the $…$ syntax (it is in fact widely
> > used outside of tex: in most markdown flavors and texmacs for
> > example). I also expect that a significant proportion of tex users
> > are not aware of the \(…\) syntax. I think here of users that are
> > less tech literate than most of this mailing list.
>
> Agreed.
>
> Best wishes,

While I can see the advantages of $..$ for equations, I think we also
need to keep in mind that org mode is NOT a latex or tex editing mode.
While it is excellent at providing a higher level abstraction which
works well with Latex, other considerations also need to come into play,
especially with respect to efficient and consistent parsing of org mode
syntax. From that perspective, $...$ seem to add complexity which is
making it much harder to get consistency and efficiency in parsing and
processing things like font locking, indentation etc.

The question then becomes "Is the slight reduction in typing and/or
possibly more readable $..$ syntax sufficient justification for more
complex and difficult to maintain code for parsing, font-locking and
indentation/filling? Furthermore, could not the readability issue be
even further enhanced with the \[...\] syntax if we are able to parse
the contents more reliably/efficiently and possibly provide other
mechanisms to improve readability of math/formula? (i.e. better
font-locking, hiding of delimiters etc).

I'm not convinced arguments regarding what authors familiar with writing
in Tex/Latex are familiar with is terribly relevant to org mode. There
are already things in org mode which are inconsistent with what you
would write in pure Tex/Latex and as mentioned, org mode is not just a
front-end for writing Tex/Latex documents. Org has its own flavoured
markup and we should work towards making the syntax of that markup as
consistent, clean and verifiable as possible. 



Re: Playing down the text in org-mode

2022-01-14 Thread Tim Cross


fatiparty--- via "General discussions about Org-mode."  
writes:

> Jan 15, 2022, 02:13 by e.fr...@ucl.ac.uk:
>
>> On Friday, 14 Jan 2022 at 14:44, fatipa...@tutanota.com wrote:
>>
>>> I would like to include the customisation using elisp code
>>>
>>
>> Have a look at, for instance,
>>
>> ,[ C-h f set-face-foreground RET ]
>> | set-face-foreground is an interactive native compiled Lisp function in
>> | ‘faces.el’.
>> | 
>> | (set-face-foreground FACE COLOR  FRAME)
>> | 
>> | Change the foreground color of face FACE to COLOR (a string).
>> | FRAME nil or not specified means change face on all frames.
>> | COLOR can be a system-defined color name (see ‘list-colors-display’)
>> | or a hex spec of the form #RRGGBB.
>> | When called interactively, prompts for the face and color.
>> | 
>> `
>>
> What is the face name associated with inline footnotes?
>

You might find the command M-x list-faces-display useful as it will show
you a list of the known faces with sample text using the attributes of
each face. You can also use the interface to inspect the current values
for a face and modify the face (via custom).

For setting attributes using elisp, you might find the function
set-face-attributes useful. 



Re: Playing down the text in org-mode

2022-01-13 Thread Tim Cross


fatiparty--- via "General discussions about Org-mode."  
writes:

> In org-mode one can use the emphasizing tags 
>
> *word*   /word/   _word_
>
> Is there anything to perform the opposite, playing down the text?


No, there are no syntax elements in org mode to do this.

Part of the challenge here is that de-emphasis or 'play down' is not well
defined. Traditionally, I think the way you handled adding text which
was considered important enough to include, but perhaps tangential or
less important was to use a footnote or margin note. The point of
emphasis in text is to draw the readers eye (and attention) to
something. It is difficult to do the opposite. Furthermore, most
techniques used to try and do this really just amount to making the text
harder to read, which doesn't seem very beneficial for those readers who
want that additional information. 

Personally, I prefer footnotes, asides, margin notes over including less
important text within the important text because even if the text is
somehow marked visually to indicate it is less important, I still have
to scan it when reading. A footnote on the other hand tells me there is
additional information available if I want to read it and all I had to
scan was a letter. This is also why I prefer Oxford over Harvard
referencing styles. 



Re: [BUG] Prefer lowercase #+results [9.5.2 (release_9.5.2-3-geb9f34 @ /Users/salutis/src/emacs/nextstep/Emacs.app/Contents/Resources/lisp/org/)]

2022-01-08 Thread Tim Cross


Rudolf Adamkovič  writes:

> If I remember well, Org decided to suggest with lowercase keywords (and
> use uppercase keywords in the manual, for some reason).  Then, we should
> change "org-babel-results-keyword" from "RESULTS" to "results".
>
> (1) Example #1
>
> #+begin_src R :results verbatim :wrap example
> […]
> #+end_src
>
> #+RESULT: [<--- inconsistent]
> #+begin_example
>
> […]
> #+end_example
>
> (2) Example #2
>
> #+begin_src R :results verbatim :wrap EXAMPLE
> […]
> #+end_src
>
>
> #+RESULT:
>
> #+begin_EXAMPLE [< still inconsistent]
> […]
> #+end_EXAMPLE
>
> P.S. Org should not leak potentially sensitive information, such as
> "/Users/salutis/", in the subject of its bug reports.
>
> Rudy
>

This is probably a can of worms not worth opening!

>From a technical perspective, case does not matter. Originally, the
'styhle' was to use upper case letters for things like #+BEGIN et. al.
(which is probably why they are in upper case in the manual). 
Then (I think it was with the change to the code to insert block
markers) there was a change to use #+begin_*. However, I think for those
who reverted to the old block insertion method (now based on tempo), you
still get things in upper case.

The point is, the begin v BEGIN issue is to some extent bordering on
becoming a religious argument and one which is unlikely to gain
consensus. While personally I agree with your point and for consistency,
I think the default should be lower case, it really doesn't matter.
Everyone will have their personal preferences and while it is trivial to
set to your preferred format, it is one of those issues which will
likely generate lots of noise with little resolution.  



Re: org table problems with sorting of some column

2021-12-31 Thread Tim Cross


Uwe Brauer  writes:

> [[S/MIME Signed Part:Undecided]]
>>>> "TC" == Tim Cross  writes:
>
>> Uwe Brauer  writes:
>
>
>> For problem 1, I suspect you will need to either change the date format
>> or tell org what the date format is. It won't work trying to sort
>> numerically as those values are not really numerical values. The date
>> format you currently have I think is ambiguous - in Europe it would
>> represent dd-mm-, but in the US it would represent mm-dd-. Best
>> solution would be to convert the values into an ISO format i.e.
>> -mm-dd.
>
> I just checked 
> I have the setting 
> ,
> | org-time-stamp-custom-formats is a variable defined in ‘org.el’.
> | Its value is (" %d.%m.%Y " . " %d.%m.%Y %a %H:%M ")
> `
>
> But this as I understand only effect via 
> ‘org-display-custom-times’ the representation. 
>
> It does not help me to recognise  28.04.2020 as a European date.
>
> [[End of S/MIME Signed Part]]

I suspect org is actually passing off the sorting/comparison to calc
(based on nothing but a guess of course). I would look into how calc
determines the format for date/time calculations and see if that can
help.



Re: org table problems with sorting of some column

2021-12-30 Thread Tim Cross


Uwe Brauer  writes:

> [[S/MIME Signed Part:Undecided]]
>
>
> * Problem 1
>
> I imported this column from a CVS file, so the date format was already chosen.
>  
> |   Date |
> ||
> | 05.05.2020 |
> | 07.01.2021 |
> | 07.01.2021 |
> | 07.05.2020 |
> | 09.03.2021 |
> | 12.05.2020 |
> | 13.02.2020 |
> | 13.03.2020 |
> | 13.03.2020 |
> | 14.05.2020 |
> | 15.02.2021 |
> | 15.02.2021 |
> | 17.02.2020 |
> | 17.04.2020 |
> | 20.04.2020 |
> | 22.04.2020 |
> | 23.06.2020 |
> | 24.02.2021 |
> | 24.04.2020 |
> | 25.09.2020 |
> | 25.11.2020 |
>
> I'd like to sort these dates, but when I do that numerically it does not work 
> as expected, any idea what to do.
>
>
> * Problem 2
> I'd like to order the column below descending 
>
> |   Income |
> |--|
> |   837.00 |
> |   681.71 |
> |   664.16 |
> |   499.46 |
> |   316.64 |
> |   258.30 |
> |   258.30 |
> |   258.30 |
> |   187.52 |
> |   173.15 |
> |   170.87 |
> |   163.50 |
> |   127.76 |
> |   117.00 |
> |94.80 |
> |91.00 |
> | 2.852.21 |
> | 2.401.74 |
> | 1.912.16 |
> | 1.735.14 |
> | 1.480.87 |
> | 1.407.73 |
>
> But again it does not work.
>
> Any idea what to do?
>
> regards
>
> Uwe Brauer 
>
> [[End of S/MIME Signed Part]]

When reporting a problem, it would really help if you made it clear what
you have tried and how it didn't work. From your description above, it
is impossible to know if you actually tried using the built-in table
sorting command and if you did, how you used it (what input, how you
tried calling it etc). You also don't provide any details on what did
not work - was it an error, was the sorting wrong, did nothing happen at
all? There is also no details regarding your Emacs version or the org
version you are running. 

For problem 1, I suspect you will need to either change the date format
or tell org what the date format is. It won't work trying to sort
numerically as those values are not really numerical values. The date
format you currently have I think is ambiguous - in Europe it would
represent dd-mm-, but in the US it would represent mm-dd-. Best
solution would be to convert the values into an ISO format i.e.
-mm-dd.

For problem 2, I suspect the issue is because all your values are not
numerical. The value 2.852.21 might be composed of numbers, but that is
not a numerical value (what is the value of 1 + 2.852.21?). Org will
interpret those values as strings and will sort them alphabetically.

The table sorting command in the manual states

‘C-c ^’ (‘org-table-sort-lines’)
 Sort the table lines in the region.  The position of point
 indicates the column to be used for sorting, and the range of lines
 is the range between the nearest horizontal separator lines, or the
 entire table.  If point is before the first column, you are
 prompted for the sorting column.  If there is an active region, the
 mark specifies the first line and the sorting column, while point
 should be in the last line to be included into the sorting.  The
 command prompts for the sorting type, alphabetically, numerically,
 or by time.  You can sort in normal or reverse order.  You can also
 supply your own key extraction and comparison functions.  When
 called with a prefix argument, alphabetic sorting is
 case-sensitive.

Note the part on providing key extraction and comparison functions. If
you cannot convert your data to data formats org is able to recognise,
then you will need to provide functions to provide the key comparison
logic i.e. how to sort 2.99 and 2.852.21 etc.



Re: block folding - should this work?

2021-12-29 Thread Tim Cross


Robert Nikander  writes:

> I started reading about “blocks" in the manual. I wanted a chunk of text that 
> I could hide, so I tried this:
>
> * Test
> Some text
> #+BEGIN 
> Hide this
> #+END
>
> Hitting TAB on the BEGIN line does nothing. But if I add a blank line before 
> it,
> then hitting TAB hides and shows the block. Is that a bug? Or am I doing it
> wrong? Seems like it should work without the blank line.
>
> * Test
> Some text
>
> #+BEGIN 
> Hide this
> #+END
>
> M-x org-version => 9.5.1.

The problem here is that your blocks are not correctly specified. If you
run M-x org-lint on your example, it will tell you the blocks may not be
correctly specified. You need to tell org what sort of block it is. Org
supports a number of different block types, such as source code blocks
(#+begin_src/#+end_src), example blocks (#+begin_example/#+end_example),
centred blocks (#+begin_center/#+end_center) etc. You can use C-c C-, to
run the command org-insert-structured-template to select the block type.
If you first highlight the region, it will be wrapped in the block you
select.




Re: C-c C-, in a terminal?

2021-12-28 Thread Tim Cross


Anssi Saari  writes:

> Hi all, while it's not an ORG mode question exactly, I have a problem
> with org-insert-structure-template and it's default shortcut, C-c C-, in
> terminals.
>
> As C-, isn't usually a control character in a terminal, terminals send
> just the comma if I hit C-,. Some terminals (at least wsltty) do send
> some kind of control sequence but I don't know if mapping other
> terminals to do the same (Konsole and urxvt at least) and changing the
> keys for Emacs makes sense. I'd have to do it for every terminal I ever
> use.
>
> Alternatively, any suggestions for a more terminal friendly key
> combination for org-insert-structure-template in Emacs?

It is almost impossible to suggest an alternative key binding because
there are just too many variables involved. What you can use will depend
on what window manager you use and the key shortcuts it uses (which will
'steal' keys before Emacs get to see them) and on the terminal emulator
and the remote access protocol (i.e. ssh, telnet, tmux, screen etc) you
use. Then there is differences depending on what Emacs packages you are
using and what (if any) key bindings you have already defined and
whether they are global or local definitions etc.

My suggestion would be to use something like the free-keys (available in
melpa) package to identify what key bindings are available when in an
org buffer and select one of those. It may also be worthwhile
considering using something like hydra. You could define an org specific
hydra and associate all your preferred bindings to that hydra. Org can
be particularly challenging because it already has some complex key
bindings, so avoiding unexpected side effects when adding/changing
bindings can be a challenge. 




Re: Dealing with growing backlog (was: next-error for agenda searches)

2021-12-24 Thread Tim Cross


Ihor Radchenko  writes:

> Samuel Wales  writes:
>
>> one issue with this great thing called capture is that there is
>> nothing quite so convenient that does the exact opposite.
>>
>> [you can regularly purge, if your life/forest is simple enough or you
>> have the physical ability to do things.  but you can't just
>> org-doneify-lower-value-stuff-i-captured-when-wasn't-sure-of-their-value-at-the-time
>> without adding energy, concentration, time, etc.]
>
> If I understand you correctly you are talking about ever-growing someday
> list.
>
> My latest solution to this problem (which I am quite happy with) is the
> following:
>
> 1. Every day/week I go through recently added someday staff and look if
>it still looks useful.  For ideas, I just check if they still make
>sense and for links, I open each link and skim through the abstract
>and sometimes link text in more details.
>
>When I first did the above, I was surprised that 50-80% of captured
>staff is just gone because it is not as interesting as it looked
>initially.
>
> 2. The ideas/links I mark for some day in future are scheduled using
>org-learn. They will appear again in my agenda a few days later and I
>can re-assess them. If still looks interesting, but someday not now -
>reschedule using org-learn utilising spaced repetition. Otherwise -
>archive.
>
>With the above approach, I only see "not sure" ideas
>days->weeks->years later. Only several times a year. More useful
>ideas remind about themselves more frequently and I often end up
>actually using them.
>
>Credit of this idea:
>
> https://www.getdnote.com/blog/how-i-built-personal-knowledge-base-for-myself/
>
> The total time needed to do the described is surprisingly small,
> especially with the ability to do bulk agenda operations to postpone all
> the maybe staff when you have no time/energy/mental power.
>

I use a very similar technique. Provided you do this type of
'house keeping" on a regular basis, it doesn't take long and the quality
of material in your knowledge base increases.

I've also learnt to be quite ruthless in making decisions about what to
archive and what to simply delete. This is possibly the hardest
behaviour to learn. In a digital age, it is way too easy to keep
everything. However, this has a hidden cost - the quality of material
degrades and results from searching etc become less productive because
there is too much 'noise' in your data. Data, like many other things,
degrades over time. It is important to learn how to recognise when
things have passed their 'use by date' and just get rid of them.

I'm often stunned by people who boast about the fact they still have all
their emails boing back to 1990 and are somewhat proud about having an
email archive with over 50Gb of messages. Reality is, the vast majority
of those email messages will never be read again and are of little or no
actual value.

Knowing when to just delete old data is almost as important as gathering
it, but a much harder skill to master.




Re: text after sub headings?

2021-12-23 Thread Tim Cross


Robert Nikander  writes:

> I see why this is not possible, given the text format of an org file. But I 
> am curious if people think it would be useful. This is a bit off-topic maybe, 
> but I’m imagining what I would do if I created something like org-mode using 
> another underlying format.
>
> Example: 
>
> * Top
>   Some text under “Top”
>   ** A level-2 heading 
> Text under level-2 heading
>   ** Another level-2 heading
> Text under the second level-2 heading
>   More text under “Top”
>   So if the level-2 headings were collapsed we would still see this.
>   ** Could have more sub-headings here
>   More top level text, etc.

While I can see how the structure you suggest could be useful in some
use cases, the big drawback is that to implement this, you would have to
add some new 'marker' to indicate where the contents of a
section/sub-section end. Personally, I would find the need to add
section end markers more inconvenient than having this feature, which is
something I would probably only use very rarely. I also suspect it would
have a processing penalty as org would now need to search for and track
matching end markers for each section rather than just searching for
another heading marker of the same level when performing folding.

In my use cases, it is extremely rare to have a situation where I have
sub headings and want to add something at the end which is part of the
parent heading, but which must follow the sub headings. Where I have
found such a structure useful, the contents of the sub headings tend to
be small and more suitable either for a list or one of the other
#+begin_* block types.

One of the original benefits of org mode was simplicity and a basis
built on simple text files. Maintaining that simplicity requires loss of
some flexibility. The more we complicate the model, the less simple it
remains and the more bugs we get. 



Re: Parens matching errors in org-babel code blocks

2021-12-21 Thread Tim Cross


I think this is a known problem (Org probably needs a PROBLEMS or
KNOWN_PROBLEMS file in addition to a NEWS file).

Problem is that org treats things like <> as special paren syntax
because they are used by org. Unfortunately, this causes problems with
things like paren matching in org files if you also have code which uses
these characters 'unmatched' e.g. (< 1 2) or (> 2 1). This is partly due
to things like check-parens, which doens't really know about babel code
blocks and so just uses the syntax table to find all characters marked
as parenthesis type syntax characters and generates an error when they
are not matched/closed.

Issue is, there is no universal and clean way to fix this which
doesn't have unwanted side effects in some use cases or has an impact on
speed/performance.

There was one solution proposed a couple of months ago which looked
promising, but it didn't seem to gain any traction. Not sure why. You
can probably find it in the archives. 

One solution which can work is to modify the syntax table for org
buffers to not mark things like < and > as syntax parens. However, this
can have other side effects, depending on how you use org.

What I do is not have paren matching or run check-parens in org buffers.
I also use org-edit-special (C-c ') to edit babel blocks as this gives
me the 'normal' language setup and things like check-parens or parens
matching work correctly.

Eddie Drury  writes:

> Inside org-babel code blocks, emacs matches parens incorrectly whenever 
> different bracket types (eg <> {} []) are used. 
>
> For example the following in an .org file will cause check-parens to throw an 
> error:
>
> #+BEGIN_SRC emacs-lisp 
>
>   (> 1 0)
>
> #+END_SRC 
>
> Where in an .el it will interpret the parens correctly. This error also 
> occurs with show-paren-mode and evil-jump-item. 
>
> Thanks very much. 
>
> Emacs  : GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.27, 
> cairo version 1.17.4)
>  of 2021-03-27
> Package: Org mode version 9.4.4 (release_9.4.4 @ 
> /usr/share/emacs/27.2/lisp/org/)




Re: format/fill of text in a cell in tables

2021-12-17 Thread Tim Cross


 writes:

> [[PGP Signed Part:Undecided]]
> On Fri, Dec 17, 2021 at 08:51:59AM +1100, Tim Cross wrote:
>
> [on flowing text whithin table cells]
>
>> I agree. This is actually a much harder problem to solve than it may
>> appear on the surface [...]
>
> Tables have an advantage if your approach is an explorative one, i.e. if
> the process is part of the result. I don't think they are as successful
> as they are for no reason (SQL, or R's data frames are about tables, after
> all, so it's not only Excel). If you want your reader to take part in
> the exploration process, a table might just be right.
>

Yes, sometimes tables are extremely useful - especially wrt 2-d
relationships I'm not against the use of tables, but do find their use
as a formatting/layout tool limited. Unfortunately, this is often how I
see them used. Much of my work has been with databases, so conceptually,
I often think in terms of tables, rows and relationships. More often
than not, while tables are good for simple entities, they are poor for
modelling because things often break down into hierarchies and related
sets etc.  

Many years ago, I worked on a system which used an interesting interface
which used 'cubes' to represent database information. At the time, all
of us who worked on the system dreamed of the day when you could have a
hologram display, true 3-d manipulation and linking of data cubes with
the ability to interact and walk around it to cut (slice) the data in
different ways. It was a lot of fun, but at the time computing power was
not quite up to the task. We even used an interesting logic based on
work done by a mathematician called Charles Peirce, who defined a
deductive logic based on graphs where your basic logic operations were
done through the union, intersection and projection of graphs etc. An
interesting approach which seems well suited to modern computer
interfaces (despite the fact he lived from 1839-1914). 

> The point you make about assistive technologies is hugely important. I
> haven't much experience with blind people myself, but I'm convinced that
> their perception of dimensionality (2D, 2D vs 3D) could be quite
> different from that of sighted people. Is a table an advantage or a
> disadvantage then? Does it depend on the strategic path they have
> chosen? Do some feel better at 3D? 5D? [2]
>

I have been legally blind my whole life and for 17 years, I had nothing
but light/dark perception (I could tell when it was day and when it was
night, or if a light was on, but that was about it). I was lucky to
regain some sight about 11 years ago and can now see colour, shapes and
even sometimes recognise people (though much of that is about other
cues, like hair length/style, clothing, deportment etc. I have a large
screen and use a large font, but rely heavily on text-to-speech. I'm
sort of between worlds - enough sight to prevent me really developing
'sightless'skills, but not enough to rely on sight for reading etc. My
braille is comprehension is terrible.

I know a few totally blind mathematicians and their skills are
impressive. Quite a few of them have ended up working in fields relating
to topology. I suspect that not having sight actually helps them in
their mental model of n-dimensions when n > 3. Sighted people seem to
find such dimensional thinking challenging and I suspect it is because
they are more accustomed to a 3-d world. For me, I have only 1 eye, so
no depth perception and a somewhat 'flat' view of the world :-).  

> * WYS ain't WYG
>
> Lastly, Org ain't WYSIWYG (well, duh). But such things as flowed cells
> are measuring it up to one, up to a point (although, at some point, I
> admit to having yearned for some). A strength is a weakness is a
> strength. I think it is the nature of Org to live with such conflicts.
> It's an interesting place, where it lives :-)
>

Yes, I can see why people like WYSWYG when editing. However, my
experience with such systems as more often than not been extremely
frustrating as I seem to end up constantly fighting with the system to
get it looking right rather than focusing on the content. I remember the
joy I had when I discovered Latex and how it allowed me to focus on
content and leave presentation to others who undoubtedly had far better
skills then me. Often the problem with WYSWYG is that it isn't always
true - more like WYSCWYG (what you see is close to what you get).  



Re: format/fill of text in a cell in tables

2021-12-16 Thread Tim Cross


Ihor Radchenko  writes:

> George Michaelson  writes:
>
>> Thank you for the continuing support for Org mode. I really appreciate
>> the work done.
>>
>> I would love to understand why there is no built-in 'fill/wrap'
>> function for text in a cell, inside an org mode table.
>>
>> Given how this is a layout mechanism, and heads to M-x ox-
>> outcomes, it would be useful to recognize the output is often table
>> edit states which do respect fill/wrap/center
>
> AFAIK, mostly technical difficulty. Emacs does not natively distinguish
> individual table cells, so we would need to emulate cell editing if it
> spans over multiple lines.
>
> Patches are welcome!
>

I agree. This is actually a much harder problem to solve than it may
appear on the surface. There have been a number of proposals regarding
how to do this over the years, but unfortunately, they have all had
significant drawbacks or limitations. One of which is that formatting
tables with multiple lines in LaTeX is actually a bit tricky (probably
the only exporter where it is relatively straight-forward is HTML). This
means you actually have two broad problems - handling cell wrapping in
the org buffers and handling it cleanly in the different exporters. 

In addition to the problem of defining some mechanism to treat each
table cell as if it was it's own 'minibuffer' (in TeX, you sometimes see
the 'minipage' environment used in this context), you also have to
define some mechanism to describe the relationship between the cells in
order to handle things like centring and justification. screen/page
width etc. To further complicate matters, you also have to consider what
this would mean for org's spreadsheet like capabilities when applied to
tables with cells wrapped over multiple lines. 

If you have lots of cells needing wrapping, the table is perhaps not the
right layout mechanism to use. While it may seem like a convenient way
to present content, often it isn't a great way to consume it. Donald
Knuth wrote a bit about why tables with multiple cell lines were a poor
choice. After years of dealing with project managers who too often use
Excel to record, present and share data, I tend to agree with his views.
I'm also old enough to remember when the table was the 'goto' solution
for managing layout in HTML files and what a mess that became.

While tables are great when you want to show 2-dimensional
relationships, for other situations, alternatives like definition lists,
nested lists and breaking the content up into subsections are often a
good alternative. It could be argued that not supporting table wrapping
is a positive aspect as it makes the author consider alternative layout
approaches which may actually improve readability of the content.

Finally, there is also an accessibility issue with multiple line tables.
For users who rely on assistive technology to consume content,
presenting information in a meaningful manner which is easy to navigate
and can represent larger 'chunks' of information with appropriate
indicators for the relationships between the cells is challenging.

As an aside, I sometimes find it useful when thinking about how to
layout information, how a typical user will consume it. We tend prefer
layouts with infinite length (pages), but with set width. Scrolling
up/down is convenient. Scrolling left/right is not. While larger screens
means we have more width to work with when reading on the screen, this
does not map well to printed pages as that 'width' has not changed -
best we can do is switch to 'landscape' rather than portrait mode. The
other problem with width is in variability - many people have wide
screens on desktops, but narrow screens on mobile devices. We can see
how quickly this becomes complex when we consider all the challenges we
have had with respect to being able to render web content on different
devices with varying screen sizes. Much of the complexity of CSS is
related to column layout and screen sizes. 



Re: [PATCH] org-test: Create a collaborative test set for Org buffer parser

2021-12-14 Thread Tim Cross


Max Nikulin  writes:

> On 11/12/2021 21:39, Ihor Radchenko wrote:
>
> Since these tests will unlikely become a part of some software, I do not think
> that GPL may be an obstacle for any project. Requirement of signed consent 
> will
> likely prevent contributing of new cases from some developers.

I agree. The test org files are input data rather than code and I don't
think they fall under the copyright restrictions (or the code
contribution guidelines). They probably don't even need to be GPL'd -
possibly a CC license would be sufficient.

As to whether they should be part of the org-mode repository or in their
own repository, I'm not sure. It would be convenient to have them int he
org-mode repository as I expect they will become part of the testing
framework and only having to checkout one repository would be useful. On
the other hand, I guess there could be cases where people want to just
checkout these samples to use to validate their own library/parser etc.
I tend towards putting them in the org-mode repository for simplicity.





Re: Concrete suggestions to improve Org mode third-party integration :: an afterthought following Karl Voit's Orgdown proposal

2021-12-11 Thread Tim Cross


Ihor Radchenko  writes:

> Tim Cross  writes:
>
>> ... while I totally agree we should work
>> very hard not to break compatibility or adversely affect other projects
>> which are built on top of org mode, like org-roam, we also don't want to
>> find ourselves in a position where we cannot improve/enhance org mode
>> because of the impact it has on other projects.
>
> Well. We have no direct control on the other projects. However, not
> doing anything about the fact that other project keep appearing is
> nothing but a call for more compatibility issues. If we do not clearly
> specify relatively stable syntax or API, the other projects will
> inevitably use internal implementation details and could be broken more
> easily. For example, my recent patch to org-element broke org-roam
> because org-roam relied on some undocumented behaviour of
> org-element-at-point.
>
>> Having thought about this whole thread and other recent posts, I still
>> feel any concern or reference to third party libraries etc is misguided
>> or at the least, irrelevant. Most of the suggestions are fine and would
>> be beneficial to org mode (such as clearly defined, consistent and
>> documented syntax). The fact 3rd party libraries would also benefit from
>> this is a bonus, but largely irrelevant.
>
> You read "Org mode third-party integration" as benefit for third-party
> libraries. I read it as benefit for Org from third-party libraries (as
> opposed to no benefit and potential complains from third-party library
> users).
>

I think 3rd party libraries are a benefit to org users. On the whole,
they are of no direct benefit to org-mode (if they are, then they are a
good candidate for inclusion into org mode). The relationship is very
similar to what Emacs has with all the external packages which are not
part of Emacs. There is no direct benefit to Emacs from all these
packages, but there is huge benefit for the Emacs community. Emacs
changes and evolves as necessary to keep it relevant, maintainable and
up-to-date with user expectations. At times, it will make changes that
are breaking on 3rd party libraries or require 3rd party libraries to
update/modify how they interact with Emacs. These changes are not done
lightly and are done so as to minimise impact to these 3rd parties.
However, Emacs does not concern itself with 3rd party libraries - it
focuses on making Emacs better and leaves 3rd party libraries to
themselves.

Placing too much focus on 3rd party libraries runs the risk of the tail
wagging the dog. Org mode should focus on what org-mode needs to do to
be performant, maintainable and minimise bugs. Having clear syntax
specifications, good unit tests, a clear and consistent API and
documentation all contribute to org-mode stability and maintainability.
Coincidentally, these are also the things which will most benefit 3rd
party libraries. However, should there be some justified reason to
change the syntax or the API in order to improve performance,
or maintainability, we should not feel constrained from doing so because
it will impact on 3rd party libraries. Instead, we should make such
changes in a staged manner and provide a reasonable time for 3rd parties
to be updated to work with the changes and only introduce breaking
change in new major versions.



Re: Concrete suggestions to improve Org mode third-party integration :: an afterthought following Karl Voit's Orgdown proposal

2021-12-09 Thread Tim Cross


"Dr. Arne Babenhauserheide"  writes:

> [[PGP Signed Part:Undecided]]
>
> Russell Adams  writes:
>
>> Did Org break your Org editing experience in Emacs for your Org files,
>> or did this change just break some of the finer formatting details of
>> your exported Org file?
>
> The change to electric indent broke my workflow badly (always having to
> undo the indentation after every new headline), and it took long until I
> found out how to avoid that.
>

That is probably a good example of how change can be imposed by external
events outside the control of org-mode. While I would agree that more
analysis of that change may have resulted in better initial
documentation and in turn, less inconvenience to users. that is only
obvious now with the benefit of hindsight. The fact it was a change
triggered by a change in Emacs rather than a change initiated by org
demonstrates that at times, org has to adapt to its evolving
environment. While this change may have 'broken' your workflow, the
previous behaviour was breaking other users workflows because org did
not honour electric-indent-mode and was not consistent with other core
emacs modes. 



Re: Concrete suggestions to improve Org mode third-party integration :: an afterthought following Karl Voit's Orgdown proposal

2021-12-08 Thread Tim Cross


Russell Adams  writes:

> On Wed, Dec 08, 2021 at 08:22:31PM +0100, Dr. Arne Babenhauserheide wrote:
>> >  - Anything outside of basic Org syntax, tables and source blocks I do
>> >directly in latex. Images are a good example. I will use latex code
>> >for the image, sizing, orientation, etc instead of relying on Org's
>> >extended syntax for image links, caption, and attributes.
>>
>> > As a result my publishing has been pretty consistent for customer
>> > documents. I also only update my Org between projects. ;]
>>
>> If I had needed a stronger argument for more backwards compatibility,
>> this list of habits is it. That should not be required to keep your
>> org-mode documents working.
>
> I think this may be a problem regarding expectations.
>
> I expect Org to be great at handling it's own format, and to give me
> the editing experience within Emacs that I have come to expect.
>
> That Org can also be used to export to other formats is both a
> blessing and a curse. Org can only do high level constructs in the
> languages it exports to, and really should only be expected to do just
> that. It's a paper thin macro or template over a much more complicated
> document language.
>
> Org's lightweight markup has had things bolted onto it repeatedly for
> years. Typically issues have resulted in changes in the export engine
> defaults (ie: html moving to using css), and not Org itself changing
> the editing experience in Emacs.
>
>> Org-mode is not just a library, it keeps user-data. It should really not
>> be volatile¹.
>
> Org's format isn't volatile. You could view those anytime in Org and
> see what you expect to see. The issue you are having is that an old
> document may not export perfectly over time.
>
> What if Org didn't diverge, the underlying format did?
>
>> If I can’t trust org-mode to keep working but have to check the
>> documents every time I come back to them — and might have to spend hours
>> fixing them — then it not suitable for writing, as much as that would
>> pain me (because it would cast into doubt most of my decisions around
>> writing of the past decade).
>
> You can absolutely trust Org to open, view, and edit it's own files
> even decades old. It's plain text, so there's no risk in experiencing
> a permanent loss of data.
>
> The exporting is the difference in expectations. Org's lightweight
> markup is quite simple, and the documents it produces should be as
> well. This is much like the original HTML specification. Look how
> complicated it is to write HTML now with CSS and Javascript emulating
> mundane functions after decades of bolt on "standards".
>
> If I had a document which had a highly sensitive output format which
> had to remain perfect over decades, I would argue that perhaps Org
> wasn't the correct markup to write it in.
>
> Much like plain text vs original simple HTML, vs Latex. Text was plain
> and simple, with little formatting. Durable and ugly at times, but
> always legible. The original HTML had more markup required, but it was
> hyperlinks and some simple fonts and formats. Prettier, variable
> fonts, colors, pictures. Latex can make pixel perfect PDFs with
> multiple medias and professional results, however it has a very
> specific format and this may be poor for writing in dynamically. HTML
> required decades of tweaks to become "pixel perfect", and HTML a
> decade old rarely renders properly in a "modern" browser.
>
> At some point with each of these languages, the formatting became more
> important than the content.
>
> I write all my customer documentation in Org, with custom Latex
> templates. I've only had to make major changes once, I think between
> v8 and v9. Yes, my old documents won't export identically without the
> changes. The likelihood they still export is high, and 100% that I can
> view and edit them correctly in Org. It's only the polished result
> which could degrade. I may have to tweak them to make them export the
> same way again, but I expect they can without too much effort. I'm OK
> with that.
>
>> Please do not make org-mode volatile.¹
>
> I think our maintainers have done an excellent job of minimizing the
> impact of any changes. However when changes are needed, I trust their
> judgement to have good reason to make a change and document it
> thoroughly.
>
> However I only export Org to be backwardly compatible with itself, not
> the languages it makes exports to.
>

Those are excellent points and highlight the fact much of what org does
is not always under the control of org. As you point out, the HTML
specification has changed a lot in the last 30 years. It use to be
'standard' to use upper case for HTML tags and closing tags were not
always required. However, html5 requires tags in lower case and now
expects closing tags.

Irony here is we are dealing with a mediam which is inherently
susceptible to change. Talk to any archivist and they will tell you
about the huge challenges they 'digital age' has created for 

Re: Concrete suggestions to improve Org mode third-party integration :: an afterthought following Karl Voit's Orgdown proposal

2021-12-08 Thread Tim Cross


"Dr. Arne Babenhauserheide"  writes:

> [[PGP Signed Part:Undecided]]
>
> Russell Adams  writes:
>
>> On Wed, Dec 08, 2021 at 05:16:20PM +0100, Dr. Arne Babenhauserheide wrote:
>>>
>>> Tim Cross  writes:
>>>
> To date, I only had a bigger problem once (and that hurt a lot, because
> it was just before giving a lecture, so I had to ditch most of the
> improvements I wanted to do and instead spend all the time fixing the
> document), but the talk here about “sometimes you have to break
> compatibility” goes into a direction I consider as very dangerous.
>
> Please do not make org-mode volatile.¹
>
> Org-Mode and Emacs have mostly been stable the past 15 years. And it is
> good to be stable; a strength that is highlighted much too seldomly.
>

Nobody is suggesting we make org-mode volatile. However, it expect that
there will never be breaking change is idealistic. I cannot think of a
single piece of software which hasn't at some point had some level of
breaking change. 

As I stated in my post, backwards compatibility is important and no
breaking change should be taken lightly. However, at times, it is
necessary and cannot be avoided. It might even be outside org-mode's
control - for example, a breaking change in Emacs might result in the
need for a change in org-mode or a security vulnerability might be
discovered which cannot be fixed without a breaking change.

Change is inevitable. It cannot be prevented. All we can do is try to
mitigate the impact of that change as best we can. Of course you also
have the choice to avoid such changes simply by not upgrading. While
this will mean you don't get bug fixes or enhancements, it is really the
only way to guarantee your documents are not impacted.

I think org-mode has a pretty good track record. There have been
breaking changes, but those changes have been in the main, justified and
never done lightly. They have bene documented and included in the NEWS
file and org has provided tools like rog-lint and conversion functions
to help with the transition required for such change.

Change is inevitable and sometimes, breaking change cannot be avoided.
It is a fact of life we have to deal with. As developers, we need to try
and ensure the impact from change is as minimal as possible and when it
is inevitable, we implement the change in a planned manner which tries
to reduce that impact (communication, conversion facilities and
conversion functions, stage implementation, deprecation periods etc).

What really doesn't help is to immediately jump to extremes and start
talking about making something volatile just because change is
mentioned.



Re: Concrete suggestions to improve Org mode third-party integration :: an afterthought following Karl Voit's Orgdown proposal

2021-12-08 Thread Tim Cross


Ihor Radchenko  writes:

> Tim Cross  writes:
>
>> Meanwhile, Emacs development continues and new features/capabilities
>> continue to be added. In particular, a new feature is added which is
>> extremely powerful and would be a huge benefit for Emacs org-mode users.
>> However, there is a problem. In order to take advantage of this new
>> feature, significant changes are required for the specification. This
>> will result in implementations requiring considerable work in order to
>> update them to the new specification.
>
> I disagree. We already need to care about back-compatibility of Org
> syntax (think of org documents written years ago). Major changes to
> syntax are very unlikely even without considering third-party software.
> And, by the way, remember the existing "third party" Elisp packages
> (think of Org roam, for example). We do not want to break them.
>

Backwards compatibility is important and changes should never be done
lightly. However, that doesn't mean they don't occur (we have already
had breaking changes, so old org files are likely to have issues
already). Backwards compatibility can also become a burden and
sometimes, needs to be sacrificed to maintain the viability or
maintainability of the system. So while I totally agree we should work
very hard not to break compatibility or adversely affect other projects
which are built on top of org mode, like org-roam, we also don't want to
find ourselves in a position where we cannot improve/enhance org mode
because of the impact it has on other projects. The priority should
always be org-mode as an Emacs mode. When there is a need for a breaking
change, that needs to be managed in a way which provides other
dependent libraries and projects a reasonable time to adapt.

Having thought about this whole thread and other recent posts, I still
feel any concern or reference to third party libraries etc is misguided
or at the least, irrelevant. Most of the suggestions are fine and would
be beneficial to org mode (such as clearly defined, consistent and
documented syntax). The fact 3rd party libraries would also benefit from
this is a bonus, but largely irrelevant. I'm not convinced that the
perceived lack of such documentation or specification is actually the
impediment to a 3rd party org mode. I think the real problem and the
real reason you are unlikely to get a version of org-mode which is
popular for non-Emacs users (and would facilitate collaboration with
non-Emacs users) is because what makes org-mode so great has little to
do with the markup. The org-mode markup is no better or worse than other
'markdown' dialects. What makes org-mode such a great system is
intrinsically interwoven with Emacs and the facilities Emacs provides.
The amount of work which would be required in another editor to get even
close to the experience and benefits of org mode is simply too high. The
best you can hope for is some baic rendering and syntax
highlighting/font-locking, which is unlikely to be sufficient to make
people switch from the existing markdown they already use.

I think a far more likely scenario is that we will see some/many of the
ideas found in org-mode adapted and implemented in other editors, but
without concern for compatibility. This has little to do with Emacs
org-mode's documentation or org-modes specification, but rather is about
how the ideas which are encapsulated in org-mode can be implemented in
other systems and within the limitations of those systems. I'm actually
surprised there hasn't been more org-mode clones already, but that could
be a reflection of the amount of work it would take to create something
which wasn't just another markdown module that renders a reasonable
HTML/PDF version of it's contents. .



Re: From macros to elisp programming?

2021-12-06 Thread Tim Cross


Ypo  writes:

> Hi
>
> I am able to make macros, but I think I am far away from Lisp programming.
>
> Is there a path to go from macros to elisp programming? For example, the last 
> macro I've made is for transforming the name of some
> headlines, adding in front of them a part from the previous headline. This is 
> the elisp code of the macro:
>
> #+BEGIN_SRC
> (fset 'SanzTema5
>(kmacro-lambda-form [?\C-a ?\M-f ?\M-b ?\C-  ?\M-f ?\M-f ?\M-f ?\M-f ?\M-f 
> ?\M-f ?\M-f ?\M-f ?\M-f ?\M-w ?\C-c ?\C-n ?\C-a ?\M-f ?\M-b ?\C-y
> ?  ?- ?  ?\C-e ?\M-b ?\M-f ?\"] 0 "%d"))
> #+END_SRC
>
> Using that code, from these headlines:
>
> *** Sanz Aparicio et al. (2019) "5 Los Motivos Adquiridos, Menéndez Balaña" 
> (pp. 95-118)
>  INTRODUCCIÓN
>
> I get modified the second headline:
>
> *** Sanz Aparicio et al. (2019) "5 Los Motivos Adquiridos, Menéndez Balaña" 
> (pp. 95-118)
>  Sanz Aparicio et al. (2019) "5 Los Motivos Adquiridos - INTRODUCCIÓN"
>
> Are macros near to elisp programming or they are two different worlds?
>

Unfortunately, the term 'macro' is somewhat overloaded and can mean
different things depending on the language. For example, C has
pre-processor 'macros' which are essentially simple string replacement
mechanisms and very different from macros as used in lisp languages like
elisp, common lisp or clojure. In a similar vain, what you are referring
to above is a keyboard macro, which is not an elisp macro. A keyboard
macro is essentially a simple 'replay' facility where you can capture a
set of keyboard input actions, give them a name and then repeat those
actions by running the macro with that name again. This is very
different from an elisp macro. An elisp macro (or a common lisp macro,
or a clojure macro etc) is a facility which allows you to manipulate
source code and can be very useful in defining new low level language
constructs which are largely indistinguishable from the built-in
constructs of the language. For example, if elisp did not have a 'when'
statement, you could define it as a macro and it would operate at the
same level as other built in functions. For this reason, macros are
often used a lot when defining a 'domain specific language' (DSL-)  a type of
language dialect which has operators specific to a specialised domain. 

While others would likely disagree, I would not consider using keyboard
macros as programming in emacs lisp. Keyboard macros are an end user
facility which can be used to automate repetitive tasks and which can be
used by anyone using Emacs - you don't need to know any emacs lisp to
create and use a keyboard macro. 

Macros as used in lisp languages are a wonderful, extremely powerful
facility that allow the programmer to do things which are near
impossible in other languages. However, they should be used very
judiciously. There is an old adage which states that if you can do it
with a function, then use a function, not a macro. A common error made
by those new to macros is that they are so amazed at their power, they
now think it is the solution to all problems.

It is difficult to recommend resources as a lot depends on your own
experience, what level you want to dive into and what you want to be
able to do. There is a big difference in knowing enough emacs lisp to
tweak your Emacs configuration and knowing enough emacs lisp to write
the next great package. The good news is that there are some great free
resources out there, so the main investment will be in time.

If you have not yet read it, a good starting point is Mastering Emacs by
Mickey Petersen  https://www.masteringemacs.org/

There is also an Introduction to Emacs Lisp, which is part of the GNU
Emacs distribution.

For getting to understand lisp programming at a much deeper level, there
are some excellent resources out there, many of which are free. The
trick is to not limit yourself to searching just for Emacs lisp. Emacs
lisp is a dialect of lisp - a domain specific language for an editor if
you like. Most of the concepts associated with lisp are applicable to
emacs lisp. almost any resource teaching lisp will teach you things
applicable to emacs lisp. Therefore, if you really want to learn how to
program in emacs lisp, search for resources on learning to program in
lisp.

There is a somewhat famous book called 'The Structure and Interpretation
of Computer Programs" (SCIP), which is based on a course in computer
science taught at MIT. It is an excellent book. Even better, videos from
the course are also available on-line. While the focus is on
programming, it uses a lisp dialect to demonstrate the core principals,
which it makes very clear. A must watch for anyone really keen to
understand programming.

Another great book about lisp which is also freely available is Paul
Graham's 'On Lisp', which I think gives a pretty good overview of the
power and flexibility of lisp and functional programming. 

A book which I really like and which I think really shows off the power
of lisp macros is 

Re: Concrete suggestions to improve Org mode third-party integration :: an afterthought following Karl Voit's Orgdown proposal

2021-12-05 Thread Tim Cross


Ihor Radchenko  writes:

> Tim Cross  writes:
>
>> I think your working off a false premise. Your view is that org mode
>> should be available in other editors/software so that others can realise
>> the power and benefits it provides. I can understand that position.
>
> A clarification: my premise is that org mode should be available in
> other _free_ editors/software. If we provide the means for other free
> software to support Org mode (either as markup format or as some subset
> of Elist implementation), it will benefit software freedom in general.
> Whatever helper information (think of tests) we provide, it should be
> licensed under GPLv3 with the following effect:
>

I don't disagree with this objective. My objection is to changing the
emphasis or priority of org mode as an Emacs mode to a general technical
specification for a small part of what is org-mode, the markup (I will
outline the concerns I have in doing this below). The other point of
course is that if you make it easier to implement org markdown in other
editors, you don't have control over whether they are implemented in
free or non-free solutions. 

> https://www.gnu.org/licenses/quick-guide-gplv3.html
>>> It's always possible to use GPLed code to write software that
>>> implements DRM. However, if someone does that with code protected by
>>> GPLv3, section 3 says that the system will not count as an effective
>>> technological "protection" measure. This means that if you break the
>>> DRM, you'll be free to distribute your own software that does that,
>>> and you won't be threatened by the DMCA or similar laws.
>
> The fact is that e.g. Github already provides support for Org markup.
> They do it for their own profit and we cannot stop them. If we have a
> controlled criteria about quality of third-party Org mode support, there
> will be means to interfere with non-free software attempting to makes
> profits out of Org mode. For example, if Github do not integrate our
> recommended test suite (with all the legal consequences defined in
> GPLv3), we will be able to have a list of third-party tools and, among
> free alternatives, mention that Github support for Org is not verified
> and most likely not consistent with other _free_ tools. We cannot do it
> now.
>

There is a fatal flaw in this argument. The GPL licenses code, not the
underlying idea (which is essentially what patents are about). We can
define all the quality criteria we like for 3rd party implementations,
but we cannot enforce them unless they are also using the GPL'd code. As
this code is elisp, it is extremely unlikely any 3rd party
implementation will be using it. In short, defining clear specifications
and minimal quality criteria will only have baring on code added to org
mode - it would, for example, have no effect on what github has/is
implementing. 

>> However, the FSF position would be exactly the opposite. They would
>> argue that orgmode is a powerful and flexible tool that is part of Emacs
>> and if you want that power and flexibility, you need to use Emacs. Org
>> mode has probably done more to bring new users to Emacs than any other
>> Emacs mode in the last 30 years. As a consequence, you will find not
>> only little support towards making it available in other editors, you
>> are likely to run into active resistance. As you say, org-mode can be
>> thought of as a brand name and that is a brand name owned by the FSF as
>> an official GNU project and a goal of the FSF is to convert people to
>> use GNU free software. Anything which has the potential to take the
>> power of org mode and make it available in non-free software (not simply
>> open source) is not going to be supported or welcomed.
>
> I am very much sceptical that third-party tools can provide the level of
> Org support Emacs does provide. Emacs is and will remain the most
> feature-full tool for people to use Org mode. Org mode's largest power
> is configurability thanks to Emacs. On the other hand, Org mode's
> support would be welcome in tools like TeXMacs or in forges like
> Sourcehut (currently only supporting markdown).
>

I don't disagree about the benefit of org markup being supported in 3rd
party tools. I disagree with the proposal to change the emphasis of the
org-mode project from being an Emacs mode to being a more general
technology.

Consider this contrived scenario.

We have adopted a change in emphasis to promote org mode as a more
general solution and clarified the specification to make it easier for
3rd parties to implement.

Meanwhile, Emacs development continues and new features/capabilities
continue to be added. In particular, a new feature is added which is
extremely powerful and would be a huge benefit for Emacs org-mode users

Re: Concrete suggestions to improve Org mode third-party integration :: an afterthought following Karl Voit's Orgdown proposal

2021-12-05 Thread Tim Cross


"Bruce D'Arcus"  writes:

> On Sun, Dec 5, 2021 at 8:42 AM Tim Cross  wrote:
>
>> I think your working off a false premise. Your view is that org mode
>> should be available in other editors/software so that others can realise
>> the power and benefits it provides. I can understand that position.
>>
>> However, the FSF position would be exactly the opposite. They would
>> argue that orgmode is a powerful and flexible tool that is part of Emacs
>> and if you want that power and flexibility, you need to use Emacs.
>
> Perhaps, but Emacs org users benefit from better support for org
> documents outside of Emacs.
>

That may be true, but it is secondary to the main goal of the FSF. It
isn't about benefit or convenience. It is about freedom. My guess is
that the FSF position is likely to be something along the lines that
while supporting/encouraging org mode in other, possibly non-free,
software might be beneficial to Emacs users, it not only fails to
promote the objectives of the FSF, it may actually work against it by
decreasing the incentives to move away from a closed non-free solution
to an open free one. 



Re: Concrete suggestions to improve Org mode third-party integration :: an afterthought following Karl Voit's Orgdown proposal

2021-12-05 Thread Tim Cross


Ihor Radchenko  writes:

> Juan Manuel Macías  writes:
>
>> Yes, sorry for not explaining myself well: I was also referring to
>> search results, not the title in the web site...
>>
>> But the question is: what need is there to remove the reference to Emacs
>> in the search result? I think the emphasis is necessary. As we say in
>> spanish, it's like putting the bandage on before the wound. If there are
>> people who think that Org Mode can 'live' in some way outside of Emacs
>> (a respectable opinion, but that I do not share, at least 100%), I think
>> the burden of the work falls on them and not on us, the users of
>> Emacs-Org-Mode. Anyway it is my simple and subjetive opinion, and it is
>> not my intention to initiate a controversy with it.
>
> Ok. Let me explain my thought process.
>
> First of all, there is no burden on users of Org mode in making edits to
> orgmode.org. It is a burden on Org contributors.
>
> One of the aims of my proposal is reducing this burden by involving
> non-emacs users to provide contributions to Org (e.g. by making more
> tests for Org-element parser). To do it, we need to make the
> contribution process for non-emacs developers easier. Ideally, without
> too much effort on our side.
>
> The idea of involving non-emacs users does have a potential because we
> do know that third-party tools are already using Org. The problem is the
> disconnect between those tools and Org mode proper.
>
> The sources of the disconnect are (1) lack of technical "blueprints" for
> Org that do not require knowing Elisp; (2) lack of discovereability of
> Org mode as something that can live outside narrow field of Emacs. In
> this branch of our discussion, I am going to talk about the second
> point.
>
> People simply do not expect to see a markup language when they encounter
> a link with "Org mode for Emacs" title. Someone looking for Org mode
> markup to be used in, say, websites will think that "Org mode for Emacs"
> is limited to Emacs. Someone just interested in plain text markup will
> find no relevance at all.
>
> Title is important. If we care at all about orgmode.org website
> appearing in search results, we want the title and the summary to have 2
> main properties: (1) Provide search keywords to make it searchable by
> potentially interested people; (2) Provide a title that immediately
> signal that our website contains the information people are looking for.
>
> Now, we need to understand what kind of people may be looking to
> orgmode.org website.
>
> 1. Existing emacs users
>
>If a Emacs user is faced with "Org mode for Emacs", the word "Emacs"
>is indeed recognisable. On the other hand, the word "Org mode" does
>not provide much further info, except that it is a major (or maybe
>minor?) mode for "Org"??
>
>Now, consider "Org mode: your life in plain text".
>For emacs users, "Org mode" is not just a strange phrase, but a very
>clear indication that we are talking about Emacs.
>The "your life in plain text" provides extra information about what
>"Org mode" refers to. Clearly, text documents and "your life in plain
>text" should resonate with every Emacs user's soul.
>
>We can change the second variant of the title to contain "Emacs", but
>will it add much value? I am not convinced. On the other hand, making
>title too long or too complex _is_ bad. Long titles tend to be
>skipped (there was even formal research on this!)
>
> 2. Non-emacs users interested in plain text markup
>
>These users know nothing about Emacs and "Org mode" has no meaning
>for them as is. So, we do need something more descriptive.
>Adding "Emacs" may be fine, but it will serve no purpose for people
>not familiar with emacs. Just another unknown term making the title
>longer.
>
> 3. Non-emacs users interested in GTD/project management, etc
>"Org mode: your life in plain text" is somewhat relevant when people
>are looking to manage "life" (typically true for GTD enthusiasts).
>
>Though we can probably do better for this category.
>Maybe "Org mode: manage your life and notes in plain text"?
>Though it makes the title less relevant to group #2.
>
> 4. Researchers looking for ipython-like environment
>
>Not covered, except by reading my proposed site summary. I am not
>sure how we can improve title for this audience.
>
> 5. ??? (Suggestions are welcome)
>
> Of course, better suggestions for the title are welcome. I just wanted
> to make it clear the reasoning I do not like the current title and how
> the proposed alternative is better (though not ideal).
>
> Finally, I want to emphasise that our aim for search engines is not
> advertising Emacs (we already do it by trapping users inside Org and
> making them switch to Emacs by force :evil_laughter:). The aim is
> encouraging people to use and contribute to Org mode in useful ways
> (even unrelated to writing Elisp or, really, any code at all).
>
> Search 

Re: Concrete suggestions to improve Org mode third-party integration :: an afterthought following Karl Voit's Orgdown proposal

2021-12-05 Thread Tim Cross


Juan Manuel Macías  writes:

> Ihor Radchenko writes:
>
>> The website title is "Org mode for Emacs", repelling users who _do
>>not want_ to use Org inside Emacs. Maybe we can do better? Something
>>with less accent on Emacs like "Org mode: your life in plain text"
>
> I am not at all in favor of separating the 'Org Mode' name from 'GNU Emacs'.
> In any case, regardless of my opinion, I see here two problems:
>
> 1. "Org mode: your life in plain text". The word "mode" remains orphan:
> mode of what? It is clear that it is an Emacs mode, therefore it doesn't
> make much sense to hide Emacs and then suggest it.
>
> 2. One possibility: "Org: your life in plain text". But here what
> remains orphaned is "Org", too generic name. Unless "Org Mode" is a
> lexicalized construct, without reference to any emacs mode.
>
> (In any case, I would be extremely saddened if the reference to GNU Emacs
> disappears in the title, just to please a minority).
>

There is another big issue which people are not considering.

Org mode is a GNU project and it is part of Emacs. This actually has
some consequences, most notably -

- It is not acceptable for the project to explicitly or implicitly
  recommend or appear to recommend any non-free solutions or provide
  support for non-free software. Products like Github and MS Visual Code
  fall squarely in the unacceptable bucket.

- It would not be possible to reference any 3rd party libraries or
  programs which are not free software in the proposed list of external
  tools

- The suggested org mode in a browser example is unlikely to be
  acceptable to the FSF (or RMS). The FSF is very much against cloud
  based computing services or any web service which uses non-free
  Javascript (which is most of them and one of the many reasons Github
  is discouraged by the FSF).

A number of the ideas proposed are good ideas for org mode generally -
for example, a repository of reference documents which could be used for
testing purposes would be extremely useful for org-mode development and
testing. Likewise, any effort to clarify the syntax and remove any
ambiguities is beneficial for org mode itself. However, the emphasis and
priority needs to remain focused on org mode as a mode for Emacs. The
use of org mode by other external programs is really outside (but
related) to the project.

As a consequence and to eliminate/remove potential conflicts with FSF
philosophy and goals, it may be worth considering spinning off a
separate project. which happens to use the same markup syntax, but is
not a GNU project (though it would be good to still be GPL'd). 

If you want ot get a feel for the sort of issues which could come up
when trying to develop/support 3rd party tools, have a look at the
recent thread on creating an LSP server for emacs-lisp. While I
personally disagree with most of the fears raised by some contributors
to that thread and disagree with RMS's view that such a server would not
be in the best interests of Emacs, the thread does give you a sample of
the sort of issues which could come up with efforts to support or
encourage 3rd party libraries for org markup, much of which could be
avoided if the work is clearly not part of, related to or supported by
the main org-mode project. 



Re: [PATCH] Fix org-comment-line-break-function

2021-12-04 Thread Tim Cross


Given that Nicholas cannot remember the reason for the original function
and suspects it was meant to be an internal only function, I think this
patch is probably the best way forward and should be applied.

Kaushal Modi  writes:

> On Tue, Nov 30, 2021 at 6:29 PM Tim Cross  wrote:
>
>  It would be good to get Nicholas' input here as I think he wrote the
>  original function back in 2012. 
>
> Just to see what happens, I tried this:
>
> M-: (setq-local comment-line-break-function #'comment-indent-new-line)
>
> .. and then M-j started working perfectly! It worked fine both: in Org 
> comment lines and out of comment lines.
>
> I see that comment-indent-new-line was added to emacs in newcomment.el back 
> in 2000. So I don't know why
> org-comment-line-break-function was added. May be Nicolas can comment more on 
> that.
>
> So would this patch work?
>
> =
>
> From 1a9187b82ed8d4e8d54ddd369a44d34295281fc3 Mon Sep 17 00:00:00 2001
> From: Kaushal Modi 
> Date: Tue, 30 Nov 2021 20:37:10 -0500
> Subject: [PATCH] org: Remove `org-comment-line-break-function'
>
> * lisp/org.el: Remove `org-comment-line-break-function' and let
> `comment-line-break-function' be the default value.
>
> This fixes the `M-j' binding issue reported by Richard Lawrence in
> <https://lists.gnu.org/r/emacs-orgmode/2021-11/msg00639.html>.
> ---
>  lisp/org.el | 17 ++---
>  1 file changed, 2 insertions(+), 15 deletions(-)
>
> diff --git a/lisp/org.el b/lisp/org.el
> index ec59ddf44..ee8ca1f03 100644
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -19624,8 +19624,7 @@ assumed to be significant there."
>  
>  ;; `org-auto-fill-function' takes care of auto-filling.  It calls
>  ;; `do-auto-fill' only on valid areas with `fill-prefix' shadowed with
> -;; `org-adaptive-fill-function' value.  Internally,
> -;; `org-comment-line-break-function' breaks the line.
> +;; `org-adaptive-fill-function' value.  
>  
>  ;; `org-setup-filling' installs filling and auto-filling related
>  ;; variables during `org-mode' initialization.
> @@ -19647,8 +19646,7 @@ assumed to be significant there."
>(setq-local fill-paragraph-function 'org-fill-paragraph)
>(setq-local auto-fill-inhibit-regexp nil)
>(setq-local adaptive-fill-function 'org-adaptive-fill-function)
> -  (setq-local normal-auto-fill-function 'org-auto-fill-function)
> -  (setq-local comment-line-break-function 'org-comment-line-break-function))
> +  (setq-local normal-auto-fill-function 'org-auto-fill-function))
>  
>  (defun org-fill-line-break-nobreak-p ()
>"Non-nil when a new line at point would create an Org line break."
> @@ -19903,17 +19901,6 @@ filling the current element."
>   (adaptive-fill-mode (not (equal fill-prefix ""
>   (when fill-prefix (do-auto-fill))
>  
> -(defun org-comment-line-break-function ( soft)
> -  "Break line at point and indent, continuing comment if within one.
> -The inserted newline is marked hard if variable
> -`use-hard-newlines' is true, unless optional argument SOFT is
> -non-nil."
> -  (if soft (insert-and-inherit ?\n) (newline 1))
> -  (save-excursion (forward-char -1) (delete-horizontal-space))
> -  (delete-horizontal-space)
> -  (indent-to-left-margin)
> -  (insert-before-markers-and-inherit fill-prefix))
> -
>  
>  ;;; Fixed Width Areas




Re: [PATCH] org-src.el: add option `org-src-native-defun-movements'

2021-12-04 Thread Tim Cross


Sébastien Miquel  writes:

> Hi,
>
> Thank you for having a look.
>
> Tim Cross writes:
>> This also seems like an edge case and I'm not convinced yet another
>> option is justified. Why have eilisp in org blocks rather than an
>> emacs-lisp block?
>
> By org src blocks I meant an org emacs-lisp src block. The point of
> the patch is to be able to eval-defun some lisp code in an emacs-lisp
> src block from the org-buffer.
>

OK, that makes it clearer. However, I'm not convinced this is something
we need or want. You can evaluate emacs lisp blocks and you can use
edit-special buffers to evaluate individual lines in a source block.
Being able to execute arbitrary lisp forms at a top level inside an org
buffer could be considered dangerous. I don't think it should be enabled
by default. 

>> As this is a breaking change, it should not be on by default.
> Currently eval-defun errors out, and fixing that will break things
> sooner or later, I think.
>
> I do not mind updating the patch to set the new option to nil by
> default, although I'll wait for a second opinion on this.
>

That is fine. However, note that this would mean your patch can only be
applied to the next version (development version) of org and not to the
current maintenance branch because you cannot add a breaking change to
an already released version.

I think you are making it harder to get the patch applied by enabling
the option. There is a (rightly) conservative stance on breaking
changes. Adding a new option which is enabled by default and which
breaks existing functionality has almost no chance of being applied.
Adding a new option which is a breaking change that needs to be enabled
by the user is far more likely to be considered.




Re: quotation marks in table cell vs. org-babel-ref-resolve

2021-12-04 Thread Tim Cross


Greg Minshall  writes:

> hi, Tim,
>
>> The key question is what is the use case for having this 'mixed' content
>> in a table cell?
>
> in my case, i am putting RFC822('ish) e-mail addresses in a column of an
> org-mode table.  and, i want to extract them.
> 
> | oxymo...@example.com  |
> | Greg Oxymoron   |
> | "Greg G. Oxymoron"  |
> 
> for the third row returns =Greg G. Oxymoron=, rather than my desired
> ="Greg G. Oxymoron" =.
>
> by the way, do you know the use case for the current behavior for
> strings that start with a ="=?  i couldn't find anything in the manual.
>
> i wonder if maybe the existing parameter =inhibit-lisp-eval= (which, in
> the path i am exercising, is non-nil) could also be used to not do the
> check for a ="=?  (maybe that's also a hack, but i think it would solve
> my problem. :)
>

I don't know. It could be related to the spreadsheet capabilities or it
could simply be an oversight in how the code extracts values from
tables. 

I tend to use the function org-table-to-list to extract the data from a
table. It gives me a nested list which I can then process with elisp in
any way I want. I don't know if that would help or how it will interpret
a cell whic contains both quoted and unquoted data.



Re: On zero width spaces and Org syntax

2021-12-04 Thread Tim Cross


Max Nikulin  writes:

> On 04/12/2021 04:48, Tim Cross wrote:
>> My vote is to simply maintain the status quo. Don't modify the syntax,
>> don't make the zero space character somewhat special or processed in any
>> special way during export. In short, accept that inner word markup has
>> only limited support and if that is a requirement which is critical to
>> your use case, accept that org mode may not be the right solution for
>> your requirements.
>
> Tim, you are skeptical concerning usage of Org markup outside of Emacs. Though
> some subscribers of this list support such idea with hope for collaboration 
> with
> colleagues and for other reasons. Status quo in respect to similar questions
> increases risk that other tools will adapt different workarounds and
> incompatible dialects will appear.

This is a misrepresentation of my position. I've never stated I'm
sceptical or org markup outside of Emacs. I'm sceptical of org mode
outside of Emacs, but have never expressed an opinion of org markup
outside of Emacs.  

However, I will now

Org markup outside Emacs is very much a secondary concern that would be
a nice to have for some workflows, but should be achieved with zero
impact on Emacs users. Org mode and the markup it uses is primarily an
Emacs mode. In fact, making it easier for non-Emacs users to use org
mode is almost certainly working against the FSF philosophy. I'm pretty
certain RMS would be very unhappy of any efforts to allow users to use
org mode in products like MS Visual Code. While it is fine for 3rd party
systems to try and mimic org mode, it is totally contrary to GNU
philosophy for a GNU project to actively support or enable such
functionality in non-free solutions. Any decisions to make changes to
org mode must be primarily for the benefit of Emacs users. When such
decisions also have benefit for non-Emacs users, that is great, but it
should not be a driving factor in making decisions regarding change or
extensions to org mode.

>
> From the point of view of popularizing Org it is better to make some decision:
> either zero-width space should become a part of syntax or some other printable
> marker should be chosen to suppress effect of Org markup or vice versa to
> activate some construct.

Chasing popularity is always a mistake and should never be used as an
argument for change. We are also talking about something where there is
little evidence of demand. We have a single post from someone asking how
to support inner word emphasis and suddenly, threads about modifying
syntax, modifying back ends and a dozen proposals on how to support this
'feature'.

A question I would ask is that if extending and adding broader support
for emphasis is so straight-forward, why do we already have so many
issues reported about incorrect application of markup? We have not been
successful in eliminating existing ambiguities with the markup and yet
some would have us charge off and add even more complexity.

Rather than extending markup syntax, lets focus on fixing the real issues we
already have. There have been far more posts to this list about that
than about inner word emphasis. For example, the many posts about markup
and links. 

With respect to the status of zero width space, I'm not convinced we
need to do anything. Would it be classified as a kludge, probably. Does
it provide an escape hatch for some situations, yes. Does that mean it
needs to be formally recognised and added to the syntax, no. Does the
existence of this kludge make implementation of org mode markup for
other tools more difficult or less clear, probably. Should that be a
primary concern for Emacs org-mode, no. Should it be something we
consider when making decisions, sure, but only as a secondary
consideration. 

What the need for the zero width space kludge really means is that in
some situations, we have some ambiguity in the existing syntax. Can we
fix those ambiguities? I don't know - so far, I've not seen a proposal
which doesn't introduce as many problems as it solves, (though Tomp's @@
proposal looks interesting, but lots more analysis is required).

The zero width kludge is certainly a symptom of limitations in the
existing syntax definition. However, I don't think it is the cure and I
don't agree it needs to be formally recognised as part of the syntax -
it is not the cure. If we can find the correct cure, the zero width
kludge will not be necessary (or will only be necessary in extreme and
rare edge cases). 



Re: Org-syntax: Intra-word markup

2021-12-03 Thread Tim Cross


Tom Gillespie  writes:

> I don't mean to be a wet blanket, but the edge cases for
> the current markup syntax are already hard enough to
> implement correctly, to the point where different parts of
> Org mode are inconsistent. Intra-word markup isn't viable
> because there simply isn't any sane way to parse something
> like *hello world*/hrm/oh no*. The other issue is that this will
> degrade parsing performance because almost every
> character could precede the start of a markup section.
>
> I recommend anyone suggesting solutions try to implement
> something that can parse the markup unambiguously with
> lots of nasty test cases. You will likely find that it is impossible
> to consistently tokenize markup, and that you have to hand
> write a whole bunch of heuristics, making Org syntax even
> harder to implement correctly.
>
> Any solution that suggests extending how =/*~+_  can be
> used gets a hard no from me. I could see teaching other
> exporters how to interpret \emph{hello}world, but trying for
> to have any sane behavior for something like
> why *hello*world oh no a wild askterisk*
> is not worth it.
>

+infinity!

Please, please can we stop trying to satisfy every edge case or extend
the markup to satisfy every possible scenario.

Org's big strength is in its simplicity. This comes at a price -
limitations in what can be done. If those limitations are unacceptable,
then use a richer markup format like Latex, XML, HTML etc.

The point about back end exporter support is very relevant. The 'richer'
the markup, the harder it is to get a consistent mapping for back end
exporters. things quickly become more complex and difficult to maintain.

In 18 years, I've seen requests for inner word markup less than 4 times.
this is not a feature we should even be considering adding to the markup
syntax. 

Org provides a light weight markup, not a fully flexible rich markup
designed to meet any need. It makes the easy stuff simple. 



Re: quotation marks in table cell vs. org-babel-ref-resolve

2021-12-03 Thread Tim Cross


Greg Minshall  writes:

> fwiw, tracing, the problem appears to be this line
> 
> ((eq (string-to-char cell) ?\") (read cell))
> 
> in =org-babel-read=.
>
> presumably there are many cases where this is the right thing to do.
>
> but, maybe look for a simple =^"[^"]*"$= (i.e., a quotation mark, some
> other stuff, a quotation mark, and *nothing* else in the cell)?
>
> or (heaven help us), something more complicated, with random escaped
> quotation marks inside the "cell"?
>
> cheers, Greg

The key question is what is the use case for having this 'mixed' content
in a table cell?

I really don't like the idea of adding even more regexp processing of
contents in order to resolve complexities in data formats which are
rare.  the benefit of the line you identified is that it is simple and
easy to maintain. There may be edge cases where it does not work
correctly, but perhaps the right solution in those situations is fix the
format of the data in the table rather than complicate the functions
used to extract the data from a table.



Re: [PATCH] org-src.el: add option `org-src-native-defun-movements'

2021-12-03 Thread Tim Cross


Sébastien Miquel  writes:

> Hi,
>
> The attached patch adds a new option ~org-src-native-defun-movements~
> that makes ~beginning-of-defun~, ~end-of-defun~ and ~eval-defun~ work
> natively when called from inside an org src block : those functions
> are called from an org src edit buffer, in the appropriate language
> mode. Without this patch, calling =eval-defun= on elisp code fails.
>
> With this option set to t by default, this is a breaking change. To
> get to the beginning/end of a src block you'd have to call
> ~org-backward-element~ or ~org-forward-element~ directly, instead of
> ~beginning-of-defun~. Or you could disable the new behaviour by
> setting ~org-src-native-defun-movements~ to nil.
>

As this is a breaking change, it should not be on by default.

This also seems like an edge case and I'm not convinced yet another
option is justified. Why have eilisp in org blocks rather than an
emacs-lisp block?



Re: On zero width spaces and Org syntax

2021-12-03 Thread Tim Cross


Juan Manuel Macías  writes:

> Hi all,
>
> It is usually recommended, as you know, to insert a zero width space
> character (Unicode U+200B) as a sort of delimiter mark to solve the
> scenarios of emphasis within a word (for example, =/meta/literature=)
> and others contexts where emphasis marks are not recognized (for example
> =[/literature/]=). I believe that as a puntual workaround it is not bad;
> however, I find it problematic that this character is part, more or less
> de facto, of the Org syntax. For two main reasons:
>
> 1. It is an invisible character, and therefore it is difficult to
> control and manage. I think it is not good practice to introduce this
> type of characters implicitly in a plain text document.
>
> 2. It is more natural that this type of space characters are part of the
> 'output' and not of the 'input'. In the input it is better to introduce
> them not implicitly but through their representation. For example, in
> LaTeX (with LuaTeX) using the command '\char"200B{}' (or '200b'),
> '' in HTML, etc.
>
> In any case, as an implicit character, I do not see it appropriate for
> the syntax of a markup language. The marks should be simply ascii
> characters, IMHO. So what if Org had a specific delimiter mark for the
> scenarios described above? For example, something like that:
>
> #+begin_example
>
> /meta/''literature
>
> *meta*''literature
>
> [''*literature*'']
>
> #+end_example
>
> WDYT?
>
> Best regards,
>
> Juan Manuel 

I think I am in agreement regarding most of your points about the use of
the zero-width character. I see it as a type of escape hatch which
provides a solution in some less frequent situations. It is a somewhat
clever kludge to enable markup in some situations not supported by the
basic markup syntax I'm happy with its status as a kludge and would not
want to see it become an official part of the syntax. Where we may
differ is in whether we actually want to add inner word markup support
at all. 

I'm somewhat surprised and more than a little concerned at how much
interest and focus on modifying the markup syntax of org the question of
inner word markup has generated. This seems to be a symptom of a more
general trend towards adding and extending org mode to meet the needs of
everyone and I'm concerned this is overlooking the key strength of org
mode - simplicity.

Consider how many times we have had requests for inner word markup in
the last 18 years. I've seen such requests only a very few times.
Certainly not frequently enough to consider modification of the markup
syntax to accommodate such a requirement.

A key philosophy of org mode is simplicity - it makes the easy stuff
simple and the hard stuff possible. The thing about simple solutions is
that they will inevitably have limitations. If you don't want those
limitations, then you use a more complex feature rich markup, such as
Latex, HTML, XML etc. Ideally, your system will provide some escape
hatches to allow you to do things not supported by the base markup
syntax. Those escape hatches will usually be less convenient and often
look quite ugly, but that is fine because they are an escape hatch
which is used infrequently. Better still is if the system provides some
way to make a specific escape hatch easier to use in a document (such as
via a macro). The basic org markup syntax has worked remarkably well for
18 years. Nearly all the proposed additions or alterations to support
inner word markup with complicate the syntax or introduce potential new
ambiguities and/or complexity in processing to support a feature which
has been rarely asked for and which has other, less convenient and often
ugly, solutions which work.

One of org's strengths has been the ability to export documents to
multiple formats. One way this has been made possible is by keeping the
markup syntax simple - a basic markup which is well supported by all
export back ends. Once you start adding more complex markup support, you
see a blow out of complexity in the export back ends. Worse yet, you get
results which are surprising to the end user or which simply don't work
correctly with some formats. to avoid this, it is critical to keep the
markup syntax as simple and straight-forward as possible, even if that
means some limitations on what can be done with the markup. 

My vote is to simply maintain the status quo. Don't modify the syntax,
don't make the zero space character somewhat special or processed in any
special way during export. In short, accept that inner word markup has
only limited support and if that is a requirement which is critical to
your use case, accept that org mode may not be the right solution for
your requirements. 



Re: Orgdown: negative feedback & attempt of a root-cause analysis

2021-12-01 Thread Tim Cross


Karl Voit  writes:

> Hi,
>
> I've summarized my current state of mind about the whole Orgdown
> fiasco into a blog article:
> https://karl-voit.at/2021/12/02/Orgdown-feedback/
>
> Don't worry, I tried to analyze my own faults as well so that others
> might be able to learn from this unfortunate situation.


Hi Karl,

thanks for writing up your experiences. I'm not surprised about the
reaction you got from reddit. I gave up on reddit some time back due to
the toxic nature of too many threads. I don't know why it is so often
toxic, but it really isn't worth the hassle.

Starting up a project is difficult. I have an open source JS library
which has turned out to be far more popular than I expected (averages
over 150k downloads each week). I'm not sure I was ready for the
commitment maintaining such a project involves - especially the long
term nature of it. At times, I ahve had problems with rather 'entitled'
users who demand ridiculous things from a free bit of software and who
can become extremely rude and somewhat nasty when I don't do what they
want. I've learnt to just ignore them.

The best advice I can give is to suggest you just put the whole thing on
the back burner for a month or so and then come back to it. During that
time, other comments are bound to come through and I find the later
comments are often far more considered and less emotional than initial
responses. Stepping back gives the subconscious part of your brain time
to process everything and will likely provide additional clarity once it
has had time to percolate. 



Re: frustrations

2021-12-01 Thread Tim Cross


Jan Ulrich Hasecke  writes:

> Hi all,
>
> after happily using straight for quite a while I am currently frustrated by a
> couple of problems.
>
> In a different thread I asked about exporting citations, which does not
> work for me at all.
>
> Today I discovered that pamparam does not work anymore with an error
> message mentioned here:
> https://github.com/nobiot/org-transclusion/issues/105
>
> Even to get to this error, as pamparam didn't start at all, I had to
> manually fix something in worf.el
> https://github.com/leotaku/worf/commit/38e901d3888e3a245a5cba14a061bffa1c5fd20b
>
> If I understand correctly, straight uses the bleeding of packages from
> github. Maybe this is not what I want. Maybe in the past I just was
> lucky not to hit a bug or an incompatible change. 
>
> There are some more issues. Startup time of my emacs is more than 30
> seconds even after optimizing something with esup. I have 10.000+ files
> in my org-roam and fear that I hit some limitation either of org-roam or
> my hardware.
>
> How do you configure your emacs using current versions like org 9.5 but
> at the same time avoiding problems with incompatible packages or newly
> introduced bugs?
>
> In a few days I'll get a new computer and I have serious doubt whether
> to copy my settings.org to the new one, because there are too many
> problems in the last couple of weeks.
>

As mentioned by others, this is really a general Emacs and software
maintenance issue. However, I'll add a couple of points to what has
already been said (most of which is good advice).

- Don't use straight. I've never used it primarily because installing
  directly from git is high risk. There is no guarantee that a git
  repository is stable or if the head of a branch is fully tested etc.
  Instead, stick with package.el and in particular, the GNU ELPA and
  NONGNU repositories.

  One reason for this is that when an updated package gets into one of
  the repositories, it does so because the maintainer has released a new
  version. This is not necessarily the case when you install directly
  from a git repository (either using straight or git directly).

- Often it is useful to steal (borrow) ideas from others. In particular,
  some of the curated 'canned' emacs setups, like spacemacs and doom
  implement a package rollback facility. You can upgrade packages and if
  you find they break your workflow in some way, you can rollback to a
  previous version.

  You may not want to use one of these curated configurations. However,
  this doesn't stop you from taking the ideas (and in most cases, even
  the code as it is typically GPL'd). These days, I use spacemacs and
  one reason is because I'm not interested in having to constantly tweak
  and review my packages and additional libraries. I now just leave that
  to spacemacs and enjoy the time it saves me.


With updates and stability, I think there are two approaches

- Update regularly - at least once a week if not more often. Doing this
  tends to result in only small changes and when things do break, it
  usually isn't too hard to fix. However, you are more likely to have
  small breakage more often. I tend to do my updates on Sunday so taht I
  ahve time to fix any problems before work on Monday.

- Only update when you encounter problems and need recent bug fixes or
  when a package you use has released a new version which has features
  you want. This results in far fewer instances of things breaking.
  However, the downside is that when you do upgrade, lots will likely
  change and tracking down the cause of problems can be more difficult.
  This approach often requires more planning and takes longer, but you
  do it less frequently. This I would do during a week off or over a
  holiday period etc.

One trick which I've used in the past is to

- Ensure all my library and package dependencies are located under
  .emacs.d
- Use something like use-package to manage installation of additional
  packages (via the :enable stanza).

Before doing an update/upgrade, I make a copy of .emacs.d called
emacs.d-mmdd (i.e. emacs.d with the date appended to the name). I
then do the update/upgrade. If things a broken, I simply delete .emacs.d
and then move the copy back to .emacs.d and I'm back to where I was
before the upgrade. It is somewhat crude, but simple and reliable. In
reality, I've put lots of my configuration files under git, so I can
easily restore a configuration on any system.

With regards to startup time, the key thing is to ensure you don't load
things until they are needed. Emacs has built in support for this with
the 'autoload' facility (get more details from the manual).
Alternatively, you can use something like use-package, which has the
:deferred stanza that will defer loading a package until a specific
command is executed (pretty sure it just uses the autoload facility
under the hood). I load approx 500 additional Emacs packages in my setup
and my load time is only a cople of seconds.


Re: [PATCH] Fix org-comment-line-break-function

2021-11-30 Thread Tim Cross


Kaushal Modi  writes:

> On Tue, Nov 30, 2021 at 6:29 PM Tim Cross  wrote:
>
>  It would be good to get Nicholas' input here as I think he wrote the
>  original function back in 2012. 
>
> Just to see what happens, I tried this:
>
> M-: (setq-local comment-line-break-function #'comment-indent-new-line)
>
> .. and then M-j started working perfectly! It worked fine both: in Org 
> comment lines and out of comment lines.
>
> I see that comment-indent-new-line was added to emacs in newcomment.el back 
> in 2000. So I don't know why
> org-comment-line-break-function was added. May be Nicolas can comment more on 
> that.
>
> So would this patch work?
>

Well, that is the big question - why was org-comment-line-break-function
added instead of just using the default comment-indent-new-line? My
only thoughts are there must be some subtle difference in org which the
default function was not sophisticated enough to work with. Problem is,
not knowing what that might be makes it hard to test and verify the real
impact of making the change.

Until this is determined, I think the only 'safe' approach would be to
just advise those who are impacted by the M-j issue to set
comment-line-break-function to comment-indent-new-line and then wait to
see if someone who has more historical context to comment. If nothing or
nobody says anything after a couple of months, then maybe apply your
patch.



Re: Orgdown: negative feedback & attempt of a root-cause analysis (was: "Orgdown", the new name for the syntax of Org-mode)

2021-11-30 Thread Tim Cross


Tom Gillespie  writes:

> Karl,
>The exact naming of a thing is nearly always the most contentious
> step in trying to promulgate it. In my own field we can easily get all
> parties to agree on a definition, but they refuse to budge on a name.
> As others have said, I wouldn't worry about kibitizing over the name.
>
> I would however worry about the larger negative reaction. From my
> perspective I think the issue is that there are many efforts working
> toward a formalized specification for Org syntax and Org mode
> functionality, and some of those stakeholders who have invested
> significant effort may feel blindsided by a public declaration
> announcing Orgdown because they were not consulted and not
> made aware that you were working on it.
>
> I appreciate the amount of work that you have put in, I have devoted
> hundreds of hours to working on an alternate implementation of org
> in Racket that uses a formal ebfn in hopes that others will be able
> to use it as a guide and as a way to talk formally about how Org
> parsers and implementations should behave.
>
> It would thus be easy for me to say that your approach has put the
> cart before the horse, because there are countless nuances in the
> specification for Org syntax which must be addressed before any
> levels of org compliance can be specified, otherwise the behavior
> between levels will be inconsistent.
>
> If I were to say this, it would not be fair to you at all. The ideas
> and motivation for Orgdown are vital and important. You have put
> in enormous thought and effort, all because you care about Org
> and want to see it succeed.
>
> The issue is that any shared specification for Org syntax is
> fundamentally about how to coordinate as a community.
> The way that Orgdown was presented to the community feels
> (to me) like it is being imposed top down or coming from an
> individual source, not from an open and visible community
> process (the subject of your original email reads as a declaration
> in english, and thus can be quite off putting, though I know that
> was not the intention).
>
> I personally haven't bothered with promulgation because I think
> that we are not technically ready as a community to approach
> outreach to other developers in a way that we can succeed.
>
> The good news is that all of this can co-exist if we want it to,
> but we need to be clear about our objectives as a community.
>
> To me these objectives are as follows (and I would love
> to hear from others about additional or alternate objectives).
>
> 1. To never fracture Org syntax so as to avoid the nightmare
> of markdown flavors. (This means being able to say clearly
> as a community that a parser is out of compliance and that
> it is up to the user to fix their files. The ruby org parser used
> by Github is a major issue here.)
> 2. To provide a clear specification for what graceful degradation
> looks like when parsing Org syntax if a parser does not support
> some portion of that syntax (e.g. should property drawer lines
> be excluded or rendered as plain text?).
> 3. Provide a solid basis on which further formal specification
> can be built. (My interests in particular are around providing
> consistent semantics for org-babel blocks across languages
> so that babel implementations can clearly communicate what
> runtime features they support.)
>
> The approach for Orgdown can absolutely meet all three of
> these objectives, however in its current form Orgdown1 is not
> sufficiently well specified to avoid fracturing the syntax.
> This is because Org syntax is extremely complex (even the
> elisp implementation of Org mode is internally inconsistent)
> and there are edge cases where behavior will diverge if parsing
> of even the simplest elements is not fully specified.
>
> There are many ways to remedy this, however they require
> a more formal approach. A number of us are working to build
> technical foundations for such a formal approach, but I do not
> think that any of those projects are ready to be used to
> specify discrete levels of Org syntax parsing compliance.
>
> If I may, I would suggest that an Orgdown0 is something that
> could be well specified, but it would avoid parsing of markup
> altogether and only deal with the major element types. Parsing
> paragraphs and all the org objects is not something that can
> be done piecemeal. There are too many interactions between
> different parts of the syntax, and in some cases the existing
> specification desperately needs to be revisited due to the
> complexity that it induces or because it is underspecified.
> Of course this would make Orgdown0 fairly useless as a
> replacement for markdown, but at least it would be a start.
>

Hi Tom,

I pretty much agree with everything you wrote. I also feel it is
unfortunate that Karl received so much negative focus on the name and
not on the underlying idea - but I'm not surprised. As you say, naming
is extremely hard and getting consensus is even 

Re: [PATCH] Fix org-comment-line-break-function

2021-11-30 Thread Tim Cross


Kaushal Modi  writes:

> On Tue, Nov 30, 2021 at 3:20 PM Marco Wahl  wrote:
>
>  diff --git a/lisp/org.el b/lisp/org.el
>  index 1a1375461..fdeec0d67 100644
>  --- a/lisp/org.el
>  +++ b/lisp/org.el
>  @@ -19695,7 +19695,8 @@ non-nil."
> (save-excursion (forward-char -1) (delete-horizontal-space))
> (delete-horizontal-space)
> (indent-to-left-margin)
>  -  (insert-before-markers-and-inherit fill-prefix))
>  +  (when fill-prefix
>  +(insert-before-markers-and-inherit fill-prefix)))
>
>  I don't have anything better.  I think this is a good patch.  It makes
>  M-j work again.
>
>  Possible refinements and improvements can follow.
>
>  +1 for applying of your patch.
>
> I am able to reproduce that M-j issue (using Emacs version: GNU Emacs 28.0.60 
> (build 9, x86_64-pc-linux-gnu, GTK+ Version 3.22.30,
> cairo version 1.15.12)
>  of 2021-11-29, built using commit c4daff9cf844ec85930bdcd2064787c92c260861, 
> and Org mode version 9.5
> (release_9.5-292-g5e35de)).
>
> And this patch fixes that for me as well.
>
> +1 for applying this patch.
>
> =
>
> Before this patch, M-j gave this backtrace with debug enabled:
>
> Debugger entered--Lisp error: (wrong-type-argument char-or-string-p nil)
>   insert-before-markers-and-inherit(nil)
>   org-comment-line-break-function(nil)
>   default-indent-new-line(nil t)
>   funcall-interactively(default-indent-new-line nil t)
>   call-interactively(default-indent-new-line nil nil)
>   command-execute(default-indent-new-line)
>
> Output of C-h k M-j:
>
> M-j runs the command default-indent-new-line (found in global-map),
> which is an interactive compiled Lisp function in ‘simple.el’.
>
> It is bound to C-M-j, M-j.
>
> (default-indent-new-line  SOFT FORCE)
>
> Break line at point and indent.
> If a comment syntax is defined, call ‘comment-line-break-function’.
>
> The inserted newline is marked hard if variable ‘use-hard-newlines’ is true,
> unless optional argument SOFT is non-nil.

I'm not sure this is the right patch to apply. While it does fix the
immediate error, it really does so by just avoiding the call to
insert-before-markers-and-inherit when fill-prefix is nil. It does not
address the question of what that function is supposed to do or whether
the correct fix is either to call the function without the fill-prefix
argument (which also works in that it avoids the error) or if instead,
the patch should be

(if fill-prefix
  (insert-before-markers-and-inherit fill-prefix)
 (insert-before-markers-and-inherit))

I note also that with or without the patch, the function does not appear
to work correctly anyway. If you hit M-j while in a comment, the new
line should be indented appropriately and have the comment character
prefix i.e. start a new comment line. It does not do that. This is
supposed to be the key difference between C-j and M-j.

Regardless, I think that unless we understand the purpose of
insert-before-markers-and-inherit, we should make the patch such that it
still calls that function. Even if fill-prefix is nil there is
probably a good reason why the markers and properties need to be
modified for some situations. 

It would be good to get Nicholas' input here as I think he wrote the
original function back in 2012. 



Re: [PATCH] Fix org-comment-line-break-function

2021-11-30 Thread Tim Cross



Marco Wahl  writes:

> Hi Richard and all,
>
> [...]
>
>> Just to be extra, super sure, I built Emacs this afternoon from a
>> checkout of the repo, and the error is *still* there, with the same
>> cause. In that build, with emacs -Q, I have:
>>
>> (org-version)
>> "9.5"
>>
>> (emacs-version)
>> "GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.5, cairo 
>> version 1.16.0)
>>  of 2021-11-30"
>>
>> At this point I've replicated the bug on my machine in four different
>> builds of Emacs (version 26.1 from Debian, 27.2 and "emacs-next" from
>> Guix, and version 29.0.50 I built myself from source) with several
>> versions of Org (the built-in ones in these Emacsen and a recent build
>> of the bugfix branch). It is robustly reproducible for me, and the cause
>> is clear: default-indent-new-line calls org-comment-line-break-function,
>> which calls
>>
>> (insert-before-markers-and-inherit nil)
>>
>> which is a type error. I'm looking for help figuring out what the right
>> fix is. I attach a patch for the simplest fix I can think of; please let
>> me know if something else would be better.
>
> diff --git a/lisp/org.el b/lisp/org.el
> index 1a1375461..fdeec0d67 100644
> --- a/lisp/org.el
> +++ b/lisp/org.el
> @@ -19695,7 +19695,8 @@ non-nil."
>(save-excursion (forward-char -1) (delete-horizontal-space))
>(delete-horizontal-space)
>(indent-to-left-margin)
> -  (insert-before-markers-and-inherit fill-prefix))
> +  (when fill-prefix
> +(insert-before-markers-and-inherit fill-prefix)))
>  
> I don't have anything better.  I think this is a good patch.  It makes
> M-j work again.
>
> Possible refinements and improvements can follow.
>
> +1 for applying of your patch.
>
>

I was finally able to reproduce the error. It depends to some degree on
the text in the buffer and where the cursor is when you hit M-j. Adding
some additional text and moving the cursor to different locations
enabled me to reproduce the error and I now agree it is a bug in
org-comment-line-break-function.

I don't know if your patch is the right fix or not because I don't
understand what the purpose of insert-before-marks-and-inherit is - in
fact, the doc string for that function doesn't even state what the @rest
args argument is for, so I don't understand why it is passing in
fill-prefix. For example, is it safe to assume
insert-before-merks-and-inherit does not need to be called if
fill-prefix is nil? Why is that function even called with the
fill-prefix as an argument? 



Re: Is M-j broken for you in Org on Emacs 27 and 28?

2021-11-29 Thread Tim Cross


Richard Lawrence  writes:

> Tim Cross  writes:
>
>> I'm running Emacs 28 and cannot reproduce the issue you observe.
>
> Hmm, the plot thickens!
>
>> Running emacs -Q I find M-j is bound to
>>
>> M-j runs the command default-indent-new-line (found in global-map),
>> which is an interactive compiled Lisp function in ‘simple.el’.
>
> I definitely see the error in emacs -Q with
>
> GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.24, cairo 
> version 1.16.0)
 o>
> which only contains Org 9.3, in my installation. So the problem has been
> around at least that long, but only surfaced for me because the binding
> of M-j changed between Emacs 26 and 27.
>
>> This binding is the same inside and outside of org mode.
>
> Yes, but inside Org mode, default-indent-new-line calls
> org-comment-line-break-function (because it is the value of
> comment-line-break-function), which is where the error originates.
>
> The last line of org-comment-line-break-function is:
>
>   (insert-before-markers-and-inherit fill-prefix)
>
> and fill-prefix is nil, at least in all the contexts where I've tried
> this.
>
> Since you're not seeing the error, could you please check in an Org
> buffer whether:
>
>   - comment-line-break-function is 'org-comment-line-break-function
>   - org-comment-line-break-function contains the line above (it should;
> it's still there in the master branch)
>   - fill-prefix is nil when you type M-j?
>   

I just checked this when running emacs -Q and get the following

comment-line-break-function is a variable defined in ‘simple.el’.

Its value is ‘org-comment-line-break-function’
Local in buffer test.org; global value is 
comment-indent-new-line

Mode-specific function that line breaks and continues a comment.
This function is called during auto-filling when a comment syntax
is defined.
The function should take a single optional argument, which is a flag
indicating whether it should use soft newlines.

  This variable may be risky if used as a file-local variable.

and fill-prefix is

fill-prefix is a variable defined in ‘simple.el’.

Its value is nil

String for filling to insert at front of new line, or nil for none.

  Automatically becomes buffer-local when set.
  This variable is safe as a file local variable if its value
  satisfies the predicate ‘string-or-null-p’.
  You can customize this variable.
  Probably introduced at or before Emacs version 1.7.

and I don't get any error with M-j and cannot reproduce the issue you
are encountering. .

Emacs is

GNU Emacs 28.0.60 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30,
cairo version 1.16.0) of 2021-11-30

and org version is

Org mode version 9.5.1 (release_9.5.1-11-g96d91b @
/usr/local/share/emacs/28.0.60/lisp/org/)

Looking at the git log, I can only find these messages relating to
default-indent-new-line

commit b41f31d2b60269bd0e7addd1081f3738f91e76bc
Author: Lars Ingebrigtsen 
Date:   Wed Aug 4 10:03:12 2021 +0200

Make `M-j' work reliably if `comment-auto-fill-only-comments' is set

* lisp/simple.el (default-indent-new-line): Force breaking the
line when called interactively (bug#49849).  (Perhaps the
interactive command should be rebound and call this function
instead...)

commit 8a11e430ec261c08cc928a7a5b05ee1027f50368
Author: Dmitry Gutov 
Date:   Thu Jun 27 16:57:47 2019 +0200

Use `default-indent-new-line' instead of `indent-new-comment-line'

* lisp/simple.el (default-indent-new-line): Doc string fix.

* lisp/textmodes/refill.el (refill-post-command-function): Make
default-indent-new-line work as indent-new-comment-line.

* lisp/textmodes/refill.el (refill-post-command-function): Bind
`M-C-j' and `M-j' to default-indent-new-line instead of
indent-new-comment-line to allow overriding via
`comment-line-break-function' (bug#12413).

commit 0b727f9d087d950c0d6614c9ec743a935d4e5fc7
Author: Richard M. Stallman 
Date:   Tue Aug 7 03:04:23 2007 +

(default-indent-new-line): New function.
It calls comment-line-break-function if there are comments.
(do-auto-fill): Use that.

which indicates the function was added in 2007 by RMS and made the
default for M-j in 2019. 

Based on your report of having org 9.3, my suspicion is that your org
version is too old for the current Emacs versions (since the change in
2019 to use default-indent-new-line for C-M-j and M-j. I don't think
this is a bug in current Org or Emacs.

The other possibility is that you have a broken "mixed" installation of
org. This can easily occur if org is upgraded when it has already been
loaded into the Emacs instance when the upgrade is performed. It is
critically important to ensure Org has not been loaded before attempting
to do an upgrade. Unfortunately, it is very easy to not realise that
something in your init file is

Re: Is M-j broken for you in Org on Emacs 27 and 28?

2021-11-29 Thread Tim Cross


Richard Lawrence  writes:

> Tim Cross  writes:
>
>> I'm running Emacs 28 and cannot reproduce the issue you observe.
>
> Hmm, the plot thickens!
>
>> Running emacs -Q I find M-j is bound to
>>
>> M-j runs the command default-indent-new-line (found in global-map),
>> which is an interactive compiled Lisp function in ‘simple.el’.
>
> I definitely see the error in emacs -Q with
>
> GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.24, cairo 
> version 1.16.0)
>
> which only contains Org 9.3, in my installation. So the problem has been
> around at least that long, but only surfaced for me because the binding
> of M-j changed between Emacs 26 and 27.
>

I think something is very wrong if your Emacs 28 has org 9.3. I'm pretty
sure the earliest version which was bundled with Emacs 28 was 9.4 - it
is certainly 9.5 now and that is the version that will be bundled with
it when it is released.

My suggestion would be to update Emacs to current version on the
emacs-28 branch of the git repo. This is at 'rc1' level and close to
what will be released. There is not much oint in trying to diagnose an
issue with both an old development version of Emacs and a version of org
which is two releases old. Update Emacs and ensure org is at least
version 9.5 (verison 9.5.1 was just released).



Re: Is M-j broken for you in Org on Emacs 27 and 28?

2021-11-28 Thread Tim Cross


Richard Lawrence  writes:

> Colin Baxter   writes:
>
>> I confirm that it also appears broken to me in emacs-27.2, with the same
>> error as you found. I have never noticed it before, possibly because I
>> use C-j rather than M-j.
>
> Thanks for confirming. Do you know what the difference between C-j and
> M-j is "supposed" to be? They both do very mode-dependent things. I
> guess M-j is more explicitly aimed at continuing comments (which is
> probably why I started using it), but it has always worked great for me
> as a newline-and-indent-like-I-want command outside of comments too.

I'm running Emacs 28 and cannot reproduce the issue you observe. Running
emacs -Q I find M-j is bound to

M-j runs the command default-indent-new-line (found in global-map),
which is an interactive compiled Lisp function in ‘simple.el’.

It is bound to C-M-j, M-j.

(default-indent-new-line  SOFT FORCE)

Break line at point and indent.
If a comment syntax is defined, call ‘comment-line-break-function’.

The inserted newline is marked hard if variable ‘use-hard-newlines’ is true,
unless optional argument SOFT is non-nil.

This binding is the same inside and outside of org mode. This is with
org version

Org mode version 9.5 (release_9.5-72-gc5d6656 @ 
/usr/local/share/emacs/28.0.60/lisp/org/)

and Emacs version

GNU Emacs 28.0.60 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30,
cairo version 1.16.0) of 2021-11-26

Note that C-j in org mode is different from 'normal' C-j in that it is
bound to org-return-and-maybe-indent. If you want M-j to act like C-j in
org mode, you would need to rebind M-j to org-return-and-maybe-indent in
an appropriate org mode startup hook. 



Re: [patch] fix ox-latex async export bug

2021-11-28 Thread Tim Cross


Nicolas Goaziou  writes:

> Hello,
>
> Rasmus  writes:
>
>> When I try to export documents asynchronously with ox-latex, I always get
>> a bug in the “org-export-processFOO” files. The last sexp is always 
>> something like this: 
>>
>> (funcall '# 
>> "test.tex")
>>
>> where the “#” and “’” are mixed around. This happens even with a very
>> simple ‘org-export-async-init-file’ just loading ELPA Org.
>>
>> This was previously reported here:
>>
>> https://lists.gnu.org/archive/html/emacs-orgmode/2021-06/msg00422.html
>>
>> Nicolas’ fix (replicated in this patch) seems to fix it.
>
> Applied. Thank you.
>
>> I don’t really understand why this bug happens to be honest.
>
> The patch is already an improvement, but the beast is still lurking,
> indeed.
>

Just a shot in the dark which might be completely misleading, but ...

I noticed a thread recently on emacs-devel which talked about one of the
problems with async calls in Emacs is that they cannot handle user input
correctly. All seems to work fine provided the async process being
generated doesn't try to wait for user input at some point. I'm
wondering if this could be a problem when exporting to latex if the
underlying latex process encounters errors and is waiting for user input
before it can continue (which happens if there are problems in the tex
source latex is trying to process)?

It might be worth checking next time you encounter an error if you can
run latex on the generated *.tex file and see if it waits for user
input?



Re: "Orgdown", the new name for the syntax of Org-mode

2021-11-28 Thread Tim Cross


Karl Voit  writes:

> Hi Org-mode community,
>
> At this year's EmascsConf, I had a 12 minute video where I explain why
> we do need a different name for the syntax of Org-mode in contrast to
> the Elisp implementation of GNU/Emacs Org-mode.
>
> I would like you to read my rationale and motivate you to use the term
> "Orgdown" for the syntax and "Orgdown1" for the first (very basic)
> level of Orgdown syntax elements.
>
> - The EmacsConf21 talk: https://emacsconf.org/2021/talks/org-outside
> - Orgdown site: https://gitlab.com/publicvoit/orgdown (please contribute!)
> - My motivation article: https://karl-voit.at/2021/11/27/orgdown/
>   - This is the longer version of my 12 minute EmacsConf21 video.
> - My personal copy of the video: 
> https://tube.graz.social/w/bgJVfjPLQAoJwLJQZoo3Hu
>
>
> Just as a sneak preview (not as a replacement for my motivation article):
>
> Orgdown is and will be defined in a set of levels, starting with very
> basic Orgdown1 (or OD1 or O↓1 or ⧬1 - depending on your coolness
> factor of choice :-) )
>
> - OD1 → 
> https://gitlab.com/publicvoit/orgdown/-/blob/master/doc/Orgdown-Levels.org
> - OD2 → will be defined in future
> - OD3 → will be defined in future
> - ...
> - OD∞ = Org-mode (by definition)
>
> Any OD-level needs to be compatible with Org-mode as implemented in
> Elisp for GNU/Emacs Org-mode according to https://orgmode.org. Any ODx
> is a sub-set of the syntax elements of ODy (with y>x).
>
> With introducing a new term specific for the syntax, we do get the
> benefit of getting a better way to handle Org-mode support in
> 3rd-party tools such as listed on
> https://gitlab.com/publicvoit/orgdown/-/blob/master/doc/Tool-Support.org
> (please extend!).
>
> Having a well-defined sub-set of Org-mode, I also do think that formal
> definitions of the Org-mode syntax will be easier to develop, starting
> with the very simple OD1 level.
>
> It would be awesome if we start referring to syntax support in
> 3rd-party tools with the corresponding OD levels.
>
> I want to emphasize that the goal of Orgdown is NOT and will never be
> something that is an alternative to our golden standard Org-mode. We
> will try hard not to get into the Markdown situation where you need to
> know the exact flavor of the markup in order to produce text.
>
> So far, the response was great at the conference and I do hope that
> this idea will get a life of its own, developing the standard further,
> bringing this magnificent lightweight markup to the digital world.
> This also eases some pain for users of GNU/Emacs when it comes to
> exchanging text-based data.
>

Hi Karl,

while I can appreciate the point you are making, I'm doubtful your
suggestion will gain the traction necessary to work. To me, it feels a
little like the frequent posts from RMS in the emacs-devel list where he
gets upset when people refer to Linux instead of GNU Linux. To some
extent, the distinction will be too subtle for many and often, it isn't
clear whether an issue is a syntax definition (orgdown) or an
implementation bug or just simply user misunderstanding.

Perhaps we just need a name for the markup syntax which doesn't actually
reference 'org' at all - it simply is the markup syntax which org
happens to use. A completely separate name might avoid confusion and
would make it very clear that the markup syntax is not org mode. Problem
is, naming is terribly difficult and I have no suggestions on what would
be a good name.

I have not yet viewed your video, but will certainly be doing so. Again,
agree with the sentiment of what your trying to do, just not convinced
it is compatible with basic human nature. 



Re: Re-installing org-mode packages due to annoying message

2021-11-28 Thread Tim Cross


"Alan E. Davis"  writes:

> I have just spent an hour trying to figure out what's going on with ELPA, GNU 
> ELPA, NONGNU ELPA packages.  I am lost.
>
> A plethora of methods exist for installing org-mode and other packages; it is 
> unnecessary to list them, even if I could.
>
> I've been using Emacs and Org-mode for many years.  I am not interested in 
> spending an hour of my time to learn a new way to install
> something that has been working well for me.  I may not use org-mode with the 
> facility of a programmer who can whip off a quick utility
> in emacs lisp, but I have come to depend on the basic tools as a core of my 
> work flow.
>
> I have tried "use package",  but I would prefer something straightforward, 
> like just list-packages then install.  I don't understand how to
> set up my init file (dot emacs) for various package repos.  It was working, 
> that's all I needed.  Now I get a 5 second delay each time I use
> org-mode.  I cannot seem to find the information I need to fix this.  On 
> reddit, on emacs wiki, on this list, I cannot find the magic search
> term.  I see advice like "the maintainer has written a very clear explanation 
> of the issue" but,this very clear explanation does not help
> me understand what I need to do.
>
> I guess I need a formula, but I have cut and pasted two or three different 
> things into the top of my .init file.  Perhaps I need to start
> again, but my .init file has been taking root for nearly 30 years; it's 
> burned into my muscle memory.
>
> I hope I will never have to write another email like this to get help for 
> something that should be simple.  Maybe I will now have to install
> from git.  I think I am already too far out at sea to abandon the packages 
> approach.  I guess it serves me right for stepping off the
> beach.
>

Hi Alan,

sorry your feeling so frustrated. Unfortunately, in order to be able to
maintain org mode with the limited resources available, it has been
necessary to make some changes and as with most transitions, there can
be some rough bits to get through initially. In the long term, things
should actually be simpler with respect to org mode as there will be
no need to add any repositories to use org mode or the org contgrib
packages as they will be available in the default package.el
configuration (Emacs 28 has both GNU ELPA and the new NONGNU
repositories defined by default). 

It is difficult to provide you with any concrete help as you did not
include some important information in your message. Things which would
help include

- What Emacs version are you using? Emacs comes with org built-in, so if
  your running a reasonably recent version of Emacs, perhaps you don't
  actually need to install org at all?

- Do you use any org mode extensions or add on packages (those which are
  not part of org mode or the org mode contrib packages)?

- Is there a reason you need the latest version of org mode rather than
  just using the version which comes bundled in Emacs? Many people just
  stick with the version which is bundled with Emacs as it is stable and
  requires no additional installation steps.

- What is the annoying message you reference in the subject but failed
  to include in the body of your message? 

The good news is that while there are many different ways of installing
packages, you really don't need any of them except those that come with
Emacs. I don't use straight, eget, or any of the many other package
management solutions for Emacs. I use just package.el and use-package,
which works on top of package.el (it can use straight and other package
managers, but your not required to). This gives you exactly what you
want - M-x list-packages, from where you can just select the packages
you want to install and install them.

I would not advise installing from git. This will likely just make
things even more complicated as then you also need to make sure your
building from the right branch/tag - you don't want to run org from the
head of the main branch as this is the development branch, which is
likely not as stable as you want. 

Assuming you are running the current Emacs stable release (Emacs 27.2),
the only repository you have to add in your init file is the new nongnu
repository. You no longer require the orgmode.org/elpa repository. As
the GNU ELPA repository has been standard in Emacs for the last few
releases, you don't need to add anything in order to get the current org
mode. If you use some of the contrib packages, then you now need to
install the nongnu repository. This repository will be configured by
default for the next Emacs release (Emacs 28), but for versions prior to
that, you need to add https://elpa.nongnu.org/nongnu/ to the package
archives list. I have the following in my init.el file

(require 'package)

(setq package-archives '(("nongnu" . "https://elpa.nongnu.org/nongnu/;)
 ("elpa" . "https://elpa.gnu.org/packages/;)
 

Re: CDATA auto aggregation

2021-11-26 Thread Tim Cross


Diego Rodriguez  writes:

> Hello,
>
> I am customizing my org-mode installation but there is something that I don't 
> understand.
>
> When I execute the following statement:
>
> ```
> (setq org-html-mathjax-template
>
> "
> 
> MathJax.Hub.Config({
> displayAlign: \"%ALIGN\",
> displayIndent: \"%INDENT\",
> \"HTML-CSS\": { scale: %SCALE,
> linebreaks: { automatic: \"%LINEBREAKS\" },
> webFont: \"%FONT\"
>},
> SVG: {scale: %SCALE,
>   linebreaks: { automatic: \"%LINEBREAKS\" },
>   font: \"%FONT\"},
> NativeMML: {scale: %SCALE},
> TeX: { equationNumbers: {autoNumber: \"%AUTONUMBER\"},
>MultLineWidth: \"%MULTLINEWIDTH\",
>TagSide: \"%TAGSIDE\",
>TagIndent: \"%TAGINDENT\"
> }
> });
> 
> 
> ")
> ```
>
> A `CDATA` tag gets appended in my HTML export as shown below:
>
> ```html
>   
>  mathjax.hub.config({
> displayalign: "center",
> displayindent: "0em",
> "html-css": { scale: 100,
> linebreaks: { automatic: "false" },
> webfont: "TeX"
>},
> svg: {scale: 100,
>   linebreaks: { automatic: "false" },
>   font: "TeX"},
> nativemml: {scale: 100},
> tex: { equationnumbers: {autonumber: "AMS"},
>multlinewidth: "85%",
>tagside: "right",
>tagindent: ".8em"
> }
> });
>   ]]>
>   
>  
> "https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS_HTML";
>   type="text/javascript">
> 
> ```
>
> As you can see above, when I set the variable I have no CDATA tag on it. 
> Where does this tag come from? The problem is that the
> CDATA tag messes up with the MathJax configuration parser, as it expects a 
> JavaScript script inside. But, instead, it finds a CDATA tag
> that, while it might be valid according to W3, the MathJax parser throws an 
> `eval` error in the console since it tries to parse the body of
> that HTML node.
>
> Where does this CDATA tag gets added automatically and how can I modify this 
> behavior?
>

Unfortunately, I don't know precisely where that tag gets added from. I
suspect it could be related to the ox-html stuff, which is still based
on xhtml rather than html5. I did suggest some months back that it was
probably about time we updated to export as html5 rather than the now
deprecated xhtml, but there were a few who felt that this would result
in the loss of key functionality they wanted to maintain.

We probably should consider moving the existing ox-html to an ox-xhtml
and implementing a new ox-html that is based on html5. 



Re: Why is an image width restricted to being between 0 and 200% of the text area

2021-11-23 Thread Tim Cross


Matt Huszagh  writes:

> I agree that requesting an image to be >2x the buffer text width is a
> strange request, and it's not one I've ever tried to give. But, I think
> the salient point is that it's a very clear request, and I think org
> should carry it out. I'm all in favor of org helping people not shoot
> themselves in the foot, but I don't think it should prevent people from
> doing so, especially when they're clear about their intentions. I also
> think this qualifies as a case where someone /might/ have a valid reason
> for doing this.

+1M. We need to ensure org does not become too opinionated regarding
what is reasonable. If there is no good reason to impose an upper limit,
we should avoid doing so. Org is so powerful and open to customisation,
it is unlikely any of us can foresee all possible scenarios, so we need
to be careful not to artificially constrain the possibilities.   , 

>
> I guess we could make the upper limit customizable and default to
> 2.0. But, this is a bit confusing because it doesn't apply to the
> original image width. I also think adding too many customizable
> variables adds to complexity. I don't know. Thoughts? This also isn't a
> feature I've ever needed... so I'm happy to concede and revisit it in
> the future if I have a valid use case for it.
>

+1M. Org already has an excessive number of custom settings. We need to
actively avoid adding mor whenever we can. At first glance, a custom
variable seems to be a good option. However, once you take testing and
maintenance into consideration and think about the basic testing
principal of ensuring all possible paths are tested, you soon see why
adding such custom options really increases maintenance overhead.

If there is a legitimate technical reason to set an upper limit, then
that is fine. However, setting a limit because you cannot imagine anyone
wanting to go above it is probably not.



Re: [PATCH] Fix window width when line numbers present

2021-11-22 Thread Tim Cross


Bastien  writes:

> Hi Ihor,
>
> Ihor Radchenko  writes:
>
>> The text is mostly clear. However, there is a slight confusion about
>> actual current Emacs version vs. planned Emacs version.
>
> I've rephrased it a bit to focus on "releases", not "versions":
>
>   We aim at keeping the latest stable version of Org compatible with
>   the *last three major releases of Emacs*.
>
>   For example, if the latest release of Emacs is 28.x, then you can
>   assume that the latest release of Org is compatible with Emacs 28.x,
>   27.x and 26.x, but not with Emacs 25.x.
>
>> Should we target "current" Emacs 28 for main? Should we target Emacs
>> version+1 as current on main all the time or maybe just when new
>> Emacs release branch is created, but not yet released? It is not
>> fully clear.
>
> These paragraphes on Worg try to explicit reasonable expectations 
> for users and are thus focusing on stable releases only.
>
> Expectactions for development versions are another story: I am in
> favor of not trying to commit to _any_ expectactions for Org's main.
>
> For example, I would not mind a feature on main that relies on a new
> feature in the Emacs development branch, as long as we manage to keep
> the promise that the next Org stable is usable with the older Emacs
> releases.

So what your saying is that if the current main branch relies on a
new feature in the current Emacs development branch, then org must also
have a compatibility function so that the feature works with at least
the last 2 released versions (possibly 3 since we don't know when the
new Emacs version will be released and we may need/want to release main
before the new Emacs version is released).


So basically, if you want to use a new feature in development Emacs, you
will also need to implement a compatibility version for the previous 2
or 3 Emacs versions. This probably sounds worse than it is in practice
because most of the time, once you have a compatibility implementation
for the previous version, it will likely work with the previous 2 or 3
versions. It further means that a new feature added into Emacs cannot be
used in org without a compatibility version until it has been in Emacs
for 3 versions. 



Re: [PATCH] Fix window width when line numbers present

2021-11-22 Thread Tim Cross


Ihor Radchenko  writes:

> Bastien  writes:
>
>> Our commitment is that the latest Org version is compatible with the
>> last three stable versions of Emacs.
>>
>> So when Emacs 28 and Org 9.6 are both out, we guarantee that Org is
>> compatible with Emacs 28, 27 and 26.
>>
>> Does that explain it better?
>>
>> Let me know if you think the text on Worg should be clarified.
>
> The text is mostly clear. However, there is a slight confusion about
> actual current Emacs version vs. planned Emacs version.
>
> Emacs 28 is have not been officially released yet. So, the "current
> major version of Emacs" is 27 and we are theoretically committed to
> Emacs 27, 26, and 25. Both on main and bugfix.
>
> On the other hand, we do know that Org 9.5 (bugfix) is going to be
> packaged together with Emacs 28 when it is out. Also, we do not yet know
> the time when Emacs 28 is going to be released (it can be even, say, a
> year from now). Will it happen before Org 9.6 (main)? Maybe. Maybe not.
> And some people are already using main branch. Should we target
> "current" Emacs 28 for main? Should we target Emacs version+1 as current
> on main all the time or maybe just when new Emacs release branch is
> created, but not yet released? It is not fully clear.
>

I agree it can be confusing. My interpretation -

As org 9.5 has been released and the latest Emacs version is still 27.2,
then org 9.5 needs to support 27.x, 26.x and 25.x.

Provided Org 9.6 is not released *before* Emacs 29, then it only needs
to support 29,.x, 28.x and 27.x. However, if org 9.6 is released before
Emacs 29, it has to support 28.x, 27.x and 26.x.

Of course, we could also interpret things differently. For example, we
could tie the support to the version of Emacs current at the time of
release. i.e. if we release org 9.6 before Emacs 29.1, then it would
have to support 28.x, 27.x and 26.x even if org 9.6 is what is released
in Emacs 29.

The difference in release schedules will mean this is always a little
challenging IMO.





Re: [PATCH] Accept more :tangle-mode specification forms

2021-11-21 Thread Tim Cross


Timothy  writes:

> Hi Tim,
>
>>> The parsing of “555” to the integer 555 is done by
>>> org-babel-parse-header-arguments, and so can’t really be changed.
>>
>> I don't understand this. Why can't it be changed?
>
> Well, it can't be changed without changing
> org-babel-parse-header-arguments, which is quite a major change and I
> suspect may be a breaking change.
>
> Maybe it's fine? I just don't feel confident about this.
>
>> When I said disable base 10 values, what I meant was generate an error
>> if the value looks like it is base 10 i.e. does not have a leading o or
>> #o. This makes it clear it has to be specified as an octal value and
>> will alert users to this fact.
>
> This is all well and good, but the point I'm making is that since due to
> the current behaviour of org-babel-parse-header-arguments we can't
> distinguish between :tangle-mode (identity #o555) and :tangle-mode 365
> putting this idea /into practice/ doesn't look like it will be easy.
>

I don't see the complication here. When you call
org-babel-parse-header-arguments, it returns an alist of argument name
and value. If you had a header like :tangle-mode 555, you get the value
(:tangle-mode . 555), if you have :tangle-mode o555, you get
(:tangle-mode . "o555"). My suggestion is simply to look at the value
and if it is not a string, reject it outright. If it is a string, then
match that string to the allowed formats (i.e. an octal value if it
starts with "#0o" or "o", a symbolic value if it matches a regexp for
symbolic representation and even possiblly your ls -l format if it
matches that. This is not a change in org-babel-parse-arguments, but
rather a change in how we interpret those arguments. 


> It's also worth noting that with regard to the recent changes that have
> been made, this is not a new issue.

Which I don't think is a valid argument for the status quo. Allowing
base 10 mode specifications is just a bad idea. The fact it has been
permitted in the past is likely just an oversight rather than
intentional behaviour. Given the potential security issues involved
here, I think we can legitimately obsolete such features (using
acceptable change management approaches with suitable notification for
users etc).


>
>>> I think the ls-style is quite valuable for two reasons. It’s
>>> well-established thanks to ls, and is particularly good for
>>> understanding permissions at a glance.
>>
>> I would agree it is an established way to display the permissions
>> associated with a filesystem object, but disagree it is a well
>> established way to set such values - I know of no other tool which uses
>> this format.
>
> The driving motivation here is that while the tools which you mention do
> not use this for setting permissions (e.g. chmod), they are only used
> for /setting/ permissions. Since Org files aren't write-only, and
> occasionally we go back and look at what we've written :P I think
> allowing a format that is optimised for ease-of-reading instead of
> ease-of-writing makes some sense. It is in this respect that I think
> ls -l style is a good idea.
>

I understand your motivation. I dispute your claim it is an established
and common convention. 

>> It is also not the same as the ls -l display (no object
>> type indicator). The ls -l also displays sticky bit, uid and gid. Does
>> your format also support setting these values (something which can be
>> done with the octal or symbolic formats) i.e. support s, S, t and T for
>> the 'executable' bit for user/group?
>
> Ah, I'm afraid that I'm not that up-together on sticky bits. I suspect
> that it's not just the ls -l style that will need tweaking. I'll read up
> on this in the next few days and update the ML.
>

This is very difficult Timothy. I really do appreciate all the work you
have done here and your patience, which makes what I'm going to say all
the more difficult. My big concern here is that you don't understand
this area with sufficient depth of understanding to really appreciate
some of the subtleties involved here. This makes me concerned that any
implementation of a new way to specify mode settings may have some
unexpected side effects or hidden issues which could be exploited in
unexpected ways (worse case) or simply have unexpected bugs which result
in mode settings which are not what the user intended. As someone who
has used Unix since 1980 and Linux since 1995, I've experienced my share
of issues relating to file modes and their interaction with different
filesystems, NFS, different platforms, character sets, locales etc.
Implementing a new approach as you are proposing is something I would be
extremely wary about.

If you do plan to go forward with this approach, please ensure you are
very confident in your understanding of the sticky bit, setuid/setgid
and associated interaction with execute etc. 

I suspect you do have the skills to implement correctly given sufficient
time, testing and motivation. However, I am concerned your initial stab
at this 

Re: [PATCH] Accept more :tangle-mode specification forms

2021-11-20 Thread Tim Cross


Timothy  writes:

> * For example, “:tangle-mode 755” will now produce the warning:
>   “1363 is not a valid file mode octal. Did you give the decimal value 755 by
>   mistake?”. Maybe it would be worth adding “if so try o755” or similar?


I think that warning will be confusing for users. They will look at the
value 1363 and be confused where that value came from, likely assuming
they have found a bug in org mode. Perhaps something simpler like

"Decimal 555 is not a valid file mode specification. Did you mean to
specify it as an octal value? If so, prefix it with a 'o'".



Re: [PATCH] Accept more :tangle-mode specification forms

2021-11-20 Thread Tim Cross


Timothy  writes:

> The parsing of “555” to the integer 555 is done by
> org-babel-parse-header-arguments, and so can’t really be changed. For a simple
> octal notation our best bet is adding non-digit characters so it is left as a
> string, e.g. “o555”.

I don't understand this. Why can't it be changed? Doing so may not be
trivial, but I cannot see any reason it cannot be changed if we wanted
to. However, perhaps all we really need to do is look at the value
returned and if the value for mode is not a string, throw an error and
if it is a string, either match symbolic format or verify it has a
leading 'o' or '#o' and treat as octal, else throw an error. 

> Giving errors when a base 10 value has been given by accident would be a nice
> idea, but in practice isn’t easy as many base 10 values are valid octal 
> values,
> e.g. #o555 = 365. Likewise, I don’t think Tim’s suggestion of disallowing base
> 10 values is feasible.
>

Did you mean many base 10 values are valid mode setting values? All base
10 values can be expressed in octal, so I'm assuming you meant some base
10 values will translate into octal values which are valid mode setting
values. My point here is that such translations are unlikely to be what
the user expected.

I have never seen anyone use base 10 to specify mode settings. It takes
a lot more mental agility to derive a base 10 version for a valid mode
setting than it does to use octal, which has a clear and natural mapping
of the octal characters to the underlying bits used to represent the
mode for user, grop and other (as well as setuid, setgid and sticky
bit).

When I said disable base 10 values, what I meant was generate an error
if the value looks like it is base 10 i.e. does not have a leading o or
#o. This makes it clear it has to be specified as an octal value and
will alert users to this fact. 

> Regarding the concern that “we are making a rod for our back by trying to make
> this overly clever” and that the change makes it too complex — I disagree.
> Having had this discussion earlier we’ve ended up with,
> • a shorthand for octal
> • ls-style
> • chmod-style
> I think this small collection of distinct and simple input methods isn’t 
> overly
> clever or complex, and feel that it strikes the right balance between too many
> options and too little flexibility.
>

We will probably have to disagree here as I remain unconvinced. I guess
time will tell.

> Octal is great, for those that are familiar with it. Likewise, chmod-style is
> quite natural for those who are used to chmod. There’s little added complexity
> to the Org code base as we simply pass this to the Emacs function
> file-modes-symbolic-to-number. I think the ls-style is quite valuable for two
> reasons. It’s well-established thanks to ls, and is particularly good for
> understanding permissions at a glance.

I would agree it is an established way to display the permissions
associated with a filesystem object, but disagree it is a well
established way to set such values - I know of no other tool which uses
this format. It is also not the same as the ls -l display (no object
type indicator). The ls -l also displays sticky bit, uid and gid. Does
your format also support setting these values (something which can be
done with the octal or symbolic formats) i.e. support s, S, t and T for
the 'executable' bit for user/group?

>For reading Org files I think this is
> advantageous compared to the other styles. I’m don’t find assertions that this
> is non-typical or unpredictable well-founded. Each style/syntax is 
> well-defined,
> simple, distinct, and taken from very common/wide spread usage.
>

Again, I know of no other tool which uses the ls -l format to set file
mode permissions, so I cannot agree with the assertion it is well
established. Personally, I prefer the symbolic form as it is shorter and
clear. I find the ls -l form too easy to get wrong (especially with
getting the number of '-' correct).

> Tim suggested that invalid forms should cause tangling to fail, but I feel 
> this
> may be a bit much. Personally I’m inclined to either
> • warn, and don’t touch the file permissions (this is what currently occurs)
> • use a very conservative file permission (e.g. rw——-).

I'm unsure on this. My concern is people may not notice the warning and
then be surprised later. Given the potential security issues, a later
surprise is something to be avoided even if it is inconvenient. With
respect to the default action to take, I would suggest we also need to
look at the default umask setting for the user and if that is more
restrictive, use that rather than some value like rw--- For all we
know, the user has set a specific umask for a valid reason and will be
surprised if emacs just ignores that to do what it wants (another
surprise to be avoided).

The user is not required to specify a mode. However, if they do and if
we cannot interpret what they have specified without ambiguity, we
should throw an error and cease 

Re: [PATCH] Accept more :tangle-mode specification forms

2021-11-19 Thread Tim Cross


Timothy  writes:

> Hi All,
>
> I thought I’d checked for this, but I’ve just noticed that :tangle-mode 755
> doesn’t actually work as expected. I assumed 755 would be passed as a string 
> but
> org-babel-parse-header-arguments actually turns it into an integer, just like
> (identity #o755). Obviously 755 != #o755 and so this causes issues.
>
> As it stands “755” works, but that isn’t great (most importantly, it’s easy to
> confuse). Since it’s easier to add than remove things like this, we could just
> get rid of this for now, but a convenient octal notation was a large chunk of
> the motivation here IIRC.
>
> We could also change the implementation to handle :tangle-mode o755, which 
> will
> make org-babel-parse-header-arguments parse the argument as a string.
>
> I’m be keen to hear other people’s thoughts on this.
>

Thanks for your work on this. I am a little concerned we are making a
rod for our back by trying to make this overly clever in order to
provide as much convenience to the user as possible. As this setting
does have significant security implications, I would favour a simple and
easily testable option which is concise and unambiguous over
convenience. I would drop the 'rwxrw-r--' format as it isn't typical,
not allow base10 mode specifications and possibly even limit what
can be set (i.e. no sticky bit etc, just read, write and execute).

Security issues are more often than not, caused by complexity. Things
become complex when we try to satisfy too many options. In this case,
rather than being liberal in what we accept and precise in what we
send/do, I think we need to be precise in what we accept and do.

I would just accept two formats, both being strings with either "o400"
(or perhaps "#o400") and "u+rwx" symbolic form and anything else
generates an error (a hard error, not a warning i.e. stop processing,
don't tangle). 
 
Making the octal version be "#o600" rather than just "o600" would
possibly make interpretation easier as it would avoid "o600" and "o+r" -
if it starts with "#o" interpret as octal, otherwise try to parse as
symbolic names.

this would mean there will be some edge cases where you cannot set the
mode precisely to the value you want. However, these will be fringe
cases and requiring the user to take additional/special steps in this
case is IMO not too much to ask in exchange for reliability and
correctness for the majority and avoiding dangerous corner cases. 



Re: [DISCUSSION] Refactoring fontification system

2021-11-19 Thread Tim Cross


Ihor Radchenko  writes:

> Dear all,
>
> Recently, there have been multiple issues related to incorrect
> fontification:
> - https://list.orgmode.org/orgmode/23707.20428.546749.44...@frac.u-strasbg.fr/
> - https://list.orgmode.org/orgmode/87fsujp7mc@web.de/
> - https://list.orgmode.org/orgmode/87czvqxdn9@gmail.com/
> - 
> https://list.orgmode.org/8735nsv9qo@nicolasgoaziou.fr/T/#me1c44b6e493dd280cca4f042b833c24749ae4fe0
>
> These issues keep appearing because our current fontification code is
> based on regexps and only approximates the actual Org grammar elements.
> It is largely a legacy from the times when org-element parser was not a
> thing.
>
> Maybe it is a time to upgrade the fontification according to our
> state-of-art parser?
>
> Instead of fontifying elements individually via regexps, we can leverage
> org-element-map, org-element-parse-buffer, org-element-cache, and
> jit-lock-mode. Each type of Org element/object can be assigned with a
> fontification function accepting a single argument - the element datum.
>
> Also, the fontification code can be move to a separate library.
>
> WDYT?
>

Sounds like a better approach to me. In addition to being
more accurate, this would mean we don't need to keep 2 separate
definitions in sync or have confusing font locking where the regexp and element
definitions are different. Should gives us increased consistency and
less maintenance. 



Re: after installing python3.6 on Ubuntu 16.04, org files open with errors

2021-11-13 Thread Tim Cross


Uwe Brauer  writes:

> [[S/MIME Signed Part:Undecided]]
>
>> Uwe Brauer  writes:
>
>
>> The whole transition from v2.x to v3.x for python has been a terrible
>> mess. Version issues are the most frustrating aspect of Python and one
>> reason I've never embraced the language.
>
>> Given that Ubuntu 16.04 was end of life in April 2021, my recommendation
>> would be to upgrade to ubuntu 21.04. That version seems to have a more
>> consistent Python environment (based on v3). It also has newer ciaro,
>> hafbuzz and other libs used by Emacs which will likely work better and
>> once Emacs 28.0 is released, will have the gccjit libs necessary for
>> native comp etc. 
>
> Well in that case I'd rather would prefer 20.04 a LTS release. From
> experience I know it will take me days till everything is working again,
> but maybe it is time to do that step.
>

I would check what the status was with Python in 20.04. I'm not sure
they had completed the migration to v3 at that point and it was still a
mixed v2/v3 setup. Pretty sure by 21.04 it was all v3.

While I do find upgrades to a new version can be very disruptive if it
is left for a few versions, updates to the next version has been
minimally disruptive for me. I guess a lot depends on how much 'extra'
stuff you install and what tweaks you do, but for the last few years,
I've found the upgrades quite painless. A lot depends on what changes -
for example, I expect the update to 22.04 or maybe 22.10 will be more
disruptive due to the move to pipewire for sound.

One thing I do find useful is to upgrade a virtual first. I use
virt-manager to create a virtual machine running my current version and
then upgrade that to see how the upgrade goes. In fact, I tend to
maintain a VM which is the same as my current version and whenever I
need to make some potentially disruptive upgrades or configuration
changes - such as updating org to a new version or try out the latest
Emacs, I do that on the virtual machine first. This way, I don't
interfere with my core system and configuration and have time to
investigate any issues before doing the same on my core system. 



Re: after installing python3.6 on Ubuntu 16.04, org files open with errors

2021-11-12 Thread Tim Cross


Uwe Brauer  writes:

> Hi
>
> I am running Ubuntu 16.04 and I installed python3.6 (which is not
> officially supported for that Ubuntu version via 
> https://askubuntu.com/questions/865554/how-do-i-install-python-3-6-using-apt-get
>
> I also installed 
>
> sudo -H python3.6 -m pip install jupyter
> sudo -H python3.6 -m pip install pexpect
> sudo -H python3.6 -m pip install matlab_kernel
>
>
> Now however I cannot open even simple org files, I obtain errors I attach
> and most of my keybinding and other things do not work.
>
> Any help is strongly appreciated.
>
> Regards
>

The whole transition from v2.x to v3.x for python has been a terrible
mess. Version issues are the most frustrating aspect of Python and one
reason I've never embraced the language.

Given that Ubuntu 16.04 was end of life in April 2021, my recommendation
would be to upgrade to ubuntu 21.04. That version seems to have a more
consistent Python environment (based on v3). It also has newer ciaro,
hafbuzz and other libs used by Emacs which will likely work better and
once Emacs 28.0 is released, will have the gccjit libs necessary for
native comp etc. 



Re: whitespace in org source files

2021-11-12 Thread Tim Cross


excalamus--- via "General discussions about Org-mode."  
writes:

> I'm hacking on ob-shell.el and I see that the source has mixed tabs and 
> spaces.
> If I were to contribute changes, would it be an issue if tabs were converted 
> to
> whitespace?
>
> I've not used the email contribution style before and have had whitespace 
> issues
> in the past with GitHub PRs.  I typically run whitespace-cleanup with the
> before-save-hook which changes tabs to spaces.

It is usually a bad idea to change whitespace in this manner. It will
not improve readibility but will cause lots of change which will likely
make merging and tracking/reviewing changes harder than necessary.

Ideally, git itslef would be configured to deal with this automatically.
Unfortunately, the tab v space arguments can often reach the level of
religious war and represents something which is difficult to get
consensus on.

My recommendation would be to use dir local variable settings to make
Emacs use the same formatting style (i.e. with tabs) for working with
org-mode sources if your normal/wider preference is just for spaces.
this will prevent Emacs from making unnecessary whitespace changes.



Re: typescript from org src buffer -- tide, lsp

2021-11-06 Thread Tim Cross


Greg Minshall  writes:

> hi.
>
> i'm wondering if anyone programming in typescript might have run into,
> and solved, this "problem", of tangling into a subdirectory, and tide
> [1] not being able to resolve "relative" imports.
>
> i have a file in path /a/b/foo.org.  it includes various source blocks,
> and they do things like `:tangle c/d/bar.ts`.  another source block,
> tangled into in c/d/fubar.ts, say, does something like:
> 
> import Bar from "./bar"
> 
>
> when i open an Org Src edit buffer from fubar.ts's source block, with
> tide mode [1] enabled, i get a complaint
> 
> Cannot find module './bar' or its corresponding type declarations. [2307]
> 
> tide (or, really, iiuc, tsserver) appears to be, reasonably, looking for
> /a/b/bar.ts, and not finding it.  but, i haven't been able to figure out
> how to talk tide into looking elsewhere.
>
> i've done things like add "rootDirs" to `tsconfig.json`, to no avail.  i
> haven't been able to get lsp-mode [2] to work for this scenario, either.
>
> anyway, in case anyone has done exactly that, i'm asking.
>

I've not tried to do this, but a complete shot in the dark

according to the org manual, you can set a :dir value in the source
block header which will specify the working directory to be used i.e.
the directory where the code will be evaluated. Perhaps setting that to
the directory where the other block has tangled the output would work?

See the Environment of a code block node under the working with source
code node in the org manual for details.



  1   2   3   4   5   6   7   >