Re: Manual on web site is not the latest version

[Daniele Nicolodi]

On 28/04/2021 07:35, Bastien wrote:
> Do you still see differences between the doc/org-manual.org and the
> one published on https://orgmode.org/manual/ ? 
> 
> If so, please point at one or two differences.  Thanks!
> 

These two sections are older versions that what is in doc/org-manual.org

https://orgmode.org/manual/Formula-syntax-for-Calc.html#Formula-syntax-for-Calc
https://orgmode.org/manual/Formula-syntax-for-Lisp.html#Formula-syntax-for-Lisp

Cheers,
Dan

Re: On the protection of emails in the archives

[Daniele Nicolodi]

On 08/04/2021 10:29, Guillaume MULLER wrote:
> Hi all,
> 
> I recently sent an email on this mailing list (see 
> https://orgmode.org/list/b1e778c5-acbf-8a17-c9bf-dcb6693e9...@univ-st-etienne.fr/
>  )
> 
> Since then, I'm receiving spams on the email address I used to send the 
> message.
> 
> After some research, it seems that this email address only appears on 2 
> websites, notably this orgmode.org mailing list archive.

A Google search demonstrates otherwise.

> Would it be possible to configure the archive to obfuscate / hide the email 
> addresses inorder to protect its users?

In this time and age, it is not an useful anti-spam measure. It is much
more likely that your email address is harvested from the address book
or the received emails of someone that has their email account
compromised because of a weak password (or, in this case, from any other
website that publishes it).

Cheers,
Dan

Re: Including Email Address in the Reply in Mailing-list

[Daniele Nicolodi]

On 02/04/2021 17:54, Husain Alshehhi wrote:
> 
> Hello all,
> 
> I just noticed that some of us here, when replying, include the email of
> the sender of the previous email in the response as part of body of the
> email. This email address shows up in plain text in the mailing
> list[1]. This is certainly done without any ill-intentions, but that
> could cause the email address to receive quite a bit of spam if spammers
> crawl these pages.
> 
> I suggest refraining from doing so, and instead use the name. Please let
> me know what you think.

The mailing list is archived here https://orgmode.org/list/ including
the sender email address of all messages. Removing the mail address from
quote headers in the replies is useless to the goal of hiding the email
address of the original message senders from web crawlers.

As expressed by others, you need to accept that writing to a public
mailing list requires public disclosure of the source email address.

It is also about 20 years that avoiding publishing someone email address
on the web is not at all an effective measure in avoiding having such
email address targeted by spam.

Cheers,
Dan

Re: Syntax Proposal: Multi-line Table Cells/Text Wrapping

[Daniele Nicolodi]

On 18/03/2021 14:38, Atlas Cove wrote:
>> technically the second table is more space efficient than the first
> 
> I was referring to screen space.

Assuming that you are looking at this with a monospaced font (and I
don't see how you could use a variable width font to look at an Org
table effectively), screen space is proportional to the number of
characters.

Cheers,
Dan

Re: Syntax Proposal: Multi-line Table Cells/Text Wrapping

[Daniele Nicolodi]

On 17/03/2021 21:29, Atlas Cove wrote:

> Allow me to give an example of the updated syntax I propose.
> 
> ```
> | Name | Description| Price |
> |--++---|
> | Orange Juice | Very Citrusy! Very \\  |  5.00 |
> |  | nice indeed!   |   |
> | Grape Juice  | It's like wine, but \\ |  6.00 |
> |  | you can have it all \\ |   |
> |  | day!   |   |
> ```
> 
> As you can see, this is a lot more space-efficient than the current, vanilla 
> solution, especially if there's a large block of text.
> 
> ```
> | Name | Description  | Price |
> |--+--+---|
> | Orange Juice | Very Citrusy! Very nice indeed!  |  5.00 |
> | Grape Juice  | It's like wine, but you can have it all day! |  6.00 |
> ```

Nitpicking: technically the second table is more space efficient than
the first: in the first table the information is represented in 350
characters, but in just 288 characters in the second table.

Cheers,
Dan

Re: [PATCH] ob-sql.el: Add support for SAP HANA

[Daniele Nicolodi]

Hello Robin,

from this patch I infer that you work with SAP. Can I bug you off-list
with a few questions unrelated to org-mode but related to
programmatically interact with SAP?

Thank you.

Best,
Dan

On 04/02/2021 08:55, Robin Campbell Joy wrote:
> * lisp/ob-sql.el (org-babel-execute:sql, org-babel-sql-dbstring-saphana):
> Add basic support for SAP HANA to SQL blocks
> * testing/lisp/test-ob-sql.el: Basic tests for generated db connection
> string
> 
> This change adds basic support for SAP HANA to SQL blocks by
> specifying saphana as :engine.
> 
> It also adds a new header arg `dbinstance' in order to specify the SAP
> HANA instance to connect to.
> 
> Signed-off-by: Robin Campbell Joy  >
> ---
>  lisp/ob-sql.el              |  25 ++-
>  testing/lisp/test-ob-sql.el | 382 
>  2 files changed, 406 insertions(+), 1 deletion(-)
>  create mode 100644 testing/lisp/test-ob-sql.el
> 
> diff --git a/lisp/ob-sql.el b/lisp/ob-sql.el
> index 902194ae8..5398c85aa 100644
> --- a/lisp/ob-sql.el
> +++ b/lisp/ob-sql.el
> @@ -40,6 +40,7 @@
>  ;; - dbuser
>  ;; - dbpassword
>  ;; - dbconnection (to reference connections in sql-connection-alist)
> +;; - dbinstance
>  ;; - database
>  ;; - colnames (default, nil, means "yes")
>  ;; - result-params
> @@ -58,6 +59,7 @@
>  ;; - postgresql (postgres)
>  ;; - oracle
>  ;; - vertica
> +;; - saphana
>  ;;
>  ;; TODO:
>  ;;
> @@ -85,6 +87,7 @@
>      (dbport       . :any)
>      (dbuser       . :any)
>      (dbpassword       . :any)
> +    (dbinstance       . :any)
>      (database       . :any))
>    "SQL-specific header arguments.")
>  
> @@ -174,6 +177,18 @@ SQL Server on Windows and Linux platform."
>    (when database (format "-d %s" database
>        " "))
>  
> +(defun org-babel-sql-dbstring-saphana (host port instance user password
> database)
> +  "Make SAP HANA command line args for database connection. Pass nil to
> omit that arg."
> +  (mapconcat #'identity
> +             (delq nil
> +                   (list (when (and host port) (format "-n %s:%s" host
> port))
> +                         (when (and host (not port)) (format "-n %s" host))
> +                         (when instance (format "-i %d" instance))
> +                         (when user (format "-u %s" user))
> +                         (when password (format "-p %s"
> (shell-quote-argument password)))
> +                         (when database (format "-d %s" database
> +             " "))
> +
>  (defun org-babel-sql-convert-standard-filename (file)
>    "Convert FILE to OS standard file name.
>  If in Cygwin environment, uses Cygwin specific function to
> @@ -197,6 +212,7 @@ database connections."
>                               (:dbport . sql-port)
>                               (:dbuser . sql-user)
>                               (:dbpassword . sql-password)
> +                             (:dbinstance . sql-dbinstance)
>                               (:database . sql-database)))
>               (mapped-name (cdr (assq name name-mapping
>          (cadr (assq mapped-name
> @@ -212,6 +228,7 @@ This function is called by
> `org-babel-execute-src-block'."
>           (dbport (org-babel-find-db-connection-param params :dbport))
>           (dbuser (org-babel-find-db-connection-param params :dbuser))
>           (dbpassword (org-babel-find-db-connection-param params
> :dbpassword))
> +         (dbinstance (org-babel-find-db-connection-param params
> :dbinstance))
>           (database (org-babel-find-db-connection-param params :database))
>           (engine (cdr (assq :engine params)))
>           (colnames-p (not (equal "no" (cdr (assq :colnames params)
> @@ -276,6 +293,12 @@ footer=off -F \"\t\"  %s -f %s -o %s %s"
>        dbhost dbport dbuser dbpassword database)
>       (org-babel-process-file-name in-file)
>       (org-babel-process-file-name out-file)))
> +    (saphana (format "hdbsql %s -I %s -o %s %s"
> +     (org-babel-sql-dbstring-saphana
> +      dbhost dbport dbinstance dbuser dbpassword database)
> +     (org-babel-process-file-name in-file)
> +     (org-babel-process-file-name out-file)
> +     (or cmdline "")))
>                      (t (user-error "No support for the %s SQL engine"
> engine)
>      (with-temp-file in-file
>        (insert
> @@ -309,7 +332,7 @@ SET COLSEP '|'
>   (progn (insert-file-contents-literally out-file) (buffer-string)))
>        (with-temp-buffer
>   (cond
> - ((memq (intern engine) '(dbi mysql postgresql postgres sqsh vertica))
> + ((memq (intern engine) '(dbi mysql postgresql postgres saphana sqsh
> vertica))
>    ;; Add header row delimiter after column-names header in first line
>    (cond
>     (colnames-p
> diff --git a/testing/lisp/test-ob-sql.el b/testing/lisp/test-ob-sql.el
> new file mode 100644
> index 0..51edd2309
> --- /dev/null
> +++ b/testing/lisp/test-ob-sql.el
> @@ -0,0 +1,382 @@
> +;;; test-ob-sql.el --- tests for ob-sql.el
> +
> +;; Copyright (C) 2021 Robin

Re: Google SoC organisation application

[Daniele Nicolodi]

On 02/02/2021 10:54, TEC wrote:
> Hello All,
> 
> Google's SoC organisation applications are currently open, and close on
> <2020-02-20>. I know that Org participated once, in 2012.
> 
> Would it be a good idea to submit an application to do so again?
> With the rise in interest in computational notebooks, blogging tools,
> and other features that Org possesses I'd imagine we have a decent shot.

Applying only makes sense if there are concrete ideas about projects
suitable for GSoC participants and if only if there are Org contributors
that can work as mentors.

Suitable projects should be of the right level of complexity for the
students to have a chance to have them implemented in the allotted time,
which this year is not much, and have a clear design approved by the Org
maintainers, so that most of the time available to the students will not
be spent discussing it.

Do you have any idea about suitable projects?

However, I I think the availability of mentors can be the hardest
requirement. I think that it would not be fair for the students if the
code they would produce does not have good chances to entering the code
base in a timely fashion. Thus, mentors (or, at least, co-mentors)
should be among the Org maintainers. Is anyone available?

Please understand that the GSoC is not a way to get labor for the
project sponsored by Google: it is very likely that the time investment
required to the mentors would be more than enough to implement what the
students will do during the GSoC. The GSoC is more an opportunity to
form a possible future long term contributor to the project.

Cheers,
Dan

Re: ol-plo.el --- Orgmode Link type for navigating Ordered Plain Lists

[Daniele Nicolodi]

On 12/01/2021 21:21, TRS-80 wrote:
> Hello *,
> 
> I just banged out a bit of Elisp implementing a new Orgmode link type
> for navigating Ordered (i.e., numbered/lettered) Plain Lists.[0]

How do you deal with the lists being re-numbered when you edit them?

Cheers,
Dan

Re: accounting

[Daniele Nicolodi]

On 04/01/2021 21:04, Uwe Brauer wrote:
>>>> "DN" == Daniele Nicolodi  writes:
> 
>> On 04/01/2021 09:54, Uwe Brauer wrote:
>>> Hi 
>>> 
>>> https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-ledger.html
>>> 
>>> Points out how to use ledger within org mode.
>>> 
>>> Is there any simpler solution?
> 
>> As other pointed out, you are not describing what you want to
>> accomplish, thus is impossible to suggest anything concrete. However, I
>> would like to suggest to look at Beancount and beancount-mode as
>> alternatives to Ledger as plain text accounting tools.
> 
> Thanks, it seems that beancount is not in Ubuntu, and beancount-mode is
> not available as a emacs package.
> 
> Do you have any pointers?

https://beancount.github.io/

is the project home page. You can find the documentation here:

https://beancount.github.io/docs/

I haven't submitted beancount-mode to any Emacs package repository yet,
there is still some work I would like to do before doing it. You can
find it here:

https://github.com/beancount/beancount-mode

with some minimal documentation.

Beancount is distributed as a Python package, thus it should be easily
installable on any system providing Python 3.6 or later.

Cheers,
Dan

Re: accounting

[Daniele Nicolodi]

On 04/01/2021 09:54, Uwe Brauer wrote:
> Hi 
> 
> https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-ledger.html
> 
> Points out how to use ledger within org mode.
> 
> Is there any simpler solution?

As other pointed out, you are not describing what you want to
accomplish, thus is impossible to suggest anything concrete. However, I
would like to suggest to look at Beancount and beancount-mode as
alternatives to Ledger as plain text accounting tools.

Cheers,
Dan

Re: what do do when multiple functions store a link

[Daniele Nicolodi]

On 02/01/2021 14:49, John Kitchin wrote:
> Recently I have had an issue where multiple functions may store a link,
> e.g. to a bibtex entry. 
>  
> In this case, org-mode seems to prompt me to ask which function to store
> the link with, with an initial input of the first function, which masks
> all the options that are available. This happens
> inside |org-store-link| in ol.el at line 1495 for me. in
> 
> (apply #'org-link-store-props
> (cdr (assoc-string
>       (completing-read
> "Which function for creating the link? "
> (mapcar #'car results-alist)
> nil t (symbol-name name))
>       results-alist))) 
> 
> because of the (symbol-name name).
>  
> Is there an easy way to avoid this, or to modify the order of the
> functions used? I want to see all the options for storing, or better, to
> just store them all and let me choose later when I use org-insert-link.

I have the exact same problem. I think it comes from having org-bibtex
and org-ref loaded at the same time. I haven't investigated a possible
solution.

Cheers,
Dan

Re: Microsoft Excel spreadsheet editing directly from within emacs.

[Daniele Nicolodi]

On 28/12/2020 20:36, Jean Louis wrote:
> * Hongyi Zhao  [2020-12-28 18:19]:
>> Is it possible for me to edit Microsoft Excel spreadsheet directly
>> from within emacs, especially utilizing the powerful capabilities of
>> orgmode?
> 
> You have high expectations but Emacs Org mode does not nearly replace
> the capabilities of a dedicated spreadsheet.
> 
> In comparison to all major known spreadsheets Org tables is not
> powerful and not even comparable.

Without specific details on which functionalities are not supported in
Org tables, this advice is not useful.

There are many use cases in which Org tables are superiors to
spreadsheets, there are use cases where a dedicated spreadsheet
application works better, and there are use cases where a spreadsheet is
an horrible solution but whoever found themselves solving the problem
didn't know better and hammered around till they got a spreadsheet to
output what they wanted.

In my personal experience, in the 80% of the cases where a spreadsheet
has been used, it was the wrong tool for the job.

Without knowing what is the problem at hand, it is impossible to know
which solution is the more appropriate.

Cheers,
Dan

Re: org-mime: markdown not html

[Daniele Nicolodi]

On 19/12/2020 21:27, Uwe Brauer wrote:
> 
> Hi
> 
> Does anybody know a package which would allow, using say gnus, to 
> markdown-ise an email, that would be useful for sending email to github
> issues.

As far as I know, GitHub does not interpret Mardown in comments added to
issues via the email interface. Are you suggesting that Github has
special handling for emails that contain text/markdown MIME parts? I
haven't seen this documented anywhere.

Cheers,
Dan

Re: Release Org 9.4.2

[Daniele Nicolodi]

On 15/12/2020 14:58, Pankaj Jangid wrote:
> Eric S Fraga  writes:
> 
>> On Monday, 14 Dec 2020 at 20:49, Pankaj Jangid wrote:
>>> I like testing Emacs on the trunk and I ‘git pull’ and ‘make bootstrap’
>>> daily and use it without any external packages. This is just to make
>>> sure that any external package is not the cause for what appears to be
>>> an Emacs bug.
>>>
>>> I can certainly add latest Org by adding it to the package-archives. 
>>
>> Or you could track org development from git just as you are tracking
>> Emacs.  That's what I do: any time I build the most recent Emacs (maybe
>> every 2-3 weeks), I also build org.
> 
> May be that I am not explaining it well. Let me put it in a question
> format. It is possible that this has been discussed and I appologise for
> my ignorance.
> 
> My question is/are: (1) Why Org is developed outside Emacs, given that
> it is a core/built-in package. (2) Are there other packages that follow
> the same process?

AFAIK also cc-mode is developed in a dedicate repository.

>From an Emacs development point of view there is the desire to move more
packages that are now included in the core to ELPA and assemble Emacs
releases from these. This will allow packages to have independent
releases and users to update them more frequently than the pace of Emacs
releases. In this process, moving more packages to separate git
repositories would be natural. Thus, the idea would be remove the need
for syncing the org and cc-mode codebases to the Emacs repository rather
than moving the development of these packages to the Emacs git.

Cheers,
Dan

Re: [PATCH] doc/org-manual.org: Extend table formulas Lisp form documentation

[Daniele Nicolodi]

On 25/11/2020 05:37, Kyle Meyer wrote:
> Daniele Nicolodi writes:
> 
>> Hello,
>>
>> I always found the description of Lisp forms in Org table formulas not
>> extremely clear, especially in regard to the use of mode flags. The
>> attached patch tries to clarify the manual a bit.
> 
> Thanks.

Thank you for the review, Kyle. An updated patch is attached.

>> Would it be worth to mention org-sbe in the same section of the manual?
> 
> Yeah, it looks like there's no mention of org-sbe in the manual, so I
> think so (as a separate patch).

After playing a bit with org-sbe, I came to the conclusion that it is
broken beyond repair, at least without breaking it for the people that
managed to make it work for them.

I think that adding mention of it in the manual and explain all the
quirks of the macro is much more work than replace it with something
better. I tried to write a better macro, please have a look here:

https://orgmode.org/list/d429d29b-42fa-7d7b-6f3a-9fe692fd6...@grinta.net/

and the parent message for an explanation of what I think is broken in
org-sbe. Would you support adding org-sbx (for a lack of a better name)
to ob-table.el and mention it in the manual instead of org-sbe? I would
not go as far as deprecating org-sbe, just yet, but maybe soon...

Cheers,
Dan
From d39ec4465605f56d5f53a36faf4e419ae1b862f0 Mon Sep 17 00:00:00 2001
From: Daniele Nicolodi 
Date: Sat, 14 Nov 2020 18:33:41 +0100
Subject: [PATCH] doc/org-manual.org: Extend table formulas Lisp form
 documentation

doc/org-manual.org (Emacs Lisp forms as formulas): Be more
explicit about how fields are interpolated into the Lisp forms,
clarify the use of mode flags, and add a couple more examples.
---
 doc/org-manual.org | 64 +-
 1 file changed, 40 insertions(+), 24 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 2f7f5f847..97018d075 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -2178,38 +2178,54 @@ It is also possible to write a formula in Emacs Lisp.  
This can be
 useful for string manipulation and control structures, if Calc's
 functionality is not enough.
 
-If a formula starts with a single-quote followed by an opening
-parenthesis, then it is evaluated as a Lisp form.  The evaluation
-should return either a string or a number.  Just as with Calc
-formulas, you can specify modes and a ~printf~ format after
-a semicolon.
+A formula is evaluated as a Lisp form when it starts with a
+single-quote followed by an opening parenthesis.  Cell table
+references are interpolated into the Lisp form before execution.  The
+evaluation should return either a string or a number.  Evaluation
+modes and a ~printf~ format used to render the returned values can be
+specified after a semicolon.
 
-With Emacs Lisp forms, you need to be conscious about the way field
-references are interpolated into the form.  By default, a reference is
-interpolated as a Lisp string (in double-quotes) containing the field.
-If you provide the =N= mode switch, all referenced elements are
-numbers---non-number fields will be zero---and interpolated as Lisp
-numbers, without quotes.  If you provide the =L= flag, all fields are
-interpolated literally, without quotes.  For example, if you want a
-reference to be interpreted as a string by the Lisp form, enclose the
-reference operator itself in double-quotes, like ="$3"=.  Ranges are
-inserted as space-separated fields, so you can embed them in list or
-vector syntax.
+By default, references are interpolated as literal Lisp strings: the
+field content is replaced in the Lisp form stripped of leading and
+trailing white space and surrounded in double-quotes.  For example:
 
-Here are a few examples---note how the =N= mode is used when we do
-computations in Lisp:
+: '(concat $1 $2)
 
-- ='(concat (substring $1 1 2) (substring $1 0 1) (substring $1 2))= ::
+#+texinfo: @noindent
+concatenates the content of columns 1 and column 2.
+
+When the =N= flag is used, all referenced elements are parsed as
+numbers and interpolated as Lisp numbers, without quotes. Fields that
+cannot be parsed as numbers are interpolated as zeros.  For example:
+
+: '(+ $1 $2);N
+
+#+texinfo: @noindent
+adds columns 1 and 2, equivalent to Calc's =$1+$2=.  Ranges are
+inserted as space-separated fields, so they can be embedded in list or
+vector syntax. For example:
 
-  Swap the first two characters of the content of column 1.
+: '(apply '+ '($1..$4));N
 
-- ='(+ $1 $2);N= ::
+#+texinfo: @noindent
+computes the sum of columns 1 to 4, like Calc's =vsum($1..$4)=.
+
+When the =L= flag is used, all fields are interpolated literally: the
+cell content is replaced in the Lisp form stripped of leading and
+trailing white space and without quotes.  If a reference is intended
+to be interpreted as a string by the Lisp form, the reference operator
+itself sho

Re: [PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

Thank you for the review, Kyle.

Another updated patch set is attached.

Cheers,
Dan


On 24/11/2020 06:35, Kyle Meyer wrote:
> Daniele Nicolodi writes:
> 
>> Subject: [PATCH 1/3] org-table: Remove unused org-tbl-calc-modes variable
>>  declaration
> 
> Looks good.
> 
>> Subject: [PATCH 2/3] org-table: Simplify mode string parsing
>> and reduce scope of local variables.
> [...]
>> -  (if (string-match ";" formula)
>> -  (let ((tmp (org-split-string formula ";")))
>> -(setq formula (car tmp)
>> -  fmt (concat (cdr (assoc "%" org-table-local-parameters))
>> -  (nth 1 tmp)))
>> +  (if (string-match "\\(.*\\);\\(.*\\)" formula)
>> +  (progn
>> +(setq fmt (concat (match-string-no-properties 2 formula)
>> +  (cdr (assoc "%" org-table-local-parameters
> 
> Hmm, the concat arguments are getting swapped.  Intentional?
> 
> The rest looks good.
> 
>> Subject: [PATCH 3/3] org-table: Add mode flag to enable Calc units
>>  simplification mode
>>
>> * org-table.el (org-table-eval-formula): Add the `u` mode flag to
>> enable Calc's units simplification mode.
>>
>> * test-org-table.el (test-org-table/mode-string-u): Add Unit test for
>> the new mode flag.
>>
>> * org-manual.org: Document new mode flag.
>> ---
>>  doc/org-manual.org |  8 
>>  etc/ORG-NEWS   |  7 ++-
>>  lisp/org-table.el  |  5 +++--
>>  testing/lisp/test-org-table.el | 12 
>>  4 files changed, 29 insertions(+), 3 deletions(-)
> 
> Thanks for the additions.
> 
>> diff --git a/doc/org-manual.org b/doc/org-manual.org
>> index be69996d5..70b748fc7 100644
>> --- a/doc/org-manual.org
>> +++ b/doc/org-manual.org
>> @@ -2075,6 +2075,14 @@ variable ~org-calc-default-modes~.
>>  
>>Fraction and symbolic modes of Calc.
>>  
>> +- =u= ::
>> +
>> +  Units simplification mode of Calc. Calc is also a symbolic
> 
> convention nit: two spaces after a period
> 
>> +  calculator and is capable of working with values having an unit
>> +  (numerals followed by an unit string in Org table cells). This mode
>> +  instructs Calc to simplify the units in the computed expression
>> +  before returning the result.
>> +
>>  - =T=, =t=, =U= ::
>>  
>>Duration computations in Calc or Lisp, [[*Durations and time values]].
>> diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
>> index 889eb4aab..6f6db8e43 100644
>> --- a/etc/ORG-NEWS
>> +++ b/etc/ORG-NEWS
>> @@ -60,7 +60,7 @@ relative links within a project as follows:
>>  #+end_src
>>  
>>  ** New features
>> -*** =ob-python= improvements to =:return= header argument 
>> +*** =ob-python= improvements to =:return= header argument
> 
> unrelated space change
> 
>>  The =:return= header argument in =ob-python= now works for session
>>  blocks as well as non-session blocks.  Also, it now works with the
>> @@ -112,6 +112,11 @@ package, to convert pandas Dataframes into orgmode 
>> tables:
>>  | 2 | 3 | 6 |
>>  #+end_src
>>  
>> +*** New =u= table formula flag to enable Calc units simplification mode
>> +
>> +A new =u= mode flat for Calc formulas in Org tables has been added to
>> +enable Calc units simplification mode.
> 
> s/flat/flag/
> 

From d2ab4d06e19620c0347425861e4534cde8656543 Mon Sep 17 00:00:00 2001
From: Daniele Nicolodi 
Date: Mon, 23 Nov 2020 23:44:51 +0100
Subject: [PATCH 1/3] org-table: Remove unused org-tbl-calc-modes variable
 declaration

* org-table.el (org-tbl-calc-modes): Remove variable declaration as
the varialble is used only within `org-table-eval-formula'.

* org-table.el (org-table-eval-formula): Rename `org-tbl-calc-modes`
local variable without the `org-tbl-` prefix and use the gained screen
real estate to avoid indirection through covenience macro. This
requires moving the mode lookup table from `org-table--set-calc-mode`
to here.

* org-table.el (org-table--set-calc-mode): Drop convenience macro.
Note that the macro was not working as intended when the caller tried
to add a new entry in the plist as in this case the macro would create
a new plist with the added entry but return the old one.
---
 lisp/org-table.el | 42 +++---
 1 file changed, 15 insertions(+), 27 deletions(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 112b1e171..a3b73a828 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -676,8 +676,6 @@ Will be filled automatically during use.")
 ("_

Re: [PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

An updated patch series is attached.

Thank you.

Cheers,
Dan
From d2ab4d06e19620c0347425861e4534cde8656543 Mon Sep 17 00:00:00 2001
From: Daniele Nicolodi 
Date: Mon, 23 Nov 2020 23:44:51 +0100
Subject: [PATCH 1/3] org-table: Remove unused org-tbl-calc-modes variable
 declaration

* org-table.el (org-tbl-calc-modes): Remove variable declaration as
the varialble is used only within `org-table-eval-formula'.

* org-table.el (org-table-eval-formula): Rename `org-tbl-calc-modes`
local variable without the `org-tbl-` prefix and use the gained screen
real estate to avoid indirection through covenience macro. This
requires moving the mode lookup table from `org-table--set-calc-mode`
to here.

* org-table.el (org-table--set-calc-mode): Drop convenience macro.
Note that the macro was not working as intended when the caller tried
to add a new entry in the plist as in this case the macro would create
a new plist with the added entry but return the old one.
---
 lisp/org-table.el | 42 +++---
 1 file changed, 15 insertions(+), 27 deletions(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 112b1e171..a3b73a828 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -676,8 +676,6 @@ Will be filled automatically during use.")
 ("_" . "Names for values in row below this one.")
 ("^" . "Names for values in row above this one.")))
 
-(defvar org-tbl-calc-modes nil)
-
 (defvar org-pos nil)
 
 
@@ -721,18 +719,6 @@ Field is restored even in case of abnormal exit."
 (org-table-goto-column ,column)
 (set-marker ,line nil)
 
-(defsubst org-table--set-calc-mode (var &optional value)
-  (if (stringp var)
-  (setq var (assoc var '(("D" calc-angle-mode deg)
-("R" calc-angle-mode rad)
-("F" calc-prefer-frac t)
-("S" calc-symbolic-mode t)))
-   value (nth 2 var) var (nth 1 var)))
-  (if (memq var org-tbl-calc-modes)
-  (setcar (cdr (memq var org-tbl-calc-modes)) value)
-(cons var (cons value org-tbl-calc-modes)))
-  org-tbl-calc-modes)
-
 
 ;;; Predicates
 
@@ -2433,7 +2419,7 @@ location of point."
equation
  (org-table-get-formula equation (equal arg '(4)
   (n0 (org-table-current-column))
-  (org-tbl-calc-modes (copy-sequence org-calc-default-modes))
+  (calc-modes (copy-sequence org-calc-default-modes))
   (numbers nil)   ; was a variable, now fixed default
   (keep-empty nil)
   n form form0 formrpl formrg bw fmt x ev orig c lispp literal
@@ -2449,14 +2435,12 @@ location of point."
  (setq c (string-to-char (match-string 1 fmt))
n (string-to-number (match-string 2 fmt)))
  (if (= c ?p)
- (setq org-tbl-calc-modes
-   (org-table--set-calc-mode 'calc-internal-prec n))
-   (setq org-tbl-calc-modes
- (org-table--set-calc-mode
-  'calc-float-format
-  (list (cdr (assoc c '((?n . float) (?f . fix)
-(?s . sci) (?e . eng
-n
+ (setf (cl-getf calc-modes 'calc-internal-prec) n)
+   (setf (cl-getf calc-modes
+  'calc-float-format)
+  (list (cdr (assoc c '((?n . float) (?f . fix)
+(?s . sci) (?e . eng
+n)))
  (setq fmt (replace-match "" t t fmt)))
(if (string-match "[tTU]" fmt)
(let ((ff (match-string 0 fmt)))
@@ -2476,9 +2460,13 @@ location of point."
(setq keep-empty t
  fmt (replace-match "" t t fmt)))
(while (string-match "[DRFS]" fmt)
- (setq org-tbl-calc-modes
-   (org-table--set-calc-mode (match-string 0 fmt)))
- (setq fmt (replace-match "" t t fmt)))
+ (let* ((c (string-to-char (match-string 0 fmt)))
+(mode (cdr (assoc c '((?D calc-angle-mode deg)
+  (?R calc-angle-mode rad)
+  (?F calc-prefer-frac t)
+  (?S calc-symbolic-mode t))
+   (setf (cl-getf calc-modes (car mode)) (cadr mode))
+   (setq fmt (replace-match "" t t fmt
(unless (string-match "\\S-" fmt)
  (setq fmt nil
   (when (and (not suppress-const) org-table-formula-use-constants)
@@ -2621,7 +2609,7 @@ location of point."
 
   

Re: [PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

On 23/11/2020 23:25, Kyle Meyer wrote:
> Daniele Nicolodi writes:
> 
>> On 23/11/2020 04:14, Kyle Meyer wrote:
>>> You already mentioned that you plan to add documentation.  It'd also be
>>> good to add a test to test-org-table.el and a NEWS entry.
>>
>> By the way, have you seen my other patch with some documentation updates
>> for Org tables?
> 
> Aside from recalling seeing some discussion about word choice, only in
> the sense that it exists in a queue of patches that I will try to get to
> if nobody else does.

I am not planning to revisit the word choice as the current choice
maintain consistency within the manual and revisit all the manual for
substituting "interpolate" with something else consistently is outside
the scope of the proposed patch.

>  Are you bringing it up because it'd be easier to
> revisit this series after that patch is applied?

No, they both touch parts of the manual/code that deal with org-table
formula mode flags, but in different context. On the other hand, they
are related enough that it may be worth to review them in close
succession to amortize cognitive context switch costs :-)

Cheers,
Dan

Re: [PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

On 23/11/2020 04:14, Kyle Meyer wrote:
> You already mentioned that you plan to add documentation.  It'd also be
> good to add a test to test-org-table.el and a NEWS entry.

By the way, have you seen my other patch with some documentation updates
for Org tables?

Cheers,
Dan

Re: [PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

Hello Kyle,

thank you for the review. It is much appreciated as lisp (and Emacs lisp
in particular) is not the language I am most fluent in.

On 23/11/2020 04:14, Kyle Meyer wrote:
> Daniele Nicolodi writes:
> 
>> Subject: [PATCH 1/4] org-table: Fix table formula mode string handling
>>
>> * lisp/org-table.el (org-table-eval-formula): Move mode lookup table
>> from org-table--set-calc-mode to here.
>>
>> * lisp/org-table.el (org-table--set-calc-mode): Use plist-put instead
>> of the buggy open coded version.
> 
> So I think the "buggy" is referring to your analysis in
> <6d8c15c2-d1b0-d913-df39-c60381cff...@grinta.net>:
> 
>   The first (if ...) does some value substitutions, however, IIUC the
>   second (if ...) sets a new value for an entry in the org-tbl-calc-modes
>   plist if the entry is already present and builds a new plist with the
>   entry prepended if the entry is not there. However, the original plist
>   is returned and not the one with the new entry prepended.
> 
> And...
> 
>> ---
>>  lisp/org-table.el | 24 ++--
>>  1 file changed, 10 insertions(+), 14 deletions(-)
>>
>> diff --git a/lisp/org-table.el b/lisp/org-table.el
>> index 112b1e171..0790dc3ca 100644
>> --- a/lisp/org-table.el
>> +++ b/lisp/org-table.el
>> @@ -721,17 +721,8 @@ Field is restored even in case of abnormal exit."
>>   (org-table-goto-column ,column)
>>   (set-marker ,line nil)
>>  
>> -(defsubst org-table--set-calc-mode (var &optional value)
>> -  (if (stringp var)
>> -  (setq var (assoc var '(("D" calc-angle-mode deg)
>> - ("R" calc-angle-mode rad)
>> - ("F" calc-prefer-frac t)
>> - ("S" calc-symbolic-mode t)))
>> -value (nth 2 var) var (nth 1 var)))
>> -  (if (memq var org-tbl-calc-modes)
>> -  (setcar (cdr (memq var org-tbl-calc-modes)) value)
>> -(cons var (cons value org-tbl-calc-modes)))
>> -  org-tbl-calc-modes)
>> +(defsubst org-table--set-calc-mode (var value)
>> +  (plist-put org-tbl-calc-modes var value))
> 
> ...that does look to be the case.  Do you have an example that triggers
> the issue?  If so, it'd be good to cover that in test-org-table.el.

calc-simplify-mode is not part of the default calc mode plist, thus
adding it to the plist does not work without this change.

> However, looking ahead, org-table--set-calc-mode is dropped in the last
> patch.  I'd suggest instead dropping org-table--set-calc-mode and moving
> to using cl-getf as part of this first patch.  (I know that'd require a
> bit of reworking since it touches changes from patches 2 and 3.)

I can do this.

>> Subject: [PATCH 2/4] org-table: Simplify mode string parsing
>>
> [...]
>>  ;;; Predicates
>> @@ -2427,54 +2427,42 @@ location of point."
>> (org-tbl-calc-modes (copy-sequence org-calc-default-modes))
>> (numbers nil)   ; was a variable, now fixed default
>> (keep-empty nil)
>> -   n form form0 formrpl formrg bw fmt x ev orig c lispp literal
>> +   form form0 formrpl formrg bw fmt ev orig lispp literal
>> duration duration-output-format)
>>;; Parse the format string.  Since we have a lot of modes, this is
>>;; a lot of work.  However, I think calc still uses most of the time.
>> -  (if (string-match ";" formula)
>> -  (let ((tmp (org-split-string formula ";")))
>> -(setq formula (car tmp)
>> -  fmt (concat (cdr (assoc "%" org-table-local-parameters))
>> -  (nth 1 tmp)))
>> +  (if (string-match "\\(.*\\);\\(.*\\)" formula)
>> +  (progn
>> +(setq fmt (concat (match-string-no-properties 2 formula)
>> +  (cdr (assoc "%" org-table-local-parameters)))
>> +  formula (match-string-no-properties 1 formula))
> 
> This patch's changes look good to me.  As a minor comment, I'd prefer if
> the rewritten parts (here and for the entire series) only used one pair
> per setq call, even if it's not worth updating the entire function to
> use this style.

Funny. This is also the style I prefer, but I wrote the code to conform
to the style used in this context. I can fix this too.

>> Subject: [PATCH 3/4] org-table: Add mode flag to enable Calc units
>>  simplification mode
>>
>> * org-table.el (org-table-eval-formula): Add the `u` mode flag to
>> enable Calc's units simplification 

Re: [PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

On 19/11/2020 06:58, Kyle Meyer wrote:
> Daniele Nicolodi writes:
> 
>> Hello,
>>
>> I don't think this is what is holding up review of these patches, but, I
>> recently completed the paperwork for copyright assignment to the FSF.
> 
> Thanks for this series (and thanks to Eric for the feedback in the
> previous thread).  I'm sorry for the slow response.
> 
> Quickly scanning through and playing around with it, it looks nicely
> done.  I'll put aside some time this weekend to give it a closer review.

Thanks for having a look, Kyle.

Cheers,
Dan

Re: Ignored bugs

[Daniele Nicolodi]

On 17/11/2020 21:15, Boruch Baum wrote:
> On 2020-11-17 20:51, Daniele Nicolodi wrote:
>> Please complain with the Emacs maintainers about their handling of the
>> bugs on the Emacs bug tracking system,
> 
> How do I know it was "their handling" and not "your handling"? It's
> unclear from my view of the thread who performed the actions but all the
> other actions on that thread were from a fellow called "Bastien" - is he
> one of "theirs" or one of "yours" (or both)?
> 
> In any event, it should be of concern to people on this list to make
> contributing to org-mode as open and hurdle-free as possible instead of
> piling on supplemental demands / requirements. As a case study, you
> can review the submissions that I referenced and the elisp content there
> that seems would have gone lost had I not taken the extra steps of
> engaging on this list. Who knows how many other contributions and
> contributors you've "lost" this way...

The page that introduces the Emacs bug tracker already reports that this
mailing list is the right place where to report Org bugs (or feature
request, as is the case for at least one of the bug you reported), and
not the Emacs bug tracker.

I went and read the exchange on bug #42484 and already Bastien told you
that this mailing list is the right place where to report bugs and
submit patches.

You are deliberately ignoring these instructions. What do you expect?

Would you like to elaborate on why making it easy for you to contribute
something to org-mode that solves your problems should be of concerns to
the people of this list, especialy if this comes at extra cost to them?

Cheers,
Dan

Re: Ignored bugs

[Daniele Nicolodi]

On 17/11/2020 19:41, Boruch Baum wrote:
> A little over a month ago, I submitted several bug reports related to
> org-mode, and sor most of them haven't seen any action whatsoever, not
> even an indication that they were noticed. In many cases, I provided
> either a total solution or a suggested implementation of a solution.
> 
> BUG REPORT: 42484

I haven't investigated further than reading this email, but the fact
that you are including what look like debbug numbers suggests that you
reported those bugs against Emacs.

Org is mostly developed outside Emacs and has an independent release
cycle. Emacs syncs irregularly with the latest released version of Org.

Bug report affecting the latest release of Org can be submitted to this
mailing list. This is also the best way to make them known to the Org
developers and maintainer. I don't think anyone here really looks into
bugs reported to the Emacs debbugs instance: time is limited and the
version of Org distributed with Emacs is very often not the most current
one, thus the bug reports may anyhow not be relevant anyhow.

I suggest you to try to reproduce these bugs with the latest Org release
(or the latest snapshot from Org ELPA) and report on this list if the
problems persist, see `org-submit-bug-report'.

Please complain with the Emacs maintainers about their handling of the
bugs on the Emacs bug tracking system, I don't think it a discussion
relevant to this mailing list.

Cheers,
Dan

Re: [PATCH] doc/org-manual.org: Extend table formulas Lisp form documentation

[Daniele Nicolodi]

On 16/11/2020 11:25, Eric S Fraga wrote:
> Daniele,
> 
> this looks good.  One minor pedantic point: I think you mean
> "interpreted" when you say "interpolated" (several times in the
> text).  Otherwise, this is a very useful addition to the manual.

Thank you for reading and for the comment.

"interpolated" looks strange to me in this context too, but it is the
word that is currently used in the manual. I decided to stick to this
term for consistency, however, I haven't check if it is used with the
same meaning elsewhere.

I don't think it is wrong to use "interpolated", but if you thing it
should be changed I can change it and check the manual for consistency.
However, I don't think "interpreted" is the right word either. Probably
"replaced" or "substituted" are better choices in this context.

Cheers,
Dan

[PATCH] doc/org-manual.org: Extend table formulas Lisp form documentation

[Daniele Nicolodi]

Hello,

I always found the description of Lisp forms in Org table formulas not
extremely clear, especially in regard to the use of mode flags. The
attached patch tries to clarify the manual a bit.

Would it be worth to mention org-sbe in the same section of the manual?

Cheers,
Dan
From aad5ba2217c30ff219069afbfdd8c65851f899d2 Mon Sep 17 00:00:00 2001
From: Daniele Nicolodi 
Date: Sat, 14 Nov 2020 18:33:41 +0100
Subject: [PATCH] doc/org-manual.org: Extend table formulas Lisp form
 documentation

Be more explicit about how fields are interpolated into the Lisp
forms, clarify the use of mode flags, and add a cuple more examples.
---
 doc/org-manual.org | 60 +++---
 1 file changed, 36 insertions(+), 24 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 040fccc21..208e53023 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -2165,38 +2165,50 @@ It is also possible to write a formula in Emacs Lisp.  
This can be
 useful for string manipulation and control structures, if Calc's
 functionality is not enough.
 
-If a formula starts with a single-quote followed by an opening
-parenthesis, then it is evaluated as a Lisp form.  The evaluation
-should return either a string or a number.  Just as with Calc
-formulas, you can specify modes and a ~printf~ format after
-a semicolon.
+A formula is evaluated as a Lisp form when it starts with a
+single-quote followed by an opening parenthesis.  Cell table
+references are interpolated into the Lisp form before execution.  The
+evaluation should return either a string or a number.  Evaluation
+modes and a ~printf~ format used to render the returned values can be
+specified after a semicolon.
 
-With Emacs Lisp forms, you need to be conscious about the way field
-references are interpolated into the form.  By default, a reference is
-interpolated as a Lisp string (in double-quotes) containing the field.
-If you provide the =N= mode switch, all referenced elements are
-numbers---non-number fields will be zero---and interpolated as Lisp
-numbers, without quotes.  If you provide the =L= flag, all fields are
-interpolated literally, without quotes.  For example, if you want a
-reference to be interpreted as a string by the Lisp form, enclose the
-reference operator itself in double-quotes, like ="$3"=.  Ranges are
-inserted as space-separated fields, so you can embed them in list or
-vector syntax.
+By default, references are interpolated as literal Lisp strings: the
+field content is replaced in the Lisp form stripped of leading and
+trailing white space and surrounded in double-quotes.  For example:
 
-Here are a few examples---note how the =N= mode is used when we do
-computations in Lisp:
+: '(concat $1 $2)
 
-- ='(concat (substring $1 1 2) (substring $1 0 1) (substring $1 2))= ::
+concatenates the content of columns 1 and column 2.
 
-  Swap the first two characters of the content of column 1.
+When the =N= flag is used, all referenced elements are parsed as
+numbers and interpolated as Lisp numbers, without quotes. Fields that
+cannot be parsed as numbers are interpolated as zeros.  For example:
 
-- ='(+ $1 $2);N= ::
+: '(+ $1 $2);N
 
-  Add columns 1 and 2, equivalent to Calc's =$1+$2=.
+adds columns 1 and 2, equivalent to Calc's =$1+$2=.  Ranges are
+inserted as space-separated fields, so they can be embedded in list or
+vector syntax. For example:
 
-- ='(apply '+ '($1..$4));N= ::
+: '(apply '+ '($1..$4));N
 
-  Compute the sum of columns 1 to 4, like Calc's =vsum($1..$4)=.
+computes the sum of columns 1 to 4, like Calc's =vsum($1..$4)=.
+
+When the =L= flag is used, all fields are interpolated literally: the
+cell content is replaced in the Lisp form stripped of leading and
+trailing white space and without quotes.  If a reference is intended
+to be interpreted as a string by the Lisp form, the reference operator
+itself should be enclosed in double-quotes, like ="$3"=.  The =L= flag
+is useful when strings and numbers are used in the same Lisp form.  For
+example:
+
+: '(substring "$1" $2 $3);L
+
+extracts the part of the string in column 1 between the character
+positions specified in the integers in column 2 and 3 and it is easier
+to read than the equivalent:
+
+: '(substring $1 (string-to-number $2) (string-to-number $3))
 
 *** Durations and time values
 :PROPERTIES:
-- 
2.29.2

Re: How is org-sbe supposed to work?

[Daniele Nicolodi]

On 13/11/2020 15:39, Daniele Nicolodi wrote:
> How are variables passed to the code block supposed to be handled? The
> macro docstring does not mention anything particular, thus I would
> imagine they are handled just like in any other lisp code. However, this
> does not seem to be the case.

After some more testing and starring at the code some more, it seems
that the odd behavior comes from the attempt of org-sbe to be clever and
do not require to use the N flag in the org table formula definition to
parse numbers as such, maybe.

Anyway, I came up with an alternative macro that does what I would
expect org-sbe to do, acting as a transparent interface to execute org
code blocks, without surprises:

(defun org-sbx1 (name header args)
  (let* ((args (mapconcat
(lambda (x)
  (format "%s=%S" (symbol-name (car x)) (cadr x)))
args ", "))
 (ctx (list 'babel-call (list :call name
  :name name
  :inside-header header
  :arguments args
  :end-header ":results silent")))
 (info (org-babel-lob-get-info ctx)))
(when info (org-babel-execute-src-block nil info

(defmacro org-sbx (name &rest args)
  (let* ((header (if (stringp (car args)) (car args) nil))
 (args (if (stringp (car args)) (cdr args) args)))
(unless (stringp name)
  (setq name (symbol-name name)))
(let ((result (org-sbx1 name header args)))
  (org-trim (if (stringp result) result (format "%S" result))

which I find significantly simpler than the implementation of org-sbe.
It works emulating what a #+call: directive does constructing by hand
the parsed version returned by org-element-parse. Demo:

#+name: sum
#+begin_src elisp :var x='(2 3)
  (apply '+ x)
#+end_src

#+name: concat
#+begin_src elisp :var x='()
  (apply 'concat x)
#+end_src

| 1 | 2 | 3 | 4 | 5 |15 |15 |
| a | b | c | d | e | abcde | abcde |
#+TBLFM: @1$6='(org-sbx sum (x '($1..$5)));N
#+TBLFM: @1$7='(apply '+ '($1..$5));N
#+TBLFM: @2$6='(org-sbx concat (x '($1..$5)))
#+TBLFM: @2$7='(apply 'concat '($1..$5))


The only problem I have with it is that the point jumps around during
evaluation and that while the macro is evaluated the formula definition
line is duplicated. I haven't yet investigated why (however the same
happens with org-sbe).

It may be that it is not too difficult to avoid having to convert the
arguments to a string representation and back, but this would require
digging a bit deeper into ob-core.el.

Cheers,
Dan

How is org-sbe supposed to work?

[Daniele Nicolodi]

Hello,

I am trying to use the org-sbe macro to execute a code block with input
from an org table, however it does not work as I would expect it to work
and the code is cryptic enough for me to not being able to understand
what the intent was when it was written. I searched for tests in the
codebase that exercise this functionality, but I haven't found any:

daniele@black:~/src/org-mode$ grep org-sbe * -r
lisp/ob-table.el:;; `org-sbe' as so...
lisp/ob-table.el:;; #+TBLFM: $2='(org-sbe "fibbd" (n $1))
lisp/ob-table.el:(defmacro org-sbe (source-block &rest variables)
lisp/ob-table.el:So this `org-sbe' construct
lisp/ob-table.el: (org-sbe \"source-block\" (n $2) (m 3))
lisp/ob-table.el:#+TBLFM: @1$4=\\='(org-sbe test-sbe $3 (x $1) (y $2))"

How are variables passed to the code block supposed to be handled? The
macro docstring does not mention anything particular, thus I would
imagine they are handled just like in any other lisp code. However, this
does not seem to be the case.

In the case that the values are integers, all works as expected:

#+name: example
#+begin_src emacs-lisp :var x=1
(format "x:%S" x)
#+end_src

#+call: example(x=2)

#+RESULTS:
: x:2

| 1 | x:2 |
#+TBLFM: $2='(org-sbe "example" (x 2))

However, strings are not handled correctly:

#+call: example(x="baz")

#+RESULTS:
: x:"baz"

| 1 | #ERROR |
#+TBLFM: $2='(org-sbe "example" (x "baz"))

unless "double quoted":

| 1 | x:"baz" |
#+TBLFM: $2='(org-sbe "example" (x "\"baz\""))

and becomes an exercise in code injection if the string needs to come
from a table cell:

| baz | x:"baz" | x:"baz" |
#+TBLFM: $2='(org-sbe "example" (x \"$1\"));L
#+TBLFM: $3='(format "x:%S" $1)

Things get even more fun if the string value is the name of a code block:

#+name: foo
#+begin_src emacs-lisp :var x=1
nil
#+end_src

#+call: example(x="foo")

#+RESULTS:
: x:"foo"

| 1 |  |
#+TBLFM: $2='(org-sbe "example" (x "foo"))

as in this case, org-sbe actually tries to execute that code block
instead than using the literal string.

Should we come up with some tests highlighting the expected behavior of
org-sbe and make the implementation work accordingly?

Thank you.

Cheers,
Dan

Re: official orgmode parser

[Daniele Nicolodi]

On 11/11/2020 10:15, Bastien wrote:
> Hi Daniele,
> 
> Daniele Nicolodi  writes:
> 
>> Would it make sense to have one "official" (or a set of) org-mode test
>> files and the corresponding syntax tree as parsed by org-elements (maybe
>> in a format easier to read from other programming languages than
>> s-expressions, json maybe?) to make testing other parser against the
>> reference implementation easier?
> 
> I think it is a very good idea.
> 
> The example file would be also good to help users track for small
> syntactic changes, when they happen.
> 
> Would you like to work on such a file?

I don't have enough motivation to see this climb high enough in my TODO
list to see any meaningful progress in a reasonable time frame.  I am
mote than happy to contribute to Org, but it is more effective to keep
these contributions related to my daily use of Org.

Cheers,
Dan

Caching answer to prompt for code block execution

[Daniele Nicolodi]

Hello,

I am trying to use the org-sbe macro to compute values in an org table.
This results in a (well, actually two, see my other recent email to the
mailing list) prompt for execution of the relevant code block. This is
unpractical when there are more than an handful of cells to be evaluated.

I have a couple of ideas for improvement.

First, I think prompting only once for code blocks evaluated multiple
times by org-sbe would make a lot of sense and would make the user
experience much better.

Second, I think it would be nice to have a way to cache the answer to
the code block execution prompt. This could be implemented caching
somewhere the answer keyed with an hash of the content of the code
block. This should not require many changes to the code. The only thing
that is not obvious is how to expire the cache to do not have it grow
unbound.

What do others think of these approaches?

Cheers,
Dan

Re: Thoughts on the standardization of Org

[Daniele Nicolodi]

On 09/11/2020 15:04, Maxim Nikulin wrote:
> 2020-11-08 Jean Louis wrote:
>> That is right, I am using it since years in ~/.mailcap that works well
>> for mutt email client.
>>
>> text/org;emacsclient %s; nametemplate=%s.org;
>> text/x-org;  emacsclient %s; nametemplate=%s.org;
> 
> Just for curiosity, couldn't it lead to execution of arbitrary code 
> placed into elisp table expressions, some macro, etc.? I have not 
> convinced myself that just opening of a file (without executing of src 
> blocks) is safe enough and there no dangerous #+startup options or other 
> tricks. Emacs is too powerful and too flexible...

As far as I know it is not any more dangerous than opening any other
file with Emacs. Why do you think the fact that the file extension is
".org" instead of something else makes a relevant difference?

Cheers,
Dan

org-sbe results in two prompts for each evaluated cell

[Daniele Nicolodi]

Hello,

using the org-sbe macro to evaluate a code block with table cells as
input results in two prompts for each cell evaluated: one to confirm the
evaluation of the code block passed as an argument to org-sbe and one to
confirm evaluation of the code block dynamically generated by org-sbe.

Is this by design or is it an unwanted consequence of how org-sbe is
implemented? I think the second prompt is redundant, not very useful,
and actually confusing because, unless they go digging to see how
org-sbe is implemented, the user is not aware that another hidden code
block is involved in the computation.

Would a patch that removes the second prompt be something desirable?

Thank you.

Cheers,
Dan

Re: [PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

Hello,

I don't think this is what is holding up review of these patches, but, I
recently completed the paperwork for copyright assignment to the FSF.

Cheers,
Dan


On 24/10/2020 17:33, Daniele Nicolodi wrote:
> Hello,
> 
> attached there are a few patches reworking the parsing of org-table
> formula mode strings and introducing a new mode flag to enable Calc's
> units simplification mode as discussed in a recent thread on the mailing
> list. I haven't updated the documentation. I can do it once we agree
> that this feature is a good idea.
> 
> I have already submitted the patches in another thread
> https://orgmode.org/list/6d8c15c2-d1b0-d913-df39-c60381cff...@grinta.net/T/#m03a426dd8476b60019dfffecb8781a2126df690f
> but it seems that woof did not pick them up, thus I am re-sending them.
> 
> Cheers,
> Dan
> 

Re: Thoughts on the standardization of Org

[Daniele Nicolodi]

On 02/11/2020 10:02, TEC wrote:
> I think there are absolutely some benefits for Org users. I am
> personally interested in registering Org as an IANA MIME type.

I don't think that registering Org as IANA MIME type will have the
consequences you hope it has.

> What will this do? Well, for starters I'd like to be able to 
> attach org
> files without the type being recognised as
> "application/vnd.lotus-organizer" 🤢.

This is not governed by the presence of an MIME type for Org, but by the
mechanism that assigns a MIME type to the file when it is attached tp a
message, and this is independent of the IANA MIME type database.

You (most probably) already can configure your email client (or
operative system) to use "text/x-org" as the mime type for files with
extension ".org". Having a IANA MIME type, would only make using
"text/org" a bit more "right" (there is nothing that stops you from
using the "text/org" mime type without it being registered).

If you want the association to be automatic, you need to convince the
maintainers of the MIME type association databases to change their
defaults. And this is a rather long list of people you have to interact
with and convince that "text/org" is a better choice than
"application/vnd.lotus-organizer" as the default MIME type for ".org" files.

Finally, even if you get your attachment automatically tagged as
"text/org", the receiving side needs to have a mime type handler
configured to display it. As far as I know, not even Emacs (on the
platforms that allow it) registers itself as an handler for any MIME
type. Therefore, what you get, assuming that the mail client on the
other side behaves correctly and uses "text/*" as a fallback for
"text/org" is that your attachment will be displayed in a generic text
editor.

> I also think it's to our benefit that non-Emacsers become more
> comfortable with seeing an org file --- as I see it that improves 
> our
> chances that we can directly share Org files with them, which they 
> might
> be comfortable editing and sending back for example, or that a 
> generic
> tool might think to support Org files.

I send Org files as "text/plain" (often even using ".txt" extension to
avoid confusion on the receiving side) and I think this is the best
choice as it puts the least burden on the receiving side to consume the
content and it is displayed inline by most email clients.

> So I'd like to assure you that my interest in improving 
> recognition and
> support for Org is motivated by selfish reasons 😛 which just so 
> happen
> to potentially benefit non-Emacsers.

I don't think that registering "text/org" with the IANA will have the
consequences that you hope it has.

Cheers,
Dan

Re: Thoughts on the standardization of Org

[Daniele Nicolodi]

On 02/11/2020 00:10, Dr. Arne Babenhauserheide wrote:
> 
> Daniele Nicolodi  writes:
>> Maybe the standardization should cover only the "static" parts of Org
>> (ie no table formulas, no babel, no agenda, no exporters, etc). However,
>> in this case, what is left is little more of a markup language with an
>> editor that allows sections folding. You can have this on top of pretty
>> much any markup language using Emacs' outline-minor-mode.
> 
> It could become stronger competition for asciidoc by being available in
> more places.

Why does Org need to compete with asciidoc? I don't see any advantage in
fighting with anyone for market share.

> Having an acceptance criterion for “supports basic org-mode
> presentation” and “can edit org-files without breaking editing in
> org-mode” could help adoption.

Acceptance criterion for what? Adoption of what?

It seems to me that some see a the adoption of a simplified version of
the Org markup language outside Emacs and the org-mode implementation as
something desirable. However, I don't see what the Org community would
gain from that.

> That would be the only part I’d really expect from standardization:
> There would be a clear-cut point when a tool could claim compatibility
> with org level N or by components (i.e. basic presentation, code-blocks,
> …).
> 
> Having org-files parsed as html on a VCS-infopage is pretty nice.

As explained many times now, you don't a formal specification for this:
the specification is the org-mode implementation itself.

However, I will not discourage anyone from working on some form of
standardization, other than pointing out that IMO it is an exercise with
very limited usefulness, impact and future.

Cheers,
Dan

Re: Thoughts on the standardization of Org

[Daniele Nicolodi]

On 01/11/2020 17:13, Russell Adams wrote:
> On Sat, Oct 31, 2020 at 08:22:01PM -0400, Asa Zeren wrote:
>> First, I would like to repeat the importance of developing standards
>> for org-mode. If we want to expand the influence of org, tooling must
>> expand beyond Emacs.
> 
> I disagree. There are other open text based formats outside of
> Emacs. That Org is so compelling is because it's tightly integrated
> into an Emacs mode which makes using Org data so easy.

I cannot agree more with this statement and a similar statement (that I
am too lazy to go search to provide correct authorship) that stated that
one of the advantages of Org is that, being implemented in Emacs, it has
infinite potential for customization, thus we would need to agree about
what org mode is before standardizing it: my Org is not your org, and,
thank to the features offered be the Emacs environment, I use different
flavors of Org in different buffers.

Reading the thread I have the impression that what wants to be
standardized is the syntax of Org, but that is not very different from
the syntax of other text based markup languages such at
reStructuredText, asciidoc, Markdown, or others. What makes Org stand
out, for some applications, is "org-mode" (as the implementation of Org
in Emacs) and the tools built on top of the syntax.

I don't think it is reasonable to expect much of org-mode being
implemented in another environment, because that quickly becomes a task
as complex as implementing Emacs. For example, Org has org-tables, and
the formula syntax allows for Emacs Calc expressions or Elisp functions
to be called: should an hypothetical implementation of Org allow the
same formulas to be executed? Wouldn't that mean that this
implementation needs to re-implement a good fraction of Emacs (or use
Emacs itself for interpreting the formulas?

Maybe the standardization should cover only the "static" parts of Org
(ie no table formulas, no babel, no agenda, no exporters, etc). However,
in this case, what is left is little more of a markup language with an
editor that allows sections folding. You can have this on top of pretty
much any markup language using Emacs' outline-minor-mode.

If this is what you are after, I think a much more interesting goal
(from the point of view of Emacs users and Org developers) would be to
fold back some of the improvements org-mode builds on top of
outline-mode back into outline-mode itself. This would immediately be
beneficial as it would probably make Org simpler and the improvements
could be actually be used by many.

I also think that, if you are in the position to choose a format to use
for collaborative projects, reStructuredText may be a better choice as
the syntax is simple, well defined, and extensible. Although, without
trying I don't know if outline-minor-mode can be made to work with
reST-style sections headers. But, if it does not, it should not be hard
to adjust it (see previous paragraph).

Cheers,
Dan

Re: Bug: headline folding broken by whitespace

[Daniele Nicolodi]

On 31/10/2020 17:49, Kyle Meyer wrote:
> Daniele Nicolodi writes:
> 
>> Hello,
>>
>> i saw a few reports of bugs related to folding and unfolding of
>> headlines recently, but I think this is not a duplicate.
> 
> I think you're referring to
> 
> https://orgmode.org/list/87h7qu10ht@eml.cc
> 
> https://orgmode.org/list/CAJj=8EFJXFu9ZQ11cO24AAu=1jykW2Zr+=R4FLuKG=bljfu...@mail.gmail.com
> 
> The cases reported there are resolved by e8070d71a (Revert "org.el:
> Allow empty subtrees to be folded back", 2020-10-26).
> 
>> I just noticed that with the latest release (Org mode version 9.4
>> (9.4-27-g47bf60-elpa @ /Users/daniele/.emacs.d/elpa/org-20201026/)
>> unfolding of headlines is broken by whitespace. In particular with a
>> buffer with this content:
>>
>> * Foo
>> ** Bar
>>bar bar bar+49 1520 7178926
>>
>>
>> * Baz
>>
>> where the line before "* Baz" contains a single whitespace character,
>> the headline "* Foo" can be folded but cannot be unfolded. Removing the
>> whitespace while the headline is folded fixes the problem.
> 
> Yep, with your test case (making sure to add a space on the line before
> Baz), I can trigger the issue on your reported version (47bf6005b), and
> it goes away with e8070d71a.  So, it looks like the same underlying
> problem and should go away with the next elpa bump.

This is funny. I have two emacs instances running: one with my
customizations and an emacs -Q one. The first exhibits the issue, the
second does not, matching what Russel reports.

By the way, how can I get a dump of org configuration similar to the one
org-submit-bug-report produces?

Cheers,
Dan

Re: Bug: headline folding broken by whitespace

[Daniele Nicolodi]

On 31/10/2020 17:28, Russell Adams wrote:
> On Sat, Oct 31, 2020 at 04:06:22PM +0100, Daniele Nicolodi wrote:
>> I just noticed that with the latest release (Org mode version 9.4
>> (9.4-27-g47bf60-elpa @ /Users/daniele/.emacs.d/elpa/org-20201026/)
>> unfolding of headlines is broken by whitespace. In particular with a
>> buffer with this content:
>>
>> * Foo
>> ** Bar
>>bar bar bar+49 1520 7178926
>>
>>
>> * Baz
>>
>> where the line before "* Baz" contains a single whitespace character,
>> the headline "* Foo" can be folded but cannot be unfolded. Removing the
>> whitespace while the headline is folded fixes the problem.
> 
> I'm on Org mode version 9.4 (9.4-27-g47bf60-elpa) and tested, cannot
> reproduce. Do you have something conflicting, or are you running an
> Emacs distribution (ie: Doom, Space)?

Thank you for trying to reproduce the issue. Indeed if I try to
reproduce it with an Emacs without my customizations it works just fine.

I'm using Emacs 27.1 from here https://emacsformacosx.com/ I'm surprised
something in my configuration breaks this: I don't use anything fancy.

Cheers,
Dan

Bug: headline folding broken by whitespace

[Daniele Nicolodi]

Hello,

i saw a few reports of bugs related to folding and unfolding of
headlines recently, but I think this is not a duplicate.

I just noticed that with the latest release (Org mode version 9.4
(9.4-27-g47bf60-elpa @ /Users/daniele/.emacs.d/elpa/org-20201026/)
unfolding of headlines is broken by whitespace. In particular with a
buffer with this content:

* Foo
** Bar
   bar bar bar+49 1520 7178926


* Baz

where the line before "* Baz" contains a single whitespace character,
the headline "* Foo" can be folded but cannot be unfolded. Removing the
whitespace while the headline is folded fixes the problem.

Cheers,
Dan

Re: Inconsistency between code and manual: org-lowest-priority or org-priority-lowest

[Daniele Nicolodi]

On 30/10/2020 05:57, Kyle Meyer wrote:
> Daniele Nicolodi writes:
> 
>> Hello,
>>
>> I found an inconsistency between the manual and implementation: the
>> Priorities section says that the range of valid priorities can be set
>> modifying the `org-priority-highest', `org-priority-lowest' and
>> `org-priority-default' variables. However, in the code the variables
>> names are `org-highest-priority', `org-lowest-priority' and
>> `org-default-priority'.
>>
>> In can submit a patch with a fix. However, in a case like this, should
>> the code modified to match the documentation, or vice-versa?
> 
> The org-X-priority -> org-priority-X rename happened in v9.4, with
> org-X-priority names retained as aliases.  So, it sounds like there are
> some leftover bits in the code.

You are absolutely right. This is what you get when you read the manual
for the latest version but look at the code for an old one...

Cheers,
Dan

Inconsistency between code and manual: org-lowest-priority or org-priority-lowest

[Daniele Nicolodi]

Hello,

I found an inconsistency between the manual and implementation: the
Priorities section says that the range of valid priorities can be set
modifying the `org-priority-highest', `org-priority-lowest' and
`org-priority-default' variables. However, in the code the variables
names are `org-highest-priority', `org-lowest-priority' and
`org-default-priority'.

In can submit a patch with a fix. However, in a case like this, should
the code modified to match the documentation, or vice-versa?

Thank you.

Cheers,
Dan

Re: New website - back to the old unicorn!

[Daniele Nicolodi]

On 28/10/2020 17:59, TEC wrote:
> 
> Let's do two at once:
> 
> Daniele Nicolodi  writes:
> 
>> Great work! it looks very nice and informative at the same time.
> That's the idea! :D
> 
>> If I can bikeshed a bit more: I like the lighter page background 
>> that is
>> currently on http://orgomode.org more than the darker one in the 
>> new
>> version.
> 
> See the current. It's slightly tweaked, but you're not getting 
> #fff :P

I like this version. Thank you.

>> 2 and more important) Use the most popular language to get the 
>> most people interested. There are probably a lot more Python 
>> programmers than shell scripters. I suggest the code example 
>> should be a minimal (~6 lines?) of Python that produces either 
>> (or both) a simple graph or ~3 column table.
> 
> Hmmm. This seems sensible. I've tweaked the example. I may make it
> something website-related later (to fit in with the other 
> content).
> Suggestions welcome.

The source block now says that the code is shell, but it looks like
Python to me :-)

Cheers,
Dan

Re: New website - back to the old unicorn!

[Daniele Nicolodi]

On 28/10/2020 13:25, TEC wrote:
> 
> Daniele Nicolodi  writes:
> 
>> - (minor) I would add a background to the example in the home 
>> page to
>> make it stand out more as an example org-mode syntax buffer
>>
>> - (very minor) why does the example on the home page need to be 
>> an SVG
>> file? It would be very cool if it could be copy and pasted, but 
>> right
>> now it is not possible (with Firefox, at least)
> 
> So, I've turned these "minor" comments into a major revamp of the 
> demo
> (that I am inordinately happy with) --- following a brilliant 
> suggestion
> from someone on HN.

Great work! it looks very nice and informative at the same time.

If I can bikeshed a bit more: I like the lighter page background that is
currently on http://orgomode.org more than the darker one in the new
version.

Cheers,
Dan

Re: New website - back to the old unicorn!

[Daniele Nicolodi]

On 26/10/2020 15:26, TEC wrote:
> 
> Daniele Nicolodi  writes:
> 
>> I like the new design. Thank you for your work!
> 
> Thanks for the kind words!
> 
>> Two things:
> *cough, Three 😆

Indeed, I thought about the third one while writing the first two...

>> - in Firefox on a macOS machine, the colophon renders as "Made 
>> with XX
>> by TEC" with the XX being an Unicode replacement character. What 
>> is it
>> supposed to be?
> 
> Currently a brown heart, though I've actually had a better, less 
> tacky
> idea. You'll see it in a day or two ;)

Do you have any idea while it does not seem to be understood by Firefox
on macOS? It seem to be an emoji character... as the unicode replacement
char does not look that nice, what about replacing it with a more
commonly supported gliph? Or an embeeded SVG, maybe?

>> - (minor) I would add a background to the example in the home 
>> page to
>> make it stand out more as an example org-mode syntax buffer
> 
> Hmm, I've got mixed feelings about that. I kinda like how smoothly 
> it
> fits in at the moment, and I wouldn't want to jeopardise that. If 
> we can
> have it fit in smoothly, and more clearly be a example that would 
> be
> great though :)

IMHO a very light gray background would not compromise the design and
make the example stand out as such.

Cheers,
Dan

Re: New website - back to the old unicorn!

[Daniele Nicolodi]

On 26/10/2020 14:54, Daniele Nicolodi wrote:
> On 26/10/2020 11:27, TEC wrote:
>>
>> TEC  writes:
>>
>>> there are a few teething issues that have appeared when 
>>> deploying the
>>> site on orgmode.org.
>>
>> These issues have now been fixed! Go wild :P
>>
>> I've taken the liberty of making a post on reddit:
>> https://www.reddit.com/r/emacs/comments/jic3ww/the_org_website_has_been_revamped/
>>
>> Once again, thanks to everyone who's helped out with this!
>>
>> Timothy.
> 
> I like the new design. Thank you for your work!

By the way, what's the theme used to create the GIFs in the Features page?

Cheers,
Dan

Re: New website - back to the old unicorn!

[Daniele Nicolodi]

On 26/10/2020 11:27, TEC wrote:
> 
> TEC  writes:
> 
>> there are a few teething issues that have appeared when 
>> deploying the
>> site on orgmode.org.
> 
> These issues have now been fixed! Go wild :P
> 
> I've taken the liberty of making a post on reddit:
> https://www.reddit.com/r/emacs/comments/jic3ww/the_org_website_has_been_revamped/
> 
> Once again, thanks to everyone who's helped out with this!
> 
> Timothy.

I like the new design. Thank you for your work!

Two things:

- in Firefox on a macOS machine, the colophon renders as "Made with XX
by TEC" with the XX being an Unicode replacement character. What is it
supposed to be?

- (minor) I would add a background to the example in the home page to
make it stand out more as an example org-mode syntax buffer

- (very minor) why does the example on the home page need to be an SVG
file? It would be very cool if it could be copy and pasted, but right
now it is not possible (with Firefox, at least)

Cheers,
Dan

Re: official orgmode parser

[Daniele Nicolodi]

On 23/09/2020 10:09, Bastien wrote:
> I disagree that a parser is too difficult to maintain because Org is 
> a moving target.  Org core syntax is not moving anymore, a parser can
> reasonably target it.  That's what is done with the Ruby parser, in
> use in this small project called github.com :)

(Just an aside: which Ruby org-mode parser does Github use? I sometime
find instances where Github does not render an org-mode file correclty
and I would be happy to file bugs to have them corrected).

> So I'd say:
> 
> - let's enhance Worg's documentation
> - yes, please go for enhancing parsing tools
> 
> I don't think we need official tools.  The official Org parser exists,
> it is Org itself.

Would it make sense to have one "official" (or a set of) org-mode test
files and the corresponding syntax tree as parsed by org-elements (maybe
in a format easier to read from other programming languages than
s-expressions, json maybe?) to make testing other parser against the
reference implementation easier?

Maybe the org-mode test suite already has something like this. I haven't
looked for it yet.

Cheers,
Dan

[PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

Hello,

attached there are a few patches reworking the parsing of org-table
formula mode strings and introducing a new mode flag to enable Calc's
units simplification mode as discussed in a recent thread on the mailing
list. I haven't updated the documentation. I can do it once we agree
that this feature is a good idea.

I have already submitted the patches in another thread
https://orgmode.org/list/6d8c15c2-d1b0-d913-df39-c60381cff...@grinta.net/T/#m03a426dd8476b60019dfffecb8781a2126df690f
but it seems that woof did not pick them up, thus I am re-sending them.

Cheers,
Dan
From c7434974897d932fe3acd182f06a98a61719e208 Mon Sep 17 00:00:00 2001
From: Daniele Nicolodi 
Date: Tue, 20 Oct 2020 11:03:14 +0200
Subject: [PATCH 1/4] org-table: Fix table formula mode string handling

* lisp/org-table.el (org-table-eval-formula): Move mode lookup table
from org-table--set-calc-mode to here.

* lisp/org-table.el (org-table--set-calc-mode): Use plist-put instead
of the buggy open coded version.
---
 lisp/org-table.el | 24 ++--
 1 file changed, 10 insertions(+), 14 deletions(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 112b1e171..0790dc3ca 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -721,17 +721,8 @@ Field is restored even in case of abnormal exit."
 (org-table-goto-column ,column)
 (set-marker ,line nil)
 
-(defsubst org-table--set-calc-mode (var &optional value)
-  (if (stringp var)
-  (setq var (assoc var '(("D" calc-angle-mode deg)
-("R" calc-angle-mode rad)
-("F" calc-prefer-frac t)
-("S" calc-symbolic-mode t)))
-   value (nth 2 var) var (nth 1 var)))
-  (if (memq var org-tbl-calc-modes)
-  (setcar (cdr (memq var org-tbl-calc-modes)) value)
-(cons var (cons value org-tbl-calc-modes)))
-  org-tbl-calc-modes)
+(defsubst org-table--set-calc-mode (var value)
+  (plist-put org-tbl-calc-modes var value))
 
 
 ;;; Predicates
@@ -2476,9 +2467,14 @@ location of point."
(setq keep-empty t
  fmt (replace-match "" t t fmt)))
(while (string-match "[DRFS]" fmt)
- (setq org-tbl-calc-modes
-   (org-table--set-calc-mode (match-string 0 fmt)))
- (setq fmt (replace-match "" t t fmt)))
+ (let* ((c (string-to-char (match-string 0 fmt)))
+(mode (cdr (assoc c '((?D calc-angle-mode deg)
+  (?R calc-angle-mode rad)
+  (?F calc-prefer-frac t)
+  (?S calc-symbolic-mode t))
+   (setq org-tbl-calc-modes (org-table--set-calc-mode
+ (car mode) (cadr mode))
+ fmt (replace-match "" t t fmt
(unless (string-match "\\S-" fmt)
  (setq fmt nil
   (when (and (not suppress-const) org-table-formula-use-constants)
-- 
2.28.0

From fb8b62e5faabca2b6c6514e25bd306f7a5e8696f Mon Sep 17 00:00:00 2001
From: Daniele Nicolodi 
Date: Tue, 20 Oct 2020 15:13:40 +0200
Subject: [PATCH 2/4] org-table: Simplify mode string parsing

* org-table.el (org-table-eval-formula): Simplify mode string parsing
and reduce scope of local variables.
---
 lisp/org-table.el | 98 +--
 1 file changed, 43 insertions(+), 55 deletions(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 0790dc3ca..4baad2600 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -722,7 +722,7 @@ Field is restored even in case of abnormal exit."
 (set-marker ,line nil)
 
 (defsubst org-table--set-calc-mode (var value)
-  (plist-put org-tbl-calc-modes var value))
+  (setq org-tbl-calc-modes (plist-put org-tbl-calc-modes var value)))
 
 
 ;;; Predicates
@@ -2427,54 +2427,42 @@ location of point."
   (org-tbl-calc-modes (copy-sequence org-calc-default-modes))
   (numbers nil)   ; was a variable, now fixed default
   (keep-empty nil)
-  n form form0 formrpl formrg bw fmt x ev orig c lispp literal
+  form form0 formrpl formrg bw fmt ev orig lispp literal
   duration duration-output-format)
   ;; Parse the format string.  Since we have a lot of modes, this is
   ;; a lot of work.  However, I think calc still uses most of the time.
-  (if (string-match ";" formula)
- (let ((tmp (org-split-string formula ";")))
-   (setq formula (car tmp)
- fmt (concat (cdr (assoc "%" org-table-local-parameters))
- (nth 1 tmp)))
+  (if (string-match "\\(.*\\);\\(.*\\)" formula)
+ (progn
+   (setq fmt (concat (mat

Re: [PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

Hello,

after working on this I realized that the org-tbl-calc-modes variables
is used only locally despite being declare globally. Maybe a remnant
from pre-lexical-binding times. Attached is a patch (on top of the
others one in this thread) that simplifies things a little.

Cheers,
Dan


On 20/10/2020 15:30, Daniele Nicolodi wrote:
> Hello,
> 
> attached there are a few patches reworking the code, fixing the bug, and
> introducing a new mode flag to enable Calc's units simplification mode
> as discussed in a recent thread on the mailing list.  I haven't updated
> the documentation.  I can do it once we agree that this feature is a
> good idea.
> 
> Cheers,
> Dan
> 
> 
> On 19/10/2020 17:38, Daniele Nicolodi wrote:
>> Hello,
>>
>> I am hacking org-table-eval-formula (see thread about monetary values in
>> org-tables) which uses this inline function:
>>
>> (defsubst org-table--set-calc-mode (var &optional value)
>>   (if (stringp var)
>>   (setq var (assoc var '(("D" calc-angle-mode deg)
>>   ("R" calc-angle-mode rad)
>>   ("F" calc-prefer-frac t)
>>   ("S" calc-symbolic-mode t)))
>>  value (nth 2 var) var (nth 1 var)))
>>   (if (memq var org-tbl-calc-modes)
>>   (setcar (cdr (memq var org-tbl-calc-modes)) value)
>> (cons var (cons value org-tbl-calc-modes)))
>>   org-tbl-calc-modes)
>>
>> which I am not able to understand or which is not correct.
>>
>> The first (if ...) does some value substitutions, however, IIUC the
>> second (if ...) sets a new value for an entry in the org-tbl-calc-modes
>> plist if the entry is already present and builds a new plist with the
>> entry prepended if the entry is not there. However, the original plist
>> is returned and not the one with the new entry prepended.
>>
>> It does not seem to be the intended behavior.
>>
>> Shouldn't this be simply:
>>
>> (defsubst org-table--set-calc-mode (var &optional value)
>>   (if (stringp var)
>>   (setq var (assoc var '(("D" calc-angle-mode deg)
>>   ("R" calc-angle-mode rad)
>>   ("F" calc-prefer-frac t)
>>   ("S" calc-symbolic-mode t)))
>>  value (nth 2 var) var (nth 1 var)))
>>   (plist-put org-tbl-calc-modes var value))
>>
>> or, better, the code refactored to do not use this function?
>>
>> Cheers,
>> Dan
>>
> 

From 2d4521a032ec3e4174c97b2b2e9a08491e9870fb Mon Sep 17 00:00:00 2001
From: Daniele Nicolodi 
Date: Wed, 21 Oct 2020 17:47:15 +0200
Subject: [PATCH 4/4] org-table: Remove unused org-tbl-calc-modes variable

* org-table.el (org-tbl-calc-modes): Remove the variable declaration
as the varialble is only used as a local variable in `org-table-eval-formula'.

* org-table.el (org-table--set-calc-mode): Drop convenience macro.

* org-table.el (org-table-eval-formula): Rename `org-tbl-calc-modes`
local variable without the org-table prefix and usr the gained screen
real estate to avoid indirection through covenience macro.
---
 lisp/org-table.el | 29 -
 1 file changed, 12 insertions(+), 17 deletions(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 6b92656bd..1651decd3 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -676,8 +676,6 @@ Will be filled automatically during use.")
 ("_" . "Names for values in row below this one.")
 ("^" . "Names for values in row above this one.")))
 
-(defvar org-tbl-calc-modes nil)
-
 (defvar org-pos nil)
 
 
@@ -721,9 +719,6 @@ Field is restored even in case of abnormal exit."
 (org-table-goto-column ,column)
 (set-marker ,line nil)
 
-(defsubst org-table--set-calc-mode (var value)
-  (setq org-tbl-calc-modes (plist-put org-tbl-calc-modes var value)))
-
 
 ;;; Predicates
 
@@ -2424,7 +2419,7 @@ location of point."
equation
  (org-table-get-formula equation (equal arg '(4)
   (n0 (org-table-current-column))
-  (org-tbl-calc-modes (copy-sequence org-calc-default-modes))
+  (calc-modes (copy-sequence org-calc-default-modes))
   (numbers nil)   ; was a variable, now fixed default
   (keep-empty nil)
   form form0 formrpl formrg bw fmt ev orig lispp literal
@@ -2440,11 +2435,11 @@ location of point."
  (let ((c (string-to-char (match-string 1 fmt)))
(n (string-to-number (match-string 2 fmt
(cl-case c
- (?p (

Re: [PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

On 20/10/2020 16:32, Daniele Nicolodi wrote:
> On 20/10/2020 16:19, Eric S Fraga wrote:
>> Hello again,
>>
>> Following up on myself.  I'm seeing some strange behaviour although unit
>> calculations are working nicely.  For instance, this table:
>>
>> #+begin_src org
>>   | stream   | a| b| c| total|   
>> x_a |   x_b |   x_c |
>>   |  |   |   |   |   |   
>>  |||
>>   
>> |--+--+--+--+--+---+---+---|
>>   | feed | 1.05 mol/s   | 1.05 mol/s   |  | 2.10 mol / s | 
>> 0.500 | 0.500 | 0 |
>>   | effluent | 0.74 mol / s | 0.74 mol / s | 0.32 mol / s | 1.80 mol / s | 
>> 0.411 | 0.411 | 0.178 |
>>   ,#+TBLFM: 
>> $5=vsum($2..$4);uf2::$6=$2/$5;uf3::$7=$3/$5;uf3::$8=$4/$5;uf3::@4$2=(1-0.3)*@-1;uf2::@4$3=(1-0.3)*@-1;uf2::@4$4=@-1+0.3*@-1$-1;uf2
>> #+end_src
>>
>> does not seem to pay attention to the f3 mode in the last column, first
>> data row.
> 
> It is something related to how Calc computes the result. The f2 mode
> specifies the formatting for floating point values, however it seems
> that Calc treats the zero (from the missing value in the fourth column)
> divided by a float (from the value in the fifth column) as an integer
> and not as a float. This may be because the org substitutes a "0" for
> the missing value, thus an integer. Still, I am not sure dividing an
> integer by a float should result in an integer (I guess zero is special
> cased here).
> 
> If you change the formula for that field to:
> 
>   #+TBLFM: $8=$4*1.0/$5;uf3
> 
> to force the $4 field to be evaluated as a float (there are other ways
> to get the same effect) you get the expected result (I think).

There are other funny Calc behaviors: if an expression results in a
number with an unit where the numerical part is exactly 1, the printed
result looses the numerical part and only the units are printed.

Cheers,
Dan

Re: [PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

On 20/10/2020 16:19, Eric S Fraga wrote:
> Hello again,
> 
> Following up on myself.  I'm seeing some strange behaviour although unit
> calculations are working nicely.  For instance, this table:
> 
> #+begin_src org
>   | stream   | a| b| c| total|   
> x_a |   x_b |   x_c |
>   |  |   |   |   |   |   
>  |||
>   
> |--+--+--+--+--+---+---+---|
>   | feed | 1.05 mol/s   | 1.05 mol/s   |  | 2.10 mol / s | 
> 0.500 | 0.500 | 0 |
>   | effluent | 0.74 mol / s | 0.74 mol / s | 0.32 mol / s | 1.80 mol / s | 
> 0.411 | 0.411 | 0.178 |
>   ,#+TBLFM: 
> $5=vsum($2..$4);uf2::$6=$2/$5;uf3::$7=$3/$5;uf3::$8=$4/$5;uf3::@4$2=(1-0.3)*@-1;uf2::@4$3=(1-0.3)*@-1;uf2::@4$4=@-1+0.3*@-1$-1;uf2
> #+end_src
> 
> does not seem to pay attention to the f3 mode in the last column, first
> data row.

It is something related to how Calc computes the result. The f2 mode
specifies the formatting for floating point values, however it seems
that Calc treats the zero (from the missing value in the fourth column)
divided by a float (from the value in the fifth column) as an integer
and not as a float. This may be because the org substitutes a "0" for
the missing value, thus an integer. Still, I am not sure dividing an
integer by a float should result in an integer (I guess zero is special
cased here).

If you change the formula for that field to:

  #+TBLFM: $8=$4*1.0/$5;uf3

to force the $4 field to be evaluated as a float (there are other ways
to get the same effect) you get the expected result (I think).

> I've also seen some (difficult to replicate) problem with column widths
> where the columns are much wider than the expected.  I'll keep playing
> to see if I can isolate the column width behaviour.

I haven't touched any code dealing with columns width (I believe).

Cheers,
Dan

Re: [PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

On 20/10/2020 15:44, Eric S Fraga wrote:
> Just to say that I have done a quick test with this and I really like
> it.  Simple calculations with units of mol/s and mol/min work perfectly.

Thank you for testing Eric.

To cover the use case of monetary quantities discussed in the other
thread I would like to also add an `m?` (or M?) flag that can combine
the effect of the existing `f?` flag and the new `u` flag. But I don't
know if sparing one character is worth one character in the small flag
characters set.

Cheers,
Dan

[PATCH] org-table: Add mode flag to enable Calc units simplification mode

[Daniele Nicolodi]

Hello,

attached there are a few patches reworking the code, fixing the bug, and
introducing a new mode flag to enable Calc's units simplification mode
as discussed in a recent thread on the mailing list.  I haven't updated
the documentation.  I can do it once we agree that this feature is a
good idea.

Cheers,
Dan


On 19/10/2020 17:38, Daniele Nicolodi wrote:
> Hello,
> 
> I am hacking org-table-eval-formula (see thread about monetary values in
> org-tables) which uses this inline function:
> 
> (defsubst org-table--set-calc-mode (var &optional value)
>   (if (stringp var)
>   (setq var (assoc var '(("D" calc-angle-mode deg)
>("R" calc-angle-mode rad)
>("F" calc-prefer-frac t)
>("S" calc-symbolic-mode t)))
>   value (nth 2 var) var (nth 1 var)))
>   (if (memq var org-tbl-calc-modes)
>   (setcar (cdr (memq var org-tbl-calc-modes)) value)
> (cons var (cons value org-tbl-calc-modes)))
>   org-tbl-calc-modes)
> 
> which I am not able to understand or which is not correct.
> 
> The first (if ...) does some value substitutions, however, IIUC the
> second (if ...) sets a new value for an entry in the org-tbl-calc-modes
> plist if the entry is already present and builds a new plist with the
> entry prepended if the entry is not there. However, the original plist
> is returned and not the one with the new entry prepended.
> 
> It does not seem to be the intended behavior.
> 
> Shouldn't this be simply:
> 
> (defsubst org-table--set-calc-mode (var &optional value)
>   (if (stringp var)
>   (setq var (assoc var '(("D" calc-angle-mode deg)
>("R" calc-angle-mode rad)
>("F" calc-prefer-frac t)
>("S" calc-symbolic-mode t)))
>   value (nth 2 var) var (nth 1 var)))
>   (plist-put org-tbl-calc-modes var value))
> 
> or, better, the code refactored to do not use this function?
> 
> Cheers,
> Dan
> 

From c7434974897d932fe3acd182f06a98a61719e208 Mon Sep 17 00:00:00 2001
From: Daniele Nicolodi 
Date: Tue, 20 Oct 2020 11:03:14 +0200
Subject: [PATCH 1/3] org-table: Fix table formula mode string handling

* lisp/org-table.el (org-table-eval-formula): Move mode lookup table
from org-table--set-calc-mode to here.

* lisp/org-table.el (org-table--set-calc-mode): Use plist-put instead
of the buggy open coded version.
---
 lisp/org-table.el | 24 ++--
 1 file changed, 10 insertions(+), 14 deletions(-)

diff --git a/lisp/org-table.el b/lisp/org-table.el
index 112b1e171..0790dc3ca 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -721,17 +721,8 @@ Field is restored even in case of abnormal exit."
 (org-table-goto-column ,column)
 (set-marker ,line nil)
 
-(defsubst org-table--set-calc-mode (var &optional value)
-  (if (stringp var)
-  (setq var (assoc var '(("D" calc-angle-mode deg)
-("R" calc-angle-mode rad)
-("F" calc-prefer-frac t)
-("S" calc-symbolic-mode t)))
-   value (nth 2 var) var (nth 1 var)))
-  (if (memq var org-tbl-calc-modes)
-  (setcar (cdr (memq var org-tbl-calc-modes)) value)
-(cons var (cons value org-tbl-calc-modes)))
-  org-tbl-calc-modes)
+(defsubst org-table--set-calc-mode (var value)
+  (plist-put org-tbl-calc-modes var value))
 
 
 ;;; Predicates
@@ -2476,9 +2467,14 @@ location of point."
(setq keep-empty t
  fmt (replace-match "" t t fmt)))
(while (string-match "[DRFS]" fmt)
- (setq org-tbl-calc-modes
-   (org-table--set-calc-mode (match-string 0 fmt)))
- (setq fmt (replace-match "" t t fmt)))
+ (let* ((c (string-to-char (match-string 0 fmt)))
+(mode (cdr (assoc c '((?D calc-angle-mode deg)
+  (?R calc-angle-mode rad)
+  (?F calc-prefer-frac t)
+  (?S calc-symbolic-mode t))
+   (setq org-tbl-calc-modes (org-table--set-calc-mode
+     (car mode) (cadr mode))
+ fmt (replace-match "" t t fmt
(unless (string-match "\\S-" fmt)
  (setq fmt nil
   (when (and (not suppress-const) org-table-formula-use-constants)
-- 
2.28.0

From aad8eb548e7a7a7fde1908a9f9c66f980da10b56 Mon Sep 17 00:00:00 2001
From: Daniele Nicolodi 
Date: Tue, 20 Oct 2020 15:22:02 +0200
Subject: [PATCH 3/3

Specification for org-table formula mode string

[Daniele Nicolodi]

Hello,

I am looking at the parsing of org-table formula mode strings and I find
it quite confusing and I have an hard time understanding if it works the
way it does by design or by accident.

Formula mode strings are documented here

https://orgmode.org/manual/Formula-syntax-for-Calc.html#Formula-syntax-for-Calc

but the specification is not very strict and the parser implements
surprising behaviors. Would it make sense to tighten the specification
and the implementation?

I think being a bit more strict is what is accepted would help catching
typos and mistakes, but I am not sure that having formulas resulting in
#ERROR upon an org-mode upgrade is desirable. Also, maybe a specific
marker (#INVALID ?) should be used for invalid syntax.

The main problem with the current parsing code is that any character
that is not recognized as a valid mode flag is used as a value format
string. For example:

| 1 | 2 | OO3.000 |
#+TBLFM: $3=$1+$2;FOO%.3f

Is this by design? If so, what is the use case?

Thus is slightly more surprising:

| 1 | 2 | 3.000 OO |
#+TBLFM: $3=$1+$2;%.3f FOO

I argue the first form should result in an error. The second can either
result in an error or in everything following the % sign to be used as a
format string, although I am not sure there is a clear use case for this.

Thank you.

Cheers,
Dan

Bug in org-table--set-calc-mode?

[Daniele Nicolodi]

Hello,

I am hacking org-table-eval-formula (see thread about monetary values in
org-tables) which uses this inline function:

(defsubst org-table--set-calc-mode (var &optional value)
  (if (stringp var)
  (setq var (assoc var '(("D" calc-angle-mode deg)
 ("R" calc-angle-mode rad)
 ("F" calc-prefer-frac t)
 ("S" calc-symbolic-mode t)))
value (nth 2 var) var (nth 1 var)))
  (if (memq var org-tbl-calc-modes)
  (setcar (cdr (memq var org-tbl-calc-modes)) value)
(cons var (cons value org-tbl-calc-modes)))
  org-tbl-calc-modes)

which I am not able to understand or which is not correct.

The first (if ...) does some value substitutions, however, IIUC the
second (if ...) sets a new value for an entry in the org-tbl-calc-modes
plist if the entry is already present and builds a new plist with the
entry prepended if the entry is not there. However, the original plist
is returned and not the one with the new entry prepended.

It does not seem to be the intended behavior.

Shouldn't this be simply:

(defsubst org-table--set-calc-mode (var &optional value)
  (if (stringp var)
  (setq var (assoc var '(("D" calc-angle-mode deg)
 ("R" calc-angle-mode rad)
 ("F" calc-prefer-frac t)
 ("S" calc-symbolic-mode t)))
value (nth 2 var) var (nth 1 var)))
  (plist-put org-tbl-calc-modes var value))

or, better, the code refactored to do not use this function?

Cheers,
Dan

Re: org-tables with monetary amounts

[Daniele Nicolodi]

On 12/10/2020 10:22, Christian Moe wrote:
> 
> Hi, Daniele,
> 
> Good that it's working for you. I'll try to explain the unclear parts.
> 
> Daniele Nicolodi writes:
> 
>> On 24/09/2020 11:17, Christian Moe wrote:
>>
>>> Now, with the Calc command to simplify units, you can add dollars to
>>> euros and get the result in whichever currency comes first in the
>>> algebraic expression
>>>
>>>   | 3 USD | 4 EUR | 6.58 EUR |
>>>   #+tblfm: $3=usimplify($2+$1)
>>>
>>>   | 3 USD | 4 EUR | 7.6511628 USD |
>>>   #+tblfm: $3=usimplify($1+$2)
>>
>> Having to explicitly use usimplify() is a bit too verbose. It would be
>> ideal if this could be somehow be implicit.
> 
> Yes, or at least with a less obtrusive syntax, like a mode flag,
> something like
> 
>   $3=$2+$1;u

I had a look and unfortunately the implementation of mode flags is not
easily extensible, thus this cannot easily implemented in independent code.

Also, it would probably make sense to have this being a Calc mode
activated with an org-table formula mode flag, rather than something
implemented in org-table. I don't know how easy it would be to extend Calc.


>>> I don't use this functionality, so I don't have answers to all the
>>> questions you'll now have -- including how to get the desired precision
>>> without lopping off the currency unit in the last example!
>>
>> Having the desired fixed precision is quite important for this to be
>> useful. In y recent tests I had to drop the units (currencies) to
>> achieve this. It would be nice to find a way to avoid it.

Turns our this is actually easy. There are two ways to specify
visualization format for table cells: a format specifier like

$3=$1+$2;%.2f

which results in the units being stripped, and a Calc formatting mode
flag, like

$3=$1+$2;f2

which does not strip the units.

Cheers,
Dan

Re: org-tables with monetary amounts

[Daniele Nicolodi]

Hello Christian,

thank you for your input. calc support for units indeed covers the
basics of working with currencies. I gave it a spin on some notes I have
been working on recently. While some things work very nicely, other are
a bit ugly. It is a good start and I'll explore this approach more.

On 24/09/2020 11:17, Christian Moe wrote:

> Now, with the Calc command to simplify units, you can add dollars to
> euros and get the result in whichever currency comes first in the
> algebraic expression
> 
>   | 3 USD | 4 EUR | 6.58 EUR |
>   #+tblfm: $3=usimplify($2+$1)
> 
>   | 3 USD | 4 EUR | 7.6511628 USD |
>   #+tblfm: $3=usimplify($1+$2)

Having to explicitly use usimplify() is a bit too verbose. It would be
ideal if this could be somehow be implicit.

> I don't use this functionality, so I don't have answers to all the
> questions you'll now have -- including how to get the desired precision
> without lopping off the currency unit in the last example!

Having the desired fixed precision is quite important for this to be
useful. In y recent tests I had to drop the units (currencies) to
achieve this. It would be nice to find a way to avoid it.

> There are ways to enter user-defined units permanently. But exchange
> rates change, so to use this functionality on a daily basis, you'll want
> to have some kind of function to pull exchange rates and update the
> currency unit definitions in the Calc init file.

Computations usually refer to a specific point in time, thus this is npt
a big problem in practice. I already have scripts to pull exchange rates
at a given date. These can the be inserted as a property that can be
easily references from the computations.

> Apart from `usimplify', most Calc functions on units appear (?) to be
> missing corresponding algebraic versions that you can use in Calc
> expressions in Org tables, which limits the usefulness.
> 
> Org tables don't seem to have any specific formula syntax for leveraging
> Calc unit operations apart from what happens to work out of the
> box. This might be an area for improvement, though I'm not sure what to
> ask for.

I don't understand what you mean in the two paragraphs above. Can you
please clarify, maybe with examples?

Thank you.

Cheers,
Dan

Re: org-tables with monetary amounts

[Daniele Nicolodi]

On 25/09/2020 11:25, Neil Jerram wrote:
> Aside: Perhaps I'm misunderstanding them, but none of the open source
> tools, including (h)ledger, seem to be of much help here.
> - They focus on data entry and reconciliation, which I don't need as I'm
> happy to download and use OFX files from my bank.

I don't think the focus of ledger (or hledger, or beancount) is on data
entry. Actually beancount even has an ingest framework to deal with
importing data from bank statements and other sources.

> - They don't offer anything intelligent and automated for automatically
> categorizing transactions.

For beancount there is smart_importer
https://github.com/beancount/smart_importer and it is not difficult to
hack your own. I indeed use my own tool based on a simple machine
learning algorithm implemented by scikit-learn (Python).

> - They don't have a sophisticated representation of a budget, and
> reporting against that.

I am not sure what you mean by "sophisticated representation of a
budged". Once you categorize expenses in a (hierarchical) set of
expenses accounts it is trivial to use something like beancount to check
the balances against a budget.

> Do you know of a good forum (other than this!) for discussing such points?

The ledger-cli and the beancount mailing lists may be the right places
to discuss the use of these tools.

> I've attached mine, in case you read Scheme and there's more detail in
> there that is of interest.

My use case is very different, thus something external to org-mode is
not really useful.

Maybe an example would clarify what my use case is. I am working on a
research proposal where I need to provide details about funding
allocation in a few different categories over a certain amount of years.
In my document this is an org-table. Right now I have two choices to
edit the table:

- have monetary amounts in the cells (example "123.00 EUR") and do the
match externally,

- do the math in the org-table, specify the right formatting for all
computed cells, and still have to post-process it to add the currency
symbols to the input cells.

Either solution is sub-optimal. It would be nice if org-tables could
support monetary fields as they support time fields.

Cheers,
Dan

Re: org-tables with monetary amounts

[Daniele Nicolodi]

On 23/09/2020 18:55, Eric S Fraga wrote:
> Not answering your question directly but, depending on why and how you
> are using org tables for monetary calculations, you may wish to look at
> ledger [1].  There is support in org for ledger via babel (ob-ledger.el,
> distributed with org).

Hello Eric,

I use beancoount (another text-based accounting software, very similar
to ledger) for accounting purposes and I am very happy with it. I use
org-mode for planning and management tasks or simply to compare quotes,
things better handled in a a spreadsheet table.

I think it is time to have a look at how org-tables can be extended to
handle monetary quantities.

Cheers,
Dan

org-tables with monetary amounts

[Daniele Nicolodi]

Hello,

I often use org-tables to work with monetary amounts. It would be very
nice to have a couple of functionalities common in this domain:

- fixed precision arithmetic, namely derive the precision of the results
from the precision of the arguments (I think that calc can do this),

- support for parsing numbers followed by currencies,

- correct alignment for monetary values.

I had a quick look around, but I haven't found anything that implements
those things. Has anyone some secret code that they would like to share?

Thank you!

Cheers,
Dan

Link to source file location with content preview

[Daniele Nicolodi]

Hello,

I would like to create links from an org-mode file to source file
locations that display a preview of the link destination in the form of
a source code block or similar with a few lines of the file content.

Is this doable with org-store-link and org-insert-link?

A related question: does anyone have some elisp code that turns a Grep
buffer (the results of running "M-x grep") into org-mode links to the
locations?

Thank you!

Cheers,
Dan

Bug in identification of links?

[Daniele Nicolodi]

Hello,

org-mode fails to recognize https://doi.org/10.1016/0370-1573(89)90087-2
as a valid URL, it breaks it after the closing parenthesis ). I don't
understand why this is the case as I would imagine that if the )
character is not allowed in URLs the link would be broken before it and
not after. I haven't tried to find the code responsible for this, thus I
don't know what exactly is going on. Does anyone have an idea?

Thank you.

Cheers,
Dan

Re: [ANN] faster org-table-to-lisp

[Daniele Nicolodi]

Hello,

On 30-04-2020 14:28, tbanelwebmin wrote:
> * Version 9.4 (not yet released)
> ** Miscellaneous
> *** Faster org-table-to-lisp
> 
> The new implementation can be more than 100 times faster. This enhances
> responsiveness of Babel or Gnuplot blocks handling thousands long tables.

Nitpicking: I think "handling thousands long tables" should be "handling
tables with thousands of (rows|columns|cells)" (pick the appropriate
one), but probably "handling very large tables" is just good enough.

Cheers,
Dan

Re: org-ref: xxx-add-bibtex-entry with an org-capture like interface

[Daniele Nicolodi]

On 09-04-2020 13:43, John Kitchin wrote:
> This is basically what doi-add-bibtex-entry does. There isn't one
> function that does arxiv too, but there is arxiv-add-bibtex-entry. Those
> are both commands in org-ref. These don't pop up a buffer for approval,
> that is something you would have to build yourself.
> 
> There are many workflows people want, the one above works for me when I
> am writing, and just want to add entries with minimal disruption. I
> usually polish the entries later if needed. I can see a slower, approach
> like you describe too.

Hello John,

I have been able to hack something combining org-capture and the utility
functions you mention that implements the workflow I described.

There is only one issue: doi-utils-insert-bibtex-entry-from-doi contains
a (save-buffer) call that does not play nicely when I call it on a
temporary buffer. Why is the (save-buffer) there?
doi-utils-insert-bibtex-entry-from-doi looks like an utility function,
thus I think it should not try to save the current buffer. What do you
think?

Thank you.

Cheers,
Dan

org-ref: xxx-add-bibtex-entry with an org-capture like interface

[Daniele Nicolodi]

Hello John,

sorry for the cryptic subject but I was unable to come up with something
better.  I am revamping my workflow to take notes about articles I read
or that I save to be read later and I decided to explore if org-ref and
the related utility packages fit what I would like to be able to do.

org-ref offers quite a few very useful facilities, however, I would like
an interface to add new entries to the bibliography more similar to
org-capture: I would like to have a command that asks for an identifier
(for example a URL, a DOI, an arXiv id) and pops up a buffer with a
populated bibtex entry that can be verified and amended before being
appended to a bibtex file (for example add some keywords).

I think most of the building blocks are there, but before starting to
hack something together myself I wanted to check if this functionality
is already there and I missed it, and if it would be interesting to add
it to the functionality shipped with org-ref. Of course, any hint on how
to implement this effectively is greatly appreciated.

Thank you.

Cheers,
Dan

Re: [PATCH] support colorful blocks display on org-agenda

[Daniele Nicolodi]

On 20/02/2020 00:15, Bastien wrote:
> Days for the "contrib/" directory are counted: before Org 9.5, I will
> extract it from Org's repository, make it an independant repository on
> code.orgmode.org and make it available through Org ELPA.

This is an excellent plan!

Cheers,
Dan

Re: change to mail-list subject line?

[Daniele Nicolodi]

On 30/10/2019 23:03, adam wrote:
> 
> Has the emacs-orgmode  mail-list ceased including a  [O]  in the subject 
> line of its mails?  Or, what has changed here? 

Yes, as announced on the list a couple of days ago.

> I was using  [O]  as a filter for my incoming mails, now they are 
> turning up unsorted amongst my general mail.  
> 
> OK. I can filter on  and    Sorry for the noise.  
> 
> Any better suggestions for filtering gratefully received. 

The Lisd-Id header is there mostly for this.

Cheers,
Dan

Re: [O] Insert time in Org-mode

[Daniele Nicolodi]

On 25/06/2019 22:54, Spenser Truex wrote:
> "Fraga, Eric"  writes:
> 
>> On Tuesday, 25 Jun 2019 at 18:41, gmx wrote:
>>> How to insert the current time? 
>>
>> Easiest immediate way is to insert an inactive time stamp by C-u C-c !
>> RET.
> 
> Robert Pluim  writes:
> 
>> C-u M-x org-time-stamp
>>
>> will prompt for a date and insert it along with the current time (I
>> bind org-time-stamp to "C-c ." for convenience)
>>
> 
> Strange, my system does not insert the current time, only the date. I
> have to insert the time manually. Same for inactive and active stamps.
> I'm on the latest master versions.

For it to insert the time you need a prefix argument (C-u) to
org-time-stamp, so it should be "C-u C-c ." to add an active time-stamp
and "C-u C-c !" for an inactive one.

Cheers,
Dan

Re: [O] [ANN] Changes to link syntax

[Daniele Nicolodi]

On 11-03-2019 10:35, Nicolas Goaziou wrote:
> Hello,
> 
> Daniele Nicolodi  writes:
> 
>> On 10/03/2019 11:19, Nicolas Goaziou wrote:
>>> When in doubt, use the function ~org-link-escape~ in order to turn
>>> a link string into its properly escaped form.
>>>
>>> The old ~org-link-escape~ and ~org-link-unescape~ functions have
>>> been renamed into ~org-link-encode~ and ~org-link-decode~.
>>
>> Aren't those two sentences slightly contradictory? If ~org-link-escape~
>> has been renamed ~org-link-encode~ I think that the later should be
>> mentioned in the first sentence.
> 
> The first paragraph is about the new `org-link-escape'. The second one
> is about the old `org-link-escape', which has become `org-link-encode'.
> Is that clearer?

I may have understood what you mean now. At the first read I understood
that org-link-escape should be used to turn strings into link form, and
that it has been renamed into org-link-encode, thus the function that
should be used to turn strings into valid links is org-link-encode. What
you meant is that the old org-link-escape is now org-link-encode and a
new function with the same name has been introduced.

Cheers,
Dan

Re: [O] [ANN] Changes to link syntax

[Daniele Nicolodi]

On 10/03/2019 11:19, Nicolas Goaziou wrote:
> When in doubt, use the function ~org-link-escape~ in order to turn
> a link string into its properly escaped form.
> 
> The old ~org-link-escape~ and ~org-link-unescape~ functions have
> been renamed into ~org-link-encode~ and ~org-link-decode~.

Aren't those two sentences slightly contradictory? If ~org-link-escape~
has been renamed ~org-link-encode~ I think that the later should be
mentioned in the first sentence.

Cheers,
Dan

[O] Styling header stats for collapsed trees

[Daniele Nicolodi]

Hello,

there is a feature of Orgzly that I like: when a headline containing
sub-haedlines is folded, the leading star (bullet in Orgzly interface)
changes to signal that subheadings are hidden below.

I've tried to see if org allows to easily achieve something similar, but
the face of the headline and of the stars do not change when the
headline is collapsed.

Does anyone have a hint about something to hook upon to implement
something like that?

Thank you.

Cheers,
Dan

[O] Refile to datetree

[Daniele Nicolodi]

Hello,

I'm not always very systematic in my note taking and sometimes I find 
the need to refile entries into a datetree. Unfortunately `org-refile` 
does not provide facilities for doing that easily. `org-archive` kind of 
does, but it does not allow to easily specify the target date.

I saw a few questions on Stack Overflow asking for this functionality, 
but all answers only provided partial solutions. I came up with the 
following:

;; org-refile to datetree
(defun dnn-refile-to-datetree (arg)
  (interactive "P")
  (require 'org-datetree)
  (let* ((loc (org-refile-get-location "Datetree root" (current-buffer)))
 (buffer (find-file-noselect (nth 1 loc)))
 (pos (nth 3 loc))
 (date
  (calendar-gregorian-from-absolute
   (if arg (time-to-days (org-read-date nil t)) (org-today
 (dest
  (with-current-buffer buffer
(save-excursion
  (goto-char pos)
  (org-datetree-find-date-create date 'subtree-at-point)
  (list (org-get-heading) (buffer-file-name) nil (point))
(org-refile nil nil dest)))

which works for me so far.  If people think it may be generally useful,
and we can agree on how to expose the functionality to the user, I would
like to integrate it in org-refile proper.

Thanks. Cheers,
Dan

Re: [O] Clean up org-faces.el

[Daniele Nicolodi]

On 08/04/2018 11:33, Rasmus wrote:
> Daniele Nicolodi  writes:
> 
>> On 08/04/2018 07:27, Rasmus wrote:
>>> I have been wanting to clean up org-faces.el for a couple of years.
>>
>> Only tangentially related: is there a way to have a different face for
>> tables content and tables structure (ie the grid)?  The default blue
>> hurts my eyes :-)
> 
> That’s a nice idea.
> 
> At the moment it’s not possible as it’s the same face for table contents
> and table borders.  It shouldn’t be too hard to add since cells are
> recognized by org-element.  It would solve another of my issues.  I
> typically display links in variable pitch, but when they then go into
> tables, it breaks the alignment.

I can look into it, but it may take a while for me to figure it out.

Cheers,
Dan

Re: [O] Clean up org-faces.el

[Daniele Nicolodi]

On 08/04/2018 07:27, Rasmus wrote:
> I have been wanting to clean up org-faces.el for a couple of years.

Only tangentially related: is there a way to have a different face for
tables content and tables structure (ie the grid)?  The default blue
hurts my eyes :-)

Cheers,
Dan

Re: [O] Insert datetree entry

[Daniele Nicolodi]

On 4/5/17 3:40 PM, Bruce V Chiarelli wrote:
> 
> Daniele Nicolodi writes:
> 
>> On 4/4/17 1:25 PM, Bruce V Chiarelli wrote:
>>>
>>> Daniele Nicolodi  writes:
>> Hi Bruce,
>>
>> I modified your code as follow to be able to insert a datetree entry
>> correctly being anywhere in an existing datetree. I also added the
>> possibility of having a prefix argument to prompt for the date. It works
>> for me, but I don't know if this is the most elegant way to obtain what
>> I want. Comments are welcome.
>>
>>
>> ;; look for datetree root
>> (defun dnn-org-datetree-root ()
>>   (let ((re
>> "^\\([12][0-9]\\{3\\}\\)\\(-\\([01][0-9]\\)\\(-\\([0123][0-9]\\)\\)?\\)?
>> \\w+$"))
>> (while (string-match re (org-get-heading))
>>   (org-up-heading-safe))
>> (org-up-heading-safe)))
>>
>> ;; add a datetree entry
>> (defun dnn-org-datetree-find-create (arg)
>>   (interactive "P")
>>   (let ((d (calendar-gregorian-from-absolute
>>  (if arg (time-to-days (org-read-date nil t)) (org-today)
>> (dnn-org-datetree-root)
>> (org-datetree-find-date-create d 'subtree-at-point)))
> 
> Nice! I'm afraid I'm unqualified to give any comments/criticism on your
> code, since I'm fairly new to elisp myself. Looks better than what I had
> though, at least to work around the missing 'subtree-at-point
> feature. Maybe someone else can chime in.

If that code works around the missiong 'subtree-at-point feature on
older org-mode releases, it is absolutely by chance, I "back-ported" the
feature into my system from current org-mode master branch :-)

Cheers,
Daniele

Re: [O] Insert datetree entry

[Daniele Nicolodi]

On 4/4/17 1:25 PM, Bruce V Chiarelli wrote:
> 
> Daniele Nicolodi  writes:
> 
>>
>> thanks for the hint, but I don't understand what `subtree-at-point` is
>> in your code, it does not seem to be defined in my emacs and
>> `org-datetree-find-date-create` has a third parameter that is
>> interpreted as a boolean. I'm confused.
> 
> Ah, yes I forgot that this is a fairly new feature. Sorry for the
> confusion. In the latest Org, the optional keep-restriction parameter can be
> 
> - t - make the datetree at the end of the current /view/ of the buffer,
> - nil - make the datetree at the end of the file, even if it was narrowed
>   to a subtree before, or
> - 'subtree-at-point - put the datetree as a subheading at the end of the
>   current heading. This one is new in Org 9.0.5.
> 
> The first two options have been around for a long time, so if you get
> rid of 'subtree-at-point it should be fine. It will default to nil.

Hi Bruce,

I modified your code as follow to be able to insert a datetree entry
correctly being anywhere in an existing datetree. I also added the
possibility of having a prefix argument to prompt for the date. It works
for me, but I don't know if this is the most elegant way to obtain what
I want. Comments are welcome.


;; look for datetree root
(defun dnn-org-datetree-root ()
  (let ((re
"^\\([12][0-9]\\{3\\}\\)\\(-\\([01][0-9]\\)\\(-\\([0123][0-9]\\)\\)?\\)?
\\w+$"))
(while (string-match re (org-get-heading))
  (org-up-heading-safe))
(org-up-heading-safe)))

;; add a datetree entry
(defun dnn-org-datetree-find-create (arg)
  (interactive "P")
  (let ((d (calendar-gregorian-from-absolute
(if arg (time-to-days (org-read-date nil t)) (org-today)
(dnn-org-datetree-root)
(org-datetree-find-date-create d 'subtree-at-point)))


Cheers,
Daniele

Re: [O] Insert datetree entry

[Daniele Nicolodi]

On 4/4/17 1:25 PM, Bruce V Chiarelli wrote:
> 
> Daniele Nicolodi  writes:
> 
>>
>> thanks for the hint, but I don't understand what `subtree-at-point` is
>> in your code, it does not seem to be defined in my emacs and
>> `org-datetree-find-date-create` has a third parameter that is
>> interpreted as a boolean. I'm confused.
> 
> Ah, yes I forgot that this is a fairly new feature. Sorry for the
> confusion. In the latest Org, the optional keep-restriction parameter can be
> 
> - t - make the datetree at the end of the current /view/ of the buffer,
> - nil - make the datetree at the end of the file, even if it was narrowed
>   to a subtree before, or
> - 'subtree-at-point - put the datetree as a subheading at the end of the
>   current heading. This one is new in Org 9.0.5.
> 
> The first two options have been around for a long time, so if you get
> rid of 'subtree-at-point it should be fine. It will default to nil.

I would like t he functionality provided specifying 'subtree-at-point,
but it has been introduced after the 3.0.5 release currently available
on melpa :(

Cheers,
Daniele

Re: [O] Insert datetree entry

[Daniele Nicolodi]

On 4/3/17 5:37 PM, Bruce V Chiarelli wrote:
> 
> Daniele Nicolodi writes:
> 
>> Hello,
>>
>> there is a way in org-mode to insert a datetree entry without going
>> through the org-capture?  Ideally the function would jump to the
>> datetree and inser an headline for the current day if one is not
>> present, or move to the end of it if one is present.
>>
>> I haven't found anything like this in the manual. I'm now digging in the
>> code. Hopefully it is not something too hard to implement with my
>> lacking elisp knowledge.
> 
> I don't believe there really is one, but I've had to do it a couple of
> times myself. This is my solution (I'm not a lisp expert either, but it
> did the job):
> 
> (defun bc/org-new-datetree-at-point
>(interactive)
>(org-up-heading-safe)
>(org-datetree-find-date-create
>  (calendar-gregorian-from-absolute (org-today))
>  'subtree-at-point))
> 
> This will make the datetree as a subheading of the current heading (or
> find it if it already exists). Getting rid of 'subtree-at-point will
> make the year a level 1 heading at the end of the file, like with
> org-capture.

Hi Bruce,

thanks for the hint, but I don't understand what `subtree-at-point` is
in your code, it does not seem to be defined in my emacs and
`org-datetree-find-date-create` has a third parameter that is
interpreted as a boolean. I'm confused.

Cheers,
Daniele

[O] Insert datetree entry

[Daniele Nicolodi]

Hello,

there is a way in org-mode to insert a datetree entry without going
through the org-capture?  Ideally the function would jump to the
datetree and inser an headline for the current day if one is not
present, or move to the end of it if one is present.

I haven't found anything like this in the manual. I'm now digging in the
code. Hopefully it is not something too hard to implement with my
lacking elisp knowledge.

Thanks. Cheers,
Daniele

Re: [O] org-link interface

[Daniele Nicolodi]

On 10/13/16 12:02 PM, John Kitchin wrote:
> I do not know if you can do that. I would do something like this.
> 
> (defun insert-foo-link (path)
>  (interactive "sfoo path: ")
>  (insert (format "[[foo:%s][%s]]" path path)))

This has the drawback that it requires a different key-bind, instead
than using the org standard interface.

If it is not possible to teach org-insert-link to do that, it would be
an useful addition, I think.  It looks like it is possible to hook into
org-store-link to derive a description from the link type and path, but
the same is not possible with org-insert-link. I may propose a patch to
add this.

Cheers,
Daniele

>  
> Daniele Nicolodi writes:
> 
>> On 10/13/16 8:14 AM, John Kitchin wrote:
>>> In org8, this is not that easy to do, but not impossible.
>>>
>>> Why do not you want to use [[foo:bar][bar]]?
>>
>> Hi John,
>>
>> I'm fine using [[foo:bar][bar]], but it is much longer to type.
>>
>> Slightly changing my question: there is a way to instruct
>> org-insert-link to derive a default value for the description from the
>> path?  I had a look at the code, and it does not seem so.
>>
>> Thanks!
>>
>> Cheers,
>> Daniele
>>
>>
>>> Daniele Nicolodi writes:
>>>
>>>> Hello,
>>>>
>>>> I would like to add a new link type to org-mode.  Following the example
>>>> in the manual it is a very easy task.  However I would like to customize
>>>> how the link is visualized, specifically I would like a link like
>>>> "[[foo:bar]]" to be visualized simply as "bar" where the protocol type
>>>> has been hidden (same effect I would obtain with a link in the form
>>>> "[[foo:bar][bar]]".
>>>>
>>>> Is it possible to hook somehow into org-mode to have that transformation
>>>> done, or there is an easy way to accomplish that?
>>>>
>>>> Thank you!
>>>>
>>>> Cheers,
>>>> Daniele
>>>
>>>
> 
> 

Re: [O] [RFC] Change visibility for bracket links

[Daniele Nicolodi]

On 10/13/16 6:07 AM, Nicolas Goaziou wrote:
> Daniele Nicolodi  writes:
> 
>> Does this solve your problem? :)
> 
> Not really. It makes my example more bearable, but using C-a is still
> a workaround, and there are also many other dances to perform in various
> other cases. So, I still think there is an usability issue that needs to
> be fixed.
> 
> As another data point, `org-hide-emphasis-markers' is nil by default,
> because emphasized text can be a pain to edit when this variable is
> non-nil. I don't think links should go the opposite way.

Hi Nicolas,

I understand the problem and I agree that editing links and text around
them is not always straightforward.  However I think that what you are
proposing is a change in behavior that does not strictly solve a
technical problem.  As such, I believe, having the [ ] markers should be
an option, and probably not enabled by default.

Cheers,
Daniele

Re: [O] org-link interface

[Daniele Nicolodi]

On 10/13/16 8:14 AM, John Kitchin wrote:
> In org8, this is not that easy to do, but not impossible.
> 
> Why do not you want to use [[foo:bar][bar]]?

Hi John,

I'm fine using [[foo:bar][bar]], but it is much longer to type.

Slightly changing my question: there is a way to instruct
org-insert-link to derive a default value for the description from the
path?  I had a look at the code, and it does not seem so.

Thanks!

Cheers,
Daniele


> Daniele Nicolodi writes:
> 
>> Hello,
>>
>> I would like to add a new link type to org-mode.  Following the example
>> in the manual it is a very easy task.  However I would like to customize
>> how the link is visualized, specifically I would like a link like
>> "[[foo:bar]]" to be visualized simply as "bar" where the protocol type
>> has been hidden (same effect I would obtain with a link in the form
>> "[[foo:bar][bar]]".
>>
>> Is it possible to hook somehow into org-mode to have that transformation
>> done, or there is an easy way to accomplish that?
>>
>> Thank you!
>>
>> Cheers,
>> Daniele
> 
> 

Re: [O] [RFC] Change visibility for bracket links

[Daniele Nicolodi]

On 10/12/16 4:37 AM, Nicolas Goaziou wrote:
> Hello,
> 
> Daniele Nicolodi  writes:
> 
>> I use org-insert-link [C-c C-l] to edit links. I think this solved your
>> problem with the difficulty in editing the link description.
> 
> No, it doesn't. The problem also arises with text _around_ the link
> itself, which can be painful to edit. You could try, e.g., to start an
> empty buffer with and try to add anything before it (no M-x visible-mode
> or M-: allowed).

I just tried: going to the beginning of the line with C-a (by default
bound to org-beginning-of-line in org-mode) puts the cursor in a
position where you can insert text before the link, without it being
part of the link description.

Does this solve your problem? :)

Cheers,
Daniele

Re: [O] [RFC] Change visibility for bracket links

[Daniele Nicolodi]

On 05/10/16 09:40, Nicolas Goaziou wrote:
> Hello,
> 
> I'd like to make the following change to bracket links display (i.e.,
> when `org-highlight-links' contains `bracket', which is the default).
> 
> Basically, I want to leave a visible pair of brackets around the link,

I don't like the brackets. If you must please make this configurable.

> The rationale behind this suggestion is that editing a mostly invisible
> link is unintuitive when point is at the boundaries of its visible part.

I use org-insert-link [C-c C-l] to edit links. I think this solved your
problem with the difficulty in editing the link description.

Cheers,
Daniele

[O] org-link interface

[Daniele Nicolodi]

Hello,

I would like to add a new link type to org-mode.  Following the example
in the manual it is a very easy task.  However I would like to customize
how the link is visualized, specifically I would like a link like
"[[foo:bar]]" to be visualized simply as "bar" where the protocol type
has been hidden (same effect I would obtain with a link in the form
"[[foo:bar][bar]]".

Is it possible to hook somehow into org-mode to have that transformation
done, or there is an easy way to accomplish that?

Thank you!

Cheers,
Daniele

Re: [O] Enginnering notation in tables exported to LaTeX

[Daniele Nicolodi]

On 4/7/16 1:15 AM, Nicolas Goaziou wrote:
> Hello,
> 
> Daniele Nicolodi  writes:
> 
>> when exporting to LaTeX, numbers expressed in engineering notation
>> contained in tables are transformed into a quite confusing notation:
>> 1.234e5 becomes 1.234\,(5) which is rendered by LaTeX as 1.234 (5) with
>> the space being a very narrow one. This notation is confusing because it
>> is usually used (at l;east in physical sciences) to indicate uncertainty.
>>
>> I haven't found mention of this in the manual. There is a way to
>> disable, or customize this conversion?
> 
> See `org-latex-table-scientific-notation'.

Thanks!

Cheers,
Daniele

[O] Enginnering notation in tables exported to LaTeX

[Daniele Nicolodi]

Hello,

when exporting to LaTeX, numbers expressed in engineering notation
contained in tables are transformed into a quite confusing notation:
1.234e5 becomes 1.234\,(5) which is rendered by LaTeX as 1.234 (5) with
the space being a very narrow one. This notation is confusing because it
is usually used (at l;east in physical sciences) to indicate uncertainty.

I haven't found mention of this in the manual. There is a way to
disable, or customize this conversion?

Thank you. Cheers,
Daniele

Re: [O] How to make a non-GPL Org-mode exporter?

[Daniele Nicolodi]

On 27/07/15 18:59, Marcin Borkowski wrote:
> On 2015-07-27, at 14:39, Daniele Nicolodi  wrote:
>> Therefore, the only extensions to org-mode that can exist (and be
>> distributed, if you write code and keep it for yourself you are not
>> affected by the licensing terms) must be GPL.
>>
>> Thus, it makes little sense to continue the discussion: even if you
>> would release the code in your tutorial under a different license, it
>> would be or no use for who will read it.
> 
> I see.  Funnily, I found a few Emacs blogs (also by renowned Emacs
> hackers, like Oleh mentioned above) which clearly violate the rule that
> any Elisp code should be GPL'd.  So my intuition that nobody cares (at
> least until explicitly asked) seems to be confirmed;-).

Only who detains the copyright in something can enforce his licensing
terms. The copyright holder can decide that a specific use of its
copyrights material is fine and allow it, despite it not following
strictly the license agreement. In this case (I believe) the FSF
recognizes that there is no point in nitpicking on the license of a few
snippets of code. However, the FSF could ask in any moment to have the
license of those snippets clarified.

>>> How do I do that?  Is that even possible?  Also, is it possible to get
>>> an actual answer to this question without spending money on lawyers?
>>
>> There is no need to have lawyers involved, if you are in doubt about the
>> interpretation of the GPL you can consult the FSF (or other
>> organizations) on the matter. They will be happy to answer your question
>> with a very high degree of authority on the matter.
> 
> As someone mentioned, lawyers will not answer any question with a "high
> degree of authority".  Although I admit that this is not entirely their
> fault.

Who spoke about lawyers? I suggested to contact the FSF (or other
organizations involved in GPL enforcement). The FSF is the organizations
that wrote the GPL and that detains the copyright of a fairly large
codebase released under the GPL. Therefore they know what they meant
when they wrote it.

Furthermore, they detain the copyright on Emacs (and its Elisp
implementation) and org-mode, the project from which you are basing (or
not) your work. Therefore, they are authoritative when they deal with
copyright issues on that code.

I will not comment on the rest of your nonsensical arguments.

Cheers,
Daniele

Re: [O] Org-mode exporters licensing

[Daniele Nicolodi]

On 27/07/15 20:20, Marcin Borkowski wrote:
> 
> On 2015-07-27, at 20:02, Daniele Nicolodi  wrote:
> 
>> On 27/07/15 19:42, Marcin Borkowski wrote:
>>> That I've already learned.  OTOH, one of the reasons to use PD might be
>>> that I explicitly state that I object the legal system I live in.  (Mind
>>> you: I'm not an anarchist, and I'm very far from that.  But this system
>>> is almost unbearable.)
>>
>> This statement confirms that you do not really understand what you are
>> talking about: as you cannot renounce your copy rights, you cannot place
>> something in the public domain. If you do not release your work with an
>> explicit license, the default copyright protection law applies and this
>> means (in all jurisdictions I know about) that you reserve all rights to
>> yourself: none can use your code, and probably not even look at it.
> 
> I do understand (at least I think so).  And I do understand that my
> declaration of putting something in PD would be technically void.
> I just don't care about it, if the declaration of intent is clear.

If you do not care about the terms in which who receives your work is
able to use it, why all the discussion?

I thought that you were arguing that a less strict license than the GPL
is better for the content of a possible tutorial and you were inquiring
if you could release your code derived or inspired from GPL code with
another license. Now you say that you do not care, or better you say
that you do not want to give any rights to who receives your code.

I think you are confused.

Cheers,
Daniele

Re: [O] Org-mode exporters licensing

[Daniele Nicolodi]

On 27/07/15 19:42, Marcin Borkowski wrote:
> That I've already learned.  OTOH, one of the reasons to use PD might be
> that I explicitly state that I object the legal system I live in.  (Mind
> you: I'm not an anarchist, and I'm very far from that.  But this system
> is almost unbearable.)

This statement confirms that you do not really understand what you are
talking about: as you cannot renounce your copy rights, you cannot place
something in the public domain. If you do not release your work with an
explicit license, the default copyright protection law applies and this
means (in all jurisdictions I know about) that you reserve all rights to
yourself: none can use your code, and probably not even look at it.

>> Other than this I would recommend you to refrain from harsh comments on
>> a matter on which you hold strong ideas but weak knowledge (as most of
>> this thread demonstrates). Especially if your positions seem detrimental
>> of the Copyleft model, and you are asking for help in a mailing-list
>> devoted to a very successful Copyleft program.
> 
> Well, as I mentioned earlier, my knowledge is less and less weak, also
> thanks to your explanations.  OTOH, the more I know about these issues,
> the more I dislike the status quo, and the more harsh my opinions about
> GPL in particular are.  (It is not a secret that I am very critical of
> the GPL and of the FSF.  Still, as I said before, I'm very hesitant
> about explicitly breaking their rules.)

You are free to think whatever you want. However, using software
released under the GPL (and profiting of the freedom that the GPL
guarantees you) is not very coherent with your position.

Cheers,
Daniele

Re: [O] Org-mode exporters licensing

[Daniele Nicolodi]

On 27/07/15 14:42, Greg Troxel wrote:
> 
> Daniele Nicolodi  writes:
> 
>> On 27/07/15 13:52, Marcin Borkowski wrote:
>>> I disagree.  Licensing a tutorial with GPL is a stupid thing to do.
>>> A tutorial may contain code which people naturally mimic (or even
>>> copy).  Such things should definitely be in PD.
> 
> [many excellent comments.   As a nit, to reuse another's work under the
> GPL under a BSD license, you need more than them not to object; you
> need their affirmative permission.   And if much of org is assigned to
> the FSF, as I believe it is, that means the FSF's permission.  That's a
> use of resources about something that doesn't really matter much.]
> 
> Indeed.  A major point of which Marcin seems unaware is that licensing
> in a project in is more than a legal matter.  The license terms are a
> declaration of intent for how the code will be shared, and people
> contirbute under an expectation that those norms will be followed.
> 
> In particular, the GPL is designed to allow sharing only when the
> recipients receive rights to further share (and more).  In other words,
> not only is the code Free Software, but any derived works (that are
> distributed) will also be Free Software.  With a BSD-style license, or
> PD, derived works may or may not be Free.
> 
> Regardless of licensing, you can't make a derived work from copyrighted
> code and have it be PD.   And as Daniele points out, new works being PD
> only works in some jurisdictions (hence CC0).

Very good points!

I really like the "declaration of intent" pint of view.

Cheers,
Daniele

Re: [O] Org-mode exporters licensing

[Daniele Nicolodi]

On 27/07/15 14:25, Oleh Krehel wrote:
> Rasmus  writes:
> 
>> Oleh Krehel  writes:
>>
>>> If anywhere in your code there's (require 'org), you have to release
>>> your code under GPL.
>>
>> Are you sure about that?  By this logic, *any* .el file should be GPL as
>> they use (defun ·), implicitly loaded from byte-run.
> 
> I'm pretty sure: you're calling a library that is GPLv3.  There was this
> whole TiVo issue about linking GPL libraries to non-GPL code, which
> resulted in GPLv3. I just checked, and `progn' is GPLv3 and not GPLv2
> (which would at least have a chance to be linked).

Hello,

I'm not sure that using an interpreter for running some code classifies
as linking, but I don't know of any "official" statement on the subject.

On the other hand, Elisp is an extension language for a GPL program,
thus it may be argued that implicitly everything coded in Elist is an
extension of Emacs and therefore linked to it.

I believe this issue must have come up before. Does anyone have a link
to some statement from the GNU Project about this?

Cheers,
Daniele

Re: [O] How to make a non-GPL Org-mode exporter?

[Daniele Nicolodi]

Hello Marcin,

On 27/07/15 14:10, Marcin Borkowski wrote:
> Assume that (for some reason) I want to write an Org-mode exporter which
> won't be GPL'd.  (Use-case: having written a few custom exporters, I'm
> writing a tutorial on them, and I consider publishing a *tutorial* with
> GPL'd code a Bad Thing™.  (The idea of a programming tutorial is that
> other people can or even should reuse the code in the tutorial, right?
> And I see no reason to impose GPL on them.))

As Oleh Krehel pointed out in a reply to another mail of yours, if your
code links to org-mode code (or other GPL code) you cannot release it
under a different license. I'm not sure about how linking is intended in
Elisp sense of ('require)ing a library, but I believe it is analog to
executable linking in machine code programs.

Therefore, the only extensions to org-mode that can exist (and be
distributed, if you write code and keep it for yourself you are not
affected by the licensing terms) must be GPL.

Thus, it makes little sense to continue the discussion: even if you
would release the code in your tutorial under a different license, it
would be or no use for who will read it.

> How do I do that?  Is that even possible?  Also, is it possible to get
> an actual answer to this question without spending money on lawyers?

There is no need to have lawyers involved, if you are in doubt about the
interpretation of the GPL you can consult the FSF (or other
organizations) on the matter. They will be happy to answer your question
with a very high degree of authority on the matter.

> The manual says:
> 
> ,
> | Your two entry points are respectively ‘org-export-define-backend’ and
> | ‘org-export-define-derived-backend’.  To grok these functions, you
> | should first have a look at ‘ox-latex.el’ (for how to define a new
> | back-end from scratch) and ‘ox-beamer.el’ (for how to derive a new
> | back-end from an existing one.
> `
> 
> So basically you are expected to use existing GPL'd code to learn to
> write new exporters.

The manual makes the thing easy for the ones that want to respect the
authors choice of license.

> Also, the overall structure of the exporters is extremally similar.  For
> instance, the :menu-entry argument of org-export-define-backend is
> almost the same for all exporters (and it should be, in order not to
> break usability!).  Should I follow such conventions, in order to
> satisfy users, or should I deliberately break it, in order to satisfy
> lawyers?

First, your goal is not to satisfy lawyers, but to comply with the
license terms. The license terms are chosen by the authors of the code,
thus you are satisfying the wishes of whom granted you access to their
code. It is the minimum you should do to thank them.

Second, if that is the only structure that makes sense, you can
re-invent it without looking at the GPL code. Thus you can use a similar
structure in your differently licensed code. However, you should really
have re-created it without looking at the original. This email already
suggests otherwise.

> Also, the names of functions (like `org-latex-export-as-latex' vs
> `org-latex-export-to-latex') are similar across exporters.  Should I use
> this convention, too, in order to satisfy fellow programmers, or should
> I deliberately break it, in order to satisfy lawyers?

Function names are not copyrightable (as far as I know).

> Also, the docstrings of many transcoders are similar.  How am I supposed
> to write a docstring which is at the same time more or less
> comprehensive and different enough from the existing ones, so that
> I don't end up in jail?  (<--- this is actually a joke.  I hope so at
> least...)

There is a minimum unit of "code" that is copyrightable (the word "the"
appears in many copyrighted works, but you can still use it in your
own). If the doc-strings are trivial you can freely write something similar.

Cheers,
Daniele

Re: [O] Org-mode exporters licensing

[Daniele Nicolodi]

On 27/07/15 13:52, Marcin Borkowski wrote:
> I disagree.  Licensing a tutorial with GPL is a stupid thing to do.
> A tutorial may contain code which people naturally mimic (or even
> copy).  Such things should definitely be in PD.

As yourself pointed out in one of your emails, in many legal
ordinations, there is no such concept as public domain: you cannot
renounce to the copyright on your intellectual production.

Therefore licensing something as public domain is not quite possible. If
you want to grant the users of your code the most freedom (but do not
care about this freedom being carried over to others) the 3-Clause BSD
license http://opensource.org/licenses/BSD-3-Clause, the 2-Clause BSD
license http://opensource.org/licenses/BSD-2-Clause, or the MIT license
http://opensource.org/licenses/mit-license.html are good candidate
licenses formulated in the framework of copyright law as accepted
internationally.

However, you cannot derive your work from some other work distributed
under GPL and license it with a more permissive license (as the ones
suggested above). What constituted a derived work is however not
scientifically defined (and you have been rather terse in describing how
your work build upon code released under the GPLv3). In one place you
explicitly mention running a query-replace on the source code:
mechanical transformations of the source code are considered derived
works, even if the end result does not resemble at all the original.

I would suggest you to do derive your work from the GPL code and then
consult with the authors about its licensing. If you are only using the
GPL code as a skeleton, I think they would not have objections (but you
could also easily re-implement it from scratch).

Other than this I would recommend you to refrain from harsh comments on
a matter on which you hold strong ideas but weak knowledge (as most of
this thread demonstrates). Especially if your positions seem detrimental
of the Copyleft model, and you are asking for help in a mailing-list
devoted to a very successful Copyleft program.

Cheers,
Daniele