Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

On 2023-09-03  08:53, Ihor Radchenko wrote:
> Jens Schmidt  writes:
> 
>> Implemented in the next version of the patch, please check.
>>
> 
> Applied, onto main.

Thanks!

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Ihor Radchenko]

Jens Schmidt  writes:

> Implemented in the next version of the patch, please check.
>
> From 11dc3ac4ff060f1ffb9dae7b35eabe526bbbc572 Mon Sep 17 00:00:00 2001
> From: Jens Schmidt 
> Date: Thu, 24 Aug 2023 22:38:02 +0200
> Subject: [PATCH] org-make-tags-matcher: Re-add quoting of property names

Applied, onto main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=234650af2
Fixed. I can no longer reproduce the recipe.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Ihor Radchenko]

Jens Schmidt  writes:

> TL;DR:
>
> - I think we cannot make "&" mandatory because of backward compatibility.

Sorry for the confusion. I did not mean that "&" should be mandatory,
just that "&" might make it easier to avoid a need to escape things. So,
it could be used _instead_ of escaping.

> - Even if we made "&" mandatory, it would not really solve the quoting
>   problem, since the parser is rather hacky and other quoting and
>   context issues would still be there.

But at this point you are more familiar with that parser than I am, so
my idea does not seem to be viable.

> This all calls for a proper parser, based on peg or bovine or whatever.
> Hopefully that parser would still keep backward compatibility, but
> that's probably wishful thinking.

Backward compatibility will be easy - just leave the current code when
old query version is detected. We should better focus on the new syntax
in future and leave the current syntax as compatibility layer that will
be eventually deprecated.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Tom Gillespie]

Ignore the previous message. I see that this was about matching
tags not about specifying them. Best,
Tom

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Tom Gillespie]

Without wading too far into this, why do we need escape syntax for this?
The only character that might need an escape would be colon :, but
my reading of the syntax doc is that colo : will immediately terminate
the property, so we would update the doc to make it clear that property
names cannot contain a colon. As written, if there is an issue with the
minus sign in property names then that is a bug, but I feel like I might
be missing something?

Tom

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

On 2023-08-27  09:43, Ihor Radchenko wrote:
> Samuel Loury  writes:
> 
>> IMHO, "-tag&-todo=TODO" is totally ok. I even imagine we could say that
>> & and | are forbidden to say anything else than AND and OR and people
>> would be ok with that.
> 
> Actually, explicit & or | might be an easier way to not worry about
> escaping things. Except escaping & or | themselves.
> It might be the preferred way to put into the manual.
> 
>> IOW, I wonder of the time and effort to deal with optional & is worth
>> it. WDYT?
> 
> We cannot remove the optionality of &, because doing so will break
> existing user setups. But we can certainly change our recommendations in
> the manual.
> 
> Though pure tag matcher makes more sense with implicit &:
>  +tag1-tag2+tag3... vs +tag1&-tag2&+tag3
> Especially in the interactive agenda filters.

I feel it's a bit hard to reply to this message in the right places, so
I'll do a plain bottom post, sorry.

TL;DR:

- I think we cannot make "&" mandatory because of backward compatibility.

- Even if we made "&" mandatory, it would not really solve the quoting
  problem, since the parser is rather hacky and other quoting and
  context issues would still be there.

- But then I cannot see any real reason why we should recommend using
  "&" in the manual.

- Finally, we should hope for and work towards a "real parser", as Ihor
  has mentioned already in a previous post:

  (beginning of thread)
  
https://list.orgmode.org/orgmode/9132e58f-d89e-f7df-bbe4-43d53a236...@vodafonemail.de/
  (mentioning of peg)
  https://list.orgmode.org/orgmode/87wmyendr7.fsf@localhost/

Details, as far as still required:

The whole tag/property query parser seems to have a long tradition.  For
example, there is still that TODO-state special case that allows for
queries like this (example ripped from the manual):

  work/WAITING
 Same as work+TODO​="WAITING"

So people *have* been worrying about a) query brevity and b) backward
compatibility, and I think we should do so as well.

Then the current parser is horribly ad-hoc-ish and, as a plain
regexp-based parser, context insensitive.  For example, a tag regexp
match

  {regexp}

is matched simply as

  {[^}]+}

that is, you have no chance whatsoever to quote the closing curly
parenthesis.  Likewise for double-quoted strings.

In other words, even if we make | or & mandatory, there will still
remain a lot of ambiguities stemming from the rest of the parser.  And
I'm too lazy to think about how we could quote & in that whole picture.
(| is sort of handled already, but again in a completely
context-insensitive way.)

This all calls for a proper parser, based on peg or bovine or whatever.
Hopefully that parser would still keep backward compatibility, but
that's probably wishful thinking.

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

On 2023-08-31  10:08, Ihor Radchenko wrote:
> Jens Schmidt  writes:
> 
>>> Implemented in the next version of the patch, please check.
>>
>> Gentle ping ...
> 
> I was waiting for your comment on 
> https://list.orgmode.org/orgmode/874jklhqw2.fsf@localhost/

Ah, ok, sorry.  I will give that branch a second thought and comment
soon.

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Ihor Radchenko]

Jens Schmidt  writes:

>> Implemented in the next version of the patch, please check.
>
> Gentle ping ...

I was waiting for your comment on 
https://list.orgmode.org/orgmode/874jklhqw2.fsf@localhost/

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

On 2023-08-26  14:54, Jens Schmidt wrote:
> On 2023-08-26  14:22, Ihor Radchenko wrote:

>> Looks good to me.
> 
> Implemented in the next version of the patch, please check.

Gentle ping ...

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Ihor Radchenko]

Samuel Loury  writes:

> IMHO, "-tag&-todo=TODO" is totally ok. I even imagine we could say that
> & and | are forbidden to say anything else than AND and OR and people
> would be ok with that.

Actually, explicit & or | might be an easier way to not worry about
escaping things. Except escaping & or | themselves.
It might be the preferred way to put into the manual.

> IOW, I wonder of the time and effort to deal with optional & is worth
> it. WDYT?

We cannot remove the optionality of &, because doing so will break
existing user setups. But we can certainly change our recommendations in
the manual.

Though pure tag matcher makes more sense with implicit &:
 +tag1-tag2+tag3... vs +tag1&-tag2&+tag3
Especially in the interactive agenda filters.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Samuel Loury]

I have a dumb question.

IIUC, it needs a lot of effort to deal with implicit & correctly. I
initially used it because de manual said it was ok, but I would have
used explicit & if the manual had said so.

I wonder if we could just stop saying that & is optional and have a
simpler parsing. 

IMHO, "-tag&-todo=TODO" is totally ok. I even imagine we could say that
& and | are forbidden to say anything else than AND and OR and people
would be ok with that.

IOW, I wonder of the time and effort to deal with optional & is worth
it. WDYT?

-- 
Konubinix
GPG Key: 7439106A
Fingerprint: 5993 BE7A DA65 E2D9 06CE  5C36 75D2 3CED 7439 106A


signature.asc
Description: PGP signature

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

On 2023-08-26  14:22, Ihor Radchenko wrote:
> Jens Schmidt  writes:

>> #+begin_example
>> boss\-prio="C"
>> boss\:prio="C"
>> boss\\prio="C"
>> #+end_example
> 
> Looks good to me.

Implemented in the next version of the patch, please check.

From 11dc3ac4ff060f1ffb9dae7b35eabe526bbbc572 Mon Sep 17 00:00:00 2001
From: Jens Schmidt 
Date: Thu, 24 Aug 2023 22:38:02 +0200
Subject: [PATCH] org-make-tags-matcher: Re-add quoting of property names

* lisp/org.el (org-make-tags-matcher):
* testing/lisp/test-org.el (test-org/map-entries): Move special cased
handling of LEVEL properties.  Add tests for other special cased
properties TODO and CATEGORY.

* lisp/org.el (org-make-tags-matcher):
* doc/org-manual.org (Matching tags and properties):
* testing/lisp/test-org.el (test-org/map-entries): Re-add and extend
quoting of property names in search strings.

Link: https://orgmode.org/list/87h6oq2nu1@gmail.com
---
 doc/org-manual.org   | 22 
 lisp/org.el  | 55 
 testing/lisp/test-org.el | 34 +
 3 files changed, 68 insertions(+), 43 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 17b25fef4..249648566 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -9320,21 +9320,21 @@ With the regular =<= operator, the search would handle entries without
 an =EFFORT= property as having a zero effort and would include them in
 the result as well.
 
-Currently, you can use only property names including alphanumeric
-characters, underscores, and minus characters in search strings.  In
-addition, if you want to search for a property whose name starts with
-a minus character, you have to "quote" that leading minus character
-with an explicit positive selection plus character, like this:
+You can use all characters valid in property names when matching
+properties.  However, you have to quote some characters in property
+names with backslashes when using them in search strings, namely all
+characters different from alphanumerics and underscores[fn:: If you
+quote alphanumeric characters or underscores with a backslash, that
+backslash is ignored.].  For example, to search for all entries having
+a property =boss-prio=, =boss:prio=, or =boss\prio=, respectively,
+with value =C=, use search strings
 
 #+begin_example
-+-long-and-twisted-property-name-="foo"
+boss\-prio="C"
+boss\:prio="C"
+boss\\prio="C"
 #+end_example
 
-#+texinfo: @noindent
-Without that extra plus character, the minus character would be taken
-to indicate a negative selection on search term
-=long-and-twisted-property-name-​="foo"=.
-
 You can configure Org mode to use property inheritance during
 a search, but beware that this can slow down searches considerably.
 See [[*Property Inheritance]], for details.
diff --git a/lisp/org.el b/lisp/org.el
index 50df1b2d9..78f8eb2e9 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -11328,15 +11328,14 @@ See also `org-scan-tags'."
   "\\(?2:"
   ;; tag regexp match
   "{[^}]+}\\|"
-  ;; LEVEL property match.  For sake of consistency,
-  ;; recognize starred operators here as well.  We do
-  ;; not need to process them below, however, since
-  ;; the LEVEL property is always present.
-  "LEVEL\\(?3:" opre "\\)\\*?\\(?4:[0-9]+\\)\\|"
-  ;; regular property match
+  ;; property match.  Try to keep this subre generic
+  ;; and rather handle special properties like LEVEL
+  ;; and CATEGORY further below.  This ensures that
+  ;; the same quoting mechanics can be used for all
+  ;; property names.
   "\\(?:"
   ;; property name [1]
-  "\\(?5:[[:alnum:]_-]+\\)"
+  "\\(?5:\\(?:[[:alnum:]_]+\\|[^[:space:]]\\)+\\)"
   ;; operator, optionally starred
   "\\(?6:" opre "\\)\\(?7:\\*\\)?"
   ;; operand (regexp, double-quoted string,
@@ -11353,13 +11352,19 @@ See also `org-scan-tags'."
  (start 0)
  tagsmatch todomatch tagsmatcher todomatcher)
 
-;; [1] The minus characters in property names do *not* conflict
-;; with the exclusion operator above, since the mandatory
-;; following operator distinguishes these both cases.
-;; Accordingly, minus characters do not need any special quoting,
-;; even if https://orgmode.org/list/87jzv67k3p.fsf@localhost and
-;; commit 19b0e03f32c6032a60150fc6cb07c6f766cb3f6c suggest
-;; otherwise.
+;; [1] The history of this particular subre:
+;; - \\([[:alnum:]_]+\\) [pre-19b0e03]
+;;   Does not allow for minus characters in property names.
+;; - "\\(\\(?:[[:alnum:]_]+\\(?:-\\)*\\)+\\)" [19b0e03]
+;;   Incomplete fix of above issue, still resulting in, e.g.,
+;;   

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Ihor Radchenko]

Jens Schmidt  writes:

>> That's partially what I meant.
>> It would help to provide a simpler example first.
>
> How about a series of simpler examples then, like this:
>
> #+begin_example
> boss\-prio="C"
> boss\:prio="C"
> boss\\prio="C"
> #+end_example

Looks good to me.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

On 2023-08-26  14:00, Ihor Radchenko wrote:
> Jens Schmidt  writes:
>> On 2023-08-26  12:16, Ihor Radchenko wrote:

>>> This actually feels rather cubersome.
>>
>> [...]  I can
>> provide a simpler example in the manual, but that's probably not
>> what you meant :-)
> 
> That's partially what I meant.
> It would help to provide a simpler example first.

How about a series of simpler examples then, like this:

#+begin_example
boss\-prio="C"
boss\:prio="C"
boss\\prio="C"
#+end_example

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Ihor Radchenko]

Jens Schmidt  writes:

> On 2023-08-26  12:16, Ihor Radchenko wrote:
>> Jens Schmidt  writes:
>>>  #+begin_example
>>> -+-long-and-twisted-property-name-="foo"
>>> +\-long\=and\\twisted\:property\.name\*="foo"
>>>  #+end_example
>> 
>> This actually feels rather cubersome.
>
> That particular example looks awkward, but pls don't forget that
> users (mostly) have been happy without any option of quoting so
> far.  IOW, there shouldn't be much to quote in real life.  I can
> provide a simpler example in the manual, but that's probably not
> what you meant :-)

That's partially what I meant.
It would help to provide a simpler example first.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

On 2023-08-26  12:16, Ihor Radchenko wrote:
> Jens Schmidt  writes:
>>  #+begin_example
>> -+-long-and-twisted-property-name-="foo"
>> +\-long\=and\\twisted\:property\.name\*="foo"
>>  #+end_example
> 
> This actually feels rather cubersome.

That particular example looks awkward, but pls don't forget that
users (mostly) have been happy without any option of quoting so
far.  IOW, there shouldn't be much to quote in real life.  I can
provide a simpler example in the manual, but that's probably not
what you meant :-)

> I am now wondering if we could instead do something like
> "-long=and\twisted:property.name*"="foo"
> Then, we will just have to quote the " itself, not all
> non-alphanumerics.
> 
> Although, using " might be tricky.

Agreed.  It introduces more context, and longer context, than a
simple backslash.  Plus double quotes are already used for other
purposes in the larger regexp, which could introduce more
ambiguities.

Finally, a term

  "foo-bar"="foo-bar"

looks, er, tautologically? at best.

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Ihor Radchenko]

Jens Schmidt  writes:

>> IOW, backslash quotes everything except whitespace, which by definition
>> cannot be part of a property name.
>> 
>> Will start on this, but with tests and documentation this might take
>> some time.
>
> Here comes a first patch ... please check.

Thanks!

>  #+begin_example
> -+-long-and-twisted-property-name-="foo"
> +\-long\=and\\twisted\:property\.name\*="foo"
>  #+end_example

This actually feels rather cubersome.
I am now wondering if we could instead do something like
"-long=and\twisted:property.name*"="foo"
Then, we will just have to quote the " itself, not all
non-alphanumerics.

Although, using " might be tricky.
  

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

On 2023-08-24  10:52, Jens Schmidt wrote:
> On 2023-08-24  09:32, Ihor Radchenko wrote:

>> I prefer (B). And we will need to allow escaping of the "\" itself. Like
>> \\.
> 
> OK.  Since backslash is not used yet in the rest of the regexp, (B)
> should be rather edge-case-free.  So the corresponding subre to match
> property names would look like
> 
>   \(?5:\(?:[[:alnum:]]+\|\\[^[:space:]]\)+\)
> 
> IOW, backslash quotes everything except whitespace, which by definition
> cannot be part of a property name.
> 
> Will start on this, but with tests and documentation this might take
> some time.

Here comes a first patch ... please check.From 830750bfa10b52ac77abe8af1f2789057f9101c1 Mon Sep 17 00:00:00 2001
From: Jens Schmidt 
Date: Thu, 24 Aug 2023 22:38:02 +0200
Subject: [PATCH] org-make-tags-matcher: Re-add quoting of property names

* lisp/org.el (org-make-tags-matcher):
* testing/lisp/test-org.el (test-org/map-entries): Move special cased
handling of LEVEL properties.  Add tests for other special cased
properties TODO and CATEGORY.

* lisp/org.el (org-make-tags-matcher):
* doc/org-manual.org (Matching tags and properties):
* testing/lisp/test-org.el (test-org/map-entries): Re-add and extend
quoting of property names in search strings.

Link: https://orgmode.org/list/87h6oq2nu1@gmail.com
---
 doc/org-manual.org   | 20 +++
 lisp/org.el  | 55 
 testing/lisp/test-org.el | 34 +
 3 files changed, 66 insertions(+), 43 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 17b25fef4..10a03b344 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -9320,21 +9320,19 @@ With the regular =<= operator, the search would handle entries without
 an =EFFORT= property as having a zero effort and would include them in
 the result as well.
 
-Currently, you can use only property names including alphanumeric
-characters, underscores, and minus characters in search strings.  In
-addition, if you want to search for a property whose name starts with
-a minus character, you have to "quote" that leading minus character
-with an explicit positive selection plus character, like this:
+You can use all characters valid in property names when matching
+properties.  However, you have to quote some characters in property
+names with backslashes when using them in search strings, namely all
+characters different from alphanumerics and underscores[fn:: If you
+quote alphanumeric characters or underscores with a backslash, that
+backslash is ignored.].  For example, to search for all entries having
+a property =-long​=and\twisted:property.name*= with value =foo=, use
+search string
 
 #+begin_example
-+-long-and-twisted-property-name-="foo"
+\-long\=and\\twisted\:property\.name\*="foo"
 #+end_example
 
-#+texinfo: @noindent
-Without that extra plus character, the minus character would be taken
-to indicate a negative selection on search term
-=long-and-twisted-property-name-​="foo"=.
-
 You can configure Org mode to use property inheritance during
 a search, but beware that this can slow down searches considerably.
 See [[*Property Inheritance]], for details.
diff --git a/lisp/org.el b/lisp/org.el
index 50df1b2d9..78f8eb2e9 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -11328,15 +11328,14 @@ See also `org-scan-tags'."
   "\\(?2:"
   ;; tag regexp match
   "{[^}]+}\\|"
-  ;; LEVEL property match.  For sake of consistency,
-  ;; recognize starred operators here as well.  We do
-  ;; not need to process them below, however, since
-  ;; the LEVEL property is always present.
-  "LEVEL\\(?3:" opre "\\)\\*?\\(?4:[0-9]+\\)\\|"
-  ;; regular property match
+  ;; property match.  Try to keep this subre generic
+  ;; and rather handle special properties like LEVEL
+  ;; and CATEGORY further below.  This ensures that
+  ;; the same quoting mechanics can be used for all
+  ;; property names.
   "\\(?:"
   ;; property name [1]
-  "\\(?5:[[:alnum:]_-]+\\)"
+  "\\(?5:\\(?:[[:alnum:]_]+\\|[^[:space:]]\\)+\\)"
   ;; operator, optionally starred
   "\\(?6:" opre "\\)\\(?7:\\*\\)?"
   ;; operand (regexp, double-quoted string,
@@ -11353,13 +11352,19 @@ See also `org-scan-tags'."
  (start 0)
  tagsmatch todomatch tagsmatcher todomatcher)
 
-;; [1] The minus characters in property names do *not* conflict
-;; with the exclusion operator above, since the mandatory
-;; following operator distinguishes these both cases.
-;; Accordingly, minus characters do not need any special quoting,
-;; even if https://orgmode.org/list/87jzv67k3p.fsf@localhost and
-;; 

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

On 2023-08-24  09:32, Ihor Radchenko wrote:
> Jens Schmidt  writes:

>> Some obvious choice would be a simpler single backslash
>>
>>   (B) "\-" => "-"
> 
> I prefer (B). And we will need to allow escaping of the "\" itself. Like
> \\.

OK.  Since backslash is not used yet in the rest of the regexp, (B)
should be rather edge-case-free.  So the corresponding subre to match
property names would look like

  \(?5:\(?:[[:alnum:]]+\|\\[^[:space:]]\)+\)

IOW, backslash quotes everything except whitespace, which by definition
cannot be part of a property name.

Will start on this, but with tests and documentation this might take
some time.

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Ihor Radchenko]

Jens Schmidt  writes:

>> When "TAG1-TAG2" is an actual tag name with dash, we may allow escaping.
>
> But I'd like to clarify that.  From the Org syntax
>
>   https://orgmode.org/worg/org-syntax.html#Headings
>
> I understood that tag names cannot contain minus characters,
> and `org-tag-re' does not match any, either.  So we are talking
> about property names only, right?

You are right.

> Then the question is what quoting scheme to use for property
> names.  The previous one used before my commit f689eb4
>
>   (A) "\\-" => "-"
>
> never has been documented and never has been working properly,
> since the *matching* of these was done on *prop* names, but
> the *unescaping* on *tag* names.  So we are basically free to
> come up with something new.
>
> Some obvious choice would be a simpler single backslash
>
>   (B) "\-" => "-"

I prefer (B). And we will need to allow escaping of the "\" itself. Like
\\.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Ihor Radchenko]

Jens Schmidt  writes:

> Yet another edge-case: In the Org syntax definition a property
> name is really delimited by a "colon-whitespace" sequence.  This
> takes out the ambiguity whether a property name is defined
> non-greedily
>
>   :[^[:space:]]+?:
>
> or greedily
>
>   :[^[:space:]]+:
>
> I'm too lazy right now to think about the consequences, but I
> hope this makes a difference only for really weird property
> matches, like this:
>
>   :foo:=1+:bar:=2

Thus, -1 to this idea with ":" delimiter. If possible, we should avoid
edge cases. Otherwise, we are getting rid of the edge case with "-" by
introducing yet another edge case.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

On 2023-08-23  16:00, Jens Schmidt wrote:

> Given the property name syntax in
> 
>   https://orgmode.org/worg/org-syntax.html#Node_Properties
> 
> the subre in `org-make-tags-matcher' to match property names
> should then look similar to
> 
>   "\\(?5:[[:alnum:]_]+\\|:[^[:space:]]+?:\\)"
> 
> , right (colons being stripped off later)?  I'm really asking
> about the trailing plus signs here, but these do not seem to
> make sense in property queries.

Yet another edge-case: In the Org syntax definition a property
name is really delimited by a "colon-whitespace" sequence.  This
takes out the ambiguity whether a property name is defined
non-greedily

  :[^[:space:]]+?:

or greedily

  :[^[:space:]]+:

I'm too lazy right now to think about the consequences, but I
hope this makes a difference only for really weird property
matches, like this:

  :foo:=1+:bar:=2

This would be parsed non-greedily as

  FOO == 1 && BAR == 2

or greedily as

  FOO:=1+:BAR == 2

I'd go for non-greedily defined property names.

Re: [RFC] Quoting property names in tag/property matches [Was: [BUG?] Matching tags: & operator no more implicit between tags and special property]

[Jens Schmidt]

> On 2023-08-23  12:31, Ihor Radchenko wrote:
> 
>> Jens, we got an edge case with "-", after all.
>>
>> What is happening here is that the new parser
>> (https://orgmode.org/list/9132e58f-d89e-f7df-bbe4-43d53a236...@vodafonemail.de)
>> treats the above match string as
>>
>> property "tag-TODO" not equal "TODO"
>>
>> instead of
>>
>> not tag equal "tag" and not TODO equal "TODO"

Right.  And you already provided in the other branch of this
discussion a direction how to resolve that:

> [...] In case of ambiguity TAG1-TAG2-TAG3-PROP="" like we see here, we
> should prefer multiple shorter conditions: TAG1 && -TAG2 && -TAG3 && PROP=""
> 
> When "TAG1-TAG2" is an actual tag name with dash, we may allow escaping.

But I'd like to clarify that.  From the Org syntax

  https://orgmode.org/worg/org-syntax.html#Headings

I understood that tag names cannot contain minus characters,
and `org-tag-re' does not match any, either.  So we are talking
about property names only, right?  At least I'll do so in the
following ...


Then the question is what quoting scheme to use for property
names.  The previous one used before my commit f689eb4

  (A) "\\-" => "-"

never has been documented and never has been working properly,
since the *matching* of these was done on *prop* names, but
the *unescaping* on *tag* names.  So we are basically free to
come up with something new.

Some obvious choice would be a simpler single backslash

  (B) "\-" => "-"

And when I have been "fixing" the parser, I also thought that

  (C) ":...:" => "..."

would give a nice and somewhat logical quoting scheme.  That is,
if a property name contains "special" characters, it could be
surrounded by colons to quote these.

So we have so far quoting schemes A, B, C, with my preference
being C.  Any other proposals?


I'll focus on C, where there is one big cave-at:  The main
regexp in `org-make-tags-matcher' also allows for (undocumented)
colons as inclusive search term separator:

  ;; exclusion and inclusion (the latter being implicit)
  "\\(?1:[-+:]\\)?"
^
   
Which collides with the colons used by quoting scheme C.  I'd
rather sacrifice that use of colon as search term separator,
TBH.

Given the property name syntax in

  https://orgmode.org/worg/org-syntax.html#Node_Properties

the subre in `org-make-tags-matcher' to match property names
should then look similar to

  "\\(?5:[[:alnum:]_]+\\|:[^[:space:]]+?:\\)"

, right (colons being stripped off later)?  I'm really asking
about the trailing plus signs here, but these do not seem to
make sense in property queries.


What do you think?