Re: [DRAFT PATCH v4] Decouple LANG= and testing (was: Test failure due to LANG)

[Ihor Radchenko]

Ruijie Yu  writes:

>> I recommend `org-element-timestamp-parser' for analysis.
>
> Thanks for pointing that out!  I didn't realize it was there.  Although,
> I don't think I can use this function as-is, nor its internal regexps:
> all the relevant tests where I used `org-test-timestamp-regexp' tried to
> remove the day-of-week part.  So I would need to know the region for the
> day-of-week portion.  At the moment, I can't think of anything other
> than making a group for DoW, and removing wrapping `save-match-data' in
> `org-element-timestamp-parser' -- which is why I had those convoluted
> group numbers in my `org-test-timestamp-regexp' for all portions that
> one may ever want to use, hoping that nobody goes beyond defining 100
> groups.

(let ((org-time-stamp-formats '("%Y-%m-%d" . "%Y-%m-%d %H:%M")))
 (org-element-timestamp-interpreter timestamp nil))

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [DRAFT PATCH v4] Decouple LANG= and testing (was: Test failure due to LANG)

[General discussions about Org-mode.]

Ihor Radchenko  writes:

> Ruijie Yu via "General discussions about Org-mode."
>  writes:
>
>> Something very troublesome for me is `test-org-clock/clocktable/lang' on
>> zh_CN.UTF-8.  What I noticed is that when I run this test in batch mode
>> (one of the two methods below), it fails:
>
> Well. Batch and non-batch modes are not the same, especially wrt display
> things. And the differences are not consistent for different Emacs
> versions. This only happens for weird edge cases though. (Or when
> displaying images).

Hmm.  I guess I won't be trying to fix this particular issue then -- it
is a bit beyond my knowledge.  Thanks for explaining it.

>> +(defconst org-test-timestamp-regexp
>
> I recommend `org-element-timestamp-parser' for analysis.

Thanks for pointing that out!  I didn't realize it was there.  Although,
I don't think I can use this function as-is, nor its internal regexps:
all the relevant tests where I used `org-test-timestamp-regexp' tried to
remove the day-of-week part.  So I would need to know the region for the
day-of-week portion.  At the moment, I can't think of anything other
than making a group for DoW, and removing wrapping `save-match-data' in
`org-element-timestamp-parser' -- which is why I had those convoluted
group numbers in my `org-test-timestamp-regexp' for all portions that
one may ever want to use, hoping that nobody goes beyond defining 100
groups.

I also don't know if it is viable to integrate my full-of-`rx' and
full-of-`group-n' regexp expression into the 10s of different internal
constants representing different possible components of a timestamp.

Alternatively (and I say this very hesitantly), we could try to rewrite
all the involved tests so that we don't need to try to remove DoW from
the timestamp.  But, I haven't figured out how to do all of them (they
are also of different styles, some with repeater, some with warning
period, some are bare, etc), and there are -- I think -- around 40 of
them that needs changing.

-- 
Best,


RY

[Please note that this mail might go to spam due to some
misconfiguration in my mail server -- still investigating.]

Re: [DRAFT PATCH v4] Decouple LANG= and testing (was: Test failure due to LANG)

[Ihor Radchenko]

Ruijie Yu via "General discussions about Org-mode."
 writes:

> Something very troublesome for me is `test-org-clock/clocktable/lang' on
> zh_CN.UTF-8.  What I noticed is that when I run this test in batch mode
> (one of the two methods below), it fails:

Well. Batch and non-batch modes are not the same, especially wrt display
things. And the differences are not consistent for different Emacs
versions. This only happens for weird edge cases though. (Or when
displaying images).

> +(defconst org-test-timestamp-regexp

I recommend `org-element-timestamp-parser' for analysis.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [DRAFT PATCH v4] Decouple LANG= and testing (was: Test failure due to LANG)

[General discussions about Org-mode.]

Axel Kielhorn  writes:

>> Am 23.04.2023 um 03:47 schrieb Ruijie Yu via General discussions about 
>> Org-mode. :
>> 
>> 
>> Ruijie Yu via "General discussions about Org-mode."  
>> writes:
>> 
>>> Axel Kielhorn  writes:
>>> 
 With this patch I get:
   FAILED  test-org-clock/clock-drawer-dwim  arrayp
   FAILED  test-org-clok/org-clock-timestamps-change  arrayp
 
 Axel
>>> 
>>> Thanks for testing.  Will take a look tomorrow.
>> 
>> Apparently `seq-map' does not preserve the type of its input.  This
>> patch be good then?
>> 
>> <0001-DRAFT-Fix-dependence-on-locale-in-org-testing-facili.patch>
>
> This works for me, thanks.
>
> Axel

Thank you for confirming that it works.

The following is an update of my progress trying to fix all issues I
found when I tested against different $LANG values.

I was able to iron out most of the errors found in my testing on
zh_CN.UTF-8.  The amount of unexpected failures went down from 8 to 2.

I also noticed that new errors are introduced in LANG=fr_FR.UTF-8 (was
1, now 5), so I will take a look at that for the next iteration.

Something very troublesome for me is `test-org-clock/clocktable/lang' on
zh_CN.UTF-8.  What I noticed is that when I run this test in batch mode
(one of the two methods below), it fails:

$ LANG=zh_CN.UTF-8 make test
$ LANG=zh_CN.UTF-8 emacs -Q -nw -batch \
   -L lisp -L testing/lisp \
   -l ert \
   -l testing/org-test.el \
   -l testing/lisp/test-org-clock.el \
   -f ert-run-tests-batch-and-exit

Whereas when I run it interactively, it succeeds:

$ LANG=zh_CN.UTF-8 emacs -Q -nw \
   -L lisp -L testing/lisp \
   -l ert \
   -l testing/org-test.el \
   -l testing/lisp/test-org-clock.el

M-x ert-i test-org-clock/clocktable/lang RET

Not sure what to make of it just yet.

The other failure I got in zh_CN.UTF-8 is
`test-org-colview/columns-width', as I mentioned somewhere up-thread.
It is caused by this s-exp:

(format "%-5.5s |" "1234…")

which returns "1234… |" when locale is en_US.UTF-8 or fr_FR.UTF-8, and
"1234 |" when locale is zh_CN.UTF-8 or ja_JA.UTF-8.

I think it might be due to the "perceived width" of the ellipsis, or
something?

Anyways, this is enough progress for now.  Here is the 4th iteration.
Note the very big rx regexp, where I asked in another thread whether my
understanding of a timestamp is complete.  Thanks.

>From d8525132d3f90623a6e8c732bd2a580ba23a2156 Mon Sep 17 00:00:00 2001
From: Ruijie Yu 
Date: Sat, 22 Apr 2023 20:36:18 +0800
Subject: [PATCH] [DRAFT] Fix dependence on locale in org testing facilities

* testing/org-test.el org-test-day-of-weeks-seconds: add values of
seconds that yield different days of week (by trial-and-error).
org-test-day-of-weeks-abbrev: add a vector of abbreviated DoW
names for testing.
org-test-day-of-weeks-full: add a vector of full DoW names for
testing.

* testing/lisp/test-org-clock.el
(test-org-clok/org-clock-timestamps-change):
(test-org-clock/clock-drawer-dwim): make use of the pre-generated
DoW names in the testing to remove assumptions on LANG=C.

* testing/lisp/test-org.el org-test-timestamp-regexp: make a comprehensive
timestamp regexp for testing.
(test-org/clone-with-time-shift): (test-org/add-planning-info):
(test-org/deadline): (test-org/schedule): replace the regexps which try to match
the DoW string in a timestamp into using `org-test-timestamp-regexp'.
---
 testing/lisp/test-org-clock.el |  62 +
 testing/lisp/test-org.el   | 231 +
 testing/org-test.el|  25 
 3 files changed, 177 insertions(+), 141 deletions(-)

diff --git a/testing/lisp/test-org-clock.el b/testing/lisp/test-org-clock.el
index 7d8123e1f..4d9306619 100644
--- a/testing/lisp/test-org-clock.el
+++ b/testing/lisp/test-org-clock.el
@@ -91,27 +91,32 @@ the buffer."
 
 (ert-deftest test-org-clok/org-clock-timestamps-change ()
   "Test `org-clock-timestamps-change' specifications."
-  (should
-   (equal
-"CLOCK: [2023-02-19 Sun 21:30]--[2023-02-19 Sun 23:35] =>  2:05"
-(org-test-with-temp-text
-"CLOCK: [2023-02-19 Sun 22:30]--[2023-02-20 Mon 00:35] =>  2:05"
-  (org-clock-timestamps-change 'down 1)
-  (buffer-string
-  (should
-   (equal
-"CLOCK: [2023-02-20 Mon 00:00]--[2023-02-20 Mon 00:40] =>  0:40"
-(org-test-with-temp-text
-"CLOCK: [2023-02-19 Sun 23:55]--[2023-02-20 Mon 00:35] =>  0:40"
-  (org-clock-timestamps-change 'up 1)
-  (buffer-string
-  (should
-   (equal
-"CLOCK: [2023-02-20 Mon 00:30]--[2023-02-20 Mon 01:35] =>  1:05"
-(org-test-with-temp-text
-"CLOCK: [2023-02-19 Sun 23:30]--[2023-02-20 Mon 00:35] =>  1:05"
-  (org-clock-timestamps-change 'up 1)
-  (buffer-string)
+  (let ((sun (aref org-test-day-of-weeks-abbrev 0))
+(mon (aref org-test-day-of-weeks-abbrev 1)))
+(should
+ (equal