Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On 10/19/2008 03:48 AM, Peter Stephenson wrote: Thanks, this tells me that the compfiles utility isn't handling Meta characters properly when performing matching. Luckily this seems to be a local problem within that command that I think the patch addresses. I can confirm that the completion problem has been solved on cvs head. Thanks for your work! -- Morita Sho [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On Wed, 15 Oct 2008 13:34:47 +0900
Morita Sho [EMAIL PROTECTED] wrote:
Thanks for detailed explanation! But I'm still confusing. Even with the old
syntax, the completion Ą/[TAB] won't work.
You're right. My explanation wasn't correct; it doesn't matter if you
use the m:{[lower]}={[upper]} syntax or the older one, so I've restored
the advice that the newer one is preferable.
BTW, to find out why replacing _path_files with the old one solves the problem
for me, I put a debug output into _path_files as below to watch the value of
$tmp1 before the $~tmp1 expansion.
print DEBUG: LINENO=$LINENO tmp1=$tmp1 /dev/stderr
tmp1=( $~tmp1 ) 2 /dev/null
When I type Ą/[TAB], I got this debug output.
DEBUG: LINENO=400 tmp1=ă�*(-/)
DEBUG: LINENO=400 tmp1=ă�*(-/)
Thanks, this tells me that the compfiles utility isn't handling Meta
characters properly when performing matching. Luckily this seems to be
a local problem within that command that I think the patch addresses.
Index: Doc/Zsh/compwid.yo
===
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/compwid.yo,v
retrieving revision 1.44
diff -u -r1.44 compwid.yo
--- Doc/Zsh/compwid.yo 14 Oct 2008 12:59:04 - 1.44
+++ Doc/Zsh/compwid.yo 18 Oct 2008 18:44:08 -
@@ -942,16 +942,14 @@
completion you can use `tt(m:{[:lower:]}={[:upper:]})'. Although the
matching system does not yet handle multibyte characters, this is likely
to be a future extension, at which point this syntax will handle
-arbitrary alphabets; until then it is safer to use the older syntax
-that only handles ASCII characters, `tt(m:{a-z}={A-Z}) as this does
-not have side effects in the case of multibyte characters.
-
-In other cases `tt([:)var(name)tt(:])' forms are allowed. If the two forms
-on the left and right are the same, the characters must match exactly. In
-remaining cases, the corresponding tests are applied to both characters,
-but they are not otherwise constrained; any matching character in one set
-goes with any matching character in the other set: this is equivalent to
-the behaviour of ordinary character classes.
+arbitrary alphabets; hence this form, rather than the use of explicit
+ranges, is the recommended form. In other cases
+`tt([:)var(name)tt(:])' forms are allowed. If the two forms on the left
+and right are the same, the characters must match exactly. In remaining
+cases, the corresponding tests are applied to both characters, but they
+are not otherwise constrained; any matching character in one set goes
+with any matching character in the other set: this is equivalent to the
+behaviour of ordinary character classes.
The pattern var(tpat) may also be one or two stars, `tt(*)' or
`tt(**)'. This means that the pattern on the command line can match
Index: Src/Zle/computil.c
===
RCS file: /cvsroot/zsh/zsh/Src/Zle/computil.c,v
retrieving revision 1.110
diff -u -r1.110 computil.c
--- Src/Zle/computil.c 14 Oct 2008 22:09:11 - 1.110
+++ Src/Zle/computil.c 18 Oct 2008 18:44:09 -
@@ -4024,7 +4024,7 @@
* management is difficult.
*/
for (;;) {
- for (mp = ms; *add; add++, mp++) {
+ for (mp = ms; *add; ) {
if (!(m = *mp)) {
/*
* No matcher, so just match the character
@@ -4034,13 +4034,13 @@
* metacharacter?
*/
if (ret) {
- if (imeta(*add)) {
+ if (*add == Meta) {
*p++ = Meta;
- *p++ = *add ^ 32;
+ *p++ = add[1];
} else
*p++ = *add;
} else
- len += imeta(*add) ? 2 : 1;
+ len += (*add == Meta) ? 2 : 1;
} else if (m-flags CMF_RIGHT) {
/*
* Right-anchored: match anything followed
@@ -4049,15 +4049,16 @@
if (ret) {
*p++ = '*';
/* TODO: quote again? */
- if (imeta(*add)) {
+ if (*add == Meta) {
*p++ = Meta;
- *p++ = *add ^ 32;
+ *p++ = add[1];
} else
*p++ = *add;
} else
- len += imeta(*add) ? 3 : 2;
+ len += (*add == Meta) ? 3 : 2;
} else {
/* The usual set of matcher possibilities. */
+ int chr = (*add == Meta) ? add[1] ^ 32 : *add;
int ind;
if (m-line-tp == CPAT_EQUIV
m-word-tp == CPAT_EQUIV) {
@@ -4072,21 +4073,21 @@
*/
if (ret) {
*p++ = '[';
- if (imeta(*add)) {
+ if (*add == Meta) {
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On Tue, 14 Oct 2008 10:46:29 +0900
Morita Sho [EMAIL PROTECTED] wrote:
zstyle ':completion:*' matcher-list 'm:{[:lower:]}={[:upper:]}'
After removing the above line, the completion Ą/[TAB] works again.
Thanks for narrowing it down... I'm aware that that syntax still needs
some work on it, it's only half-converted to supporting non-ASCII
characters, so it's probably not surprising this sort of thing happens.
The problem is the way characters are stored is quite specific to a
single-byte representation; I've made it so that it can be extended to
multibyte/wide characters, but it hasn't been.
I'm not going to get it properly converted before the next release since
it's a big job (I started it once back in June and got bogged down). When
I get a chance, I'll look and see if there's any thing obvious I can do
with Meta-encoded characters that will work round the problem for now.
Until then you should be able to get away with the old syntax,
zstyle ':completion:*' matcher-list 'm:{a-z}={A-Z}'
since before I finish the job it won't actually handle non-ASCII character
conversions properly anyway.
I will document my way round this as follows...
Index: Doc/Zsh/compwid.yo
===
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/compwid.yo,v
retrieving revision 1.43
diff -u -r1.43 compwid.yo
--- Doc/Zsh/compwid.yo 14 Jul 2008 17:39:26 - 1.43
+++ Doc/Zsh/compwid.yo 14 Oct 2008 12:01:54 -
@@ -942,14 +942,16 @@
completion you can use `tt(m:{[:lower:]}={[:upper:]})'. Although the
matching system does not yet handle multibyte characters, this is likely
to be a future extension, at which point this syntax will handle
-arbitrary alphabets; hence this form, rather than the use of explicit
-ranges, is the recommended form. In other cases
-`tt([:)var(name)tt(:])' forms are allowed. If the two forms on the left
-and right are the same, the characters must match exactly. In remaining
-cases, the corresponding tests are applied to both characters, but they
-are not otherwise constrained; any matching character in one set goes
-with any matching character in the other set: this is equivalent to the
-behaviour of ordinary character classes.
+arbitrary alphabets; until then it is safer to use the older syntax
+that only handles ASCII characters, `tt(m:{a-z}={A-Z}) as this does
+not have side effects in the case of multibyte characters.
+
+In other cases `tt([:)var(name)tt(:])' forms are allowed. If the two forms
+on the left and right are the same, the characters must match exactly. In
+remaining cases, the corresponding tests are applied to both characters,
+but they are not otherwise constrained; any matching character in one set
+goes with any matching character in the other set: this is equivalent to
+the behaviour of ordinary character classes.
The pattern var(tpat) may also be one or two stars, `tt(*)' or
`tt(**)'. This means that the pattern on the command line can match
--
Peter Stephenson [EMAIL PROTECTED] Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On 10/14/2008 09:04 PM, Peter Stephenson wrote:
Until then you should be able to get away with the old syntax,
zstyle ':completion:*' matcher-list 'm:{a-z}={A-Z}'
since before I finish the job it won't actually handle non-ASCII character
conversions properly anyway.
Thanks for detailed explanation! But I'm still confusing. Even with the old
syntax, the completion Ą/[TAB] won't work. As far as I understand, the following
configuration should fix the completion problem, but it won't.
% cat ~/.zshrc
# Lines configured by zsh-newuser-install
HISTFILE=~/.histfile
HISTSIZE=1000
SAVEHIST=1000
bindkey -e
# End of lines configured by zsh-newuser-install
# The following lines were added by compinstall
#zstyle ':completion:*' matcher-list 'm:{[:lower:]}={[:upper:]}'
zstyle ':completion:*' matcher-list 'm:{a-z}={A-Z}'
zstyle :compinstall filename '/home/qw/.zshrc'
autoload -Uz compinit
compinit
# End of lines added by compinstall
BTW, to find out why replacing _path_files with the old one solves the problem
for me, I put a debug output into _path_files as below to watch the value of
$tmp1 before the $~tmp1 expansion.
print DEBUG: LINENO=$LINENO tmp1=$tmp1 /dev/stderr
tmp1=( $~tmp1 ) 2 /dev/null
When I type Ą/[TAB], I got this debug output.
DEBUG: LINENO=400 tmp1=ă�*(-/)
DEBUG: LINENO=400 tmp1=ă�*(-/)
Afterward, I replaced _path_files with the old one (which is from zsh 4.3.6),
and put the same debug output, then I got this debug output.
DEBUG: LINENO=372 tmp1=ă�*(-/)
DEBUG: LINENO=372 tmp1=Ą/*
It seems that tmp1 has a invalid (meta-encoded?) character. In _path_files which
is from zsh 4.3.6, they seems to retry with a normal (non-meta-encoded?)
character.
Unfortunately, I couldn't understand why they are differ
--
Morita Sho [EMAIL PROTECTED]
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On Mon, Oct 13, 2008 at 03:32, Morita Sho [EMAIL PROTECTED] wrote: Replacing _path_files with previous one (zsh 4.3.6) seems to solve the problem. I'm not sure, but it might be a regression in _path_files. Just to be sure, you are talking about the second part here, not both ones, correct? Richard -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On Mon, Oct 13, 2008 at 11:30, Peter Stephenson [EMAIL PROTECTED] wrote: Please send the output from running ^X? if there no obvious difference. Whoah, neat! I 'fear' I will never stop being amazed at what the ZSH devs think of.. :) Richard -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On Mon, 13 Oct 2008 10:32:05 +0900 Morita Sho [EMAIL PROTECTED] wrote: Additionally, Ą/[TAB] completion doesn't work on cvs head. (It was working on zsh 4.3.6) % mkdir Ą % touch Ą/foo % ls Ą/[TAB] = Nothing completes. This works fine for me (as does the version with () after the previous fix). Does the same still happen after zsh -f; autoload -U compinit; compinit? Is the character the composed version, i.e. Unicode 0x0104, which is 0xc4 0x84 in UTF-8, or could it be some decomposed version with combining characters? Please send the output from running ^X? if there no obvious difference. -- Peter Stephenson [EMAIL PROTECTED] Software Engineer CSR PLC, Churchill House, Cambridge Business Park, Cowley Road Cambridge, CB4 0WZ, UK Tel: +44 (0)1223 692070 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On 10/13/2008 03:25 PM, Richard Hartmann wrote:
Just to be sure, you are talking about the second part here, not both ones,
correct?
Sorry for ambiguous. I'm talking about both ones. Replacing _path_files solves
both '()Ą'/[TAB] and Ą/[TAB] problems.
On 10/13/2008 06:30 PM, Peter Stephenson wrote:
This works fine for me (as does the version with () after the previous
fix). Does the same still happen after zsh -f; autoload -U compinit;
compinit? Is the character the composed version, i.e. Unicode 0x0104,
which is 0xc4 0x84 in UTF-8, or could it be some decomposed version with
combining characters?
Please send the output from running ^X? if there no obvious difference.
Thanks! The problem gone after zsh -f; autoload -U compinit; compinit.
The directory 'Ą' is exactly 0xc4 0x84.
% ls
Ą
% ls | hexdump -C
c4 84 0a |...|
I removed and reconstructed ~/.zshrc to find out what settings in my ~/.zshrc
break the completion.
The completion Ą/[TAB] works fine with the following .zshrc.
% cat ~/.zshrc
# Lines configured by zsh-newuser-install
HISTFILE=~/.histfile
HISTSIZE=1000
SAVEHIST=1000
bindkey -e
# End of lines configured by zsh-newuser-install
# The following lines were added by compinstall
zstyle :compinstall filename '/home/qw/.zshrc'
autoload -Uz compinit
compinit
# End of lines added by compinstall
However, if I run `compinstall` and enable Case-insensitive completion
(lowercase matches uppercase) from 2. Matching control menu, the completion
Ą/[TAB] stop to working.
I see the following line were added to ~/.zshrc.
zstyle ':completion:*' matcher-list 'm:{[:lower:]}={[:upper:]}'
After removing the above line, the completion Ą/[TAB] works again.
--
Morita Sho [EMAIL PROTECTED]
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On 10/12/2008 06:52 AM, Peter Stephenson wrote: The pattern gets divided up into chunks because of the backslashed characters, but we don't report that it wasn't a pure string on that basis, so the directory trise to match the wrong string. This was triggered in this case because of the character in a range that zsh has to quote internally to avoid clashing with tokens, which is why it occurred with some forms of accent but not others. I could optimise this better but this is a simple change for now. Thanks for your work. I have checkouted the latest source code from cvs head. I can see $~tmp1 expansion works correctly, but '()Ą'/[TAB] completion still not works. % mkdir '()Ą' % touch '()Ą'/foo % ls '()Ą'/[TAB] = Nothing completes. % tmp1='\(\)Ą/*' % print $~tmp1 ()Ą/foo Additionally, Ą/[TAB] completion doesn't work on cvs head. (It was working on zsh 4.3.6) % mkdir Ą % touch Ą/foo % ls Ą/[TAB] = Nothing completes. Replacing _path_files with previous one (zsh 4.3.6) seems to solve the problem. I'm not sure, but it might be a regression in _path_files. Regards, -- Morita Sho [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On Sat, Oct 11, 2008 at 01:17, Clint Adams [EMAIL PROTECTED] wrote: = I expected it completes foo, but it completes nothing. Confirmed for 4.3.6 on Debian unstable. Richard -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
Thanks for your quick reply. I found that the difference between '()Ą'/[TAB] and '()Ā'/[TAB] is at line 372 of /usr/share/zsh/functions/Completion/Unix/_path_files. tmp1=( $~tmp1 ) 2 /dev/null When I typed '()Ą'/[TAB], tmp1 will be '\(\)Ą/*', and $~tmp1 is expanded to nothing. When I typed '()Ā'/[TAB], tmp1 will be '\(\)Ā/*', and $~tmp1 is expanded to ()Ā/foo. $~tmp1 expansion seems not working correctly for a pattern '\(\)Ą/*'. % ls -R .: ()Ā ()Ą ./()Ā: foo ./()Ą: foo % tmp1='\(\)Ą/*'; print $~tmp1; zsh: no matches found: \(\)Ą/* % tmp1='\(\)Ā/*'; print $~tmp1; ()Ā/foo But `print \(\)Ą/*` is correctly expanded to ()Ą/foo. % print \(\)Ą/* ()Ą/foo Regards, -- Morita Sho [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On Sat, 11 Oct 2008 22:47:52 +0900
Morita Sho [EMAIL PROTECTED] wrote:
$~tmp1 expansion seems not working correctly for a pattern '\(\)Ą/*'.
Thanks for the clear analysis.
The pattern gets divided up into chunks because of the backslashed
characters, but we don't report that it wasn't a pure string on that
basis, so the directory trise to match the wrong string. This was triggered
in this case because of the character in a range that zsh has to quote
internally to avoid clashing with tokens, which is why it occurred with
some forms of accent but not others.
I could optimise this better but this is a simple change for now.
Index: Src/pattern.c
===
RCS file: /cvsroot/zsh/zsh/Src/pattern.c,v
retrieving revision 1.47
diff -u -r1.47 pattern.c
--- Src/pattern.c 8 Jun 2008 17:53:55 - 1.47
+++ Src/pattern.c 11 Oct 2008 21:44:35 -
@@ -1446,7 +1446,13 @@
* Marker for restoring a backslash in output:
* does not match a character.
*/
- return patcomppiece(flagp);
+ next = patcomppiece(flagp);
+ /*
+* Can't match a pure string since we need to do this
+* as multiple chunks.
+*/
+ *flagp = ~P_PURESTR;
+ return next;
break;
#ifdef DEBUG
default:
Index: Test/D07multibyte.ztst
===
RCS file: /cvsroot/zsh/zsh/Test/D07multibyte.ztst,v
retrieving revision 1.25
diff -u -r1.25 D07multibyte.ztst
--- Test/D07multibyte.ztst 26 Mar 2008 15:26:20 - 1.25
+++ Test/D07multibyte.ztst 11 Oct 2008 21:44:35 -
@@ -395,3 +395,14 @@
OK
OK
OK
+
+ mkdir glob
+ mkdir glob/'()Ą' glob/'()Ā'
+ mkdir glob/'()Ą'/foo glob/'()Ā'/bar
+ tmp1=('glob/\(\)Ą/*')
+ print $~tmp1
+ tmp1=('glob/\(\)Ā/*')
+ print $~tmp1
+0:Backslashes and metafied characters in patterns
+glob/()Ą/foo
+glob/()Ā/bar
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
Package: zsh
Version: 4.3.6-7
Severity: normal
Hi,
Completion fails if a directory name contains '(', ')' and 'Ą '.
'Ą ' is U+0104 in unicode.
Following is a small example to reproduce the problem.
% mkdir '()Ą'
% touch '()Ą'/foo
% ls '()Ą'/[TAB]
= I expected it completes foo, but it completes nothing.
If I renamed the directory name slightly, e.g. remove a parenthesis or
change 'Ą' to 'Ā ' (U+0100), it correctly completes foo.
Regards,
-- System Information:
Debian Release: lenny/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: i386 (i686)
Kernel: Linux 2.6.26-1-686 (SMP w/1 CPU core)
Locale: LANG=ja_JP.UTF-8, LC_CTYPE=ja_JP.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Versions of packages zsh depends on:
ii libc6 2.7-14 GNU C Library: Shared libraries
ii libcap2 2.11-2 support for getting/setting POSIX.
ii libncursesw5 5.6+20080925-1 shared libraries for terminal hand
Versions of packages zsh recommends:
ii libpcre3 7.8-2 Perl 5 Compatible Regular Expressi
Versions of packages zsh suggests:
pn zsh-doc none (no description available)
-- no debconf information
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#501851: zsh: Completion fails if a directory name contains '(', ')' and 'Ą '.
On Sat, Oct 11, 2008 at 08:06:11AM +0900, Morita Sho wrote:
Completion fails if a directory name contains '(', ')' and 'Ą '.
'Ą ' is U+0104 in unicode.
Following is a small example to reproduce the problem.
% mkdir '()Ą'
% touch '()Ą'/foo
% ls '()Ą'/[TAB]
= I expected it completes foo, but it completes nothing.
If I renamed the directory name slightly, e.g. remove a parenthesis or
change 'Ą' to 'Ā ' (U+0100), it correctly completes foo.
I get different but also unuseful behavior trying to reproduce that.
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
