Re: Strange compgen behaviour
I decided to make this into a blog post to in some way contribute back to the community for all the help I got. http://muublog.blogspot.com/2009/10/bash-completion-for-playing-movies.html Enjoy! /Mathias
Re: Strange compgen behaviour
Chet Ramey schrieb:
Bernd Eggink wrote:
Chet Ramey schrieb:
Hm, compgen appears to behave strange if words contain whitespace.
Well, it splits the argument to -W on $IFS as documented. What other
strange behavior do you see?
For example, this:
function _aha
{
local list="a b:c d:e f"
COMPREPLY=($(IFS=: compgen -W "$list"))
}
complete -F _aha aha
Typing aha cycles through 6 items a, b, c, d, e, f, whereas I
would expect 3 items 'a b', 'c d', 'e f'. It looks like compgen splits
the argument to -W on $IFS _and_ whitespace. Or am I missing something?
I don't think you're considering the interaction between compgen,
command substitution, and compound array assignment.
I'll use the assignment to `list' you used above for the explanation.
compgen performs word splitting using $IFS and then outputs the results,
one word per line:
$ IFS=: compgen -W "$list"
a b
c d
e f
Command substitution takes that, chops off the last newline, and, since
it's unquoted, performs word splitting using $IFS on the result:
$ recho $(IFS=: compgen -W "$list")
argv[1] =
argv[2] =
argv[3] =
argv[4] =
argv[5] =
argv[6] =
If you want to preserve the embedded newlines, you have to quote the
expansion (but see below):
$ recho "$(IFS=: compgen -W "$list")"
argv[1] =
When performing compound array assignments, though, the words between
the parens undergo all the shell word expansions, including word splitting:
$ COMPREPLY=( $(IFS=: compgen -W "$list") )
$ recho "${comprep...@]}"
argv[1] =
argv[2] =
argv[3] =
argv[4] =
argv[5] =
argv[6] =
So you can split on just the embedded newlines by setting $IFS:
$ oifs="$IFS"
$ IFS=$'\n' COMPREPLY=( $(IFS=: compgen -W "$list") )
$ IFS="$oifs"
$ recho "${comprep...@]}"
argv[1] =
argv[2] =
argv[3] =
Remember to save and restore $IFS, though, since the assignment to it
preceding the assignment to COMPREPLY will persist.
Chet,
thanks for the clarification. It's actually quite obvious and I should
have seen it myself.
Regards,
Bernd
Re: Strange compgen behaviour
> printf "%q\n" works fine here.
It does here too :), I misunderstood the syntax.
> Also, grep "${cur}" could probably be replaced by -path "*/${cur}*".
Indeed it can.
Smaller = better, thanks!
/Mathias
Re: Strange compgen behaviour
Bernd Eggink wrote:
> Chet Ramey schrieb:
>>> Hm, compgen appears to behave strange if words contain whitespace.
>>
>> Well, it splits the argument to -W on $IFS as documented. What other
>> strange behavior do you see?
>
> For example, this:
>
> function _aha
> {
> local list="a b:c d:e f"
> COMPREPLY=($(IFS=: compgen -W "$list"))
> }
>
> complete -F _aha aha
>
> Typing aha cycles through 6 items a, b, c, d, e, f, whereas I
> would expect 3 items 'a b', 'c d', 'e f'. It looks like compgen splits
> the argument to -W on $IFS _and_ whitespace. Or am I missing something?
I don't think you're considering the interaction between compgen,
command substitution, and compound array assignment.
I'll use the assignment to `list' you used above for the explanation.
compgen performs word splitting using $IFS and then outputs the results,
one word per line:
$ IFS=: compgen -W "$list"
a b
c d
e f
Command substitution takes that, chops off the last newline, and, since
it's unquoted, performs word splitting using $IFS on the result:
$ recho $(IFS=: compgen -W "$list")
argv[1] =
argv[2] =
argv[3] =
argv[4] =
argv[5] =
argv[6] =
If you want to preserve the embedded newlines, you have to quote the
expansion (but see below):
$ recho "$(IFS=: compgen -W "$list")"
argv[1] =
When performing compound array assignments, though, the words between
the parens undergo all the shell word expansions, including word splitting:
$ COMPREPLY=( $(IFS=: compgen -W "$list") )
$ recho "${comprep...@]}"
argv[1] =
argv[2] =
argv[3] =
argv[4] =
argv[5] =
argv[6] =
So you can split on just the embedded newlines by setting $IFS:
$ oifs="$IFS"
$ IFS=$'\n' COMPREPLY=( $(IFS=: compgen -W "$list") )
$ IFS="$oifs"
$ recho "${comprep...@]}"
argv[1] =
argv[2] =
argv[3] =
Remember to save and restore $IFS, though, since the assignment to it
preceding the assignment to COMPREPLY will persist.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Strange compgen behaviour
Mathias Dahl writes:
> Got this to work:
>
> files=$(find /home/mathias/Videos/movies/ -iname "*.avi" -type f -
> printf "%P\n" | grep "${cur}" | while read file; do
> printf %q "$file"
> echo
> done)
>
> With the %q option to printf it no longer accepts a \n so I needed the
> extra echo.
printf "%q\n" works fine here.
Also, grep "${cur}" could probably be replaced by -path "*/${cur}*".
Andreas.
--
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."
Re: Strange compgen behaviour
> Interesting! Can I make it part of my pipe (in place of `sed') or need
> I change it to a for or while loop? This is what I have now:
>
> files=$(find /home/mathias/Videos/movies/ -iname "*.avi" -type f -
> printf "%P\n" | grep "${cur}" | sed "s/\\([][\\(\\) ,\']\\)/\\1/
> g")
Got this to work:
files=$(find /home/mathias/Videos/movies/ -iname "*.avi" -type f -
printf "%P\n" | grep "${cur}" | while read file; do
printf %q "$file"
echo
done)
With the %q option to printf it no longer accepts a \n so I needed the
extra echo.
/Mathias
Re: Strange compgen behaviour
> printf %q "$filename"
>
> will either insert backslashes in front of all the shell metacharacters,
> or $'...' quote the whole thing, or take some other action which renders
> a string "safe". It's basically the opposite of eval.
Interesting! Can I make it part of my pipe (in place of `sed') or need
I change it to a for or while loop? This is what I have now:
files=$(find /home/mathias/Videos/movies/ -iname "*.avi" -type f -
printf "%P\n" | grep "${cur}" | sed "s/\\([][\\(\\) ,\']\\)/\\1/
g")
/Mathias
Re: Strange compgen behaviour
> This has been an interesting topic!
I thought I should share the final version as well:
_mm() {
local cur files
COMPREPLY=()
cur="${COMP_WORDS[COMP_CWORD]}"
files=$(find /home/mathias/Videos/movies/ -iname "*.avi" -type f -
printf "%P\n" | grep "${cur}" | sed "s/\\([][\\(\\) ,\']\\)/\\1/
g")
local IFS=$'\n'
COMPREPLY=(${files})
}
complete -F _mm mm
And the `mm' script needs to append the root folder, of course, like
so:
#!/bin/bash
mplayer "/home/mathias/Videos/movies/$1"
Works quite well. Maybe it will be useful to someone else too.
/Mathias
Re: Strange compgen behaviour
On Thu, Sep 24, 2009 at 03:05:07PM -0700, Mathias Dahl wrote: > I did not find any generic way to quote/escape file names so I > hardcoded some chars I know exist in my file names and used sed: printf %q "$filename" will either insert backslashes in front of all the shell metacharacters, or $'...' quote the whole thing, or take some other action which renders a string "safe". It's basically the opposite of eval.
Re: Strange compgen behaviour
> Hm, I can't see any problem here. My version lets you pick any file in > any subdir by simply typing the name (or part of it) without the > directory part. After all, 'find -name' matches names, not paths (if you > want to match full paths, use 'find -path'). I'd also rather use printf > "%P\n" (capital P) instead of %p, the results look nicer (IMHO). When I try with %P it works (better). However, when I do that and use the -o filenames option I see only the filenames of the sub folders and sometimes that name does not reflect what it is as well as the folder does (if for example I have a folder called Cool Movie 2 and in it a file called thefile.avi). So, it seems I did not really want what I thought I wanted from the beginning :) The output I get now looks like this: ... filie_in_root_folder.avi Some sub folder/file_in_sub_folder.avi ... The only annoyance now is the hardcoded quoting (to handle spaces and other "bad" chars) I do using sed, but I guess I have to live with it. This has been an interesting topic! /Mathias
Re: Strange compgen behaviour
Mathias Dahl schrieb:
Hm, compgen appears to behave strange if words contain whitespace.
However, you don't need it, as you build the list yourself. Try this:
_mm2() {
local cur files
cur=${COMP_WORDS[COMP_CWORD]}
files=$(find /home/mathias/Videos/movies/ -iname "$cur*.avi" -type
f -printf "%P\n")
local IFS=$'\n'
COMPREPLY=($files)
}
Ah, you're right of course, I can do the matching myself. I have yet
another version working now (had to change your latest suggestion and
use grep for matching because -name does not like full paths which
becomes the case here):
_mm() {
local cur files
COMPREPLY=()
cur="${COMP_WORDS[COMP_CWORD]}"
files=$(find /home/mathias/Videos/movies/ -iname "*.avi" -type f -
printf "%p\n" | grep "${cur}")
local IFS=$'\n'
COMPREPLY=(${files})
}
complete -o filenames -F _mm mm
Now, this works almost. The remaining problem is that because `find'
finds file in subdirs (which I want, otherwise I could add the -
maxdepth option) as well, the `-o filenames' argument to `complete'
does not play well with it. I see the names of files in subdirs listed
when I type TAB (without path) but can never pick them without knowing
the name of the folder they are in.
Hm, I can't see any problem here. My version lets you pick any file in
any subdir by simply typing the name (or part of it) without the
directory part. After all, 'find -name' matches names, not paths (if you
want to match full paths, use 'find -path'). I'd also rather use printf
"%P\n" (capital P) instead of %p, the results look nicer (IMHO).
Regards,
Bernd
--
Bernd Eggink
http://sudrala.de
Re: Strange compgen behaviour
> ...but then I have to shell quote the file name myself to handle
> spaces, brackets of various sorts, comma characters etc. Will hunt for
> such a function and see. There are all sorts of crazy helper functions
> in /etc/bash_completion, of which I barely understand anything.
I did not find any generic way to quote/escape file names so I
hardcoded some chars I know exist in my file names and used sed:
_mm() {
local cur files
COMPREPLY=()
cur="${COMP_WORDS[COMP_CWORD]}"
files=$(find /home/mathias/Videos/movies/ -iname "*.avi" -type f -
printf "%p\n" | grep "${cur}" | sed 's/\([\(\) ,]\)/\\\1/g')
local IFS=$'\n'
COMPREPLY=(${files})
}
complete -F _mm mm
However, I don't like it, is feels ugly and one of these days there
will be some other funky char in some filename... :(
/Mathias
Re: Strange compgen behaviour
> Hm, compgen appears to behave strange if words contain whitespace.
> However, you don't need it, as you build the list yourself. Try this:
>
> _mm2() {
> local cur files
> cur=${COMP_WORDS[COMP_CWORD]}
> files=$(find /home/mathias/Videos/movies/ -iname "$cur*.avi" -type
> f -printf "%P\n")
> local IFS=$'\n'
> COMPREPLY=($files)
> }
Ah, you're right of course, I can do the matching myself. I have yet
another version working now (had to change your latest suggestion and
use grep for matching because -name does not like full paths which
becomes the case here):
_mm() {
local cur files
COMPREPLY=()
cur="${COMP_WORDS[COMP_CWORD]}"
files=$(find /home/mathias/Videos/movies/ -iname "*.avi" -type f -
printf "%p\n" | grep "${cur}")
local IFS=$'\n'
COMPREPLY=(${files})
}
complete -o filenames -F _mm mm
Now, this works almost. The remaining problem is that because `find'
finds file in subdirs (which I want, otherwise I could add the -
maxdepth option) as well, the `-o filenames' argument to `complete'
does not play well with it. I see the names of files in subdirs listed
when I type TAB (without path) but can never pick them without knowing
the name of the folder they are in. So I have to get rid of that
option, but then I have to shell quote the file name myself to handle
spaces, brackets of various sorts, comma characters etc. Will hunt for
such a function and see. There are all sorts of crazy helper functions
in /etc/bash_completion, of which I barely understand anything.
Thanks!
Re: Strange compgen behaviour
On Thu, 24 Sep 2009, Bernd Eggink wrote:
> Chet Ramey schrieb:
> > > Hm, compgen appears to behave strange if words contain whitespace.
> >
> > Well, it splits the argument to -W on $IFS as documented. What other
> > strange behavior do you see?
>
> For example, this:
>
> function _aha
> {
> local list="a b:c d:e f"
> COMPREPLY=($(IFS=: compgen -W "$list"))
That expands to:
COMPREPLY=( a b c d e f )
> }
>
> complete -F _aha aha
>
> Typing aha cycles through 6 items a, b, c, d, e, f, whereas I would
> expect 3 items 'a b', 'c d', 'e f'. It looks like compgen splits the argument
> to -W on $IFS _and_ whitespace. Or am I missing something?
--
Chris F.A. Johnson, webmaster
===
Author:
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
Re: Strange compgen behaviour
Bernd Eggink wrote:
> Chet Ramey schrieb:
>>> Hm, compgen appears to behave strange if words contain whitespace.
>>
>> Well, it splits the argument to -W on $IFS as documented. What other
>> strange behavior do you see?
>
> For example, this:
>
> function _aha
> {
> local list="a b:c d:e f"
> COMPREPLY=($(IFS=: compgen -W "$list"))
> }
>
> complete -F _aha aha
>
> Typing aha cycles through 6 items a, b, c, d, e, f, whereas I
> would expect 3 items 'a b', 'c d', 'e f'. It looks like compgen splits
> the argument to -W on $IFS _and_ whitespace. Or am I missing something?
Hmmm...let me take a look.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Strange compgen behaviour
Chet Ramey schrieb:
Hm, compgen appears to behave strange if words contain whitespace.
Well, it splits the argument to -W on $IFS as documented. What other
strange behavior do you see?
For example, this:
function _aha
{
local list="a b:c d:e f"
COMPREPLY=($(IFS=: compgen -W "$list"))
}
complete -F _aha aha
Typing aha cycles through 6 items a, b, c, d, e, f, whereas I
would expect 3 items 'a b', 'c d', 'e f'. It looks like compgen splits
the argument to -W on $IFS _and_ whitespace. Or am I missing something?
Regards,
Bernd
--
Bernd Eggink
http://sudrala.de
Re: Strange compgen behaviour
> Hm, compgen appears to behave strange if words contain whitespace. Well, it splits the argument to -W on $IFS as documented. What other strange behavior do you see? Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Strange compgen behaviour
Mathias Dahl schrieb:
It depends heavily on how the variables IFS and zf are set. From 'man bash':
-W wordlist
The wordlist is split using the characters in the IFS special
variable as delimiters, and each resultant word is expanded.
The possible completions are the members of the resultant
list which match the word being completed.
I used a newline since the original listing comes from `find'.
You didn't say how you assigned the variable zf. If you simply did
zf=$(ls /home/mathias/Videos/movies/*), the "Brazil" line will be split
into 4 words instead of 1. However, your output suggest that you somehow
managed to combine all file names to a single word starting with
Harry.Potter.
Yes, that could be the case.
Try this: Choose a character which doesn't appear in any file name,
e.g., ':'.
list=$(printf "%s:" /home/mathias/Videos/movies/*)
IFS=: compgen -W "$list" -- $zc
That works, thanks! However, I also want files from sub folders to be
found, so I use `find' to list them.
Here is my latest attempt, using the idea of setting IFS to `:':
_mm2() {
local cur files
COMPREPLY=()
cur="${COMP_WORDS[COMP_CWORD]}"
files=$(find /home/mathias/Videos/movies/ -iname '*.avi' -type f -
printf "%p:")
OLDIFS=$IFS
IFS=:
COMPREPLY=($(compgen -W "${files}" -- ${cur}))
IFS=$OLDIFS
}
complete -o filenames -F _mm2 mm
Looks like it should work but it does not. Typing mm gives
the listing and completes all the way to the path, but if I add B
again it does not match Brazil.
Hm, compgen appears to behave strange if words contain whitespace.
However, you don't need it, as you build the list yourself. Try this:
_mm2() {
local cur files
cur=${COMP_WORDS[COMP_CWORD]}
files=$(find /home/mathias/Videos/movies/ -iname "$cur*.avi" -type
f -printf "%P\n")
local IFS=$'\n'
COMPREPLY=($files)
}
Regards,
Bernd
--
Bernd Eggink
http://sudrala.de
Re: Strange compgen behaviour
> It depends heavily on how the variables IFS and zf are set. From 'man bash':
>
> -W wordlist
> The wordlist is split using the characters in the IFS special
> variable as delimiters, and each resultant word is expanded.
> The possible completions are the members of the resultant
> list which match the word being completed.
I used a newline since the original listing comes from `find'.
> You didn't say how you assigned the variable zf. If you simply did
> zf=$(ls /home/mathias/Videos/movies/*), the "Brazil" line will be split
> into 4 words instead of 1. However, your output suggest that you somehow
> managed to combine all file names to a single word starting with
> Harry.Potter.
Yes, that could be the case.
> Try this: Choose a character which doesn't appear in any file name,
> e.g., ':'.
>
> list=$(printf "%s:" /home/mathias/Videos/movies/*)
> IFS=: compgen -W "$list" -- $zc
That works, thanks! However, I also want files from sub folders to be
found, so I use `find' to list them.
Here is my latest attempt, using the idea of setting IFS to `:':
_mm2() {
local cur files
COMPREPLY=()
cur="${COMP_WORDS[COMP_CWORD]}"
files=$(find /home/mathias/Videos/movies/ -iname '*.avi' -type f -
printf "%p:")
OLDIFS=$IFS
IFS=:
COMPREPLY=($(compgen -W "${files}" -- ${cur}))
IFS=$OLDIFS
}
complete -o filenames -F _mm2 mm
Looks like it should work but it does not. Typing mm gives
the listing and completes all the way to the path, but if I add B
again it does not match Brazil.
Any ideas?
Re: Strange compgen behaviour
Mathias Dahl schrieb:
Hi fellow bashers!
I am trying to add some completion to a command. The completion should
list all movies I have in a certain folder, regardless if I am in that
folder or not. I have kind of got it to work in several variants but
all have some issue. The current problem I am looking at is very
strange. I have isolated it down to a strange behaviour with compgen.
Let's see if I can describe it clearly enough:
zf contains the list of movie file names
zc is the current input, in my case "/home/mathias/Videos/movies/H"
$ compgen -W "${zf}" -- ${zc}
Here is the output:
/home/mathias/Videos/movies/Harry.Potter...
/home/mathias/Videos/movies/Harry.Potter...
/home/mathias/Videos/movies/Harry.Potter...
/home/mathias/Videos/movies/Harry.Potter...
/home/mathias/Videos/movies/Harry.Potter...
/home/mathias/Videos/movies/Brazil (Terry Gilliam, 1985).avi
/home/mathias/Videos/movies/Ice.Age.2...
/home/mathias/Videos/movies/True.Blood.S01...
/home/mathias/Videos/movies/True.Blood.S01...
It depends heavily on how the variables IFS and zf are set. From 'man bash':
-W wordlist
The wordlist is split using the characters in the IFS special
variable as delimiters, and each resultant word is expanded.
The possible completions are the members of the resultant
list which match the word being completed.
You didn't say how you assigned the variable zf. If you simply did
zf=$(ls /home/mathias/Videos/movies/*), the "Brazil" line will be split
into 4 words instead of 1. However, your output suggest that you somehow
managed to combine all file names to a single word starting with
Harry.Potter.
Try this: Choose a character which doesn't appear in any file name,
e.g., ':'.
list=$(printf "%s:" /home/mathias/Videos/movies/*)
IFS=: compgen -W "$list" -- $zc
Regards,
Bernd
--
Bernd Eggink
http://sudrala.de
Strange compgen behaviour
Hi fellow bashers!
I am trying to add some completion to a command. The completion should
list all movies I have in a certain folder, regardless if I am in that
folder or not. I have kind of got it to work in several variants but
all have some issue. The current problem I am looking at is very
strange. I have isolated it down to a strange behaviour with compgen.
Let's see if I can describe it clearly enough:
zf contains the list of movie file names
zc is the current input, in my case "/home/mathias/Videos/movies/H"
$ compgen -W "${zf}" -- ${zc}
Here is the output:
/home/mathias/Videos/movies/Harry.Potter...
/home/mathias/Videos/movies/Harry.Potter...
/home/mathias/Videos/movies/Harry.Potter...
/home/mathias/Videos/movies/Harry.Potter...
/home/mathias/Videos/movies/Harry.Potter...
/home/mathias/Videos/movies/Brazil (Terry Gilliam, 1985).avi
/home/mathias/Videos/movies/Ice.Age.2...
/home/mathias/Videos/movies/True.Blood.S01...
/home/mathias/Videos/movies/True.Blood.S01...
I have shortended the names to make it more readable and also removed
some hits.
The main issue here is how for example the line with Brazil can match
the input. Same goes for True Blood. And, to make it even more scary,
an input such as "/home/mathias/Videos/movies/B" does not match the
Brazil line.
What's up with this?
My bash version:
$ bash --version
GNU bash, version 3.2.39(1)-release (i486-pc-linux-gnu)
Running under Ubuntu 8.04.
Thanks!
/Mathias
PS. If you like I can post the whole function that takes care of the
completion, but I think it is not important here.
PPS. The main cause of the issues I have seens to be that some file
names have spaces, and as I said earlier, every way I have tried, even
the ones that workes so so, have some issues. Any tips to get this
relatively fool proof?
