Re: [dev] Shell style guide

[Anselm R Garbe]

Hi there,

On 6 September 2016 at 20:35, Evan Gates  wrote:
> suckless.org projects have traditionally been small amounts of pure C.
> The code tends towards simplicity and correctness. I value this and
> have learned much over the past years from reading and contributing to
> various projects.
>
> The addition of stali means there will probably be a fair amount of
> shell scripting. In my experience the vast majority of shell scripts
> are complete crap. Worse than poor style are poor practices that
> create broken code. As such I propose that we add a shell scripting
> style guide to go along with the existing C style guide in hopes of
> keeping suckless.org's shell scripts as clean, simple, and correct as
> the C code.
>
> I think it should include the following, and probably some more. Many
> of these rules are covered in the only bash guide I've found that
> doesn't include bad practices. It also has a lot of information
> pertaining to POSIX sh. Please check out the guide[0], faq[1], and
> common pitfalls[2].

I appreciate your efforts of coming up with some sh styleguide. A side
note to that, I prefer calling test explicitely, instead of using the
weirdo [ ] symlinks in while/if statements.

Nevertheless, after an excursion to sh for several years, I'm kind of
favouring 9base/rc again, after all. For stali I now tend to adopt rc
as primary scripting language for the target system as well. For the
build host environment I would rather stick to sh+make instead of
rc+mk. We have to live with the fact that a build host environment is
poisoned with crap bloat to hell anyways. But for ideal target system,
we should stick to technologies that have been designed with clarity
and cleanliness in mind. rc is the perfect example for a decent shell
environment.

When I started 9base and Uriel started werc and other rc-based stuff,
we concluded that one cannot really write portable scripts with sh.
You have to rely on a defined userland. The Plan 9 derived userland
offers this definition. I agree that sbase goes into a similar
direction, but the danger remains when using sh, that some stuff will
end up in pretty mixed environments.

For serious stuff like werc, rc was the right choice, because it
relies on this definite userland environment and hence does not allow
for ambiguity.

I just wanted to mention this, as this is one of my very recent
decisions about stali's future.

BR,
Anselm

Re: [dev] [st] bitmap font and chscale 1< line drawing bug

[Eric Pruitt]

On Wed, Sep 07, 2016 at 11:36:47PM -0700, le...@bitmessage.ch wrote:
> On Wed, Sep 07, 2016 at 09:39:08PM -0700, Eric Pruitt wrote:
> > Even if you don't debug and / or fix the problem, at least post the
> > offending commit, and I will take a look.
>
> I was able to fix it the problem by rolling back part of commit
> 528241aa3835e2f1f052abeeaf891737712955a0
> and all of
> 0e48a1995eee1c2babc58523ef0be296e4b1c3e8

I don't know enough about FontCofig and related things to comment with
any certainty, but this looks like one of those situations where a bug
(or less-than-ideal implementation of a feature) resulted in desirable
behavior. Hopefully someone more knowledgeably will chime in.

Eric

Re: [dev] Shell style guide

[hiro]

You're wrong kamil. my computer has a working border patrol that
ensures your crappy files don't end up here. I never have to deal with
such shitty file names.

On 9/8/16, Kamil Cholewiński  wrote:
> On Thu, 08 Sep 2016, Nick  wrote:
>> I am against writing scripts that can deal with filenames with
>> newlines. Fuck such filenames. If you have to deal with them, shell
>> scripting is a terrible technology to use.
>
> And since you never know when you'll have to deal with them...
>
>

Re: [dev] Shell style guide

[Kamil Cholewiński]

On Thu, 08 Sep 2016, Nick  wrote:
> I am against writing scripts that can deal with filenames with 
> newlines. Fuck such filenames. If you have to deal with them, shell 
> scripting is a terrible technology to use.

And since you never know when you'll have to deal with them...

Re: [dev] Shell style guide

[Evan Gates]

On Thu, Sep 8, 2016 at 7:30 AM, Nick  wrote:
> I am against writing scripts that can deal with filenames with
> newlines. Fuck such filenames. If you have to deal with them, shell
> scripting is a terrible technology to use.

I too think newlines in filenames is stupid. I never put newlines in
my filenames. But I don't know if someone who uses a script I release
will have newlines in their filenames. It's not difficult to write
scripts that can handle all legal filenames. Also don't forget files
that start with -, or have * in the name or a whole bunch of others. I
have an evilfiles directory I run scripts in to test sometimes. It'll
scare you. (I tried to link to the directory on bitbucket, but it
seems their code can't handle it, I find that highly amusing)

Re: [dev] Shell style guide

[Evan Gates]

On Thu, Sep 8, 2016 at 7:26 AM, Greg Reagle  wrote:
> On Thu, Sep 8, 2016, at 10:11 AM, Greg Reagle wrote:
>> On Thu, Sep 8, 2016, at 10:05 AM, Kamil Cholewiński wrote:
>> > Wow, shell quoting is just fucked up crazy. I was mostly using unquoted
>> > $@ my whole life. And here I thought I knew enough not to screw up a
>> > simple script,
>>
>> It is common knowledge to always double-quote $@, i.e. "$@", in
>> Bourne/Posix shell.
>
> At the same time, it is something that everyone has to learn at some
> point.  We are not born knowing this.  There was a time before I knew it
> too.  So I don't mean to be critical of you personally.
>

The problem (as I see it) is that no one takes shell scripting
seriously. They never take the time to sit down and study it the way
they do for other languages. That and 99% of the scripts you'll find
in the wild are pure crap.

I'll take this chance to plug the bash guide[0] again. And especially
the pitfalls[1] and faq[2]. If you don't want to work through the
whole guide (which you should), at least read the pitfalls and faq.

[0]http://mywiki.wooledge.org/BashGuide
[1]http://mywiki.wooledge.org/BashPitfalls
[2]http://mywiki.wooledge.org/BashFAQ

Re: [dev] Shell style guide

[Nick]

Quoth Evan Gates:
> On Thu, Sep 8, 2016 at 5:44 AM, Nick  wrote:
> > I think this is something one learns with time. There are several
> > good reasons not to quote substitutions, such as passing multiple
> > arguments to another program (e.g. cmd $@), or a for or case
> > statement. But yes, quoting is essential most of the time.  Shell
> > escaping sucks, though, obviously. Another reason to keep bourne
> > shell very simple.
> 
> No. $@ is another example of something that _must_ be quoted every
> time. When inside quotes the shell expands it correctly to a word for
> each parameter. Without quotes the shell will do extra word splitting
> and globbing. For example, try this:
> 
> nargs() { echo "$#"; }
> set -- foo 'bar baz'
> echo "$#"
> nargs "$@"
> nargs $@

Ah, my bad, I did not know that, thanks. I thought that passing "$@" 
would just pass the contents of $@ as one string. And there I was 
considering myself well-versed on the ways of shell.

> When in a case statement should you not quote? I can't think of any examples.

Me neither. Ignore me :p

> I'm opposed to this. If you want convert, write convert. If you want
> gm convert, write gm convert. If you want to be able to change it
> everywhere in your script at once, put it in a function. Functions
> exist for exactly this reason. (That was strongly worded. If you write
> code that won't break, that's fine, but putting commands in variables
> obfuscates the code.)

How does it obfuscate the code to have "$convert a.png a.tif" versus 
"convert a.png a.tif" (where convert has been (re)defined as a 
function earlier in the script)? The former makes clear that you're 
just using the command in the convert variable, the latter does not.

> > I disagree. GNU ls may be, but it isn't supposed to be that way.
> > Something like `ls -tr | tail -n 1` seems perfectly reasonable to
> > me, for example. Sure you could use find, but for some cases ls is
> > simpler.
> 
> Simpler maybe, but incorrect. Filenames can contain newlines
> (filenames can contain everything but a / and a nul byte).

I am against writing scripts that can deal with filenames with 
newlines. Fuck such filenames. If you have to deal with them, shell 
scripting is a terrible technology to use.

Re: [dev] Shell style guide

[Greg Reagle]

On Thu, Sep 8, 2016, at 10:11 AM, Greg Reagle wrote:
> On Thu, Sep 8, 2016, at 10:05 AM, Kamil Cholewiński wrote:
> > Wow, shell quoting is just fucked up crazy. I was mostly using unquoted
> > $@ my whole life. And here I thought I knew enough not to screw up a
> > simple script,
> 
> It is common knowledge to always double-quote $@, i.e. "$@", in
> Bourne/Posix shell.

At the same time, it is something that everyone has to learn at some
point.  We are not born knowing this.  There was a time before I knew it
too.  So I don't mean to be critical of you personally.

Re: [dev] Shell style guide

[Greg Reagle]

On Thu, Sep 8, 2016, at 10:05 AM, Kamil Cholewiński wrote:
> Wow, shell quoting is just fucked up crazy. I was mostly using unquoted
> $@ my whole life. And here I thought I knew enough not to screw up a
> simple script,

It is common knowledge to always double-quote $@, i.e. "$@", in
Bourne/Posix shell.

> but there just always seems to be a new hidden gotcha.

Indeed.  It sucks.  Use an unbroken shell like rc and these troubles are
behind you.

Re: [dev]

[Martin Kühne]

Please disregard. My email skills failed me.

cheers!
mar77i

[dev]

[Martin Kühne]

Bash really is a special little snowflake.
The space trimming feature, as I recently discovered is made available
for all but the implicit REPLY.

$ read -r <<<"   TEST   "; printf '<%s>\n' "$REPLY"
<   TEST   >
$ read -r REPLY <<<"   TEST   "; printf '<%s>\n' "$REPLY"


As a heavy user of bash and freenode #bash padawan, I must say that
I'd be very in favor of keeping bash out of suckless.
It's a good learning instrument wrt this kind of nitpickery which is
useful for discussing the C standard, but it's definitely not
suckless.

cheers!
mar77i

Re: [dev] Shell style guide

[Martin Kühne]

On Thu, Sep 8, 2016 at 4:11 PM, Greg Reagle  wrote:
> On Thu, Sep 8, 2016, at 10:05 AM, Kamil Cholewiński wrote:
>> Wow, shell quoting is just fucked up crazy. I was mostly using unquoted
>> $@ my whole life. And here I thought I knew enough not to screw up a
>> simple script,
>
> It is common knowledge to always double-quote $@, i.e. "$@", in
> Bourne/Posix shell.
>

Bash really is a special little snowflake.
The space trimming feature, as I recently discovered is made available
for all but the implicit REPLY.

$ read -r <<<"   TEST   "; printf '<%s>\n' "$REPLY"
<   TEST   >
$ read -r REPLY <<<"   TEST   "; printf '<%s>\n' "$REPLY"


As a heavy user of bash and freenode #bash padawan, I must say that
I'd be very in favor of keeping bash out of suckless.
It's a good learning instrument wrt this kind of nitpickery which is
useful for discussing the C standard, but it's definitely not
suckless.

cheers!
mar77i

Re: [dev] Shell style guide

[Kamil Cholewiński]

On Thu, 08 Sep 2016, Evan Gates  wrote:
> No. $@ is another example of something that _must_ be quoted every
> time. When inside quotes the shell expands it correctly to a word for
> each parameter. Without quotes the shell will do extra word splitting
> and globbing. For example, try this:
>
> nargs() { echo "$#"; }
> set -- foo 'bar baz'
> echo "$#"
> nargs "$@"
> nargs $@
>
> When in a case statement should you not quote? I can't think of any examples.

Wow, shell quoting is just fucked up crazy. I was mostly using unquoted
$@ my whole life. And here I thought I knew enough not to screw up a
simple script, but there just always seems to be a new hidden gotcha.

Perhaps we should start with a simple, minimalistic list of allowed
constructs, and make it a policy to use rc/c for anything else?

(Also, isn't this how C++ was finally outlawed here? long long time ago?
People trying to agree which subset was "sane" and ultimately deciding
none of it was?)

<3,K.

Re: [dev] Shell style guide

[Evan Gates]

On Wed, Sep 7, 2016 at 11:21 PM, Martin Kühne  wrote:
> As with read's -r flag, use <<'EOF' unless silly/potentially
> devastating side effects are desired.

To add on to this, if reading lines, use IFS= as well so you don't
lose leading/trailing whitespace. e.g.:

while IFS= read -r line; do
something with "$line"
done < file

Re: [dev] Shell style guide

[Evan Gates]

On Thu, Sep 8, 2016 at 12:27 AM,   wrote:
> is there a special reason people use `EOF` rather than anything else.
> FWIW, I use a dot. looks very clear and natural to me, saves some
> keystrokes, feels right, just as ending a sentence with full stop.
>
> cat <<.
> 1st line
> 2nd line
> 3rd line
> 4th line
> .

I wouldn't use . because I often use heredocs as input for ed. I do
like using DOG because it makes the 12 year old in me giggle.

cat << DOG

Re: [dev] Shell style guide

[Evan Gates]

On Thu, Sep 8, 2016 at 5:44 AM, Nick  wrote:
> I think this is something one learns with time. There are several
> good reasons not to quote substitutions, such as passing multiple
> arguments to another program (e.g. cmd $@), or a for or case
> statement. But yes, quoting is essential most of the time.  Shell
> escaping sucks, though, obviously. Another reason to keep bourne
> shell very simple.

No. $@ is another example of something that _must_ be quoted every
time. When inside quotes the shell expands it correctly to a word for
each parameter. Without quotes the shell will do extra word splitting
and globbing. For example, try this:

nargs() { echo "$#"; }
set -- foo 'bar baz'
echo "$#"
nargs "$@"
nargs $@

When in a case statement should you not quote? I can't think of any examples.

> I actually never use functions in shell scripts; I consider any
> script that needs them too complex to use shell. I wouldn't be
> opposed to storing a command name in a string, though, say to store
> something along the lines of 'convert' vs 'gm convert'.

I'm opposed to this. If you want convert, write convert. If you want
gm convert, write gm convert. If you want to be able to change it
everywhere in your script at once, put it in a function. Functions
exist for exactly this reason. (That was strongly worded. If you write
code that won't break, that's fine, but putting commands in variables
obfuscates the code.)

> I prefer backticks, because nesting commands is a terrible idea in
> shell scripts. Keep intermediate output in other variables, if you
> need to. But if I see nested commands in a shell script there's a
> high chance I'll rewrite it, if I'm likely to need to actually use /
> maintain it.

If you want to use intermediate variables that's fine. Sometimes they
make the code more clear, sometimes they make it less clear. I'm just
against all the syntax horror that goes on with backticks.

> I disagree. GNU ls may be, but it isn't supposed to be that way.
> Something like `ls -tr | tail -n 1` seems perfectly reasonable to
> me, for example. Sure you could use find, but for some cases ls is
> simpler.

Simpler maybe, but incorrect. Filenames can contain newlines
(filenames can contain everything but a / and a nul byte).

Re: [dev] Shell style guide

[Nick]

Hi Evan, I agree with most of your thoughts, I'll just add my own 
where they differ.

> Shebang: Use #!/bin/sh and only use POSIX shell features. If you need
> bash features use the proper shebang, either #!/path/to/bash or
> #!/usr/bin/env bash

No suckless project should be using bash. If you need the 
complexity, use rc or something else more fit for the job.

> Quoting: Quote all expansions/substitutions. e.g. always use "$foo"
> and never use $foo. There are an extremely small number of acceptable
> reasons to break this rule, e.g. $CFLAGS (note that some parts of the
> grammar do not require quotes for safe expansion such as assignment
> and case $var in, we should discuss what to require in these cases)

I think this is something one learns with time. There are several 
good reasons not to quote substitutions, such as passing multiple 
arguments to another program (e.g. cmd $@), or a for or case 
statement. But yes, quoting is essential most of the time.  Shell 
escaping sucks, though, obviously. Another reason to keep bourne 
shell very simple.

> Storing Commands: Do not store commands in strings. This is what
> functions are for. Storing complex commands in strings and trying 
> to execute or eval them is fragile and needlessly complex.

I actually never use functions in shell scripts; I consider any 
script that needs them too complex to use shell. I wouldn't be 
opposed to storing a command name in a string, though, say to store 
something along the lines of 'convert' vs 'gm convert'.

> Command Substitution: Always use "$()", never use backticks. This
> makes for easier nesting and fewer surprises. Remember these should
> always be quoted.

I prefer backticks, because nesting commands is a terrible idea in 
shell scripts. Keep intermediate output in other variables, if you 
need to. But if I see nested commands in a shell script there's a 
high chance I'll rewrite it, if I'm likely to need to actually use / 
maintain it.

> Checking exit status: Do not run a command and then check against $?.
> This is pointless. Instead check the exit status directly withif
> cmd; then or by using a boolean operator such ascmd && ...

Generally yes, but there can be times when it's clearer to have the 
test in a separate statement, for example if the command is long and 
complex.

> Do not parse ls: ls is a tool to view files in a human readable
> format. Most often when someone tries to use the output of ls they
> really just wanted a glob anyway.

I disagree. GNU ls may be, but it isn't supposed to be that way.  
Something like `ls -tr | tail -n 1` seems perfectly reasonable to 
me, for example. Sure you could use find, but for some cases ls is 
simpler.

> Test: Do not use parens or boolean operators inside test expressions.
> They are deprecated and useless. Instead of [ "$a" = foo -a "$b" = bar
> ] use [ "$a" = foo ] && [ "$b" = bar ]

Yep. I actually prefer just using the test command rather than [ / 
], as I find it clearer, rather than some pseudo-syntactical sugar.  
But definitely eschewing boolean operators from test is sensible.

> Echo and printf: Do not use echo if your input includes a variable or
> backslash. There is no safe way to do so. Use printf and %s instead.

Agreed. I'd furthermore say that the first argument of printf should 
always be single-quoted, to ensure no unexpected substitutions can 
occur.

Nick

Re: [dev] Shell style guide

[ssd]

* Marc Collin 2016-09-08 05:04
> What people here think of heredoc?
> 
> cat << EOF
> 1st line
> 2nd line
> 3rd line
> 4th line
> EOF

is there a special reason people use `EOF` rather than anything else.
FWIW, I use a dot. looks very clear and natural to me, saves some
keystrokes, feels right, just as ending a sentence with full stop.

cat <<.
1st line
2nd line
3rd line
4th line
.

Re: [dev] [st] bitmap font and chscale 1< line drawing bug

[lemon]

On Wed, Sep 07, 2016 at 09:39:08PM -0700, Eric Pruitt wrote:
> Even if you don't debug and / or fix the problem, at least post the
> offending commit, and I will take a look.

I was able to fix it the problem by rolling back part of commit
528241aa3835e2f1f052abeeaf891737712955a0
and all of 
0e48a1995eee1c2babc58523ef0be296e4b1c3e8

Here is my patch, it works with my setup but I haven't tested it
rigorously. 

diff --git a/st.c b/st.c
index 2594c65..ca126d7 100644
--- a/st.c
+++ b/st.c
@@ -3345,6 +3346,9 @@ xloadfonts(char *fontstr, double fontsize)
defaultfontsize = usedfontsize;
}
 
+   FcConfigSubstitute(0, pattern, FcMatchPattern);
+   FcDefaultSubstitute(pattern);
+
if (xloadfont(, pattern))
die("st: can't open font %s\n", fontstr);
 
@@ -3667,7 +3671,7 @@ xmakeglyphfontspecs(XftGlyphFontSpec *specs, const Glyph 
*glyphs, int len, int x
specs[numspecs].font = frc[f].font;
specs[numspecs].glyph = glyphidx;
specs[numspecs].x = (short)xp;
-   specs[numspecs].y = (short)yp;
+   specs[numspecs].y = (short)(winy + frc[f].font->ascent);
xp += runewidth;
numspecs++;
}

--
lemon

Re: [dev] Shell style guide

[Martin Kühne]

As with read's -r flag, use <<'EOF' unless silly/potentially
devastating side effects are desired.

cheers!
mar77i