Re: [Gta04-owner] Helldriving with basic and essential scripts

[H. Nikolaus Schaller]

Hi Sven,
sorry for starting with a top-post but it is easier to summarize.

First of all, I completely understand that you are upset and angry because
makesd is contrary to your expectations.

Secondly, please let me note that makesd has been developed for some years.

The first visible git commit was 2014-12-03, but it has roots dating back
at least to 2010: 
http://download.goldelico.com/letux400/20100430bootstrap-lenny/
Since then it has evolved to what it is today. And it will evolve further
as long as someone works on its improvement.

During that time, new features have been added, many people have successfully
used it, and have given a lot of feedback.

Users did have some trouble (e.g. recently with incompatible sfdisk versions)
and have made mistakes. We always tried to improve that as fast as possible.
So it has evolved to what it is today by incorporating a lot of feedback.

Unfortunately, everyone using software may be the first one to discover another
flaw that nobody else did find or discover the same way before.

Unfortunately you were the first to have such an issue with combining DEV= and 
sudo.

But also: fortunately you were the first and only (even if it hurts) because
it gives us the chance to discuss and improve on this issue.


> Am 02.03.2018 um 13:53 schrieb Sven Dyroff :
> 
> Hello Nikolaus and Andreas, 
> 
> > sudo does not pass the variables on (security reasons, you could do funny 
> > things). 
> > So makesd did not know about your DEV=/dev/sdg 
> 
> indeed I aghastly realized that this was the mistake I made. Admittedly a 
> stupid oversight. But easily to understand: sudo is a command that if you use 
> it at all, you'll ever use it in the most restrictively way possible! So of 
> course especially if you're in a hurry or stupid (or both) you don't think 
> about sudo'ing a whole command line but only a single command!

Indeed! I mostly use it in the "sudo su" way and then start typing commands. 
But admittedly not always.

> > makesd should simply have no default for DEV and not do anything if not 
> > set. 

This will be fixed with the next update.

> 
> To tell it in clear words: NEVER NEVER NEVER a script should access any path 
> that the user never had provided by himself except "." !!! 

Yes, that is a good goal. But AFAIK no Linux system can fulfill it completely.

> 
> It's in no way acceptable that a script - especially if it's such basic and 
> essential - contains hardcoded paths that are valid on no other machine than 
> the one of the developer!!! Especially not for doing actions like deleting 
> and/or formatting!!! 
> 
> It was an incredible luck that my now erased harddisc /dev/sdc mostly had 
> been backuped. But still it contained some data that are lost now! By a hair 
> it could have affected another external harddisc of mine that I need for my 
> daily work. It would have been an horrific catastrophe if this would have 
> been affected!!! I currently don't have time left over to we wasted for doing 
> huge recovery cycles!!! 
> 
> > do not automatically use /dev/sdc if no device is specified 
> > instead print usage and an error message. 
> 
> I would even accept if in such a case the script would use ".", resulting in 
> writing the rootfs into the local directory from where the script had been 
> called, because this would just follow the principle GIGO: Garbage in - 
> garbage out! 

This could also overwrite valuable information. For example if you load makesd 
to your home directory and have some private ~/bin for whatever reason.

I have updated this default to /dev/null. And check if the device is really a 
writeable block device.

> 
> But why does it want to have such an essential parameter like the path to be 
> erased and formatted as environment variable at all??? Why not pass it as 
> commandline parameter?

AFAIR the original intention for not using a command line parameter was that
you can add "export DEV=..." to your .profile so that you do not have to type
it again for every makesd command.

This wasn't used any more for a long time on my setups but the DEV= approach
remained and it was forgotten over time that it additionally has a potentially
dangerous default.

And AFAIR it was also intended to put it in a prominent position and in
front of the command so that the user thinks about it.

If you ask me how I am using it currently most of the time, I find myself
permanently using the DEV=/dev/sdb prefix (on an OMAP5 EVM with an external
USB SD card reader) and not the /dev/sdc default... So event he /dev/sdc
was correct for a different ./makesd host.

So I find myself typing it every time (unless using it explicitly by another
wrapper around it) - making it almost the equal number of key strokes as with
e.g. ./makesd --dev=/dev/sdb or similar. And contrary to the original goals.

That can happen for long-living code (8 years now) that users develop behavior
that works around a problem - and if the developer is also 

[Gta04-owner] Helldriving with basic and essential scripts

[Sven Dyroff]

Hello Nikolaus and Andreas,

> sudo does not pass the variables on (security reasons, you could do 
funny things).
> So makesd did not know about your DEV=/dev/sdg

indeed I aghastly realized that this was the mistake I made. Admittedly a 
stupid oversight. But easily to understand: sudo is a command that if you 
use it at all, you'll ever use it in the most restrictively way possible! 
So of course especially if you're in a hurry or stupid (or both) you don't 
think about sudo'ing a whole command line but only a single command!

> makesd should simply have no default for DEV and not do anything if not 
set.

To tell it in clear words: NEVER NEVER NEVER a script should access any 
path that the user never had provided by himself except "." !!!

It's in no way acceptable that a script - especially if it's such basic 
and essential - contains hardcoded paths that are valid on no other 
machine than the one of the developer!!! Especially not for doing actions 
like deleting and/or formatting!!!

It was an incredible luck that my now erased harddisc /dev/sdc mostly had 
been backuped. But still it contained some data that are lost now! By a 
hair it could have affected another external harddisc of mine that I need 
for my daily work. It would have been an horrific catastrophe if this 
would have been affected!!! I currently don't have time left over to we 
wasted for doing huge recovery cycles!!! 

> do not automatically use /dev/sdc if no device is specified
> instead print usage and an error message.

I would even accept if in such a case the script would use ".", resulting 
in writing the rootfs into the local directory from where the script had 
been called, because this would just follow the principle GIGO: Garbage in 
- garbage out!

But why does it want to have such an essential parameter like the path to 
be erased and formatted as environment variable at all??? Why not pass it 
as commandline parameter? You can easily also pass an environment variable 
by that way!

> Well, it has to have some default because if you make 20 sd cards
> per day, every keystroke counts. And in 19 times you type
> /dev/sdg correctly and then you mistype /dev/sdh + Enter.
>
> So it needs some default you do not have to think about too much.

Ok, I understand. But then I strongly suggest a parameter like "--hell" 
for activating such hardcoded values that are valid on your machine 
only!!!

> There is already a secondary default which reads the file ~/.makesd.conf
> to set such a default. And the /dev/sdc is only used if that file
> does not exist or is empty.
>
> I have already updated the documentation page because that
> feature is available for a while.

Sorry Nikolaus, but for using a command that had been suggested per mail 
as a quick hack, I don't expect to be expected to read a documentation 
first. I already hesitated several days before I decided to overcome my 
doubts and just try it, although already at the first glance I could see 
that the line was suspicious. Then I did one small step after the other in 
order to avoid exactly such disasters!

> Well, it is impossible to perfectly prevent erasing the wrong device, 
because only the user can know what the SD card reader with an SD really 
is.

Exactly because of that a script never should dare to fool you!!!

> I usually use an SD card reader of some SBC where I did ssh root access 
to so I never thought
> about needing sudo.

Ok, that explains many things. Especially it confirms that my decision was 
right to first get a night of sleep before I tell you how exorbitant 
furious I am currently be. Seems to need the whole weekend in order to 
come down again!

> Maybe we should add a check that you really have write access to the 
specified device.

As a Linux user you don't expect such actually self-evident things 
anymore, especially not on the raw developer's side. But meanwhile you can 
expect Linux to reliably act in the already mentioned way GIGO: Stupid 
users will get stupid results. But normally no disasters anymore like in 
early days.

> There is at least one protection: you can't overwrite /dev/sda.

Under Linux indeed this can already be seen as luxury. Every other user 
would expect this as minimum safety.

> So it is equally risky as calling fdisk directly.

The risk here is a completely other: From experience the parameters of 
fdisk are highly dependent on the Linux version that you're using. So my 
first question was: Will the script work properly on my system? But once 
again: Never you would expect to get devices accessed that you never 
specified!!!

>> Why "fsck.ext2"? My QtMoko v55 and v57 SD cards had already been 
formatted with "fsck.ext4". So even "fsck.ext3" would have been outdated.
>
> No idea. The script only calls fsck.ext3. Seems to be a bug in 
util-linux 2.27.1 when printing the error message.

A suggestion: Please provide a parameter like -3 or -4. So that users are 
able to choose ext3 or ext4 by themselfes. I for example