Hi Sven, sorry for starting with a top-post but it is easier to summarize. First of all, I completely understand that you are upset and angry because makesd is contrary to your expectations.
Secondly, please let me note that makesd has been developed for some years. The first visible git commit was 2014-12-03, but it has roots dating back at least to 2010: http://download.goldelico.com/letux400/20100430bootstrap-lenny/ Since then it has evolved to what it is today. And it will evolve further as long as someone works on its improvement. During that time, new features have been added, many people have successfully used it, and have given a lot of feedback. Users did have some trouble (e.g. recently with incompatible sfdisk versions) and have made mistakes. We always tried to improve that as fast as possible. So it has evolved to what it is today by incorporating a lot of feedback. Unfortunately, everyone using software may be the first one to discover another flaw that nobody else did find or discover the same way before. Unfortunately you were the first to have such an issue with combining DEV= and sudo. But also: fortunately you were the first and only (even if it hurts) because it gives us the chance to discuss and improve on this issue. > Am 02.03.2018 um 13:53 schrieb Sven Dyroff <s.dyr...@phytec.de>: > > Hello Nikolaus and Andreas, > > > sudo does not pass the variables on (security reasons, you could do funny > > things). > > So makesd did not know about your DEV=/dev/sdg > > indeed I aghastly realized that this was the mistake I made. Admittedly a > stupid oversight. But easily to understand: sudo is a command that if you use > it at all, you'll ever use it in the most restrictively way possible! So of > course especially if you're in a hurry or stupid (or both) you don't think > about sudo'ing a whole command line but only a single command! Indeed! I mostly use it in the "sudo su" way and then start typing commands. But admittedly not always. > > makesd should simply have no default for DEV and not do anything if not > > set. This will be fixed with the next update. > > To tell it in clear words: NEVER NEVER NEVER a script should access any path > that the user never had provided by himself except "." !!! Yes, that is a good goal. But AFAIK no Linux system can fulfill it completely. > > It's in no way acceptable that a script - especially if it's such basic and > essential - contains hardcoded paths that are valid on no other machine than > the one of the developer!!! Especially not for doing actions like deleting > and/or formatting!!! > > It was an incredible luck that my now erased harddisc /dev/sdc mostly had > been backuped. But still it contained some data that are lost now! By a hair > it could have affected another external harddisc of mine that I need for my > daily work. It would have been an horrific catastrophe if this would have > been affected!!! I currently don't have time left over to we wasted for doing > huge recovery cycles!!! > > > do not automatically use /dev/sdc if no device is specified > > instead print usage and an error message. > > I would even accept if in such a case the script would use ".", resulting in > writing the rootfs into the local directory from where the script had been > called, because this would just follow the principle GIGO: Garbage in - > garbage out! This could also overwrite valuable information. For example if you load makesd to your home directory and have some private ~/bin for whatever reason. I have updated this default to /dev/null. And check if the device is really a writeable block device. > > But why does it want to have such an essential parameter like the path to be > erased and formatted as environment variable at all??? Why not pass it as > commandline parameter? AFAIR the original intention for not using a command line parameter was that you can add "export DEV=..." to your .profile so that you do not have to type it again for every makesd command. This wasn't used any more for a long time on my setups but the DEV= approach remained and it was forgotten over time that it additionally has a potentially dangerous default. And AFAIR it was also intended to put it in a prominent position and in front of the command so that the user thinks about it. If you ask me how I am using it currently most of the time, I find myself permanently using the DEV=/dev/sdb prefix (on an OMAP5 EVM with an external USB SD card reader) and not the /dev/sdc default... So event he /dev/sdc was correct for a different ./makesd host. So I find myself typing it every time (unless using it explicitly by another wrapper around it) - making it almost the equal number of key strokes as with e.g. ./makesd --dev=/dev/sdb or similar. And contrary to the original goals. That can happen for long-living code (8 years now) that users develop behavior that works around a problem - and if the developer is also a user we may run into blind spots like we discuss here. This does not happen if the development team is bigger and one person takes the role of the product owner/product manager who just takes care about features - and actively decides about optimizing or removing them. > You can easily also pass an environment variable by that way! > > > Well, it has to have some default because if you make 20 sd cards > > per day, every keystroke counts. And in 19 times you type > > /dev/sdg correctly and then you mistype /dev/sdh + Enter. > > > > So it needs some default you do not have to think about too much. > > Ok, I understand. But then I strongly suggest a parameter like "--hell" for > activating such hardcoded values that are valid on your machine only!!! We already have a different solution. > > > There is already a secondary default which reads the file ~/.makesd.conf > > to set such a default. And the /dev/sdc is only used if that file > > does not exist or is empty. > > > > I have already updated the documentation page because that > > feature is available for a while. > > Sorry Nikolaus, but for using a command that had been suggested per mail as a > quick hack, I don't expect to be expected to read a documentation first. Well, I must give that back to you: if you use sudo you should be aware of big dangers. And always take quick hacks carefully. Even distrust me please, I may also write stupid things in a hurry. I am not perfect. Nobody is. > I already hesitated several days before I decided to overcome my doubts and > just try it, although already at the first glance I could see that the line > was suspicious. Then I did one small step after the other in order to avoid > exactly such disasters! > > > Well, it is impossible to perfectly prevent erasing the wrong device, > > because only the user can know what the SD card reader with an SD really > > is. > > Exactly because of that a script never should dare to fool you!!! > > > I usually use an SD card reader of some SBC where I did ssh root access to > > so I never thought > > about needing sudo. > > Ok, that explains many things. Especially it confirms that my decision was > right to first get a night of sleep before I tell you how exorbitant furious > I am currently be. Seems to need the whole weekend in order to come down > again! > > > Maybe we should add a check that you really have write access to the > > specified device. > > As a Linux user you don't expect such actually self-evident things anymore, > especially not on the raw developer's side. But meanwhile you can expect > Linux to reliably act in the already mentioned way GIGO: Stupid users will > get stupid results. But normally no disasters anymore like in early days. How does this come? It comes by more and more people really using and testing it. And some falling into a pit. But reporting such issues. This allows developers to fix it. The more people test and the more people develop the better the system becomes. But also important is that Linux and GNU tools (or more generally FLOSS projects) rarely do have a quality assurance departments which tests everything against a clear specification and is paid for it. On the other hand our Letux/Tinkerphones project simply does not have a secretly working quality assurance department which makes sure that only code is released to the public that has already passed through intensive alpha and beta tests. There is no money available and my requests for donations to support it have not been fruitful so far. My dream is that we, the tinkerphones/gta04/pyraphone-owners can permanently collect enough money to support developers (you included?) who regularly and responsibly work on quality improvement. BTW: this is *not* something for a kickstarter because it needs continuous funding to keep an existing project alive. The Linux kernel has a much better quality assurance model having separate roles of maintainers and developers proposing patches which need discussion and agreement by maintainers. And a last veto by top quality manager Linus. Nevertheless they have to revert patches from time to time because they break systems. BTW: quality (testing) is what you really pay for in commercial software. There you can expect and request that there are no significant faults because you pay for. But even then you have no guarantee. See Windows updates for Meltdown/Spectre patches which freeze the PC. Or MacOS updates which looses the complete mailbox. So my summary: Lubarsky's Law of Cybernetic Entomology: there is always one more bug. My extension: there is always one more helldriving bug. Unfortunately somebody has to discover it which hurts. > > > There is at least one protection: you can't overwrite /dev/sda. > > Under Linux indeed this can already be seen as luxury. Every other user would > expect this as minimum safety. > > > So it is equally risky as calling fdisk directly. > > The risk here is a completely other: From experience the parameters of fdisk > are highly dependent on the Linux version that you're using. IMHO they aren't. At least for recent distributions. Otherwise we couldn't use it inside of makesd So far we only had sfdisk incompatibilities. Either everyone is using Ubuntu&Debian or they are sufficiently compatible. > So my first question was: Will the script work properly on my system? But > once again: Never you would expect to get devices accessed that you never > specified!!! The good thing is that you can inspect the source code to answer this question. In Non-FLOSS you would have to believe it. > > >> Why "fsck.ext2"? My QtMoko v55 and v57 SD cards had already been formatted > >> with "fsck.ext4". So even "fsck.ext3" would have been outdated. > > > > No idea. The script only calls fsck.ext3. Seems to be a bug in util-linux > > 2.27.1 when printing the error message. > > A suggestion: Please provide a parameter like -3 or -4. So that users are > able to choose ext3 or ext4 by themselfes. I for example prefer ext4. If you run ./makesd --help you see (amongst many other options): -f fat -f ext3 -f ext4 -f none It is a little tricky to use it properly, especially if you use the predefined macros like "gta04" or "qtmoko". The reason is that -f starts the next partition. To modify the current partition, use ./makesd qtmoko -F -f ext4 > > > Thanks for feedback about makesd. It can only become better :) > > Indeed!!! > > I totally accept if you say that you're providing tinkerer alpha versions and > don't intend to serve them on silver plates. As said above, I basically intend to serve it on a silver plate, but our tiny team simply can't without some funding. So it is the best we can provide. Even if it is not good, it is more than nothing. > But please understand that my intention is to be the one who would be able to > provide the GTA04 on silver plates to others. It's a hard job because the > reaction of de facto everyone who is just an interested smartphone user > looking for an alternative to Apple, Samsung etc. but is not a computer > freak, and who did only one short look at your internet pages was: What a > crap! If you (and others) give more help, documentation and internet pages *can* be improved. If someone takes care of it. At the moment we are simply not enough people *doing* something. We have more complaining that something is wrong or tell us what *we* should do... > But still I'm trying to find a way to get some acceptance for what you're > doing in the wider public. Yes, this is why I have invested so much personal time to get QtMoko 2 compiling and running again. Because I think that it may be 8 years old, but it still is the best Phone GUI we have which is completely community driven. Replicant may be technically much more mature and has Fdroid with many apps, but we are still depending on API definitions set by Google. > > But never I would dare to provide the current makesd script to others. Not on > a silver plate, but even not on pitchfork! > > Please tidy it up, thoroughly. Its too basic and too essential to be left in > the state as it currently is! I completely agree that it is basic and essential, but the state isn't as bad as you make it look. > > Sorry, but will not continue testing until this work has been done! Please contribute to that work! You are really welcome. As soon as you have more time to spend on it. We did wait 5 years so there is no need to hurry. > By that opportunity one more well-intentioned suggestion: > > The default way of using such a script should not be to have a swiss army > knife (German: "eierlegende Wollmilchsau"). For an ordinary user it's better > to use such a script in several separated but straightforward steps: > > 1.) Download the big tarball manually, either using "wget" or a browser. This > isolates all the problems that you can get with wrongly written URLs (e.g. > whitespaces), network problems (e.g. slow networks for such huge tarballs, > including possible timeouts), etc. Well, people failed in the past more often with that step than with using makesd. Especially if it comes to creating 2-partition SD cards which include a new u-boot image and a rootfs. Or 3-partition cards. You could replace the makesd script with a good blog entry and describe all steps to be done manually, but why not automate it as good as possible? And a major problem is that recommendations for "download this or that URL" in a mail or tweet or where eever is archived everywhere and you end up in a plethora of contradicting download locations. You never know what is current and what is old. This was the situation I found many years ago when trying to boot a BeagleBoard for the first time. It was very difficult to locate these tarballs. And they were incompatible or incomplete. I.e. not ending in a working SD card without manual tweaks. This is where makesd uses "latest" defaults for URLs. You can always be sure to get an SD card that has a recent setup and tested combination. Randomly mixed tarballs have the tendency to be not compatible. > 2.) Don't call the script with the URL, but with the filename of the tarball > on your harddisc. This widely eliminates the problems with wrongly written > filenames, because you can use the tab key for auto-completion. So the whole > command needs much less keystrokes. So problems like wrongly copy-and-pasted > whitespaces can much more easily be solved. And in case that the script > failed for any reason, you don't have to download the whole tarball again. Intersting idea. I'll think about what the implications are. Most likely people get lost to know which tarball to download for which device. It is easy to type ./makesd pyra; ./makesd gta04; ./makesd openpandora -r qtmoko Instead of finding out which tarballs to download. This is the major benefit of makesd that it knows which tarballs to fetch from where. So unless I missed some aspects it is like telling people to remove apt-get tools and download .deb directly and then install them manually by dpkg. Which ruins the core benefit of the makesd script. BTW: I haven't tried but you should be able to specify file://localhost/path URLs. So in summary, makesd is capable of doing it your proposed way. It is just not documented, because I consider it much less user-friendly. But for sure, it might save your butt in some cases. > > 3.) You wrote that the script automatically adds a Linux kernel to QtMoko. > Which kernel version? The "latest". Unless you instruct it to do something else. "latest" is a symlink on the server to mark the stable/latest from http://projects.goldelico.com/p/gta04-kernel/ > From which link? Currently there's not any transparency for that. Better way > would be if also for the kernel there would be a parameter so that it can be > downloaded and provided from outside of the sript by the user. You can specify a parameter -k to load a specific kernel package. Phew. Got an even longer answer but I hope it helps all of us to get back patience and enthusiasm. BR and sorry again for the trouble. If we meet some time in person, I'll spend you a beer :) Nikolaus _______________________________________________ Gta04-owner mailing list Gta04-owner@goldelico.com http://lists.goldelico.com/mailman/listinfo.cgi/gta04-owner
