Re: [ksh?] quote missing exit error message
On 8/30/24 12:09, Anon Loli wrote: On Thu, Aug 29, 2024 at 07:03:11PM -0400, Nick Holland wrote: On 8/29/24 16:11, Anon Loli wrote: > Okay, I have an vague idea about what happens, let me 1st add this to the bug > report: > > So I launched a half a dozen tmux windows and they had archivemedia script > running and while I started all the windows, I went on to edit the script and > add a few lines, so...you were editing a running script? I do believe that falls under the category, "undefined behavior". Yeah, I kinda expect things to go bad when you do that. You are probably assuming the file is loaded in RAM at start and stays there until done. That's not how it works, I am pretty sure I've tried it and not got away with it before, too. Nick. Of course I assumed that the file would be loaded in RAM lol why wouldn't I? oh, perhaps because you understood Unix? Silly me, of course not. It's the only thing that makes sense to me, but then again I don't know how executing scripts works exactly *shrug*. I would like to know why that is, perhaps this is already answered in e-mails I'm yet to read, but my OS will not have this unlogical and unreliable and actually I'm assuming that this is also UNDOCUMENTED undefined behavior, and as such, ksh.1 manual page is at fault here, this is a bugs. Kid, there's an unlimited number of ways to do things wrong. man pages can't cover all of them, especially when you make shit up. I'd invite you to switch to Linux, except bash does the exact same thing. I'd invent you to return to Windows, except windows batch does the exact same thing (I once had to explain this to a person at my company's headquarters. She'd just crashed the point of sale system at every one of our stores by updating the nightly processing batch file while by having the batch file copy over a new version of itself. "D, you can't change a running batch file". Fortunately, I had ignored her threats of firing me for messing with the Point of Sale system enough that I knew how to recover her error. We got along a LOT better after that!) Dunno about PowerShell. But I'd be slightly surprised if it isn't the same "read from disk as you go". Here's a fun demo: $ cat a.ksh #!/bin/ksh echo "#!/bin/ksh" >b.ksh echo "sleep 0.5" >>b.ksh chmod +x b.ksh ./b.ksh & for N in $(jot 20 1); do print -n '.' print "print $N" >>b.ksh print "sleep 1" >>b.ksh sleep 0.5 done It's a script (a.ksh) that creates a new script (b.ksh), then executes that new script in the background AND keeps adding to the new script. But it adds to the new script faster than it executes it. So most of the code that b.ksh runs doesn't exist when it was first started. And...works exactly the same on linux+bash (with appropriate code changes) Nick.
Re: [ksh?] quote missing exit error message
On 2024/08/30 17:51:55 +, Anon Loli wrote: > On Fri, Aug 30, 2024 at 06:41:50PM +0200, Omar Polo wrote: > > On 30/08/24 18:09, Anon Loli wrote: > > > On Thu, Aug 29, 2024 at 07:03:11PM -0400, Nick Holland wrote: > > >> On 8/29/24 16:11, Anon Loli wrote: > > >>> Okay, I have an vague idea about what happens, let me 1st add this to > > >>> the bug > > >>> report: > > >>> > > >>> So I launched a half a dozen tmux windows and they had archivemedia > > >>> script > > >>> running and while I started all the windows, I went on to edit the > > >>> script and > > >>> add a few lines, > > >> so...you were editing a running script? I do believe that falls under > > >> the > > >> category, "undefined behavior". Yeah, I kinda expect things to go bad > > >> when you do that. > > >> > > >> You are probably assuming the file is loaded in RAM at start and stays > > >> there > > >> until done. That's not how it works, I am pretty sure I've tried it and > > >> not > > >> got away with it before, too. > > >> > > >> Nick. > > > Of course I assumed that the file would be loaded in RAM lol why wouldn't > > > I? > > > > Because it's a waste of both memory and time to load an entire file in > > memory before executing it, especially since the nature of the shell. > > > > Also, IIRC the Thompson shell relied on this to implement conditionals. > > I am not fully aware of "the nature of the shell", but I believe that the > current implementation is a hazard to reliability, brings confusion and > undefined behavior, and is an inconvenience. Before complaining about "hazards to reliability" & co, I'd suggest you to step down, study the system, understand the expectations and how the various primitives work, and only then, eventually, come back talking about how to improve. If you do, you would know how we handle these things on UNIX. Because none of this is OpenBSD specific. > Waste of memory? Look I don't know how many scripts OpenBSD actually executes > like for root/system and what-not, but for the userland, this would be a > extremely minor problem. > I'm assuming that non-root or non-system shells like user-executed shells will > have a extremely small memory and time-to-load resource requirements. > I might as well say that an old computer could function like this. I'd be interested in knowing the research you did to come to this conclusion. I'm also sure that you are greatly missing the point by not realizing how nothing of this is really ksh(1) specific. (I'm serious. do you know you can crash patch(1) by concurrently modifying the file it is patching? Oh, the horror!) You can't pick up a tool you don't know how to use, only to then complain how it is a security hazard. Learn first how it is expected to be used. > > > It's the only thing that makes sense to me, but then again I don't know > > > how > > > executing scripts works exactly *shrug*. > > > > > > I would like to know why that is, perhaps this is already answered in > > > e-mails > > > I'm yet to read, but my OS will not have this unlogical and unreliable and > > > actually I'm assuming that this is also UNDOCUMENTED undefined behavior, > > > and as > > > > > > such, ksh.1 manual page is at fault here, this is a bugs. > > > > unfortunately, this is how unix actually works. Even if ksh used > > memory-mapped I/O, it could still concurred edits to the files. Heck, in > > some cases using memory-mapped I/O could lead to worst issues (imagine a > > concurrent process truncating the file while you're reading it, and say > > hello to SIGBUS.) > > If the issue is as serious as you make it, and if there is no other approach > to > this which can prevent this issue, then: > you could get the file modes of the file, change it to read-only to the > users/groups who already could read it, read it to memory, and then chmod it > to > the original file modes. how does "atomicity" fit into this scheme? You're trying to fix a problem by opening another (but way bigger!) can of worms. What we really do on unix is to do what install(1) does. or at least that's a way. so, ksh parsing bugs notwithstanding, I'd file what you saw with concurred edits to script as "working as indended." > By the way, what is this "\xc2\xa0" twice, some Android version of quotes? probably some unicode space. I replied from (gasp!) linux with thunderbird, and I still haven't found a way to fully addomesticate it. Luckily i only have to use it for work :p
Re: [ksh?] quote missing exit error message
So you've got an opinion. And you are loud. But noone cares. Anon Loli wrote: > On Fri, Aug 30, 2024 at 06:41:50PM +0200, Omar Polo wrote: > > On 30/08/24 18:09, Anon Loli wrote: > > > On Thu, Aug 29, 2024 at 07:03:11PM -0400, Nick Holland wrote: > > >> On 8/29/24 16:11, Anon Loli wrote: > > >>> Okay, I have an vague idea about what happens, let me 1st add this to > > >>> the bug > > >>> report: > > >>> > > >>> So I launched a half a dozen tmux windows and they had archivemedia > > >>> script > > >>> running and while I started all the windows, I went on to edit the > > >>> script and > > >>> add a few lines, > > >> so...you were editing a running script? I do believe that falls under > > >> the > > >> category, "undefined behavior". Yeah, I kinda expect things to go bad > > >> when you do that. > > >> > > >> You are probably assuming the file is loaded in RAM at start and stays > > >> there > > >> until done. That's not how it works, I am pretty sure I've tried it and > > >> not > > >> got away with it before, too. > > >> > > >> Nick. > > > Of course I assumed that the file would be loaded in RAM lol why wouldn't > > > I? > > > > Because it's a waste of both memory and time to load an entire file in > > memory before executing it, especially since the nature of the shell. > > > > Also, IIRC the Thompson shell relied on this to implement conditionals. > > I am not fully aware of "the nature of the shell", but I believe that the > current implementation is a hazard to reliability, brings confusion and > undefined behavior, and is an inconvenience. > > Waste of memory? Look I don't know how many scripts OpenBSD actually executes > like for root/system and what-not, but for the userland, this would be a > extremely minor problem. > I'm assuming that non-root or non-system shells like user-executed shells will > have a extremely small memory and time-to-load resource requirements. > I might as well say that an old computer could function like this. > > > > > > It's the only thing that makes sense to me, but then again I don't know > > > how > > > executing scripts works exactly *shrug*. > > > > > > I would like to know why that is, perhaps this is already answered in > > > e-mails > > > I'm yet to read, but my OS will not have this unlogical and unreliable and > > > actually I'm assuming that this is also UNDOCUMENTED undefined behavior, > > > and as > > > > > > such, ksh.1 manual page is at fault here, this is a bugs. > > > > unfortunately, this is how unix actually works. Even if ksh used > > memory-mapped I/O, it could still concurred edits to the files. Heck, in > > some cases using memory-mapped I/O could lead to worst issues (imagine a > > concurrent process truncating the file while you're reading it, and say > > hello to SIGBUS.) > > If the issue is as serious as you make it, and if there is no other approach > to > this which can prevent this issue, then: > you could get the file modes of the file, change it to read-only to the > users/groups who already could read it, read it to memory, and then chmod it > to > the original file modes. > > By the way, what is this "\xc2\xa0" twice, some Android version of quotes? > > > > > > Note that this also depends on how the editor you're using writes the files > > back to the disk. Some editors will just rewrite the file in-place, which > > means that other processes can see these concurred edits, while others will > > use a tempfile + rename scheme. No one is strictly better than the other. > > Interesting. > The issue happens when editing with vi(1) as well as neatvi. >
Re: [ksh?] quote missing exit error message
On Fri, Aug 30, 2024 at 06:41:50PM +0200, Omar Polo wrote: > On 30/08/24 18:09, Anon Loli wrote: > > On Thu, Aug 29, 2024 at 07:03:11PM -0400, Nick Holland wrote: > >> On 8/29/24 16:11, Anon Loli wrote: > >>> Okay, I have an vague idea about what happens, let me 1st add this to the > >>> bug > >>> report: > >>> > >>> So I launched a half a dozen tmux windows and they had archivemedia script > >>> running and while I started all the windows, I went on to edit the script > >>> and > >>> add a few lines, > >> so...you were editing a running script? I do believe that falls under the > >> category, "undefined behavior". Yeah, I kinda expect things to go bad > >> when you do that. > >> > >> You are probably assuming the file is loaded in RAM at start and stays > >> there > >> until done. That's not how it works, I am pretty sure I've tried it and > >> not > >> got away with it before, too. > >> > >> Nick. > > Of course I assumed that the file would be loaded in RAM lol why wouldn't I? > > Because it's a waste of both memory and time to load an entire file in memory > before executing it, especially since the nature of the shell. > > Also, IIRC the Thompson shell relied on this to implement conditionals. I am not fully aware of "the nature of the shell", but I believe that the current implementation is a hazard to reliability, brings confusion and undefined behavior, and is an inconvenience. Waste of memory? Look I don't know how many scripts OpenBSD actually executes like for root/system and what-not, but for the userland, this would be a extremely minor problem. I'm assuming that non-root or non-system shells like user-executed shells will have a extremely small memory and time-to-load resource requirements. I might as well say that an old computer could function like this. > > It's the only thing that makes sense to me, but then again I don't know how > > executing scripts works exactly *shrug*. > > > > I would like to know why that is, perhaps this is already answered in > > e-mails > > I'm yet to read, but my OS will not have this unlogical and unreliable and > > actually I'm assuming that this is also UNDOCUMENTED undefined behavior, > > and as > > > > such, ksh.1 manual page is at fault here, this is a bugs. > > unfortunately, this is how unix actually works. Even if ksh used > memory-mapped I/O, it could still concurred edits to the files. Heck, in > some cases using memory-mapped I/O could lead to worst issues (imagine a > concurrent process truncating the file while you're reading it, and say hello > to SIGBUS.) If the issue is as serious as you make it, and if there is no other approach to this which can prevent this issue, then: you could get the file modes of the file, change it to read-only to the users/groups who already could read it, read it to memory, and then chmod it to the original file modes. By the way, what is this "\xc2\xa0" twice, some Android version of quotes? > Note that this also depends on how the editor you're using writes the files > back to the disk. Some editors will just rewrite the file in-place, which > means that other processes can see these concurred edits, while others will > use a tempfile + rename scheme. No one is strictly better than the other. Interesting. The issue happens when editing with vi(1) as well as neatvi.
Re: [ksh?] quote missing exit error message
On 30/08/24 18:09, Anon Loli wrote: > On Thu, Aug 29, 2024 at 07:03:11PM -0400, Nick Holland wrote: >> On 8/29/24 16:11, Anon Loli wrote: >>> Okay, I have an vague idea about what happens, let me 1st add this to the >>> bug >>> report: >>> >>> So I launched a half a dozen tmux windows and they had archivemedia script >>> running and while I started all the windows, I went on to edit the script >>> and >>> add a few lines, >> so...you were editing a running script? I do believe that falls under the >> category, "undefined behavior". Yeah, I kinda expect things to go bad >> when you do that. >> >> You are probably assuming the file is loaded in RAM at start and stays there >> until done. That's not how it works, I am pretty sure I've tried it and not >> got away with it before, too. >> >> Nick. > Of course I assumed that the file would be loaded in RAM lol why wouldn't I? Because it's a waste of both memory and time to load an entire file in memory before executing it, especially since the nature of the shell. Also, IIRC the Thompson shell relied on this to implement conditionals. > It's the only thing that makes sense to me, but then again I don't know how > executing scripts works exactly *shrug*. > > I would like to know why that is, perhaps this is already answered in e-mails > I'm yet to read, but my OS will not have this unlogical and unreliable and > actually I'm assuming that this is also UNDOCUMENTED undefined behavior, and > as > > such, ksh.1 manual page is at fault here, this is a bugs. unfortunately, this is how unix actually works. Even if ksh used memory-mapped I/O, it could still concurred edits to the files. Heck, in some cases using memory-mapped I/O could lead to worst issues (imagine a concurrent process truncating the file while you're reading it, and say hello to SIGBUS.) Note that this also depends on how the editor you're using writes the files back to the disk. Some editors will just rewrite the file in-place, which means that other processes can see these concurred edits, while others will use a tempfile + rename scheme. No one is strictly better than the other. (if you think this is only related to weird and/or "ancient" stuff well, I was VERY surprised while trying the vis editor (vise on openbsd), and seeing the content on the monitor getting corrupted after I did a "echo > file" :-)
Re: [ksh?] quote missing exit error message
On 2024/08/29 19:03, Nick Holland wrote: > On 8/29/24 16:11, Anon Loli wrote: > > Okay, I have an vague idea about what happens, let me 1st add this to the > > bug > > report: > > > > So I launched a half a dozen tmux windows and they had archivemedia script > > running and while I started all the windows, I went on to edit the script > > and > > add a few lines, > so...you were editing a running script? I do believe that falls under the > category, "undefined behavior". Yeah, I kinda expect things to go bad > when you do that. That falls under the "you are damn lucky if things _don't_ break". Edit a copy and move it into place.
Re: [ksh?] quote missing exit error message
On Thu, Aug 29, 2024 at 07:03:11PM -0400, Nick Holland wrote: > On 8/29/24 16:11, Anon Loli wrote: > > Okay, I have an vague idea about what happens, let me 1st add this to the > > bug > > report: > > > > So I launched a half a dozen tmux windows and they had archivemedia script > > running and while I started all the windows, I went on to edit the script > > and > > add a few lines, > so...you were editing a running script? I do believe that falls under the > category, "undefined behavior". Yeah, I kinda expect things to go bad > when you do that. > > You are probably assuming the file is loaded in RAM at start and stays there > until done. That's not how it works, I am pretty sure I've tried it and not > got away with it before, too. > > Nick. Of course I assumed that the file would be loaded in RAM lol why wouldn't I? It's the only thing that makes sense to me, but then again I don't know how executing scripts works exactly *shrug*. I would like to know why that is, perhaps this is already answered in e-mails I'm yet to read, but my OS will not have this unlogical and unreliable and actually I'm assuming that this is also UNDOCUMENTED undefined behavior, and as such, ksh.1 manual page is at fault here, this is a bugs.
Re: [ksh?] quote missing exit error message
On 8/29/24 16:11, Anon Loli wrote: Okay, I have an vague idea about what happens, let me 1st add this to the bug report: So I launched a half a dozen tmux windows and they had archivemedia script running and while I started all the windows, I went on to edit the script and add a few lines, so...you were editing a running script? I do believe that falls under the category, "undefined behavior". Yeah, I kinda expect things to go bad when you do that. You are probably assuming the file is loaded in RAM at start and stays there until done. That's not how it works, I am pretty sure I've tried it and not got away with it before, too. Nick.
Re: [ksh?] quote missing exit error message
> /home/user/bin/archivemedia[421]: syntax error: `)' unexpected This smells like the quoting bug mentioned in BUGS of ksh. It's not meant to be like this...
Re: [ksh?] quote missing exit error message
Okay, I have an vague idea about what happens, let me 1st add this to the bug report: So I launched a half a dozen tmux windows and they had archivemedia script running and while I started all the windows, I went on to edit the script and add a few lines, and in these next 2 bugs you can see the [440], which I believe that at the time of the script execution, exceeded the line length of the script by about 10. /home/user/bin/archivemedia[440]: d-metadata: not found another tmux window at about the same time also gave this: /home/user/bin/archivemedia[421]: syntax error: `)' unexpected and this is ultra rare to see, the '/home/user/bin/script[num]' remains more-less the same This makes me think that the main problem is editing scripts while they have been executed by some process. Why does this happen? And before someone says, no, the script doesn't run another clone script like recursives execution or something like that. And why do I have a feeling that Theo De Raadt or someone will say "This is not a bug, this is a feature; It is meant to be like this!" On Sun, Aug 25, 2024 at 02:21:51PM +, Anon Loli wrote: > Hi list! > > I am puzzled about this as to why - this has kept on happening to me for many > months if not year(s). > It has also happened across many different scripts of mine. > > So upon exit of my scripts, sometimes, and I mean sometimes like 1 out of 20 > or > so runs of my scripts, it happens. > > How I ran these scripts: > I have a "ksh" line at the end of .cshrc and that basically starts my Ksh > shells on every terminal window. > And I ran these scripts by hand per-say, as they're located in ~/bin folder > which is, of course, added to PATH via > PATH=$HOME/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin:/usr/local/bin:/usr/local/sbin > which can be found in ~/.profile file. > ALSO IT COULD BE POSSIBLE: I usually also run these scripts within tmux(1) > becaues the ST doesn't have a buffer scrollback even with the scrollback > applied, on OpenBSD, for some reason... it works perfectly well on FreeBSD > last > time I used it. > I also have "#!/bin/sh" in all of my scripts. > > I have of course modified some things like the exported PS1 in ~/.kshrc, but > other than that I don't see what can be the problem. > > It could be, which I doubt, the terminal problem, and the terminal used is ST > from [1]. > > Sadly I instinctively closed my terminal window with the exact error message > so > I can only give you as much of it as I remember. > I tried reproducing it right now but I can't, perhaps it's even rarer than > 1/20 > chance. > Remember that this happens on the EXIT of a script which was running perfectly > fine. > > The error message is something like this: > "/home/user/bin/script: quotation mark not closed", > or > "/home/user/bin/script: closing quotation mark missing", > and in the script that I have attached you can clearly see that all quotation > marks have been closed. > And no, not all scripts use find(1) or mpv(1), so I don't think they have a > specific utility in common... unless maybe echo? > > [1] http://st.suckless.org/ > #!/bin/sh > # 2024/08/25, I deny the existence of Intellectual Property including > licenses. > tmpfile=".playvideos.tmp" > echo > "$tmpfile" > find . -type f -name "*mp4" -o -name "*webm" -o -name "*mkv" -o -name "*avi" > -o -name "*m4v" -o -name "*mov" -o -name "*3gp" -o -name "*flv" -o -name > "*wmv" -o -name "*vob" >> "$tmpfile" > mpv --vo=gpu --loop --shuffle --playlist="$tmpfile" > rm "$tmpfile"
