Re: port & root
On 2018-04-05 19:39, Joshua Root wrote: > On 2018-4-6 03:25 , Rainer Müller wrote: >> On 2018-04-05 19:14, Joshua Root wrote: >>> My guess is that this is a bug in Tcl's file command. Deleting a >>> nonexistent file is not supposed to be considered an error. But by >>> starting two instances of port(1) at the same time, a race condition >>> arises. The 'file delete' command probably checks whether the file >>> exists, returns TCL_OK if not, and then attempts to delete the file. If >>> the file has been deleted by the other running instance (or indeed >>> anything else) between these two steps, then the actual deletion fails, >>> and it incorrectly returns an error. >> >> You are right. We hit a race condition in the implementation of 'file >> delete' in Tcl. >> >> It is missing another check for ENOENT here, as done in line 381: >> >> https://github.com/macports/macports-base/blob/4b13207d9f7f9aba1cc9eba266b3071318637a8c/vendor/tcl8.5.19/generic/tclFCmd.c#L415 > > Still seems broken in 8.6 at first glance too. I filed a patch with Tcl upstream: https://core.tcl.tk/tcl/tktview?name=27b6822849 Rainer
Re: port & root
On Thu, Apr 05, 2018 at 07:25:10PM +0200, Rainer Müller wrote: On 2018-04-05 19:14, Joshua Root wrote: My guess is that this is a bug in Tcl's file command. Deleting a nonexistent file is not supposed to be considered an error. But by starting two instances of port(1) at the same time, a race condition arises. The 'file delete' command probably checks whether the file exists, returns TCL_OK if not, and then attempts to delete the file. If the file has been deleted by the other running instance (or indeed anything else) between these two steps, then the actual deletion fails, and it incorrectly returns an error. You are right. We hit a race condition in the implementation of 'file delete' in Tcl. It is missing another check for ENOENT here, as done in line 381: https://github.com/macports/macports-base/blob/4b13207d9f7f9aba1cc9eba266b3071318637a8c/vendor/tcl8.5.19/generic/tclFCmd.c#L415 @l2dy, Is our Travis job executing multiple 'port lint' commands in parallel? Then this is the same cause. Not multiple lint commands, but lint and building ports (including listing subports) are executed in parallel. Rainer -- Best regards, Zero King smime.p7s Description: S/MIME cryptographic signature
Re: port & root
> On 5 Apr 2018, at 6:39 pm, Bjarne D Mathiesen > wrote: > > Rainer Müller wrote: >>> On 2018-04-05 18:26, Bjarne D Mathiesen wrote: >>> When running './diffPorts php72-' as root I get this : >> >> Just to be specific, did you open a root shell first or did you run it >> with sudo? > > root shell through 'ssh root@localhost' The recommendation is not to do this. Just use your regular user account with sudo. > > > -- > Bjarne D Mathiesen > Korsør ; Danmark ; Europa > -- > denne besked er skrevet i et totalt M$-frit miljø > macOS 10.13.4 High Sierra (17E199) > 2 x 3,46 GHz 6-Core Intel Xeon ; 48 GB 1333 MHz DDR3 ECC > ATI Radeon HD 5770 1024 MB
Re: port & root
Rainer Müller wrote: > On 2018-04-05 18:26, Bjarne D Mathiesen wrote: >> When running './diffPorts php72-' as root I get this : > > Just to be specific, did you open a root shell first or did you run it > with sudo? root shell through 'ssh root@localhost' -- Bjarne D Mathiesen Korsør ; Danmark ; Europa -- denne besked er skrevet i et totalt M$-frit miljø macOS 10.13.4 High Sierra (17E199) 2 x 3,46 GHz 6-Core Intel Xeon ; 48 GB 1333 MHz DDR3 ECC ATI Radeon HD 5770 1024 MB
Re: port & root
On 2018-4-6 03:25 , Rainer Müller wrote: > On 2018-04-05 19:14, Joshua Root wrote: >> My guess is that this is a bug in Tcl's file command. Deleting a >> nonexistent file is not supposed to be considered an error. But by >> starting two instances of port(1) at the same time, a race condition >> arises. The 'file delete' command probably checks whether the file >> exists, returns TCL_OK if not, and then attempts to delete the file. If >> the file has been deleted by the other running instance (or indeed >> anything else) between these two steps, then the actual deletion fails, >> and it incorrectly returns an error. > > You are right. We hit a race condition in the implementation of 'file > delete' in Tcl. > > It is missing another check for ENOENT here, as done in line 381: > > https://github.com/macports/macports-base/blob/4b13207d9f7f9aba1cc9eba266b3071318637a8c/vendor/tcl8.5.19/generic/tclFCmd.c#L415 Still seems broken in 8.6 at first glance too. - Josh
Re: port & root
On 2018-04-05 19:14, Joshua Root wrote: > My guess is that this is a bug in Tcl's file command. Deleting a > nonexistent file is not supposed to be considered an error. But by > starting two instances of port(1) at the same time, a race condition > arises. The 'file delete' command probably checks whether the file > exists, returns TCL_OK if not, and then attempts to delete the file. If > the file has been deleted by the other running instance (or indeed > anything else) between these two steps, then the actual deletion fails, > and it incorrectly returns an error. You are right. We hit a race condition in the implementation of 'file delete' in Tcl. It is missing another check for ENOENT here, as done in line 381: https://github.com/macports/macports-base/blob/4b13207d9f7f9aba1cc9eba266b3071318637a8c/vendor/tcl8.5.19/generic/tclFCmd.c#L415 @l2dy, Is our Travis job executing multiple 'port lint' commands in parallel? Then this is the same cause. Rainer
Re: port & root
On 2018-4-6 02:26 , Bjarne D Mathiesen wrote:
> When running port in certain situations under the root account, it fails
>
> In particular, I've got at script
>
> diffPorts :
> #!/macports/bin/bash
> diff <( port -q search ${1} ) \
> <( port -q installed name:^${1} | awk '{print $1}' ) \
> | grep '<'
>
> which I'm using to get a list of sub-ports not installed;
> eg : ./diffPorts php72-
>
> When running './diffPorts php72-' as root I get this :
>
> Error: /macports/bin/port: Failed to initialize MacPorts, error deleting
> "/macports/var/macports/home/Library/Preferences/com.apple.dt.Xcode.plist":
> no such file or directory
> < error deleting
> "/macports/var/macports/home/Library/Preferences/com.apple.dt.Xcode.plist":
> no such file or directory
> < while executing
> < "file delete -force "${target_dir}/com.apple.dt.Xcode.plist""
> < (procedure "macports::copy_xcode_plist" line 5)
> < invoked from within
> < "macports::copy_xcode_plist $env(HOME)"
> < (procedure "mportinit" line 540)
> < invoked from within
> < "mportinit ui_options global_options global_variations"
>
> each sub-part of the script works perfectly; that is
> port -q search php72-
> port -q installed name:^php72- | awk '{print $1}'
> have no issues; but when combining them in the redirection for 'diff' I
> get the error; but only when the script is being run under the root
> account. When running it under an admin or normal user account, there
> are no issues.
My guess is that this is a bug in Tcl's file command. Deleting a
nonexistent file is not supposed to be considered an error. But by
starting two instances of port(1) at the same time, a race condition
arises. The 'file delete' command probably checks whether the file
exists, returns TCL_OK if not, and then attempts to delete the file. If
the file has been deleted by the other running instance (or indeed
anything else) between these two steps, then the actual deletion fails,
and it incorrectly returns an error.
You don't see the problem when running as another user because we only
copy the Xcode plist when running as root. I guess the good news is that
there's little reason to be running this command as root?
- Josh
Re: port & root
On 2018-04-05 18:26, Bjarne D Mathiesen wrote:
> When running './diffPorts php72-' as root I get this :
Just to be specific, did you open a root shell first or did you run it
with sudo?
> Error: /macports/bin/port: Failed to initialize MacPorts, error deleting
> "/macports/var/macports/home/Library/Preferences/com.apple.dt.Xcode.plist":
> no such file or directory
> < error deleting
> "/macports/var/macports/home/Library/Preferences/com.apple.dt.Xcode.plist":
> no such file or directory
> < while executing
> < "file delete -force "${target_dir}/com.apple.dt.Xcode.plist""
> < (procedure "macports::copy_xcode_plist" line 5)
> < invoked from within
> < "macports::copy_xcode_plist $env(HOME)"
> < (procedure "mportinit" line 540)
> < invoked from within
> < "mportinit ui_options global_options global_variations"
This looks like the error we have also seen on Travis, but were never
able to reproduce:
https://lists.macports.org/pipermail/macports-dev/2017-June/035945.html
Rainer
