Adding to my previous post... My expectation of a "process wait" is that it returns when the process is finished executing. I don't think it necessarily means that the OS has released the executable file. In Windows, this is clearly the case. It's unfortunate that the Windows characteristic that an open file cannot be removed (as it can in Unix) causes build-execute-delete tools to take care that the delete succeeds and take some action if it did not. I think the most efficient solution is one that costs nothing in the more common case where the delete succeeds, and costs an inevitable short delay *only* in the less common case.
On Sun, Aug 16, 2020 at 11:27 AM Bob Alexander <bobja...@gmail.com> wrote: > Note that the "retry loop for deleting the executable" technique has zero > wait time if the delete succeeds. > > A year or so ago I submitted a bug report because I had a program that ran > hundreds of external processes (one at a time), and my Go program was way > slower that a Python program that did the same thing. Turns out that it was > because of the unconditional 5ms delay that was built into the Windows > incantation of os (Go\src\os\exec_windows.go). This is an excerpt from that > file: > > // NOTE(brainman): It seems that sometimes process is not dead > // when WaitForSingleObject returns. But we do not know any > // other way to wait for it. Sleeping for a while seems to do > // the trick sometimes. > // See https://golang.org/issue/25965 for details. > defer time.Sleep(5 * time.Millisecond) > defer p.Release() > > Seems like a great idea to have zero delay except in those few times that > it is necessary. > > On Sat, Aug 15, 2020 at 9:31 AM Bob Alexander <bobja...@gmail.com> wrote: > >> Here's what I think is really going on: >> >> At the end of a process's execution, 2 things happen: >> - The process's code finishes its execution -- wait returns. >> - The OS closes the executable file. >> >> The second item always "comes after" the first. On Windows the delay >> might be a few milliseconds, which can cause a problem, since an attempt to >> delete the executable right after wait returns can fail because the >> executable is still open. On Unix, this is not a problem because it's OK to >> "remove" an open file -- the file doesn't actually get deleted until all >> openers have closed the file. >> >> At one time, Go's os/exec for Windows had a built-in unconditional 5ms >> delay to prevent this from happening. Not sure if it still does that. >> >> It seems that, on Windows, if a program wants to delete the executable >> right after its process has finished, if should put the delete in a little >> retry loop: >> loop a few times (10?) >> try do delete the executable >> if the delete succeeds >> exit this loop >> wait a short time (1 ms ?) >> announce an error -- executable could not be deleted in reasonable >> time after process completion >> >> This retry loop would be the responsibility of any Windows program that >> wants to delete the executable file after its execution finishes. >> In reality, this is not done in very many places, done only by some tools >> like "go run" (build, run, delete), and the occasional user-written tool. >> The vast majority of places where external processes are run leave the >> executable file alone after process completion. >> >> Is it the responsibility of an OS like Windows the guarantee the the >> executable is closed when a process wait returns? I would say not, because >> that might cause a (small) delay in the vast majority of external process >> executions where the executable is not deleted immediately after. >> >> >> On Friday, August 14, 2020 at 8:55:55 AM UTC-7, jake...@gmail.com wrote: >>> >>> > Turns out it takes some time to release the lock on the folder, so we >>> should do some time.Sleep before the os.Remove, so that Windows can release >>> the lock. >>> >>> I do not believe that should be the case under normal Windows operation. >>> Using a Sleep in a case like this is always a hack. Sometimes it is the >>> only way, but it can fail randomly, especially under stress. There is >>> likely something else going on. Could be poorly written antivirus software, >>> or a finalizer that has not run in your go app, or something else. If it is >>> ok for it to work "most of the time", then maybe your Sleep() solution is >>> sufficient. But if you need real reliability, I suggest figuring out what >>> is really going on. >>> >>> On Friday, August 14, 2020 at 10:10:44 AM UTC-4 atakanc...@gmail.com >>> wrote: >>> >>>> Hello guys, I have solved the issue. >>>> >>>> Turns out it takes some time to release the lock on the folder, so we >>>> should do some time.Sleep before the os.Remove, so that Windows can release >>>> the lock. >>>> >>>> Thank you both for replying. >>>> >>>> 14 Ağustos 2020 Cuma tarihinde saat 16:21:17 UTC+3 itibarıyla >>>> jake...@gmail.com şunları yazdı: >>>> >>>>> This works fine for me on Windows 10. >>>>> What is "my.exe" doing? >>>>> Do you have third party antivirus software? If so, try turning it off. >>>>> They are notorious for causing this kind of problem. >>>>> >>>>> On Friday, August 14, 2020 at 5:02:36 AM UTC-4 atakanc...@gmail.com >>>>> wrote: >>>>> >>>>>> Hello dear fellow gophers, >>>>>> >>>>>> I had a relatively simple yet quite inconvenient issue which I felt >>>>>> the need to ask here. In my main() function; >>>>>> >>>>>> os.Remove("my.exe") // err is nil, my.exe is removed >>>>>> >>>>>> works in Windows without any errors, but when I call exec beforehand, >>>>>> I get access is denied error; >>>>>> >>>>>> buffer, err := exec.Command("my.exe", myArgs...).Output() // err is >>>>>> nil here, I get desired output >>>>>> os.Remove("my.exe") // remove "C:\\.......\my.exe": Access is denied >>>>>> >>>>>> I tried using cmd.Process.Kill(), cmd.Process.Wait(), >>>>>> cmd.Start()-ioutil.ReadlAll()-cmd.Wait() alternatives as well. I kept >>>>>> getting no errors until 'Access is denied'. >>>>>> >>>>>> I'm using go1.14 linux/amd64 for my compiler and Windows 10 >>>>>> Enterprise 10.0.18362. >>>>>> >>>>>> Thank you. >>>>>> >>>>> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "golang-nuts" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/golang-nuts/XglcNW0USuc/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> golang-nuts+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/f0a33b3f-a342-4a9a-9ccc-b0265bd2d60ao%40googlegroups.com >> <https://groups.google.com/d/msgid/golang-nuts/f0a33b3f-a342-4a9a-9ccc-b0265bd2d60ao%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAKyHfRNOAj-EFc_4NDtGMNqTw96f2PtJCp_XU%3Duc-eCH1NbY%2Bg%40mail.gmail.com.
