Re: [windows] Can't delete a closed file?
On Friday, 10 May 2019 at 19:10:05 UTC, Machine Code wrote: Well, I've had similar issue. The error message says "access denied" which I believe refers to the tmp directory; i.e, the user that is running your executable has no permissions to delete that file. Well, this has nothing to do with access rights, the user have full access to his own temp folder even in the most restrictive windows configuration. According to winapi docs, FreeLibrary doesn't guarantee file close op, and that's why you cannot delete it. Internally, windows maps a memory handle directly into the dll file and that pointer remains active until the application exits. Actually, I ran your example with several sequential calls to FreeLibrary, all returning success, even if it's clear that the internal reference count touched zero after the first call. The lock of the file disappears effectively after the application has exited and the dll can be safely deleted afterwards. If you want this dll deploying mechanism (which I consider a bad approach, contributing to the well known dll hell) check at startup if the dll already exists, delete it and redeploy if the existing one doesn't satisfy your version requirements. To avoid dll hell, dlls must be deployed in the system dir, or at least in the application folder through a normal installation process. In our network environment governed by a restrictive GPO, your application will never run.
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 10:09:23 UTC, Cym13 wrote:
Hi,
this is likely not related to D itself but hopefully someone
can help me with this since I'm rather new to windows
programming, I mainly work on linux. I'm trying to bundle a DLL
in a binary, write it in a temp folder, use it and remove the
dangling file.
So far I have the following file:
import std;
void main(string[] args) {
import core.runtime;
static immutable libcurl = import("libcurl.dll");
import std.file: write;
auto libpath = tempDir.buildPath("libcurl.dll");
libpath.write(libcurl);
auto libcurlMem = rt_loadLibrary(libpath.toStringz);
import std.net.curl;
"https://dlang.org/".byLine.count.writeln;
rt_unloadLibrary(libcurlMem);
remove(libpath);
}
Compiled with: dmd.exe -Jlibdir test.d
It almost work, I can write, load and use the library, but when
it comes to
removing it nothing works.
std.file.FileException@std\file.d(1045):
C:\users\cym13\Temp\libcurl.dll: Access denied.
0x00402377 in EntryPoint
0x00413BC7 in EntryPoint
0x00413B49 in EntryPoint
0x004139E3 in EntryPoint
0x0040B77F in EntryPoint
0x7B4754C2 in call_process_entry
0x7B477FC6 in ExitProcess
0x7B4754CE in call_process_entry
I tried using an explicit File handle to explicitely close the
file after
writing to it but that doesn't change anything.
I'm pretty sure I'm missing something basic about the way
windows handles
open files but I don't know what, could someone explain why
this doesn't work
the way I expect it to?
Well, I've had similar issue. The error message says "access
denied" which I believe refers to the tmp directory; i.e, the
user that is running your executable has no permissions to delete
that file. To be honest, I find all the permissions issues on
Windows a pain the pass so I did a simple tmp files manager. My
application create a tmp folder in same folder as in the
executable's directory then delete it when the application
finishs.
I also did write my own function to geneate a random string meant
to be used as tmp filename.
To delete the file, it might be finishing something and the OS
may hold it for some while; so try to delete the file a couple of
times, with some interval between the attemps, I do something
like this:
void deleteFile(string filename)
{
import std.file : remove, FileException;
enum int nAttempts = 5;
enum int interval = 200; // value in ms
int n = nAttempts;
do
{
try
{
remove(filename);
}
catch(FileException)
{
// the file may be temporary busy or something, so we try
again after a while, few times.
sleep(interval);
}
} while(n --> 0);
}
Re: [windows] Can't delete a closed file?
On Friday, 10 May 2019 at 15:06:46 UTC, Temtaime wrote: Lol, you don't have to load and unload the curl dll. std.net.curl have its own lazy libcurl loader. But i'm not sure if it tries to find the dll in the temp directory. If it is the case, then it simply doesn't unload the dll when you have called some function from it. As I said it's more about the general idea of bundling DLLs in than libcurl itself, but even in that case it doesn't search the temp directory for it. Besides, if it did, it would be a pretty important vulnerability.
Re: [windows] Can't delete a closed file?
Lol, you don't have to load and unload the curl dll. std.net.curl have its own lazy libcurl loader. But i'm not sure if it tries to find the dll in the temp directory. If it is the case, then it simply doesn't unload the dll when you have called some function from it.
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 10:09:23 UTC, Cym13 wrote: Hi, this is likely not related to D itself but hopefully someone can help me with this since I'm rather new to windows programming, I mainly work on linux. I'm trying to bundle a DLL in a binary, write it in a temp folder, use it and remove the dangling file. [...] That's a windows "feature". You can't delete files that are in use. The fact that you close the file in your program isn't a guarantee that there's no other open handles around. For the same reason you have to restart windows after every update to apply it.
Re: [windows] Can't delete a closed file?
On Friday, 10 May 2019 at 08:07:32 UTC, Cym13 wrote: Which C runtime are you using? The old and buggy DigitalMars one or the official MS one? I really don't know, how can I find out? I litterally just used dmd on the script above with no special options then ran the resulting exe. Yeah, so you're using the old and buggy DigitalMars C runtime. So you can use: - use LDC - use -m64 - use -ms32mscoff and there's a high chance that your problem will go away ;-) They all use the universal C runtime.
Re: [windows] Can't delete a closed file?
On Friday, 10 May 2019 at 07:09:45 UTC, Seb wrote: On Thursday, 9 May 2019 at 13:18:44 UTC, Cym13 wrote: On Thursday, 9 May 2019 at 13:02:51 UTC, Rene Zwanenburg wrote: On Thursday, 9 May 2019 at 12:33:37 UTC, Cym13 wrote: [...] You could try to use the find handle function in Process Explorer to figure out what process has the file open: https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer I did just that and my test program truly is the only process on the system having that file open. Unless I'm missing something that process-explorer doesn't see and is even true in wine's ultra light environment comprised of a single process, this is definitely not the issue. Which C runtime are you using? The old and buggy DigitalMars one or the official MS one? I really don't know, how can I find out? I litterally just used dmd on the script above with no special options then ran the resulting exe.
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 15:05:10 UTC, Andre Pany wrote: Can you reproduce the issue with other Dlls or is it only reproducible with curl dll? Does the issue with curl dll also exists if you do not call the curl function? Kind regards Andre I didn't have the time to test with another dll just yet but in that specific case the issue disappears if I don't call its functions. That's interesting.
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 13:18:44 UTC, Cym13 wrote: On Thursday, 9 May 2019 at 13:02:51 UTC, Rene Zwanenburg wrote: On Thursday, 9 May 2019 at 12:33:37 UTC, Cym13 wrote: [...] You could try to use the find handle function in Process Explorer to figure out what process has the file open: https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer I did just that and my test program truly is the only process on the system having that file open. Unless I'm missing something that process-explorer doesn't see and is even true in wine's ultra light environment comprised of a single process, this is definitely not the issue. Which C runtime are you using? The old and buggy DigitalMars one or the official MS one?
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 13:18:44 UTC, Cym13 wrote: On Thursday, 9 May 2019 at 13:02:51 UTC, Rene Zwanenburg wrote: On Thursday, 9 May 2019 at 12:33:37 UTC, Cym13 wrote: On Thursday, 9 May 2019 at 11:31:20 UTC, Cym13 wrote: ... To dismiss any doubt about AV or other processes coming into play I took the binary and ran it with wine on linux with the exact same end result. For reference my windows system is a 64b windows 10. You could try to use the find handle function in Process Explorer to figure out what process has the file open: https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer I did just that and my test program truly is the only process on the system having that file open. Unless I'm missing something that process-explorer doesn't see and is even true in wine's ultra light environment comprised of a single process, this is definitely not the issue. Can you reproduce the issue with other Dlls or is it only reproducible with curl dll? Does the issue with curl dll also exists if you do not call the curl function? Kind regards Andre
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 13:02:51 UTC, Rene Zwanenburg wrote: On Thursday, 9 May 2019 at 12:33:37 UTC, Cym13 wrote: On Thursday, 9 May 2019 at 11:31:20 UTC, Cym13 wrote: ... To dismiss any doubt about AV or other processes coming into play I took the binary and ran it with wine on linux with the exact same end result. For reference my windows system is a 64b windows 10. You could try to use the find handle function in Process Explorer to figure out what process has the file open: https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer I did just that and my test program truly is the only process on the system having that file open. Unless I'm missing something that process-explorer doesn't see and is even true in wine's ultra light environment comprised of a single process, this is definitely not the issue.
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 12:33:37 UTC, Cym13 wrote: On Thursday, 9 May 2019 at 11:31:20 UTC, Cym13 wrote: ... To dismiss any doubt about AV or other processes coming into play I took the binary and ran it with wine on linux with the exact same end result. For reference my windows system is a 64b windows 10. You could try to use the find handle function in Process Explorer to figure out what process has the file open: https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 11:31:20 UTC, Cym13 wrote: ... To dismiss any doubt about AV or other processes coming into play I took the binary and ran it with wine on linux with the exact same end result. For reference my windows system is a 64b windows 10.
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 11:07:53 UTC, Andre Pany wrote: On Thursday, 9 May 2019 at 10:09:23 UTC, Cym13 wrote: Hi, this is likely not related to D itself but hopefully someone can help me with this since I'm rather new to windows programming, I mainly work on linux. I'm trying to bundle a DLL in a binary, write it in a temp folder, use it and remove the dangling file. [...] I can't explain the behaviour, but you could store the temp file name in a string array and remove these file in the module destructor. That's the way how dub handles temp files/folders. https://github.com/dlang/dub/blob/master/source/dub/internal/utils.d#L98 Kind regards Andre No luck there.
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 11:11:56 UTC, Rumbu wrote: Since deploying a dll is a suspect behaviour outside a normal installation process, most probably you have a lock on the file put by windows defender or an antivirus if installed. Thanks for your input but I'm absolutely certain that it's not related to AV since I made sure to test in an AV-free environment (and I'd expect AV to kick in before loading the DLL, but I may be wrong on that). FreeLibrary doesn't guarantee that the dll file is closed, maybe other windows processes are accessing it (like prefetch). Anyway, you are deploying a dll just to read a file, It's over engineering, use WinAPI UrlDownloadToFile instead or any other winapi functions. Well, that's just a test script, it's feature or DLL doesn't matter.
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 10:09:23 UTC, Cym13 wrote:
Hi,
this is likely not related to D itself but hopefully someone
can help me with this since I'm rather new to windows
programming, I mainly work on linux. I'm trying to bundle a DLL
in a binary, write it in a temp folder, use it and remove the
dangling file.
So far I have the following file:
import std;
void main(string[] args) {
import core.runtime;
static immutable libcurl = import("libcurl.dll");
import std.file: write;
auto libpath = tempDir.buildPath("libcurl.dll");
libpath.write(libcurl);
auto libcurlMem = rt_loadLibrary(libpath.toStringz);
import std.net.curl;
"https://dlang.org/".byLine.count.writeln;
rt_unloadLibrary(libcurlMem);
remove(libpath);
}
Compiled with: dmd.exe -Jlibdir test.d
It almost work, I can write, load and use the library, but when
it comes to
removing it nothing works.
std.file.FileException@std\file.d(1045):
C:\users\cym13\Temp\libcurl.dll: Access denied.
0x00402377 in EntryPoint
0x00413BC7 in EntryPoint
0x00413B49 in EntryPoint
0x004139E3 in EntryPoint
0x0040B77F in EntryPoint
0x7B4754C2 in call_process_entry
0x7B477FC6 in ExitProcess
0x7B4754CE in call_process_entry
I tried using an explicit File handle to explicitely close the
file after
writing to it but that doesn't change anything.
I'm pretty sure I'm missing something basic about the way
windows handles
open files but I don't know what, could someone explain why
this doesn't work
the way I expect it to?
Since deploying a dll is a suspect behaviour outside a normal
installation process, most probably you have a lock on the file
put by windows defender or an antivirus if installed.
FreeLibrary doesn't guarantee that the dll file is closed, maybe
other windows processes are accessing it (like prefetch).
Anyway, you are deploying a dll just to read a file, It's over
engineering, use WinAPI UrlDownloadToFile instead or any other
winapi functions.
Re: [windows] Can't delete a closed file?
On Thursday, 9 May 2019 at 10:09:23 UTC, Cym13 wrote: Hi, this is likely not related to D itself but hopefully someone can help me with this since I'm rather new to windows programming, I mainly work on linux. I'm trying to bundle a DLL in a binary, write it in a temp folder, use it and remove the dangling file. [...] I can't explain the behaviour, but you could store the temp file name in a string array and remove these file in the module destructor. That's the way how dub handles temp files/folders. https://github.com/dlang/dub/blob/master/source/dub/internal/utils.d#L98 Kind regards Andre
