Re: request assistance resolving a std.net.curl segmentation fault
On Friday, 19 May 2023 at 12:40:29 UTC, Danny Arends wrote:
On Friday, 19 May 2023 at 11:07:01 UTC, anonymouse wrote:
What am I doing wrong here?
[SNIP]
You're running the whole thing in a while(TRUE) loop,
recreating the curl object re-initiating the transfer and file
pointer, etc.
The reason I used a while loop was to detect loss of internet
connection and resume the process once the connection is
re-established. What would have been a better approach?
furthermore, the curl.set(CurlOption.writedata,
); doesn't work as you expect..
The idea was to detect an incomplete download and continue from
where it left off. I'm sometimes downloading files 15Gb or
greater. Reaching 80% and having to restart the process is a
nogo. As I understand it, `CurlOption.writedata` allows me to
achieve that goal. Is there a better option to accomplish the
same?
After fiddling a bit, this works:
curl.onReceive = (ubyte[] data) { fp.rawWrite(data); return
data.length;};
Thank you for your assistance thus far.
--anonymouse
Re: Can dmd compile a favicon.ico to exe file ?
On 19/05/2023 9:39 PM, John Xu wrote: On Thursday, 18 May 2023 at 15:39:05 UTC, Richard (Rikki) Andrew Cattermole wrote: On 19/05/2023 2:19 AM, John Xu wrote: On Monday, 15 May 2023 at 03:54:03 UTC, Richard (Rikki) Andrew Cattermole wrote: That is only for OMF target. You need rc that comes with Visual Studio C++ build tools. Alternatively windres from mingw may work (I haven't tested). How can I add this step to dub.sdl ? You would use one of the build commands and then put the result file name into source files. However you may not want to require it in normal builds (since the file it outputs won't change you can just commit that) just to ensure your build is reproducible. sourceFiles ? or libFiles ? sourceFiles. libFiles is for shared libraries.
Re: Lockstep iteration in parallel: Error: cannot have parameter of type `void`
On 5/19/23 02:17, kdevel wrote: Should this compile? dmd says Multiple points: - lockstep works only with foreach loops but it's not a range. - std.range.zip can be used instead but it does not provide 'ref' access to its elements. - However, since slices are already references to groups of elements, you don't need 'ref' anyway. - You seem to want to assign to elements in parallel; such functionality already exists in std.parallelism. - Phobos documentation is not very useful these days as it's not clear from the following page that there are goodies like amap, asyncBuf, etc. It just shows parallel() and a few friends at the top: https://dlang.org/phobos/std_parallelism.html One needs to know to click TaskPool: https://dlang.org/phobos/std_parallelism.html#.TaskPool The following amap example there may be useful for your case but I could not make the types work: // Same thing, but explicitly allocate an array // to return the results in. The element type // of the array may be either the exact type // returned by functions or an implicit conversion // target. auto squareRoots = new float[numbers.length]; taskPool.amap!sqrt(numbers, squareRoots); // Multiple functions, explicit output range, and // explicit work unit size. auto results = new Tuple!(float, real)[numbers.length]; taskPool.amap!(sqrt, log)(numbers, 100, results); https://dlang.org/phobos/std_parallelism.html#.TaskPool.amap Ali
Re: Best way to use C library
On Friday, 19 May 2023 at 18:31:45 UTC, Maximilian Naderer wrote: Hello guys, So what’s currently the best way to use a big C library? Let’s assume something like cglm assimp glfw ImportC doesn’t really work for such huge libraries, I’ll investigate further. Deimos is outdated or there are no bindings. I know that there is a dub package for glfw which works fine. But how would I do something for assimp or cglm. The dub assimp package is quite outdated. Am I stuck with manually creating interface files either by hand or automation? I’m hope somebody could give me some insights. Thank you ! Kind regards from Austria Max If you aim static linking or static binding then you can give [dstep](https://github.com/jacob-carlborg/dstep) a shot. Otherwise and if you have the time to then dont forget to report the problems founds when you have tried ImportC.
Re: Best way to use C library
On Friday, 19 May 2023 at 18:31:45 UTC, Maximilian Naderer wrote: Hello guys, So what’s currently the best way to use a big C library? Let’s assume something like cglm assimp glfw ImportC doesn’t really work for such huge libraries, I’ll investigate further. Deimos is outdated or there are no bindings. I know that there is a dub package for glfw which works fine. But how would I do something for assimp or cglm. The dub assimp package is quite outdated. Am I stuck with manually creating interface files either by hand or automation? I’m hope somebody could give me some insights. Thank you ! Kind regards from Austria Max If there are issues using those libraries, you should report the bugs.
Best way to use C library
Hello guys, So what’s currently the best way to use a big C library? Let’s assume something like cglm assimp glfw ImportC doesn’t really work for such huge libraries, I’ll investigate further. Deimos is outdated or there are no bindings. I know that there is a dub package for glfw which works fine. But how would I do something for assimp or cglm. The dub assimp package is quite outdated. Am I stuck with manually creating interface files either by hand or automation? I’m hope somebody could give me some insights. Thank you ! Kind regards from Austria Max
Re: request assistance resolving a std.net.curl segmentation fault
On Friday, 19 May 2023 at 12:28:20 UTC, kdevel wrote: On Friday, 19 May 2023 at 11:07:01 UTC, anonymouse wrote: What am I doing wrong here? [...] curl.set(CurlOption.writedata, ); According to [1] this line must read ``` curl.set(CurlOption.writedata, cast (void *) fp.getFP()); ``` [1] https://curl.se/libcurl/c/CURLOPT_WRITEDATA.html Thank you so much.
Re: request assistance resolving a std.net.curl segmentation fault
On Friday, 19 May 2023 at 12:28:20 UTC, kdevel wrote: On Friday, 19 May 2023 at 11:07:01 UTC, anonymouse wrote: What am I doing wrong here? [...] curl.set(CurlOption.writedata, ); According to [1] this line must read ``` curl.set(CurlOption.writedata, cast (void *) fp.getFP()); ``` [1] https://curl.se/libcurl/c/CURLOPT_WRITEDATA.html Thank you so much.
Re: request assistance resolving a std.net.curl segmentation fault
On Friday, 19 May 2023 at 11:07:01 UTC, anonymouse wrote:
What am I doing wrong here?
```D
import std.net.curl: Curl, CurlOption, CurlException;
import std.file: exists;
import std.stdio: File, writefln;
import core.thread: Thread;
void downloadFile(string url, string filename)
{
while (true) {
try {
File fp;
if (filename.exists())
fp.open(filename, "a");
else
fp.open(filename, "w");
Curl curl;
curl.initialize();
curl.onProgress = delegate int(size_t dltotal,
size_t dlnow, size_t ultotal, size_t ulnow)
{
writefln("Progress: %s of %s", dlnow, dltotal);
return 0;
};
curl.set(CurlOption.url, url~filename);
curl.set(CurlOption.resume_from_large, fp.size());
// Start the download
curl.set(CurlOption.writedata, );
curl.perform();
// Close the file
fp.close();
writefln("Download as %s complete.", filename);
break;
} catch (CurlException e) {
writefln("Error while downloading: %s", e.msg);
// Wait for a bit before retrying
Thread.sleep(imported!"core.time".seconds(10));
}
}
}
void main()
{
string url =
"https://downloads.dlang.org/releases/2.x/2.103.1/;;
string filename = "dmd.2.103.1.dmg";
downloadFile(url, filename);
}
```
Output:
```
./download_file
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
zsh: segmentation fault ./download_file
```
Thanks.
--anonymouse
You're running the whole thing in a while(TRUE) loop, recreating
the curl object re-initiating the transfer and file pointer, etc.
furthermore, the curl.set(CurlOption.writedata,
); doesn't work as you expect..
After fiddling a bit, this works:
```D
import std.net.curl: Curl, CurlOption, CurlException;
import std.file: exists;
import std.stdio: File, writefln;
import core.thread: Thread;
void downloadFile(string url, string filename){
try {
File fp;
fp.open(filename, "w");
Curl curl;
curl.initialize();
curl.onProgress = delegate int(size_t dltotal, size_t dlnow,
size_t ultotal, size_t ulnow){
writefln("Progress: %s of %s", dlnow, dltotal);
return 0;
};
curl.onReceive = (ubyte[] data) { fp.rawWrite(data); return
data.length;};
curl.set(CurlOption.url, url~filename);
// Start the download
curl.perform();
writefln("Download as %s complete.", filename);
} catch (CurlException e) {
writefln("Error while downloading: %s", e.msg);
}
}
void main(){
string url =
"https://downloads.dlang.org/releases/2.x/2.103.1/;;
string filename = "dmd.2.103.1.dmg";
downloadFile(url, filename);
}
```
Re: request assistance resolving a std.net.curl segmentation fault
On Friday, 19 May 2023 at 11:07:01 UTC, anonymouse wrote: What am I doing wrong here? [...] curl.set(CurlOption.writedata, ); According to [1] this line must read ``` curl.set(CurlOption.writedata, cast (void *) fp.getFP()); ``` [1] https://curl.se/libcurl/c/CURLOPT_WRITEDATA.html
request assistance resolving a std.net.curl segmentation fault
What am I doing wrong here?
```D
import std.net.curl: Curl, CurlOption, CurlException;
import std.file: exists;
import std.stdio: File, writefln;
import core.thread: Thread;
void downloadFile(string url, string filename)
{
while (true) {
try {
File fp;
if (filename.exists())
fp.open(filename, "a");
else
fp.open(filename, "w");
Curl curl;
curl.initialize();
curl.onProgress = delegate int(size_t dltotal, size_t
dlnow, size_t ultotal, size_t ulnow)
{
writefln("Progress: %s of %s", dlnow, dltotal);
return 0;
};
curl.set(CurlOption.url, url~filename);
curl.set(CurlOption.resume_from_large, fp.size());
// Start the download
curl.set(CurlOption.writedata, );
curl.perform();
// Close the file
fp.close();
writefln("Download as %s complete.", filename);
break;
} catch (CurlException e) {
writefln("Error while downloading: %s", e.msg);
// Wait for a bit before retrying
Thread.sleep(imported!"core.time".seconds(10));
}
}
}
void main()
{
string url =
"https://downloads.dlang.org/releases/2.x/2.103.1/;;
string filename = "dmd.2.103.1.dmg";
downloadFile(url, filename);
}
```
Output:
```
./download_file
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
zsh: segmentation fault ./download_file
```
Thanks.
--anonymouse
request assistance resolving a std.net.curl sementation fault
What am I doing wrong here?
```D
import std.net.curl: Curl, CurlOption, CurlException;
import std.file: exists;
import std.stdio: File, writefln;
import core.thread: Thread;
void downloadFile(string url, string filename)
{
while (true) {
try {
File fp;
if (filename.exists())
fp.open(filename, "a");
else
fp.open(filename, "w");
Curl curl;
curl.initialize();
curl.onProgress = delegate int(size_t dltotal, size_t
dlnow, size_t ultotal, size_t ulnow)
{
writefln("Progress: %s of %s", dlnow, dltotal);
return 0;
};
curl.set(CurlOption.url, url~filename);
curl.set(CurlOption.resume_from_large, fp.size());
// Start the download
curl.set(CurlOption.writedata, );
curl.perform();
// Close the file
fp.close();
writefln("Download as %s complete.", filename);
break;
} catch (CurlException e) {
writefln("Error while downloading: %s", e.msg);
// Wait for a bit before retrying
Thread.sleep(imported!"core.time".seconds(10));
}
}
}
void main()
{
string url =
"https://downloads.dlang.org/releases/2.x/2.103.1/;;
string filename = "dmd.2.103.1.dmg";
downloadFile(url, filename);
}
```
Output:
```
./download_file
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
Progress: 0 of 0
zsh: segmentation fault ./download_file
```
Thanks.
--anonymouse
Re: Can dmd compile a favicon.ico to exe file ?
On Thursday, 18 May 2023 at 15:39:05 UTC, Richard (Rikki) Andrew Cattermole wrote: On 19/05/2023 2:19 AM, John Xu wrote: On Monday, 15 May 2023 at 03:54:03 UTC, Richard (Rikki) Andrew Cattermole wrote: That is only for OMF target. You need rc that comes with Visual Studio C++ build tools. Alternatively windres from mingw may work (I haven't tested). How can I add this step to dub.sdl ? You would use one of the build commands and then put the result file name into source files. However you may not want to require it in normal builds (since the file it outputs won't change you can just commit that) just to ensure your build is reproducible. sourceFiles ? or libFiles ?
Lockstep iteration in parallel: Error: cannot have parameter of type `void`
```
import std.range;
import std.parallelism;
void vec_op (double [] outp, const double [] inp,
double function (double) f)
{
foreach (ref a, b; parallel (lockstep (outp, inp)))
a = f (b);
}
```
Should this compile? dmd says
```
[...]/src/phobos/std/parallelism.d(4094): Error: cannot have
parameter of type `void`
[...]/src/phobos/std/parallelism.d(4095): Error: cannot have
parameter of type `void`
[...]/src/phobos/std/parallelism.d(3619): Error: template
instance `std.parallelism.ParallelForeach!(Lockstep!(double[],
const(double)[]))` error instantiating
lspar.d(7):instantiated from here:
`parallel!(Lockstep!(double[], const(double)[]))`
```
