Re: array index out of bound may not throw exception?
On Friday, 21 July 2023 at 23:40:44 UTC, mw wrote: Is there a way to let it report on the spot when it happens? The best way is to wrap your thread's main function in a try-catch block and just print whatever error/exception is caught.
Re: Advice on debugging possible exception or crash
On Thursday, 6 July 2023 at 06:00:04 UTC, Cecil Ward wrote: My program is instrumented with a load of writeflns. At one point it looks as though it suddenly quits prematurely because the expected writeflns are not seen in the output. It could be that I am just reading the flow of control wrong as it goes ret, ret etc. I’m wondering if it is throwing an exception, or has a fault initiating a crash, perhaps even due to the fetching of arguments of one of the writeflns. In my limited experience, exceptions produce an error message though, and I’m not seeing anything. Any advice on how to debug this, silent termination ? I don’t have a debugger on this machine, but on an x86-64 box I could use gdb if I first take the time to work out how. Just some advice on if you're spawning threads, you should try and catch exceptions and errors at the top-most level and log them, otherwise they'll just vanish as the thread dies.
Options for Cross-Platform 3D Game Development
So, I've gotten the itch to have a go at game development in D, after doing a bit of it in Java last year. I've previously used LWJGL, which is a java wrapper for OpenGL, OpenAL, GLFW, and some other useful libs. The problem is, apparently OpenGL is deprecated for apple devices, so I don't really want to use that unless there are no decent alternatives. So far, the most promising I've seen is [bindbc-bgfx](https://code.dlang.org/packages/bindbc-bgfx), but it's been a pain to set up due to having to build the bgfx codebase, which requires a specific version of glibc that my distro (Linux Mint) doesn't offer yet. Are there any other recommendations for cross-platform rendering libraries? Of course I could use a pre-made game engine like Unity or Godot, but for me, most of the fun is in making the engine.
Re: How does D’s ‘import’ work?
On Thursday, 1 June 2023 at 03:47:00 UTC, Cecil Ward wrote: I have another question if I may, what do we do about getting makefiles right given that we have imports ? Others can correct me if I'm wrong, but I don't think that it is a priority for D to be specially compatible with makefiles in any way beyond the whole separation of compilation and linking.
Re: Log rotation in std.logger.filelogger
On Tuesday, 23 May 2023 at 11:21:07 UTC, apz28 wrote: On Wednesday, 17 May 2023 at 21:11:41 UTC, Vitalii wrote: Hello! Please tell me how to enable log rotation in std.logger.filelogger? Without log rotation, it seems that the std.logger.filelogger is useless, because it will quickly take up all the free space on the hard drive. Or can try my log package https://github.com/apz28/dlang/tree/main https://github.com/apz28/dlang/tree/main/source/pham/external/std/log log_logger.d RollingFileLogger Happy coding! Please don't take this as an insult, as I've also written my own logging library lol. But I have a feeling that our efforts would be better spent in improving the standard library's implementation rather than making our own logger. Hopefully this will become easier in the future as issues are migrated to github and more effort is put into organization.
Re: iopipe code to count lines in gzipped file works with v 0.1.7 but fails with 0.2.1
On Saturday, 8 August 2020 at 02:06:36 UTC, Steven Schveighoffer wrote: On 8/7/20 9:31 PM, Steven Schveighoffer wrote: On 8/7/20 8:57 PM, Steven Schveighoffer wrote: I think this is an issue with dub when using an inline recipe file, but I don't know? ugh. This is an issue with iopipe specifying io version 0.2.x. I will fix this. OK, iopipe 0.2.2 is released, with no specific io dependency. It should work with io 0.3.1 as a separate listed dependency. -Steve Thank you for looking into this. I assumed the problem was at my end, as you have actually saved my bacon before, processing exactly the same file, the problem being that it was a blocked gzip file (ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz) I will raise an issue, as I think io is being imported and linked ok.
iopipe code to count lines in gzipped file works with v 0.1.7 but fails with 0.2.1
Hi,
This code to count lines in a gzipped file exits with "Program
exited with code -9" when run with the latest version of the
library, I guess because I am doing unsafe things. Could someone
tell me how to change it to make it work? The actual program I'm
writing processes a file line by line, so ideally I'd like to
keep the structure of open a file, then foreach over it.
Thanks very much
Andrew
/+ dub.sdl:
name "hello"
dependency "iopipe" version="~>0.2.0"
+/
import std.stdio;
import std.typecons;
import iopipe.textpipe;
import iopipe.zip;
import iopipe.bufpipe;
import std.io : File = File;
void main()
{
auto counter = 0;
auto fileToRead =
File("file.gz").refCounted.bufd.unzip(CompressionFormat.gzip);
foreach (line; fileToRead.assumeText.byLineRange!false)
{
counter++;
}
writeln(counter);
}
Good examples for dmd.frontend?
Hello, I'm doing some experiments trying to use the DMD front-end as a library to generate code (Why3ML) that can be used in a formal verification tool. However, documentation is a little sparse. I've looked through the examples in dmd/src/examples, and they were a fine starting point but now I'm getting to the point where I need to snag function arguments, types, and contracts out of the AST. Can anybody point me to some good examples of other projects that use the DMD front-end that I might be able to reference? Thanks, Doc
curl: can't send email?
What wrong?
```
import std.net.curl;
void main() {
auto smtp = SMTP("smtps://smtp.gmail.com:465");
smtp.setAuthentication("qwe...@gmail.com", "password");
smtp.mailTo = ["std.net.curl.CurlException@std/net/curl.d(4364): Failed sending
data to the peer on handle 55FACCB58AC0
??:? pure @safe void
std.exception.bailOut!(std.net.curl.CurlException).bailOut(immutable(char)[], ulong, scope const(char)[]) [0x55facbd8db5e]
??:? pure @safe bool
std.exception.enforce!(std.net.curl.CurlException).enforce!(bool).enforce(bool, lazy const(char)[], immutable(char)[], ulong) [0x55facbd8dad1]
??:? void std.net.curl.Curl._check(int) [0x55facbd7209b]
??:? int
std.net.curl.Curl.perform(std.typecons.Flag!("throwOnError").Flag) [0x55facbd724c7]
??:? int
std.net.curl.SMTP.perform(std.typecons.Flag!("throwOnError").Flag) [0x55facbd71a80]
??:? _Dmain [0x55facbd69b6b]
Re: D threading and shared variables
On Sunday, 7 April 2019 at 14:08:07 UTC, Archie Allison wrote: I have written an industrial control program which uses serial ports to communicate with hardware but am having problems, perhaps with shared memory, on Windows. The SerialPort class calls C object-file functions. Transmits are on one thread and receives on another (all within a class derived from Thread), with a signal(created from a mutex) notifying the transmit thread when a packet has arrived. This generally works OK when tied to a Console but when link options are changed to be SUBSYSTEM:WINDOWS and ENTRY:mainCRTStartup it rarely does. Compiling with -vtls shows the serial port is in thread local storage. As a hardware resource, I wasn't sure if this matters. I've tried casting to a shared object so it no longer appears in the -vtls list, with no difference. Does anyone have ideas about where I may have misunderstood the threading and shared-memory design of D or what I can look at? When you say it "rarely works" when run as a GUI app (vs console), it makes me think that there's probably a race condition going on, and the extra context switching that takes place in GUI mode makes it more likely. I haven't tried it in D, but you may be able use Application Verifier with your app to add checks for lock contention. Without more information about the way your threads are synchronized it's hard to say for sure though.
Re: countUntil to print all the index of a given string.
On Sunday, 25 February 2018 at 13:25:56 UTC, Vino wrote:
On Sunday, 25 February 2018 at 03:41:27 UTC, Jonathan M Davis
wrote:
On Sunday, February 25, 2018 02:58:33 Seb via
Digitalmars-d-learn wrote:
[...]
That will help eventually, but it requires a compiler flag, so
it's really not going to help for code in general right now,
and the fact that that DIP does nothing to solve the problem
of how to create exception messages without allocating them on
the GC heap means that exceptions in general are still
frequently going to result in allocations unless you jump
through several hoops to be able to create an exception
message that's in a static array or malloc-ed or something.
So, I don't know how much it's going to help in practice
outside of code where the programmer is absolutely determined
to have no GC allocations.
[...]
Yeah. There does tend to be a correlation between @nogc and
whether a range is lazy, but it's not guaranteed, so I'm
inclined to think that it's a poor idea to rely on it and that
it's just ultimately better to look at the documentation or
even the code.
- Jonathan M Davis
Hi All,
Sorry, I am not able to see any correlation between the
raised topic and the conversation that is happening in this
forum, could any one please explain on of what is going on and
how do you thing that this conversation is related to the topic
raised, if not would suggest you to open a new topic.
From,
Vino.B
I think this will work:
import std.container;
import std.algorithm;
import std.range;
import std.stdio;
void main () {
auto a = Array!string("Test1", "Test2", "Test3", "Test1",
"Test2");
auto b = Array!string("Test1", "Test2", "Test3");
foreach(i; b[])
a[].enumerate.filter!(a => a[1] == i).map!(a =>
a[0]).SList!size_t[].writeln;
Re: Using iopipe to stream a gzipped file
On Thursday, 4 January 2018 at 15:48:21 UTC, Steven Schveighoffer wrote: It's now been updated, see version 0.0.3. Note, the performance isn't something I focused on. I'll note that gzcat | wc -l is 2x faster than your simple example on that file. I can think of a couple reasons for this: 1. gzcat may use mmap to increase read speed 2. gzcat may read larger chunks at once (this can be tuned using iopipe as well, just set the optimizedReadSize). 3. gzcat file.gz | iopipe_byline -nooutput is about 20% faster than using wc -l, so it's definitely not the line parsing. Let me know if this works correctly for your other test cases! If not, file an issue: https://github.com/schveiguy/iopipe/issues -Steve That works perfectly, thank you very much!
Re: Using iopipe to stream a gzipped file
On Thursday, 4 January 2018 at 12:15:27 UTC, Steven Schveighoffer wrote: On 1/4/18 7:01 AM, Andrew wrote: Ah thank you, that makes sense. These types of files are compressed using the bgzip utility so that the file can be indexed meaning specific rows extracted quickly (there's more details of this here http://www.htslib.org/doc/tabix.html and the code can be found here: https://github.com/samtools/htslib/blob/develop/bgzf.c) Hm... that utility seems to say it will result in bgz file extension? So this must be an extraction from one of those files? In any case, I'll figure out how to deal with concatenated gzip file, and update iopipe. Next version will focus on a bunch of stuff relating to the 2 zip threads recently posted here. Thanks! -Steve That would be really great for me, thank you! By default bgzip produces a file with the standard .gz extension. Looking at the code it adds an extra field to the standard gzip header: /* BGZF/GZIP header (speciallized from RFC 1952; little endian): +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ | 31|139| 8| 4| 0| 0|255| 6| 66| 67| 2|BLK_LEN| +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+ BGZF extension: ^ ^ ^ ^ | | | | FLG.EXTRA XLEN B C BGZF format is compatible with GZIP. It limits the size of each compressed block to 2^16 bytes and adds and an extra "BC" field in the gzip header which records the size. */ Thanks again! Andrew
Re: Using iopipe to stream a gzipped file
On Thursday, 4 January 2018 at 02:44:09 UTC, Steven Schveighoffer wrote: On 1/3/18 12:03 PM, Andrew wrote: Thanks for looking into this. So it looks like the file you have is a concatenated gzip file. If I gunzip the file and recompress it, it works properly. Looking at the docs of zlib inflate [1]: " Unlike the gunzip utility and gzread() ..., inflate() will not automatically decode concatenated gzip streams. inflate() will return Z_STREAM_END at the end of the gzip stream. The state would need to be reset to continue decoding a subsequent gzip stream." So what is happening is the inflate function is returning Z_STREAM_END, and I'm considering the stream done from that return code. I'm not sure yet how to fix this. I suppose I can check if any more data exists, and then re-init and continue. I have to look up what a concatenated gzip file is. gzread isn't good for generic purposes, because it requires an actual file input (I want to support any input type, including memory data). -Steve [1] https://github.com/dlang/phobos/blob/master/etc/c/zlib.d#L874 Ah thank you, that makes sense. These types of files are compressed using the bgzip utility so that the file can be indexed meaning specific rows extracted quickly (there's more details of this here http://www.htslib.org/doc/tabix.html and the code can be found here: https://github.com/samtools/htslib/blob/develop/bgzf.c)
Re: Using iopipe to stream a gzipped file
On Wednesday, 3 January 2018 at 16:09:19 UTC, Steven
Schveighoffer wrote:
On 1/3/18 9:45 AM, Andrew wrote:
Hi,
I have a very large gziped text file (all ASCII characters and
~500GB) that I want to stream and process line-by-line, and I
thought the iopipe library would be perfect for this, but I
can't seem to get it to work. So far, this is the closest I
have to getting it to work:
import iopipe.textpipe;
import iopipe.zip;
import iopipe.bufpipe;
import iopipe.stream;
void main()
{
auto fileToRead =
openDev("file.gz").bufd.unzip(CompressionFormat.gzip);
foreach (line; fileToRead.assumeText.byLineRange!false)
{
\\ do stuff
}
}
but this only processes the first ~200 odd lines (I guess the
initial read into the buffer). Can anyone help me out?
Do you have a sample file I can play with? Your iopipe chain
looks correct, so I'm not sure why it wouldn't work.
-Steve
A sample file (about 250MB) can be found here:
ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz
It should have 1,103,800 lines, but the following code only
reports 256:
import iopipe.textpipe;
import iopipe.zip;
import iopipe.bufpipe;
import iopipe.stream;
import std.stdio;
void main()
{
auto fileToRead =
openDev("ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz").bufd.unzip(CompressionFormat.gzip);
auto counter = 0;
foreach (line; fileToRead.assumeText.byLineRange!false)
{
counter++;
}
writeln(counter);
}
Thanks for looking into this.
Andrew
Using iopipe to stream a gzipped file
Hi,
I have a very large gziped text file (all ASCII characters and
~500GB) that I want to stream and process line-by-line, and I
thought the iopipe library would be perfect for this, but I can't
seem to get it to work. So far, this is the closest I have to
getting it to work:
import iopipe.textpipe;
import iopipe.zip;
import iopipe.bufpipe;
import iopipe.stream;
void main()
{
auto fileToRead =
openDev("file.gz").bufd.unzip(CompressionFormat.gzip);
foreach (line; fileToRead.assumeText.byLineRange!false)
{
\\ do stuff
}
}
but this only processes the first ~200 odd lines (I guess the
initial read into the buffer). Can anyone help me out?
Thanks very much
Andrew
Re: static array internal & dangling reference
On Saturday, 12 November 2016 at 11:03:31 UTC, Mike Parker wrote:
[...]
You *have* created a dangling pointer. It's just that for such
a simple little program, the part of the stack where the
original array was allocated isn't stomped at the point where
you access it after the function call. The same sort of program
will also work in C, where it's not uncommon for functions to
return string literals that are immediately printed to stdout
or a log file.
One difference from C in this case is that it's still possible
to make things work even after the stack has been stomped and
the memory is no longer valid simply by increasing the length
of the returned slice: sb ~= 0, or perhaps sb.length+=n. This
will cause an allocation and sb will then own the memory block
to which it points.
Bear in mind that static arrays are implicitly sliced when
passed to or returned from a function anywhere a dynamic array
is expected. If you modify f() to look like this:
int[] f()
{
int[10] sa;
foreach(int i, ref sa_i;sa){
sa_i=i;
}
return sa;
}
I thought that if you changed the function signature to int[10]
f() to return a static array, this should be returned by value,
and so should be safe to use, but it seems that this too points
to the same memory.
To make it safe, should I be using:
auto sb = f().dup;
Thanks
Andrew
Re: Get date at compile time
On Saturday, 1 October 2016 at 14:43:31 UTC, Adam D. Ruppe wrote: On Saturday, 1 October 2016 at 14:41:22 UTC, Andrew wrote: Is there any way to get the system date at compile time. Not exactly, but the special symbol __TIMESTAMP__ gets a string out of the compiler at build time. http://dlang.org/spec/lex.html#specialtokens perfect, works a treat. Thanks both Andrew
Get date at compile time
Hi, Is there any way to get the system date at compile time. I want something like: static string compileDate = Clock.currTime.toString; but that fails. Thanks very much Andrew
Re: Configuring of dub for the application reading enviroment variable
On Friday, 30 September 2016 at 10:31:52 UTC, MGW wrote: My STARTING application shall read the enviroment variable. For example MY_VARIABLE= "I'm Gena". The MY_VARIABLE variable needs to be set in dub.json so what she would be visible in case of start of my application. Purpose: to set LD_LIBRARY_PATH having specified a certain directory in dub packet. Please, write an example of dub.json for it. Can you use preBuildCommands, e.g.: "preBuildCommands" : ["export LD_LIBRARY_PATH=dir"]
Re: Pipe one shell command into another
On Saturday, 30 January 2016 at 15:57:49 UTC, Griffon26 wrote: On Saturday, 30 January 2016 at 15:12:26 UTC, Andrew wrote: foreach(line; pipesLs.stdout.byLine) pipesSort.stdin.writeln(line); Because you write sort's input first and read its output later, it might end up blocking if ls generates too much data. The output pipe of sort will fill up, causing sort to block and not read data from its input pipe anymore, resulting in your program blocking when the input pipe has filled up. I have a piece of code using poll to write to two input pipes and read from one output pipe, but it's a bit more complex than what you need. However, it also uses pipeShell to specify multiple commands at once. Since your program only needs to read the output (and not generate input), you may find pipeShell useful. https://github.com/Griffon26/tdiff3/blob/87709dd51c279e9896f37ba1cef477a525e44562/source/gnudiff.d Disclaimer: I'm not an experienced D programmer. My D code may not be pretty =) pipeShell did the trick. Thank you very much! Andrew
Pipe one shell command into another
Hi,
I'd like to run a shell command which involves piping one thing
into another and then processes the output line by line, i.e.
something like "ls -l | sort -k5,5n"
What I've come up so far with is:
import std.process;
import std.stdio;
void main(){
auto pipesLs = pipeProcess(["ls", "-l"], Redirect.stdout);
auto pipesSort = pipeProcess(["sort", "-k5,5n"], Redirect.all);
scope (exit) wait(pipesSort.pid);
foreach(line; pipesLs.stdout.byLine)
pipesSort.stdin.writeln(line);
pipesSort.stdin.close;
foreach(line; pipesSort.stdout.byLine)
writeln(line);
}
This seems to work on this simple example, but is there a better
way to do it, and if not, is this reliable? Also, could someone
explain to me the necessity of the wait command?
Thanks very much
Andrew
demangle()
When I run the DMD profile, the "overview" at the end of the trace.log contains some mangles names (such as: _D3std5stdio4File17LockingTextWriter12__T3putTAyaZ3putMFAyaZ13trustedFwriteFNbNiNexPvmmPOS4core4stdc5stdio7__sFILEZm When I call demangle() on those mangled names it returns the mangled name, presumably because it can't detangle the name (according to the spec). How do I get the demangled name? Why can't it demangle? What is the consequence to my program of its inability to demangle (is it a many to 1?)? Thanks
Re: Binding to GSL library
On Wednesday, 25 November 2015 at 16:45:51 UTC, Radek wrote: i have found bug. It shoul be alias gsl_complex = _gsl_complex; not alias gsl_complex = _gsl_complex*; On Wednesday, 25 November 2015 at 16:35:06 UTC, drug wrote: A little bit offtopic but do you know about https://github.com/abrown25/gsld? It would be nice to join efforts. Sure, I'll share my code :) Ask me next month. That what I'm doing is my student project and first i need to complete it :) Hi, I've been a little slow updating that repository because of work, and I got a little stuck with function pointers, I hope to get back to it soon. It would be lovely if it's all finished by that time :)
Arty of Constructor
The documentation gives plenty of examples of how to use a static if with the arity trait, but how do I specify the constructor of an object as the parameter to arity? Thanks
Re: Efficiency of immutable vs mutable
This: On Tuesday, 3 November 2015 at 04:08:09 UTC, TheFlyingFiddle wrote: __gshared char[4] lookup = ['a', 't', 'g', 'c]; Has the same efficiency gain as immutable, so it looks like a thread-local vs global difference and the extra cost is going through the thread-local lookup. Thanks
Efficiency of immutable vs mutable
I've written a short D program that involves many lookups into a static array. When I make the array immutable the program runs faster. This must mean that immutable is more than a restriction on access, it must affect the compiler output. But why and how? Thanks Andrew
