What prevents ImportC from using .h directly?
Hi D Import D is working quite well so far. One limitation is that module definitions require a tiny *.c file which is often just: ```C #include ``` Creating this file is trivial and has almost no knock-on effects, except when dealing with single file programs. I have quite a few small utilities with dub headers. It would be handy if I could set an import path and use ImportC on the header file directly, skipping the small C file all together. A specific example in my case follows. Given a C-lib interface defined in the header file: ``` /usr/include/cspice/SpiceUsr.h ``` it would be neat if a dub header similar to the following worked: ```d #!/usr/bin/env dub /+ dub.sdl: name "some_app" cImportPaths "/usr/local/include/cspice" +/ import SpiceUsr; // ... source continues ``` So, why does ImportC need *.c files exclusively? I'm sure it solves a problem, but I don't know what that problem is.
Re: Recommendations on porting Python to D
On Thursday, 25 April 2024 at 16:57:53 UTC, mw wrote: On Wednesday, 24 April 2024 at 22:07:41 UTC, Chris Piker wrote: Python-AST to D source converter may already exist? https://github.com/joortcom/eiffel_rename/tree/main/yi A rudimentary converter from (extended) Python to D. Maybe you can use it as a starting point. Thanks for the suggestions. I put the question aside for a bit, but yesterday ran across a python transpiler here: https://github.com/py2many/py2many It already has support for C++, Go and others. Since I have mountains of python code created over many years, maybe it would be worth contributing to this project out of self interest. Can you take a look at py2many and see what you think about it? Getting D on the support list might be good.
Re: Recommendations on porting Python to D
On Thursday, 25 April 2024 at 07:04:13 UTC, Sergey wrote: On Wednesday, 24 April 2024 at 22:07:41 UTC, Chris Piker wrote: Python-AST to D source converter may already exist? Another possible way maybe is using C :) Python -> C -> D https://wiki.python.org/moin/PythonImplementations#Compilers Thanks for the info, though I think going to a low-level language in the middle will make code that has almost no high level structure. Converting that back in to class-style D would likely take a while, in which case a manual port is a better option. Both python and D offer garbage collection so going through a non-GC language first would probably obfuscate the intended structure. Maybe converting this AST to another AST format and then using a D code generator would be a better route. So backing up, are there any AST -> D source generators in existence?
Re: Recommendations on porting Python to D
On Wednesday, 24 April 2024 at 20:13:26 UTC, Lance Bachmeier wrote: I haven't used Python much in recent years, but my recollection is that Python 2 had an ast module that would spit out the ast for you. Thanks for the pointer! So I ran one of my modules through and generated an AST, and get results similar to: ``` Module( body=[ Import( names=[ alias(name='sys')]), FunctionDef( name='pout', args=arguments( posonlyargs=[], args=[ arg(arg='item')], kwonlyargs=[], kw_defaults=[], defaults=[]), body=[ ``` etc. I presume I'll now need to write something that parses this into D source (maybe with the assistance of a module provided above). Before I do that, is this syntax general enough that a Python-AST to D source converter may already exist? Obvious searches in google and the D package index didn't turn up anything. I have no background at all in working with ASTs, in fact my formal education is not even in CS, so I'm way outside my wheelhouse at this point.
Recommendations on porting Python to D
Hi D I have a somewhat extensive CGI based web service written in Python and I'd like to port it to D. I can do this manually of course, and maybe that's the best way, but for a rough start, is anyone aware of any tools that generate an abstract syntax tree which could then be converted to somewhat equivalent D code? This might give me a jump-start on the manual conversion process. Then later I can work on removing the CGI dependency. I'm aware that this wouldn't work in general due to all the third party modules typically used in python, but most of this code is self contained Python2 and doesn't depend on many imports. I can just call my old C code from D, but the old Python is another story. Thanks for any advice you may have,
Re: Best way to use large C library in D as of 2024
On Monday, 1 April 2024 at 02:08:20 UTC, Lance Bachmeier wrote: On Saturday, 30 March 2024 at 05:01:32 UTC, harakim wrote: It works well if you only need to work with a header. There are still a few rough edges that get in the way if you're compiling the full C sources (I filed bugs for all of them): - Can't handle va_arg - Can't cast to a pointer of a struct that's typedef'd - Can't use complex numbers with the ternary operator ... Once these final odds and ends are working, we have a killer language feature. Even though DMD can't compile some C code, that's pretty much a non-issue for me anyway. In my environment the servers are all Linux so "apt-get" (or equivalent) typically provides a pre-compiled dependency. Being able to list a package as a system dependency and then just call it from D with no interface code is a Big Freaking Deal! Compared to Python interfaces this is a huge improvement. It makes D an even better replacement for the mixed mode python + C development I was doing before switching to D for new projects.
Re: Best way to use large C library in D as of 2024
On Saturday, 30 March 2024 at 07:11:49 UTC, Mike Parker wrote: Though I appreciate the sentiment, it's much more effective and efficient for people actually using the feature, and who appreciate it, to write up a blog post about it somewhere and share that on Twitter/Reddit/HN, etc. I would, but I'm just not a social media person. I pretty much only post online content here and at github.com. You may have the problem that D doesn't attract "very-online" personality types. I do mention D in work presentations, but those are not visible to the public.
Re: Best way to use large C library in D as of 2024
On Tuesday, 26 March 2024 at 20:19:27 UTC, bachmeier wrote: Should be able to just use it, as described here: https://forum.dlang.org/post/qxctappnigkwvaqak...@forum.dlang.org Create a .c file that includes the header files and then call the functions you need. Wow. **That just worked the first time!** Holy &^@$ that's easy! So why does the 2nd page returned from the google search ``` interfacing with C site:dlang.org ``` (which happens to be: https://dlang.org/spec/interfaceToC.html) still have this text: Since D can call C code directly, it can also call any C library functions, giving D access to the smorgasbord of existing C libraries. To do so, however, one needs to write a D interface (.di) file, which is a translation of the C .h header file for the C library into D. For popular C libraries, the first place to look for the corresponding D interface file is the Deimos Project. If it isn't there already, please write and contribute one to the Deimos Project. ? This lead me to believe that interfacing was a chore and I was considering going back to C for a small program I need.
Best way to use large C library in D as of 2024
Hi D I have a C library I use for work, it's maintained by an external organization that puts it through a very through test framework. Though source code is supplied, the intended use is to include the header files and link against pre-compiled code. What is the best way, as of 2024 to use this library with D, in particular dmd? ImportC doesn't seem like the go-to option since the C source does not need to be complied. I've seen some forum post indicating that manually generated D wrappers are no longer needed, but I'm fuzzy on the particulars. If there's any blog posts or other instructions you'd like to reference I'd be happy to check them out. For reference here's the C library in question: https://naif.jpl.nasa.gov/naif/toolkit_C_PC_Linux_GCC_64bit.html Thanks for any guidance you can provide,
SumType structure wrapping seems to fail (dmd v2.105.2)
Hi D
As suggested in other threads I've tried wrapping a SumType in a
structure to add functionality and used `alias ... this` to make
assignment, etc. easier. However the following code fails in dmd
2.105.2.
```d
import std.sumtype;
struct Item{
SumType!(void*, byte[3], ubyte[3], string[3]) value;
alias value this;
this(T)(T thing){ value = thing;}
}
void main(){
Item item;
byte[3] byte_vec = [0x01, 0x02, 0x03];
item.value = byte_vec; // <-- works
item = byte_vec; // <-- fails to compile
}
```
Is there some important detail I'm missing? The compiler error
message is:
```
Error: generated function `test_sumtype3.Item.opAssign(Item p)`
is not
callable using argument types `(byte[3])`
cannot pass argument `stuff` of type `byte[3]` to parameter
`Item p`
```
Thanks for any suggestions,
Re: How to print current type of a SumType?
We posted at the same time! Thanks for help nonetheless. I do hope over time, SumTypes get the Ali Çehreli treatment. I keep his book open on my desk all the time.
Re: How to print current type of a SumType?
On Sunday, 1 October 2023 at 01:17:50 UTC, Chris Piker wrote:
```d
alias Vec3 = SumType!(void* /* invalid vector */, byte[3],
short[3], char[][3]);
```
I know it's bad form to reply to my own question, but I think I
found a reasonably simple way:
```d
string prnType(Vec3 vec){
return vec.match!( t => typeof(t).stringof);
}
```
which is hardly worth making into a function of it's own. Other
suggestions are welcome of course.
SumTypes are really tripping me up. Unlearning years of writing
C takes more time then I'd prefer.
How to print current type of a SumType?
Hi D
I've a simple question but it's bedeviling me anyway. How do I
get a string representation of the current type of a SumType?
I'm trying to avoid something like this:
```d
alias Vec3 = SumType!(void* /* invalid vector */, byte[3],
short[3], char[][3]);
string prnType(Vec3 vec){
return vec.match!(
(void* _) => "void*",
(byte[3] _) => "byte[3]",
(short[3] _) => "short[3]",
(char[][3] _) => "char[][3]"
);
}
```
I'm sure there's a much easier way, especially once there's a
largish number of vector types.
Thanks,
Re: Recommendation on plotting library
Thanks for the both of the long replies. I've ready them twice and will do so again. To focus in on one aspect of D package support: On Saturday, 22 July 2023 at 02:24:08 UTC, Greggor wrote: In general whenever possible I think its better for everyone that stuff is built from source. It ensures that builds can be re-produced by anyone and that issues with building are caught fast by anyone consuming the library. I agree 100%. I've run into similar python package hell dealing with Tensor flow on Rocky Linux 8, since it wasn't Ubuntu. The problem is we're small, and many dependencies are already written in C, and dub is not setup to handle C dependencies. If I could turn my legacy C projects (pre-processor macros and all) into dub projects I'd do so as fast as possible. If dub supported two things: 1. Building C projects 2. Installing the resulting binaries in "site-packages" or similar. many, many project kickstart issues would be solved. Like C++, D is designed as a successor to C. It would be nice to bring C into the party with full dub support.
Re: Recommendation on plotting library
On Friday, 21 July 2023 at 17:40:25 UTC, Greggor wrote: Up to date versions of Windows 10 should have curl included and dub can run commands before building, so you could try downloading a prebuilt lib for windows via curl. https://everything.curl.dev/get/windows Hey, nice! This might be a winner. Unless I'm missing something libcairo-2.dll, libpng13.dll and zlib1.dll could be downloaded, and then ggplotd plot can build against those (probably?). The key thing that dub is missing compared to pip is some concept of an "install" directory. Compared to everything else dub handles, not adding this capability mostly comes down to self imposed limitations. I wonder how many in the community would like to see those limitations removed? I presume it's just as easy to add commands that run after the build as before. I could have an alternate dub configuration that's short for "setup_build_test_install" that runs everything. If multiple projects start defining an install target the same way, then you have a convention, which can eventually lead to a standard.
Re: Recommendation on plotting library
On Friday, 21 July 2023 at 06:15:10 UTC, Jonathan M Davis wrote: On Thursday, July 20, 2023 10:57:22 PM MDT Chris Piker via Digitalmars-d-learn wrote: Regardless though, dub really isn't designed with packaging anything in mind. Rather, it's designed to build your code as well as pull in D libraries that it usees and build those too. Anyone looking to actually package stuff would create a package from what was built with dub (e.g. with deb, rpm, flatpacks, etc.). So as far as I can tell, python pip originally only dealt with python code, but eventually wheels were added for binary support. Just as a wild guess, do you see dub ever evolving in that direction? All the reasons for not supporting pre-compiled binaries in pip apply to dub, but yet support was added anyway, and it's been wildly successful. I know it's hard to make predictions (especially about the future), but I'd be interesting in your opinion on the matter.
Re: Recommendation on plotting library
On Friday, 21 July 2023 at 02:40:10 UTC, harakim wrote: On Thursday, 20 July 2023 at 02:37:54 UTC, Chris Piker wrote: If you happen upon a basic charting library for D during this hunt, please let me know! Last year, I rolled my own and it got the job done, but I wasn't concerned about how they looked. :D Well, just for fun, if you have a plotting library on github or some other public place I wouldn't mind taking a look. Given the state of things, it doesn't have to be awesome to be a good seed point. As for plotting, I imagine it's c bindings only because there's no need to re-invent the wheel. (Warning, possible ill-informed opinions ahead...) In a way there is a need to reinvent the wheel. With python I can run `pip install matplotlib` and get whatever binaries I need to get the job done. D runs on dub, which I only see handling source code, and as far as I can tell, only D source code at that. So unless it's D code, it can't be packaged and delivered easily within the D ecosystem. If dub supports either pre-built binaries, or C code (such as libcairo2), I'd be interested in seeing how that's done. With the wizardry I've see around here, it's probably easy, I just don't know about it. Going wy out on a limb for a minute, I think D shines as a scripting language replacement. Most of my programs are single file projects these days with dub set as the interpreter. Also Rust seems to be crowding the system level space and so focusing on it's "compiled scripts" capability avoids that competition. (If any of the statements above are faulty, I invite correction.)
Re: Pre-import version statements
On Thursday, 20 July 2023 at 06:44:30 UTC, Jonathan M Davis wrote: D has nothing equivalent to that. You compile your code with whichever version of dmd (or ldc, gdc, etc.) that you want, and it either compiles or it doesn't. Thanks :) As I developer that doesn't bother me too much, though I can imagine that management would be concerned. For larger projects it's appreciated when I can point to a particular fixed standard that I'm following. In addition to the `#define` statement above, my libraries were also compiled with `gcc -std=c99`. I'll move over to a compiler specific area and broach the topic.
Pre-import version statements
Hi D In my C code I used to typically put the line: ``` #define _POSIX_C_SOURCE 200112L ``` in the source before importing any standard library headers. Is there something equivalent for phobos? Say `version(phobos2.100)` or similar? I don't particularly need this functionality, just checking to see if there as a formal way to denote what library standard the application code is expect to work against. Thanks,
Re: Recommendation on plotting library
On Thursday, 20 July 2023 at 03:58:05 UTC, Andrew wrote: If you're already using python, it's probably best to keep using that. Oh of course. Examples *have* to be provided in python, since that's the default language of science these days. But extra examples don't hurt, and it would be nice to see D along side other options. I find these days that my D "scripts" are about the same size as my python scripts, which is really surprising for a compiled language. I just tried ggplotd and it was easy to make it work on Linux, only one external apt command needed, but on Windows, even that is a deal breaker. Package management on Windows seems to be wild-west/nonexistent. Side musing... Since I've seen many people use anaconda on Windows, I wonder how hard it would be to make a conda package that provided dmd+dub?
Recommendation on plotting library
Hi D One of my jobs is to release and maintain public data archives from long-running scientific instruments. In order to help people understand how to process the data, sample code is often included with the archive. Recently this has been in the form of short programs that generate a plot of a representative time period. Python works well for this task as matplotlib support is pretty much universal. Matlab is also a reasonable choice for many. Were I to also provide sample code in D, what library would you recommend for scientific plotting, especially for dynamic power spectral densities generated from time-series data (basically heatmaps)? Since dub can pull down dependencies easily enough and since it supports single file projects it seems that D would be well suited to this task, but I've never plotted any data directly from a D program. Many GUI libraries I see for D are wrappers around external C or C++ based libraries that would need to be installed separately. I'd like to avoid such complications if that's possible. Thanks for any advice,
Re: Proper way to handle "alias this" deprecation for classes
On Wednesday, 10 May 2023 at 20:25:48 UTC, H. S. Teoh wrote: On Wed, May 10, 2023 at 07:56:10PM +, Chris Piker via Digitalmars-d-learn wrote: [...] I also suffer from left/right confusion, and always have to pause to think about which is the right(!) word before uttering it. Oh, I though was the only one with that difficulty. Glad to hear I'm not alone. :-) I have a tendency to think of things by their purpose when programming but not by their location on the line or page. So terms such as "writable" versus "ephemeral" or "addressable" versus "temporary" (or "register"), make so much more sense to me. Back on the ref issue for a moment... I'd imagine that asking the compiler to delay creating a writable variable until it finds out that a storage location is actually needed by subsequent statements, is a tall order. So D chose to introduce programmers to lvalues and rvalues head-on, instead of creating a leaky abstraction.
Re: Proper way to handle "alias this" deprecation for classes
On Wednesday, 10 May 2023 at 16:01:40 UTC, H. S. Teoh wrote: x = y; ^ ^ | | lvalue rvalue ... // This is OK: x = y + 1; // This is not OK: (y + 1) = x; Thanks for the clear explanation. My problem with the terms lvalue and rvalue is much more basic, and is just a personal one that only affects probably 0.1% of people. I just can't keep left vs. right straight in real life. "Right" in my head always means "correct". My daughter hates it when I'm telling her which way to turn the car since I've said the wrong direction so many times. :)
Re: Proper way to handle "alias this" deprecation for classes
On Wednesday, 10 May 2023 at 14:42:50 UTC, Inkrementator wrote: On Sunday, 7 May 2023 at 21:12:22 UTC, Chris Piker wrote: https://gist.github.com/run-dlang/9b7aec72710b1108fc8277789776962a Thanks for posting that. Reading over the code I'm reminded that I never cared whether something was an rvalue or lvalue before writing D code. It's off topic, but I forget why managing memory for rvalues* was pushed onto the programmer and not handled by the compiler. I'm sure there is a good reason but it does seem like a symmetry breaking requirement. -- *or was it lvalues, I can never keep the two separate. Wish the some other terminology was adopted long ago, such as "named" vs. "ephemeral".
Re: Proper way to handle "alias this" deprecation for classes
On Sunday, 7 May 2023 at 21:04:05 UTC, Inkrementator wrote: On Sunday, 7 May 2023 at 18:19:04 UTC, Ali Çehreli wrote: alias this is for implicit type conversions, which can be achieved explicitly as well. Open question to everybody: What you're opinion on using opCast for this? Since it's a type conversion, it seems fitting to me. And another suggestion: Wrap the class in a struct that has visibility on the class members via the "package" access specifier and continue using "alias this". Hi Inkrementator In this case, ResponseContainer is already a wrapper structure, so on the surface, putting a wrapper on a wrapper feels like over-engineering. On the other hand, your first suggestion of using opCast() does seem like a reasonable choice to me. Can you provide a short code snippet using opCast to achieve the same result?
Re: Proper way to handle "alias this" deprecation for classes
On Sunday, 7 May 2023 at 18:19:04 UTC, Ali Çehreli wrote:
auto main() {
auto c = new C();
// The same type conversion is now explicit:
foo(c.asIntPtr);
}
Hi Ali
Ah, very clear explanation, thanks! So basically to fix the
problem I just delete the alias this line from dpq2, see what
unit tests and app code it breaks, then fix each of those.
Actually seems straightforward, for a limited code base anyway.
On a side note, with all the free help you've provided, it's
about time I gave back. Since I've no free time or expertise to
offer, I picked up a hardcover copy of "Programming in D" for a
bright young programming student I know in appreciation.
Cheers,
Proper way to handle "alias this" deprecation for classes
Hi D
One of the dependencies for my project has a class that makes use
of the `alias x this` construct. According to dmd 2.103, alias
this is deprecated for classes, so I'd like to correct the
problem.
Is there a specific paragraph or two that I can read to find out
what is the appropriate replacement construct? On a related
note, has anyone created a code maintenance guide to help
work-a-day programmers navigate recent changes to the D language?
For reference, here's a the code in question (from dpq2,
result.d):
```d
package immutable final class ResultContainer
{
version(Dpq2_Dynamic)
{
import dpq2.dynloader: ReferenceCounter;
private ReferenceCounter dynLoaderRefCnt;
}
// ResultContainer allows only one copy of PGresult* due to
avoid
// double free. For the same reason this class is declared as
final.
private PGresult* result;
alias result this; //< Deprecation Warning Here
package this(immutable PGresult* r)
{
assert(r);
result = r;
version(Dpq2_Dynamic) dynLoaderRefCnt =
ReferenceCounter(true);
}
...
```
Thanks for any pointers,
Re: Use dub to create source lib and executables
On Saturday, 4 March 2023 at 21:31:09 UTC, Richard (Rikki) Andrew Cattermole wrote: Yes dub was setup to cover the most common cases, but ignores when you have multiple outputs. Not ideal. There is a PR to add build steps currently, which will help improve things, so there is work to make dub better at these less common use cases. The simplest solution is to do one package per binary, then use a shell script to trigger them in succession, not great but a workable solution. It's interesting to hear that multiple outputs are rare for other developers. So I was hoping to do away with shell scripts (or make) since dub is common on all systems, but alas it's not worth the effort, so I won't continue to try and make `"targetType":"none"` work. By the way, do you know if there will be a dub "build-all" type command in the future?
Re: Use dub to create source lib and executables
On Saturday, 4 March 2023 at 20:23:29 UTC, Steven Schveighoffer wrote: On 3/4/23 1:33 PM, Chris Piker wrote: If you mean that you have multiple subprojects inside your main dub project, my advice is to follow what other such projects do. I always look at vibe for my example. I have been trying to do that for hours, and all I get is cryptic error messages from dub. I've tried all in one top level dub.json, I've tried three dub.json files, one at the top level and one for each of lib and utilities. The fact that dub considers the output of a project to only be a single lib, exc, etc. is quite restrictive and it means we have to have a target of "none" and then subprojects, which is cumbersome at best. Maybe it's just the frustration talking, but it seems wrong that a build rule that is so trivial to construct using 1980s makefiles is so hard today. If I wasn't trying to blend in I would have just given up a long time ago. But thanks for mentioning vibe.d as an example I'll take a look.
Re: Using Windbg to debug D applications and unittests
On Monday, 27 February 2023 at 12:09:50 UTC, Basile B. wrote: At least this is what is done for the Dexed GDB widget, so that gdb breaks automatically when an Error or an Exception is new'd (https://gitlab.com/basile.b/dexed/-/blob/master/src/u_gdb.pas#L2072). Glad you mentioned Dexed. I Had been meaning to try it out but forgot about it. I just downloaded the deb and will give it a go. Thanks!
Use dub to create source lib and executables
Hi D I normally work in a *nix environment, typically on server-side code. For many projects I have gnu makefiles that build a small lib along with command line utilities. Up to now I've been creating a dub.json file for just the sourceLibrary, and then putting embedded dub comments at the top of each of the utility programs. The utilities are built via `dub build --single` and a thin makefile triggers all the dub commands. This method is inefficient, and it won't work in a typical Windows dev environment, so today I'm trying to remove gnu make from the picture. Going back to basics, does anyone know of a clear example of how to convert this simplified makefile into 1-N dub files?: ```makefile # Build library + utility programs. # Lib sources are in "libsrc", 1-file programs in "utils" # Implicit rule to make a utility program outdir/%:utils/%.d outdir/mylib.a dmd -I libsrc -od=outdir -of=$@ $^ # Top level build rule for everything build:outdir/mylib.a outdir/prog1 outdir/prog2 | outdir # Explicit rule to make output directory if not present outdir: @if [ ! -e outdir ]; then mkdir outdir; fi # Explicit build rule for the library outdir/mylib.a:libsrc/mod1.d libsrc/mod2.d dmd -lib -od=outdir -of=$@ $^ ``` This is a paired down example since `make test` and `make install` aren't present, but I hope the main ideas are apparent. I've been at it now for about four hours, with lots of web-searches (btw, thanks to schveiguy for getting me this far) but I haven't figured out what hierarchy of dub constructs I need to create to order to get the effect of the small make file above. Thanks for considering the question,
Re: Deciding one member of iteration chain at runtime
On Friday, 17 February 2023 at 17:42:19 UTC, Ali Çehreli wrote: // Two different steps auto g1 = r.map!((int n) => n * n); auto g2 = r.map!((int n) => n * 10); // The rest of the algoritm auto result = choose(condition, g1, g2) .array; Now that's a handy construct. There's many little goodies in phobos I've yet to learn, thanks for the tip!
Re: Deciding one member of iteration chain at runtime
On Friday, 17 February 2023 at 17:44:20 UTC, H. S. Teoh wrote: Here's an actual function taken from my own code, that returns a different range type depending on a runtime condition, maybe this will help you? Thanks, this was helpful. I keep forgetting to expand my horizons on what can be returned from an auto function. I'm still unlearning C & Java. Cheers,
Deciding one member of iteration chain at runtime
Hi D I have a main "loop" for a data processing program that looks much as follows: ```d sourceRange .operatorA .operatorB .operatorC .operatorD .operatorE .operatorF .operatorG .operatorH .copy(destination); ``` Where all `operator` items above are InputRange structs that take an upstream range, and the pipeline really is 9 operations deep. In order to handle new functionality it turns out that operatorG needs to be of one of two different types at runtime. How would I do something like the following: ```d auto virtualG; // <-- probably invalid code, illustrating the idea if(runtime_condition) virtualG = operatorG1; else virtualG = operatorG2; sourceRange .operatorA .operatorB .operatorC .operatorD .operatorE .operatorF .virtualG .operatorH .copy(destination); ``` ? I've tried various usages of `range.InputRangeObject` but haven't been able to get the syntax right. Any suggestions on the best way to proceed? Maybe the whole chain should be wrapped in InputRangeObject classes, I don't know. Thanks,
Re: Is there such a JSON parser?
On Monday, 2 January 2023 at 14:56:27 UTC, SealabJaster wrote: Are you asking for a SAX-styled parser for JSON? I have an upcoming project (about 3-6 months away) that could make use of this as well. If you need someone to try it out please let me know and I'll give it a spin. Good luck on your library. Cheers,
Re: importC and cmake
On Wednesday, 28 September 2022 at 06:04:36 UTC, zjh wrote:
On Wednesday, 28 September 2022 at 05:29:41 UTC, Chris Piker
wrote:
`Xmake` is indeed simpler.
`Xmake` is really nice!
zjh
Sorry to go off topic for a moment, but do you happen to know how
to tell xmake that my project is C only, and thus it shouldn't
add the /TP flag to cl.exe? The obvious statement:
```lua
set_languages("c99")
```
doesn't accomplish that task.
Thanks for the help,
Re: importC and cmake
On Wednesday, 7 September 2022 at 00:31:53 UTC, zjh wrote: `xmake` is simpler. Thanks for the recommendation. Was struggling with cmake for a dependent clib. Xmake is indeed simpler.
Re: What are (were) the most difficult parts of D?
On Sunday, 22 May 2022 at 20:11:12 UTC, rikki cattermole wrote: On 23/05/2022 8:05 AM, Chris Piker wrote: Vibe.d is well tested against the frontend. Its part of dmd's test suite. See: https://buildkite.com/dlang/dmd/builds/26775 Thanks, that's handy. Do you know where the equivalent test suite is for gdc? No idea. I've pinged Iain, because you are certainly at that point that is well past my knowledge of GDC. Not something I use, as it isn't exactly easy to use on Windows, plus LDC has some pretty awesome features. Hey thanks! I bet LDC is pretty cool, have to look into it sometime. For now at my site just introducing D is a bit radical, don't want to capsize the boat by leaving the gcc toolchain altogether. I'm willing to help out with GDC work were I can, at least by being a bleeding-edge tester if nothing else.
Re: What are (were) the most difficult parts of D?
On Sunday, 22 May 2022 at 19:33:21 UTC, rikki cattermole wrote: I should probably jump back to another thread, but maybe one more reply isn't too much off topic discussion... DMD and LDC would have produced the same set of issues, because its the same frontend. Oh, the compile stage works okay (after building a custom gcc), its linking that's the problem now. ``` dub build --single my_prog.d # Works env LD_RUN_PATH=/usr/local/gcc12/lib64 DFLAGS="-Wno-deprecated" dub build --compiler=gdc-12 --single my_prog.d # Fails with link errors. ``` An example link error is: ``` /usr/bin/ld: libvibe_core.a: in function `_D3std6format8internal5write__T8getWidthTAwZQnFNaNbNiNfQoZl': args.d:(.text+0x657e): undefined reference to `_D3std9algorithm9searching__T3allSQBg6format8internal5write__T8getWidthTAwZQnFQgZ9__lambda2Z__TQCoTQBbZQCwMFNaNbNiNfQBsZb' ``` I've made sure libgphobos.so is in the LD_RUN_PATH above. Since there's no substitute for end-to-end testing, a CI that weekly grabbed packages off of code.dlang.org and tried them with gdc would be handy. Now that gdc-12 is out, it might be something to consider. I know that it's the package provider's job, but a automated friendly nudge wouldn't hurt. Vibe.d is well tested against the frontend. Its part of dmd's test suite. See: https://buildkite.com/dlang/dmd/builds/26775 Thanks, that's handy. Do you know where the equivalent test suite is for gdc?
Re: What are (were) the most difficult parts of D?
On Sunday, 22 May 2022 at 19:01:41 UTC, rikki cattermole wrote: On 23/05/2022 6:06 AM, Chris Piker wrote: Iain's workload should be decreasing now that it is using the up to date frontend. Rather than the older C++ version with backports that he has been maintaining. Hats off to Iain for sure. Wish I had the resources to pay for his work, would certainly do so. 2. Testing common packages against gdc Why? Mostly because I've put in about 15 hours effort so far trying to get a vibe.d based project to build using gdc, with no success. I'm about to give up and declare either gdc or vibe.d unsuitable for use in a large upcoming project. That's too bad really because D only has any credibility at all in my work group due to the existence of gdc, and vibe.d is a nice platform to build on. 3. Updating dub so that it could assist with creating deb and yum packages Packaging is something dub doesn't do right now, it is something that would be very nice to have. For this item, if there was just a "dub install --prefix=" type command it would be good enough. Let the packaging tools take over from there and scoop-up the output. 4. Having a standard GUI library (maybe). If you have the many tens of millions to build it, sure. Ah, if only. Let me check the couch cushions, should be a trust fund hiding in there somewhere ;)
Re: What are (were) the most difficult parts of D?
On Wednesday, 11 May 2022 at 05:41:35 UTC, Ali Çehreli wrote: What are you stuck at? What was the most difficult features to understand? etc. To make it more meaningful, what is your experience with other languages? Ali Hi Ali, thanks for asking. Coming from C background I had problems with voldemort types. I kept wanting to know the storage type for items so that I could use it in interfaces, but that's not as big a deal anymore. Currently my problem is integration into common tool chains in a Linux environment. On Linux the common development tools are built around gcc & gdb. The common package managers are apt and dnf. Right now I can't compile vibe.d using gdc, and I can't `apt install vibe-d` So I'd say lack of Linux integration is my most difficult problem using D. I'd certainly contribute to a fund that focused on D Linux usability, namely: 1. Supporting gdc development and distribution. 2. Testing common packages against gdc 3. Updating dub so that it could assist with creating deb and yum packages 4. Having a standard GUI library (maybe). These days the language itself is not so much of a concern. The development days for D2 are long over. What exists is excellent. I think the central focus should on libraries and integration with larger ecosystems.
Re: Write UTF-8 bytes directly to stack buffer
On Thursday, 10 March 2022 at 17:59:33 UTC, H. S. Teoh wrote:
Probably what you're looking for is std.format.formattedWrite.
For example:
```d
import std;
void main() {
ubyte[65536] buf;
char[] usable_buf = cast(char[]) buf[];
usable_buf.formattedWrite!"Blah %d blah %s"(123, "Это UTF-8
строка.");
auto used = buf.length - usable_buf.length;
writefln("%(%02X %)", buf[0 .. used]);
}
```
Hey thanks! That does work with recent versions of dmd+phobos,
but doesn't work in gdc-10. For some reason it produces this
error:
```d
error: static assert "Cannot put a const(char)[] into a char[]."
```
Is there a work around involving `.representation` as alluded to
in this
[thread](https://forum.dlang.org/post/zmehmpithifbgfuef...@forum.dlang.org) ?
To get around the issue I built gdc-11.2 from source code at the
GNU site but the old version of phobos is still included, so no
dice.
Re: gdc or ldc for faster programs?
On Tuesday, 25 January 2022 at 20:04:04 UTC, Adam D Ruppe wrote: Not surprising at all: gdc is excellent and underrated in the community. The performance metrics are just a bonus. Gdc is the main reason I can get my worksite to take D seriously since we're a traditional unix shop (solaris -> linux). The gcd crew are doing a *huge* service for the community.
Write UTF-8 bytes directly to stack buffer
Hi D
There are quite a few string, array and range functions in phobos
so I'm getting confused as to the right way to encode string data
as UTF-8 directly into a stack buffer while keeping track of the
write point.
I have some output packets I'm building up in a tight loop. For
speed I'm using the a priori knowledge that output packets will
never be larger then 64K. So what's the best way to do this:
```d
ubyte[65536] buf;
ubyte[] usable_buf = buf;
// part of some tight loop, how to create function writef_utf8 ?
foreach(input_thing; things){
usable_buf.writef_utf8!"format str"(input_thing.fieldA,
input_thing.fieldB);
}
size_t used = buf.length - usable_buf.length;
stdout.write(buf[0.. used]);
```
Re: Source code for vibe.d listenTCP()
On Sunday, 27 February 2022 at 01:45:35 UTC, Adam D Ruppe wrote: My dpldocs.info search engine is not great right now but it can sometimes help find these things: http://search.dpldocs.info/?q=listenTCP Hi Adam Your site has been super helpful given the state of the vibe.d docs. It only has one issue from my point of view. The source code listing pages, for example this one: https://vibe-core.dpldocs.info/source/vibe.core.stream.d.html#L288 don't expand horizontally to use the given space in the browser window. Since the vibe.d sources contain many long lines I have to scroll down, find the horizontal scroll bar at the bottom of the page, and then scroll back up and find the function I was looking at. Do you observe the same behavior in your preferred browser, or is this just a problem in Firefox 91? No matter what, thanks for the site. I've been using it a lot!
Re: Set output location for dub --single
On Monday, 28 February 2022 at 08:03:24 UTC, Basile B. wrote: That 's not exactly what you ask for but you can define the path in the embedded recipe (targetPath) ```d #!/usr/bin/env dub /+ dub.sdl: dependency "mypackage" version="*" path=".." targetPath "./bin" +/ ``` Hey thanks! Not perfect, but it'll probably work. I'll bring up the idea of overriding some settings (such as targetPath) on the command line in a similar manner to ssh in the dub [discussions](https://github.com/dlang/dub/discussions) area and see how well that goes over.
Set output location for dub --single
Hi D Coming from a python background it's worked well to organize my D projects as a dub `sourceLibrary` and then to put top level programs in a directory named `scripts` that are just dub single file projects. So the layout looks like this: ``` rootdir/ | +- mypackage/ || |+- libfile1.d |+- libfile2.d | +- scripts/ || |+- prog1.d |+- prog2.d | +- dub.json ``` In dub.json "scripts" are ignored via `"excludedSourceFiles":["scripts/*"]`. Each "script" includes `mypackage` via: ```d #!/usr/bin/env dub /+ dub.sdl: dependency "mypackage" version="*" path=".." dependency (other nonlocal packages) +/ ``` ...and all seems rather familiar to a python programmer and I avoid the complexities of dub sub-projects. For building the individual "scripts" as binaries, it be nice output the binaries to another directory, say `bin`. Is there an override for the dub command line that would specify the output location? Maybe something like: ```bash dub --single -o targetPath=./bin ./script/prog1.d ``` ? (Here the mythical argument `-o` overrides a single build configuration setting.) After reading over the dub documentation I don't see a general way to override project options via the command line, but maybe it's there and I couldn't understand what the dub docs were trying to say.
Simple way to handle rvalues and templates.
Hi D I have bit of code that was tripping me up. I need to parse small fields out of a big binary read, and it looks like some operations just can't be composed when it comes to using templates. So this works: ```d import std.bitmanip, std.system; ubyte[8192] data; ubyte[] temp = data[4..6]; ushort us = temp.read!(ushort, Endian.bigEndian); // intentionally provided the default byte order for readability ``` But this doesn't work: ```d import std.bitmanip, std.system; ubyte[8192] data; ushort us = data[4..6].read!(ushort, Endian.bigEndian); ``` The reasons for this are probably old hat for seasoned D programmers by this is really confusing for newbies. Is there a better way to handle this instead of making a bunch of temporary variables that I don't care about? Matlab has this behavior too, statements that should be composable aren't, and it drives me crazy since Java and Python don't seem to suffer from this problem near a much.
Re: Source code for vibe.d listenTCP()
On Sunday, 27 February 2022 at 01:45:35 UTC, Adam D Ruppe wrote: and my website also offers a "see implementation" link at the bottom which has some inline code navigation jump links too to help. Yes! Freaking awesome! This needs to be a link on the regular D pages, or at least in the package index pages. (Maybe it is already?)
Re: Source code for vibe.d listenTCP()
On Saturday, 26 February 2022 at 22:25:46 UTC, Chris Piker wrote: Anyway if someone can just help me find the source code to listenTCP inside vibe.d I'd be grateful. Sorry to reply to myself, but I found the function. It was separated out into a separate package: https://github.com/vibe-d/vibe-core on which vibe-d now depends. D has so many code obscuring features that I didn't think to check for dependencies first.
Source code for vibe.d listenTCP()
Hi D I'm trying out the vibe.d framework for the first time and it looks like many of the functions mutate some hidden global state. Take for example `listenTCP`. To help me build a mental picuture of the framework I'd like to see what global state is mutated, but for the life of me I can't even find the source code to listenTCP(). The obvious method of: ```bash git clone g...@github.com:vibe-d/vibe.d.git cd vibe.d grep -R -n listenTCP ``` returns many instances where listenTCP is used, but none that look like a definition. It's quite possible I just overlooked it, or maybe it's implemented as a mixin or something weird like that. Anyway if someone can just help me find the source code to listenTCP inside vibe.d I'd be grateful. Thanks for your time,
Re: Tips on TCP socket to postgresql middleware
On Monday, 21 February 2022 at 07:00:52 UTC, eugene wrote: On Monday, 21 February 2022 at 04:46:53 UTC, Chris Piker wrote: On Sunday, 20 February 2022 at 18:00:26 UTC, eugene wrote: I'm adverse to reading it closely since there was no license file and don't want to accidentally violate copyright. :) I think WTFPL will do :) If you want to add this file (or similar) to your sources, http://www.wtfpl.net/txt/copying/, but with your name in the copyright line I'd be happy to put the D version through it's paces and credit you for the basic ideas. I would not have thought of this formalism on my own (at least not right away) and want to give credit where it's due.
Re: Tips on TCP socket to postgresql middleware
On Sunday, 20 February 2022 at 18:36:21 UTC, eugene wrote: I often use two connections, one for perform main task (upload some data and alike) and the second for getting notifications from PG, 'cause it very incovinient to do both in a single connection. Ah, a very handy tip. It would be convoluted to multiplex notifications on the data connection.
Re: Tips on TCP socket to postgresql middleware
On Sunday, 20 February 2022 at 18:00:26 UTC, eugene wrote: Yes, here is my engine with example (echo client/server pair): - [In D (for Linux & FreeBSD)](http://zed.karelia.ru/0/e/edsm-2022-02-20.tar.gz) The code is terse and clean, thanks for sharing :) I'm adverse to reading it closely since there was no license file and don't want to accidentally violate copyright. I noticed there were no dub files in the package. Not surprised. Dub is such a restrictive tool compared to say, setup.py/.cfg in python.
Re: Tips on TCP socket to postgresql middleware
On Sunday, 20 February 2022 at 17:58:41 UTC, Ali Çehreli wrote: Another one is to set the message box sizes to throttle. Message sizes and rates are relatively well know so it will be easy to pick a throttle point that's unlikely to backup the source yet provide for some quick DB maintenance in the middle of a testing session. In case you haven't seen yet, the recipe for std.concurrency that works for me is summarized here: https://www.youtube.com/watch?v=dRORNQIB2wA=1735s Thanks! I like your simple exception wrapping pattern, will use that.
Re: Tips on TCP socket to postgresql middleware
On Sunday, 20 February 2022 at 15:20:17 UTC, eugene wrote: Most people will probably say this is crazy, but as to PG, one can do without libraries. I am doing so during years (in C, not D) and did not expierienced extremely complex troubles. I mean I do not use libpq - instead I implement some subset of the protocol, which is needed for particular program. Very interesting. I need to stand-up this program and two others in one week, so it looks like dpq2 and message passing is the good short term solution to reduce implementation effort. But I would like to return to your idea in a couple months so that I can try a fiber based implementation instead. Usually I design more or less complex (network) programs using event-driven paradigm (reactor pattern) plus state machines. In other words programs designed this way are, so to say, hierarchical team of state machines, interacting with each other as well as with outer world (signals, timers, events from sockets etc) It sounds like you might have a rigorous way of defining and keeping track of your state machines. I could probably learn quite a bit from reading your source code, or the source for similarly implemented programs. Are there examples you would recommend?
Tips on TCP socket to postgresql middleware
Hi D I'm about to start a small program to whose job is: 1. Connect to a server over a TCP socket 2. Read a packetized real-time data stream 3. Update/insert to a postgresql database as data arrive. In general it should buffer data in RAM to avoid exerting back pressure on the input socket and to allow for dropped connections to the PG database. Since the data stream is at most 1.5 megabits/sec (not bytes) I can buffer for quite some time before running out of space. So far, I've never written a multi-threaded D program where one thread writes a FIFO and the other reads it so I'm reading the last few chapters of Ali Cehreli's book as background. On top of that preparation, I'm looking for: * general tips on which libraries to examine * gotchas you've run into in your multi-threaded (or just concurrent) programs, * other resources to consult etc. Thanks for any advice you want to share. Best,
Re: UFC creating name conflict
On Saturday, 9 October 2021 at 21:37:27 UTC, Paul Backus wrote: On Saturday, 9 October 2021 at 21:26:52 UTC, Chris Piker wrote: A struct member always takes priority over a UFCS function, so there must be something else going on that you've left out of your explanation. Can you post a complete example that we can use to reproduce your issue? Ah. That's good to know, and comforting. Turns out this was an embarrassingly hasty post on my part. I was going by memory on the struct names and was just plain wrong. Struct member was `.second` not `.seconds`. The fact that I didn't get a "no member named seconds..." style error confused me for a bit. Problem solved. Thanks for the quick response,
UFC creating name conflict
Hi D I have and old C structure that I have to wrap that has a member named '.seconds', and in the module that handles this I also have conversion functions to go from an internal time representation to struct SysTime values. Unfortunately importing `core.time` brings in a seconds function, which due to UFC is confused with a structure member of the same name. How can I explicitly tell the compiler that I'm referring to: ```d thing.seconds # The structure member ``` and not ``` seconds(thing) # the function ``` ? Currently my code fails to compile due do this ambiguity. Thanks for any advice you can give. My google searches with the qualifier `site:dlang.org` weren't turning up any hits.
Re: Using D "rocket" logo in outside presentation
On Wednesday, 29 September 2021 at 05:44:59 UTC, Mike Parker wrote: On Wednesday, 29 September 2021 at 04:24:13 UTC, Chris Piker wrote: I'm to give a presentation to a combined NASA/ESA group in a few hours and would like to include a copy of the D "rocket" logo when mentioning new server side tools that I've written in D. Is such use of this particular [D logo](https://i0.wp.com/dlang.org/blog/wp-content/uploads/2021/08/logo_256.png?w=750=1) permissible? Yes! I ran out of time to convert the image to png format* and size it for the slide, but I really appreciate everyone's freely given time in answering my new-user questions as well as efforts to improve compiler error messages for lambda functions. The bottom right corner of the last slide in [this presentation](https://issues.cosmos.esa.int/socciwiki/display/IHDEA/IHDEA+-+2021?preview=/86739043/98635262/2021_IHDEA_das2_update.pdf) contains an acknowledgement of your freely given time. (Use the scroll wheel, not the arrow icons, to move to the last slide) Over the next couple years I hope to use D to help me make the most advanced space-physics data server in the world, though that's not saying much. Technology used in academia trails commercial equivalents due to drastically different funding scales. However, D does help me close the gap. Best, *and to correct some grammatical errors
Using D "rocket" logo in outside presentation
Hi D I'm to give a presentation to a combined NASA/ESA group in a few hours and would like to include a copy of the D "rocket" logo when mentioning new server side tools that I've written in D. Is such use of this particular [D logo](https://i0.wp.com/dlang.org/blog/wp-content/uploads/2021/08/logo_256.png?w=750=1) permissible? Thanks,
Recommendations on parsing XML via an InputRange
Hi D I just finished a ~1K line project using `dxml` as the XML reader for my data streams. It works well in my test examples using memory mapped files, but like an impulse shopper I didn't notice that dxml requires `ForwardRange` objects. That's unfortunate, because my next enhancement was to start parsing streams as they come in from stdin. (doh!) So I've learned my lesson and will RTFM closer next time, but now I'm casting about for a solution. Two ideas, either: 1. Find a different StAX-ish parser that works with `InputRange` (and buffers internally a bit if needed), or 2. Find a way to represent standard input as a ForwardRange without saving the whole stream in memory. (iopipe?) Dxml is very nice, as I have small sections of the stream that I parse into a DOM, but the majority of the items are handled and discarded element by element. Any recommendations?
Re: Is std.variant useful for types only known at run time?
On Wednesday, 8 September 2021 at 08:39:53 UTC, jfondren wrote:
so I'd look at a std.sumtype of them first:
Wow, this forum is like a CS department with infinite office
hours!
Interesting. I presume that the big win for using std.sumtype
over a class set is value semantics instead of reference
semantics?
So out of curiosity, say each structure implemented a function to
provide the desired broken-down-time, would the following
"virtual function" style call work?
```d
import std.sumtype;
struct BDTime { int y, int m, int d, int h, int m, double s };
struct ISO8601 { BDTime bdTime(){ ... } }
struct FloatEpoch { BDTime bdTime(){ ... } }
struct DoubleEpoch { BDTime bdTime(){ ... } }
struct LongEpoch { BDTime bdTime(){ ... } }
alias Time = SumType!(ISO8601, FloatEpoch, DoubleEpoch,
LongEpoch);
void main() {
import std.stdio : writeln;
import std.format : format;
Time e = ISO8601();
BDTime = e.bdTime();
}
```
or would I need to use `match!` to get the right structure type
and then generate the internal time representation?
Is std.variant useful for types only known at run time?
Hi D I'm working on data streaming reading module where the encoding of each input array isn't known until runtime. For example date-time column values may be encoded as: * An ISO-8601 UTC time string (aka char[]) * A ASCII floating point value with an indicated unit size and epoch (aka char[]) * A IEEE double with an indicated endianness, unit size, and epoch. (aka double[]) * A 64-bit signed in with an indicated endianness, unit size, and epoch. (aka long[]) My job when encountering a date-time array in the stream is to just to properly convert the info into a broken down time structure, regardless of the encoding. Initially I've been reading chunks of the stream into a `ubyte[]` and then using `cast` and `to` as needed, but then I stumbled across std.variant which can hold any type. I'm wondering if std.variant is useful in cases where type information is only known at run-time, since many of the flexible data structures I've run across so far in D require compile-time information. Thanks,
Re: Looking to get typeof parseXML return value
On Tuesday, 7 September 2021 at 04:40:25 UTC, jfondren wrote:
typeof(parseXML!simpleXML("")) xml;
Hey, I like this trick!
I was wondering what to use for the const(char)[] variable in the
typeof statement. It's blindingly obvious in retrospect.
Wouldn't work so well if there wasn't a literal for the input
data range type.
Sorry, didn't see your response before I posted mine, but thanks
for the tip anyway :)
Re: Looking to get typeof parseXML return value
On Tuesday, 7 September 2021 at 04:13:08 UTC, Chris Piker wrote: Any ideas on how to get the return type of `parseXML` below: ``` import dxml.parser; const(char)[] _mmfile; //_mmfile initialization TYPE??? _entityRng = parseXML!(simpleXML)(_mmfile); ``` Though it's ususally bad form to respond to one's own question. I hope to avoid wasting your time on this question. Just reading the source instead of trying to pull some typeof wizardry gives the answer: ``` EntityRange!(simpleXML, const(char)[]) _rEntity; _rEntity = parseXML!(simpleXML)(_data); ``` Sorry for the forum noise, carry on.
Looking to get typeof parseXML return value
Hi D I'm using the **dxml** library since I like it's "pull here for more data" mentality. I've come across the need to save an entity range created by the `parseXML` function as a class member so that I can tuck it away and pull more data as needed. Like almost all new users to D I'm tripping over how to save and pass around variables since nothing has an understandable type anymore and you can't use "auto" for *class member* storage types. Any ideas on how to get the return type of `parseXML` below: ``` import dxml.parser; const(char)[] _mmfile; //_mmfile initialization TYPE??? _entityRng = parseXML!(simpleXML)(_mmfile); ``` *before* calling parseXML, so that it can be a class member variable? I've tried variations on `typeof` and `.inputRangeObject` etc. with no success so far. Thanks for any advice :) All this would be so much easier if dxml just defined `Entity` at the top level of the parser module instead of burying it inside a templated struct. Then the type could just be `InputRange!Entity` which is easy to work with.
Re: wanting to try a GUI toolkit: needing some advice on which one to choose
On Thursday, 27 May 2021 at 07:00:32 UTC, Imperatorn wrote: I would like to recommend DlangUI [1], but we have tried now for months to get in contact with the owner of it (to take over development) and are getting no reponse. 1. https://github.com/buggins/dlangui Of the 107 forks of dlangui last seen on github, do you know if any contain more recent changes/features than the buggins original repo? If there's a quick way to check without paging through all of them I'm curious to know how it would be done. I'm using D for efficient server-side processes only, since at work we're heavily invested in Java for our end-user GUIs, but I'm curious about the state of D GUI toolkits nonetheless.
State of D for webassembly
Hi D Our group has some spectragram (aka dynamic spectra) creation algorithms that are fast in Java and since D has many Java-ish concepts it looks like it would be do-able to port the code to D. If I take on this project my target would be to run as a webassembly program. What is the general state of support for GC dependent D-code running as webassembly? If the runtime is not ready that's okay, just wanted to inquire about the state of things. Thanks,
Re: Recommendations on avoiding range pipeline type hell
On Monday, 17 May 2021 at 00:27:01 UTC, SealabJaster wrote: On Sunday, 16 May 2021 at 23:52:06 UTC, Adam D. Ruppe wrote: ... I've opened a PR (https://github.com/dlang/dmd/pull/12526) with a super hacked together proof-of-concept. As I say in the PR I don't know if I'm actually capable of pushing it forward if the idea gets accepted, but I've decided to at least try to see if this kind of thing is even palatable to the compiler devs. Wow! That's good news. Thanks! Here's hoping that future versions of dmd do a bit more whitespace formatting of error messages. The current undifferentiated text walls are an early low quality user experience that can drive people away before they are invested in the language. --- A final note on the initial problem that started this thread: My overall my program works now. (None too soon, management meeting is tomorrow) To solve my previous Type Hell(tm) problems I've found that `std.range.interfaces` is my new best friend.
Re: Recommendations on avoiding range pipeline type hell
On Sunday, 16 May 2021 at 13:35:02 UTC, Adam D. Ruppe wrote:
Wait, what's the bug there? The typeof DOES tell you they are
separate.
Error: cannot implicitly convert expression `b` of type
`S!(func2)` to `S!(func1)`
Sorry, it's a forum post, so I really should have been more
explicit.
It seems there's a broken symmetry in compiler error reporting
for the following, ostensibly identical, cases:
```d
struct S(alias Func){ }
int func1(int a){ return a*2; }
int func2(int a){ return a*2; }
void main()
{
auto a = S!func1();
auto b = S!func2();
a = b; // Error message is understandable
auto c = S!((int a) => a*2)();
auto d = S!((int a) => a*2)();
c = d; // Error message says a thing != to a same thing
}
```
Error messages formatted below for readability:
```
test.d(12): Error: cannot implicitly convert expression b of type
S!(func2)
to
S!(func1)
test.d(17): Error: cannot implicitly convert expression d of type
test.S!(function (int a) pure nothrow @nogc @safe => a * 2)
to
test.S!(function (int a) pure nothrow @nogc @safe => a * 2)
```
As new D programmer, this really threw me off the debugging
trail. For the second case, had the compiler reported some in the
order of:
```
test.d(17): Error: cannot implicitly convert expression d of type
test.S!(__lambda_temp_main_1)
to
test.S!(__lambda_temp_main_2)
Hint: All lambda function instances have unique auto-generated
names
```
It would have saved a lot of head scratching.
Oh well, I learned quite a bit from this exchange so that's
productive. Thanks all
Re: Recommendations on avoiding range pipeline type hell
On Sunday, 16 May 2021 at 10:10:54 UTC, SealabJaster wrote:
It's due to a quirk with passing lambdas as template arguments.
Each lambda is actually separated into its own function.
Hey that was a very well laid out example.
Okay, I think the light is starting do dawn. So if I use lambdas
as real arguments (not type arguments) then I'm just storing
function pointers and I'm back to C land where I understand
what's going on. Though I may lose some safety, I gain some
sanity.
For example:
```d
alias EXPLICIT_TYPE = int function (int);
struct Tplt(FT)
{
FT f;
}
void main()
{
auto a = Tplt!(EXPLICIT_TYPE)( a => a+3);
auto b = Tplt!(EXPLICIT_TYPE)( a => a*2);
a = b; // Lambdas as arguments instead of types works
}
```
Here's the non-lambda version of your example that helped me to
understand what's going on, and how the function called get's
mashed into the type (even though `typeid` doesn't tell us that's
what happens):
```d
struct S(alias Func)
{
pragma(msg, __traits(identifier, Func));
}
int func1(int a){ return a*2; }
int func2(int a){ return a*2; }
void main()
{
auto a = S!func1();
auto b = S!func2();
pragma(msg, typeof(a));
pragma(msg, typeof(b));
a = b;
}
```
I'm going to go above my station and call this a bug in
typeof/typeid. If the function name is part of the type, that
should be stated explicitly to make the error messages more
clear. We depend on those type names in compiler messages to
understand what's going on.
Cheers,
Re: Recommendations on avoiding range pipeline type hell
On Sunday, 16 May 2021 at 09:17:47 UTC, Jordan Wilson wrote: Another example: ```d auto r = [iota(1,10).map!(a => a.to!int),iota(1,10).map!(a => a.to!int)]; # compile error ``` Hi Jordan Nice succinct example. Thanks for looking at the code :) So, honest question. Does it strike you as odd that the exact same range definition is considered to be two different types? Maybe that's eminently reasonable to those with deep knowledge, but it seems crazy to a new D programmer. It breaks a general assumption about programming when copying and pasting a definition yields two things that aren't the same type. (except in rare cases like SQL where null != null.) On a side note, I appreciate that `.array` solves the problem, but I'm writing pipelines that are supposed to work on arbitrarily long data sets (> 1.4 TB is not uncommon).
Re: Recommendations on avoiding range pipeline type hell
On Saturday, 15 May 2021 at 14:05:34 UTC, Paul Backus wrote: If you post your code (or at least a self-contained subset of it) someone can probably help you figure out where you're running into trouble. The error messages by themselves do not provide enough information--all I can say from them is, "you must be doing something wrong." I just tacked on `.array` in the the unittest and moved on for now, but for those who may be interested in the "equivalent but not equivalent" dmd error message mentioned above, the code is up on github. To trigger the error message: ```bash git clone g...@github.com:das-developers/das2D.git cd das2D rdmd -unittest --main das2/range.d # This works ``` In file `das2/range.d`, comment out lines 550 & 553 and uncomment lines 557 & 558 to get alternate definitions of `coarse_recs` and `fine_recs` then run rdmd again: ```bash rdmd -unittest --main das2/range.d # No longer works ``` In addition to the issue mentioned above, comments on any style issues, best practices or design choices are invited. By the way the writeln calls in the unittests just temporary.
Re: Recommendations on avoiding range pipeline type hell
Thanks to everyone who has replied. You've given me a lot to think about, and since I'm not yet fluent in D it will take a bit to digest it all, though one thing is clear. This community is one of the strong features of D. I will mention it to others as a selling point. Best,
Re: Recommendations on avoiding range pipeline type hell
On Saturday, 15 May 2021 at 14:05:34 UTC, Paul Backus wrote: If you post your code (or at least a self-contained subset of it) someone can probably help you figure out where you're running into trouble. Smart idea. It's all on github. I'll fix a few items and send a link soon as I get a little shut eye. all I can say from them is, "you must be doing something wrong." I bet you're right :) Take Care,
Re: Recommendations on avoiding range pipeline type hell
On Saturday, 15 May 2021 at 13:43:29 UTC, Mike Parker wrote: On Saturday, 15 May 2021 at 11:25:10 UTC, Chris Piker wrote: In addition to what Adam said, if you do need to store the result for use in a friendlier form, just import `std.array` and append `.array` to the end of the pipeline. This will eagerly allocate space for and copy the range elements to an array, i.e., convert the range to a container: Thanks for the suggestion. Unfortunately the range is going to be 40+ years of Voyager magnetometer data processed in a pipeline. I am trying to do everything in functional form, but the deep type dependencies (and my lack of knowledge) are crushing my productivity. I might have to stop trying to write idiomatic D and start writing Java-in-D just to move this project along. Fortunately, D supports that too.
Re: Recommendations on avoiding range pipeline type hell
On Saturday, 15 May 2021 at 11:51:11 UTC, Adam D. Ruppe wrote: On Saturday, 15 May 2021 at 11:25:10 UTC, Chris Piker wrote: The idea is you aren't supposed to care what the type is, just what attributes it has, e.g., can be indexed, or can be assigned, etc. (Warning, new user rant ahead. Eye rolling warranted and encouraged) I'm trying to do that, but range3 and range2 are written by me not a Phobos wizard, and there's a whole library of template functions a person needs to learn to make their own pipelines. For example: ```d // From std/range/package.d CommonType!(staticMap!(ElementType, staticMap!(Unqual, Ranges)) alias RvalueElementType = CommonType!(staticMap!(.ElementType, R)); // ... what's with the . before the ElementType statement? Line 921 says // .ElementType depends on RvalueElementType. How can they depend on // each other? Is this a recursive template thing? ``` and all the other automagic stuff that phobos pulls off to make ranges work. If that's what's needed to make a custom range type, then D ranges should come with the warning **don't try this at home**. (Ali's book made it look so easy that I got sucker in) Every time I slightly change the inputs to range2, then a function that operates on *range3* output types blows up with a helpful message similar to: ``` template das2.range.PrioritySelect!(PriorityRange!(DasRange!(Tuple!(int, int)[], int function(Tuple!(int, int)) pure nothrow @nogc @safe, int function(Tuple!(int, int)) pure nothrow @nogc @safe, Tuple!(int, int), int), int function() pure nothrow @nogc @safe), PriorityRange!(DasRange!(Tuple!(int, int)[], int function(Tuple!(int, int)) pure nothrow @nogc @safe, int function(Tuple!(int, int)) pure nothrow @nogc @safe, Tuple!(int, int), int), int function() pure nothrow @nogc @safe)).PrioritySelect.getReady.filter!((rng) => !rng.empty).filter cannot deduce function from argument types !()(PriorityRange!(DasRange!(Tuple!(int, int)[], int function(Tuple!(int, int)) pure nothrow @nogc @safe, int function(Tuple!(int, int)) pure nothrow @nogc @safe, Tuple!(int, int), int), int function() pure nothrow @nogc @safe), PriorityRange!(DasRange!(Tuple!(int, int)[], int function(Tuple!(int, int)) pure nothrow @nogc @safe, int function(Tuple!(int, int)) pure nothrow @nogc @safe, Tuple!(int, int), int), int function() pure nothrow @nogc @safe)) ``` What the heck is that? Anyway, you put it all in one bit thing and this is kinda important: avoid assigning it to anything. You'd ideally do all the work, from creation to conclusion, all in the big pipeline. I fell back to using assignments just to make sure range2 values were saved in a concrete variable so that range3 didn't break when I changed the lambda that was run by range2 to mutate it's output elements. What went in to getting the element to range3's doorstep is a detail that I shouldn't have to care about inside range3 code, but am forced to care about it, because changing range2's type, changes range3's type and triggers really obscure error messages. (Using interfaces or *gasp* casts, would break the TMI situation.) So say you want to write it auto mega_range = range1.range2!(lambda2).range3!(lambda3); writeln(mega_range); that'd prolly work, writeln is itself flexible enough, but you'd prolly be better off doing like Sure it will work, because writeln isn't some function written by a new user, it's got all the meta magic. This way the concrete type never enters into things, it is all just a detail the compiler tracks to ensure the next consumer doesn't try to do things the previous step does not support. It's all just a detail the compiler tracks, until you're not sending to writeln, but to your own data consumer. Then, you'd better know all of std.traits and std.meta cause you're going to need them too implement a range-of-ranges consumer. And by the way you have to use a range of ranges instead of an array of ranges because two ranges that look to be identical types, actually are not identical types and so can't go into the same array. Here's an actual (though formatted by me) error message I got stating that two things were different and thus couldn't share an array. Can you see the difference? I can't. Please point it out if you do. ```d das2/range.d(570,39): Error: incompatible types for (dr_fine) : (dr_coarse): das2.range.PriorityRange!( DasRange!( Take!( ZipShortest!( cast(Flag)false, Result, Generator!(function () @safe => uniform(0, 128)) ) ), int function(Tuple!(int, int)) pure nothrow @nogc @safe, int function(Tuple!(int, int)) pure nothrow @nogc @safe, Tuple!(int, int), int ), int function() pure nothrow @nogc @safe ) and das2.range.PriorityRange!( DasRange!( Take!( ZipShortest!( cast(Flag)false, Result, Generator!(function () @safe => uniform(0, 128)) ) ), int function(Tuple!(int, int))
Recommendations on avoiding range pipeline type hell
Hi D
Since the example of piping the output of one range to another
looked pretty cool, I've tried my own hand at it for my current
program, and the results have been... sub optimal.
Basically the issue is that if one attempts to make a range based
pipeline aka:
```d
auto mega_range = range1.range2!(lambda2).range3!(lambda3);
```
Then the type definition of mega_range is something in the order
of:
```d
TYPE_range3!( TYPE_range2!( TYPE_range1, TYPE_lamba2 ),
TYPE_lambda3));
```
So the type tree builds to infinity and the type of `range3` is
very much determined by the lambda I gave to `range2`. To me
this seems kinda crazy.
To cut through all the clutter, I need something more like a unix
command line:
```bash
prog1 | prog2 some_args | prog3 some_args
```
Here prog2 doesn't care what prog1 *is* just what it produces.
So pipelines that are more like:
```d
ET2 front2(ET1, FT)(ET1 element, FT lambda){ /* stuff */ }
ET3 front3(ET2, FT)(ET2 element, FT lambda){ /* stuff */ }
void main(){
for(; !range1.empty; range1.popFront() )
{
ET3 el3 = front3( front2(range1.front, lambda2), lamda3) );
writeln(el3);
}
}
```
But, loops are bad. On the D blog I've seen knowledgeable people
say all loops are bugs. But how do you get rid of them without
descending into Type Hell(tm). Is there anyway to get some type
erasure on the stack?
The only thing I can think of is to use Interfaces and Classes
like Java, but we don't have the automagical JVM reordering the
heap at runtime, so that means living life on a scattered heap,
just like python.
Is there some obvious trick or way of looking at the problem that
I'm missing?
Thanks for your patience with a potentially dumb question. I've
been working on the code for well over 12 hours so I'm probably
not thinking straight it this point.
Cheers all,
Re: Any suggestions on dmd error message formatting?
On Saturday, 15 May 2021 at 06:12:25 UTC, SealabJaster wrote: On Saturday, 15 May 2021 at 04:54:15 UTC, Chris Piker wrote: T_T My eyes burn. Good, it's not just me. If figured the Deities out there visually parse these messages even hung over. Seems the final `int function` parameter needs to accept a `Tuple!(int, int)` I did much the same as you and reformatted the error message to find the bug. As to the larger question of how to automatically process compiler output... got any ideas? Hey since you're pretty good at this, can you tell me why how a person fixes this error? I've already formatted it, but I haven't changed any of the non-whitespace text. ``` das2/range.d(570,39): Error: incompatible types for (dr_fine) : (dr_coarse): ``` ```d das2.range.PriorityRange!( DasRange!( Take!( ZipShortest!( cast(Flag)false, Result, Generator!(function () @safe => uniform(0, 128)) ) ), int function(Tuple!(int, int)) pure nothrow @nogc @safe, int function(Tuple!(int, int)) pure nothrow @nogc @safe, Tuple!(int, int), int ), int function() pure nothrow @nogc @safe ) and das2.range.PriorityRange!( DasRange!( Take!( ZipShortest!( cast(Flag)false, Result, Generator!(function () @safe => uniform(0, 128)) ) ), int function(Tuple!(int, int)) pure nothrow @nogc @safe, int function(Tuple!(int, int)) pure nothrow @nogc @safe, Tuple!(int, int), int ), int function() pure nothrow @nogc @safe ) ``` To get around the problem I used `.array` for a bit of type erasure, so for now this error isn't messing with my unittests, but danged if I can spot the problem, even formatted. My guess is that the `Result` item is a hint. `Result` probably never equals another `Result` no matter what.
Any suggestions on dmd error message formatting?
Hi D So the compile error messages getting from dmd are starting to remind me of the notorious 130 line error message I once got from a C++ compiler for missing a comma in a template. :-/ (After fixing that bug, I left work early and came back the next day with a python book.) So, like the noob I am, I've been copying error messages to a text editor, deleting all the @ annotations, replacing common patterns with short tokens, indenting the result, and then trying to understand the problem. I figured this community is so inventive that I'm obviously doing it wrong and that there is a better way. So does anyone have any suggestions on how I make error messages like this one more grokable? ``` das2/range.d(359,31): Error: constructor das2.range.PriorityRange!(DasRange!(Tuple!(int, int)[], int function(Tuple!(int, int)) pure nothrow @nogc @safe, int function(Tuple!(int, int)) pure nothrow @nogc @safe, Tuple!(int, int), int), int function() pure nothrow @nogc @safe).PriorityRange.this(DasRange!(Tuple!(int, int)[], int function(Tuple!(int, int)) pure nothrow @nogc @safe, int function(Tuple!(int, int)) pure nothrow @nogc @safe, Tuple!(int, int), int) range, int function(Tuple!(int, int)) priority) is not callable using argument types (DasRange!(Tuple!(int, int)[], int function(Tuple!(int, int)) pure nothrow @nogc @safe, int function(Tuple!(int, int)) pure nothrow @nogc @safe, Tuple!(int, int), int), int function() pure nothrow @nogc @safe) ``` As always, your advice is much appreciated
Re: Testing for object property supporting "<" comparison
On Wednesday, 12 May 2021 at 00:06:52 UTC, Paul Backus wrote: On Tuesday, 11 May 2021 at 19:42:34 UTC, Chris Piker wrote: std.traits.isOrderingComparable https://phobos.dpldocs.info/std.traits.isOrderingComparable.html Well I feel sheepish, don't know how I missed that one. Hey thanks for pointing it out.
Testing for object property supporting "<" comparison
Hi D I'm working on a bit of code that handles selecting one *.front from multiple range-ish objects. It's a select-or-drop algorithm for a data streaming service, the details aren't important. The algorithm takes a range of something I'll call "PriorityRange" objects. PriorityRange objects are like input ranges, but have a few extra members. The set is: .empty : Returns bool .front : Returns something non-void .popFront(): Return void .priority : Returns something comparable via "<" to other .priority values for all input PriortyRanges .min : Returns something comparable to .max below via "<" for all input PriorityRanges .max : Returns something comparable to .min above via "<" for all input PriorityRanges I noticed the isInputRange(R) enum from std/range/primitives.d and am wondering how to setup a similar check, call it "isPriorityRange(PR)". My problem is that I don't know how to specify that properties must be comparable via "<". I took a look at the Traits module, but there are more things that are comparable then Arithmetic or Floating point types. Also, since the compiler will catch objects that don't have these properties, maybe an enum check is not worth implementing. Thanks very much for any advice you might have, Cheers
Re: Selected elements from splitter output
On Tuesday, 4 May 2021 at 22:02:11 UTC, Ali Çehreli wrote: On 5/4/21 1:40 PM, Chris Piker wrote: > I only care about columns 0, 2, 3, 4, 8, 9, 10. That's std.range.stride. > char[][] wanted = string_range.get( [1, 5, 7] ); // pseudo-code element That's std.range.indexed. Hey Thanks! And even more, thanks for the book. It's very well organized. I keep a paper copy of "Programming in D" open on my desk all the time these days. Looking forward to getting a few copies for work once we're back in the office.
Selected elements from splitter output
Hi D
I have a white-space delimited file with quite a few columns, but
I only care about columns 0, 2, 3, 4, 8, 9, 10. Since I don't
need most of the 60+ columns it seemed like:
std.algorithm.iteration.splitter()
would be a better function to use then std.array.split(). My
problem is that I don't know how to get the elements I care about
from the splitter, for example:
char[] line;
char[][] cols_needed;
while(file.readln(line)){
auto a = line.splitter()
cols_needed = ???
}
On a related note, are there any standard library functions that
select specific elements of a range by index without a loop? So
the logical equivalent of:
auto string_range
char[][] wanted = string_range.get( [1, 5, 7] ); // pseudo-code
element selection
It's not a big deal if there's not something standard.
Thanks for the help,
Re: Need for speed
On Thursday, 1 April 2021 at 16:52:17 UTC, Nestor wrote: I was hoping to beat my dear Python and get similar results to Go, but that is not the case neither using rdmd nor running the executable generated by dmd. I am getting values between 350-380 ms, and 81ms in Python. Nice test. I'm new to D as well and can't comment on needed refactoring. To confirm your results I compiled the D example using: ``` gdc -O2 speed.d -o speed ``` and measured 129 ms for the D program and 63 ms for the python3 equivalent. I'll be keen to see how this plays out since I'm using D as a faster alternative to python.
Re: Creating a .di file for a custom C library
On Tuesday, 30 March 2021 at 04:01:12 UTC, Brad wrote: I would like to use an updated version of the Termbox library (written in C) with D. I have the .h file. This is new territory for me (why try something easy - right?). I think I need to create a .di file that corresponds to the .h file. I also suspect that I need to put the library file (C code) into the project as a file somehow. I am probably just not looking in the right place for the documentation, but I cannot seem to find a lot of guidance in this area. Thanks in advance. Hi Brad I used dstep https://code.dlang.org/packages/dstep to help me with this process. Though I ended up checking the output and fixing a couple oddities in the "extern (C):" line, it was *huge* time saver. The result was a D module file (i.e. thing.d, not thing.di) that I could include like any other, so long as the C library was specified on the compiler command line. In addition, you might want to check out https://wiki.dlang.org/Deimos where standard patterns are discussed for D prototypes of C libraries. I ended up using the command line: dub init --format=deimos to start off the wrapper lib package, then copied in the C header, and then ran dstep. Cheers,
Re: Contributing CDF bindings to Deimos
On Friday, 26 March 2021 at 00:50:36 UTC, Jordan Wilson wrote: Nice one. I've used HDF5/NetCDF, will have to check out and see what CDF offers. AFAIK CDF is much simpler than NetCDF. If you do generate CDF files and want other common tools to understand your data structures, use the ISTP metadata scheme: https://spdf.gsfc.nasa.gov/istp_guide/istp_guide.html Since deimos.cdf just provides C bindings and type safety, a higher level module would be nicer. Currently, the most natural interface to CDF files that I've used is the spacepy.pycdf python package.
Re: Contributing CDF bindings to Deimos
On Thursday, 25 March 2021 at 16:57:05 UTC, Bastiaan Veelo wrote: On Thursday, 25 March 2021 at 04:00:33 UTC, Chris Piker wrote: As an aside, software developers at NASA Goddard have now heard of D which is nice. They were pleased to see that it was supported by gcc. (Hat tip to the GDC team) That’s cool. I’d love to see NASA on https://dlang.org/orgs-using-d.html one day. — Bastiaan. I'd like that too, we'll see. The next set of NASA C code that needs bindings is the navigation library at https://naif.jpl.nasa.gov/naif/index.html. This is an older library and so it isn't listed in the expected location https://software.nasa.gov/ A next step for the the CDF bindings would be to parse the existing external text documentation for each function and output ddoc comments. Text processing is not my forte so any hints or volunteers are more than welcome. I'll open an issue for that on github.
Re: Contributing CDF bindings to Deimos
On Tuesday, 23 March 2021 at 05:54:13 UTC, mw wrote: On Tuesday, 23 March 2021 at 05:34:57 UTC, Chris Piker wrote: Create a github repo, and create an account on: https://code.dlang.org/ Then you can register your project there, and supported by dub build. Okay, that's done. The repo https://github.com/das-developers/deimos.cdf and package https://code.dlang.org/packages/cdf have been drafted and tested on Linux, I'm about to test on Windows and MacOS. As an aside, software developers at NASA Goddard have now heard of D which is nice. They were pleased to see that it was supported by gcc. (Hat tip to the GDC team) I've attempted to follow all guidelines as best I understood them, but this is my first package. It likely has some style and functionality issues. Is there a peer review stage to this process that is triggered automatically or could be requested?
Contributing CDF bindings to Deimos
Hi D There is a C library that's important for my line of work, https://cdf.gsfc.nasa.gov/ but it's not exactly a popular library in the general sense. I willing to contribute & maintain D bindings for this library following the Deimos guidelines but am wondering if it's too specific to be allowed into the repository. If D bindings for the Common Data Format library are an acceptable addition, can you point me to a procedure document for issuing pull requests for package submissions? Thanks for the guidance,
Re: noobie question, dub or meson?
On Saturday, 20 March 2021 at 18:33:20 UTC, James Blachly wrote: Chris: for one of my (D) libraries that also links in an .o file that's built from C source, I have a makefile for the C and call `make` during the dub build process. It is not incredibly sophisticated, but works well for us. Here is the dubfile with `preBuildCommands`: https://github.com/blachlylab/intervaltree/blob/master/dub.json James: Nice concrete example, thanks! My code is cross platform, so I'll try to adapt what you have to a version that invokes nmake.exe if run in a visual studio environment. Take Care,
Re: noobie question, dub or meson?
On Thursday, 18 March 2021 at 06:02:03 UTC, Elronnd wrote: Meson doesn't track dependencies properly for d, so your dirty builds will be wrong if you go that route. You might consider keeping the c and d code in the same repository, but with separate build systems; using dub to build the d code and and whatever tool you prefer for c. Or try reggae. Hi Elronnd You might be right. Since Visual D is highly spoken of on these forums I decided to give it a go. I don't usually write software on Windows, and I have to say that once I stepped outside the D ecosystem the experience was not good. There's no standard locations for anything. Meson worked well on Linux, but I can't figure out how to get it to find vcpkg dependencies on Windows. Outside Visual D, the one thing that worked great on windows was: dub I know everyone knocks the idea of downloading your dependencies off the web, but when starting from scratch you have to get them from somewhere and dub comes to the rescue, and "add-path" is there for local packages. Has there ever been talk of adding C source code support to dub, or is that a forbidden topic? I know if dub supported C, all my C libs and all my necessary dependencies (openssl, expat, etc.) would have dub.json files before the weekend was over.
noobie question, dub or meson?
Hi D I've started a D layer for one of my C libraries that's adds some new functionality and a bit of an interface upgrade. In turn I'm using this combined code-base as a dependency for D "scripts". Since my software is used by a few outside groups in my field, it seems I should get used to packaging D modules, even if those packages never make it to the central dub repo. Given that source code for the combined library is some D but mostly C, would you recommend that I: 1) Keep the D sources and C sources in separate projects? 2) Use meson to create a combined package? 3) Use dub to create a combined package? 4) Some other option? The D code is useless without it's C core, so a dub package that just includes the D parts would be disappointing. The library's not huge, only about 25K lines, but I don't think I have time for a straight conversion of the whole thing to D at this point. Thanks for your opinions on the matter,
Re: rdmd and D equivalent for PYTHONPATH?
On Wednesday, 17 March 2021 at 20:24:19 UTC, Tobias Pankrath
wrote:
For scripts this could be a good way, but it does not really
work with most dub packages:
1. put all your dependencies into a single location, like
/home//dstuff
2. add -I /home//dstuff to your call to rdmd/dmd (or put
into /etc/dmd.conf
3. add -i (lowercase) to your call of rdmd/dmd
4. profit
I tried that route, and it's not too bad, but I have C library
dependencies so I start getting shebangs that look like this:
#!/usr/bin/env -S rdmd -i -I${D_PATH} -L-L${LD_LIBRARY_PATH}
-L-ldas2.3 -L-lexpat -L-lssl -L-lfftw3
So kinda messy, though the rdmd -i option is nice. Since the dub
packages have the linker info builtin, it seemed better to use
this
instead:
#!/usr/bin/env dub
and to assist with finding local packages, throw in some sort of
local search path via an environment variable reference in the
dub.sdl section.
If everyone used the same environment variable in dub.sdl comment
it could become a defacto standard for scripts, similar to
PYTHONPATH or MATLABPATH, though not nearly as fundamental.
To make this work the dependencies must have the correct
project layout, e.g. sources should be in the top-level project
directory and not in a subdirectory source. This rules out most
dub packages :/
Yea, this seemed strange to me. I'd think that a group would want
the compiler's module lookup semantics to match the common package
layout scheme and vice-versa. But since I'm new around here I'll
just assume that the mismatch came about for a reason. The dub
designers were probably trying to solve some problem that I'm
unaware
of.
Re: rdmd and D equivalent for PYTHONPATH?
On Wednesday, 17 March 2021 at 20:13:49 UTC, Imperatorn wrote: On Wednesday, 17 March 2021 at 19:33:26 UTC, Chris Piker wrote: On Wednesday, 17 March 2021 at 09:34:21 UTC, Mike Parker wrote: [...] Sure will, thanks for the invite to contribute in a specific way. [...] You probably already know this, just sharing: https://atilaoncode.blog/2020/02/19/want-to-call-c-from-python-use-d/ Hey, that looks nice. I'm trying to get away from python as the front end language, but I do have a fair bit of C code running around and there are a lot of python users. Thanks for the tip :)
Re: rdmd and D equivalent for PYTHONPATH?
On Wednesday, 17 March 2021 at 09:34:21 UTC, Mike Parker wrote: On Wednesday, 17 March 2021 at 07:13:31 UTC, Chris Piker wrote: Very handy example. Unfortunately it means that paths are embedded in scripts, which is usually a bad idea. The ability to use D source modules “script style” is something that has grown organically over time largely as a convenience. I doubt anyone is using it heavily enough to have tested the boundaries, In your exploration of those boundaries, please take note of what you discover so they can potentially be expanded where possible. Sure will, thanks for the invite to contribute in a specific way. D looks to be a good replacement for split Python/C development though I don't want to drag all my python baggage in here. I'm trying to understand the D way of doing things before suggesting changes. Interpreted languages like Python MATLAB, IDL are the norm in my field. So anything that makes D easier to use for "quick-and-dirty" data analysis tasks would make it more palatable to the casual programmers I interact with. I general dub seems fantastic! I was stunned yesterday by a three-line vibe.d test script I ran that produced a compiled running web-server. So, if I could do the equivalent of: dub add-path via an environment variable (not a permanent change under ~/.dub), or have some environment variable that tells dub where to read a "system-level" local-packages.json file and merge it's paths in with any personal settings, that would likely handle our internal code sharing needs.
Re: rdmd and D equivalent for PYTHONPATH?
On Wednesday, 17 March 2021 at 06:07:01 UTC, user1234 wrote: You can use local a specific local version too, for example the git repository #!/usr/bin/env dub /+ dub.sdl: dependency "mir-algorithm" path="/home/x/repositories/mir/mir-algorithm" +/ In addition with --nodeps, no internet is required. Very handy example. Unfortunately it means that paths are embedded in scripts, which is usually a bad idea. I'm still looking for environment variables or config files that affect dub's module include path. Is there dub variable to give a general path for all dependencies? Since dub can read environment variables, this may be a way to get a top-level module directory known to scripts without hard coding paths. Also, what do people do when generating .deb or .rpm packages for D libraries? They must reference some local library path in a general fashion (I would think). The only module paths I see referenced in: /etc/dmd.conf are for phobos and the runtime import. I guess I could just add another one there. Not sure if gdc also uses that file.
Re: rdmd and D equivalent for PYTHONPATH?
On Wednesday, 17 March 2021 at 03:43:22 UTC, Chris Piker wrote: Note: I'm aware of dub. This isn't a question about dub. I'm making scripts for local use, not redistributable binaries, so I would like to "install" mir-algorithm and similar libraries for my rdmd scripts to use. Sorry to reply to myself, but I found something that will run the scripts, though it still doesn't make use of local libraries. #!/usr/bin/env dub /+ dub.sdl: dependency "mir-algorithm" version="~>1.0" +/ import mir.ndslice ... Since dub seems to work a bit better as the "interpreter" then rdmd (assuming you're connected to the Internet) why is this not promoted here: https://tour.dlang.org/tour/en/welcome/run-d-program-locally instead of rdmd? Thanks,
rdmd and D equivalent for PYTHONPATH?
Hi D I've writing little test scripts using rdmd to understand what various functions are really doing (ex: .take(5)). I'm up to the point where I need to write sample code to understand mir-algorithm a little better, but of course the library is not installed on my system. So two related questions: 1) After a "git clone" and "dub build" I'm left with redistributables that need to be put ... somewhere. But where? I tried this: cp -r -p source/mir /usr/local/include cp -p libmir-algorithm.a /usr/local/lib but no joy. So where can I install libs so that rdmd will find them? 2) On a related topic, is there an environment variable I can set (similar to PYTHONPATH, MATLABPATH, IDL_PATH, etc.) or a config file I can alter that will add to the module include path? I tried: #!/usr/bin/env DFLAGS="-I/usr/local/include" rdmd at the top of the file, but that just hung rdmd and nothing ran. Note: I'm aware of dub. This isn't a question about dub. I'm making scripts for local use, not redistributable binaries, so I would like to "install" mir-algorithm and similar libraries for my rdmd scripts to use. Thanks for the help,
