Re: Recommendation on plotting library
On Thursday, 20 July 2023 at 02:37:54 UTC, Chris Piker wrote: Hi D One of my jobs is to release and maintain public data archives from long-running scientific instruments. In order to help people understand how to process the data, sample code is often included with the archive. Recently this has been in the form of short programs that generate a plot of a representative time period. Python works well for this task as matplotlib support is pretty much universal. Matlab is also a reasonable choice for many. [...] https://forum.dlang.org/post/mgxcrmjcgasbdaipd...@forum.dlang.org
Re: Recommendation on plotting library
Thanks for the both of the long replies. I've ready them twice and will do so again. To focus in on one aspect of D package support: On Saturday, 22 July 2023 at 02:24:08 UTC, Greggor wrote: In general whenever possible I think its better for everyone that stuff is built from source. It ensures that builds can be re-produced by anyone and that issues with building are caught fast by anyone consuming the library. I agree 100%. I've run into similar python package hell dealing with Tensor flow on Rocky Linux 8, since it wasn't Ubuntu. The problem is we're small, and many dependencies are already written in C, and dub is not setup to handle C dependencies. If I could turn my legacy C projects (pre-processor macros and all) into dub projects I'd do so as fast as possible. If dub supported two things: 1. Building C projects 2. Installing the resulting binaries in "site-packages" or similar. many, many project kickstart issues would be solved. Like C++, D is designed as a successor to C. It would be nice to bring C into the party with full dub support.
Re: Recommendation on plotting library
On Friday, 21 July 2023 at 22:51:16 UTC, Jonathan M Davis wrote: On Friday, July 21, 2023 11:40:25 AM MDT Greggor via Digitalmars-d-learn wrote: >> So as far as I can tell, python pip originally only dealt >> with python code, but eventually wheels were added for >> binary support. >> >> Just as a wild guess, do you see dub ever evolving in that >> >> direction? All the reasons for not supporting pre-compiled >> binaries in pip apply to dub, but yet support was added >> anyway, and it's been wildly successful. >> >> I know it's hard to make predictions (especially about the >> future), but I'd be interesting in your opinion on the >> matter. > > I'd be very surprised if dub added support for pre-compiled > binaries - particularly since D isn't generally binary > compatible across releases - but I really don't know what > the folks working on dub want to do with it. > > - Jonathan M Davis Dependency management sucks for windows and I understand wanting the ability to just do dub run and have it “just work tm”. Up to date versions of Windows 10 should have curl included and dub can run commands before building, so you could try downloading a prebuilt lib for windows via curl. https://everything.curl.dev/get/windows Well, from what I recall (though it's been a while since I messed with anything like it), it's possible with dub to run more or less arbitrary stuff (e.g. use cmake from dub). It's a pain, but it can be done. And if that's the case, then you should be able to design a dub project that pulled in pretty much whatever you want with curl. And simply being able to build and run a D program as part of the build would be enough to be able to use curl in general without Windows having added it, since dmd comes with it because of std.net.curl. But of course, that's a rather different thing from dub providing a clean way to handle that sort of thing. dub can do a lot, but because it's really not designed around doing much beyond pulling in dependencies from code.dlang.org and then building all of the code, doing stuff beyond that gets to be problematic even if it can be done. So, if we wanted something more fully featured, then dub would need a bit of an overhaul. And maybe dub will get that at some point (I don't know), but as things stand, if anyone really wants to do complex stuff with their builds, dub can be pretty awkward to use. And that's why projects such as https://code.dlang.org/packages/reggae exist, though it's geared more towards providing a more fully-feature build system than providing a way to pull in pre-built binaries and the like. What additional features we get in the future will likely be highly dependent on how motivated the people are who want dub to be able to do more. - Jonathan M Davis I'll be honest, I actually really don't want this, pip and especially prebuilt wheels deps have often made my life miserable. On the linux side of things. Pulling in dynamic libraries that are not from the distro package manager or freshly built locally is simply not a good idea. Things like different stdlibC(s) will break even the simplest of things. Python is actually a good example of what NOT to do, I don't program in python so take what I am about to say with a grain of salt. But not too long ago I installed onto my machine stable diffusion and it was a harrowing process that perfectly encapsulates my problems with the python ecosystem. Building the deps from source was broken and the prebuilt deps where not matching my distro's python version. I tried all sorts of things like venv and building as much stuff locally all the other "correct" ways and nothing worked. Eventually I went down a path of super jank and essentially frankensteined a working runtime by hand. I built a specific version of python from src and installed it to its own folder, I used a bunch of LD env vars and used patchelf on deps to basically make an isolated install of python and glued together a bunch of the native pre build packages, because building them on my machine was a pain. What should of been a simple "pip install X" is now a cargo cult-ed install. I dread the day when it will stop working. Whats even more upsetting is that non of it is some ancient out of date packages, it's all popular and maintained stuff. Maybe part of the problem is that I when I say "linux", I am not referring or implying some version of Ubuntu and that my understanding of the word includes distros that use different stdlibC(s) and could also not use things like system-D. Despite the amount of users and active support, I guess a lot of headaches related to building are flying under the radar due to pre-built packages. Comparing this with Dub & D is a night and day difference, the current ecosystem is very easy to use and has little issues, even with our smaller community. I feel comfortable that anytime I ran into an issue with a
Re: Recommendation on plotting library
On Friday, 21 July 2023 at 17:40:25 UTC, Greggor wrote: Up to date versions of Windows 10 should have curl included and dub can run commands before building, so you could try downloading a prebuilt lib for windows via curl. https://everything.curl.dev/get/windows Hey, nice! This might be a winner. Unless I'm missing something libcairo-2.dll, libpng13.dll and zlib1.dll could be downloaded, and then ggplotd plot can build against those (probably?). The key thing that dub is missing compared to pip is some concept of an "install" directory. Compared to everything else dub handles, not adding this capability mostly comes down to self imposed limitations. I wonder how many in the community would like to see those limitations removed? I presume it's just as easy to add commands that run after the build as before. I could have an alternate dub configuration that's short for "setup_build_test_install" that runs everything. If multiple projects start defining an install target the same way, then you have a convention, which can eventually lead to a standard.
Re: Recommendation on plotting library
On Friday, July 21, 2023 11:40:25 AM MDT Greggor via Digitalmars-d-learn wrote: > >> So as far as I can tell, python pip originally only dealt with > >> python code, but eventually wheels were added for binary > >> support. > >> > >> Just as a wild guess, do you see dub ever evolving in that > >> > >> direction? All the reasons for not supporting pre-compiled > >> binaries in pip apply to dub, but yet support was added anyway, > >> and it's been wildly successful. > >> > >> I know it's hard to make predictions (especially about the > >> future), but I'd be interesting in your opinion on the matter. > > > > I'd be very surprised if dub added support for pre-compiled > > binaries - particularly since D isn't generally binary > > compatible across releases - but I really don't know what the > > folks working on dub want to do with it. > > > > - Jonathan M Davis > > Dependency management sucks for windows and I understand wanting > the ability to just do dub run and have it “just work tm”. > > Up to date versions of Windows 10 should have curl included and > dub can run commands before building, so you could try > downloading a prebuilt lib for windows via curl. > https://everything.curl.dev/get/windows Well, from what I recall (though it's been a while since I messed with anything like it), it's possible with dub to run more or less arbitrary stuff (e.g. use cmake from dub). It's a pain, but it can be done. And if that's the case, then you should be able to design a dub project that pulled in pretty much whatever you want with curl. And simply being able to build and run a D program as part of the build would be enough to be able to use curl in general without Windows having added it, since dmd comes with it because of std.net.curl. But of course, that's a rather different thing from dub providing a clean way to handle that sort of thing. dub can do a lot, but because it's really not designed around doing much beyond pulling in dependencies from code.dlang.org and then building all of the code, doing stuff beyond that gets to be problematic even if it can be done. So, if we wanted something more fully featured, then dub would need a bit of an overhaul. And maybe dub will get that at some point (I don't know), but as things stand, if anyone really wants to do complex stuff with their builds, dub can be pretty awkward to use. And that's why projects such as https://code.dlang.org/packages/reggae exist, though it's geared more towards providing a more fully-feature build system than providing a way to pull in pre-built binaries and the like. What additional features we get in the future will likely be highly dependent on how motivated the people are who want dub to be able to do more. - Jonathan M Davis
Re: Recommendation on plotting library
On Thursday, 20 July 2023 at 04:41:48 UTC, Chris Piker wrote: [SNIP] I just tried ggplotd and it was easy to make it work on Linux, only one external apt command needed, but on Windows, even that is a deal breaker. Package management on Windows seems to be wild-west/nonexistent. Try MinGW. There should be SOME way to use GTK with it, though I haven't been able to get it to work. [SNIP]
Re: Recommendation on plotting library
On Friday, 21 July 2023 at 15:12:57 UTC, Jonathan M Davis wrote: On Friday, July 21, 2023 1:03:47 AM MDT Chris Piker via Digitalmars-d-learn wrote: On Friday, 21 July 2023 at 06:15:10 UTC, Jonathan M Davis wrote: > On Thursday, July 20, 2023 10:57:22 PM MDT Chris Piker via > Digitalmars-d-learn wrote: > > Regardless though, dub really isn't designed with packaging > anything in mind. Rather, it's designed to build your code > as well as pull in D libraries that it usees and build those > too. Anyone looking to actually package stuff would create a > package from what was built with dub (e.g. with deb, rpm, > flatpacks, etc.). So as far as I can tell, python pip originally only dealt with python code, but eventually wheels were added for binary support. Just as a wild guess, do you see dub ever evolving in that direction? All the reasons for not supporting pre-compiled binaries in pip apply to dub, but yet support was added anyway, and it's been wildly successful. I know it's hard to make predictions (especially about the future), but I'd be interesting in your opinion on the matter. I'd be very surprised if dub added support for pre-compiled binaries - particularly since D isn't generally binary compatible across releases - but I really don't know what the folks working on dub want to do with it. - Jonathan M Davis Dependency management sucks for windows and I understand wanting the ability to just do dub run and have it “just work tm”. Up to date versions of Windows 10 should have curl included and dub can run commands before building, so you could try downloading a prebuilt lib for windows via curl. https://everything.curl.dev/get/windows
Re: Recommendation on plotting library
On Friday, July 21, 2023 1:03:47 AM MDT Chris Piker via Digitalmars-d-learn wrote: > On Friday, 21 July 2023 at 06:15:10 UTC, Jonathan M Davis wrote: > > On Thursday, July 20, 2023 10:57:22 PM MDT Chris Piker via > > Digitalmars-d-learn wrote: > > > > Regardless though, dub really isn't designed with packaging > > anything in mind. Rather, it's designed to build your code as > > well as pull in D libraries that it usees and build those too. > > Anyone looking to actually package stuff would create a package > > from what was built with dub (e.g. with deb, rpm, flatpacks, > > etc.). > > So as far as I can tell, python pip originally only dealt with > python code, but eventually wheels were added for binary support. > Just as a wild guess, do you see dub ever evolving in that > direction? All the reasons for not supporting pre-compiled > binaries in pip apply to dub, but yet support was added anyway, > and it's been wildly successful. > > I know it's hard to make predictions (especially about the > future), but I'd be interesting in your opinion on the matter. I'd be very surprised if dub added support for pre-compiled binaries - particularly since D isn't generally binary compatible across releases - but I really don't know what the folks working on dub want to do with it. - Jonathan M Davis
Re: Recommendation on plotting library
On Friday, 21 July 2023 at 06:15:10 UTC, Jonathan M Davis wrote: On Thursday, July 20, 2023 10:57:22 PM MDT Chris Piker via Digitalmars-d-learn wrote: Regardless though, dub really isn't designed with packaging anything in mind. Rather, it's designed to build your code as well as pull in D libraries that it usees and build those too. Anyone looking to actually package stuff would create a package from what was built with dub (e.g. with deb, rpm, flatpacks, etc.). So as far as I can tell, python pip originally only dealt with python code, but eventually wheels were added for binary support. Just as a wild guess, do you see dub ever evolving in that direction? All the reasons for not supporting pre-compiled binaries in pip apply to dub, but yet support was added anyway, and it's been wildly successful. I know it's hard to make predictions (especially about the future), but I'd be interesting in your opinion on the matter.
Re: Recommendation on plotting library
On Thursday, July 20, 2023 10:57:22 PM MDT Chris Piker via Digitalmars-d-learn wrote: > (Warning, possible ill-informed opinions ahead...) > > In a way there is a need to reinvent the wheel. With python I > can run `pip install matplotlib` and get whatever binaries I need > to get the job done. D runs on dub, which I only see handling > source code, and as far as I can tell, only D source code at > that. So unless it's D code, it can't be packaged and delivered > easily within the D ecosystem. > > If dub supports either pre-built binaries, or C code (such as > libcairo2), I'd be interested in seeing how that's done. With > the wizardry I've see around here, it's probably easy, I just > don't know about it. Well, dub is certainly designed around building projects that are pure D without anything fancy going on, but with effort, it's possible to do far more complicated stuff (though it's certainly far more of a pain than would be desirable). That being said, if the C libraries are already on your system, it's trivial to have a dub project just use them via bindings in the D code. And there are libraries on code.dlang.org which are basically just bindings for C libraries. Regardless though, dub really isn't designed with packaging anything in mind. Rather, it's designed to build your code as well as pull in D libraries that it usees and build those too. Anyone looking to actually package stuff would create a package from what was built with dub (e.g. with deb, rpm, flatpacks, etc.). > Going wy out on a limb for a minute, I think D shines as a > scripting language replacement. Most of my programs are single > file projects these days with dub set as the interpreter. Also > Rust seems to be crowding the system level space and so focusing > on it's "compiled scripts" capability avoids that competition. > > (If any of the statements above are faulty, I invite correction.) D does work quite well as a scripting language replacement for a lot stuff. In fact, the compiler and standard library have largely replaced their Makefiles with D scripts. It's definitely worse if you want to write a script that needs to pull in dependencies that aren't part of the standard library, but if Phobos has what you need, it works quite well - and of course, you can always run other shell commands from within a D program. - Jonathan M Davis
Re: Recommendation on plotting library
On Friday, 21 July 2023 at 02:40:10 UTC, harakim wrote: On Thursday, 20 July 2023 at 02:37:54 UTC, Chris Piker wrote: If you happen upon a basic charting library for D during this hunt, please let me know! Last year, I rolled my own and it got the job done, but I wasn't concerned about how they looked. :D Well, just for fun, if you have a plotting library on github or some other public place I wouldn't mind taking a look. Given the state of things, it doesn't have to be awesome to be a good seed point. As for plotting, I imagine it's c bindings only because there's no need to re-invent the wheel. (Warning, possible ill-informed opinions ahead...) In a way there is a need to reinvent the wheel. With python I can run `pip install matplotlib` and get whatever binaries I need to get the job done. D runs on dub, which I only see handling source code, and as far as I can tell, only D source code at that. So unless it's D code, it can't be packaged and delivered easily within the D ecosystem. If dub supports either pre-built binaries, or C code (such as libcairo2), I'd be interested in seeing how that's done. With the wizardry I've see around here, it's probably easy, I just don't know about it. Going wy out on a limb for a minute, I think D shines as a scripting language replacement. Most of my programs are single file projects these days with dub set as the interpreter. Also Rust seems to be crowding the system level space and so focusing on it's "compiled scripts" capability avoids that competition. (If any of the statements above are faulty, I invite correction.)
Re: Recommendation on plotting library
On Thursday, 20 July 2023 at 02:37:54 UTC, Chris Piker wrote: Hi D One of my jobs is to release and maintain public data archives from long-running scientific instruments. In order to help people understand how to process the data, sample code is often included with the archive. Recently this has been in the form of short programs that generate a plot of a representative time period. Python works well for this task as matplotlib support is pretty much universal. Matlab is also a reasonable choice for many. Were I to also provide sample code in D, what library would you recommend for scientific plotting, especially for dynamic power spectral densities generated from time-series data (basically heatmaps)? Since dub can pull down dependencies easily enough and since it supports single file projects it seems that D would be well suited to this task, but I've never plotted any data directly from a D program. Many GUI libraries I see for D are wrappers around external C or C++ based libraries that would need to be installed separately. I'd like to avoid such complications if that's possible. Thanks for any advice, If you happen upon a basic charting library for D during this hunt, please let me know! Last year, I rolled my own and it got the job done, but I wasn't concerned about how they looked. :D As for plotting, I imagine it's c bindings only because there's no need to re-invent the wheel.
Re: Recommendation on plotting library
20.07.2023 05:37, Chris Piker пишет: Hi D One of my jobs is to release and maintain public data archives from long-running scientific instruments. In order to help people understand how to process the data, sample code is often included with the archive. Recently this has been in the form of short programs that generate a plot of a representative time period. Python works well for this task as matplotlib support is pretty much universal. Matlab is also a reasonable choice for many. Were I to also provide sample code in D, what library would you recommend for scientific plotting, especially for dynamic power spectral densities generated from time-series data (basically heatmaps)? Since dub can pull down dependencies easily enough and since it supports single file projects it seems that D would be well suited to this task, but I've never plotted any data directly from a D program. Many GUI libraries I see for D are wrappers around external C or C++ based libraries that would need to be installed separately. I'd like to avoid such complications if that's possible. Thanks for any advice, https://code.dlang.org/packages/ggplotd
Re: Recommendation on plotting library
On Thursday, 20 July 2023 at 04:41:48 UTC, Chris Piker wrote: On Thursday, 20 July 2023 at 03:58:05 UTC, Andrew wrote: I just tried ggplotd and it was easy to make it work on Linux, only one external apt command needed, but on Windows, even that is a deal breaker. Package management on Windows seems to be wild-west/nonexistent. Have you tried https://github.com/koji-kojiro/matplotlib-d? There is a slightly more updated fork out there but I can't remember exactly who maintains it. Side musing... Since I've seen many people use anaconda on Windows, I wonder how hard it would be to make a conda package that provided dmd+dub? Have you tried chocolatey.org? It's similar to homebrew for macOS but for Windows.
Re: Recommendation on plotting library
On Thursday, 20 July 2023 at 03:58:05 UTC, Andrew wrote: If you're already using python, it's probably best to keep using that. Oh of course. Examples *have* to be provided in python, since that's the default language of science these days. But extra examples don't hurt, and it would be nice to see D along side other options. I find these days that my D "scripts" are about the same size as my python scripts, which is really surprising for a compiled language. I just tried ggplotd and it was easy to make it work on Linux, only one external apt command needed, but on Windows, even that is a deal breaker. Package management on Windows seems to be wild-west/nonexistent. Side musing... Since I've seen many people use anaconda on Windows, I wonder how hard it would be to make a conda package that provided dmd+dub?
Recommendation on plotting library
Hi D One of my jobs is to release and maintain public data archives from long-running scientific instruments. In order to help people understand how to process the data, sample code is often included with the archive. Recently this has been in the form of short programs that generate a plot of a representative time period. Python works well for this task as matplotlib support is pretty much universal. Matlab is also a reasonable choice for many. Were I to also provide sample code in D, what library would you recommend for scientific plotting, especially for dynamic power spectral densities generated from time-series data (basically heatmaps)? Since dub can pull down dependencies easily enough and since it supports single file projects it seems that D would be well suited to this task, but I've never plotted any data directly from a D program. Many GUI libraries I see for D are wrappers around external C or C++ based libraries that would need to be installed separately. I'd like to avoid such complications if that's possible. Thanks for any advice,
