Re: Dub support was added to Meson
On Thu, 2018-08-16 at 22:44 +, Filipe Laíns via Digitalmars-d-announce wrote: […] Apologies for the delay in replying to this one. > This is obviously bad. Your distro has a package manager, you > should use it, not create a separated language-specific one. If I'm afraid you are onto a lost cause on this one. The whole JVM-based milieu, Ruby, Python, Go, D, Rust, etc. all have language specific repositories. Debian, Fedora, etc. pick and choose which bits they choose to package based on some algorithm almost, but not quite, totally unrelated to what is the latest version. Operating system package managers are providing the operating system, not the development tools and dependencies needed for software development. Go, Rust, and indeed D, are going the route of static compilation as much because operating system dependencies can never be guaranteed, and are often wrong. It is not clear to me why Debian spend so much time packaging bits of the Go universe that no-one uses even if the dependencies are in fact used. On the other hand, having GtkD (and GStreamerD) packaged is great since there are shared objects for use with D codes. There is nothing quite so depressing as waiting for LDC or DMD to statically link to GtkD. So static linking is not something I want. But waiting for Debian and Fedora to package things is often like Waiting for Godot. Hence "build it yourself" becomes a bit of a must. This is not a simple situation, and every individuals positions on it is likely inconsistent and full of holes. > you are doing this locally, either but using the user's home of > by installing to /usr/local, I don't think it's much of a > problem. If you are implementing something like this at least do > it in a way that the package managing feature is optional. I > don't know if I'm being biased by being an Archlinux TU but from > my perspective, it's not something we should do, at the very > least globally. It may be that Arch stays more up to date than Debian and Fedora (because it is less centralised and more like Homebrew/Linuxbrew, but Debian (and Fedora?) is where the bulk of Linux programs get executed, and so is the obvious place to develop. > Weirdly enough, I can reproduce. This was working when I wrote > the patch. I've opened an issue in the upstream. Excellent. I have a clone of Meson so can try stuff out on master/HEAD as needed. -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Dub support was added to Meson
On Wednesday, 15 August 2018 at 09:31:48 UTC, Russel Winder wrote: That's the battle isn't it. Should the system that builds be the system that also manages the packages for the build (cf. Dub, Cargo, Go, Mason), or should the two be kept separate (cf. CMake, Meson, SCons, Make). The debate has been had on the D mailing lists a number of times and at great length. But in the end D, Rust, and Go are going with a single system for everything. This is obviously bad. Your distro has a package manager, you should use it, not create a separated language-specific one. If you are doing this locally, either but using the user's home of by installing to /usr/local, I don't think it's much of a problem. If you are implementing something like this at least do it in a way that the package managing feature is optional. I don't know if I'm being biased by being an Archlinux TU but from my perspective, it's not something we should do, at the very least globally. I am not sure how "dub describe" tells you which compiler is being used, it always says "compiler": "dmd" on my projects even though I only use ldc2 for build. Weirdly enough, I can reproduce. This was working when I wrote the patch. I've opened an issue in the upstream.
Re: Dub support was added to Meson
On 15/08/2018 9:31 PM, Russel Winder wrote: Maybe then Dub should be better able to handle shared objects. Whilst the whole static linking vs. shared objects is another topic that descends into flame wars of prejudice, the de facto standard tool should not really be taking sides in the war but should support both sides. Dub supports shared objects as dependencies, but just fails to be able to build them. Until shared library support is actually designed and only then implemented, supporting it in dub any further is a lost cause.
Re: Dub support was added to Meson
On Tue, 2018-08-14 at 17:51 +, Filipe Laíns via Digitalmars-d-announce wrote: […] > > Yeah, Meson is a build system, not a package manager. This > problem can be solved by simply creating a script that fetches > and builds de dependencies. When creating a script, you might > also want to compile the dependencies against the system's > default D compiler. This way, you won't have any issues. That's the battle isn't it. Should the system that builds be the system that also manages the packages for the build (cf. Dub, Cargo, Go, Mason), or should the two be kept separate (cf. CMake, Meson, SCons, Make). The debate has been had on the D mailing lists a number of times and at great length. But in the end D, Rust, and Go are going with a single system for everything. Building for Debian/Fedora/etc. can be seen as separate from building a project independent of platform. Whilst build-only clearly works for Debian/Fedora/etc. builds, and possibly for big corporate builds, for small projects the all-in-one approach is clearly winning the battle for hearts and minds. […] > It uses 'dub describe' to find the files, this also allows us to > check the compiler. Meson won't even let you try to compile the > project if there's an compiler mismatch. I am not sure how "dub describe" tells you which compiler is being used, it always says "compiler": "dmd" on my projects even though I only use ldc2 for build. […] > I mean, you can do this but as you said, it's not recommended. Certainly it is not recommended for Meson builds but I see no reason why. It is just duplicating information that is held on the filestore, and DRY is supposed to be a thing in computing. Convention over configuration really makes things a lot easier. But I guess this debate would turn into a flame war of competing prejudices, so probably best left alone. […] > Dub is implemented this way for a reason. We really *shouldn't* > use dynamic libraries, at least for now. Maybe then Dub should be better able to handle shared objects. Whilst the whole static linking vs. shared objects is another topic that descends into flame wars of prejudice, the de facto standard tool should not really be taking sides in the war but should support both sides. Dub supports shared objects as dependencies, but just fails to be able to build them. -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
Re: Dub support was added to Meson
On Friday, 10 August 2018 at 14:38:10 UTC, Russel Winder wrote: First, you need to fetch and build the dependency. dub fetch vibe-d dub build vibe-d This feels a little unsatisfactory. Dub (and soon SCons I hope) builds handle getting the dependency as well as building it and thence linking the project to it. Meson both depends on the dub executable and yet isn't doing the getting, it is assuming already got. Yeah, Meson is a build system, not a package manager. This problem can be solved by simply creating a script that fetches and builds de dependencies. When creating a script, you might also want to compile the dependencies against the system's default D compiler. This way, you won't have any issues. I haven't finished reading the Meson source to find out which Dub built archive is used in the Meson build but if it is the one in ~/.dub/packages/-/ then stuff can go wrong since it is the result of the last build. In order to get the build with the specific compiler on the specific platform you have to delve into ~/.dub/packages/-//.dub/ and know the Dub naming strategy. For SCons I am having problems with this because of the extra stuff in the directory name. It uses 'dub describe' to find the files, this also allows us to check the compiler. Meson won't even let you try to compile the project if there's an compiler mismatch. I have been getting projects such as Unit-Threaded, DInotify, TinyEndian, and D_YAML to have Meson builds so as to create shared libraries prior to using pkg-config to handle D project being dependent on these libraries. It's working nicely (for me anyway) except Dub uses GitHub tags for versioning and Meson requires version numbers in the meson.build file. This duplication is getting quite annoying for all concerned. As is the Meson requirement to list all source files, rather than have a convention over configuration approach as Dub does – SCons has a different approach but still avoids explicit source lists. I can get round it with Meson, but I am informed it is naughty and should not be done. I mean, you can do this but as you said, it's not recommended. These builds create shared objects and allow for dynamic linking. Using Dub built archives means static linking. Unless Meson somehow manages to get shared objects out of the Dub build. Unless Dub can generate shared objects, having the dependencies build locally and creating pkg-config files actually seems a better way that accessing Dub builds. Dub is implemented this way for a reason. We really *shouldn't* use dynamic libraries, at least for now.
Re: Dub support was added to Meson
On Mon, 2018-08-13 at 14:33 +, Matthias Klumpp via Digitalmars-d-announce
wrote:
[…]
> This is intentional, see this comment from Jussi:
> https://github.com/mesonbuild/meson/pull/3592#issuecomment-390421754
I am not entirely sure I agree with everything there, but he is BDFL of the
project so his view wins.
> Unfortunately, this makes the dub feature almost useless for
> Linux distro integration.
> However, I hope we might be able to implement reading dub.json
> files that are locally installed and incorporate D sources from
> the dependency into the regular Meson build process.
I am rapidly heading this route for SCons: instead of using Dub to create the
libraries just use it to get the sources and then compile the sources as a
sub-project. However I still prefer having a Meson build for the library and
building that way. If however there is a way of using dub.sdl or dub.json that
would be interesting.
> This would mean that Meson's dlang plugin would have to implement
> quite a bit of dub and dub.json parsing, which is some effort.
> The benefit would be the creation of a near-seamless bridge
> between the dub and Meson worlds (you'd only have to locally
> install a dub package), that is also useful for Linux
> distribution packaging.
Perhaps there is a route whereby there can be a separate Python package to do
this that both Meson and SCons can use? Otherwise it risks there being
implementation of the same code for Meson and SCons separately.
[…]
> Jup, deterministic source lists are useful for split-building
> stuff efficiently. And globbing sources is possible with Meson,
> but highly discouraged (and even looks ugly).
I am not entirely convinced explicit manual lists are better, nor are they
required for consistent builds. If the source files structure changes you have
to create a new build anyway.
What is the ugly globbing way you know – I'm wondering if it better than the
hack I have.
[…]
> At the moment, I think for a final project, adding Meson files to
> dependencies still is the better way. However, for quickly
> testing out things and for building something locally, the new
> dub support is quite valuable.
> Also, we can expand it in future, this is a first step :-)
I get the feeling Meson Rust/Cargo support is easier and better than D/Dub
support, even though D seems to get more publicity in the Meson documentation.
I think a really good way forward for Meson (and analogously SCons) would be
to have the Dub dub.{json,sdl} file usable to create (and deploy if using
shared objects) a Meson (Scons) build of the dependency and then building the
project based on the created dependencies.
--
Russel.
===
Dr Russel Winder t: +44 20 7585 2200
41 Buckmaster Roadm: +44 7770 465 077
London SW11 1EN, UK w: www.russel.org.uk
signature.asc
Description: This is a digitally signed message part
Re: Dub support was added to Meson
On Friday, 10 August 2018 at 14:38:10 UTC, Russel Winder wrote: On Sat, 2018-08-04 at 16:07 +, Filipe Laíns via Digitalmars-d-announce wrote: Hello, Dub support was finally merged to the Meson's upstream. For the ones that don't know, Meson[1] is a fast build system that uses ninja[2] as a backend. Until now it was impossible to use dependencies via the Dub and many many packages only supported Dub. Now you can import dependencies that already exist in Dub's local repo. You still need to fetch and build them though. Sadly not in Meson 0.47.1 is a new release due soon? There is no fixed release schedule, but from experience, this might take a few months. First, you need to fetch and build the dependency. dub fetch vibe-d dub build vibe-d This feels a little unsatisfactory. Dub (and soon SCons I hope) builds handle getting the dependency as well as building it and thence linking the project to it. Meson both depends on the dub executable and yet isn't doing the getting, it is assuming already got. Of course Meson relies on all dependencies being present at build specification time, so nothing inconsistent, it just seems a wee bit incomplete. This is intentional, see this comment from Jussi: https://github.com/mesonbuild/meson/pull/3592#issuecomment-390421754 Unfortunately, this makes the dub feature almost useless for Linux distro integration. However, I hope we might be able to implement reading dub.json files that are locally installed and incorporate D sources from the dependency into the regular Meson build process. This would mean that Meson's dlang plugin would have to implement quite a bit of dub and dub.json parsing, which is some effort. The benefit would be the creation of a near-seamless bridge between the dub and Meson worlds (you'd only have to locally install a dub package), that is also useful for Linux distribution packaging. [...] I have been getting projects such as Unit-Threaded, DInotify, TinyEndian, and D_YAML to have Meson builds so as to create shared libraries prior to using pkg-config to handle D project being dependent on these libraries. It's working nicely (for me anyway) except Dub uses GitHub tags for versioning and Meson requires version numbers in the meson.build file. This duplication is getting quite annoying for all concerned. As is the Meson requirement to list all source files, rather than have a convention over configuration approach as Dub does – SCons has a different approach but still avoids explicit source lists. I can get round it with Meson, but I am informed it is naughty and should not be done. Jup, deterministic source lists are useful for split-building stuff efficiently. And globbing sources is possible with Meson, but highly discouraged (and even looks ugly). These builds create shared objects and allow for dynamic linking. Using Dub built archives means static linking. Unless Meson somehow manages to get shared objects out of the Dub build. Unless Dub can generate shared objects, having the dependencies build locally and creating pkg-config files actually seems a better way that accessing Dub builds. [...] At the moment, I think for a final project, adding Meson files to dependencies still is the better way. However, for quickly testing out things and for building something locally, the new dub support is quite valuable. Also, we can expand it in future, this is a first step :-)
Re: Dub support was added to Meson
On Sat, 2018-08-04 at 16:07 +, Filipe Laíns via Digitalmars-d-announce
wrote:
> Hello,
>
> Dub support was finally merged to the Meson's upstream.
> For the ones that don't know, Meson[1] is a fast build system
> that uses ninja[2] as a backend. Until now it was impossible to
> use dependencies via the Dub and many many packages only
> supported Dub. Now you can import dependencies that already exist
> in Dub's local repo. You still need to fetch and build them
> though.
Sadly not in Meson 0.47.1 is a new release due soon?
> First, you need to fetch and build the dependency.
>
>dub fetch vibe-d
>dub build vibe-d
This feels a little unsatisfactory. Dub (and soon SCons I hope) builds handle
getting the dependency as well as building it and thence linking the project
to it. Meson both depends on the dub executable and yet isn't doing the
getting, it is assuming already got.
Of course Meson relies on all dependencies being present at build
specification time, so nothing inconsistent, it just seems a wee bit
incomplete.
I haven't finished reading the Meson source to find out which Dub built
archive is used in the Meson build but if it is the one in
~/.dub/packages/-/ then stuff can go wrong since it
is the result of the last build. In order to get the build with the specific
compiler on the specific platform you have to delve into
~/.dub/packages/-//.dub/ and know the Dub naming
strategy. For SCons I am having problems with this because of the extra stuff
in the directory name.
> Then you just need to specify Dub as the dependency method.
>
>vibed_dep = dependency('vibe-d', method: 'dub')
I have been getting projects such as Unit-Threaded, DInotify, TinyEndian, and
D_YAML to have Meson builds so as to create shared libraries prior to using
pkg-config to handle D project being dependent on these libraries. It's
working nicely (for me anyway) except Dub uses GitHub tags for versioning and
Meson requires version numbers in the meson.build file. This duplication is
getting quite annoying for all concerned. As is the Meson requirement to list
all source files, rather than have a convention over configuration approach as
Dub does – SCons has a different approach but still avoids explicit source
lists. I can get round it with Meson, but I am informed it is naughty and
should not be done.
These builds create shared objects and allow for dynamic linking. Using Dub
built archives means static linking. Unless Meson somehow manages to get
shared objects out of the Dub build.
Unless Dub can generate shared objects, having the dependencies build locally
and creating pkg-config files actually seems a better way that accessing Dub
builds.
But I may be missing something!
[…]
--
Russel.
===
Dr Russel Winder t: +44 20 7585 2200
41 Buckmaster Roadm: +44 7770 465 077
London SW11 1EN, UK w: www.russel.org.uk
signature.asc
Description: This is a digitally signed message part
Re: Dub support was added to Meson
On Wednesday, 8 August 2018 at 17:26:39 UTC, Mike Wey wrote: On 07-08-18 22:33, Atila Neves wrote: How does it track dependencies given that ninja needs functionality akin to gcc's to do that? Or does it always compile everything if any file changes? It currently only tracks dependencies when using gdc, for dmd and ldc dmd pull 6961[1] would have to be merged so that support can be extended to the other compilers. So when using ldc or dmd you will currently have to call 'ninja clean && ninja' to compile everything. [1] https://github.com/dlang/dmd/pull/6961 Makes sense, thanks for the info.
Re: Dub support was added to Meson
On 07-08-18 22:33, Atila Neves wrote: How does it track dependencies given that ninja needs functionality akin to gcc's to do that? Or does it always compile everything if any file changes? It currently only tracks dependencies when using gdc, for dmd and ldc dmd pull 6961[1] would have to be merged so that support can be extended to the other compilers. So when using ldc or dmd you will currently have to call 'ninja clean && ninja' to compile everything. [1] https://github.com/dlang/dmd/pull/6961 -- Mike Wey
Re: Dub support was added to Meson
On Saturday, 4 August 2018 at 16:07:45 UTC, Filipe Laíns wrote: Hello, Dub support was finally merged to the Meson's upstream. For the ones that don't know, Meson[1] is a fast build system that uses ninja[2] as a backend. Until now it was impossible to use dependencies via the Dub and many many packages only supported Dub. Now you can import dependencies that already exist in Dub's local repo. You still need to fetch and build them though. [...] How does it track dependencies given that ninja needs functionality akin to gcc's to do that? Or does it always compile everything if any file changes? reggae has to wrap dmd to be able to use ninja for anything. I can't see any other way but to add the necessary dependencies generation as you compile to dmd itself.
Re: Dub support was added to Meson
On Saturday, 4 August 2018 at 19:01:50 UTC, Russel Winder wrote: Great stuff. I have failed to date finishing Dub support in SCons, I shall have to put more effort in now. :-) The single biggest problem with D and Meson is that unit-threaded doesn't work with the way Meson works. Which is why I introduced the ProgramAllAtOnce builder into SCons. If there is a plan to try and make D/unit-threaded/Meson all work together, count me in as a tester at the very least. I'll keep an eye out for that. Maybe I'll try to implement something if my free time allows it :) PS.: Sorry for the grammar errors in the original post (mainly the "the Dub" :facepalm:), I rewrote some of the sentences and I didn't properly read the whole post again. Thanks, Filipe Laíns
Re: Dub support was added to Meson
Great stuff. I have failed to date finishing Dub support in SCons, I shall have to put more effort in now. :-) The single biggest problem with D and Meson is that unit-threaded doesn't work with the way Meson works. Which is why I introduced the ProgramAllAtOnce builder into SCons. If there is a plan to try and make D/unit-threaded/Meson all work together, count me in as a tester at the very least. -- Russel. === Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Roadm: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk signature.asc Description: This is a digitally signed message part
