Re: For projects switching to Meson *only*

[mcatanzaro]

On Thu, Apr 27, 2017 at 7:00 PM, Peter Hutterer 
 wrote:
I will, but I'll keep the two parallel for at least a release or two. 
If you

need me to add anything specific to keep continuous happy for the time
being, let me know.

Cheers,
   Peter


Here's a request for maintainers who are supporting both meson and 
Autotools: please consider adding the meson build files to your release 
tarballs (add them to EXTRA_DIST) if you want people to actually use 
the meson build system. You don't have to, but it'd be nice.


For people helping maintain the jhbuild modulesets, please do not 
switch modules to use meson until the meson build files have appeared 
in a release tarball, or you're confident they will appear in the next 
tarball. (Unless Autotools support has already been dropped, in which 
case maintainers, please follow the GNOME release cycle in making your 
next release!) This will reduce the amount of manual hacking required 
to produce release modulesets that actually build. I've temporarily 
switched GStreamer and Grilo back to using Autotools for this reason.


Thanks,

Michael

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: For projects switching to Meson *only*

[Peter Hutterer]

On Thu, Apr 27, 2017 at 03:24:24PM +0100, Emmanuele Bassi wrote:
> Replying to both Michael's and Iain at the same time, to reduce the
> amount of email.
> 
> On 27 April 2017 at 15:04, Michael Catanzaro  wrote:
> > On Thu, Apr 27, 2017 at 4:11 AM, Iain Lane  wrote:
> >>
> >> As it happens I interacted with this script (in gnome-software)
> >> yesterday. I hadn't got a new enough jhbuild - the one I had was trying
> >> to call ./configure instead of meson directly.
> >>
> >> The script AFAICS ignores unknown arguments. In particular, I was
> >> interested in passing some --enable/--disable flags to select features
> >> but I didn't find out how to do that short of explicitly extending it
> >> with those particular options. If you expect distributors to be using
> >> this, or if this is interesting for users of the build API, is having
> >> some support for ./configure <-> meson_options integration a reasonable
> >> request?
> 
> Each module has its own set of options with their own name and
> semantics; the build-api only defines a specific subset, so if you
> want to have a `configure` script to paper over the Meson switch, you
> also get to add the options you want to port over.
> 
> Mostly because if I end up patching this script into Continuous, then
> I get to be the one who decides which options get ported over and
> which one don't; maintainers of a project are usually best suited at
> that.
> 
> For instance, the libinput maintainer has started dropping all
> auto-detection checks and now the build relies on specifying all
> options; a worthy goal, and I'd actually hope more modules followed
> suit. If he also switched to Meson-only, I'd have to write a patch for
> Continuous that manually ported over the build options as a
> compatibility layer; I would do this only for the options Continuous
> supports, though, and it would take me slightly more time than just
> copying the file over.

I will, but I'll keep the two parallel for at least a release or two. If you
need me to add anything specific to keep continuous happy for the time
being, let me know.

Cheers,
   Peter

> 
> >> Otherwise, and this is what happens now that I upgraded jhbuild, using
> >> meson directly works fine. But if a goal of this is to smooth the
> >> transition path and avoid a requirement for tooling to be updated, maybe
> >> it would be useful.
> 
> Of course.
> 
> > This compatibility issue seems like a very good argument for shipping the
> > "compatibility" script in Continuous patches rather than application
> > repositories.
> 
> As I said, this takes me (in the absolute simplest case) about 90
> seconds of my life, so I don't mind doing a patch.
> 
> I'd be happier, though, if maintainers that are planning to drop
> autotools also dropped me a line so that I don't wake up to a string
> of failed builds and then have to figure out whether or not this was a
> planned break, or just general CI failure. *Especially* if the
> maintainer also fixes the jhbuild moduleset.
> 
> So, I guess the real question is: communicate these changes
> beforehand, instead of pushing to master and then going home without
> looking at the explosion?
> 
> Ciao,
>  Emmanuele.
> 
> 
> -- 
> https://www.bassi.io
> [@] ebassi [@gmail.com]
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: For projects switching to Meson *only*

[Michael Catanzaro]

On Thu, Apr 27, 2017 at 9:24 AM, Emmanuele Bassi  
wrote:

I'd be happier, though, if maintainers that are planning to drop
autotools also dropped me a line so that I don't wake up to a string
of failed builds and then have to figure out whether or not this was a
planned break, or just general CI failure. *Especially* if the
maintainer also fixes the jhbuild moduleset.

So, I guess the real question is: communicate these changes
beforehand, instead of pushing to master and then going home without
looking at the explosion?


Yes, and my apologies for failing to do this yesterday when I switched 
over Epiphany. I even left JHBuild broken. Maintainers, please don't do 
that. :)


Michael

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: For projects switching to Meson *only*

[Emmanuele Bassi]

Replying to both Michael's and Iain at the same time, to reduce the
amount of email.

On 27 April 2017 at 15:04, Michael Catanzaro  wrote:
> On Thu, Apr 27, 2017 at 4:11 AM, Iain Lane  wrote:
>>
>> As it happens I interacted with this script (in gnome-software)
>> yesterday. I hadn't got a new enough jhbuild - the one I had was trying
>> to call ./configure instead of meson directly.
>>
>> The script AFAICS ignores unknown arguments. In particular, I was
>> interested in passing some --enable/--disable flags to select features
>> but I didn't find out how to do that short of explicitly extending it
>> with those particular options. If you expect distributors to be using
>> this, or if this is interesting for users of the build API, is having
>> some support for ./configure <-> meson_options integration a reasonable
>> request?

Each module has its own set of options with their own name and
semantics; the build-api only defines a specific subset, so if you
want to have a `configure` script to paper over the Meson switch, you
also get to add the options you want to port over.

Mostly because if I end up patching this script into Continuous, then
I get to be the one who decides which options get ported over and
which one don't; maintainers of a project are usually best suited at
that.

For instance, the libinput maintainer has started dropping all
auto-detection checks and now the build relies on specifying all
options; a worthy goal, and I'd actually hope more modules followed
suit. If he also switched to Meson-only, I'd have to write a patch for
Continuous that manually ported over the build options as a
compatibility layer; I would do this only for the options Continuous
supports, though, and it would take me slightly more time than just
copying the file over.

>> Otherwise, and this is what happens now that I upgraded jhbuild, using
>> meson directly works fine. But if a goal of this is to smooth the
>> transition path and avoid a requirement for tooling to be updated, maybe
>> it would be useful.

Of course.

> This compatibility issue seems like a very good argument for shipping the
> "compatibility" script in Continuous patches rather than application
> repositories.

As I said, this takes me (in the absolute simplest case) about 90
seconds of my life, so I don't mind doing a patch.

I'd be happier, though, if maintainers that are planning to drop
autotools also dropped me a line so that I don't wake up to a string
of failed builds and then have to figure out whether or not this was a
planned break, or just general CI failure. *Especially* if the
maintainer also fixes the jhbuild moduleset.

So, I guess the real question is: communicate these changes
beforehand, instead of pushing to master and then going home without
looking at the explosion?

Ciao,
 Emmanuele.


-- 
https://www.bassi.io
[@] ebassi [@gmail.com]
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GNOME Build situation and BuildStream

[Tristan Van Berkom]

On Thu, 2017-04-27 at 14:41 +0200, Sébastien Wilmet wrote:
> Hi Tristan,
> 

[...]
> With jhbuild, when we enter into a jhbuild shell we are still in the
> same directory, usually inside the git repository. With builddir ==
> srcdir we have all the files that we can directly open with our
> preferred text editor. When we open a new terminal tab, we are in the
> same directory where we are able to 1) do git commands 2) building
> (with
> recursive make) 3) launching executables 4) editing files, etc.
> 
> With BuildStream, will it be similar?

Hi Sébastien,

Good question :)

There are some things which will inevitably be different. I think the
most disruptive thing is that you will not have the experience of
having a single, persistent filesystem tree where the things you've
built "are".

This is because BuildStream does not have a serial build model but
rather will parallelize builds where possible; every build result is
stored in a separate "artifact", and sandboxed environments are created
on demand.

So, first of all to talk about VMs, launching a full VM is the
preferred way to:

  o Test how some software interacts in a full GNOME environment,
usually the bleeding edge of development.

  o Work on modules like GNOME Shell, GDM, GNOME Session etc which
is very difficult to isolate and work on in your host environment.


That said, today BuildStream has a `bst shell` option to stage a given
module's dependencies in a sandbox and run shell on demand.

There are two semantics for this, first of all let's assume that you
have a checkout of the GNOME build metadata (or "modulesets"), your
current working directory is at the root of that checkout, and the
module you want to hack on is called "foo".


  bst shell --scope build foo.bst
  ~~~
  Stage all of the build time dependencies for the module `foo.bst`,
  and also stage the sources for building `foo.bst`, and drop you
  into a shell in the build directory.

  Useful for debugging failed builds (however, when a build fails
  you will be presented an option to shell into the actual failed
  build sandbox anyway).


  bst shell --scope run foo.bst
  ~
  Stage all of the runtime dependencies for the module `foo.bst`,
  including the built `foo.bst` module itself, and drop you
  into a shell in that sandboxed environment.

  Useful for debugging applications to a certain degree, can
  run gdb and strace and similar things in here.

  This shell differs from the actual build sandbox because it allows
  some pass through of the host environment variables. This makes
  it possible to launch graphical applications like say, gedit.

  This will _only_ work well on systems which have a somewhat
  conforming environment, i.e. your host should be running dbus
  and you should have DBUS_SESSION_BUS_ADDRESS set in your environment,
  similarly you want to have DISPLAY in your environment.

  So essentially, launching graphical applications inside
  `bst shell --scope run foo.bst` should work only for the cases that
  it would have worked when using jhbuild, so no loss there really.


Now that part is already working, and dont worry about speed; even if
hundreds of "artifacts" need to be staged into a directory, this is
lightning fast and uses hardlinks to get it done.


But what you will be more interested in is your edit/compile/debug
cycles, for that we have a hard blocker before BuildStream can be
really efficient for the type of workflow you want; we're calling this
"Workspaces"[0].

With workspaces, you will be able to use a directory of your choosing
to build a specific module from (and you can have more than one "active
workspace" at a time, so you might open a workspace to hack on glib,
and another one to hack on GTK+, and have your local trees for both be
effective for any builds).

This is not done yet but here's an approximate mock shell of what I
think the UX would look like:

  # First get a local copy of the modulesets
  host$ git clone 
  host$ cd gnome-modulesets

  # Now lets create some workspaces
  host$ bst init-workspace glib.bst ~/glib
  host$ bst init-workspace gtk+.bst ~/gtk+

  # Open your favorite text editor, and edit
  # files directly in ~/glib and/or ~/gtk+
  #
  # Now build something, maybe we want to just test with gtk3-demo
  host$ bst build gtk+.bst

  # Lets test it
  host$ bst shell --scope run gtk+.bst

  # We're in the sandbox now
  sandbox$ gtk3-demo

  # Hmmm, why did it crash ?
  sandbox$ gdb gtk3-demo
  
  # Ah, I see what I did there...
  sandbox$ exit

  # Edit some files in ~/glib and/or ~/gtk+ and try again
  #
  host$ bst build gtk+.bst
  host$ bst shell --scope run gtk+.bst
  sandbox$ gtk3-demo
  sandbox$ exit

  # Ok that worked !
  host$ cd ~/gtk+
  host$ git commit -a -m "Its a patch !"

  # Do appropriate thing, maybe you push, maybe you
  # do `git format-patch` and post some patch
  #
  # At this point you may want to continuously leave
  # the 

Re: For projects switching to Meson *only*

[Michael Catanzaro]

On Thu, Apr 27, 2017 at 4:11 AM, Iain Lane  
wrote:

As it happens I interacted with this script (in gnome-software)
yesterday. I hadn't got a new enough jhbuild - the one I had was 
trying

to call ./configure instead of meson directly.

The script AFAICS ignores unknown arguments. In particular, I was
interested in passing some --enable/--disable flags to select features
but I didn't find out how to do that short of explicitly extending it
with those particular options. If you expect distributors to be using
this, or if this is interesting for users of the build API, is having
some support for ./configure <-> meson_options integration a 
reasonable

request?

Otherwise, and this is what happens now that I upgraded jhbuild, using
meson directly works fine. But if a goal of this is to smooth the
transition path and avoid a requirement for tooling to be updated, 
maybe

it would be useful.


This compatibility issue seems like a very good argument for shipping 
the "compatibility" script in Continuous patches rather than 
application repositories.


Michael

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GNOME Build situation and BuildStream

[Sébastien Wilmet]

Hi Tristan,

For application or library developers (libraries used by applications),
I'm a bit struggling to see what the workflow will look like with
BuildStream.

I've described two examples of my current workflow in this mail:
https://mail.gnome.org/archives/desktop-devel-list/2016-August/msg00047.html
"builddir != srcdir in jhbuild breaks my workflow"

See also:
https://mail.gnome.org/archives/desktop-devel-list/2017-February/msg00018.html
"Equivalent of recursive make with meson/ninja?"

With BuildStream you're talking about launching a VM. It's quite a big
change compared to how applications are launched with jhbuild.

So, can you describe a little more how the workflow would look like for
application developers using the terminal (not an IDE)?

With jhbuild, when we enter into a jhbuild shell we are still in the
same directory, usually inside the git repository. With builddir ==
srcdir we have all the files that we can directly open with our
preferred text editor. When we open a new terminal tab, we are in the
same directory where we are able to 1) do git commands 2) building (with
recursive make) 3) launching executables 4) editing files, etc.

With BuildStream, will it be similar?

--
Sébastien
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GNOME Build situation and BuildStream

[Tristan Van Berkom]

Hi Matthias,

I realize now that this was too much information at once (even for the
involved reader as opposed to a fly-by reader).

So I'd like to thank you for your mind share.

On Wed, 2017-04-26 at 16:39 -0400, Matthias Clasen wrote:
> Tristan,
> 
> again, it is impossible to reply to an email of this length. I can
> only give a few general comments, beyond that, we really need to sit
> down face-to-face and discuss this. I hope you are going to be at
> Guadec ?

I will certainly be around all week at GUADEC to meet with you and
anyone who wants to discuss :)

I am preparing a talk on this subject, but perhaps I should also try to
organize something more hands on, maybe a BoF or such would be good.

> My general comments:
> 
> What you are describing here (and in your previous communications)
> looks like a big, all-encompassing system, with lots of its own
> terminology and a complete worldview of how things should be built. I
> prefer a system that starts small and solves one problem initially,
> and then maybe grows over time.

I can see how it can come across this way, we are trying to break the
trend of having a build system as something that is tied to any
particular deployment/use case.

As such, I needed to give consideration to a lot of use cases to be
sure that this is something that fits, and is also an improvement over
what exists. These considerations will be reflected in my
communications and I can see how one might think this appears to be
some kind of huge monolith which does everything.

However this is exactly the opposite of what I'm trying to achieve,
instead we are striving to "do one thing well" and are making an effort
to ensure we're doing it the right way, for any use case.

So, the core codebase itself should remain small over time, with really
the sole purpose of being:

   "A format and engine for modeling and executing pipelines of 
    elements which perform mutations on filesystem data within an
    isolated sandboxed environment"

In time, I expect that an ecosystem of plugins and projects will grow
around this, and use cases I had not even foreseen will come to light.
This has already started to happen in some ways, as Jochen Breuer's
commented on my blog here:

   https://blogs.gnome.org/tvb/2017/02/06/introducing-buildstream-2/

And as a result has started to work on a plugin which would allow
importing distro packages and building on top of those bases:

   https://gitlab.com/BuildStream/buildstream/issues/10

> The system you describe seems to be all about centralization, and
> about introducing a new language to describe what we build. That is
> by-and-large what we already have in various incarnations that you
> describe: jhbuild modulesets, the continuous manifest, flatpak
> runtimes. I can get behind the idea of unifying these into a single
> way of describing a multi-module build.
> 
> But I've also seen things mentioned like 'conversion scripts for
> flatpak'. And I think that is exactly the opposite of what we need
> for application building.

I may be mistaken, but I have a feeling you are getting the same
impression which Christian had last month, which I tried to explain in
this email:

   https://mail.gnome.org/archives/desktop-devel-list/2017-March/msg3.html

> If we want to convince 3rd party applications to use flatpak, we
> can't treat it as an intermediate format that we generate from some
> other source, and just use somewhere in a centralized build process.
> We need to embrace it ourselves and use it, just like we expect 3rd
> party applications to use it.

So at the risk of being repetitive, I am completely behind application
authors maintaining their own build metadata themselves, building
flatpaks themselves and/or submitting build metadata to a "flathub" to
have them built and published to users.

Of course this makes sense, because the application authors themselves
are usually best situated to know what should be in their bundling
build metadata.

So let me try to break down how I would see this work (I realize,
already a long email):


  GNOME core modules and services (excluding Flatpak apps)
  
  These can be expressed in a single format/repository for all the
  interesting purposes:

    o Performing CI
    o Creating bootable GNOME images on top of some base system,
      mostly for developers
    o Release modulesets
    o Producing the GNOME Flatpak runtime and SDK

  This is of course, one centralization.


  Flatpak Applications
  
  Considering the benefits which the GNOME core modules and services
  get by representing build metadata for multiple purposes, a given
  application developer team may also benefit in similar ways.

  This is without centralizing everything into one big build blob, or
  using intermediate formats or anything like this.

  Now, this may be more of a personal goal, a bit more ambitious,
  maybe best punted to later on, but 

Re: For projects switching to Meson *only*

[Iain Lane]

Hiya,

Apologies for any ignorance about the scope and intent of the build API.

On Thu, Apr 27, 2017 at 09:45:04AM +0100, Emmanuele Bassi wrote:
> Hi everyone;
> A simple script is available here[2], and it has nice properties, like
> being able to work with a simple:
> 
>   ./configure
>   make
>   sudo make install
> 
> which makes distributors and integrators happy.
> […]
> [2]: https://github.com/ebassi/graphene/blob/master/configure

As it happens I interacted with this script (in gnome-software)
yesterday. I hadn't got a new enough jhbuild - the one I had was trying
to call ./configure instead of meson directly.

The script AFAICS ignores unknown arguments. In particular, I was
interested in passing some --enable/--disable flags to select features
but I didn't find out how to do that short of explicitly extending it
with those particular options. If you expect distributors to be using
this, or if this is interesting for users of the build API, is having
some support for ./configure <-> meson_options integration a reasonable
request?

Otherwise, and this is what happens now that I upgraded jhbuild, using
meson directly works fine. But if a goal of this is to smooth the
transition path and avoid a requirement for tooling to be updated, maybe
it would be useful.

Cheers,

-- 
Iain Lane  [ i...@orangesquash.org.uk ]
Debian Developer   [ la...@debian.org ]
Ubuntu Developer   [ la...@ubuntu.com ]


signature.asc
Description: PGP signature
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: For projects switching to Meson *only*

[Emmanuele Bassi]

The issue is not writing a simple patch: it takes me about 90 seconds
of my life to copy the file, create a commit, format-patch it, and add
the patch to gnome-continuous.

The whole point would be to avoid adding a patch to Continuous.

Additionally, conforming to the build-api is a nice transitional tool
for distributors that are still trying to catch up to Meson.

Ciao,
 Emmanuele.


On 27 April 2017 at 09:48, Carlos Soriano  wrote:
> Hello Emmanuele,
>
> Would be fine if the maintainer does the patch for continuous instead of
> doing the build-API?
>
> Carlos Soriano
>
>  Original Message 
> Subject: For projects switching to Meson *only*
> Local Time: 27 April 2017 10:45 AM
> UTC Time: 27 April 2017 08:45
> From: eba...@gmail.com
> To: Desktop Development List 
>
> Hi everyone;
>
> since maintainers have started switching to Meson for the development
> cycle (awesome!) I'd like to ask for a favour: if you're dropping
> autotools entirely, could you please add a `configure` wrapper script
> that conforms to the build-api[1] that GNOME Continuous uses?
>
> A simple script is available here[2], and it has nice properties, like
> being able to work with a simple:
>
> ./configure
> make
> sudo make install
>
> which makes distributors and integrators happy.
>
> Before you ask: yes, I'm looking at adding Meson support to
> Continuous, as part of a larger rework of the manifest file to adapt
> to the Flatpak builder manifest format; that will take time, though,
> so in the interim it'd be great if we didn't have to ship a patch for
> every project, when the alternative is simply dropping a script into
> the project's top-level.
>
> Ciao,
> Emmanuele.
>
> [1]: https://github.com/cgwalters/build-api
> [2]: https://github.com/ebassi/graphene/blob/master/configure
>
> --
> https://www.bassi.io
> [@] ebassi [@gmail.com]
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list
>
>



-- 
https://www.bassi.io
[@] ebassi [@gmail.com]
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: For projects switching to Meson *only*

[Carlos Soriano via desktop-devel-list]

Hello Emmanuele,

Would be fine if the maintainer does the patch for continuous instead of doing 
the build-API?

Carlos Soriano

 Original Message 
Subject: For projects switching to Meson *only*
Local Time: 27 April 2017 10:45 AM
UTC Time: 27 April 2017 08:45
From: eba...@gmail.com
To: Desktop Development List 

Hi everyone;

since maintainers have started switching to Meson for the development
cycle (awesome!) I'd like to ask for a favour: if you're dropping
autotools entirely, could you please add a `configure` wrapper script
that conforms to the build-api[1] that GNOME Continuous uses?

A simple script is available here[2], and it has nice properties, like
being able to work with a simple:

./configure
make
sudo make install

which makes distributors and integrators happy.

Before you ask: yes, I'm looking at adding Meson support to
Continuous, as part of a larger rework of the manifest file to adapt
to the Flatpak builder manifest format; that will take time, though,
so in the interim it'd be great if we didn't have to ship a patch for
every project, when the alternative is simply dropping a script into
the project's top-level.

Ciao,
Emmanuele.

[1]: https://github.com/cgwalters/build-api
[2]: https://github.com/ebassi/graphene/blob/master/configure

--
https://www.bassi.io
[@] ebassi [@gmail.com]
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

For projects switching to Meson *only*

[Emmanuele Bassi]

Hi everyone;

since maintainers have started switching to Meson for the development
cycle (awesome!) I'd like to ask for a favour: if you're dropping
autotools entirely, could you please add a `configure` wrapper script
that conforms to the build-api[1] that GNOME Continuous uses?

A simple script is available here[2], and it has nice properties, like
being able to work with a simple:

  ./configure
  make
  sudo make install

which makes distributors and integrators happy.

Before you ask: yes, I'm looking at adding Meson support to
Continuous, as part of a larger rework of the manifest file to adapt
to the Flatpak builder manifest format; that will take time, though,
so in the interim it'd be great if we didn't have to ship a patch for
every project, when the alternative is simply dropping a script into
the project's top-level.

Ciao,
 Emmanuele.

[1]: https://github.com/cgwalters/build-api
[2]: https://github.com/ebassi/graphene/blob/master/configure

-- 
https://www.bassi.io
[@] ebassi [@gmail.com]
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list