Port your module from g_memdup() to g_memdup2() now

[Philip Withnall]

Hi all,

GLib 2.67.3 will be released shortly with a new g_memdup2() API to
replace the old g_memdup() API. Please port to using g_memdup2() as
soon as possible: g_memdup() will be deprecated in GLib 2.67.4.

The old API took the size of the memory to duplicate as a guint,
whereas most memory functions take memory sizes as a gsize. This made
it easy to accidentally pass a gsize to g_memdup(). For large values,
that would lead to a silent truncation of the size from 64 to 32 bits,
and result in a heap area being returned which is significantly smaller
than what the caller expects. This can likely be exploited in various
modules to cause a heap buffer overflow.

Details here:
https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1926

GLib 2.66.6 will also be released shortly, which will fix the bug
internally in GLib. It will not export g_memdup2() as a new public API
though. You’ll need to depend on GLib ≥2.67.3 for that, or copy the
`static inline` version of g_memdup2() from here:

https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1927/diffs

Sorry for the short notice about this. Due to some oversights, the
vulnerability was made public today at the same time as we first heard
about it.

Thanks to Kevin Backhouse from GHSL for finding it and providing a
detailed report.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Reminder: action required when updating dependencies or build options

[Philip Withnall]

On Thu, 2021-01-14 at 11:57 +0100, Benjamin Berg wrote:
> On Wed, 2021-01-13 at 15:57 -0600, Michael Catanzaro wrote:
> On Wed, Jan 13, 2021 at 9:28 pm, Philip Withnall 
>  wrote:
> > Given that you’ve just committed to submitting MRs and waiting for 
> > CI
> > to pass, rather than pushing directly to master, perhaps this rule
> > should be rethought?
> 
> Hm... as long as we have permission to merge the MR after CI has 
> passed, or to bypass the CI if it's broken due to a preexisting
> issue,
> then we should be good. However, I'm not sure GitLab allows this?
> E.g. we discovered yesterday that I didn't have permission to commit
> to
> gnome-remote-desktop until Jonas played with the settings; there, I
> had
> created a MR, but only project maintainers were able to merge it.
> 
> I think, to do this you would effectively need to add the release
> team
> with Maintainer privileges to all projects (or, I suppose have a
> Release Team account for this purpose).
> 
> I suppose that should be pretty simple to hook into the .doap
> synchronisation. Not sure how people people feel about having the
> Release team marked as a maintainer everywhere.
> 
> 
> Note that I am not really concerned about anyone merging a branch
> even
> if CI is failing without having a good reason to do so.
> 
> Benjamin
> 
> The worst-case scenario would be: need to revert a commit, CI is 
> already broken due to some preexisting unrelated issue that we don't 
> know how to fix, can't land revert via MR because it requires CI to 
> pass, can't change the setting to allow bypassing CI because the only
> way to get permission to change the setting is to update the doap, 
> can't update the doap without first passing CI

I don’t know anything about what the release team is doing with all
these modules, or why, but perhaps rather than the default being “pull
in all the modules into an OS build and urgently push fixes to the
modules if something fails”, could the default be “only pull module
updates into an OS build if they don’t fail”? i.e. Essentially pinning
each module in the OS build by default, and regularly updating those
pins via a CI gate.

Then individual module maintainers can work at their own pace, and
updates (e.g. API changes) can land in different modules without
needing prior consultation with the release team.

I might be wide off the mark since I don’t know the problem space. Just
carping from the sidelines.

Philip

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Reminder: action required when updating dependencies or build options

[Philip Withnall]

On Wed, 2021-01-13 at 15:15 -0600, Michael Catanzaro wrote:
>  > And similar behaviour over the years where you seemed to be under
> the
>  > impression that I was the only person in GNOME that felt that
> peer-
>  > reviewed patches and CI-gated merge requests were a requirement.
>  >
>  > Until the main development branch is protected and it's not
> possible 
> to
>  > push directly there, I don't see myself being interested in
>  > contributing to modules where I filled the maintainer gap.
> 
> This is disappointing. If you restrict commits, please let release
> team 
> know, because we have a rule that we only build modules from git
> master 
> if we can commit to them to fix build failures. We would just need to
> switch your modules to build from tarball instead. Building core 
> software from tarball instead of git would reduce the value of GNOME
> OS 
> and the master runtime for everyone, but it is an option.

Given that you’ve just committed to submitting MRs and waiting for CI
to pass, rather than pushing directly to master, perhaps this rule
should be rethought?

Philip

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Reminder: action required when updating dependencies or build options

[Philip Withnall]

On Tue, 2021-01-12 at 09:31 -0600, Michael Catanzaro wrote:
> Hi developers,
> 
> Please remember that action is required when updating your
> dependencies 
> or build options. You need to either make sure gnome-build-meta is OK
> with your changes, or ask release team to investigate on your behalf.
> 
> We've had at least four separate breakages from four separate
> projects 
> in the past few days. This is too much. Please be careful. Thanks!

Sorry to have caused one of those breakages. I think this is going to
keep happening, though, since remembering to ping an external party
when dependencies change is hard. It’s something that happens
infrequently, and is not part of people’s normal
code/compile/review/merge cycle, so it’s not surprising they forget.

Perhaps there’s scope for a CI bot which reminds developers to look at
updating gnome-build-meta whenever a meson.build is changed. That bot
could be set up for every project which is listed in gnome-build-meta.

Or, if it’s appropriate, the bot could file an issue against gnome-
build-meta and assign the developer who’s touching meson.build to that
issue. Or something.

Philip

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Changes to GitLab runners configuration

[Philip Withnall]

On Wed, 2020-02-19 at 13:14 -0600, Michael Catanzaro wrote:
> On Wed, Feb 19, 2020 at 8:00 pm, Sam Thursfield  
> wrote:
> > We've been using podman successfully to build the Tracker CI
> > images.
> > The exact build instructions are here:
> > 
> >   
> > https://gitlab.gnome.org/GNOME/tracker-oci-images/blob/master/.gitlab-ci.yml
> > 
> > These containers are working fine in GitLab CI, see the
> > tracker.git 
> > and
> > tracker-miners.git pipelines
> 
> I'll look at that. Thanks!

Bastien fixed podman support for GLib recently, too. Take a look at 
https://gitlab.gnome.org/GNOME/glib/commit/36aacb8ae36db6352a36ffe1bbaf15c79fe41532
.

Philip

___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Maintainers should announce build-related changes in their modules

[Philip Withnall]

On Thu, 2019-09-12 at 19:14 +0100, Emmanuele Bassi wrote:
> On Thu, 12 Sep, 2019 at 19:08, Philip Withnall <
> phi...@tecnocode.co.uk> wrote:
> > That sounds like something people are going to forget to do. Would
> > it be possible to use computers to automate this?
> 
> It's software: anything is possible.
> 
> As to whether we can automate this **right now**, the answer is: no.
> 
> I'm not going to block on a feature that may or may not appear in
> Gitlab's enterprise edition and then may or may not be backported to
> the community edition we have. Of course, enterprising hackers are
> strongly encouraged to work on that.

The link to the GitLab EE issue was illustrative, not definitive. If it
solves the problem, a cronjob which polls every module’s `/meson.build`
and `/meson_options.txt` files every 30 minutes and uses sendmail to
send you an e-mail about changes would work.

> From my perspective, I don't think is much to ask to the community of
> GNOME maintainers to help us on the release team (and all the people
> that build GNOME components) in ensuring their projects actually
> build instead of deploying hopes and prayers to users.

I don’t think anyone is going to begrudge you their time, it’s just the
practicality of having ~30 humans remember to send an e-mail to a list
every time they (infrequently and rarely) touch a particular part of
their code base.

I will endeavour to remember to do so, but I can pretty much guarantee
I’ll forget at some point. I apologise in advance.

Philip
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Maintainers should announce build-related changes in their modules

[Philip Withnall]

On Thu, 2019-09-12 at 18:54 +0100, Emmanuele Bassi via desktop-devel-
list wrote:
> Hi all;
> 
> the 3.34 release is out of the door, but before we go into the 3.35
> development cycle, the release team would like to kindly ask **all**
> GNOME maintainers to send an email to release-t...@gnome.org (and
> possibly Cc: distributor-l...@gnome.org) every time their project(s)
> introduce a new dependency, or update the version requirement of
> existing dependencies, or change the build options of their
> project(s).

That sounds like something people are going to forget to do. Would it
be possible to use computers to automate this?

For example, https://gitlab.com/gitlab-org/gitlab-ee/issues/1817

Philip
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

gtester support deprecated in GLib

[Philip Withnall]

Hi all,

A quick notification: the `gtester` utility, and support for its format
for outputting results from unit tests, has been deprecated in GLib.

https://gitlab.gnome.org/GNOME/glib/issues/1441

`gtester` is a test harness, and has been de-facto deprecated for a
long time, but now it will output deprecation warnings everywhere. We
won’t drop it from GLib until we break API (which is not planned to
happen), but it’s not going to be maintained either.

Instead of `gtester`, you should be using TAP output and an external
test harness which can parse it. Unit tests now default to using TAP
(unless you run them under `gtester`):

https://gitlab.gnome.org/GNOME/glib/issues/1619

This was previously available by passing the `--tap` option to unit
tests — you now no longer need to do that (but it’s harmless to
continue doing so, for backwards compatibility).

TAP is a structured format for passing unit test results from the tests
to the test harness:

https://testanything.org/

It’s supported natively by Meson since 0.50 using the `protocol`
argument to `test()`:

https://mesonbuild.com/Reference-manual.html#test

It’s supported in autotools by using the glib-tap.mk helper, as
documented on this StackOverflow answer:

https://stackoverflow.com/a/19975406/2931197

Let me know if you have any questions.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab postmortem

[Philip Withnall]

On Sun, 2018-12-23 at 17:21 +, Philip Withnall wrote:
> On Wed, 2018-12-19 at 16:48 +0000, Philip Withnall wrote:
> > On Wed, 2018-12-19 at 14:37 +0000, Philip Withnall wrote:
> > > On Tue, 2018-12-11 at 14:22 +0100, Carlos Soriano wrote:
> > > > Hey,
> > > > It has been a few months since we moved to GitLab. Apart of
> > > > spurious issues, specific annoyances and frustrations, seems it
> > > > has been generally good. I would like to gather some general
> > > > feeling about it. Things that really made a constant impact to
> > > > you and your work, both bad or good. Feel free to provide
> > > > feedback about the transition or the administration of GitLab
> > > > instance too. Free form.
> > > 
> > > It’s all been pretty excellent. I can’t fault the transition or
> > > the effort that people have put into it.
> > > 
> > > A few larger annoyances about GitLab, having now worked with it
> > > for a while:
> 
> To follow up on these, I’ve now filed bugs for some of them. See
> below:
> 
> > >  1. Being able to draft review comments and submit them all at
> > > once would reduce e-mail overload on people, and make it easier
> > > to draft coherent code reviews. I quite like how GitHub does this
> > > (although I dislike most other things about GitHub).
> 
> Paid-for feature. ☹
> 
> > >  2. Hiding the diff of a large file when it’s the only file
> > > changed in an MR is not helpful. I should file a bug about this.
> 
> I can’t currently find the MR I saw this with, but will file an issue
> with GitLab if I see it again. I really should have filed it when I
> first saw the problem, sorry.
> 
> > >  4. Starting to type while the tag popover is loading will still
> > > execute global hotkeys, which normally refreshes the page or does
> > > something unexpected. I should file a bug about this too.
> 
> Filed https://gitlab.com/gitlab-org/gitlab-ce/issues/55679.
> 
> > >  5. Changing branches when creating an MR loses your
> > > title/description/tags, and since the branch drop-down is quite
> > > far down the form, I often forget to do that first before filling
> > > out the title/description/tags. This makes backports a bit more
> > > annoying. I should file a bug about this too.
> 
> Filed https://gitlab.com/gitlab-org/gitlab-ce/issues/55680.
> 
> > >  6. GitHub recently acquired a way to suggest minor one-line
> > > changes to MRs, and allow the MR author to press a button to
> > > accept them. This would be really good for minor typo fixes and
> > > cleanups. It would be less intrusive than having to write a
> > > nitpick comment for each one and making the MR author really
> > > bored or frustrated with the review.
> 
> Apparently we just got this with 11.6. Looking forward to trying it
> out!
> 
> > 7. The ability for a maintainer to push fixups to an old MR, or
> > rerun failed CI pipelines on it, so that we don’t have to clone MRs
> > to resurrect ones where the original author has wandered off; and
> > people don’t have to remember to tick the “allow others to push to
> > my branch” tickbox when creating an MR to allow the CI pipelines to
> > be retried.
> 
> There already seems to be an upstream issue about it: 
> https://gitlab.com/gitlab-org/gitlab-ce/issues/55680.

That was a copy/paste error. I actually meant this link for #7: 
https://gitlab.com/gitlab-org/gitlab-ce/issues/49323
Sorry,Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab postmortem

[Philip Withnall]

On Wed, 2019-01-02 at 15:15 +0100, Milan Crha via desktop-devel-list
wrote:
> On Wed, 2018-12-19 at 14:37 +0000, Philip Withnall wrote:
> > 3. I’d like to see continued movement towards disallowing direct
> > pushes to git, and requiring all commits to go through MRs (and
> > CI).
> 
>   Hi,
> I hope this won't go through without a good research and reasoning.

I only care that the option is available for each maintainer to
disallow direct pushes; not that the policy is applied to all modules.

> Any such requirement would be kind of showstopper for me personally.
> It
> would be just double work for me when coding (issue is not merge
> request), causing awful commit history, resulting in unbelievably
> harder effort required to produce NEWS file content (yes, I do use
> commit messages to fill the NEWS files in a semi-automated way saving
> my time, which I can spend on more productive things). To name a few
> major obstacles at least.

I’ve just written this script for generating a NEWS entry from `git
log` plus information from GitLab, which you might find useful:

https://gitlab.gnome.org/pwithnall/gitlab-changelog

Philip



signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab postmortem

[Philip Withnall]

On Wed, 2018-12-19 at 16:48 +, Philip Withnall wrote:
> On Wed, 2018-12-19 at 14:37 +0000, Philip Withnall wrote:
> > On Tue, 2018-12-11 at 14:22 +0100, Carlos Soriano wrote:
> > > Hey,
> > > It has been a few months since we moved to GitLab. Apart of
> > > spurious issues, specific annoyances and frustrations, seems it
> > > has been generally good. I would like to gather some general
> > > feeling about it. Things that really made a constant impact to
> > > you and your work, both bad or good. Feel free to provide
> > > feedback about the transition or the administration of GitLab
> > > instance too. Free form.
> > 
> > It’s all been pretty excellent. I can’t fault the transition or the
> > effort that people have put into it.
> > 
> > A few larger annoyances about GitLab, having now worked with it for
> > a while:

To follow up on these, I’ve now filed bugs for some of them. See below:

> >  1. Being able to draft review comments and submit them all at once
> > would reduce e-mail overload on people, and make it easier to draft
> > coherent code reviews. I quite like how GitHub does this (although
> > I dislike most other things about GitHub).

Paid-for feature. ☹

> >  2. Hiding the diff of a large file when it’s the only file changed
> > in an MR is not helpful. I should file a bug about this.

I can’t currently find the MR I saw this with, but will file an issue
with GitLab if I see it again. I really should have filed it when I
first saw the problem, sorry.

> >  4. Starting to type while the tag popover is loading will still
> > execute global hotkeys, which normally refreshes the page or does
> > something unexpected. I should file a bug about this too.

Filed https://gitlab.com/gitlab-org/gitlab-ce/issues/55679.

> >  5. Changing branches when creating an MR loses your
> > title/description/tags, and since the branch drop-down is quite far
> > down the form, I often forget to do that first before filling out
> > the title/description/tags. This makes backports a bit more
> > annoying. I should file a bug about this too.

Filed https://gitlab.com/gitlab-org/gitlab-ce/issues/55680.

> >  6. GitHub recently acquired a way to suggest minor one-line
> > changes to MRs, and allow the MR author to press a button to accept
> > them. This would be really good for minor typo fixes and cleanups.
> > It would be less intrusive than having to write a nitpick comment
> > for each one and making the MR author really bored or frustrated
> > with the review.

Apparently we just got this with 11.6. Looking forward to trying it
out!

> 7. The ability for a maintainer to push fixups to an old MR, or rerun
> failed CI pipelines on it, so that we don’t have to clone MRs to
> resurrect ones where the original author has wandered off; and people
> don’t have to remember to tick the “allow others to push to my
> branch” tickbox when creating an MR to allow the CI pipelines to be
> retried.

There already seems to be an upstream issue about it: 
https://gitlab.com/gitlab-org/gitlab-ce/issues/55680.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab postmortem

[Philip Withnall]

On Wed, 2018-12-19 at 14:37 +, Philip Withnall wrote:
> On Tue, 2018-12-11 at 14:22 +0100, Carlos Soriano wrote:
> > Hey,
> > It has been a few months since we moved to GitLab. Apart of
> > spurious issues, specific annoyances and frustrations, seems it has
> > been generally good. I would like to gather some general feeling
> > about it. Things that really made a constant impact to you and your
> > work, both bad or good. Feel free to provide feedback about the
> > transition or the administration of GitLab instance too. Free form.
> 
> It’s all been pretty excellent. I can’t fault the transition or the
> effort that people have put into it.
> 
> A few larger annoyances about GitLab, having now worked with it for a
> while:
> 
>  1. Being able to draft review comments and submit them all at once
> would reduce e-mail overload on people, and make it easier to draft
> coherent code reviews. I quite like how GitHub does this (although I
> dislike most other things about GitHub).
> 
>  2. Hiding the diff of a large file when it’s the only file changed
> in an MR is not helpful. I should file a bug about this.
> 
>  3. I’d like to see continued movement towards disallowing direct
> pushes to git, and requiring all commits to go through MRs (and CI).
> I think the remaining barrier to this is the translation
> workflow/Damned Lies. It would be good to get Damned Lies to create
> MRs with submitted translation changes, so that it doesn’t need
> direct push access to git, and so that translators don’t have to faff
> with MRs themselves.
> 
>  4. Starting to type while the tag popover is loading will still
> execute global hotkeys, which normally refreshes the page or does
> something unexpected. I should file a bug about this too.
> 
>  5. Changing branches when creating an MR loses your
> title/description/tags, and since the branch drop-down is quite far
> down the form, I often forget to do that first before filling out the
> title/description/tags. This makes backports a bit more annoying. I
> should file a bug about this too.
> 
>  6. GitHub recently acquired a way to suggest minor one-line changes
> to MRs, and allow the MR author to press a button to accept them.
> This would be really good for minor typo fixes and cleanups. It would
> be less intrusive than having to write a nitpick comment for each one
> and making the MR author really bored or frustrated with the review.

7. The ability for a maintainer to push fixups to an old MR, or rerun
failed CI pipelines on it, so that we don’t have to clone MRs to
resurrect ones where the original author has wandered off; and people
don’t have to remember to tick the “allow others to push to my branch”
tickbox when creating an MR to allow the CI pipelines to be retried.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab postmortem

[Philip Withnall]

On Tue, 2018-12-11 at 14:22 +0100, Carlos Soriano wrote:
> Hey,
> It has been a few months since we moved to GitLab. Apart of spurious
> issues, specific annoyances and frustrations, seems it has been
> generally good. I would like to gather some general feeling about it.
> Things that really made a constant impact to you and your work, both
> bad or good. Feel free to provide feedback about the transition or
> the administration of GitLab instance too. Free form.

It’s all been pretty excellent. I can’t fault the transition or the
effort that people have put into it.
A few larger annoyances about GitLab, having now worked with it for a
while:
 1. Being able to draft review comments and submit them all at once
would reduce e-mail overload on people, and make it easier to draft
coherent code reviews. I quite like how GitHub does this (although I
dislike most other things about GitHub).
 2. Hiding the diff of a large file when it’s the only file changed in
an MR is not helpful. I should file a bug about this.
 3. I’d like to see continued movement towards disallowing direct
pushes to git, and requiring all commits to go through MRs (and CI). I
think the remaining barrier to this is the translation workflow/Damned
Lies. It would be good to get Damned Lies to create MRs with submitted
translation changes, so that it doesn’t need direct push access to git,
and so that translators don’t have to faff with MRs themselves.
 4. Starting to type while the tag popover is loading will still
execute global hotkeys, which normally refreshes the page or does
something unexpected. I should file a bug about this too.

 5. Changing branches when creating an MR loses your
title/description/tags, and since the branch drop-down is quite far
down the form, I often forget to do that first before filling out the
title/description/tags. This makes backports a bit more annoying. I
should file a bug about this too.

 6. GitHub recently acquired a way to suggest minor one-line changes to
MRs, and allow the MR author to press a button to accept them. This
would be really good for minor typo fixes and cleanups. It would be
less intrusive than having to write a nitpick comment for each one and
making the MR author really bored or frustrated with the review.

Ta,Philip




signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: [GitLab] Fast CI is now available for all projects under gitlab.gnome.org

[Philip Withnall]

Great, thanks to Packet.net, OSU and the sysadmin/GitLab team! Let’s
test more things.
On Fri, 2018-11-16 at 13:29 +0100, Carlos Soriano wrote:
> We still have other donations that are very useful to us, as we
> really shouldn't rely on a single or only a few sources. So still,
> many thanks to GitLab Inc, AWS, Canonical, and the individual
> contributors with the Windows and MacOSX machines giving it for free
> as shared runners.
> 
> On Fri, 16 Nov 2018 at 13:24, Carlos Soriano 
> wrote:
> > So that's it, we got fast CI for all projects now, including GNOME/
> > World/ and all forks.
> > This is thanks to donations from Packet.net and Oregon State
> > University Open Source Lab. We will check if they want some PR and
> > do so soon. We will also update https://wiki.gnome.org/GitLab/CI
> > with the info soon. 
> > 
> > Thanks to Rob, Javier and Andrea for the ideas and set up!
> 
> ___desktop-devel-list
> mailing listdesktop-devel-l...@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Heads up: potential hash table ordering changes in GLib

[Philip Withnall]

Hi all,

As a heads up, we’ve just merged some performance improvements to
GHashTable, done by Hans Petter Jansson:

https://gitlab.gnome.org/GNOME/glib/merge_requests/208
https://gitlab.gnome.org/GNOME/glib/issues/1198
https://hpjansson.org/blag/2018/08/29/what-ails-ghashtable/
https://hpjansson.org/blag/2018/07/24/a-hash-table-re-hash/

These may change the hash table iteration order. If you are using
GHashTable incorrectly by relying on it to iterate over its entries in
a particular order, this may break your code.

You should fix this by either making your code independent of order, or
sorting the entries once you’ve extracted them from the GHashTable (and
before your order-dependent iteration over them).

This is not a bug in GLib.

To the best of my knowledge, we last changed iteration order in 2008,
so some code written since then may break.

Have fun,
Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

GLib BoF session at GUADEC

[Philip Withnall]

There will be a GLib BoF session all day on the 9th of July in room 2
of GUADEC, subverting part of the GTK+ BoF:

https://wiki.gnome.org/GUADEC/2018/Hacking%20days/GtkBOF

Come along if you’re interested in GLib, GIO, GObject. We’ll be
assessing the status of the 2.58 release, and starting to plan for
2.60.

Ta,
Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab CI runners for non-Linux systems

[Philip Withnall]

On Wed, 2018-05-30 at 00:59 +0800, 藍挺瑋 wrote:
> 於 星期一，2018-05-28 於 12:09 -0400，xclae...@gmail.com 提到：
> > Hi,
> > 
> > We now have 6 arch tested for glib, all with Meson.
> > 
> > - macosx-10.13-meson-x86_64
> >   * Native macosx 10.13 build
> >   * Fails to build with --werror, if anyone wants to take a look.
> >   * Some unit tests fails and are ignored, if anyone wants to take
> > a
> > look.
> >   * Machine currently hosted at Collabora Montreal office, will
> > setup
> > another runner on another machine we have in a proper data center.
> 
> It looks like this one was temporarily removed.
> 
> > Missing, if someone wants to contribute:
> >  - Some autotools runners? IMHO (I'm not maintainer) we should
> > recommend meson for 2.58 and drop autotools for 2.60.
> >  - Some 32 bits archs?
> >  - More macosx/android/windows/distros versions?
> >  - *BSD
> 
> After spending a few days on fixing GLib tests, I finally get all
> tests pass on
> FreeBSD: https://gitlab.gnome.org/lantw/glib/-/jobs/38840
> 
> The test ran in a jail with FreeBSD 11.1-RELEASE-p10 userland on
> FreeBSD 11.2-
> BETA3 amd64 kernel.
> 
> In fact, 'collate' test fails but it is marked as 'SKIP' because of
> lacking
> en_US locale, 'testfilemonitor' is modified to allow some missing
> events because
> kqueue isn't as good as inotify on file monitoring. Even if not all
> tests work
> perfectly, running these tests already let me find a few bugs and fix
> problems
> in GKqueueFileMonitor. The code and commits in my GLib fork are just
> for quick
> testing and I will make them look better before submitting a merge
> request.
> 
> However, the runner I use in my GLib fork runs in a VM whose host is
> very
> unreliable and not suitable for use as an official CI runner. This
> host crashes
> often and its uptime is usually less than 2 weeks. It also gives me
> random
> segfault and corrupted files sometimes. I can document steps required
> to run
> GitLab CI runner on FreeBSD, so people who are able to provide
> machines can
> setup a runner quickly. It is easy and can be done in about 5
> commands.

What are those 5 commands? I think it should be possible to use a
separate Docker image for the FreeBSD CI, but run it on the same Docker
hosts as the Fedora image which we use for the Linux CI and
crossbuilds.

See https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#defin
e-image-and-services-from-gitlab-ci-yml.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Migration of GNOME to GitLab is now completed

[Philip Withnall]

On Tue, 2018-05-29 at 19:00 +0200, Carlos Soriano wrote:
> Hello all,
> 
> I'm glad to announce that the GNOME migration to GitLab is now
> completed!

Big thanks Carlos! And thank you too to everyone else who helped out,
including the sysadmins and the original team who looked at code
hosting options (Alberto, Allan and others). GitLab has already made my
life a lot easier.

Philip
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab CI runners for non-Linux systems

[Philip Withnall]

On Wed, 2018-05-23 at 10:03 -0700, Alan Coopersmith wrote:
> On 05/23/18 09:40 AM, Philip Withnall wrote:
> > On Wed, 2018-05-23 at 16:19 +0200, Christoph Reiter wrote:
> > > On Mon, May 21, 2018 at 5:54 PM, Alan Coopersmith
> > > <alan.coopersm...@oracle.com> wrote:
> > > > On 05/18/18 02:52 AM, Philip Withnall wrote:
> > > > > Can anybody else provide and maintain CI runners for other
> > > > > platforms?
> > > > 
> > > > Are there instructions somewhere on what's needed to do this?
> > > 
> > > 1) Install the gitlab runner on a server:
> > > https://docs.gitlab.com/runner/#install-gitlab-runner
> > 
> > I’m not sure of the latest status of this, but it looks like gitab-
> > runner might need some porting work to Solaris to be done before it
> > can
> > be used. There’s some information about it here:
> > 
> > https://gitlab.com/gitlab-org/gitlab-runner/issues/1764
> > 
> > > 2) Fork glib on gitlab and register your runner in the forked
> > > repo
> > > settings
> > > 3) Adjust the .gitlab-ci.yml file to add a job using your runner.
> > > See
> > > Xavier's recent patch for
> > >   macos for what's roughly needed there:
> > > https://bugzilla.gnome.org/show_bug.cgi?id=796343
> > 
> > Additionally, once all that is working, file an ‘infrastructure’
> > issue
> > (like https://gitlab.gnome.org/Infrastructure/GitLab/issues/141) to
> > get
> > a CI token for the runner to be used by GitLab.
> 
> Thanks - I'm not going to have time until after we get Solaris 11.4
> shipped, but hopefully later this year I'll have some time to see if
> we can get a Solaris runner set up in the Oracle cloud.

Sure. Whenever you get time, it would be appreciated (and I hope it
would reduce your porting workload, in the long run). Let me know when
you pick this up, or if you have any questions then.

Thanks,
Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab CI runners for non-Linux systems

[Philip Withnall]

On Wed, 2018-05-23 at 16:19 +0200, Christoph Reiter wrote:
> On Mon, May 21, 2018 at 5:54 PM, Alan Coopersmith
> <alan.coopersm...@oracle.com> wrote:
> > On 05/18/18 02:52 AM, Philip Withnall wrote:
> > > Can anybody else provide and maintain CI runners for other
> > > platforms?
> > 
> > Are there instructions somewhere on what's needed to do this?
> 
> 1) Install the gitlab runner on a server:
> https://docs.gitlab.com/runner/#install-gitlab-runner

I’m not sure of the latest status of this, but it looks like gitab-
runner might need some porting work to Solaris to be done before it can
be used. There’s some information about it here:

https://gitlab.com/gitlab-org/gitlab-runner/issues/1764

> 2) Fork glib on gitlab and register your runner in the forked repo
> settings
> 3) Adjust the .gitlab-ci.yml file to add a job using your runner. See
> Xavier's recent patch for
>   macos for what's roughly needed there:
> https://bugzilla.gnome.org/show_bug.cgi?id=796343

Additionally, once all that is working, file an ‘infrastructure’ issue
(like https://gitlab.gnome.org/Infrastructure/GitLab/issues/141) to get
a CI token for the runner to be used by GitLab.

Thanks, and please keep me in the loop!
Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab CI runners for non-Linux systems

[Philip Withnall]

On Sat, 2018-05-19 at 20:41 +1000, Jonathan Matthew wrote:
> On Fri, May 18, 2018 at 10:52:18AM +0100, Philip Withnall wrote:
> > 
> > Can anybody else provide and maintain CI runners for other
> > platforms?
> > I’d particularly like to see:
> >  • *BSD (probably OpenBSD and NetBSD)
> >  • macOS (ideally several versions, since we support from OS X 10.7
> > upwards[2])
> >  • Android (probably a cross-build)
> >  • More Windows configurations (currently we have MSYS2 on Windows
> > Server 2012; ideally we’d have a MinGW-w64 runner too)
> 
> I can do OpenBSD and SmartOS, assuming I can get the gitlab CI runner
> to
> work on either of them.

That would be great, thanks. Here are the instructions Christoph
provided elsewhere in the thread about how to set things up, in case
you haven’t tried this before:

1) Install the gitlab runner on a server:
https://docs.gitlab.com/runner/#install-gitlab-runner
2) Fork glib on gitlab and register your runner in the forked repo
settings
3) Adjust the .gitlab-ci.yml file to add a job using your runner. See
Xavier's recent patch for
  macos for what's roughly needed there:
https://bugzilla.gnome.org/show_bug.cgi?id=796343

Once all that is working, file an ‘infrastructure’ issue (like https://
gitlab.gnome.org/Infrastructure/GitLab/issues/141) to get a CI token
for the runner to be used by GitLab.

Thanks,
Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab CI runners for non-Linux systems

[Philip Withnall]

On Fri, 2018-05-18 at 09:21 -0400, philip.chime...@gmail.com wrote:
> On Fri, May 18, 2018 at 5:52 AM Philip Withnall <phi...@tecnocode.co.
> uk> wrote:
> > Can anybody else provide and maintain CI runners for other
> > platforms?
> > I’d particularly like to see:
> >  • *BSD (probably OpenBSD and NetBSD)
> >  • macOS (ideally several versions, since we support from OS X 10.7
> > upwards[2])
> >  • Android (probably a cross-build)
> >  • More Windows configurations (currently we have MSYS2 on Windows
> > Server 2012; ideally we’d have a MinGW-w64 runner too)
> 
> I can help write the CI job configurations for macOS, but I don't
> know how to host or set up a runner.
> 
> (For a shortcut solution, we could consider farming out the macOS
> builds to Travis CI, which has macOS runners already available)

Xavier has now provided a configuration here:
https://bugzilla.gnome.org/show_bug.cgi?id=796343

It looks like work is needed to get all the unit tests passing. I know
there are some unreviewed patches in Bugzilla for fixing various things
on OS X (they’re on my radar). Help with those, and with fixing other
failures from bug #796343, would be appreciated!

---

Copying the instructions Christoph provided for configuring a runner
from elsewhere in the thread, for anyone who’s interested:

1) Install the gitlab runner on a server:
https://docs.gitlab.com/runner/#install-gitlab-runner
2) Fork glib on gitlab and register your runner in the forked repo
settings
3) Adjust the .gitlab-ci.yml file to add a job using your runner. See
Xavier's recent patch for
  macos for what's roughly needed there:
https://bugzilla.gnome.org/show_bug.cgi?id=796343

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab CI runners for non-Linux systems

[Philip Withnall]

On Sun, 2018-05-20 at 00:34 -0400, xclae...@gmail.com wrote:
> Ok, spent way too much time on this, but finally got it working to
> cross build glib on our fedora docker image.
> 
> https://bugzilla.gnome.org/show_bug.cgi?id=796264

Fantastic, thanks. I will review this (and your other CI submissions)
as soon as I can.

> Similar setup could be done to cross build for windows using mingw.
> Tests could maybe run with exe_wrapper=wine, that would be fun :P

That would indeed be cool to try, if anyone is interested in
bootstrapping and supporting that as a supported platform. I honestly
don’t know whether that would be a little, or a lot, of work.

Philip

> Le vendredi 18 mai 2018 à 19:16 +0530, Arun Raghavan a écrit :
> > On 18 May 2018 at 18:51,  <philip.chime...@gmail.com> wrote:
> > > On Fri, May 18, 2018 at 5:52 AM Philip Withnall <philip@tecnocode
> > > .c
> > > o.uk>
> > > wrote:
> > > > 
> > > > Can anybody else provide and maintain CI runners for other
> > > > platforms?
> > > > I’d particularly like to see:
> > > >  • *BSD (probably OpenBSD and NetBSD)
> > > >  • macOS (ideally several versions, since we support from OS X
> > > > 10.7
> > > > upwards[2])
> > > >  • Android (probably a cross-build)
> > > >  • More Windows configurations (currently we have MSYS2 on
> > > > Windows
> > > > Server 2012; ideally we’d have a MinGW-w64 runner too)
> > > 
> > > 
> > > I can help write the CI job configurations for macOS, but I don't
> > > know how
> > > to host or set up a runner.
> > > 
> > > (For a shortcut solution, we could consider farming out the macOS
> > > builds to
> > > Travis CI, which has macOS runners already available)
> > 
> > If anyone can point me to how to set up an Android runner I could
> > try
> > to pitch in there.
> > 
> > Cheers,
> > Arun
> > ___
> > desktop-devel-list mailing list
> > desktop-devel-list@gnome.org
> > https://mail.gnome.org/mailman/listinfo/desktop-devel-list
> 
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

GitLab CI runners for non-Linux systems

[Philip Withnall]

Hi all,

tl;dr: Want to provide us with a GitLab CI runner for a non-Linux
platform?

There’s been a surge of interest recently, from various directions, in
getting GLib better tested on non-Linux architectures. This is great,
and we’ve got various people to thank for doing the thankless work of
porting and testing. Particularly:
 • macOS: Ryan Schmidt, Patrick Griffis, Michael Lauer, John Ralls
 • Windows/MinGW/MSYS2: LRN, Christoph Reiter, Xavier Claessens, Chun-
wei Fan
 • Android: Xavier Claessens
 • *BSD: Ting-Wei Lan

There’s a real risk of this all bitrotting, though, since the main GLib
developers only have regular access to Linux machines (and don’t have
the bandwidth or expertise to regularly test on other architectures).

The solution here is CI. Christoph Reiter has already got a Windows
MSYS2 CI runner set up on GitLab[1], which is producing helpful results
for GLib.

Can anybody else provide and maintain CI runners for other platforms?
I’d particularly like to see:
 • *BSD (probably OpenBSD and NetBSD)
 • macOS (ideally several versions, since we support from OS X 10.7
upwards[2])
 • Android (probably a cross-build)
 • More Windows configurations (currently we have MSYS2 on Windows
Server 2012; ideally we’d have a MinGW-w64 runner too)

After the initial setup and porting work, I hope these wouldn’t require
much effort to maintain. The fast turnaround on testing GLib on non-
Linux platforms would help us ensure cross-platform compatibility
during development.

Thanks,
Philip

[1]: https://gitlab.gnome.org/creiter/gitlab-ci-win32-runner
[2]: https://wiki.gnome.org/Projects/GLib/SupportedPlatforms

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Dropping python 2.7 support in gtk-doc (requiring python 3.X)

[Philip Withnall]

On Fri, 2018-03-30 at 13:24 +0200, Stefan Sauer wrote:
> To also give some extra motivation - Since a few month I am working
> on
> gtkdoc-mkhtml which is a pure python drop-in replacement for
> gtkdoc-mkhtml+gtkdoc-fixxref. This is not using docbook xslt anymore
> and
> it won't shell out to tools to do syntax highlighting. This new tool
> would need some changes to work with python 2.7 too.
> 
> I'll write more about gtkdoc-mkhtml2 soon, especially how you can
> test
> this with your modules, but if you have questions please ask (on
> gtk-doc-list@ or in #gtkdoc in irc).

I don’t have any questions; I just want to say that this sounds
fantastic!

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Let's kill gnome-common!

[Philip Withnall]

On Tue, 2018-02-13 at 10:08 +0100, Carlos Soriano wrote:
> Hi Michael,
> 
> Could you give some context and explanation on this? I could take the
> gnome-autoar, but need to know why this is wanted and the alternative
> to gnome-common.

Handily all written up a while ago:

https://wiki.gnome.org/Projects/GnomeCommon/Migration

This was written up before Meson was a thing; porting the build systems
of those modules to Meson is another legitimate way to port away from
gnome-common. It would be the better choice in the long run for modules
we are going to be maintaining for a while.

Philip

> On 13 February 2018 at 02:08,  wrote:
> > Hi,
> > 
> > I want to remove gnome-common from our BuildStream projects, but a
> > few modules are still depending on it: gcr, gnome-autoar,
> > libnotify, adwaita-icon-theme, gnome-menus, and gsettings-desktop-
> > schemas.
> > 
> > If you help fix these sad modules, you'll earn the right to say
> > that you helped fix these sad modules.
> > 
> > Thanks,
> > 
> > Michael
> > 
> > ___
> > desktop-devel-list mailing list
> > desktop-devel-list@gnome.org
> > https://mail.gnome.org/mailman/listinfo/desktop-devel-list
> 
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: About lib version and git

[Philip Withnall]

On Wed, 2018-01-17 at 11:49 +0100, Daniel Garcia Moreno wrote:
> On Tue, Jan 16, 2018 at 9:35 PM, Philip Withnall <phi...@tecnocode.co
> .uk> wrote:
> > On Tue, 2018-01-16 at 12:17 -0600, Federico Mena Quintero wrote:
> > > It doesn't really matter when you do it.  There are some
> > important
> > > things:
> > >
> > > * Make sure the new header files are parallel-installable with
> > the
> > > old
> > > ones.
> > >
> > > * Make sure the soname in the library changes.  Grep for
> > LT_VERSION
> > > here: https://wiki.gnome.org/MaintainersCorner/Releasing -
> > although
> > > I'm
> > > not sure how this works in Meson.
> > >
> > > Just to avoid apps that use the old API to inadvertently try to
> > > compile/link with the new one.
> > 
> > There is a documentation page covering this in its entirety:
> > 
> > https://developer.gnome.org/programming-guidelines/unstable/paralle
> > l-in
> > stallability.html.en
> > 
> > Pay attention to all of it; if there’s only one part of your
> > library
> > which is not parallel installable, the whole thing is effectively
> > not
> > parallel installable.
> 
> I've changed the version number and the build names so now for the
> next version all goes with a different name, so I think that this
> will allow different versions of libgepub running in the same system.
> 
> https://gitlab.gnome.org/GNOME/libgepub/commit/d721c7ebba040b935d3c8e
> 0456ccf5a4a674e531
> 
> These are all files that will install libgepub:
> 
> local
> local/lib
> local/lib/libgepub-0.6.so
> local/lib/girepository-1.0
> local/lib/girepository-1.0/Gepub-0.6.typelib
> local/lib/libgepub-0.6.so.0.0.0
> local/lib/libgepub-0.6.so.0
> local/lib/pkgconfig
> local/lib/pkgconfig/libgepub-0.6.pc
> local/share
> local/share/gir-1.0
> local/share/gir-1.0/Gepub-0.6.gir
> local/include
> local/include/libgepub-0.6
> local/include/libgepub-0.6/gepub-widget.h
> local/include/libgepub-0.6/gepub-archive.h
> local/include/libgepub-0.6/gepub-text-chunk.h
> local/include/libgepub-0.6/gepub-doc.h
> local/include/libgepub-0.6/gepub.h
> 
> With this change, anyone that uses libgepub should specify the
> version in his code and link with -lgepub-0.6 for example.
> 
> Is this the recommended way to do that?

Well, if you install that in parallel with the old version of the
library on a system, will any of the files conflict? :-)

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: About lib version and git

[Philip Withnall]

On Tue, 2018-01-16 at 12:17 -0600, Federico Mena Quintero wrote:
> It doesn't really matter when you do it.  There are some important
> things:
> 
> * Make sure the new header files are parallel-installable with the
> old
> ones.
> 
> * Make sure the soname in the library changes.  Grep for LT_VERSION
> here: https://wiki.gnome.org/MaintainersCorner/Releasing - although
> I'm
> not sure how this works in Meson.
> 
> Just to avoid apps that use the old API to inadvertently try to
> compile/link with the new one.

There is a documentation page covering this in its entirety:

https://developer.gnome.org/programming-guidelines/unstable/parallel-in
stallability.html.en

Pay attention to all of it; if there’s only one part of your library
which is not parallel installable, the whole thing is effectively not
parallel installable.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

g_object_ref() now propagates types

[Philip Withnall]

Hi all,

We just landed a patch in GLib which propagates the type from the
argument of g_object_ref() to its return type:

https://git.gnome.org/browse/glib/commit/?id=3fae39a5d
https://bugzilla.gnome.org/show_bug.cgi?id=790697

The idea here is that it will catch invalid implicit casts which the
compiler otherwise wouldn’t have noticed because g_object_ref()
previously operated entirely on gpointers. This will eliminate a whole
class of bugs: bugs which are unlikely to exist, but are a complete
pain to track down if they do.

The downside of this is that some legitimate implicit casts may now
cause compiler warnings with -Wincompatible-pointer-types. For example,
in the situation below, a warning will be introduced since parent_type
isn’t guaranteed to be an instance of child_type:

ParentType *parent_type;
ChildType *child_type;

child_type = g_object_ref (parent_type);

To fix this warning, first double-check that parent_type is actually
guaranteed to always be an instance of child_type at runtime, and then
change the ref to:

child_type = CHILD_TYPE (g_object_ref (parent_type));

That will add a compile-time explicit cast, and a runtime type check.
(As always, the runtime type check is disabled if GLib is built without
debugging enabled (or with G_DISABLE_CAST_CHECKS defined.)

Note that the new behaviour requires GCC, and is only enabled if you
have defined GLIB_VERSION_MAX_ALLOWED >= GLIB_VERSION_2_56.

If anybody encounters any problems with this, please comment on the bug
report:

https://bugzilla.gnome.org/show_bug.cgi?id=790697

Thanks,
Philip
on behalf of the GLib maintainership cabal

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Meson support in Continuous

[Philip Withnall]

On Fri, 2017-12-01 at 14:39 +, Emmanuele Bassi wrote:
> On 29 November 2017 at 21:12, Emmanuele Bassi 
> wrote:
> 
> > The build bot is currently busy rebuilding the whole of the
> > manifest,
> > and I'm going to babysit it for a little while, so don't start
> > removing build-api wrappers or patches from the gnome-continuous
> > repository just yet; once the build runs through, I'll give the all
> > clear.
> 
> All clear; you can start removing the `configure` build API wrappers
> from your projects, if you have them.

But if you do so, please make sure your module configuration is up to
date (using Meson) in jhbuild, or builds will break for people.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: gtk-doc output options

[Philip Withnall]

On Sun, 2017-11-12 at 17:08 +0100, Stefan Sauer wrote:
> Since people regular illy complain about the slowness of gtk-doc I'd
> like to evaluate some options. First the slowness is due to using the
> docbook-xsl stylesheets to produce html/pdf and optionally man pages.
> One main to get a huge speedup would be to drop the whole docbook
> toolchain and generate the html output directly from the comment
> blocks
> in the code. A nice side effect would be much higher portability.
> 
> Below is a link to a survey to learn how important pdf/man support
> would
> be. Please note that man support means outputting one man-page for
> each
> refentry. This is not about dropping output of man-pages for PROGRAM
> sections (though we need to rethink the tooling here). Please fill
> the
> poll, it will be up until end of this year.
> 
> http://www.polljunkie.com/poll/tiazer/gtkdoc-output-formats
> 
> Getting there is quite a bit of work. I am working on coming up with
> a
> smooth transition plan. For questions, feel free to ask, come to
> #gtkdoc
> to discuss etc.

Thanks for putting time and thought into this, Stefan. I appreciate
your continued efforts on gtk-doc.

Perhaps the poll could be clarified by pointing to some example output
in each format (HTML, PDF, man). Pointing to some man output,
especially, would help, since I suspect many people aren’t familiar
with what’s available there.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab status update

[Philip Withnall]

On Fri, 2017-09-08 at 18:11 +0200, Carlos Soriano wrote:
> Hey Philip,
> 
> Glad you like it! :)
> 
> A question from the ignorance, does two factor authentication plays a
> role if you use LDAP? If not, probably it's not very useful for the
> contributors here around, since we are supposed to use LDAP and our
> GNOME account.

I have no idea. I guess I’ve missed the memo, because I ended up
registering without using my GNOME LDAP account. Do I win a prize for
falling into that trap?

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitLab status update

[Philip Withnall]

On Mon, 2017-09-04 at 15:42 +0200, Carlos Soriano wrote:
> Hello all,
> 
> I'm happy to announce our GitLab instance at https://gitlab.gnome.org
> /GNOME/ is now officially open to host GNOME projects!

Yay! Thanks to everyone who’s worked on this. :-)

A reminder that gitlab supports 2-factor authentication, and it would
be a good idea to enable it on your account when you register on
gitlab.gnome.org.

You can do so at https://gitlab.gnome.org/profile/two_factor_auth.

It supports standard TOTP apps, and U2F devices (if your browser
supports them; there’s a Firefox extension for that here: https://addon
s.mozilla.org/en-GB/firefox/addon/u2f-support-add-on/).

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Updating GNOME Goals?

[Philip Withnall]

On Thu, 2017-08-24 at 09:20 +0200, Daniel Mustieles García wrote:
> Hi,
> 
> Some additional comments:
> 
> - /RemoveMarkupInMessages: it makes no sense to set a goal for this,
> because a fixed module today might change tomorrow, introducing new
> strings with markup. Maybe should be set as a guideline (and the goal
> moved to the Expired section)

Doesn’t mean it can’t be a goal. The benefit of doing it as a goal
means that it can be done on loads of modules at the same time in the
cycle (to fit in with translation work).

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GNOME Initiatives

[Philip Withnall]

On Tue, 2017-08-22 at 18:50 +, Sriram Ramkrishna wrote:
> Looks like GNOME Initiatives could use with some updating as well. 
> There are some that are current and some that might need to be
> revisited.  For instance:
> 
> Document Centric GNOME
> Memory Reduction

Memory reduction would be a good one to revisit and update. I was
looking at that in frustration today as my machine hit swap and an
Intel driver bug took down my desktop session as a result. It looks
like we’ve gone sufficiently long without focusing on reducing memory
usage that there should be some good low-hanging fruit there.

(For example, it seems like half the shell search providers persist in
memory long (indefinitely?) after a shell search is done, at about 50MB
RSS each.)

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Proposal to deploy GitLab on gnome.org

[Philip Withnall]

On Tue, 2017-05-16 at 14:22 +0100, Allan Day wrote:
> In recent months we have got together to examine the possibilities
> for GNOME’s development infrastructure. We’ve spent a lot of time on
> this, because we want the community to have faith in our conclusions.
> If you are interested in this, you can read our research on the wiki
> [1].

I’m excited for this. I think my main concerns have already been raised
by others, but I’ll raise them as a +1:
 - Dependencies between issues (the workaround of listing them all in a
comment isn’t very satisfying, but will do in the interim if we are
sure a proper solution is coming)
 - First-class support for a fast-forward merge workflow, including not
losing comments when doing a force-push to an MR
 - Not breaking any old links when migrating cgit/Bugzilla
bugs/Bugzilla attachments

With those concerns in mind, I’m enthusiastic for gitlab.

I would also be supportive of a solution using Phabricator+cgit. Phab
for task management and patch review, since its task management is more
powerful than gitlab’s, and its patch review workflow doesn’t have the
problems of gitlab’s branch-and-merge approach (its inter-diff reviews
are great). cgit for viewing the repositories, as normal.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: About maintenance of Bijiben

[Philip Withnall]

Hey,

On Fri, 2017-03-03 at 06:09 +, Jonathan Kang wrote:
> Michael Catanzaro mentioned that Bijiben(GNOME Notes) is the state of
> lacking
> maintenance in his post[1]. And I'd like to spend some time to make
> it a better
> application(at least one that CAN be released with GNOME release
> circle).
> 
> Currently, I don't know where to start. If you have any idea or know
> which bug reports
> of Bijiben should be fixed in a very high priority. Please let me
> know.

That’s good to hear!

I posted a patch set which made various code and build system fixes a
while ago. Probably best to review and merge it before you do too much
else to the code base (or end up fixing any of the same issues).

https://bugzilla.gnome.org/show_bug.cgi?id=762648

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GitHub Development Platform for GNOME

[Philip Withnall]

On Mon, 2017-04-10 at 14:09 -0400, Nicolas Dufresne wrote:
> Le lundi 10 avril 2017 à 01:25 +0530, Nirbheek Chauhan a écrit :
> > On Mon, Apr 10, 2017 at 1:14 AM, Walter Vargas  > co
> > m> wrote:
> > > I want to share my humble opinion and thoughts about
> > > GitHub/GitLab:
> > > 
> > 
> > From what I've been hearing, people within GNOME have been
> > evaluating
> > the possibility of running our own GitLab instance, so I would wait
> > and see what the results of their testing is.
> 
> And we need not to forget that a lot of the freedesktop community [0]
> projects are moving to Phabricator (even though it does not come with
> an easy patch submission mechanism).

There’s git-phab, which is pretty good. You do need to install it
though (`pip3 install git-phab`).

If you don’t install git-phab, the patch upload process is basically
the same as Bugzilla: `git format-patch …` then attach it to a form and
submit.

Phabricator explicitly doesn’t support pull requests, and there’s some
justification for that in nudging people towards code review: https://s
ecure.phabricator.com/phame/post/view/766/write_review_merge_publish_ph
abricator_review_workflow/ (written by one of the Phabricator authors).

Phabricator’s patch review system is unsurpassed (in my experience of
GitHub, GitLab, Phabricator and Bugzilla) in its support for patch
sets, inter-diffs, and tracking review comments through multiple
iterations of a review. It’s beautiful. I think I’m in love.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GNOME goal candidates

[Philip Withnall]

On Wed, 2017-03-01 at 13:40 +, Emmanuele Bassi wrote:
> On 1 March 2017 at 13:26, Michael Catanzaro <mcatanz...@gnome.org>
> wrote:
> > It sounds like most everyone else supports installed tests. OK,
> > then.
> > 
> > On Wed, 2017-03-01 at 10:22 +, Philip Withnall wrote:
> > > I agree that developers need to be engaged with adding more unit
> > > tests
> > > and code coverage if such a goal is to be useful. I wonder if
> > > making
> > > it
> > > a goal would kick-start some people to do that? I don’t think we
> > > can
> > > ever expect the majority of maintainers to care about (or have
> > > enough
> > > time to care about) code coverage and unit testing — but GNOME
> > > goals
> > > have been useful catalysts in the past. I guess a suitably well
> > > publicised and tutorialised blog post would work just as well
> > > though.
> > > 
> > 
> > This is the other thing. The goals should be achievable, something
> > we
> > can look at in a year or two and say "all apps meet the goal" and
> > close
> > it, not a longstanding epic that stays open forever. The installed
> > tests and coverage goals do not really qualify. Even though more
> > tests
> > are definitely desirable, I don't think it's reasonable to use the
> > GNOME Goals project for this, even if it would be nice to see as
> > many
> > projects as possible adopting it.
> > 
> > Maybe I am being too negative here. It does seem odd to say that
> > doing
> > something desirable should not be a goal. But a longstanding pie-
> > in-
> > the-sky project is very different from existing goals. Switching to
> > g_timeout_add_seconds() or adding a GtkHeaderBar are quick tasks
> > that
> > all apps should be able to accomplish easily. Adding a
> > comprehensive
> > testsuite, not so much. And adding just one or two installed tests,
> > while a good starting point, is not very useful on its own.
> 
> At some point, Gnome Goals become "best practices for GNOME projects"
> — especially because new projects should conform to these goals by
> default.
> 
> I'm all for taking all the present and past GNOME Goals pages on the
> wiki and turning them into "Best Practices for GNOME projects" —
> where
> applicable. Additionally, every cycle we can evaluate where we are on
> the completion of every goal, and if the completion rate passes a
> certain threshold we simply close the goal and move the page to the
> "best practices" section.

+1, although I think such documentation should go in gnome-devel-docs,
rather than on the wiki. Cross-referencing it and finding it is a lot
easier in gnome-devel-docs.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GNOME goal candidates

[Philip Withnall]

On Wed, 2017-03-01 at 07:26 -0600, Michael Catanzaro wrote:
> It sounds like most everyone else supports installed tests. OK, then.
> 
> On Wed, 2017-03-01 at 10:22 +0000, Philip Withnall wrote:
> > I agree that developers need to be engaged with adding more unit
> > tests
> > and code coverage if such a goal is to be useful. I wonder if
> > making
> > it
> > a goal would kick-start some people to do that? I don’t think we
> > can
> > ever expect the majority of maintainers to care about (or have
> > enough
> > time to care about) code coverage and unit testing — but GNOME
> > goals
> > have been useful catalysts in the past. I guess a suitably well
> > publicised and tutorialised blog post would work just as well
> > though.
> > 
> 
> This is the other thing. The goals should be achievable, something we
> can look at in a year or two and say "all apps meet the goal" and
> close
> it, not a longstanding epic that stays open forever. The installed
> tests and coverage goals do not really qualify. Even though more
> tests
> are definitely desirable, I don't think it's reasonable to use the
> GNOME Goals project for this, even if it would be nice to see as many
> projects as possible adopting it.
> 
> Maybe I am being too negative here. It does seem odd to say that
> doing
> something desirable should not be a goal. But a longstanding pie-in-
> the-sky project is very different from existing goals. Switching to
> g_timeout_add_seconds() or adding a GtkHeaderBar are quick tasks that
> all apps should be able to accomplish easily. Adding a comprehensive
> testsuite, not so much. And adding just one or two installed tests,
> while a good starting point, is not very useful on its own.

Porting a module to installed-tests is a fairly quick job. I’m not
suggesting the goal includes “write more tests” in its description. If
a module doesn’t have any tests already, then it doesn’t need porting
to the installed-tests infrastructure (instead, the infrastructure
should be added when tests *are* added to that module in future). The
advantage of porting everything to the infrastructure now is that all
new tests are then installed-tests by default, and hence appear in
Continuous, etc.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GNOME goal candidates

[Philip Withnall]

Hi,

On Tue, 2017-02-28 at 17:12 -0600, Michael Catanzaro wrote:
> Hi,

*snip*

> GtkBuilder validation looks like more gook to add to our Automake
> files, when we really want less gook there. Even if it's only a small
> amount of code, I'd rather it be implemented as an autoconf archive
> macro and re-proposed. I'm not sure if it's really necessary anymore
> anyway, since GTK+ almost always warns about XML problems at runtime,
> right?

Who cares how much ‘gook’ we have in the build system? What we care
about is how useful it is. The value of adding validation for files at
build time is that it catches errors *at build time*, not at runtime if
a certain code path is taken. For GtkBuilder files, the usefulness of
this depends entirely on the project — if the project uses a single
massive .ui file, any errors in that are going to be caught when the
program is started. But if a project uses a separate .ui file for each
dialogue, you have to test every dialogue in the program at runtime
before you know all the .ui files are valid.

This is a textbook example of the tradeoff between build time and
runtime testing.

The example rules given on the goal page are not the tidiest. There is
a simpler way to do this:

https://git.gnome.org/browse/hitori/tree/Makefile.am#n95 (four lines)
https://git.gnome.org/browse/hitori/tree/configure.ac#n51 (two lines)

I don’t think that needs to be shipped out to an autoconf-archive macro
— including such a macro in a project would be two lines at the least,
so it would save a total of four lines. This one is better off being
cargo-culted.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: GNOME goal candidates

[Philip Withnall]

On Tue, 2017-02-28 at 18:05 -0600, Michael Catanzaro wrote:
> Ah yeah, don't cancel your plans for nautilus.
> 
> Regarding coverage. Most of our modules are core modules; we have a
> lot
> of them. I don't think we have the resources right now to write tests
> and obtain high coverage for more than a couple of these modules,
> unfortunately. I'd like to keep the GNOME goals manageable and
> achievable. You should still go ahead and add coverage support to
> nautilus, though. It's an excellent way to improve quality.

I agree that developers need to be engaged with adding more unit tests
and code coverage if such a goal is to be useful. I wonder if making it
a goal would kick-start some people to do that? I don’t think we can
ever expect the majority of maintainers to care about (or have enough
time to care about) code coverage and unit testing — but GNOME goals
have been useful catalysts in the past. I guess a suitably well
publicised and tutorialised blog post would work just as well though.

> Regarding installed tests. The benefits of installed tests versus
> make
> check tests are not very clear to me. I don't think it should be
> necessary to install the tests in order to be able to run and test
> applications. That indicates a failure somewhere in the test
> infrastructure.

The comparison of installed-tests and `make check` is given on the goal
page:

https://wiki.gnome.org/Initiatives/GnomeGoals/InstalledTests#Issues_wit
h_.22make_check.22

Briefly:
 • installed-tests give a more consistent test environment, eliminating
spurious test passes or failures due to differences in developers’
laptop configurations
 • installed-tests allows reverse-dependency testing: find test
failures from new versions of libraries your project depends on,
without having to rebuild your project (useful in a CI environment)
 • Much easier to integrate into a system like gnome-continuous
 • Allows for integration tests as well as just unit tests

Importantly, there seems to be a misconception that it’s “installed-
tests *or* `make check`”. That’s not true. Adding support for
installed-tests to a module doesn’t mean `make check` goes away or
becomes less useful.

I’m strongly in favour of adding installed-tests to our modules. With
the glib-tap.mk helper file, it’s pretty trivial. Before accepting it
as a goal, though, the wiki page should be updated to give a howto on
adding installed-tests support to a module.

This is the commit I did for libgdata a while ago. It’s probably not
perfect, but it’s an indication of what needs to be done:

https://git.gnome.org/browse/libgdata/commit/?id=802fad78

Note that installed-tests also fits in nicely with distribution testing
efforts like Debian’s autopkgtest:

http://packaging.ubuntu.com/html/auto-pkg-test.html

i.e. It’s a case of adding the following two files to a package:

https://git.apertis.org/cgit/rhosydd.git/tree/debian/tests/control
https://git.apertis.org/cgit/rhosydd.git/tree/debian/tests/gnome-deskto
p-testing

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Documentation for build system setup

[Philip Withnall]

On Thu, 2017-02-09 at 08:52 -0600, Michael Catanzaro wrote:
> On Thu, 2017-02-09 at 15:12 +0100, Sebastian Geiger (Lanoxx) wrote:
> > One more question:
> > 
> > The sample autogen.sh on https://wiki.gnome.org/Projects/GnomeCommo
> > n/
> > Migration also explicitly calls aclocal like this:
> > 
> > aclocal --install || exit 1
> > 
> > But the man page for autoreconf says that autoreconf also calls
> > aclocal, is this redundant? If so, can it be removed?
> 
> I think it can indeed be removed if you're not using any of the other
> -ize programs (glib-gettextize, intltoolize, gtkdocize).

Why should it be necessary to call aclocal if you’re using the -ize
programs? All they do is copy files into place, which shouldn’t depend
on any other files (or aclocal.m4) being in place, right?

Giving this a spin with a project which currently uses `aclocal;
intltoolize; autoreconf`, intltoolize seems to need aclocal to create
$srcdir/m4, but nothing else. *facepalm*

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Documentation for build system setup

[Philip Withnall]

Looks fine to me, thanks.

On Thu, 2017-02-09 at 16:14 +0100, Sebastian Geiger (Lanoxx) wrote:
> I have just updated the wiki, please let me know if something I
> changed
> is not right:
> 
> https://wiki.gnome.org/Projects/GnomeCommon/Migration
> 
> 
> On 09/02/17 15:03, Emmanuele Bassi wrote:
> > On 9 February 2017 at 13:57, Sebastian Geiger (Lanoxx) <lanoxx@gmx.
> > net> wrote:
> > > What confuses me is that this paragraph starts with the phrase
> > > "it does
> > > not need any modifications made".
> > > 
> > > > It does not need any modifications made, unless you need to run
> > > > another tool before configure, or do not use one of glib-
> > > > gettextize,
> > > > gtkdocize or intltoolize. (Note that you should not use both
> > > > glib-gettextize and intltoolize in the same module, and it is
> > > > better
> > > > to use neither; see the FAQ entry below for details.)
> > > 
> > > It might be more helpful to provide a autogen.sh listing that
> > > omits both
> > > glib-gettextize and intltoolize and then add a paragraph
> > > explaining to
> > > add these for project that use either of these tools. If I
> > > understand
> > > right both a deprecated anyway and should not be needed for
> > > state-of-the-art projects.
> > > 
> > > Maybe write something like this:
> > > 
> > > If you are still using the deprecated glib-gettextize, then add
> > > the following line immediately before
> > > autoreconf:
> > > 
> > > glib-gettextize --force --copy || exit 1
> > > 
> > > If you are still using the almost obsolete intltoolize, then add
> > > the following line immediately before
> > > autoreconf:
> > > 
> > > intltoolize --force --copy --automake || exit 1
> > 
> > Those look like good changes, indeed. Feel free to update the wiki.
> > 
> > Ciao,
> >  Emmanuele.
> > 
> > > On 09/02/17 11:45, Philip Withnall wrote:
> > > > Can we please standardise on the autogen.sh given in
> > > > https://wiki.gnome.org/Projects/GnomeCommon/Migration#autogen.s
> > > > h ?
> > > > 
> > > > It does everything we need, and meets all the standards (like
> > > > build-
> > > > api). If you’ve got improvements to make, please make them on
> > > > that
> > > > page.
> > > > 
> > > > Sebastian: the paragraph immediately above the example does say
> > > > that
> > > > you should remove one or both of the intltoolize/glib-
> > > > gettextize calls
> > > > as appropriate.
> > > 
> > > ___
> > > desktop-devel-list mailing list
> > > desktop-devel-list@gnome.org
> > > https://mail.gnome.org/mailman/listinfo/desktop-devel-list
> > 
> > 
> 
> ___
> desktop-devel-list mailing list
> desktop-devel-list@gnome.org
> https://mail.gnome.org/mailman/listinfo/desktop-devel-list

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Documentation for build system setup

[Philip Withnall]

On Thu, 2017-02-09 at 10:32 +, Emmanuele Bassi wrote:
> Hi;
> 
> On 9 February 2017 at 09:56, Sebastian Geiger (Lanoxx)  t> wrote:
> 
> > It would be great to have an additional entry on this
> > page that is maybe named "build automation" or "building the
> > application" and that contains
> > some information about how to setup the build system.
> 
> I'm not entirely sure GNOME should have a section on best practices
> for the build system you may or may not be using. GNOME doesn't
> really
> mandate *how* you build your application or library code, as long as:
> 
>   - if it's an application
> - it installs a desktop file under XDG_DATA_DIRS/applications
> - it installs a DBus service file in the DBus service directory
> - it installs an appdata XML file
>   - if it's a library
> - it installs a pkg-config file under PKG_CONFIG_PATH
> - the pkg-config file contains the appropriate compiler and
> linker
> flags to find the headers and shared objects
> 
> All of these best practices are really general ones, for any build
> system
> 
> If you want to learn the best practices for autotools, I can strongly
> recommend reading the "Autotools Mythbuster" book:
> 
>   https://autotools.io/index.html
> 
> > I know we already have a bunch of good documentation on the wiki,
> > for example:
> > 
> >  https://wiki.gnome.org/Initiatives/GnomeGoals/ModernAutotools
> >  https://wiki.gnome.org/Initiatives/GnomeGoals/NicerBuilds
> >  https://wiki.gnome.org/Projects/GnomeCommon/Migration
> 
> These are mostly goals for GNOME projects hosted on git.gnome.org —
> and they are "nice to have", not strictly mandatory.

It would be good if the first two were consolidated a bit. They
probably also need updating, since they’re now quite old.

> > However it seems to me that none of these pages offers
> > a really up to date information and consistent summary
> > of how a typical gnome application should be setup
> > today.
> 
> Again, the Autotools Mythbuster book is probably what you're looking
> for.
> 
> > A few example questions for which I would hope to find answers in
> > the documentation are:
> > 
> > 1. What are best practices when setting up autogen.sh, is there are
> > recommended template
> > to start with. I know the GnomeCommon migration guide lists
> > one, but
> > I did not find it useful.
> > In particular because it contains calls to both glib-gettextize 
> > and
> > intltoolize and the page later
> > mentions that exactly this should not be done.
> 
> Your autogen.sh should only call `autoreconf`; that will take care of
> things like libtoolize and gettextize.
> 
> If you are writing a library and using gtk-doc, then you'll need to
> call `gtkdocize` prior to `autoreconf`.
> 
> There are a couple of additional things for helping non-srcdir builds
> and a autobuilders like Continuous:
> 
>  * switch to "$srcdir" before calling anything that will need to
> modify the srcdir, e.g. autoreconf and gtkdocize
>  * if your autogen.sh wants to helpfully call the generated configure
> script, then check if the NOCONFIGURE environment variable is set
> first
> 
> The smallest autogen.sh I can think of is:
> 
> ```sh
> test -n "$srcdir" || srcdir=`dirname "$0"`
> test -n "$srcdir" || srcdir=.
> 
> olddir=`pwd`
> cd "$srcdir"
> 
> autoreconf --force --install --verbose || exit $?
> 
> cd "$olddir"
> test -n "$NOCONFIGURE" || exec "$srcdir/configure" "$@"
> ```

Can we please standardise on the autogen.sh given in
https://wiki.gnome.org/Projects/GnomeCommon/Migration#autogen.sh ?

It does everything we need, and meets all the standards (like build-
api). If you’ve got improvements to make, please make them on that
page.

Sebastian: the paragraph immediately above the example does say that
you should remove one or both of the intltoolize/glib-gettextize calls
as appropriate.

> > 2. What are best practices regarding builddir!=srcdir builds? Are
> > there
> > any special steps required when using jhbuild, when I want it to do
> > out-of-tree builds?
> 
> Making sure that autogen.sh switches to srcdir before modifying it is
> enough.
> 
> When writing Makefile.am rules, always remember to include paths
> under
> top_builddir if you're including a path from top_srcdir; also
> remember
> that if you're generating files, their source is in srcdir, but their
> results are in builddir.

If you want JHBuild to build with srcdir ≠ builddir, set checkoutroot,
buildroot and prefix to different paths. For example, I have them set
to:

checkoutroot = '/opt/gnome/source'
buildroot = '/opt/gnome/build'
prefix = '/opt/gnome/install'

https://developer.gnome.org/jhbuild/stable/config-reference.html.en

> > 3. What are some recommended macros from autoconf-archive that
> > should be
> > used?
> 
> This depends entirely on what you're trying to achieve.
> 
> The recommendations are usually for compiler arguments checks and
> code
> coverage for test suites. Appdata and Yelp have their own 

Re: Tracker as a security risks

[Philip Withnall]

Hey,

On Mon, 2016-12-05 at 16:42 +0100, Carlos Garnacho wrote:
> On Mon, Dec 5, 2016 at 3:01 PM, Hanno Böck  wrote:
> > On Mon, 5 Dec 2016 13:44:40 +
> > Sam Thursfield  wrote:
> > 
> > > The design of Tracker takes the risks into account. Metadata
> > > extraction is isolated in its own process (tracker-extract) which
> > > can
> > > crash without (theoretically) causing any harm.
> > 
> > I don't see how that helps against security vulnerabilities.
> > 
> > Having an isolated process probably helps in a way that a crash
> > won't
> > cause the whole tracker service to malfunction. Thus parsing broken
> > files won't cause a service disruption. But as long as this process
> > runs with normal user rights this doesn't protect in a security
> > sense.
> > 
> > > > I think there needs to be a wider discussion about this and the
> > > > fundamental design choices done here need to be questioned.
> > > 
> > > What questions do you have in particular?
> > 
> > Quite frankly, I don't claim to have all the answers here, that's
> > why I
> > formulated it in an open "needs discussion" way.
> > 
> > I think sandboxing the tracker parser (which you already indicated
> > in your mail) is probably the most reasonable way to go forward.
> > This
> > isn't exactly my area of expertise, so I can't comment on which
> > technique here is most promising.
> 
> It indeed sounds possible to lift extraction into a separate process
> with limited access to the filesystem, we essentially need to pass an
> fd to mmap() and an output one to receive sparql back. There's just
> two things to consider:
> 
> - The extraction process sometimes needs access to misc files (eg.
> CUE
> files, XMP sidecar files, ...), those might be passed along too, but
> then we need detecting those cases beforehand.
> 
> - Ideally we wouldn't spawn one process per file being extracted,
> although if we go to defcon 1 level of paranoia, that's probably what
> should happen.

I would suggest a single sandboxed extraction process, which has read-
only access to the whole of ~/, and write access to the Tracker
database. No network access. That means that regardless of whether or
how the extraction process gets compromised, it cannot compromise the
integrity of any of the files in your home directory (except the
Tracker database, which I assume people aren’t too precious about), and
it can’t compromise the confidentiality of any of your data (except by
leaking it through the Tracker database — can we assume the database
format is sufficiently prescribed to be able to prevent this?).

This should be easily accomplished by using an AppArmor (or
equivalently, SELinux) profile for tracker-extract. seccomp-bpf could
also be used to achieve much the same thing, regardless of whether an
LSM is enabled.

The Apertis project has such an AppArmor file already, and it would be
great if that were pulled upstream:

https://git.apertis.org/cgit/packaging/tracker.git/tree/debian/apparmor
.d/usr.lib.tracker

(I am not claiming this profile is perfect, but it’s a start.)

> Anyway, this goes IMHO too much on the technical side for this ML, we
> already have https://bugzilla.gnome.org/show_bug.cgi?id=764786 filed
> to Tracker, and it's already high in my list for fixing on 1.12, feel
> free to join there.
> 
> And I should add... Tracker is not alone here, if it's not Tracker
> stumbling on infected content, with varying but still rather low
> levels of interaction it may be a thumbnailer, a previewer like
> sushi,
> or the web browser itself streaming content which hit this. So
> there's
> more places in need of further isolation when dealing with untrusted
> content.
> 
> And still, the chain is only as strong as its weakest link, as soon
> as
> there is anything opening that file with wide enough permissions to
> cause any harm, you're essentially screwed. This might sound like an
> argument to running every app through flatpak, although I think the
> long term answer always is "fix the vulnerability!".

Agreed. Thumbnailers are another big target here.

> > The other issue I think is that the quality of huge parts of the
> > foss
> > ecosystem needs to be improved. The good news here is that we got
> > some
> > powerful tools in terms of fuzzing (afl, libfuzzer) and memory
> > safety
> > bug detection (asan) in the past years. Ideally all free software
> > devs
> > should be aware of those tools and use them in their development
> > process. I'm trying to help here where I can, see e.g. also my
> > recent
> > post on this list [1]. If our libraries would be better tested we
> > could
> > be more comfortable feeding it with untrusted inputs.
> 
> I agree some more active prevention would be positive, sounds like
> something to tackle in the libraries dealing with file formats
> though,
> Tracker is a strawman here, in the sense that filesystem extraction
> it's only exploitable through its tracker-extract's modules, and
> those
> are 

Re: Proposal for Gnome Goal (was Re: Switching from Autotools to CMake for core evolution products)

[Philip Withnall]

On Mon, 2016-10-10 at 10:49 +0200, Sebastian Geiger (Lanoxx) wrote:
> On 05/10/16 15:39, Michael Biebl wrote:
> 
> > 
> > As much as I hate autotools and its arcane syntax, it does bring
> > uniformity and consistency.
> > Atm I'm counting waf (for some non-core modules), autotools, cmake and
> > some are discussing to use meson/ninja.
> > 
> > So while I'm not tied to autotools, I would hate to see if every
> > modules maintainer chooses his/her own build system of choice. This
> > makes it really cumbersome as downstream/integrator.
> Maybe it would make sense to introduce an official Gnome Goal that 
> encourages every module maintainer to switch over to
> meson.
> 
> This would of course mean that we already know meson will be the build 
> system of choice and that it fits the
> needs of all modules. Of course individual module maintainers would 
> still be free to make a different choice or to stick with autotools but 
> we would at least have something to motivate the migration, to track its 
> status across all modules and it would be a push to towards some level 
> of consistency. Also we could collect some best practices from modules 
> that have already done the conversion on a Gnome Goal page.

I don’t think we’ve ported enough modules as testbeds yet. Meson is too new
to jump into encouraging everyone to port GNOME modules en-masse.

Maybe the goal could be proposed in 6 months once Meson has matured a bit
more and more modules have dogfooded it?

Philip
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Testing for memory safety issues with Address Sanitizer

[Philip Withnall]

Hi,

On Mon, 2016-09-19 at 12:03 +0200, Hanno Böck wrote:
> I recently reported a couple of bugs to GNOME-components that were
> easily discovered with a feature called Address Sanitizer (asan):
> https://bugzilla.gnome.org/show_bug.cgi?id=762417 (glib)
> https://bugzilla.gnome.org/show_bug.cgi?id=762493 (glib)
> https://bugzilla.gnome.org/show_bug.cgi?id=768441 (gnome-session)
> 
> The first two are out of bounds reads in glib that get triggered by
> running the test suite with "make check". The third is a buffer
> overflow in gnome-session that gets triggered on every startup.

Nice work!

> Things that could be done, by increasing complexity:
> 
> * Run the existing test suites with address sanitizer
> This could be mostly automated, it's usually as simple as doing
> something like:
> ./configure CFLAGS="-fsanitize=address -g -fno-common
> -U_FORTIFY_SOURCE" CXXFLAGS="-fsanitize=address -g -fno-common
> -U_FORTIFY_SOURCE" LDFLAGS="-fsanitize=address -g -fno-common
> -U_FORTIFY_SOURCE"
> make
> ASAN_OPTIONS="log_path=/var/log/asan/asan-error" make check

It’s something which would be good to have running as part of CI, along
with the other -fsanitize options (address, thread, leak, undefined) and
things like -D_FORTIFY_SOURCE=1.

Unfortunately, these flags are mostly mutually exclusive, which means that we’d 
need 5 new CI pipelines.

I guess the right approach is to set up a few more parallel instances of 
gnome-continuous, each running with different compiler options. I’ll have a 
play with that tomorrow. I guess eventually it could be integrated into 
gnome-continuous as a new set of tree suffixes.

> * Run GNOME itself and its applications with asan enabled.
> Test suites are fine, but they don't replace real testing, i.e.
> actually running the software.
> Ideally one would built the full software stack with address sanitizer
> enabled to find bugs both in the applications themselve and in
> libraries.

Automated tests complement manual testing, and require a lot less effort
over time. (While it would be lovely in theory, in practice, very few
maintainers are going to recompile each of their projects with 5 different
sets of compiler options and manually test every feature in the UI before a
release — that’s a huge amount of work.)

Enabling asan in the jhbuild configuration by default would be one option,
with the idea that people would then develop using it by default. However,
asan interferes with Valgrind, and I think that Valgrind is probably the
more useful tool to have working in a development environment like jhbuild.
Additionally, this means consciously prioritising asan over any other
-fsanitize or hardening flags (which is fine, but it is a tradeoff).

> * Use Fuzzing.
> This is especially interesting for everything that is parsing complex
> data formats.
> There has been a lot of improvement in fuzzing techniques lately and
> advanced fuzzing tools like american fuzzy lop or libfuzzer can greatly
> help in improving and securing code.
> 
> I have some introductions to these things on the fuzzing project
> webpage:
> https://fuzzing-project.org/tutorials.html

Not really related to asan, but good advice all the same. afl is pretty
amazing.

> I discussed this already with Tobias Mueller recently and he already
> started finding a couple of issues with asan:
> 
> https://bugzilla.gnome.org/show_bug.cgi?id=771607
> https://bugzilla.gnome.org/show_bug.cgi?id=771608
> https://bugzilla.gnome.org/show_bug.cgi?id=771609
> https://bugzilla.gnome.org/show_bug.cgi?id=771610
> https://bugzilla.gnome.org/show_bug.cgi?id=771612
> https://bugzilla.gnome.org/show_bug.cgi?id=771298
> https://bugzilla.gnome.org/show_bug.cgi?id=771289

I should reiterate that I think doing testing like this manually is a trap.
For this to be useful, it needs to be automated — that will rely on each
module having good test coverage, but that’s a good goal to aim for anyway.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: g_log_structured() and introspection

[Philip Withnall]

Hey,

On Sat, 2016-08-13 at 01:49 -0400, Ray Strode wrote:
> > 
> > During the talk on GLib's new structured logging API at GUADEC
> > today,
> > it was pointed out that g_log_structured() (and the rest of the
> > interesting bits of the new structured logging API) are not
> > introspectable.
> > 
> > I'm not sure what we can do about this. The API is based around
> > GLogField, which is basically a pointer and a length, and hence is
> > not
> > introspectable:
> > 
> > struct _GLogField
> > {
> >   const gchar *key;
> >   gconstpointer value;
> >   gssize length;
> > };
> 
> One idea:
> 
> 1) g_log_variant (GLogLevelFlags log_level, GVariant *variant);
> 
> The variant would have to be of type G_VARIANT_TYPE_VARDICT
> 
> In javascript for instance it could look like this:
> 
> fields = { 'MESSAGE': new GLib.Variant('s', 'checking for optional
> sweep modulation'),
>   'GLIB_DOMAIN': new GLib.Variant('s', 'Foo'),
>   'FOO_STATE': new GLib.Variant('s',
> JSON.stringify(fooState)),
>   'FOO_AGE': new GLib.Variant('u', 3) };
> 
> GLib.log_variant(GLib.LogLevelFlags.level_debug, new
> GLib.Variant('a{sv}', fields))
> 
> which is a little wordy, but not more awful than how dbus is done.
> Then Gjs could ship a string only convenience
> api like g_log_structured by doing:
> 
> GLib.log_structured = function(logDomain, logLevel, stringFields) = {
>    fields = {};
>    for (let key in stringFields) {
>  fields[key] = new GLib.Variant('s', stringFields[key]);
>    }
>    fields['GLIB_DOMAIN'] = new GLib.Variant('s', logDomain);
> 
>    GLib.log_variant(logLevel, new GLib.Variant('a{sv}', fields);
> }
> 
> then the javascript would be:
> 
> GLib.log_structured(GLib.LogLevelFlags.level_debug,
>  { 'MESSAGE': 'checking for optional
> sweep
> modulation',
>    'FOO_STATE':
> JSON.stringify(fooState)});

Seems reasonable to me. pygobject, gjs people: how does that look to
you?

> One complication with g_log_variant, is figuring out how to send the
> variant off to journald. If you just use g_variant_print, you'll end
> up putting quotes around all the strings, which is probably wrong.
> It's probably have to unpack the variant manually, with fall back to
> g_variant_print

Yeah, the conversion from each a{sv} value to a GLogField value would
be the least satisfying part: I guess we'd do something like:
  if it's a string: use the string directly
  if it's a byte array ('ay'): use the byte array with its length
  if it's anything else: g_variant_print() and hope it looks reasonable

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

g_log_structured() and introspection

[Philip Withnall]

Hi all,

During the talk on GLib's new structured logging API at GUADEC today,
it was pointed out that g_log_structured() (and the rest of the
interesting bits of the new structured logging API) are not
introspectable.

I'm not sure what we can do about this. The API is based around
GLogField, which is basically a pointer and a length, and hence is not
introspectable:

struct _GLogField
{
  const gchar *key;
  gconstpointer value;
  gssize length;
};

My suggestion would be for bindings to manually bind
g_log_structured_array() with whatever language-specific syntax or API
they see fit, and to ignore the GIR definitions of g_log_structured()
and g_log_structured_array().

Thoughts?

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: librest API/ABI breakage

[Philip Withnall]

Hey,

On Thu, 2016-07-14 at 14:55 +0200, Christophe Fergeau wrote:
> This is just a headsup that we are about to merge some changes to
> librest master which are going to break API/ABI. Main goal is to
> change the async API to use GTask, but we are using this break as an
> opportunity for more cleanups.
> 
> At the moment, the API in librest git master is not going to be
> stable,
> so I would not recommend porting your module to it yet. The librest-
> 0-7
> branch is going to be the stable branch from which we'll cut releases
> if
> needed. This one keeps API/ABI stable.
> 
> I've updated the jhbuild modulesets so that they use this stable
> librest-0-7 branch as well.

Nice!

OOI, are you going to take the opportunity ensure it follows all the
latest parallel installability guidelines as well?

https://developer.gnome.org/programming-guidelines/unstable/parallel-in
stallability.html.en

Highly recommended, as it allows for future API breaks more easily, and
makes sure the headers are in a logical layout in srcdir and when
installed.

Some of the other programming-guidelines pages may also be relevant
(e.g. versioning).

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Keep shipping also generated gtk-doc html/ folder?

[Philip Withnall]

On Thu, 2016-06-23 at 10:35 +0200, Milan Crha wrote:
>   Hello,
> while playing with developer documentation I noticed that the source
> tarballs (eventually `make dist` result) contain also developer
> documentation in generated form. These documentation html/ files are
> not small, in case of the glib it makes around 10MB. The thing is
> that
> when I configure with --enable-gtk-doc, then the shipped html/ files
> are regenerated, thus it looks like a waste of space and bandwidth to
> distribute them.
> 
> I do not know the history behind it, maybe I just overlooked
> something
> and it does make sense to distribute that too. That's why I raised it
> here.
> 
> It would be interesting to know whether anyone uses the html/ files
> without --enable-gtk-doc these days, but I understand it's a hard
> question.

If I remember correctly, it’s so that the tarballs can be unpacked to
give documentation on developer.gnome.org without having to build
anything.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: gettext

[Philip Withnall]

On Fri, 2016-04-22 at 08:49 +0100, Patrick Welche wrote:
> All the docs for the maintainer side of i18n I found use intltool and
> glib-gettext, yet I see bugzilla bugs open to use upstream gettext /
> autopoint.
> 
> Do you have a description of The Right Way(tm)?

Upstream gettext and autopoint are the recommended tools to use,
although you will need the latest versions of them in order to get
support for all the things which intltool supports (namely, AppData,
support for which was added in gettext 0.19.6).

intltool and glib-gettext still work OK, but they are basically
unmaintained.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Developer experience (DX) hackfest 2016

[Philip Withnall]

Hi all,

A reminder that the DX hackfest is happening in a little over a month,
so please make sure you are signed up on the wiki page, and your
transport and accommodation are sorted out!

Philip

On Sun, 2015-12-13 at 13:50 +, Philip Withnall wrote:
> Hi all,
> 
> There’s going to be a developer experience (DX) hackfest at the end
> of
> January 2016 (27–29th January) in Brussels, just before FOSDEM. This
> should be a really useful hackfest for pushing forward the roadmap
> for
> developer tools and toolkits in GNOME!
> 
> https://wiki.gnome.org/Hackfests/DeveloperExperience2016
> 
> Anyone who has relevant interests in the developer experience is
> welcome, so please sign up ASAP on the wiki page.
> 
> If you plan to apply for travel or accommodation sponsorship, please
> do
> so in the next week (and certainly no later than the end of December)
> to the travel committee, so they have time to process requests.
> 
> https://wiki.gnome.org/Travel
> 
> The venue will be the betacowork coworking centre in Brussels. It is
> kindly being sponsored by betacowork and ICAB. Thanks!
>  • http://www.betacowork.com/
>  • http://www.icabrussel.be/
> 
> Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Developer experience (DX) hackfest 2016

[Philip Withnall]

Hi all,

There’s going to be a developer experience (DX) hackfest at the end of
January 2016 (27–29th January) in Brussels, just before FOSDEM. This
should be a really useful hackfest for pushing forward the roadmap for
developer tools and toolkits in GNOME!

https://wiki.gnome.org/Hackfests/DeveloperExperience2016

Anyone who has relevant interests in the developer experience is
welcome, so please sign up ASAP on the wiki page.

If you plan to apply for travel or accommodation sponsorship, please do
so in the next week (and certainly no later than the end of December)
to the travel committee, so they have time to process requests.

https://wiki.gnome.org/Travel

The venue will be the betacowork coworking centre in Brussels. It is
kindly being sponsored by betacowork and ICAB. Thanks!
 • http://www.betacowork.com/
 • http://www.icabrussel.be/

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Maintainers, please read this. [Re: GNOME 3.19.2 unstable tarballs due]

[Philip Withnall]

On Wed, 2015-11-25 at 21:40 +0100, Frederic Peters wrote:
> Hi,
> 
> Philip Withnall wrote:
> 
> > On Wed, 2015-11-25 at 11:07 +0100, Frederic Peters wrote:
> > > I'm a bit surprised by 1) but we could certainly automatically
> > > produce
> > > a list of maintainers / modules/ time/commits since last release,
> > > if
> > > that could be useful.
> > 
> > I think 1) would be useful because I have a load of modules which I
> > am
> > not sure if they need to follow the main GNOME release schedule. I
> > had
> > a nagging fear they should, but then nobody poked me about doing a
> > release, so I forgot to check up further. As a result, a lot of my
> > modules haven’t had releases following the schedule. (Sorry if I
> > have
> > been a total pain because of this.)
> > 
> > Would such a list be useful for the release team, as a way of
> > tracking
> > who needs nagging? If so, then I hope producing it should not be
> > too
> > much of a drain on your time — if it is a drain, then you probably
> > shouldn’t do it.
> 
> This was mostly done manually (at least as far I am concerned), but
> here is an experiment,
>   https://people.gnome.org/~fpeters/health/wanted-releases.html
> 
> And the red modules are first targets.
> 
> (this has been generated from my local clones, updated a few hours
> ago, if it's something we want to pursue it would be quite nice to
> have this automated and running on GNOME infrastructure) (the disk
> requirements exclude openshift).

Nice! If it's not much effort to get this running on GNOME
infrastructure, I think it would be useful, and have it linked from the
release reminder e-mails.

What do the x/y numbers mean?

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Maintainers, please read this. [Re: GNOME 3.19.2 unstable tarballs due]

[Philip Withnall]

On Wed, 2015-11-25 at 11:07 +0100, Frederic Peters wrote:
> Bastien Nocera wrote:
> 
> > > It's been some months we have those reminder emails sent to
> > > devel-announce-list.  Maintainers, make sure you are subscribed.
> > > 
> > > Maintainers (bis), please do try to respect the Monday 23:59 UTC
> > > deadline, it's really not fun to chase maintainers for days
> > > during
> > > release weeks.  If you know you will be late, tell us, either by
> > > email
> > > at release-t...@gnome.org or simply by pinging us on #release-
> > > team.
> > 
> > I'd rather somebody helped me 1) know which of my modules need
> > releasing 2) do releases.
> 
> We can of course assist in producing tarballs, just ping us.
> 
> I'm a bit surprised by 1) but we could certainly automatically
> produce
> a list of maintainers / modules/ time/commits since last release, if
> that could be useful.

I think 1) would be useful because I have a load of modules which I am
not sure if they need to follow the main GNOME release schedule. I had
a nagging fear they should, but then nobody poked me about doing a
release, so I forgot to check up further. As a result, a lot of my
modules haven’t had releases following the schedule. (Sorry if I have
been a total pain because of this.)

Would such a list be useful for the release team, as a way of tracking
who needs nagging? If so, then I hope producing it should not be too
much of a drain on your time — if it is a drain, then you probably
shouldn’t do it.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Death of gnome-common

[Philip Withnall]

Hey,

On Sun, 2015-08-16 at 11:31 +0200, Sébastien Wilmet wrote:
 On Thu, May 28, 2015 at 12:47:47PM +0100, Philip Withnall wrote:
  There’s a migration guide here:
  
  https://wiki.gnome.org/Projects/GnomeCommon/Migration
  
  We’ve tried to make the transition as easy and smooth as possible,
  but
  there will inevitably be hiccups. Please let me know about anything
  which breaks or doesn’t make sense. First person to complain about
  -Wswitch-enum gets a prize.
 
 I've migrated GtkSourceView, and now when running autogen.sh I see
 those
 messages:
 
 + glib-gettextize --force --copy
 Copying file po/Makefile.in.in
  
 Please add the files
   codeset.m4 gettext.m4 glibc21.m4 iconv.m4 isc-posix.m4
 lcmessage.m4
   progtest.m4
 from the /home/seb/jhbuild/share/aclocal directory to your
 autoconf macro directory
 or directly to your aclocal.m4 file.
 You will also need config.guess and config.sub, which you can get
 from
 ftp://ftp.gnu.org/pub/gnu/config/.
 
 Is it normal? We don't want to add those m4 files on git. For
 autoconf-archive macros, we have chosen the preferred option, i.e.
 adding a dependency on autoconf-archive instead of copying the
 macros.
 So for gettext we want to do the same.
 
 'make distcheck' passes, but the listed macros above are not
 distributed
 in tarballs. --install has been added to the ACLOCAL_AMFLAGS.

They should be automatically concatenated to the aclocal.m4 script in
the tarball, if they are actually used by any of the build system.

 Do you know what's wrong? Do we need to do something about it?

Just ignore the message.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Feedback from a happy user

[Philip Withnall]

One of my friends just sent me this, and I thought people might like to
share in it:

Since GNOME has turned 18 so is now eligible to die for its country
and get a tattoo I thought it prudent to thank its developers for
making my everyday computing experience a goddamn delight!

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Death of gnome-common

[Philip Withnall]

On Tue, 2015-06-23 at 22:24 -0500, Michael Catanzaro wrote:
 On Tue, 2015-06-23 at 08:33 +0100, Philip Withnall wrote:
  As Kalev says, we should ensure this still builds with various 
  distros.
  RHEL7 is fine. Debian stable (Jessie) has 1.14. Fedora has had ≥ 
  1.13
  for a long time. Arch has 1.15. FreeBSD has 1.15.
  
  So unless anybody objects, I think a hard dependency on automake 
  1.13
  would be fine.
  
  Would you mind updating the wiki page to match your suggestions?
 
 I guess we can simply remove the ACLOCAL_AMFLAGS line from 
 Makefile.am,
 since autogen.sh calls aclocal --install. Correct? (This works for 
 me.)

Sounds good.

 libtoolize complains about it:
 
 libtoolize: Consider adding `-I m4' to ACLOCAL_AMFLAGS in 
 Makefile.am.
 
 But I guess libtoolize is wrong.

Is there an upstream bug report?

  Probably also makes sense to update
  
  https://wiki.gnome.org/Initiatives/GnomeGoals/ModernAutotools
  
  which is a separate, though related, goal.
 
 We should also remove mention of AX_REQUIRE_DEFINED([APPDATA_XML])
 since APPDATA_XML is obsoleted by APPSTREAM_XML.

Preferably change it to AX_REQUIRE_DEFINED([APPSTREAM_XML]).

 I guess aclocal --install and autoreconf --install are redundant, and
 we just need to run aclocal first because we run glib-gettextize,
 intltoolize, and gtkdocize before autoreconf? Moving autoreconf to 
 the
 top of the  allows me to drop use of aclocal from my autogen.sh:
 
 set -x
 autoreconf --verbose --force --install -Wno-portability || exit 1
 glib-gettextize --force --copy || exit 1
 intltoolize --force --copy --automake || exit 1
 set +x
 
 This works with epiphany's build system (though it doesn't use
 gtkdocize).
 
 I also suspect it's wrong to use both glib-gettextize and intltoolize
 in autogen.sh, but I don't pretend to understand these tools

It is, but that’s an example file and the instructions do say to drop
any -ize programs which your module doesn’t use. Suggestions welcome
for ways to make this clearer on the wiki page (or just edit it!).

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Death of gnome-common

[Philip Withnall]

On Wed, 2015-06-24 at 22:38 +0200, Michael Biebl wrote:
 2015-06-24 9:56 GMT+02:00 Emmanuele Bassi eba...@gmail.com:
  On 23 June 2015 at 23:41, Philip Withnall phi...@tecnocode.co.uk 
  wrote:
   What if your module uses glib-gettext, gtk-doc, or intltool?
  
  I do hope that, if you're using glib-gettext or intltool, you spend 
  a
  little bit of time to port to upstream gettext instead.
 
 Interesting. Is intltool considered deprecated nowadays? If so, where
 can I read more about it?

I would consider it discouraged because it’s effectively a wrapper
around gettext, from the days when gettext was unmaintained and didn’t
support the tools people wanted. Now gettext is actively (and well!)
maintained, intltool is no longer needed.

At least, that’s how I understand it.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Death of gnome-common

[Philip Withnall]

On Wed, 2015-06-24 at 08:57 -0500, Michael Catanzaro wrote:
 On Wed, 2015-06-24 at 14:17 +0100, Emmanuele Bassi wrote:
  Sadly, it's still in use for AppData XML, but for that (and any 
  other
  XML-based format) there's itstool, which is also used by the
  documentation team for the application help:
  
 http://itstool.org/
 
 I will wait for someone to replace intltool with itstool in some 
 module
 that uses appdata, so I have an example to follow. I'm definitely not
 up for learning yet another language to figure out how to do this 
 from
 scratch.

I’ve got a patch which adds support for AppData to gettext, but it’s
stalled on FSF copyright assignment faff. I’m not sure I’ve ever come
across such a big barrier to contribution.

https://savannah.gnu.org/bugs/?45305

Anyway, ranting aside, here’s the associated branch to port Hitori to
gettext, including AppData:

https://git.gnome.org/browse/hitori/log/?h=wip/pwithnall/gettext

Upstream gettext also has a useful gtk-3 example project:

http://git.savannah.gnu.org/cgit/gettext.git/tree/gettext
-tools/examples/hello-c-gnome3

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Death of gnome-common

[Philip Withnall]

On Wed, 2015-06-24 at 08:31 -0500, Michael Catanzaro wrote:
 On Wed, 2015-06-24 at 09:21 +0100, Philip Withnall wrote:
   libtoolize complains about it:
   
   libtoolize: Consider adding `-I m4' to ACLOCAL_AMFLAGS in 
   Makefile.am.
   
   But I guess libtoolize is wrong.
  
  Is there an upstream bug report?
 
 No need. I have libtool 2.4.2 provided by Fedora 22, but I see in the
 libtool 2.4.3 changelog:
 
 - The libtoolize program now advises use of the new Autoconf
 AC_CONFIG_MACRO_DIRS declaration. If you follow that advice, all
 your developers will need at least autoconf-2.70 and automake-1.13
 to rebootstrap your probject. If you still need to support
 bootstrap with older Autotools, then you should add the following
 to your configure.ac file:
 
 m4_ifndef([AC_CONFIG_MACRO_DIRS],
 [m4_define([AC_CONFIG_MACRO_DIRS],
 m4_defn([AC_CONFIG_MACRO_DIR]))])
 
 
 
 I guess that should have read your developers will need at least
 autoconf-2.70 OR automake-1.13 since AC_CONFIG_MACRO_DIRS is 
 provided
 by Automake, but actually I see no reason to use AC_CONFIG_MACRO_DIRS
 at all: AC_CONFIG_MACRO_DIR seems to work just fine.

Yes, as I understand it, AC_CONFIG_MACRO_DIR is just a single-directory
version of AC_CONFIG_MACRO_DIRS, and they function equivalently.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Death of gnome-common

[Philip Withnall]

On Mon, 2015-06-22 at 10:44 -0500, Michael Catanzaro wrote:
 On Mon, 2015-06-22 at 16:36 +0100, Philip Withnall wrote:
  Do we require automake 1.13 though?
 
 Looks like it was released January 1, 2013... so we should.

We *could*; doesn’t necessarily mean we *should*.

   Automake 2.0 isn’t out yet (though
  it looks cool), so I’d be tempted to leave in the ACLOCAL_AMFLAGS
  recommendation for backwards compatibility until automake 2.0 is
  released, and then bump everyone’s dependency to automake 1.13 and 
  drop
  use of ACLOCAL_AMFLAGS.
  
  Unless you think automake 2.0 is being released imminently, and we
  should instead be adding a hard dependency on automake 1.13 and 
  using
  AC_CONFIG_MACRO_DIRS to begin with?
 
 Automake 2.0 is supposed to be released later this year, and automake
 1.13 has been around for a while, so I would definitely go straight 
 to
 AC_CONFIG_MACRO_DIRS.

As Kalev says, we should ensure this still builds with various distros.
RHEL7 is fine. Debian stable (Jessie) has 1.14. Fedora has had ≥ 1.13
for a long time. Arch has 1.15. FreeBSD has 1.15.

So unless anybody objects, I think a hard dependency on automake 1.13
would be fine.

Would you mind updating the wiki page to match your suggestions?
Probably also makes sense to update

https://wiki.gnome.org/Initiatives/GnomeGoals/ModernAutotools

which is a separate, though related, goal.

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Death of gnome-common

[Philip Withnall]

Hey Emmanuele, Philip

On Tue, 2015-06-23 at 17:08 +0100, Emmanuele Bassi wrote:
 that's usually better replaced by:
 
 autoreconf -if || exit $?
 test -n $NOCONFIGURE  ./configure $@
 
 Which isn't shorter, but it's pretty much to the point and works fine
 in jhbuild and autobuilders.

What if your module uses glib-gettext, gtk-doc, or intltool? As I
understand it, autoreconf doesn’t know about running glib-gettextize,
gtkdocize or intltoolize.

I guess the best answer to that would be:
 • “use gettext rather than glib-gettext or intltool”; and
 • “fix gtk-doc to not need gtkdocize”[0].

Philip (the other Philip)

[0]: https://bugzilla.gnome.org/show_bug.cgi?id=744864

 On 23 June 2015 at 16:54, Philip Chimento philip.chime...@gmail.com 
 wrote:
  On Mon, Jun 22, 2015 at 12:01 AM, Philip Withnall 
  phi...@tecnocode.co.uk
  wrote:
   
   On Sun, 2015-06-21 at 17:56 -0700, Jasper St. Pierre wrote:
On Thu, May 28, 2015 at 9:05 AM, Philip Withnall 
phi...@tecnocode.co.uk wrote:
 See literally the first migration item on
 https://wiki.gnome.org/Projects/GnomeCommon/Migration
 
 tl;dr: You can open-code it with essentially:
 aclocal --install || exit 1
 glib-gettextize --force --copy || exit 1
 gtkdocize --copy || exit 1
 intltoolize --force --copy --automake || exit 1
 autoreconf --verbose --force --install -Wno
 -portability ||
 exit
 1
 
 (removing the technologies which you don’t use).

Er, I meant to reply to this earlier, but forgot to, I guess. 
Is
there
a simpler thing than this, because I still like . gnome
-autogen.sh
more. Seems much simpler.
   
   There is not. You are welcome to continue using gnome-autogen.sh 
   if you
   want to keep a dependency around purely for a build-time shell 
   script.
   Or you could copy gnome-autogen.sh into your project. We are 
   planning
   no further changes to gnome-autogen.sh upstream.
   
   Note that using gnome-autogen.sh isn’t actually as simple as 
   that;
   you’re supposed to set various environment variables indicating 
   your
   dependencies. So a realistic invocation is more like:
   
   #!/bin/sh
   srcdir=`dirname $0`
   test -z $srcdir  srcdir=.
   
   PKG_NAME=libgdata
   
   (test -f $srcdir/configure.ac) || {
   echo **Error**: Directory \`$srcdir\' does not look 
   like the
   top-level $PKG_NAME directory
   exit 1
   }
   
   which gnome-autogen.sh || {
   echo You need to install gnome-common from GNOME 
   Git
   exit 1
   }
   
   REQUIRED_PKG_CONFIG_VERSION=0.17.1 
   REQUIRED_AUTOMAKE_VERSION=1.13
   REQUIRED_GTK_DOC_VERSION=1.14 . gnome-autogen.sh $@
  
  
  In fact for most projects you can probably just replace . gnome
  -autogen.sh
  with autoreconf -if. It's fewer keystrokes ;-)
  
  Or autoreconf -if  ./configure $@ if you want to retain the 
  behaviour of
  automatically running configure (which I would prefer not to do, if 
  only
  jhbuild didn't rely on it.)
  
  Philip C.
  
 On Thu, 2015-05-28 at 08:43 -0700, Jasper St. Pierre wrote:
  What about the gnome-autogen.sh script? Most of my 
  autogen.sh
  files
  just run . gnome-autogen.sh
  
  On Thu, May 28, 2015 at 6:55 AM, Daniel Mustieles García
  daniel.mustie...@gmail.com wrote:
   Ok, thanks for clarifying! :)
   
   2015-05-28 14:40 GMT+02:00 Philip Withnall 
   phi...@tecnocode.co.uk:

It’s already sort of tracked as part of the 
ModernAutotools
goal,
although that one lost momentum a while ago, so its 
status
needs to be
reset:

https://wiki.gnome.org/Initiatives/GnomeGoals/ModernAut
otools

However, since there’s no flag-day-changeover for gnome
-common, I’m not
sure it’s necessary to have a GNOME Goal. Maintainers 
hate
touching
build systems unnecessarily.

Philip

On Thu, 2015-05-28 at 14:03 +0200, Daniel Mustieles 
García
wrote:
 Would this make sense to create a GNOME Goal[1] to
 ease/track the
 change?
 
 [1]
 
 https://wiki.gnome.org/action/show/Initiatives/GnomeG
 oals?a
 ction=showredirect=GnomeGoals
 
 
 2015-05-28 13:47 GMT+02:00 Philip Withnall 
 phi...@tecnocode.co.uk:
 Hi all,
 
 This cycle, Dave and I are planning for the 
 last
 ever release
 of
 gnome-common. A lot of its macros for 
 deprecated
 technologies
 (scrollkeeper?!) have been removed, and the
 remainder of its
 macros have
 found better replacements in autoconf

Re: Death of gnome-common

[Philip Withnall]

On Sun, 2015-06-21 at 19:54 -0500, Michael Catanzaro wrote:
 OK, next problem. The porting guide says:
 
 Firstly, add
 
 AC_CONFIG_MACRO_DIR([m4])
 
 to your configure.ac, and
 
 ACLOCAL_AMFLAGS = --install -I m4 ${ACLOCAL_FLAGS}
 
 to your top-level Makefile.am, regardless of which of the two options
 you choose below.
 
 But I see in Automake's NEWS file:
 
   - The ACLOCAL_AMFLAGS special make variable will be fully 
 deprecated
 in
 Automake 2.0: it will raise warnings in the obsolete category
 (but
 still no hard error of course, for compatibilities with the many,
 many
 packages that still relies on that variable).  You are advised to
 start relying on the new Automake support for 
 AC_CONFIG_MACRO_DIRS
 instead (which was introduced in Automake 1.13).

Do we require automake 1.13 though? Automake 2.0 isn’t out yet (though
it looks cool), so I’d be tempted to leave in the ACLOCAL_AMFLAGS
recommendation for backwards compatibility until automake 2.0 is
released, and then bump everyone’s dependency to automake 1.13 and drop
use of ACLOCAL_AMFLAGS.

Unless you think automake 2.0 is being released imminently, and we
should instead be adding a hard dependency on automake 1.13 and using
AC_CONFIG_MACRO_DIRS to begin with?

Philip

signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Death of gnome-common

[Philip Withnall]

On Sun, 2015-06-21 at 17:56 -0700, Jasper St. Pierre wrote:
 On Thu, May 28, 2015 at 9:05 AM, Philip Withnall 
 phi...@tecnocode.co.uk wrote:
  See literally the first migration item on
  https://wiki.gnome.org/Projects/GnomeCommon/Migration
  
  tl;dr: You can open-code it with essentially:
  aclocal --install || exit 1
  glib-gettextize --force --copy || exit 1
  gtkdocize --copy || exit 1
  intltoolize --force --copy --automake || exit 1
  autoreconf --verbose --force --install -Wno-portability || 
  exit
  1
  
  (removing the technologies which you don’t use).
 
 Er, I meant to reply to this earlier, but forgot to, I guess. Is 
 there
 a simpler thing than this, because I still like . gnome-autogen.sh
 more. Seems much simpler.

There is not. You are welcome to continue using gnome-autogen.sh if you
want to keep a dependency around purely for a build-time shell script.
Or you could copy gnome-autogen.sh into your project. We are planning
no further changes to gnome-autogen.sh upstream.

Note that using gnome-autogen.sh isn’t actually as simple as that;
you’re supposed to set various environment variables indicating your
dependencies. So a realistic invocation is more like:

#!/bin/sh
srcdir=`dirname $0`
test -z $srcdir  srcdir=.

PKG_NAME=libgdata

(test -f $srcdir/configure.ac) || {
echo **Error**: Directory \`$srcdir\' does not look like the
top-level $PKG_NAME directory
exit 1
}

which gnome-autogen.sh || {
echo You need to install gnome-common from GNOME Git
exit 1
}

REQUIRED_PKG_CONFIG_VERSION=0.17.1 REQUIRED_AUTOMAKE_VERSION=1.13
REQUIRED_GTK_DOC_VERSION=1.14 . gnome-autogen.sh $@

Philip

  On Thu, 2015-05-28 at 08:43 -0700, Jasper St. Pierre wrote:
   What about the gnome-autogen.sh script? Most of my autogen.sh 
   files
   just run . gnome-autogen.sh
   
   On Thu, May 28, 2015 at 6:55 AM, Daniel Mustieles García
   daniel.mustie...@gmail.com wrote:
Ok, thanks for clarifying! :)

2015-05-28 14:40 GMT+02:00 Philip Withnall 
phi...@tecnocode.co.uk:
 
 It’s already sort of tracked as part of the ModernAutotools 
 goal,
 although that one lost momentum a while ago, so its status 
 needs to be
 reset:
 
 https://wiki.gnome.org/Initiatives/GnomeGoals/ModernAutotools
 
 However, since there’s no flag-day-changeover for gnome
 -common, I’m not
 sure it’s necessary to have a GNOME Goal. Maintainers hate 
 touching
 build systems unnecessarily.
 
 Philip
 
 On Thu, 2015-05-28 at 14:03 +0200, Daniel Mustieles García 
 wrote:
  Would this make sense to create a GNOME Goal[1] to 
  ease/track the
  change?
  
  [1]
  
  https://wiki.gnome.org/action/show/Initiatives/GnomeGoals?a
  ction=showredirect=GnomeGoals
  
  
  2015-05-28 13:47 GMT+02:00 Philip Withnall 
  phi...@tecnocode.co.uk:
  Hi all,
  
  This cycle, Dave and I are planning for the last 
  ever release
  of
  gnome-common. A lot of its macros for deprecated 
  technologies
  (scrollkeeper?!) have been removed, and the 
  remainder of its
  macros have
  found better replacements in autoconf-archive[1], 
  where they
  can be used
  by everyone, not just GNOME.
  
  We plan to make one last release, and people are 
  welcome to
  depend on it
  for as long as they like. However, if you want new 
  hotness,
  port to the
  autoconf-archive versions of the macros; but please 
  do it in
  your own
  time. There will be no flag day port away from 
  gnome-common.
  
  Note that, for example, porting to 
  AX_COMPILER_FLAGS is
  valuable, but
  will probably require fixing a number of new 
  compiler warnings
  in your
  code due to increased warning flags. We hope this 
  will make
  your code
  better in the long run.
  
  There’s a migration guide here:
  
  
  https://wiki.gnome.org/Projects/GnomeCommon/Migration
  
  We’ve tried to make the transition as easy and 
  smooth as
  possible, but
  there will inevitably be hiccups. Please let me 
  know about
  anything
  which breaks or doesn’t make sense. First person to 
  complain
  about
  -Wswitch-enum gets a prize.
  
  
  For developers
  ---
  
  When building from a tarball of a module which uses 
  the new
  macros, you
  will no longer need gnome-common installed. 
  (Although

Re: Death of gnome-common

[Philip Withnall]

On Sat, 2015-05-30 at 08:09 -0500, Michael Catanzaro wrote:
 This is a nit, but I think the 'set -x' to 'set +x' at the end of the
 sample autogen.sh is too expansive, since it results in the conditional
 tests being printed as independent statements:

Good nit. Please feel free to update the wiki page. :-)

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Death of gnome-common

[Philip Withnall]

Hi all,

This cycle, Dave and I are planning for the last ever release of
gnome-common. A lot of its macros for deprecated technologies
(scrollkeeper?!) have been removed, and the remainder of its macros have
found better replacements in autoconf-archive[1], where they can be used
by everyone, not just GNOME.

We plan to make one last release, and people are welcome to depend on it
for as long as they like. However, if you want new hotness, port to the
autoconf-archive versions of the macros; but please do it in your own
time. There will be no flag day port away from gnome-common.

Note that, for example, porting to AX_COMPILER_FLAGS is valuable, but
will probably require fixing a number of new compiler warnings in your
code due to increased warning flags. We hope this will make your code
better in the long run.

There’s a migration guide here:

https://wiki.gnome.org/Projects/GnomeCommon/Migration

We’ve tried to make the transition as easy and smooth as possible, but
there will inevitably be hiccups. Please let me know about anything
which breaks or doesn’t make sense. First person to complain about
-Wswitch-enum gets a prize.


For developers
---

When building from a tarball of a module which uses the new macros, you
will no longer need gnome-common installed. (Although you may not have
needed it before.)

When building from git, you will need m4-common[2] or
autoconf-archive[1] installed.

JHBuild bootstrap installs m4-common automatically, as does
gnome-continuous; so you don’t need to worry about that.

For packagers
---

In the 3.14.0 release, gnome-common installed some early versions of the
autoconf-archive macros which conflicted with what autoconf-archive
itself installs. It now has a --[with|without]-autoconf-archive
configure option to control this. We suggest that all packagers pass
--with-autoconf-archive if (and only if) autoconf-archive is packaged on
the distribution. See bug #747920[3].

m4-common *must not* be packaged. See its README[2]. m4-common is
essentially a caching subset of autoconf-archive.

For continuous integrators
---

Modules which use the new AX_COMPILER_FLAGS macro gain a new standard
--disable-Werror configure flag, which should be used in CI systems (and
any other system where spurious compiler warnings should _not_ cause
total failure of a build) to disable -Werror. The idea here is that
-Werror is enabled by default when building from git, and disabled by
default when building from release tarballs and in buildbots.

Philip

[1]: http://www.gnu.org/software/autoconf-archive/
[2]: https://github.com/desrt/m4-common/
[3]: https://bugzilla.gnome.org/show_bug.cgi?id=747920


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Death of gnome-common

[Philip Withnall]

It’s already sort of tracked as part of the ModernAutotools goal,
although that one lost momentum a while ago, so its status needs to be
reset:

https://wiki.gnome.org/Initiatives/GnomeGoals/ModernAutotools

However, since there’s no flag-day-changeover for gnome-common, I’m not
sure it’s necessary to have a GNOME Goal. Maintainers hate touching
build systems unnecessarily.

Philip

On Thu, 2015-05-28 at 14:03 +0200, Daniel Mustieles García wrote:
 Would this make sense to create a GNOME Goal[1] to ease/track the
 change?
 
 [1]
 https://wiki.gnome.org/action/show/Initiatives/GnomeGoals?action=showredirect=GnomeGoals
 
 
 2015-05-28 13:47 GMT+02:00 Philip Withnall phi...@tecnocode.co.uk:
 Hi all,
 
 This cycle, Dave and I are planning for the last ever release
 of
 gnome-common. A lot of its macros for deprecated technologies
 (scrollkeeper?!) have been removed, and the remainder of its
 macros have
 found better replacements in autoconf-archive[1], where they
 can be used
 by everyone, not just GNOME.
 
 We plan to make one last release, and people are welcome to
 depend on it
 for as long as they like. However, if you want new hotness,
 port to the
 autoconf-archive versions of the macros; but please do it in
 your own
 time. There will be no flag day port away from gnome-common.
 
 Note that, for example, porting to AX_COMPILER_FLAGS is
 valuable, but
 will probably require fixing a number of new compiler warnings
 in your
 code due to increased warning flags. We hope this will make
 your code
 better in the long run.
 
 There’s a migration guide here:
 
 https://wiki.gnome.org/Projects/GnomeCommon/Migration
 
 We’ve tried to make the transition as easy and smooth as
 possible, but
 there will inevitably be hiccups. Please let me know about
 anything
 which breaks or doesn’t make sense. First person to complain
 about
 -Wswitch-enum gets a prize.
 
 
 For developers
 ---
 
 When building from a tarball of a module which uses the new
 macros, you
 will no longer need gnome-common installed. (Although you may
 not have
 needed it before.)
 
 When building from git, you will need m4-common[2] or
 autoconf-archive[1] installed.
 
 JHBuild bootstrap installs m4-common automatically, as does
 gnome-continuous; so you don’t need to worry about that.
 
 For packagers
 ---
 
 In the 3.14.0 release, gnome-common installed some early
 versions of the
 autoconf-archive macros which conflicted with what
 autoconf-archive
 itself installs. It now has a --[with|
 without]-autoconf-archive
 configure option to control this. We suggest that all
 packagers pass
 --with-autoconf-archive if (and only if) autoconf-archive is
 packaged on
 the distribution. See bug #747920[3].
 
 m4-common *must not* be packaged. See its README[2]. m4-common
 is
 essentially a caching subset of autoconf-archive.
 
 For continuous integrators
 ---
 
 Modules which use the new AX_COMPILER_FLAGS macro gain a new
 standard
 --disable-Werror configure flag, which should be used in CI
 systems (and
 any other system where spurious compiler warnings should _not_
 cause
 total failure of a build) to disable -Werror. The idea here is
 that
 -Werror is enabled by default when building from git, and
 disabled by
 default when building from release tarballs and in buildbots.
 
 Philip
 
 [1]: http://www.gnu.org/software/autoconf-archive/
 [2]: https://github.com/desrt/m4-common/
 [3]: https://bugzilla.gnome.org/show_bug.cgi?id=747920
 
 ___
 desktop-devel-list mailing list
 desktop-devel-list@gnome.org
 https://mail.gnome.org/mailman/listinfo/desktop-devel-list
 
 



signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Death of gnome-common

[Philip Withnall]

See literally the first migration item on
https://wiki.gnome.org/Projects/GnomeCommon/Migration

tl;dr: You can open-code it with essentially:
aclocal --install || exit 1
glib-gettextize --force --copy || exit 1
gtkdocize --copy || exit 1
intltoolize --force --copy --automake || exit 1
autoreconf --verbose --force --install -Wno-portability || exit
1

(removing the technologies which you don’t use).

Philip

On Thu, 2015-05-28 at 08:43 -0700, Jasper St. Pierre wrote:
 What about the gnome-autogen.sh script? Most of my autogen.sh files
 just run . gnome-autogen.sh
 
 On Thu, May 28, 2015 at 6:55 AM, Daniel Mustieles García
 daniel.mustie...@gmail.com wrote:
  Ok, thanks for clarifying! :)
 
  2015-05-28 14:40 GMT+02:00 Philip Withnall phi...@tecnocode.co.uk:
 
  It’s already sort of tracked as part of the ModernAutotools goal,
  although that one lost momentum a while ago, so its status needs to be
  reset:
 
  https://wiki.gnome.org/Initiatives/GnomeGoals/ModernAutotools
 
  However, since there’s no flag-day-changeover for gnome-common, I’m not
  sure it’s necessary to have a GNOME Goal. Maintainers hate touching
  build systems unnecessarily.
 
  Philip
 
  On Thu, 2015-05-28 at 14:03 +0200, Daniel Mustieles García wrote:
   Would this make sense to create a GNOME Goal[1] to ease/track the
   change?
  
   [1]
  
   https://wiki.gnome.org/action/show/Initiatives/GnomeGoals?action=showredirect=GnomeGoals
  
  
   2015-05-28 13:47 GMT+02:00 Philip Withnall phi...@tecnocode.co.uk:
   Hi all,
  
   This cycle, Dave and I are planning for the last ever release
   of
   gnome-common. A lot of its macros for deprecated technologies
   (scrollkeeper?!) have been removed, and the remainder of its
   macros have
   found better replacements in autoconf-archive[1], where they
   can be used
   by everyone, not just GNOME.
  
   We plan to make one last release, and people are welcome to
   depend on it
   for as long as they like. However, if you want new hotness,
   port to the
   autoconf-archive versions of the macros; but please do it in
   your own
   time. There will be no flag day port away from gnome-common.
  
   Note that, for example, porting to AX_COMPILER_FLAGS is
   valuable, but
   will probably require fixing a number of new compiler warnings
   in your
   code due to increased warning flags. We hope this will make
   your code
   better in the long run.
  
   There’s a migration guide here:
  
   https://wiki.gnome.org/Projects/GnomeCommon/Migration
  
   We’ve tried to make the transition as easy and smooth as
   possible, but
   there will inevitably be hiccups. Please let me know about
   anything
   which breaks or doesn’t make sense. First person to complain
   about
   -Wswitch-enum gets a prize.
  
  
   For developers
   ---
  
   When building from a tarball of a module which uses the new
   macros, you
   will no longer need gnome-common installed. (Although you may
   not have
   needed it before.)
  
   When building from git, you will need m4-common[2] or
   autoconf-archive[1] installed.
  
   JHBuild bootstrap installs m4-common automatically, as does
   gnome-continuous; so you don’t need to worry about that.
  
   For packagers
   ---
  
   In the 3.14.0 release, gnome-common installed some early
   versions of the
   autoconf-archive macros which conflicted with what
   autoconf-archive
   itself installs. It now has a --[with|
   without]-autoconf-archive
   configure option to control this. We suggest that all
   packagers pass
   --with-autoconf-archive if (and only if) autoconf-archive is
   packaged on
   the distribution. See bug #747920[3].
  
   m4-common *must not* be packaged. See its README[2]. m4-common
   is
   essentially a caching subset of autoconf-archive.
  
   For continuous integrators
   ---
  
   Modules which use the new AX_COMPILER_FLAGS macro gain a new
   standard
   --disable-Werror configure flag, which should be used in CI
   systems (and
   any other system where spurious compiler warnings should _not_
   cause
   total failure of a build) to disable -Werror. The idea here is
   that
   -Werror is enabled by default when building from git, and
   disabled by
   default when building from release tarballs and in buildbots.
  
   Philip
  
   [1]: http://www.gnu.org/software/autoconf-archive/
   [2]: https

Re: Current status of the GNOME Github mirror

[Philip Withnall]

On Fri, 2015-05-08 at 18:47 +0200, Andrea Veri wrote:
 I still have to find out a way to map
 maintainers with their Github account as it probably differs in many
 cases from their respective GNOME handle.

The first thing to come to mind is another element in the DOAP file,
similar to how we have gnome:userid — something like github:userid?

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Current status of the GNOME Github mirror

[Philip Withnall]

On Wed, 2015-05-06 at 10:10 +1000, Ben Finney wrote:
 Philip Withnall phi...@tecnocode.co.uk writes:
 
  […] given the choice between:
   • a contributor giving up because Bugzilla is another account to create
 and another tool to learn;
   • a contributor using Bugzilla but badly, and uploading a terrible
 patch which doesn’t apply cleanly; or
   • a contributor submitting a pull request;
 
  I would much rather receive the pull request, pull it in git, fix it up,
  and push it.
 
 What of the contributor who has no GitHub account, and wants to discuss
 their submission on an equal footing?

If they have a Bugzilla account already, I assume they will know to use
that.

If they don’t, then they’re going to have to create at least one account
(GitHub or Bugzilla), so I don’t think it makes much odds.

We could have a CONTRIBUTING.md which suggests that bug _discussion_
happens in Bugzilla, I guess.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Current status of the GNOME Github mirror

[Philip Withnall]

On Tue, 2015-05-05 at 18:51 +0200, Andrea Veri wrote:
 That said, I would like to invite anyone interested to include the
 sample file above on the module/s they contribute to. If you have a
 different opinion on how this should be managed, please follow up to
 this thread.

I used to think that all discussion, code reviews and patches should be
handled through Bugzilla, but I’ve come to change my opinion. I think we
should allow pull requests (although perhaps not actively encourage
them), because given the choice between:
 • a contributor giving up because Bugzilla is another account to create
   and another tool to learn;
 • a contributor using Bugzilla but badly, and uploading a terrible
   patch which doesn’t apply cleanly; or
 • a contributor submitting a pull request;

I would much rather receive the pull request, pull it in git, fix it up,
and push it. If further discussion is required, I think there’s a
reasonable chance of engaging the contributor at that point and moving
discussion across to Bugzilla.

The main reasoning behind a patch should be summarised in the commit
message anyway.

So I would be in favour of not discouraging pull requests, on the
condition that we can guarantee all module maintainers will be notified
of new pull requests from GitHub on the modules they maintain —
otherwise pull requests are going to stagnate, and that’s not good for
anyone.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Binary compatibility and padding

[Philip Withnall]

On Sat, 2015-04-11 at 15:43 +0200, Juan R. Garcia Blanco wrote:
 Hi,
 
 In GNOME Dictionary 3.16.0, _GdictDefboxClass provided four members that
 are function pointers 'for future expansion'. If I understand this
 correctly, these four extra members act as a place holder in case we
 want to add a new meaningful function pointer member, therefore removing
 one of the extra ones. This is done in order to provide binary
 compatibility I think.
 
 On the road to GNOME Dictionary 3.16.1, I have added a new signal to
 GdictDefbox, therefore adding a new function pointer member to
 _GdictDefboxClass, but I have not removed any of the extra members. So
 my question is: should I remove one of the extra members to preserve
 binary compatibility? Or should I just simply increment LT_VERSION
 appropriately?

You should remove one of the padding members, otherwise the size of the
GdictDefboxClass structure has changed, and binary compatibility is
broken.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Discouraging use of sync APIs

[Philip Withnall]

On Thu, 2015-02-12 at 12:55 -0800, Philip Chimento wrote:
 On Thu, Feb 12, 2015 at 5:22 AM, Philip Withnall
 phi...@tecnocode.co.uk wrote:
 On Thu, 2015-02-12 at 11:22 +0100, Alexander Larsson wrote:
  On ons, 2015-02-11 at 12:17 +, Philip Withnall wrote:
  
   I understand where you’re coming from; we should not be
 irritating
   experienced developers. I completely agree.
  
   What do you think of the proposal to use sync gstdio.h for
   small/local/pseudo-file-system I/O and async GIO for all
 other I/O?
 
  I don't think this is a great alternative in many cases. The
 gstdio
  stuff may work fine from C, but the GObjects etc of gio
 makes such use
  much nicer from language bindings.
 
 True. Wouldn’t the languages typically have their own native
 gstdio-like
 functions for local file I/O though, which would tend to be
 integrated
 nicely?
 
 
 Javascript doesn't; one of the main reasons behind the decision to
 make JS the first-class citizen for new Gnome apps developers was
 precisely that it didn't already have its own platform. I got the
 impression that using, for example, Python's native local file I/O
 facilities in the Gnome stack was to be discouraged.

Bother, good point. Scrap that idea then.


In summary, options we’ve discussed so far are:
 1. Use gstdio.h for small reads, GIO async for others:
   - gstdio.h is rubbish from bindings
 2. Warn if sync API is used from the main context, enabling the warning
on G_ENABLE_DIAGNOSTIC, and only warning if the call takes too long:
   + Helps in tracking down bugs at the source (rather than just ‘GTK+
 is being blocked too long’) 
   + Shouldn’t have any false positives in the cases discussed so far
 (small reads from console programs, application startup)
   - Requires active effort to enable (G_ENABLE_DIAGNOSTICS), limiting
 its uptake
 3. Rearranging the docs to promote async functions more
   ? Has not been discussed
   - Big red warnings in the docs may just be ignored by people

Seems like option #2 is worth going ahead with. I’ve filed:
https://bugzilla.gnome.org/show_bug.cgi?id=744458

Anybody have any thoughts about option #3? I’m wary about filling the
docs up with big red warnings.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Discouraging use of sync APIs

[Philip Withnall]

On Fri, 2015-02-13 at 04:52 -0500, Colin Walters wrote:
 On Wed, Feb 11, 2015, at 02:33 PM, Jim Nelson wrote:
 
  One of Philip's earlier suggestions was to print a console warning
  if a sync call is used.  That seems like overkill to me, but it does
  lead to another possibility.
  
   
  Technically the issue is long synchronous calls blocking the event
  loop, but in practice the problem is GTK+'s events being starved.
   Perhaps a more feasible solution would be to issue a console
  warning when a paint or resize event sits on the event loop for too
  long.
  
  
 Agree with this.  
 
  
 Also, this kind of warning should really only occur after the first
 paint.  Or we could *try* changing it to happen only after the first
 main context iteration, but I suspect that would still be too
 annoying.

Only warning after the first main context iteration would allow things
like:
g_file_read()  /* sync startup call to load config */
…
gtk_main()

 Changing glib to spew errors for sync IO unconditionally during app
 startup or hiding the sync API docs are just an obviously, totally
 ridiculous nonstarters - particularly console apps.  How is it even a
 topic of discussion?

The suggestion has since been refined to be conditional on the I/O
taking ‘a while’, being in the main thread, and G_ENABLE_DIAGNOSTICS
being enabled.

See: https://bugzilla.gnome.org/show_bug.cgi?id=744458

It was originally a strawman idea, precisely for the reason of gauging
how ridiculous people thought it was. The idea of unconditional warnings
is ridiculous.

 Not to mention of course that there's unavoidable synchronous disk IO
 before GLib is already loaded, in the kernel and dynamic linker.

But that doesn’t block the main loop, so isn’t causing interactivity
problems and we don’t care.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Discouraging use of sync APIs

[Philip Withnall]

On Thu, 2015-02-12 at 11:22 +0100, Alexander Larsson wrote:
 On ons, 2015-02-11 at 12:17 +, Philip Withnall wrote:
  
  I understand where you’re coming from; we should not be irritating
  experienced developers. I completely agree.
  
  What do you think of the proposal to use sync gstdio.h for
  small/local/pseudo-file-system I/O and async GIO for all other I/O?
 
 I don't think this is a great alternative in many cases. The gstdio
 stuff may work fine from C, but the GObjects etc of gio makes such use
 much nicer from language bindings.

True. Wouldn’t the languages typically have their own native gstdio-like
functions for local file I/O though, which would tend to be integrated
nicely?

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Discouraging use of sync APIs

[Philip Withnall]

On Wed, 2015-02-11 at 11:33 -0800, Jim Nelson wrote:
 One of Philip's earlier suggestions was to print a console warning if
 a sync call is used.  That seems like overkill to me, but it does lead
 to another possibility.

Emmanuele refined that to printing a warning if G_ENABLE_DIAGNOSTIC is
enabled in the environment, which seems like a good idea to me.

 Technically the issue is long synchronous calls blocking the event
 loop, but in practice the problem is GTK+'s events being starved.
 Perhaps a more feasible solution would be to issue a console warning
 when a paint or resize event sits on the event loop for too long.

Federico suggested there might already be such a warning, although
grepping through the GTK+ code I can only find a warning about excessive
re-layouts being requested (in gdkframeclockidle.c). I think a warning
about delayed frames would be a good idea, though I’m not a GTK+
developer and they might have other ideas.

One problem I see with this is that although it tells you there is a
problem, it doesn’t let you know *what* is blocking the loop. You
probably already knew there was a problem due to the UI freezing, in
fact.

 At the end of the day, async is something you design for, it doesn't
 just happen (especially in straight-up imperative languages like C and
 C++).  No matter how many console warnings, documentation exclamation
 marks, or API changes are made, it's all about getting developers to
 treat I/O like event-driven programming.  I know Jasper wishes apps
 would only paint in the draw signal; getting everyone on-board with
 pure-async code is a similar crusade.

True, and there might not be a solution. I would like to check the
possibilities though.

Philip


 On Wed, Feb 11, 2015 at 7:04 AM, Ross Lagerwall
 rosslagerw...@gmail.com wrote:
  On Wed, Feb 11, 2015 at 2:46 PM, Ray Strode halfl...@gmail.com
  wrote: 
  Hi, On Wed, Feb 11, 2015 at 7:17 AM, Philip Withnall
  phi...@tecnocode.co.uk wrote: 
  It might turn out that runtime checks are just not
  feasible, but in that case I think we still need
  some way of solving the original problem: that
  people are using sync calls and blocking up the main
  loop.
  I'm all for discouraging sync use in the main thread after
  the application is up, but are stalled applications actually
  a wide spread problem? I don't really remember any apps I
  use regularly locking up (except for maybe hexchat when
  connecting to my irc proxy). Granted, it's harder to notice
  these days now that we have a compositor and applications
  don't need to redraw after getting uncovered, so it could be
  it's happening more than is obvious. But, I just wonder if
  we really need to do anything. It seems like the bad/obvious
  cases would get bug reports and fixes pretty quickly, and so
  the problem should regulate itself.
  There are quite often gvfs or Nautilus bug reports that say network
  mount causes desktop to slow down. I tracked it down to some
  gnome-shell extension which somehow does sync calls to the remote fs
  which makes everything crawl. In general though I think severe
  warnings on the documentation for each sync call is better than
  runtime warnings or compile-time warnings. Ross
  ___ desktop-devel-list
  mailing list desktop-devel-list@gnome.org
  https://mail.gnome.org/mailman/listinfo/desktop-devel-list
 ___
 desktop-devel-list mailing list
 desktop-devel-list@gnome.org
 https://mail.gnome.org/mailman/listinfo/desktop-devel-list



signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Discouraging use of sync APIs

[Philip Withnall]

On Wed, 2015-02-11 at 13:10 -0800, Jasper St. Pierre wrote:
 That's one example of low-hanging fruit. Calling query_info_async is
 the same thing. Creating a GFileMonitor for a GResourceFile
 (admittedly nonsense, but arbitrary code that tries to watch any GFile
 passed into it might hurt) will call query_info_async once every 5
 seconds, creating a new thread for each.

I’ve filed a bug so this doesn’t get forgotten:
https://bugzilla.gnome.org/show_bug.cgi?id=744392

(For the record, Jasper has already reported the file monitoring bug
here:)
https://bugzilla.gnome.org/show_bug.cgi?id=744010

 If we want to clean this up by playing whack-a-mole and writing a lot
 of special-case code, we can certainly do that. I'm widely convinced
 doing so is a waste of time.

What would you suggest instead? I think these problems can be solved by
creating one or two new interfaces, like the GMemoryBased I just
proposed in bug #744392.

Anyway, this is getting a little off-topic. We should defer discussion
of these performance problems to relevant bug reports, since they’re
fixable bugs.

Philip

 Another example: for some odd reason, GLocalFileInputStream isn't a
 pollable output stream, so calling write_async on it spawns a thread,
 even when it should be using a write poll integrated with the
 mainloop. Why was this done? I don't know. Should this be fixed? Yes.
 
 
 On Wed, Feb 11, 2015 at 4:10 AM, Philip Withnall
 phi...@tecnocode.co.uk wrote:
 On Tue, 2015-02-10 at 10:30 -0800, Jasper St. Pierre wrote:
  One quick example: calling g_file_read_async on a
 GResourceFile spawns
  a new thread and does a synchronous stream read from a block
 already
  in memory.
 
 
  It should just be a single g_bytes_ref, but we have three
 different
  classes created, a thread spawned, and a large amount of
 locks to do
  the equivalent of memcpy.
 
 That’s not good at all, and seems like it should be easy to
 fix by
 overriding the read_async vfunc for GResourceFile. Is there a
 bug filed
 about it?
 
 Philip
 
  On Tue, Feb 10, 2015 at 8:49 AM, Jasper St. Pierre
  jstpie...@mecheye.net wrote:
  Right now the way g_file_read_async works is by
 scheduling a
  task on a worker thread, having the worker thread do
 the async
  read, and then returning a result.
 
  As such, it's impossible to have two async reads
 done at the
  same time, which is really unfortunate from my
 understanding.
  If I'm reading a large file, and then a small file,
 the large
  file needs to fully complete before the small file
 is
  dispatched from the async queue.
 
  Additionally, when profiling GNOME on ARM, I've been
 seeing a
  lot of cases of users using g_file_read_async()
 just in case
  for no particular reason, which causes several locks
 to be
  taken, severely slowing performance.
 
 
  We need to seriously improve our async performance
 before
  asking people to use it.
 
 
  On Tue, Feb 10, 2015 at 6:48 AM, Lennart Poettering
  mzta...@0pointer.de wrote:
  On Tue, 10.02.15 13:59, Philip Withnall
  (phi...@tecnocode.co.uk) wrote:
 
I am pretty sure if you do async IO like
 gio does
  for every single
file access you'll just complicate your
 program
  and make it
substantially slower. For small files
 normal,
  synchronous disk access
is a ton faster than dispatching things
 to
  background threads, and
back...
  
   The problem is that GIO can’t know which
 accesses
  are to small, local
   files, and which aren’t. It already
 optimises reads
  from pollable
   streams (sockets) by keeping them in the
 main thread
  and adding them
   into the main poll() call.
 
  Well, but the developer frequently knows
 that. He
  knows that the
  config file in ~/.config is not going to be
 more than
  a few K

Re: Discouraging use of sync APIs

[Philip Withnall]

On Tue, 2015-02-10 at 08:49 -0800, Jasper St. Pierre wrote:
 Right now the way g_file_read_async works is by scheduling a task on a
 worker thread, having the worker thread do the async read, and then
 returning a result.
 
 As such, it's impossible to have two async reads done at the same
 time, which is really unfortunate from my understanding. If I'm
 reading a large file, and then a small file, the large file needs to
 fully complete before the small file is dispatched from the async
 queue.

From my reading of the GTask code, it should support 10 parallel reads
(or other operations) since it uses a thread pool with maximum 10
threads. See g_task_thread_pool_init(). Is this not the case?

 Additionally, when profiling GNOME on ARM, I've been seeing a lot of
 cases of users using g_file_read_async() just in case for no
 particular reason, which causes several locks to be taken, severely
 slowing performance.

I guess the particular reason depends on what’s being read, as discussed
in the rest of the thread.

 We need to seriously improve our async performance before asking
 people to use it.

That’s important, but I wouldn’t say it’s a blocker. The original idea
of this thread was to try and address the situation where non-expert
developers are using sync APIs when they should be using async ones, and
then tying themselves in all sorts of knots. I suspect that hitting
these sorts of performance problems is not going to be an issue for them
— but freezing up the main loop is.

I could be wrong, but that’s how I see it.

Philip

 On Tue, Feb 10, 2015 at 6:48 AM, Lennart Poettering
 mzta...@0pointer.de wrote:
 On Tue, 10.02.15 13:59, Philip Withnall
 (phi...@tecnocode.co.uk) wrote:
 
   I am pretty sure if you do async IO like gio does for
 every single
   file access you'll just complicate your program and make
 it
   substantially slower. For small files normal, synchronous
 disk access
   is a ton faster than dispatching things to background
 threads, and
   back...
 
  The problem is that GIO can’t know which accesses are to
 small, local
  files, and which aren’t. It already optimises reads from
 pollable
  streams (sockets) by keeping them in the main thread and
 adding them
  into the main poll() call.
 
 Well, but the developer frequently knows that. He knows that
 the
 config file in ~/.config is not going to be more than a few K.
 And
 that it hence is fine to access it synchronously...
 
   Also, glib has wrappers for making mmaping available to
 programs, to
   improve seldom-accessed sparse databases efficient, do you
 want to
   prohibit that too?
 
  No, mmap() is clearly a tool for a different kind of
 problem. If you’re
  accessing an mmap()ed file, you need to be sure it’s local
 anyway, I
  think? GMappedFile doesn’t have async versions of its
 methods,
  presumably for this reason.
 
 mmap() works pretty Ok these days over NFS. Concurrent access
 doesn't. But as long as you just want to access something,
 it's
 fine...
 
 That said it's probably not a good idea to use mmap() for
 stuff below
 $HOME...
 
  As above, how about making that line the distinction between
 calling
  functions from gstdio.h and using GIO? In the former case,
 you know
  you’re operating on local files. In the latter, you could be
 operating
  on files from the moon.
 
 I'd always leave some freedom for the developers. It is
 certainly good
 to document things and push people into the right directions,
 but I
 think there are many cases where the developer should have
 every right
 to prefer sync access for valid reasons, even from the main
 loop...
 
 Lennart
 
 --
 Lennart Poettering, Red Hat
 ___
 desktop-devel-list mailing list
 desktop-devel-list@gnome.org
 https://mail.gnome.org/mailman/listinfo/desktop-devel-list
 
 
 
 -- 
   Jasper
 



signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Discouraging use of sync APIs

[Philip Withnall]

On Tue, 2015-02-10 at 10:30 -0800, Jasper St. Pierre wrote:
 One quick example: calling g_file_read_async on a GResourceFile spawns
 a new thread and does a synchronous stream read from a block already
 in memory.
 
 
 It should just be a single g_bytes_ref, but we have three different
 classes created, a thread spawned, and a large amount of locks to do
 the equivalent of memcpy.

That’s not good at all, and seems like it should be easy to fix by
overriding the read_async vfunc for GResourceFile. Is there a bug filed
about it?

Philip

 On Tue, Feb 10, 2015 at 8:49 AM, Jasper St. Pierre
 jstpie...@mecheye.net wrote:
 Right now the way g_file_read_async works is by scheduling a
 task on a worker thread, having the worker thread do the async
 read, and then returning a result.
 
 As such, it's impossible to have two async reads done at the
 same time, which is really unfortunate from my understanding.
 If I'm reading a large file, and then a small file, the large
 file needs to fully complete before the small file is
 dispatched from the async queue.
 
 Additionally, when profiling GNOME on ARM, I've been seeing a
 lot of cases of users using g_file_read_async() just in case
 for no particular reason, which causes several locks to be
 taken, severely slowing performance.
 
 
 We need to seriously improve our async performance before
 asking people to use it.
 
 
 On Tue, Feb 10, 2015 at 6:48 AM, Lennart Poettering
 mzta...@0pointer.de wrote:
 On Tue, 10.02.15 13:59, Philip Withnall
 (phi...@tecnocode.co.uk) wrote:
 
   I am pretty sure if you do async IO like gio does
 for every single
   file access you'll just complicate your program
 and make it
   substantially slower. For small files normal,
 synchronous disk access
   is a ton faster than dispatching things to
 background threads, and
   back...
 
  The problem is that GIO can’t know which accesses
 are to small, local
  files, and which aren’t. It already optimises reads
 from pollable
  streams (sockets) by keeping them in the main thread
 and adding them
  into the main poll() call.
 
 Well, but the developer frequently knows that. He
 knows that the
 config file in ~/.config is not going to be more than
 a few K. And
 that it hence is fine to access it synchronously...
 
   Also, glib has wrappers for making mmaping
 available to programs, to
   improve seldom-accessed sparse databases
 efficient, do you want to
   prohibit that too?
 
  No, mmap() is clearly a tool for a different kind of
 problem. If you’re
  accessing an mmap()ed file, you need to be sure it’s
 local anyway, I
  think? GMappedFile doesn’t have async versions of
 its methods,
  presumably for this reason.
 
 mmap() works pretty Ok these days over NFS. Concurrent
 access
 doesn't. But as long as you just want to access
 something, it's
 fine...
 
 That said it's probably not a good idea to use mmap()
 for stuff below
 $HOME...
 
  As above, how about making that line the distinction
 between calling
  functions from gstdio.h and using GIO? In the former
 case, you know
  you’re operating on local files. In the latter, you
 could be operating
  on files from the moon.
 
 I'd always leave some freedom for the developers. It
 is certainly good
 to document things and push people into the right
 directions, but I
 think there are many cases where the developer should
 have every right
 to prefer sync access for valid reasons, even from the
 main loop...
 
 Lennart
 
 --
 Lennart Poettering, Red Hat
 ___
 desktop-devel-list mailing list
 desktop-devel-list@gnome.org
 https

Re: Discouraging use of sync APIs

[Philip Withnall]

On Tue, 2015-02-10 at 15:48 +0100, Lennart Poettering wrote:
   Also, glib has wrappers for making mmaping available to programs, to
   improve seldom-accessed sparse databases efficient, do you want to
   prohibit that too?
  
  No, mmap() is clearly a tool for a different kind of problem. If you’re
  accessing an mmap()ed file, you need to be sure it’s local anyway, I
  think? GMappedFile doesn’t have async versions of its methods,
  presumably for this reason.
 
 mmap() works pretty Ok these days over NFS. Concurrent access
 doesn't. But as long as you just want to access something, it's
 fine...
 
 That said it's probably not a good idea to use mmap() for stuff below
 $HOME...

OK, that’s good to know. In any case, GMappedFile doesn’t have async
methods, so it’s a moot point.

  As above, how about making that line the distinction between calling
  functions from gstdio.h and using GIO? In the former case, you know
  you’re operating on local files. In the latter, you could be operating
  on files from the moon.
 
 I'd always leave some freedom for the developers. It is certainly good
 to document things and push people into the right directions, but I
 think there are many cases where the developer should have every right
 to prefer sync access for valid reasons, even from the main loop...

I understand where you’re coming from; we should not be irritating
experienced developers. I completely agree.

What do you think of the proposal to use sync gstdio.h for
small/local/pseudo-file-system I/O and async GIO for all other I/O?

It might turn out that runtime checks are just not feasible, but in that
case I think we still need some way of solving the original problem:
that people are using sync calls and blocking up the main loop.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Feedback from downstreams presentation from DX hackfest 2015

[Philip Withnall]

On Mon, 2015-02-02 at 12:19 +0100, Olav Vitters wrote:
 On Mon, Feb 02, 2015 at 08:05:00AM +, Philip Withnall wrote:
  It was suggested that I send the presentation to DDL, since it might be
  of general interest. I haven’t modified it from the hackfest version, so
  please let me know if you have any questions.
 
 Can we assume that most still needs to be actioned? Also interested what
 discussions there were during the hackfest to improving this. E.g.
 Should we maybe reach out to our advisory board? Some things mentioned
 lack of documentation. So with DX hackfest and documentation at same
 time I also wonder if there were any possibilities to improve this.

Here’s my list of what has been done for each point, and what still
needs doing. Apologies for the messy list, but since a lot of points
have been filed as bugs already, there seems little point in writing
them up nicely. They should roughly correspond to the points in the
presentation, with duplicates removed. If anything’s unclear, please
ask!

The ‘TODO’ items I still need to follow up on, but there are some open
questions next to them (and a few other points). I’ll bring some of
those up on DDL in the near future, but any ideas people have about
fixing them are welcome any time.

SVGs in GtkImages don’t scale with the widget
 • https://bugzilla.gnome.org/show_bug.cgi?id=728476
Some properties are confusing, such as
GtkCellRendererText:editable[-set]
 • https://bugzilla.gnome.org/show_bug.cgi?id=744052
Document GtkBuilder behaving like a factory
 • https://bugzilla.gnome.org/show_bug.cgi?id=744053
gtkmm documentation search in Devhelp
 • https://bugzilla.gnome.org/show_bug.cgi?id=743918
Documentation of widget event masks and event propagation is opaque
 • https://bugzilla.gnome.org/show_bug.cgi?id=744054
Deprecations should be documented with a rationale and replacement API,
and hopefully also a porting guide
 • https://bugzilla.gnome.org/show_bug.cgi?id=744055
 • https://bugzilla.gnome.org/show_bug.cgi?id=744057
Project bootstrapping
 • To be done in Builder
Bootstrapping problems with gettext, intltool, gtk-doc
 • To be done in Builder
Indicating licencing in files
 • Tartan bug 844/3c7
Knowing which files to put in git
 • To be done in Builder
Parallel installability of libraries
 • https://bugzilla.gnome.org/show_bug.cgi?id=376123
 • TODO: See if any other documentation/examples needs updating
Total confusion over LT versioning
 • Not sure there’s anything which can be done about this — ideas?
Namespacing confusion for GObject
 • (random assortment of related bugs which I will try and go through)
 • https://bugzilla.gnome.org/show_bug.cgi?id=742587
 • https://bugzilla.gnome.org/show_bug.cgi?id=658766
 • https://bugzilla.gnome.org/show_bug.cgi?id=705056
 • https://bugzilla.gnome.org/show_bug.cgi?id=731997
Encapsulation of members in GObject private structs
 • https://bugzilla.gnome.org/show_bug.cgi?id=744060
 • TODO: Write a HowDoI on subclassing?
Repeated, inefficient use of GList
 • https://bugzilla.gnome.org/show_bug.cgi?id=741024
 • https://bugzilla.gnome.org/show_bug.cgi?id=741779
 • TODO: Can we do more? Stop promoting linked lists somehow?
Confusion between sync, async and threads
 • https://bugzilla.gnome.org/show_bug.cgi?id=741779
 •
https://mail.gnome.org/archives/desktop-devel-list/2015-February/msg00038.html
- Static analysis
- Dynamic checking (warn if sync functions called in the main loop)
- Documentation: hide sync functions unless explicitly asked for;
  maybe only show async functions in index, add link from them to
  sync functions to show that documentation
- Documentation for best practices for structuring async calls?
Massive over-use of g_strdup()
 • https://bugzilla.gnome.org/show_bug.cgi?id=622721
Persistent confusion over g_free(NULL)
 • https://bugzilla.gnome.org/show_bug.cgi?id=741779
No compiler, GIR or gtk-doc warnings enabled
 • https://wiki.gnome.org/Projects/GnomeCommon/Migration
 • https://wiki.gnome.org/Initiatives/GnomeGoals/ModernAutotools
 • http://www.gnu.org/software/autoconf-archive/ax_compiler_flags.html
No gtk-doc tests enabled
 • https://git.gnome.org/browse/gtk-doc/commit/?id=034942538432979f51e74
Confusion over documentation generation with gdbus-codegen
 • https://bugzilla.gnome.org/show_bug.cgi?id=743294
Tests were not automatable
 • For example, some tests in the wild ask for user input, and print
   output rather than asserting it’s some known, expected value. The
   user is expected to check the output manually.
 • Not sure if there’s anything which can be done upstream to discourage
   this.
Tests were not hooked up to CI or other tooling
 • TODO: Somehow make GTest easier to hook up to CI? TAP support is a
   good step. What else can be done?
APIs were untestable by design
 • This arises from use of global resources, such as databases in the
   home directory, or system services, by code under test. The fix is to
   structure the code to allow those resources

Re: Discouraging use of sync APIs

[Philip Withnall]

On Mon, 2015-02-09 at 14:55 +0100, Bastien Nocera wrote:
 On Mon, 2015-02-09 at 13:42 +, Philip Withnall wrote:
  On Mon, 2015-02-09 at 14:02 +0100, Bastien Nocera wrote:
   On Mon, 2015-02-09 at 12:53 +, Emmanuele Bassi wrote:
   snip
I do agree with Philip's proposal of warning if the sync API is called
inside the default main context, even if there's the obvious issue of
console-only code that still uses a main loop, but does not have
interactivity issues.
   
   I wouldn't want that. I can certainly see that as a necessary evil (say
   on application startup), and would rather it threw an error if that sync
   call took too long instead.
  
  Why can the startup code not use a GMainLoop? Is there some pattern I’ve
  missed? I thought in all such situations you could still do with a main
  loop so you could handle Ctrl+C nicely.
 
 What's handling Ctrl+C nicely? I can Ctrl+C stuck applications fine in
 most cases, and I wouldn't want the handling of such a feature to be the
 main driver behind my API decisions.

Cancelling any ongoing operations and rolling back any in-progress
changes, I guess. That’s a pretty vague answer. One example in my head
at the moment is git-bz: it needs to handle sigint to try and keep its
changes to .git and Bugzilla atomic, by preventing termination while
modifying them.

But yes, in the case you’re just loading resources from disk, I don’t
suppose there’s any need to handle sigint specially.

Though if you are (for example) loading resources synchronously on
startup, you could parallelise the I/O using the main loop.

 On startup, you should try to get the maximum amount of work done in the
 minimum amount of time. In some cases, pre-populating the UI (while
 blocking the mainloop) is a better idea than seeing jarring effects
 after the fact, once the UI has been loaded.

True. Should that not be implemented by freezing those individual
widgets while populating them (for example, using the standard populate
a GtkTreeModel before setting GtkTreeView:model trick), rather than
blocking the entire main loop? That would allow other widgets to remain
responsive.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Discouraging use of sync APIs

[Philip Withnall]

On Tue, 2015-02-10 at 11:20 +0100, Lennart Poettering wrote:
 On Mon, 09.02.15 10:47, Philip Withnall (phi...@tecnocode.co.uk) wrote:
 
  Hi all,
  
  From some feedback from real-world users of GLib/GIO (see the ‘Feedback
  from downstreams’ thread), and just from looking at our own code in
  GNOME, it seems that people persistently use sync versions of APIs in
  the main thread when they should be using async ones.*
 
 [...]
 
  * There are definitely legitimate uses of sync APIs, but not from the
  main thread, ignoring trivial command line utilities (and even then, if
  they want to handle signals properly, they should be running a main
  loop).
 
 Uh, oh. This is really oversimplifying things. Note that on Linux disk
 IO is generally synchronous, and it's good that way, and you cannot
 really avoid it. I mean, never forget that your executable and its
 shared libraries are all mapped into memory and access to their pages
 results in synchronous IO. Even if you wanted you couldn't make that
 async... 

The difference there is that you cannot do _anything_ until the code you
want to execute is paged in. For I/O operations on files, you can be
redrawing the UI, accessing other files, etc.

 I am pretty sure if you do async IO like gio does for every single
 file access you'll just complicate your program and make it
 substantially slower. For small files normal, synchronous disk access
 is a ton faster than dispatching things to background threads, and
 back... 

The problem is that GIO can’t know which accesses are to small, local
files, and which aren’t. It already optimises reads from pollable
streams (sockets) by keeping them in the main thread and adding them
into the main poll() call.

How about using the distinction between GIO and gstdio.h? Functions like
g_file_get_contents(), or g_open() + read() + g_close() which can safely
be used on small, local files, can continue to be called from the main
thread? That would be fine for system utilities which _know_ they will
operate on a local file system.

For typical desktop applications, though, the home directory could be an
NFS share and all the ~/.config files could be hidden behind noticeable
latency. For those applications, I think GIO should continue to be used,
and used asynchronously.

 Also, glib has wrappers for making mmaping available to programs, to
 improve seldom-accessed sparse databases efficient, do you want to
 prohibit that too?

No, mmap() is clearly a tool for a different kind of problem. If you’re
accessing an mmap()ed file, you need to be sure it’s local anyway, I
think? GMappedFile doesn’t have async versions of its methods,
presumably for this reason.

 Moreover on Linux file systems like /proc, /sys, /run, /tmp are known
 to not be backed by slow physical IO, hence its really pointless
 accessing them via async IO...

I suggest gstdio.h + normal POSIX read() (as above).

 Then, during start-up of your app, when you need to read some config
 file or so before you can do your first steps, why bother with async
 stuff? You need to wait for for reading/parsing that file anyway
 before you can proceed?

This seems to be the only use case where sync I/O calls still seem, to
me, to be reasonable. But in my opinion we could suffer the loss of that
convenience if doing so means we can easily detect other sync calls from
the main thread which _will_ cause problems.

 Hence, my recommendation would be to draw the line somewhere between:
 potentially unbounded user payload data and configuration/control
 data. For the former it would be wise to encourage async IO but for
 the latter certainly not. If you follow what I want to say...

As above, how about making that line the distinction between calling
functions from gstdio.h and using GIO? In the former case, you know
you’re operating on local files. In the latter, you could be operating
on files from the moon.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Discouraging use of sync APIs

[Philip Withnall]

On Mon, 2015-02-09 at 10:57 +, Debarshi Ray wrote:
 On Mon, Feb 09, 2015 at 10:47:37AM +, Philip Withnall wrote:
  I guess there are two approaches: making async APIs easier to use; and
  discouraging use of sync ones. I think the GAsyncResult framework we?ve
  got is pretty good, and I can?t think of a way to simplify it.
 
 One convention that I like is to use a _sync suffix for sync APIs,
 instead of an _async suffix for async ones, because it lets me spot
 synchronous calls with grep.
 
 A little sad that there are things like g_file_read that are sync, not
 async.

We had a brief discussion about that at the DX hackfest, and I think the
consensus was that the current naming scheme (no suffix = sync, ‘_async’
suffix = async) is unfortunate, but has to be kept for consistency
reasons. It would be really confusing to switch to a new scheme while
keeping the old one. :-(

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Discouraging use of sync APIs

[Philip Withnall]

Hi all,

From some feedback from real-world users of GLib/GIO (see the ‘Feedback
from downstreams’ thread), and just from looking at our own code in
GNOME, it seems that people persistently use sync versions of APIs in
the main thread when they should be using async ones.*

How can we fix this?

I guess there are two approaches: making async APIs easier to use; and
discouraging use of sync ones. I think the GAsyncResult framework we’ve
got is pretty good, and I can’t think of a way to simplify it.
(Suggestions welcome.)

For discouraging use of sync APIs, I have two strawman ideas:
 1. Hide the sync APIs in the documentation index, and only allow their
documentation to be viewed from a link from the async versions. That
would potentially fix the case where someone searches for ‘file
read’, finds g_file_read() as the first result, and never looks as
far as g_file_read_async().
 2. A bit more extreme: emit a warning every time a sync API is run in
the main thread (but not if it’s run in a worker thread). This
should capture precisely the situations we want to avoid (sync calls
blocking the main thread’s loop), but is a bit ugly, and people
don’t always check their code for runtime warnings.
 2b. Similarly, we could potentially statically analyse which threads
 each sync function could potentially be called from, and warn if
 sync functions are potentially called from the main thread. But I
 haven’t put much thought into how such an analysis would work, and
 it wouldn’t be trivial to implement.

What are people’s thoughts? 

Philip

* There are definitely legitimate uses of sync APIs, but not from the
main thread, ignoring trivial command line utilities (and even then, if
they want to handle signals properly, they should be running a main
loop).


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Discouraging use of sync APIs

[Philip Withnall]

On Mon, 2015-02-09 at 14:02 +0100, Bastien Nocera wrote:
 On Mon, 2015-02-09 at 12:53 +, Emmanuele Bassi wrote:
 snip
  I do agree with Philip's proposal of warning if the sync API is called
  inside the default main context, even if there's the obvious issue of
  console-only code that still uses a main loop, but does not have
  interactivity issues.
 
 I wouldn't want that. I can certainly see that as a necessary evil (say
 on application startup), and would rather it threw an error if that sync
 call took too long instead.

Why can the startup code not use a GMainLoop? Is there some pattern I’ve
missed? I thought in all such situations you could still do with a main
loop so you could handle Ctrl+C nicely.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Feedback from downstreams presentation from DX hackfest 2015

[Philip Withnall]

On Mon, 2015-02-02 at 16:31 +0100, Olav Vitters wrote:
 On Mon, Feb 02, 2015 at 02:37:32PM +, Philip Withnall wrote:
  What do you mean by reaching out to the advisory board? Reaching out for
  further feedback from them as downstreams, or reaching out for resources
  to fix such issues? I think the former would be interesting. I’m not
  sure the latter is worth their time, since it’s a very loosely defined
  goal.
 
 I meant for resources.
 
 Though we should have a team to continuously reach out, find out issues
 and act on them. If we don't act/respond (which could be about going
 back and saying we cannot change it), then no point in asking.

Agreed, there is no point in asking if we can’t follow up. Other than
that, I don’t really have an opinion about asking the advisory board.

   4. Instant gratification: documentation changes should be visible
  instantly, rather than waiting 6 months for a GNOME release before
  the docs hit developer.gnome.org.
 
 The current infrastructure really requires tarballs. We could reuse
 continuous integration builds to spit out tarballs to feed to
 developer.gnome.org. It would not be instant, but you'd cut it down to
 15 minutes maybe? It would be a huge improvement.

I think 15 minutes is still long enough for someone writing docs to lose
interest and start looking at pictures of cats instead. I think we
should aim to get it down to the time taken to rebuild the docs, so
around 1 minute.

If that requires a big change in the current infrastructure, then I
guess we need to make a big change in the current infrastructure.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Feedback from downstreams presentation from DX hackfest 2015

[Philip Withnall]

On Mon, 2015-02-02 at 17:11 +0100, Sébastien Wilmet wrote:
 On Mon, Feb 02, 2015 at 04:51:32PM +0100, Aleksander Morgado wrote:
  Devhelp could also have a 'next' or 'latest' default profile which could
  sync the daily? built documentation from gnome.org. Although not sure
  how useful this could be if we setup the default jhbuild-based profile.
  
  I personally think that the profiles idea could be a good one, also as a
  first step for the next stuff. What do others think?
 
 Having only the 'latest' profile should be sufficient for most
 developers, for each major version of a library of course (e.g. the
 latest gtk 2.x and the latest gtk 3.x). Each symbol has the Since:
 version information anyway. And by having the latest documentations,
 developers are aware of new features and don't reinvent the wheel or
 don't use APIs that are already deprecated since a later version.
 
 But 'latest' should have two variants: stable and unstable.
 
 Of course, having other profiles for specific versions of GNOME would be
 nice, but the 'latest' profiles should be the default and recommended in
 my opinion.

I believe there are various APIs where the ‘Since’ annotations have not
been, and are not, added correctly. I suspect we will have to aim for
having profiles for each minor release of each library — but that should
not be any more complex than supporting latest stable and latest
unstable profiles, which I think is a great idea.

One question: does ‘latest unstable’ mean git master, or the latest
release in the unstable branch? There are plenty of times I’ve needed
documentation for unreleased functions newly added in git master; I
can’t think of any time I’ve needed documentation for the latest
unstable release instead.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Feedback from downstreams presentation from DX hackfest 2015

[Philip Withnall]

On Mon, 2015-02-02 at 19:56 -0800, Philip Chimento wrote:
 On Mon, Feb 2, 2015 at 6:37 AM, Philip Withnall
 phi...@tecnocode.co.uk wrote:
 On Mon, 2015-02-02 at 12:19 +0100, Olav Vitters wrote:
  On Mon, Feb 02, 2015 at 08:05:00AM +, Philip Withnall
 wrote:
   It was suggested that I send the presentation to DDL,
 since it might be
   of general interest. I haven’t modified it from the
 hackfest version, so
   please let me know if you have any questions.
 
  Can we assume that most still needs to be actioned? Also
 interested what
  discussions there were during the hackfest to improving
 this. E.g.
  Should we maybe reach out to our advisory board? Some things
 mentioned
  lack of documentation. So with DX hackfest and documentation
 at same
  time I also wonder if there were any possibilities to
 improve this.
 
 Some of the items need actioning via bugs, which I will sort
 out. Some
 of them have already been fixed (either as part of the client
 work by
 Collabora, or by others in the time since). Some of them are
 unfixable,
 and can only be used as general guidelines for trying to avoid
 such
 problems in future (e.g. in new API designs).
 
 
 Thanks for sharing! One point from the slides was familiar enough to
 jump out at me, and it's a bit of a strange one because it's not
 actionable via a bug report nor a matter of API design:
 
 
  New functionality needs documenting: GtkBuilder templates were
 blogged about extensively, but are given a single sentence in the
 manual
 
 
 
 This actually happens quite a lot, in my experience. For someone just
 starting out, blogs are not very discoverable, and they won't know the
 blogger by name as maintainer of X. I wonder if we could take this
 existing enthusiasm for documenting something in a blog post and
 channel it into the API docs. For example I really like your own
 occasional series of posts about GObject stuff (e.g. [1]) but it would
 probably help more people if something like these code examples
 existed in the API docs.
 
 
 I do get that blog posts are more fun to write because they are
 informal, and they also build your reputation online whereas API docs
 are anonymous.

Totally agreed, and I’m very guilty of this. :-(

The only solution I can think of is for people to comment on such blog
posts and gently remind the author to ensure everything’s documented
upstream.

Matthias did this quite well with one of his GUADEC GTK+ presentations
(IIRC [1]) where the presentation was basically just various bits of
upstream documentation which he had updated specially. Maybe we could
see more of that in future?

Philip

[1]: http://blogs.gnome.org/mclasen/2014/07/28/a-talk-in-9-images/



signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Feedback from downstreams presentation from DX hackfest 2015

[Philip Withnall]

On Tue, 2015-02-03 at 11:26 +0100, Murray Cumming wrote:
 On Tue, 2015-02-03 at 09:23 +0100, Murray Cumming wrote:
  Searching of gtkmm documentation has been working in devhelp for
  years,
  and I've just tested it here both in regular Ubuntu and in jhbuild. I
  mean, searching of the index, of course. I don't think devhelp offers
  full text search for any documentation. If there's any particular
  problem, or if I've misunderstood, we'd like to hear about it.
 
 Actually, I think I just found a big problem with gtkmm's devhelp index:
 https://bugzilla.gnome.org/show_bug.cgi?id=743918
 
 It seems to be a doxygen bug - hopefully we can figure out a simple test
 case.

That might explain things! The point I made in the presentation about
this contains all the information I know about the problems with gtkmm
documentation — someone else was on that project, so all my information
is second-hand. I’m getting in touch with them to see if we can clarify
what the problem is, and whether it’s the one you’ve just found.

Cheers,
Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Feedback from downstreams presentation from DX hackfest 2015

[Philip Withnall]

On Mon, 2015-02-02 at 14:47 +0100, Bastien Nocera wrote:
 On Mon, 2015-02-02 at 08:05 +, Philip Withnall wrote:
  At the DX hackfest last week[1] I gave an informal presentation about
  some of the work Collabora has been doing with GLib, GStreamer, Clutter,
  GTK+ and related technologies on client projects, specifically focusing
  on the feedback from two client projects, and the problems we
  encountered downstream which could motivate upstream development in the
  future.
  
  We discussed various issues around those in the presentation, and have
  some ideas for fixing some of the problems which we did not manage to
  fix at the time of doing the work downstream. Some of these problems are
  inherently unfixable, though, and are mentioned purely to motivate
  future API designs to direct people towards a single, correct, intended
  usage of the API, as much as that is possible.
  
  It was suggested that I send the presentation to DDL, since it might be
  of general interest. I haven’t modified it from the hackfest version, so
  please let me know if you have any questions.
 
 I think a lot of the problems in this report need bugs created, so that
 the information trickles down to the appropriate maintainers. It would
 also greatly help with tracking of the overall goals of improving our
 stack.

Yup, that’s definitely on my to-do list. It will take an hour or two
though, so it might take me a few days to get to.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Feedback from downstreams presentation from DX hackfest 2015

[Philip Withnall]

On Mon, 2015-02-02 at 07:55 -0800, Jasper St. Pierre wrote:
 
 On Feb 2, 2015 6:37 AM, Philip Withnall phi...@tecnocode.co.uk
 wrote:
 
  On Mon, 2015-02-02 at 12:19 +0100, Olav Vitters wrote:
   On Mon, Feb 02, 2015 at 08:05:00AM +, Philip Withnall wrote:
It was suggested that I send the presentation to DDL, since it
 might be
of general interest. I haven’t modified it from the hackfest
 version, so
please let me know if you have any questions.
  
   Can we assume that most still needs to be actioned? Also
 interested what
   discussions there were during the hackfest to improving this. E.g.
   Should we maybe reach out to our advisory board? Some things
 mentioned
   lack of documentation. So with DX hackfest and documentation at
 same
   time I also wonder if there were any possibilities to improve
 this.
 
  Some of the items need actioning via bugs, which I will sort out.
 Some
  of them have already been fixed (either as part of the client work
 by
  Collabora, or by others in the time since). Some of them are
 unfixable,
  and can only be used as general guidelines for trying to avoid such
  problems in future (e.g. in new API designs).
 
  What do you mean by reaching out to the advisory board? Reaching out
 for
  further feedback from them as downstreams, or reaching out for
 resources
  to fix such issues? I think the former would be interesting. I’m not
  sure the latter is worth their time, since it’s a very loosely
 defined
  goal.
 
  There were some DX–documentation discussions, although I wasn’t
 involved
  in all of them so I can’t report fully. One interesting discussion
 came
  up with a set of requirements for any replacement for gtk-doc:
   1. Do not want to write XML in documentation comments. Too painful,
 and
  a steep learning curve.
   2. No version control program (but wikis’ version control is fine).
 Too
  much of a barrier for contribution.
   3. No waiting for review of documentation changes — post-hoc gives
 a
  much lower barrier to contribution instead.
   4. Instant gratification: documentation changes should be visible
  instantly, rather than waiting 6 months for a GNOME release
 before
  the docs hit developer.gnome.org.
   5. Documents need to be available offline in Devhelp.
   6. Devhelp needs to give you documentation for the version of the
  library you’re using (e.g. in JHBuild), not the version
 installed on
  your system, which is invariably outdated.
   7. Automatically generate documentation from annotations as much as
  possible (e.g. eliminate ‘Free return value with g_free()’ in
 favour
  of (transfer full)).
   8. Topic-based help which can be reorganised dynamically; eliminate
  in-order Docbook indexes. Basically the Mallard approach for
  reference documentation.
   9. Don’t put big code examples in C comments; move them to separate
 C
  files instead, which can be compiled stand-alone. Have a way of
  limiting what gets rendered in the docs, plus a link to the full
  source.
  10. Don’t parse C with regexps; use documentation from GIR files,
 and
  allow g-ir-scanner to do the C parsing legwork.
 
 I'm confused. You want both a wiki, and docs embedded in comments?
 What are you imagining here?

Sorry, I should have explained that more clearly.

Documentation is generated from C comments as at the moment,
reformatted, uploaded to developer.gnome.org, and then some kind of
online interface can be used to edit the documentation (and possibly add
comments to it) with instant gratification. That’s what the first few
points are about.

There are no requirements covering how these online edits would be fed
back into the C comments, but presumably that would happen somehow. The
fact that there are no requirements means nobody in the hackfest
discussion had strong feelings about how it should be done.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Feedback from downstreams presentation from DX hackfest 2015

[Philip Withnall]

At the DX hackfest last week[1] I gave an informal presentation about
some of the work Collabora has been doing with GLib, GStreamer, Clutter,
GTK+ and related technologies on client projects, specifically focusing
on the feedback from two client projects, and the problems we
encountered downstream which could motivate upstream development in the
future.

We discussed various issues around those in the presentation, and have
some ideas for fixing some of the problems which we did not manage to
fix at the time of doing the work downstream. Some of these problems are
inherently unfixable, though, and are mentioned purely to motivate
future API designs to direct people towards a single, correct, intended
usage of the API, as much as that is possible.

It was suggested that I send the presentation to DDL, since it might be
of general interest. I haven’t modified it from the hackfest version, so
please let me know if you have any questions.

Slides:
http://people.collabora.com/~pwith/feedback-from-downstreams/presentation.pdf
Handout:
http://people.collabora.com/~pwith/feedback-from-downstreams/handout.pdf
Source:
http://cgit.collabora.com/git/user/pwith/feedback-from-downstreams.git/
Tarball:
http://people.collabora.com/~pwith/feedback-from-downstreams/source.tar.xz

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Installing DBus interface files for services

[Philip Withnall]

On Wed, 2015-01-28 at 20:59 +0100, Lennart Poettering wrote:
 On Tue, 13.01.15 10:43, Cosimo Cecchi (cosi...@gnome.org) wrote:
 
  Hi all,
  
  I was wondering if there's any reason we typically don't install on the
  system DBus XML interface files for services. On my system, I can see a
  bunch of definitions in /usr/share/dbus-1/interfaces, but it's by no means
  a complete list of all the services in the system.
 
 Well, systemd at least used to install them, but we don't do this
 anymore, since there's really no clear consumer of it. Also, it's not clear
 how precisely to put them there. Most projects that put the files
 their, put a full descriptition of an object node there, which might
 have multiple interfaces. So you end up encoding multiple interfaces
 in a single file. But now, if you have 8 different interfaces, and use
 them in different combinations, what would you even place on disk?
 each interface only once? or every possible combination of interfaces
 (which would be highly redundant?
 
 We eventually decided to remove all this from systemd since this also
 created issues with cross-platform builds, since the introspection was
 created dynamically, and thus required built code to be executed to
 get them...

True, but not a problem in the (common?) case that the module installing
an introspection XML file is actually generating its code from that
file, rather than generating the file from its code.

 I think it's probably a better idea to pull out the stuff from running
 daemons. 

Since Cosimo’s goal here is using gdbus-codegen to generate code using
those interfaces, I don’t think it makes sense to pull the introspection
XML from processes which would have to be running at build time.

 Or, if you really want to keep the dir, then at least standaridze what
 should be in it, how the files are named, and such. AMong other things
 that means that it really should only include interface, but no
 node elements, if you follow what i mean...

Sounds better to me.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Proposal for personal WIP branches in git

[Philip Withnall]

On Thu, 2015-01-29 at 10:03 +0100, Jens Georg wrote:
  There’s a bit of discussion going on at the DX hackfest about personal
  branches of other people’s modules.
  
  People here agree that it would be pretty good if we standardised on:
  wip/$nick/$branch_name
  as the name for your branches on someone else’s module. These could be
  pushed to git.gnome.org without worrying about asking permission from
  the module maintainer.
  
  ‘wip’ to allow force-pushes, and $nick to disambiguate.
  
  If nobody has any objections, I could add this to
  https://wiki.gnome.org/Git/FAQ
 
 +1 from me. This would embrace the people that don't like to use github 
 for such things because reasons.

Thanks all for the feedback. I’ve added the proposal to the wiki as an
FAQ entry:

https://wiki.gnome.org/Git/FAQ#How_do_I_create_a_personal_WIP_branch.3F

Please discuss changes here rather than silently editing the wiki.

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Search Providers

[Philip Withnall]

On Fri, 2015-01-30 at 09:43 -0500, Erick Pérez Castellanos wrote:
  You could get the best of both worlds by doing what gnome-documents
  does (ie. use an inactivity-timeout):
  https://git.gnome.org/browse/gnome-documents/tree/src/application.js#n133
 
  That way, your process does not linger for ever, but it stays around
  long enough to cache a quick series of searches.
 
 That would present the same problem initially. We would want the app
 to return the search results as fast as it can, but since it takes
 some times to load the machiney initially, the user would have to type
 his query, wait a few seconds for the first results and then, after
 that, the querying will get a better, snappier.
 
 And this makes me think on when does gnome-shell span its
 gnome-shell-calendar-server. I'm thinking about those applications on
 Windows that registers itself to be started at launch. Do we want that
 for our users? Or it's better to slow search at times.

As I see it, the correct solution is to optimise the startup of both
search result providers (and their dependencies, e.g. folks and EDS).

Philip


signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list

Re: Installing DBus interface files for services

[Philip Withnall]

On Thu, 2015-01-15 at 14:40 +0800, Cosimo Cecchi wrote:
 On Tue, Jan 13, 2015 at 4:08 PM, Philip Withnall
 phi...@tecnocode.co.uk wrote:
 I can’t think of any reasons why not. Perhaps a GDBus automake
 snippet
 
 could be installed by GLib which:
  1. Installs D-Bus XML interface files.
  2. Includes rules for building documentation and C/H files
 from them.
  3. Validates the XML interface files for well-formedness.
 
 
 I think that would be really helpful; note that my goal here would be
 just to make things more convenient for developers and not to use it
 as a way to guarantee API stability of a service through
 auto-generated code.
 In that sense, I see your points 1), 3) and the first half of 2) as
 something typically used by the service, and the second half of 2) as
 something a consumer would call.

Yes, making sure the snippet doesn’t confuse the two use cases is a good
point.

 Another thing that to take into consideration is that some projects
 currently encode the location of the DBus interface xml into their
 pkgconfig file [1].
 Is that something we should recommend? I believe if we had this
 snippet, the macro could just do a lookup following the usual rules of
 XDG_DATA_DIRS and we could avoid using pkgconfig, but perhaps I'm
 missing something.

IMO, no, we shouldn’t recommend that. The ‘${datadir}/dbus-1/interfaces’
path should be well known, and the interface files should follow the
interface names, which are already hard-coded into the client code
anyway (since it calls them). So we can avoid running pkg-config here.

Philip



signature.asc
Description: This is a digitally signed message part
___
desktop-devel-list mailing list
desktop-devel-list@gnome.org
https://mail.gnome.org/mailman/listinfo/desktop-devel-list