Re: [Tracker] Meson build instructions for Tracker

[Sam Thursfield]

I decided to do the same for libmediaart. This was a much nicer job --
about 2 hours work in total :-)

https://bugzilla.gnome.org/show_bug.cgi?id=783562
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Sam Thursfield]

On Mon, May 22, 2017 at 12:22 PM, Carlos Garnacho  wrote:
> Right, most failing tests are the functional ones. I'm more baffled at
> the tracker-steroids test timing out here. Running it manually leaves
> it stuck awaiting for the reply of the first update, while the
> autotools generated one just succeeds.

There are a few ones that fail for me due to locale mismatches trying
to use the libtracker-direct backend. I tried setting XDG_CACHE_HOME
and XDG_DATA_HOME to an empty directory for these but I still get
issues. They seem to pass with Autotools though -- it's weird.

The functional tests need a bit of reworking to run the right
executables from the source tree. Probably the test runner should pass
these flags in through the environment.

> Hopefully functional tests will get better when we have
> domain-ontologies in place :).

Yes

> Go go go! I guess we'll have to do
> https://mail.gnome.org/archives/desktop-devel-list/2017-April/msg00091.html
> before the next release, but I can go ahead after you merge it.

Done :-)

I also added the meson.build files to EXTRA_DIST so they will appear
in future tarballs.

Sam
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Carlos Garnacho]

Hey :),

On Sun, May 21, 2017 at 11:35 PM, Sam Thursfield  wrote:
> On 5/21/17, Carlos Garnacho  wrote:
>> On Thu, Mar 30, 2017 at 11:54 PM, Sam Thursfield  wrote:
>> FWIW, the meson step fails here because I lack the gee library, which
>> we don't need anymore :), those bits can be removed from the root dir
>> and utils/tracker-resdump meson.build files.
>
> OK, done.
>
>>> You can run the test suite:
>>>
>>> ninja-build test
>>
>> Hmm, amusingly "ninja test" fails on some tests in weird ways here. I
>> must investigate further, but won't make this hold the merge.
>
> Interesting .. I see some failures as well. I'll see if I can solve
> any. The functional tests mostly fail but that's been the case anyway,
> it's just that previously they weren't run unless you manually did
> `make functional-test`.

Right, most failing tests are the functional ones. I'm more baffled at
the tracker-steroids test timing out here. Running it manually leaves
it stuck awaiting for the reply of the first update, while the
autotools generated one just succeeds.

Hopefully functional tests will get better when we have
domain-ontologies in place :).

>
>> As long as autotools config is preserved so far, I think this can be
>> merged soon so it gets more coverage through jhbuild/whatnot. At a
>> later point we can definitely consider going meson-only :).
> ...
>>
>> Please let's get this merged ASAP :), I think it looks and works good
>> enough that we get further coverage.
>
> I've pushed a new 'meson-final' branch with everything in single
> commit; I can merge this tomorrow if you're happy with it.

Go go go! I guess we'll have to do
https://mail.gnome.org/archives/desktop-devel-list/2017-April/msg00091.html
before the next release, but I can go ahead after you merge it.

Cheers,
  Carlos
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Sam Thursfield]

On 3/30/17, Sam Thursfield  wrote:
>   * There's no `make dist` equivalent. We can use `git archive` to
> produce tarballs, but these won't have the .c files generated by
> valac. That said, shipping the generated .c files does make the
> Vala preprocessor useless so it would be good if we can stop,
> but that could break things for downstreams unexpectedly.

Meson 0.41 is going to gain a `dist` target. It won't allow us to
carry on shipping pregenerated Vala code, but looks useful:

> Creating distribution tarballs is simple:
>
> ninja dist
>
> This will create a .tar.xz archive of the source code including submodules 
> without any revision
> control information. This command also verifies that the resulting archive 
> can be built, tested
> and installed. This is roughly equivalent to the distcheck target in other 
> build systems. Currently
> this only works for projects using Git and only with the Ninja backend.

>From http://mesonbuild.com/Release-notes-for-0-41-0.html

Sam
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Sam Thursfield]

On 5/21/17, Carlos Garnacho  wrote:
> On Thu, Mar 30, 2017 at 11:54 PM, Sam Thursfield  wrote:
> FWIW, the meson step fails here because I lack the gee library, which
> we don't need anymore :), those bits can be removed from the root dir
> and utils/tracker-resdump meson.build files.

OK, done.

>> You can run the test suite:
>>
>> ninja-build test
>
> Hmm, amusingly "ninja test" fails on some tests in weird ways here. I
> must investigate further, but won't make this hold the merge.

Interesting .. I see some failures as well. I'll see if I can solve
any. The functional tests mostly fail but that's been the case anyway,
it's just that previously they weren't run unless you manually did
`make functional-test`.

> As long as autotools config is preserved so far, I think this can be
> merged soon so it gets more coverage through jhbuild/whatnot. At a
> later point we can definitely consider going meson-only :).
...
>
> Please let's get this merged ASAP :), I think it looks and works good
> enough that we get further coverage.

I've pushed a new 'meson-final' branch with everything in single
commit; I can merge this tomorrow if you're happy with it.

Sam
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Carlos Garnacho]

Hi!,

On Thu, Mar 30, 2017 at 11:54 PM, Sam Thursfield  wrote:
> Hello
> I've been working on build instructions for Tracker using Meson.
> They are now pretty much ready for use!
>
> You can find them in the branch wip/sam/meson:
> https://git.gnome.org/browse/tracker/commit/?h=wip/sam/meson
>
> To use, you need to install Meson, which can either be done from Pip
> (pip3 install --user meson), from a distro package (but beware that it
> might be old), or you can run it right from Git
> (git://github.com/mesonbuild/meson).
>
> Then you do something like this:
>
> mkdir build
> cd build
> meson ..
> ninja-build

FWIW, the meson step fails here because I lack the gee library, which
we don't need anymore :), those bits can be removed from the root dir
and utils/tracker-resdump meson.build files.

>
> You can run the test suite:
>
> ninja-build test

Hmm, amusingly "ninja test" fails on some tests in weird ways here. I
must investigate further, but won't make this hold the merge.

>
> And `ninja-build install` to install of course.
>
> Meson accepts standard arguments like --prefix, but to change the
> Tracker-specific options you need to run mesonconf. Run it in the build
> directory with no arguments to see all the available options, and pass
> `-D option=value` to set something.
>
> I've compared an install of this with an equivalent Autotools build. So
> I'm confident there aren't major regressions, but it does require more
> testing. Help with that is appreciated!

As long as autotools config is preserved so far, I think this can be
merged soon so it gets more coverage through jhbuild/whatnot. At a
later point we can definitely consider going meson-only :).

>
> Here are a few remaining issues:
>
>   * There's no `make dist` equivalent. We can use `git archive` to
> produce tarballs, but these won't have the .c files generated by
> valac. That said, shipping the generated .c files does make the
> Vala preprocessor useless so it would be good if we can stop,
> but that could break things for downstreams unexpectedly.

I agree that shipping generated files is a bad idea, there's very
little scenarios where I can see this helping (perhaps embedded?), and
at that point I don't think additionally compiling vala is a big leap.

I agree with Michael in that reproducible builds are a vala problem
and we just tape over it. Introducing issues here is not quite ideal,
but there's also chances we 1) don't hit the code paths that might
result in flipped C code lines, or 2) we expose new places where this
happens, so I'd say it's more beneficial in the long term to just do
it.

>
>   * The Firefox, Thunderbird, Evolution and Nautilus plugins don't
> have Meson build rules. I'm not sure if any of these are actually
> still used, we can easily fix that if they are.

Sounds like a good time to kill them all :). They were also gone in my
past effort to divide tracker stuff (which would be nice to try again
after this is merged).

>
>   * Not every single configure flag has an equivalent in meson_options.txt

I see you've mainly killed tracker-extract toggles that had an "auto"
fallback, makes sense.

>
> Meson is still kinda beta quality but it's under active development, the
> maintainers are very helpful and responsive and the build is literally
> twice as fast than with Autotools. Plus the output is a lot clearer so all the
> compile warnings are there.
>
> If you try this, please let me know any issues and I'll be happy try and 
> triage
> and hopefully fix them.

Please let's get this merged ASAP :), I think it looks and works good
enough that we get further coverage.

Thanks,
  Carlos
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Philip Van Hoof]

Hello Michael,

I would agree with that. Just like it would be silly to ship the
generated .o files in a source package.

I think that, given that valac is a compiler like any other, the
generated .c files should not be part of the dist tarball. Instead
should the build-dependencies include a specified version of valac.

My personal recommendation would be, to all upstream packages, not to
include generated .c files to the make-dist tarball.

Before I guess it was needed, because valac was not considered a
truthful real dependency. But nowadays it is. It is a compiler like any
other. What files it generates should be of no concern to the packagers
(in my opinion), just like .o files: the packager wants the
resulting .pc, .a and .so files. Those files are more or less the same
for vala sources.

I added Jürg in CC to comment on this.


Kind regards,

Philip


On Sun, 2017-04-09 at 19:34 +0200, Michael Biebl wrote:
> 2017-04-09 19:31 GMT+02:00 Philip Van Hoof :
> > I wonder if removing those .stamp files should be part of any
> > make-target that is ~ standard for packaging tooling? In my opinion
> > should such generated .c files share the rules of .o files.
> 
> Well, that would defeat the point of shipping the generated .c files
> in the dist tarball in the first place, no?



signature.asc
Description: This is a digitally signed message part
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Michael Biebl]

2017-04-09 19:31 GMT+02:00 Philip Van Hoof :
> I wonder if removing those .stamp files should be part of any
> make-target that is ~ standard for packaging tooling? In my opinion
> should such generated .c files share the rules of .o files.

Well, that would defeat the point of shipping the generated .c files
in the dist tarball in the first place, no?
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Philip Van Hoof]

I wonder if removing those .stamp files should be part of any
make-target that is ~ standard for packaging tooling? In my opinion
should such generated .c files share the rules of .o files.

On Sun, 2017-04-09 at 19:28 +0200, Michael Biebl wrote:
> 2017-04-09 19:04 GMT+02:00 Philip Van Hoof :
> > Correct. Is there anything holding you back from rerunning valac after
> > applying a patch during dpkg-buildpackage?
> 
> No, not really. I guess I would just remove the vala.stamp files to
> force the regeneration of the .c files. That should be all that's
> needed afaics.
> 
> 



signature.asc
Description: This is a digitally signed message part
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Michael Biebl]

2017-04-09 19:04 GMT+02:00 Philip Van Hoof :
> Correct. Is there anything holding you back from rerunning valac after
> applying a patch during dpkg-buildpackage?

No, not really. I guess I would just remove the vala.stamp files to
force the regeneration of the .c files. That should be all that's
needed afaics.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Philip Van Hoof]

On Sun, 2017-04-09 at 18:54 +0200, Michael Biebl wrote:

> > Hmmrr. Patching .c files generated by valac sounds absurd. Is there ever
> > a circumstance where you want to do that instead of patching the .vapi
> > or .vala file? Because if you do, that sounds like a bug in valac
> > instead.

> Maybe just a misunderstanding. I wasn't suggesting to patch the
> generated .c files. Of course one should apply such a change to the
> .vala file. In such a case though, you need to rerun valac. And once
> you setup debian/rules to do that, you can just do it always and
> forget about the generated .c files.

Correct. Is there anything holding you back from rerunning valac after
applying a patch during dpkg-buildpackage? Consider valac to be the
compiler, in these cases. Consider patching the .c files similar to
patching the .o file.

As for Tracker's distributed release: we don't expect downstream to
patch our generated .c files (whenever they got generated by, among
others, valac). If you need to do this, we really want to know about
this for a variety of reasons (among others, understanding stack traces
from reported bugs).


Kind regards,

Philip





signature.asc
Description: This is a digitally signed message part
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Michael Biebl]

2017-04-09 17:36 GMT+02:00 Philip Van Hoof :
> On Fri, 2017-03-31 at 00:32 +0200, Michael Biebl wrote:
>
>> You are right, there is indeed an issue with valac and reproducible builds:
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802528
>>
>> I guess this should be fixed in vala proper though.
>
> nod.
>
>> Personally, I'm not a huge fan of shipping pre-generated files in a
>> dist tarball. As soon you need to patch one of the source files you
>> need to regenerate it anyway. So it's better to do it always.
>
> Hmmrr. Patching .c files generated by valac sounds absurd. Is there ever
> a circumstance where you want to do that instead of patching the .vapi
> or .vala file? Because if you do, that sounds like a bug in valac
> instead.

Maybe just a misunderstanding. I wasn't suggesting to patch the
generated .c files. Of course one should apply such a change to the
.vala file. In such a case though, you need to rerun valac. And once
you setup debian/rules to do that, you can just do it always and
forget about the generated .c files.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Philip Van Hoof]

On Fri, 2017-03-31 at 00:32 +0200, Michael Biebl wrote:

> You are right, there is indeed an issue with valac and reproducible builds:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802528
> 
> I guess this should be fixed in vala proper though.

nod.

> Personally, I'm not a huge fan of shipping pre-generated files in a
> dist tarball. As soon you need to patch one of the source files you
> need to regenerate it anyway. So it's better to do it always.

Hmmrr. Patching .c files generated by valac sounds absurd. Is there ever
a circumstance where you want to do that instead of patching the .vapi
or .vala file? Because if you do, that sounds like a bug in valac
instead.

Kind regards,

Philip






signature.asc
Description: This is a digitally signed message part
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Michael Biebl]

2017-03-31 0:12 GMT+02:00 Sam Thursfield :
> On 3/30/17, Michael Biebl  wrote:

>> So a git archive tarball would be fine with me.
>
> That's good to know! We did wonder if regenerating the .c files each
> build would have implications for reproducible-builds, but I guess if
> there are issues there it's better to fix them now than ignore them.

You are right, there is indeed an issue with valac and reproducible builds:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802528

I guess this should be fixed in vala proper though.

Personally, I'm not a huge fan of shipping pre-generated files in a
dist tarball. As soon you need to patch one of the source files you
need to regenerate it anyway. So it's better to do it always.

Regards,
Michael

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Sam Thursfield]

On 3/30/17, Michael Biebl  wrote:
> 2017-03-30 23:54 GMT+02:00 Sam Thursfield :
>> I've been working on build instructions for Tracker using Meson.
>> They are now pretty much ready for use!
>
> Nice work!

Thanks Michael!

>> Then you do something like this:
>>
>> mkdir build
>> cd build
>> meson ..
>> ninja-build
>
> Hm, did you mean "ninja" here?

Ah yes, on Red Hat/Fedora it's renamed to `ninja-build`, but on Debian
it's just `ninja`.

> As one of your downstreams (Debian), I wouldn't have a problem of not
> shipping pre-generated files (for gtk-doc and valac).
>
> So a git archive tarball would be fine with me.

That's good to know! We did wonder if regenerating the .c files each
build would have implications for reproducible-builds, but I guess if
there are issues there it's better to fix them now than ignore them.

Sam
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list

Re: [Tracker] Meson build instructions for Tracker

[Michael Biebl]

Hi Sam


2017-03-30 23:54 GMT+02:00 Sam Thursfield :
> Hello
> I've been working on build instructions for Tracker using Meson.
> They are now pretty much ready for use!

Nice work!

> You can find them in the branch wip/sam/meson:
> https://git.gnome.org/browse/tracker/commit/?h=wip/sam/meson
>
> To use, you need to install Meson, which can either be done from Pip
> (pip3 install --user meson), from a distro package (but beware that it
> might be old), or you can run it right from Git
> (git://github.com/mesonbuild/meson).
>
> Then you do something like this:
>
> mkdir build
> cd build
> meson ..
> ninja-build

Hm, did you mean "ninja" here?

> You can run the test suite:
>
> ninja-build test
>
> And `ninja-build install` to install of course.
>
> Meson accepts standard arguments like --prefix, but to change the
> Tracker-specific options you need to run mesonconf. Run it in the build
> directory with no arguments to see all the available options, and pass
> `-D option=value` to set something.
>
> I've compared an install of this with an equivalent Autotools build. So
> I'm confident there aren't major regressions, but it does require more
> testing. Help with that is appreciated!
>
> Here are a few remaining issues:
>
>   * There's no `make dist` equivalent. We can use `git archive` to
> produce tarballs, but these won't have the .c files generated by
> valac. That said, shipping the generated .c files does make the
> Vala preprocessor useless so it would be good if we can stop,
> but that could break things for downstreams unexpectedly.

As one of your downstreams (Debian), I wouldn't have a problem of not
shipping pre-generated files (for gtk-doc and valac).

So a git archive tarball would be fine with me.



-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
___
tracker-list mailing list
tracker-list@gnome.org
https://mail.gnome.org/mailman/listinfo/tracker-list