Bug#871619: Packaging

[Aron Xu]

On Wed, Oct 25, 2017 at 8:32 PM, Antonio Russo
 wrote:
> The two github repositories
>
> https://github.com/aerusso/pkg-zfsonlinux-zfs
> https://github.com/aerusso/pkg-zfsonlinux-spl
>
> have upstream branches representing an import of the zfs/spl
> sources, and a "proposed-updates" branch which includes now
> a changelog entry. It builds, installs, and runs on a few
> machines, and (as best as I can tell) matches the style
> of the alioth repository. Can the changes in these repositories
> just be pulled into alioth? Is there anything I can do to
> help?
>

I've pulled the changes, but still some minor issues need to be fixed,
I'm working on that right now.

Regards,
Aron

Bug#871619: Packaging

[Antonio Russo]

The two github repositories

https://github.com/aerusso/pkg-zfsonlinux-zfs
https://github.com/aerusso/pkg-zfsonlinux-spl

have upstream branches representing an import of the zfs/spl
sources, and a "proposed-updates" branch which includes now
a changelog entry. It builds, installs, and runs on a few
machines, and (as best as I can tell) matches the style
of the alioth repository. Can the changes in these repositories
just be pulled into alioth? Is there anything I can do to
help?

Antonio

Bug#871619: Packaging

[Fabian Grünbichler]

On Thu, Oct 12, 2017 at 06:51:05AM -0400, Antonio Russo wrote:
> On 10/12/17 06:12, Fabian Grünbichler wrote:
> > 
> > I would also suggest cherry-picking/backporting the following two
> > upstream PRs on top of 0.7.2:
> > 
> > #6616: send/recv compatibility with 0.6.5.x [1]
> > #6695: receive_freeobjects() skips freeing some objects [2]
> > 
> > the first (DISCLAIMER: written by me :P) fixes an incompatibility
> > between zfs send on 0.7 and zfs recv on 0.6.5 which would otherwise lead
> > to lots of never-terminating, 100% cpu usage zfs recv threads on
> > not-yet-upgraded replication/backup targets.
> > 
> > the second fixes a potential inconsistency when receiving incremental
> > streams, which would cause subsequent receives to fail.
> > 
> > both are slated for inclusion in 0.7.3 ([3]).
> > 
> > 1: https://github.com/zfsonlinux/zfs/pull/6616
> > 2: https://github.com/zfsonlinux/zfs/pull/6695
> > 3: https://github.com/zfsonlinux/zfs/projects/12
> >
> 
> Just as a follow up, all of these proposed stable additions are
> currently under review for inclusion into the stable branch by
> upstream [1]. I've included them tentatively in the
> repositories [2,3].
> 
> 1: https://github.com/zfsonlinux/zfs/pull/6752
> 2: https://github.com/aerusso/pkg-zfsonlinux-zfs
> 3: https://github.com/aerusso/pkg-zfsonlinux-spl
> 

and there has now been a 0.7.3 release incorporating those and other
fixes:

https://github.com/zfsonlinux/zfs/releases/tag/zfs-0.7.3

Bug#871619: Packaging

[Antonio Russo]

On 10/12/17 06:12, Fabian Grünbichler wrote:
> 
> I would also suggest cherry-picking/backporting the following two
> upstream PRs on top of 0.7.2:
> 
> #6616: send/recv compatibility with 0.6.5.x [1]
> #6695: receive_freeobjects() skips freeing some objects [2]
> 
> the first (DISCLAIMER: written by me :P) fixes an incompatibility
> between zfs send on 0.7 and zfs recv on 0.6.5 which would otherwise lead
> to lots of never-terminating, 100% cpu usage zfs recv threads on
> not-yet-upgraded replication/backup targets.
> 
> the second fixes a potential inconsistency when receiving incremental
> streams, which would cause subsequent receives to fail.
> 
> both are slated for inclusion in 0.7.3 ([3]).
> 
> 1: https://github.com/zfsonlinux/zfs/pull/6616
> 2: https://github.com/zfsonlinux/zfs/pull/6695
> 3: https://github.com/zfsonlinux/zfs/projects/12
>

Just as a follow up, all of these proposed stable additions are
currently under review for inclusion into the stable branch by
upstream [1]. I've included them tentatively in the
repositories [2,3].

1: https://github.com/zfsonlinux/zfs/pull/6752
2: https://github.com/aerusso/pkg-zfsonlinux-zfs
3: https://github.com/aerusso/pkg-zfsonlinux-spl

Bug#871619: Packaging

[Fabian Grünbichler]

On Sat, Oct 07, 2017 at 12:19:35AM +0800, Aron Xu wrote:
> Hi,
> 
> On Fri, Oct 6, 2017 at 4:59 PM, Petter Reinholdtsen  wrote:
> >
> > [Antonio Russo]
> >> How can I help out the packaging team on this? Who should I email?
> >
> > The best way to help is to continue contributing and participating on
> > the team mailing list and IRC channel.
> >
> > Aron Xu has been most active with maintenance, and I hope he can comment
> > on how you best can contribute.  I am not competent do to so. :)
> >
> > Any project admin on
> > https://alioth.debian.org/projects/pkg-zfsonlinux/ > can grant
> > access, ie Aron, Liang and Carlos.
> >
> > CC to Aron, and email to the bts also go to the project mailing list.
> >
> 
> I've started to update the package to 0.7.2 release and it appears
> there are some build system changes that installs quite some files
> into non-FHS compliant paths, and possibly a new package (could be
> "zfs-tests") is needed. If you are eager to help, please start from
> these changes and my latest work is available on alioth git.
> 
> Regards,
> Aron

I would also suggest cherry-picking/backporting the following two
upstream PRs on top of 0.7.2:

#6616: send/recv compatibility with 0.6.5.x [1]
#6695: receive_freeobjects() skips freeing some objects [2]

the first (DISCLAIMER: written by me :P) fixes an incompatibility
between zfs send on 0.7 and zfs recv on 0.6.5 which would otherwise lead
to lots of never-terminating, 100% cpu usage zfs recv threads on
not-yet-upgraded replication/backup targets.

the second fixes a potential inconsistency when receiving incremental
streams, which would cause subsequent receives to fail.

both are slated for inclusion in 0.7.3 ([3]).

1: https://github.com/zfsonlinux/zfs/pull/6616
2: https://github.com/zfsonlinux/zfs/pull/6695
3: https://github.com/zfsonlinux/zfs/projects/12

Bug#871619: Packaging

[Fabian Grünbichler]

CC-ing bug/list again, IMHO this should be discussed publicly.

On Wed, Oct 11, 2017 at 06:41:54PM -0400, Antonio Russo wrote:
> On 10/10/17 17:43, Antonio Russo wrote:
> 
> > I'll check the FHS concerns, time-permitting.
> 
> With Fabian's changes, the only things that maybe looked questionable
> were the files installed to
> 
> /usr/lib/${ARCH}/zfs
> 
> which seem to be architecture independent shellscript. Is this what
> you are talking about? Should these files instead be located in
> 
> /usr/lib/zfsutils-linux
> 
> Antonio
> 

either that (see attached patch, I reused /usr/lib/zfs-linux which
already contains the scrub script referenced in the cron job), or move
them to /usr/share/zfs like the zfs-test scripts? IMHO both places are
fine, /usr/lib/ARCH/zfs is definitely wrong ;)
From: =?utf-8?q?Fabian_Gr=C3=BCnbichler?= 
Date: Thu, 12 Oct 2017 08:57:48 +0200
Subject: fix install path of zpool.d scripts
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit

Signed-off-by: Fabian Grünbichler 
---
 cmd/zpool/Makefile.am | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/cmd/zpool/Makefile.am b/cmd/zpool/Makefile.am
index 6eff1d1..872a223 100644
--- a/cmd/zpool/Makefile.am
+++ b/cmd/zpool/Makefile.am
@@ -22,11 +22,11 @@ zpool_LDADD = \
 	-lm $(LIBBLKID)
 
 zpoolconfdir = $(sysconfdir)/zfs/zpool.d
-zpoolexecdir = $(libexecdir)/zfs/zpool.d
+zpoollibdir = /usr/lib/zfs-linux/zpool.d
 
 EXTRA_DIST = zpool.d/README
 
-dist_zpoolexec_SCRIPTS = \
+dist_zpoollib_SCRIPTS = \
 	zpool.d/enc \
 	zpool.d/encdev \
 	zpool.d/fault_led \
@@ -107,5 +107,5 @@ install-data-hook:
 	for f in $(zpoolconfdefaults); do \
 	  test -f "$(DESTDIR)$(zpoolconfdir)/$${f}" -o \
 	   -L "$(DESTDIR)$(zpoolconfdir)/$${f}" || \
-	ln -s "$(zpoolexecdir)/$${f}" "$(DESTDIR)$(zpoolconfdir)"; \
+	ln -s "$(zpoollibdir)/$${f}" "$(DESTDIR)$(zpoolconfdir)"; \
 	done

Bug#871619: [Pkg-zfsonlinux-devel] Bug#871619: Bug#871619: Packaging

[Fabian Grünbichler]

On Fri, 6 Oct 2017 08:49:51 +0200 Dmitry Galenko  wrote:
> I successful build deb packages from upstream repo, for kernel 4.10
> (custom) without any changes with this instruction:
> 
> [snip]

this is not a good idea for production use IMHO - those converted packages 
using alien lack all of the integration into Debian proper (including missing 
all the systemd service stuff, among other things)..

Bug#871619: Packaging

[Fabian Grünbichler]

> Aron Xu  hat am 6. Oktober 2017 um 18:19 geschrieben:
> 
> 
> Hi,
> 
> On Fri, Oct 6, 2017 at 4:59 PM, Petter Reinholdtsen  wrote:
> >
> > [Antonio Russo]
> >> How can I help out the packaging team on this? Who should I email?
> >
> > The best way to help is to continue contributing and participating on
> > the team mailing list and IRC channel.
> >
> > Aron Xu has been most active with maintenance, and I hope he can comment
> > on how you best can contribute.  I am not competent do to so. :)
> >
> > Any project admin on
> > https://alioth.debian.org/projects/pkg-zfsonlinux/ > can grant
> > access, ie Aron, Liang and Carlos.
> >
> > CC to Aron, and email to the bts also go to the project mailing list.
> >
> 
> I've started to update the package to 0.7.2 release and it appears
> there are some build system changes that installs quite some files
> into non-FHS compliant paths, and possibly a new package (could be
> "zfs-tests") is needed. If you are eager to help, please start from
> these changes and my latest work is available on alioth git.
> 
> Regards,
> Aron
> 

did you see my WIP branch on github, linked earlier here (and also separately 
on pkg-zfsonlinux-de...@lists.alioth.debian.org)[1,2]? I already did most of 
the work (including splitting out zfs-test) except for updating 
debian/copyright.. note that I merged the actual upstream tags and not the 
upstream release tar balls, so you probably only want to take a closer look at 
commits touching debian/ ;) I have been running this branch (recently with the 
addition of upstream PR#6616[3]) for quite some time on my personal machines 
(Sid and Stretch), and have been testing a variant of it with a 4.13 kernel at 
work as well - all without any problems so far (except for the send/receive 
incompatibility with 0.6.5, for which I made the PR ;))

the only other point up for discussion besides splitting out zfs-test was 
whether the zfs and zpool binaries should move to /bin instead of /sbin, as 
they are now usable by unprivileged users for certain read-only operations..

I would be glad to assist in any future work here, both for the 0.7.2 release 
as well as further packaging (I am responsible for most of the downstream ZFS 
packaging in Proxmox VE, which is based on Debian - so I am familiar with both 
ZoL and Debian packaging ;)).

if you have any questions about my branch or want to discuss stuff, just drop 
me an e-mail (here, on-list or directly) or catch me on IRC (f_g on OFTC and 
freenode)

1: https://github.com/Fabian-Gruenbichler/zfs/commits/debian/wip-0.7
2: 
https://lists.alioth.debian.org/pipermail/pkg-zfsonlinux-devel/2017-August/001196.html
3: https://github.com/zfsonlinux/zfs/pull/6616

Bug#871619: Packaging

[Aron Xu]

Hi,

On Fri, Oct 6, 2017 at 4:59 PM, Petter Reinholdtsen  wrote:
>
> [Antonio Russo]
>> How can I help out the packaging team on this? Who should I email?
>
> The best way to help is to continue contributing and participating on
> the team mailing list and IRC channel.
>
> Aron Xu has been most active with maintenance, and I hope he can comment
> on how you best can contribute.  I am not competent do to so. :)
>
> Any project admin on
> https://alioth.debian.org/projects/pkg-zfsonlinux/ > can grant
> access, ie Aron, Liang and Carlos.
>
> CC to Aron, and email to the bts also go to the project mailing list.
>

I've started to update the package to 0.7.2 release and it appears
there are some build system changes that installs quite some files
into non-FHS compliant paths, and possibly a new package (could be
"zfs-tests") is needed. If you are eager to help, please start from
these changes and my latest work is available on alioth git.

Regards,
Aron

Bug#871619: Packaging

[Petter Reinholdtsen]

[Antonio Russo]
> How can I help out the packaging team on this? Who should I email?

The best way to help is to continue contributing and participating on
the team mailing list and IRC channel.

Aron Xu has been most active with maintenance, and I hope he can comment
on how you best can contribute.  I am not competent do to so. :)

Any project admin on
https://alioth.debian.org/projects/pkg-zfsonlinux/ > can grant
access, ie Aron, Liang and Carlos.

CC to Aron, and email to the bts also go to the project mailing list.

-- 
Happy hacking
Petter Reinholdtsen

Bug#871619: [Pkg-zfsonlinux-devel] Bug#871619: Bug#871619: Packaging

[Dmitry Galenko]

I successful build deb packages from upstream repo, for kernel 4.10
(custom) without any changes with this instruction:

apt-get install -y build-essential autoconf libtool gawk alien
fakeroot linux-headers-$(uname -r) zlib1g-dev uuid-dev libattr1-dev
libblkid-dev libselinux-dev libudev-dev libdevmapper-dev parted lsscsi
ksh

export ZFS_BUILD_PATH=/tmp/zfs
rm -rf $ZFS_BUILD_PATH
mkdir $ZFS_BUILD_PATH

cd $ZFS_BUILD_PATH && rm -rf spl && git clone
https://github.com/zfsonlinux/spl && cd spl && git checkout
remotes/origin/spl-0.7-release && sh autogen.sh && ./configure && make
-s -j2 && make deb

cd $ZFS_BUILD_PATH && rm -rf zfs && git clone
https://github.com/zfsonlinux/zfs && cd zfs && git checkout
remotes/origin/zfs-0.7-release && sh autogen.sh && ./configure && make
-s -j2 && make deb

# Delete the many ZFS kernel modules non-optionally installed by
Ubuntu and instead install ours:
cd $ZFS_BUILD_PATH && dpkg -i spl/*.deb zfs/*.deb
rm -rf /lib/modules/*/kernel/zfs
depmod -a


2017-10-06 7:41 GMT+02:00 Fabian Grünbichler :

> On Thu, Oct 05, 2017 at 08:10:16PM -0400, Antonio Russo wrote:
> > Hi!
> >
> > I forked the Alioth repository, and got the debian packaging able
> > to build 0.7.2 (the spl ./debian required no changes).
> >
> > The fork is available on github:
> >
> > https://github.com/aerusso/pkg-zfsonlinux-zfs/tree/
> proposed-changes/debian
> >
> > I dropped a patch that seemed to diverge from upstream's support for
> powerpc,
> > mostly because I have no ability to test that platform (and the code
> around
> > that seemed to have quite a bit of churn). Also several patches were
> applied
> > upstream, so they also got removed.
> >
> > How can I help out the packaging team on this? Who should I email?
> >
>
> note: I am not the maintainer or even a DM in general..
>
> you could have probably saved yourself a lot of effort if you had read
> the full history of this bug, and noticed I already have a working 0.7.2
> branch for SPL and ZFS up on my github ;)
>
> I am planning to do an NMU for 0.6.5.11 with compat patches for 4.13 if
> I find a sponsor (this weekend), and then a bigger NMU for 0.7.2 (in
> which I will introduce a new binary package zfs-test in accordance with
> upstream's packaging, which means a trip through NEW) soon after.
>
> the zfsonlinux maintainers for Debian are pretty much inactive
> currently, so if you want to help out you are of course more than
> welcome.
>
> ___
> Pkg-zfsonlinux-devel mailing list
> pkg-zfsonlinux-de...@lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/
> pkg-zfsonlinux-devel
>



-- 
С уважением,
Дмитрий.

Bug#871619: [Pkg-zfsonlinux-devel] Bug#871619: Packaging

[Fabian Grünbichler]

On Thu, Oct 05, 2017 at 08:10:16PM -0400, Antonio Russo wrote:
> Hi!
> 
> I forked the Alioth repository, and got the debian packaging able
> to build 0.7.2 (the spl ./debian required no changes).
> 
> The fork is available on github:
> 
> https://github.com/aerusso/pkg-zfsonlinux-zfs/tree/proposed-changes/debian
> 
> I dropped a patch that seemed to diverge from upstream's support for powerpc,
> mostly because I have no ability to test that platform (and the code around
> that seemed to have quite a bit of churn). Also several patches were applied
> upstream, so they also got removed.
> 
> How can I help out the packaging team on this? Who should I email?
> 

note: I am not the maintainer or even a DM in general..

you could have probably saved yourself a lot of effort if you had read
the full history of this bug, and noticed I already have a working 0.7.2
branch for SPL and ZFS up on my github ;)

I am planning to do an NMU for 0.6.5.11 with compat patches for 4.13 if
I find a sponsor (this weekend), and then a bigger NMU for 0.7.2 (in
which I will introduce a new binary package zfs-test in accordance with
upstream's packaging, which means a trip through NEW) soon after.

the zfsonlinux maintainers for Debian are pretty much inactive
currently, so if you want to help out you are of course more than
welcome.

Bug#871619: Packaging

[Antonio Russo]

Hi!

I forked the Alioth repository, and got the debian packaging able
to build 0.7.2 (the spl ./debian required no changes).

The fork is available on github:

https://github.com/aerusso/pkg-zfsonlinux-zfs/tree/proposed-changes/debian

I dropped a patch that seemed to diverge from upstream's support for powerpc,
mostly because I have no ability to test that platform (and the code around
that seemed to have quite a bit of churn). Also several patches were applied
upstream, so they also got removed.

How can I help out the packaging team on this? Who should I email?

Thanks,
Antonio Russo