Re: ABI changes within stable branch

[Patrick M. Hausen]

Morning,

> Am 20.09.2017 um 19:27 schrieb Mark Linimon :
> 
> On Tue, Sep 19, 2017 at 10:15:32AM +0200, Kurt Jaeger wrote:
>> A pointer to the official policy would be nice 8-}
> 
> 3rd paragraph of:
> 
>  http://www.freebsd.org/portmgr/policies_eol.html

One comment: it's easy to overlook the implications of the word "supported" in

"Package builds will use the oldest supported minor release within each 
major branch ..."

Thanks for the link.

Patrick
--
punkt.de GmbH   Internet - Dienstleistungen - Beratung
Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100
76133 Karlsruhe i...@punkt.de   http://punkt.de
AG Mannheim 108285  Gf: Juergen Egeling


signature.asc
Description: Message signed with OpenPGP

Re: ABI changes within stable branch

[Mark Linimon]

On Tue, Sep 19, 2017 at 07:33:20PM -0600, Warner Losh wrote:
> FreeBSD has always had a policy of backwards compatibility. By that
> definition we are stable. What we don't promise is full forwards
> compatibility, which is what you are asking for.

In particular, "we add things to the ABI" sometimes changes the way
packages build (e.g. if a config script auto-detects the change).
Because of this, you have to be very particular that you are running
either 11.0-built things, or 11.1-built things, but not a combination.

If you're running 11.X-stable across the thing-added-to-ABI boundary,
then you can get into the same bad state.

Ports rarely get into that state across that boundary, and only a few
of them, but it is very annoying when it does happen (e.g. rsync).

The only way to make sure that doesn't happen is to cancel the developers'
ability to modify the ABI in any way.  Unfortunately, I don't see that
happening.

mcl
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ABI changes within stable branch

[Mark Linimon]

On Tue, Sep 19, 2017 at 10:15:32AM +0200, Kurt Jaeger wrote:
> A pointer to the official policy would be nice 8-}

3rd paragraph of:

  http://www.freebsd.org/portmgr/policies_eol.html

mcl
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ABI changes within stable branch

[Patrick M. Hausen]

Hi!

> Am 20.09.2017 um 04:09 schrieb Aristedes Maniatis :
> At the very least I need to remember to keep poudriere on the x.0 release 
> even after it is EOL,
> until every one of my servers has been upgraded

Not necessarily. You can run build jails with lower OS versions on an up-to-date
poudriere system.

In your specific case just build 11.0 and 11.1 packages (until 11.0 breaks ;-) 
and
use the appropriate package repos on your various servers.

You *cannot* build 11.1 packages on an 11.0 poudriere, at least not reliably.

HTH,
Patrick
--
punkt.de GmbH   Internet - Dienstleistungen - Beratung
Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100
76133 Karlsruhe i...@punkt.de   http://punkt.de
AG Mannheim 108285  Gf: Juergen Egeling


signature.asc
Description: Message signed with OpenPGP

Re: ABI changes within stable branch

[Aristedes Maniatis]

On 20/9/17 11:33AM, Warner Losh wrote:
> FreeBSD has always had a policy of backwards compatibility. By that 
> definition we are stable. What we don't promise is full forwards 
> compatibility, which is what you are asking for. 

Correct. Within the stable branch I'd always assumed forward compatibility was 
the case and haven't been bitten by this since my days of FreeBSD 3.0.

But even if this is no longer the case (or was never a goal), I'm still 
confused by versioning packages like this: http://pkg.FreeBSD.org/${ABI}/ which 
is clearly not correct. There is just no way for me to discover which package 
is compatible with which OS version.

Anyhow, thanks for listening. This is putting a dent in my adoption of the 
accelerated EOL of minor releases. At the very least I need to remember to keep 
poudriere on the x.0 release even after it is EOL, until every one of my 
servers has been upgraded (which is rarely before the new accelerated EOL for 
machines that don't face the internet).

Ari


-- 
-->
Aristedes Maniatis
CEO, ish
https://www.ish.com.au
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ABI changes within stable branch

[Warner Losh]

On Sep 19, 2017 6:05 PM, "Aristedes Maniatis"  wrote:

Matthew Seaman wrote:
>
> Ports are still being built according to the same policy -- on the
> earliest still-supported release of each major branch.
>
> It's just that now, for 11.x and subsequent, 11.0 goes out of support a
> month or so after 11.1-RELEASE comes out.  You're meant to have upgraded
> by now.  The 11.0 -> 11.1 upgrade is intended to be a pretty routine
> thing that you can do about as freely as you can apply a security patch
> or other update within the 11.0 series.

I'm afraid this hasn't made things clearer for me at all.

1. What does the "stable" branch mean if the ABI is no longer stable


FreeBSD has always had a policy of backwards compatibility. By that
definition we are stable. What we don't promise is full forwards
compatibility, which is what you are asking for.


2. This policy of changing the ABI means that upgrading from 11.0 to 11.1
is now less routine than it used to be in the old days. Each minor update
is more like the effort involved in upgrading 10 -> 11. So I'll be doing it
less often, not more often.


How so? All the old binaries work. It's running new binaries on old systems
that's a problem.

3. Packages are located in a namespace like this: https://pkg.freebsd.org/
freebsd:11:x86:64  But now I don't know which release this is actually
pointing to or which packages will work.

4. /etc/pkg/repos/FreeBSD.conf points to url: "pkg+http://pkg.FreeBSD.org/${
ABI}/quarterly" However this is now wrong. If I am delayed in upgrading my
system, downloading packages from there will sometimes break things. And I
will not know until runtime.

5. The package MANIFEST contains information about system compatibility.
That is just the major version, but we need the minor release version now
too.


Here are some possible solutions from where I'm sitting on the edges:

a. Go back to 'stable' meaning the ABI doesn't change. Not just the kernel,
but the whole OS.


The definition hasn't changed in a decade.

b. Since there is no different in breakage and effort when going from 11.0
-> 11.1 or when going from 11.0 -> 12.0, just get rid of the point releases
entirely. Then the existing packaging system still works.

c. Add point releases to the package manifest. We've have something like
https://pkg.freebsd.org/freebsd:11.0:x86:64

d. Wait for some new base packaging magic to solve things.


Have I summarised this effectively?


Apart from the whole forwards backwards thing, which is sadly critical...

Warner

Ari


--
-->
Aristedes Maniatis
CEO, ish
https://www.ish.com.au
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ABI changes within stable branch

[Aristedes Maniatis]

Matthew Seaman wrote:
> 
> Ports are still being built according to the same policy -- on the
> earliest still-supported release of each major branch.
> 
> It's just that now, for 11.x and subsequent, 11.0 goes out of support a
> month or so after 11.1-RELEASE comes out.  You're meant to have upgraded
> by now.  The 11.0 -> 11.1 upgrade is intended to be a pretty routine
> thing that you can do about as freely as you can apply a security patch
> or other update within the 11.0 series.

I'm afraid this hasn't made things clearer for me at all.

1. What does the "stable" branch mean if the ABI is no longer stable

2. This policy of changing the ABI means that upgrading from 11.0 to 11.1 is 
now less routine than it used to be in the old days. Each minor update is more 
like the effort involved in upgrading 10 -> 11. So I'll be doing it less often, 
not more often.

3. Packages are located in a namespace like this: 
https://pkg.freebsd.org/freebsd:11:x86:64  But now I don't know which release 
this is actually pointing to or which packages will work.

4. /etc/pkg/repos/FreeBSD.conf points to url: 
"pkg+http://pkg.FreeBSD.org/${ABI}/quarterly; However this is now wrong. If I 
am delayed in upgrading my system, downloading packages from there will 
sometimes break things. And I will not know until runtime.

5. The package MANIFEST contains information about system compatibility. That 
is just the major version, but we need the minor release version now too.


Here are some possible solutions from where I'm sitting on the edges:

a. Go back to 'stable' meaning the ABI doesn't change. Not just the kernel, but 
the whole OS.

b. Since there is no different in breakage and effort when going from 11.0 -> 
11.1 or when going from 11.0 -> 12.0, just get rid of the point releases 
entirely. Then the existing packaging system still works.

c. Add point releases to the package manifest. We've have something like  
https://pkg.freebsd.org/freebsd:11.0:x86:64

d. Wait for some new base packaging magic to solve things.


Have I summarised this effectively?

Ari


-- 
-->
Aristedes Maniatis
CEO, ish
https://www.ish.com.au
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ABI changes within stable branch

[Patrick M. Hausen]

Hi all,

> Am 19.09.2017 um 10:32 schrieb Aristedes Maniatis :
> Then we have a problem since 
> https://pkg.freebsd.org/freebsd:11:x86:64/latest/All/ has been built on 11.1, 
> not on 11.0 (I just tested it with csync2 which I know fails). Packages there 
> may fail to run on 11.0, but there is no clear indication, just random 
> failures at runtime.
> 
> Maybe we'd need specific 11.0, 11.1, 11.2 releases instead of quarterly 
> releases?

This is precisely what we do on our own poudriere - build the quarterly ports 
branches
on various FreeBSD release versions.

Patrick


signature.asc
Description: Message signed with OpenPGP

Re: ABI changes within stable branch

[Matthew Seaman]

On 19/09/2017 09:32, Aristedes Maniatis wrote:
> On 19/9/17 6:15PM, Kurt Jaeger wrote:
>> Hi!
>>
>>> Now that we are on a faster upgrade policy for minor branches, it is 
>>> expected that we'll upgrade from 11.0 to 11.1 to 11.2 much faster than in 
>>> the old days. I can cope with that, but it appears that functional changes 
>>> are also being made within the stable branch as seen here:
>>>
>>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221672
>>>
>>> A new fdatasync()  method is available in 11.1 but not in 11.0 which means 
>>> that I now need to maintain separate ports trees for each minor update. 
>>> I've never done this before, assuming (correctly for me until now) that all 
>>> ports build on the latest minor release within the stable branch would work 
>>> on older releases until I was ready to upgrade them.
>>
>> I think it was the other way around: All ports build on the .0 of
>> a RELEASE work on all later .x of that RELEASE. Which makes it a bit
>> difficult, if a .0 is no longer supported/patched by the secteam.
>>
>> A pointer to the official policy would be nice 8-}
> 
> Then we have a problem since 
> https://pkg.freebsd.org/freebsd:11:x86:64/latest/All/ has been built on 11.1, 
> not on 11.0 (I just tested it with csync2 which I know fails). Packages there 
> may fail to run on 11.0, but there is no clear indication, just random 
> failures at runtime.
> 
> Maybe we'd need specific 11.0, 11.1, 11.2 releases instead of quarterly 
> releases?

Ports are still being built according to the same policy -- on the
earliest still-supported release of each major branch.

It's just that now, for 11.x and subsequent, 11.0 goes out of support a
month or so after 11.1-RELEASE comes out.  You're meant to have upgraded
by now.  The 11.0 -> 11.1 upgrade is intended to be a pretty routine
thing that you can do about as freely as you can apply a security patch
or other update within the 11.0 series.

Yes, there should be some sort of warning about your system being older
than what the package was built for.  Ideally it would be intelligent
enough to understand about things like the new fdatasync meaning libc
was incompatible.

Once we do finally get base system packages this problem should largely
disappear, as the normal pkg(8) dependency handling should pull in an
updated libc as a dependnecy of anything expecting the new fdatasync().

Cheers,

Matthew



___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ABI changes within stable branch

[Aristedes Maniatis]

On 19/9/17 6:15PM, Kurt Jaeger wrote:
> Hi!
> 
>> Now that we are on a faster upgrade policy for minor branches, it is 
>> expected that we'll upgrade from 11.0 to 11.1 to 11.2 much faster than in 
>> the old days. I can cope with that, but it appears that functional changes 
>> are also being made within the stable branch as seen here:
>>
>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221672
>>
>> A new fdatasync()  method is available in 11.1 but not in 11.0 which means 
>> that I now need to maintain separate ports trees for each minor update. I've 
>> never done this before, assuming (correctly for me until now) that all ports 
>> build on the latest minor release within the stable branch would work on 
>> older releases until I was ready to upgrade them.
> 
> I think it was the other way around: All ports build on the .0 of
> a RELEASE work on all later .x of that RELEASE. Which makes it a bit
> difficult, if a .0 is no longer supported/patched by the secteam.
> 
> A pointer to the official policy would be nice 8-}

Then we have a problem since 
https://pkg.freebsd.org/freebsd:11:x86:64/latest/All/ has been built on 11.1, 
not on 11.0 (I just tested it with csync2 which I know fails). Packages there 
may fail to run on 11.0, but there is no clear indication, just random failures 
at runtime.

Maybe we'd need specific 11.0, 11.1, 11.2 releases instead of quarterly 
releases?

Ari



-- 
-->
Aristedes Maniatis
CEO, ish
https://www.ish.com.au
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ABI changes within stable branch

[Kurt Jaeger]

Hi!

> Now that we are on a faster upgrade policy for minor branches, it is expected 
> that we'll upgrade from 11.0 to 11.1 to 11.2 much faster than in the old 
> days. I can cope with that, but it appears that functional changes are also 
> being made within the stable branch as seen here:
> 
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221672
> 
> A new fdatasync()  method is available in 11.1 but not in 11.0 which means 
> that I now need to maintain separate ports trees for each minor update. I've 
> never done this before, assuming (correctly for me until now) that all ports 
> build on the latest minor release within the stable branch would work on 
> older releases until I was ready to upgrade them.

I think it was the other way around: All ports build on the .0 of
a RELEASE work on all later .x of that RELEASE. Which makes it a bit
difficult, if a .0 is no longer supported/patched by the secteam.

A pointer to the official policy would be nice 8-}

-- 
p...@opsec.eu+49 171 3101372 3 years to go !
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

ABI changes within stable branch

[Aristedes Maniatis]

Now that we are on a faster upgrade policy for minor branches, it is expected 
that we'll upgrade from 11.0 to 11.1 to 11.2 much faster than in the old days. 
I can cope with that, but it appears that functional changes are also being 
made within the stable branch as seen here:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221672

A new fdatasync()  method is available in 11.1 but not in 11.0 which means that 
I now need to maintain separate ports trees for each minor update. I've never 
done this before, assuming (correctly for me until now) that all ports build on 
the latest minor release within the stable branch would work on older releases 
until I was ready to upgrade them.

Is this instance a mistake or am I misunderstanding the new policy?

If I need to treat each release within the stable branch as a whole new 
platform for ports, that means a bunch of extra testing and maintenance work 
for me.

Cheers
Ari


-- 
-->
Aristedes Maniatis
CEO, ish
https://www.ish.com.au
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"