[dpdk-dev] On DPDK ABI policy

[Marc Sune]

2016-04-07 13:51 GMT+02:00 Panu Matilainen :

> [ change of subject since this is about ABI policy, not namespacing ]
>
> On 04/07/2016 01:16 PM, Marc Sune wrote:
>
>>
>>
>> 2016-04-07 11:33 GMT+02:00 Panu Matilainen > >:
>>
>> On 04/07/2016 12:18 PM, Thomas Monjalon wrote:
>>
>> Thank you everyone for the feedbacks.
>>
>> 2016-04-05 15:56, Thomas Monjalon:
>>
>> The goal of this email is to get some feedback on how
>> important it is
>> to fix the DPDK namespace.
>>
>>
>> Everybody agree every symbols must be prefixed. Checking and
>> fixing the
>> namespace consistency will be in the roadmap.
>>
>> It seems most of you agree renaming would be a nice improvement
>> but not
>> so important.
>> The main drawback is the induced backporting pain, even if we have
>> some scripts to convert the patches to the old namespace.
>> Note: the backports can be in DPDK itself or in the applications.
>>
>> If there is enough agreement that we should do something, I
>> suggest to
>> introduce the "dpdk_" prefix slowly and live with both
>> "rte_" and "dpdk_"
>> during some time.
>> We could start using the new prefix for the new APIs
>> (example: crypto)
>> or when there is a significant API break (example: mempool).
>>
>>
>> The slow change has been clearly rejected in favor of a complete
>> change
>> in one patch.
>> The timing was also discussed as it could impact the pending
>> patches.
>> So it would be done at the end or the beginning of a release.
>> Marc suggests to do it for 16.04 as the numbering scheme has
>> changed.
>>
>>
>> Just noting that it cannot be done in 16.04 because the ABI policy
>> requires a deprecation cycle of at least one major release for every
>> breakage. And we're discussing a total 100% breakage of everything
>> here, even if its just a simple rename.
>>
>>
>> I keep not understanding the ABI policy, and particularly why ABI
>> changes have to be announced once cycle before _if_ there is already at
>> least one ABI change proposed. DPDK applications will have to recompile
>> anyway.
>>
>
> The point is to allow API/ABI consumers to assess in advance what sort of
> pains can they expect when moving their applications from one version to
> another. Otherwise all sorts of massive changes could ride the wave of
> whatever "change 16bit struct member to 32bit" trivialities that are
> nevertheless ABI breaks.
>
> There have already been quite a few exceptions to the rule when the ABI is
> already being broken, so its not entirely rigid. Another point that migth
> warrant some tweaking to the policy is the "core" libraries depending on
> each other so an ABI break in any one of them forces recompile of
> everything anyway.
>

In addition to what Matthew said:

I don't understand which sort of pains an announcement saying "we are going
to change this structure and this other, for those high level reasons, but
we don't know exactly how yet, because it is not fully implemented" can be
of any help to a DPDK user. At least it does not to me. This information
has to be in the release notes and users can read that before deciding to
upgrade to a new release.

On the other side, bug fixes still go to NEXT_VERSION. So in 1 ouf ot 2
cases (so far), next release is breaking ABI, so you will have to anyway
recompile your application.

And about ABI breakages; DPDK is not a standard library/library set. For
performance reasons it has a lot of inline code and other optimizations, so
even for small bug fixings can brake ABI, or to be precise, some bug fixes
in inline functions may be silently ignored. I don't know how many users
really use dynamic libraries for DPDK and if there is some warning
somewhere in the documentation for that.


> This aspect of the policy only slows down DPDK development and it
>>
>
> One could also think that slowing down development and forcing people to
> think ahead are not entirely unintentional or unwanted side-effects :)
>
> Look at the latest librte_vhost initiative to remove unnecessarily exposed
> structs to avoid having to deal with ABI breakages all the time: the policy
> is effectively encouraging people into better library design.
>
> pollutes the repository with commits announcing ABI changes that are
>> irrelevant after 2 cycles, as (code) diffs show that already (not
>> mentioning NEXT_ABI complexity and extra LOCs).
>>
>
> Fully agreed on NEXT_ABI, I never liked it at all.
>
> Maintaining LTS releases, and enforcing bug fixing in old LTS first,
>> upstreaming bugfixes is to me a much better approach to solve backwards
>> compatibility issues.
>>
>
> LTS releases could help the situation somewhat, but then again people tend
> to still 

[dpdk-dev] On DPDK ABI policy

[Marc Sune]

2016-04-07 23:52 GMT+02:00 Matthew Hall :

> On Thu, Apr 07, 2016 at 02:51:35PM +0300, Panu Matilainen wrote:
> > LTS releases could help the situation somewhat, but then again
> > people tend to still want those new fancy things backported (you
> > know, have the cake and eat it too) but that can't be done because
> > of ABI breakage, so they're forced to run the latest version anyway.
>
> RH and Debian / Ubuntu don't put features in LTS except extremely rarely.
> Generally only if there's severe functionality breakage or security issues
> and
> the rest is ignored, and for good reason, as this is much more reliable and
> simple and predictable.
>
> If people are so irrational they can't deal with that simple of a policy,
> NEXT_ABI, LTS, etc. is never going to help them.
>
> If people like to have backported stuff, yes of course we can make trees
> and
> branches for this, they are basically free in Git. But at that point
> community
> people in need of LTS forks of features need to step up to the plate to
> help
> out.
>

Completely agree.

Marc


>
> Matthew.
>

[dpdk-dev] On DPDK ABI policy

[Matthew Hall]

On Thu, Apr 07, 2016 at 02:51:35PM +0300, Panu Matilainen wrote:
> LTS releases could help the situation somewhat, but then again
> people tend to still want those new fancy things backported (you
> know, have the cake and eat it too) but that can't be done because
> of ABI breakage, so they're forced to run the latest version anyway.

RH and Debian / Ubuntu don't put features in LTS except extremely rarely. 
Generally only if there's severe functionality breakage or security issues and 
the rest is ignored, and for good reason, as this is much more reliable and 
simple and predictable.

If people are so irrational they can't deal with that simple of a policy, 
NEXT_ABI, LTS, etc. is never going to help them.

If people like to have backported stuff, yes of course we can make trees and 
branches for this, they are basically free in Git. But at that point community 
people in need of LTS forks of features need to step up to the plate to help 
out.

Matthew.

[dpdk-dev] On DPDK ABI policy

[Panu Matilainen]

[ change of subject since this is about ABI policy, not namespacing ]

On 04/07/2016 01:16 PM, Marc Sune wrote:
>
>
> 2016-04-07 11:33 GMT+02:00 Panu Matilainen  >:
>
> On 04/07/2016 12:18 PM, Thomas Monjalon wrote:
>
> Thank you everyone for the feedbacks.
>
> 2016-04-05 15:56, Thomas Monjalon:
>
> The goal of this email is to get some feedback on how
> important it is
> to fix the DPDK namespace.
>
>
> Everybody agree every symbols must be prefixed. Checking and
> fixing the
> namespace consistency will be in the roadmap.
>
> It seems most of you agree renaming would be a nice improvement
> but not
> so important.
> The main drawback is the induced backporting pain, even if we have
> some scripts to convert the patches to the old namespace.
> Note: the backports can be in DPDK itself or in the applications.
>
> If there is enough agreement that we should do something, I
> suggest to
> introduce the "dpdk_" prefix slowly and live with both
> "rte_" and "dpdk_"
> during some time.
> We could start using the new prefix for the new APIs
> (example: crypto)
> or when there is a significant API break (example: mempool).
>
>
> The slow change has been clearly rejected in favor of a complete
> change
> in one patch.
> The timing was also discussed as it could impact the pending
> patches.
> So it would be done at the end or the beginning of a release.
> Marc suggests to do it for 16.04 as the numbering scheme has
> changed.
>
>
> Just noting that it cannot be done in 16.04 because the ABI policy
> requires a deprecation cycle of at least one major release for every
> breakage. And we're discussing a total 100% breakage of everything
> here, even if its just a simple rename.
>
>
> I keep not understanding the ABI policy, and particularly why ABI
> changes have to be announced once cycle before _if_ there is already at
> least one ABI change proposed. DPDK applications will have to recompile
> anyway.

The point is to allow API/ABI consumers to assess in advance what sort 
of pains can they expect when moving their applications from one version 
to another. Otherwise all sorts of massive changes could ride the wave 
of whatever "change 16bit struct member to 32bit" trivialities that are 
nevertheless ABI breaks.

There have already been quite a few exceptions to the rule when the ABI 
is already being broken, so its not entirely rigid. Another point that 
migth warrant some tweaking to the policy is the "core" libraries 
depending on each other so an ABI break in any one of them forces 
recompile of everything anyway.

> This aspect of the policy only slows down DPDK development and it

One could also think that slowing down development and forcing people to 
think ahead are not entirely unintentional or unwanted side-effects :)

Look at the latest librte_vhost initiative to remove unnecessarily 
exposed structs to avoid having to deal with ABI breakages all the time: 
the policy is effectively encouraging people into better library design.

> pollutes the repository with commits announcing ABI changes that are
> irrelevant after 2 cycles, as (code) diffs show that already (not
> mentioning NEXT_ABI complexity and extra LOCs).

Fully agreed on NEXT_ABI, I never liked it at all.

> Maintaining LTS releases, and enforcing bug fixing in old LTS first,
> upstreaming bugfixes is to me a much better approach to solve backwards
> compatibility issues.

LTS releases could help the situation somewhat, but then again people 
tend to still want those new fancy things backported (you know, have the 
cake and eat it too) but that can't be done because of ABI breakage, so 
they're forced to run the latest version anyway.

> But this is probably another discussion.

Yup, changed subject to avoid mixing it up with the namespace discussion 
too much.

- Panu -