04/04/2019 16:07, Burakov, Anatoly: > On 04-Apr-19 1:52 PM, Ray Kinsella wrote: > > On 04/04/2019 11:54, Bruce Richardson wrote: > >> On Thu, Apr 04, 2019 at 10:29:19AM +0100, Burakov, Anatoly wrote: > >>> On 03-Apr-19 4:42 PM, Ray Kinsella wrote: > >> Actually, I think we *do* need to constrain the pace of development for the > >> sake of ABI stability. At this stage DPDK has been around for quite a > >> number of years and so should be considered a fairly mature project - it > >> should just start acting like it. > > > > I 100% agree. > > > > If you break your users stuff regularly enough, they will eventually > > start looking around for an alternative that doesn't break their stuff > > quiet so regularly. > > > > We often use the pace of innovation in DPDK as justification for ABI/API > > breakages, but that approach is a real rarity among the Open Source > > community. I can't think of any mature project off-hand that share's it. > > > > I would ask is Linux any less innovative because they offer a stable API > > and have an absolute commitment to never breaking userspace? Would Linux > > have ever been as popular as it is today it they broke userspace every > > quarter? > > > > They reality is that they (Linux) find workarounds and compromise > > because there is an uber-maintainer Linus who had a strong ethos from > > the start not to break their users stuff - we need the same ethos in DPDK. > > > >> > >> Now, in terms of features like the memory rework, that is indeed a case > >> that there was no alternative other than a massive ABI break. However, for > >> that rework there was a strong need for improvement in that area that we > >> can make the case for an ABI break to support it - and it is of a scale > >> that nothing other than an ABI change would do. For other areas and > >> examples, I doubt there are many in the last couple of years that are of > >> that scale. > > > > I would also be inclined to agree with Bruce's points on memory rework > > was somewhat of an outlier, we don't see many like it. > >> My thoughts on the matter are: > >> 1. I think we really need to do work to start hiding more of our data > >> structures - like what Stephen's latest RFC does. This hiding should reduce > >> the scope for ABI breaks. > >> 2. Once done, I think we should commit to having an ABI break only in the > >> rarest of circumstances, and only with very large justification. I want us > >> to get to the point where DPDK releases can immediately be picked up by all > >> linux distros and rolled out because they are ABI compatible. > > > > The work that Anatoly describes removing APIs that exposed internal > > structures and Stephen H's RFC similarly are good examples of the kind > > of work required to prepare for this change. We need to take a good look > > at the API and reduce the number of unnecessary internal structures > > exposed. > > > > I never expected it going to to be a big bang - but is a definite > > direction we need to move towards over the next few release. > > ...in this case, we have to think long and hard about the fabled EAL > rework/split, and in general *specifying* what is it that we want to > support, and the use cases that we want to target. Right now there is a > huge mountain of technical debt and kludges and workarounds that has > accumulated over the years, and it exists precisely because "every > change breaks someone's workflow". > > For example, just in memory subsystem alone, we have legacy mem, because > some use cases require huge amounts of contiguous memory, and not > everyone is using VFIO; there's all of the 32-bit related workarounds > and hacks; there's the single-file-segments stuff that could have been > the default if not for the fact that we support kernels that don't > support fallocate(); there are two different ways of doing in-memory > mode, because not all kernels support memfd's; there is a gargantuan > pile of workarounds (and "known issues", and just code in general) all > over the DPDK codebase just to support our multiprocess model and all of > the various warts that come with it. > > In fact, i would even go as far as to say that *most* of EAL ABI breaks > have been due to the fact that we store data in shared memory because of > multiprocess - so there is simply no way we can change these internal > data structures without ABI breaks, because even if they're not exposed > through user-facing API, they are still exposed by virtue of secondary > processes basically having an ABI contract with primary process instances. > > So, if we are to cement our core API - we have to make a concrete effort > to specify what goes and what stays, if we want it to be maintainable. > The DPDK 1.0 specification, if you will :)
"DPDK 1.0 specification", that's a great project name :-)
