--> On Mon, Feb 27, 2017 at 1:27 PM, Pramod Immaneni <pra...@datatorrent.com> wrote:
> For malhar, for existing operators, I prefer we do this as part of the > planned refactoring for breaking the monolith modules into baby packages > and would also prefer deprecating the existing operators in place. Refactor into smaller modules was discussed for malhar-contrib and given the overall state of that module I think it is OK to defer package renaming there. I do however prefer to see the package rename addressed for other modules, especially for the main library module. > This > will help us achieve two things. First, the user will see all the new > changes at once as opposed to dealing with it twice (with package rename > and dependency changes) and second it will allow for a smoother transition > as the existing code will still work in a deprecated state. It will also > give a more consistent structure to malhar. For new operators, we can go > with the new package path but we need to ensure they will get moved into > the baby packages as well. > I think existing operators should be renamed so that git history remains. A possible solution for backward compatibility could be to subsequently add empty subclasses in the previous location (for existing concrete operators that we know are actually in use) to simplify migration for users. > For demos, we can modify the paths as the apps are typically used wholesale > and the interface is typically manual interaction. > > For core, if we are adding new api subsystems, like the launcher api we > added recently for example, we can go with new package path but if we are > making incremental additions to existing functionality, I feel it is better > to keep it in the same package. I also prefer we keep the package of the > implementation classes consistent with api, for understandability and > readability of the code. So, for example, we don't change package path of > LogicalPlan as it is an implementation of DAG. It is subjective, but it > will be good if we can also do the same with classes closely related to the > implementation classes as well. Maybe we can moving these on a package by > package basis, like everything in com.datatorrent.stram.engine could be > moved. For completely internal components like buffer server, we can move > them wholesale. We can consider moving all api and classes, when we go to > next major release but would like to see if we can find a way to support > existing api for one more major release in deprecated mode. > > The point of the major release is to enable backward incompatible changes and I don't think it is realistic to support the existing API for another major release. IMO it is also not necessary as most existing application code refers to operators, attributes and the application interface. Perhaps it is possible to keep those around as interface extensions to help migration. Custom operators may need to be migrated to reflect API changes, and I would consider that a reasonable task for operator developers as part of a major upgrade. API and implementation in engine are kept separate intentionally. They reside in different packages today, so I don't see a problem renaming com.datatorrent.stram.engine as you say, even when the API cannot be touched right away. Thanks > > On Mon, Feb 27, 2017 at 7:39 AM, Thomas Weise <t...@apache.org> wrote: > > > Hi, > > > > This topic has come up on several PRs and I think it warrants a broader > > discussion. > > > > At the time of incubation, the decision was to defer change of Java > > packages from com.datatorrent to org.apache.apex till next major release > to > > ensure backward compatibility for users. > > > > Unfortunately that has lead to some confusion, as contributors continue > to > > add new code under legacy packages. > > > > It is also a wider issue that examples for using Apex continue to refer > to > > com.datatorrent packages, nearly one year after graduation. More and more > > user code is being built on top of something that needs to change, the > can > > is being kicked down the road and users will face more changes later. > > > > I would like to propose the following: > > > > 1. All new code has to be submitted under org.apache.apex packages > > > > 2. Not all code is under backward compatibility restriction and in those > > cases we can rename the packages right away. Examples: buffer server, > > engine, demos/examples, benchmarks > > > > 3. Discuss when the core API and operators can be changed. For operators > we > > have a bit more freedom to do changes before a major release as most of > > them are marked @Evolving and users have the ability to continue using > > prior version of Malhar with newer engine due to engine backward > > compatibility guarantee. > > > > Thanks, > > Thomas > > >
