--> On Tue, Feb 9, 2016 at 9:05 AM, Chinmay Kolhatkar <[email protected]> wrote:
> Hello Everyone!! > > Continuing with packaging effort (rpm+deb) of apex, here are some proposals > about package structure etc.. > Before posting it on bbigtop mailing list, I have some question for apex > community. > > Proposed Directory structure of apex package for both deb & rpm: > > /usr/lib/apex/bin/dtcli > /usr/lib/apex/lib/apex-api-<version>.jar > /usr/lib/apex/lib/apex-engine-<version>.jar > /usr/lib/apex/lib/apex-bufferserver-<version>.jar > /usr/lib/apex/lib/apex-common-<version>.jar > /usr/lib/apex/lib/<other dependent jars> /usr/bin/dtcli -> /usr/lib/apex/bin/dtcli > /usr/share/doc/man/man1/dtcli.1.gz > /usr/share/doc/apex/license/LICENSE.txt > /usr/share/doc/apex/license/<package>-LICENSE.txt > /usr/share/doc/apex/CHANGELOG > /usr/share/doc/apex/NOTICE > > > <version> = 3.3.0-incubating. > <other dependent jars> = All the 3rd party jars which are required for apex > to run. Usually the dependencies are packaged as part of rpm/deb by any > software in bigtop. > Can you specify what those jars are in Bigtop context? Same as shipped with DT community addition under lib/ ? > <package LICENSE> = Licenses of corresponding of 3rd party jars which needs > to included while packaging. > > Questions related to this: > 1. Should we call the cli of apex as "apex" instead of "dtcli" in bigtop > package? > I think we should keep the name until we are able to change it in Apex. Otherwise this may get confusing. > 2. I see that all softwares in bigtop have man page for their executables. > I think we should have it too for dtcli. Is there any documentation which I > can convert to man page? or can I use output of "dtcli --help"? > 3. Do we want to call version of apex in Bigtop as 3.3.0 OR > 3.3.0-incubating? > Has to be -incubating > 4. Is it ok for apex package of bigtop to depend on 2.7.1 version of > bigtop hadoop? Any problems that we see with this dependency? > Should work without changed. I thought we certified against 2.7.0? > 5. Following is the apache mirror from which bigtop will pick the apex > source for compilation and packaging. Please correct if incorrect: > http://apache.osuosl.org/incubator/apex/v3.3.0-incubating/ > NOTE: apache.osuosl.org is the mirror used by all the softwares in > bigtop. > > > Please share your opinion. > > Thanks, > Chinmay. > > > > > > On Mon, Feb 8, 2016 at 11:19 AM, Chinmay Kolhatkar < > [email protected]> > wrote: > > > Yes.. Starting to work on the packaging. > > > > I've already started discussion on bigtop dev mailing list for > > integration. Also created a Jira for the same. For this communities > > reference, here is the bigtop Jira: BIGTOP-2313. > > > > > > > > On Mon, Feb 8, 2016 at 1:13 AM, Thomas Weise <[email protected]> > > wrote: > > > >> Chinmay, > >> > >> I don't see anything under prerequisites that would be a problem. We > >> looked > >> at the ASF licencing compatibility as part of becoming an incubator > >> project. > >> > >> Please focus on the packaging during the next weeks. Since the work will > >> be > >> part of Bigtop, related discussions and JIRAs should also be there. > >> > >> Would be good to have the packaging in place by end Feb. > >> > >> Thanks, > >> Thomas > >> > >> > >> On Thu, Feb 4, 2016 at 10:14 AM, Chinmay Kolhatkar < > >> [email protected]> > >> wrote: > >> > >> > Hi All, > >> > > >> > We're planning a work on adding Apache Apex as a component to Apache > >> > Bigtop. > >> > Bigtop is the packaging system for the Apache big data ecosystem. > >> Several > >> > Hadoop distros use it, most recently EMR. > >> > > >> > Here is the tracking Jira task in APEXCORE for the same: > >> > https://issues.apache.org/jira/browse/APEXCORE-331 > >> > > >> > Proposed plan of execution is as follows: > >> > *Step 1) Handle prerequisites* > >> > Apache bigtop has some hard and soft expectation for new components to > >> get > >> > integrated into Bigtop. > >> > Here is the list of it: > >> > > >> > > >> > https://cwiki.apache.org/confluence/display/BIGTOP/Requirement+for+adding+a+new+component+to+Bigtop+distribution > >> > > >> > Most of them seems to be standard ASF based requirements, but few need > >> to > >> > be checked for: > >> > 1. Software projects are expected to be Licensed under Apache License, > >> > Version 2.0 (and their dependencies are expected to be compatible with > >> this > >> > license) > >> > - Apex is under ASL 2.0 but need to check if dependencies of Apex > >> are > >> > compatible with ASL 2.0. This I guess would be a verification check. > >> > 2. Software projects are expected to be compatible with all of the > >> > supported platforms that Bigtop distribution is targeting > >> > - This needs verifying whether our software runs fine in centos-6 > >> > centos-7 fedora-20 ubuntu-14.04 debian-8 opensuse-13.2. > >> > 3. What smoke tests that should be added for deployment. > >> > 4. Identifying the test artifacts which goes beyond smoke test > >> > - These are basically the integration tests for verification after > >> the > >> > deployment. This is a soft requirement, but aim is to achieve this as > >> well > >> > or at least have explanation why not to include. > >> > > >> > If there are any from the link which explicitly needs to be checked > >> other > >> > than above 4, please let us know. > >> > > >> > *Step 2) Adding Apex as component to Bigtop* > >> > From one of the mail archive of Bigtop, it was learnt that the bigtop > >> > community want to see the addition of new components in phases. Here > are > >> > the phases: > >> > 1. Packaging > >> > - This needs creating of package i.e. rpm & deb files. > >> > - documentations/READMEs, LICENSE, DISCLAMER, NOTES etc if any > >> needed. > >> > - Any documentation that need to be added to distribution of our > >> > software. > >> > - Any license information of dependencies required to be added to > >> > package > >> > 2. Smoke tests (at very least) > >> > - Adding smoke test for packaging. > >> > 3. Puppet recipes for automatic deployment and configuration > >> > - Add puppet recipes for automatic deployment > >> > 4. Integration tests > >> > - For verification of deployments. > >> > 5. license clearance: > >> > Run 'gradle rat' to make sure all new stuff is compliant with ASF > >> > license requirements. If you add code licenses under different > licenses, > >> > those would need to be listed in the NOTICE. > >> > > >> > Please share your thoughts on the approach. > >> > We'll start corresponding communication on bigtop mailing list as > well. > >> > > >> > We have some specific questions/suggestions related to what should be > >> the > >> > content of the package and what should be the smoke tests, but in the > >> > interest of not having too much content here, we'll put the questions > >> as a > >> > separate mail in this mailthread. > >> > > >> > Thanks, > >> > Chinmay. > >> > > >> > > > > >
