Yes. Made that change and also those suggested by pradeep. Thanks, Chinmay. On 18 Feb 2016 7:15 p.m., "Munagala Ramanath" <[email protected]> wrote:
> Tushar is right: apex (formerly dtcli) talks directly to Yarn and does not > depend on dtGateway or the REST API. > > Ram > > On Thu, Feb 18, 2016 at 12:07 AM, Tushar Gosavi <[email protected]> > wrote: > > > In following statement, can you replace dtGateway with ApplicationMaster > > and Yarn. or may be remove this statement entirely. > > apex is a wrapper around the REST API provided by dtGatway, > > > > - Tushar. > > > > > > > > > > On Thu, Feb 18, 2016 at 12:41 PM, Chinmay Kolhatkar <[email protected]> > > wrote: > > > > > Hi, > > > > > > I've attached man page for apex in Jira. Here is the link to it: > > > https://issues.apache.org/jira/secure/attachment/12788403/apex.1 > > > > > > Please view it by: man -l apex.1 > > > > > > Thanks, > > > Chinmay. > > > > > > > > > On Thu, Feb 18, 2016 at 12:51 AM, Tushar Gosavi < > [email protected]> > > > wrote: > > > > > > > Hi Chinmay, > > > > > > > > I am not able to find the attachment, may be attachments are not > > allowed > > > in > > > > the mailing list. Can you attach the man page to the Jira? > > > > > > > > - Tushar. > > > > > > > > > > > > On Wed, Feb 17, 2016 at 2:52 PM, Chinmay Kolhatkar < > [email protected] > > > > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > I've attached man page file for the purpose of bigtop integration. > > > > > Please have a look and let me know if there any any feedback about > > it. > > > > > > > > > > You can see the man page content of this file with: > > > > > man -l apex.1 > > > > > > > > > > Thanks, > > > > > Chinmay. > > > > > > > > > > > > > > > > > > > > On Wed, Feb 17, 2016 at 11:04 AM, Chinmay Kolhatkar < > > > [email protected]> > > > > > wrote: > > > > > > > > > >> Thanks. Will follow bigtop recommendation. i.e. remove > > "-incubating". > > > > >> > > > > >> On Tue, Feb 16, 2016 at 9:52 PM, Thomas Weise < > > [email protected] > > > > > > > > >> wrote: > > > > >> > > > > >>> Please follow the recommendation and continue discussion on the > > > Bigtop > > > > >>> list. > > > > >>> > > > > >>> -- > > > > >>> sent from mobile > > > > >>> On Feb 16, 2016 8:17 AM, "Chinmay Kolhatkar" <[email protected] > > > > > > wrote: > > > > >>> > > > > >>> > I got a recommendation from bigtop community to drop > > "-incubating" > > > > from > > > > >>> > version. > > > > >>> > > > > > >>> > Here is the mail thread: > > > > >>> > > > > > >>> > > > > > >>> > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/bigtop-dev/201602.mbox/%3C20160216154548.GA4472%40tpx%3E > > > > >>> > > > > > >>> > Please let me know if removing "-incubating" is fine. > > > > >>> > > > > > >>> > Thanks, > > > > >>> > Chinmay. > > > > >>> > > > > > >>> > > > > > >>> > On Tue, Feb 16, 2016 at 9:06 PM, Chinmay Kolhatkar < > > > > [email protected] > > > > >>> > > > > > >>> > wrote: > > > > >>> > > > > > >>> > > I've already started communication there. > > > > >>> > > > > > > >>> > > On Tue, Feb 16, 2016 at 8:33 PM, Thomas Weise < > > > > >>> [email protected]> > > > > >>> > > wrote: > > > > >>> > > > > > > >>> > >> This is a question you should ask on the Bigtop mailing > list. > > > > >>> > >> > > > > >>> > >> -- > > > > >>> > >> sent from mobile > > > > >>> > >> On Feb 16, 2016 5:25 AM, "Pradeep A. Dalvi" < > [email protected]> > > > > >>> wrote: > > > > >>> > >> > > > > >>> > >> > Yes. In case of RPM, the only restriction on version is > that > > > it > > > > >>> can > > > > >>> > not > > > > >>> > >> > contain '-'. > > > > >>> > >> > However, as a workaround, one can use 'incubating' as a > part > > > of > > > > >>> > >> %{release}. > > > > >>> > >> > > > > > >>> > >> > Version: 3.3.0 > > > > >>> > >> > Release: incubating > > > > >>> > >> > > > > > >>> > >> > apex-3.3.0-incubating.noarch.rpm > > > > >>> > >> > > > > > >>> > >> > It is the same practice followed for pre-release packages > > e.g. > > > > >>> with > > > > >>> > rcX, > > > > >>> > >> > alphaX, betaX being used in %{release}. > > > > >>> > >> > > > > > >>> > >> > Regards, > > > > >>> > >> > Pradeep A. Dalvi > > > > >>> > >> > > > > > >>> > >> > On Tue, Feb 16, 2016 at 6:35 PM, Chinmay Kolhatkar < > > > > >>> > [email protected]> > > > > >>> > >> > wrote: > > > > >>> > >> > > > > > >>> > >> > > Hi Everyone, > > > > >>> > >> > > > > > > >>> > >> > > In one of the previous mails in this thread, we decided > > that > > > > >>> version > > > > >>> > >> of > > > > >>> > >> > the > > > > >>> > >> > > package will be "3.3.0-incubating". > > > > >>> > >> > > The problem that I'm facing is in RPM base version "-" > is > > > not > > > > >>> > allowed. > > > > >>> > >> > > > > > > >>> > >> > > Can we have package version to "3.3.0"? > > > > >>> > >> > > i.e. the packages created will be as follows: > > > > >>> > >> > > For RPM: apex-3.3.0-1.noarch.rpm > > > > >>> > >> > > For DEB: apex_3.3.0-1_all.deb > > > > >>> > >> > > > > > > >>> > >> > > There is another integration for Zepplin in Bigtop which > > is > > > > >>> > incubating > > > > >>> > >> > and > > > > >>> > >> > > is having base version as "0.5.6". > > > > >>> > >> > > > > > > >>> > >> > > Please let me know your thoughts. > > > > >>> > >> > > > > > > >>> > >> > > Thanks, > > > > >>> > >> > > Chinmay. > > > > >>> > >> > > > > > > >>> > >> > > > > > > >>> > >> > > > > > > >>> > >> > > On Tue, Feb 16, 2016 at 11:48 AM, Chinmay Kolhatkar < > > > > >>> > >> [email protected]> > > > > >>> > >> > > wrote: > > > > >>> > >> > > > > > > >>> > >> > > > Thanks everyone for the feedback. > > > > >>> > >> > > > > > > > >>> > >> > > > Also as Ram and David mentioned, I'll start creating > the > > > man > > > > >>> page > > > > >>> > as > > > > >>> > >> > > well. > > > > >>> > >> > > > > > > > >>> > >> > > > Thanks, > > > > >>> > >> > > > Chinmay. > > > > >>> > >> > > > > > > > >>> > >> > > > > > > > >>> > >> > > > On Tue, Feb 16, 2016 at 6:30 AM, Thomas Weise < > > > > >>> > >> [email protected]> > > > > >>> > >> > > > wrote: > > > > >>> > >> > > > > > > > >>> > >> > > >> I think we already agreed that we will retain the > > current > > > > >>> script > > > > >>> > >> name > > > > >>> > >> > in > > > > >>> > >> > > >> this bigtop iteration. There is a ticket for > renaming, > > > add > > > > >>> you > > > > >>> > >> > > suggestions > > > > >>> > >> > > >> there. > > > > >>> > >> > > >> > > > > >>> > >> > > >> -- > > > > >>> > >> > > >> sent from mobile > > > > >>> > >> > > >> On Feb 15, 2016 4:56 PM, "Sandesh Hegde" < > > > > >>> > [email protected]> > > > > >>> > >> > > wrote: > > > > >>> > >> > > >> > > > > >>> > >> > > >> > +1 for Symlink from apex to dtcli or start the > > > transition > > > > >>> > >> process by > > > > >>> > >> > > >> making > > > > >>> > >> > > >> > dtcli as the symlink to apex cli. > > > > >>> > >> > > >> > > > > > >>> > >> > > >> > Renaming is a breaking change should be done during > > > major > > > > >>> > version > > > > >>> > >> > > number > > > > >>> > >> > > >> > change. > > > > >>> > >> > > >> > > > > > >>> > >> > > >> > On Mon, Feb 15, 2016 at 9:57 AM David Yan < > > > > >>> > [email protected] > > > > >>> > >> > > > > > >>> > >> > > >> wrote: > > > > >>> > >> > > >> > > > > > >>> > >> > > >> > > Apologies for hitting the "Send" button too > early. > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > What is the scope of the name change? If we > simply > > > > have > > > > >>> a > > > > >>> > >> symlink > > > > >>> > >> > > >> from > > > > >>> > >> > > >> > > apex to dtcli, then it should have no backward > > > > >>> compatibility > > > > >>> > >> > problem > > > > >>> > >> > > >> at > > > > >>> > >> > > >> > > all. > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > But the proper name change would involve: > > > > >>> > >> > > >> > > - Change the name of DTCli class to ApexCli > > > > >>> > >> > > >> > > - Change the prompt from dt> to apex> > > > > >>> > >> > > >> > > - Change the initial greeting when running the > CLI > > > from > > > > >>> > >> something > > > > >>> > >> > > like > > > > >>> > >> > > >> > "DT > > > > >>> > >> > > >> > > CLI 3.3.0-incubating ..." to "Apex CLI ...". > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > The above changes may break existing scripts that > > > rely > > > > >>> on the > > > > >>> > >> the > > > > >>> > >> > > >> string > > > > >>> > >> > > >> > > "DT". > > > > >>> > >> > > >> > > But if folks are okay with this potential > problem, > > I > > > am > > > > >>> okay > > > > >>> > >> with > > > > >>> > >> > > the > > > > >>> > >> > > >> > name > > > > >>> > >> > > >> > > change. > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > For the man page, we can start with the content > of > > > this > > > > >>> page: > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > http://docs.datatorrent.com/dtcli/ > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > Plus the command line options when doing dtcli > -h: > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > usage: DTCli > > > > >>> > >> > > >> > > -e <arg> Commands are read from the argument > > > > >>> > >> > > >> > > -f <arg> Use the specified prompt at all time > > > > >>> > >> > > >> > > -h Print this help > > > > >>> > >> > > >> > > -kp <arg> Use the specified kerberos principal > > > > >>> > >> > > >> > > -kt <arg> Use the specified kerberos keytab > > > > >>> > >> > > >> > > -p <arg> JSONP padding function > > > > >>> > >> > > >> > > -r JSON Raw mode > > > > >>> > >> > > >> > > -v Verbose mode level 1 > > > > >>> > >> > > >> > > -vv Verbose mode level 2 > > > > >>> > >> > > >> > > -vvv Verbose mode level 3 > > > > >>> > >> > > >> > > -vvvv Verbose mode level 4 > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > David > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > On Mon, Feb 15, 2016 at 9:56 AM, David Yan < > > > > >>> > >> [email protected] > > > > >>> > >> > > > > > > >>> > >> > > >> > wrote: > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > > What is the scope of the name change? If we > > simply > > > > >>> have a > > > > >>> > >> > symlink > > > > >>> > >> > > >> from > > > > >>> > >> > > >> > > > apex to dtcli, then it should have no backward > > > > >>> > compatibility > > > > >>> > >> > > >> problem at > > > > >>> > >> > > >> > > all. > > > > >>> > >> > > >> > > > > > > > >>> > >> > > >> > > > But the proper name change would involve: > > > > >>> > >> > > >> > > > - Change the name of DTCli class to ApexCli > > > > >>> > >> > > >> > > > - Change the prompt from dt> to apex> > > > > >>> > >> > > >> > > > - Change the initial greeting when running the > > CLI > > > > from > > > > >>> > >> > something > > > > >>> > >> > > >> like > > > > >>> > >> > > >> > > "DT > > > > >>> > >> > > >> > > > CLI 3.3.0-incubating ..." to "Apex CLI ...". > > > > >>> > >> > > >> > > > > > > > >>> > >> > > >> > > > The above changes may break existing scripts > that > > > > rely > > > > >>> on > > > > >>> > the > > > > >>> > >> > the > > > > >>> > >> > > >> > string > > > > >>> > >> > > >> > > > "DT". > > > > >>> > >> > > >> > > > But if folks are okay with this potential > > problem, > > > I > > > > am > > > > >>> > okay > > > > >>> > >> > with > > > > >>> > >> > > >> the > > > > >>> > >> > > >> > > name > > > > >>> > >> > > >> > > > change. > > > > >>> > >> > > >> > > > > > > > >>> > >> > > >> > > > For the man page, we can start with the content > > of > > > > this > > > > >>> > page: > > > > >>> > >> > > >> > > > > > > > >>> > >> > > >> > > > > > > > >>> > >> > > >> > > > > > > > >>> > >> > > >> > > > > > > > >>> > >> > > >> > > > On Fri, Feb 12, 2016 at 7:04 PM, Chinmay > > Kolhatkar > > > < > > > > >>> > >> > > >> [email protected] > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > > wrote: > > > > >>> > >> > > >> > > > > > > > >>> > >> > > >> > > >> Here is a suggestion I have related to name > > > change: > > > > >>> > >> > > >> > > >> > > > > >>> > >> > > >> > > >> 1. While packaging we change the name of > "dtcli" > > > > >>> script to > > > > >>> > >> > > "apex". > > > > >>> > >> > > >> > This > > > > >>> > >> > > >> > > is > > > > >>> > >> > > >> > > >> done so that in future when we change the name > > to > > > > >>> "apex", > > > > >>> > >> users > > > > >>> > >> > > of > > > > >>> > >> > > >> > > bigtop > > > > >>> > >> > > >> > > >> apex don't have to transition much. > > > > >>> > >> > > >> > > >> 2. We also keep a symlink named dtcli which > > points > > > > to > > > > >>> apex > > > > >>> > >> > > script. > > > > >>> > >> > > >> > This > > > > >>> > >> > > >> > > is > > > > >>> > >> > > >> > > >> for backward compatibility. > > > > >>> > >> > > >> > > >> 3. We still keep conf/dt-env.sh for first > > > > integration > > > > >>> in > > > > >>> > >> > bigtop. > > > > >>> > >> > > >> This > > > > >>> > >> > > >> > is > > > > >>> > >> > > >> > > >> in > > > > >>> > >> > > >> > > >> the interest of not changing the content of > > dtcli > > > or > > > > >>> apex > > > > >>> > >> > script. > > > > >>> > >> > > >> > > >> 4. In apex 3.4.0, we take care of all these > > naming > > > > >>> related > > > > >>> > >> > > changes > > > > >>> > >> > > >> and > > > > >>> > >> > > >> > > >> update bigtop repository later to remove > > > references > > > > to > > > > >>> > dtcli > > > > >>> > >> > all > > > > >>> > >> > > >> > > together. > > > > >>> > >> > > >> > > >> > > > > >>> > >> > > >> > > >> Please share your thoughts on above approach. > > > > >>> > >> > > >> > > >> > > > > >>> > >> > > >> > > >> Also, please share what could be the man page > > > > content > > > > >>> for > > > > >>> > >> > "apex". > > > > >>> > >> > > >> > > >> > > > > >>> > >> > > >> > > >> Thanks, > > > > >>> > >> > > >> > > >> Chinmay. > > > > >>> > >> > > >> > > >> > > > > >>> > >> > > >> > > >> > > > > >>> > >> > > >> > > >> > > > > >>> > >> > > >> > > >> On Sat, Feb 13, 2016 at 8:17 AM, Chinmay > > > Kolhatkar < > > > > >>> > >> > > >> > [email protected]> > > > > >>> > >> > > >> > > >> wrote: > > > > >>> > >> > > >> > > >> > > > > >>> > >> > > >> > > >> > Yes.. It'll be build from source tar > > downloaded > > > > >>> from one > > > > >>> > >> of > > > > >>> > >> > the > > > > >>> > >> > > >> > apache > > > > >>> > >> > > >> > > >> > mirror: apache.osuosl.org > > > > >>> > >> > > >> > > >> > > > > > >>> > >> > > >> > > >> > The reason for dt-env.sh is it's sourced > from > > > > dtcli. > > > > >>> > This > > > > >>> > >> > > leaves > > > > >>> > >> > > >> us > > > > >>> > >> > > >> > 3 > > > > >>> > >> > > >> > > >> > options: > > > > >>> > >> > > >> > > >> > If we should not have any dt-env.sh, dtcli > > would > > > > >>> need a > > > > >>> > >> > change > > > > >>> > >> > > >> while > > > > >>> > >> > > >> > > >> > packaging. > > > > >>> > >> > > >> > > >> > > > > > >>> > >> > > >> > > >> > But if we want to use dtcli as it is, then > we > > > > would > > > > >>> need > > > > >>> > >> > > >> dt-env.sh > > > > >>> > >> > > >> > > >> > > > > > >>> > >> > > >> > > >> > > > > > >>> > >> > > >> > > >> > > > > > >>> > >> > > >> > > >> > On Sat, Feb 13, 2016 at 8:07 AM, Thomas > Weise > > < > > > > >>> > >> > > >> > [email protected] > > > > >>> > >> > > >> > > > > > > > >>> > >> > > >> > > >> > wrote: > > > > >>> > >> > > >> > > >> > > > > > >>> > >> > > >> > > >> >> Looks good overall, though there shouldn't > be > > > any > > > > >>> > >> dt-env.sh > > > > >>> > >> > > >> > > >> >> > > > > >>> > >> > > >> > > >> >> What will the bigtop package be built from, > > the > > > > >>> source > > > > >>> > >> tar? > > > > >>> > >> > > >> > > >> >> > > > > >>> > >> > > >> > > >> >> > > > > >>> > >> > > >> > > >> >> > > > > >>> > >> > > >> > > >> >> On Fri, Feb 12, 2016 at 6:31 PM, Chinmay > > > > Kolhatkar > > > > >>> < > > > > >>> > >> > > >> > > [email protected] > > > > >>> > >> > > >> > > >> > > > > > >>> > >> > > >> > > >> >> wrote: > > > > >>> > >> > > >> > > >> >> > > > > >>> > >> > > >> > > >> >> > Hi Thomas, > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > Thanks for the feedback. > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > First, we're not changing the name > > anywhere. > > > > >>> We'll > > > > >>> > >> follow > > > > >>> > >> > > what > > > > >>> > >> > > >> > > >> >> currently is > > > > >>> > >> > > >> > > >> >> > in source tarball for 3.3.0-incubating > > > version. > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > Secondly, I've mentioned a directory > > > structure > > > > >>> below > > > > >>> > >> which > > > > >>> > >> > > is > > > > >>> > >> > > >> > > inline > > > > >>> > >> > > >> > > >> >> with > > > > >>> > >> > > >> > > >> >> > other existing integrations with bigtop. > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > The need for each file is as follows: > > > > >>> > >> > > >> > > >> >> > 1. bin/dtcli -> CLI for apex picked from > > > > >>> > >> > > >> engine/src/main/scripts/ > > > > >>> > >> > > >> > > of > > > > >>> > >> > > >> > > >> >> source > > > > >>> > >> > > >> > > >> >> > code apex 3.3.0-incubating. > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > 2. conf/dt-env.sh -> This is source from > > > dtcli. > > > > >>> This > > > > >>> > >> > > searches > > > > >>> > >> > > >> for > > > > >>> > >> > > >> > > >> hadoop > > > > >>> > >> > > >> > > >> >> > binary path and exports an env variable > for > > > > >>> dtcli to > > > > >>> > >> use > > > > >>> > >> > it. > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > 3. lib/ -> Libraries for CLI. This list > is > > > > >>> extracted > > > > >>> > >> in a > > > > >>> > >> > > >> similar > > > > >>> > >> > > >> > > >> way to > > > > >>> > >> > > >> > > >> >> > how DT Community Edition finds dependency > > > jars. > > > > >>> > >> > > >> > > >> >> > Please note that this is the first > > iteration > > > > >>> list of > > > > >>> > >> jars. > > > > >>> > >> > > I'm > > > > >>> > >> > > >> > > >> trying to > > > > >>> > >> > > >> > > >> >> > narrow this down to only those which are > > > really > > > > >>> > >> required. > > > > >>> > >> > > >> > > >> >> > The test that I'm running to check if > dtcli > > > > runs > > > > >>> fine > > > > >>> > >> with > > > > >>> > >> > > >> given > > > > >>> > >> > > >> > > >> >> dependency > > > > >>> > >> > > >> > > >> >> > is to launch+shutdown+kill+status for pi > > > demo. > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > Above is the directory structure which is > > > > >>> required > > > > >>> > for > > > > >>> > >> CLI > > > > >>> > >> > > to > > > > >>> > >> > > >> > work. > > > > >>> > >> > > >> > > >> >> > What aniruddha and pradeep has mentioned > > are > > > > some > > > > >>> > >> > additional > > > > >>> > >> > > >> > files > > > > >>> > >> > > >> > > >> which > > > > >>> > >> > > >> > > >> >> > makes the like of administrator easier. > For > > > eg. > > > > >>> > >> /etc/skel > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > Please let me know if this is inline with > > > what > > > > >>> you're > > > > >>> > >> > > >> thinking. > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > Thanks, > > > > >>> > >> > > >> > > >> >> > Chinmay. > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > . > > > > >>> > >> > > >> > > >> >> > |-- bin > > > > >>> > >> > > >> > > >> >> > | `-- dtcli > > > > >>> > >> > > >> > > >> >> > |-- conf > > > > >>> > >> > > >> > > >> >> > | `-- dt-env.sh > > > > >>> > >> > > >> > > >> >> > `-- lib > > > > >>> > >> > > >> > > >> >> > |-- ant-1.9.2.jar > > > > >>> > >> > > >> > > >> >> > |-- ant-launcher-1.9.2.jar > > > > >>> > >> > > >> > > >> >> > |-- apex-api-3.3.0-incubating.jar > > > > >>> > >> > > >> > > >> >> > |-- > > > apex-bufferserver-3.3.0-incubating.jar > > > > >>> > >> > > >> > > >> >> > |-- apex-common-3.3.0-incubating.jar > > > > >>> > >> > > >> > > >> >> > |-- apex-engine.jar > > > > >>> > >> > > >> > > >> >> > |-- apex-engine-tests.jar > > > > >>> > >> > > >> > > >> >> > |-- async-http-client-1.7.20.jar > > > > >>> > >> > > >> > > >> >> > |-- bval-core-0.5.jar > > > > >>> > >> > > >> > > >> >> > |-- bval-jsr303-0.5.jar > > > > >>> > >> > > >> > > >> >> > |-- commons-beanutils-1.8.3.jar > > > > >>> > >> > > >> > > >> >> > |-- commons-codec-1.10.jar > > > > >>> > >> > > >> > > >> >> > |-- commons-lang3-3.1.jar > > > > >>> > >> > > >> > > >> >> > |-- commons-logging-1.1.3.jar > > > > >>> > >> > > >> > > >> >> > |-- grizzly-http-servlet-2.1.2.jar > > > > >>> > >> > > >> > > >> >> > |-- hadoop-common-2.2.0-tests.jar > > > > >>> > >> > > >> > > >> >> > |-- httpclient-4.3.5.jar > > > > >>> > >> > > >> > > >> >> > |-- httpcore-4.3.2.jar > > > > >>> > >> > > >> > > >> >> > |-- jackson-core-asl-1.9.2.jar > > > > >>> > >> > > >> > > >> >> > |-- jackson-mapper-asl-1.9.2.jar > > > > >>> > >> > > >> > > >> >> > |-- javax.servlet-3.1.jar > > > > >>> > >> > > >> > > >> >> > |-- javax.servlet-api-3.0.1.jar > > > > >>> > >> > > >> > > >> >> > |-- jersey-apache-client4-1.9.jar > > > > >>> > >> > > >> > > >> >> > |-- jersey-client-1.9.jar > > > > >>> > >> > > >> > > >> >> > |-- jetty-http-8.1.10.v20130312.jar > > > > >>> > >> > > >> > > >> >> > |-- jetty-io-8.1.10.v20130312.jar > > > > >>> > >> > > >> > > >> >> > |-- jetty-util-8.1.10.v20130312.jar > > > > >>> > >> > > >> > > >> >> > |-- > > jetty-websocket-8.1.10.v20130312.jar > > > > >>> > >> > > >> > > >> >> > |-- jline-2.11.jar > > > > >>> > >> > > >> > > >> >> > |-- kryo-2.24.0.jar > > > > >>> > >> > > >> > > >> >> > |-- mbassador-1.1.9.jar > > > > >>> > >> > > >> > > >> >> > |-- minlog-1.2.jar > > > > >>> > >> > > >> > > >> >> > |-- netlet-1.2.0.jar > > > > >>> > >> > > >> > > >> >> > |-- netty-3.6.6.Final.jar > > > > >>> > >> > > >> > > >> >> > |-- objenesis-2.1.jar > > > > >>> > >> > > >> > > >> >> > |-- validation-api-1.1.0.Final.jar > > > > >>> > >> > > >> > > >> >> > |-- xbean-asm5-shaded-4.3.jar > > > > >>> > >> > > >> > > >> >> > `-- zip4j-1.3.2.jar > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > 3 directories, 40 files > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > On Sat, Feb 13, 2016 at 3:24 AM, Thomas > > > Weise < > > > > >>> > >> > > >> > > >> [email protected]> > > > > >>> > >> > > >> > > >> >> > wrote: > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > > Chinmay, > > > > >>> > >> > > >> > > >> >> > > > > > > >>> > >> > > >> > > >> >> > > Before discussing where to put the > files, > > > > let's > > > > >>> > make > > > > >>> > >> > sure > > > > >>> > >> > > >> they > > > > >>> > >> > > >> > > are > > > > >>> > >> > > >> > > >> >> really > > > > >>> > >> > > >> > > >> >> > > needed for the operation of the CLI. As > > for > > > > >>> names, > > > > >>> > >> > > anything > > > > >>> > >> > > >> new > > > > >>> > >> > > >> > > >> needs > > > > >>> > >> > > >> > > >> >> to > > > > >>> > >> > > >> > > >> >> > > reflect Apex in the name and should > > follow > > > > >>> common > > > > >>> > >> > > >> conventions, > > > > >>> > >> > > >> > > >> >> especially > > > > >>> > >> > > >> > > >> >> > > in Bigtop where there are many existing > > > > >>> > integrations > > > > >>> > >> to > > > > >>> > >> > > look > > > > >>> > >> > > >> > at. > > > > >>> > >> > > >> > > >> >> > > > > > > >>> > >> > > >> > > >> >> > > Thanks, > > > > >>> > >> > > >> > > >> >> > > Thomas > > > > >>> > >> > > >> > > >> >> > > > > > > >>> > >> > > >> > > >> >> > > > > > > >>> > >> > > >> > > >> >> > > On Wed, Feb 10, 2016 at 8:07 AM, > Chinmay > > > > >>> Kolhatkar > > > > >>> > < > > > > >>> > >> > > >> > > >> >> > > [email protected]> > > > > >>> > >> > > >> > > >> >> > > wrote: > > > > >>> > >> > > >> > > >> >> > > > > > > >>> > >> > > >> > > >> >> > > > Really good points Pradeep and > > Aniruddha. > > > > >>> > >> > > >> > > >> >> > > > > > > > >>> > >> > > >> > > >> >> > > > 1. I believe we won't need to change > > the > > > > >>> dtcli > > > > >>> > >> > > >> considering it > > > > >>> > >> > > >> > > >> works > > > > >>> > >> > > >> > > >> >> > with > > > > >>> > >> > > >> > > >> >> > > DT > > > > >>> > >> > > >> > > >> >> > > > Community edition. We can keep the > > > > directory > > > > >>> > >> structure > > > > >>> > >> > > >> > similar > > > > >>> > >> > > >> > > to > > > > >>> > >> > > >> > > >> >> that. > > > > >>> > >> > > >> > > >> >> > > > dt-env.sh has variables which > contains > > > > >>> > information > > > > >>> > >> > > >> required > > > > >>> > >> > > >> > for > > > > >>> > >> > > >> > > >> >> dtcli > > > > >>> > >> > > >> > > >> >> > to > > > > >>> > >> > > >> > > >> >> > > > launch. > > > > >>> > >> > > >> > > >> >> > > > > > > > >>> > >> > > >> > > >> >> > > > 2. Let me check with bigtop community > > > that > > > > >>> > whether > > > > >>> > >> > they > > > > >>> > >> > > >> > > >> facilitate > > > > >>> > >> > > >> > > >> >> the > > > > >>> > >> > > >> > > >> >> > > > installation of rpms/debs before user > > is > > > > >>> created. > > > > >>> > >> In > > > > >>> > >> > > >> either > > > > >>> > >> > > >> > > case, > > > > >>> > >> > > >> > > >> >> > current > > > > >>> > >> > > >> > > >> >> > > > dtcli creates a .dt folder in home > > > > directory. > > > > >>> > Also > > > > >>> > >> > > before > > > > >>> > >> > > >> > > putting > > > > >>> > >> > > >> > > >> >> > > anything > > > > >>> > >> > > >> > > >> >> > > > in /etc/skel we need to define what > are > > > the > > > > >>> > default > > > > >>> > >> > > >> contents > > > > >>> > >> > > >> > > that > > > > >>> > >> > > >> > > >> >> > should > > > > >>> > >> > > >> > > >> >> > > go > > > > >>> > >> > > >> > > >> >> > > > to ~/.dt/ folder. If there is no > > > defaults, > > > > >>> > >> probably we > > > > >>> > >> > > >> should > > > > >>> > >> > > >> > > not > > > > >>> > >> > > >> > > >> >> > > > explicitly add it in /etc/skel. > > > > >>> > >> > > >> > > >> >> > > > > > > > >>> > >> > > >> > > >> >> > > > 3. /etc/profile.d approach looks > nice. > > > > This > > > > >>> way > > > > >>> > >> > > contents > > > > >>> > >> > > >> of > > > > >>> > >> > > >> > > >> >> dt-env.sh > > > > >>> > >> > > >> > > >> >> > > are > > > > >>> > >> > > >> > > >> >> > > > present as env variables. But I see a > > > catch > > > > >>> > there. > > > > >>> > >> > > Adding > > > > >>> > >> > > >> > > >> dt-env.sh > > > > >>> > >> > > >> > > >> >> to > > > > >>> > >> > > >> > > >> >> > > > /etc/profile.d would make all the > > > variables > > > > >>> > >> available > > > > >>> > >> > at > > > > >>> > >> > > >> > > runtime > > > > >>> > >> > > >> > > >> all > > > > >>> > >> > > >> > > >> >> > the > > > > >>> > >> > > >> > > >> >> > > > time. I feel a little skeptical about > > > that. > > > > >>> > Maybe a > > > > >>> > >> > > >> possible > > > > >>> > >> > > >> > > >> >> collision > > > > >>> > >> > > >> > > >> >> > > can > > > > >>> > >> > > >> > > >> >> > > > occur with other application vars. > > > > >>> > >> > > >> > > >> >> > > > Moreover, current dtcli does source > > > > >>> > >> > "../conf/dt-env.sh" > > > > >>> > >> > > >> and > > > > >>> > >> > > >> > > >> >> > > > "~/.dt/dt-env.sh". So those variables > > are > > > > >>> anyway > > > > >>> > >> > > >> available. > > > > >>> > >> > > >> > > >> >> > > > > > > > >>> > >> > > >> > > >> >> > > > @Aniruddha, the Jira and PR effort is > > > > >>> happening > > > > >>> > at > > > > >>> > >> > > Bigtop > > > > >>> > >> > > >> > > >> >> > > Jira/repository. > > > > >>> > >> > > >> > > >> >> > > > Packaging related code usually goes > > > there. > > > > >>> > (That's > > > > >>> > >> > what > > > > >>> > >> > > >> all > > > > >>> > >> > > >> > the > > > > >>> > >> > > >> > > >> >> > > components > > > > >>> > >> > > >> > > >> >> > > > in bigtop does). > > > > >>> > >> > > >> > > >> >> > > > Having said that, once a PR is > created, > > > > I'll > > > > >>> be > > > > >>> > >> > sharing > > > > >>> > >> > > >> the > > > > >>> > >> > > >> > > link > > > > >>> > >> > > >> > > >> of > > > > >>> > >> > > >> > > >> >> PR > > > > >>> > >> > > >> > > >> >> > > > here, so that apex community as well > > can > > > > >>> review > > > > >>> > it. > > > > >>> > >> > > >> > > >> >> > > > > > > > >>> > >> > > >> > > >> >> > > > Thanks, > > > > >>> > >> > > >> > > >> >> > > > Chinmay. > > > > >>> > >> > > >> > > >> >> > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >>> > >> > > >> > > >> >> > > > On Wed, Feb 10, 2016 at 7:57 PM, > > > Aniruddha > > > > >>> > >> Thombare < > > > > >>> > >> > > >> > > >> >> > > > [email protected]> wrote: > > > > >>> > >> > > >> > > >> >> > > > > > > > >>> > >> > > >> > > >> >> > > > > +1 on suggestions and approach. > > > > >>> > >> > > >> > > >> >> > > > > We may need to iron out details > about > > > > exact > > > > >>> > paths > > > > >>> > >> > etc. > > > > >>> > >> > > >> > > >> >> > > > > Which can be done on jira / PR > > > comments. > > > > >>> > >> > > >> > > >> >> > > > > Is that right @dev? > > > > >>> > >> > > >> > > >> >> > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >>> > >> > > >> > > >> >> > > > > On Wed, 10 Feb 2016 7:53 pm Pradeep > > A. > > > > >>> Dalvi < > > > > >>> > >> > > >> > > >> >> > [email protected]> > > > > >>> > >> > > >> > > >> >> > > > > wrote: > > > > >>> > >> > > >> > > >> >> > > > > > > > > >>> > >> > > >> > > >> >> > > > > > Inline comments... > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > On Wed, Feb 10, 2016 at 2:51 PM, > > > > Chinmay > > > > >>> > >> > Kolhatkar < > > > > >>> > >> > > >> > > >> >> > > > > > [email protected]> > > > > >>> > >> > > >> > > >> >> > > > > > wrote: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > @Thomas, Not all the jar > present > > in > > > > DT > > > > >>> > >> community > > > > >>> > >> > > >> > edition > > > > >>> > >> > > >> > > >> will > > > > >>> > >> > > >> > > >> >> be > > > > >>> > >> > > >> > > >> >> > > > > included > > > > >>> > >> > > >> > > >> >> > > > > > > there. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > Here is the list of jars I > found > > > from > > > > >>> > common > > > > >>> > >> > > >> > dependencies > > > > >>> > >> > > >> > > >> >> between > > > > >>> > >> > > >> > > >> >> > > of > > > > >>> > >> > > >> > > >> >> > > > > apex > > > > >>> > >> > > >> > > >> >> > > > > > > and DT Community edition: > > > > >>> > >> > > >> > > >> >> > > > > > > netlet-1.2.0.jar > > > > >>> > >> > > >> > > >> >> > > > > > > kryo-2.24.0.jar > > > > >>> > >> > > >> > > >> >> > > > > > > jackson-core-asl-1.9.2.jar > > > > >>> > >> > > >> > > >> >> > > > > > > jackson-mapper-asl-1.9.2.jar > > > > >>> > >> > > >> > > >> >> > > > > > > async-http-client-1.7.20.jar > > > > >>> > >> > > >> > > >> >> > > > > > > netty-3.6.6.Final.jar > > > > >>> > >> > > >> > > >> >> > > > > > > validation-api-1.1.0.Final.jar > > > > >>> > >> > > >> > > >> >> > > > > > > bval-jsr303-0.5.jar > > > > >>> > >> > > >> > > >> >> > > > > > > bval-core-0.5.jar > > > > >>> > >> > > >> > > >> >> > > > > > > commons-lang3-3.1.jar > > > > >>> > >> > > >> > > >> >> > > > > > > commons-beanutils-1.8.3.jar > > > > >>> > >> > > >> > > >> >> > > > > > > httpclient-4.3.5.jar > > > > >>> > >> > > >> > > >> >> > > > > > > commons-codec-1.10.jar > > > > >>> > >> > > >> > > >> >> > > > > > > zip4j-1.3.2.jar > > > > >>> > >> > > >> > > >> >> > > > > > > > > > jetty-websocket-8.1.10.v20130312.jar > > > > >>> > >> > > >> > > >> >> > > > > > > xbean-asm5-shaded-4.3.jar > > > > >>> > >> > > >> > > >> >> > > > > > > jersey-apache-client4-1.9.jar > > > > >>> > >> > > >> > > >> >> > > > > > > jline-2.11.jar > > > > >>> > >> > > >> > > >> >> > > > > > > ant-1.9.2.jar > > > > >>> > >> > > >> > > >> >> > > > > > > ant-launcher-1.9.2.jar > > > > >>> > >> > > >> > > >> >> > > > > > > mbassador-1.1.9.jar > > > > >>> > >> > > >> > > >> >> > > > > > > jackson-jaxrs-1.9.2.jar > > > > >>> > >> > > >> > > >> >> > > > > > > jackson-xc-1.9.2.jar > > > > >>> > >> > > >> > > >> >> > > > > > > hadoop-common-2.2.0-tests.jar > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > Ofcourse, I'll be running some > > > tests > > > > do > > > > >>> > check > > > > >>> > >> > that > > > > >>> > >> > > >> > dtcli > > > > >>> > >> > > >> > > >> works > > > > >>> > >> > > >> > > >> >> > > > properly > > > > >>> > >> > > >> > > >> >> > > > > > for > > > > >>> > >> > > >> > > >> >> > > > > > > launch+shutdown+kill of apps > with > > > > only > > > > >>> > these > > > > >>> > >> > > >> libraries > > > > >>> > >> > > >> > > >> >> present in > > > > >>> > >> > > >> > > >> >> > > > > > isolation > > > > >>> > >> > > >> > > >> >> > > > > > > without dependency on local m2. > > > > >>> > >> > > >> > > >> >> > > > > > > I'm believe that there are > > unwanted > > > > >>> jars > > > > >>> > >> which > > > > >>> > >> > are > > > > >>> > >> > > >> used > > > > >>> > >> > > >> > > for > > > > >>> > >> > > >> > > >> >> > compile > > > > >>> > >> > > >> > > >> >> > > > > time > > > > >>> > >> > > >> > > >> >> > > > > > > dependency and not runtime in > > above > > > > >>> list > > > > >>> > >> which I > > > > >>> > >> > > can > > > > >>> > >> > > >> > drop > > > > >>> > >> > > >> > > >> to > > > > >>> > >> > > >> > > >> >> keep > > > > >>> > >> > > >> > > >> >> > > > > package > > > > >>> > >> > > >> > > >> >> > > > > > > size minimal. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > @Anniruddha, > > > > >>> > >> > > >> > > >> >> > > > > > > 1. dt-env.sh -> I'm not sure if > > all > > > > of > > > > >>> the > > > > >>> > >> > > >> dt-env.sh is > > > > >>> > >> > > >> > > >> >> required. > > > > >>> > >> > > >> > > >> >> > > > Only > > > > >>> > >> > > >> > > >> >> > > > > > > required values I see in it are > > > > >>> classpath. > > > > >>> > >> and > > > > >>> > >> > > >> dtcli is > > > > >>> > >> > > >> > > >> >> capable > > > > >>> > >> > > >> > > >> >> > of > > > > >>> > >> > > >> > > >> >> > > > > > building > > > > >>> > >> > > >> > > >> >> > > > > > > it on the fly. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > 2. dt-site.xml -> Yes, we'll > have > > > to > > > > >>> see > > > > >>> > how > > > > >>> > >> > dtcli > > > > >>> > >> > > >> can > > > > >>> > >> > > >> > > find > > > > >>> > >> > > >> > > >> >> the > > > > >>> > >> > > >> > > >> >> > jar > > > > >>> > >> > > >> > > >> >> > > > > path, > > > > >>> > >> > > >> > > >> >> > > > > > > dt-env.sh and all such conf > > files. > > > If > > > > >>> we > > > > >>> > >> need a > > > > >>> > >> > > >> change > > > > >>> > >> > > >> > in > > > > >>> > >> > > >> > > >> >> dtcli, > > > > >>> > >> > > >> > > >> >> > > then > > > > >>> > >> > > >> > > >> >> > > > > > > community's opinion is required > > for > > > > >>> whether > > > > >>> > >> > dtcli > > > > >>> > >> > > >> > should > > > > >>> > >> > > >> > > >> >> change > > > > >>> > >> > > >> > > >> >> > in > > > > >>> > >> > > >> > > >> >> > > > our > > > > >>> > >> > > >> > > >> >> > > > > > repo > > > > >>> > >> > > >> > > >> >> > > > > > > or a copy of that with required > > > > changes > > > > >>> > >> exist in > > > > >>> > >> > > >> bigtop > > > > >>> > >> > > >> > > >> repo > > > > >>> > >> > > >> > > >> >> > until > > > > >>> > >> > > >> > > >> >> > > we > > > > >>> > >> > > >> > > >> >> > > > > > make > > > > >>> > >> > > >> > > >> >> > > > > > > the dtcli generic enough. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > Can we have these file paths set > > > using > > > > >>> > >> environment > > > > >>> > >> > > >> > > variables > > > > >>> > >> > > >> > > >> >> with > > > > >>> > >> > > >> > > >> >> > > some > > > > >>> > >> > > >> > > >> >> > > > > > default values in dtcli? > > > > >>> > >> > > >> > > >> >> > > > > > Then we can set such params in > > > > dt-env.sh. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > 3. dt-sited.xml -> How is this > > file > > > > >>> > different > > > > >>> > >> > from > > > > >>> > >> > > >> > > >> >> dt-site.xml? > > > > >>> > >> > > >> > > >> >> > Not > > > > >>> > >> > > >> > > >> >> > > > > sure > > > > >>> > >> > > >> > > >> >> > > > > > if > > > > >>> > >> > > >> > > >> >> > > > > > > adding in /etc/skel would help. > > > > Files & > > > > >>> > Dirs > > > > >>> > >> > from > > > > >>> > >> > > >> > > /etc/skel > > > > >>> > >> > > >> > > >> >> are > > > > >>> > >> > > >> > > >> >> > > > copied > > > > >>> > >> > > >> > > >> >> > > > > to > > > > >>> > >> > > >> > > >> >> > > > > > > home of new user when useradd > > > program > > > > >>> is > > > > >>> > >> called. > > > > >>> > >> > > But > > > > >>> > >> > > >> > for > > > > >>> > >> > > >> > > >> >> existing > > > > >>> > >> > > >> > > >> >> > > > users > > > > >>> > >> > > >> > > >> >> > > > > > > this won't be of any use. > > Moreover, > > > > >>> dtcli > > > > >>> > >> > creates > > > > >>> > >> > > >> > ~/.dt/ > > > > >>> > >> > > >> > > on > > > > >>> > >> > > >> > > >> >> the > > > > >>> > >> > > >> > > >> >> > fly > > > > >>> > >> > > >> > > >> >> > > > if > > > > >>> > >> > > >> > > >> >> > > > > > not > > > > >>> > >> > > >> > > >> >> > > > > > > encountered for the first time. > > > Again > > > > >>> > >> correct me > > > > >>> > >> > > if > > > > >>> > >> > > >> I'm > > > > >>> > >> > > >> > > >> wrong. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > Yes, you are right. However this > > > would > > > > >>> > >> probably be > > > > >>> > >> > > >> > > necessary > > > > >>> > >> > > >> > > >> >> step > > > > >>> > >> > > >> > > >> >> > for > > > > >>> > >> > > >> > > >> >> > > > > > rpm/deb, as they may also get > > > installed > > > > >>> > during > > > > >>> > >> OS > > > > >>> > >> > > >> install > > > > >>> > >> > > >> > > and > > > > >>> > >> > > >> > > >> >> > before > > > > >>> > >> > > >> > > >> >> > > > user > > > > >>> > >> > > >> > > >> >> > > > > > accounts were created. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > 4. Changing bashrc & > bash_profile > > > -> > > > > >>> I'm > > > > >>> > not > > > > >>> > >> > sure > > > > >>> > >> > > >> its > > > > >>> > >> > > >> > the > > > > >>> > >> > > >> > > >> best > > > > >>> > >> > > >> > > >> >> > > idea. > > > > >>> > >> > > >> > > >> >> > > > > > We'll > > > > >>> > >> > > >> > > >> >> > > > > > > anyway put dtcli in location > > which > > > is > > > > >>> by > > > > >>> > >> default > > > > >>> > >> > > >> > present > > > > >>> > >> > > >> > > in > > > > >>> > >> > > >> > > >> >> path > > > > >>> > >> > > >> > > >> >> > > i.e. > > > > >>> > >> > > >> > > >> >> > > > > > > /usr/bin. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > For other env variables specific > to > > > > apex, > > > > >>> > I'll > > > > >>> > >> > > prefer > > > > >>> > >> > > >> to > > > > >>> > >> > > >> > > use > > > > >>> > >> > > >> > > >> >> > > dt-env.sh > > > > >>> > >> > > >> > > >> >> > > > > and > > > > >>> > >> > > >> > > >> >> > > > > > > source it in dtcli rather than > > > > changing > > > > >>> > >> bashrc > > > > >>> > >> > > >> etc... > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > Instead, coping dt-env.sh to > > > > >>> /etc/profile.d/ > > > > >>> > >> shall > > > > >>> > >> > > be > > > > >>> > >> > > >> the > > > > >>> > >> > > >> > > >> >> cleaner > > > > >>> > >> > > >> > > >> >> > way > > > > >>> > >> > > >> > > >> >> > > > to > > > > >>> > >> > > >> > > >> >> > > > > > achieve the same. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > Thanks, > > > > >>> > >> > > >> > > >> >> > > > > > > Chinmay. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > On Wed, Feb 10, 2016 at 1:07 > PM, > > > > >>> Aniruddha > > > > >>> > >> > > Thombare > > > > >>> > >> > > >> < > > > > >>> > >> > > >> > > >> >> > > > > > > [email protected]> > > wrote: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > Hi, > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > @Chinmay, > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > We need to consider > following: > > > > >>> > >> > > >> > > >> >> > > > > > > > System wide default config > > files > > > > can > > > > >>> be > > > > >>> > >> > located > > > > >>> > >> > > at > > > > >>> > >> > > >> > > >> following > > > > >>> > >> > > >> > > >> >> > > > > locations: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > /etc/apex/dt-env.sh (we may > > have > > > > to > > > > >>> > change > > > > >>> > >> > > dtcli > > > > >>> > >> > > >> > > >> behaviour > > > > >>> > >> > > >> > > >> >> on > > > > >>> > >> > > >> > > >> >> > > how > > > > >>> > >> > > >> > > >> >> > > > it > > > > >>> > >> > > >> > > >> >> > > > > > > finds > > > > >>> > >> > > >> > > >> >> > > > > > > > those locations) > > > > >>> > >> > > >> > > >> >> > > > > > > > /etc/apex/dt-site.xml (we may > > > have > > > > to > > > > >>> > >> change > > > > >>> > >> > > dtcli > > > > >>> > >> > > >> > > >> >> behaviour on > > > > >>> > >> > > >> > > >> >> > > how > > > > >>> > >> > > >> > > >> >> > > > > it > > > > >>> > >> > > >> > > >> >> > > > > > > > finds those locations) > > > > >>> > >> > > >> > > >> >> > > > > > > > /etc/skel/.dt/dt-sited.xml > and > > > > other > > > > >>> > files > > > > >>> > >> > (for > > > > >>> > >> > > >> new > > > > >>> > >> > > >> > > users > > > > >>> > >> > > >> > > >> >> that > > > > >>> > >> > > >> > > >> >> > > will > > > > >>> > >> > > >> > > >> >> > > > > be > > > > >>> > >> > > >> > > >> >> > > > > > > > created in system in future.) > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > We may also have to modify > > > bashrc / > > > > >>> > >> > bashprofile > > > > >>> > >> > > >> for > > > > >>> > >> > > >> > > >> >> population > > > > >>> > >> > > >> > > >> >> > > any > > > > >>> > >> > > >> > > >> >> > > > > > > > variables that are required. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > @dev, please put in your > > > comments / > > > > >>> > >> > suggestion. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > Thanks, > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > Aniruddha > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > On Wed, Feb 10, 2016 at 1:16 > > AM, > > > > >>> Thomas > > > > >>> > >> Weise > > > > >>> > >> > < > > > > >>> > >> > > >> > > >> >> > > > > [email protected]> > > > > >>> > >> > > >> > > >> >> > > > > > > > wrote: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > --> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > On Tue, Feb 9, 2016 at 9:05 > > AM, > > > > >>> Chinmay > > > > >>> > >> > > >> Kolhatkar < > > > > >>> > >> > > >> > > >> >> > > > > > > > [email protected] > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > wrote: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Hello Everyone!! > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Continuing with packaging > > > > effort > > > > >>> > >> (rpm+deb) > > > > >>> > >> > > of > > > > >>> > >> > > >> > apex, > > > > >>> > >> > > >> > > >> here > > > > >>> > >> > > >> > > >> >> > are > > > > >>> > >> > > >> > > >> >> > > > some > > > > >>> > >> > > >> > > >> >> > > > > > > > > proposals > > > > >>> > >> > > >> > > >> >> > > > > > > > > > about package structure > > etc.. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Before posting it on > > bbigtop > > > > >>> mailing > > > > >>> > >> > list, I > > > > >>> > >> > > >> have > > > > >>> > >> > > >> > > >> some > > > > >>> > >> > > >> > > >> >> > > question > > > > >>> > >> > > >> > > >> >> > > > > for > > > > >>> > >> > > >> > > >> >> > > > > > > > apex > > > > >>> > >> > > >> > > >> >> > > > > > > > > > community. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Proposed Directory > > structure > > > of > > > > >>> apex > > > > >>> > >> > package > > > > >>> > >> > > >> for > > > > >>> > >> > > >> > > both > > > > >>> > >> > > >> > > >> >> deb & > > > > >>> > >> > > >> > > >> >> > > > rpm: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > /usr/lib/apex/bin/dtcli > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> /usr/lib/apex/lib/apex-api-<version>.jar > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > /usr/lib/apex/lib/apex-engine-<version>.jar > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> /usr/lib/apex/lib/apex-bufferserver-<version>.jar > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > /usr/lib/apex/lib/apex-common-<version>.jar > > > > >>> > >> > > >> > > >> >> > > > > > > > > > /usr/lib/apex/lib/<other > > > > >>> dependent > > > > >>> > >> jars> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > /usr/bin/dtcli -> > > > > >>> > /usr/lib/apex/bin/dtcli > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> /usr/share/doc/man/man1/dtcli.1.gz > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > /usr/share/doc/apex/license/LICENSE.txt > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> /usr/share/doc/apex/license/<package>-LICENSE.txt > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > /usr/share/doc/apex/CHANGELOG > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > /usr/share/doc/apex/NOTICE > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > <version> = > > 3.3.0-incubating. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > <other dependent jars> = > > All > > > > the > > > > >>> 3rd > > > > >>> > >> party > > > > >>> > >> > > >> jars > > > > >>> > >> > > >> > > which > > > > >>> > >> > > >> > > >> >> are > > > > >>> > >> > > >> > > >> >> > > > > required > > > > >>> > >> > > >> > > >> >> > > > > > > for > > > > >>> > >> > > >> > > >> >> > > > > > > > > apex > > > > >>> > >> > > >> > > >> >> > > > > > > > > > to run. Usually the > > > > dependencies > > > > >>> are > > > > >>> > >> > > packaged > > > > >>> > >> > > >> as > > > > >>> > >> > > >> > > >> part of > > > > >>> > >> > > >> > > >> >> > > > rpm/deb > > > > >>> > >> > > >> > > >> >> > > > > by > > > > >>> > >> > > >> > > >> >> > > > > > > any > > > > >>> > >> > > >> > > >> >> > > > > > > > > > software in bigtop. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > Can you specify what those > > jars > > > > >>> are in > > > > >>> > >> > Bigtop > > > > >>> > >> > > >> > > context? > > > > >>> > >> > > >> > > >> >> Same > > > > >>> > >> > > >> > > >> >> > as > > > > >>> > >> > > >> > > >> >> > > > > > shipped > > > > >>> > >> > > >> > > >> >> > > > > > > > with > > > > >>> > >> > > >> > > >> >> > > > > > > > > DT community addition under > > > lib/ > > > > ? > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > <package LICENSE> = > > Licenses > > > of > > > > >>> > >> > > corresponding > > > > >>> > >> > > >> of > > > > >>> > >> > > >> > > 3rd > > > > >>> > >> > > >> > > >> >> party > > > > >>> > >> > > >> > > >> >> > > jars > > > > >>> > >> > > >> > > >> >> > > > > > which > > > > >>> > >> > > >> > > >> >> > > > > > > > > needs > > > > >>> > >> > > >> > > >> >> > > > > > > > > > to included while > > packaging. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Questions related to > this: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 1. Should we call the cli > > of > > > > >>> apex as > > > > >>> > >> > "apex" > > > > >>> > >> > > >> > instead > > > > >>> > >> > > >> > > >> of > > > > >>> > >> > > >> > > >> >> > > "dtcli" > > > > >>> > >> > > >> > > >> >> > > > in > > > > >>> > >> > > >> > > >> >> > > > > > > > bigtop > > > > >>> > >> > > >> > > >> >> > > > > > > > > > package? > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > I think we should keep the > > name > > > > >>> until > > > > >>> > we > > > > >>> > >> are > > > > >>> > >> > > >> able > > > > >>> > >> > > >> > to > > > > >>> > >> > > >> > > >> >> change > > > > >>> > >> > > >> > > >> >> > it > > > > >>> > >> > > >> > > >> >> > > in > > > > >>> > >> > > >> > > >> >> > > > > > Apex. > > > > >>> > >> > > >> > > >> >> > > > > > > > > Otherwise this may get > > > confusing. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 2. I see that all > softwares > > > in > > > > >>> bigtop > > > > >>> > >> have > > > > >>> > >> > > man > > > > >>> > >> > > >> > page > > > > >>> > >> > > >> > > >> for > > > > >>> > >> > > >> > > >> >> > their > > > > >>> > >> > > >> > > >> >> > > > > > > > > executables. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > I think we should have it > > too > > > > for > > > > >>> > >> dtcli. > > > > >>> > >> > Is > > > > >>> > >> > > >> there > > > > >>> > >> > > >> > > any > > > > >>> > >> > > >> > > >> >> > > > > documentation > > > > >>> > >> > > >> > > >> >> > > > > > > > > which I > > > > >>> > >> > > >> > > >> >> > > > > > > > > > can convert to man page? > or > > > can > > > > >>> I use > > > > >>> > >> > output > > > > >>> > >> > > >> of > > > > >>> > >> > > >> > > >> "dtcli > > > > >>> > >> > > >> > > >> >> > > --help"? > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 3. Do we want to call > > version > > > > of > > > > >>> apex > > > > >>> > >> in > > > > >>> > >> > > >> Bigtop > > > > >>> > >> > > >> > as > > > > >>> > >> > > >> > > >> >> 3.3.0 OR > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 3.3.0-incubating? > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > Has to be -incubating > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 4. Is it ok for apex > > package > > > of > > > > >>> > bigtop > > > > >>> > >> to > > > > >>> > >> > > >> depend > > > > >>> > >> > > >> > on > > > > >>> > >> > > >> > > >> >> 2.7.1 > > > > >>> > >> > > >> > > >> >> > > > > version > > > > >>> > >> > > >> > > >> >> > > > > > of > > > > >>> > >> > > >> > > >> >> > > > > > > > > > bigtop hadoop? Any > problems > > > > that > > > > >>> we > > > > >>> > see > > > > >>> > >> > with > > > > >>> > >> > > >> this > > > > >>> > >> > > >> > > >> >> > dependency? > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > Should work without > changed. > > I > > > > >>> thought > > > > >>> > we > > > > >>> > >> > > >> certified > > > > >>> > >> > > >> > > >> >> against > > > > >>> > >> > > >> > > >> >> > > > 2.7.0? > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > 5. Following is the > apache > > > > mirror > > > > >>> > from > > > > >>> > >> > which > > > > >>> > >> > > >> > bigtop > > > > >>> > >> > > >> > > >> will > > > > >>> > >> > > >> > > >> >> > pick > > > > >>> > >> > > >> > > >> >> > > > the > > > > >>> > >> > > >> > > >> >> > > > > > > apex > > > > >>> > >> > > >> > > >> >> > > > > > > > > > source for compilation > and > > > > >>> packaging. > > > > >>> > >> > Please > > > > >>> > >> > > >> > > correct > > > > >>> > >> > > >> > > >> if > > > > >>> > >> > > >> > > >> >> > > > > incorrect: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > >>> > >> > > > > http://apache.osuosl.org/incubator/apex/v3.3.0-incubating/ > > > > >>> > >> > > >> > > >> >> > > > > > > > > > NOTE: > > apache.osuosl.org > > > is > > > > >>> the > > > > >>> > >> mirror > > > > >>> > >> > > >> used > > > > >>> > >> > > >> > by > > > > >>> > >> > > >> > > >> all > > > > >>> > >> > > >> > > >> >> the > > > > >>> > >> > > >> > > >> >> > > > > > softwares > > > > >>> > >> > > >> > > >> >> > > > > > > in > > > > >>> > >> > > >> > > >> >> > > > > > > > > > bigtop. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Please share your > opinion. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Thanks, > > > > >>> > >> > > >> > > >> >> > > > > > > > > > Chinmay. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > On Mon, Feb 8, 2016 at > > 11:19 > > > > AM, > > > > >>> > >> Chinmay > > > > >>> > >> > > >> > Kolhatkar > > > > >>> > >> > > >> > > < > > > > >>> > >> > > >> > > >> >> > > > > > > > > > [email protected]> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > wrote: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > Yes.. Starting to work > on > > > the > > > > >>> > >> packaging. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > I've already started > > > > >>> discussion on > > > > >>> > >> > bigtop > > > > >>> > >> > > >> dev > > > > >>> > >> > > >> > > >> mailing > > > > >>> > >> > > >> > > >> >> > list > > > > >>> > >> > > >> > > >> >> > > > for > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > integration. Also > > created a > > > > >>> Jira > > > > >>> > for > > > > >>> > >> the > > > > >>> > >> > > >> same. > > > > >>> > >> > > >> > > For > > > > >>> > >> > > >> > > >> >> this > > > > >>> > >> > > >> > > >> >> > > > > > communities > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > reference, here is the > > > bigtop > > > > >>> Jira: > > > > >>> > >> > > >> > BIGTOP-2313. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > On Mon, Feb 8, 2016 at > > 1:13 > > > > AM, > > > > >>> > >> Thomas > > > > >>> > >> > > >> Weise < > > > > >>> > >> > > >> > > >> >> > > > > > > [email protected] > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > wrote: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Chinmay, > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> I don't see anything > > under > > > > >>> > >> > prerequisites > > > > >>> > >> > > >> that > > > > >>> > >> > > >> > > >> would > > > > >>> > >> > > >> > > >> >> be a > > > > >>> > >> > > >> > > >> >> > > > > > problem. > > > > >>> > >> > > >> > > >> >> > > > > > > We > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> looked > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> at the ASF licencing > > > > >>> compatibility > > > > >>> > >> as > > > > >>> > >> > > part > > > > >>> > >> > > >> of > > > > >>> > >> > > >> > > >> >> becoming > > > > >>> > >> > > >> > > >> >> > an > > > > >>> > >> > > >> > > >> >> > > > > > > incubator > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> project. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Please focus on the > > > > packaging > > > > >>> > during > > > > >>> > >> > the > > > > >>> > >> > > >> next > > > > >>> > >> > > >> > > >> weeks. > > > > >>> > >> > > >> > > >> >> > Since > > > > >>> > >> > > >> > > >> >> > > > the > > > > >>> > >> > > >> > > >> >> > > > > > > work > > > > >>> > >> > > >> > > >> >> > > > > > > > > will > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> be > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> part of Bigtop, > related > > > > >>> > discussions > > > > >>> > >> and > > > > >>> > >> > > >> JIRAs > > > > >>> > >> > > >> > > >> should > > > > >>> > >> > > >> > > >> >> > also > > > > >>> > >> > > >> > > >> >> > > be > > > > >>> > >> > > >> > > >> >> > > > > > > there. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Would be good to have > > the > > > > >>> > packaging > > > > >>> > >> in > > > > >>> > >> > > >> place > > > > >>> > >> > > >> > by > > > > >>> > >> > > >> > > >> end > > > > >>> > >> > > >> > > >> >> Feb. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Thanks, > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Thomas > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> On Thu, Feb 4, 2016 at > > > 10:14 > > > > >>> AM, > > > > >>> > >> > Chinmay > > > > >>> > >> > > >> > > >> Kolhatkar < > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > [email protected] > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> wrote: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Hi All, > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > We're planning a > work > > on > > > > >>> adding > > > > >>> > >> > Apache > > > > >>> > >> > > >> Apex > > > > >>> > >> > > >> > > as a > > > > >>> > >> > > >> > > >> >> > > component > > > > >>> > >> > > >> > > >> >> > > > > to > > > > >>> > >> > > >> > > >> >> > > > > > > > Apache > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Bigtop. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Bigtop is the > > packaging > > > > >>> system > > > > >>> > for > > > > >>> > >> > the > > > > >>> > >> > > >> > Apache > > > > >>> > >> > > >> > > >> big > > > > >>> > >> > > >> > > >> >> data > > > > >>> > >> > > >> > > >> >> > > > > > > ecosystem. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> Several > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Hadoop distros use > it, > > > > most > > > > >>> > >> recently > > > > >>> > >> > > EMR. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Here is the tracking > > > Jira > > > > >>> task > > > > >>> > in > > > > >>> > >> > > >> APEXCORE > > > > >>> > >> > > >> > for > > > > >>> > >> > > >> > > >> the > > > > >>> > >> > > >> > > >> >> > same: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> > > > https://issues.apache.org/jira/browse/APEXCORE-331 > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Proposed plan of > > > execution > > > > >>> is as > > > > >>> > >> > > follows: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > *Step 1) Handle > > > > >>> prerequisites* > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Apache bigtop has > some > > > > hard > > > > >>> and > > > > >>> > >> soft > > > > >>> > >> > > >> > > expectation > > > > >>> > >> > > >> > > >> >> for > > > > >>> > >> > > >> > > >> >> > new > > > > >>> > >> > > >> > > >> >> > > > > > > > components > > > > >>> > >> > > >> > > >> >> > > > > > > > > to > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> get > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > integrated into > > Bigtop. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Here is the list of > > it: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >>> > >> > > >> > > >> >> > > > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > > > > >>> > >> > > >> > > >> > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > > > > >>> > >> > > >> > > > > >>> > >> > > > > > > >>> > >> > > > > > >>> > >> > > > > >>> > > > > > >>> > > > > > > > > > > https://cwiki.apache.org/confluence/display/BIGTOP/Requirement+for+adding+a+new+component+to+Bigtop+distribution > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Most of them seems > to > > be > > > > >>> > standard > > > > >>> > >> ASF > > > > >>> > >> > > >> based > > > > >>> > >> > > >> > > >> >> > > requirements, > > > > >>> > >> > > >> > > >> >> > > > > but > > > > >>> > >> > > >> > > >> >> > > > > > > few > > > > >>> > >> > > >> > > >> >> > > > > > > > > need > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> to > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > be checked for: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 1. Software projects > > are > > > > >>> > expected > > > > >>> > >> to > > > > >>> > >> > be > > > > >>> > >> > > >> > > Licensed > > > > >>> > >> > > >> > > >> >> under > > > > >>> > >> > > >> > > >> >> > > > > Apache > > > > >>> > >> > > >> > > >> >> > > > > > > > > License, > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Version 2.0 (and > their > > > > >>> > >> dependencies > > > > >>> > >> > are > > > > >>> > >> > > >> > > expected > > > > >>> > >> > > >> > > >> >> to be > > > > >>> > >> > > >> > > >> >> > > > > > > compatible > > > > >>> > >> > > >> > > >> >> > > > > > > > > with > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> this > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > license) > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > - Apex is under > > ASL > > > > 2.0 > > > > >>> but > > > > >>> > >> need > > > > >>> > >> > to > > > > >>> > >> > > >> > check > > > > >>> > >> > > >> > > if > > > > >>> > >> > > >> > > >> >> > > > > dependencies > > > > >>> > >> > > >> > > >> >> > > > > > of > > > > >>> > >> > > >> > > >> >> > > > > > > > > Apex > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> are > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > compatible with ASL > > 2.0. > > > > >>> This I > > > > >>> > >> guess > > > > >>> > >> > > >> would > > > > >>> > >> > > >> > > be a > > > > >>> > >> > > >> > > >> >> > > > > verification > > > > >>> > >> > > >> > > >> >> > > > > > > > check. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 2. Software projects > > are > > > > >>> > expected > > > > >>> > >> to > > > > >>> > >> > be > > > > >>> > >> > > >> > > >> compatible > > > > >>> > >> > > >> > > >> >> > with > > > > >>> > >> > > >> > > >> >> > > > all > > > > >>> > >> > > >> > > >> >> > > > > of > > > > >>> > >> > > >> > > >> >> > > > > > > the > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > supported platforms > > that > > > > >>> Bigtop > > > > >>> > >> > > >> distribution > > > > >>> > >> > > >> > > is > > > > >>> > >> > > >> > > >> >> > > targeting > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > - This needs > > > verifying > > > > >>> > whether > > > > >>> > >> > our > > > > >>> > >> > > >> > > software > > > > >>> > >> > > >> > > >> >> runs > > > > >>> > >> > > >> > > >> >> > > fine > > > > >>> > >> > > >> > > >> >> > > > in > > > > >>> > >> > > >> > > >> >> > > > > > > > > centos-6 > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > centos-7 fedora-20 > > > > >>> ubuntu-14.04 > > > > >>> > >> > > debian-8 > > > > >>> > >> > > >> > > >> >> > opensuse-13.2. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 3. What smoke tests > > that > > > > >>> should > > > > >>> > be > > > > >>> > >> > > added > > > > >>> > >> > > >> for > > > > >>> > >> > > >> > > >> >> > deployment. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 4. Identifying the > > test > > > > >>> > artifacts > > > > >>> > >> > which > > > > >>> > >> > > >> goes > > > > >>> > >> > > >> > > >> beyond > > > > >>> > >> > > >> > > >> >> > > smoke > > > > >>> > >> > > >> > > >> >> > > > > test > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > - These are > > > basically > > > > >>> the > > > > >>> > >> > > integration > > > > >>> > >> > > >> > > tests > > > > >>> > >> > > >> > > >> for > > > > >>> > >> > > >> > > >> >> > > > > > verification > > > > >>> > >> > > >> > > >> >> > > > > > > > > after > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> the > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > deployment. This is > a > > > soft > > > > >>> > >> > requirement, > > > > >>> > >> > > >> but > > > > >>> > >> > > >> > > aim > > > > >>> > >> > > >> > > >> is > > > > >>> > >> > > >> > > >> >> to > > > > >>> > >> > > >> > > >> >> > > > > achieve > > > > >>> > >> > > >> > > >> >> > > > > > > this > > > > >>> > >> > > >> > > >> >> > > > > > > > > as > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> well > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > or at least have > > > > >>> explanation why > > > > >>> > >> not > > > > >>> > >> > to > > > > >>> > >> > > >> > > include. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > If there are any > from > > > the > > > > >>> link > > > > >>> > >> which > > > > >>> > >> > > >> > > explicitly > > > > >>> > >> > > >> > > >> >> needs > > > > >>> > >> > > >> > > >> >> > to > > > > >>> > >> > > >> > > >> >> > > > be > > > > >>> > >> > > >> > > >> >> > > > > > > > checked > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> other > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > than above 4, please > > let > > > > us > > > > >>> > know. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > *Step 2) Adding Apex > > as > > > > >>> > component > > > > >>> > >> to > > > > >>> > >> > > >> Bigtop* > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > From one of the mail > > > > >>> archive of > > > > >>> > >> > Bigtop, > > > > >>> > >> > > >> it > > > > >>> > >> > > >> > was > > > > >>> > >> > > >> > > >> >> learnt > > > > >>> > >> > > >> > > >> >> > > that > > > > >>> > >> > > >> > > >> >> > > > > the > > > > >>> > >> > > >> > > >> >> > > > > > > > > bigtop > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > community want to > see > > > the > > > > >>> > >> addition of > > > > >>> > >> > > new > > > > >>> > >> > > >> > > >> >> components > > > > >>> > >> > > >> > > >> >> > in > > > > >>> > >> > > >> > > >> >> > > > > > phases. > > > > >>> > >> > > >> > > >> >> > > > > > > > Here > > > > >>> > >> > > >> > > >> >> > > > > > > > > > are > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > the phases: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 1. Packaging > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > - This needs > > > creating > > > > of > > > > >>> > >> package > > > > >>> > >> > > i.e. > > > > >>> > >> > > >> > rpm > > > > >>> > >> > > >> > > & > > > > >>> > >> > > >> > > >> deb > > > > >>> > >> > > >> > > >> >> > > files. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > - > > > > >>> documentations/READMEs, > > > > >>> > >> > LICENSE, > > > > >>> > >> > > >> > > >> DISCLAMER, > > > > >>> > >> > > >> > > >> >> > NOTES > > > > >>> > >> > > >> > > >> >> > > > etc > > > > >>> > >> > > >> > > >> >> > > > > if > > > > >>> > >> > > >> > > >> >> > > > > > > any > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> needed. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > - Any > > documentation > > > > that > > > > >>> > need > > > > >>> > >> to > > > > >>> > >> > be > > > > >>> > >> > > >> > added > > > > >>> > >> > > >> > > to > > > > >>> > >> > > >> > > >> >> > > > > distribution > > > > >>> > >> > > >> > > >> >> > > > > > of > > > > >>> > >> > > >> > > >> >> > > > > > > > our > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > software. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > - Any license > > > > >>> information of > > > > >>> > >> > > >> > dependencies > > > > >>> > >> > > >> > > >> >> required > > > > >>> > >> > > >> > > >> >> > > to > > > > >>> > >> > > >> > > >> >> > > > be > > > > >>> > >> > > >> > > >> >> > > > > > > added > > > > >>> > >> > > >> > > >> >> > > > > > > > > to > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > package > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 2. Smoke tests (at > > very > > > > >>> least) > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > - Adding smoke > > test > > > > for > > > > >>> > >> > packaging. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 3. Puppet recipes > for > > > > >>> automatic > > > > >>> > >> > > >> deployment > > > > >>> > >> > > >> > and > > > > >>> > >> > > >> > > >> >> > > > configuration > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > - Add puppet > > recipes > > > > for > > > > >>> > >> > automatic > > > > >>> > >> > > >> > > >> deployment > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 4. Integration tests > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > - For > verification > > > of > > > > >>> > >> > deployments. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > 5. license > clearance: > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Run 'gradle rat' > > to > > > > make > > > > >>> > sure > > > > >>> > >> all > > > > >>> > >> > > new > > > > >>> > >> > > >> > > stuff > > > > >>> > >> > > >> > > >> is > > > > >>> > >> > > >> > > >> >> > > > compliant > > > > >>> > >> > > >> > > >> >> > > > > > > with > > > > >>> > >> > > >> > > >> >> > > > > > > > > ASF > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > license > requirements. > > If > > > > >>> you add > > > > >>> > >> code > > > > >>> > >> > > >> > licenses > > > > >>> > >> > > >> > > >> >> under > > > > >>> > >> > > >> > > >> >> > > > > different > > > > >>> > >> > > >> > > >> >> > > > > > > > > > licenses, > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > those would need to > be > > > > >>> listed in > > > > >>> > >> the > > > > >>> > >> > > >> NOTICE. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Please share your > > > thoughts > > > > >>> on > > > > >>> > the > > > > >>> > >> > > >> approach. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > We'll start > > > corresponding > > > > >>> > >> > communication > > > > >>> > >> > > >> on > > > > >>> > >> > > >> > > >> bigtop > > > > >>> > >> > > >> > > >> >> > > mailing > > > > >>> > >> > > >> > > >> >> > > > > list > > > > >>> > >> > > >> > > >> >> > > > > > > as > > > > >>> > >> > > >> > > >> >> > > > > > > > > > well. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > We have some > specific > > > > >>> > >> > > >> questions/suggestions > > > > >>> > >> > > >> > > >> >> related to > > > > >>> > >> > > >> > > >> >> > > > what > > > > >>> > >> > > >> > > >> >> > > > > > > should > > > > >>> > >> > > >> > > >> >> > > > > > > > > be > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> the > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > content of the > package > > > and > > > > >>> what > > > > >>> > >> > should > > > > >>> > >> > > be > > > > >>> > >> > > >> > the > > > > >>> > >> > > >> > > >> smoke > > > > >>> > >> > > >> > > >> >> > > tests, > > > > >>> > >> > > >> > > >> >> > > > > but > > > > >>> > >> > > >> > > >> >> > > > > > > in > > > > >>> > >> > > >> > > >> >> > > > > > > > > the > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > interest of not > having > > > too > > > > >>> much > > > > >>> > >> > content > > > > >>> > >> > > >> > here, > > > > >>> > >> > > >> > > >> we'll > > > > >>> > >> > > >> > > >> >> > put > > > > >>> > >> > > >> > > >> >> > > > the > > > > >>> > >> > > >> > > >> >> > > > > > > > > questions > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> as a > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > separate mail in > this > > > > >>> > mailthread. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Thanks, > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > Chinmay. > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >> > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > > >>> > >> > > >> > > >> >> > > > > > > > >>> > >> > > >> > > >> >> > > > > > > >>> > >> > > >> > > >> >> > > > > > >>> > >> > > >> > > >> >> > > > > >>> > >> > > >> > > >> > > > > > >>> > >> > > >> > > >> > > > > > >>> > >> > > >> > > >> > > > > >>> > >> > > >> > > > > > > > >>> > >> > > >> > > > > > > > >>> > >> > > >> > > > > > > >>> > >> > > >> > > > > > >>> > >> > > >> > > > > >>> > >> > > > > > > > >>> > >> > > > > > > > >>> > >> > > > > > > >>> > >> > > > > > >>> > >> > > > > >>> > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >> > > > > >> > > > > > > > > > > > > > > >
