On Fri, Feb 26, 2016 at 02:26PM, Chinmay Kolhatkar wrote: > Thank you Bigtop community for efficiently consuming the first patch > (packing of apex).
yeah, you don't want to see the regurgitation process ;) > For the second phase, I'll soon be creating a PR for Smoke tests for apex > package. Keep 'em coming and thanks for the contributions! Cos > On Thu, Feb 25, 2016 at 12:30 PM, Chinmay Kolhatkar <[email protected]> > wrote: > > > Thanks Jay. > > There are already some comments from YoungWoo Kim which are taken care of > > in that PR. > > > > On Tue, Feb 23, 2016 at 6:05 PM, Jay Vyas <[email protected]> > > wrote: > > > >> Thanks again for this , don't apologize for the email. > >> > >> We actually are excited about apex, thanks for this! We will get it > >> reviewed within the next couple of days I'm sure (I'll look some time after > >> work if nobody else gets to it first). > >> > >> > On Feb 23, 2016, at 6:09 AM, Chinmay Kolhatkar <[email protected]> > >> wrote: > >> > > >> > Sorry for another mail. Just want to mention how I tested the patch > >> before > >> > creating PR: > >> > > >> > Apex package was built using with following docker images: > >> > 1) Ubuntu trusty: > >> > docker run --rm -v `pwd`:/ws --workdir /ws > >> bigtop/slaves:trunk-ubuntu-14.04 > >> > bash -l -c './gradlew allclean ; ./gradlew apex-pkg' > >> > > >> > 2) CentOS 7 > >> > docker run --rm -v `pwd`:/ws --workdir /ws bigtop/slaves:trunk-centos-7 > >> > bash -l -c './gradlew allclean ; ./gradlew apex-pkg' > >> > > >> > 3) CentOS 6 > >> > docker run --rm -v `pwd`:/ws --workdir /ws bigtop/slaves:trunk-centos-6 > >> > bash -l -c './gradlew allclean ; ./gradlew apex-pkg' > >> > > >> > 4) OpenSUSE 13.2 > >> > docker run --rm -v `pwd`:/ws --workdir /ws > >> > bigtop/slaves:trunk-opensuse-13.2 -l -c './gradlew allclean ; ./gradlew > >> > apex-pkg' > >> > > >> > > >> > Apex package was installed on following docker containers and verified > >> for > >> > sanity: > >> > 1) bigtop/slaves:trunk-ubuntu-14.04 > >> > 2) bigtop/slaves:trunk-centos-6 > >> > 3) bigtop/slaves:trunk-centos-7 > >> > 4) bigtop/slaves:trunk-opensuse-13.2 > >> > > >> > When you have some time, please provide the feedback on the pull > >> request ( > >> > https://github.com/apache/bigtop/pull/92) > >> > > >> > Thank you, > >> > Chinmay. > >> > > >> > > >> > > >> > > >> > On Mon, Feb 22, 2016 at 9:43 PM, Chinmay Kolhatkar <[email protected]> > >> > wrote: > >> > > >> >> Hello Bigtop Community, > >> >> > >> >> I've created a pull request for packing of Apache Apex into bigtop that > >> >> I'd like to get reviewd and committed to Bigtop repository: > >> >> https://github.com/apache/bigtop/pull/92 > >> >> > >> >> The pull request contains code for packing of apex for both deb and rpm > >> >> packing. > >> >> This does not include smoke tests which will be the very next steps > >> once > >> >> this gets accepted. > >> >> > >> >> If you have some time, could you please review this pull request and > >> >> provide feedback so that we can get these merged in? > >> >> > >> >> Also, there is a question related to what should be included in package > >> >> with respect to packing of runtime dependent jars. > >> >> Currently, I've included license files of non-ASF based licenses in > >> >> package. > >> >> Is there any specific guidelines from bigtop that integrating projects > >> >> needs to follow on that front? > >> >> > >> >> In the meantime, I would like to get some feedback on pull request for > >> if > >> >> there are any code change required. > >> >> > >> >> Thank you in advance!! > >> >> > >> >> Chinmay. > >> >> > >> >> > >> >> > >> >> On Wed, Feb 17, 2016 at 11:10 AM, Chinmay Kolhatkar < > >> [email protected]> > >> >> wrote: > >> >> > >> >>> Cos, > >> >>> Thank for a quick reply. We (apex community) has decided to follow > >> your > >> >>> recommendation i.e. drop "-incubating". > >> >>> > >> >>> Roman, > >> >>> Apologies, I did not understand your last question earlier. > >> >>> Yes, I can certainly provide the prototype for rpm+deb packing of Apex > >> >>> with bigtop. > >> >>> Here is the link to GH where development is WIP: > >> >>> https://github.com/chinmaykolhatkar/bigtop/tree/BIGTOP-2314 > >> >>> > >> >>> Thanks, > >> >>> Chinmay. > >> >>> > >> >>> > >> >>> On Tue, Feb 16, 2016 at 9:15 PM, Konstantin Boudnik <[email protected]> > >> >>> wrote: > >> >>> > >> >>>> Dude to known limitations of RPMs "-" aren't allowed for sure. Deb > >> >>>> doesn't > >> >>>> have such idiosyncrasy, but there are ppl who are using RPM, which is > >> >>>> beyond > >> >>>> me. Anyway... > >> >>>> > >> >>>> This is a long standing issue for us as well, as we have a few > >> incubating > >> >>>> projects in the stack. My recommendation would be to drop > >> "-incubating" > >> >>>> part > >> >>>> from the package version. Check out how it is done for zeppelin. > >> >>>> > >> >>>> Cos > >> >>>> > >> >>>>> On Tue, Feb 16, 2016 at 09:04PM, Chinmay Kolhatkar wrote: > >> >>>>> Hello Everyone, > >> >>>>> > >> >>>>> I'm facing an issue while setting base version for rpm package. > >> >>>>> We want to have version of Apache Apex as "3.3.0-incubating". > >> >>>>> But when I try to set version in bigtop.bom file, I get error while > >> >>>>> generating rpm package as rpm does not allow hyphen ('-') in > >> version. > >> >>>>> > >> >>>>> I also tried to set 'incubating' as a value as release: version { > >> base > >> >>>> = > >> >>>>> '3.3.0'; pkg = base; release = 'incubating' } > >> >>>>> But that doesn't seem to take any effect. > >> >>>>> > >> >>>>> Can you please suggest how can we achieve this? > >> >>>>> > >> >>>>> Thanks, > >> >>>>> Chinmay. > >> >>>>> > >> >>>>> > >> >>>>> On Mon, Feb 15, 2016 at 11:38 AM, Chinmay Kolhatkar < > >> >>>> [email protected]> > >> >>>>> wrote: > >> >>>>> > >> >>>>>> Hi Roman, > >> >>>>>> > >> >>>>>> Thanks for the feedback. > >> >>>>>> Here are the answers to the question in the same order: > >> >>>>>> 1. Apex is a YARN-native platform. Hence apex needs YARN to be > >> there. > >> >>>>>> 2. conf/ directory recently got removed. This is not needed anymore > >> >>>> in the > >> >>>>>> package. > >> >>>>>> 3. Here is the link to GH repo for apex platform: > >> >>>>>> https://github.com/apache/incubator-apex-core > >> >>>>>> The source as of now does not output any package. do-component > >> >>>> build > >> >>>>>> would compile the apex platform and generate a rpm/deb package > >> using > >> >>>> files > >> >>>>>> present in source code. Hence there is no prototype of the package. > >> >>>>>> Is that the prototype you're looking for? > >> >>>>>> > >> >>>>>> Thanks, > >> >>>>>> Chinmay. > >> >>>>>> > >> >>>>>> > >> >>>>>> On Mon, Feb 15, 2016 at 11:11 AM, Roman Shaposhnik < > >> >>>> [email protected]> > >> >>>>>> wrote: > >> >>>>>> > >> >>>>>>> Looks good to me. Three questions: > >> >>>>>>> 1. Does apex provide standalone daemon or does only run on YARN > >> ? > >> >>>>>>> 2. What's the intended use for content under /usr/lib/apex/conf > >> ? > >> >>>>>>> 3. Any chance you can provide a pointer to GH repo with a > >> >>>> prototype? > >> >>>>>>> > >> >>>>>>> Thanks, > >> >>>>>>> Roman. > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> On Sun, Feb 14, 2016 at 8:36 PM, Chinmay Kolhatkar < > >> >>>> [email protected]> > >> >>>>>>> wrote: > >> >>>>>>>> Hi Everyone, > >> >>>>>>>> > >> >>>>>>>> Here is some details regarding packaging apex component into > >> >>>> bigtop: > >> >>>>>>>> 1. Apex component will dependent on hadoop. This will be > >> >>>> mentioned in > >> >>>>>>>> bigtop.bom file. > >> >>>>>>>> 2. The component added to bom file will look like following: > >> >>>>>>>> > >> >>>>>>>> 'apex' { > >> >>>>>>>> name = 'apex' > >> >>>>>>>> relNotes = 'Apache Apex (incubating)' > >> >>>>>>>> version { base = '3.3.0-incubating'; pkg = base; release = > >> >>>> 1 } > >> >>>>>>>> tarball { source = > >> >>>>>>> "$name-${version.base}-source-release.tar.gz" > >> >>>>>>>> destination = "$name-${version.base}.tar.gz" } > >> >>>>>>>> url { download_path = > >> >>>> "/incubator/$name/v${version.base}/" > >> >>>>>>>> site = "${apache.APACHE_MIRROR}/${download_path}" > >> >>>>>>>> archive = > >> >>>> "${apache.APACHE_ARCHIVE}/${download_path}" } > >> >>>>>>>> } > >> >>>>>>>> > >> >>>>>>>> 3. After installation of package (deb+rpm), the directory > >> >>>> structure > >> >>>>>>> related > >> >>>>>>>> to apex will look like following: > >> >>>>>>>> > >> >>>>>>>> / > >> >>>>>>>> `-- usr > >> >>>>>>>> |-- bin > >> >>>>>>>> | |-- apex -> ../lib/apex/bin/apex > >> >>>>>>>> | `-- dtcli -> ../lib/apex/bin/apex > >> >>>>>>>> |-- lib > >> >>>>>>>> | `-- apex > >> >>>>>>>> | |-- bin > >> >>>>>>>> | | |-- apex > >> >>>>>>>> | | `-- dtcli -> apex > >> >>>>>>>> | |-- conf > >> >>>>>>>> | | `-- dt-env.sh > >> >>>>>>>> | `-- lib > >> >>>>>>>> | |-- apex-api-3.3.0-incubating.jar > >> >>>>>>>> | |-- apex-bufferserver-3.3.0-incubating.jar > >> >>>>>>>> | |-- apex-common-3.3.0-incubating.jar > >> >>>>>>>> | |-- apex-engine.jar > >> >>>>>>>> | `-- <other dependency jars> > >> >>>>>>>> `-- share > >> >>>>>>>> `-- doc > >> >>>>>>>> `-- apex > >> >>>>>>>> |-- changelog.Debian.gz > >> >>>>>>>> |-- copyright > >> >>>>>>>> `-- license > >> >>>>>>>> `-- LICENSE.gz > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> Please share your thoughts on above approach. > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> Thanks, > >> >>>>>>>> Chinmay. > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>>> On Sat, Feb 13, 2016 at 8:38 AM, Thomas Weise <[email protected]> > >> >>>> wrote: > >> >>>>>>>> > >> >>>>>>>>> Chinmay, > >> >>>>>>>>> > >> >>>>>>>>> Can you share some details regarding the packaging? > >> >>>>>>>>> > >> >>>>>>>>> Thanks > >> >>>>>>>>> > >> >>>>>>>>> On Thu, Feb 11, 2016 at 12:35 AM, Chinmay Kolhatkar < > >> >>>>>>>>> [email protected] > >> >>>>>>>>>> wrote: > >> >>>>>>>>> > >> >>>>>>>>>> Dear Community, > >> >>>>>>>>>> > >> >>>>>>>>>> Here is the Jira that is created for adding Apex into Bigtop. > >> >>>>>>>>>> https://issues.apache.org/jira/browse/BIGTOP-2313 > >> >>>>>>>>>> > >> >>>>>>>>>> I've started working on creating packages for Apex. > >> >>>>>>>>>> > >> >>>>>>>>>> Please share your thoughts on approach as well as on questions > >> >>>>>>> related to > >> >>>>>>>>>> packaging in my previous mail. > >> >>>>>>>>>> > >> >>>>>>>>>> Thanks, > >> >>>>>>>>>> Chinmay. > >> >>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>>> > >> >>>>>>>>>> On Wed, Feb 10, 2016 at 9:55 PM, Chinmay Kolhatkar < > >> >>>>>>>>>> [email protected]> > >> >>>>>>>>>> wrote: > >> >>>>>>>>>> > >> >>>>>>>>>>> Hello Bigtop Community, > >> >>>>>>>>>>> > >> >>>>>>>>>>> Continuing with the packing effort, here is how we're > >> >>>> planning to > >> >>>>>>>>> package > >> >>>>>>>>>>> Apache Apex. > >> >>>>>>>>>>> > >> >>>>>>>>>>> Tentative package directory structure for apex would look > >> >>>> like > >> >>>>>>>>> following: > >> >>>>>>>>>>> > >> >>>>>>>>>>> /usr/lib/apex/bin/dtcli > >> >>>>>>>>>>> /usr/lib/apex/lib/apex-api-3.3.0-incubating.jar > >> >>>>>>>>>>> /usr/lib/apex/lib/apex-engine-3.3.0-incubating.jar > >> >>>>>>>>>>> /usr/lib/apex/lib/apex-bufferserver-3.3.0-incubating.jar > >> >>>>>>>>>>> /usr/lib/apex/lib/apex-common-3.3.0-incubating.jar > >> >>>>>>>>>>> /usr/lib/apex/lib/<other dependent jars> > >> >>>>>>>>>>> /usr/lib/apex/conf/dt-env.sh > >> >>>>>>>>>>> /usr/bin/dtcli -> /usr/lib/apex/bin/dtcli > >> >>>>>>>>>>> /usr/share/doc/man/man1/dtcli.1.gz > >> >>>>>>>>>>> /usr/share/doc/apex/license/LICENSE.txt > >> >>>>>>>>>>> /usr/share/doc/apex/license/<package>-LICENSE.txt > >> >>>>>>>>>>> /usr/share/doc/apex/CHANGELOG > >> >>>>>>>>>>> /usr/share/doc/apex/NOTICE > >> >>>>>>>>>>> > >> >>>>>>>>>>> Addition contents to bigtop.bom: > >> >>>>>>>>>>> 1. Dependency of apex will be on "hadoop" > >> >>>>>>>>>>> 2. Component addition would looks like this: > >> >>>>>>>>>>> 'apex' { > >> >>>>>>>>>>> name = 'apex' > >> >>>>>>>>>>> relNotes = 'Apache Apex (incubating)' > >> >>>>>>>>>>> version { base = '3.3.0-incubating'; pkg = base; > >> >>>> release = 1 > >> >>>>>>> } > >> >>>>>>>>>>> tarball { source = > >> >>>>>>>>>> "$name-${version.base}-source-release.tar.gz" > >> >>>>>>>>>>> destination = "$name-${version.base}.tar.gz" > >> >>>> } > >> >>>>>>>>>>> url { download_path = > >> >>>>>>> "/incubator/$name/v${version.base}/" > >> >>>>>>>>>>> site = > >> >>>> "${apache.APACHE_MIRROR}/${download_path}" > >> >>>>>>>>>>> archive = > >> >>>>>>> "${apache.APACHE_ARCHIVE}/${download_path}" } > >> >>>>>>>>>>> } > >> >>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>>> I have couple of questions related to packaging: > >> >>>>>>>>>>> 1. What is the best possible way to test the package locally? > >> >>>>>>>>>>> 2. Do you have any specific size limitation on size of the > >> >>>> package > >> >>>>>>> (rpm > >> >>>>>>>>>> OR > >> >>>>>>>>>>> deb)? > >> >>>>>>>>>>> 3. Is above proposed packaging information inline with bigtop > >> >>>>>>>>> standards? > >> >>>>>>>>>>> 4. Am I missing anything there? > >> >>>>>>>>>>> > >> >>>>>>>>>>> Thanks, > >> >>>>>>>>>>> Chinmay. > >> >>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>>> > >> >>>>>>>>>>> On Fri, Feb 5, 2016 at 12:01 AM, Chinmay Kolhatkar < > >> >>>>>>>>>>> [email protected]> wrote: > >> >>>>>>>>>>> > >> >>>>>>>>>>>> Hello Bigtop community! > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> My name is Chinmay Kolhatkar and I am a committer in Apache > >> >>>> Apex > >> >>>>>>>>>>>> (incubating) project. > >> >>>>>>>>>>>> Our website is at: http://apex.incubator.apache.org/ > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> We want to explore and integrate Apache Apex into the > >> >>>> Bigtop as a > >> >>>>>>>>>>>> component. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> Please treat this mail as a continuation of a mail thread: > >> >>>> > >> http://mail-archives.apache.org/mod_mbox/bigtop-dev/201601.mbox/%3CCA%2B5xAo20CAHuYtdxw3mtH2X9whREKd7fLdz4xmuTMqkaTiiskg%40mail.gmail.com%3E > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> We're currently confirming with requirements for components > >> >>>> and > >> >>>>>>> making > >> >>>>>>>>>>>> sure that we meet the expectations: > >> >>>> > >> https://cwiki.apache.org/confluence/display/BIGTOP/Requirement+for+adding+a+new+component+to+Bigtop+distribution > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> Once requirements are through, we're planning to add > >> >>>> component > >> >>>>>>> into > >> >>>>>>>>>>>> Apache Bigtop in following phases (as separate patches/pull > >> >>>>>>> requests): > >> >>>>>>>>>>>> 1. Packaging > >> >>>>>>>>>>>> - Creating of package i.e. rpm & deb files. > >> >>>>>>>>>>>> - documentations/READMEs, LICENSE, DISCLAMER, NOTES etc > >> >>>> if any > >> >>>>>>>>>> needed. > >> >>>>>>>>>>>> - Any documentation that need to be added to > >> >>>> distribution of > >> >>>>>>> our > >> >>>>>>>>>>>> software. > >> >>>>>>>>>>>> - Any license information of dependencies required to be > >> >>>>>>> added to > >> >>>>>>>>>>>> package > >> >>>>>>>>>>>> 2. Smoke tests > >> >>>>>>>>>>>> - Adding smoke test for packaging. > >> >>>>>>>>>>>> 3. Puppet recipes for automatic deployment and configuration > >> >>>>>>>>>>>> - Add puppet recipes for automatic deployment > >> >>>>>>>>>>>> 4. license clearance: > >> >>>>>>>>>>>> - Run 'gradle rat' to make sure all new stuff is > >> >>>> compliant > >> >>>>>>> with > >> >>>>>>>>> ASF > >> >>>>>>>>>>>> license requirements. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> Please share your thoughts on above approach. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> If the approach seems fine, we can create a Jira items and > >> >>>>>>>>> corresponding > >> >>>>>>>>>>>> subtasks to start working. > >> >>>>>>>>>>>> > >> >>>>>>>>>>>> Thanks, > >> >>>>>>>>>>>> Chinmay. > >> >> > >> > > > >
signature.asc
Description: Digital signature
