Hi All, We're planning a work on adding Apache Apex as a component to Apache Bigtop. Bigtop is the packaging system for the Apache big data ecosystem. Several Hadoop distros use it, most recently EMR.
Here is the tracking Jira task in APEXCORE for the same: https://issues.apache.org/jira/browse/APEXCORE-331 Proposed plan of execution is as follows: *Step 1) Handle prerequisites* Apache bigtop has some hard and soft expectation for new components to get integrated into Bigtop. Here is the list of it: https://cwiki.apache.org/confluence/display/BIGTOP/Requirement+for+adding+a+new+component+to+Bigtop+distribution Most of them seems to be standard ASF based requirements, but few need to be checked for: 1. Software projects are expected to be Licensed under Apache License, Version 2.0 (and their dependencies are expected to be compatible with this license) - Apex is under ASL 2.0 but need to check if dependencies of Apex are compatible with ASL 2.0. This I guess would be a verification check. 2. Software projects are expected to be compatible with all of the supported platforms that Bigtop distribution is targeting - This needs verifying whether our software runs fine in centos-6 centos-7 fedora-20 ubuntu-14.04 debian-8 opensuse-13.2. 3. What smoke tests that should be added for deployment. 4. Identifying the test artifacts which goes beyond smoke test - These are basically the integration tests for verification after the deployment. This is a soft requirement, but aim is to achieve this as well or at least have explanation why not to include. If there are any from the link which explicitly needs to be checked other than above 4, please let us know. *Step 2) Adding Apex as component to Bigtop* >From one of the mail archive of Bigtop, it was learnt that the bigtop community want to see the addition of new components in phases. Here are the phases: 1. Packaging - This needs creating of package i.e. rpm & deb files. - documentations/READMEs, LICENSE, DISCLAMER, NOTES etc if any needed. - Any documentation that need to be added to distribution of our software. - Any license information of dependencies required to be added to package 2. Smoke tests (at very least) - Adding smoke test for packaging. 3. Puppet recipes for automatic deployment and configuration - Add puppet recipes for automatic deployment 4. Integration tests - For verification of deployments. 5. license clearance: Run 'gradle rat' to make sure all new stuff is compliant with ASF license requirements. If you add code licenses under different licenses, those would need to be listed in the NOTICE. Please share your thoughts on the approach. We'll start corresponding communication on bigtop mailing list as well. We have some specific questions/suggestions related to what should be the content of the package and what should be the smoke tests, but in the interest of not having too much content here, we'll put the questions as a separate mail in this mailthread. Thanks, Chinmay.
