IMHO, the easiest and fastest way to get the distribution aspect to be more useful to more folks is to add a build target that generates plain tarballs instead of distro-specific Linux packaging. People like us can take the tarballs and unpack them to environments where for various reasons we don't want to do RPM management. Vendors like Cloudera can convert tarballs to parcels, or whatever proprietary format is desired.
On Sun, Jun 18, 2017 at 12:13 PM, Evans Ye <[email protected]> wrote: > Hi folks, > > Many things happened during DataWorks Summit San Jose 2017. Some of the > folks(Cos, Roman, Amir, Nate, Mike, etc) gathered together to discuss 1.2.1 > and the future 1.3 release of Bigtop. I'd like to get back those > discussions to the mailing list so that who can't make it there can still > be with us for further discussions: > > * 1.2.1 release > a). Some of the folks expecting Docker on YARN to be back ported to 2.7.4 > and included in the release > b). Get rotted code out of our code base: packaging, deployment, testing, > etc > c). Get integration test to work in CI > > * 1.3.0 release > a). More machine learning integrations > b). K8S integration will be an interesting topic > > Please help me to complete the list if I miss something. :) > > > OTOH, for me specifically, I visited Cloudera for doing a tech talk. I meet > Sean Mackrory and there Hadoop and HBase lead. The pain point they're > having for a long time is not having an integration test framework for > there work on the bleeding edge. For example, whether a specific patch from > Hadoop breaks HBase or Hive? > > My thinking towards this is this is what Bigtop tries to solve at the very > beginning. We supposed to have folks from multiple projects to work with us > to upgrade packages, and use our frameworks to properly integrate, test > their code with other components. > > So, the future of Bigtop. I think tightly work with the other communities > is a better way we move forward. But, that means something need to be > changed. For example, our distribution is somehow, from developers > perspective, old. Which can not support the integration and testing on the > bleeding edge. If we still like to release something suggested for > Production only, one of the solution is to have both dev and stable > releases in Bigtop, so developers can work on the dev branch and test > against newest components. In that case, people from other communities > might be possible to help us upgrade the package to the newer version, > which makes things easier. > > What do you guys think? Please join me for the discussion. > -- Best regards, - Andy If you are given a choice, you believe you have acted freely. - Raymond Teller (via Peter Watts)
