I follow the guide on readme.md for building from git. However I got no luck. Does someone tried it recently?
Konstantin Boudnik <[email protected]>於 2017年6月29日 週四，上午10:53寫道： > On Wed, Jun 21, 2017 at 10:53PM, Olaf Flebbe wrote: > > Andrew, > > > > sorry to be sounding dismissive. > > For it worth - I don't think you were dismissive. But whatever... > > I actually agree with your point: tarballs could be easily derived from the > packages. And if anyone needs them we can have a target in the build system > (please submit the patch), but I would be really against having them as a > part > of the standard artifacts' set. > > > rpm can be converted to cpio payload. deb is an ar archive containing > > data.tar,gz with the files as payload. > > > > Details can be looked up. > > > > Nevertheless, please do not call "tar" an "improvement" with respect to > > deployment compared to "rpm" or "deb". I see the usecase to extract just > the > > content without all the dependency management, service management, > > verification and so on. Please just use rpm2cpio or ar. > > +1 > > Cos > > > > Am 21.06.2017 um 22:08 schrieb Andrew Purtell <[email protected]>: > > > > > >> you surely are making jokes when you are saying TAR is an improvement > > > with respect to RPM/DEB.You surely know that you can unpack every RPM > > > straight to the filesystem (DEB requires two steps), in case you'll > like to. > > > > > > Not a joke and the condescension isn't helpful either. > > > > > > > > > On Wed, Jun 21, 2017 at 10:02 AM, Olaf Flebbe <[email protected]> wrote: > > > > > >> Hi Andrew, > > >> > > >> you surely are making jokes when you are saying TAR is an improvement > with > > >> respect to RPM/DEB.You surely know that you can unpack every RPM > straight > > >> to the filesystem (DEB requires two steps), in case you'll like to. > > >> > > >> You surely know that one can easily host a complete docker based > hadoop > > >> cluster on a developer machine in the current git of bigtop. And that > > >> docker toolbox, docker engine, docker for mac integrates really well > with > > >> Windows, Linux and MacOSX, working right out of the box (at least on > MacOSX > > >> and Linux) as it is right now within bigtop without manually tweaking > > >> config files. > > >> > > >> I see no point in reproducing hive, hbase, ... hadoop tests -- most of > > >> them single machine, fake cluster environment -- when we can have > the real > > >> thing, a cluster where we use docker for isolating nodes. When the > tests do > > >> not really work portable, that's a problem of other projects, not > ours. > > >> Let's fix it there. > > >> > > >> IMHO, if we could orchestrate k8s (kubernetes) (or docker-swarm, my > > >> favorite) we could even chose to use a single host with some docker > > >> instances or scale out to a cloud environment and have a reproducable > > >> system without tweaking files. Of course there is much work to do to > port > > >> tests to the cloud environment, but these would be a tremendous value > added. > > >> > > >> Olaf > > >> > > >> > > >> > > >> > > >>> Am 20.06.2017 um 23:12 schrieb Andrew Purtell < > [email protected] > > >>> : > > >>> > > >>> Yeah, we can build from git repos. Instead of archive URL you can > > >> specify for each component a repo and reference by git-URL and > branch, tag, > > >> or SHA. > > >>> > > >>> Regarding tarball build targets, I was thinking of it as a packaging > > >> improvement, an additional packaging target. It could make integration > > >> testing more convenient too if you are not using containers or bare > metal > > >> systems where you own the whole filesystem. > > >>> > > >>>> On Jun 19, 2017, at 6:13 AM, Evans Ye <[email protected]> wrote: > > >>>> > > >>>> Hi Andy, > > >>>> > > >>>> Is it easier to have multiple tarballs to setup a cluster for > > >> integration > > >>>> tests? > > >>>> I'm not on the Hadoop/HBase developer side so I have zero context. > I was > > >>>> just assuming that deploying a cluster for integration tests would > be a > > >>>> beneficial feature for them. > > >>>> > > >>>> Bringing up my discussion with Hadoop and HBase guys at Cloudera, > them > > >>>> mentioned two things specifically for Bigtop: > > >>>> > > >>>> a). build from git (which I think you've contributed that in Bigtop > > >> already) > > >>>> b). easy to run integration test framework > > >>>> > > >>>> I'm happy to have b). because either way we need to have it in our > CI. > > >>>> > > >>>> > > >>>> 2017-06-19 5:04 GMT+08:00 Andrew Purtell <[email protected]>: > > >>>> > > >>>>> IMHO, the easiest and fastest way to get the distribution aspect > to be > > >> more > > >>>>> useful to more folks is to add a build target that generates plain > > >> tarballs > > >>>>> instead of distro-specific Linux packaging. People like us can > take the > > >>>>> tarballs and unpack them to environments where for various reasons > we > > >> don't > > >>>>> want to do RPM management. Vendors like Cloudera can convert > tarballs > > >> to > > >>>>> parcels, or whatever proprietary format is desired. > > >>>>> > > >>>>> > > >>>>> > > >>>>>> On Sun, Jun 18, 2017 at 12:13 PM, Evans Ye <[email protected]> > > >> wrote: > > >>>>>> > > >>>>>> Hi folks, > > >>>>>> > > >>>>>> Many things happened during DataWorks Summit San Jose 2017. Some > of > > >> the > > >>>>>> folks(Cos, Roman, Amir, Nate, Mike, etc) gathered together to > discuss > > >>>>> 1.2.1 > > >>>>>> and the future 1.3 release of Bigtop. I'd like to get back those > > >>>>>> discussions to the mailing list so that who can't make it there > can > > >> still > > >>>>>> be with us for further discussions: > > >>>>>> > > >>>>>> * 1.2.1 release > > >>>>>> a). Some of the folks expecting Docker on YARN to be back ported > to > > >> 2.7.4 > > >>>>>> and included in the release > > >>>>>> b). Get rotted code out of our code base: packaging, deployment, > > >> testing, > > >>>>>> etc > > >>>>>> c). Get integration test to work in CI > > >>>>>> > > >>>>>> * 1.3.0 release > > >>>>>> a). More machine learning integrations > > >>>>>> b). K8S integration will be an interesting topic > > >>>>>> > > >>>>>> Please help me to complete the list if I miss something. :) > > >>>>>> > > >>>>>> > > >>>>>> OTOH, for me specifically, I visited Cloudera for doing a tech > talk. I > > >>>>> meet > > >>>>>> Sean Mackrory and there Hadoop and HBase lead. The pain point > they're > > >>>>>> having for a long time is not having an integration test > framework for > > >>>>>> there work on the bleeding edge. For example, whether a specific > patch > > >>>>> from > > >>>>>> Hadoop breaks HBase or Hive? > > >>>>>> > > >>>>>> My thinking towards this is this is what Bigtop tries to solve at > the > > >>>>> very > > >>>>>> beginning. We supposed to have folks from multiple projects to > work > > >> with > > >>>>> us > > >>>>>> to upgrade packages, and use our frameworks to properly > integrate, > > >> test > > >>>>>> their code with other components. > > >>>>>> > > >>>>>> So, the future of Bigtop. I think tightly work with the other > > >> communities > > >>>>>> is a better way we move forward. But, that means something need > to be > > >>>>>> changed. For example, our distribution is somehow, from developers > > >>>>>> perspective, old. Which can not support the integration and > testing on > > >>>>> the > > >>>>>> bleeding edge. If we still like to release something suggested > for > > >>>>>> Production only, one of the solution is to have both dev and > stable > > >>>>>> releases in Bigtop, so developers can work on the dev branch and > test > > >>>>>> against newest components. In that case, people from other > communities > > >>>>>> might be possible to help us upgrade the package to the newer > version, > > >>>>>> which makes things easier. > > >>>>>> > > >>>>>> What do you guys think? Please join me for the discussion. > > >>>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> Best regards, > > >>>>> > > >>>>> - Andy > > >>>>> > > >>>>> If you are given a choice, you believe you have acted freely. - > Raymond > > >>>>> Teller (via Peter Watts) > > >>>>> > > >> > > >> > > > > > > > > > -- > > > Best regards, > > > Andrew > > > > > > Words like orphans lost among the crosstalk, meaning torn from truth's > > > decrepit hands > > > - A23, Crosstalk > > > > >
