I do not think this is appropriate yet because of how early in the process we are.
On Wed, Apr 26, 2017 at 11:13 PM, Valencia Serrao <[email protected]> wrote: > Hi All, > > I have submitted the second patchset of ppc64le-native-toolchain for > gerrit review. Also, I have the build logs of x86 and ppc64le runs of the > native-toolchain ready. Earlier in thread, there was a comment that a new > email list "*[email protected]* > <[email protected]>" could be created to share the same. > Is it created already ? > > Please let me know the best way I could share the logs for reference. > > Regards, > Valencia > > [image: Inactive hide details for Valencia Serrao---04/20/2017 02:36:33 > PM---Hi Jim, That's OK. Discussing on the testing infra issue,]Valencia > Serrao---04/20/2017 02:36:33 PM---Hi Jim, That's OK. Discussing on the > testing infra issue, having more clarity on it and deciding to > > From: Valencia Serrao/Austin/Contr/IBM > To: Jim Apple <[email protected]> > Cc: "dev@impala" <[email protected]>, Manish > Patil/Austin/Contr/IBM@IBMUS, Nishidha Panpaliya/Austin/Contr/IBM@IBMUS, > Sudarshan Jagadale/Austin/Contr/IBM@IBMUS, David Clissold/Austin/IBM@IBMUS, > Valencia Serrao/Austin/Contr/IBM@IBMUS > Date: 04/20/2017 02:36 PM > Subject: Re: Upstreaming ppc64le patches for native-toolchain > ------------------------------ > > > Hi Jim, > > That's OK. Discussing on the testing infra issue, having more clarity on > it and deciding to go ahead with reviewing the ppc64le patches, in this we > all have progressed. I have taken following key points from the earlier > discussion. > > 1. Jenkins ppc64le testing infra not required for now. > 2. ppc64le patch submission can resume, with every patch accompanied with > test results on x86 and ppc64le. > 3. All ppc64le-related modifications will be termed as experimental until > they are confirmed to work by community. > > Please let me know if I've missed out on anything. > > Also, I'm focusing on getting the native-toolchain reviewed and upstreamed > first, then breakpad, kudu, followed by Impala. > > Regards, > Valencia > > > [image: Inactive hide details for Jim Apple ---04/20/2017 03:53:19 > AM---OK, then it sounds like for now the ppc64le work can proceed fo]Jim > Apple ---04/20/2017 03:53:19 AM---OK, then it sounds like for now the > ppc64le work can proceed for now without a Jenkins testing avail > > From: Jim Apple <[email protected]> > To: "dev@impala" <[email protected]> > Cc: Manish Patil/Austin/Contr/IBM@IBMUS, Nishidha > Panpaliya/Austin/Contr/IBM@IBMUS, Sudarshan Jagadale/Austin/Contr/IBM@IBMUS, > Valencia Serrao/Austin/Contr/IBM@IBMUS > Date: 04/20/2017 03:53 AM > Subject: Re: Upstreaming ppc64le patches for native-toolchain > ------------------------------ > > > > OK, then it sounds like for now the ppc64le work can proceed for now > without a Jenkins testing available to the whole community, unless anyone > else objects. > > Valencia et al.: apologies if I delayed your progress > > Silvius & Tim: thanks for the good suggestions. > > On Mon, Apr 17, 2017 at 4:19 PM, Tim Armstrong <*[email protected]* > <[email protected]>> wrote: > > Makes sense to me > > On Mon, Apr 17, 2017 at 3:11 PM, Jim Apple <*[email protected]* > <[email protected]>> wrote: > > > I'm OK with us calling this "experimental" (once the patches are in > and we > > think it works) and letting patches continue to land with our current > > testing regime. > > > > Later, if we add daily test reports with contributor-managed > hardware or > > even pre-commit testing on *jenkins.impala.io* > <http://jenkins.impala.io/> in a VM, maybe we could > > upgrade to other names than "experimental", like "beta" or > "best-effort", > > if we still want to have caveats. > > > > How does that sound to everyone else? > > > > On Mon, Apr 17, 2017 at 2:54 PM, Tim Armstrong < > *[email protected]* <[email protected]>> > > wrote: > > > > > Something like that makes sense - I think it should be an > experimental or > > > unsupported feature until if/when we have a critical mass of > committers > > to > > > maintain it. > > > > > > On Mon, Apr 17, 2017 at 2:46 PM, Jim Apple <*[email protected]* > <[email protected]>> wrote: > > > > > > > That makes sense to me. It would be good for us to provide > support > > > without > > > > completely focusing all development effort on a HW platform with > fewer > > > > users. > > > > > > > > If ppc64le support lands in 2.10, and between 2.10 and 2.11 the > ppc64le > > > > tests start breaking for reasons nobody with HW access can debug, > > should > > > we > > > > say in 2.11 release notes that ppc64le is no longer supported? > > > > > > > > Or perhaps, even in the first release where we think it works, we > > should > > > > spell it out as a platform with only "best-effort" support, that > way we > > > > don't have to retract support? > > > > > > > > On Mon, Apr 17, 2017 at 2:41 PM, Marcel Kornacker < > *[email protected]* <[email protected]> > > > > > > > wrote: > > > > > > > > > My main concern is that we don't unduly burden the development > > process. > > > > As > > > > > such, it makes a lot of sense to keep the PPC tests out of the > > regular > > > > > tests and have the party that's interested in the PPC tests > create > > the > > > > > infrastructure and run those tests. > > > > > > > > > > On Mon, Apr 17, 2017 at 2:30 PM, Jim Apple < > *[email protected]* <[email protected]>> > > > wrote: > > > > > > > > > > > One concern I have is sustainability. If only one Impala > > contributor > > > > can > > > > > > work with ppc64le, and that contributor is not as seasoned > as some > > of > > > > the > > > > > > other committers, what happens if ppc64le breaks and the one > person > > > > with > > > > > VM > > > > > > access can't fix it? > > > > > > > > > > > > Part of my concern is just how flaky the current tests are, > too. It > > > > takes > > > > > > some time to be able to distinguish broken tests that are > flaky and > > > > > broken > > > > > > tests that are the result of a specific commit. > > > > > > > > > > > > My hope was that with a ppc64le VM (maybe through Qemu?) > that runs > > on > > > > > > x86-64 Linux, the other contributors could help fix things > that > > broke > > > > on > > > > > > that platform. > > > > > > > > > > > > On Mon, Apr 17, 2017 at 12:51 PM, Marcel Kornacker < > > > > *[email protected]* <[email protected]>> > > > > > > wrote: > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > On Mon, Apr 17, 2017 at 11:56 AM, Henry Robinson < > > > *[email protected]* <[email protected]> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > On Mon, Apr 17, 2017 at 9:09 AM Tim Armstrong < > > > > > *[email protected]* <[email protected]> > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > I feel like we shouldn't make PPC part of pre-commit > at least > > > > > > > initially - > > > > > > > > > it's an unreasonable barrier if > contributors/committers to > > > debug > > > > > > issues > > > > > > > > on > > > > > > > > > a platform they don't have easy access to. Having the > testing > > > > infra > > > > > > is > > > > > > > > > still important because we don't want to have code in > there > > > that > > > > > will > > > > > > > > > gradually bit-rot without us noticing. > > > > > > > > > > > > > > > > > > On Mon, Apr 17, 2017 at 8:51 AM, Silvius Rus < > > > *[email protected]* <[email protected]>> > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Would it make sense to _not_ run PPC tests as part of > > > > presubmit? > > > > > > > > Instead > > > > > > > > > > Valencia could set up nightly tests using in-house > > > > > infrastructure. > > > > > > > And > > > > > > > > > > share the test results, e.g., by sending them to a > new > > email > > > > list > > > > > > > > > > *[email protected]* > <[email protected]> (that we'd need to > > create) > > > > so > > > > > > > > everyone > > > > > > > > > > can see when there are failures or if coverage stops > for > > > > whatever > > > > > > > > reason. > > > > > > > > > > GCC has been doing something like this for long, > > > > > > > > > > *https://gcc.gnu.org/ml/gcc-testresults/2017-04/* > <https://gcc.gnu.org/ml/gcc-testresults/2017-04/>. > > > > > > > > > > > > > > > > > > > > On Tue, Apr 11, 2017 at 9:44 AM, Jim Apple < > > > > *[email protected]* <[email protected]> > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Locally, I work on native-toolchain using a VM > > configured > > > > > with > > > > > > > > > > > > Ubuntu16.04ppc64le, 4GB RAM and 50GB of HDD. If > we > > > provide > > > > > > you a > > > > > > > > VM > > > > > > > > > > with > > > > > > > > > > > > this config, will it be sufficient ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What hypervisor/emulator will it use? > > > > > > > > > > > > > > > > > > > > > > What are the requirements of the host OS and host > > hardware? > > > > > > > > > > > > > > > > > > > > > > Why is the config you have it set to so important > that > > you > > > > > > mention > > > > > > > it > > > > > > > > > in > > > > > > > > > > > your email - will the config be locked down into > that > > > config > > > > or > > > > > > can > > > > > > > > it > > > > > > > > > be > > > > > > > > > > > reconfigured later? > > > > > > > > > > > > > > > > > > > > > > How is the VM controlled from the host OS? Keep in > mind > > > that > > > > a > > > > > > GUI > > > > > > > > > cannot > > > > > > > > > > > be the only option for automated tests. > > > > > > > > > > > > > > > > > > > > > > FWIW, Impala's test suite probably cannot fully > complete > > > > > without > > > > > > at > > > > > > > > > least > > > > > > > > > > > 8, and I suspect 16, GB of RAM, and we might need > more > > disk > > > > > > space, > > > > > > > > too, > > > > > > > > > > but > > > > > > > > > > > these should be reconfigurable with most > > > > hypervisors/emulators. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Henry Robinson > > > > > > > > Software Engineer > > > > > > > > Cloudera > > > > > > > > *415-994-6679* <415-994-6679> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
