Makes sense to me On Mon, Apr 17, 2017 at 3:11 PM, Jim Apple <jbap...@cloudera.com> wrote:
> I'm OK with us calling this "experimental" (once the patches are in and we > think it works) and letting patches continue to land with our current > testing regime. > > Later, if we add daily test reports with contributor-managed hardware or > even pre-commit testing on jenkins.impala.io in a VM, maybe we could > upgrade to other names than "experimental", like "beta" or "best-effort", > if we still want to have caveats. > > How does that sound to everyone else? > > On Mon, Apr 17, 2017 at 2:54 PM, Tim Armstrong <tarmstr...@cloudera.com> > wrote: > > > Something like that makes sense - I think it should be an experimental or > > unsupported feature until if/when we have a critical mass of committers > to > > maintain it. > > > > On Mon, Apr 17, 2017 at 2:46 PM, Jim Apple <jbap...@cloudera.com> wrote: > > > > > That makes sense to me. It would be good for us to provide support > > without > > > completely focusing all development effort on a HW platform with fewer > > > users. > > > > > > If ppc64le support lands in 2.10, and between 2.10 and 2.11 the ppc64le > > > tests start breaking for reasons nobody with HW access can debug, > should > > we > > > say in 2.11 release notes that ppc64le is no longer supported? > > > > > > Or perhaps, even in the first release where we think it works, we > should > > > spell it out as a platform with only "best-effort" support, that way we > > > don't have to retract support? > > > > > > On Mon, Apr 17, 2017 at 2:41 PM, Marcel Kornacker <mar...@cloudera.com > > > > > wrote: > > > > > > > My main concern is that we don't unduly burden the development > process. > > > As > > > > such, it makes a lot of sense to keep the PPC tests out of the > regular > > > > tests and have the party that's interested in the PPC tests create > the > > > > infrastructure and run those tests. > > > > > > > > On Mon, Apr 17, 2017 at 2:30 PM, Jim Apple <jbap...@cloudera.com> > > wrote: > > > > > > > > > One concern I have is sustainability. If only one Impala > contributor > > > can > > > > > work with ppc64le, and that contributor is not as seasoned as some > of > > > the > > > > > other committers, what happens if ppc64le breaks and the one person > > > with > > > > VM > > > > > access can't fix it? > > > > > > > > > > Part of my concern is just how flaky the current tests are, too. It > > > takes > > > > > some time to be able to distinguish broken tests that are flaky and > > > > broken > > > > > tests that are the result of a specific commit. > > > > > > > > > > My hope was that with a ppc64le VM (maybe through Qemu?) that runs > on > > > > > x86-64 Linux, the other contributors could help fix things that > broke > > > on > > > > > that platform. > > > > > > > > > > On Mon, Apr 17, 2017 at 12:51 PM, Marcel Kornacker < > > > mar...@cloudera.com> > > > > > wrote: > > > > > > > > > > > +1 > > > > > > > > > > > > On Mon, Apr 17, 2017 at 11:56 AM, Henry Robinson < > > he...@cloudera.com > > > > > > > > > > wrote: > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > On Mon, Apr 17, 2017 at 9:09 AM Tim Armstrong < > > > > tarmstr...@cloudera.com > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > I feel like we shouldn't make PPC part of pre-commit at least > > > > > > initially - > > > > > > > > it's an unreasonable barrier if contributors/committers to > > debug > > > > > issues > > > > > > > on > > > > > > > > a platform they don't have easy access to. Having the testing > > > infra > > > > > is > > > > > > > > still important because we don't want to have code in there > > that > > > > will > > > > > > > > gradually bit-rot without us noticing. > > > > > > > > > > > > > > > > On Mon, Apr 17, 2017 at 8:51 AM, Silvius Rus < > > s...@cloudera.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > Would it make sense to _not_ run PPC tests as part of > > > presubmit? > > > > > > > Instead > > > > > > > > > Valencia could set up nightly tests using in-house > > > > infrastructure. > > > > > > And > > > > > > > > > share the test results, e.g., by sending them to a new > email > > > list > > > > > > > > > te...@impala.incubator.apache.org (that we'd need to > create) > > > so > > > > > > > everyone > > > > > > > > > can see when there are failures or if coverage stops for > > > whatever > > > > > > > reason. > > > > > > > > > GCC has been doing something like this for long, > > > > > > > > > https://gcc.gnu.org/ml/gcc-testresults/2017-04/. > > > > > > > > > > > > > > > > > > On Tue, Apr 11, 2017 at 9:44 AM, Jim Apple < > > > jbap...@cloudera.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Locally, I work on native-toolchain using a VM > configured > > > > with > > > > > > > > > > > Ubuntu16.04ppc64le, 4GB RAM and 50GB of HDD. If we > > provide > > > > > you a > > > > > > > VM > > > > > > > > > with > > > > > > > > > > > this config, will it be sufficient ? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > What hypervisor/emulator will it use? > > > > > > > > > > > > > > > > > > > > What are the requirements of the host OS and host > hardware? > > > > > > > > > > > > > > > > > > > > Why is the config you have it set to so important that > you > > > > > mention > > > > > > it > > > > > > > > in > > > > > > > > > > your email - will the config be locked down into that > > config > > > or > > > > > can > > > > > > > it > > > > > > > > be > > > > > > > > > > reconfigured later? > > > > > > > > > > > > > > > > > > > > How is the VM controlled from the host OS? Keep in mind > > that > > > a > > > > > GUI > > > > > > > > cannot > > > > > > > > > > be the only option for automated tests. > > > > > > > > > > > > > > > > > > > > FWIW, Impala's test suite probably cannot fully complete > > > > without > > > > > at > > > > > > > > least > > > > > > > > > > 8, and I suspect 16, GB of RAM, and we might need more > disk > > > > > space, > > > > > > > too, > > > > > > > > > but > > > > > > > > > > these should be reconfigurable with most > > > hypervisors/emulators. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > Henry Robinson > > > > > > > Software Engineer > > > > > > > Cloudera > > > > > > > 415-994-6679 > > > > > > > > > > > > > > > > > > > > > > > > > > > >
