Re: [DISCUSS] Replace ptest with hive-test-kube

2020-06-05 Thread Zoltan Haindrich
Peter: I think that's great practice - but if its not safeguarded by something at all the time; eventually it might be overlooked. for example some time in the past we had javadoc fixed on some modules - but all that work is in vain because someone have added javadoc errors again... David:

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-06-05 Thread David Mollitor
There are a couple Apache projects that use the "spotless" plugin to format the code so that it all adheres to the same style. I'd be in favor of adding that to make that code automatically adhere to the check style requirements. On Fri, Jun 5, 2020, 5:59 AM Peter Vary wrote: > I personally

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-06-05 Thread Peter Vary
I personally checked the "soft" errors and fixed them if a second run was needed in my patches (which was the case most often than not ,because of the flaky tests anyway :() Also often checked the output before my reviews as well, and asked contributors to fix them. Running them manually for

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-06-05 Thread Stamatis Zampetakis
+1 for failing fast starting with findbugs and eventually covering the important points from checkstyle. Bes, Stamatis On Fri, Jun 5, 2020 at 9:35 AM Zoltan Haindrich wrote: > > Hey Mustafa! > > Those checks are not executed anymore in the new system. I always feeled > it a bit confusing to

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-06-05 Thread Zoltan Haindrich
Hey Mustafa! Those checks are not executed anymore in the new system. I always feeled it a bit confusing to have a comment which reports about checkstyle/finbugs/etc issues; while getting a green test run was almost impossible due to the high number of randomly falling tests. So I don't

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-06-04 Thread Mustafa IMAN
Thank you Zoltan for all this work. I see many PRs are merged based on the new workflow already. The old workflow generates many reports like ASF license/findbugs/checkstyle etc. I don't see these in the new Github PR workflow. I am concerned the codebase is going to suffer from lack of these

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-06-02 Thread Zoltan Haindrich
Hello, I would like to note that you may login to the jenkins instance - to start/kill builds (or create new jobs). I've configured github oauth - but since team membership can't be queried from the "apache organization" - it's harder to configure all "hive committers". However...I think I've

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-05-29 Thread Zoltan Haindrich
Hey all! The patch is now in master - so every new PR or a push on it will trigger a new run. Please decide which one would you like to use - open a PR to see the new one work...or upload a patch file to the jira - but please don't do both; because in that case 2 execution will happen. The

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-05-27 Thread Zoltan Haindrich
Hello all! The new stuff is ready to be switched on-to. It needs to be merged into master - and after that anyone who opens a PR will get a run by the new HiveQA infra. I propose to run the 2 systems side-by-side for some time - the regular master builds will start; and we will see how

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-05-18 Thread Zoltan Haindrich
Hey! On 5/18/20 11:51 AM, Zoltan Chovan wrote: Thank you for all of your efforts, this looks really promising. With moving to github PRs, would that also mean that we move away from the reviewboard for code review? I didn't thinked about that. I think using github's review interface will remain

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-05-18 Thread Zoltan Chovan
Hi Zoltan! Thank you for all of your efforts, this looks really promising. With moving to github PRs, would that also mean that we move away from the reviewboard for code review? Also, what happens if a PR is updated? Will the tests run for both or just for the latest version? Regards, Zoltan

Re: [DISCUSS] Replace ptest with hive-test-kube

2020-05-17 Thread Zoltan Haindrich
Hello all! The proposed system have become more stable lately - and I think I've solved a few sources of flakiness. To be really usable I also wanted to add a way to dynamically enable/disable a set of tests (for example the replication tests take ~7 hours to execute from the total of 24 hours

[DISCUSS] Replace ptest with hive-test-kube

2020-04-29 Thread Zoltan Haindrich
Hey All! I was planning to replace the ptest stuff with something less complex for a while now - I see that we struggle a lot because of ptest is more complicated than it should be... It would be much better if it would be constructed from well made existing CI piece. - because of that I've