Thank you, Steve and Hyukjin. And, don't worry, Ted.
Travis launches new VMs for every PR. Apache Spark repository uses the following setting. VM: Google Compute Engine OS: Ubuntu 14.04.3 LTS Server Edition 64bit CPU: ~2 CORE RAM: 7.5GB FYI, you can find more information about this here. https://docs.travis-ci.com/user/ci-environment/#Virtualization-environments Dongjoon. On Mon, May 23, 2016 at 6:32 AM, Ted Yu <yuzhih...@gmail.com> wrote: > Do you know if more than one PR would be verified on the same machine ? > > I wonder whether the 'mvn install' from two simultaneous PR builds may > have conflict. > > On Sun, May 22, 2016 at 9:21 PM, Dongjoon Hyun <dongj...@apache.org> > wrote: > >> Thank you for feedback. Sure, correctly, that's the reason why the >> current SparkPullRequestBuilder do not run `lint-java`. :-) >> >> In addition, that's the same reason why contributors are reluctant to run >> `lint-java` and causes breaking on JDK7 builds. >> >> Such a tedious and time-consuming job should be done by CI without human >> interventions. >> >> By the way, why do you think we need to wait for that? We should not wait >> for any CIs, we should continue our own work. >> >> My proposal isn't for making you wait to watch the result. There are two >> use cases I want for us to focus here. >> >> Case 1: When you make a PR to Spark PR queue. >> >> Travis CI will finish before SparkPullRequestBuilder. >> We will run the followings in parallel mode. >> 1. Current SparkPullRequestBuilder: JDK8 + sbt build + (no Java >> Linter) >> 2. Travis: JDK7 + mvn build + Java Linter >> 3. Travis: JDK8 + mvn build + Java Linter >> As we know, 1 is the longest time-consuming one which have lots of >> works (except maven building or lint- java). You don't need to wait more >> in many cases. Yes, in many cases, not all the cases. >> >> >> Case 2: When you prepare a PR on your branch. >> >> If you are at the final commit (maybe already-squashed), just go to >> case 1. >> >> However, usually, we makes lots of commits locally while making >> preparing our PR. >> And, finally we squashed them into one and send a PR to Spark. >> I mean you can use Travis CI during preparing your PRs. >> Again, don't wait for Travis CI. Just push it sometime or at every >> commit, and continue your work. >> >> At the final stage when you finish your coding, squash your commits >> into one, >> and amend your commit title or messages, see the Travis CI. >> Or, you can monitor Travis CI result on status menu bar. >> If it shows green icon, you have nothing to do. >> >> https://docs.travis-ci.com/user/apps/ >> >> To sum up, I think we don't need to wait for any CIs. It's like an email. >> `Send and back to work.` >> >> Dongjoon. >> >> >> On Sun, May 22, 2016 at 8:32 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> >>> Without Zinc, 'mvn -DskipTests clean install' takes ~30 minutes. >>> >>> Maybe not everyone is willing to wait that long. >>> >>> On Sun, May 22, 2016 at 1:30 PM, Dongjoon Hyun <dongj...@apache.org> >>> wrote: >>> >>>> Oh, Sure. My bad! >>>> >>>> - For Oracle JDK7, mvn -DskipTests install and run `dev/lint-java`. >>>> - For Oracle JDK8, mvn -DskipTests install and run `dev/lint-java`. >>>> >>>> Thank you, Ted. >>>> >>>> Dongjoon. >>>> >>>> On Sun, May 22, 2016 at 1:29 PM, Ted Yu <yuzhih...@gmail.com> wrote: >>>> >>>>> The following line was repeated twice: >>>>> >>>>> - For Oracle JDK7, mvn -DskipTests install and run `dev/lint-java`. >>>>> >>>>> Did you intend to cover JDK 8 ? >>>>> >>>>> Cheers >>>>> >>>>> On Sun, May 22, 2016 at 1:25 PM, Dongjoon Hyun <dongj...@apache.org> >>>>> wrote: >>>>> >>>>>> Hi, All. >>>>>> >>>>>> I want to propose the followings. >>>>>> >>>>>> - Turn on Travis CI for Apache Spark PR queue. >>>>>> - Recommend this for contributors, too >>>>>> >>>>>> Currently, Spark provides Travis CI configuration file to help >>>>>> contributors check Scala/Java style conformance and JDK7/8 compilation >>>>>> easily during their preparing pull requests. Please note that it's only >>>>>> about static analysis. >>>>>> >>>>>> - For Oracle JDK7, mvn -DskipTests install and run `dev/lint-java`. >>>>>> - For Oracle JDK7, mvn -DskipTests install and run `dev/lint-java`. >>>>>> Scalastyle is included in the step 'mvn install', too. >>>>>> >>>>>> Yep, if you turn on your Travis CI configuration, you can already see >>>>>> the results on your branches before making PR. I wrote this email to >>>>>> prevent more failures proactively and community-widely. >>>>>> >>>>>> For stability, I have been monitoring that for two weeks. It detects >>>>>> the failures or recovery on JDK7 builds or Java linter on Spark master >>>>>> branch correctly. The only exceptional case I observed rarely is >>>>>> `timeout` >>>>>> failure due to hangs of maven. But, as we know, it's happen in our >>>>>> Jenkins >>>>>> SparkPullRequestBuilder, too. I think we can ignore that. >>>>>> >>>>>> I'm sure that this will save much more community's efforts on the >>>>>> static errors by preventing them at the very early stage. But, there >>>>>> might >>>>>> be another reason not to do this. I'm wondering about your thoughts. >>>>>> >>>>>> I can make a Apache INFRA Jira issue for this if there is some >>>>>> consensus. >>>>>> >>>>>> Warmly, >>>>>> Dongjoon. >>>>>> >>>>> >>>>> >>>> >>> >> >
