PR testing and flaky tests (triggering executions separately)

Hello everybody, A commit for a PR triggers three different test executions: - SparkPullRequestBuilder - SparkPullRequestBuilder-K8s - Github actions triggered checks Each one of these could contain a flaky test. *Is it already possible to trigger these executions separately?* If not

Re: minikube and kubernetes cluster versions for integration testing

Thanks Shane! As I promised: - the PR about documenting the change - my Spark PR with checking Minikube versions and using a simpler way to configure kubernetes client for integration testing - the Jira

Re: Shutdown cleanup of disk-based resources that Spark creates

same situation. There is clearly some demand for Spark to > automatically clean up checkpoints on shutdown. But perhaps that should > be... a new config? a rejected feature? something else? I dunno. > > Does anyone else have thoughts on how to approach this? > > On Wed, Mar 10

Re: Shutdown cleanup of disk-based resources that Spark creates

to pick up the checkpointed data. > > The use case you are describing potentially makes sense. But preserving > checkpoint data after an unexpected shutdown -- even when > `cleanCheckpoints` is set to true -- is a new guarantee that is not > currently expressed in the API or document

Re: Shutdown cleanup of disk-based resources that Spark creates

Hi Nick! I am not sure you are fixing a problem here. I think what you see is as problem is actually an intended behaviour. Checkpoint data should outlive the unexpected shutdowns. So there is a very important difference between the reference goes out of scope during a normal execution (in this c

minikube and kubernetes cluster versions for integration testing

Hi All, I am working on PR to change kubernetes integration testing and use the `minikube kubectl -- config view --minify` output to build the kubernetes client config. This solution has the advantage of not using hardcoded values like 8443 for server port (which is wrong when the vm-driver is doc

Re: Please use Jekyll via "bundle exec" from now on

Ls for the last versions of the markdown files) - whether the right version of Jekyll was used On Thu, Feb 18, 2021 at 10:17 AM Attila Zsolt Piros < piros.attila.zs...@gmail.com> wrote: > Hello everybody, > > To pin the exact same version of Jekyll across all the contributo

Please use Jekyll via "bundle exec" from now on

Hello everybody, To pin the exact same version of Jekyll across all the contributors, Ruby Bundler is introduced. This way the differences in the generated documentation, which were caused by using different Jekyll versions, are avoided. Regarding its usage this simply means an extra prefix "*bun