On Wed, Jan 13, 2016 at 3:49 PM, Niels de Vos <[email protected]> wrote:
> On Wed, Jan 13, 2016 at 10:35:42AM +0100, Xavier Hernandez wrote: > > The same has happened to slave34.cloud.gluster.org. I've disabled it to > > allow regressions to be run on other slaves. > > > > There are two files owned by root inside > > /home/jenkins/root/workspace/rackspace-regression-2GB-triggered: > > > > -rwxr-xr-x 1 root root 10124 Jan 7 17:54 file_lock > > drwxr-xr-x 3 root root 4096 Jan 7 18:31 > slave34.cloud.gluster.org: > > Thanks! > > I've looked into this a little more now, and might have identified the > problem. > > This one failed with an unrelated error: > > > https://build.gluster.org/job/rackspace-regression-2GB-triggered/17413/console > > ... > Building remotely on slave34.cloud.gluster.org > (rackspace_regression_2gb) in workspace > /home/jenkins/root/workspace/rackspace-regression-2GB-triggered > > git rev-parse --is-inside-work-tree # timeout=10 > Fetching changes from the remote Git repository > > git config remote.origin.url git://review.gluster.org/glusterfs.git > # timeout=10 > Fetching upstream changes from git://review.gluster.org/glusterfs.git > ... > > The next run on slave34 failed because of the weird directory: > > > https://build.gluster.org/job/rackspace-regression-2GB-triggered/17440/console > > ... > Building remotely on slave34.cloud.gluster.org > (rackspace_regression_2gb) in workspace > /home/jenkins/root/workspace/rackspace-regression-2GB-triggered > Wiping out workspace first. > java.io.IOException: remote file operation failed: > /home/jenkins/root/workspace/rackspace-regression-2GB-triggered at > hudson.remoting.Channel@62ecdacb:slave34.cloud.gluster.org: > ... > > Note the "Wiping out workspace first." line. This comes from an option > in the regression job. This seems to be a recently added "Additional > Behaviour" in the Jenkins job. Did anyone add this on purpose, or was > that automatically done with a Jenkins update or something? > The three additional behaviour added in regression configuration were added by me and I simply copied whatever was there in smoke configuration page. We can try removing this configuration line but the tests weren't getting started without this(we got it running after a restart so this might be false requirement). Technically, it is not a harmful configuration and wiki pages recommend it. It says such errors occur only if files were created and left open/locked by some tests or were created with different permissions. We still need to identify what tests are responsible for this. > > Niels > > > > > Xavi > > > > On 12/01/16 12:06, Niels de Vos wrote: > > >Hi, > > > > > >I've disabled slave32.cloud.gluster.org because it failed multiple > > >regression tests with a weird error. After disabling slave32 and > > >retriggering the failed run, the same job executed fine on a different > > >slave. > > > > > >The affected directory is owned by root, so the jenkins user is not > > >allowed to wipe it. Does anyone know how this could happen? The dirname > > >is rather awkward too... > > > > > > > > /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/slave32.cloud.gluster.org: > /d > > > > > >I think we can just remove that dir and the slave can be enabled again. > > >Leaving the status as is for further investigation. > > > > > >Thanks, > > >Niels > > > > > > > > >Full error: > > > > > > Wiping out workspace first. > > > java.io.IOException: remote file operation failed: > /home/jenkins/root/workspace/rackspace-regression-2GB-triggered at > hudson.remoting.Channel@7bc1e07d:slave32.cloud.gluster.org: > java.nio.file.AccessDeniedException: > /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/slave32.cloud.gluster.org: > /d > > > at hudson.FilePath.act(FilePath.java:986) > > > at hudson.FilePath.act(FilePath.java:968) > > > at hudson.FilePath.deleteContents(FilePath.java:1183) > > > at > hudson.plugins.git.extensions.impl.WipeWorkspace.beforeCheckout(WipeWorkspace.java:28) > > > at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1040) > > > at hudson.scm.SCM.checkout(SCM.java:485) > > > at > hudson.model.AbstractProject.checkout(AbstractProject.java:1276) > > > at > hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:607) > > > at > jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86) > > > at > hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:529) > > > at hudson.model.Run.execute(Run.java:1738) > > > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) > > > at > hudson.model.ResourceController.execute(ResourceController.java:98) > > > at hudson.model.Executor.run(Executor.java:410) > > > Caused by: java.nio.file.AccessDeniedException: > /home/jenkins/root/workspace/rackspace-regression-2GB-triggered/slave32.cloud.gluster.org: > /d > > > at > sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) > > > at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) > > > at > sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) > > > at > sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244) > > > at > sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103) > > > at java.nio.file.Files.delete(Files.java:1079) > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > at java.lang.reflect.Method.invoke(Method.java:606) > > > at hudson.Util.deleteFile(Util.java:255) > > > at hudson.FilePath.deleteRecursive(FilePath.java:1203) > > > at > hudson.FilePath.deleteContentsRecursive(FilePath.java:1212) > > > at hudson.FilePath.deleteRecursive(FilePath.java:1194) > > > at > hudson.FilePath.deleteContentsRecursive(FilePath.java:1212) > > > at hudson.FilePath.access$1100(FilePath.java:190) > > > at hudson.FilePath$15.invoke(FilePath.java:1186) > > > at hudson.FilePath$15.invoke(FilePath.java:1183) > > > at > hudson.FilePath$FileCallableWrapper.call(FilePath.java:2719) > > > at > hudson.remoting.UserRequest.perform(UserRequest.java:120) > > > at hudson.remoting.UserRequest.perform(UserRequest.java:48) > > > at hudson.remoting.Request$2.run(Request.java:326) > > > at > hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68) > > > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > at java.lang.Thread.run(Thread.java:745) > > > at ......remote call to slave32.cloud.gluster.org(Native > Method) > > > at > hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1416) > > > at > hudson.remoting.UserResponse.retrieve(UserRequest.java:220) > > > at hudson.remoting.Channel.call(Channel.java:781) > > > at hudson.FilePath.act(FilePath.java:979) > > > ... 13 more > > > Finished: FAILURE > > > > > > > > > > > >_______________________________________________ > > >Gluster-infra mailing list > > >[email protected] > > >http://www.gluster.org/mailman/listinfo/gluster-infra > > > >
_______________________________________________ Gluster-infra mailing list [email protected] http://www.gluster.org/mailman/listinfo/gluster-infra
