Re: ConcurrentModificationException saving artifacts to GCP

2020-10-29 Thread Ivan Fernandez Calvo 
Well, we really did not workaround the issue, it simply happens fewer 
times, the real solution will 
be https://github.com/jenkinsci/google-storage-plugin/pull/120

El jueves, 29 de octubre de 2020 a las 16:53:38 UTC+1, Ivan Fernandez Calvo 
escribió:

> We have workaround the issue by using a semaphore to call the GCP storage 
> step
>
>
> import groovy.transform.Field
> @Field def lock = false
>
> @NonCPS
> def setLock(){
>   synchronized(lock) {
> lock = true
>   }
> }
>
> @NonCPS
> def setUnlock(){
>   synchronized(lock) {
> lock = false
>   }
> }
>
> @NonCPS
> def isLock(){
>   synchronized(lock) {
> return lock
>   }
> }
>
> def waitForUnlock(){
>   while(isLock()){
> sleep 10
>   }
> }
>
> def uploadPackages(bucketUri, baseDir){
>   waitForUnlock()
>   setLock()
>   googleStorageUpload(bucket: bucketUri,
> credentialsId: "${JOB_GCS_CREDENTIALS}",
> pathPrefix: "${baseDir}/build/distributions/",
> pattern: "${baseDir}/build/distributions/**/*",
> sharedPublicly: true,
> showInline: true
>   )
>   setUnlock()
> }
> El miércoles, 28 de octubre de 2020 a las 22:34:59 UTC+1, Ivan Fernandez 
> Calvo escribió:
>
>> Hi,
>>
>> We are facing some issues with a job that has about 100 stages in 
>> parallel, we track down the issue and it happens when we storage artifacts 
>> on GCP in parallel, the thing is that I am not sure it is related to the 
>> plugin directly, Could someone take a look to this stack trace and confirm 
>> that it is not related to the Core? BTW we are using Jenkins 2.252 
>>
>> *19:12:54*  java.util.ConcurrentModificationException*19:12:54*  at 
>> java.util.HashMap$HashIterator.nextNode(HashMap.java:1445)*19:12:54* 
>> at java.util.HashMap$KeyIterator.next(HashMap.java:1469)*19:12:54*  at 
>> com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:73)*19:12:54*
>>at 
>> com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)*19:12:54*
>>  at 
>> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)*19:12:54*
>> at 
>> com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)*19:12:54*
>> at 
>> hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:264)*19:12:54*
>> at 
>> hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:251)*19:12:54*
>>   Caused: java.lang.RuntimeException: Failed to serialize 
>> com.google.jenkins.plugins.storage.reports.BuildGcsUploadReport#files for 
>> class 
>> com.google.jenkins.plugins.storage.reports.BuildGcsUploadReport*19:12:54*
>>   at 
>> hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:255)*19:12:54*
>>  at 
>> hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:223)*19:12:54*
>>   at 
>> com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)*19:12:54*
>>  at 
>> hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)*19:12:54*
>> at 
>> hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)*19:12:54*
>>   at 
>> com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)*19:12:54*
>>  at 
>> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)*19:12:54*
>> at 
>> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)*19:12:54*
>> at 
>> com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:88)*19:12:54*
>> at 
>> com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.writeItem(AbstractCollectionConverter.java:64)*19:12:54*
>>  at 
>> com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:74)*19:12:54*
>>at 
>> com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)*19:12:54*
>>  at 
>> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)*19:12:54*
>> at 
>> com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)*19:12:54*
>> at 
>> hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:264)*19:12:54*
>> at 
>> hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:251)*19:12:54*
>>   Caused: java.lang.RuntimeException: Failed to serialize 
>> hudson.model.Actionable#actions for class 
>> org.jenkinsci.plugins.workflow.job.WorkflowRun*19:12:54* at 
>> hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:255)*19:12:54*
>>

Re: ConcurrentModificationException saving artifacts to GCP

2020-10-29 Thread Ivan Fernandez Calvo 
We have workaround the issue by using a semaphore to call the GCP storage 
step


import groovy.transform.Field
@Field def lock = false

@NonCPS
def setLock(){
  synchronized(lock) {
lock = true
  }
}

@NonCPS
def setUnlock(){
  synchronized(lock) {
lock = false
  }
}

@NonCPS
def isLock(){
  synchronized(lock) {
return lock
  }
}

def waitForUnlock(){
  while(isLock()){
sleep 10
  }
}

def uploadPackages(bucketUri, baseDir){
  waitForUnlock()
  setLock()
  googleStorageUpload(bucket: bucketUri,
credentialsId: "${JOB_GCS_CREDENTIALS}",
pathPrefix: "${baseDir}/build/distributions/",
pattern: "${baseDir}/build/distributions/**/*",
sharedPublicly: true,
showInline: true
  )
  setUnlock()
}
El miércoles, 28 de octubre de 2020 a las 22:34:59 UTC+1, Ivan Fernandez 
Calvo escribió:

> Hi,
>
> We are facing some issues with a job that has about 100 stages in 
> parallel, we track down the issue and it happens when we storage artifacts 
> on GCP in parallel, the thing is that I am not sure it is related to the 
> plugin directly, Could someone take a look to this stack trace and confirm 
> that it is not related to the Core? BTW we are using Jenkins 2.252 
>
> *19:12:54*  java.util.ConcurrentModificationException*19:12:54*   at 
> java.util.HashMap$HashIterator.nextNode(HashMap.java:1445)*19:12:54* 
> at java.util.HashMap$KeyIterator.next(HashMap.java:1469)*19:12:54*  at 
> com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:73)*19:12:54*
>at 
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)*19:12:54*
>  at 
> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)*19:12:54*
> at 
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)*19:12:54*
> at 
> hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:264)*19:12:54*
> at 
> hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:251)*19:12:54*
>   Caused: java.lang.RuntimeException: Failed to serialize 
> com.google.jenkins.plugins.storage.reports.BuildGcsUploadReport#files for 
> class 
> com.google.jenkins.plugins.storage.reports.BuildGcsUploadReport*19:12:54* 
>  at 
> hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:255)*19:12:54*
>  at 
> hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:223)*19:12:54*
>   at 
> com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)*19:12:54*
>  at 
> hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)*19:12:54*
> at 
> hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)*19:12:54*
>   at 
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)*19:12:54*
>  at 
> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)*19:12:54*
> at 
> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)*19:12:54*
> at 
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:88)*19:12:54*
> at 
> com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.writeItem(AbstractCollectionConverter.java:64)*19:12:54*
>  at 
> com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:74)*19:12:54*
>at 
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)*19:12:54*
>  at 
> com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)*19:12:54*
> at 
> com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)*19:12:54*
> at 
> hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:264)*19:12:54*
> at 
> hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:251)*19:12:54*
>   Caused: java.lang.RuntimeException: Failed to serialize 
> hudson.model.Actionable#actions for class 
> org.jenkinsci.plugins.workflow.job.WorkflowRun*19:12:54* at 
> hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:255)*19:12:54*
>  at 
> hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:223)*19:12:54*
>   at 
> com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)*19:12:54*
>  at 
> hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)*19:12:54*
> at 
> hudson.util.Robus

ConcurrentModificationException saving artifacts to GCP

2020-10-28 Thread kuisathaverat 
Hi,

We are facing some issues with a job that has about 100 stages in parallel,
we track down the issue and it happens when we storage artifacts on GCP in
parallel, the thing is that I am not sure it is related to the plugin
directly, Could someone take a look to this stack trace and confirm that it
is not related to the Core? BTW we are using Jenkins 2.252

*19:12:54*  java.util.ConcurrentModificationException*19:12:54* at
java.util.HashMap$HashIterator.nextNode(HashMap.java:1445)*19:12:54*
at java.util.HashMap$KeyIterator.next(HashMap.java:1469)*19:12:54*
at 
com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:73)*19:12:54*
at 
com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)*19:12:54*
at 
com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)*19:12:54*
at 
com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)*19:12:54*
at 
hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:264)*19:12:54*
at 
hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:251)*19:12:54*
 Caused: java.lang.RuntimeException: Failed to serialize
com.google.jenkins.plugins.storage.reports.BuildGcsUploadReport#files
for class 
com.google.jenkins.plugins.storage.reports.BuildGcsUploadReport*19:12:54*
at 
hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:255)*19:12:54*
at 
hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:223)*19:12:54*
at 
com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)*19:12:54*
at 
hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)*19:12:54*
at 
hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)*19:12:54*
at 
com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)*19:12:54*
at 
com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)*19:12:54*
at 
com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)*19:12:54*
at 
com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:88)*19:12:54*
at 
com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.writeItem(AbstractCollectionConverter.java:64)*19:12:54*
at 
com.thoughtworks.xstream.converters.collections.CollectionConverter.marshal(CollectionConverter.java:74)*19:12:54*
at 
com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)*19:12:54*
at 
com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)*19:12:54*
at 
com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84)*19:12:54*
at 
hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:264)*19:12:54*
at 
hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:251)*19:12:54*
 Caused: java.lang.RuntimeException: Failed to serialize
hudson.model.Actionable#actions for class
org.jenkinsci.plugins.workflow.job.WorkflowRun*19:12:54*at
hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:255)*19:12:54*
at 
hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:223)*19:12:54*
at 
com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:138)*19:12:54*
at 
hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:209)*19:12:54*
at 
hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:150)*19:12:54*
at 
com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69)*19:12:54*
at 
com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:58)*19:12:54*
at 
com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:43)*19:12:54*
at 
com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:82)*19:12:54*
at 
com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)*19:12:54*
at com.thoughtworks.xstream.XStream.marshal(XStream.java:1026)*19:12:54*
at com.thoughtworks.xstream.XStream.marshal(XStream.java:1015)*19:12:54*
at com.thoughtworks.xstream.XStream.toXML(XStream.java:988)*19:12:54*
at hudson.util.XStream2.toXMLUTF8(XStre