Re: Fwd: [GUMP@vmgump]: Project ooxml-schemas (in module poi) failed

[Stefan Bodewig]

On 2016-10-23, Dominik Stadler wrote:

> Thanks for applying this quickly and fixing the missing import.

You're welcome. Most likely the import wasn't strictly necessary as
imports sometimes seem to work transitively in Python - I'm in no way a
Python developer myself.

> The run now finished after 1 hour and 9 minutes, so it seems it took
> just a bit too much time before.

Yes, I've seen it. Glad it worked.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org
For additional commands, e-mail: dev-h...@poi.apache.org

Re: Fwd: [GUMP@vmgump]: Project ooxml-schemas (in module poi) failed

[Dominik Stadler]

Thanks for applying this quickly and fixing the missing import. The run now
finished after 1 hour and 9 minutes, so it seems it took just a bit too
much time before.

Thanks... Dominik

On Oct 21, 2016 5:31 PM, "Stefan Bodewig"  wrote:

> Hi Dominik
>
> On 2016-10-20, Dominik Stadler wrote:
>
> > Since some time one of the steps in the POI builds inside Gump is
> failing,
> > see below. We think it is related to the 1 hour timeout that is applied
> by
> > default.
>
> Yes, looks that way.
>
> > The step compiles a big number of files, but finishes in a few minutes
> > usually. Our builds on the Apache Jenkins instance usually take up to 40
> > minutes for the complete build which includes a number of additional
> steps,
> > so we are not sure why it is taking so long here, maybe the machine is
> > quite limited on memory/CPU?
>
> Yes, vmgump is a pretty limited. We are in the process of replacing it,
> but it may take a while until we get there.
>
> > Is there a simple way to try to increase the timeout for this specific
> > build?
>
> This is not implemented, although most of the infrastructure would be
> there.
>
> In case anybody with some working Python knowledge can spare a bit of
> time: https://svn.apache.org/repos/asf/gump/live/ holds the code.
>
> All builders create an instance of gump.util.process.Cmd and the timeout
> value could be configured via a constructor arg. For POI it would be
> enough to add an optional timeout property to
> gump.core.model.builder.Ant (likely Builder would be the better place)
> and use that from gump.core.build.Ant.getAntCommand - the builder
> instance is project.ant.
>
> I'm not sure how quick I'd be able to make the changes myself, I'm
> switching between a couple of tasks (including the new VM for gump)
> right now.
>
> Stefan
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org
> For additional commands, e-mail: dev-h...@poi.apache.org
>
>

Re: Fwd: [GUMP@vmgump]: Project ooxml-schemas (in module poi) failed

[Dominik Stadler]

Hi Andi,

Yes, you are right, this might be. But as it did work for a long time
before, and I think even sometimes now, I thought it might just exceed the
1 hour timeout by a small margin and so I wanted to try with a longer
timeout first before going the route of more investigation into the actual
build-times here.

Dominik.

On Sat, Oct 22, 2016 at 8:00 PM, Andreas Beeker 
wrote:

> Hi Dominik,
>
> we actually don't know which step of downloading/generating/compiling
> takes so long.
> I thought about echoing timestamps, so we can find the culprit.
> My wild guess is, that it is a memory limitation and it will simply
> garbage collect too much while compiling ...
>
> Andi
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org
> For additional commands, e-mail: dev-h...@poi.apache.org
>
>

Re: Fwd: [GUMP@vmgump]: Project ooxml-schemas (in module poi) failed

[Andreas Beeker]

Hi Dominik,

we actually don't know which step of downloading/generating/compiling takes so 
long.
I thought about echoing timestamps, so we can find the culprit.
My wild guess is, that it is a memory limitation and it will simply garbage 
collect too much while compiling ...

Andi


-
To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org
For additional commands, e-mail: dev-h...@poi.apache.org

Re: Fwd: [GUMP@vmgump]: Project ooxml-schemas (in module poi) failed

[Dominik Stadler]

Hi,

thanks for the hints, I added this locally, it seems to work when I run the
ooxml-schemas target. I am not a Python expert, but the changes seemed easy
enough, see attached diffs.

I had to split it into two files as the metadata directory is an
svn-external.

Regards... Dominik.

On Fri, Oct 21, 2016 at 5:31 PM, Stefan Bodewig  wrote:

> Hi Dominik
>
> On 2016-10-20, Dominik Stadler wrote:
>
> > Since some time one of the steps in the POI builds inside Gump is
> failing,
> > see below. We think it is related to the 1 hour timeout that is applied
> by
> > default.
>
> Yes, looks that way.
>
> > The step compiles a big number of files, but finishes in a few minutes
> > usually. Our builds on the Apache Jenkins instance usually take up to 40
> > minutes for the complete build which includes a number of additional
> steps,
> > so we are not sure why it is taking so long here, maybe the machine is
> > quite limited on memory/CPU?
>
> Yes, vmgump is a pretty limited. We are in the process of replacing it,
> but it may take a while until we get there.
>
> > Is there a simple way to try to increase the timeout for this specific
> > build?
>
> This is not implemented, although most of the infrastructure would be
> there.
>
> In case anybody with some working Python knowledge can spare a bit of
> time: https://svn.apache.org/repos/asf/gump/live/ holds the code.
>
> All builders create an instance of gump.util.process.Cmd and the timeout
> value could be configured via a constructor arg. For POI it would be
> enough to add an optional timeout property to
> gump.core.model.builder.Ant (likely Builder would be the better place)
> and use that from gump.core.build.Ant.getAntCommand - the builder
> instance is project.ant.
>
> I'm not sure how quick I'd be able to make the changes myself, I'm
> switching between a couple of tasks (including the new VM for gump)
> right now.
>
> Stefan
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org
> For additional commands, e-mail: dev-h...@poi.apache.org
>
>
Index: /opt/apache/gump-live/metadata/dtd/project.dtd
===
--- /opt/apache/gump-live/metadata/dtd/project.dtd	(Revision 1766220)
+++ /opt/apache/gump-live/metadata/dtd/project.dtd	(Arbeitskopie)
@@ -96,6 +96,7 @@
   basedir   CDATA #IMPLIED
   verbose   CDATA #IMPLIED
   debug CDATA #IMPLIED
+  timeout   CDATA #IMPLIED
   vmCDATA #IMPLIED >
 
 
 

-  
+  
 
   
 
Index: /opt/apache/gump-live/python/gump/core/build/ant.py
===
--- /opt/apache/gump-live/python/gump/core/build/ant.py	(Revision 1766220)
+++ /opt/apache/gump-live/python/gump/core/build/ant.py	(Arbeitskopie)
@@ -101,6 +101,12 @@
 # The ant build file (or none == build.xml)
 buildfile = ant.getBuildFile()
 
+# Optional 'timeout'
+if ant.hasTimeout:
+timeout = ant.getTimeout()
+else:
+timeout = setting.TIMEOUT
+
 # Optional 'verbose' or 'debug'
 verbose=ant.isVerbose()
 debug=ant.isDebug()
@@ -119,7 +125,7 @@

 # Run java on apache Ant...
 cmd=Cmd(javaCommand,'build_'+project.getModule().getName()+'_'+project.getName(),
-basedir,{'CLASSPATH':classpath})
+basedir,{'CLASSPATH':classpath}, timeout)
 
 # These are workspace + project system properties
 cmd.addNamedParameters(sysproperties)
Index: /opt/apache/gump-live/python/gump/core/model/builder.py
===
--- /opt/apache/gump-live/python/gump/core/model/builder.py	(Revision 1766220)
+++ /opt/apache/gump-live/python/gump/core/model/builder.py	(Arbeitskopie)
@@ -236,6 +236,10 @@
 self.target = self.getDomAttributeValue('target')
 # Import the buildfile
 self.buildfile = self.getDomAttributeValue('buildfile')
+# Import the timeout
+self.timeout = self.getDomAttributeValue('timeout')
+if self.timeout is not None:
+			self.timeout = float(self.timeout)
 
 def hasTarget(self):
 if self.target:
@@ -253,6 +257,14 @@
 def getBuildFile(self):
 return self.buildfile
 
+def hasTimeout(self):
+if self.timeout:
+return True
+return False
+
+def getTimeout(self):
+		return self.timeout
+
 def dump(self, indent = 0, output = sys.stdout):
 """ Display the contents of this object """
 Builder.dump(self, indent, output)
@@ -261,6 +273,8 @@
 output.write(i+'Target: ' + self.getTarget() + '\n')
 if self.hasBuildFile():
 output.write(i+'BuildFile: ' + self.getBuildFile() + '\n')
+if self.timeout():
+output.write(i+'Timeout: ' + self.getTimeout() + '\n')
 
 class Ant(BaseAnt): 
 

Re: Fwd: [GUMP@vmgump]: Project ooxml-schemas (in module poi) failed

[Stefan Bodewig]

Hi Dominik

On 2016-10-20, Dominik Stadler wrote:

> Since some time one of the steps in the POI builds inside Gump is failing,
> see below. We think it is related to the 1 hour timeout that is applied by
> default.

Yes, looks that way.

> The step compiles a big number of files, but finishes in a few minutes
> usually. Our builds on the Apache Jenkins instance usually take up to 40
> minutes for the complete build which includes a number of additional steps,
> so we are not sure why it is taking so long here, maybe the machine is
> quite limited on memory/CPU?

Yes, vmgump is a pretty limited. We are in the process of replacing it,
but it may take a while until we get there.

> Is there a simple way to try to increase the timeout for this specific
> build?

This is not implemented, although most of the infrastructure would be
there.

In case anybody with some working Python knowledge can spare a bit of
time: https://svn.apache.org/repos/asf/gump/live/ holds the code.

All builders create an instance of gump.util.process.Cmd and the timeout
value could be configured via a constructor arg. For POI it would be
enough to add an optional timeout property to
gump.core.model.builder.Ant (likely Builder would be the better place)
and use that from gump.core.build.Ant.getAntCommand - the builder
instance is project.ant.

I'm not sure how quick I'd be able to make the changes myself, I'm
switching between a couple of tasks (including the new VM for gump)
right now.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@poi.apache.org
For additional commands, e-mail: dev-h...@poi.apache.org