Jenkins build is back to normal : UIMA-DUCC #1317

[Apache Jenkins Server]

See 

[jira] [Closed] (UIMA-3476) Automate version mgmt in DUCCs README and RELEASE_NOTES

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik closed UIMA-3476.
---
Resolution: Fixed

Added ${project.version} placeholder in both README and RELEASE_NOTES.html to 
automated version updates when building DUCC. 

Added new plugin to do the replace and copy updated files to target. Modified 
assembly to pull both files from target.

> Automate version mgmt in DUCCs README and RELEASE_NOTES
> ---
>
> Key: UIMA-3476
> URL: https://issues.apache.org/jira/browse/UIMA-3476
> Project: UIMA
>  Issue Type: Improvement
>  Components: DUCC
>Reporter: Jerry Cwiklik
>Assignee: Jerry Cwiklik
>Priority: Minor
> Fix For: 2.2.2-Ducc
>
>
> Currently, DUCCs README and RELEASE_NOTES contain DUCCs version as static 
> text. This becomes a chore to maintain going forward. Find a way to automate 
> version mgmt in both of these files.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (UIMA-3476) Automate version mgmt in DUCCs README and RELEASE_NOTES

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik updated UIMA-3476:

Fix Version/s: (was: future-DUCC)
   2.2.2-Ducc

> Automate version mgmt in DUCCs README and RELEASE_NOTES
> ---
>
> Key: UIMA-3476
> URL: https://issues.apache.org/jira/browse/UIMA-3476
> Project: UIMA
>  Issue Type: Improvement
>  Components: DUCC
>Reporter: Jerry Cwiklik
>Assignee: Jerry Cwiklik
>Priority: Minor
> Fix For: 2.2.2-Ducc
>
>
> Currently, DUCCs README and RELEASE_NOTES contain DUCCs version as static 
> text. This becomes a chore to maintain going forward. Find a way to automate 
> version mgmt in both of these files.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (UIMA-4524) DUCC build produces javadoc warnings

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-4524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik updated UIMA-4524:

Fix Version/s: (was: future-DUCC)
   2.2.2-Ducc

> DUCC build produces javadoc warnings 
> -
>
> Key: UIMA-4524
> URL: https://issues.apache.org/jira/browse/UIMA-4524
> Project: UIMA
>  Issue Type: Bug
>  Components: DUCC
>Reporter: Jerry Cwiklik
>Assignee: Jerry Cwiklik
>Priority: Major
> Fix For: 2.2.2-Ducc
>
>
> When building DUCC, the javadoc produces warnings for misspelled things like 
> @returns instead of @return, parameters, etc. Seems like most DUCC projects 
> show this problem. We should clean these up.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (UIMA-3479) DUCC assembly script should fetch saxon from maven repo instead of svn

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-3479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik closed UIMA-3479.
---
Resolution: Won't Fix

Future UIMA-AS 3.0 will not use saxon. Dont see a need to fix this

> DUCC assembly script should fetch saxon from maven repo instead of svn
> --
>
> Key: UIMA-3479
> URL: https://issues.apache.org/jira/browse/UIMA-3479
> Project: UIMA
>  Issue Type: Bug
>  Components: DUCC
>Reporter: Jerry Cwiklik
>Assignee: Jerry Cwiklik
>Priority: Minor
> Fix For: future-DUCC
>
>
> Currently saxon jar is stored in the svn. Since saxon is *now* in maven repo 
> we should remove saxon from svn and fetch it from the repo. Modify Ducc's 
> assembly script and parent pom.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (UIMA-3817) Refactor Ducc Agent to collect process metrics from ps command not from /proc filesystem

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-3817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik closed UIMA-3817.
---
Resolution: Duplicate

Duplicate of UIMA-5245

> Refactor Ducc Agent to collect process metrics from ps command not from /proc 
> filesystem
> 
>
> Key: UIMA-3817
> URL: https://issues.apache.org/jira/browse/UIMA-3817
> Project: UIMA
>  Issue Type: Bug
>  Components: DUCC
>Affects Versions: 1.0.0-Ducc
>Reporter: Jerry Cwiklik
>Assignee: Jerry Cwiklik
>Priority: Major
> Fix For: future-DUCC
>
>
> Looks like the current approach for collecting process metrics (CPU, RSS, 
> SWAP) is not portable across different flavors of Linux. The agent currently 
> collects some of the process stats from /proc//statm file but it looks 
> like the syntax of this file is OS specific thus current approach is not 
> working right. 
> There is some evidence that on RedHat the stats for RSS and Swap are wrong. 
> Change agent's code to collect process stats from ps command



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (UIMA-4332) Job process termination on Java Exception should be an option

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-4332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik closed UIMA-4332.
---
Resolution: Duplicate

Duplicate of UIMA-5700

> Job process termination on Java Exception should be an option
> -
>
> Key: UIMA-4332
> URL: https://issues.apache.org/jira/browse/UIMA-4332
> Project: UIMA
>  Issue Type: Bug
>  Components: DUCC
>Reporter: Eddie Epstein
>Assignee: Jerry Cwiklik
>Priority: Major
> Fix For: future-DUCC
>
>
> In DUCC v1.x any uncaught exception thrown in user's Job Process code causes 
> the JP to be terminated and any other work items concurrently being processed 
> in the JP to be resubmitted. In the DUCC v2 design the decision can and 
> should be moved to the job's error handler running in the Job Driver. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (UIMA-4901) UIMA DUCC: Improve process cleanup code on deallocate request

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik closed UIMA-4901.
---
Resolution: Duplicate

Duplicate of UIMA-5611

> UIMA DUCC: Improve process cleanup code on deallocate request
> -
>
> Key: UIMA-4901
> URL: https://issues.apache.org/jira/browse/UIMA-4901
> Project: UIMA
>  Issue Type: Improvement
>  Components: DUCC
>Reporter: Jerry Cwiklik
>Assignee: Jerry Cwiklik
>Priority: Major
> Fix For: future-DUCC
>
>
> There is a suggestion to improve process cleanup strategy. On deallocate 
> request, the agent should cleanup owner of the  container along with all 
> dependent processes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (UIMA-3890) DUCC agent rogue process detector doesnt call cleanup script at consistent intervals

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-3890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik closed UIMA-3890.
---
Resolution: Duplicate

Duplicate UIMA-4907

> DUCC agent rogue process detector doesnt call cleanup script at consistent 
> intervals
> 
>
> Key: UIMA-3890
> URL: https://issues.apache.org/jira/browse/UIMA-3890
> Project: UIMA
>  Issue Type: Bug
>  Components: DUCC
>Affects Versions: 1.0.0-Ducc
>Reporter: Jerry Cwiklik
>Assignee: Jerry Cwiklik
>Priority: Minor
> Fix For: future-DUCC
>
>
> It was reported that the Ducc agent rogue process detector does not call 
> configured clean up script at regular intervals. For some PIDs it is 
> consistent while for other it is not. Review the code to see why there are 
> delays and find the cause of inconsistency.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (UIMA-5246) Serialize re-registration of DUCC services to reduce chance of duplicate services

[Burn Lewis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Burn Lewis resolved UIMA-5246.
--
Resolution: Duplicate

The fix in 5372 has done the trick ... no more problems observed

> Serialize re-registration of DUCC services to reduce chance  of duplicate 
> services
> --
>
> Key: UIMA-5246
> URL: https://issues.apache.org/jira/browse/UIMA-5246
> Project: UIMA
>  Issue Type: Bug
>  Components: DUCC
>Reporter: Burn Lewis
>Assignee: Burn Lewis
>Priority: Major
> Fix For: future-DUCC
>
>
> After an unregister + register ducc-mon has sometimes shown 2 service with 
> the same endpoint.  Apparently the older unregistered one was not marked as 
> "archived" in the DB.  Currently new instances of the service may be created 
> before all of the old ones have been removed - would be better if the old 
> service is completely removed before the new instances created.
> Should also handle failures to update the DB.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (UIMA-5246) Serialize re-registration of DUCC services to reduce chance of duplicate services

[Burn Lewis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Burn Lewis closed UIMA-5246.


> Serialize re-registration of DUCC services to reduce chance  of duplicate 
> services
> --
>
> Key: UIMA-5246
> URL: https://issues.apache.org/jira/browse/UIMA-5246
> Project: UIMA
>  Issue Type: Bug
>  Components: DUCC
>Reporter: Burn Lewis
>Assignee: Burn Lewis
>Priority: Major
> Fix For: future-DUCC
>
>
> After an unregister + register ducc-mon has sometimes shown 2 service with 
> the same endpoint.  Apparently the older unregistered one was not marked as 
> "archived" in the DB.  Currently new instances of the service may be created 
> before all of the old ones have been removed - would be better if the old 
> service is completely removed before the new instances created.
> Should also handle failures to update the DB.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (UIMA-5037) UIMA-DUCC: Agent not reporting child process GC sometimes

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik closed UIMA-5037.
---
Resolution: Duplicate

Duplicate of UIMA-5537

> UIMA-DUCC: Agent not reporting child process GC sometimes
> -
>
> Key: UIMA-5037
> URL: https://issues.apache.org/jira/browse/UIMA-5037
> Project: UIMA
>  Issue Type: Bug
>  Components: DUCC
>Reporter: Jerry Cwiklik
>Assignee: Jerry Cwiklik
>Priority: Major
> Fix For: future-DUCC
>
>
> Looks like another race in agent process metrics gathering. This time, for 
> some processes the garbage collection metrics are missing. An agent collects 
> these by connecting to a remote process using jmx and querring the MBean 
> server. 
> Looking at the agent's log it appears that the class responsible for GC 
> metrics gathering is initialized before the remote process sends its first 
> status update which includes jmx connect string. If the jmx string is missing 
> at the time of class initialization, GC metrics are not collected.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: UIMA-DUCC #1316

[Apache Jenkins Server]

See 

--
Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on H22 (ubuntu xenial) in workspace 

java.io.IOException: Failed to mkdirs: 

at hudson.FilePath.mkdirs(FilePath.java:1170)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1200)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
Retrying after 10 seconds
java.io.IOException: Failed to mkdirs: 

at hudson.FilePath.mkdirs(FilePath.java:1170)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1200)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
Retrying after 10 seconds
java.io.IOException: Failed to mkdirs: 

at hudson.FilePath.mkdirs(FilePath.java:1170)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1200)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:574)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1724)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
[WARNINGS] Skipping publisher since build result is FAILURE

[jira] [Updated] (UIMA-5527) UIMA-DUCC: require java 1.8 for builds and runtime

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-5527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik updated UIMA-5527:

Fix Version/s: (was: future-DUCC)
   2.2.2-Ducc

> UIMA-DUCC: require java 1.8 for builds and runtime
> --
>
> Key: UIMA-5527
> URL: https://issues.apache.org/jira/browse/UIMA-5527
> Project: UIMA
>  Issue Type: Task
>  Components: DUCC
>Reporter: Jerry Cwiklik
>Assignee: Jerry Cwiklik
>Priority: Major
> Fix For: 2.2.2-Ducc
>
>
> Currently DUCC requires java 7 for maven builds. This version of java is 
> quite old. We should 
>  move up DUCC's java version to java 8 at minimum. Besides pom changes, the 
> DUCC's javadoc method headers must be changed to comply with a stricter 
> syntax checking in version 8. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (UIMA-5727) UIMA-DUCC: fix XStream warning msgs

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik closed UIMA-5727.
---
Resolution: Fixed

Modified to reset xstream to avoid warning msgs on stdout

> UIMA-DUCC: fix XStream warning msgs
> ---
>
> Key: UIMA-5727
> URL: https://issues.apache.org/jira/browse/UIMA-5727
> Project: UIMA
>  Issue Type: Bug
>  Components: DUCC
>Reporter: Jerry Cwiklik
>Assignee: Jerry Cwiklik
>Priority: Major
> Fix For: 2.2.2-Ducc
>
>
> After upgrading xstream to 1.4.10 (bundled with AMQ 5.15.2) msgs are dumped 
> to stdout when running various ducc things:
> "Security framework of XStream not initialized, XStream is probably 
> vulnerable."
> Seeing these when running ducc_submit. Also in JD log. The new XStream is 
> configured by default  to run without security but dumps the above every time 
> xml serialization/deserialization is done. All is working fine except for 
> these warning msgs.
> The simplest way to fix that is to override XStream defaults and to whitelist 
> everything. I actually tried that by changing XStreamUtils and 
> DuccEventHttpDispatcherCl. No more annoying msgs.
> Perhaps a better (more secure way) is to white list specific classes/packages 
> when serializing/deserializing ducc msgs. This may take time to get it right. 
> We need to list all types which are allowed including java classes. I think 
> we only serialize DUCC classes (event classes) + java primitives + java 
> collections (Map, Lists, etc)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (UIMA-5726) UIMA-DUCC: JD TestSuite fails in checkCas()

[Jerry Cwiklik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-5726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jerry Cwiklik closed UIMA-5726.
---
Resolution: Fixed

Fixed checkCas in JD TestSuite to accept new CAS serialization format

> UIMA-DUCC: JD TestSuite fails in checkCas() 
> 
>
> Key: UIMA-5726
> URL: https://issues.apache.org/jira/browse/UIMA-5726
> Project: UIMA
>  Issue Type: Bug
>  Components: DUCC
>Reporter: Jerry Cwiklik
>Assignee: Jerry Cwiklik
>Priority: Major
> Fix For: 2.2.2-Ducc
>
>
> After upgrading ducc to use uima 2.10.2, the ducc build fails while running 
> JD junit tests. The reported error is:
> test_16(org.apache.uima.ducc.container.jd.test.Test1) Time elapsed: 0.326 sec 
> <<< FAILURE!
> java.lang.AssertionError: cas = ' encoding="UTF-8"?>http://www.omg.org/XMI; 
> xmlns:examples="http:///org/apache/uima/examples.ecore; 
> xmlns:tcas="http:///uima/tcas.ecore; xmlns:cas="http:///uima/cas.ecore; 
> xmi:version="2.0"> sofa="1" begin="0" end="1" language="x-unspecified"/> sofaNum="1" sofaID="_InitialView" mimeType="text" sofaString="1"/> sofa="1" members="8"/>'
>  at org.junit.Assert.fail(Assert.java:91)
>  at 
> org.apache.uima.ducc.container.jd.test.TestSuite.checkCas(TestSuite.java:43)
>  at 
> org.apache.uima.ducc.container.jd.test.TestSuite.getMetaCases(TestSuite.java:161)
>  at 
> org.apache.uima.ducc.container.jd.test.TestSuite.getMetaCases(TestSuite.java:174)
>  at org.apache.uima.ducc.container.jd.test.Test1.test_16(Test1.java:71)
>  
> The TestSuite.checkCas() expects either of these 3 xml preambles:
>   String prefix2 = " xmlns:examples=\"http:///org/apache/uima/examples.ecore\";;
>  String prefix1 = " xmlns:tcas=\"http:///uima/tcas.ecore\";;
>  String prefix0 = " xmlns:cas=\"http:///uima/cas.ecore\";;
>  
> It looks like CAS serialization produces a different preamble:
>  xmlns:xmi="http://www.omg.org/XMI; 
> xmlns:examples="http:///org/apache/uima/examples.ecore; 
> xmlns:tcas="http:///uima/tcas.ecore; xmlns:cas="http:///uima/cas.ecore; 
> xmi:version="2.0">
> Not sure about the cause of this. Perhaps the new uima (2.10.2) is 
> serializing differently now. I was able to fix the problem by adding a new 
> prefix (prefix3) to make sure the new xml preamble matches. Perhaps I am 
> fixing the wrong thing here. Why there is this preamble different from before 
> is the question. Looks like code doing the CAS serialization is producing a 
> different xml. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (UIMA-5647) %CPU always reports 0 for managed reservations

[Lou DeGenaro (JIRA)]

 [ 
https://issues.apache.org/jira/browse/UIMA-5647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lou DeGenaro closed UIMA-5647.
--

> %CPU always reports 0 for managed reservations
> --
>
> Key: UIMA-5647
> URL: https://issues.apache.org/jira/browse/UIMA-5647
> Project: UIMA
>  Issue Type: Bug
>  Components: DUCC
>Affects Versions: 2.2.1-Ducc
>Reporter: Eddie Epstein
>Assignee: Lou DeGenaro
>Priority: Major
> Fix For: 2.2.2-Ducc
>
>
> %CPU reporting works for service and job processes, but not managed 
> reservations.
> To reproduce the problem, create a bash script, say eatcpu.sh:
> #!/bin/bash
> while true; do for f in $(seq 1 10); do echo 555 > /dev/null; done; done;
> and then run it viaducc:
> $DUCC_HOME/bin/viaducc eatcpu.sh
> The duccmon reservation details page always shows %CPU=0, but top shows 
> something just under 100%



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)