[jira] [Commented] (OODT-1003) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/OODT-1003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16713785#comment-16713785 ] ASF GitHub Bot commented on OODT-1003: -- lewismc commented on issue #89: OODT-1003 Upgrade Apache parent POM to version 21 URL: https://github.com/apache/oodt/pull/89#issuecomment-445489145 Any objections to merge? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade Apache parent POM to version 21 > --- > > Key: OODT-1003 > URL: https://issues.apache.org/jira/browse/OODT-1003 > Project: OODT > Issue Type: Improvement > Components: build proces >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > Version 21 came out some time ago, we should upgrade > https://blogs.apache.org/maven/entry/apache-software-foundation-parent-pom1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-999) File.toURL fails when their are spaces in project/repository paths
[
https://issues.apache.org/jira/browse/OODT-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16712385#comment-16712385
]
ASF GitHub Bot commented on OODT-999:
-
IMS94 commented on issue #75: [OODT-999] Fix for "File.toURL fails when their
are spaces in project/repository paths"
URL: https://github.com/apache/oodt/pull/75#issuecomment-445135107
@chrismattmann Can this be merged?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> File.toURL fails when their are spaces in project/repository paths
> --
>
> Key: OODT-999
> URL: https://issues.apache.org/jira/browse/OODT-999
> Project: OODT
> Issue Type: Bug
>Reporter: Madhawa Vidanapathirana
>Priority: Major
>
> The method File.toURL which is now deprecated by Java has been used in many
> places of the code.
> The javadoc for File.toURL method explains as follows
> ??@deprecated This method does not automatically escape characters that??
> ??* are illegal in URLs. It is recommended that new code convert an??
> ??* abstract pathname into a URL by first converting it into a URI, via the??
> ??* \{@link #toURI() toURI} method, and then converting the URI into a URL??
> ??* via the \{@link java.net.URI#toURL() URI.toURL} method.??
> This method causes ingest operations to fail when repository path contains
> spaces.
> E.g. If specified in product_types.xml as follows,
> {noformat}
> path="file:///home/user/Apache%20OODT/cas-filemgr-1.9-SNAPSHOT-dist/data"/>{noformat}
> the ingest operations via client fails with the following log message by
> filemgr.
> {code:java}
> WARNING: URISyntaxException getting URI from URI str:
> [file:/home/madhawa/Apache
> OODT/cas-filemgr-1.9-SNAPSHOT-dist/data/blah.txt/blah.txt]
> java.lang.NullPointerException
> at java.io.File.(File.java:277)
> at
> org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor.doExtract(FinalFileLocationExtractor.java:90)
> at
> org.apache.oodt.cas.filemgr.metadata.extractors.AbstractFilemgrMetExtractor.extractMetadata(AbstractFilemgrMetExtractor.java:61)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.runExtractors(FileManager.java:875)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.addMetadata(FileManager.java:829)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.ingestProduct(FileManager.java:553)
> at
> org.apache.oodt.cas.filemgr.system.AvroFileManagerServer.ingestProduct(AvroFileManagerServer.java:468)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:91)
> at org.apache.avro.ipc.Responder.respond(Responder.java:151)
> at
> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:188)
> {code}
> A possible way to fix this issue of ingest operation is modifying
> BasicVersioner.java as follows.
> {noformat}
> // old code
> // dataStoreRef = new File(new URI(productRepoPath)).toURL()
> // .toExternalForm();
> // new code
> dataStoreRef = new File(new URI(productRepoPath)).toURI().toURL()
> .toExternalForm();{noformat}
> However, other locations where this deprecated method is being used may cause
> other operations to fail as well. For example, multiple unit tests of the
> project fails if the code is cloned to a path that contain spaces.
> I am currently working on a fix for this issue. But I may have to modify some
> unit tests as well in order to make them work when source path contains
> spaces.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-1003) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/OODT-1003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16708957#comment-16708957 ] ASF GitHub Bot commented on OODT-1003: -- lewismc commented on issue #89: OODT-1003 Upgrade Apache parent POM to version 21 URL: https://github.com/apache/oodt/pull/89#issuecomment-444163446 @chrismattmann @buggtb am I submitting pull requests to the correct branch? I am using the 1.9-SNAPSHOT artifacts in my COAL-SDS project and able to use all of the Avro functionality just fine. Should I keep sending PR's to master branch or development? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade Apache parent POM to version 21 > --- > > Key: OODT-1003 > URL: https://issues.apache.org/jira/browse/OODT-1003 > Project: OODT > Issue Type: Improvement > Components: build proces >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > Version 21 came out some time ago, we should upgrade > https://blogs.apache.org/maven/entry/apache-software-foundation-parent-pom1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-1003) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/OODT-1003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705888#comment-16705888 ] ASF GitHub Bot commented on OODT-1003: -- lewismc commented on issue #89: OODT-1003 Upgrade Apache parent POM to version 21 URL: https://github.com/apache/oodt/pull/89#issuecomment-443437396 By the way, this issue also fixes a NoDefClassFound Exception which arises in Workflow Manager due to a method return value change in Lucene's IndexWriter.addDocuement method. This occurs when you use Lucene as the data store for capturing workflow information. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade Apache parent POM to version 21 > --- > > Key: OODT-1003 > URL: https://issues.apache.org/jira/browse/OODT-1003 > Project: OODT > Issue Type: Improvement > Components: build proces >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > Version 21 came out some time ago, we should upgrade > https://blogs.apache.org/maven/entry/apache-software-foundation-parent-pom1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-1003) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/OODT-1003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705890#comment-16705890 ] ASF GitHub Bot commented on OODT-1003: -- lewismc edited a comment on issue #89: OODT-1003 Upgrade Apache parent POM to version 21 URL: https://github.com/apache/oodt/pull/89#issuecomment-443437396 By the way, this issue also fixes a NoDefClassFound Exception which arises in Workflow Manager due to a method return value change in Lucene's IndexWriter.addDocuement method (I think this happened between Lucene 6.1.0 and 6.6.5). This occurs when you use Lucene as the data store for capturing workflow information and could be attributed to incorrect dependency management. This issue fixes that. You will also see that I've synchronized the Avro dependencies to most recent 1.8.2 across the board. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade Apache parent POM to version 21 > --- > > Key: OODT-1003 > URL: https://issues.apache.org/jira/browse/OODT-1003 > Project: OODT > Issue Type: Improvement > Components: build proces >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > Version 21 came out some time ago, we should upgrade > https://blogs.apache.org/maven/entry/apache-software-foundation-parent-pom1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-1003) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/OODT-1003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705889#comment-16705889 ] ASF GitHub Bot commented on OODT-1003: -- lewismc edited a comment on issue #89: OODT-1003 Upgrade Apache parent POM to version 21 URL: https://github.com/apache/oodt/pull/89#issuecomment-443437396 By the way, this issue also fixes a NoDefClassFound Exception which arises in Workflow Manager due to a method return value change in Lucene's IndexWriter.addDocuement method. This occurs when you use Lucene as the data store for capturing workflow information and could be attributed to incorrect dependency management. This issue fixes that. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade Apache parent POM to version 21 > --- > > Key: OODT-1003 > URL: https://issues.apache.org/jira/browse/OODT-1003 > Project: OODT > Issue Type: Improvement > Components: build proces >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > Version 21 came out some time ago, we should upgrade > https://blogs.apache.org/maven/entry/apache-software-foundation-parent-pom1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-1003) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/OODT-1003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16705632#comment-16705632 ] ASF GitHub Bot commented on OODT-1003: -- lewismc opened a new pull request #89: OODT-1003 Upgrade Apache parent POM to version 21 URL: https://github.com/apache/oodt/pull/89 This issue addresses https://issues.apache.org/jira/browse/OODT-1003 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade Apache parent POM to version 21 > --- > > Key: OODT-1003 > URL: https://issues.apache.org/jira/browse/OODT-1003 > Project: OODT > Issue Type: Improvement > Components: build proces >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > Version 21 came out some time ago, we should upgrade > https://blogs.apache.org/maven/entry/apache-software-foundation-parent-pom1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-1002) Rename CoreMetKeys.java in filemgr and workflow to CoreFilemgrMetKeys.java and CoreWorkflowMetKeys.java respectively
[ https://issues.apache.org/jira/browse/OODT-1002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704421#comment-16704421 ] ASF GitHub Bot commented on OODT-1002: -- lewismc opened a new pull request #88: OODT-1002 Rename CoreMetKeys.java in filemgr and workflow to CoreFilemgrMetKeys.java and CoreWorkflowMetKeys.java respectively URL: https://github.com/apache/oodt/pull/88 This issue addresses https://issues.apache.org/jira/browse/OODT-1002 Quite a few files affected here however it is a simple change to resolve ClassPath issues. Additionally, I've also removed rogue, unnecessary file pge/velocity.log.1 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Rename CoreMetKeys.java in filemgr and workflow to CoreFilemgrMetKeys.java > and CoreWorkflowMetKeys.java respectively > > > Key: OODT-1002 > URL: https://issues.apache.org/jira/browse/OODT-1002 > Project: OODT > Issue Type: Task > Components: file manager, workflow manager >Affects Versions: 1.2.4 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Blocker > Fix For: 1.9 > > > Primary context for this thread can be found > [here|https://lists.apache.org/thread.html/deb7cc02e41afbc70e76583cc42623ee359a871c5e58271a14bf3234@%3Cdev.oodt.apache.org%3E]. > There is a clear ClassLoading issue for the CoreMetKeys.java file present in > filemgr and workflow. I propose to rename the files to > CoreFilemgrMetKeys.java and CoreWorkflowMetKeys.java respectively. There are > however still a few issues in the following classes > ./tomcat/webapps/fmprod/WEB-INF/classes/org/apache/oodt/cas/product/CASProductHandler.class > ./tomcat/webapps/opsui/WEB-INF/classes/org/apache/oodt/cas/product/CASProductHandler.class > ./tomcat/webapps/pcs/WEB-INF/classes/org/apache/oodt/pcs/services/HealthResource.class > I have a PR to address this. Coming up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-990) Enable OODT to build under JDK 10.X
[
https://issues.apache.org/jira/browse/OODT-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704313#comment-16704313
]
ASF GitHub Bot commented on OODT-990:
-
lewismc closed pull request #71: OODT-990 Enable OODT to build under JDK 10.X
URL: https://github.com/apache/oodt/pull/71
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/cli/src/main/java/org/apache/oodt/cas/cli/CmdLineUtility.java
b/cli/src/main/java/org/apache/oodt/cas/cli/CmdLineUtility.java
index 9fbb6d8cd..9d2a166d4 100644
--- a/cli/src/main/java/org/apache/oodt/cas/cli/CmdLineUtility.java
+++ b/cli/src/main/java/org/apache/oodt/cas/cli/CmdLineUtility.java
@@ -31,7 +31,6 @@
import java.util.HashSet;
import java.util.List;
import java.util.Set;
-import java.util.logging.Logger;
//Apache imports
import org.apache.commons.lang.Validate;
@@ -47,7 +46,6 @@
import org.apache.oodt.cas.cli.exception.CmdLineActionStoreException;
import org.apache.oodt.cas.cli.exception.CmdLineConstructionException;
import org.apache.oodt.cas.cli.exception.CmdLineOptionStoreException;
-import org.apache.oodt.cas.cli.exception.CmdLineParserException;
import org.apache.oodt.cas.cli.option.ActionCmdLineOption;
import org.apache.oodt.cas.cli.option.CmdLineOption;
import org.apache.oodt.cas.cli.option.CmdLineOptionInstance;
@@ -80,7 +78,6 @@
* @author bfoster (Brian Foster)
*/
public class CmdLineUtility {
- private static Logger LOG =
Logger.getLogger(CmdLineUtility.class.getName());
private boolean debugMode;
private CmdLineParser parser;
private CmdLineConstructor constructor;
diff --git a/cli/src/main/java/org/apache/oodt/cas/cli/util/CmdLineUtils.java
b/cli/src/main/java/org/apache/oodt/cas/cli/util/CmdLineUtils.java
index 87b8d8e83..f22d70b3f 100755
--- a/cli/src/main/java/org/apache/oodt/cas/cli/util/CmdLineUtils.java
+++ b/cli/src/main/java/org/apache/oodt/cas/cli/util/CmdLineUtils.java
@@ -18,6 +18,7 @@
//JDK imports
import java.io.File;
+import java.lang.reflect.InvocationTargetException;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.Collections;
@@ -1113,7 +1114,7 @@ public static String getFormattedString(String string,
int startIndex,
} else if (type.equals(Double.class) || type.equals(Double.TYPE)) {
List doubles = new LinkedList();
for (String value : values) {
-doubles.add(new Double(value));
+doubles.add(Double.valueOf(value));
}
return doubles;
} else if (type.equals(String.class)) {
@@ -1125,7 +1126,12 @@ public static String getFormattedString(String string,
int startIndex,
} else {
List objects = new LinkedList();
for (String value : values) {
-Object object = Class.forName(value).newInstance();
+Object object = null;
+try {
+ object = Class.forName(value).getConstructor().newInstance();
+} catch (IllegalArgumentException | InvocationTargetException |
NoSuchMethodException | SecurityException e) {
+ e.printStackTrace();
+}
if (!type.isAssignableFrom(object.getClass())) {
throw new RuntimeException(object.getClass() + " is not a valid"
+ " type or sub-type of " + type);
diff --git a/commons/src/main/java/org/apache/oodt/commons/AvroExecServer.java
b/commons/src/main/java/org/apache/oodt/commons/AvroExecServer.java
index c779acd6f..d60b56570 100644
--- a/commons/src/main/java/org/apache/oodt/commons/AvroExecServer.java
+++ b/commons/src/main/java/org/apache/oodt/commons/AvroExecServer.java
@@ -37,7 +37,7 @@
import java.net.UnknownHostException;
import java.rmi.server.RemoteObject;
import java.rmi.server.RemoteRef;
-import java.rmi.server.RemoteStub;
+import java.rmi.server.UnicastRemoteObject;
import java.util.Date;
import java.util.Iterator;
@@ -168,8 +168,8 @@ public static void runInitializers() throws EDAException {
for (Iterator i =
org.apache.oodt.commons.util.Utility.parseCommaList(initList); i.hasNext();) {
String iname = (String) i.next();
try {
-Class initClass = Class.forName(iname);
-Initializer init = (Initializer) initClass.newInstance();
+Class initClass = Class.forName(iname);
+Initializer init = (Initializer)
initClass.getConstructor().newInstance();
init.initialize();
} catch (ClassNotFoundException ex) {
System.err.println("Initializer \"" + iname + "\" not found;
aborting");
@@ -180,6 +180,14 @@ public static void runInitializers() throws EDAExc
[jira] [Commented] (OODT-999) File.toURL fails when their are spaces in project/repository paths
[
https://issues.apache.org/jira/browse/OODT-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16690787#comment-16690787
]
ASF GitHub Bot commented on OODT-999:
-
madhawav commented on issue #75: [OODT-999] Fix for "File.toURL fails when
their are spaces in project/repository paths"
URL: https://github.com/apache/oodt/pull/75#issuecomment-439665464
Thank you @buggtb @chrismattmann
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> File.toURL fails when their are spaces in project/repository paths
> --
>
> Key: OODT-999
> URL: https://issues.apache.org/jira/browse/OODT-999
> Project: OODT
> Issue Type: Bug
>Reporter: Madhawa Vidanapathirana
>Priority: Major
>
> The method File.toURL which is now deprecated by Java has been used in many
> places of the code.
> The javadoc for File.toURL method explains as follows
> ??@deprecated This method does not automatically escape characters that??
> ??* are illegal in URLs. It is recommended that new code convert an??
> ??* abstract pathname into a URL by first converting it into a URI, via the??
> ??* \{@link #toURI() toURI} method, and then converting the URI into a URL??
> ??* via the \{@link java.net.URI#toURL() URI.toURL} method.??
> This method causes ingest operations to fail when repository path contains
> spaces.
> E.g. If specified in product_types.xml as follows,
> {noformat}
> path="file:///home/user/Apache%20OODT/cas-filemgr-1.9-SNAPSHOT-dist/data"/>{noformat}
> the ingest operations via client fails with the following log message by
> filemgr.
> {code:java}
> WARNING: URISyntaxException getting URI from URI str:
> [file:/home/madhawa/Apache
> OODT/cas-filemgr-1.9-SNAPSHOT-dist/data/blah.txt/blah.txt]
> java.lang.NullPointerException
> at java.io.File.(File.java:277)
> at
> org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor.doExtract(FinalFileLocationExtractor.java:90)
> at
> org.apache.oodt.cas.filemgr.metadata.extractors.AbstractFilemgrMetExtractor.extractMetadata(AbstractFilemgrMetExtractor.java:61)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.runExtractors(FileManager.java:875)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.addMetadata(FileManager.java:829)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.ingestProduct(FileManager.java:553)
> at
> org.apache.oodt.cas.filemgr.system.AvroFileManagerServer.ingestProduct(AvroFileManagerServer.java:468)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:91)
> at org.apache.avro.ipc.Responder.respond(Responder.java:151)
> at
> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:188)
> {code}
> A possible way to fix this issue of ingest operation is modifying
> BasicVersioner.java as follows.
> {noformat}
> // old code
> // dataStoreRef = new File(new URI(productRepoPath)).toURL()
> // .toExternalForm();
> // new code
> dataStoreRef = new File(new URI(productRepoPath)).toURI().toURL()
> .toExternalForm();{noformat}
> However, other locations where this deprecated method is being used may cause
> other operations to fail as well. For example, multiple unit tests of the
> project fails if the code is cloned to a path that contain spaces.
> I am currently working on a fix for this issue. But I may have to modify some
> unit tests as well in order to make them work when source path contains
> spaces.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-999) File.toURL fails when their are spaces in project/repository paths
[
https://issues.apache.org/jira/browse/OODT-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16690617#comment-16690617
]
ASF GitHub Bot commented on OODT-999:
-
buggtb commented on issue #75: [OODT-999] Fix for "File.toURL fails when their
are spaces in project/repository paths"
URL: https://github.com/apache/oodt/pull/75#issuecomment-439627223
Good PR, nothing wrong that I can see and I've hit the encoding issues
before so +1 from me.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> File.toURL fails when their are spaces in project/repository paths
> --
>
> Key: OODT-999
> URL: https://issues.apache.org/jira/browse/OODT-999
> Project: OODT
> Issue Type: Bug
>Reporter: Madhawa Vidanapathirana
>Priority: Major
>
> The method File.toURL which is now deprecated by Java has been used in many
> places of the code.
> The javadoc for File.toURL method explains as follows
> ??@deprecated This method does not automatically escape characters that??
> ??* are illegal in URLs. It is recommended that new code convert an??
> ??* abstract pathname into a URL by first converting it into a URI, via the??
> ??* \{@link #toURI() toURI} method, and then converting the URI into a URL??
> ??* via the \{@link java.net.URI#toURL() URI.toURL} method.??
> This method causes ingest operations to fail when repository path contains
> spaces.
> E.g. If specified in product_types.xml as follows,
> {noformat}
> path="file:///home/user/Apache%20OODT/cas-filemgr-1.9-SNAPSHOT-dist/data"/>{noformat}
> the ingest operations via client fails with the following log message by
> filemgr.
> {code:java}
> WARNING: URISyntaxException getting URI from URI str:
> [file:/home/madhawa/Apache
> OODT/cas-filemgr-1.9-SNAPSHOT-dist/data/blah.txt/blah.txt]
> java.lang.NullPointerException
> at java.io.File.(File.java:277)
> at
> org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor.doExtract(FinalFileLocationExtractor.java:90)
> at
> org.apache.oodt.cas.filemgr.metadata.extractors.AbstractFilemgrMetExtractor.extractMetadata(AbstractFilemgrMetExtractor.java:61)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.runExtractors(FileManager.java:875)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.addMetadata(FileManager.java:829)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.ingestProduct(FileManager.java:553)
> at
> org.apache.oodt.cas.filemgr.system.AvroFileManagerServer.ingestProduct(AvroFileManagerServer.java:468)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:91)
> at org.apache.avro.ipc.Responder.respond(Responder.java:151)
> at
> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:188)
> {code}
> A possible way to fix this issue of ingest operation is modifying
> BasicVersioner.java as follows.
> {noformat}
> // old code
> // dataStoreRef = new File(new URI(productRepoPath)).toURL()
> // .toExternalForm();
> // new code
> dataStoreRef = new File(new URI(productRepoPath)).toURI().toURL()
> .toExternalForm();{noformat}
> However, other locations where this deprecated method is being used may cause
> other operations to fail as well. For example, multiple unit tests of the
> project fails if the code is cloned to a path that contain spaces.
> I am currently working on a fix for this issue. But I may have to modify some
> unit tests as well in order to make them work when source path contains
> spaces.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-999) File.toURL fails when their are spaces in project/repository paths
[
https://issues.apache.org/jira/browse/OODT-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16678660#comment-16678660
]
ASF GitHub Bot commented on OODT-999:
-
madhawav opened a new pull request #75: [OODT-999] Fix for "File.toURL fails
when their are spaces in project/repository paths"
URL: https://github.com/apache/oodt/pull/75
Fix for https://issues.apache.org/jira/projects/OODT/issues/OODT-999
1) Modified unit tests involving data repository path to expose the issue.
2) Replaced the deprecated method File.toURL() which causes the issue by
File.toURI().toURL().
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> File.toURL fails when their are spaces in project/repository paths
> --
>
> Key: OODT-999
> URL: https://issues.apache.org/jira/browse/OODT-999
> Project: OODT
> Issue Type: Bug
>Reporter: Madhawa Vidanapathirana
>Priority: Major
>
> The method File.toURL which is now deprecated by Java has been used in many
> places of the code.
> The javadoc for File.toURL method explains as follows
> ??@deprecated This method does not automatically escape characters that??
> ??* are illegal in URLs. It is recommended that new code convert an??
> ??* abstract pathname into a URL by first converting it into a URI, via the??
> ??* \{@link #toURI() toURI} method, and then converting the URI into a URL??
> ??* via the \{@link java.net.URI#toURL() URI.toURL} method.??
> This method causes ingest operations to fail when repository path contains
> spaces.
> E.g. If specified in product_types.xml as follows,
> {noformat}
> path="file:///home/user/Apache%20OODT/cas-filemgr-1.9-SNAPSHOT-dist/data"/>{noformat}
> the ingest operations via client fails with the following log message by
> filemgr.
> {code:java}
> WARNING: URISyntaxException getting URI from URI str:
> [file:/home/madhawa/Apache
> OODT/cas-filemgr-1.9-SNAPSHOT-dist/data/blah.txt/blah.txt]
> java.lang.NullPointerException
> at java.io.File.(File.java:277)
> at
> org.apache.oodt.cas.filemgr.metadata.extractors.examples.FinalFileLocationExtractor.doExtract(FinalFileLocationExtractor.java:90)
> at
> org.apache.oodt.cas.filemgr.metadata.extractors.AbstractFilemgrMetExtractor.extractMetadata(AbstractFilemgrMetExtractor.java:61)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.runExtractors(FileManager.java:875)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.addMetadata(FileManager.java:829)
> at
> org.apache.oodt.cas.filemgr.system.FileManager.ingestProduct(FileManager.java:553)
> at
> org.apache.oodt.cas.filemgr.system.AvroFileManagerServer.ingestProduct(AvroFileManagerServer.java:468)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:91)
> at org.apache.avro.ipc.Responder.respond(Responder.java:151)
> at
> org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:188)
> {code}
> A possible way to fix this issue of ingest operation is modifying
> BasicVersioner.java as follows.
> {noformat}
> // old code
> // dataStoreRef = new File(new URI(productRepoPath)).toURL()
> // .toExternalForm();
> // new code
> dataStoreRef = new File(new URI(productRepoPath)).toURI().toURL()
> .toExternalForm();{noformat}
> However, other locations where this deprecated method is being used may cause
> other operations to fail as well. For example, multiple unit tests of the
> project fails if the code is cloned to a path that contain spaces.
> I am currently working on a fix for this issue. But I may have to modify some
> unit tests as well in order to make them work when source path contains
> spaces.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-998) The startup script of cas-filemgr fails if distribution is located in a path with blank spaces
[
https://issues.apache.org/jira/browse/OODT-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16674117#comment-16674117
]
ASF GitHub Bot commented on OODT-998:
-
IMS94 closed pull request #74: OODT-998 Fix for "The startup script of
cas-filemgr fails if distribution is located in a path with blank spaces"
URL: https://github.com/apache/oodt/pull/74
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/filemgr/src/main/bin/filemgr b/filemgr/src/main/bin/filemgr
index 1a8c4164e..8981580c3 100644
--- a/filemgr/src/main/bin/filemgr
+++ b/filemgr/src/main/bin/filemgr
@@ -46,7 +46,7 @@ export PATH
## make sure that casfile manager has a run directory
## just to be on the safe side
-mkdir -p ${RUN_HOME}
+mkdir -p "${RUN_HOME}"
for file in `find ../lib/*.jar`; do
LIB_DEPS="${file}:${LIB_DEPS}"
@@ -73,19 +73,19 @@ case "$1" in
$JAVA_HOME/bin/java \
-cp ${LIB_DEPS} \
${DISTRIBUTED_CONF_PROPERTIES} \
- -Dlog4j.configurationFile=${FILEMGR_HOME}/etc/log4j2.xml \
-
-Djava.util.logging.config.file=${FILEMGR_HOME}/etc/logging.properties \
+ -Dlog4j.configurationFile="${FILEMGR_HOME}/etc/log4j2.xml" \
+
-Djava.util.logging.config.file="${FILEMGR_HOME}/etc/logging.properties" \
-Dorg.apache.oodt.cas.filemgr.properties=${CAS_FILEMGR_PROPS} \
org.apache.oodt.cas.filemgr.system.FileManagerServerMain \
--portNum $SERVER_PORT &
-echo $! > ${RUN_HOME}/cas.filemgr.pid
+echo $! > "${RUN_HOME}/cas.filemgr.pid"
echo "OK"
sleep 5
;;
stop)
echo -n "Shutting down cas file manager: "
-kill `cat ${RUN_HOME}/cas.filemgr.pid`
-rm -f ${RUN_HOME}/cas.filemgr.pid
+kill `cat "${RUN_HOME}/cas.filemgr.pid"`
+rm -f "${RUN_HOME}/cas.filemgr.pid"
echo "OK"
;;
restart)
@@ -93,8 +93,8 @@ case "$1" in
$0 start
;;
status)
-if [ -e ${RUN_HOME}/cas.filemgr.pid ] ; then
- pid=`cat ${RUN_HOME}/cas.filemgr.pid`
+if [ -e "${RUN_HOME}/cas.filemgr.pid" ] ; then
+ pid=`cat "${RUN_HOME}/cas.filemgr.pid"`
echo "cas filemgr is running with pid: $pid"
else
echo "cas filemgr is not running"
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> The startup script of cas-filemgr fails if distribution is located in a path
> with blank spaces
> --
>
> Key: OODT-998
> URL: https://issues.apache.org/jira/browse/OODT-998
> Project: OODT
> Issue Type: Bug
>Reporter: Madhawa Vidanapathirana
>Priority: Minor
>
> The script bin/filemgr of cas-filemgr fails if distribution is located in a
> path with space characters.
> E.g.
> # The distribution archive is extracted to _/home/user/Apache
> OODT/cas-filemgr-1.9-SNAPSHOT-dist_
> # The command _./filemgr start_ is executed from working directory
> _/home/user/Apache
> OODT/cas-filemgr-1.9-SNAPSHOT-dist/cas-filemgr-1.9-SNAPSHOT/bin_
> # The following output is observed.
> {noformat}
> Using standalone configuration management
> Starting cas file manager: OK
> Error: Could not find or load main class
> OODT.cas-filemgr-1.9-SNAPSHOT-dist.cas-filemgr-1.9-SNAPSHOT.etc.log4j2.xml{noformat}
> I will provide a fix for this issue soon. Already working on it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-998) The startup script of cas-filemgr fails if distribution is located in a path with blank spaces
[
https://issues.apache.org/jira/browse/OODT-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16674116#comment-16674116
]
ASF GitHub Bot commented on OODT-998:
-
IMS94 commented on issue #74: OODT-998 Fix for "The startup script of
cas-filemgr fails if distribution is located in a path with blank spaces"
URL: https://github.com/apache/oodt/pull/74#issuecomment-435602563
@madhawav Thanks for the PR. Looks good to me. Merging.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> The startup script of cas-filemgr fails if distribution is located in a path
> with blank spaces
> --
>
> Key: OODT-998
> URL: https://issues.apache.org/jira/browse/OODT-998
> Project: OODT
> Issue Type: Bug
>Reporter: Madhawa Vidanapathirana
>Priority: Minor
>
> The script bin/filemgr of cas-filemgr fails if distribution is located in a
> path with space characters.
> E.g.
> # The distribution archive is extracted to _/home/user/Apache
> OODT/cas-filemgr-1.9-SNAPSHOT-dist_
> # The command _./filemgr start_ is executed from working directory
> _/home/user/Apache
> OODT/cas-filemgr-1.9-SNAPSHOT-dist/cas-filemgr-1.9-SNAPSHOT/bin_
> # The following output is observed.
> {noformat}
> Using standalone configuration management
> Starting cas file manager: OK
> Error: Could not find or load main class
> OODT.cas-filemgr-1.9-SNAPSHOT-dist.cas-filemgr-1.9-SNAPSHOT.etc.log4j2.xml{noformat}
> I will provide a fix for this issue soon. Already working on it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-998) The startup script of cas-filemgr fails if distribution is located in a path with blank spaces
[
https://issues.apache.org/jira/browse/OODT-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16673969#comment-16673969
]
ASF GitHub Bot commented on OODT-998:
-
madhawav opened a new pull request #74: OODT-998 Fix for "The startup script of
cas-filemgr fails if distribution is located in a path with blank spaces"
URL: https://github.com/apache/oodt/pull/74
This PR is a fix for issue
https://issues.apache.org/jira/projects/OODT/issues/OODT-998.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> The startup script of cas-filemgr fails if distribution is located in a path
> with blank spaces
> --
>
> Key: OODT-998
> URL: https://issues.apache.org/jira/browse/OODT-998
> Project: OODT
> Issue Type: Bug
>Reporter: Madhawa Vidanapathirana
>Priority: Minor
>
> The script bin/filemgr of cas-filemgr fails if distribution is located in a
> path with space characters.
> E.g.
> # The distribution archive is extracted to _/home/user/Apache
> OODT/cas-filemgr-1.9-SNAPSHOT-dist_
> # The command _./filemgr start_ is executed from working directory
> _/home/user/Apache
> OODT/cas-filemgr-1.9-SNAPSHOT-dist/cas-filemgr-1.9-SNAPSHOT/bin_
> # The following output is observed.
> {noformat}
> Using standalone configuration management
> Starting cas file manager: OK
> Error: Could not find or load main class
> OODT.cas-filemgr-1.9-SNAPSHOT-dist.cas-filemgr-1.9-SNAPSHOT.etc.log4j2.xml{noformat}
> I will provide a fix for this issue soon. Already working on it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-976) Upgrade Apache Tika to 1.19.1
[ https://issues.apache.org/jira/browse/OODT-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672689#comment-16672689 ] ASF GitHub Bot commented on OODT-976: - IMS94 commented on issue #61: OODT-976 Upgrade Apache Tika to 1.18 URL: https://github.com/apache/oodt/pull/61#issuecomment-435296165 @lewismc I saw Tika 1.19.1 has been released. It will be great if you can resend this PR with the desired upgrade when you find some free time. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade Apache Tika to 1.19.1 > - > > Key: OODT-976 > URL: https://issues.apache.org/jira/browse/OODT-976 > Project: OODT > Issue Type: Improvement > Components: core, file manager, radix >Affects Versions: 1.2.2 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > tika-core and tika-parsers 1.13 dependencies are pretty outdated now. A > release is pending for Tika 1.19.1. We should upgrade when it is available. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-976) Upgrade Apache Tika to 1.18
[ https://issues.apache.org/jira/browse/OODT-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16638544#comment-16638544 ] ASF GitHub Bot commented on OODT-976: - lewismc commented on issue #61: OODT-976 Upgrade Apache Tika to 1.18 URL: https://github.com/apache/oodt/pull/61#issuecomment-427095031 The recent Tika 1.19.1 VOTE was cancelled due to pending upgrade of PDFBox. I would like to upgrade to a more recent version of Tika as it has significant improvements over 1.13 which we are using. I'll go ahead and close this issue then wait for the pending 1.19.1 release before I work on submitting the PR which we can then test. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade Apache Tika to 1.18 > --- > > Key: OODT-976 > URL: https://issues.apache.org/jira/browse/OODT-976 > Project: OODT > Issue Type: Improvement > Components: core, file manager, radix >Affects Versions: 1.2.2 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > tika-core and tika-parsers 1.13 dependencies are pretty outdated now. Tika > 1.18 is available so we should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-994) Replace deprecated -Djava.ext.dirs with -classpath
[
https://issues.apache.org/jira/browse/OODT-994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614363#comment-16614363
]
ASF GitHub Bot commented on OODT-994:
-
lewismc commented on issue #73: OODT-994 Replace deprecated -Djava.ext.dirs
with -classpath
URL: https://github.com/apache/oodt/pull/73#issuecomment-421229618
This PR will not work, it looks like I need to add a wild card to each entry
when using the -cp, -classpath or --class-path notation for the replacement.
The command syntax is going to be slightly more tricky.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Replace deprecated -Djava.ext.dirs with -classpath
> --
>
> Key: OODT-994
> URL: https://issues.apache.org/jira/browse/OODT-994
> Project: OODT
> Issue Type: Improvement
> Components: radix
> Environment: Apache Maven 3.5.4
> (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T11:33:14-07:00)
> Maven home: /usr/local/Cellar/maven/3.5.4/libexec
> Java version: 10.0.2, vendor: Oracle Corporation, runtime:
> /Library/Java/JavaVirtualMachines/jdk-10.0.2.jdk/Contents/Home
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "Mac"
> OODT master branch
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Blocker
> Fix For: 1.9
>
>
> I'm back working with [coal-sds|https://github.com/capstone-coal/coal-sds]
> and will be running it under JDK 10 if possible. Immediately upon launching
> the RADiX deployment, all hell breaks loose
> {code}
> Resource Manager started PID file
> (/usr/local/coal-sds-deploy/resmgr/run/cas.resmgr.pid).
> Workflow Manager started PID file
> (/usr/local/coal-sds-deploy/workflow/run/cas.workflow.pid).
> -Djava.ext.dirs=/usr/local/coal-sds-deploy/filemgr/lib is not supported. Use
> -classpath instead.
> -Djava.ext.dirs=/usr/local/coal-sds-deploy/workflow/lib is not supported.
> Use -classpath instead.
> -Djava.ext.dirs=/usr/local/coal-sds-deploy/resmgr/lib is not supported. Use
> -classpath instead.
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.Error: Could not
> create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> {code}
> On my macOS Sierra v10.12.6 the following statement does a find and replace
> for all instances of '-Djava.ext.dirs', replacing with '-classpath'
> {code}
> git grep -lz '\-Djava.ext.dirs' | xargs -0 perl -i'' -pE
> "s/-Djava.ext.dirs/-classpath/g"
> {code}
> PR coming up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-994) Replace deprecated -Djava.ext.dirs with -classpath
[
https://issues.apache.org/jira/browse/OODT-994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614364#comment-16614364
]
ASF GitHub Bot commented on OODT-994:
-
lewismc closed pull request #73: OODT-994 Replace deprecated -Djava.ext.dirs
with -classpath
URL: https://github.com/apache/oodt/pull/73
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/config/src/main/scripts/conf-publisher
b/config/src/main/scripts/conf-publisher
index fd1476726..75e4d0f97 100644
--- a/config/src/main/scripts/conf-publisher
+++ b/config/src/main/scripts/conf-publisher
@@ -28,7 +28,7 @@ echo -n "Starting Configuration Publisher\n"
$JAVA_HOME/bin/java \
-cp ${LIB_DEPS} \
-Dlog4j.configurationFile=${CONF_HOME}/etc/log4j2.xml \
- -Djava.ext.dirs=. \
+ -classpath=. \
-Djava.util.logging.config.file=${CONF_HOME}/etc/logging.properties \
-Dorg.apache.oodt.cas.cli.action.spring.config=${CONF_HOME}/policy/cmd-line-actions.xml
\
-Dorg.apache.oodt.cas.cli.option.spring.config=${CONF_HOME}/policy/cmd-line-options.xml
\
diff --git a/crawler/src/main/bin/crawlctl b/crawler/src/main/bin/crawlctl
index 0b693135d..c09f96dfc 100644
--- a/crawler/src/main/bin/crawlctl
+++ b/crawler/src/main/bin/crawlctl
@@ -18,4 +18,4 @@
java \
-Dlog4j.configurationFile=../etc/log4j2.xml \
-Djava.util.logging.config.file=../etc/logging.properties \
--Djava.ext.dirs=../lib
org.apache.oodt.cas.crawl.daemon.CrawlDaemonController $*
+-classpath=../lib org.apache.oodt.cas.crawl.daemon.CrawlDaemonController $*
diff --git a/crawler/src/main/bin/crawler_launcher
b/crawler/src/main/bin/crawler_launcher
index 5693dfc91..b4a193aa6 100644
--- a/crawler/src/main/bin/crawler_launcher
+++ b/crawler/src/main/bin/crawler_launcher
@@ -25,7 +25,7 @@ fi
export JAVA_HOME
$JAVA_HOME/bin/java \
--Djava.ext.dirs=../lib \
+-classpath=../lib \
-Dlog4j.configurationFile=../etc/log4j2.xml \
-Djava.util.logging.config.file=../etc/logging.properties \
-Dorg.apache.oodt.cas.crawl.bean.repo=../policy/crawler-config.xml \
diff --git a/filemgr/src/main/bin/query_tool b/filemgr/src/main/bin/query_tool
index f6610cf0f..aeafe58de 100644
--- a/filemgr/src/main/bin/query_tool
+++ b/filemgr/src/main/bin/query_tool
@@ -20,4 +20,4 @@
# NOTE: This has been deprecated... use SqlQuery and LuceneQuery actions
instead.
-$JAVA_HOME/bin/java -Djava.ext.dirs=../lib
org.apache.oodt.cas.filemgr.tools.QueryTool "$@"
+$JAVA_HOME/bin/java -classpath=../lib
org.apache.oodt.cas.filemgr.tools.QueryTool "$@"
diff --git a/filemgr/src/main/resources/bash_aliases.txt
b/filemgr/src/main/resources/bash_aliases.txt
index 17ca48000..1f526c361 100644
--- a/filemgr/src/main/resources/bash_aliases.txt
+++ b/filemgr/src/main/resources/bash_aliases.txt
@@ -22,9 +22,9 @@
# For complete documentation see:
#
https://cwiki.apache.org/confluence/display/OODT/BASH+and+TCSH+shell+tools+for+File+Manager
#
-alias lucenequery="java
-Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
-Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.QueryTool --url
http://localhost:9000 --lucene -query "
-alias sqlquery="java
-Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
-Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.QueryTool --url
http://localhost:9000 --sql -query "
-alias fmdel="java
-Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
-Djava.ext.dirs=../lib org.apache.oodt.cas.filemgr.tools.DeleteProduct
--fileManagerUrl http://localhost:9000 --read"
-alias metdump="java -Djava.ext.dirs=../lib
org.apache.oodt.cas.filemgr.tools.MetadataDumper --url http://localhost:9000
--out . --productId "
+alias lucenequery="java
-Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
-classpath=../lib org.apache.oodt.cas.filemgr.tools.QueryTool --url
http://localhost:9000 --lucene -query "
+alias sqlquery="java
-Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
-classpath=../lib org.apache.oodt.cas.filemgr.tools.QueryTool --url
http://localhost:9000 --sql -query "
+alias fmdel="java
-Dorg.apache.oodt.cas.filemgr.properties=../etc/filemgr.properties
-classpath=../lib org.apache.oodt.cas.filemgr.tools.DeleteProduct
--fileManagerUrl http://localhost:9000 --read"
+alias metdump="java -classpath=../lib
org.apache.oodt.cas.filemgr.tools.MetadataDumper --url http://localhost:9000
--out . --productId "
#
\ No newline at end of file
diff --git a/filemgr/src/m
[jira] [Commented] (OODT-994) Replace deprecated -Djava.ext.dirs with -classpath
[
https://issues.apache.org/jira/browse/OODT-994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614359#comment-16614359
]
ASF GitHub Bot commented on OODT-994:
-
lewismc commented on issue #73: OODT-994 Replace deprecated -Djava.ext.dirs
with -classpath
URL: https://github.com/apache/oodt/pull/73#issuecomment-421228843
The PR as it stands is not a fix...
```
Workflow Manager started PID file
(/usr/local/coal-sds-deploy/workflow/run/cas.workflow.pid).
Resource Manager started PID file
(/usr/local/coal-sds-deploy/resmgr/run/cas.resmgr.pid).
Unrecognized option: -classpath=/usr/local/coal-sds-deploy/resmgr/lib
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Unrecognized option: -classpath=/usr/local/coal-sds-deploy/filemgr/lib
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Unrecognized option: -classpath=/usr/local/coal-sds-deploy/workflow/lib
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
```
Still debugging.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Replace deprecated -Djava.ext.dirs with -classpath
> --
>
> Key: OODT-994
> URL: https://issues.apache.org/jira/browse/OODT-994
> Project: OODT
> Issue Type: Improvement
> Components: radix
> Environment: Apache Maven 3.5.4
> (1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T11:33:14-07:00)
> Maven home: /usr/local/Cellar/maven/3.5.4/libexec
> Java version: 10.0.2, vendor: Oracle Corporation, runtime:
> /Library/Java/JavaVirtualMachines/jdk-10.0.2.jdk/Contents/Home
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "Mac"
> OODT master branch
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Blocker
> Fix For: 1.9
>
>
> I'm back working with [coal-sds|https://github.com/capstone-coal/coal-sds]
> and will be running it under JDK 10 if possible. Immediately upon launching
> the RADiX deployment, all hell breaks loose
> {code}
> Resource Manager started PID file
> (/usr/local/coal-sds-deploy/resmgr/run/cas.resmgr.pid).
> Workflow Manager started PID file
> (/usr/local/coal-sds-deploy/workflow/run/cas.workflow.pid).
> -Djava.ext.dirs=/usr/local/coal-sds-deploy/filemgr/lib is not supported. Use
> -classpath instead.
> -Djava.ext.dirs=/usr/local/coal-sds-deploy/workflow/lib is not supported.
> Use -classpath instead.
> -Djava.ext.dirs=/usr/local/coal-sds-deploy/resmgr/lib is not supported. Use
> -classpath instead.
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.Error: Could not
> create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> {code}
> On my macOS Sierra v10.12.6 the following statement does a find and replace
> for all instances of '-Djava.ext.dirs', replacing with '-classpath'
> {code}
> git grep -lz '\-Djava.ext.dirs' | xargs -0 perl -i'' -pE
> "s/-Djava.ext.dirs/-classpath/g"
> {code}
> PR coming up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-994) Replace deprecated -Djava.ext.dirs with -classpath
[
https://issues.apache.org/jira/browse/OODT-994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16614354#comment-16614354
]
ASF GitHub Bot commented on OODT-994:
-
lewismc opened a new pull request #73: OODT-994 Replace deprecated
-Djava.ext.dirs with -classpath
URL: https://github.com/apache/oodt/pull/73
Issue addresses https://issues.apache.org/jira/browse/OODT-994
My environment is as follows
```
Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe;
2018-06-17T11:33:14-07:00)
Maven home: /usr/local/Cellar/maven/3.5.4/libexec
Java version: 10.0.2, vendor: Oracle Corporation, runtime:
/Library/Java/JavaVirtualMachines/jdk-10.0.2.jdk/Contents/Home
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.12.6", arch: "x86_64", family: "Mac"
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Replace deprecated -Djava.ext.dirs with -classpath
> --
>
> Key: OODT-994
> URL: https://issues.apache.org/jira/browse/OODT-994
> Project: OODT
> Issue Type: Improvement
> Components: radix
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Blocker
> Fix For: 1.9
>
>
> I'm back working with [coal-sds|https://github.com/capstone-coal/coal-sds]
> and will be running it under JDK 10 if possible. Immediately upon launching
> the RADiX deployment, all hell breaks loose
> {code}
> Resource Manager started PID file
> (/usr/local/coal-sds-deploy/resmgr/run/cas.resmgr.pid).
> Workflow Manager started PID file
> (/usr/local/coal-sds-deploy/workflow/run/cas.workflow.pid).
> -Djava.ext.dirs=/usr/local/coal-sds-deploy/filemgr/lib is not supported. Use
> -classpath instead.
> -Djava.ext.dirs=/usr/local/coal-sds-deploy/workflow/lib is not supported.
> Use -classpath instead.
> -Djava.ext.dirs=/usr/local/coal-sds-deploy/resmgr/lib is not supported. Use
> -classpath instead.
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.Error: Could not
> create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> {code}
> On my macOS Sierra v10.12.6 the following statement does a find and replace
> for all instances of '-Djava.ext.dirs', replacing with '-classpath'
> {code}
> git grep -lz '\-Djava.ext.dirs' | xargs -0 perl -i'' -pE
> "s/-Djava.ext.dirs/-classpath/g"
> {code}
> PR coming up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-990) Enable OODT to build under JDK 10.X
[ https://issues.apache.org/jira/browse/OODT-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16606237#comment-16606237 ] ASF GitHub Bot commented on OODT-990: - lewismc commented on issue #71: OODT-990 Enable OODT to build under JDK 10.X URL: https://github.com/apache/oodt/pull/71#issuecomment-419198230 The issue I had was with the stalling TestXmlRpcWorkflowManager.java when run as part of the Unit test suite. When I ran it individually, it passed no problems. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Enable OODT to build under JDK 10.X > --- > > Key: OODT-990 > URL: https://issues.apache.org/jira/browse/OODT-990 > Project: OODT > Issue Type: Improvement > Components: build proces >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Critical > Fix For: 1.9 > > > When reviewing the 1.2.4 RC, I decided to build under JDK 10.X. This issue > will document and contain a fix for any deprecation and code improvements > required to build and test master branch under JDK 10.X. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-990) Enable OODT to build under JDK 10.X
[ https://issues.apache.org/jira/browse/OODT-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599355#comment-16599355 ] ASF GitHub Bot commented on OODT-990: - chrismattmann commented on issue #71: OODT-990 Enable OODT to build under JDK 10.X URL: https://github.com/apache/oodt/pull/71#issuecomment-417808346 @buggtb @lewismc can we get this and #61 / OODT-976 closed in the next week? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Enable OODT to build under JDK 10.X > --- > > Key: OODT-990 > URL: https://issues.apache.org/jira/browse/OODT-990 > Project: OODT > Issue Type: Improvement > Components: build proces >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Critical > Fix For: 1.9 > > > When reviewing the 1.2.4 RC, I decided to build under JDK 10.X. This issue > will document and contain a fix for any deprecation and code improvements > required to build and test master branch under JDK 10.X. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-993) Upgrade aws-java-sdk dependency and address deprecated api usage
[
https://issues.apache.org/jira/browse/OODT-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599340#comment-16599340
]
ASF GitHub Bot commented on OODT-993:
-
chrismattmann closed pull request #72: OODT-993 Upgrade aws-java-sdk dependency
and address deprecated api usage
URL: https://github.com/apache/oodt/pull/72
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/core/pom.xml b/core/pom.xml
index 69bf75272..fa86caa3e 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -72,7 +72,7 @@ the License.
com.amazonaws
aws-java-sdk
-1.7.4
+1.11.396
org.apache.avro
diff --git
a/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/S3DataTransferer.java
b/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/S3DataTransferer.java
index e7f9385c7..28471639b 100644
---
a/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/S3DataTransferer.java
+++
b/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/S3DataTransferer.java
@@ -70,7 +70,7 @@ public void transferProduct(Product product) throws
DataTransferException, IOExc
bucketName, dataStoreRef, new File(origRef));
if (encrypt) {
ObjectMetadata requestMetadata = new
ObjectMetadata();
-
requestMetadata.setServerSideEncryption(AES_256_SERVER_SIDE_ENCRYPTION);
+
requestMetadata.setSSEAlgorithm(AES_256_SERVER_SIDE_ENCRYPTION);
request.setMetadata(requestMetadata);
}
s3Client.putObject(request);
diff --git
a/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/S3DataTransfererFactory.java
b/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/S3DataTransfererFactory.java
index d87dd3763..86fc9703d 100644
---
a/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/S3DataTransfererFactory.java
+++
b/filemgr/src/main/java/org/apache/oodt/cas/filemgr/datatransfer/S3DataTransfererFactory.java
@@ -16,10 +16,10 @@
*/
package org.apache.oodt.cas.filemgr.datatransfer;
+import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
-import com.amazonaws.regions.Region;
-import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3Client;
+import com.amazonaws.services.s3.AmazonS3ClientBuilder;
/**
* {@link DataTransferFactory} which creates {@link S3DataTransferer}s.
@@ -47,8 +47,12 @@ public S3DataTransferer createDataTransfer() {
String secretKey = System.getProperty(SECRET_KEY_PROPERTY);
boolean encrypt = Boolean.getBoolean(ENCRYPT_PROPERTY);
- AmazonS3Client s3 = new AmazonS3Client(new
BasicAWSCredentials(accessKey, secretKey));
-s3.setRegion(Region.getRegion(Regions.valueOf(region)));
+ AmazonS3Client s3 = (AmazonS3Client)
AmazonS3ClientBuilder.standard()
+ .withRegion(region)
+ .withCredentials(
+ new AWSStaticCredentialsProvider(
+ new BasicAWSCredentials(accessKey,
secretKey)))
+ .build();
return new S3DataTransferer(s3, bucketName, encrypt);
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Upgrade aws-java-sdk dependency and address deprecated api usage
>
>
> Key: OODT-993
> URL: https://issues.apache.org/jira/browse/OODT-993
> Project: OODT
> Issue Type: Improvement
> Components: file manager
>Affects Versions: 1.9
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Major
> Fix For: 1.9
>
>
> Small but important patch for upgrading the AWS s3 SDK dependency. Patch
> coming up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-993) Upgrade aws-java-sdk dependency and address deprecated api usage
[ https://issues.apache.org/jira/browse/OODT-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599341#comment-16599341 ] ASF GitHub Bot commented on OODT-993: - chrismattmann commented on issue #72: OODT-993 Upgrade aws-java-sdk dependency and address deprecated api usage URL: https://github.com/apache/oodt/pull/72#issuecomment-417806163 thanks @lewismc This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade aws-java-sdk dependency and address deprecated api usage > > > Key: OODT-993 > URL: https://issues.apache.org/jira/browse/OODT-993 > Project: OODT > Issue Type: Improvement > Components: file manager >Affects Versions: 1.9 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > Small but important patch for upgrading the AWS s3 SDK dependency. Patch > coming up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-993) Upgrade aws-java-sdk dependency and address deprecated api usage
[ https://issues.apache.org/jira/browse/OODT-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16592688#comment-16592688 ] ASF GitHub Bot commented on OODT-993: - lewismc opened a new pull request #72: OODT-993 Upgrade aws-java-sdk dependency and address deprecated api usage URL: https://github.com/apache/oodt/pull/72 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Upgrade aws-java-sdk dependency and address deprecated api usage > > > Key: OODT-993 > URL: https://issues.apache.org/jira/browse/OODT-993 > Project: OODT > Issue Type: Improvement > Components: file manager >Affects Versions: 1.9 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > Small but important patch for upgrading the AWS s3 SDK dependency. Patch > coming up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-990) Enable OODT to build under JDK 10.X
[ https://issues.apache.org/jira/browse/OODT-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588157#comment-16588157 ] ASF GitHub Bot commented on OODT-990: - lewismc commented on issue #71: OODT-990 Enable OODT to build under JDK 10.X URL: https://github.com/apache/oodt/pull/71#issuecomment-414853693 NP @buggtb just take a look at the issues with the test I've mentioned above. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Enable OODT to build under JDK 10.X > --- > > Key: OODT-990 > URL: https://issues.apache.org/jira/browse/OODT-990 > Project: OODT > Issue Type: Improvement > Components: build proces >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Critical > Fix For: 1.9 > > > When reviewing the 1.2.4 RC, I decided to build under JDK 10.X. This issue > will document and contain a fix for any deprecation and code improvements > required to build and test master branch under JDK 10.X. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-990) Enable OODT to build under JDK 10.X
[ https://issues.apache.org/jira/browse/OODT-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588146#comment-16588146 ] ASF GitHub Bot commented on OODT-990: - buggtb commented on issue #71: OODT-990 Enable OODT to build under JDK 10.X URL: https://github.com/apache/oodt/pull/71#issuecomment-414850314 Thanks Lewis, we had this on our ticket, you've beaten us to it. I'll test it for you later this week/early next week. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Enable OODT to build under JDK 10.X > --- > > Key: OODT-990 > URL: https://issues.apache.org/jira/browse/OODT-990 > Project: OODT > Issue Type: Improvement > Components: build proces >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Critical > Fix For: 1.9 > > > When reviewing the 1.2.4 RC, I decided to build under JDK 10.X. This issue > will document and contain a fix for any deprecation and code improvements > required to build and test master branch under JDK 10.X. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-990) Enable OODT to build under JDK 10.X
[ https://issues.apache.org/jira/browse/OODT-990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587713#comment-16587713 ] ASF GitHub Bot commented on OODT-990: - lewismc opened a new pull request #71: OODT-990 Enable OODT to build under JDK 10.X URL: https://github.com/apache/oodt/pull/71 PR addresses https://issues.apache.org/jira/browse/OODT-990. The more I dug here, the more unmanageable this patch became/is becoming so what is represented here is a cleanup patch (complying with Type safety, removing unused imports, etc.) as well as addressing use of deprecated API's peppered throughout the OODT codebase. I should mention that when I run the tests from the parent pom, [TestXmlRpcWorkflowManager.java](https://github.com/apache/oodt/blob/master/workflow/src/test/java/org/apache/oodt/cas/workflow/system/TestXmlRpcWorkflowManager.java) hangs. I would like others to let me know if it hangs when they test this patch. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Enable OODT to build under JDK 10.X > --- > > Key: OODT-990 > URL: https://issues.apache.org/jira/browse/OODT-990 > Project: OODT > Issue Type: Improvement > Components: build proces >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Critical > Fix For: 1.9 > > > When reviewing the 1.2.4 RC, I decided to build under JDK 10.X. This issue > will document and contain a fix for any deprecation and code improvements > required to build and test master branch under JDK 10.X. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-992) AvroRPC WorkflowManager executeDynamicWorkflow() accepts a Map
[
https://issues.apache.org/jira/browse/OODT-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585417#comment-16585417
]
ASF GitHub Bot commented on OODT-992:
-
chrismattmann closed pull request #70: [OODT-992] Modified
AvroRPCWorkflowManager to accept a Map when creating dynamic
workflows
URL: https://github.com/apache/oodt/pull/70
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/core/pom.xml b/core/pom.xml
index 49486c76b..69bf75272 100644
--- a/core/pom.xml
+++ b/core/pom.xml
@@ -517,19 +517,16 @@ the License.
org.mortbay.jetty
jetty
${jetty.version}
-provided
org.mortbay.jetty
jetty-management
${jetty.version}
-provided
org.mortbay.jetty
jetty-util
${jetty.version}
-provided
org.python
diff --git
a/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/CASProductHandler.java
b/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/CASProductHandler.java
index 00ef50fc6..d930d8f2d 100644
---
a/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/CASProductHandler.java
+++
b/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/CASProductHandler.java
@@ -18,7 +18,6 @@
package org.apache.oodt.cas.product;
-//JDK imports
import org.apache.oodt.cas.filemgr.structs.Product;
import org.apache.oodt.cas.filemgr.structs.Reference;
@@ -49,8 +48,6 @@
import static
org.apache.oodt.cas.product.CASProductHandlerMetKeys.CAS_PROFILE_ID;
import static org.apache.oodt.cas.product.CASProductHandlerMetKeys.FILE_HEADER;
-//OODT imports
-
/**
* @author mattmann
* @version $Revision$
diff --git
a/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DatasetDeliveryServlet.java
b/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DatasetDeliveryServlet.java
index 9685e36d3..d04232c46 100644
---
a/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DatasetDeliveryServlet.java
+++
b/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/data/DatasetDeliveryServlet.java
@@ -51,8 +51,7 @@
* @author mattmann
* @version $Revision$
*/
-public class DatasetDeliveryServlet extends HttpServlet implements
-DataDeliveryKeys {
+public class DatasetDeliveryServlet extends HttpServlet implements
DataDeliveryKeys {
/* our log stream */
private static final Logger LOG = Logger
@@ -266,6 +265,14 @@ public void init(ServletConfig config) throws
ServletException {
}
+ public void destroy() {
+if (client != null) {
+ try {
+client.close();
+ } catch (IOException ignored) { }
+}
+ }
+
private boolean alreadyZipped(Product p, Map hash) {
return hash.containsKey(p.getProductName());
}
diff --git
a/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rdf/RDFDatasetServlet.java
b/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rdf/RDFDatasetServlet.java
index e15faf4f8..bf4df9d41 100644
---
a/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rdf/RDFDatasetServlet.java
+++
b/webapp/fmprod/src/main/java/org/apache/oodt/cas/product/rdf/RDFDatasetServlet.java
@@ -18,7 +18,7 @@
package org.apache.oodt.cas.product.rdf;
-//JDK imports
+
import org.apache.commons.lang.StringEscapeUtils;
import org.apache.commons.lang.StringUtils;
import org.apache.commons.lang.WordUtils;
@@ -32,15 +32,6 @@
import org.w3c.dom.Document;
import org.w3c.dom.Element;
-import java.io.FileNotFoundException;
-import java.io.IOException;
-import java.net.MalformedURLException;
-import java.net.URL;
-import java.util.List;
-import java.util.Vector;
-import java.util.logging.Level;
-import java.util.logging.Logger;
-
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
@@ -53,8 +44,14 @@
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;
-
-//OODT imports
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.util.List;
+import java.util.Vector;
+import java.util.logging.Level;
+import java.util.logging.Logger;
/**
*
@@ -266,4 +263,13 @@ public void outputRDF(List productTypes,
String base, HttpServletRe
return types;
}
+ public void destroy() {
+super.destroy();
+
+if (fClient != null) {
+ try {
+fClient.close();
+ } catch (IOException ignored) { }
+}
+ }
}
diff --git a/workflow/pom.xml b/workflow/pom.xml
index 99f12df19..55259ddac 100644
--- a/workflow/p
[jira] [Commented] (OODT-992) AvroRPC WorkflowManager executeDynamicWorkflow() accepts a Map
[ https://issues.apache.org/jira/browse/OODT-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585415#comment-16585415 ] ASF GitHub Bot commented on OODT-992: - chrismattmann commented on issue #70: [OODT-992] Modified AvroRPCWorkflowManager to accept a Map when creating dynamic workflows URL: https://github.com/apache/oodt/pull/70#issuecomment-414195138 LGTM, and all tests passed locally, with merge to master and development branches! ``` [INFO] Installing /Users/mattmann/git/oodt/pom.xml to /Users/mattmann/.m2/repository/org/apache/oodt/oodt/1.9-SNAPSHOT/oodt-1.9-SNAPSHOT.pom [INFO] [INFO] Reactor Summary: [INFO] [INFO] OODT Core .. SUCCESS [ 3.130 s] [INFO] Common Utilities ... SUCCESS [ 25.421 s] [INFO] CAS Command Line Interface . SUCCESS [ 8.795 s] [INFO] OODT - Configuration Management SUCCESS [ 28.416 s] [INFO] Process Control System Input Data Package .. SUCCESS [ 3.432 s] [INFO] Catalog and Archive Service Generic Multi-valued Metadata Container SUCCESS [ 6.208 s] [INFO] Catalog and Archive File Management Component .. SUCCESS [04:20 min] [INFO] Catalog and Archive Resource Management Component .. SUCCESS [ 51.807 s] [INFO] Catalog and Archive Workflow Management Component .. SUCCESS [01:11 min] [INFO] Catalog and Archive Crawling Framework . SUCCESS [ 37.896 s] [INFO] OODT CAS Curator Single Sign On Security Package ... SUCCESS [ 1.941 s] [INFO] CAS Curation Web Services .. SUCCESS [ 12.557 s] [INFO] Process Control System Core Package SUCCESS [ 26.957 s] [INFO] OODT Wicket Web Components . SUCCESS [ 7.039 s] [INFO] CAS Curation Interface . SUCCESS [ 16.833 s] [INFO] CAS PGE Adaptor Framework .. SUCCESS [ 35.021 s] [INFO] CAS Installer Maven Mojo ... SUCCESS [ 4.901 s] [INFO] OODT :: Archetypes :: OpsUI SUCCESS [ 1.406 s] [INFO] OODT :: Archetypes :: RADiX SUCCESS [ 1.549 s] [INFO] OODT :: Archetypes . SUCCESS [ 0.059 s] [INFO] CAS File Manager Browser Web App ... SUCCESS [ 13.497 s] [INFO] CAS Workflow Manager Monitor Web App ... SUCCESS [ 6.362 s] [INFO] CAS Product Server Web Application . SUCCESS [ 14.273 s] [INFO] CAS Workflow REST Services . SUCCESS [ 4.860 s] [INFO] Process Control System Operator Interface Webapp ... SUCCESS [ 27.878 s] [INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [ 11.487 s] [INFO] Apache OODT SUCCESS [ 0.044 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 11:25 min [INFO] Finished at: 2018-08-19T18:31:51-07:00 [INFO] Final Memory: 141M/1550M [INFO] nonas:oodt mattmann$ ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > AvroRPC WorkflowManager executeDynamicWorkflow() accepts a Map > - > > Key: OODT-992 > URL: https://issues.apache.org/jira/browse/OODT-992 > Project: OODT > Issue Type: Bug > Components: workflow manager >Affects Versions: 1.2, 1.9 >Reporter: Imesha Sudasingha >Assignee: Chris A. Mattmann >Priority: Major > > AvroRPC Workflow manager's *executeDynamicWorkflow()* accepts a Map String> which should be accepting Map where the value can be > either a String or a List. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-992) AvroRPC WorkflowManager executeDynamicWorkflow() accepts a Map
[ https://issues.apache.org/jira/browse/OODT-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16584827#comment-16584827 ] ASF GitHub Bot commented on OODT-992: - IMS94 opened a new pull request #70: [OODT-992] Modified AvroRPCWorkflowManager to accept a Map when creating dynamic workflows URL: https://github.com/apache/oodt/pull/70 [OODT-992] Modified AvroRPCWorkflowManager to accept a Map as metadata when creating dynamic workflows. Changes other associated avro and java method signatures to match the modification. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > AvroRPC WorkflowManager executeDynamicWorkflow() accepts a Map > - > > Key: OODT-992 > URL: https://issues.apache.org/jira/browse/OODT-992 > Project: OODT > Issue Type: Bug > Components: workflow manager >Affects Versions: 1.2, 1.9 >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Major > > AvroRPC Workflow manager's *executeDynamicWorkflow()* accepts a Map String> which should be accepting Map where the value can be > either a String or a List. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-693) Consolidate logging in OODT
[ https://issues.apache.org/jira/browse/OODT-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528923#comment-16528923 ] ASF GitHub Bot commented on OODT-693: - chrismattmann commented on issue #66: [OODT-693] Consolidating logging in OODT Components URL: https://github.com/apache/oodt/pull/66#issuecomment-401571927 passed great! I am going to push now: ``` [INFO] [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ pcs-services --- [INFO] Installing /Users/mattmann/git/oodt/pcs/services/target/pcs-services-1.9-SNAPSHOT.war to /Users/mattmann/.m2/repository/org/apache/oodt/pcs-services/1.9-SNAPSHOT/pcs-services-1.9-SNAPSHOT.war [INFO] Installing /Users/mattmann/git/oodt/pcs/services/pom.xml to /Users/mattmann/.m2/repository/org/apache/oodt/pcs-services/1.9-SNAPSHOT/pcs-services-1.9-SNAPSHOT.pom [INFO] [INFO] [INFO] Building Apache OODT 1.9-SNAPSHOT [INFO] [INFO] [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ oodt --- [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ oodt --- [INFO] [INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ oodt --- [INFO] [INFO] >>> maven-javadoc-plugin:2.10.3:javadoc (attach-javadocs) > generate-sources @ oodt >>> [INFO] [INFO] <<< maven-javadoc-plugin:2.10.3:javadoc (attach-javadocs) < generate-sources @ oodt <<< [INFO] [INFO] [INFO] --- maven-javadoc-plugin:2.10.3:javadoc (attach-javadocs) @ oodt --- [INFO] [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ oodt --- [INFO] Installing /Users/mattmann/git/oodt/pom.xml to /Users/mattmann/.m2/repository/org/apache/oodt/oodt/1.9-SNAPSHOT/oodt-1.9-SNAPSHOT.pom [INFO] [INFO] Reactor Summary: [INFO] [INFO] OODT Core .. SUCCESS [ 4.908 s] [INFO] Common Utilities ... SUCCESS [ 28.830 s] [INFO] CAS Command Line Interface . SUCCESS [ 8.927 s] [INFO] OODT - Configuration Management SUCCESS [ 27.971 s] [INFO] Process Control System Input Data Package .. SUCCESS [ 2.203 s] [INFO] Catalog and Archive Service Generic Multi-valued Metadata Container SUCCESS [ 4.372 s] [INFO] Catalog and Archive File Management Component .. SUCCESS [04:17 min] [INFO] Catalog and Archive Resource Management Component .. SUCCESS [ 42.786 s] [INFO] Catalog and Archive Workflow Management Component .. SUCCESS [01:01 min] [INFO] Catalog and Archive Crawling Framework . SUCCESS [ 29.711 s] [INFO] OODT CAS Curator Single Sign On Security Package ... SUCCESS [ 1.714 s] [INFO] CAS Curation Web Services .. SUCCESS [ 10.667 s] [INFO] Process Control System Core Package SUCCESS [ 22.090 s] [INFO] OODT Wicket Web Components . SUCCESS [ 6.147 s] [INFO] CAS Curation Interface . SUCCESS [ 14.479 s] [INFO] CAS PGE Adaptor Framework .. SUCCESS [ 30.094 s] [INFO] CAS Installer Maven Mojo ... SUCCESS [ 12.110 s] [INFO] OODT :: Archetypes :: OpsUI SUCCESS [ 2.737 s] [INFO] OODT :: Archetypes :: RADiX SUCCESS [ 1.615 s] [INFO] OODT :: Archetypes . SUCCESS [ 0.074 s] [INFO] CAS File Manager Browser Web App ... SUCCESS [ 15.881 s] [INFO] CAS Workflow Manager Monitor Web App ... SUCCESS [ 6.242 s] [INFO] CAS Product Server Web Application . SUCCESS [ 13.907 s] [INFO] CAS Workflow REST Services . SUCCESS [ 4.266 s] [INFO] Process Control System Operator Interface Webapp ... SUCCESS [ 28.865 s] [INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [ 8.972 s] [INFO] Apache OODT SUCCESS [ 0.046 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 10:50 min [INFO] Finished at: 2018-06-30T13:51:50-07:00 [INFO] Final Memory: 140M/1534M [INFO] nonas:oodt mattmann$ ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use t
[jira] [Commented] (OODT-693) Consolidate logging in OODT
[ https://issues.apache.org/jira/browse/OODT-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528862#comment-16528862 ] ASF GitHub Bot commented on OODT-693: - IMS94 commented on issue #66: [OODT-693] Consolidating logging in OODT Components URL: https://github.com/apache/oodt/pull/66#issuecomment-401560521 Fixed the problem. Tests are passing in my machine now. Can you check? @chrismattmann This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Consolidate logging in OODT > --- > > Key: OODT-693 > URL: https://issues.apache.org/jira/browse/OODT-693 > Project: OODT > Issue Type: Improvement > Components: file manager >Affects Versions: 0.6 >Reporter: Lewis John McGibbney >Assignee: Imesha Sudasingha >Priority: Major > Fix For: 1.1 > > > Right now we seem to be using an array of inconsistent logging frameworks. > Personally I really like the Slf4j over Log4j setup. Extremely easy to work > with, easy to configure and Log4j 2.X is dynamite so if we can implement that > then we are laughing. > This is by no means a trivial task. > It is however an important one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-693) Consolidate logging in OODT
[ https://issues.apache.org/jira/browse/OODT-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528814#comment-16528814 ] ASF GitHub Bot commented on OODT-693: - chrismattmann commented on issue #66: [OODT-693] Consolidating logging in OODT Components URL: https://github.com/apache/oodt/pull/66#issuecomment-401552598 can't commit, I tried to build and am getting this error: ``` Running org.apache.oodt.cas.filemgr.metadata.extractors.examples.TestMimeTypeExtractor Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.634 sec - in org.apache.oodt.cas.filemgr.metadata.extractors.examples.TestMimeTypeExtractor Running org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.668 sec - in org.apache.oodt.cas.filemgr.validation.TestXMLValidationLayer Results : Tests in error: TestDeleteProductByIdCliAction.testValidateErrors:78 » NullPointer Tests run: 230, Failures: 0, Errors: 1, Skipped: 0 [INFO] [INFO] Reactor Summary: [INFO] [INFO] OODT Core .. SUCCESS [ 3.272 s] [INFO] Common Utilities ... SUCCESS [ 26.911 s] [INFO] CAS Command Line Interface . SUCCESS [ 14.958 s] [INFO] OODT - Configuration Management SUCCESS [ 29.047 s] [INFO] Process Control System Input Data Package .. SUCCESS [ 2.032 s] [INFO] Catalog and Archive Service Generic Multi-valued Metadata Container SUCCESS [ 3.255 s] [INFO] Catalog and Archive File Management Component .. FAILURE [03:41 min] [INFO] Catalog and Archive Resource Management Component .. SKIPPED [INFO] Catalog and Archive Workflow Management Component .. SKIPPED [INFO] Catalog and Archive Crawling Framework . SKIPPED [INFO] OODT CAS Curator Single Sign On Security Package ... SKIPPED [INFO] CAS Curation Web Services .. SKIPPED [INFO] Process Control System Core Package SKIPPED [INFO] OODT Wicket Web Components . SKIPPED [INFO] CAS Curation Interface . SKIPPED [INFO] CAS PGE Adaptor Framework .. SKIPPED [INFO] CAS Installer Maven Mojo ... SKIPPED [INFO] OODT :: Archetypes :: OpsUI SKIPPED [INFO] OODT :: Archetypes :: RADiX SKIPPED [INFO] OODT :: Archetypes . SKIPPED [INFO] CAS File Manager Browser Web App ... SKIPPED [INFO] CAS Workflow Manager Monitor Web App ... SKIPPED [INFO] CAS Product Server Web Application . SKIPPED [INFO] CAS Workflow REST Services . SKIPPED [INFO] Process Control System Operator Interface Webapp ... SKIPPED [INFO] OODT Process Control System JAX-RS service layer ... SKIPPED [INFO] Apache OODT SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 05:01 min [INFO] Finished at: 2018-06-30T09:28:36-07:00 [INFO] Final Memory: 77M/1276M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project cas-filemgr: There are test failures. [ERROR] [ERROR] Please refer to /Users/mattmann/git/oodt/filemgr/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :cas-filemgr nonas:oodt mattmann$ ``` @IMS94 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Consolidate logging in OODT > --- > > Key: OODT-693 > URL: https://issues.apache.o
[jira] [Commented] (OODT-693) Consolidate logging in OODT
[
https://issues.apache.org/jira/browse/OODT-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528810#comment-16528810
]
ASF GitHub Bot commented on OODT-693:
-
IMS94 commented on a change in pull request #66: [OODT-693] Consolidating
logging in OODT Components
URL: https://github.com/apache/oodt/pull/66#discussion_r199326612
##
File path:
filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/FileManagerCliAction.java
##
@@ -16,52 +16,47 @@
*/
package org.apache.oodt.cas.filemgr.cli.action;
-//JDK imports
import org.apache.commons.lang.Validate;
import org.apache.oodt.cas.cli.action.CmdLineAction;
import org.apache.oodt.cas.filemgr.structs.exceptions.ConnectionException;
import org.apache.oodt.cas.filemgr.system.FileManagerClient;
import org.apache.oodt.cas.filemgr.util.RpcCommunicationFactory;
-import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
-//Apache imports
-//OODT imports
-
/**
* Base {@link CmdLineAction} for File Manager.
*
* @author bfoster (Brian Foster)
*/
public abstract class FileManagerCliAction extends CmdLineAction {
- private FileManagerClient client;
+private FileManagerClient fmc;
- public String getUrl() {
- return System.getProperty("org.apache.oodt.cas.filemgr.url");
- }
+public String getUrl() {
+return System.getProperty("org.apache.oodt.cas.filemgr.url");
+}
- protected FileManagerClient getClient() throws MalformedURLException,
ConnectionException {
- Validate.notNull(getUrl(), "Must specify url");
+/**
+ * TODO(imesha) Fix client closing problem which makes it usable only once
Review comment:
Actually this problem is still there. Let it be for now. The problem is, we
need to remove the `fmc` attribute of `FileManagerCLIAction`, but cannot be
done since the tests are using it to add a mock filemanager client.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Consolidate logging in OODT
> ---
>
> Key: OODT-693
> URL: https://issues.apache.org/jira/browse/OODT-693
> Project: OODT
> Issue Type: Improvement
> Components: file manager
>Affects Versions: 0.6
>Reporter: Lewis John McGibbney
>Assignee: Imesha Sudasingha
>Priority: Major
> Fix For: 1.1
>
>
> Right now we seem to be using an array of inconsistent logging frameworks.
> Personally I really like the Slf4j over Log4j setup. Extremely easy to work
> with, easy to configure and Log4j 2.X is dynamite so if we can implement that
> then we are laughing.
> This is by no means a trivial task.
> It is however an important one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-693) Consolidate logging in OODT
[
https://issues.apache.org/jira/browse/OODT-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528807#comment-16528807
]
ASF GitHub Bot commented on OODT-693:
-
IMS94 commented on a change in pull request #66: [OODT-693] Consolidating
logging in OODT Components
URL: https://github.com/apache/oodt/pull/66#discussion_r199326466
##
File path:
filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/FileManagerCliAction.java
##
@@ -16,52 +16,47 @@
*/
package org.apache.oodt.cas.filemgr.cli.action;
-//JDK imports
import org.apache.commons.lang.Validate;
import org.apache.oodt.cas.cli.action.CmdLineAction;
import org.apache.oodt.cas.filemgr.structs.exceptions.ConnectionException;
import org.apache.oodt.cas.filemgr.system.FileManagerClient;
import org.apache.oodt.cas.filemgr.util.RpcCommunicationFactory;
-import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
-//Apache imports
-//OODT imports
-
/**
* Base {@link CmdLineAction} for File Manager.
*
* @author bfoster (Brian Foster)
*/
public abstract class FileManagerCliAction extends CmdLineAction {
- private FileManagerClient client;
+private FileManagerClient fmc;
- public String getUrl() {
- return System.getProperty("org.apache.oodt.cas.filemgr.url");
- }
+public String getUrl() {
+return System.getProperty("org.apache.oodt.cas.filemgr.url");
+}
- protected FileManagerClient getClient() throws MalformedURLException,
ConnectionException {
- Validate.notNull(getUrl(), "Must specify url");
+/**
+ * TODO(imesha) Fix client closing problem which makes it usable only once
Review comment:
This TODO item can be removed now. Because I have addressed this in each CLI
action by adding a **try with resources**:`try (FileManagerClient client =
getClient())` Will remove this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Consolidate logging in OODT
> ---
>
> Key: OODT-693
> URL: https://issues.apache.org/jira/browse/OODT-693
> Project: OODT
> Issue Type: Improvement
> Components: file manager
>Affects Versions: 0.6
>Reporter: Lewis John McGibbney
>Priority: Major
> Fix For: 1.1
>
>
> Right now we seem to be using an array of inconsistent logging frameworks.
> Personally I really like the Slf4j over Log4j setup. Extremely easy to work
> with, easy to configure and Log4j 2.X is dynamite so if we can implement that
> then we are laughing.
> This is by no means a trivial task.
> It is however an important one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-693) Consolidate logging in OODT
[
https://issues.apache.org/jira/browse/OODT-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528804#comment-16528804
]
ASF GitHub Bot commented on OODT-693:
-
chrismattmann commented on a change in pull request #66: [OODT-693]
Consolidating logging in OODT Components
URL: https://github.com/apache/oodt/pull/66#discussion_r199326290
##
File path: pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java
##
@@ -135,30 +148,27 @@ public void run(Metadata metadata,
WorkflowTaskConfiguration config) throws Work
// Commit dynamic metadata.
updateDynamicMetadata();
} catch (Exception e) {
- logger.log(Level.SEVERE, "PGETask FAILED!!! : " + e.getMessage(), e);
- throw new WorkflowTaskInstanceException("PGETask FAILED!!! : "
- + e.getMessage(), e);
+ logger.error("PGETask FAILED!!! Error occurred when running", e);
+ throw new WorkflowTaskInstanceException("PGETask FAILED!!! : " +
e.getMessage(), e);
}
}
protected void updateStatus(String status) throws Exception {
logger.info("Updating status to workflow as [" + status + "]");
if
(!getWorkflowManagerClient().updateWorkflowInstanceStatus(workflowInstId,
status)) {
- throw new PGEException(
- "Failed to update workflow status : client returned false");
+ throw new PGEException("Failed to update workflow status : client
returned false");
}
}
- protected Logger createLogger() throws IOException, PGEException {
+ protected java.util.logging.Logger createLogger() throws IOException,
PGEException {
File logDir = new File(pgeConfig.getExeDir(), "logs");
if (!(logDir.exists() || logDir.mkdirs())) {
throw new PGEException("mkdirs for logs directory return false");
}
- Logger logger = Logger.getLogger(PGETaskInstance.class.getName()
-+ "." + workflowInstId);
- FileHandler handler = new FileHandler(
-new File(logDir, createLogFileName()).getAbsolutePath());
+ java.util.logging.Logger logger =
java.util.logging.Logger.getLogger(PGETaskInstance.class.getName() + "." +
workflowInstId);
+ // TODO Need to find an alternative way to add a dynamic handler to
write workflowInstance logs to a separate file
Review comment:
create a JIRA issue for this
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Consolidate logging in OODT
> ---
>
> Key: OODT-693
> URL: https://issues.apache.org/jira/browse/OODT-693
> Project: OODT
> Issue Type: Improvement
> Components: file manager
>Affects Versions: 0.6
>Reporter: Lewis John McGibbney
>Priority: Major
> Fix For: 1.1
>
>
> Right now we seem to be using an array of inconsistent logging frameworks.
> Personally I really like the Slf4j over Log4j setup. Extremely easy to work
> with, easy to configure and Log4j 2.X is dynamite so if we can implement that
> then we are laughing.
> This is by no means a trivial task.
> It is however an important one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-693) Consolidate logging in OODT
[
https://issues.apache.org/jira/browse/OODT-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528805#comment-16528805
]
ASF GitHub Bot commented on OODT-693:
-
Github user chrismattmann commented on a diff in the pull request:
https://github.com/apache/oodt/pull/66#discussion_r199326290
--- Diff: pge/src/main/java/org/apache/oodt/cas/pge/PGETaskInstance.java ---
@@ -135,30 +148,27 @@ public void run(Metadata metadata,
WorkflowTaskConfiguration config) throws Work
// Commit dynamic metadata.
updateDynamicMetadata();
} catch (Exception e) {
- logger.log(Level.SEVERE, "PGETask FAILED!!! : " + e.getMessage(),
e);
- throw new WorkflowTaskInstanceException("PGETask FAILED!!! : "
- + e.getMessage(), e);
+ logger.error("PGETask FAILED!!! Error occurred when running", e);
+ throw new WorkflowTaskInstanceException("PGETask FAILED!!! : " +
e.getMessage(), e);
}
}
protected void updateStatus(String status) throws Exception {
logger.info("Updating status to workflow as [" + status + "]");
if
(!getWorkflowManagerClient().updateWorkflowInstanceStatus(workflowInstId,
status)) {
- throw new PGEException(
- "Failed to update workflow status : client returned false");
+ throw new PGEException("Failed to update workflow status : client
returned false");
}
}
- protected Logger createLogger() throws IOException, PGEException {
+ protected java.util.logging.Logger createLogger() throws IOException,
PGEException {
File logDir = new File(pgeConfig.getExeDir(), "logs");
if (!(logDir.exists() || logDir.mkdirs())) {
throw new PGEException("mkdirs for logs directory return false");
}
- Logger logger = Logger.getLogger(PGETaskInstance.class.getName()
-+ "." + workflowInstId);
- FileHandler handler = new FileHandler(
-new File(logDir, createLogFileName()).getAbsolutePath());
+ java.util.logging.Logger logger =
java.util.logging.Logger.getLogger(PGETaskInstance.class.getName() + "." +
workflowInstId);
+ // TODO Need to find an alternative way to add a dynamic handler to
write workflowInstance logs to a separate file
--- End diff --
create a JIRA issue for this
> Consolidate logging in OODT
> ---
>
> Key: OODT-693
> URL: https://issues.apache.org/jira/browse/OODT-693
> Project: OODT
> Issue Type: Improvement
> Components: file manager
>Affects Versions: 0.6
>Reporter: Lewis John McGibbney
>Priority: Major
> Fix For: 1.1
>
>
> Right now we seem to be using an array of inconsistent logging frameworks.
> Personally I really like the Slf4j over Log4j setup. Extremely easy to work
> with, easy to configure and Log4j 2.X is dynamite so if we can implement that
> then we are laughing.
> This is by no means a trivial task.
> It is however an important one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-693) Consolidate logging in OODT
[
https://issues.apache.org/jira/browse/OODT-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528801#comment-16528801
]
ASF GitHub Bot commented on OODT-693:
-
chrismattmann commented on a change in pull request #66: [OODT-693]
Consolidating logging in OODT Components
URL: https://github.com/apache/oodt/pull/66#discussion_r199326255
##
File path:
filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/FileManagerCliAction.java
##
@@ -16,52 +16,47 @@
*/
package org.apache.oodt.cas.filemgr.cli.action;
-//JDK imports
import org.apache.commons.lang.Validate;
import org.apache.oodt.cas.cli.action.CmdLineAction;
import org.apache.oodt.cas.filemgr.structs.exceptions.ConnectionException;
import org.apache.oodt.cas.filemgr.system.FileManagerClient;
import org.apache.oodt.cas.filemgr.util.RpcCommunicationFactory;
-import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
-//Apache imports
-//OODT imports
-
/**
* Base {@link CmdLineAction} for File Manager.
*
* @author bfoster (Brian Foster)
*/
public abstract class FileManagerCliAction extends CmdLineAction {
- private FileManagerClient client;
+private FileManagerClient fmc;
- public String getUrl() {
- return System.getProperty("org.apache.oodt.cas.filemgr.url");
- }
+public String getUrl() {
+return System.getProperty("org.apache.oodt.cas.filemgr.url");
+}
- protected FileManagerClient getClient() throws MalformedURLException,
ConnectionException {
- Validate.notNull(getUrl(), "Must specify url");
+/**
+ * TODO(imesha) Fix client closing problem which makes it usable only once
Review comment:
should we open up a separate issue for this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Consolidate logging in OODT
> ---
>
> Key: OODT-693
> URL: https://issues.apache.org/jira/browse/OODT-693
> Project: OODT
> Issue Type: Improvement
> Components: file manager
>Affects Versions: 0.6
>Reporter: Lewis John McGibbney
>Priority: Major
> Fix For: 1.1
>
>
> Right now we seem to be using an array of inconsistent logging frameworks.
> Personally I really like the Slf4j over Log4j setup. Extremely easy to work
> with, easy to configure and Log4j 2.X is dynamite so if we can implement that
> then we are laughing.
> This is by no means a trivial task.
> It is however an important one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-693) Consolidate logging in OODT
[
https://issues.apache.org/jira/browse/OODT-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528802#comment-16528802
]
ASF GitHub Bot commented on OODT-693:
-
Github user chrismattmann commented on a diff in the pull request:
https://github.com/apache/oodt/pull/66#discussion_r199326255
--- Diff:
filemgr/src/main/java/org/apache/oodt/cas/filemgr/cli/action/FileManagerCliAction.java
---
@@ -16,52 +16,47 @@
*/
package org.apache.oodt.cas.filemgr.cli.action;
-//JDK imports
import org.apache.commons.lang.Validate;
import org.apache.oodt.cas.cli.action.CmdLineAction;
import org.apache.oodt.cas.filemgr.structs.exceptions.ConnectionException;
import org.apache.oodt.cas.filemgr.system.FileManagerClient;
import org.apache.oodt.cas.filemgr.util.RpcCommunicationFactory;
-import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URL;
-//Apache imports
-//OODT imports
-
/**
* Base {@link CmdLineAction} for File Manager.
*
* @author bfoster (Brian Foster)
*/
public abstract class FileManagerCliAction extends CmdLineAction {
- private FileManagerClient client;
+private FileManagerClient fmc;
- public String getUrl() {
- return System.getProperty("org.apache.oodt.cas.filemgr.url");
- }
+public String getUrl() {
+return System.getProperty("org.apache.oodt.cas.filemgr.url");
+}
- protected FileManagerClient getClient() throws MalformedURLException,
ConnectionException {
- Validate.notNull(getUrl(), "Must specify url");
+/**
+ * TODO(imesha) Fix client closing problem which makes it usable only
once
--- End diff --
should we open up a separate issue for this?
> Consolidate logging in OODT
> ---
>
> Key: OODT-693
> URL: https://issues.apache.org/jira/browse/OODT-693
> Project: OODT
> Issue Type: Improvement
> Components: file manager
>Affects Versions: 0.6
>Reporter: Lewis John McGibbney
>Priority: Major
> Fix For: 1.1
>
>
> Right now we seem to be using an array of inconsistent logging frameworks.
> Personally I really like the Slf4j over Log4j setup. Extremely easy to work
> with, easy to configure and Log4j 2.X is dynamite so if we can implement that
> then we are laughing.
> This is by no means a trivial task.
> It is however an important one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OODT-976) Upgrade Apache Tika to 1.18
[ https://issues.apache.org/jira/browse/OODT-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528794#comment-16528794 ] ASF GitHub Bot commented on OODT-976: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/61 @IMS94 can you take a look at this? > Upgrade Apache Tika to 1.18 > --- > > Key: OODT-976 > URL: https://issues.apache.org/jira/browse/OODT-976 > Project: OODT > Issue Type: Improvement > Components: core, file manager, radix >Affects Versions: 1.2.2 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > tika-core and tika-parsers 1.13 dependencies are pretty outdated now. Tika > 1.18 is available so we should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-693) Consolidate logging in OODT
[ https://issues.apache.org/jira/browse/OODT-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528725#comment-16528725 ] ASF GitHub Bot commented on OODT-693: - GitHub user IMS94 opened a pull request: https://github.com/apache/oodt/pull/66 [OODT-693] Consolidating logging in OODT Components This is the first PR of consolidating logging support in OODT. Configuration management, file manager, workflow manager, crawler and PGE have been updated to use SLF4J. Log redirection from Java util logs to SLF4J has been also used as a temporary solution which will be removed in future. You can merge this pull request into a Git repository by running: $ git pull https://github.com/IMS94/oodt OODT-693 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/oodt/pull/66.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #66 commit 86b877da6e6e81818cf5915095f10cf9ca7b8685 Author: Imesha Sudasingha Date: 2018-06-02T11:08:06Z [OODT-987] Upgraded oodt-conf package from log4j to log4j2 commit 6ef3e9bb6170b9c7113e0b356e3446323b27517a Author: Imesha Sudasingha Date: 2018-06-03T04:27:43Z [OODT-982] Updated crawler logging to log4j2 commit ecd1756e94dfb99cd8df887e7fbce17ebff4015f Author: Imesha Sudasingha Date: 2018-06-03T04:28:46Z [OODT-693] Added maven dependency to redirect JUL to slf4j commit a5dcd9e7fb13c253fe6bcb830ed5099fe344bc60 Author: Imesha Sudasingha Date: 2018-06-03T04:30:32Z [OODT-981] slf4j logging improvements and workflow/file manager client usage improvements to close once done commit 94af8ccbb95a95722d4372269512db459ca07b00 Author: Imesha Sudasingha Date: 2018-06-03T04:32:14Z [OODT-980] slf4j logging improvements and file manager CLI improvements for closing client commit e5d1b7d190b8ba431137e1f780880e7c4905c3d2 Author: Imesha Sudasingha Date: 2018-06-09T17:56:23Z [OODT-981] Further logging improvements for workflow manager commit c77ca81c5fccdcab6c072853d7be051e869a0a35 Author: Imesha Sudasingha Date: 2018-06-10T06:42:07Z [OODT-984] Logging improvements for PGE and fixed NPE in workflowInstanceId in PGETaskInstance > Consolidate logging in OODT > --- > > Key: OODT-693 > URL: https://issues.apache.org/jira/browse/OODT-693 > Project: OODT > Issue Type: Improvement > Components: file manager >Affects Versions: 0.6 >Reporter: Lewis John McGibbney >Priority: Major > Fix For: 1.1 > > > Right now we seem to be using an array of inconsistent logging frameworks. > Personally I really like the Slf4j over Log4j setup. Extremely easy to work > with, easy to configure and Log4j 2.X is dynamite so if we can implement that > then we are laughing. > This is by no means a trivial task. > It is however an important one. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-987) Update Configuration Management to use Log4J2 over Log4J
[ https://issues.apache.org/jira/browse/OODT-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528715#comment-16528715 ] ASF GitHub Bot commented on OODT-987: - Github user IMS94 commented on the issue: https://github.com/apache/oodt/pull/65 Closing this one to open a new PR containing all the logging related changes > Update Configuration Management to use Log4J2 over Log4J > > > Key: OODT-987 > URL: https://issues.apache.org/jira/browse/OODT-987 > Project: OODT > Issue Type: Sub-task >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Minor > > OODT Configuration Management module already use SLF4J logging API, but with > Log4J as the implementation. The task is to update logging implementation to > Log4J2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-987) Update Configuration Management to use Log4J2 over Log4J
[ https://issues.apache.org/jira/browse/OODT-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528716#comment-16528716 ] ASF GitHub Bot commented on OODT-987: - Github user IMS94 closed the pull request at: https://github.com/apache/oodt/pull/65 > Update Configuration Management to use Log4J2 over Log4J > > > Key: OODT-987 > URL: https://issues.apache.org/jira/browse/OODT-987 > Project: OODT > Issue Type: Sub-task >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Minor > > OODT Configuration Management module already use SLF4J logging API, but with > Log4J as the implementation. The task is to update logging implementation to > Log4J2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-987) Update Configuration Management to use Log4J2 over Log4J
[ https://issues.apache.org/jira/browse/OODT-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16528708#comment-16528708 ] ASF GitHub Bot commented on OODT-987: - GitHub user IMS94 opened a pull request: https://github.com/apache/oodt/pull/65 [OODT-987] Upgraded oodt-conf package from log4j to log4j2 You can merge this pull request into a Git repository by running: $ git pull https://github.com/IMS94/oodt OODT-987 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/oodt/pull/65.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #65 commit e8965e17b1e0c8f31cabf4181b64fb1c7fadf8ff Author: Imesha Sudasingha Date: 2018-06-30T13:35:00Z Upgraded oodt-conf package from log4j to log4j2 > Update Configuration Management to use Log4J2 over Log4J > > > Key: OODT-987 > URL: https://issues.apache.org/jira/browse/OODT-987 > Project: OODT > Issue Type: Sub-task >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Minor > > OODT Configuration Management module already use SLF4J logging API, but with > Log4J as the implementation. The task is to update logging implementation to > Log4J2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-979) [AvroRPC] WorkflowManagerClient CLI hangs after starting a dynamic workflow
[ https://issues.apache.org/jira/browse/OODT-979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16490917#comment-16490917 ] ASF GitHub Bot commented on OODT-979: - Github user asfgit closed the pull request at: https://github.com/apache/oodt/pull/63 > [AvroRPC] WorkflowManagerClient CLI hangs after starting a dynamic workflow > --- > > Key: OODT-979 > URL: https://issues.apache.org/jira/browse/OODT-979 > Project: OODT > Issue Type: Bug > Components: workflow manager >Affects Versions: 1.9 >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Major > Fix For: 1.9 > > > When WM client is upgraded to use AvroRPCWorkflowManagerClient, the CLI hangs > after starting the workflow instance. This happens because the underlying > Netty Transceiver used has a running threadpool even after the communication > is complete. Closing the NettyTranceiver fixes this issue which was > implemented in OODT-978. > So, updating workflow manager components to close the WMClient once done is > the solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-979) [AvroRPC] WorkflowManagerClient CLI hangs after starting a dynamic workflow
[ https://issues.apache.org/jira/browse/OODT-979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16490915#comment-16490915 ] ASF GitHub Bot commented on OODT-979: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/63 works fine! ``` [INFO] [INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-descriptor) @ oodt --- [INFO] [INFO] >>> maven-javadoc-plugin:2.10.3:javadoc (attach-javadocs) > generate-sources @ oodt >>> [INFO] [INFO] <<< maven-javadoc-plugin:2.10.3:javadoc (attach-javadocs) < generate-sources @ oodt <<< [INFO] [INFO] [INFO] --- maven-javadoc-plugin:2.10.3:javadoc (attach-javadocs) @ oodt --- [INFO] [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ oodt --- [INFO] Installing /Users/mattmann/git/oodt/pom.xml to /Users/mattmann/.m2/repository/org/apache/oodt/oodt/1.9-SNAPSHOT/oodt-1.9-SNAPSHOT.pom [INFO] [INFO] Reactor Summary: [INFO] [INFO] OODT Core .. SUCCESS [ 2.966 s] [INFO] Common Utilities ... SUCCESS [ 25.553 s] [INFO] CAS Command Line Interface . SUCCESS [ 9.238 s] [INFO] OODT - Configuration Management SUCCESS [ 26.115 s] [INFO] Process Control System Input Data Package .. SUCCESS [ 2.539 s] [INFO] Catalog and Archive Service Generic Multi-valued Metadata Container SUCCESS [ 3.944 s] [INFO] Catalog and Archive File Management Component .. SUCCESS [04:00 min] [INFO] Catalog and Archive Resource Management Component .. SUCCESS [ 34.679 s] [INFO] Catalog and Archive Workflow Management Component .. SUCCESS [ 44.468 s] [INFO] Catalog and Archive Crawling Framework . SUCCESS [ 25.364 s] [INFO] OODT CAS Curator Single Sign On Security Package ... SUCCESS [ 1.768 s] [INFO] CAS Curation Web Services .. SUCCESS [ 10.596 s] [INFO] Process Control System Core Package SUCCESS [ 22.956 s] [INFO] OODT Wicket Web Components . SUCCESS [ 6.353 s] [INFO] CAS Curation Interface . SUCCESS [ 17.284 s] [INFO] CAS PGE Adaptor Framework .. SUCCESS [ 31.880 s] [INFO] CAS Installer Maven Mojo ... SUCCESS [ 5.200 s] [INFO] OODT :: Archetypes :: OpsUI SUCCESS [ 1.831 s] [INFO] OODT :: Archetypes :: RADiX SUCCESS [ 1.496 s] [INFO] OODT :: Archetypes . SUCCESS [ 0.067 s] [INFO] CAS File Manager Browser Web App ... SUCCESS [ 14.612 s] [INFO] CAS Workflow Manager Monitor Web App ... SUCCESS [ 5.818 s] [INFO] CAS Product Server Web Application . SUCCESS [ 13.191 s] [INFO] CAS Workflow REST Services . SUCCESS [ 4.052 s] [INFO] Process Control System Operator Interface Webapp ... SUCCESS [ 24.666 s] [INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [ 9.585 s] [INFO] Apache OODT SUCCESS [ 0.020 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 09:47 min [INFO] Finished at: 2018-05-25T09:02:12-07:00 [INFO] Final Memory: 137M/1556M [INFO] nonas:oodt mattmann$ ``` commiting now! > [AvroRPC] WorkflowManagerClient CLI hangs after starting a dynamic workflow > --- > > Key: OODT-979 > URL: https://issues.apache.org/jira/browse/OODT-979 > Project: OODT > Issue Type: Bug > Components: workflow manager >Affects Versions: 1.9 >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Major > Fix For: 1.9 > > > When WM client is upgraded to use AvroRPCWorkflowManagerClient, the CLI hangs > after starting the workflow instance. This happens because the underlying > Netty Transceiver used has a running threadpool even after the communication > is complete. Closing the NettyTranceiver fixes this issue which was > implemented in OODT-978. > So, updating workflow manager components to close the WMClient once done is > the solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-979) [AvroRPC] WorkflowManagerClient CLI hangs after starting a dynamic workflow
[ https://issues.apache.org/jira/browse/OODT-979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16490289#comment-16490289 ] ASF GitHub Bot commented on OODT-979: - GitHub user IMS94 opened a pull request: https://github.com/apache/oodt/pull/63 [OODT-979] Stabilized AvroRpc workflow manager client You can merge this pull request into a Git repository by running: $ git pull https://github.com/IMS94/oodt master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/oodt/pull/63.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #63 commit 836499df6cb95b646fa557048ae3dd73ebde2928 Author: Imesha Sudasingha Date: 2018-05-25T05:50:46Z [OODT-979] Removed XML RPC versions of workflow manager client from all possible places and added client.close() wherever necessary. Now the WM client CLI is working without any hanging. This commit is the last of many which were to stabilize Avro RPC implementations. > [AvroRPC] WorkflowManagerClient CLI hangs after starting a dynamic workflow > --- > > Key: OODT-979 > URL: https://issues.apache.org/jira/browse/OODT-979 > Project: OODT > Issue Type: Bug > Components: workflow manager >Affects Versions: 1.9 >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Major > Fix For: 1.9 > > > When WM client is upgraded to use AvroRPCWorkflowManagerClient, the CLI hangs > after starting the workflow instance. This happens because the underlying > Netty Transceiver used has a running threadpool even after the communication > is complete. Closing the NettyTranceiver fixes this issue which was > implemented in OODT-978. > So, updating workflow manager components to close the WMClient once done is > the solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-976) Upgrade Apache Tika to 1.18
[ https://issues.apache.org/jira/browse/OODT-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488277#comment-16488277 ] ASF GitHub Bot commented on OODT-976: - Github user IMS94 commented on the issue: https://github.com/apache/oodt/pull/61 @chrismattmann Sure. Will look at this. > Upgrade Apache Tika to 1.18 > --- > > Key: OODT-976 > URL: https://issues.apache.org/jira/browse/OODT-976 > Project: OODT > Issue Type: Improvement > Components: core, file manager, radix >Affects Versions: 1.2.2 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > tika-core and tika-parsers 1.13 dependencies are pretty outdated now. Tika > 1.18 is available so we should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-976) Upgrade Apache Tika to 1.18
[ https://issues.apache.org/jira/browse/OODT-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488050#comment-16488050 ] ASF GitHub Bot commented on OODT-976: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/61 @IMS94 can you take a look at this? Do tests pass with the upgrade? > Upgrade Apache Tika to 1.18 > --- > > Key: OODT-976 > URL: https://issues.apache.org/jira/browse/OODT-976 > Project: OODT > Issue Type: Improvement > Components: core, file manager, radix >Affects Versions: 1.2.2 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > tika-core and tika-parsers 1.13 dependencies are pretty outdated now. Tika > 1.18 is available so we should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-976) Upgrade Apache Tika to 1.18
[ https://issues.apache.org/jira/browse/OODT-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480924#comment-16480924 ] ASF GitHub Bot commented on OODT-976: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/61 @lewismc can't you simply monkey patch DRAT to use the new Tika transitively that OODT would install should this PR be merged, and test it and report back? If I get time today I'll try that myself. > Upgrade Apache Tika to 1.18 > --- > > Key: OODT-976 > URL: https://issues.apache.org/jira/browse/OODT-976 > Project: OODT > Issue Type: Improvement > Components: core, file manager, radix >Affects Versions: 1.2.2 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > tika-core and tika-parsers 1.13 dependencies are pretty outdated now. Tika > 1.18 is available so we should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-978) Crawler Launcher hangs after ingesting files when Avro File Manager client is used
[ https://issues.apache.org/jira/browse/OODT-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480846#comment-16480846 ] ASF GitHub Bot commented on OODT-978: - Github user asfgit closed the pull request at: https://github.com/apache/oodt/pull/62 > Crawler Launcher hangs after ingesting files when Avro File Manager client is > used > -- > > Key: OODT-978 > URL: https://issues.apache.org/jira/browse/OODT-978 > Project: OODT > Issue Type: Bug > Components: crawler, file manager >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Critical > > When we run the crawler with Avro File manager client (within StdIngester), > the crawler hangs even after ingesting all the files with the directory. > This was caused by Netty Transceiver still running in the background which is > used by Avro FM client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-978) Crawler Launcher hangs after ingesting files when Avro File Manager client is used
[ https://issues.apache.org/jira/browse/OODT-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480839#comment-16480839 ] ASF GitHub Bot commented on OODT-978: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/62 Works great now, I'm going to merge it! ``` [INFO] --- maven-javadoc-plugin:2.10.3:javadoc (attach-javadocs) @ oodt --- [INFO] [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ oodt --- [INFO] Installing /Users/mattmann/git/oodt/pom.xml to /Users/mattmann/.m2/repository/org/apache/oodt/oodt/1.9-SNAPSHOT/oodt-1.9-SNAPSHOT.pom [INFO] [INFO] Reactor Summary: [INFO] [INFO] OODT Core .. SUCCESS [ 4.124 s] [INFO] Common Utilities ... SUCCESS [ 30.957 s] [INFO] CAS Command Line Interface . SUCCESS [ 12.048 s] [INFO] OODT - Configuration Management SUCCESS [ 34.007 s] [INFO] Process Control System Input Data Package .. SUCCESS [ 3.404 s] [INFO] Catalog and Archive Service Generic Multi-valued Metadata Container SUCCESS [ 4.103 s] [INFO] Catalog and Archive File Management Component .. SUCCESS [03:58 min] [INFO] Catalog and Archive Resource Management Component .. SUCCESS [ 31.632 s] [INFO] Catalog and Archive Workflow Management Component .. SUCCESS [ 41.087 s] [INFO] Catalog and Archive Crawling Framework . SUCCESS [ 28.410 s] [INFO] OODT CAS Curator Single Sign On Security Package ... SUCCESS [ 1.701 s] [INFO] CAS Curation Web Services .. SUCCESS [ 11.687 s] [INFO] Process Control System Core Package SUCCESS [ 19.636 s] [INFO] OODT Wicket Web Components . SUCCESS [ 6.191 s] [INFO] CAS Curation Interface . SUCCESS [ 16.762 s] [INFO] CAS PGE Adaptor Framework .. SUCCESS [ 27.703 s] [INFO] CAS Installer Maven Mojo ... SUCCESS [ 6.616 s] [INFO] OODT :: Archetypes :: OpsUI SUCCESS [ 2.411 s] [INFO] OODT :: Archetypes :: RADiX SUCCESS [ 1.142 s] [INFO] OODT :: Archetypes . SUCCESS [ 0.025 s] [INFO] CAS File Manager Browser Web App ... SUCCESS [ 13.762 s] [INFO] CAS Workflow Manager Monitor Web App ... SUCCESS [ 6.312 s] [INFO] CAS Product Server Web Application . SUCCESS [ 13.493 s] [INFO] CAS Workflow REST Services . SUCCESS [ 5.431 s] [INFO] Process Control System Operator Interface Webapp ... SUCCESS [ 31.500 s] [INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [ 8.884 s] [INFO] Apache OODT SUCCESS [ 0.021 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 10:02 min [INFO] Finished at: 2018-05-18T08:58:23-07:00 [INFO] Final Memory: 136M/1566M [INFO] nonas:oodt mattmann$ ``` > Crawler Launcher hangs after ingesting files when Avro File Manager client is > used > -- > > Key: OODT-978 > URL: https://issues.apache.org/jira/browse/OODT-978 > Project: OODT > Issue Type: Bug > Components: crawler, file manager >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Critical > > When we run the crawler with Avro File manager client (within StdIngester), > the crawler hangs even after ingesting all the files with the directory. > This was caused by Netty Transceiver still running in the background which is > used by Avro FM client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-978) Crawler Launcher hangs after ingesting files when Avro File Manager client is used
[ https://issues.apache.org/jira/browse/OODT-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480820#comment-16480820 ] ASF GitHub Bot commented on OODT-978: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/62 nice one @IMS94 I will pull and test now. > Crawler Launcher hangs after ingesting files when Avro File Manager client is > used > -- > > Key: OODT-978 > URL: https://issues.apache.org/jira/browse/OODT-978 > Project: OODT > Issue Type: Bug > Components: crawler, file manager >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Critical > > When we run the crawler with Avro File manager client (within StdIngester), > the crawler hangs even after ingesting all the files with the directory. > This was caused by Netty Transceiver still running in the background which is > used by Avro FM client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-978) Crawler Launcher hangs after ingesting files when Avro File Manager client is used
[ https://issues.apache.org/jira/browse/OODT-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480313#comment-16480313 ] ASF GitHub Bot commented on OODT-978: - Github user IMS94 commented on the issue: https://github.com/apache/oodt/pull/62 @chrismattmann Can you check now? > Crawler Launcher hangs after ingesting files when Avro File Manager client is > used > -- > > Key: OODT-978 > URL: https://issues.apache.org/jira/browse/OODT-978 > Project: OODT > Issue Type: Bug > Components: crawler, file manager >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Critical > > When we run the crawler with Avro File manager client (within StdIngester), > the crawler hangs even after ingesting all the files with the directory. > This was caused by Netty Transceiver still running in the background which is > used by Avro FM client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-978) Crawler Launcher hangs after ingesting files when Avro File Manager client is used
[ https://issues.apache.org/jira/browse/OODT-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480183#comment-16480183 ] ASF GitHub Bot commented on OODT-978: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/62 @IMS94 OK here's what I did. This is definitely something related to your PR/patch: ``` 517 git reset --hard 3cac96e4d51c19e0fa2467be3cb66350bc2dfe01 518 mvn clean install 519 git apply fed87b2bccc9890b0f5e45883936cc31fcd12f8f 520 git cherry-pick fed87b2bccc9890b0f5e45883936cc31fcd12f8f 521 mvn clean install 522 git cherry-pick 40ed14e48510b9e6993caaa97c7d69bad9ff99a7 523 mvn clean install 524 git cherry-pick 1c49a9bbbc8e7c95bd23106c3dc58560c5af7488 525 git cherry-pick 2d19275c1f142d4d42f9765aad8904ebbe7e5a12 526 mvn clean install ``` Output from my history. That resets to the DOAP file commit pre commits from @lewismc and you. Then cherry picks and applies commits without merge commits. You can see that it breaks after I apply !525: ``` INFO: Loading workflow context metadata... Tests run: 15, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 1.24 sec <<< FAILURE! - in org.apache.oodt.cas.pge.TestPGETaskInstance testCreatePgeConfig(org.apache.oodt.cas.pge.TestPGETaskInstance) Time elapsed: 0.007 sec <<< ERROR! java.io.IOException: Failed to build PgeConfig : Failed to parse value: /var/folders/n5/1d_k3z4s2293q8ntx_n8sw54mm5n_8/T/904bacbf-0121-47a9-981a-a0a5db5e0e85/pgeConfig.xml at java.net.URL.(URL.java:532) at java.net.URL.(URL.java:490) at java.net.URL.(URL.java:439) at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:444) at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:438) at org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:58) at org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:218) at org.apache.oodt.cas.pge.TestPGETaskInstance.testCreatePgeConfig(TestPGETaskInstance.java:260) Running org.apache.oodt.cas.pge.metadata.TestPgeMetadata Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in org.apache.oodt.cas.pge.metadata.TestPgeMetadata Running org.apache.oodt.cas.pge.metadata.TestPgeTaskMetKeys Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in org.apache.oodt.cas.pge.metadata.TestPgeTaskMetKeys Results : Tests in error: TestXmlHelper.testGetImports:52 » PGE Failed to parse value: blank TestXmlHelper.testGetNamespace:65 » PGE Failed to parse value: blank TestXmlFilePgeConfigBuilder.testBuild:54 » IO Failed to build PgeConfig : Fail... TestPGETaskInstance.testCreatePgeConfig:260 » IO Failed to build PgeConfig : F... Tests run: 31, Failures: 0, Errors: 4, Skipped: 1 [INFO] [INFO] Reactor Summary: [INFO] [INFO] OODT Core .. SUCCESS [ 2.580 s] [INFO] Common Utilities ... SUCCESS [ 25.469 s] [INFO] CAS Command Line Interface . SUCCESS [ 10.728 s] [INFO] OODT - Configuration Management SUCCESS [ 28.755 s] [INFO] Process Control System Input Data Package .. SUCCESS [ 4.786 s] [INFO] Catalog and Archive Service Generic Multi-valued Metadata Container SUCCESS [ 4.022 s] [INFO] Catalog and Archive File Management Component .. SUCCESS [04:25 min] [INFO] Catalog and Archive Resource Management Component .. SUCCESS [ 37.578 s] [INFO] Catalog and Archive Workflow Management Component .. SUCCESS [ 40.678 s] [INFO] Catalog and Archive Crawling Framework . SUCCESS [ 28.660 s] [INFO] OODT CAS Curator Single Sign On Security Package ... SUCCESS [ 1.679 s] [INFO] CAS Curation Web Services .. SUCCESS [ 11.258 s] [INFO] Process Control System Core Package SUCCESS [ 21.118 s] [INFO] OODT Wicket Web Components . SUCCESS [ 5.078 s] [INFO] CAS Curation Interface . SUCCESS [ 15.037 s] [INFO] CAS PGE Adaptor Framework .. FAILURE [ 6.886 s] [INFO] CAS Installer Maven Mojo ... SKIPPED [INFO] OODT :: Archetypes :: OpsUI SKIPPED [INFO] OODT :: Archetypes :: RADiX SKIPPED [INFO] OODT :: Archetypes . SKIPPED [INFO] CAS File Manager Browser Web App ... SKIPPED [INFO] CAS Workflow Manager Monitor Web App ... SK
[jira] [Commented] (OODT-975) Upgrade Tomcat in radix
[ https://issues.apache.org/jira/browse/OODT-975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480163#comment-16480163 ] ASF GitHub Bot commented on OODT-975: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/60 so I was able to get around this by excluding merge commits...and by cherry picking the commit. > Upgrade Tomcat in radix > --- > > Key: OODT-975 > URL: https://issues.apache.org/jira/browse/OODT-975 > Project: OODT > Issue Type: Improvement > Components: radix >Affects Versions: 1.2.2 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Minor > Fix For: 1.9 > > > Tomcat is sitting at 5.X prior to org.apache.tomcat package renaming. This > issue will make the upgrade to the most recent Tomcat version... which I > assume will also improve security for radix installations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-975) Upgrade Tomcat in radix
[ https://issues.apache.org/jira/browse/OODT-975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480150#comment-16480150 ] ASF GitHub Bot commented on OODT-975: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/60 @lewismc I'm seeing some errors running `mvn clean install` with Apache OODT and this patch and running the tests. > Upgrade Tomcat in radix > --- > > Key: OODT-975 > URL: https://issues.apache.org/jira/browse/OODT-975 > Project: OODT > Issue Type: Improvement > Components: radix >Affects Versions: 1.2.2 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Minor > Fix For: 1.9 > > > Tomcat is sitting at 5.X prior to org.apache.tomcat package renaming. This > issue will make the upgrade to the most recent Tomcat version... which I > assume will also improve security for radix installations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-978) Crawler Launcher hangs after ingesting files when Avro File Manager client is used
[ https://issues.apache.org/jira/browse/OODT-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480149#comment-16480149 ] ASF GitHub Bot commented on OODT-978: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/62 @IMS94 I think you are right. In fact I think it's related to @lewismc and the update for OODT-976 in #60 > Crawler Launcher hangs after ingesting files when Avro File Manager client is > used > -- > > Key: OODT-978 > URL: https://issues.apache.org/jira/browse/OODT-978 > Project: OODT > Issue Type: Bug > Components: crawler, file manager >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Critical > > When we run the crawler with Avro File manager client (within StdIngester), > the crawler hangs even after ingesting all the files with the directory. > This was caused by Netty Transceiver still running in the background which is > used by Avro FM client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-978) Crawler Launcher hangs after ingesting files when Avro File Manager client is used
[ https://issues.apache.org/jira/browse/OODT-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480071#comment-16480071 ] ASF GitHub Bot commented on OODT-978: - Github user IMS94 commented on the issue: https://github.com/apache/oodt/pull/62 @chrismattmann I will look at that. Seems like this is not related to my changes. Let's see. > Crawler Launcher hangs after ingesting files when Avro File Manager client is > used > -- > > Key: OODT-978 > URL: https://issues.apache.org/jira/browse/OODT-978 > Project: OODT > Issue Type: Bug > Components: crawler, file manager >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Critical > > When we run the crawler with Avro File manager client (within StdIngester), > the crawler hangs even after ingesting all the files with the directory. > This was caused by Netty Transceiver still running in the background which is > used by Avro FM client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-978) Crawler Launcher hangs after ingesting files when Avro File Manager client is used
[ https://issues.apache.org/jira/browse/OODT-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479478#comment-16479478 ] ASF GitHub Bot commented on OODT-978: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/62 @IMS94 I get an error building this: ``` INFO: Loading workflow context metadata... Tests run: 15, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 1.242 sec <<< FAILURE! - in org.apache.oodt.cas.pge.TestPGETaskInstance testCreatePgeConfig(org.apache.oodt.cas.pge.TestPGETaskInstance) Time elapsed: 0.01 sec <<< ERROR! java.io.IOException: Failed to build PgeConfig : Failed to parse value: /var/folders/n5/1d_k3z4s2293q8ntx_n8sw54mm5n_8/T/3ea2e425-c368-4fc3-b73b-fe45724a17b5/pgeConfig.xml at java.net.URL.(URL.java:532) at java.net.URL.(URL.java:490) at java.net.URL.(URL.java:439) at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:444) at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:438) at org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePgeConfigBuilder.java:58) at org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance.java:218) at org.apache.oodt.cas.pge.TestPGETaskInstance.testCreatePgeConfig(TestPGETaskInstance.java:260) Running org.apache.oodt.cas.pge.metadata.TestPgeMetadata Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in org.apache.oodt.cas.pge.metadata.TestPgeMetadata Running org.apache.oodt.cas.pge.metadata.TestPgeTaskMetKeys Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in org.apache.oodt.cas.pge.metadata.TestPgeTaskMetKeys Results : Tests in error: TestXmlHelper.testGetImports:52 » PGE Failed to parse value: blank TestXmlHelper.testGetNamespace:65 » PGE Failed to parse value: blank TestXmlFilePgeConfigBuilder.testBuild:54 » IO Failed to build PgeConfig : Fail... TestPGETaskInstance.testCreatePgeConfig:260 » IO Failed to build PgeConfig : F... Tests run: 31, Failures: 0, Errors: 4, Skipped: 1 [INFO] [INFO] Reactor Summary: [INFO] [INFO] OODT Core .. SUCCESS [ 3.146 s] [INFO] Common Utilities ... SUCCESS [ 27.736 s] [INFO] CAS Command Line Interface . SUCCESS [ 6.426 s] [INFO] OODT - Configuration Management SUCCESS [ 23.711 s] [INFO] Process Control System Input Data Package .. SUCCESS [ 2.284 s] [INFO] Catalog and Archive Service Generic Multi-valued Metadata Container SUCCESS [ 3.407 s] [INFO] Catalog and Archive File Management Component .. SUCCESS [03:50 min] [INFO] Catalog and Archive Resource Management Component .. SUCCESS [ 33.547 s] [INFO] Catalog and Archive Workflow Management Component .. SUCCESS [ 42.013 s] [INFO] Catalog and Archive Crawling Framework . SUCCESS [ 26.329 s] [INFO] OODT CAS Curator Single Sign On Security Package ... SUCCESS [ 2.354 s] [INFO] CAS Curation Web Services .. SUCCESS [ 11.634 s] [INFO] Process Control System Core Package SUCCESS [ 20.501 s] [INFO] OODT Wicket Web Components . SUCCESS [ 6.201 s] [INFO] CAS Curation Interface . SUCCESS [ 18.657 s] [INFO] CAS PGE Adaptor Framework .. FAILURE [ 7.536 s] [INFO] CAS Installer Maven Mojo ... SKIPPED [INFO] OODT :: Archetypes :: OpsUI SKIPPED [INFO] OODT :: Archetypes :: RADiX SKIPPED [INFO] OODT :: Archetypes . SKIPPED [INFO] CAS File Manager Browser Web App ... SKIPPED [INFO] CAS Workflow Manager Monitor Web App ... SKIPPED [INFO] CAS Product Server Web Application . SKIPPED [INFO] CAS Workflow REST Services . SKIPPED [INFO] Process Control System Operator Interface Webapp ... SKIPPED [INFO] OODT Process Control System JAX-RS service layer ... SKIPPED [INFO] Apache OODT SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 07:46 min [INFO] Finished at: 2018-05-17T10:35:12-07:00 [INFO] Final Memory: 116M/1554M [INFO] [ERROR] Failed to execute go
[jira] [Commented] (OODT-978) Crawler Launcher hangs after ingesting files when Avro File Manager client is used
[ https://issues.apache.org/jira/browse/OODT-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479427#comment-16479427 ] ASF GitHub Bot commented on OODT-978: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/62 testing this now locally > Crawler Launcher hangs after ingesting files when Avro File Manager client is > used > -- > > Key: OODT-978 > URL: https://issues.apache.org/jira/browse/OODT-978 > Project: OODT > Issue Type: Bug > Components: crawler, file manager >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Critical > > When we run the crawler with Avro File manager client (within StdIngester), > the crawler hangs even after ingesting all the files with the directory. > This was caused by Netty Transceiver still running in the background which is > used by Avro FM client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-978) Crawler Launcher hangs after ingesting files when Avro File Manager client is used
[ https://issues.apache.org/jira/browse/OODT-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479425#comment-16479425 ] ASF GitHub Bot commented on OODT-978: - Github user chrismattmann commented on the issue: https://github.com/apache/oodt/pull/62 LGTM! > Crawler Launcher hangs after ingesting files when Avro File Manager client is > used > -- > > Key: OODT-978 > URL: https://issues.apache.org/jira/browse/OODT-978 > Project: OODT > Issue Type: Bug > Components: crawler, file manager >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Critical > > When we run the crawler with Avro File manager client (within StdIngester), > the crawler hangs even after ingesting all the files with the directory. > This was caused by Netty Transceiver still running in the background which is > used by Avro FM client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-978) Crawler Launcher hangs after ingesting files when Avro File Manager client is used
[ https://issues.apache.org/jira/browse/OODT-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16477217#comment-16477217 ] ASF GitHub Bot commented on OODT-978: - GitHub user IMS94 opened a pull request: https://github.com/apache/oodt/pull/62 [OODT-978] Fix for Crawler Launcher hanging even after ingesting all the files (when avro fm client is used) You can merge this pull request into a Git repository by running: $ git pull https://github.com/IMS94/oodt master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/oodt/pull/62.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #62 commit ecfe0a78849c928d904eeb129cfd7d24f9600afc Author: Imesha Sudasingha Date: 2018-04-18T13:15:47Z Updated startup scripts of resmgr and workflowmanager to use AvroRpc implementations. commit 3fcd4249a146415e6925a2c839de2f5dbf31aeb3 Author: Imesha Sudasingha Date: 2018-04-18T13:16:21Z Merge remote-tracking branch 'origin/master' commit 19e51621268fc52b0355b8203a9dc42d61dd8ce4 Author: Imesha Sudasingha Date: 2018-04-18T13:17:05Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/oodt commit 1c49a9bbbc8e7c95bd23106c3dc58560c5af7488 Author: Imesha Sudasingha Date: 2018-05-16T10:26:29Z [OODT-978] Fixed crawler launcher hanging issue in file manager avro version by adding a close() method to the FileManagerClient commit dc794944b7362d87821c42ae91de8c39df849de5 Author: Imesha Sudasingha Date: 2018-05-16T10:30:16Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/oodt > Crawler Launcher hangs after ingesting files when Avro File Manager client is > used > -- > > Key: OODT-978 > URL: https://issues.apache.org/jira/browse/OODT-978 > Project: OODT > Issue Type: Bug > Components: crawler, file manager >Reporter: Imesha Sudasingha >Assignee: Imesha Sudasingha >Priority: Critical > > When we run the crawler with Avro File manager client (within StdIngester), > the crawler hangs even after ingesting all the files with the directory. > This was caused by Netty Transceiver still running in the background which is > used by Avro FM client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-976) Upgrade Apache Tika to 1.18
[ https://issues.apache.org/jira/browse/OODT-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460502#comment-16460502 ] ASF GitHub Bot commented on OODT-976: - Github user lewismc commented on the issue: https://github.com/apache/oodt/pull/61 @chrismattmann there is an issue in testing this over in DRAT. OODT 1.2.2 modules use Tika 1.10 and 1.7 dependencies... hence DRAT distribution packages a deployment which inherits the older Tika dependencies transitively. We need to upgrade Tika in OODT, as per this PR, test and push a release before we can move forward elsewhere. > Upgrade Apache Tika to 1.18 > --- > > Key: OODT-976 > URL: https://issues.apache.org/jira/browse/OODT-976 > Project: OODT > Issue Type: Improvement > Components: core, file manager, radix >Affects Versions: 1.2.2 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > tika-core and tika-parsers 1.13 dependencies are pretty outdated now. Tika > 1.18 is available so we should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OODT-976) Upgrade Apache Tika to 1.18
[ https://issues.apache.org/jira/browse/OODT-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16460410#comment-16460410 ] ASF GitHub Bot commented on OODT-976: - Github user lewismc commented on the issue: https://github.com/apache/oodt/pull/61 @chrismattmann https://github.com/apache/tika/pull/235 > Upgrade Apache Tika to 1.18 > --- > > Key: OODT-976 > URL: https://issues.apache.org/jira/browse/OODT-976 > Project: OODT > Issue Type: Improvement > Components: core, file manager, radix >Affects Versions: 1.2.2 >Reporter: Lewis John McGibbney >Assignee: Lewis John McGibbney >Priority: Major > Fix For: 1.9 > > > tika-core and tika-parsers 1.13 dependencies are pretty outdated now. Tika > 1.18 is available so we should upgrade. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
