[GitHub] metron pull request #985: METRON-1515: Errors loading stellar functions curr...
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/985#discussion_r180576511
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/resolver/ClasspathFunctionResolver.java
---
@@ -254,18 +266,24 @@ public void initialize(Context context) {
Set classes = new HashSet<>();
Set ret = new HashSet<>();
for(ClassLoader cl : cls) {
- for(Class c : ClassIndex.getAnnotated(Stellar.class, cl)) {
-LOG.debug("{}: Found class: {}", cl.getClass().getCanonicalName(),
c.getCanonicalName());
-boolean isAssignable = StellarFunction.class.isAssignableFrom(c);
-boolean isFiltered = filterBuilder.apply(c.getCanonicalName());
-if( isAssignable && isFiltered ) {
- String className = c.getName();
- if(!classes.contains(className)) {
-LOG.debug("{}: Added class: {}",
cl.getClass().getCanonicalName(), className);
-ret.add((Class) c);
-classes.add(className);
+ for(Class c : getStellarClasses(cl)) {
+try {
+ LOG.debug("{}: Found class: {}",
cl.getClass().getCanonicalName(), c.getCanonicalName());
+ if (includeClass(c, filterBuilder)) {
+String className = c.getName();
+if (!classes.contains(className)) {
+ LOG.debug("{}: Added class: {}",
cl.getClass().getCanonicalName(), className);
+ ret.add((Class) c);
+ classes.add(className);
+}
}
}
+catch(Error le) {
+ //we have had some error loading a stellar function. This could
mean that
+ //the classpath is unstable (e.g. old copies of jars are on the
classpath).
+ LOG.error("Skipping class: " + le.getMessage()
--- End diff --
Yeah, this is tricky, getting the classname may actually cause the
exception (this is precisely the scenario that motivated this, sadly). That's
why I forewent that approach.
---
[GitHub] metron pull request #985: METRON-1515: Errors loading stellar functions curr...
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/985#discussion_r180577592
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/resolver/ClasspathFunctionResolver.java
---
@@ -254,18 +266,24 @@ public void initialize(Context context) {
Set classes = new HashSet<>();
Set ret = new HashSet<>();
for(ClassLoader cl : cls) {
- for(Class c : ClassIndex.getAnnotated(Stellar.class, cl)) {
-LOG.debug("{}: Found class: {}", cl.getClass().getCanonicalName(),
c.getCanonicalName());
-boolean isAssignable = StellarFunction.class.isAssignableFrom(c);
-boolean isFiltered = filterBuilder.apply(c.getCanonicalName());
-if( isAssignable && isFiltered ) {
- String className = c.getName();
- if(!classes.contains(className)) {
-LOG.debug("{}: Added class: {}",
cl.getClass().getCanonicalName(), className);
-ret.add((Class) c);
-classes.add(className);
+ for(Class c : getStellarClasses(cl)) {
+try {
+ LOG.debug("{}: Found class: {}",
cl.getClass().getCanonicalName(), c.getCanonicalName());
+ if (includeClass(c, filterBuilder)) {
+String className = c.getName();
+if (!classes.contains(className)) {
+ LOG.debug("{}: Added class: {}",
cl.getClass().getCanonicalName(), className);
+ ret.add((Class) c);
+ classes.add(className);
+}
}
}
+catch(Error le) {
+ //we have had some error loading a stellar function. This could
mean that
+ //the classpath is unstable (e.g. old copies of jars are on the
classpath).
+ LOG.error("Skipping class: " + le.getMessage()
--- End diff --
For posterity, this is the stack trace that motivated this:
```
org.apache.metron.stellar.dsl.functions.MathFunctions and
org.apache.metron.stellar.dsl.functions.MathFunctions$Sqrt disagree on
InnerClasses attribute
at java.lang.Class.getDeclaringClass0(Native Method)
at java.lang.Class.getDeclaringClass(Class.java:1235)
at java.lang.Class.getEnclosingClass(Class.java:1277)
at java.lang.Class.getCanonicalName(Class.java:1392)
at
org.apache.metron.stellar.dsl.functions.resolver.ClasspathFunctionResolver.resolvables(ClasspathFunctionResolver.java:250)
```
The problem is in the `getCanonicalName`. I suppose I can do a `getName`,
but I'm a bit wary of even that, frankly.
---
[jira] [Commented] (METRON-1515) Errors loading stellar functions currently bomb the entire topology, they should be recoverable
[
https://issues.apache.org/jira/browse/METRON-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433046#comment-16433046
]
ASF GitHub Bot commented on METRON-1515:
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/985#discussion_r180577592
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/resolver/ClasspathFunctionResolver.java
---
@@ -254,18 +266,24 @@ public void initialize(Context context) {
Set classes = new HashSet<>();
Set ret = new HashSet<>();
for(ClassLoader cl : cls) {
- for(Class c : ClassIndex.getAnnotated(Stellar.class, cl)) {
-LOG.debug("{}: Found class: {}", cl.getClass().getCanonicalName(),
c.getCanonicalName());
-boolean isAssignable = StellarFunction.class.isAssignableFrom(c);
-boolean isFiltered = filterBuilder.apply(c.getCanonicalName());
-if( isAssignable && isFiltered ) {
- String className = c.getName();
- if(!classes.contains(className)) {
-LOG.debug("{}: Added class: {}",
cl.getClass().getCanonicalName(), className);
-ret.add((Class) c);
-classes.add(className);
+ for(Class c : getStellarClasses(cl)) {
+try {
+ LOG.debug("{}: Found class: {}",
cl.getClass().getCanonicalName(), c.getCanonicalName());
+ if (includeClass(c, filterBuilder)) {
+String className = c.getName();
+if (!classes.contains(className)) {
+ LOG.debug("{}: Added class: {}",
cl.getClass().getCanonicalName(), className);
+ ret.add((Class) c);
+ classes.add(className);
+}
}
}
+catch(Error le) {
+ //we have had some error loading a stellar function. This could
mean that
+ //the classpath is unstable (e.g. old copies of jars are on the
classpath).
+ LOG.error("Skipping class: " + le.getMessage()
--- End diff --
For posterity, this is the stack trace that motivated this:
```
org.apache.metron.stellar.dsl.functions.MathFunctions and
org.apache.metron.stellar.dsl.functions.MathFunctions$Sqrt disagree on
InnerClasses attribute
at java.lang.Class.getDeclaringClass0(Native Method)
at java.lang.Class.getDeclaringClass(Class.java:1235)
at java.lang.Class.getEnclosingClass(Class.java:1277)
at java.lang.Class.getCanonicalName(Class.java:1392)
at
org.apache.metron.stellar.dsl.functions.resolver.ClasspathFunctionResolver.resolvables(ClasspathFunctionResolver.java:250)
```
The problem is in the `getCanonicalName`. I suppose I can do a `getName`,
but I'm a bit wary of even that, frankly.
> Errors loading stellar functions currently bomb the entire topology, they
> should be recoverable
> ---
>
> Key: METRON-1515
> URL: https://issues.apache.org/jira/browse/METRON-1515
> Project: Metron
> Issue Type: Bug
>Reporter: Casey Stella
>Priority: Major
>
> If a bad stellar function gets on the classpath, the entire enrichment
> topology (or shell) bombs. We should just log an error and skip the function.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (METRON-1515) Errors loading stellar functions currently bomb the entire topology, they should be recoverable
[
https://issues.apache.org/jira/browse/METRON-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433048#comment-16433048
]
ASF GitHub Bot commented on METRON-1515:
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/985#discussion_r180578439
--- Diff:
metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/resolver/ClasspathFunctionResolver.java
---
@@ -254,18 +266,24 @@ public void initialize(Context context) {
Set classes = new HashSet<>();
Set ret = new HashSet<>();
for(ClassLoader cl : cls) {
- for(Class c : ClassIndex.getAnnotated(Stellar.class, cl)) {
-LOG.debug("{}: Found class: {}", cl.getClass().getCanonicalName(),
c.getCanonicalName());
-boolean isAssignable = StellarFunction.class.isAssignableFrom(c);
-boolean isFiltered = filterBuilder.apply(c.getCanonicalName());
-if( isAssignable && isFiltered ) {
- String className = c.getName();
- if(!classes.contains(className)) {
-LOG.debug("{}: Added class: {}",
cl.getClass().getCanonicalName(), className);
-ret.add((Class) c);
-classes.add(className);
+ for(Class c : getStellarClasses(cl)) {
+try {
+ LOG.debug("{}: Found class: {}",
cl.getClass().getCanonicalName(), c.getCanonicalName());
+ if (includeClass(c, filterBuilder)) {
+String className = c.getName();
+if (!classes.contains(className)) {
+ LOG.debug("{}: Added class: {}",
cl.getClass().getCanonicalName(), className);
+ ret.add((Class) c);
+ classes.add(className);
+}
}
}
+catch(Error le) {
+ //we have had some error loading a stellar function. This could
mean that
+ //the classpath is unstable (e.g. old copies of jars are on the
classpath).
+ LOG.error("Skipping class: " + le.getMessage()
--- End diff --
On second thought, I think I might've squared the circle there in the
latest commit. How do you like what I came up with?
> Errors loading stellar functions currently bomb the entire topology, they
> should be recoverable
> ---
>
> Key: METRON-1515
> URL: https://issues.apache.org/jira/browse/METRON-1515
> Project: Metron
> Issue Type: Bug
>Reporter: Casey Stella
>Priority: Major
>
> If a bad stellar function gets on the classpath, the entire enrichment
> topology (or shell) bombs. We should just log an error and skip the function.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (METRON-1510) Update Metron website to include info about github update subscription
[ https://issues.apache.org/jira/browse/METRON-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433416#comment-16433416 ] ASF GitHub Bot commented on METRON-1510: Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/981 > Update Metron website to include info about github update subscription > -- > > Key: METRON-1510 > URL: https://issues.apache.org/jira/browse/METRON-1510 > Project: Metron > Issue Type: Bug >Reporter: Anand Subramanian >Priority: Major > > Per the discussion > [here|https://lists.apache.org/thread.html/9a450a46701eb0924d431a042737b904ab344eb052f630b112a7e5a6@%3Cdev.metron.apache.org%3E], > github updates have been moved to a separate issues list at > https://lists.apache.org/list.html?issues@metron.apache.org > The Metron website needs an update to reflect the same. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] metron pull request #981: METRON-1510: Update Metron website to include info...
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/981 ---
