I think a better solution might be to invert the normal class loader resolution order with an extension of URLClassLoader that overrides loadClass(), which tries resolving classes internally before delegating to the parent. Boot classpath classes can still be forced to load correctly. I think this will solve the use case without the regression due to inability to resolve *db classes.
Thanks, -MDB On Sat, Jul 2, 2016 at 11:44 PM Xikui Wang <[email protected]> wrote: > The error log is as follow: > > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at > > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111) > at > > org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143) > at > > org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39) > at > > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127) > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341) > at org.apache.hyracks.control.nc.Task.run(Task.java:297) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at > > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75) > at > > org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53) > at > > org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39) > at > > org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41) > at > > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109) > ... 8 more > Caused by: java.lang.ClassCastException: > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to > org.apache.asterix.external.api.IFunctionFactory > at > > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72) > ... 12 more > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at > > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111) > at > > org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143) > at > > org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39) > at > > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127) > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341) > at org.apache.hyracks.control.nc.Task.run(Task.java:297) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at > > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75) > at > > org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53) > at > > org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39) > at > > org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41) > at > > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109) > ... 8 more > Caused by: java.lang.ClassCastException: > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to > org.apache.asterix.external.api.IFunctionFactory > at > > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72) > ... 12 more > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > at org.apache.hyracks.control.nc.Task.run(Task.java:297) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at > > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130) > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341) > ... 4 more > Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at > > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111) > at > > org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143) > at > > org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39) > at > > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127) > ... 5 more > Caused by: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at > > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75) > at > > org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53) > at > > org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39) > at > > org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41) > at > > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109) > ... 8 more > Caused by: java.lang.ClassCastException: > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to > org.apache.asterix.external.api.IFunctionFactory > at > > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72) > ... 12 more > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > at org.apache.hyracks.control.nc.Task.run(Task.java:297) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at > > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130) > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341) > ... 4 more > Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at > > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111) > at > > org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143) > at > > org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39) > at > > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127) > ... 5 more > Caused by: > org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > Unable to load/instantiate class > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > at > > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75) > at > > org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53) > at > > org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39) > at > > org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41) > at > > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109) > ... 8 more > Caused by: java.lang.ClassCastException: > edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to > org.apache.asterix.external.api.IFunctionFactory > at > > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72) > ... 12 more > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.api.exceptions.HyracksDataException: > java.lang.InterruptedException > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > at org.apache.hyracks.control.nc.Task.run(Task.java:297) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > java.lang.InterruptedException > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126) > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43) > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344) > ... 4 more > Caused by: java.lang.InterruptedException > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:502) > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124) > ... 6 more > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.api.exceptions.HyracksDataException: > java.lang.InterruptedException > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > at org.apache.hyracks.control.nc.Task.run(Task.java:297) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > java.lang.InterruptedException > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126) > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43) > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344) > ... 4 more > Caused by: java.lang.InterruptedException > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:502) > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124) > ... 6 more > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.api.exceptions.HyracksDataException: > java.lang.InterruptedException > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > at org.apache.hyracks.control.nc.Task.run(Task.java:297) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > java.lang.InterruptedException > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126) > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43) > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344) > ... 4 more > Caused by: java.lang.InterruptedException > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:502) > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124) > ... 6 more > org.apache.hyracks.api.exceptions.HyracksDataException: > org.apache.hyracks.api.exceptions.HyracksDataException: > java.lang.InterruptedException > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > at org.apache.hyracks.control.nc.Task.run(Task.java:297) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > java.lang.InterruptedException > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126) > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43) > at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344) > ... 4 more > Caused by: java.lang.InterruptedException > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:502) > at > > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124) > ... 6 more > testFile > > src/test/resources/runtimets/queries/external-library/tweetGeoTag/tweetGeoTag.3.update.aql > raised an exception: > java.lang.Exception: HTTP operation failed: 2 > STATUS LINE: HTTP/1.1 500 Server Error > SUMMARY: Error: Failure in feed > > STACKTRACE: org.apache.asterix.common.exceptions.AsterixException: Failure > in feed > at > > org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertNoFailure(FeedLifecycleEventSubscriber.java:62) > at > > org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertEvent(FeedLifecycleEventSubscriber.java:55) > at > > org.apache.asterix.aql.translator.QueryTranslator.handleConnectFeedStatement(QueryTranslator.java:2238) > at > > org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:366) > at > > org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:253) > at > > org.apache.asterix.api.http.servlet.RESTAPIServlet.handleRequest(RESTAPIServlet.java:195) > at > > org.apache.asterix.api.http.servlet.UpdateAPIServlet.handleRequest(UpdateAPIServlet.java:30) > at > > org.apache.asterix.api.http.servlet.RESTAPIServlet.doPost(RESTAPIServlet.java:162) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483) > at > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:228) > at > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:956) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411) > at > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:188) > at > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:891) > at > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > at > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114) > at org.eclipse.jetty.server.Server.handle(Server.java:353) > at > > org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:598) > at > > org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1076) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:805) > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:218) > at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:427) > at > > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:510) > at > > org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34) > at > > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40) > at > > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:450) > at java.lang.Thread.run(Thread.java:745) > > at > > org.apache.asterix.test.aql.TestExecutor.executeHttpMethod(TestExecutor.java:275) > at > > org.apache.asterix.test.aql.TestExecutor.executeUpdate(TestExecutor.java:331) > at > org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:496) > at > org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:762) > at > org.apache.asterix.test.runtime.ExecutionTest.test(ExecutionTest.java:106) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > at > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > at > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at org.junit.runners.Suite.runChild(Suite.java:127) > at org.junit.runners.Suite.runChild(Suite.java:26) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > at > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at org.junit.runner.JUnitCore.run(JUnitCore.java:160) > at > > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119) > at > > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42) > at > > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234) > at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) > ...Unexpected! > > On Sat, Jul 2, 2016 at 7:23 PM, Raman Grover <[email protected]> > wrote: > > > i am missing the attachment > > On Jul 2, 2016 6:39 PM, "Xikui Wang" <[email protected]> wrote: > > > > > Hi Raman, > > > > > > Thanks for your help. I tried this quick fix on my branch, but it > > > introduces some new exceptions. I think this causes Asterix fails at > > > entering the external function. The error message is attached. > > > > > > Best, > > > Xikui > > > > > > On Fri, Jul 1, 2016 at 10:11 AM, Raman Grover <[email protected] > > > > > wrote: > > > > > >> Operations related to setting up an external library are contained in > > >> ExternalLibraryUtil > > >> < > > >> > > > https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java > > >> > > > >> > > >> At line 382, we have > > >> // create and return the class loader > > >> > > >> ClassLoader classLoader = new URLClassLoader(urls, parentClassLoader); > > >> return classLoader; > > >> > > >> Above, we have the parentClassLoader set to the classloader for > > >> ExternalLibraryUtil which is the application class loader (AsterixDB's > > >> classloader that loads the dependencies from pom.xml). The proposed > > >> solution (a) in earlier thread - skipping application classloader > would > > >> translate to replacing the above code with > > >> > > >> ClassLoader classLoader = new URLClassLoader(urls, null); > > >> > > >> Regards, > > >> Raman > > >> > > >> On Thu, Jun 30, 2016 at 4:41 PM, Xikui Wang <[email protected]> wrote: > > >> > > >> > > > > >> > > Hi Abdullah, > > >> > > > >> > I reverted my code to reproduce the problem. Noticing this external > > >> > function has a couple of other bugs but the dependency one is > blocking > > >> > others, so this should be enough to reproduce the problem. > > >> > > > >> > > > > >> > The external function package is geoTag.zip. > > >> > > > >> > > > > >> > > > > >> > > Test scripts are in tweetGeoTag.zip > > >> > > > >> > External function is loading data from data/, i.e.: data/state.json > . > > So > > >> > all json files in data.zip need to be placed under > > >> > ../asterixdb/asterix-app/data/ > > >> > > > >> > The real_tweets_adm.adm used in ddl is also attached. > > >> > > > >> > This setting will cause > > >> > > > >> > java.lang.NoSuchMethodError: > > >> > > com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z > > >> > > at > > >> > > > > >> > > com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:541) > > >> > > at > > >> > > > > >> > > com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:452) > > >> > > at > > org.wololo.geojson.GeoJSONFactory.<clinit>(GeoJSONFactory.java:17) > > >> > > at > > >> > > > > >> > > > >> > > > edu.uci.ics.cloudberry.gnosis.USGeoJSONIndex.loadShape(IGeoIndex.scala:29) > > >> > > at > > >> > > > > >> > > > >> > > > edu.uci.ics.cloudberry.gnosis.USGeoGnosis$.loadShape(USGeoGnosis.scala:101) > > >> > > at > > >> > > > > >> > > > >> > > > edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:20) > > >> > > at > > >> > > > > >> > > > >> > > > edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:18) > > >> > > at > > >> > > > > >> > > > >> > > > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245) > > >> > > at > > >> > > > > >> > > > >> > > > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245) > > >> > > at scala.collection.immutable.List.foreach(List.scala:381) > > >> > > at > > >> scala.collection.TraversableLike$class.map(TraversableLike.scala:245) > > >> > > at scala.collection.immutable.List.map(List.scala:285) > > >> > > at > > >> edu.uci.ics.cloudberry.gnosis.USGeoGnosis.load(USGeoGnosis.scala:18) > > >> > > at > > >> edu.uci.ics.cloudberry.gnosis.USGeoGnosis.<init>(USGeoGnosis.scala:15) > > >> > > at > > >> > > > > >> > > > >> > > > edu.uci.ics.cloudberry.noah.feed.GeoTagFunction.initialize(GeoTagFunction.java:34) > > >> > > at > > >> > > > > >> > > > >> > > > org.apache.asterix.external.library.ExternalFunction.initialize(ExternalFunction.java:113) > > >> > > > > >> > > > >> > The attachments exceed the size limits. I moved them under this > google > > >> > drive directory: > > >> > > > >> > > > >> > > > >> > > > https://drive.google.com/folderview?id=0B_6Dzy3OTjaNRUVJWTRvWEtKSU0&usp=sharing > > >> > > > >> > Let me know if you need anything else. Thanks. > > >> > > > >> > Best, > > >> > Xikui > > >> > > > >> > On Thu, Jun 30, 2016 at 12:06 PM, abdullah alamoudi < > > [email protected] > > >> > > > >> > wrote: > > >> > > > >> > > I will look into that. > > >> > >> > > >> > >> @Xikui, > > >> > >> Do you have a reproducable use case where such a behavior can be > > >> > observed? > > >> > >> > > >> > >> Cheers, > > >> > >> Abdullah. > > >> > >> > > >> > >> On Thu, Jun 30, 2016 at 7:53 PM, Raman Grover < > > >> [email protected]> > > >> > >> wrote: > > >> > >> > > >> > >> > Hi > > >> > >> > > > >> > >> > Each external library has an separately associated custom class > > >> loader > > >> > >> > (URLClassLoader) that loads the contained functions and any > > >> > dependencies > > >> > >> > (jar files) packaged inside the library. The custom loader fits > > >> into > > >> > the > > >> > >> > natural hierarchy with system (classpath based) classloader as > > the > > >> > >> > immediate parent and bootstrap classloader as the root. As the > > >> > >> delegation > > >> > >> > model works, a class loader attempts to load a class only after > > >> > >> parent(s) > > >> > >> > in the lineage have failed. > > >> > >> > > > >> > >> > In your specific case, the system class loader is able to find > > the > > >> > >> jackson > > >> > >> > library from the configured classpath and so the library > > >> classloader > > >> > is > > >> > >> not > > >> > >> > involved at all for this jar. The problem arising due to > > different > > >> > >> versions > > >> > >> > of a jar present across the hierarchy of classloaders is > similar > > to > > >> > one > > >> > >> > faced in web containers such as Tomcat or application servers > > such > > >> as > > >> > >> > Websphere where system loaded classes can override any > > application > > >> > >> specific > > >> > >> > classes due to name conflict. > > >> > >> > > > >> > >> > The way you are defining a library is correct. The fix needs to > > be > > >> in > > >> > >> the > > >> > >> > way the classes are loaded by AsterixDB (here acting as an > > >> application > > >> > >> > server). > > >> > >> > > > >> > >> > I see two ways:- > > >> > >> > > > >> > >> > a) Skipping the system class loader: AsterixDB uses a > > >> URLClassloader > > >> > to > > >> > >> > load libraries. Setting the parent of the this classloader as > > null > > >> > >> implies > > >> > >> > the bootrstrap classloader (root) as the immediate parent. The > > >> > bootstrap > > >> > >> > classloader loads rt.jar (java runtime classes) and i18n.jar > and > > >> these > > >> > >> > definitely do not have conflicts with your library dependencies > > (if > > >> > they > > >> > >> > do, not sure why one would override the classes in rt.jar and > so > > we > > >> > >> need to > > >> > >> > fix the library). Note that the system classloader is out of > the > > >> > >> hierarchy > > >> > >> > here and so any other jars loaded by AsterixDB from its > classpath > > >> > >> (pom.xml) > > >> > >> > would not be visible. > > >> > >> > > > >> > >> > b) Defining a Custom Class loader: Inside AsterixDB, we > provide a > > >> > custom > > >> > >> > implementation of classloader that attempts to load a class > prior > > >> to > > >> > >> > delegating to the parent. This way, the library classes and > > >> packaged > > >> > >> > dependencies override any system level classes from the class > > path > > >> and > > >> > >> even > > >> > >> > the classes contained in rt.jar and i18n.jar. > > >> > >> > > > >> > >> > I am opening the discussion here to suggest further > alternatives > > or > > >> > >> provide > > >> > >> > preferences. > > >> > >> > > > >> > >> > I have a preference for (a) (skipping the system class loader) > > for > > >> > two > > >> > >> > reasons: > > >> > >> > > > >> > >> > a) it is simpler > > >> > >> > > > >> > >> > b) the other option allows a custom class loader to override > > >> classes > > >> > in > > >> > >> > rt.jar., which is ok but not how classloaders are supposed to > > work > > >> in > > >> > >> > principle. > > >> > >> > > > >> > >> > Regards, > > >> > >> > > > >> > >> > Raman > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > > > >> > >> > On Jun 29, 2016 11:07 PM, "Mike Carey" <[email protected]> > > wrote: > > >> > >> > > > >> > >> > > Any classloader experts have suggestions...? > > >> > >> > > On Jun 29, 2016 10:26 PM, "Xikui Wang" <[email protected]> > wrote: > > >> > >> > > > > >> > >> > > > Hi Devs, > > >> > >> > > > > > >> > >> > > > We found a problem when trying to build external functions > > for > > >> the > > >> > >> > > > cloudberry demo. > > >> > >> > > > > > >> > >> > > > When the external function is depend on certain library, > the > > >> > library > > >> > >> > that > > >> > >> > > > comes with the external function will be blocked by same > > >> library > > >> > in > > >> > >> > > > AsterixDB. In our case, our external function 'geoTag' uses > > >> > jackson > > >> > >> > > v2.7.1, > > >> > >> > > > and we packed all dependencies into one single jar. When > > >> running > > >> > >> > 'geoTag' > > >> > >> > > > on Asterix, it will call jackson v2.0.0 in AsterixDB which > > >> causes > > >> > >> > > > NullPointerException. We have to manually change pom.xml in > > >> > >> AsterixDB > > >> > >> > to > > >> > >> > > > fix that. > > >> > >> > > > > > >> > >> > > > We are wondering is that because we load the external > > function > > >> in > > >> > a > > >> > >> > wrong > > >> > >> > > > way, or this could be one possible interesting problem > which > > is > > >> > >> worth > > >> > >> > > > noticing. Thanks. > > >> > >> > > > > > >> > >> > > > Best, > > >> > >> > > > Xikui > > >> > >> > > > > > >> > >> > > > > >> > >> > > > >> > >> > > >> > > > > >> > > > > >> > > > >> > > >> > > >> > > >> -- > > >> Raman > > >> > > > > > > > > >
