I've submitted a patchset to Gerrit with a proposed fix for this: https://asterix-gerrit.ics.uci.edu/#/c/973/
Thanks, -MDB On Mon, Jul 4, 2016 at 2:37 AM Till Westmann <[email protected]> wrote: > Sounds good to me! > > Cheers, > Till > > On 3 Jul 2016, at 23:34, Michael Blow wrote: > > > I think a better solution might be to invert the normal class loader > > resolution order with an extension of URLClassLoader that overrides > > loadClass(), which tries resolving classes internally before > > delegating to > > the parent. Boot classpath classes can still be forced to load > > correctly. I > > think this will solve the use case without the regression due to > > inability > > to resolve *db classes. > > > > Thanks, > > > > -MDB > > > > On Sat, Jul 2, 2016 at 11:44 PM Xikui Wang <[email protected]> wrote: > > > >> The error log is as follow: > >> > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111) > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143) > >> at > >> > >> > org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39) > >> at > >> > >> > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127) > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341) > >> at org.apache.hyracks.control.nc.Task.run(Task.java:297) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > >> at java.lang.Thread.run(Thread.java:745) > >> Caused by: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75) > >> at > >> > >> > org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53) > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39) > >> at > >> > >> > org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41) > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109) > >> ... 8 more > >> Caused by: java.lang.ClassCastException: > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to > >> org.apache.asterix.external.api.IFunctionFactory > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72) > >> ... 12 more > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111) > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143) > >> at > >> > >> > org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39) > >> at > >> > >> > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127) > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341) > >> at org.apache.hyracks.control.nc.Task.run(Task.java:297) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > >> at java.lang.Thread.run(Thread.java:745) > >> Caused by: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75) > >> at > >> > >> > org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53) > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39) > >> at > >> > >> > org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41) > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109) > >> ... 8 more > >> Caused by: java.lang.ClassCastException: > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to > >> org.apache.asterix.external.api.IFunctionFactory > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72) > >> ... 12 more > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > >> at org.apache.hyracks.control.nc.Task.run(Task.java:297) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > >> at java.lang.Thread.run(Thread.java:745) > >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at > >> > >> > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130) > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341) > >> ... 4 more > >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111) > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143) > >> at > >> > >> > org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39) > >> at > >> > >> > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127) > >> ... 5 more > >> Caused by: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75) > >> at > >> > >> > org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53) > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39) > >> at > >> > >> > org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41) > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109) > >> ... 8 more > >> Caused by: java.lang.ClassCastException: > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to > >> org.apache.asterix.external.api.IFunctionFactory > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72) > >> ... 12 more > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > >> at org.apache.hyracks.control.nc.Task.run(Task.java:297) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > >> at java.lang.Thread.run(Thread.java:745) > >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at > >> > >> > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:130) > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:341) > >> ... 4 more > >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:111) > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.meta.AlgebricksMetaOperatorDescriptor$2.open(AlgebricksMetaOperatorDescriptor.java:143) > >> at > >> > >> > org.apache.asterix.external.feed.dataflow.SyncFeedRuntimeInputHandler.open(SyncFeedRuntimeInputHandler.java:39) > >> at > >> > >> > org.apache.asterix.external.operators.FeedMetaComputeNodePushable.open(FeedMetaComputeNodePushable.java:127) > >> ... 5 more > >> Caused by: > >> org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: > >> Unable to load/instantiate class > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:75) > >> at > >> > >> > org.apache.asterix.external.library.ExternalScalarFunction.<init>(ExternalFunctionProvider.java:53) > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunctionProvider.getExternalFunctionEvaluator(ExternalFunctionProvider.java:39) > >> at > >> > >> > org.apache.asterix.external.library.ExternalScalarFunctionEvaluatorFactory.createScalarEvaluator(ExternalScalarFunctionEvaluatorFactory.java:41) > >> at > >> > >> > org.apache.hyracks.algebricks.runtime.operators.std.AssignRuntimeFactory$1.open(AssignRuntimeFactory.java:109) > >> ... 8 more > >> Caused by: java.lang.ClassCastException: > >> edu.uci.ics.cloudberry.noah.feed.GeoTagFactory cannot be cast to > >> org.apache.asterix.external.api.IFunctionFactory > >> at > >> > >> > org.apache.asterix.external.library.ExternalFunction.<init>(ExternalFunction.java:72) > >> ... 12 more > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> java.lang.InterruptedException > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > >> at org.apache.hyracks.control.nc.Task.run(Task.java:297) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > >> at java.lang.Thread.run(Thread.java:745) > >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > >> java.lang.InterruptedException > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126) > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43) > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344) > >> ... 4 more > >> Caused by: java.lang.InterruptedException > >> at java.lang.Object.wait(Native Method) > >> at java.lang.Object.wait(Object.java:502) > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124) > >> ... 6 more > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> java.lang.InterruptedException > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > >> at org.apache.hyracks.control.nc.Task.run(Task.java:297) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > >> at java.lang.Thread.run(Thread.java:745) > >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > >> java.lang.InterruptedException > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126) > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43) > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344) > >> ... 4 more > >> Caused by: java.lang.InterruptedException > >> at java.lang.Object.wait(Native Method) > >> at java.lang.Object.wait(Object.java:502) > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124) > >> ... 6 more > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> java.lang.InterruptedException > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > >> at org.apache.hyracks.control.nc.Task.run(Task.java:297) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > >> at java.lang.Thread.run(Thread.java:745) > >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > >> java.lang.InterruptedException > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126) > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43) > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344) > >> ... 4 more > >> Caused by: java.lang.InterruptedException > >> at java.lang.Object.wait(Native Method) > >> at java.lang.Object.wait(Object.java:502) > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124) > >> ... 6 more > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> org.apache.hyracks.api.exceptions.HyracksDataException: > >> java.lang.InterruptedException > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:365) > >> at org.apache.hyracks.control.nc.Task.run(Task.java:297) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > >> at > >> > >> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > >> at java.lang.Thread.run(Thread.java:745) > >> Caused by: org.apache.hyracks.api.exceptions.HyracksDataException: > >> java.lang.InterruptedException > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:126) > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicFrameReader.nextFrame(NonDeterministicFrameReader.java:43) > >> at org.apache.hyracks.control.nc.Task.pushFrames(Task.java:344) > >> ... 4 more > >> Caused by: java.lang.InterruptedException > >> at java.lang.Object.wait(Native Method) > >> at java.lang.Object.wait(Object.java:502) > >> at > >> > >> > org.apache.hyracks.dataflow.std.collectors.NonDeterministicChannelReader.findNextSender(NonDeterministicChannelReader.java:124) > >> ... 6 more > >> testFile > >> > >> > src/test/resources/runtimets/queries/external-library/tweetGeoTag/tweetGeoTag.3.update.aql > >> raised an exception: > >> java.lang.Exception: HTTP operation failed: 2 > >> STATUS LINE: HTTP/1.1 500 Server Error > >> SUMMARY: Error: Failure in feed > >> > >> STACKTRACE: org.apache.asterix.common.exceptions.AsterixException: > >> Failure > >> in feed > >> at > >> > >> > org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertNoFailure(FeedLifecycleEventSubscriber.java:62) > >> at > >> > >> > org.apache.asterix.external.feed.management.FeedLifecycleEventSubscriber.assertEvent(FeedLifecycleEventSubscriber.java:55) > >> at > >> > >> > org.apache.asterix.aql.translator.QueryTranslator.handleConnectFeedStatement(QueryTranslator.java:2238) > >> at > >> > >> > org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:366) > >> at > >> > >> > org.apache.asterix.aql.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:253) > >> at > >> > >> > org.apache.asterix.api.http.servlet.RESTAPIServlet.handleRequest(RESTAPIServlet.java:195) > >> at > >> > >> > org.apache.asterix.api.http.servlet.UpdateAPIServlet.handleRequest(UpdateAPIServlet.java:30) > >> at > >> > >> > org.apache.asterix.api.http.servlet.RESTAPIServlet.doPost(RESTAPIServlet.java:162) > >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) > >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) > >> at > >> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546) > >> at > >> > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483) > >> at > >> > >> > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:228) > >> at > >> > >> > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:956) > >> at > >> > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411) > >> at > >> > >> > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:188) > >> at > >> > >> > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:891) > >> at > >> > >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > >> at > >> > >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114) > >> at org.eclipse.jetty.server.Server.handle(Server.java:353) > >> at > >> > >> > org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:598) > >> at > >> > >> > org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1076) > >> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:805) > >> at > >> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:218) > >> at > >> org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:427) > >> at > >> > >> > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:510) > >> at > >> > >> > org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34) > >> at > >> > >> > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40) > >> at > >> > >> > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:450) > >> at java.lang.Thread.run(Thread.java:745) > >> > >> at > >> > >> > org.apache.asterix.test.aql.TestExecutor.executeHttpMethod(TestExecutor.java:275) > >> at > >> > >> > org.apache.asterix.test.aql.TestExecutor.executeUpdate(TestExecutor.java:331) > >> at > >> > org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:496) > >> at > >> > org.apache.asterix.test.aql.TestExecutor.executeTest(TestExecutor.java:762) > >> at > >> > org.apache.asterix.test.runtime.ExecutionTest.test(ExecutionTest.java:106) > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> at > >> > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >> at > >> > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >> at java.lang.reflect.Method.invoke(Method.java:497) > >> at > >> > >> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > >> at > >> > >> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > >> at > >> > >> > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > >> at > >> > >> > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > >> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) > >> at > >> > >> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) > >> at > >> > >> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) > >> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > >> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >> at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >> at org.junit.runners.Suite.runChild(Suite.java:127) > >> at org.junit.runners.Suite.runChild(Suite.java:26) > >> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) > >> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) > >> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) > >> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) > >> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) > >> at > >> > >> > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > >> at > >> > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > >> at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > >> at org.junit.runner.JUnitCore.run(JUnitCore.java:160) > >> at > >> > >> > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:119) > >> at > >> > >> > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42) > >> at > >> > >> > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234) > >> at > >> com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74) > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> at > >> > >> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > >> at > >> > >> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > >> at java.lang.reflect.Method.invoke(Method.java:497) > >> at > >> com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) > >> ...Unexpected! > >> > >> On Sat, Jul 2, 2016 at 7:23 PM, Raman Grover > >> <[email protected]> > >> wrote: > >> > >>> i am missing the attachment > >>> On Jul 2, 2016 6:39 PM, "Xikui Wang" <[email protected]> wrote: > >>> > >>>> Hi Raman, > >>>> > >>>> Thanks for your help. I tried this quick fix on my branch, but it > >>>> introduces some new exceptions. I think this causes Asterix fails > >>>> at > >>>> entering the external function. The error message is attached. > >>>> > >>>> Best, > >>>> Xikui > >>>> > >>>> On Fri, Jul 1, 2016 at 10:11 AM, Raman Grover > >>>> <[email protected] > >>> > >>>> wrote: > >>>> > >>>>> Operations related to setting up an external library are contained > >>>>> in > >>>>> ExternalLibraryUtil > >>>>> < > >>>>> > >>> > >> > https://github.com/apache/asterixdb/blob/master/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/external/ExternalLibraryUtils.java > >>>>>> > >>>>> > >>>>> At line 382, we have > >>>>> // create and return the class loader > >>>>> > >>>>> ClassLoader classLoader = new URLClassLoader(urls, > >>>>> parentClassLoader); > >>>>> return classLoader; > >>>>> > >>>>> Above, we have the parentClassLoader set to the classloader for > >>>>> ExternalLibraryUtil which is the application class loader > >>>>> (AsterixDB's > >>>>> classloader that loads the dependencies from pom.xml). The > >>>>> proposed > >>>>> solution (a) in earlier thread - skipping application classloader > >> would > >>>>> translate to replacing the above code with > >>>>> > >>>>> ClassLoader classLoader = new URLClassLoader(urls, null); > >>>>> > >>>>> Regards, > >>>>> Raman > >>>>> > >>>>> On Thu, Jun 30, 2016 at 4:41 PM, Xikui Wang <[email protected]> > >>>>> wrote: > >>>>> > >>>>>>> > >>>>>>> Hi Abdullah, > >>>>>> > >>>>>> I reverted my code to reproduce the problem. Noticing this > >>>>>> external > >>>>>> function has a couple of other bugs but the dependency one is > >> blocking > >>>>>> others, so this should be enough to reproduce the problem. > >>>>>> > >>>>>>> > >>>>>> The external function package is geoTag.zip. > >>>>>> > >>>>>>> > >>>>>>> > >>>>>>> Test scripts are in tweetGeoTag.zip > >>>>>> > >>>>>> External function is loading data from data/, i.e.: > >>>>>> data/state.json > >> . > >>> So > >>>>>> all json files in data.zip need to be placed under > >>>>>> ../asterixdb/asterix-app/data/ > >>>>>> > >>>>>> The real_tweets_adm.adm used in ddl is also attached. > >>>>>> > >>>>>> This setting will cause > >>>>>> > >>>>>> java.lang.NoSuchMethodError: > >>>>>>> com.fasterxml.jackson.core.JsonFactory.requiresPropertyOrdering()Z > >>>>>>> at > >>>>>>> > >>>>> > >>> > com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:541) > >>>>>>> at > >>>>>>> > >>>>> > >>> > com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:452) > >>>>>>> at > >>> org.wololo.geojson.GeoJSONFactory.<clinit>(GeoJSONFactory.java:17) > >>>>>>> at > >>>>>>> > >>>>>> > >>>>> > >>> > >> > edu.uci.ics.cloudberry.gnosis.USGeoJSONIndex.loadShape(IGeoIndex.scala:29) > >>>>>>> at > >>>>>>> > >>>>>> > >>>>> > >>> > >> > edu.uci.ics.cloudberry.gnosis.USGeoGnosis$.loadShape(USGeoGnosis.scala:101) > >>>>>>> at > >>>>>>> > >>>>>> > >>>>> > >>> > >> > edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:20) > >>>>>>> at > >>>>>>> > >>>>>> > >>>>> > >>> > >> > edu.uci.ics.cloudberry.gnosis.USGeoGnosis$$anonfun$load$1.apply(USGeoGnosis.scala:18) > >>>>>>> at > >>>>>>> > >>>>>> > >>>>> > >>> > >> > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245) > >>>>>>> at > >>>>>>> > >>>>>> > >>>>> > >>> > >> > scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:245) > >>>>>>> at scala.collection.immutable.List.foreach(List.scala:381) > >>>>>>> at > >>>>> scala.collection.TraversableLike$class.map(TraversableLike.scala:245) > >>>>>>> at scala.collection.immutable.List.map(List.scala:285) > >>>>>>> at > >>>>> edu.uci.ics.cloudberry.gnosis.USGeoGnosis.load(USGeoGnosis.scala:18) > >>>>>>> at > >>>>> > edu.uci.ics.cloudberry.gnosis.USGeoGnosis.<init>(USGeoGnosis.scala:15) > >>>>>>> at > >>>>>>> > >>>>>> > >>>>> > >>> > >> > edu.uci.ics.cloudberry.noah.feed.GeoTagFunction.initialize(GeoTagFunction.java:34) > >>>>>>> at > >>>>>>> > >>>>>> > >>>>> > >>> > >> > org.apache.asterix.external.library.ExternalFunction.initialize(ExternalFunction.java:113) > >>>>>>> > >>>>>> > >>>>>> The attachments exceed the size limits. I moved them under this > >> google > >>>>>> drive directory: > >>>>>> > >>>>>> > >>>>>> > >>>>> > >>> > >> > https://drive.google.com/folderview?id=0B_6Dzy3OTjaNRUVJWTRvWEtKSU0&usp=sharing > >>>>>> > >>>>>> Let me know if you need anything else. Thanks. > >>>>>> > >>>>>> Best, > >>>>>> Xikui > >>>>>> > >>>>>> On Thu, Jun 30, 2016 at 12:06 PM, abdullah alamoudi < > >>> [email protected] > >>>>>> > >>>>>> wrote: > >>>>>> > >>>>>>> I will look into that. > >>>>>>>> > >>>>>>>> @Xikui, > >>>>>>>> Do you have a reproducable use case where such a behavior can > >>>>>>>> be > >>>>>> observed? > >>>>>>>> > >>>>>>>> Cheers, > >>>>>>>> Abdullah. > >>>>>>>> > >>>>>>>> On Thu, Jun 30, 2016 at 7:53 PM, Raman Grover < > >>>>> [email protected]> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Hi > >>>>>>>>> > >>>>>>>>> Each external library has an separately associated custom > >>>>>>>>> class > >>>>> loader > >>>>>>>>> (URLClassLoader) that loads the contained functions and any > >>>>>> dependencies > >>>>>>>>> (jar files) packaged inside the library. The custom loader > >>>>>>>>> fits > >>>>> into > >>>>>> the > >>>>>>>>> natural hierarchy with system (classpath based) classloader as > >>> the > >>>>>>>>> immediate parent and bootstrap classloader as the root. As the > >>>>>>>> delegation > >>>>>>>>> model works, a class loader attempts to load a class only > >>>>>>>>> after > >>>>>>>> parent(s) > >>>>>>>>> in the lineage have failed. > >>>>>>>>> > >>>>>>>>> In your specific case, the system class loader is able to find > >>> the > >>>>>>>> jackson > >>>>>>>>> library from the configured classpath and so the library > >>>>> classloader > >>>>>> is > >>>>>>>> not > >>>>>>>>> involved at all for this jar. The problem arising due to > >>> different > >>>>>>>> versions > >>>>>>>>> of a jar present across the hierarchy of classloaders is > >> similar > >>> to > >>>>>> one > >>>>>>>>> faced in web containers such as Tomcat or application servers > >>> such > >>>>> as > >>>>>>>>> Websphere where system loaded classes can override any > >>> application > >>>>>>>> specific > >>>>>>>>> classes due to name conflict. > >>>>>>>>> > >>>>>>>>> The way you are defining a library is correct. The fix needs > >>>>>>>>> to > >>> be > >>>>> in > >>>>>>>> the > >>>>>>>>> way the classes are loaded by AsterixDB (here acting as an > >>>>> application > >>>>>>>>> server). > >>>>>>>>> > >>>>>>>>> I see two ways:- > >>>>>>>>> > >>>>>>>>> a) Skipping the system class loader: AsterixDB uses a > >>>>> URLClassloader > >>>>>> to > >>>>>>>>> load libraries. Setting the parent of the this classloader as > >>> null > >>>>>>>> implies > >>>>>>>>> the bootrstrap classloader (root) as the immediate parent. The > >>>>>> bootstrap > >>>>>>>>> classloader loads rt.jar (java runtime classes) and i18n.jar > >> and > >>>>> these > >>>>>>>>> definitely do not have conflicts with your library > >>>>>>>>> dependencies > >>> (if > >>>>>> they > >>>>>>>>> do, not sure why one would override the classes in rt.jar and > >> so > >>> we > >>>>>>>> need to > >>>>>>>>> fix the library). Note that the system classloader is out of > >> the > >>>>>>>> hierarchy > >>>>>>>>> here and so any other jars loaded by AsterixDB from its > >> classpath > >>>>>>>> (pom.xml) > >>>>>>>>> would not be visible. > >>>>>>>>> > >>>>>>>>> b) Defining a Custom Class loader: Inside AsterixDB, we > >> provide a > >>>>>> custom > >>>>>>>>> implementation of classloader that attempts to load a class > >> prior > >>>>> to > >>>>>>>>> delegating to the parent. This way, the library classes and > >>>>> packaged > >>>>>>>>> dependencies override any system level classes from the class > >>> path > >>>>> and > >>>>>>>> even > >>>>>>>>> the classes contained in rt.jar and i18n.jar. > >>>>>>>>> > >>>>>>>>> I am opening the discussion here to suggest further > >> alternatives > >>> or > >>>>>>>> provide > >>>>>>>>> preferences. > >>>>>>>>> > >>>>>>>>> I have a preference for (a) (skipping the system class > >>>>>>>>> loader) > >>> for > >>>>>> two > >>>>>>>>> reasons: > >>>>>>>>> > >>>>>>>>> a) it is simpler > >>>>>>>>> > >>>>>>>>> b) the other option allows a custom class loader to override > >>>>> classes > >>>>>> in > >>>>>>>>> rt.jar., which is ok but not how classloaders are supposed to > >>> work > >>>>> in > >>>>>>>>> principle. > >>>>>>>>> > >>>>>>>>> Regards, > >>>>>>>>> > >>>>>>>>> Raman > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Jun 29, 2016 11:07 PM, "Mike Carey" <[email protected]> > >>> wrote: > >>>>>>>>> > >>>>>>>>>> Any classloader experts have suggestions...? > >>>>>>>>>> On Jun 29, 2016 10:26 PM, "Xikui Wang" <[email protected]> > >> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hi Devs, > >>>>>>>>>>> > >>>>>>>>>>> We found a problem when trying to build external functions > >>> for > >>>>> the > >>>>>>>>>>> cloudberry demo. > >>>>>>>>>>> > >>>>>>>>>>> When the external function is depend on certain library, > >> the > >>>>>> library > >>>>>>>>> that > >>>>>>>>>>> comes with the external function will be blocked by same > >>>>> library > >>>>>> in > >>>>>>>>>>> AsterixDB. In our case, our external function 'geoTag' uses > >>>>>> jackson > >>>>>>>>>> v2.7.1, > >>>>>>>>>>> and we packed all dependencies into one single jar. When > >>>>> running > >>>>>>>>> 'geoTag' > >>>>>>>>>>> on Asterix, it will call jackson v2.0.0 in AsterixDB which > >>>>> causes > >>>>>>>>>>> NullPointerException. We have to manually change pom.xml in > >>>>>>>> AsterixDB > >>>>>>>>> to > >>>>>>>>>>> fix that. > >>>>>>>>>>> > >>>>>>>>>>> We are wondering is that because we load the external > >>> function > >>>>> in > >>>>>> a > >>>>>>>>> wrong > >>>>>>>>>>> way, or this could be one possible interesting problem > >> which > >>> is > >>>>>>>> worth > >>>>>>>>>>> noticing. Thanks. > >>>>>>>>>>> > >>>>>>>>>>> Best, > >>>>>>>>>>> Xikui > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> Raman > >>>>> > >>>> > >>>> > >>> > >> >
