As Ian mentioned test case already exists (which I thought was good
enough to prevent regressions like this one), but for some reason it’s not
triggering the error, whereas if the same ddl\query is executed on a local
cluster, which was prepared by managix, the error will trigger.
On Jun 9, 2015, at 08:07, Mike Carey <[email protected]> wrote:
Ildar, can you try to nail this one down and also add a test case if you
can create one once this is making more sense...?
On Jun 8, 2015 11:58 PM, "Ildar Absalyamov" <[email protected]
wrote:
Interesting, I guess step 1 here would be to investigate what’s
different
in test framework\AsterixHyracksIntegrationUntil setup and regular
cluster
configuration, created by managix.
On Jun 8, 2015, at 23:37, Ian Maxon <[email protected]> wrote:
I was playing around with the latest snapshot version, trying to
create a
fresher version of my demo docker container, and so I ran through the
ADM/AQL 101. However I couldn't get it to work, which was really
surprising
(and distressing). The error appears after trying to create an RTree
index
on
sender-location:
SEVERE: java.lang.NullPointerException
edu.uci.ics.asterix.metadata.MetadataException:
java.lang.NullPointerException
at
edu.uci.ics.asterix.metadata.MetadataNode.addIndex(MetadataNode.java:229)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$97(TCPTransport.java:683)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$2/1502895914.run(Unknown
Source)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at
sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276)
at
sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
at
java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:194)
at
java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:148)
at com.sun.proxy.$Proxy11.addIndex(Unknown Source)
at
edu.uci.ics.asterix.metadata.MetadataManager.addIndex(MetadataManager.java:417)
at
edu.uci.ics.asterix.aql.translator.AqlTranslator.handleCreateIndexStatement(AqlTranslator.java:920)
at
edu.uci.ics.asterix.aql.translator.AqlTranslator.compileAndExecute(AqlTranslator.java:261)
at
edu.uci.ics.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:97)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:970)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:904)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
at org.eclipse.jetty.server.Server.handle(Server.java:347)
at
org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:439)
at
org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:924)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:781)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220)
at
org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:43)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:545)
at
org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:43)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at java.io.DataOutputStream.writeUTF(DataOutputStream.java:347)
at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323)
at
edu.uci.ics.hyracks.dataflow.common.data.marshalling.UTF8StringSerializerDeserializer.serialize(UTF8StringSerializerDeserializer.java:44)
at
edu.uci.ics.asterix.dataflow.data.nontagged.serde.AStringSerializerDeserializer.serialize(AStringSerializerDeserializer.java:47)
at
edu.uci.ics.asterix.dataflow.data.nontagged.serde.AStringSerializerDeserializer.serialize(AStringSerializerDeserializer.java:26)
at
edu.uci.ics.asterix.formats.nontagged.AqlSerializerDeserializerProvider$1.serialize(AqlSerializerDeserializerProvider.java:208)
at
edu.uci.ics.asterix.formats.nontagged.AqlSerializerDeserializerProvider$1.serialize(AqlSerializerDeserializerProvider.java:189)
at
edu.uci.ics.asterix.metadata.entitytupletranslators.IndexTupleTranslator.getTupleFromMetadataEntity(IndexTupleTranslator.java:260)
at
edu.uci.ics.asterix.metadata.MetadataNode.addIndex(MetadataNode.java:224)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$97(TCPTransport.java:683)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$2/1502895914.run(Unknown
Source)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
I started going back, trying this on older and older revisions until
it
stopped happening. It looks like this somehow has been lurking in the
Open/nested indexing code for a long time. The error does not
reproduce
in
the testing framework, for some reason I don't understand.
The source of the bug seems to be that the RTree index is on a type of
UNION(sender-location, null). The name of the type is not set like it
should be, it is null. Then in IndexTupleTranslator, the 9th field of
the
new metadata record is written with the name of the type- which is
null,
so
everything explodes.
What is especially puzzling- is that this doesn't reproduce in
AsterixHyracksIntegrationUtil. There, the name of the type is
"Field_sender-location_in_FacebookMessageType". This type is created
too
in
the deployed version, however it doesn't seem to be picked up.
I will keep digging on this, but I imagine someone more familiar with
the
Open/Nested indexing patch might have an idea about why this is
happening.
To reproduce just try deploying the snapshot version from the website
normally on your machine, and run through ADM/AQL 101.
- Ian
Best regards,
Ildar
Best regards,
Ildar
