I was playing around with the latest snapshot version, trying to create a fresher version of my demo docker container, and so I ran through the ADM/AQL 101. However I couldn't get it to work, which was really surprising (and distressing). The error appears after trying to create an RTree index on sender-location:
SEVERE: java.lang.NullPointerException > edu.uci.ics.asterix.metadata.MetadataException: > java.lang.NullPointerException > at > edu.uci.ics.asterix.metadata.MetadataNode.addIndex(MetadataNode.java:229) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323) > at sun.rmi.transport.Transport$1.run(Transport.java:200) > at sun.rmi.transport.Transport$1.run(Transport.java:197) > at java.security.AccessController.doPrivileged(Native Method) > at sun.rmi.transport.Transport.serviceCall(Transport.java:196) > at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$97(TCPTransport.java:683) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$2/1502895914.run(Unknown > Source) > at java.security.AccessController.doPrivileged(Native Method) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > at > sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:276) > at > sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:253) > at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162) > at > java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:194) > at > java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:148) > at com.sun.proxy.$Proxy11.addIndex(Unknown Source) > at > edu.uci.ics.asterix.metadata.MetadataManager.addIndex(MetadataManager.java:417) > at > edu.uci.ics.asterix.aql.translator.AqlTranslator.handleCreateIndexStatement(AqlTranslator.java:920) > at > edu.uci.ics.asterix.aql.translator.AqlTranslator.compileAndExecute(AqlTranslator.java:261) > at > edu.uci.ics.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:97) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:754) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:546) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:483) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:970) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:411) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:904) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110) > at org.eclipse.jetty.server.Server.handle(Server.java:347) > at > org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:439) > at > org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:924) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:781) > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:220) > at > org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:43) > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:545) > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:43) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:529) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at java.io.DataOutputStream.writeUTF(DataOutputStream.java:347) > at java.io.DataOutputStream.writeUTF(DataOutputStream.java:323) > at > edu.uci.ics.hyracks.dataflow.common.data.marshalling.UTF8StringSerializerDeserializer.serialize(UTF8StringSerializerDeserializer.java:44) > at > edu.uci.ics.asterix.dataflow.data.nontagged.serde.AStringSerializerDeserializer.serialize(AStringSerializerDeserializer.java:47) > at > edu.uci.ics.asterix.dataflow.data.nontagged.serde.AStringSerializerDeserializer.serialize(AStringSerializerDeserializer.java:26) > at > edu.uci.ics.asterix.formats.nontagged.AqlSerializerDeserializerProvider$1.serialize(AqlSerializerDeserializerProvider.java:208) > at > edu.uci.ics.asterix.formats.nontagged.AqlSerializerDeserializerProvider$1.serialize(AqlSerializerDeserializerProvider.java:189) > at > edu.uci.ics.asterix.metadata.entitytupletranslators.IndexTupleTranslator.getTupleFromMetadataEntity(IndexTupleTranslator.java:260) > at > edu.uci.ics.asterix.metadata.MetadataNode.addIndex(MetadataNode.java:224) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323) > at sun.rmi.transport.Transport$1.run(Transport.java:200) > at sun.rmi.transport.Transport$1.run(Transport.java:197) > at java.security.AccessController.doPrivileged(Native Method) > at sun.rmi.transport.Transport.serviceCall(Transport.java:196) > at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$97(TCPTransport.java:683) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$2/1502895914.run(Unknown > Source) > at java.security.AccessController.doPrivileged(Native Method) > at > sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > ... 1 more I started going back, trying this on older and older revisions until it stopped happening. It looks like this somehow has been lurking in the Open/nested indexing code for a long time. The error does not reproduce in the testing framework, for some reason I don't understand. The source of the bug seems to be that the RTree index is on a type of UNION(sender-location, null). The name of the type is not set like it should be, it is null. Then in IndexTupleTranslator, the 9th field of the new metadata record is written with the name of the type- which is null, so everything explodes. What is especially puzzling- is that this doesn't reproduce in AsterixHyracksIntegrationUtil. There, the name of the type is "Field_sender-location_in_FacebookMessageType". This type is created too in the deployed version, however it doesn't seem to be picked up. I will keep digging on this, but I imagine someone more familiar with the Open/Nested indexing patch might have an idea about why this is happening. To reproduce just try deploying the snapshot version from the website normally on your machine, and run through ADM/AQL 101. - Ian
