Mingda Li created ASTERIXDB-1735:
------------------------------------
Summary: Unable to load HDFS to AsterixDB
Key: ASTERIXDB-1735
URL: https://issues.apache.org/jira/browse/ASTERIXDB-1735
Project: Apache AsterixDB
Issue Type: Bug
Reporter: Mingda Li
I met a problem when loading data use following query:
use dataverse tpcds3;
load dataset inventory
using
hdfs(("hdfs"="hdfs://SCAI01.CS.UCLA.EDU:9000"),("path"="/clash/datasets/tpcds/10/inventory"),("format"="delimited-text"),("delimiter"="|"));
The Error in web interface is:
Internal error. Please check instance logs for further details.
[NullPointerException]
I check the cc.log (cluster controller) and find the following problem:
SEVERE: Unable to create adapter\
org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: Unable to
create adapter\
at
org.apache.asterix.metadata.declared.AqlMetadataProvider.getConfiguredAdapterFactory(AqlMetadataProvider.java:990)\
at
org.apache.asterix.metadata.declared.LoadableDataSource.buildDatasourceScanRuntime(LoadableDataSource.java:141)\
at
org.apache.asterix.metadata.declared.AqlMetadataProvider.getScannerRuntime(AqlMetadataProvider.java:383)\
at
org.apache.hyracks.algebricks.core.algebra.operators.physical.DataSourceScanPOperator.contributeRuntimeOperator(DataSourceScanPOperator.java:112)\
at
org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator.contributeRuntimeOperator(AbstractLogicalOperator.java:166)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:98)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)\
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compilePlan(PlanCompiler.java:61)\
at
org.apache.hyracks.algebricks.compiler.api.HeuristicCompilerFactoryBuilder$1$1.createJob(HeuristicCompilerFactoryBuilder.java:107)\
at
org.apache.asterix.api.common.APIFramework.compileQuery(APIFramework.java:344)\
at
org.apache.asterix.app.translator.QueryTranslator.handleLoadStatement(QueryTranslator.java:1845)\
at
org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:341)\
at
org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:268)\
at
org.apache.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:132)\
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)\
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)\
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)\
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)\
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)\
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\
at org.eclipse.jetty.server.Server.handle(Server.java:524)\
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)\
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)\
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\
at java.lang.Thread.run(Thread.java:745)\
Caused by: java.lang.NullPointerException\
at
org.apache.asterix.external.util.HDFSUtils.getInputFormatClassName(HDFSUtils.java:155)\
at
org.apache.asterix.external.util.HDFSUtils.configureHDFSJobConf(HDFSUtils.java:186)\
at
org.apache.asterix.external.input.HDFSDataSourceFactory.configure(HDFSDataSourceFactory.java:83)\
at
org.apache.asterix.external.adapter.factory.GenericAdapterFactory.configure(GenericAdapterFactory.java:139)\
at
org.apache.asterix.external.provider.AdapterFactoryProvider.getAdapterFactory(AdapterFactoryProvider.java:49)\
at
org.apache.asterix.metadata.declared.AqlMetadataProvider.getConfiguredAdapterFactory(AqlMetadataProvider.java:969)\
... 45 more\
\
8267828 [qtp1281025083-167] DEBUG org.eclipse.jetty.server.Server - RESPONSE
for / h=true\
200 null\
Date: Wed, 16 Nov 2016 04:34:31 GMT
I change the query by adding the ("input-format"="text-input-format"), and the
error becomes:
Message
Internal error. Please check instance logs for further details. [EOFException]
The log file for it is:
SEVERE: Unable to create adapter
org.apache.hyracks.algebricks.common.exceptions.AlgebricksException: Unable to
create adapter
at
org.apache.asterix.metadata.declared.AqlMetadataProvider.getConfiguredAdapterFactory(AqlMetadataProvider.java:990)
at
org.apache.asterix.metadata.declared.LoadableDataSource.buildDatasourceScanRuntime(LoadableDataSource.java:141)
at
org.apache.asterix.metadata.declared.AqlMetadataProvider.getScannerRuntime(AqlMetadataProvider.java:383)
at
org.apache.hyracks.algebricks.core.algebra.operators.physical.DataSourceScanPOperator.contributeRuntimeOperator(DataSourceScanPOperator.java:112)
at
org.apache.hyracks.algebricks.core.algebra.operators.logical.AbstractLogicalOperator.contributeRuntimeOperator(AbstractLogicalOperator.java:166)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:98)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compileOpRef(PlanCompiler.java:85)
at
org.apache.hyracks.algebricks.core.jobgen.impl.PlanCompiler.compilePlan(PlanCompiler.java:61)
at
org.apache.hyracks.algebricks.compiler.api.HeuristicCompilerFactoryBuilder$1$1.createJob(HeuristicCompilerFactoryBuilder.java:107)
at
org.apache.asterix.api.common.APIFramework.compileQuery(APIFramework.java:344)
at
org.apache.asterix.app.translator.QueryTranslator.handleLoadStatement(QueryTranslator.java:1845)
at
org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:341)
at
org.apache.asterix.app.translator.QueryTranslator.compileAndExecute(QueryTranslator.java:268)
at
org.apache.asterix.api.http.servlet.APIServlet.doPost(APIServlet.java:132)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at
org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:524)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.asterix.common.exceptions.AsterixException:
java.io.IOException: Failed on local exception: java.io.EOFException; Host
Details : local host is: "SCAI01/131.179.64.20"; destination host is:
"SCAI01.CS.UCLA.EDU":9000;
at
org.apache.asterix.external.input.HDFSDataSourceFactory.configure(HDFSDataSourceFactory.java:112)
at
org.apache.asterix.external.adapter.factory.GenericAdapterFactory.configure(GenericAdapterFactory.java:139)
at
org.apache.asterix.external.provider.AdapterFactoryProvider.getAdapterFactory(AdapterFactoryProvider.java:49)
at
org.apache.asterix.metadata.declared.AqlMetadataProvider.getConfiguredAdapterFactory(AqlMetadataProvider.java:969)
... 45 more
Caused by: java.io.IOException: Failed on local exception:
java.io.EOFException; Host Details : local host is: "SCAI01/131.179.64.20";
destination host is: "SCAI01.CS.UCLA.EDU":9000;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:764)
at org.apache.hadoop.ipc.Client.call(Client.java:1351)
at org.apache.hadoop.ipc.Client.call(Client.java:1300)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at com.sun.proxy.$Proxy18.getFileInfo(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy18.getFileInfo(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:651)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1679)
at
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1106)
at
org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1102)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1102)
at
org.apache.hadoop.fs.FileSystem.globStatusInternal(FileSystem.java:1701)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1647)
at
org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:222)
at
org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:270)
at
org.apache.asterix.external.input.HDFSDataSourceFactory.configure(HDFSDataSourceFactory.java:90)
... 48 more
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:392)
at
org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:995)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:891)
11147300 [qtp1281025083-525] DEBUG org.eclipse.jetty.server.Server - RESPONSE
for / h=true
200 null
Date: Wed, 16 Nov 2016 05:22:30 GMT
Content-Type: text/html;charset=utf-8
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
