Hi Ivan,
Many thanks!
1. *I run Hive query with IFGS without Ignite MapReduce (on TEZ)*
beeline --hiveconf fs.default.name=igfs://dev-dn1:10500
set hive.execution.engine = tez;
*Errors in Hive log:*
2017-04-28 11:38:16,409 [INFO] [main] |service.AbstractService|: Service
org.apache.tez.dag.app.DAGAppMaster failed in state INITED; cause:
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2228)
at
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2780)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2793)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:99)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2829)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2811)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:390)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at
org.apache.tez.common.TezCommonUtils.getTezBaseStagingPath(TezCommonUtils.java:86)
at
org.apache.tez.common.TezCommonUtils.getTezSystemStagingPath(TezCommonUtils.java:145)
at
org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:427)
at
org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.tez.dag.app.DAGAppMaster$7.run(DAGAppMaster.java:2389)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at
org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2386)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2190)
... 17 more
*Errors in Ignite log*
Exception in thread "igfs-#64%null%" java.lang.NullPointerException
at
org.apache.ignite.internal.processors.hadoop.impl.igfs.HadoopIgfsSecondaryFileSystemPositionedReadable.read(HadoopIgfsSecondaryFileSystemPositionedReadable.java:104)
at
org.apache.ignite.internal.processors.igfs.IgfsLazySecondaryFileSystemPositionedReadable.read(IgfsLazySecondaryFileSystemPositionedReadable.java:64)
at
org.apache.ignite.internal.processors.igfs.IgfsDataManager.secondaryDataBlock(IgfsDataManager.java:405)
at
org.apache.ignite.internal.processors.igfs.IgfsDataManager$4.applyx(IgfsDataManager.java:343)
at
org.apache.ignite.internal.processors.igfs.IgfsDataManager$4.applyx(IgfsDataManager.java:332)
at
org.apache.ignite.internal.util.lang.IgniteClosureX.apply(IgniteClosureX.java:38)
at
org.apache.ignite.internal.util.future.GridFutureChainListener.applyCallback(GridFutureChainListener.java:78)
at
org.apache.ignite.internal.util.future.GridFutureChainListener.access$000(GridFutureChainListener.java:30)
at
org.apache.ignite.internal.util.future.GridFutureChainListener$1.run(GridFutureChainListener.java:65)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
Jars (ignite-core-1.9.0.jar, ignite-hadoop-1.9.0.jar,
ignite-shmem-1.0.0.jar) are available on every cluster node in
/usr/hdp/2.6.0.3-8/hadoop/lib.
Environment variables are set on every cluster node:
export JAVA_HOME=/usr/java/jdk1.8.0_101
export IGNITE_HOME=/home/ignite/apache-ignite-hadoop-1.9.0-bin
export HADOOP_HOME=/usr/hdp/current/hadoop-client
export HADOOP_COMMON_HOME=/usr/hdp/2.6.0.3-8/hadoop
export HADOOP_HDFS_HOME=/usr/hdp/current/hadoop-hdfs-client/
export HADOOP_MAPRED_HOME=/usr/hdp/current/hadoop-mapreduce-client/
What could be wrong?
2. *I run Hive query with IgniteMR but without IGFS*
beeline
set mapreduce.jobtracker.address=dev-dn1.co.vectis.local:11211;
set hive.execution.engine=mr;
*Error*
][FATAL][Hadoop-task-2ccf111e-16a9-4f1d-9b40-84166f5bc7d7_1-MAP-2-0-#68%null%][ExecMapper]
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:52)
Then I set hive.vectorized.execution.enabled = false;
*Error*
][ERROR][Hadoop-task-2ccf111e-16a9-4f1d-9b40-84166f5bc7d7_2-MAP-1-0-#199%null%][HadoopRunnableTask]
Task execution failed.
class org.apache.ignite.IgniteCheckedException: class
org.apache.ignite.IgniteCheckedException: Error in configuring object
at
org.apache.ignite.internal.processors.hadoop.impl.v1.HadoopV1MapTask.run(HadoopV1MapTask.java:128)
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row {…}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:565)
3. * I run Hive query with Ignite MapReduce and --hiveconf
ignite.job.shared.classloader but without IFGS*
beeline --hiveconf ignite.job.shared.classloader=false
0: jdbc:hive2://dev-nn2:10000/price> set hive.execution.engine=mr;
0: jdbc:hive2://dev-nn2:10000/price> set
hive.vectorized.execution.enabled=false;
0: jdbc:hive2://dev-nn2:10000/price> set
mapreduce.jobtracker.address=dev-dn1.co.vectis.local:11211;
0: jdbc:hive2://dev-nn2:10000/price> select calday, count(*) from
price.toprice where calday between '2017-03-20' and '2017-03-21' group by
calday order by calday;
First attempt it succeeded with correct result (surprisingly!).
Second attempt it succeeded with wrong result (12 mnl instead of 37 mln)
Third attempt it failed with "Caused by: java.lang.OutOfMemoryError:
Metaspace"
Do you have any cases when Hive + Ignite MapReduce work correctly and stably
on HDP platform (or other distributions)?
I found this discussion on Hortonworks forum about problem with Hive on
MapReduce, so I have doubts that this is possible
https://community.hortonworks.com/questions/24953/solution-for-hive-runtime-error-while-processing-r.html
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/HDP-Hive-Ignite-tp12195p12304.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
