Ignite-hadoop accelerator is configured as
https://apacheignite.readme.io/v1.0/docs/hadoop-accelerator#section-secondary-file-system.
Idea is to run hive queries on IGFS but looks like its not working that way.
Not sure how to confirm if hive is connecting to HDFS or IGFS. Also, I think
creating partition by passing comma separated string to fs.append is not
creating partition correctly which is resulting in no result on hive query.
Below are some of the configurations I have. Please let me know if I need to
share any more details.
core-site.xml is having these properties:
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>fs.igfs.impl</name>
<value>org.apache.ignite.hadoop.fs.v1.IgniteHadoopFileSystem</value>
</property>
<property>
<name>fs.AbstractFileSystem.igfs.impl</name>
<value>org.apache.ignite.hadoop.fs.v2.IgniteHadoopFileSystem</value>
</property>
<property>
<name>dfs.client.block.write.replace-datanode-on-failure.policy</name>
<value>NEVER</value>
</property>
ignite's default-config has below configuration
<property name="secondaryFileSystem">
<bean
class="org.apache.ignite.hadoop.fs.IgniteHadoopIgfsSecondaryFileSystem">
<property name="fileSystemFactory">
<bean
class="org.apache.ignite.hadoop.fs.CachingHadoopFileSystemFactory">
<property name="uri"
value="hdfs://localhost:9000/"/>
<property name="configPaths">
<list>
<value>D:/hadoop/etc/hadoop/core-site.xml</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
--
View this message in context:
http://apache-ignite-users.70518.x6.nabble.com/how-to-append-data-to-IGFS-so-that-data-gets-saved-to-Hive-partitioned-table-tp15725p15887.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
