[ 
https://issues.apache.org/jira/browse/NIFI-2873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15575324#comment-15575324
 ] 

ASF subversion and git services commented on NIFI-2873:
-------------------------------------------------------

Commit e969a5ffe3a1e03b0b6a0f0a11e954dc396bf25a in nifi's branch 
refs/heads/master from d810146
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=e969a5f ]

NIFI-2873: Nifi throws UnknownHostException with HA NameNode

Signed-off-by: Matt Burgess <mattyb...@apache.org>

NIFI-2873: Changed test hive-site.xml to use local FS, fixed Checkstyle 
violations

This closes #1113


> PutHiveStreaming throws UnknownHostException with HA NameNode
> -------------------------------------------------------------
>
>                 Key: NIFI-2873
>                 URL: https://issues.apache.org/jira/browse/NIFI-2873
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.0.0
>            Reporter: Franco
>             Fix For: 1.1.0
>
>
> This is the same issue that previously affected Spark:
> https://github.com/Jianfeng-chs/spark/commit/9f2b2bf001262215742be418f24d5093c92ff10f
> We are experiencing this issue consistently when trying to use 
> PutHiveStreaming. In theory this should be a problem with GetHDFS but for 
> whatever reason it is not.
> The fix is identical namely preloading the Hadoop configuration during the 
> processor setup phase. Pull request forthcoming.
> {code:title=Stack Trace|borderStyle=solid}
> 2016-10-06 16:07:59,225 ERROR [Timer-Driven Process Thread-9] 
> o.a.n.processors.hive.PutHiveStreaming
> java.lang.IllegalArgumentException: java.net.UnknownHostException: tdcdv2
>         at 
> org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:374)
>  ~[hadoop-common-2.6.2.jar:na]
>         at 
> org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310)
>  ~[hadoop-hdfs-2.6.2.jar:na]
>         at 
> org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176) 
> ~[hadoop-hdfs-2.6.2.jar:na]
>         at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:668) 
> ~[hadoop-hdfs-2.6.2.jar:na]
>         at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:604) 
> ~[hadoop-hdfs-2.6.2.jar:na]
>         at 
> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:148)
>  ~[hadoop-hdfs-2.6.2.jar:na]
>         at 
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596) 
> ~[hadoop-common-2.6.2.jar:na]
>         at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) 
> ~[hadoop-common-2.6.2.jar:na]
>         at 
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) 
> ~[hadoop-common-2.6.2.jar:na]
>         at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) 
> ~[hadoop-common-2.6.2.jar:na]
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) 
> ~[hadoop-common-2.6.2.jar:na]
>         at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) 
> ~[hadoop-common-2.6.2.jar:na]
>         at 
> org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.<init>(OrcRecordUpdater.java:221)
>  ~[hive-exec-1.2.1.jar:1.2.1]
>         at 
> org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat.getRecordUpdater(OrcOutputFormat.java:292)
>  ~[hive-exec-1.2.1.jar:1.2.1]
>         at 
> org.apache.hive.hcatalog.streaming.AbstractRecordWriter.createRecordUpdater(AbstractRecordWriter.java:141)
>  ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
>         at 
> org.apache.hive.hcatalog.streaming.AbstractRecordWriter.newBatch(AbstractRecordWriter.java:121)
>  ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
>         at 
> org.apache.hive.hcatalog.streaming.StrictJsonWriter.newBatch(StrictJsonWriter.java:37)
>  ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
>         at 
> org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.<init>(HiveEndPoint.java:509)
>  ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
>         at 
> org.apache.hive.hcatalog.streaming.HiveEndPoint$TransactionBatchImpl.<init>(HiveEndPoint.java:461)
>  ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
>         at 
> org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.fetchTransactionBatchImpl(HiveEndPoint.java:345)
>  ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
>         at 
> org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.fetchTransactionBatch(HiveEndPoint.java:325)
>  ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
>         at 
> org.apache.nifi.util.hive.HiveWriter.lambda$nextTxnBatch$1(HiveWriter.java:250)
>  ~[nifi-hive-processors-1.0.0.jar:1.0.0]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to