[jira] [Commented] (NIFI-2828) SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport

2016-10-21 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15596181#comment-15596181
 ] 

ASF subversion and git services commented on NIFI-2828:
---

Commit f5f132b1abc63a0b3612e4efe6a3faa129c337e2 in nifi's branch 
refs/heads/appveyor-improvement from [~mattyb149]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f5f132b ]

NIFI-2828: Fixed issue with transitive Hadoop dependencies in Hive NAR

This closes #1146


> SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP 
> transport 
> 
>
> Key: NIFI-2828
> URL: https://issues.apache.org/jira/browse/NIFI-2828
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joey Frazee
>Assignee: Matt Burgess
> Fix For: 1.1.0
>
>
> SelectHiveQL and PutHiveQL don't currently work with HTTP transport. There 
> appears to be a class loader problem resulting in 
> java.lang.NoClassDefFoundError: Could not initialize class 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> This looks like a conflict with the Apache commons httpclient version in 
> hadoop-common. Removing the hadoop-libraries .nar dependency and provided 
> scope for hadoop-common appears to fix the issue, but I haven't done any 
> rigorous testing so I'm not sure if there are other consequences or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2828) SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport

2016-10-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592666#comment-15592666
 ] 

ASF GitHub Bot commented on NIFI-2828:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1146


> SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP 
> transport 
> 
>
> Key: NIFI-2828
> URL: https://issues.apache.org/jira/browse/NIFI-2828
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joey Frazee
>Assignee: Matt Burgess
> Fix For: 1.1.0
>
>
> SelectHiveQL and PutHiveQL don't currently work with HTTP transport. There 
> appears to be a class loader problem resulting in 
> java.lang.NoClassDefFoundError: Could not initialize class 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> This looks like a conflict with the Apache commons httpclient version in 
> hadoop-common. Removing the hadoop-libraries .nar dependency and provided 
> scope for hadoop-common appears to fix the issue, but I haven't done any 
> rigorous testing so I'm not sure if there are other consequences or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2828) SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport

2016-10-20 Thread ASF subversion and git services (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592664#comment-15592664
 ] 

ASF subversion and git services commented on NIFI-2828:
---

Commit f5f132b1abc63a0b3612e4efe6a3faa129c337e2 in nifi's branch 
refs/heads/master from [~mattyb149]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f5f132b ]

NIFI-2828: Fixed issue with transitive Hadoop dependencies in Hive NAR

This closes #1146


> SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP 
> transport 
> 
>
> Key: NIFI-2828
> URL: https://issues.apache.org/jira/browse/NIFI-2828
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joey Frazee
>Assignee: Matt Burgess
> Fix For: 1.1.0
>
>
> SelectHiveQL and PutHiveQL don't currently work with HTTP transport. There 
> appears to be a class loader problem resulting in 
> java.lang.NoClassDefFoundError: Could not initialize class 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> This looks like a conflict with the Apache commons httpclient version in 
> hadoop-common. Removing the hadoop-libraries .nar dependency and provided 
> scope for hadoop-common appears to fix the issue, but I haven't done any 
> rigorous testing so I'm not sure if there are other consequences or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2828) SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport

2016-10-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15592013#comment-15592013
 ] 

ASF GitHub Bot commented on NIFI-2828:
--

Github user olegz commented on the issue:

https://github.com/apache/nifi/pull/1146
  
Merging. . .


> SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP 
> transport 
> 
>
> Key: NIFI-2828
> URL: https://issues.apache.org/jira/browse/NIFI-2828
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joey Frazee
>Assignee: Matt Burgess
> Fix For: 1.1.0
>
>
> SelectHiveQL and PutHiveQL don't currently work with HTTP transport. There 
> appears to be a class loader problem resulting in 
> java.lang.NoClassDefFoundError: Could not initialize class 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> This looks like a conflict with the Apache commons httpclient version in 
> hadoop-common. Removing the hadoop-libraries .nar dependency and provided 
> scope for hadoop-common appears to fix the issue, but I haven't done any 
> rigorous testing so I'm not sure if there are other consequences or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2828) SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport

2016-10-18 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586572#comment-15586572
 ] 

ASF GitHub Bot commented on NIFI-2828:
--

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/1146

NIFI-2828: Fixed issue with transitive Hadoop dependencies in Hive NAR





You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-2828_2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1146.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1146


commit 7b987df3b092280401cfc1115aa82a0b668a24cb
Author: Matt Burgess 
Date:   2016-10-18T20:22:15Z

NIFI-2828: Fixed issue with transitive Hadoop dependencies in Hive NAR




> SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP 
> transport 
> 
>
> Key: NIFI-2828
> URL: https://issues.apache.org/jira/browse/NIFI-2828
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joey Frazee
>Assignee: Matt Burgess
> Fix For: 1.1.0
>
>
> SelectHiveQL and PutHiveQL don't currently work with HTTP transport. There 
> appears to be a class loader problem resulting in 
> java.lang.NoClassDefFoundError: Could not initialize class 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> This looks like a conflict with the Apache commons httpclient version in 
> hadoop-common. Removing the hadoop-libraries .nar dependency and provided 
> scope for hadoop-common appears to fix the issue, but I haven't done any 
> rigorous testing so I'm not sure if there are other consequences or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2828) SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport

2016-10-04 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15546250#comment-15546250
 ] 

ASF GitHub Bot commented on NIFI-2828:
--

Github user bbende commented on the issue:

https://github.com/apache/nifi/pull/1075
  
Tried this out on against an HDP 2.4 sandbox and got an exception that 
boiled down to:

```
Caused by: java.io.IOException: No FileSystem for scheme: hdfs
at 
org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584) 
~[hadoop-common-2.6.2.jar:na]
at 
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591) 
~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) 
~[hadoop-common-2.6.2.jar:na]
at 
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) 
~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) 
~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) 
~[hadoop-common-2.6.2.jar:na]
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) 
~[hadoop-common-2.6.2.jar:na]
at 
org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.(OrcRecordUpdater.java:221)
 ~[hive-exec-1.2.1.jar:1.2.1]
at 
org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat.getRecordUpdater(OrcOutputFormat.java:292)
 ~[hive-exec-1.2.1.jar:1.2.1]
at 
org.apache.hive.hcatalog.streaming.AbstractRecordWriter.createRecordUpdater(AbstractRecordWriter.java:141)
 ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
at 
org.apache.hive.hcatalog.streaming.AbstractRecordWriter.newBatch(AbstractRecordWriter.java:121)
 ~[hive-hcatalog-streaming-1.2.1.jar:1.2.1]
... 10 common frames omitted
```
Comparing the JARs that get included with Hadoop Libraries NAR vs the Hive 
NAR there appear to be some differences.

Hadoop Libraries NAR:
hadoop-annotations-2.6.2.jar
hadoop-auth-2.6.2.jar
hadoop-client-2.6.2.jar
hadoop-common-2.6.2.jar
hadoop-hdfs-2.6.2.jar
hadoop-mapreduce-client-app-2.6.2.jar
hadoop-mapreduce-client-common-2.6.2.jar
hadoop-mapreduce-client-core-2.6.2.jar
hadoop-mapreduce-client-jobclient-2.6.2.jar
hadoop-mapreduce-client-shuffle-2.6.2.jar
hadoop-yarn-api-2.6.2.jar
hadoop-yarn-client-2.6.2.jar
hadoop-yarn-common-2.6.2.jar
hadoop-yarn-server-common-2.6.2.jar

Hive NAR:
hadoop-annotations-2.6.2.jar
hadoop-auth-2.6.2.jar
hadoop-common-2.6.2.jar
hadoop-mapreduce-client-core-2.6.2.jar
hadoop-yarn-api-2.6.2.jar
hadoop-yarn-common-2.6.2.jar
hadoop-yarn-server-applicationhistoryservice-2.6.0.jar
hadoop-yarn-server-common-2.6.0.jar
hadoop-yarn-server-resourcemanager-2.6.0.jar
hadoop-yarn-server-web-proxy-2.6.0.jar

I think the Hive NAR at least needs the hadoop-hdfs jar, but not sure what 
else.


> SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP 
> transport 
> 
>
> Key: NIFI-2828
> URL: https://issues.apache.org/jira/browse/NIFI-2828
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joey Frazee
>Assignee: Matt Burgess
> Fix For: 1.1.0
>
>
> SelectHiveQL and PutHiveQL don't currently work with HTTP transport. There 
> appears to be a class loader problem resulting in 
> java.lang.NoClassDefFoundError: Could not initialize class 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> This looks like a conflict with the Apache commons httpclient version in 
> hadoop-common. Removing the hadoop-libraries .nar dependency and provided 
> scope for hadoop-common appears to fix the issue, but I haven't done any 
> rigorous testing so I'm not sure if there are other consequences or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2828) SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport

2016-09-27 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15527822#comment-15527822
 ] 

ASF GitHub Bot commented on NIFI-2828:
--

GitHub user mattyb149 opened a pull request:

https://github.com/apache/nifi/pull/1075

NIFI-2828: Fix Hive NAR dependencies

The approach is to remove the parent dependency and put all Hadoop/Hive 
dependencies in the Hive NAR instead of expecting them in a parent classloader.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattyb149/nifi NIFI-2828

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1075.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1075


commit d245a66f0495d416d556fc2b53cc00c288780ad8
Author: Matt Burgess 
Date:   2016-09-27T23:51:49Z

NIFI-2828: Fix Hive NAR dependencies




> SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP 
> transport 
> 
>
> Key: NIFI-2828
> URL: https://issues.apache.org/jira/browse/NIFI-2828
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joey Frazee
>Assignee: Matt Burgess
> Fix For: 1.1.0
>
>
> SelectHiveQL and PutHiveQL don't currently work with HTTP transport. There 
> appears to be a class loader problem resulting in 
> java.lang.NoClassDefFoundError: Could not initialize class 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> This looks like a conflict with the Apache commons httpclient version in 
> hadoop-common. Removing the hadoop-libraries .nar dependency and provided 
> scope for hadoop-common appears to fix the issue, but I haven't done any 
> rigorous testing so I'm not sure if there are other consequences or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2828) SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport

2016-09-27 Thread Andrew Grande (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526688#comment-15526688
 ] 

Andrew Grande commented on NIFI-2828:
-

I saw the same stacktrace for the PutHiveQL processor when connecting to Hive 
over HTTP.

> SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP 
> transport 
> 
>
> Key: NIFI-2828
> URL: https://issues.apache.org/jira/browse/NIFI-2828
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joey Frazee
>Assignee: Matt Burgess
> Fix For: 1.1.0
>
>
> SelectHiveQL and PutHiveQL don't currently work with HTTP transport. There 
> appears to be a class loader problem resulting in 
> java.lang.NoClassDefFoundError: Could not initialize class 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> This looks like a conflict with the Apache commons httpclient version in 
> hadoop-common. Removing the hadoop-libraries .nar dependency and provided 
> scope for hadoop-common appears to fix the issue, but I haven't done any 
> rigorous testing so I'm not sure if there are other consequences or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2828) SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport

2016-09-27 Thread Matt Burgess (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526627#comment-15526627
 ] 

Matt Burgess commented on NIFI-2828:


Initial tests appear to only increase the size of the Hive NAR by 10-12 MB

> SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP 
> transport 
> 
>
> Key: NIFI-2828
> URL: https://issues.apache.org/jira/browse/NIFI-2828
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joey Frazee
>Assignee: Matt Burgess
>
> SelectHiveQL and PutHiveQL don't currently work with HTTP transport. There 
> appears to be a class loader problem resulting in 
> java.lang.NoClassDefFoundError: Could not initialize class 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> This looks like a conflict with the Apache commons httpclient version in 
> hadoop-common. Removing the hadoop-libraries .nar dependency and provided 
> scope for hadoop-common appears to fix the issue, but I haven't done any 
> rigorous testing so I'm not sure if there are other consequences or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (NIFI-2828) SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP transport

2016-09-27 Thread Matt Burgess (JIRA) 

[ 
https://issues.apache.org/jira/browse/NIFI-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526613#comment-15526613
 ] 

Matt Burgess commented on NIFI-2828:


There are definitely issues with Hadoop versions between the 
nifi-hadoop-libraries-nar (usually the parent), nifi-hdfs-processors, 
nifi-hive-nar, HBase stuff, etc.  For Hive it seems fairly picky about its 
dependencies, so my suggested fix is to remove the dependency from the Hive NAR 
on the Hadoop Libraries NAR, remove any "provided" scopes, and remove all 
exclusions. This should involve complete regression testing of the Hive 
components in NiFi.

> SelectHiveQL and PutHiveQL fail with NoClassDefFoundError when using HTTP 
> transport 
> 
>
> Key: NIFI-2828
> URL: https://issues.apache.org/jira/browse/NIFI-2828
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Joey Frazee
>
> SelectHiveQL and PutHiveQL don't currently work with HTTP transport. There 
> appears to be a class loader problem resulting in 
> java.lang.NoClassDefFoundError: Could not initialize class 
> org.apache.http.conn.ssl.SSLConnectionSocketFactory.
> This looks like a conflict with the Apache commons httpclient version in 
> hadoop-common. Removing the hadoop-libraries .nar dependency and provided 
> scope for hadoop-common appears to fix the issue, but I haven't done any 
> rigorous testing so I'm not sure if there are other consequences or not.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)