[jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover

2017-09-11 Thread Naveen Gangam (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16161402#comment-16161402
 ] 

Naveen Gangam commented on HIVE-13817:
--

Thanks [~SINGHVJD] . I am closing this jira as duplicate and link the other 
jira as well. Thank you for your contribution.

> Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow 
> for failover
> 
>
> Key: HIVE-13817
> URL: https://issues.apache.org/jira/browse/HIVE-13817
> Project: Hive
>  Issue Type: New Feature
>  Components: Beeline
>Affects Versions: 1.2.1
>Reporter: Vijay Singh
> Attachments: HIVE-13817.1.patch, HIVE-13817.2.patch, 
> HIVE-13817.3.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Currently, in case of BDR clusters, DNS CNAME alias based connections fail. 
> As _HOST resolves to exact endpoint specified in connection string and that 
> may not be intended SPN for kerberos based on reverse DNS lookup. 
> Consequently this JIRA proposes that client specific setting be used to 
> resolv _HOST from CNAME DNS alias to A record entry on the fly in beeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover

2017-09-02 Thread Vijay Singh (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151429#comment-16151429
 ] 

Vijay Singh commented on HIVE-13817:


It appears that Hive-17218 has implemented the same logic for fixing this issue 
recently, So good news is no patch may be required in this case for Canonical 
name resolution.

> Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow 
> for failover
> 
>
> Key: HIVE-13817
> URL: https://issues.apache.org/jira/browse/HIVE-13817
> Project: Hive
>  Issue Type: New Feature
>  Components: Beeline
>Affects Versions: 1.2.1
>Reporter: Vijay Singh
> Attachments: HIVE-13817.1.patch, HIVE-13817.2.patch, 
> HIVE-13817.3.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Currently, in case of BDR clusters, DNS CNAME alias based connections fail. 
> As _HOST resolves to exact endpoint specified in connection string and that 
> may not be intended SPN for kerberos based on reverse DNS lookup. 
> Consequently this JIRA proposes that client specific setting be used to 
> resolv _HOST from CNAME DNS alias to A record entry on the fly in beeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover

2017-09-01 Thread Vijay Singh (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16151417#comment-16151417
 ] 

Vijay Singh commented on HIVE-13817:


Will rebase the patch as asked. Next ETA for patch is 09/03/17

> Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow 
> for failover
> 
>
> Key: HIVE-13817
> URL: https://issues.apache.org/jira/browse/HIVE-13817
> Project: Hive
>  Issue Type: New Feature
>  Components: Beeline
>Affects Versions: 1.2.1
>Reporter: Vijay Singh
> Attachments: HIVE-13817.1.patch, HIVE-13817.2.patch, 
> HIVE-13817.3.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Currently, in case of BDR clusters, DNS CNAME alias based connections fail. 
> As _HOST resolves to exact endpoint specified in connection string and that 
> may not be intended SPN for kerberos based on reverse DNS lookup. 
> Consequently this JIRA proposes that client specific setting be used to 
> resolv _HOST from CNAME DNS alias to A record entry on the fly in beeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover

2017-08-14 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126428#comment-16126428
 ] 

Hive QA commented on HIVE-13817:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12805512/HIVE-13817.3.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6389/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6389/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6389/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-08-14 21:28:04.644
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-6389/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-08-14 21:28:04.647
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
   4f042cc..06d9a6b  master -> origin/master
+ git reset --hard HEAD
HEAD is now at 4f042cc HIVE-17260: Typo: exception has been created and lost in 
the ThriftJDBCBinarySerDe (Oleg Danilov via Peter Vary)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
  (use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 06d9a6b HIVE-16873: Remove Thread Cache From Logging (BELUGA 
BEHR reviewed by Aihua Xu)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-08-14 21:28:10.753
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: patch failed: jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java:100
error: jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java: patch does not 
apply
error: patch failed: jdbc/src/java/org/apache/hive/jdbc/Utils.java:118
error: jdbc/src/java/org/apache/hive/jdbc/Utils.java: patch does not apply
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12805512 - PreCommit-HIVE-Build

> Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow 
> for failover
> 
>
> Key: HIVE-13817
> URL: https://issues.apache.org/jira/browse/HIVE-13817
> Project: Hive
>  Issue Type: New Feature
>  Components: Beeline
>Affects Versions: 1.2.1
>Reporter: Vijay Singh
> Attachments: HIVE-13817.1.patch, HIVE-13817.2.patch, 
> HIVE-13817.3.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Currently, in case of BDR clusters, DNS CNAME alias based connections fail. 
> As _HOST resolves to exact endpoint specified in connection string and that 
> may not be intended SPN for kerberos based on reverse DNS lookup. 
> Consequently this JIRA proposes that client specific setting be used to 
> resolv _HOST from CNAME DNS alias to A record entry on the fly in beeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover

2017-08-14 Thread Aihua Xu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126359#comment-16126359
 ] 

Aihua Xu commented on HIVE-13817:
-

[~SINGHVJD] The patch makes sense to me. Can you attach a new patch from the 
new rebase since it has been a while? Thanks.

> Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow 
> for failover
> 
>
> Key: HIVE-13817
> URL: https://issues.apache.org/jira/browse/HIVE-13817
> Project: Hive
>  Issue Type: New Feature
>  Components: Beeline
>Affects Versions: 1.2.1
>Reporter: Vijay Singh
> Attachments: HIVE-13817.1.patch, HIVE-13817.2.patch, 
> HIVE-13817.3.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Currently, in case of BDR clusters, DNS CNAME alias based connections fail. 
> As _HOST resolves to exact endpoint specified in connection string and that 
> may not be intended SPN for kerberos based on reverse DNS lookup. 
> Consequently this JIRA proposes that client specific setting be used to 
> resolv _HOST from CNAME DNS alias to A record entry on the fly in beeline.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover

2016-05-23 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296999#comment-15296999
 ] 

Hive QA commented on HIVE-13817:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12805512/HIVE-13817.3.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 56 failed/errored test(s), 9933 tests 
executed
*Failed tests:*
{noformat}
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniTezCliDriver-auto_join30.q-vector_decimal_10_0.q-acid_globallimit.q-and-12-more
 - did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_coalesce.q-cbo_windowing.q-tez_join.q-and-12-more - 
did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_interval_2.q-schema_evol_text_nonvec_mapwork_part_all_primitive.q-tez_fsstat.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-auto_join_reordering_values.q-ptf_seqfile.q-auto_join18.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-bucketsortoptimize_insert_7.q-smb_mapjoin_15.q-mapreduce1.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-join9.q-join_casesensitive.q-filter_join_breaktask.q-and-12-more
 - did not produce a TEST-*.xml file
TestSparkCliDriver-join_cond_pushdown_3.q-groupby7.q-auto_join17.q-and-12-more 
- did not produce a TEST-*.xml file
TestSparkCliDriver-order.q-auto_join18_multi_distinct.q-union2.q-and-12-more - 
did not produce a TEST-*.xml file
TestSparkCliDriver-script_pipe.q-stats12.q-auto_join24.q-and-12-more - did not 
produce a TEST-*.xml file
TestSparkCliDriver-smb_mapjoin_4.q-groupby8_map.q-groupby4_map.q-and-12-more - 
did not produce a TEST-*.xml file
TestSparkCliDriver-stats13.q-stats2.q-ppd_gby_join.q-and-12-more - did not 
produce a TEST-*.xml file
TestSparkCliDriver-vector_distinct_2.q-join15.q-load_dyn_part3.q-and-12-more - 
did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver_hbase_queries
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_bucket4
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_bucket5
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_bucket6
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_constprog_partitioner
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_disable_merge_for_bucketing
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_infer_bucket_sort_map_operators
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_infer_bucket_sort_num_buckets
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_infer_bucket_sort_reducers_power_two
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_list_bucket_dml_10
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge1
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge2
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge9
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_orc_merge_diff_fs
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_reduce_deduplicate
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join1
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join2
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join3
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join4
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_vector_outer_join5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_vector_complex_all
org.apache.hadoop.hive.llap.tez.TestConverters.testFragmentSpecToTaskSpec
org.apache.hadoop.hive.llap.tezplugins.TestLlapTaskCommunicator.testFinishableStateUpdateFailure
org.apache.hadoop.hive.metastore.TestFilterHooks.org.apache.hadoop.hive.metastore.TestFilterHooks
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs
org.apache.hadoop.hive.metastore.TestMetaStoreEventListenerOnlyOnCommit.testEventStatus
org.apache.hadoop.hive.metastore.TestMetaStoreInitListener.testMetaStoreInitListener
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.org.apache.hadoop.hive.metastore.TestMetaStoreMetrics
org.apache.hadoop.hive.ql.security.TestClientSideAuthorizationProvider.testSimplePrivileges
org.apache.hadoop.hive.ql.security.TestExtendedAcls.org.apache.hadoop.hive.ql.security.TestExtendedAcls

[jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover

2016-05-22 Thread Vijay Singh (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295939#comment-15295939
 ] 

Vijay Singh commented on HIVE-13817:


Also verified that hive2 action invoked from oozie workflow with CNAME alias 
string and useCanonicalName=true successfully obtains delegationtokens and 
executes the workflow. Please see invocation and verfication log excerpts.
{code}
[root@vjd-1 hive2]# cat hive2job.properties 
nameNode=hdfs://vjd-1.gce.cloudera.com:8020
jobTracker=yarnRM
queueName=default
jdbcURL=jdbc:hive2://vjlb.gce.cloudera.com:1/default;useCanonicalName=true;
jdbcPrincipal=hive/_h...@gce.cloudera.com
examplesRoot=/testproject/code/examplesoozie

oozie.use.system.libpath=true

oozie.wf.application.path=${nameNode}${examplesRoot}/hive2
[root@vjd-1 hive2]# cat workflow.xml 




hive2.server.principal
${jdbcPrincipal}


hive2.jdbc.url
${jdbcURL}







${jobTracker}
${nameNode}






mapred.job.queue.name
${queueName}


${jdbcURL}
script.q
INPUT=${examplesRoot}/input-data/table
OUTPUT=${examplesRoot}/output-data/hive2






Hive2 (Beeline) action failed, error 
message[${wf:errorMessage(wf:lastErrorNode())}]



[root@vjd-1 hive2]# oozie job -oozie 
http://vjtest-local.gce.cloudera.com:11000/oozie -config hive2job.properties 
-run
job: 041-16052507746-oozie-oozi-W
[root@vjd-1 hive2]#  oozie job -oozie 
http://vjtest-local.gce.cloudera.com:11000/oozie -info 
041-16052507746-oozie-oozi-W 
Job ID : 041-16052507746-oozie-oozi-W

Workflow Name : hive2-wf
App Path  : 
hdfs://vjd-1.gce.cloudera.com:8020/testproject/code/examplesoozie/hive2
Status: SUCCEEDED
Run   : 0
User  : vijay
Group : -
Created   : 2016-05-23 05:38 GMT
Started   : 2016-05-23 05:38 GMT
Last Modified : 2016-05-23 05:38 GMT
Ended : 2016-05-23 05:38 GMT
CoordAction ID: -

Actions

ID
StatusExt ID Ext Status Err Code  

041-16052507746-oozie-oozi-W@:start:  
OK-  OK - 

041-16052507746-oozie-oozi-W@hive2-node   
OKjob_1463802572747_0018 SUCCEEDED  - 

041-16052507746-oozie-oozi-W@end  
OK-  OK - 


[root@vjd-1 hive2]# cat script.q
DROP TABLE IF EXISTS test;
SELECT * FROM sample_07;
[root@vjd-1 hive2]# 
{code}

> Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow 
> for failover
> 
>
> Key: HIVE-13817
> URL: https://issues.apache.org/jira/browse/HIVE-13817
> Project: Hive
>  Issue Type: New Feature
>  Components: Beeline
>Affects Versions: 1.2.1
>Reporter: Vijay Singh
> Attachments: HIVE-13817.1.patch, HIVE-13817.2.patch, 
> HIVE-13817.3.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Currently, in case of BDR clusters, DNS CNAME alias based connections fail. 
> As _HOST resolves to exact endpoint specified in connection string and that 
> may not be intended SPN for kerberos based on reverse DNS lookup. 
> Consequently this JIRA proposes that client specific setting be used to 
> resolv _HOST from CNAME DNS alias to A record entry on the fly in beeline.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover

2016-05-22 Thread Vijay Singh (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295707#comment-15295707
 ] 

Vijay Singh commented on HIVE-13817:


Also ran the same tests with impala using modified jdbc driver and SSL turned 
on successfully.
{code}
[root@vjd-1 ~]# beeline -u 
"jdbc:hive2://vjlb.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true"
 -e "show tables;"
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated 
and will likely be removed in a future release
16/05/22 13:02:47 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree 
module jar containing PrefixTreeCodec is not present.  Continuing without it.
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
scan complete in 2ms
Connecting to 
jdbc:hive2://vjlb.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true
Error: Could not open client transport with JDBC Uri: 
jdbc:hive2://vjlb.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true:
 null (state=08S01,code=0)
No current connection
Error: Could not open client transport with JDBC Uri: 
jdbc:hive2://vjlb.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true:
 null (state=08S01,code=0)
[root@vjd-1 ~]# beeline -u 
"jdbc:hive2://vjtest-local.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true"
 -e "show tables;"
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated 
and will likely be removed in a future release
16/05/22 13:03:16 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree 
module jar containing PrefixTreeCodec is not present.  Continuing without it.
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
scan complete in 2ms
Connecting to 
jdbc:hive2://vjtest-local.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true
Connected to: Impala (version 2.5.0-cdh5.7.0)
Driver: Hive JDBC (version 1.1.0-cdh5.7.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
customers
sample_07
sample_08
web_logs
+---+--+
| name  |
+---+--+
+---+--+
No rows selected (2.014 seconds)
Beeline version 1.1.0-cdh5.7.0 by Apache Hive
Closing: 0: 
jdbc:hive2://vjtest-local.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true
[root@vjd-1 ~]# beeline -u 
"jdbc:hive2://vjtest-local.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true;useCanonicalName=true"
 -e "show tables;"
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated 
and will likely be removed in a future release
16/05/22 13:03:42 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree 
module jar containing PrefixTreeCodec is not present.  Continuing without it.
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
scan complete in 2ms
Connecting to 
jdbc:hive2://vjtest-local.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true;useCanonicalName=true
Connected to: Impala (version 2.5.0-cdh5.7.0)
Driver: Hive JDBC (version 1.1.0-cdh5.7.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
customers
sample_07
sample_08
web_logs
+---+--+
| name  |
+---+--+
+---+--+
No rows selected (2.027 seconds)
Beeline version 1.1.0-cdh5.7.0 by Apache Hive
Closing: 0: 
jdbc:hive2://vjtest-local.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true;useCanonicalName=true
[root@vjd-1 ~]# beeline -u 
"jdbc:hive2://vjlb.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true;useCanonicalName=true"
 -e "show tables;"
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated 
and will likely be removed in a future release
16/05/22 13:04:05 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree 
module jar containing PrefixTreeCodec is not present.  Continuing without it.
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
scan complete in 2ms
Connecting to 
jdbc:hive2://vjlb.gce.cloudera.com:21050/default;principal=impala/_h...@gce.cloudera.com;ssl=true;useCanonicalName=true
Connected to: Impala (version 2.5.0-cdh5.7.0)
Driver: Hive JDBC (version 1.1.0-cdh5.7.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
customers
sample_07
sample_08
web_logs
+---+--+
|

[jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover

2016-05-22 Thread Vijay Singh (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295473#comment-15295473
 ] 

Vijay Singh commented on HIVE-13817:


The last combination, where we use Canonicalname and canonicalname resolution 
is turned on in connect string has been pasted below for reference as well.
{code}
Closing: 0: 
jdbc:hive2://vjtest-local.gce.cloudera.com:1/default;principal=hive/_h...@gce.cloudera.com
[root@vjd-1 ~]# beeline -u 
"jdbc:hive2://vjtest-local.gce.cloudera.com:1/default;principal=hive/_h...@gce.cloudera.com;useCanonicalName=true"
 -e "show tables;"
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated 
and will likely be removed in a future release
16/05/22 01:33:24 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree 
module jar containing PrefixTreeCodec is not present.  Continuing without it.
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
scan complete in 2ms
Connecting to 
jdbc:hive2://vjtest-local.gce.cloudera.com:1/default;principal=hive/_h...@gce.cloudera.com;useCanonicalName=true
Connected to: Apache Hive (version 1.1.0-cdh5.7.0)
Driver: Hive JDBC (version 1.1.0-cdh5.7.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
INFO  : Compiling 
command(queryId=hive_2016052201_cd09f83a-5ce1-4cf9-8a3b-9175b00ed4bb): show 
tables
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, 
type:string, comment:from deserializer)], properties:null)
INFO  : Completed compiling 
command(queryId=hive_2016052201_cd09f83a-5ce1-4cf9-8a3b-9175b00ed4bb); Time 
taken: 0.07 seconds
INFO  : Executing 
command(queryId=hive_2016052201_cd09f83a-5ce1-4cf9-8a3b-9175b00ed4bb): show 
tables
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing 
command(queryId=hive_2016052201_cd09f83a-5ce1-4cf9-8a3b-9175b00ed4bb); Time 
taken: 0.135 seconds
INFO  : OK
++--+
|  tab_name  |
++--+
| customers  |
| sample_07  |
| sample_08  |
| web_logs   |
++--+
4 rows selected (0.357 seconds)
Beeline version 1.1.0-cdh5.7.0 by Apache Hive
Closing: 0: 
jdbc:hive2://vjtest-local.gce.cloudera.com:1/default;principal=hive/_h...@gce.cloudera.com;useCanonicalName=true
[root@vjd-1 ~]# 
{code}


> Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow 
> for failover
> 
>
> Key: HIVE-13817
> URL: https://issues.apache.org/jira/browse/HIVE-13817
> Project: Hive
>  Issue Type: New Feature
>  Components: Beeline
>Affects Versions: 1.2.1
>Reporter: Vijay Singh
> Attachments: HIVE-13817.1.patch, HIVE-13817.2.patch, 
> HIVE-13817.3.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Currently, in case of BDR clusters, DNS CNAME alias based connections fail. 
> As _HOST resolves to exact endpoint specified in connection string and that 
> may not be intended SPN for kerberos based on reverse DNS lookup. 
> Consequently this JIRA proposes that client specific setting be used to 
> resolv _HOST from CNAME DNS alias to A record entry on the fly in beeline.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover

2016-05-22 Thread Vijay Singh (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295470#comment-15295470
 ] 

Vijay Singh commented on HIVE-13817:


Please see the following SPN values available to HiveServer2.
{code}
[root@vjd-1 ~]# klist -kt 
/var/run/cloudera-scm-agent/process/2121-hive-HIVESERVER2/hive.keytab 
Keytab name: 
FILE:/var/run/cloudera-scm-agent/process/2121-hive-HIVESERVER2/hive.keytab
KVNO Timestamp   Principal
 --- --
   1 05/22/2016 01:07:19 hive/vjtest-local.gce.cloudera@gce.cloudera.com
   1 05/22/2016 01:07:19 hive/vjtest-local.gce.cloudera@gce.cloudera.com
   1 05/22/2016 01:07:19 hive/vjtest-local.gce.cloudera@gce.cloudera.com
   1 05/22/2016 01:07:19 hive/vjd-1.gce.cloudera@gce.cloudera.com
   1 05/22/2016 01:07:19 hive/vjd-1.gce.cloudera@gce.cloudera.com
   1 05/22/2016 01:07:19 hive/vjd-1.gce.cloudera@gce.cloudera.com
   1 05/22/2016 01:07:19 HTTP/vjd-1.gce.cloudera@gce.cloudera.com
   1 05/22/2016 01:07:19 HTTP/vjd-1.gce.cloudera@gce.cloudera.com
   1 05/22/2016 01:07:19 HTTP/vjd-1.gce.cloudera@gce.cloudera.com
{code}
Please see the following program that determines canonical name.
{code}
[root@vjd-1 ~]# java NSLookupFwd vjlb
Host: vjlb
Canonical HostName: vjtest-local.gce.cloudera.com
{code}

Please find the log excerpt from executing beeline with new driver with cname 
alias functionality turned on in connection string.
{code}
[root@vjd-1 ~]# hadoop version
Hadoop 2.6.0-cdh5.7.0
Subversion http://github.com/cloudera/hadoop -r 
c00978c67b0d3fe9f3b896b5030741bd40bf541a
Compiled by jenkins on 2016-03-23T18:36Z
Compiled with protoc 2.5.0
>From source with checksum b2eabfa328e763c88cb14168f9b372
This command was run using 
/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/jars/hadoop-common-2.6.0-cdh5.7.0.jar
[root@vjd-1 ~]# beeline -u 
"jdbc:hive2://vjlb:1/default;principal=hive/_h...@gce.cloudera.com;useCanonicalName=true"
 -e "show tables;"
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated 
and will likely be removed in a future release
16/05/22 01:14:45 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree 
module jar containing PrefixTreeCodec is not present.  Continuing without it.
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
scan complete in 2ms
Connecting to 
jdbc:hive2://vjlb:1/default;principal=hive/_h...@gce.cloudera.com;useCanonicalName=true
Connected to: Apache Hive (version 1.1.0-cdh5.7.0)
Driver: Hive JDBC (version 1.1.0-cdh5.7.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
INFO  : Compiling 
command(queryId=hive_20160522011414_690deeda-df58-469d-b18f-33890162cf6f): show 
tables
INFO  : Semantic Analysis Completed
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, 
type:string, comment:from deserializer)], properties:null)
INFO  : Completed compiling 
command(queryId=hive_20160522011414_690deeda-df58-469d-b18f-33890162cf6f); Time 
taken: 0.069 seconds
INFO  : Executing 
command(queryId=hive_20160522011414_690deeda-df58-469d-b18f-33890162cf6f): show 
tables
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing 
command(queryId=hive_20160522011414_690deeda-df58-469d-b18f-33890162cf6f); Time 
taken: 0.163 seconds
INFO  : OK
++--+
|  tab_name  |
++--+
| customers  |
| sample_07  |
| sample_08  |
| web_logs   |
++--+
4 rows selected (0.376 seconds)
Beeline version 1.1.0-cdh5.7.0 by Apache Hive
Closing: 0: 
jdbc:hive2://vjlb:1/default;principal=hive/_h...@gce.cloudera.com;useCanonicalName=true
[root@vjd-1 ~]#
{code}

Please find the log excerpt for connection without canonical name functionality 
turned on in connection string.
{code}
[root@vjd-1 ~]# beeline -u 
"jdbc:hive2://vjlb:1/default;principal=hive/_h...@gce.cloudera.com" -e 
"show tables;"
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated 
and will likely be removed in a future release
16/05/22 01:20:41 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree 
module jar containing PrefixTreeCodec is not present.  Continuing without it.
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; 
support was removed in 8.0
scan complete in 3ms
Connecting to 
jdbc:hive2://vjlb:1/default;principal=hive/_h...@gce.cloudera.com
16/05/22 01:20:43 [main]: ERROR transport.TSaslTransport: SASL negotiation 
failure
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: 
No valid credentials provided (Mechanism level: Server not found in Kerberos

[jira] [Commented] (HIVE-13817) Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow for failover

2016-05-22 Thread Vijay Singh (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-13817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15295442#comment-15295442
 ] 

Vijay Singh commented on HIVE-13817:



{panel:title=Test Setup for CNAME alias resolution to A record based on reverse 
dns lookup 
|borderStyle=dashed|borderColor=#ccd|titleBGColor=#F0D6C1|bgColor=#BBAFCE}
[ec2-user@ip-10-0-0-18 ~]$ nslookup testcnamerecord.gce.cloudera.com
Server: 10.0.0.11
Address:10.0.0.11#53

testcnamerecord.gce.cloudera.comcanonical name = 
testarecord.gce.cloudera.com.
Name:   testarecord.gce.cloudera.com
Address: 10.0.0.19

[ec2-user@ip-10-0-0-18 ~]$ java NSLookupFwd testcnamerecord.gce.cloudera.com
Host: testcnamerecord.gce.cloudera.com
Canonical HostName: testarecord.gce.cloudera.com
{panel}

Please find the code snippet  for above app that successfully performs CNAME 
alias to A record resolution based on reverse dns lookup. 

{code:title= NSLookupFwd.java|borderStyle=solid}
[ec2-user@ip-10-0-0-18 ~]$ cat NSLookupFwd.java 
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.logging.Level;
import java.util.logging.Logger;

public class NSLookupFwd
{
   public String canonicalize(String host) {
String cname=null;
try {
  cname = InetAddress.getByName(host).getCanonicalHostName();
  System.out.println("Host: " +
  host);
  System.out.println("Canonical HostName: " +
  cname);
} catch (UnknownHostException ex) {
  Logger.getLogger(NSLookupFwd.class.getName()).log(Level.SEVERE, null, ex);
}
return cname;
  }
  public static void main(String[] args)
  {
new NSLookupFwd().canonicalize(args[0]);
  }
}
{code}

> Allow DNS CNAME ALIAS Resolution from apache hive beeline JDBC URL to allow 
> for failover
> 
>
> Key: HIVE-13817
> URL: https://issues.apache.org/jira/browse/HIVE-13817
> Project: Hive
>  Issue Type: New Feature
>  Components: Beeline
>Affects Versions: 1.2.1
>Reporter: Vijay Singh
> Attachments: HIVE-13817.1.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Currently, in case of BDR clusters, DNS CNAME alias based connections fail. 
> As _HOST resolves to exact endpoint specified in connection string and that 
> may not be intended SPN for kerberos based on reverse DNS lookup. 
> Consequently this JIRA proposes that client specific setting be used to 
> resolv _HOST from CNAME DNS alias to A record entry on the fly in beeline.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)