from:"Rajkumar Singh $Jira$"

[jira] [Commented] (HIVE-25198) CTAS external table with camelcase and HMS translation ON is returning 0 records

2021-06-03 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-25198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17357063#comment-17357063
 ] 

Rajkumar Singh commented on HIVE-25198:
---

[~nareshpr] I think this can be taken care by 
https://issues.apache.org/jira/browse/HIVE-24951

> CTAS external table with camelcase and HMS translation ON is returning 0 
> records
> 
>
> Key: HIVE-25198
> URL: https://issues.apache.org/jira/browse/HIVE-25198
> Project: Hive
>  Issue Type: Bug
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> create external table TarGet as select * from source;
> Above query creates tableLocation with CamelCase if HMS Translation is ON, 
> whereas MoveTask will use lowerCase table path.
> eg., 
> {code:java}
> ==> Desc formatted target <==
> Location:  hdfs:///warehouse/tablespace/external/hive/test.db/TarGet
> ==> MoveTask <==
> INFO : Moving data to directory 
> hdfs:///warehouse/tablespace/external/hive/test.db/target from 
> hdfs:///warehouse/tablespace/external/hive/test.db/.hive-staging_hive_2021-06-04_03-02-36_272_669287187808252905-12/-ext-10002
> ==> HMS Translation <==
> 2021-06-04 03:02:45,772 INFO  
> org.apache.hadoop.hive.metastore.MetastoreDefaultTransformer: 
> [pool-7-thread-8]: Transformer returning table:Table(tableName:TarGet, 
> dbName:test, owner:hive, createTime:1622775765, lastAccessTime:0, 
> retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:id, 
> type:varchar(10), comment:null)], location: 
> hdfs:///warehouse/tablespace/external/hive/test.db/TarGet,{code}
> After CTAS, Select query on target table will return 0 rows.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-24994) get_aggr_stats_for call fail with "Tried to send an out-of-range integer"

2021-04-08 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-24994:
-


> get_aggr_stats_for call fail with "Tried to send an out-of-range integer"
> -
>
> Key: HIVE-24994
> URL: https://issues.apache.org/jira/browse/HIVE-24994
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Fix For: 4.0.0
>
>
> aggrColStatsForPartitions call fail with the Postgres LIMIT if the no of 
> partitions passed in the direct sql goes beyond the 32767
> {code:java}
> postgresql.util.PSQLException: An I/O error occurred while sending to the 
> backend.
>  at 
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:337) 
> ~[postgresql-42.2.8.jar:42.2.8]
>  at 
> org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446) 
> ~[postgresql-42.2.8.jar:42.2.8]
>  at 
> org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370) 
> ~[postgresql-42.2.8.jar:42.2.8]
>  at 
> org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:149)
>  ~[postgresql-42.2.8.jar:42.2.8]
>  at 
> org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:108)
>  ~[postgresql-42.2.8.jar:42.2.8]
>  at 
> com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
>  ~[HikariCP-2.6.1.jar:?]
>  at 
> com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
>  [HikariCP-2.6.1.jar:?]
>  at 
> org.datanucleus.store.rdbms.ParamLoggingPreparedStatement.executeQuery(ParamLoggingPreparedStatement.java:375)
>  [datanucleus-rdbms-4.1.19.jar:?]
>  at 
> org.datanucleus.store.rdbms.SQLController.executeStatementQuery(SQLController.java:552)
>  [datanucleus-rdbms-4.1.19.jar:?]
>  at 
> org.datanucleus.store.rdbms.query.SQLQuery.performExecute(SQLQuery.java:645) 
> [datanucleus-rdbms-4.1.19.jar:?]
>  at 
> org.datanucleus.store.query.Query.executeQuery(Query.java:1855) 
> [datanucleus-core-4.1.17.jar:?]
>  at 
> org.datanucleus.store.rdbms.query.SQLQuery.executeWithArray(SQLQuery.java:807)
>  [datanucleus-rdbms-4.1.19.jar:?]
>  at 
> org.datanucleus.api.jdo.JDOQuery.executeInternal(JDOQuery.java:368) 
> [datanucleus-api-jdo-4.2.4.jar:?]
>  at 
> org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:267) 
> [datanucleus-api-jdo-4.2.4.jar:?]
>  at 
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:2058)
>  [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.executeWithArray(MetaStoreDirectSql.java:2050)
>  [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.access$1500(MetaStoreDirectSql.java:110)
>  [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql$15$1.run(MetaStoreDirectSql.java:1530)
>  [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
> org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:73) 
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql$15.run(MetaStoreDirectSql.java:1521)
>  [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
> org.apache.hadoop.hive.metastore.Batchable.runBatched(Batchable.java:73) 
> [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.partsFoundForPartitions(MetaStoreDirectSql.java:1518)
>  [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.aggrColStatsForPartitions(MetaStoreDirectSql.java:1489)
>  [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
> org.apache.hadoop.hive.metastore.ObjectStore$20.getSqlResult(ObjectStore.java:8966)
>  [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
> org.apache.hadoop.hive.metastore.ObjectStore$20.getSqlResult(ObjectStore.java:8962)
>  [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3757)
>  [hive-exec-3.1.0.3.1.5.6019-4.jar:3.1.0.3.1.5.6019-4]
>  at 
>

[jira] [Assigned] (HIVE-24951) Table created with Uppercase name using CTAS does not produce result for select queries

2021-03-27 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-24951:
-


> Table created with Uppercase name using CTAS does not produce result for 
> select queries
> ---
>
> Key: HIVE-24951
> URL: https://issues.apache.org/jira/browse/HIVE-24951
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> Steps to repro:
> {code:java}
> CREATE EXTERNAL TABLE MY_TEST AS SELECT * FROM source
> Table created with Location but does not have any data moved to it.
> /warehouse/tablespace/external/hive/MY_TEST
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-24950) Fixing the logger for TaskQueue

2021-03-26 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-24950:
-


> Fixing the logger for TaskQueue
> ---
>
> Key: HIVE-24950
> URL: https://issues.apache.org/jira/browse/HIVE-24950
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-24916) EXPORT TABLE command to ADLS Gen2/s3 failing

2021-03-19 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305249#comment-17305249
 ] 

Rajkumar Singh commented on HIVE-24916:
---

Pull request: https://github.com/apache/hive/pull/2095

> EXPORT TABLE command to ADLS Gen2/s3 failing
> 
>
> Key: HIVE-24916
> URL: https://issues.apache.org/jira/browse/HIVE-24916
> Project: Hive
>  Issue Type: Bug
>  Components: repl
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> "EXPORT TABLE" command invoked using distcp command failed with following 
> error -
> org.apache.hadoop.tools.CopyListing$XAttrsNotSupportedException: XAttrs not 
> supported for file system: abfs://storage...@xx.core.windows.net



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-24916) EXPORT TABLE command to ADLS Gen2/s3 failing

2021-03-19 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-24916:
--
Summary: EXPORT TABLE command to ADLS Gen2/s3 failing  (was: EXPORT TABLE 
command to ADLS Gen2/s3 fail with 
org.apache.hadoop.tools.CopyListing$XAttrsNotSupportedException: XAttrs not 
supported for file system: abfs://)

> EXPORT TABLE command to ADLS Gen2/s3 failing
> 
>
> Key: HIVE-24916
> URL: https://issues.apache.org/jira/browse/HIVE-24916
> Project: Hive
>  Issue Type: Bug
>  Components: repl
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> "EXPORT TABLE" command invoked using distcp command failed with following 
> error -
> org.apache.hadoop.tools.CopyListing$XAttrsNotSupportedException: XAttrs not 
> supported for file system: abfs://storage...@xx.core.windows.net



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HIVE-24898) Beeline does not honor the credential provided in property-file

2021-03-19 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh resolved HIVE-24898.
---
Fix Version/s: 4.0.0
   Resolution: Fixed

> Beeline does not honor the credential provided in property-file
> ---
>
> Key: HIVE-24898
> URL: https://issues.apache.org/jira/browse/HIVE-24898
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Fix For: 4.0.0
>
>
> Beeline read the param correctly from the properties files but again fallback 
> to the default beeline connection which require user to provide username and 
> password.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-24898) Beeline does not honor the credential provided in property-file

2021-03-19 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305248#comment-17305248
 ] 

Rajkumar Singh commented on HIVE-24898:
---

https://issues.apache.org/jira/browse/HIVE-24307 can resolve this so closing 
this as duplicate.

> Beeline does not honor the credential provided in property-file
> ---
>
> Key: HIVE-24898
> URL: https://issues.apache.org/jira/browse/HIVE-24898
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> Beeline read the param correctly from the properties files but again fallback 
> to the default beeline connection which require user to provide username and 
> password.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-24916) EXPORT TABLE command to ADLS Gen2/s3 fail with org.apache.hadoop.tools.CopyListing$XAttrsNotSupportedException: XAttrs not supported for file system: abfs://

2021-03-19 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-24916:
-


> EXPORT TABLE command to ADLS Gen2/s3 fail with 
> org.apache.hadoop.tools.CopyListing$XAttrsNotSupportedException: XAttrs not 
> supported for file system: abfs://
> -
>
> Key: HIVE-24916
> URL: https://issues.apache.org/jira/browse/HIVE-24916
> Project: Hive
>  Issue Type: Bug
>  Components: repl
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> "EXPORT TABLE" command invoked using distcp command failed with following 
> error -
> org.apache.hadoop.tools.CopyListing$XAttrsNotSupportedException: XAttrs not 
> supported for file system: abfs://storage...@xx.core.windows.net



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-24898) Beeline does not honor the credential provided in property-file

2021-03-17 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-24898:
-


> Beeline does not honor the credential provided in property-file
> ---
>
> Key: HIVE-24898
> URL: https://issues.apache.org/jira/browse/HIVE-24898
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> Beeline read the param correctly from the properties files but again fallback 
> to the default beeline connection which require user to provide username and 
> password.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-24848) CBO failed with NPE

2021-03-04 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17295569#comment-17295569
 ] 

Rajkumar Singh commented on HIVE-24848:
---

upon further investigating realized that it's same 
https://issues.apache.org/jira/browse/HIVE-24113 but error is in different 
flavor. closing the JIRA marking as duplicate.

> CBO failed with NPE
> ---
>
> Key: HIVE-24848
> URL: https://issues.apache.org/jira/browse/HIVE-24848
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Priority: Major
>
> CBO failed for query having a predicate based on from_unixtime udf
> select * from classification where 
> CAST(from_unixtime(unix_timestamp(cast(partition_batch_ts as 
> string),'MMddHHmmss')) AS TIMESTAMP) = '2021-02-26 02:00:00';
> {code:java}
> 2021-03-04 10:08:58,844 ERROR org.apache.hadoop.hive.ql.parse.CalcitePlanner: 
> [4d92f6e5-9a53-41fb-b53f-9003c338ab52 etp2107079200-38767]: CBO failed, 
> skipping CBO.
> java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.parse.SemanticException: 
> java.lang.NullPointerException
> at 
> org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:159) 
> ~[calcite-core-1.19.0.7.1.3.0-100.jar:1.19.0.7.1.3.0-100]
> at 
> org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:114) 
> ~[calcite-core-1.19.0.7.1.3.0-100.jar:1.19.0.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1544)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:529)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12667)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:422)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:221) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:188) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:598) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:544) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:538) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:127)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:199)
>  ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:260)
>  ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:274) 
> ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:565)
>  ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:551)
>  ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at sun.reflect.GeneratedMethodAccessor207.invoke(Unknown Source) 
> ~[?:?]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_282]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_282]
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HIVE-24848) CBO failed with NPE

2021-03-04 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh resolved HIVE-24848.
---
Fix Version/s: 3.2.0
   Resolution: Duplicate

> CBO failed with NPE
> ---
>
> Key: HIVE-24848
> URL: https://issues.apache.org/jira/browse/HIVE-24848
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Priority: Major
> Fix For: 3.2.0
>
>
> CBO failed for query having a predicate based on from_unixtime udf
> select * from classification where 
> CAST(from_unixtime(unix_timestamp(cast(partition_batch_ts as 
> string),'MMddHHmmss')) AS TIMESTAMP) = '2021-02-26 02:00:00';
> {code:java}
> 2021-03-04 10:08:58,844 ERROR org.apache.hadoop.hive.ql.parse.CalcitePlanner: 
> [4d92f6e5-9a53-41fb-b53f-9003c338ab52 etp2107079200-38767]: CBO failed, 
> skipping CBO.
> java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.parse.SemanticException: 
> java.lang.NullPointerException
> at 
> org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:159) 
> ~[calcite-core-1.19.0.7.1.3.0-100.jar:1.19.0.7.1.3.0-100]
> at 
> org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:114) 
> ~[calcite-core-1.19.0.7.1.3.0-100.jar:1.19.0.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1544)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:529)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12667)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:422)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:221) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:188) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:598) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:544) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:538) 
> ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:127)
>  ~[hive-exec-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:199)
>  ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:260)
>  ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:274) 
> ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:565)
>  ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at 
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:551)
>  ~[hive-service-3.1.3000.7.1.3.0-100.jar:3.1.3000.7.1.3.0-100]
> at sun.reflect.GeneratedMethodAccessor207.invoke(Unknown Source) 
> ~[?:?]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_282]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_282]
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (HIVE-24491) setting custom job name is ineffective if the tez session pool is configured or in case of session reuse.

2021-01-20 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh resolved HIVE-24491.
---
Fix Version/s: 4.0.0
   Resolution: Fixed

> setting custom job name is ineffective if the tez session pool is configured 
> or in case of session reuse.
> -
>
> Key: HIVE-24491
> URL: https://issues.apache.org/jira/browse/HIVE-24491
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> HIVE-23026 add capability to set tez.job.name but it's not effective if tez 
> session pool manager is configured or tez session reuse.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-24491) setting custom job name is ineffective if the tez session pool is configured or in case of session reuse.

2021-01-20 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17268710#comment-17268710
 ] 

Rajkumar Singh commented on HIVE-24491:
---

Thanks [~ashutoshc] for review, PR is merged into the master.

> setting custom job name is ineffective if the tez session pool is configured 
> or in case of session reuse.
> -
>
> Key: HIVE-24491
> URL: https://issues.apache.org/jira/browse/HIVE-24491
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> HIVE-23026 add capability to set tez.job.name but it's not effective if tez 
> session pool manager is configured or tez session reuse.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-24491) setting custom job name is ineffective if the tez session pool is configured or in case of session reuse.

2020-12-04 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-24491:
-


> setting custom job name is ineffective if the tez session pool is configured 
> or in case of session reuse.
> -
>
> Key: HIVE-24491
> URL: https://issues.apache.org/jira/browse/HIVE-24491
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> HIVE-23026 add capability to set tez.job.name but it's not effective if tez 
> session pool manager is configured or tez session reuse.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-24359) Hive Compaction hangs because of doAs when worker set to HS2

2020-11-10 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17229536#comment-17229536
 ] 

Rajkumar Singh commented on HIVE-24359:
---

even without  HIVE-24089 this will fail/hung if doAs is enabled as we are 
creating the proxy user here 
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Worker.java#L539
 if table/sds path owner is different than the hive.

> Hive Compaction hangs because of doAs when worker set to HS2
> 
>
> Key: HIVE-24359
> URL: https://issues.apache.org/jira/browse/HIVE-24359
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Transactions
>Reporter: Chiran Ravani
>Priority: Critical
>
> When creating a managed table and inserting data using Impala, with 
> compaction worker set to HiveServer2 - in secured environment (Kerberized 
> Cluster). Worker thread hangs indefinitely expecting user to provide kerberos 
> credentials from STDIN
> The problem appears to be because of no login context being sent from HS2 to 
> HMS as part of QueryCompactor and HS2 JVM has property 
> javax.security.auth.useSubjectCredsOnly is set to false. Which is causing it 
> to prompt for logins via stdin, however setting to true also does not helo as 
> the context does not seem to be passed in any case.
> Below is observed in HS2 Jstack. If you see the the thread is waiting for 
> stdin "com.sun.security.auth.module.Krb5LoginModule.promptForName"
> {code}
> "c570-node2.abc.host.com-44_executor" #47 daemon prio=1 os_prio=0 
> tid=0x01506000 nid=0x1348 runnable [0x7f1beea95000]
>java.lang.Thread.State: RUNNABLE
> at java.io.FileInputStream.readBytes(Native Method)
> at java.io.FileInputStream.read(FileInputStream.java:255)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> - locked <0x9fa38c90> (a java.io.BufferedInputStream)
> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
> - locked <0x8c7d5010> (a java.io.InputStreamReader)
> at java.io.InputStreamReader.read(InputStreamReader.java:184)
> at java.io.BufferedReader.fill(BufferedReader.java:161)
> at java.io.BufferedReader.readLine(BufferedReader.java:324)
> - locked <0x8c7d5010> (a java.io.InputStreamReader)
> at java.io.BufferedReader.readLine(BufferedReader.java:389)
> at 
> com.sun.security.auth.callback.TextCallbackHandler.readLine(TextCallbackHandler.java:153)
> at 
> com.sun.security.auth.callback.TextCallbackHandler.handle(TextCallbackHandler.java:120)
> at 
> com.sun.security.auth.module.Krb5LoginModule.promptForName(Krb5LoginModule.java:862)
> at 
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:708)
> at 
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
> at 
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
> at java.security.AccessController.doPrivileged(Native Method)
> at 
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
> at sun.security.jgss.GSSUtil.login(GSSUtil.java:258)
> at sun.security.jgss.krb5.Krb5Util.getInitialTicket(Krb5Util.java:175)
> at 
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:341)
> at 
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:337)
> at java.security.AccessController.doPrivileged(Native Method)
> at 
> sun.security.jgss.krb5.Krb5InitCredential.getTgt(Krb5InitCredential.java:336)
> at 
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:146)
> at 
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
> at 
>

[jira] [Commented] (HIVE-24276) HiveServer2 loggerconf jsp Cross-Site Scripting (XSS) Vulnerability

2020-10-14 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214235#comment-17214235
 ] 

Rajkumar Singh commented on HIVE-24276:
---

pull request created :- https://github.com/apache/hive/pull/1581

> HiveServer2 loggerconf jsp Cross-Site Scripting (XSS) Vulnerability 
> 
>
> Key: HIVE-24276
> URL: https://issues.apache.org/jira/browse/HIVE-24276
> Project: Hive
>  Issue Type: Bug
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-24276) HiveServer2 loggerconf jsp Cross-Site Scripting (XSS) Vulnerability

2020-10-14 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-24276:
-


> HiveServer2 loggerconf jsp Cross-Site Scripting (XSS) Vulnerability 
> 
>
> Key: HIVE-24276
> URL: https://issues.apache.org/jira/browse/HIVE-24276
> Project: Hive
>  Issue Type: Bug
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-23867) Truncate table fail with AccessControlException if doAs enabled and tbl database has source of replication

2020-10-01 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-23867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205704#comment-17205704
 ] 

Rajkumar Singh commented on HIVE-23867:
---

Thanks [~aasha] but HIVE-22736 is not resolving the issue, FYI.. the error here 
is for the managed table.

> Truncate table fail with AccessControlException if doAs enabled and tbl 
> database has source of replication
> --
>
> Key: HIVE-23867
> URL: https://issues.apache.org/jira/browse/HIVE-23867
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, repl
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Priority: Major
>
> Steps to repro:
> 1. enable doAs
> 2. with some user (not a super user) create database 
> create database sampledb with dbproperties('repl.source.for'='1,2,3');
> 3. create table using create table sampledb.sampletble (id int);
> 4. insert some data into it insert into sampledb.sampletble values (1), 
> (2),(3);
> 5. Run truncate command on the table which fail with following error
> {code:java}
>  org.apache.hadoop.ipc.RemoteException: User username is not a super user 
> (non-super user cannot change owner).
>  at 
> org.apache.hadoop.hdfs.server.namenode.FSDirAttrOp.setOwner(FSDirAttrOp.java:85)
>  at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setOwner(FSNamesystem.java:1907)
>  at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setOwner(NameNodeRpcServer.java:866)
>  at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setOwner(ClientNamenodeProtocolServerSideTranslatorPB.java:531)
>  at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
>  at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1025)
>  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:876)
>  at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:822)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at javax.security.auth.Subject.doAs(Subject.java:422)
>  at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
>  at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2682)
>  
>  at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1498) 
> ~[hadoop-common-3.1.1.3.1.5.0-152.jar:?]
>  at org.apache.hadoop.ipc.Client.call(Client.java:1444) 
> ~[hadoop-common-3.1.1.3.1.5.0-152.jar:?]
>  at org.apache.hadoop.ipc.Client.call(Client.java:1354) 
> ~[hadoop-common-3.1.1.3.1.5.0-152.jar:?]
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
>  ~[hadoop-common-3.1.1.3.1.5.0-152.jar:?]
>  at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
>  ~[hadoop-common-3.1.1.3.1.5.0-152.jar:?]
>  at com.sun.proxy.$Proxy31.setOwner(Unknown Source) ~[?:?]
>  at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.setOwner(ClientNamenodeProtocolTranslatorPB.java:470)
>  ~[hadoop-hdfs-client-3.1.1.3.1.5.0-152.jar:?]
>  at sun.reflect.GeneratedMethodAccessor151.invoke(Unknown Source) ~[?:?]
>  at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_232]
>  at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_232]
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
>  [hadoop-common-3.1.1.3.1.5.0-152.jar:?]
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
>  ~[hadoop-common-3.1.1.3.1.5.0-152.jar:?]
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
>  ~[hadoop-common-3.1.1.3.1.5.0-152.jar:?]
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
>  [hadoop-common-3.1.1.3.1.5.0-152.jar:?]
>  at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
>  [hadoop-common-3.1.1.3.1.5.0-152.jar:?]
>  at com.sun.proxy.$Proxy32.setOwner(Unknown Source) [?:?]
>  at org.apache.hadoop.hdfs.DFSClient.setOwner(DFSClient.java:1914) 
> [hadoop-hdfs-client-3.1.1.3.1.5.0-152.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem$36.doCall(DistributedFileSystem.java:1764)
>  [hadoop-hdfs-client-3.1.1.3.1.5.0-152.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem$36.doCall(DistributedFileSystem.java:1761)
>  [hadoop-hdfs-client-3.1.1.3.1.5.0-152.jar:?]
>  at 
>

[jira] [Commented] (HIVE-24193) Select query on renamed hive acid table does not produce any output

2020-09-23 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17201016#comment-17201016
 ] 

Rajkumar Singh commented on HIVE-24193:
---

created pull request https://github.com/apache/hive/pull/1520

> Select query on renamed hive acid table does not produce any output
> ---
>
> Key: HIVE-24193
> URL: https://issues.apache.org/jira/browse/HIVE-24193
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> During onRename, HMS update COMPLETED_TXN_COMPONENTS which fail with 
> CTC_DATABASE column does not exist, upon investigation I found that enclosing 
> quotes are missing for columns thats db query fail with this exception
> Steps to repro:
> 1. create table test(id int);
> 2. insert into table test values(1);
> 3. alter table test rename to test1;
> 3. select * from test1 produce no output



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-24193) Select query on renamed hive acid table does not produce any output

2020-09-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-24193:
--
Description: 
During onRename, HMS update COMPLETED_TXN_COMPONENTS which fail with 
CTC_DATABASE column does not exist, upon investigation I found that enclosing 
quotes are missing for columns thats db query fail with this exception

Steps to repro:
1. create table test(id int);
2. insert into table test values(1);
3. alter table test rename to test1;
3. select * from test1 produce no output

  was:During onRename, HMS update COMPLETED_TXN_COMPONENTS which fail with 
CTC_DATABASE column does not exist, upon investigation I found that enclosing 
quotes are missing for columns thats db query fail with this exception


> Select query on renamed hive acid table does not produce any output
> ---
>
> Key: HIVE-24193
> URL: https://issues.apache.org/jira/browse/HIVE-24193
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> During onRename, HMS update COMPLETED_TXN_COMPONENTS which fail with 
> CTC_DATABASE column does not exist, upon investigation I found that enclosing 
> quotes are missing for columns thats db query fail with this exception
> Steps to repro:
> 1. create table test(id int);
> 2. insert into table test values(1);
> 3. alter table test rename to test1;
> 3. select * from test1 produce no output



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-24193) Select query on renamed hive acid table does not produce any output

2020-09-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-24193:
-


> Select query on renamed hive acid table does not produce any output
> ---
>
> Key: HIVE-24193
> URL: https://issues.apache.org/jira/browse/HIVE-24193
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> During onRename, HMS update COMPLETED_TXN_COMPONENTS which fail with 
> CTC_DATABASE column does not exist, upon investigation I found that enclosing 
> quotes are missing for columns thats db query fail with this exception



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-24163) Dynamic Partitioning Insert fail for MM table fail during MoveTask

2020-09-14 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-24163:
--
Summary: Dynamic Partitioning Insert fail for MM table fail during MoveTask 
 (was: Dynamic Partitioning Insert fail for MM table fail while Move Operation)

> Dynamic Partitioning Insert fail for MM table fail during MoveTask
> --
>
> Key: HIVE-24163
> URL: https://issues.apache.org/jira/browse/HIVE-24163
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Rajkumar Singh
>Priority: Major
> Fix For: 3.1.2
>
>
> -- DDLs and Query
> {code:java}
> create table `class` (name varchar(8), sex varchar(1), age double precision, 
> height double precision, weight double precision);
> insert into table class values ('RAJ','MALE',28,12,12);
> CREATE TABLE `PART1` (`id` DOUBLE,`N` DOUBLE,`Name` VARCHAR(8),`Sex` 
> VARCHAR(1)) PARTITIONED BY(Weight string, Age
> string, Height string)  ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' 
> LINES TERMINATED BY '\012' STORED AS TEXTFILE;
> INSERT INTO TABLE `part1` PARTITION (`Weight`,`Age`,`Height`)  SELECT 0, 0, 
> `Name`,`Sex`,`Weight`,`Age`,`Height` FROM `class`;
> {code}
> it fail during the MoveTask execution:
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: partition 
> hdfs://hostname:8020/warehouse/tablespace/managed/hive/part1/.hive-staging_hive_2020-09-02_13-29-58_765_4475282758764123921-1/-ext-1/tmpstats-0_FS_3
>  is not a directory!
> at 
> org.apache.hadoop.hive.ql.metadata.Hive.getValidPartitionsInPath(Hive.java:2769)
>  ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at 
> org.apache.hadoop.hive.ql.metadata.Hive.loadDynamicPartitions(Hive.java:2837) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at 
> org.apache.hadoop.hive.ql.exec.MoveTask.handleDynParts(MoveTask.java:562) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:440) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:359) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:330) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:721) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:488) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:482) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at 
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at 
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:225)
>  ~[hive-service-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> {code}
> The reason is Task write the fsstat during the FileSinkOperator closing, HS2 
> ran the MoveTask to move data into the destination partition directory, while 
> getting the partition location hive check whether destination is directory or 
> not and failing.
> -- hive set the stat location during 
> https://github.com/apache/hive/blob/d700ea54ec5da5364d92a9faaa58f89ea03181e0/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L8135
> which is relative to the  hive-staging directory:
> https://github.com/apache/hive/blob/fecad5b0f72c535ed1c53f2cc62b0d6649b651ae/ql/src/java/org/apache/hadoop/hive/ql/Context.java#L617



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-24163) Dynamic Partitioning Insert fail for MM table fail while Move Operation

2020-09-14 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-24163:
--
Description: 
-- DDLs and Query
{code:java}
create table `class` (name varchar(8), sex varchar(1), age double precision, 
height double precision, weight double precision);

insert into table class values ('RAJ','MALE',28,12,12);
CREATE TABLE `PART1` (`id` DOUBLE,`N` DOUBLE,`Name` VARCHAR(8),`Sex` 
VARCHAR(1)) PARTITIONED BY(Weight string, Age
string, Height string)  ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' LINES 
TERMINATED BY '\012' STORED AS TEXTFILE;

INSERT INTO TABLE `part1` PARTITION (`Weight`,`Age`,`Height`)  SELECT 0, 0, 
`Name`,`Sex`,`Weight`,`Age`,`Height` FROM `class`;
{code}



it fail during the MoveTask execution:

{code:java}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: partition 
hdfs://hostname:8020/warehouse/tablespace/managed/hive/part1/.hive-staging_hive_2020-09-02_13-29-58_765_4475282758764123921-1/-ext-1/tmpstats-0_FS_3
 is not a directory!
at 
org.apache.hadoop.hive.ql.metadata.Hive.getValidPartitionsInPath(Hive.java:2769)
 ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at 
org.apache.hadoop.hive.ql.metadata.Hive.loadDynamicPartitions(Hive.java:2837) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at 
org.apache.hadoop.hive.ql.exec.MoveTask.handleDynParts(MoveTask.java:562) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:440) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at 
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:359) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:330) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:721) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:488) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:482) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at 
org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166) 
~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
at 
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:225)
 ~[hive-service-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]

{code}

The reason is Task write the fsstat during the FileSinkOperator closing, HS2 
ran the MoveTask to move data into the destination partition directory, while 
getting the partition location hive check whether destination is directory or 
not and failing.

-- hive set the stat location during 
https://github.com/apache/hive/blob/d700ea54ec5da5364d92a9faaa58f89ea03181e0/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L8135

which is relative to the  hive-staging directory:

https://github.com/apache/hive/blob/fecad5b0f72c535ed1c53f2cc62b0d6649b651ae/ql/src/java/org/apache/hadoop/hive/ql/Context.java#L617






  was:
-- create MM table 
{code:java}
CREATE TABLE `part1`(  |
|   `id` double, |
|   `n` double,  |
|   `name` varchar(8),   |
|   `sex` varchar(1))|
| PARTITIONED BY (   |
|   `weight` string, |
|   `age` string,|
|   `height` string) |
| ROW FORMAT SERDE   |
|   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'  |
| WITH SERDEPROPERTIES ( |
|   'field.delim'='\u0001',  |
|   'line.delim'='\n',   |
|   'serialization.format'='\u0001') |
| STORED AS INPUTFORMAT  |
|   'org.apache.hadoop.mapred.TextInputFormat'   |
| OUTPUTFORMAT   |
|   'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' |
| LOCATION   |

[jira] [Commented] (HIVE-24163) Dynamic Partitioning Insert fail for MM table fail while Move Operation

2020-09-14 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17195741#comment-17195741
 ] 

Rajkumar Singh commented on HIVE-24163:
---

this seems regression of https://issues.apache.org/jira/browse/HIVE-21164

> Dynamic Partitioning Insert fail for MM table fail while Move Operation
> ---
>
> Key: HIVE-24163
> URL: https://issues.apache.org/jira/browse/HIVE-24163
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Rajkumar Singh
>Priority: Major
> Fix For: 3.1.2
>
>
> -- create MM table 
> {code:java}
> CREATE TABLE `part1`(  |
> |   `id` double, |
> |   `n` double,  |
> |   `name` varchar(8),   |
> |   `sex` varchar(1))|
> | PARTITIONED BY (   |
> |   `weight` string, |
> |   `age` string,|
> |   `height` string) |
> | ROW FORMAT SERDE   |
> |   'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'  |
> | WITH SERDEPROPERTIES ( |
> |   'field.delim'='\u0001',  |
> |   'line.delim'='\n',   |
> |   'serialization.format'='\u0001') |
> | STORED AS INPUTFORMAT  |
> |   'org.apache.hadoop.mapred.TextInputFormat'   |
> | OUTPUTFORMAT   |
> |   'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' |
> | LOCATION   |
> |   'hdfs://hostname:8020/warehouse/tablespace/managed/hive/part1' |
> | TBLPROPERTIES (|
> |   'bucketing_version'='2', |
> |   'transactional'='true',  |
> |   'transactional_properties'='insert_only',|
> |   'transient_lastDdlTime'='1599053368')
> {code}
> -- create managed table 
> {code:java}
> CREATE TABLE `class`(  |
> |   `name` varchar(8),   |
> |   `sex` varchar(1),|
> |   `age` double,|
> |   `height` double, |
> |   `weight` double) |
> | ROW FORMAT SERDE   |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcSerde'  |
> | STORED AS INPUTFORMAT  |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'  |
> | OUTPUTFORMAT   |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION   |
> |   'hdfs://hostname:8020/warehouse/tablespace/managed/hive/class' |
> | TBLPROPERTIES (|
> |   'bucketing_version'='2', |
> |   'transactional'='true',  |
> |   'transactional_properties'='default',|
> |   'transient_lastDdlTime'='1599053345')  
> {code}
> -- Run Insert query
> {code:java}
> INSERT INTO TABLE `part1` PARTITION (`Weight`,`Age`,`Height`)  SELECT 0, 0, 
> `Name`,`Sex`,`Weight`,`Age`,`Height` FROM `class`;
> {code}
> it fail during the MoveTask execution:
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: partition 
> hdfs://hostname:8020/warehouse/tablespace/managed/hive/part1/.hive-staging_hive_2020-09-02_13-29-58_765_4475282758764123921-1/-ext-1/tmpstats-0_FS_3
>  is not a directory!
> at 
> org.apache.hadoop.hive.ql.metadata.Hive.getValidPartitionsInPath(Hive.java:2769)
>  ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at 
> org.apache.hadoop.hive.ql.metadata.Hive.loadDynamicPartitions(Hive.java:2837) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at 
> org.apache.hadoop.hive.ql.exec.MoveTask.handleDynParts(MoveTask.java:562) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:440) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:359) 
> ~[hive-exec-3.1.3000.7.2.0.0-237.jar:3.1.3000.7.2.0.0-237]
> at

[jira] [Comment Edited] (HIVE-24039) Update jquery version to mitigate CVE-2020-11023

2020-09-02 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189789#comment-17189789
 ] 

Rajkumar Singh edited comment on HIVE-24039 at 9/3/20, 3:21 AM:


Hi Kishen, The pull request is already available for this, 
https://github.com/apache/hive/pull/1403, can you please review it?


was (Author: rajkumar singh):
Hi Kishen, The pul request is already available for this, 
https://github.com/apache/hive/pull/1403, can you please review it?

> Update jquery version to mitigate CVE-2020-11023
> 
>
> Key: HIVE-24039
> URL: https://issues.apache.org/jira/browse/HIVE-24039
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Rajkumar Singh
>Assignee: Kishen Das
>Priority: Major
>
> there is known vulnerability in jquery version used by hive, with this jira 
> plan is to upgrade the jquery version 3.5.0 where it's been fixed. more 
> details about the vulnerability can be found here.
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-11023



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-24039) Update jquery version to mitigate CVE-2020-11023

2020-09-02 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189789#comment-17189789
 ] 

Rajkumar Singh commented on HIVE-24039:
---

Hi Kishen, The pul request is already available for this, 
https://github.com/apache/hive/pull/1403, can you please review it?

> Update jquery version to mitigate CVE-2020-11023
> 
>
> Key: HIVE-24039
> URL: https://issues.apache.org/jira/browse/HIVE-24039
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Rajkumar Singh
>Assignee: Kishen Das
>Priority: Major
>
> there is known vulnerability in jquery version used by hive, with this jira 
> plan is to upgrade the jquery version 3.5.0 where it's been fixed. more 
> details about the vulnerability can be found here.
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-11023



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-24113) NPE in GenericUDFToUnixTimeStamp

2020-09-02 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-24113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189688#comment-17189688
 ] 

Rajkumar Singh commented on HIVE-24113:
---

created pull request https://github.com/apache/hive/pull/1460/files

> NPE in GenericUDFToUnixTimeStamp
> 
>
> Key: HIVE-24113
> URL: https://issues.apache.org/jira/browse/HIVE-24113
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> Following query will trigger the getPartitionsByExpr call at HMS, HMS will 
> try to evaluate the filter based on the PartitionExpressionForMetastore 
> proxy, this proxy uses the QL packages to evaluate the filter and call 
> GenericUDFToUnixTimeStamp.
> select * from table_name where hour between 
> from_unixtime(unix_timestamp('2020090120', 'MMddHH') - 1*60*60, 
> 'MMddHH') and from_unixtime(unix_timestamp('2020090122', 'MMddHH') + 
> 2*60*60, 'MMddHH');
> I think SessionState in the code path will always be NULL thats why it hit 
> the NPE.
> {code:java}
> java.lang.NullPointerException: null
> at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUnixTimeStamp.initializeInput(GenericUDFToUnixTimeStamp.java:126)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUnixTimeStamp.initialize(GenericUDFToUnixTimeStamp.java:75)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:148)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:146)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.optimizer.ppr.PartExprEvalUtils.prepareExpr(PartExprEvalUtils.java:119)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner.prunePartitionNames(PartitionPruner.java:551)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.optimizer.ppr.PartitionExpressionForMetastore.filterPartitionsByExpr(PartitionExpressionForMetastore.java:82)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.getPartitionNamesPrunedByExprNoTxn(ObjectStore.java:3527)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.access$1400(ObjectStore.java:252)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore$10.getJdoResult(ObjectStore.java:3493)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore$10.getJdoResult(ObjectStore.java:3464)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3764)
>  [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByExprInternal(ObjectStore.java:3499)
>  [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByExpr(ObjectStore.java:3452)
>  [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[?:1.8.0_112]
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[?:1.8.0_112]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_112]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
> at 
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) 
> [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at

[jira] [Assigned] (HIVE-24113) NPE in GenericUDFToUnixTimeStamp

2020-09-02 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-24113:
-


> NPE in GenericUDFToUnixTimeStamp
> 
>
> Key: HIVE-24113
> URL: https://issues.apache.org/jira/browse/HIVE-24113
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> Following query will trigger the getPartitionsByExpr call at HMS, HMS will 
> try to evaluate the filter based on the PartitionExpressionForMetastore 
> proxy, this proxy uses the QL packages to evaluate the filter and call 
> GenericUDFToUnixTimeStamp.
> select * from table_name where hour between 
> from_unixtime(unix_timestamp('2020090120', 'MMddHH') - 1*60*60, 
> 'MMddHH') and from_unixtime(unix_timestamp('2020090122', 'MMddHH') + 
> 2*60*60, 'MMddHH');
> I think SessionState in the code path will always be NULL thats why it hit 
> the NPE.
> {code:java}
> java.lang.NullPointerException: null
> at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUnixTimeStamp.initializeInput(GenericUDFToUnixTimeStamp.java:126)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUnixTimeStamp.initialize(GenericUDFToUnixTimeStamp.java:75)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:148)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:146)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.optimizer.ppr.PartExprEvalUtils.prepareExpr(PartExprEvalUtils.java:119)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner.prunePartitionNames(PartitionPruner.java:551)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.ql.optimizer.ppr.PartitionExpressionForMetastore.filterPartitionsByExpr(PartitionExpressionForMetastore.java:82)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.getPartitionNamesPrunedByExprNoTxn(ObjectStore.java:3527)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.access$1400(ObjectStore.java:252)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore$10.getJdoResult(ObjectStore.java:3493)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore$10.getJdoResult(ObjectStore.java:3464)
>  ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3764)
>  [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByExprInternal(ObjectStore.java:3499)
>  [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at 
> org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByExpr(ObjectStore.java:3452)
>  [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
> ~[?:1.8.0_112]
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
> ~[?:1.8.0_112]
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  ~[?:1.8.0_112]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
> at 
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) 
> [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
> at com.sun.proxy.$Proxy28.getPartitionsByExpr(Unknown Source) [?:?]
> at 
>

[jira] [Updated] (HIVE-24039) Update jquery version to mitigate CVE-2020-11023

2020-08-13 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-24039:
--
Summary: Update jquery version to mitigate CVE-2020-11023  (was: update 
jquery version to mitigate CVE-2020-11023)

> Update jquery version to mitigate CVE-2020-11023
> 
>
> Key: HIVE-24039
> URL: https://issues.apache.org/jira/browse/HIVE-24039
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> there is known vulnerability in jquery version used by hive, with this jira 
> plan is to upgrade the jquery version 3.5.0 where it's been fixed. more 
> details about the vulnerability can be found here.
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-11023



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-24039) update jquery version to mitigate CVE-2020-11023

2020-08-13 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-24039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-24039:
-


> update jquery version to mitigate CVE-2020-11023
> 
>
> Key: HIVE-24039
> URL: https://issues.apache.org/jira/browse/HIVE-24039
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> there is known vulnerability in jquery version used by hive, with this jira 
> plan is to upgrade the jquery version 3.5.0 where it's been fixed. more 
> details about the vulnerability can be found here.
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-11023



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-23968) CTAS with TBLPROPERTIES ('transactional'='false') does not entertain translated table location

2020-07-31 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-23968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17169115#comment-17169115
 ] 

Rajkumar Singh commented on HIVE-23968:
---

created pull request https://github.com/apache/hive/pull/1348

> CTAS with TBLPROPERTIES ('transactional'='false') does not entertain 
> translated table location
> --
>
> Key: HIVE-23968
> URL: https://issues.apache.org/jira/browse/HIVE-23968
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> HMS translation layer convert the table to external based on the 
> transactional property set to false but MoveTask does not entertain the 
> translated table location and move the data to the managed table location;
> steps to repro:
> {code:java}
> create table nontxnal TBLPROPERTIES ('transactional'='false') as select * 
> from abc;
> {code}
> select query on table return nothing t but the source table has data in it.
> {code:java}
> select * from nontxnal;
> +--+
> | nontxnal.id  |
> +--+
> +--+
> {code}
> --show create table
> {code:java}
> CREATE EXTERNAL TABLE `nontxnal`(  |
> |   `id` int)|
> | ROW FORMAT SERDE   |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcSerde'  |
> | STORED AS INPUTFORMAT  |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'  |
> | OUTPUTFORMAT   |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION   |
> |   'hdfs://hostname:8020/warehouse/tablespace/external/hive/nontxnal' |
> | TBLPROPERTIES (|
> |   'TRANSLATED_TO_EXTERNAL'='TRUE', |
> |   'bucketing_version'='2', |
> |   'external.table.purge'='TRUE',   |
> |   'transient_lastDdlTime'='1596215634')|
> {code}
> table data is moved to the managed location:
> ```
> dfs -ls -R  hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal
> . . . . . . . . . . . . . . . . . . . . . . .> ;
> ++
> | DFS Output |
> ++
> | -rw-rw+  3 hive hadoop201 2020-07-31 17:05 
> hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal/00_0 |
> ++
> ```
> The problem seems to be here
> isExternal evaluates to false since the statement is missing external
> https://github.com/apache/hive/blob/d4bfd2ea1ee797f53227f447749cbc97803cd5dc/ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java#L446
> and location return to the managed location
> https://github.com/apache/hive/blob/d4bfd2ea1ee797f53227f447749cbc97803cd5dc/ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java#L455



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-23968) CTAS with TBLPROPERTIES ('transactional'='false') does not entertain translated table location

2020-07-31 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-23968:
-

Assignee: Rajkumar Singh

> CTAS with TBLPROPERTIES ('transactional'='false') does not entertain 
> translated table location
> --
>
> Key: HIVE-23968
> URL: https://issues.apache.org/jira/browse/HIVE-23968
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> HMS translation layer convert the table to external based on the 
> transactional property set to false but MoveTask does not entertain the 
> translated table location and move the data to the managed table location;
> steps to repro:
> {code:java}
> create table nontxnal TBLPROPERTIES ('transactional'='false') as select * 
> from abc;
> {code}
> select query on table return nothing t but the source table has data in it.
> {code:java}
> select * from nontxnal;
> +--+
> | nontxnal.id  |
> +--+
> +--+
> {code}
> --show create table
> {code:java}
> CREATE EXTERNAL TABLE `nontxnal`(  |
> |   `id` int)|
> | ROW FORMAT SERDE   |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcSerde'  |
> | STORED AS INPUTFORMAT  |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'  |
> | OUTPUTFORMAT   |
> |   'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION   |
> |   'hdfs://hostname:8020/warehouse/tablespace/external/hive/nontxnal' |
> | TBLPROPERTIES (|
> |   'TRANSLATED_TO_EXTERNAL'='TRUE', |
> |   'bucketing_version'='2', |
> |   'external.table.purge'='TRUE',   |
> |   'transient_lastDdlTime'='1596215634')|
> {code}
> table data is moved to the managed location:
> ```
> dfs -ls -R  hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal
> . . . . . . . . . . . . . . . . . . . . . . .> ;
> ++
> | DFS Output |
> ++
> | -rw-rw+  3 hive hadoop201 2020-07-31 17:05 
> hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal/00_0 |
> ++
> ```
> The problem seems to be here
> isExternal evaluates to false since the statement is missing external
> https://github.com/apache/hive/blob/d4bfd2ea1ee797f53227f447749cbc97803cd5dc/ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java#L446
> and location return to the managed location
> https://github.com/apache/hive/blob/d4bfd2ea1ee797f53227f447749cbc97803cd5dc/ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java#L455



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23968) CTAS with TBLPROPERTIES ('transactional'='false') does not entertain translated table location

2020-07-31 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23968:
--
Description: 
HMS translation layer convert the table to external based on the transactional 
property set to false but MoveTask does not entertain the translated table 
location and move the data to the managed table location;

steps to repro:

{code:java}
create table nontxnal TBLPROPERTIES ('transactional'='false') as select * from 
abc;
{code}

select query on table return nothing t but the source table has data in it.
{code:java}
select * from nontxnal;
+--+
| nontxnal.id  |
+--+
+--+
{code}

--show create table

{code:java}
CREATE EXTERNAL TABLE `nontxnal`(  |
|   `id` int)|
| ROW FORMAT SERDE   |
|   'org.apache.hadoop.hive.ql.io.orc.OrcSerde'  |
| STORED AS INPUTFORMAT  |
|   'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'  |
| OUTPUTFORMAT   |
|   'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
| LOCATION   |
|   'hdfs://hostname:8020/warehouse/tablespace/external/hive/nontxnal' |
| TBLPROPERTIES (|
|   'TRANSLATED_TO_EXTERNAL'='TRUE', |
|   'bucketing_version'='2', |
|   'external.table.purge'='TRUE',   |
|   'transient_lastDdlTime'='1596215634')|

{code}

table data is moved to the managed location:
```
dfs -ls -R  hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal
. . . . . . . . . . . . . . . . . . . . . . .> ;
++
| DFS Output |
++
| -rw-rw+  3 hive hadoop201 2020-07-31 17:05 
hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal/00_0 |
++

```
The problem seems to be here
isExternal evaluates to false since the statement is missing external
https://github.com/apache/hive/blob/d4bfd2ea1ee797f53227f447749cbc97803cd5dc/ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java#L446

and location return to the managed location
https://github.com/apache/hive/blob/d4bfd2ea1ee797f53227f447749cbc97803cd5dc/ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java#L455



  was:
HMS translation layer convert the table to external based on the transactional 
property set to false but MoveTask does not entertain the translated table 
location and move the data to the managed table location;

steps to repro:

{code:java}
create table nontxnal TBLPROPERTIES ('transactional'='false') as select * from 
abc;
{code}

select query on table return nothing t but the source table has data in it.
{code:java}
select * from nontxnal;
+--+
| nontxnal.id  |
+--+
+--+
{code}

--show create table

{code:java}
CREATE EXTERNAL TABLE `nontxnal`(  |
|   `id` int)|
| ROW FORMAT SERDE   |
|   'org.apache.hadoop.hive.ql.io.orc.OrcSerde'  |
| STORED AS INPUTFORMAT  |
|   'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'  |
| OUTPUTFORMAT   |
|   'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
| LOCATION   |
|   'hdfs://hostname:8020/warehouse/tablespace/external/hive/nontxnal' |
| TBLPROPERTIES (|
|   'TRANSLATED_TO_EXTERNAL'='TRUE', |
|   'bucketing_version'='2', |
|   'external.table.purge'='TRUE',   |
|   'transient_lastDdlTime'='1596215634')|

{code}

table data is moved to the managed location:
```
dfs -ls -R  hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal
. . . . . . . . . . . . . . . . . . . . . . .> ;
++
| DFS Output |
++
| -rw-rw+  3 hive hadoop201 2020-07-31 17:05 
hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal/00_0 |
++

```




> CTAS with TBLPROPERTIES ('transactional'='false') does not entertain 
> translated table location
> --
>
> Key: HIVE-23968
> URL: https://issues.apache.org/jira/browse/HIVE-23968
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter:

[jira] [Commented] (HIVE-23962) Make bin/hive pick user defined jdbc url

2020-07-30 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-23962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17168389#comment-17168389
 ] 

Rajkumar Singh commented on HIVE-23962:
---

beeline-site.xml serve the same purpose, where you can define what all hs2 URL 
you want to use in connection string, please check BeelineSiteParser for same.

> Make bin/hive pick user defined jdbc url 
> -
>
> Key: HIVE-23962
> URL: https://issues.apache.org/jira/browse/HIVE-23962
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Xiaomeng Zhang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently hive command will trigger bin/hive which run "beeline" by default.
> We want to pass a env variable so that user can define which url beeline use.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-23753) Make LLAP Secretmanager token path configurable

2020-07-10 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-23753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155636#comment-17155636
 ] 

Rajkumar Singh commented on HIVE-23753:
---

https://github.com/apache/hive/pull/1171

> Make LLAP Secretmanager token path configurable
> ---
>
> Key: HIVE-23753
> URL: https://issues.apache.org/jira/browse/HIVE-23753
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> In a very Busy LLAP cluster if for some reason the Tokens under 
> zkdtsm_hive_llap0 zk path are not cleaned then LLAP Daemon startup takes a 
> very long time to startup, this may lead to service outage if LLAP daemons 
> are not started and the number of retries while checking LLAP app status 
> exceeds. upon looking the jstack of llap daemon it seems to traverse the 
> zkdtsm_hive_llap0 zk path before starting the secret manager.
> {code:java}
>java.lang.Thread.State: WAITING (on object monitor)
>   at java.lang.Object.wait(Native Method)
>   at java.lang.Object.wait(Object.java:502)
>   at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1386)
>   - locked <0x7fef36cdd338> (a org.apache.zookeeper.ClientCnxn$Packet)
>   at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1153)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:302)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:291)
>   at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl$2.forPath(GetDataBuilderImpl.java:142)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl$2.forPath(GetDataBuilderImpl.java:138)
>   at 
> org.apache.curator.framework.recipes.cache.PathChildrenCache.internalRebuildNode(PathChildrenCache.java:591)
>   at 
> org.apache.curator.framework.recipes.cache.PathChildrenCache.rebuild(PathChildrenCache.java:331)
>   at 
> org.apache.curator.framework.recipes.cache.PathChildrenCache.start(PathChildrenCache.java:300)
>   at 
> org.apache.hadoop.security.token.delegation.ZKDelegationTokenSecretManager.startThreads(ZKDelegationTokenSecretManager.java:370)
>   at 
> org.apache.hadoop.hive.llap.security.SecretManager.startThreads(SecretManager.java:82)
>   at 
> org.apache.hadoop.hive.llap.security.SecretManager$1.run(SecretManager.java:223)
>   at 
> org.apache.hadoop.hive.llap.security.SecretManager$1.run(SecretManager.java:218)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:360)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1846)
>   at 
> org.apache.hadoop.hive.llap.security.SecretManager.createSecretManager(SecretManager.java:218)
>   at 
> org.apache.hadoop.hive.llap.security.SecretManager.createSecretManager(SecretManager.java:212)
>   at 
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.(LlapDaemon.java:279)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-23753) Make LLAP Secretmanager token path configurable

2020-06-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-23753:
-


> Make LLAP Secretmanager token path configurable
> ---
>
> Key: HIVE-23753
> URL: https://issues.apache.org/jira/browse/HIVE-23753
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> In a very Busy LLAP cluster if for some reason the Tokens under 
> zkdtsm_hive_llap0 zk path are not cleaned then LLAP Daemon startup takes a 
> very long time to startup, this may lead to service outage if LLAP daemons 
> are not started and the number of retries while checking LLAP app status 
> exceeds. upon looking the jstack of llap daemon it seems to traverse the 
> zkdtsm_hive_llap0 zk path before starting the secret manager.
> {code:java}
>java.lang.Thread.State: WAITING (on object monitor)
>   at java.lang.Object.wait(Native Method)
>   at java.lang.Object.wait(Object.java:502)
>   at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1386)
>   - locked <0x7fef36cdd338> (a org.apache.zookeeper.ClientCnxn$Packet)
>   at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1153)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:302)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:291)
>   at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl$2.forPath(GetDataBuilderImpl.java:142)
>   at 
> org.apache.curator.framework.imps.GetDataBuilderImpl$2.forPath(GetDataBuilderImpl.java:138)
>   at 
> org.apache.curator.framework.recipes.cache.PathChildrenCache.internalRebuildNode(PathChildrenCache.java:591)
>   at 
> org.apache.curator.framework.recipes.cache.PathChildrenCache.rebuild(PathChildrenCache.java:331)
>   at 
> org.apache.curator.framework.recipes.cache.PathChildrenCache.start(PathChildrenCache.java:300)
>   at 
> org.apache.hadoop.security.token.delegation.ZKDelegationTokenSecretManager.startThreads(ZKDelegationTokenSecretManager.java:370)
>   at 
> org.apache.hadoop.hive.llap.security.SecretManager.startThreads(SecretManager.java:82)
>   at 
> org.apache.hadoop.hive.llap.security.SecretManager$1.run(SecretManager.java:223)
>   at 
> org.apache.hadoop.hive.llap.security.SecretManager$1.run(SecretManager.java:218)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:360)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1846)
>   at 
> org.apache.hadoop.hive.llap.security.SecretManager.createSecretManager(SecretManager.java:218)
>   at 
> org.apache.hadoop.hive.llap.security.SecretManager.createSecretManager(SecretManager.java:212)
>   at 
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.(LlapDaemon.java:279)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23752) Cast as Date for invalid date produce the valid output

2020-06-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23752:
--
Labels: hive  (was: )

> Cast as Date for invalid date produce the valid output
> --
>
> Key: HIVE-23752
> URL: https://issues.apache.org/jira/browse/HIVE-23752
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Priority: Major
>  Labels: hive
>
> Hive-3:
> {code:java}
> select cast("-00-00" as date) 
> 0002-11-30 
> select cast("2010-27-54" as date)
>  2012-04-23
> select cast("1992-00-74" as date) ;
> 1992-02-12
> {code}
> The reason Hive allowing is because Parser formatted is set to LENIENT 
> https://github.com/apache/hive/blob/ae008b79b5d52ed6a38875b73025a505725828eb/common/src/java/org/apache/hadoop/hive/common/type/Date.java#L50,
>  this seems to be an intentional change as changing the ResolverStyle to 
> STRICT start failing the tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23752) Cast as Date for invalid date produce the valid output

2020-06-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23752:
--
Affects Version/s: 4.0.0

> Cast as Date for invalid date produce the valid output
> --
>
> Key: HIVE-23752
> URL: https://issues.apache.org/jira/browse/HIVE-23752
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Priority: Major
>
> Hive-3:
> {code:java}
> select cast("-00-00" as date) 
> 0002-11-30 
> select cast("2010-27-54" as date)
>  2012-04-23
> select cast("1992-00-74" as date) ;
> 1992-02-12
> {code}
> The reason Hive allowing is because Parser formatted is set to LENIENT 
> https://github.com/apache/hive/blob/ae008b79b5d52ed6a38875b73025a505725828eb/common/src/java/org/apache/hadoop/hive/common/type/Date.java#L50,
>  this seems to be an intentional change as changing the ResolverStyle to 
> STRICT start failing the tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23675) WebHcat: java level deadlock in hcat in presence of InMemoryJAAS

2020-06-10 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23675:
--
Issue Type: Bug  (was: Improvement)

> WebHcat: java level deadlock in hcat in presence of InMemoryJAAS
> 
>
> Key: HIVE-23675
> URL: https://issues.apache.org/jira/browse/HIVE-23675
> Project: Hive
>  Issue Type: Bug
>  Components: HCatalog
>Affects Versions: 1.3.0
>Reporter: Rajkumar Singh
>Priority: Major
>
> ENV: Keberos/SPNEGO enabled
> set hive.exec.post.hook;
> org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.atlas.hive.hook.HiveHook
> ATLAS Hook use InMemoryJAASConfiguration
> This is a sequence of the event while issue reproduces:
> WebHcat -> hcat -> Hive Driver -> post hook execution create ATSHook  -> hook 
> start the spnego auth and stuck while finding InMemoryJAASConfiguration used 
> by the AtlasHook (this happens in separate thread ATS Logger)
> Hcat jstack
> {code:java}
> Found one Java-level deadlock:
>  =
>  "ATS Logger 0":
>waiting to lock monitor 0x7efdc8003a38 (object 0xf3fcfe28, a 
> org.apache.atlas.plugin.classloader.AtlasPluginClassLoader),
>which is held by "main"
>  "main":
>waiting to lock monitor 0x7efdc8003da8 (object 0xc0050d40, a 
> org.apache.hadoop.hive.ql.exec.UDFClassLoader),
>which is held by "ATS Logger 0"
> Java stack information for the threads listed above:
>  ===
>  "ATS Logger 0":
>  at 
> org.apache.atlas.security.InMemoryJAASConfiguration.getAppConfigurationEntry(InMemoryJAASConfiguration.java:238)
>  at 
> sun.security.jgss.LoginConfigImpl.getAppConfigurationEntry(LoginConfigImpl.java:145)
>  at javax.security.auth.login.LoginContext.init(LoginContext.java:251)
>  at javax.security.auth.login.LoginContext.(LoginContext.java:512)
>  at sun.security.jgss.GSSUtil.login(GSSUtil.java:256)
>  at sun.security.jgss.krb5.Krb5Util.getTicket(Krb5Util.java:158)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:335)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:331)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential.getTgt(Krb5InitCredential.java:330)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:145)
>  at 
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
>  at 
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
>  at 
> sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>  at 
> sun.security.jgss.spnego.SpNegoContext.GSS_initSecContext(SpNegoContext.java:882)
>  at 
> sun.security.jgss.spnego.SpNegoContext.initSecContext(SpNegoContext.java:317)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>  at 
> sun.net.www.protocol.http.spnego.NegotiatorImpl.init(NegotiatorImpl.java:108)
>  at 
> sun.net.www.protocol.http.spnego.NegotiatorImpl.(NegotiatorImpl.java:117)
>  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>  at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>  at sun.net.www.protocol.http.Negotiator.getNegotiator(Negotiator.java:63)
>  at 
> sun.net.www.protocol.http.NegotiateAuthentication.isSupportedImpl(NegotiateAuthentication.java:130)
>  - locked <0xf48c4d90> (a java.lang.Class for 
> sun.net.www.protocol.http.NegotiateAuthentication)
>  at 
> sun.net.www.protocol.http.NegotiateAuthentication.isSupported(NegotiateAuthentication.java:102)
>  - locked <0xc0050d40> (a 
> org.apache.hadoop.hive.ql.exec.UDFClassLoader)
>  at 
> sun.net.www.protocol.http.AuthenticationHeader.parse(AuthenticationHeader.java:180)
>  at 
> sun.net.www.protocol.http.AuthenticationHeader.(AuthenticationHeader.java:126)
>  at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1660)
>  - locked <0xf47b7298> (a 
> sun.net.www.protocol.https.DelegateHttpsURLConnection)
>  at 
>

[jira] [Updated] (HIVE-23675) WebHcat: java level deadlock in hcat in presence of InMemoryJAAS

2020-06-10 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23675:
--
Component/s: WebHCat

> WebHcat: java level deadlock in hcat in presence of InMemoryJAAS
> 
>
> Key: HIVE-23675
> URL: https://issues.apache.org/jira/browse/HIVE-23675
> Project: Hive
>  Issue Type: Bug
>  Components: HCatalog, WebHCat
>Affects Versions: 1.3.0
>Reporter: Rajkumar Singh
>Priority: Major
>
> ENV: Keberos/SPNEGO enabled
> set hive.exec.post.hook;
> org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.atlas.hive.hook.HiveHook
> ATLAS Hook use InMemoryJAASConfiguration
> This is a sequence of the event while issue reproduces:
> WebHcat -> hcat -> Hive Driver -> post hook execution create ATSHook  -> hook 
> start the spnego auth and stuck while finding InMemoryJAASConfiguration used 
> by the AtlasHook (this happens in separate thread ATS Logger)
> Hcat jstack
> {code:java}
> Found one Java-level deadlock:
>  =
>  "ATS Logger 0":
>waiting to lock monitor 0x7efdc8003a38 (object 0xf3fcfe28, a 
> org.apache.atlas.plugin.classloader.AtlasPluginClassLoader),
>which is held by "main"
>  "main":
>waiting to lock monitor 0x7efdc8003da8 (object 0xc0050d40, a 
> org.apache.hadoop.hive.ql.exec.UDFClassLoader),
>which is held by "ATS Logger 0"
> Java stack information for the threads listed above:
>  ===
>  "ATS Logger 0":
>  at 
> org.apache.atlas.security.InMemoryJAASConfiguration.getAppConfigurationEntry(InMemoryJAASConfiguration.java:238)
>  at 
> sun.security.jgss.LoginConfigImpl.getAppConfigurationEntry(LoginConfigImpl.java:145)
>  at javax.security.auth.login.LoginContext.init(LoginContext.java:251)
>  at javax.security.auth.login.LoginContext.(LoginContext.java:512)
>  at sun.security.jgss.GSSUtil.login(GSSUtil.java:256)
>  at sun.security.jgss.krb5.Krb5Util.getTicket(Krb5Util.java:158)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:335)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:331)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential.getTgt(Krb5InitCredential.java:330)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:145)
>  at 
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
>  at 
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
>  at 
> sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>  at 
> sun.security.jgss.spnego.SpNegoContext.GSS_initSecContext(SpNegoContext.java:882)
>  at 
> sun.security.jgss.spnego.SpNegoContext.initSecContext(SpNegoContext.java:317)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>  at 
> sun.net.www.protocol.http.spnego.NegotiatorImpl.init(NegotiatorImpl.java:108)
>  at 
> sun.net.www.protocol.http.spnego.NegotiatorImpl.(NegotiatorImpl.java:117)
>  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>  at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>  at sun.net.www.protocol.http.Negotiator.getNegotiator(Negotiator.java:63)
>  at 
> sun.net.www.protocol.http.NegotiateAuthentication.isSupportedImpl(NegotiateAuthentication.java:130)
>  - locked <0xf48c4d90> (a java.lang.Class for 
> sun.net.www.protocol.http.NegotiateAuthentication)
>  at 
> sun.net.www.protocol.http.NegotiateAuthentication.isSupported(NegotiateAuthentication.java:102)
>  - locked <0xc0050d40> (a 
> org.apache.hadoop.hive.ql.exec.UDFClassLoader)
>  at 
> sun.net.www.protocol.http.AuthenticationHeader.parse(AuthenticationHeader.java:180)
>  at 
> sun.net.www.protocol.http.AuthenticationHeader.(AuthenticationHeader.java:126)
>  at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1660)
>  - locked <0xf47b7298> (a 
> sun.net.www.protocol.https.DelegateHttpsURLConnection)
>  at 
>

[jira] [Updated] (HIVE-23675) WebHcat: java level deadlock in hcat in presence of InMemoryJAAS

2020-06-10 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23675:
--
Affects Version/s: 1.3.0

> WebHcat: java level deadlock in hcat in presence of InMemoryJAAS
> 
>
> Key: HIVE-23675
> URL: https://issues.apache.org/jira/browse/HIVE-23675
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 1.3.0
>Reporter: Rajkumar Singh
>Priority: Major
>
> ENV: Keberos/SPNEGO enabled
> set hive.exec.post.hook;
> org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.atlas.hive.hook.HiveHook
> ATLAS Hook use InMemoryJAASConfiguration
> This is a sequence of the event while issue reproduces:
> WebHcat -> hcat -> Hive Driver -> post hook execution create ATSHook  -> hook 
> start the spnego auth and stuck while finding InMemoryJAASConfiguration used 
> by the AtlasHook (this happens in separate thread ATS Logger)
> Hcat jstack
> {code:java}
> Found one Java-level deadlock:
>  =
>  "ATS Logger 0":
>waiting to lock monitor 0x7efdc8003a38 (object 0xf3fcfe28, a 
> org.apache.atlas.plugin.classloader.AtlasPluginClassLoader),
>which is held by "main"
>  "main":
>waiting to lock monitor 0x7efdc8003da8 (object 0xc0050d40, a 
> org.apache.hadoop.hive.ql.exec.UDFClassLoader),
>which is held by "ATS Logger 0"
> Java stack information for the threads listed above:
>  ===
>  "ATS Logger 0":
>  at 
> org.apache.atlas.security.InMemoryJAASConfiguration.getAppConfigurationEntry(InMemoryJAASConfiguration.java:238)
>  at 
> sun.security.jgss.LoginConfigImpl.getAppConfigurationEntry(LoginConfigImpl.java:145)
>  at javax.security.auth.login.LoginContext.init(LoginContext.java:251)
>  at javax.security.auth.login.LoginContext.(LoginContext.java:512)
>  at sun.security.jgss.GSSUtil.login(GSSUtil.java:256)
>  at sun.security.jgss.krb5.Krb5Util.getTicket(Krb5Util.java:158)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:335)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:331)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential.getTgt(Krb5InitCredential.java:330)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:145)
>  at 
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
>  at 
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
>  at 
> sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>  at 
> sun.security.jgss.spnego.SpNegoContext.GSS_initSecContext(SpNegoContext.java:882)
>  at 
> sun.security.jgss.spnego.SpNegoContext.initSecContext(SpNegoContext.java:317)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>  at 
> sun.net.www.protocol.http.spnego.NegotiatorImpl.init(NegotiatorImpl.java:108)
>  at 
> sun.net.www.protocol.http.spnego.NegotiatorImpl.(NegotiatorImpl.java:117)
>  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>  at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>  at sun.net.www.protocol.http.Negotiator.getNegotiator(Negotiator.java:63)
>  at 
> sun.net.www.protocol.http.NegotiateAuthentication.isSupportedImpl(NegotiateAuthentication.java:130)
>  - locked <0xf48c4d90> (a java.lang.Class for 
> sun.net.www.protocol.http.NegotiateAuthentication)
>  at 
> sun.net.www.protocol.http.NegotiateAuthentication.isSupported(NegotiateAuthentication.java:102)
>  - locked <0xc0050d40> (a 
> org.apache.hadoop.hive.ql.exec.UDFClassLoader)
>  at 
> sun.net.www.protocol.http.AuthenticationHeader.parse(AuthenticationHeader.java:180)
>  at 
> sun.net.www.protocol.http.AuthenticationHeader.(AuthenticationHeader.java:126)
>  at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1660)
>  - locked <0xf47b7298> (a 
> sun.net.www.protocol.https.DelegateHttpsURLConnection)
>  at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
>

[jira] [Updated] (HIVE-23675) WebHcat: java level deadlock in hcat in presence of InMemoryJAAS

2020-06-10 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23675:
--
Component/s: HCatalog

> WebHcat: java level deadlock in hcat in presence of InMemoryJAAS
> 
>
> Key: HIVE-23675
> URL: https://issues.apache.org/jira/browse/HIVE-23675
> Project: Hive
>  Issue Type: Improvement
>  Components: HCatalog
>Affects Versions: 1.3.0
>Reporter: Rajkumar Singh
>Priority: Major
>
> ENV: Keberos/SPNEGO enabled
> set hive.exec.post.hook;
> org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.atlas.hive.hook.HiveHook
> ATLAS Hook use InMemoryJAASConfiguration
> This is a sequence of the event while issue reproduces:
> WebHcat -> hcat -> Hive Driver -> post hook execution create ATSHook  -> hook 
> start the spnego auth and stuck while finding InMemoryJAASConfiguration used 
> by the AtlasHook (this happens in separate thread ATS Logger)
> Hcat jstack
> {code:java}
> Found one Java-level deadlock:
>  =
>  "ATS Logger 0":
>waiting to lock monitor 0x7efdc8003a38 (object 0xf3fcfe28, a 
> org.apache.atlas.plugin.classloader.AtlasPluginClassLoader),
>which is held by "main"
>  "main":
>waiting to lock monitor 0x7efdc8003da8 (object 0xc0050d40, a 
> org.apache.hadoop.hive.ql.exec.UDFClassLoader),
>which is held by "ATS Logger 0"
> Java stack information for the threads listed above:
>  ===
>  "ATS Logger 0":
>  at 
> org.apache.atlas.security.InMemoryJAASConfiguration.getAppConfigurationEntry(InMemoryJAASConfiguration.java:238)
>  at 
> sun.security.jgss.LoginConfigImpl.getAppConfigurationEntry(LoginConfigImpl.java:145)
>  at javax.security.auth.login.LoginContext.init(LoginContext.java:251)
>  at javax.security.auth.login.LoginContext.(LoginContext.java:512)
>  at sun.security.jgss.GSSUtil.login(GSSUtil.java:256)
>  at sun.security.jgss.krb5.Krb5Util.getTicket(Krb5Util.java:158)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:335)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:331)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential.getTgt(Krb5InitCredential.java:330)
>  at 
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:145)
>  at 
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
>  at 
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
>  at 
> sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>  at 
> sun.security.jgss.spnego.SpNegoContext.GSS_initSecContext(SpNegoContext.java:882)
>  at 
> sun.security.jgss.spnego.SpNegoContext.initSecContext(SpNegoContext.java:317)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
>  at 
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
>  at 
> sun.net.www.protocol.http.spnego.NegotiatorImpl.init(NegotiatorImpl.java:108)
>  at 
> sun.net.www.protocol.http.spnego.NegotiatorImpl.(NegotiatorImpl.java:117)
>  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>  at 
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>  at sun.net.www.protocol.http.Negotiator.getNegotiator(Negotiator.java:63)
>  at 
> sun.net.www.protocol.http.NegotiateAuthentication.isSupportedImpl(NegotiateAuthentication.java:130)
>  - locked <0xf48c4d90> (a java.lang.Class for 
> sun.net.www.protocol.http.NegotiateAuthentication)
>  at 
> sun.net.www.protocol.http.NegotiateAuthentication.isSupported(NegotiateAuthentication.java:102)
>  - locked <0xc0050d40> (a 
> org.apache.hadoop.hive.ql.exec.UDFClassLoader)
>  at 
> sun.net.www.protocol.http.AuthenticationHeader.parse(AuthenticationHeader.java:180)
>  at 
> sun.net.www.protocol.http.AuthenticationHeader.(AuthenticationHeader.java:126)
>  at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1660)
>  - locked <0xf47b7298> (a 
> sun.net.www.protocol.https.DelegateHttpsURLConnection)
>  at 
>

[jira] [Updated] (HIVE-22255) Hive don't trigger Major Compaction automatically if table contains only base files

2020-06-08 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22255:
--
Status: Open  (was: Patch Available)

> Hive don't trigger Major Compaction automatically if table contains only base 
> files 
> 
>
> Key: HIVE-22255
> URL: https://issues.apache.org/jira/browse/HIVE-22255
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, Transactions
>Affects Versions: 3.1.2
> Environment: Hive-3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22255.01.patch, HIVE-22255.02.patch, 
> HIVE-22255.patch
>
>
> user may run into the issue if the table consists of all base files but no 
> delta, then the following condition will yield false and automatic major 
> compaction will be skipped.
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java#L313]
>  
> Steps to Reproduce:
>  # create Acid table 
> {code:java}
> //  create table myacid(id int);
> {code}
>  # Run multiple insert table 
> {code:java}
> // insert overwrite table myacid values(1);insert overwrite table myacid 
> values(2),(3),(4){code}
>  # DFS ls output
> {code:java}
> // dfs -ls -R /warehouse/tablespace/managed/hive/myacid;
> ++
> |                     DFS Output                     |
> ++
> | drwxrwx---+  - hive hadoop          0 2019-09-27 16:42 
> /warehouse/tablespace/managed/hive/myacid/base_001 |
> | -rw-rw+  3 hive hadoop          1 2019-09-27 16:42 
> /warehouse/tablespace/managed/hive/myacid/base_001/_orc_acid_version |
> | -rw-rw+  3 hive hadoop        610 2019-09-27 16:42 
> /warehouse/tablespace/managed/hive/myacid/base_001/bucket_0 |
> | drwxrwx---+  - hive hadoop          0 2019-09-27 16:43 
> /warehouse/tablespace/managed/hive/myacid/base_002 |
> | -rw-rw+  3 hive hadoop          1 2019-09-27 16:43 
> /warehouse/tablespace/managed/hive/myacid/base_002/_orc_acid_version |
> | -rw-rw+  3 hive hadoop        633 2019-09-27 16:43 
> /warehouse/tablespace/managed/hive/myacid/base_002/bucket_0 |
> ++{code}
>  
> you will see that Major compaction will not be trigger until you run alter 
> table compact MAJOR.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22255) Hive don't trigger Major Compaction automatically if table contains only base files

2020-06-08 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22255:
--
Attachment: HIVE-22255.02.patch
Status: Patch Available  (was: Open)

> Hive don't trigger Major Compaction automatically if table contains only base 
> files 
> 
>
> Key: HIVE-22255
> URL: https://issues.apache.org/jira/browse/HIVE-22255
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, Transactions
>Affects Versions: 3.1.2
> Environment: Hive-3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22255.01.patch, HIVE-22255.02.patch, 
> HIVE-22255.patch
>
>
> user may run into the issue if the table consists of all base files but no 
> delta, then the following condition will yield false and automatic major 
> compaction will be skipped.
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java#L313]
>  
> Steps to Reproduce:
>  # create Acid table 
> {code:java}
> //  create table myacid(id int);
> {code}
>  # Run multiple insert table 
> {code:java}
> // insert overwrite table myacid values(1);insert overwrite table myacid 
> values(2),(3),(4){code}
>  # DFS ls output
> {code:java}
> // dfs -ls -R /warehouse/tablespace/managed/hive/myacid;
> ++
> |                     DFS Output                     |
> ++
> | drwxrwx---+  - hive hadoop          0 2019-09-27 16:42 
> /warehouse/tablespace/managed/hive/myacid/base_001 |
> | -rw-rw+  3 hive hadoop          1 2019-09-27 16:42 
> /warehouse/tablespace/managed/hive/myacid/base_001/_orc_acid_version |
> | -rw-rw+  3 hive hadoop        610 2019-09-27 16:42 
> /warehouse/tablespace/managed/hive/myacid/base_001/bucket_0 |
> | drwxrwx---+  - hive hadoop          0 2019-09-27 16:43 
> /warehouse/tablespace/managed/hive/myacid/base_002 |
> | -rw-rw+  3 hive hadoop          1 2019-09-27 16:43 
> /warehouse/tablespace/managed/hive/myacid/base_002/_orc_acid_version |
> | -rw-rw+  3 hive hadoop        633 2019-09-27 16:43 
> /warehouse/tablespace/managed/hive/myacid/base_002/bucket_0 |
> ++{code}
>  
> you will see that Major compaction will not be trigger until you run alter 
> table compact MAJOR.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23439) Hive sessions over 24 hours encounter Kerberos-related StatsTask errors

2020-06-01 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23439:
--
Attachment: HIVE-23439.patch
Status: Patch Available  (was: Open)

> Hive sessions over 24 hours encounter Kerberos-related StatsTask errors
> ---
>
> Key: HIVE-23439
> URL: https://issues.apache.org/jira/browse/HIVE-23439
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Standalone Metastore
>Affects Versions: 3.1.0
>Reporter: Chiran Ravani
>Assignee: Rajkumar Singh
>Priority: Critical
> Attachments: HIVE-23439.patch
>
>
> We have an application that uses Hive via JDBC. The interesting thing about 
> them is that they have sessions that are established with HiveServer2 for 
> multiple days. After 24 hours, their queries are failing with 
> StatsTask-related errors. From looking in the logs, it looks like the 
> communication breaks down between HiveServer2 and the MetaStore.
> Below is error seen:
> {code}
> 2020-04-22T21:25:53,248 ERROR [Thread-1202599]: exec.StatsTask (:()) - Failed 
> to run stats task
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch table 
> tennis. Unable to instantiate 
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
>  at 
> org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:4927)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.ql.stats.ColStatsProcessor.persistColumnStats(ColStatsProcessor.java:189)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.ql.stats.ColStatsProcessor.process(ColStatsProcessor.java:86)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.exec.StatsTask.execute(StatsTask.java:108) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:82) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch 
> table tennis. Unable to instantiate 
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1387) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1336) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1316) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1298) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:4918)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  ... 6 more
> Caused by: java.lang.RuntimeException: Unable to instantiate 
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
>  at 
> org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:95)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:4790) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4858) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4838) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1378) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1336) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1316) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1298) 
>

[jira] [Assigned] (HIVE-23439) Hive sessions over 24 hours encounter Kerberos-related StatsTask errors

2020-06-01 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-23439:
-

Assignee: Rajkumar Singh

> Hive sessions over 24 hours encounter Kerberos-related StatsTask errors
> ---
>
> Key: HIVE-23439
> URL: https://issues.apache.org/jira/browse/HIVE-23439
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2, Standalone Metastore
>Affects Versions: 3.1.0
>Reporter: Chiran Ravani
>Assignee: Rajkumar Singh
>Priority: Critical
>
> We have an application that uses Hive via JDBC. The interesting thing about 
> them is that they have sessions that are established with HiveServer2 for 
> multiple days. After 24 hours, their queries are failing with 
> StatsTask-related errors. From looking in the logs, it looks like the 
> communication breaks down between HiveServer2 and the MetaStore.
> Below is error seen:
> {code}
> 2020-04-22T21:25:53,248 ERROR [Thread-1202599]: exec.StatsTask (:()) - Failed 
> to run stats task
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch table 
> tennis. Unable to instantiate 
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
>  at 
> org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:4927)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.ql.stats.ColStatsProcessor.persistColumnStats(ColStatsProcessor.java:189)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.ql.stats.ColStatsProcessor.process(ColStatsProcessor.java:86)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.exec.StatsTask.execute(StatsTask.java:108) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:82) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch 
> table tennis. Unable to instantiate 
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1387) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1336) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1316) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1298) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.ql.metadata.Hive.setPartitionColumnStatistics(Hive.java:4918)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  ... 6 more
> Caused by: java.lang.RuntimeException: Unable to instantiate 
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
>  at 
> org.apache.hadoop.hive.metastore.utils.JavaUtils.newInstance(JavaUtils.java:86)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:95)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:148)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:119)
>  ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
> org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:4790) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4858) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:4838) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1378) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1336) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1316) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1298) 
> ~[hive-exec-3.1.0.3.1.4.39-6.jar:3.1.0.3.1.4.39-6]
>  at 
>

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-06-01 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Fix Version/s: 4.0.0
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, 
> HIVE-23498.02.patch, HIVE-23498.03.patch, HIVE-23498.04.patch, 
> HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-06-01 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17121519#comment-17121519
 ] 

Rajkumar Singh commented on HIVE-23498:
---

Thanks [~ngangam] for review, pushed the patch to master.

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, 
> HIVE-23498.02.patch, HIVE-23498.03.patch, HIVE-23498.04.patch, 
> HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-31 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Attachment: HIVE-23498.04.patch
Status: Patch Available  (was: Open)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, 
> HIVE-23498.02.patch, HIVE-23498.03.patch, HIVE-23498.04.patch, 
> HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-31 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Status: Open  (was: Patch Available)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, 
> HIVE-23498.02.patch, HIVE-23498.03.patch, HIVE-23498.04.patch, 
> HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-31 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120632#comment-17120632
 ] 

Rajkumar Singh commented on HIVE-23498:
---

Tests failure are unrelated but uploading the fresh patch for a clean run.

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, 
> HIVE-23498.02.patch, HIVE-23498.03.patch, HIVE-23498.04.patch, 
> HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-30 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Attachment: HIVE-23498.03.patch
Status: Patch Available  (was: Open)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, 
> HIVE-23498.02.patch, HIVE-23498.03.patch, HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-30 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Status: Open  (was: Patch Available)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, 
> HIVE-23498.02.patch, HIVE-23498.03.patch, HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23542) Query based compaction failing with ClassCastException

2020-05-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23542:
--
Attachment: HIVE-23542.01.patch
Status: Patch Available  (was: Open)

> Query based compaction failing with ClassCastException 
> ---
>
> Key: HIVE-23542
> URL: https://issues.apache.org/jira/browse/HIVE-23542
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23542.01.patch, HIVE-23542.patch
>
>
> Steps to repro:
> create table test(id int);
> insert into table test1 values (1),(2),(3);
> insert into table test1 values (4),(5),(6);
> run query-based compactor and it will fail with the following exception:-
> {code:java}
> alter table test compact 'major'; -- query based compaction 
> Caused by: java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
> processing row
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:101)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
>   ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime 
> Error while processing row
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
>   ... 19 more
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to 
> org.apache.hadoop.io.IntWritable
>   at 
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:967)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:938)
>   at 
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:938)
>   at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23542) Query based compaction failing with ClassCastException

2020-05-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23542:
--
Status: Open  (was: Patch Available)

> Query based compaction failing with ClassCastException 
> ---
>
> Key: HIVE-23542
> URL: https://issues.apache.org/jira/browse/HIVE-23542
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23542.patch
>
>
> Steps to repro:
> create table test(id int);
> insert into table test1 values (1),(2),(3);
> insert into table test1 values (4),(5),(6);
> run query-based compactor and it will fail with the following exception:-
> {code:java}
> alter table test compact 'major'; -- query based compaction 
> Caused by: java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
> processing row
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:101)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
>   ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime 
> Error while processing row
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
>   ... 19 more
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to 
> org.apache.hadoop.io.IntWritable
>   at 
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:967)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:938)
>   at 
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:938)
>   at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23542) Query based compaction failing with ClassCastException

2020-05-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23542:
--
Issue Type: Bug  (was: Improvement)

> Query based compaction failing with ClassCastException 
> ---
>
> Key: HIVE-23542
> URL: https://issues.apache.org/jira/browse/HIVE-23542
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23542.patch
>
>
> Steps to repro:
> create table test(id int);
> insert into table test1 values (1),(2),(3);
> insert into table test1 values (4),(5),(6);
> run query-based compactor and it will fail with the following exception:-
> {code:java}
> alter table test compact 'major'; -- query based compaction 
> Caused by: java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
> processing row
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:101)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
>   ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime 
> Error while processing row
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
>   ... 19 more
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to 
> org.apache.hadoop.io.IntWritable
>   at 
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:967)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:938)
>   at 
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:938)
>   at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23542) Query based compaction failing with ClassCastException

2020-05-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23542:
--
Attachment: HIVE-23542.patch
Status: Patch Available  (was: Open)

> Query based compaction failing with ClassCastException 
> ---
>
> Key: HIVE-23542
> URL: https://issues.apache.org/jira/browse/HIVE-23542
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23542.patch
>
>
> Steps to repro:
> create table test(id int);
> insert into table test1 values (1),(2),(3);
> insert into table test1 values (4),(5),(6);
> run query-based compactor and it will fail with the following exception:-
> {code:java}
> alter table test compact 'major'; -- query based compaction 
> Caused by: java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
> processing row
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:101)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
>   ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime 
> Error while processing row
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
>   ... 19 more
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to 
> org.apache.hadoop.io.IntWritable
>   at 
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:967)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:938)
>   at 
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:938)
>   at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-23542) Query based compaction failing with ClassCastException

2020-05-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-23542:
-


> Query based compaction failing with ClassCastException 
> ---
>
> Key: HIVE-23542
> URL: https://issues.apache.org/jira/browse/HIVE-23542
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> Steps to repro:
> create table test(id int);
> insert into table test1 values (1),(2),(3);
> insert into table test1 values (4),(5),(6);
> run query-based compactor and it will fail with the following exception:-
> {code:java}
> alter table test compact 'major'; -- query based compaction 
> Caused by: java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while 
> processing row
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:101)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:76)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
>   ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime 
> Error while processing row
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:573)
>   at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
>   ... 19 more
> Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to 
> org.apache.hadoop.io.IntWritable
>   at 
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:967)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:938)
>   at 
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:938)
>   at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:128)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:152)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:552)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Status: Open  (was: Patch Available)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, 
> HIVE-23498.02.patch, HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Attachment: HIVE-23498.02.patch
Status: Patch Available  (was: Open)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, 
> HIVE-23498.02.patch, HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-20 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Attachment: HIVE-23498.01.patch
Status: Patch Available  (was: Open)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, 
> HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-20 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Status: Open  (was: Patch Available)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.01.patch, 
> HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-19 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Attachment: HIVE-23498.01.patch
Status: Patch Available  (was: Open)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-19 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Status: Open  (was: Patch Available)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.01.patch, HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23512) ReplDumpTask: Adding debug log to print opentxn for debugging opentxn issue blocking repldump task

2020-05-19 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23512:
--
Summary: ReplDumpTask: Adding debug log to print opentxn for debugging 
opentxn issue blocking repldump task  (was: ReplDumpTask: Adding debug to print 
opentxn for debugging perspective)

> ReplDumpTask: Adding debug log to print opentxn for debugging opentxn issue 
> blocking repldump task
> --
>
> Key: HIVE-23512
> URL: https://issues.apache.org/jira/browse/HIVE-23512
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23512.patch
>
>
> Often time we see that ReplDumpTask waiting for 
> hive.repl.bootstrap.dump.open.txn.timeout  (1h) to kill open txns and make 
> progress, the only way to know for what txns it is waiting on is query the 
> Metastore DB and backtrack the txns in HS2 logs to know if open txns are 
> genuinely open for this long or any other issue.
> I am adding the debug log to print these txns which can help in debugging 
> such issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23512) ReplDumpTask: Adding debug to print opentxn for debugging perspective

2020-05-19 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23512:
--
Attachment: HIVE-23512.patch
Status: Patch Available  (was: Open)

> ReplDumpTask: Adding debug to print opentxn for debugging perspective
> -
>
> Key: HIVE-23512
> URL: https://issues.apache.org/jira/browse/HIVE-23512
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23512.patch
>
>
> Often time we see that ReplDumpTask waiting for 
> hive.repl.bootstrap.dump.open.txn.timeout  (1h) to kill open txns and make 
> progress, the only way to know for what txns it is waiting on is query the 
> Metastore DB and backtrack the txns in HS2 logs to know if open txns are 
> genuinely open for this long or any other issue.
> I am adding the debug log to print these txns which can help in debugging 
> such issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23512) ReplDumpTask: Adding debug to print opentxn for debugging perspective

2020-05-19 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23512:
--
Attachment: (was: HIVE-23512.patch)

> ReplDumpTask: Adding debug to print opentxn for debugging perspective
> -
>
> Key: HIVE-23512
> URL: https://issues.apache.org/jira/browse/HIVE-23512
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23512.patch
>
>
> Often time we see that ReplDumpTask waiting for 
> hive.repl.bootstrap.dump.open.txn.timeout  (1h) to kill open txns and make 
> progress, the only way to know for what txns it is waiting on is query the 
> Metastore DB and backtrack the txns in HS2 logs to know if open txns are 
> genuinely open for this long or any other issue.
> I am adding the debug log to print these txns which can help in debugging 
> such issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23512) ReplDumpTask: Adding debug to print opentxn for debugging perspective

2020-05-19 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23512:
--
Attachment: HIVE-23512.patch
Status: Patch Available  (was: Open)

> ReplDumpTask: Adding debug to print opentxn for debugging perspective
> -
>
> Key: HIVE-23512
> URL: https://issues.apache.org/jira/browse/HIVE-23512
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23512.patch
>
>
> Often time we see that ReplDumpTask waiting for 
> hive.repl.bootstrap.dump.open.txn.timeout  (1h) to kill open txns and make 
> progress, the only way to know for what txns it is waiting on is query the 
> Metastore DB and backtrack the txns in HS2 logs to know if open txns are 
> genuinely open for this long or any other issue.
> I am adding the debug log to print these txns which can help in debugging 
> such issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23512) ReplDumpTask: Adding debug to print opentxn for debugging perspective

2020-05-19 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23512:
--
Status: Open  (was: Patch Available)

> ReplDumpTask: Adding debug to print opentxn for debugging perspective
> -
>
> Key: HIVE-23512
> URL: https://issues.apache.org/jira/browse/HIVE-23512
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23512.patch
>
>
> Often time we see that ReplDumpTask waiting for 
> hive.repl.bootstrap.dump.open.txn.timeout  (1h) to kill open txns and make 
> progress, the only way to know for what txns it is waiting on is query the 
> Metastore DB and backtrack the txns in HS2 logs to know if open txns are 
> genuinely open for this long or any other issue.
> I am adding the debug log to print these txns which can help in debugging 
> such issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-23512) ReplDumpTask: Adding debug to print opentxn for debugging perspective

2020-05-19 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-23512:
-


> ReplDumpTask: Adding debug to print opentxn for debugging perspective
> -
>
> Key: HIVE-23512
> URL: https://issues.apache.org/jira/browse/HIVE-23512
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> Often time we see that ReplDumpTask waiting for 
> hive.repl.bootstrap.dump.open.txn.timeout  (1h) to kill open txns and make 
> progress, the only way to know for what txns it is waiting on is query the 
> Metastore DB and backtrack the txns in HS2 logs to know if open txns are 
> genuinely open for this long or any other issue.
> I am adding the debug log to print these txns which can help in debugging 
> such issues.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-18 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Summary: Disable HTTP Trace method on ThriftHttpCliService  (was: Disable 
HTTP Trace method on ThriftServer)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftServer

2020-05-18 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Status: Open  (was: Patch Available)

> Disable HTTP Trace method on ThriftServer
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftHttpCliService

2020-05-18 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Attachment: HIVE-23498.patch
Status: Patch Available  (was: Open)

> Disable HTTP Trace method on ThriftHttpCliService
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-23498.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23498) Disable HTTP Trace method on ThriftServer

2020-05-18 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23498:
--
Status: Patch Available  (was: Open)

> Disable HTTP Trace method on ThriftServer
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-23498) Disable HTTP Trace method on ThriftServer

2020-05-18 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-23498:
-


> Disable HTTP Trace method on ThriftServer
> -
>
> Key: HIVE-23498
> URL: https://issues.apache.org/jira/browse/HIVE-23498
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 3.1.2
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-23408) Hive on Tez : Kafka storage handler broken in secure environment

2020-05-07 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-23408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-23408:
--
Summary: Hive on Tez :  Kafka storage handler broken in secure environment  
(was: hive on Tez :  Kafka storage handler broken in secure environment)

> Hive on Tez :  Kafka storage handler broken in secure environment
> -
>
> Key: HIVE-23408
> URL: https://issues.apache.org/jira/browse/HIVE-23408
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Rajkumar Singh
>Priority: Major
>
> hive.server2.authentication.kerberos.principal set in the form of 
> hive/_HOST@REALM,
> Tez task can start at the random NM host and unfold the value of _HOST with 
> the value of fqdn where it is running. this leads to an authentication issue.
> for LLAP there is fallback for LLAP daemon keytab/principal, Kafka 1.1 
> onwards support delegation token and we should take advantage of it for hive 
> on tez.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22855) Do not change DB location to external if DB URI already exists or it's referring to non-managed location

2020-02-12 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22855:
--
Attachment: HIVE-22855.patch
Status: Patch Available  (was: Open)

> Do not change DB location to external if DB URI already exists or it's 
> referring to non-managed location
> 
>
> Key: HIVE-22855
> URL: https://issues.apache.org/jira/browse/HIVE-22855
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22855.patch, HIVE-22855.patch
>
>
> from Spark:
> {code:java}
> spark.sql("CREATE DATABASE IF NOT EXISTS test LOCATION '/tmp/test'")
> spark.sql("describe database test").show(false)
> {code}
> describe output suggests that DB URI is updated to the external warehouse 
> path, all data will be written to hive warehouse external path which is 
> undesired.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22855) Do not change DB location to external if DB URI already exists or it's referring to non-managed location

2020-02-12 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22855:
--
Status: Open  (was: Patch Available)

> Do not change DB location to external if DB URI already exists or it's 
> referring to non-managed location
> 
>
> Key: HIVE-22855
> URL: https://issues.apache.org/jira/browse/HIVE-22855
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22855.patch, HIVE-22855.patch
>
>
> from Spark:
> {code:java}
> spark.sql("CREATE DATABASE IF NOT EXISTS test LOCATION '/tmp/test'")
> spark.sql("describe database test").show(false)
> {code}
> describe output suggests that DB URI is updated to the external warehouse 
> path, all data will be written to hive warehouse external path which is 
> undesired.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22855) Do not change DB location to external if DB URI already exists or it's referring to non-managed location

2020-02-07 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22855:
--
Summary: Do not change DB location to external if DB URI already exists or 
it's referring to non-managed location  (was: Do not change DB location to 
external if DB URI already exists or already referring to non-managed locattion)

> Do not change DB location to external if DB URI already exists or it's 
> referring to non-managed location
> 
>
> Key: HIVE-22855
> URL: https://issues.apache.org/jira/browse/HIVE-22855
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22855.patch
>
>
> from Spark:
> {code:java}
> spark.sql("CREATE DATABASE IF NOT EXISTS test LOCATION '/tmp/test'")
> spark.sql("describe database test").show(false)
> {code}
> describe output suggests that DB URI is updated to the external warehouse 
> path, all data will be written to hive warehouse external path which is 
> undesired.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22855) Do not change DB location to external if DB URI already exists or already referring to non-managed locattion

2020-02-07 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22855:
--
Attachment: HIVE-22855.patch
Status: Patch Available  (was: Open)

> Do not change DB location to external if DB URI already exists or already 
> referring to non-managed locattion
> 
>
> Key: HIVE-22855
> URL: https://issues.apache.org/jira/browse/HIVE-22855
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22855.patch
>
>
> from Spark:
> {code:java}
> spark.sql("CREATE DATABASE IF NOT EXISTS test LOCATION '/tmp/test'")
> spark.sql("describe database test").show(false)
> {code}
> describe output suggests that DB URI is updated to the external warehouse 
> path, all data will be written to hive warehouse external path which is 
> undesired.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (HIVE-22855) Do not change DB location to external if DB URI already exists or already referring to non-managed locattion

2020-02-07 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh reassigned HIVE-22855:
-


> Do not change DB location to external if DB URI already exists or already 
> referring to non-managed locattion
> 
>
> Key: HIVE-22855
> URL: https://issues.apache.org/jira/browse/HIVE-22855
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22855.patch
>
>
> from Spark:
> {code:java}
> spark.sql("CREATE DATABASE IF NOT EXISTS test LOCATION '/tmp/test'")
> spark.sql("describe database test").show(false)
> {code}
> describe output suggests that DB URI is updated to the external warehouse 
> path, all data will be written to hive warehouse external path which is 
> undesired.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Attachment: HIVE-22712.07.patch
Status: Patch Available  (was: Open)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.05.patch, 
> HIVE-22712.06.patch, HIVE-22712.06.patch, HIVE-22712.07.patch, 
> HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-23 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Status: Open  (was: Patch Available)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.05.patch, 
> HIVE-22712.06.patch, HIVE-22712.06.patch, HIVE-22712.07.patch, 
> HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-20 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Attachment: HIVE-22712.06.patch
Status: Patch Available  (was: Open)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.05.patch, 
> HIVE-22712.06.patch, HIVE-22712.06.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-20 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Status: Open  (was: Patch Available)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.05.patch, 
> HIVE-22712.06.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-18 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Status: Open  (was: Patch Available)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.05.patch, 
> HIVE-22712.06.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-18 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Attachment: HIVE-22712.06.patch
Status: Patch Available  (was: Open)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.05.patch, 
> HIVE-22712.06.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-18 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Attachment: HIVE-22712.05.patch
Status: Patch Available  (was: Open)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.05.patch, 
> HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-18 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Status: Open  (was: Patch Available)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-15 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Attachment: HIVE-22712.04.patch
Status: Patch Available  (was: Open)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.04.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-15 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Status: Open  (was: Patch Available)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-14 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015236#comment-17015236
 ] 

Rajkumar Singh commented on HIVE-22712:
---

Thanks [~kgyrtkirk] for review, uploaded the new patch with the suggested 
change.

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-14 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Status: Open  (was: Patch Available)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-14 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Attachment: HIVE-22712.03.patch
Status: Patch Available  (was: Open)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.03.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-10 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Attachment: HIVE-22712.02.patch
Status: Patch Available  (was: Open)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.02.patch, 
> HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-10 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Status: Open  (was: Patch Available)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-10 Thread Rajkumar Singh (Jira)



[ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013227#comment-17013227
 ] 

Rajkumar Singh commented on HIVE-22712:
---

test failures seem unrelated to the patch, I ran the test manually which 
succeeded, uploading the patch again for a clean run.

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-10 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Status: Open  (was: Patch Available)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (HIVE-22712) ReExec Driver execute submit the query in default queue irrespective of user defined queue

2020-01-10 Thread Rajkumar Singh (Jira)



 [ 
https://issues.apache.org/jira/browse/HIVE-22712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajkumar Singh updated HIVE-22712:
--
Attachment: HIVE-22712.01.patch
Status: Patch Available  (was: Open)

> ReExec Driver execute submit the query in default queue irrespective of user 
> defined queue
> --
>
> Key: HIVE-22712
> URL: https://issues.apache.org/jira/browse/HIVE-22712
> Project: Hive
>  Issue Type: Bug
>  Components: Hive, HiveServer2
>Affects Versions: 3.1.2
> Environment: Hive-3
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22712.01.patch, HIVE-22712.patch
>
>
> we unset the queue name intentionally in 
> TezSessionState#startSessionAndContainers, 
> as a result reexec create a new session in the default queue and create a 
> problem, its a cumbersome to add reexec.overlay.tez.queue.name at session 
> level.
> I could not find a better way of setting the queue name (I am open for the 
> suggestion here) since it can create a  conflict with the Global queue name 
> vs user-defined queue that's why setting while initialization of 
> ReExecutionOverlayPlugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

1 2 3 4 5 >

1 - 100 of 470 matches

Mail list logo