[jira] [Updated] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang updated HIVE-24571:
--
Description: 
In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
redundant codes,Other branche-2 also have this problem

!image-2020-12-27-16-42-59-875.png|width=781,height=411!

  was:
In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
redundant codes,in other branch-2, it's the same problem

!image-2020-12-27-16-42-59-875.png|width=781,height=411!


> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Miklos Gergely
>Priority: Minor
>  Labels: pull-request-available
> Attachments: image-2020-12-27-16-42-59-875.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes,Other branche-2 also have this problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang updated HIVE-24571:
--
Description: 
In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
redundant codes,in other branch-2, it's the same problem

!image-2020-12-27-16-42-59-875.png|width=781,height=411!

  was:
In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
redundant codes And in other branch-2, it's the same problem

!image-2020-12-27-16-42-59-875.png|width=781,height=411!


> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Miklos Gergely
>Priority: Minor
>  Labels: pull-request-available
> Attachments: image-2020-12-27-16-42-59-875.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes,in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-19981) Managed tables converted to external tables by the HiveStrictManagedMigration utility should be set to delete data when the table is dropped

2020-12-27 Thread chenruotao (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-19981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255363#comment-17255363
 ] 

chenruotao commented on HIVE-19981:
---

I have one problem about this with hive 2.3.4,by using hive jdbc handler,i 
found that though the ext table of jdbc handler has been droped, the location 
of table in HDFS extis. As time goes by, there are so many empty directory,  
can I fix the problem using this patch?

> Managed tables converted to external tables by the HiveStrictManagedMigration 
> utility should be set to delete data when the table is dropped
> 
>
> Key: HIVE-19981
> URL: https://issues.apache.org/jira/browse/HIVE-19981
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19981.1.patch
>
>
> Using the HiveStrictManagedMigration utility, tables can be converted to 
> conform to the Hive strict managed tables mode.
> For managed tables that are converted to external tables by the utility, 
> these tables should keep the "drop data on delete" semantics they had when 
> they were managed tables.
> One way to do this is to introduce a table property "external.table.purge", 
> which if true (and if the table is an external table), will let Hive know to 
> delete the table data when the table is dropped. This property will be set by 
> the HiveStrictManagedMigration utility when managed tables are converted to 
> external tables.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang reassigned HIVE-24571:
-

Assignee: Miklos Gergely  (was: Sun Chao)

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Miklos Gergely
>Priority: Minor
>  Labels: pull-request-available
> Attachments: image-2020-12-27-16-42-59-875.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes And in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-24568) Fix guice compatibility issues

2020-12-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24568?focusedWorklogId=528704&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-528704
 ]

ASF GitHub Bot logged work on HIVE-24568:
-

Author: ASF GitHub Bot
Created on: 28/Dec/20 02:38
Start Date: 28/Dec/20 02:38
Worklog Time Spent: 10m 
  Work Description: wangyum closed pull request #1812:
URL: https://github.com/apache/hive/pull/1812


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 528704)
Time Spent: 1h  (was: 50m)

> Fix guice compatibility issues
> --
>
> Key: HIVE-24568
> URL: https://issues.apache.org/jira/browse/HIVE-24568
> Project: Hive
>  Issue Type: Improvement
>Reporter: Yuming Wang
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> {noformat}
> Exception in thread "main" java.lang.NoSuchMethodError: 
> com.google.inject.util.Types.collectionOf(Ljava/lang/reflect/Type;)Ljava/lang/reflect/ParameterizedType;
> » at 
> com.google.inject.multibindings.Multibinder.collectionOfProvidersOf(Multibinder.java:202)
> » at 
> com.google.inject.multibindings.Multibinder$RealMultibinder.(Multibinder.java:283)
> » at 
> com.google.inject.multibindings.Multibinder$RealMultibinder.(Multibinder.java:258)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-21961) Update jetty version to 9.4.x

2020-12-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-21961?focusedWorklogId=528705&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-528705
 ]

ASF GitHub Bot logged work on HIVE-21961:
-

Author: ASF GitHub Bot
Created on: 28/Dec/20 02:38
Start Date: 28/Dec/20 02:38
Worklog Time Spent: 10m 
  Work Description: wangyum closed pull request #1813:
URL: https://github.com/apache/hive/pull/1813


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 528705)
Time Spent: 20m  (was: 10m)

> Update jetty version to 9.4.x
> -
>
> Key: HIVE-21961
> URL: https://issues.apache.org/jira/browse/HIVE-21961
> Project: Hive
>  Issue Type: Task
>Reporter: Oleksiy Sayankin
>Assignee: László Bodor
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-21961.02.patch, HIVE-21961.03.patch, 
> HIVE-21961.patch
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Update jetty version to 9.4.x



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-24275) Configurations to delay the deletion of obsolete files by the Cleaner

2020-12-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24275?focusedWorklogId=528694&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-528694
 ]

ASF GitHub Bot logged work on HIVE-24275:
-

Author: ASF GitHub Bot
Created on: 28/Dec/20 01:03
Start Date: 28/Dec/20 01:03
Worklog Time Spent: 10m 
  Work Description: github-actions[bot] closed pull request #1583:
URL: https://github.com/apache/hive/pull/1583


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 528694)
Time Spent: 2h  (was: 1h 50m)

> Configurations to delay the deletion of obsolete files by the Cleaner
> -
>
> Key: HIVE-24275
> URL: https://issues.apache.org/jira/browse/HIVE-24275
> Project: Hive
>  Issue Type: New Feature
>Reporter: Kishen Das
>Assignee: Peter Varga
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Whenever compaction happens, the cleaner immediately deletes older obsolete 
> files. In certain cases it would be beneficial to retain these for certain 
> period. For example : if you are serving the file metadata from cache and 
> don't want to invalidate the cache during compaction because of performance 
> reasons. 
> For this purpose we should introduce a configuration 
> hive.compactor.delayed.cleanup.enabled, which if enabled will delay the 
> cleaning up obsolete files. There should be a separate configuration 
> CLEANER_RETENTION_TIME to specify the duration till which we should retain 
> these older obsolete files. 
> It might be beneficial to have one more configuration to decide whether to 
> retain files involved in an aborted transaction 
> hive.compactor.aborted.txn.delayed.cleanup.enabled . 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-24302) Cleaner shouldn't run if it can't remove obsolete files

2020-12-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24302?focusedWorklogId=528693&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-528693
 ]

ASF GitHub Bot logged work on HIVE-24302:
-

Author: ASF GitHub Bot
Created on: 28/Dec/20 01:02
Start Date: 28/Dec/20 01:02
Worklog Time Spent: 10m 
  Work Description: github-actions[bot] commented on pull request #1612:
URL: https://github.com/apache/hive/pull/1612#issuecomment-751539632


   This pull request has been automatically marked as stale because it has not 
had recent activity. It will be closed if no further activity occurs.
   Feel free to reach out on the d...@hive.apache.org list if the patch is in 
need of reviews.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 528693)
Time Spent: 40m  (was: 0.5h)

> Cleaner shouldn't run if it can't remove obsolete files
> ---
>
> Key: HIVE-24302
> URL: https://issues.apache.org/jira/browse/HIVE-24302
> Project: Hive
>  Issue Type: Bug
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Example:
>  # open txn 5, leave it open (maybe it's a long-running compaction)
>  # insert into table t in txns 6, 7 with writeids 1, 2
>  # compactor.Worker runs on table t and compacts writeids 1, 2
>  # compactor.Cleaner picks up the compaction queue entry, but doesn't delete 
> any files because the min global open txnid is 5, which cannot see writeIds 
> 1, 2.
>  # Cleaner marks the compactor queue entry as cleaned and removes the entry 
> from the queue.
> delta_1 and delta_2 will remain in the file system until another compaction 
> is run on table t.
> Step 5 should not happen, we should skip calling markCleaned() and leave it 
> in the queue in "ready to clean" state. MarkCleaned() should be called only 
> after txn 5 is closed and, following that, the cleaner runs successfully.
> This will potentially slow down the cleaner, but on the other hand it won't 
> silently "fail" i.e. not do its job.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-24278) Implement an UDF for throwing exception in arbitrary vertex

2020-12-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24278?focusedWorklogId=528656&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-528656
 ]

ASF GitHub Bot logged work on HIVE-24278:
-

Author: ASF GitHub Bot
Created on: 27/Dec/20 16:48
Start Date: 27/Dec/20 16:48
Worklog Time Spent: 10m 
  Work Description: abstractdog opened a new pull request #1817:
URL: https://github.com/apache/hive/pull/1817


   
   
   ### What changes were proposed in this pull request?
   
   
   
   ### Why are the changes needed?
   
   
   
   ### Does this PR introduce _any_ user-facing change?
   
   
   
   ### How was this patch tested?
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 528656)
Remaining Estimate: 0h
Time Spent: 10m

> Implement an UDF for throwing exception in arbitrary vertex
> ---
>
> Key: HIVE-24278
> URL: https://issues.apache.org/jira/browse/HIVE-24278
> Project: Hive
>  Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> For testing purposes sometimes we need to make the query fail in a vertex, so 
> assuming that we already know the plan, it could be something like:
> on mapper side
> {code}
> select a.col1, exception_in_vertex("Map 1") from a
> join b on b.id = a.id
> {code}
> or on reducer side
> {code}
> select a.col1, exception_in_vertex("Reducer 2") from a
> join b on b.id = a.id
> {code}
> more advanced configuration is possible, e.g we want to make Reducer 2's 50th 
> task's first (0th) attempt fail with an exception:
> {code}
> exception_in_vertex("Reducer 2", 50, 0)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-24278) Implement an UDF for throwing exception in arbitrary vertex

2020-12-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-24278:
--
Labels: pull-request-available  (was: )

> Implement an UDF for throwing exception in arbitrary vertex
> ---
>
> Key: HIVE-24278
> URL: https://issues.apache.org/jira/browse/HIVE-24278
> Project: Hive
>  Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> For testing purposes sometimes we need to make the query fail in a vertex, so 
> assuming that we already know the plan, it could be something like:
> on mapper side
> {code}
> select a.col1, exception_in_vertex("Map 1") from a
> join b on b.id = a.id
> {code}
> or on reducer side
> {code}
> select a.col1, exception_in_vertex("Reducer 2") from a
> join b on b.id = a.id
> {code}
> more advanced configuration is possible, e.g we want to make Reducer 2's 50th 
> task's first (0th) attempt fail with an exception:
> {code}
> exception_in_vertex("Reducer 2", 50, 0)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22944) Upgrade to Kryo5

2020-12-27 Thread Jira


 [ 
https://issues.apache.org/jira/browse/HIVE-22944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

László Bodor updated HIVE-22944:

Description: 
Maybe we should consider upgrading to kryo5 (plan ser/deser). Not sure about 
performance benefits, but looking at the code, e.g. FieldSerializer in Kryo5 
seems to let us extend it easier (less private fields), which could be a 
benefit if we want to change its behavior, e.g. defining different logic for 
different fields of an object.

Kryo 4 FieldSerializer: 
https://github.com/EsotericSoftware/kryo/blob/kryo-4/src/com/esotericsoftware/kryo/serializers/FieldSerializer.java
Kryo 5 FieldSerialier: 
https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/serializers/FieldSerializer.java

UPDATE: currently we are at kryo 5.0.3

TODO: why kryo-shaded artifact has been used so far?
https://javalibs.com/artifact/com.esotericsoftware/kryo-shaded
"This contains the shaded reflectasm jar to prevent conflicts with other 
versions of asm."



  was:
experimental ticket

Maybe we should consider upgrading to kryo5 (plan ser/deser). Not sure about 
performance benefits, but looking at the code, e.g. FieldSerializer in Kryo5 
seems to let us extend it easier (less private fields), which could be a 
benefit if we want to change its behavior, e.g. defining different logic for 
different fields of an object.

Kryo 4 FieldSerializer: 
https://github.com/EsotericSoftware/kryo/blob/kryo-4/src/com/esotericsoftware/kryo/serializers/FieldSerializer.java
Kryo 5 FieldSerialier: 
https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/serializers/FieldSerializer.java

versions, according to maven central:

5.0.0-RC4 - Apr, 2019
5.0.0-RC3 - Apr, 2019
5.0.0-RC2 - Feb, 2019
5.0.0-RC1 - Jun, 2018

there is no "5.0.0" yet

TODO: why kryo-shaded artifact has been used so far?
https://javalibs.com/artifact/com.esotericsoftware/kryo-shaded
"This contains the shaded reflectasm jar to prevent conflicts with other 
versions of asm."

UPDATE: currently we are at kryo 5.0.3


> Upgrade to Kryo5
> 
>
> Key: HIVE-22944
> URL: https://issues.apache.org/jira/browse/HIVE-22944
> Project: Hive
>  Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22944.01.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Maybe we should consider upgrading to kryo5 (plan ser/deser). Not sure about 
> performance benefits, but looking at the code, e.g. FieldSerializer in Kryo5 
> seems to let us extend it easier (less private fields), which could be a 
> benefit if we want to change its behavior, e.g. defining different logic for 
> different fields of an object.
> Kryo 4 FieldSerializer: 
> https://github.com/EsotericSoftware/kryo/blob/kryo-4/src/com/esotericsoftware/kryo/serializers/FieldSerializer.java
> Kryo 5 FieldSerialier: 
> https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/serializers/FieldSerializer.java
> UPDATE: currently we are at kryo 5.0.3
> TODO: why kryo-shaded artifact has been used so far?
> https://javalibs.com/artifact/com.esotericsoftware/kryo-shaded
> "This contains the shaded reflectasm jar to prevent conflicts with other 
> versions of asm."



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-16958) Setting hive.merge.sparkfiles=true will retrun an error when generating parquet databases

2020-12-27 Thread laixiong (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-16958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17255231#comment-17255231
 ] 

laixiong commented on HIVE-16958:
-

Why this patch not applied into branch 2.3.0 , is there any historical reasons 
or rules?

As a new user of hive, I download the source code tar ball 2.3.7 from 
[https://mirror.bit.edu.cn/apache/hive/] and build it and then met this bug , 
so I'm confused.

> Setting hive.merge.sparkfiles=true will retrun an error when generating 
> parquet databases 
> --
>
> Key: HIVE-16958
> URL: https://issues.apache.org/jira/browse/HIVE-16958
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 2.2.0, 2.3.0
> Environment: centos7 hadoop2.7.3 spark2.0.0
>Reporter: Liu Chunxiao
>Assignee: liyunzhang
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-16958.patch, Main.java, RowGenerator.java, 
> hive-site.xml, parquet-hivemergesparkfiles.txt, pom.xml, sale.sql
>
>
> The process will return 
> Job failed with java.lang.NullPointerException
> FAILED: Execution Error, return code 3 from 
> org.apache.hadoop.hive.ql.exec.spark.SparkTask. 
> java.util.concurrent.ExecutionException: Exception thrown by job
> at org.apache.spark.JavaFutureActionWrapper.getImpl(FutureAction.scala:272)
> at org.apache.spark.JavaFutureActionWrapper.get(FutureAction.scala:277)
> at 
> org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:362)
> at 
> org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:323)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: 
> Task 1 in stage 1.0 failed 4 times, most recent failure: Lost task 1.3 in 
> stage 1.0 (TID 31, bdpe822n1): java.io.IOException: 
> java.lang.reflect.InvocationTargetException
> at 
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at 
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at 
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:271)
> at 
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.(HadoopShimsSecure.java:217)
> at 
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:345)
> at 
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:695)
> at org.apache.spark.rdd.HadoopRDD$$anon$1.(HadoopRDD.scala:246)
> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:209)
> at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:102)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
> at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:319)
> at org.apache.spark.rdd.RDD.iterator(RDD.scala:283)
> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
> at org.apache.spark.scheduler.Task.run(Task.scala:85)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.GeneratedConstructorAccessor26.newInstance(Unknown Source)
> at 
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at 
> org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:257)
> ... 17 more
> Caused by: java.lang.NullPointerException
> at java.util.AbstractCollection.addAll(AbstractCollection.java:343)
> at 
> org.apache.hadoop.hive.ql.io.parquet.ProjectionPusher.pushProjectionsAndFilters(ProjectionPusher.java:118)
> at 
> org.apache.hadoop.hive.ql.io.parquet.ProjectionPusher.pushProjectionsAndFilters(ProjectionPusher.java:189)
> at 
> org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:84)
> at 
> org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.(P

[jira] [Work logged] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?focusedWorklogId=528602&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-528602
 ]

ASF GitHub Bot logged work on HIVE-24571:
-

Author: ASF GitHub Bot
Created on: 27/Dec/20 08:57
Start Date: 27/Dec/20 08:57
Worklog Time Spent: 10m 
  Work Description: zzccctv edited a comment on pull request #1770:
URL: https://github.com/apache/hive/pull/1770#issuecomment-751442114


   OK,To solve the problem, I have set up an issues in 
JIRA,@kgyrtkirk,HIVE-24571,The master branch does not have this problem,Which 
branch should I submit to



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 528602)
Time Spent: 20m  (was: 10m)

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Sun Chao
>Priority: Minor
>  Labels: pull-request-available
> Attachments: image-2020-12-27-16-42-59-875.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes And in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?focusedWorklogId=528601&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-528601
 ]

ASF GitHub Bot logged work on HIVE-24571:
-

Author: ASF GitHub Bot
Created on: 27/Dec/20 08:55
Start Date: 27/Dec/20 08:55
Worklog Time Spent: 10m 
  Work Description: zzccctv commented on pull request #1770:
URL: https://github.com/apache/hive/pull/1770#issuecomment-751442114


   OK,To solve the problem, I have set up an issues in JIRA,@kgyrtkirk



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 528601)
Remaining Estimate: 0h
Time Spent: 10m

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Sun Chao
>Priority: Minor
> Attachments: image-2020-12-27-16-42-59-875.png
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes And in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-24571:
--
Labels: pull-request-available  (was: )

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Sun Chao
>Priority: Minor
>  Labels: pull-request-available
> Attachments: image-2020-12-27-16-42-59-875.png
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes And in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang reassigned HIVE-24571:
-

Assignee: Sun Chao  (was: Alan Gates)

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Sun Chao
>Priority: Minor
> Attachments: image-2020-12-27-16-42-59-875.png
>
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes And in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang reassigned HIVE-24571:
-

Assignee: Alan Gates  (was: Sun Chao)

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Alan Gates
>Priority: Minor
> Attachments: image-2020-12-27-16-42-59-875.png
>
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes And in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang reassigned HIVE-24571:
-

Assignee: zhangzhanchang  (was: Sun Chao)

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: zhangzhanchang
>Priority: Minor
> Attachments: image-2020-12-27-16-42-59-875.png
>
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes And in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang reassigned HIVE-24571:
-

Assignee: Sun Chao  (was: zhangzhanchang)

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Sun Chao
>Priority: Minor
> Attachments: image-2020-12-27-16-42-59-875.png
>
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes And in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang updated HIVE-24571:
--
Description: 
In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
redundant codes And in other branch-2, it's the same problem

!image-2020-12-27-16-42-59-875.png|width=781,height=411!

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Sun Chao
>Priority: Major
> Attachments: image-2020-12-27-16-42-59-875.png
>
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes And in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang updated HIVE-24571:
--
Issue Type: Improvement  (was: Bug)

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Sun Chao
>Priority: Minor
> Attachments: image-2020-12-27-16-42-59-875.png
>
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes And in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang updated HIVE-24571:
--
Priority: Minor  (was: Major)

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Sun Chao
>Priority: Minor
> Attachments: image-2020-12-27-16-42-59-875.png
>
>
> In the newly released release-2.3.7, the file SemanticAnalyzer.java There are 
> redundant codes And in other branch-2, it's the same problem
> !image-2020-12-27-16-42-59-875.png|width=781,height=411!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang updated HIVE-24571:
--
Attachment: image-2020-12-27-16-42-59-875.png

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Sun Chao
>Priority: Major
> Attachments: image-2020-12-27-16-42-59-875.png
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-24571) Redundant code in SemanticAnalyzer.java

2020-12-27 Thread zhangzhanchang (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

zhangzhanchang reassigned HIVE-24571:
-

Assignee: Sun Chao

> Redundant code in SemanticAnalyzer.java
> ---
>
> Key: HIVE-24571
> URL: https://issues.apache.org/jira/browse/HIVE-24571
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 2.3.7
>Reporter: zhangzhanchang
>Assignee: Sun Chao
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)