[jira] [Work logged] (HIVE-26049) Inconsistent TBL_NAME lengths in HMS schema
[ https://issues.apache.org/jira/browse/HIVE-26049?focusedWorklogId=744547=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744547 ] ASF GitHub Bot logged work on HIVE-26049: - Author: ASF GitHub Bot Created on: 19/Mar/22 01:49 Start Date: 19/Mar/22 01:49 Worklog Time Spent: 10m Work Description: kovjanos commented on pull request #3119: URL: https://github.com/apache/hive/pull/3119#issuecomment-1072915735 The upgrade script was adjusted for MSSQL and MYSQL as these engines lost the NOT NULL constraint on ALTER. The others - Derby, Postgres and Oracle - don't need this, actually this would be invalid ALTER statement for those engines. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 744547) Time Spent: 0.5h (was: 20m) > Inconsistent TBL_NAME lengths in HMS schema > > > Key: HIVE-26049 > URL: https://issues.apache.org/jira/browse/HIVE-26049 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 4.0.0-alpha-1 >Reporter: Janos Kovacs >Assignee: Janos Kovacs >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > The transaction specific tables have TBL_NAME length set to 128 while the > TBLS itself allows 256. When a table is created with such a long name hive > fails e.g. to lock the table. > TBL_NAME needs to be consistent across the whole schema. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-25860) Optimize the synchronized scope of the renewToken method to improve concurrency
[ https://issues.apache.org/jira/browse/HIVE-25860?focusedWorklogId=744509=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744509 ] ASF GitHub Bot logged work on HIVE-25860: - Author: ASF GitHub Bot Created on: 19/Mar/22 00:15 Start Date: 19/Mar/22 00:15 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #2937: URL: https://github.com/apache/hive/pull/2937#issuecomment-1072899883 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 744509) Time Spent: 20m (was: 10m) > Optimize the synchronized scope of the renewToken method to improve > concurrency > --- > > Key: HIVE-25860 > URL: https://issues.apache.org/jira/browse/HIVE-25860 > Project: Hive > Issue Type: Bug >Reporter: dzcxzl >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > HIVE-22033 Using tokenStore in renewToken method does not need to be in > synchronized scope. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-25867) Partition filter condition should pushed down to metastore query if it is equivalence Predicate
[ https://issues.apache.org/jira/browse/HIVE-25867?focusedWorklogId=744508=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744508 ] ASF GitHub Bot logged work on HIVE-25867: - Author: ASF GitHub Bot Created on: 19/Mar/22 00:15 Start Date: 19/Mar/22 00:15 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #2947: URL: https://github.com/apache/hive/pull/2947#issuecomment-1072899876 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 744508) Time Spent: 0.5h (was: 20m) > Partition filter condition should pushed down to metastore query if it is > equivalence Predicate > --- > > Key: HIVE-25867 > URL: https://issues.apache.org/jira/browse/HIVE-25867 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: shezm >Assignee: shezm >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The colnum type of the partition is different from the column type of the hql > query, the metastore will not push down the query to the RDBMS, but will > instead get all PARTITIONS.PART_NAME of the hive table then filter it > according to the hql Expression. > https://github.com/apache/hive/blob/5b112aa6dcc4e374c0a7c2b24042f24ae6815da1/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java#L1316 > If the hive table has too many partitions and there are multiple hql queries > at the same time,RDBMS will increasing CPU IO_WAIT and affect performance. > If the partition filter condition in hql is an equivalent predicate, the > metastore should be pushed down to RDBMS, which can optimize the query > performance of hive large tables. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-25292) to_unix_timestamp & unix_timestamp should support ENGLISH format by default
[
https://issues.apache.org/jira/browse/HIVE-25292?focusedWorklogId=744510=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744510
]
ASF GitHub Bot logged work on HIVE-25292:
-
Author: ASF GitHub Bot
Created on: 19/Mar/22 00:15
Start Date: 19/Mar/22 00:15
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on pull request #2433:
URL: https://github.com/apache/hive/pull/2433#issuecomment-1072899897
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the d...@hive.apache.org list if the patch is in
need of reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 744510)
Time Spent: 1h 10m (was: 1h)
> to_unix_timestamp & unix_timestamp should support ENGLISH format by default
> ---
>
> Key: HIVE-25292
> URL: https://issues.apache.org/jira/browse/HIVE-25292
> Project: Hive
> Issue Type: Improvement
> Components: Clients
>Reporter: shezm
>Assignee: shezm
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.2.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Hei
> The to_unix_timestamp function is implemented by GenericUDFToUnixTimeStamp.
> It uses SimpleDateFormat to parse the time of the string type.
> But SimpleDateFormat does not specify the Locale parameter, that is, the
> default locale of the jvm machine will be used. This will cause some
> non-English local machines to be unable to run similar sql like :
>
> {code:java}
> hive> select to_unix_timestamp('16/Mar/2017:12:25:01', 'dd/MMM/yyy:HH:mm:ss');
> OK
> NULL
> hive> select unix_timestamp('16/Mar/2017:12:25:01', 'dd/MMM/yyy:HH:mm:ss');
> OK
> NULL
> {code}
>
> At the same time, I found that in spark, to_unix_timestamp & unix_timestamp
> also use SimpleDateFormat, and spark uses Locale.US by default, but this will
> make it impossible to use local language syntax. For example, in the Chinese
> environment, I can parse this result correctly in hive,
>
> {code:java}
> hive> select to_unix_timestamp('16/三月/2017:12:25:01', 'dd//yyy:HH:mm:ss');
> OK
> 1489638301
> Time taken: 0.147 seconds, Fetched: 1 row(s)
> OK
> {code}
> But spark will return Null.
> Because English dates are more common dates, I think two SimpleDateFormats
> are needed. The new SimpleDateFormat is initialized with the Locale.ENGLISH
> parameter.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-26049) Inconsistent TBL_NAME lengths in HMS schema
[ https://issues.apache.org/jira/browse/HIVE-26049?focusedWorklogId=744455=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744455 ] ASF GitHub Bot logged work on HIVE-26049: - Author: ASF GitHub Bot Created on: 18/Mar/22 22:07 Start Date: 18/Mar/22 22:07 Worklog Time Spent: 10m Work Description: kovjanos commented on pull request #3119: URL: https://github.com/apache/hive/pull/3119#issuecomment-1072850355 Just found an issue in the MSSQL upgrade code: compared to others it has an extra PK referencing the _TABLE column which makes the ALTER to fail. I'll fix it in the next commit... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 744455) Time Spent: 20m (was: 10m) > Inconsistent TBL_NAME lengths in HMS schema > > > Key: HIVE-26049 > URL: https://issues.apache.org/jira/browse/HIVE-26049 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 4.0.0-alpha-1 >Reporter: Janos Kovacs >Assignee: Janos Kovacs >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > The transaction specific tables have TBL_NAME length set to 128 while the > TBLS itself allows 256. When a table is created with such a long name hive > fails e.g. to lock the table. > TBL_NAME needs to be consistent across the whole schema. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-26049) Inconsistent TBL_NAME lengths in HMS schema
[ https://issues.apache.org/jira/browse/HIVE-26049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-26049: -- Labels: pull-request-available (was: ) > Inconsistent TBL_NAME lengths in HMS schema > > > Key: HIVE-26049 > URL: https://issues.apache.org/jira/browse/HIVE-26049 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 4.0.0-alpha-1 >Reporter: Janos Kovacs >Assignee: Janos Kovacs >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The transaction specific tables have TBL_NAME length set to 128 while the > TBLS itself allows 256. When a table is created with such a long name hive > fails e.g. to lock the table. > TBL_NAME needs to be consistent across the whole schema. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Assigned] (HIVE-26049) Inconsistent TBL_NAME lengths in HMS schema
[ https://issues.apache.org/jira/browse/HIVE-26049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janos Kovacs reassigned HIVE-26049: --- > Inconsistent TBL_NAME lengths in HMS schema > > > Key: HIVE-26049 > URL: https://issues.apache.org/jira/browse/HIVE-26049 > Project: Hive > Issue Type: Bug > Components: Database/Schema >Affects Versions: 4.0.0-alpha-1 >Reporter: Janos Kovacs >Assignee: Janos Kovacs >Priority: Major > > The transaction specific tables have TBL_NAME length set to 128 while the > TBLS itself allows 256. When a table is created with such a long name hive > fails e.g. to lock the table. > TBL_NAME needs to be consistent across the whole schema. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26002) Preparing for 4.0.0-alpha-1 development
[ https://issues.apache.org/jira/browse/HIVE-26002?focusedWorklogId=744137=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744137 ] ASF GitHub Bot logged work on HIVE-26002: - Author: ASF GitHub Bot Created on: 18/Mar/22 15:09 Start Date: 18/Mar/22 15:09 Worklog Time Spent: 10m Work Description: pvary merged pull request #3081: URL: https://github.com/apache/hive/pull/3081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 744137) Time Spent: 1h 40m (was: 1.5h) > Preparing for 4.0.0-alpha-1 development > --- > > Key: HIVE-26002 > URL: https://issues.apache.org/jira/browse/HIVE-26002 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > For the release we need to create the appropriate sql scripts for HMS db > initialization -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26042) Fix flaky streaming tests
[
https://issues.apache.org/jira/browse/HIVE-26042?focusedWorklogId=744146=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744146
]
ASF GitHub Bot logged work on HIVE-26042:
-
Author: ASF GitHub Bot
Created on: 18/Mar/22 15:09
Start Date: 18/Mar/22 15:09
Worklog Time Spent: 10m
Work Description: pvary opened a new pull request #3114:
URL: https://github.com/apache/hive/pull/3114
### What changes were proposed in this pull request?
We should set the warehouse locations in the tests, so the managed location
will also be inside the temporary folder. This will prevent the tests to fail,
if the previous ones left some directories behind
### Why are the changes needed?
These tests are often failing on the CI
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Run the tests manually also used a breakpoint to make sure that the
directories are correct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 744146)
Time Spent: 20m (was: 10m)
> Fix flaky streaming tests
> -
>
> Key: HIVE-26042
> URL: https://issues.apache.org/jira/browse/HIVE-26042
> Project: Hive
> Issue Type: Test
>Reporter: Peter Vary
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The TestStreamingDynamicPartitioning / TestStreaming tests are often failing
> because of problems with creating directories.
> Example:
> {code}
> 2022-03-17T04:03:39,024 ERROR [main] metastore.RetryingHMSHandler:
> MetaException(message:Unable to create database managed directory
> pfile:/home/jenkins/agent/workspace/hive-precommit_PR-3081/streaming/target/warehouse/testing5.db,
> failed to create database testing5)
> at
> org.apache.hadoop.hive.metastore.HMSHandler.create_database_core(HMSHandler.java:1269)
> at
> org.apache.hadoop.hive.metastore.HMSHandler.create_database(HMSHandler.java:1389)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:146)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy38.create_database(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createDatabase(HiveMetaStoreClient.java:1144)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:218)
> at com.sun.proxy.$Proxy47.createDatabase(Unknown Source)
> at org.apache.hadoop.hive.ql.metadata.Hive.createDatabase(Hive.java:608)
> at
> org.apache.hadoop.hive.ql.ddl.database.create.CreateDatabaseOperation.execute(CreateDatabaseOperation.java:68)
> at org.apache.hadoop.hive.ql.ddl.DDLTask.execute(DDLTask.java:84)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
> at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:361)
> at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:334)
> at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:245)
> at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:106)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:348)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:204)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:153)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:148)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:185)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:228)
> at
> org.apache.hive.streaming.TestStreaming.runDDL(TestStreaming.java:3202)
> at
[jira] [Work logged] (HIVE-26029) Upgrade netty to 4.1.75.Final due to CVE
[ https://issues.apache.org/jira/browse/HIVE-26029?focusedWorklogId=744134=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744134 ] ASF GitHub Bot logged work on HIVE-26029: - Author: ASF GitHub Bot Created on: 18/Mar/22 15:08 Start Date: 18/Mar/22 15:08 Worklog Time Spent: 10m Work Description: nrg4878 commented on pull request #3097: URL: https://github.com/apache/hive/pull/3097#issuecomment-1071113615 Fix has been merged to master. Please close the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 744134) Time Spent: 50m (was: 40m) > Upgrade netty to 4.1.75.Final due to CVE > > > Key: HIVE-26029 > URL: https://issues.apache.org/jira/browse/HIVE-26029 > Project: Hive > Issue Type: Improvement >Reporter: Yu-Wen Lai >Assignee: Yu-Wen Lai >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > Time Spent: 50m > Remaining Estimate: 0h > > As title. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26040) Fix DirectSqlUpdateStat.getNextCSIdForMPartitionColumnStatistics for mssql
[
https://issues.apache.org/jira/browse/HIVE-26040?focusedWorklogId=744068=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744068
]
ASF GitHub Bot logged work on HIVE-26040:
-
Author: ASF GitHub Bot
Created on: 18/Mar/22 15:02
Start Date: 18/Mar/22 15:02
Worklog Time Spent: 10m
Work Description: pvary merged pull request #3112:
URL: https://github.com/apache/hive/pull/3112
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 744068)
Time Spent: 0.5h (was: 20m)
> Fix DirectSqlUpdateStat.getNextCSIdForMPartitionColumnStatistics for mssql
> --
>
> Key: HIVE-26040
> URL: https://issues.apache.org/jira/browse/HIVE-26040
> Project: Hive
> Issue Type: Sub-task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code}
> mvn test -Dtest=TestMiniLlapLocalCliDriver -Dqfile=list_bucket_dml_9.q
> -Dtest.metastore.db=mssql
> {code}
> Fails with
> {code}
> 2022-03-15T07:57:17,078 ERROR [2b933b88-6083-4750-b151-2d2c7e04ccce main]
> metastore.DirectSqlUpdateStat: Unable to
> getNextCSIdForMPartitionColumnStatistics
> com.microsoft.sqlserver.jdbc.SQLServerException: Line 1: FOR UPDATE clause
> allowed only for DECLARE CURSOR.
> at
> com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:258)
> ~[mssql-jdbc-6.2.1.jre8.jar:?]
> at
> com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1535)
> ~[mssql-jdbc-6.2.1.jre8.jar:?]
> at
> com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:845)
> ~[mssql-jdbc-6.2.1.jre8.jar:?]
> at
> com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:752)
> ~[mssql-jdbc-6.2.1.jre8.jar:?]
> at
> com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7151)
> ~[mssql-jdbc-6.2.1.jre8.jar:?]
> at
> com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2478)
> ~[mssql-jdbc-6.2.1.jre8.jar:?]
> at
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:219)
> ~[mssql-jdbc-6.2.1.jre8.jar:?]
> at
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:199)
> ~[mssql-jdbc-6.2.1.jre8.jar:?]
> at
> com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQuery(SQLServerStatement.java:654)
> ~[mssql-jdbc-6.2.1.jre8.jar:?]
> at
> com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> ~[HikariCP-2.6.1.jar:?]
> at
> com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> ~[HikariCP-2.6.1.jar:?]
> at
> org.apache.hadoop.hive.metastore.DirectSqlUpdateStat.getNextCSIdForMPartitionColumnStatistics(DirectSqlUpdateStat.java:676)
> [hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.MetaStoreDirectSql.updatePartitionColumnStatisticsBatch(MetaStoreDirectSql.java:2966)
> [hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.metastore.ObjectStore.updatePartitionColumnStatisticsInBatch(ObjectStore.java:9849)
> [hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_261]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_261]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_261]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_261]
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> [hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
> com.sun.proxy.$Proxy60.updatePartitionColumnStatisticsInBatch(Unknown Source)
> [?:?]
> at
> org.apache.hadoop.hive.metastore.HMSHandler.updatePartitionColStatsForOneBatch(HMSHandler.java:7060)
> [hive-standalone-metastore-server-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
> at
>
[jira] [Work logged] (HIVE-26044) Remove hardcoded version references from the tests
[ https://issues.apache.org/jira/browse/HIVE-26044?focusedWorklogId=744059=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744059 ] ASF GitHub Bot logged work on HIVE-26044: - Author: ASF GitHub Bot Created on: 18/Mar/22 15:01 Start Date: 18/Mar/22 15:01 Worklog Time Spent: 10m Work Description: pvary opened a new pull request #3115: URL: https://github.com/apache/hive/pull/3115 ### What changes were proposed in this pull request? Remove hardcoded version references from the tests ### Why are the changes needed? For easier version changes ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Running the unit tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 744059) Time Spent: 20m (was: 10m) > Remove hardcoded version references from the tests > -- > > Key: HIVE-26044 > URL: https://issues.apache.org/jira/browse/HIVE-26044 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > There are several places where there are hardcoded version references in the > tests. > We should remove them to so it is easier to change versions. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26028) Upgrade pac4j-saml-opensamlv3 to 4.5.5 due to CVE
[ https://issues.apache.org/jira/browse/HIVE-26028?focusedWorklogId=744070=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744070 ] ASF GitHub Bot logged work on HIVE-26028: - Author: ASF GitHub Bot Created on: 18/Mar/22 15:02 Start Date: 18/Mar/22 15:02 Worklog Time Spent: 10m Work Description: nrg4878 commented on pull request #3096: URL: https://github.com/apache/hive/pull/3096#issuecomment-1071113114 Fix has been merged. Please close the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 744070) Time Spent: 1h (was: 50m) > Upgrade pac4j-saml-opensamlv3 to 4.5.5 due to CVE > - > > Key: HIVE-26028 > URL: https://issues.apache.org/jira/browse/HIVE-26028 > Project: Hive > Issue Type: Improvement >Reporter: Yu-Wen Lai >Assignee: Yu-Wen Lai >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > As title. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-25575) Add support for JWT authentication in HTTP mode
[
https://issues.apache.org/jira/browse/HIVE-25575?focusedWorklogId=744027=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-744027
]
ASF GitHub Bot logged work on HIVE-25575:
-
Author: ASF GitHub Bot
Created on: 18/Mar/22 14:58
Start Date: 18/Mar/22 14:58
Worklog Time Spent: 10m
Work Description: sourabh912 commented on a change in pull request #3006:
URL: https://github.com/apache/hive/pull/3006#discussion_r829414588
##
File path: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
##
@@ -4127,7 +4127,9 @@ private static void populateLlapDaemonVarsSet(Set
llapDaemonVarsSetLocal
" (Use with property
hive.server2.custom.authentication.class)\n" +
" PAM: Pluggable authentication module\n" +
" NOSASL: Raw transport\n" +
-" SAML: SAML 2.0 compliant authentication. This is only supported in
http transport mode."),
+" SAML: SAML 2.0 compliant authentication. This is only supported in
http transport mode.\n" +
+" JWT: JWT based authentication, JWT needs to contain the user name
as subject. This is only supported in\n" +
Review comment:
nit: Here we should also document that HS2 expects Asymmetric key for
JWT signature verification.
##
File path:
service/src/java/org/apache/hive/service/auth/jwt/URLBasedJWKSProvider.java
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.service.auth.jwt;
+
+import com.nimbusds.jose.JWSHeader;
+import com.nimbusds.jose.jwk.JWK;
+import com.nimbusds.jose.jwk.JWKMatcher;
+import com.nimbusds.jose.jwk.JWKSelector;
+import com.nimbusds.jose.jwk.JWKSet;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URL;
+import java.text.ParseException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Provides a way to get JWKS json. Hive will use this to verify the incoming
JWTs.
+ */
+public class URLBasedJWKSProvider {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(URLBasedJWKSProvider.class.getName());
+ private final HiveConf conf;
+ private List jwkSets = new ArrayList<>();
+
+ public URLBasedJWKSProvider(HiveConf conf) throws IOException,
ParseException {
+this.conf = conf;
+loadJWKSets();
+ }
+
+ /**
+ * Fetches the JWKS and stores into memory. The JWKS are expected to be in
the standard form as defined here -
+ * https://datatracker.ietf.org/doc/html/rfc7517#appendix-A.
+ */
+ private void loadJWKSets() throws IOException, ParseException {
+String jwksURL = HiveConf.getVar(conf,
HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION_JWT_JWKS_URL);
+String[] jwksURLs = jwksURL.split(",");
+for (String urlString : jwksURLs) {
+ URL url = new URL(urlString);
+ jwkSets.add(JWKSet.load(url));
+ LOG.info("Loaded JWKS from " + urlString);
+}
+ }
+
+ /**
+ * Returns filtered JWKS by one or more criteria, such as kid, typ, alg.
+ */
+ public List getJWKs(JWSHeader header) {
+List jwks = new ArrayList<>();
+JWKSelector selector = new JWKSelector(JWKMatcher.forJWSHeader(header));
+for (JWKSet jwkSet : jwkSets) {
+ List selectedJwks = selector.select(jwkSet);
+ if (selectedJwks != null) {
Review comment:
`select` api in selector never returns null. Therefore we don't need a
null check
##
File path: service/src/java/org/apache/hive/service/auth/jwt/JWTValidator.java
##
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ *
[jira] [Work logged] (HIVE-26047) Vectorized LIKE UDF should use Re2J regex to address JDK-8203458
[
https://issues.apache.org/jira/browse/HIVE-26047?focusedWorklogId=743966=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-743966
]
ASF GitHub Bot logged work on HIVE-26047:
-
Author: ASF GitHub Bot
Created on: 18/Mar/22 14:52
Start Date: 18/Mar/22 14:52
Worklog Time Spent: 10m
Work Description: nareshpr opened a new pull request #3117:
URL: https://github.com/apache/hive/pull/3117
### What changes were proposed in this pull request?
Vectorized LIKE udf is taking proportionately higher time depending on the
size of input string in UDF.
### Why are the changes needed?
To support filter condition based on input data.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Added testcase as part of PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 743966)
Time Spent: 20m (was: 10m)
> Vectorized LIKE UDF should use Re2J regex to address JDK-8203458
>
>
> Key: HIVE-26047
> URL: https://issues.apache.org/jira/browse/HIVE-26047
> Project: Hive
> Issue Type: Bug
>Reporter: Naresh P R
>Assignee: Naresh P R
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Below pattern is taking a long time to validate regex in java8 with same
> trace as shown in java bug
> [JDK-8203458|https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8203458]
>
> {code:java}
> import java.util.regex.Pattern;
> public class Test {
> public static void main(String args[]) {
> String pattern =
> "a_b";
>
> Pattern CHAIN_PATTERN = Pattern.compile("(%?[^%_]+%?)+");
> CHAIN_PATTERN.matcher(pattern).matches();
> }
> }
> {code}
> Same is reproducible with following SQL
> {code:java}
> create table table1(name string);
> insert into table1 (name) values
> ('a_b');
> select * from table1 where name like
> "a_b";{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-25575) Add support for JWT authentication in HTTP mode
[
https://issues.apache.org/jira/browse/HIVE-25575?focusedWorklogId=743944=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-743944
]
ASF GitHub Bot logged work on HIVE-25575:
-
Author: ASF GitHub Bot
Created on: 18/Mar/22 14:50
Start Date: 18/Mar/22 14:50
Worklog Time Spent: 10m
Work Description: hsnusonic commented on a change in pull request #3006:
URL: https://github.com/apache/hive/pull/3006#discussion_r829592567
##
File path:
service/src/java/org/apache/hive/service/auth/jwt/URLBasedJWKSProvider.java
##
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.service.auth.jwt;
+
+import com.nimbusds.jose.JWSHeader;
+import com.nimbusds.jose.jwk.JWK;
+import com.nimbusds.jose.jwk.JWKMatcher;
+import com.nimbusds.jose.jwk.JWKSelector;
+import com.nimbusds.jose.jwk.JWKSet;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URL;
+import java.text.ParseException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Implementation of {@link JWKSProvider} which reads JWKS from URL.
+ */
+public class URLBasedJWKSProvider implements JWKSProvider {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(URLBasedJWKSProvider.class.getName());
+ private final HiveConf conf;
+ private List jwkSets = new ArrayList<>();
+
+ public URLBasedJWKSProvider(HiveConf conf) {
+this.conf = conf;
+loadJWKSets();
+ }
+
+ private void loadJWKSets() {
+String jwksURL = HiveConf.getVar(conf,
HiveConf.ConfVars.HIVE_SERVER2_THRIFT_HTTP_JWT_JWKS_URL);
+List jwksURLs =
Arrays.stream(jwksURL.split(",")).collect(Collectors.toList());
+for (String urlString : jwksURLs) {
+ try {
+URL url = new URL(urlString);
+jwkSets.add(JWKSet.load(url));
+LOG.info("Loaded JWKS from " + urlString);
+ } catch (IOException | ParseException e) {
+LOG.info("Failed to retrieve JWKS from {}: {}", urlString,
e.getMessage());
+ }
+}
+ }
+
+ @Override
+ public List getJWKs(JWSHeader header) {
+List jwks = new ArrayList<>();
+JWKSelector selector = new JWKSelector(JWKMatcher.forJWSHeader(header));
Review comment:
Good catch! Thanks.
##
File path:
service/src/java/org/apache/hive/service/auth/jwt/URLBasedJWKSProvider.java
##
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hive.service.auth.jwt;
+
+import com.nimbusds.jose.JWSHeader;
+import com.nimbusds.jose.jwk.JWK;
+import com.nimbusds.jose.jwk.JWKMatcher;
+import com.nimbusds.jose.jwk.JWKSelector;
+import com.nimbusds.jose.jwk.JWKSet;
+import org.apache.hadoop.hive.conf.HiveConf;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URL;
+import java.text.ParseException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.stream.Collectors;
+
+/**
+ * Provides a way to get JWKS json. Hive will use this to verify the incoming
JWTs.
+ */
+public class URLBasedJWKSProvider {
+
+ private static final Logger LOG =
LoggerFactory.getLogger(URLBasedJWKSProvider.class.getName());
+
[jira] [Work logged] (HIVE-26036) NPE caused by getMTable() in ObjectStore
[ https://issues.apache.org/jira/browse/HIVE-26036?focusedWorklogId=743929=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-743929 ] ASF GitHub Bot logged work on HIVE-26036: - Author: ASF GitHub Bot Created on: 18/Mar/22 14:49 Start Date: 18/Mar/22 14:49 Worklog Time Spent: 10m Work Description: wecharyu commented on pull request #3104: URL: https://github.com/apache/hive/pull/3104#issuecomment-1072364148 @kgyrtkirk @marton-bod Could you pls review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 743929) Time Spent: 0.5h (was: 20m) > NPE caused by getMTable() in ObjectStore > > > Key: HIVE-26036 > URL: https://issues.apache.org/jira/browse/HIVE-26036 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0 >Reporter: Yu Weiqiang >Assignee: Yu Weiqiang >Priority: Minor > Labels: pull-request-available > Attachments: NPE_From_getMTable.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > *Issue* > Some api in ObjectStore invoke getMTable() but not check that if the returned > value is null, which caused the NPE, like addPartitions(), addPartition(), > alterPartition() et. > *Reason* > Such api described above will check that whether the table exists in > HMSHandler (first check), but if the table is dropped by other threads after > the first check, the NPE will happen. > The simple idea is that we can check the table each time we get from > getMTable(). > *Stack example* > !NPE_From_getMTable.png! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26029) Upgrade netty to 4.1.75.Final due to CVE
[ https://issues.apache.org/jira/browse/HIVE-26029?focusedWorklogId=743856=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-743856 ] ASF GitHub Bot logged work on HIVE-26029: - Author: ASF GitHub Bot Created on: 18/Mar/22 14:42 Start Date: 18/Mar/22 14:42 Worklog Time Spent: 10m Work Description: hsnusonic closed pull request #3097: URL: https://github.com/apache/hive/pull/3097 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 743856) Time Spent: 40m (was: 0.5h) > Upgrade netty to 4.1.75.Final due to CVE > > > Key: HIVE-26029 > URL: https://issues.apache.org/jira/browse/HIVE-26029 > Project: Hive > Issue Type: Improvement >Reporter: Yu-Wen Lai >Assignee: Yu-Wen Lai >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-1 > > Time Spent: 40m > Remaining Estimate: 0h > > As title. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-25667) Unify code managing JDBC databases in tests
[ https://issues.apache.org/jira/browse/HIVE-25667?focusedWorklogId=743851=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-743851 ] ASF GitHub Bot logged work on HIVE-25667: - Author: ASF GitHub Bot Created on: 18/Mar/22 14:41 Start Date: 18/Mar/22 14:41 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #2919: URL: https://github.com/apache/hive/pull/2919#issuecomment-1071892104 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 743851) Time Spent: 2h 40m (was: 2.5h) > Unify code managing JDBC databases in tests > --- > > Key: HIVE-25667 > URL: https://issues.apache.org/jira/browse/HIVE-25667 > Project: Hive > Issue Type: Task > Components: Testing Infrastructure >Affects Versions: 4.0.0 >Reporter: Stamatis Zampetakis >Assignee: Mark Bathori >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > Currently there are two class hierarchies managing JDBC databases in tests, > [DatabaseRule|https://github.com/apache/hive/blob/d35de014dd49fdcfe0aacb68e6c587beff6d1dea/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/dbinstall/rules/DatabaseRule.java] > and > [AbstractExternalDB|https://github.com/apache/hive/blob/d35de014dd49fdcfe0aacb68e6c587beff6d1dea/itests/util/src/main/java/org/apache/hadoop/hive/ql/externalDB/AbstractExternalDB.java]. > There are many similarities between these hierarchies and certain parts are > duplicated. > The goal of this JIRA is to refactor the aforementioned hierarchies to reduce > code duplication and improve extensibility. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26048) Missing quotation mark in findReadyToClean query
[
https://issues.apache.org/jira/browse/HIVE-26048?focusedWorklogId=743846=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-743846
]
ASF GitHub Bot logged work on HIVE-26048:
-
Author: ASF GitHub Bot
Created on: 18/Mar/22 14:41
Start Date: 18/Mar/22 14:41
Worklog Time Spent: 10m
Work Description: asinkovits opened a new pull request #3118:
URL: https://github.com/apache/hive/pull/3118
### What changes were proposed in this pull request?
### Why are the changes needed?
To prevent missing column error
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
Manual sanity testing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 743846)
Time Spent: 20m (was: 10m)
> Missing quotation mark in findReadyToClean query
>
>
> Key: HIVE-26048
> URL: https://issues.apache.org/jira/browse/HIVE-26048
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
> Labels: ACID, pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Missing quotation mark causes postgres column error
> {code}
> 2022-03-18T00:53:43,314 ERROR [Thread-651] compactor.Cleaner: Caught an
> exception in the main loop of compactor cleaner, MetaException(message:Unable
> to connect to transaction database org.postgresql.util.PSQLException: ERROR:
> column "cq_retry_retention" does not exist
> Position: 485
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
> at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
> at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:224)
> at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:374)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> )
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:397)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-26028) Upgrade pac4j-saml-opensamlv3 to 4.5.5 due to CVE
[ https://issues.apache.org/jira/browse/HIVE-26028?focusedWorklogId=743814=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-743814 ] ASF GitHub Bot logged work on HIVE-26028: - Author: ASF GitHub Bot Created on: 18/Mar/22 14:37 Start Date: 18/Mar/22 14:37 Worklog Time Spent: 10m Work Description: hsnusonic closed pull request #3096: URL: https://github.com/apache/hive/pull/3096 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 743814) Time Spent: 50m (was: 40m) > Upgrade pac4j-saml-opensamlv3 to 4.5.5 due to CVE > - > > Key: HIVE-26028 > URL: https://issues.apache.org/jira/browse/HIVE-26028 > Project: Hive > Issue Type: Improvement >Reporter: Yu-Wen Lai >Assignee: Yu-Wen Lai >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > As title. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Work logged] (HIVE-26036) NPE caused by getMTable() in ObjectStore
[ https://issues.apache.org/jira/browse/HIVE-26036?focusedWorklogId=743702=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-743702 ] ASF GitHub Bot logged work on HIVE-26036: - Author: ASF GitHub Bot Created on: 18/Mar/22 12:28 Start Date: 18/Mar/22 12:28 Worklog Time Spent: 10m Work Description: wecharyu commented on pull request #3104: URL: https://github.com/apache/hive/pull/3104#issuecomment-1072364148 @kgyrtkirk @marton-bod Could you pls review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 743702) Time Spent: 20m (was: 10m) > NPE caused by getMTable() in ObjectStore > > > Key: HIVE-26036 > URL: https://issues.apache.org/jira/browse/HIVE-26036 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0 >Reporter: Yu Weiqiang >Assignee: Yu Weiqiang >Priority: Minor > Labels: pull-request-available > Attachments: NPE_From_getMTable.png > > Time Spent: 20m > Remaining Estimate: 0h > > *Issue* > Some api in ObjectStore invoke getMTable() but not check that if the returned > value is null, which caused the NPE, like addPartitions(), addPartition(), > alterPartition() et. > *Reason* > Such api described above will check that whether the table exists in > HMSHandler (first check), but if the table is dropped by other threads after > the first check, the NPE will happen. > The simple idea is that we can check the table each time we get from > getMTable(). > *Stack example* > !NPE_From_getMTable.png! -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HIVE-26048) Missing quotation mark in findReadyToClean query
[
https://issues.apache.org/jira/browse/HIVE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Antal Sinkovits updated HIVE-26048:
---
Labels: ACID (was: )
> Missing quotation mark in findReadyToClean query
>
>
> Key: HIVE-26048
> URL: https://issues.apache.org/jira/browse/HIVE-26048
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
> Labels: ACID
>
> Missing quotation mark causes postgres column error
> {code}
> 2022-03-18T00:53:43,314 ERROR [Thread-651] compactor.Cleaner: Caught an
> exception in the main loop of compactor cleaner, MetaException(message:Unable
> to connect to transaction database org.postgresql.util.PSQLException: ERROR:
> column "cq_retry_retention" does not exist
> Position: 485
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
> at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
> at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:224)
> at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:374)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> )
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:397)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HIVE-26048) Missing quotation mark in findReadyToClean query
[
https://issues.apache.org/jira/browse/HIVE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Antal Sinkovits updated HIVE-26048:
---
Status: Patch Available (was: In Progress)
> Missing quotation mark in findReadyToClean query
>
>
> Key: HIVE-26048
> URL: https://issues.apache.org/jira/browse/HIVE-26048
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
> Labels: ACID, pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Missing quotation mark causes postgres column error
> {code}
> 2022-03-18T00:53:43,314 ERROR [Thread-651] compactor.Cleaner: Caught an
> exception in the main loop of compactor cleaner, MetaException(message:Unable
> to connect to transaction database org.postgresql.util.PSQLException: ERROR:
> column "cq_retry_retention" does not exist
> Position: 485
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
> at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
> at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:224)
> at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:374)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> )
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:397)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HIVE-26048) Missing quotation mark in findReadyToClean query
[
https://issues.apache.org/jira/browse/HIVE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-26048:
--
Labels: ACID pull-request-available (was: ACID)
> Missing quotation mark in findReadyToClean query
>
>
> Key: HIVE-26048
> URL: https://issues.apache.org/jira/browse/HIVE-26048
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
> Labels: ACID, pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Missing quotation mark causes postgres column error
> {code}
> 2022-03-18T00:53:43,314 ERROR [Thread-651] compactor.Cleaner: Caught an
> exception in the main loop of compactor cleaner, MetaException(message:Unable
> to connect to transaction database org.postgresql.util.PSQLException: ERROR:
> column "cq_retry_retention" does not exist
> Position: 485
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
> at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
> at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:224)
> at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:374)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> )
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:397)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (HIVE-26048) Missing quotation mark in findReadyToClean query
[
https://issues.apache.org/jira/browse/HIVE-26048?focusedWorklogId=743625=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-743625
]
ASF GitHub Bot logged work on HIVE-26048:
-
Author: ASF GitHub Bot
Created on: 18/Mar/22 08:06
Start Date: 18/Mar/22 08:06
Worklog Time Spent: 10m
Work Description: asinkovits opened a new pull request #3118:
URL: https://github.com/apache/hive/pull/3118
### What changes were proposed in this pull request?
### Why are the changes needed?
To prevent missing column error
### Does this PR introduce _any_ user-facing change?
no
### How was this patch tested?
Manual sanity testing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 743625)
Remaining Estimate: 0h
Time Spent: 10m
> Missing quotation mark in findReadyToClean query
>
>
> Key: HIVE-26048
> URL: https://issues.apache.org/jira/browse/HIVE-26048
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
> Labels: ACID
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Missing quotation mark causes postgres column error
> {code}
> 2022-03-18T00:53:43,314 ERROR [Thread-651] compactor.Cleaner: Caught an
> exception in the main loop of compactor cleaner, MetaException(message:Unable
> to connect to transaction database org.postgresql.util.PSQLException: ERROR:
> column "cq_retry_retention" does not exist
> Position: 485
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
> at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
> at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:224)
> at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:374)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> )
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:397)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work started] (HIVE-26048) Missing quotation mark in findReadyToClean query
[
https://issues.apache.org/jira/browse/HIVE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-26048 started by Antal Sinkovits.
--
> Missing quotation mark in findReadyToClean query
>
>
> Key: HIVE-26048
> URL: https://issues.apache.org/jira/browse/HIVE-26048
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
> Labels: ACID
>
> Missing quotation mark causes postgres column error
> {code}
> 2022-03-18T00:53:43,314 ERROR [Thread-651] compactor.Cleaner: Caught an
> exception in the main loop of compactor cleaner, MetaException(message:Unable
> to connect to transaction database org.postgresql.util.PSQLException: ERROR:
> column "cq_retry_retention" does not exist
> Position: 485
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
> at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
> at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:224)
> at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:374)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> )
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:397)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (HIVE-26048) Missing quotation mark in findReadyToClean query
[
https://issues.apache.org/jira/browse/HIVE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Antal Sinkovits updated HIVE-26048:
---
Affects Version/s: 4.0.0
> Missing quotation mark in findReadyToClean query
>
>
> Key: HIVE-26048
> URL: https://issues.apache.org/jira/browse/HIVE-26048
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
>
> Missing quotation mark causes postgres column error
> {code}
> 2022-03-18T00:53:43,314 ERROR [Thread-651] compactor.Cleaner: Caught an
> exception in the main loop of compactor cleaner, MetaException(message:Unable
> to connect to transaction database org.postgresql.util.PSQLException: ERROR:
> column "cq_retry_retention" does not exist
> Position: 485
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
> at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
> at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:224)
> at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:374)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> )
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:397)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (HIVE-26048) Missing quotation mark in findReadyToClean query
[
https://issues.apache.org/jira/browse/HIVE-26048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Antal Sinkovits reassigned HIVE-26048:
--
> Missing quotation mark in findReadyToClean query
>
>
> Key: HIVE-26048
> URL: https://issues.apache.org/jira/browse/HIVE-26048
> Project: Hive
> Issue Type: Bug
>Reporter: Antal Sinkovits
>Assignee: Antal Sinkovits
>Priority: Major
>
> Missing quotation mark causes postgres column error
> {code}
> 2022-03-18T00:53:43,314 ERROR [Thread-651] compactor.Cleaner: Caught an
> exception in the main loop of compactor cleaner, MetaException(message:Unable
> to connect to transaction database org.postgresql.util.PSQLException: ERROR:
> column "cq_retry_retention" does not exist
> Position: 485
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2433)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2178)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:306)
> at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
> at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307)
> at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293)
> at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270)
> at org.postgresql.jdbc.PgStatement.executeQuery(PgStatement.java:224)
> at
> org.apache.hive.com.zaxxer.hikari.pool.ProxyStatement.executeQuery(ProxyStatement.java:108)
> at
> org.apache.hive.com.zaxxer.hikari.pool.HikariProxyStatement.executeQuery(HikariProxyStatement.java)
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:374)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> )
> at
> org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler.findReadyToClean(CompactionTxnHandler.java:397)
> at org.apache.hadoop.hive.ql.txn.compactor.Cleaner.run(Cleaner.java:146)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (HIVE-18786) NPE in Hive windowing functions
[ https://issues.apache.org/jira/browse/HIVE-18786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17508610#comment-17508610 ] enderyang#tt commented on HIVE-18786: - very useful > NPE in Hive windowing functions > --- > > Key: HIVE-18786 > URL: https://issues.apache.org/jira/browse/HIVE-18786 > Project: Hive > Issue Type: Bug >Affects Versions: 2.3.2, 2.3.3 >Reporter: Michael Bieniosek >Assignee: Dongwook Kwon >Priority: Critical > Labels: patch > Fix For: 4.0.0 > > Attachments: HIVE-18786.1.patch, HIVE-18786.2.patch > > > When I run a Hive query with windowing functions, if there's enough data I > get an NPE. > For example something like this query might break: > select id, created_date, max(created_date) over (partition by id) > latest_created_any from ... > The only workaround I've found is to remove the windowing functions entirely. > The stacktrace looks suspiciously similar to +HIVE-15278+, but I'm in > hive-2.3.2 which appears to have the bugfix applied. > > Caused by: java.lang.RuntimeException: > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while > processing row (tag=0) > at > org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:297) > at > org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:317) > at > org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185) > ... 14 more > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime > Error while processing row (tag=0) > at > org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:365) > at > org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:287) > ... 16 more > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.exec.persistence.PTFRowContainer.first(PTFRowContainer.java:115) > at > org.apache.hadoop.hive.ql.exec.PTFPartition.iterator(PTFPartition.java:114) > at > org.apache.hadoop.hive.ql.udf.ptf.BasePartitionEvaluator.getPartitionAgg(BasePartitionEvaluator.java:200) > at > org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.evaluateFunctionOnPartition(WindowingTableFunction.java:155) > at > org.apache.hadoop.hive.ql.udf.ptf.WindowingTableFunction.iterator(WindowingTableFunction.java:538) > at > org.apache.hadoop.hive.ql.exec.PTFOperator$PTFInvocation.finishPartition(PTFOperator.java:349) > at > org.apache.hadoop.hive.ql.exec.PTFOperator.process(PTFOperator.java:123) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897) > at > org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95) > at > org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:356) -- This message was sent by Atlassian Jira (v8.20.1#820001)
