[jira] [Updated] (HIVE-27819) Iceberg: Upgrade iceberg version to 1.4.2
[ https://issues.apache.org/jira/browse/HIVE-27819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao updated HIVE-27819: -- Summary: Iceberg: Upgrade iceberg version to 1.4.2 (was: Iceberg: Upgrade iceberg version to 1.4.1) > Iceberg: Upgrade iceberg version to 1.4.2 > - > > Key: HIVE-27819 > URL: https://issues.apache.org/jira/browse/HIVE-27819 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > > Iceberg latest version 1.4.2 has been released out. we need upgrade iceberg > depdency from 1.3.9 to 1.4.2. Meantime, we should port some Hive catalog > changes from Iceberg repo to Hive repo. > [https://iceberg.apache.org/releases/#142-release] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27819) Iceberg: Upgrade iceberg version to 1.4.2
[ https://issues.apache.org/jira/browse/HIVE-27819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao updated HIVE-27819: -- Description: Iceberg latest version 1.4.2 has been released out. we need upgrade iceberg depdency from 1.3.0 to 1.4.2. Meantime, we should port some Hive catalog changes from Iceberg repo to Hive repo. [https://iceberg.apache.org/releases/#142-release] was: Iceberg latest version 1.4.2 has been released out. we need upgrade iceberg depdency from 1.3.9 to 1.4.2. Meantime, we should port some Hive catalog changes from Iceberg repo to Hive repo. [https://iceberg.apache.org/releases/#142-release] > Iceberg: Upgrade iceberg version to 1.4.2 > - > > Key: HIVE-27819 > URL: https://issues.apache.org/jira/browse/HIVE-27819 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > > Iceberg latest version 1.4.2 has been released out. we need upgrade iceberg > depdency from 1.3.0 to 1.4.2. Meantime, we should port some Hive catalog > changes from Iceberg repo to Hive repo. > [https://iceberg.apache.org/releases/#142-release] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27819) Iceberg: Upgrade iceberg version to 1.4.1
[ https://issues.apache.org/jira/browse/HIVE-27819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao updated HIVE-27819: -- Description: Iceberg latest version 1.4.2 has been released out. we need upgrade iceberg depdency from 1.3.9 to 1.4.2. Meantime, we should port some Hive catalog changes from Iceberg repo to Hive repo. [https://iceberg.apache.org/releases/#142-release] was: Iceberg latest version 1.4.2 has been released out. we need upgrade iceberg depdency from 1.3.9 to 1.4.2. Meantime, we should port some Hive catalog changes from Iceberg repo to Hive repo. [https://iceberg.apache.org/releases/#142-release|https://iceberg.apache.org/releases/#141-release] > Iceberg: Upgrade iceberg version to 1.4.1 > - > > Key: HIVE-27819 > URL: https://issues.apache.org/jira/browse/HIVE-27819 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > > Iceberg latest version 1.4.2 has been released out. we need upgrade iceberg > depdency from 1.3.9 to 1.4.2. Meantime, we should port some Hive catalog > changes from Iceberg repo to Hive repo. > [https://iceberg.apache.org/releases/#142-release] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27819) Iceberg: Upgrade iceberg version to 1.4.1
[ https://issues.apache.org/jira/browse/HIVE-27819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao updated HIVE-27819: -- Description: Iceberg latest version 1.4.2 has been released out. we need upgrade iceberg depdency from 1.3.9 to 1.4.2. Meantime, we should port some Hive catalog changes from Iceberg repo to Hive repo. [https://iceberg.apache.org/releases/#142-release|https://iceberg.apache.org/releases/#141-release] was: Iceberg latest version 1.4.1 has been released out. we need upgrade iceberg depdency from 1.3.1 to 1.4.1. Meantime, we should port some Hive catalog changes from Iceberg repo to Hive repo. [https://iceberg.apache.org/releases/#141-release] > Iceberg: Upgrade iceberg version to 1.4.1 > - > > Key: HIVE-27819 > URL: https://issues.apache.org/jira/browse/HIVE-27819 > Project: Hive > Issue Type: Improvement > Components: Iceberg integration >Reporter: zhangbutao >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > > Iceberg latest version 1.4.2 has been released out. we need upgrade iceberg > depdency from 1.3.9 to 1.4.2. Meantime, we should port some Hive catalog > changes from Iceberg repo to Hive repo. > [https://iceberg.apache.org/releases/#142-release|https://iceberg.apache.org/releases/#141-release] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-24815) Remove "IDXS" Table from Metastore Schema
[ https://issues.apache.org/jira/browse/HIVE-24815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam resolved HIVE-24815. -- Fix Version/s: 4.0.0 Resolution: Fixed Fix has been merged to master. [~rtrivedi12] do you plan on porting this change to branch-3? If so, please create a new jira as this change would not work as-is on branch-3 due to the schema file changes. > Remove "IDXS" Table from Metastore Schema > - > > Key: HIVE-24815 > URL: https://issues.apache.org/jira/browse/HIVE-24815 > Project: Hive > Issue Type: Improvement > Components: Metastore, Standalone Metastore >Affects Versions: 3.1.0, 3.0.0, 3.1.1, 3.1.2, 3.2.0, 4.0.0 >Reporter: Hunter Logan >Assignee: Riju Trivedi >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > In Hive 3 the rarely used "INDEXES" was removed from the DDL > https://issues.apache.org/jira/browse/HIVE-18448 > > There are a few issues here: > # The Standalone-Metastore schema for Hive 3+ all include the "IDXS" table, > which has no function. > ** > [https://github.com/apache/hive/tree/master/standalone-metastore/metastore-server/src/main/sql/mysql] > # The upgrade schemas from 2.x -> 3.x do not do any cleanup of the IDXS table > ** If a user used the "INDEXES" feature in 2.x and then upgrades their > metastore to 3.x+ they cannot drop any table that has an index on it due to > "IDXS_FK1" constraint since the TBLS entry is referenced in the IDXS table > ** Since INDEX is no longer in the DDL they cannot run any command from Hive > to drop the index. > ** Users can manually connect to the metastore and either drop the IDXS > table or the foreign key constraint > > Since indexes provide no benefits in Hive 3+ it should be fine to drop them > completely in the schema upgrade scripts. At the very least the 2.x -> 3.x+ > scripts should drop the fk constraint. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27499) Add optional table name filter to the HMS notification fetch API
[
https://issues.apache.org/jira/browse/HIVE-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sai Hemanth Gantasala resolved HIVE-27499.
--
Fix Version/s: 4.0.0-beta-1
Resolution: Fixed
> Add optional table name filter to the HMS notification fetch API
>
>
> Key: HIVE-27499
> URL: https://issues.apache.org/jira/browse/HIVE-27499
> Project: Hive
> Issue Type: New Feature
>Reporter: Quanlong Huang
>Assignee: Sai Hemanth Gantasala
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-beta-1
>
>
> The current notification fetch API does not support filters on table name:
> {code:java}
> struct NotificationEventRequest {
> 1: required i64 lastEvent,
> 2: optional i32 maxEvents,
> 3: optional list eventTypeSkipList,
> }
> NotificationEventResponse get_next_notification(1:NotificationEventRequest
> rqst)
> {code}
> Engines that cache HMS metadata and file metadata might want to know whether
> they have synced to the latest events on some given tables.
> In order to efficiently fetch events specific to a table, we should enhance
> the notification fetch API to include an optional table name in the request
> so that only events for that given table are returned. The NOTIFICATION_LOG
> table in the HMS should also have an index on table_name so that API calls
> with the filter do not do a table scan.
> With the index added, we can also extend the NotificationEventsCountRequest
> with an optional table name, and extend the response to carry the largest
> event id on that table. Currently they are:
> {code:java}
> struct NotificationEventsCountRequest {
> 1: required i64 fromEventId,
> 2: required string dbName,
> 3: optional string catName,
> 4: optional i64 toEventId,
> 5: optional i64 limit
> }
> struct NotificationEventsCountResponse {
> 1: required i64 eventsCount,
> }
> NotificationEventsCountResponse
> get_notification_events_count(1:NotificationEventsCountRequest rqst)
> {code}
> Similar API for multiple tables (given a table lists) will also be helpful
> since a query usually involves several tables.
> CC [~hemanth619], [~ngangam]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27499) Add optional table name filter to the HMS notification fetch API
[
https://issues.apache.org/jira/browse/HIVE-27499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17782221#comment-17782221
]
Sai Hemanth Gantasala commented on HIVE-27499:
--
The patch has been merged into the master branch. Thanks for the review
[~ngangam], [~dengzh]
> Add optional table name filter to the HMS notification fetch API
>
>
> Key: HIVE-27499
> URL: https://issues.apache.org/jira/browse/HIVE-27499
> Project: Hive
> Issue Type: New Feature
>Reporter: Quanlong Huang
>Assignee: Sai Hemanth Gantasala
>Priority: Major
> Labels: pull-request-available
>
> The current notification fetch API does not support filters on table name:
> {code:java}
> struct NotificationEventRequest {
> 1: required i64 lastEvent,
> 2: optional i32 maxEvents,
> 3: optional list eventTypeSkipList,
> }
> NotificationEventResponse get_next_notification(1:NotificationEventRequest
> rqst)
> {code}
> Engines that cache HMS metadata and file metadata might want to know whether
> they have synced to the latest events on some given tables.
> In order to efficiently fetch events specific to a table, we should enhance
> the notification fetch API to include an optional table name in the request
> so that only events for that given table are returned. The NOTIFICATION_LOG
> table in the HMS should also have an index on table_name so that API calls
> with the filter do not do a table scan.
> With the index added, we can also extend the NotificationEventsCountRequest
> with an optional table name, and extend the response to carry the largest
> event id on that table. Currently they are:
> {code:java}
> struct NotificationEventsCountRequest {
> 1: required i64 fromEventId,
> 2: required string dbName,
> 3: optional string catName,
> 4: optional i64 toEventId,
> 5: optional i64 limit
> }
> struct NotificationEventsCountResponse {
> 1: required i64 eventsCount,
> }
> NotificationEventsCountResponse
> get_notification_events_count(1:NotificationEventsCountRequest rqst)
> {code}
> Similar API for multiple tables (given a table lists) will also be helpful
> since a query usually involves several tables.
> CC [~hemanth619], [~ngangam]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27846) Tests under hive-unit module are not running
[
https://issues.apache.org/jira/browse/HIVE-27846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27846:
--
Labels: pull-request-available (was: )
> Tests under hive-unit module are not running
>
>
> Key: HIVE-27846
> URL: https://issues.apache.org/jira/browse/HIVE-27846
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Labels: pull-request-available
>
> All the tests under hive-unit module are not running currently in master
> neither locally nor in Jenkins CI.
> {noformat}
> mvn test -pl itests/hive-unit -Pitests
> {noformat}
> {noformat}
> [INFO] — maven-surefire-plugin:3.0.0-M4:test (default-test) @ hive-it-unit —
> [INFO]
> [INFO] ---
> [INFO] T E S T S
> [INFO] ---
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
>
> [INFO] BUILD SUCCESS
> [INFO]
>
> [INFO] Total time: 10.444 s
> [INFO] Finished at: 2023-11-02T12:21:51+01:00
> [INFO]
>
> {noformat}
> The problem is caused by HIVE-27757 as it can be seen by comparing the test
> reports in master:
> * http://ci.hive.apache.org/job/hive-precommit/job/master/1907/testReport/
> * http://ci.hive.apache.org/job/hive-precommit/job/master/1906/testReport/
> Observe that when HIVE-27757 was merged the total number of tests dropped
> from 49,200 (1906) to 47,663 (1907) reducing the total number of tests by
> 1537. Comparing the two test reports together it becomes clear that all the
> tests under hive-unit module are now missing.
> The problem seems to be caused by the new junit-jupiter-engine dependency
> that was added in hive-unit module.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-27846) Tests under hive-unit module are not running
Stamatis Zampetakis created HIVE-27846:
--
Summary: Tests under hive-unit module are not running
Key: HIVE-27846
URL: https://issues.apache.org/jira/browse/HIVE-27846
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
All the tests under hive-unit module are not running currently in master
neither locally nor in Jenkins CI.
{noformat}
mvn test -pl itests/hive-unit -Pitests
{noformat}
{noformat}
[INFO] — maven-surefire-plugin:3.0.0-M4:test (default-test) @ hive-it-unit —
[INFO]
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 10.444 s
[INFO] Finished at: 2023-11-02T12:21:51+01:00
[INFO]
{noformat}
The problem is caused by HIVE-27757 as it can be seen by comparing the test
reports in master:
* http://ci.hive.apache.org/job/hive-precommit/job/master/1907/testReport/
* http://ci.hive.apache.org/job/hive-precommit/job/master/1906/testReport/
Observe that when HIVE-27757 was merged the total number of tests dropped from
49,200 (1906) to 47,663 (1907) reducing the total number of tests by 1537.
Comparing the two test reports together it becomes clear that all the tests
under hive-unit module are now missing.
The problem seems to be caused by the new junit-jupiter-engine dependency that
was added in hive-unit module.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
