[
https://issues.apache.org/jira/browse/HIVE-24828?focusedWorklogId=575600&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-575600
]
ASF GitHub Bot logged work on HIVE-24828:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 01/Apr/21 16:28
Start Date: 01/Apr/21 16:28
Worklog Time Spent: 10m
Work Description: klcopp commented on a change in pull request #2073:
URL: https://github.com/apache/hive/pull/2073#discussion_r605790475
##########
File path:
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java
##########
@@ -3646,6 +3649,86 @@ public ShowCompactResponse
showCompact(ShowCompactRequest rqst) throws MetaExcep
}
}
+ /**
+ * We assume this is only called by metadata cache server to know if there
are new base/delta files should be read.
+ * The query filters compactions by state and only returns SUCCEEDED or
READY_FOR_CLEANING compactions because
+ * only these two states means there are new files ready to be read.
+ */
+ @RetrySemantics.ReadOnly
+ public GetLatestCompactionInfoResponse
getLatestCompactionInfo(GetLatestCompactionInfoRequest rqst)
+ throws MetaException {
+ GetLatestCompactionInfoResponse response = new
GetLatestCompactionInfoResponse(new ArrayList<>());
+ Connection dbConn = null;
+ PreparedStatement pst = null;
+ ResultSet rs = null;
+ try {
+ try {
+ dbConn = getDbConn(Connection.TRANSACTION_READ_COMMITTED);
+
+ List<String> params = new ArrayList<>();
+ // This query combines the result sets of SUCCEEDED compactions and
READY_FOR_CLEANING compactions
+ // We also sort the result by CC_ID in descending order so that we can
keep only the latest record
+ // according to the order in result set
+ StringBuilder sb = new StringBuilder()
+ .append("SELECT * FROM (")
Review comment:
No, never mind then
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 575600)
Time Spent: 5h 10m (was: 5h)
> [HMS] Provide new HMS API to return latest committed compaction record for a
> given table
> ----------------------------------------------------------------------------------------
>
> Key: HIVE-24828
> URL: https://issues.apache.org/jira/browse/HIVE-24828
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2
> Reporter: Kishen Das
> Assignee: Yu-Wen Lai
> Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> We need a new HMS API to return the latest committed compaction record for a
> given table. This can be used by a remote cache to decide whether a given
> table's file metadata has been compacted or not, in order to decide whether
> file metadata has to be refreshed from the file system before serving or it
> can serve the current data from the cache.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
