nateradtke opened a new pull request #3053:
URL: https://github.com/apache/hudi/pull/3053


   ## What is the purpose of the pull request
   
   Addressing [HUDI-1932](https://issues.apache.org/jira/browse/HUDI-1932); 
only update the last commit timestamp on the Hive table when the table schema 
has changed or a partition is created/updated.
   
   When using AWS Glue Data Catalog as the metastore for Hive this will ensure 
that table versions are substantive (including schema and/or partition 
changes). Prior to this change when a Hive sync is performed without schema or 
partition changes the table in the Glue Data Catalog would have a new version 
published with the only change being the timestamp property.
   
   ## Brief change log
   
   - Update Hive table's `last_commit_time_sync` property only when the table 
schema or a partition has changed
   
   ## Verify this pull request
   
   This change added tests and can be verified as follows:
   
   - Added `TestHiveSyncTool#testSyncWithoutDiffs` unit test to verify the 
change.
   - Verified existing tests in `TestHiveSyncTool` remained passing.
   
   ## Committer checklist
   
    - [ ] Has a corresponding JIRA in PR title & commit 
(https://issues.apache.org/jira/browse/HUDI-1932)
    
    - [ ] Commit message is descriptive of the change
    
    - [ ] CI is green
   
    - [ ] Necessary doc changes done or have another open PR
          
    - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to