[jira] [Assigned] (HUDI-5300) Optimize initial commit w/ metadata table

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-5300: - Assignee: sivabalan narayanan > Optimize initial commit w/ metadata table >

[jira] [Updated] (HUDI-5299) Fix schema evolution docs to clarify whats supported out of the box

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5299: -- Fix Version/s: 0.13.0 > Fix schema evolution docs to clarify whats supported out of the

[jira] [Updated] (HUDI-5299) Fix schema evolution docs to clarify whats supported out of the box

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5299: -- Labels: docs (was: ) > Fix schema evolution docs to clarify whats supported out of the

[jira] [Assigned] (HUDI-5299) Fix schema evolution docs to clarify whats supported out of the box

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-5299: - Assignee: Bhavani Sudha Saktheeswaran > Fix schema evolution docs to clarify

[jira] [Created] (HUDI-5299) Fix schema evolution docs to clarify whats supported out of the box

2022-11-29 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5299: - Summary: Fix schema evolution docs to clarify whats supported out of the box Key: HUDI-5299 URL: https://issues.apache.org/jira/browse/HUDI-5299 Project:

[jira] [Updated] (HUDI-5298) Optimize WriteStatus storing HoodieRecord

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5298: -- Fix Version/s: 0.13.0 > Optimize WriteStatus storing HoodieRecord >

[jira] [Created] (HUDI-5298) Optimize WriteStatus storing HoodieRecord

2022-11-29 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5298: - Summary: Optimize WriteStatus storing HoodieRecord Key: HUDI-5298 URL: https://issues.apache.org/jira/browse/HUDI-5298 Project: Apache Hudi Issue

[jira] [Updated] (HUDI-5297) Deprecate InternalWriteStatus and re-use WriteStatus

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5297: -- Fix Version/s: 0.13.0 > Deprecate InternalWriteStatus and re-use WriteStatus >

[jira] [Created] (HUDI-5297) Deprecate InternalWriteStatus and re-use WriteStatus

2022-11-29 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5297: - Summary: Deprecate InternalWriteStatus and re-use WriteStatus Key: HUDI-5297 URL: https://issues.apache.org/jira/browse/HUDI-5297 Project: Apache Hudi

[jira] [Assigned] (HUDI-5296) Support disabling schema on read if not required

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-5296: - Assignee: sivabalan narayanan > Support disabling schema on read if not required

[jira] [Updated] (HUDI-5296) Support disabling schema on read if not required

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5296: -- Priority: Blocker (was: Major) > Support disabling schema on read if not required >

[jira] [Updated] (HUDI-5296) Support disabling schema on read if not required

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5296: -- Fix Version/s: 0.12.2 > Support disabling schema on read if not required >

[jira] [Created] (HUDI-5296) Support disabling schema on read if not required

2022-11-29 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5296: - Summary: Support disabling schema on read if not required Key: HUDI-5296 URL: https://issues.apache.org/jira/browse/HUDI-5296 Project: Apache Hudi

[jira] [Comment Edited] (HUDI-5293) Schema on read + reconcile schema fails w/ 0.12.1

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640455#comment-17640455 ] sivabalan narayanan edited comment on HUDI-5293 at 11/29/22 11:19 PM: --

[jira] [Comment Edited] (HUDI-5293) Schema on read + reconcile schema fails w/ 0.12.1

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640455#comment-17640455 ] sivabalan narayanan edited comment on HUDI-5293 at 11/29/22 10:59 PM: --

[jira] [Comment Edited] (HUDI-5293) Schema on read + reconcile schema fails w/ 0.12.1

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640455#comment-17640455 ] sivabalan narayanan edited comment on HUDI-5293 at 11/29/22 10:58 PM: --

[jira] [Comment Edited] (HUDI-5293) Schema on read + reconcile schema fails w/ 0.12.1

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640455#comment-17640455 ] sivabalan narayanan edited comment on HUDI-5293 at 11/29/22 10:55 PM: --

[jira] [Commented] (HUDI-5293) Schema on read + reconcile schema fails w/ 0.12.1

2022-11-29 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640455#comment-17640455 ] sivabalan narayanan commented on HUDI-5293: --- steps to reproduce {code:java} import

[jira] [Created] (HUDI-5293) Schema on read + reconcile schema fails w/ 0.12.1

2022-11-28 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5293: - Summary: Schema on read + reconcile schema fails w/ 0.12.1 Key: HUDI-5293 URL: https://issues.apache.org/jira/browse/HUDI-5293 Project: Apache Hudi

[jira] [Updated] (HUDI-5293) Schema on read + reconcile schema fails w/ 0.12.1

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5293: -- Fix Version/s: 0.12.2 > Schema on read + reconcile schema fails w/ 0.12.1 >

[jira] [Commented] (HUDI-4919) Sql MERGE INTO incurs too much memory overhead

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640363#comment-17640363 ] sivabalan narayanan commented on HUDI-4919: --- We have also put out

[jira] [Closed] (HUDI-4919) Sql MERGE INTO incurs too much memory overhead

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4919. - Resolution: Done > Sql MERGE INTO incurs too much memory overhead >

[jira] [Commented] (HUDI-4919) Sql MERGE INTO incurs too much memory overhead

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640360#comment-17640360 ] sivabalan narayanan commented on HUDI-4919: --- w/ dataproc, users have to set

[jira] [Created] (HUDI-5288) Optimize drop duplicates by avoiding index look up twice

2022-11-28 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5288: - Summary: Optimize drop duplicates by avoiding index look up twice Key: HUDI-5288 URL: https://issues.apache.org/jira/browse/HUDI-5288 Project: Apache Hudi

[jira] [Updated] (HUDI-5288) Optimize drop duplicates by avoiding index look up twice

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5288: -- Story Points: 1 > Optimize drop duplicates by avoiding index look up twice >

[jira] [Updated] (HUDI-5288) Optimize drop duplicates by avoiding index look up twice

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5288: -- Fix Version/s: 0.13.0 > Optimize drop duplicates by avoiding index look up twice >

[jira] [Assigned] (HUDI-5288) Optimize drop duplicates by avoiding index look up twice

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-5288: - Assignee: sivabalan narayanan > Optimize drop duplicates by avoiding index look

[jira] [Assigned] (HUDI-5254) Revisit ExpressionPayload to avoid parsing Avro schemas for every record

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-5254: - Assignee: Jonathan Vexler (was: sivabalan narayanan) > Revisit

[jira] [Updated] (HUDI-5242) Do not fail Meta sync in Deltastreamer when inline table service fails

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5242: -- Story Points: 2 > Do not fail Meta sync in Deltastreamer when inline table service

[jira] [Updated] (HUDI-5231) Address checkstyle warnings while building hudi

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5231: -- Story Points: 4 > Address checkstyle warnings while building hudi >

[jira] [Updated] (HUDI-5231) Address checkstyle warnings while building hudi

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5231: -- Status: In Progress (was: Open) > Address checkstyle warnings while building hudi >

[jira] [Updated] (HUDI-5269) Enhancing core user flow tests for spark-sql writes

2022-11-28 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5269: -- Story Points: 6 > Enhancing core user flow tests for spark-sql writes >

[jira] [Updated] (HUDI-5163) Failure handling w/ spark ds write failures

2022-11-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5163: -- Sprint: 2022/11/01, 2022/11/29 (was: 2022/11/01) > Failure handling w/ spark ds write

[jira] [Updated] (HUDI-4991) Make sure DeltaStreamer passes SSL key/truststore configs connecting to Schema Registry

2022-11-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4991: -- Sprint: 2022/10/04, 2022/10/18, 2022/11/01, 2022/11/29 (was: 2022/10/04, 2022/10/18,

[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios

2022-11-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4631: -- Sprint: 2022/10/04, 2022/10/18, 2022/11/01 (was: 2022/10/04, 2022/10/18, 2022/11/01,

[jira] [Updated] (HUDI-4878) Fix incremental cleaning for clean based on LATEST_FILE_VERSIONS

2022-11-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4878: -- Sprint: 2022/09/05, 2022/10/04, 2022/10/18, 2022/11/01, 2022/11/29 (was: 2022/09/05,

[jira] [Updated] (HUDI-5163) Failure handling w/ spark ds write failures

2022-11-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5163: -- Sprint: 2022/11/01 (was: 2022/11/01, 2022/11/15) > Failure handling w/ spark ds write

[jira] [Updated] (HUDI-5012) Fix clean planning for very large partitions

2022-11-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5012: -- Sprint: 2022/10/04, 2022/10/18, 2022/11/01, 2022/11/29 (was: 2022/10/04, 2022/10/18,

[jira] [Updated] (HUDI-4991) Make sure DeltaStreamer passes SSL key/truststore configs connecting to Schema Registry

2022-11-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4991: -- Sprint: 2022/10/04, 2022/10/18, 2022/11/01 (was: 2022/10/04, 2022/10/18, 2022/11/01,

[jira] [Updated] (HUDI-4631) Enhance retries for failed writes w/ write conflicts in a multi writer scenarios

2022-11-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4631: -- Sprint: 2022/10/04, 2022/10/18, 2022/11/01, 2022/11/29 (was: 2022/10/04, 2022/10/18,

[jira] [Updated] (HUDI-5012) Fix clean planning for very large partitions

2022-11-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5012: -- Sprint: 2022/10/04, 2022/10/18, 2022/11/01 (was: 2022/10/04, 2022/10/18, 2022/11/01,

[jira] [Updated] (HUDI-4878) Fix incremental cleaning for clean based on LATEST_FILE_VERSIONS

2022-11-24 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4878: -- Sprint: 2022/09/05, 2022/10/04, 2022/10/18, 2022/11/01 (was: 2022/09/05, 2022/10/04,

[jira] [Updated] (HUDI-5028) Handle conficting writes for multi-writers with OCC enabled

2022-11-23 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5028: -- Sprint: 2022/10/18, 2022/11/01 (was: 2022/10/18, 2022/11/01, 2022/11/15) > Handle

[jira] [Updated] (HUDI-5028) Handle conficting writes for multi-writers with OCC enabled

2022-11-23 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5028: -- Sprint: 2022/10/18, 2022/11/01, 2022/11/29 (was: 2022/10/18, 2022/11/01) > Handle

[jira] [Commented] (HUDI-4919) Sql MERGE INTO incurs too much memory overhead

2022-11-23 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17638011#comment-17638011 ] sivabalan narayanan commented on HUDI-4919: --- User tried 0.12.1 and not seeing the perf issue w/

[jira] [Updated] (HUDI-5245) Honor pruned partitions while looking up in col stats partition in MDT

2022-11-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5245: -- Fix Version/s: 0.13.0 > Honor pruned partitions while looking up in col stats partition

[jira] [Updated] (HUDI-5245) Honor pruned partitions while looking up in col stats partition in MDT

2022-11-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5245: -- Priority: Critical (was: Major) > Honor pruned partitions while looking up in col

[jira] [Updated] (HUDI-5245) Honor pruned partitions while looking up in col stats partition in MDT

2022-11-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5245: -- Description: When looking up in col stats for data skipping, we are passing in only the

[jira] [Updated] (HUDI-5245) Honor pruned partitions while looking up in col stats partition in MDT

2022-11-19 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5245: -- Description: When looking up in col stats for data skipping, we are passing in only the

[jira] [Created] (HUDI-5245) Honor pruned partitions while looking up in col stats partition in MDT

2022-11-19 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5245: - Summary: Honor pruned partitions while looking up in col stats partition in MDT Key: HUDI-5245 URL: https://issues.apache.org/jira/browse/HUDI-5245

[jira] [Updated] (HUDI-5231) Address checkstyle warnings while building hudi

2022-11-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5231: -- Priority: Blocker (was: Major) > Address checkstyle warnings while building hudi >

[jira] [Assigned] (HUDI-5231) Address checkstyle warnings while building hudi

2022-11-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan reassigned HUDI-5231: - Assignee: Jonathan Vexler > Address checkstyle warnings while building hudi >

[jira] [Updated] (HUDI-5231) Address checkstyle warnings while building hudi

2022-11-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5231: -- Fix Version/s: 0.12.2 > Address checkstyle warnings while building hudi >

[jira] [Updated] (HUDI-5231) Address checkstyle warnings while building hudi

2022-11-16 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5231: -- Sprint: 2022/11/15 > Address checkstyle warnings while building hudi >

[jira] [Created] (HUDI-5231) Address checkstyle warnings while building hudi

2022-11-16 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5231: - Summary: Address checkstyle warnings while building hudi Key: HUDI-5231 URL: https://issues.apache.org/jira/browse/HUDI-5231 Project: Apache Hudi

[jira] [Closed] (HUDI-4986) Enhance hudi integ test readme for multi-writer tests

2022-11-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4986. - Resolution: Fixed > Enhance hudi integ test readme for multi-writer tests >

[jira] [Updated] (HUDI-5157) Duplicate partition path for chained hudi tables.

2022-11-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5157: -- Status: In Progress (was: Open) > Duplicate partition path for chained hudi tables. >

[jira] [Updated] (HUDI-5157) Duplicate partition path for chained hudi tables.

2022-11-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5157: -- Status: Patch Available (was: In Progress) > Duplicate partition path for chained hudi

[jira] [Updated] (HUDI-4919) Sql MERGE INTO incurs too much memory overhead

2022-11-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4919: -- Description: When using spark-sql MERGE INTO, memory requirement shoots up. To merge

[jira] [Updated] (HUDI-4919) Sql MERGE INTO incurs too much memory overhead

2022-11-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4919: -- Description: When using spark-sql MERGE INTO, memory requirement shoots up. To merge

[jira] [Updated] (HUDI-4919) Sql MERGE INTO incurs too much memory overhead

2022-11-15 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4919: -- Description: When using spark-sql MERGE INTO, memory requirement shoots up. To merge

[jira] [Updated] (HUDI-4919) Sql MERGE INTO incurs too much memory overhead

2022-11-11 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4919: -- Priority: Critical (was: Major) > Sql MERGE INTO incurs too much memory overhead >

[jira] [Updated] (HUDI-4919) Sql MERGE INTO incurs too much memory overhead

2022-11-11 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4919: -- Description: When using spark-sql MERGE INTO, memory requirement shoots up. To merge

[jira] [Updated] (HUDI-4919) Sql MERGE INTO incurs too much memory overhead

2022-11-11 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4919: -- Description: When using spark-sql MERGE INTO, memory requirement shoots up. To merge

[jira] [Created] (HUDI-5193) Enhancing core user flow tests for spark-datasource writes

2022-11-10 Thread sivabalan narayanan (Jira)
sivabalan narayanan created HUDI-5193: - Summary: Enhancing core user flow tests for spark-datasource writes Key: HUDI-5193 URL: https://issues.apache.org/jira/browse/HUDI-5193 Project: Apache Hudi

[jira] [Closed] (HUDI-2613) Fix usages of RealtimeSplit to use the new getDeltaLogFileStatus

2022-11-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-2613. - Resolution: Done > Fix usages of RealtimeSplit to use the new getDeltaLogFileStatus >

[jira] [Commented] (HUDI-2613) Fix usages of RealtimeSplit to use the new getDeltaLogFileStatus

2022-11-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631217#comment-17631217 ] sivabalan narayanan commented on HUDI-2613: --- Looks like we got this covered already. closing it

[jira] [Updated] (HUDI-5171) Ensure validateTableConfig also checks for partition path field value switch

2022-11-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5171: -- Story Points: 1 > Ensure validateTableConfig also checks for partition path field value

[jira] [Closed] (HUDI-5048) add CopyToTempView support

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5048. - Resolution: Fixed > add CopyToTempView support > -- > >

[jira] [Closed] (HUDI-5085) When a flink job has multiple sink tables, the index loading status is abnormal.

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5085. - Resolution: Fixed > When a flink job has multiple sink tables, the index loading status

[jira] [Updated] (HUDI-5048) add CopyToTempView support

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5048: -- Fix Version/s: 0.12.2 > add CopyToTempView support > -- > >

[jira] [Closed] (HUDI-5089) Refactor HoodieCommitMetadata deserialization

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5089. - Resolution: Fixed > Refactor HoodieCommitMetadata deserialization >

[jira] [Closed] (HUDI-5058) The primary key cannot be empty when Flink reads an error from the hudi table

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5058. - Resolution: Fixed > The primary key cannot be empty when Flink reads an error from the

[jira] [Closed] (HUDI-4946) merge into with no preCombineField has dup row in only insert

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4946. - Resolution: Fixed > merge into with no preCombineField has dup row in only insert >

[jira] [Closed] (HUDI-5072) Extract transform duplicate code

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5072. - Resolution: Fixed > Extract transform duplicate code > >

[jira] [Updated] (HUDI-5090) throw runtime Exception when flink streming job checkpoint abort

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5090: -- Fix Version/s: 0.12.2 > throw runtime Exception when flink streming job checkpoint

[jira] [Closed] (HUDI-5090) throw runtime Exception when flink streming job checkpoint abort

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5090. - Resolution: Fixed > throw runtime Exception when flink streming job checkpoint abort >

[jira] [Updated] (HUDI-4946) merge into with no preCombineField has dup row in only insert

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4946: -- Fix Version/s: 0.12.2 > merge into with no preCombineField has dup row in only insert >

[jira] [Updated] (HUDI-5072) Extract transform duplicate code

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5072: -- Fix Version/s: 0.12.2 > Extract transform duplicate code >

[jira] [Updated] (HUDI-5072) Extract transform duplicate code

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5072: -- Status: Open (was: In Progress) > Extract transform duplicate code >

[jira] [Closed] (HUDI-5000) Support schema evolution for Hive

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5000. - Resolution: Fixed > Support schema evolution for Hive > -

[jira] [Updated] (HUDI-1570) Add Avg record size in commit metadata

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1570: -- Fix Version/s: 0.12.2 > Add Avg record size in commit metadata >

[jira] [Updated] (HUDI-5035) Remove deprecated API usage in SparkPreCommitValidator#validate

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5035: -- Fix Version/s: 0.12.2 > Remove deprecated API usage in SparkPreCommitValidator#validate

[jira] [Updated] (HUDI-5035) Remove deprecated API usage in SparkPreCommitValidator#validate

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-5035: -- Status: Open (was: In Progress) > Remove deprecated API usage in

[jira] [Closed] (HUDI-5035) Remove deprecated API usage in SparkPreCommitValidator#validate

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5035. - Resolution: Fixed > Remove deprecated API usage in SparkPreCommitValidator#validate >

[jira] [Closed] (HUDI-5102) source operator(monitor and reader) support user uid

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5102. - Resolution: Fixed > source operator(monitor and reader) support user uid >

[jira] [Closed] (HUDI-4839) rocksdbjni is not compatible with apple silicon

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4839. - Resolution: Fixed > rocksdbjni is not compatible with apple silicon >

[jira] [Closed] (HUDI-5057) Fix msck repair hudi table

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5057. - Fix Version/s: 0.12.2 Resolution: Fixed > Fix msck repair hudi table >

[jira] [Updated] (HUDI-4839) rocksdbjni is not compatible with apple silicon

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4839: -- Fix Version/s: 0.12.2 > rocksdbjni is not compatible with apple silicon >

[jira] [Closed] (HUDI-4741) Deadlock when restarting failed TM in AbstractStreamWriteFunction

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4741. - Resolution: Fixed > Deadlock when restarting failed TM in AbstractStreamWriteFunction >

[jira] [Closed] (HUDI-5105) Add Call show_commit_extra_metadata for spark sql

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5105. - Fix Version/s: 0.12.2 Resolution: Fixed > Add Call show_commit_extra_metadata for

[jira] [Updated] (HUDI-4293) Implement Create/Drop/Show/Refresh Index Command for Secondary Index

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-4293: -- Status: Open (was: In Progress) > Implement Create/Drop/Show/Refresh Index Command for

[jira] [Closed] (HUDI-5107) Fix hadoop config in DirectWriteMarkers, HoodieFlinkEngineContext and StreamerUtil are not consistent issue

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5107. - Resolution: Fixed > Fix hadoop config in DirectWriteMarkers, HoodieFlinkEngineContext and

[jira] [Closed] (HUDI-4293) Implement Create/Drop/Show/Refresh Index Command for Secondary Index

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-4293. - Resolution: Fixed > Implement Create/Drop/Show/Refresh Index Command for Secondary Index

[jira] [Closed] (HUDI-5124) Fix HoodieInternalRowFileWriter#canWrite error return tag

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5124. - Resolution: Fixed > Fix HoodieInternalRowFileWriter#canWrite error return tag >

[jira] [Resolved] (HUDI-3963) [RFC-53] Use Lock-Free Message Queue Improving Hoodie Writing Efficiency

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan resolved HUDI-3963. --- > [RFC-53] Use Lock-Free Message Queue Improving Hoodie Writing Efficiency >

[jira] [Closed] (HUDI-3963) [RFC-53] Use Lock-Free Message Queue Improving Hoodie Writing Efficiency

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-3963. - Resolution: Fixed > [RFC-53] Use Lock-Free Message Queue Improving Hoodie Writing

[jira] [Closed] (HUDI-5076) Non serializable path used with engineContext with metadata table initialization

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5076. - Resolution: Fixed > Non serializable path used with engineContext with metadata table >

[jira] [Closed] (HUDI-5045) Add tests to integ test to test bulk_insert followed by upsert

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5045. - Fix Version/s: 0.12.2 Assignee: sivabalan narayanan Resolution: Fixed >

[jira] [Closed] (HUDI-5147) Flink data skipping doesn't work when HepPlanner calls copy() on HoodieTableSource

2022-11-08 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan closed HUDI-5147. - Resolution: Fixed > Flink data skipping doesn't work when HepPlanner calls copy() on >

<    5   6   7   8   9   10   11   12   13   14   >