[jira] [Updated] (HUDI-6964) Fix HoodieStreamer writing json value issue

2023-10-20 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-6964: --- Summary: Fix HoodieStreamer writing json value issue (was: Fix writing json value issue) > Fix

[jira] [Updated] (HUDI-6964) Fix writing json value issue

2023-10-20 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-6964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-6964: --- Summary: Fix writing json value issue (was: Fix writing json format issue) > Fix writing json value

[jira] [Created] (HUDI-6964) Fix writing json format issue

2023-10-20 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-6964: -- Summary: Fix writing json format issue Key: HUDI-6964 URL: https://issues.apache.org/jira/browse/HUDI-6964 Project: Apache Hudi Issue Type: Bug

[jira] [Closed] (HUDI-3518) Make HiveSchemaProvider support AWS Glue Catalog

2023-03-29 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3518. -- Resolution: Invalid > Make HiveSchemaProvider support AWS Glue Catalog >

[jira] [Closed] (HUDI-5470) DFSPropertiesConfiguration support read include confs from diferenet DFS path

2023-03-29 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-5470. -- Resolution: Abandoned > DFSPropertiesConfiguration support read include confs from diferenet DFS path >

[jira] [Closed] (HUDI-3305) Drop deprecated util HDFSParquetImporter

2023-03-29 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3305. -- Resolution: Abandoned > Drop deprecated util HDFSParquetImporter >

[jira] [Closed] (HUDI-4894) Fix ClassCastException when using fixed type defining decimal column

2023-03-29 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-4894. -- Fix Version/s: (was: 0.13.1) Resolution: Abandoned > Fix ClassCastException when using fixed

[jira] [Updated] (HUDI-5470) DFSPropertiesConfiguration support read include confs from diferenet DFS path

2022-12-23 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-5470: --- Description: For now, if set `include=xxx.properties` in our props file, we have to make sure these

[jira] [Created] (HUDI-5470) DFSPropertiesConfiguration support read include confs from diferenet DFS path

2022-12-23 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-5470: -- Summary: DFSPropertiesConfiguration support read include confs from diferenet DFS path Key: HUDI-5470 URL: https://issues.apache.org/jira/browse/HUDI-5470 Project:

[jira] [Closed] (HUDI-1904) Make SchemaProvider spark free and move it to hudi-client-common

2022-10-11 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-1904. -- Resolution: Abandoned > Make SchemaProvider spark free and move it to hudi-client-common >

[jira] [Closed] (HUDI-4903) TestHoodieLogFormat has a minor typo

2022-09-23 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-4903. -- Fix Version/s: 0.13.0 Resolution: Fixed > TestHoodieLogFormat has a minor typo >

[jira] [Closed] (HUDI-3523) Introduce AddColumnSchemaPostProcessor to support add columns to the end of a schema

2022-09-23 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3523. -- Resolution: Fixed > Introduce AddColumnSchemaPostProcessor to support add columns to the end of a >

[jira] [Updated] (HUDI-4894) Fix ClassCastException when using fixed type defining decimal column

2022-09-21 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-4894: --- Description: schema for decimal column : {code:java} {     "name": "column_name",     "type":

[jira] [Updated] (HUDI-4894) Fix ClassCastException when using fixed type defining decimal column

2022-09-21 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-4894: --- Description: schema for decimal column : {     "name": "column_name",     "type": ["null",{    

[jira] [Updated] (HUDI-4894) Fix ClassCastException when using fixed type defining decimal column

2022-09-21 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-4894: --- Description: schema for decimal column : {     "name": "column_name",     "type": ["null", {        

[jira] [Created] (HUDI-4894) Fix ClassCastException when using fixed type defining decimal column

2022-09-21 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-4894: -- Summary: Fix ClassCastException when using fixed type defining decimal column Key: HUDI-4894 URL: https://issues.apache.org/jira/browse/HUDI-4894 Project: Apache Hudi

[jira] [Closed] (HUDI-4160) Make database regex of MaxwellJsonKafkaSourcePostProcessor optional

2022-05-28 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-4160. -- Resolution: Fixed > Make database regex of MaxwellJsonKafkaSourcePostProcessor optional >

[jira] [Closed] (HUDI-4164) Make the partitionpath value of TimestampBasedAvroKeyGenerator nullable configurable

2022-05-27 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-4164. -- Fix Version/s: (was: 0.12.0) Resolution: Abandoned > Make the partitionpath value of

[jira] [Updated] (HUDI-4164) Make the partitionpath value of TimestampBasedAvroKeyGenerator nullable configurable

2022-05-27 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-4164: --- Summary: Make the partitionpath value of TimestampBasedAvroKeyGenerator nullable configurable (was:

[jira] [Created] (HUDI-4164) Make the partitionpath value nullable configurable

2022-05-27 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-4164: -- Summary: Make the partitionpath value nullable configurable Key: HUDI-4164 URL: https://issues.apache.org/jira/browse/HUDI-4164 Project: Apache Hudi Issue Type:

[jira] [Created] (HUDI-4160) Make database regex of MaxwellJsonKafkaSourcePostProcessor optional

2022-05-26 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-4160: -- Summary: Make database regex of MaxwellJsonKafkaSourcePostProcessor optional Key: HUDI-4160 URL: https://issues.apache.org/jira/browse/HUDI-4160 Project: Apache Hudi

[jira] [Closed] (HUDI-2137) Make hudi examples runnable locally

2022-05-26 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-2137. -- Resolution: Abandoned > Make hudi examples runnable locally > --- > >

[jira] [Closed] (HUDI-1438) Move DataSourceOptions to hudi-client-common to reuse more code

2022-05-26 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-1438. -- Resolution: Abandoned > Move DataSourceOptions to hudi-client-common to reuse more code >

[jira] [Closed] (HUDI-4122) Fix NPE caused by adding kafka nodes

2022-05-21 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-4122. -- Fix Version/s: 0.12.0 Resolution: Fixed Fixed via master :

[jira] [Created] (HUDI-4122) Fix NPE caused by adding kafka nodes

2022-05-19 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-4122: -- Summary: Fix NPE caused by adding kafka nodes Key: HUDI-4122 URL: https://issues.apache.org/jira/browse/HUDI-4122 Project: Apache Hudi Issue Type: Improvement

[jira] [Updated] (HUDI-4122) Fix NPE caused by adding kafka nodes

2022-05-19 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-4122: --- Description: Yesterday, when we add more nodes to kafka cluster, some of our hudi tasks failed with

[jira] [Closed] (HUDI-3607) Support backend switch in HoodieFlinkStreamer

2022-03-16 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3607. -- Resolution: Fixed > Support backend switch in HoodieFlinkStreamer >

[jira] [Closed] (HUDI-3589) flink sync hive metadata supports table properties and serde properties

2022-03-15 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3589. -- Fix Version/s: 0.11.0 Resolution: Fixed > flink sync hive metadata supports table properties and

[jira] [Closed] (HUDI-3633) TypedProperties can not set non-string values using its original type

2022-03-15 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3633. -- Resolution: Fixed > TypedProperties can not set non-string values using its original type >

[jira] [Closed] (HUDI-3547) Introduce MaxwellSourcePostProcessor to extract data from Maxwell json string

2022-03-15 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3547. -- Resolution: Fixed > Introduce MaxwellSourcePostProcessor to extract data from Maxwell json string >

[jira] [Closed] (HUDI-3619) Fix HoodieOperation fromValue using wrong constant value

2022-03-15 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3619. -- Fix Version/s: 0.11.0 Resolution: Fixed > Fix HoodieOperation fromValue using wrong constant

[jira] [Updated] (HUDI-2137) Make hudi examples runnable locally

2022-03-14 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-2137: --- Issue Type: Improvement (was: Bug) > Make hudi examples runnable locally >

[jira] [Closed] (HUDI-3501) Support savepoints command based on Call Produce Command

2022-03-13 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3501. -- Fix Version/s: 0.11.0 Resolution: Fixed > Support savepoints command based on Call Produce

[jira] [Closed] (HUDI-3569) Introduce ChainedJsonKafkaSourePostProcessor to support setting multi processors at once

2022-03-11 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3569. -- Resolution: Fixed > Introduce ChainedJsonKafkaSourePostProcessor to support setting multi > processors

[jira] [Comment Edited] (HUDI-3607) Support backend switch in HoodieFlinkStreamer

2022-03-11 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17504872#comment-17504872 ] Xianghu Wang edited comment on HUDI-3607 at 3/11/22, 11:39 AM: --- hi

[jira] [Commented] (HUDI-3607) Support backend switch in HoodieFlinkStreamer

2022-03-11 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17504872#comment-17504872 ] Xianghu Wang commented on HUDI-3607: hi [~liufangqi]  The reasion you can't assign it to yourself is

[jira] [Closed] (HUDI-3575) Use HoodieTestDataGenerator#TRIP_SCHEMA as example schema in TestSchemaPostProcessor

2022-03-11 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3575. -- Resolution: Fixed > Use HoodieTestDataGenerator#TRIP_SCHEMA as example schema in >

[jira] [Closed] (HUDI-3522) Introduce DropColumnSchemaPostProcessor to support drop columns from schema

2022-03-10 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3522. -- Resolution: Fixed Resolved via master : 83cff3afee15e129034eb51e68a1734c55d85da2 > Introduce

[jira] [Closed] (HUDI-3568) Introduce ChainedSchemaPostProcessor to support setting multi processors at once

2022-03-08 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3568. -- Resolution: Fixed Fixed via master : 548000b0d635067acf7574c4cf5122759e79b52b > Introduce

[jira] [Created] (HUDI-3575) Use HoodieTestDataGenerator#TRIP_SCHEMA as example schema in TestSchemaPostProcessor

2022-03-07 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3575: -- Summary: Use HoodieTestDataGenerator#TRIP_SCHEMA as example schema in TestSchemaPostProcessor Key: HUDI-3575 URL: https://issues.apache.org/jira/browse/HUDI-3575

[jira] [Closed] (HUDI-909) [UMBRELLA]Integrate hudi with flink engine

2022-03-07 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-909. - Resolution: Fixed > [UMBRELLA]Integrate hudi with flink engine >

[jira] [Closed] (HUDI-1403) Decouple HoodieFlinkStreamer from Kafka to support more sources

2022-03-07 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-1403. -- Resolution: Abandoned > Decouple HoodieFlinkStreamer from Kafka to support more sources >

[jira] [Updated] (HUDI-3569) Introduce ChainedJsonKafkaSourePostProcessor to support setting multi processors at once

2022-03-07 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3569: --- Summary: Introduce ChainedJsonKafkaSourePostProcessor to support setting multi processors at once

[jira] [Updated] (HUDI-3568) Introduce ChainedSchemaPostProcessor to support setting multi processors at once

2022-03-06 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3568: --- Summary: Introduce ChainedSchemaPostProcessor to support setting multi processors at once (was:

[jira] [Created] (HUDI-3569) Introduce ChainedJsonKafkaSoureProcessor to support setting multi processors at once

2022-03-06 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3569: -- Summary: Introduce ChainedJsonKafkaSoureProcessor to support setting multi processors at once Key: HUDI-3569 URL: https://issues.apache.org/jira/browse/HUDI-3569

[jira] [Created] (HUDI-3568) Introduce ChainedSchemaProcessor to support setting multi processors at once

2022-03-06 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3568: -- Summary: Introduce ChainedSchemaProcessor to support setting multi processors at once Key: HUDI-3568 URL: https://issues.apache.org/jira/browse/HUDI-3568 Project: Apache

[jira] [Closed] (HUDI-3525) Introduce JsonkafkaSourcePostProcessor to support data preprocess before it is transformed to DataSet

2022-03-06 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3525. -- Fix Version/s: 0.11.0 Resolution: Fixed Resolved via master :

[jira] [Closed] (HUDI-3520) Introduce DeleteSupportSchemaPostProcessor to support adding _hoodie_is_deleted column to schema

2022-03-06 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3520. -- Resolution: Fixed Resolved via master : 4b471772aa130dc4bb577edbd42f2a617f205da0 > Introduce

[jira] [Closed] (HUDI-3348) HoodieRealtimeFileSplit losing info when serialized/deserialized

2022-03-03 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3348. -- Resolution: Fixed > HoodieRealtimeFileSplit losing info when serialized/deserialized >

[jira] [Updated] (HUDI-3547) Introduce MaxwellSourcePostProcessor to extract data from Maxwell json string

2022-03-02 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3547: --- Summary: Introduce MaxwellSourcePostProcessor to extract data from Maxwell json string (was:

[jira] [Created] (HUDI-3547) Introduce MaxwellSourcePostProcessor to make extract data from Maxwell json string

2022-03-02 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3547: -- Summary: Introduce MaxwellSourcePostProcessor to make extract data from Maxwell json string Key: HUDI-3547 URL: https://issues.apache.org/jira/browse/HUDI-3547 Project:

[jira] [Assigned] (HUDI-2137) Make hudi examples runnable locally

2022-03-01 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang reassigned HUDI-2137: -- Assignee: Xianghu Wang > Make hudi examples runnable locally >

[jira] [Updated] (HUDI-3525) Introduce JsonkafkaSourcePostProcessor to support data preprocess before it is transformed to DataSet

2022-03-01 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3525: --- Summary: Introduce JsonkafkaSourcePostProcessor to support data preprocess before it is transformed

[jira] [Commented] (HUDI-3522) Introduce DropColumnSchemaPostProcessor to support drop columns from schema

2022-02-28 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17499354#comment-17499354 ] Xianghu Wang commented on HUDI-3522: [~shivnarayan] thanks for your response. there is a scene that

[jira] [Commented] (HUDI-3525) Introduce JsonkafkaSourceProcessor to support data preprocess before it is transformed to DataSet

2022-02-28 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17498756#comment-17498756 ] Xianghu Wang commented on HUDI-3525: hi [~shivnarayan], [~xushiyan] any ideas about this ? >

[jira] [Updated] (HUDI-3525) Introduce JsonkafkaSourceProcessor to support data preprocess before it is transformed to DataSet

2022-02-28 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3525: --- Description: currently we have `Transform` to transform source to target dataset before writing, but

[jira] [Updated] (HUDI-3525) Introduce JsonkafkaSourceProcessor to support data preprocess before it is transformed to DataSet

2022-02-28 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3525: --- Description: currently we have `Transform` to transform source to target dataset before writing, but

[jira] [Created] (HUDI-3525) Introduce JsonkafkaSourceProcessor to support data preprocess before it is transformed to DataSet

2022-02-28 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3525: -- Summary: Introduce JsonkafkaSourceProcessor to support data preprocess before it is transformed to DataSet Key: HUDI-3525 URL: https://issues.apache.org/jira/browse/HUDI-3525

[jira] [Created] (HUDI-3523) Introduce AddColumnSchemaPostProcessor to support add columns to the end of a schema

2022-02-27 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3523: -- Summary: Introduce AddColumnSchemaPostProcessor to support add columns to the end of a schema Key: HUDI-3523 URL: https://issues.apache.org/jira/browse/HUDI-3523

[jira] [Updated] (HUDI-3522) Introduce DropColumnSchemaPostProcessor to support drop columns from schema

2022-02-27 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3522: --- Summary: Introduce DropColumnSchemaPostProcessor to support drop columns from schema (was: Introduce

[jira] [Updated] (HUDI-3522) Introduce ColumnDropSchemaPostProcessor to support drop columns from schema

2022-02-27 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3522: --- Description: A SchemaPostProcessor to drop columns from given schema > Introduce

[jira] [Updated] (HUDI-3522) Introduce ColumnDropSchemaPostProcessor to support drop columns from schema

2022-02-27 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3522: --- Fix Version/s: 0.11.0 > Introduce ColumnDropSchemaPostProcessor to support drop columns from schema >

[jira] [Created] (HUDI-3522) Introduce ColumnDropSchemaPostProcessor to support drop columns from schema

2022-02-27 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3522: -- Summary: Introduce ColumnDropSchemaPostProcessor to support drop columns from schema Key: HUDI-3522 URL: https://issues.apache.org/jira/browse/HUDI-3522 Project: Apache

[jira] [Created] (HUDI-3520) Introduce DeleteSupportSchemaPostProcessor to support adding _hoodie_is_deleted column to schema

2022-02-27 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3520: -- Summary: Introduce DeleteSupportSchemaPostProcessor to support adding _hoodie_is_deleted column to schema Key: HUDI-3520 URL: https://issues.apache.org/jira/browse/HUDI-3520

[jira] [Created] (HUDI-3518) Make HiveSchemaProvider support AWS Glue Catalog

2022-02-25 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3518: -- Summary: Make HiveSchemaProvider support AWS Glue Catalog Key: HUDI-3518 URL: https://issues.apache.org/jira/browse/HUDI-3518 Project: Apache Hudi Issue Type:

[jira] [Closed] (HUDI-3464) Fix wrong exception thrown from HiveSchemaProvider

2022-02-21 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3464. -- Resolution: Fixed Fixed via master : 4d1f74ebeaee857380f69d7c596eaaf0135ca59e > Fix wrong exception

[jira] [Updated] (HUDI-3464) Fix wrong exception thrown from HiveSchemaProvider

2022-02-21 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3464: --- Fix Version/s: 0.11.0 > Fix wrong exception thrown from HiveSchemaProvider >

[jira] [Updated] (HUDI-3464) Fix wrong exception thrown from HiveSchemaProvider

2022-02-21 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3464: --- Summary: Fix wrong exception thrown from HiveSchemaProvider (was: Fix wrong exception message thrown

[jira] [Assigned] (HUDI-3464) Fix wrong exception message thrown from HiveSchemaProvider

2022-02-21 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang reassigned HUDI-3464: -- Assignee: Xianghu Wang > Fix wrong exception message thrown from HiveSchemaProvider >

[jira] [Created] (HUDI-3464) Fix wrong exception message thrown from HiveSchemaProvider

2022-02-21 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3464: -- Summary: Fix wrong exception message thrown from HiveSchemaProvider Key: HUDI-3464 URL: https://issues.apache.org/jira/browse/HUDI-3464 Project: Apache Hudi

[jira] [Closed] (HUDI-1928) [UMBRELLA] Improve user experience by simplifying hudi task parameter configuration

2022-02-18 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-1928. -- Fix Version/s: 0.10.1 Resolution: Fixed > [UMBRELLA] Improve user experience by simplifying hudi

[jira] [Assigned] (HUDI-1438) Move DataSourceOptions to hudi-client-common to reuse more code

2022-02-18 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang reassigned HUDI-1438: -- Assignee: Xianghu Wang > Move DataSourceOptions to hudi-client-common to reuse more code >

[jira] [Closed] (HUDI-1089) Refactor hudi-client to support multi-engine

2022-02-18 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-1089. -- Assignee: Xianghu Wang (was: wangxianghu#1) > Refactor hudi-client to support multi-engine >

[jira] [Assigned] (HUDI-1328) Introduce HoodieFlinkEngineContext to hudi-flink-client

2022-02-18 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang reassigned HUDI-1328: -- Assignee: Xianghu Wang (was: wangxianghu#1) > Introduce HoodieFlinkEngineContext to

[jira] [Closed] (HUDI-912) Refactor and relocate KeyGenerator to support more engines

2022-02-18 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-912. - Assignee: Xianghu Wang (was: wangxianghu#1) > Refactor and relocate KeyGenerator to support more engines

[jira] [Closed] (HUDI-1439) Remove scala dependency from hudi-client-common

2022-02-18 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-1439. -- > Remove scala dependency from hudi-client-common > --- > >

[jira] [Closed] (HUDI-1328) Introduce HoodieFlinkEngineContext to hudi-flink-client

2022-02-18 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-1328. -- > Introduce HoodieFlinkEngineContext to hudi-flink-client >

[jira] [Closed] (HUDI-1929) Support configure KeyGenerator by type

2022-02-18 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-1929. -- Resolution: Fixed (was: Resolved) > Support configure KeyGenerator by type >

[jira] [Closed] (HUDI-3283) Bootstrap support overwrite existing table

2022-02-18 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3283. -- Fix Version/s: 0.11.0 Resolution: Fixed > Bootstrap support overwrite existing table >

[jira] [Closed] (HUDI-3440) HoodieSortedMergeHandle#close write data disorder

2022-02-18 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3440. -- Resolution: Fixed Fixed via master : de8161ae96c4cec1e4c9e40069f286439f0c3261 >

[jira] [Closed] (HUDI-3413) fix jackson parse error when empty message from JsonKafkaSource Using HoodieDeltaStreamer

2022-02-11 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang closed HUDI-3413. -- Fix Version/s: 0.11.0 Resolution: Fixed Fixed via master

[jira] [Resolved] (HUDI-3413) fix jackson parse error when empty message from JsonKafkaSource Using HoodieDeltaStreamer

2022-02-11 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang resolved HUDI-3413. > fix jackson parse error when empty message from JsonKafkaSource Using > HoodieDeltaStreamer >

[jira] [Created] (HUDI-3305) Drop deprecated util HDFSParquetImporter

2022-01-24 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3305: -- Summary: Drop deprecated util HDFSParquetImporter Key: HUDI-3305 URL: https://issues.apache.org/jira/browse/HUDI-3305 Project: Apache Hudi Issue Type: Task

[jira] [Commented] (HUDI-3285) Drop unused method SparkBootstrapCommitActionExecutor#handleMetadataBootstrap

2022-01-20 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479480#comment-17479480 ] Xianghu Wang commented on HUDI-3285: Resolved via master : a66004a34072766674c56082edb1962ee61b2e83 >

[jira] [Resolved] (HUDI-3285) Drop unused method SparkBootstrapCommitActionExecutor#handleMetadataBootstrap

2022-01-20 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang resolved HUDI-3285. > Drop unused method SparkBootstrapCommitActionExecutor#handleMetadataBootstrap >

[jira] [Updated] (HUDI-3285) Drop unused method SparkBootstrapCommitActionExecutor#handleMetadataBootstrap

2022-01-20 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3285: --- Status: In Progress (was: Open) > Drop unused method

[jira] [Created] (HUDI-3285) Drop unused method SparkBootstrapCommitActionExecutor#handleMetadataBootstrap

2022-01-20 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3285: -- Summary: Drop unused method SparkBootstrapCommitActionExecutor#handleMetadataBootstrap Key: HUDI-3285 URL: https://issues.apache.org/jira/browse/HUDI-3285 Project:

[jira] [Resolved] (HUDI-3283) Bootstrap support overwrite existing table

2022-01-20 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang resolved HUDI-3283. > Bootstrap support overwrite existing table > -- > >

[jira] [Commented] (HUDI-3283) Bootstrap support overwrite existing table

2022-01-20 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479251#comment-17479251 ] Xianghu Wang commented on HUDI-3283: Resolved via master : b7a79aa94301a80c396476878fadeea5debb799a >

[jira] [Updated] (HUDI-3283) Bootstrap support overwrite existing table

2022-01-20 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3283: --- Status: In Progress (was: Open) > Bootstrap support overwrite existing table >

[jira] [Assigned] (HUDI-3283) Bootstrap support overwrite existing table

2022-01-19 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang reassigned HUDI-3283: -- Assignee: Xianghu Wang > Bootstrap support overwrite existing table >

[jira] [Created] (HUDI-3283) Bootstrap support overwrite existing table

2022-01-19 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3283: -- Summary: Bootstrap support overwrite existing table Key: HUDI-3283 URL: https://issues.apache.org/jira/browse/HUDI-3283 Project: Apache Hudi Issue Type: Task

[jira] [Resolved] (HUDI-3277) Filter non-parquet files in bootstrap procedure

2022-01-19 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang resolved HUDI-3277. > Filter non-parquet files in bootstrap procedure > --- > >

[jira] [Reopened] (HUDI-3277) Filter non-parquet files in bootstrap procedure

2022-01-19 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang reopened HUDI-3277: > Filter non-parquet files in bootstrap procedure > --- > >

[jira] [Resolved] (HUDI-3277) Filter non-parquet files in bootstrap procedure

2022-01-19 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang resolved HUDI-3277. > Filter non-parquet files in bootstrap procedure > --- > >

[jira] [Commented] (HUDI-3277) Filter non-parquet files in bootstrap procedure

2022-01-19 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17478875#comment-17478875 ] Xianghu Wang commented on HUDI-3277: Resolved via master : db93ad2f4bc9b3c357f065882c3d8cbb1c0b9c07 >

[jira] [Updated] (HUDI-3277) Filter non-parquet files in bootstrap procedure

2022-01-19 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang updated HUDI-3277: --- Description: when we have a file not endwith ".parquet" in path "hoodie.bootstrap.base.path",   it

[jira] [Created] (HUDI-3277) Filter non-parquet files in bootstrap procedure

2022-01-19 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-3277: -- Summary: Filter non-parquet files in bootstrap procedure Key: HUDI-3277 URL: https://issues.apache.org/jira/browse/HUDI-3277 Project: Apache Hudi Issue Type:

[jira] [Commented] (HUDI-2411) Remove unnecessary method overriden and note

2021-11-04 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17438509#comment-17438509 ] Xianghu Wang commented on HUDI-2411: ok, thanks for the reminder [~shivnarayan] > Remove unnecessary

[jira] [Created] (HUDI-2411) Remove unnecessary method overriden and note

2021-09-10 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-2411: -- Summary: Remove unnecessary method overriden and note Key: HUDI-2411 URL: https://issues.apache.org/jira/browse/HUDI-2411 Project: Apache Hudi Issue Type: Task

  1   2   >