[jira] [Assigned] (HUDI-5835) spark cannot read mor table after execute update statement

2023-02-23 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng reassigned HUDI-5835: -- Assignee: Tao Meng > spark cannot read mor table after execute update statement >

[jira] [Updated] (HUDI-5835) spark cannot read mor table after execute update statement

2023-02-23 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng updated HUDI-5835: --- Description: avro schema create by sparksql miss avro name and namespace,  This will lead the read schema

[jira] [Updated] (HUDI-5835) spark cannot read mor table after execute update statement

2023-02-23 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng updated HUDI-5835: --- Description: avro schema create by sparksql will miss avro name and namespace,  This will lead the read

[jira] [Updated] (HUDI-5835) spark cannot read mor table after execute update statement

2023-02-23 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng updated HUDI-5835: --- Description: avro schema create by sparksql will miss avro name and namespace,  This will lead the read

[jira] [Updated] (HUDI-5835) spark cannot read mor table after execute update statement

2023-02-23 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng updated HUDI-5835: --- Description: avro schema create by sparksql will miss avro name and namespace,  This will lead the read

[jira] [Created] (HUDI-5835) spark cannot read mor table after execute update statement

2023-02-23 Thread Tao Meng (Jira)
Tao Meng created HUDI-5835: -- Summary: spark cannot read mor table after execute update statement Key: HUDI-5835 URL: https://issues.apache.org/jira/browse/HUDI-5835 Project: Apache Hudi Issue Type:

[jira] [Created] (HUDI-5294) Support type change for schema on read enable + reconcile schema

2022-11-29 Thread Tao Meng (Jira)
Tao Meng created HUDI-5294: -- Summary: Support type change for schema on read enable + reconcile schema Key: HUDI-5294 URL: https://issues.apache.org/jira/browse/HUDI-5294 Project: Apache Hudi

[jira] [Updated] (HUDI-5194) fix schema evolution bugs

2022-11-10 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng updated HUDI-5194: --- Description: # Fix the bug, history schema files cannot be cleaned by FileBasedInternalSchemaStorageManager

[jira] [Updated] (HUDI-5194) fix schema evolution bugs

2022-11-10 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng updated HUDI-5194: --- Issue Type: Bug (was: New Feature) > fix schema evolution bugs > - > >

[jira] [Created] (HUDI-5194) fix schema evolution bugs

2022-11-10 Thread Tao Meng (Jira)
Tao Meng created HUDI-5194: -- Summary: fix schema evolution bugs Key: HUDI-5194 URL: https://issues.apache.org/jira/browse/HUDI-5194 Project: Apache Hudi Issue Type: New Feature

[jira] [Assigned] (HUDI-5000) Support schema evolution for Hive

2022-10-18 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng reassigned HUDI-5000: -- Assignee: Tao Meng > Support schema evolution for Hive > - > >

[jira] [Assigned] (HUDI-5000) Support schema evolution for Hive

2022-10-18 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-5000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng reassigned HUDI-5000: -- Assignee: (was: Tao Meng) > Support schema evolution for Hive > -

[jira] [Created] (HUDI-5000) Support schema evolution for Hive

2022-10-09 Thread Tao Meng (Jira)
Tao Meng created HUDI-5000: -- Summary: Support schema evolution for Hive Key: HUDI-5000 URL: https://issues.apache.org/jira/browse/HUDI-5000 Project: Apache Hudi Issue Type: New Feature

[jira] [Created] (HUDI-4898) for mor table, presto/hive shoud respect payload class during merge parquet file and log file

2022-09-22 Thread Tao Meng (Jira)
Tao Meng created HUDI-4898: -- Summary: for mor table, presto/hive shoud respect payload class during merge parquet file and log file Key: HUDI-4898 URL: https://issues.apache.org/jira/browse/HUDI-4898

[jira] [Closed] (HUDI-1675) Externalize all Hudi configurations

2022-06-26 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng closed HUDI-1675. -- Resolution: Fixed close it, as Hudi currently has this capability > Externalize all Hudi configurations >

[jira] [Commented] (HUDI-4184) Creating external table in Spark SQL modifies "hoodie.properties"

2022-06-07 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17550983#comment-17550983 ] Tao Meng commented on HUDI-4184: I suggest moving the schema from hoodie.properties.   1) we has already 

[jira] [Created] (HUDI-3921) Fixed schema evolution cannot work with HUDI-3855

2022-04-20 Thread Tao Meng (Jira)
Tao Meng created HUDI-3921: -- Summary: Fixed schema evolution cannot work with HUDI-3855 Key: HUDI-3921 URL: https://issues.apache.org/jira/browse/HUDI-3921 Project: Apache Hudi Issue Type: Bug

[jira] [Closed] (HUDI-1816) when query incr view of hudi table by using spark-sql, the query result is wrong

2022-03-30 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng closed HUDI-1816. -- Resolution: Not A Problem not a problem, close it > when query incr view of hudi table by using spark-sql, the

[jira] [Closed] (HUDI-3408) fixed the bug that BUCKET_INDEX cannot process special characters

2022-03-29 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng closed HUDI-3408. -- Resolution: Won't Fix This is not a common problem, let me close it > fixed the bug that BUCKET_INDEX cannot

[jira] [Created] (HUDI-3742) Enable parquet enableVectorizedReader for spark incremental read to prevent pef regression

2022-03-29 Thread Tao Meng (Jira)
Tao Meng created HUDI-3742: -- Summary: Enable parquet enableVectorizedReader for spark incremental read to prevent pef regression Key: HUDI-3742 URL: https://issues.apache.org/jira/browse/HUDI-3742 Project:

[jira] [Created] (HUDI-3719) High performance costs of AvroSerializer in Datasource writing

2022-03-26 Thread Tao Meng (Jira)
Tao Meng created HUDI-3719: -- Summary: High performance costs of AvroSerializer in Datasource writing Key: HUDI-3719 URL: https://issues.apache.org/jira/browse/HUDI-3719 Project: Apache Hudi Issue

[jira] [Updated] (HUDI-3646) The Hudi update syntax should not modify the nullability attribute of a column

2022-03-16 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Meng updated HUDI-3646: --- Description: now, when we use sparksql to update hudi table, we find that  hudi will change the nullability

[jira] [Created] (HUDI-3646) The Hudi update syntax should not modify the nullability attribute of a column

2022-03-16 Thread Tao Meng (Jira)
Tao Meng created HUDI-3646: -- Summary: The Hudi update syntax should not modify the nullability attribute of a column Key: HUDI-3646 URL: https://issues.apache.org/jira/browse/HUDI-3646 Project: Apache Hudi

[jira] [Created] (HUDI-3603) Support read DateType for hive2/hive3

2022-03-10 Thread Tao Meng (Jira)
Tao Meng created HUDI-3603: -- Summary: Support read DateType for hive2/hive3 Key: HUDI-3603 URL: https://issues.apache.org/jira/browse/HUDI-3603 Project: Apache Hudi Issue Type: Bug

[jira] [Commented] (HUDI-3355) Issue with out of order commits in the timeline when ingestion writers using SparkAllowUpdateStrategy

2022-03-06 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17501953#comment-17501953 ] Tao Meng commented on HUDI-3355: [~suryaprasanna]  if you have free time, can you try this pr

[jira] [Commented] (HUDI-3355) Issue with out of order commits in the timeline when ingestion writers using SparkAllowUpdateStrategy

2022-03-04 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17501653#comment-17501653 ] Tao Meng commented on HUDI-3355: [~suryaprasanna]    I tried to understand the problem,do you mean that:

[jira] [Commented] (HUDI-3355) Issue with out of order commits in the timeline when ingestion writers using SparkAllowUpdateStrategy

2022-03-04 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17501645#comment-17501645 ] Tao Meng commented on HUDI-3355: no problem > Issue with out of order commits in the timeline when

[jira] [Commented] (HUDI-2762) Ensure hive can query insert only logs in MOR

2022-02-18 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17494502#comment-17494502 ] Tao Meng commented on HUDI-2762: [~alexey.kudinkin]  i This problem is hive's problem. Hive will filter

[jira] [Comment Edited] (HUDI-2762) Ensure hive can query insert only logs in MOR

2022-02-18 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17494502#comment-17494502 ] Tao Meng edited comment on HUDI-2762 at 2/18/22, 9:59 AM: -- [~alexey.kudinkin] 

[jira] [Created] (HUDI-3408) fixed the bug that BUCKET_INDEX cannot process special characters

2022-02-10 Thread Tao Meng (Jira)
Tao Meng created HUDI-3408: -- Summary: fixed the bug that BUCKET_INDEX cannot process special characters Key: HUDI-3408 URL: https://issues.apache.org/jira/browse/HUDI-3408 Project: Apache Hudi

[jira] [Commented] (HUDI-3347) Updating table schema fails w/ hms mode w/ schema evolution

2022-02-08 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17489291#comment-17489291 ] Tao Meng commented on HUDI-3347: [~shivnarayan]  This problem seems to be caused by the wrong version of

[jira] [Comment Edited] (HUDI-3347) Updating table schema fails w/ hms mode w/ schema evolution

2022-01-31 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484721#comment-17484721 ] Tao Meng edited comment on HUDI-3347 at 1/31/22, 2:51 PM: -- [~shivnarayan]   yes,

[jira] [Commented] (HUDI-3347) Updating table schema fails w/ hms mode w/ schema evolution

2022-01-31 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17484721#comment-17484721 ] Tao Meng commented on HUDI-3347: yes, will do it. > Updating table schema fails w/ hms mode w/ schema

[jira] [Commented] (HUDI-2873) Support optimize data layout by sql and make the build more fast

2022-01-18 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477743#comment-17477743 ] Tao Meng commented on HUDI-2873: [~shibei]  do you have wechat,  pls add me 1037817390 > Support optimize

[jira] [Commented] (HUDI-2873) Support optimize data layout by sql and make the build more fast

2022-01-17 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477214#comment-17477214 ] Tao Meng commented on HUDI-2873: [~alexey.kudinkin]  [~shibei]  1)  support optimize data by sparksql,

[jira] [Commented] (HUDI-2645) Rewrite Zoptimize and other files in scala into Java

2022-01-17 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477205#comment-17477205 ] Tao Meng commented on HUDI-2645: [~shibei]   vc means we should better remove scala files in

[jira] [Commented] (HUDI-3237) ALTER TABLE column type change fails select query

2022-01-13 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475398#comment-17475398 ] Tao Meng commented on HUDI-3237: [~biyan900...@gmail.com]  [~xushiyan]  i agree that we should close this

[jira] [Commented] (HUDI-2874) hudi should remove the temp file which create by HoodieMergedLogRecordScanner, when we use hive/presto

2022-01-12 Thread Tao Meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17474530#comment-17474530 ] Tao Meng commented on HUDI-2874: [~shivnarayan]   yes we have areadly fixed this issue in 0.10 > hudi

[jira] [Commented] (HUDI-3164) CTAS fails w/ UnsupportedOperationException when trying to modify immutable map in DataSourceUtils.mayBeOverwriteParquetWriteLegacyFormatProp

2022-01-04 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468646#comment-17468646 ] tao meng commented on HUDI-3164: [~shivnarayan]   leesf has already solved this problem see

[jira] [Created] (HUDI-3096) fixed the bug that the cow table(contains decimalType) write by flink cannot be read by spark

2021-12-21 Thread Tao Meng (Jira)
Tao Meng created HUDI-3096: -- Summary: fixed the bug that the cow table(contains decimalType) write by flink cannot be read by spark Key: HUDI-3096 URL: https://issues.apache.org/jira/browse/HUDI-3096

[jira] [Commented] (HUDI-2059) When log exists in mor table, clustering is triggered. The query result shows that the update record in log is lost

2021-12-13 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17458858#comment-17458858 ] tao meng commented on HUDI-2059: [~shivnarayan]   as HUDI-2170 merged, it's ok to close   > When log

[jira] [Created] (HUDI-3001) clean up temp marker directory when finish bootstrap operation.

2021-12-13 Thread tao meng (Jira)
tao meng created HUDI-3001: -- Summary: clean up temp marker directory when finish bootstrap operation. Key: HUDI-3001 URL: https://issues.apache.org/jira/browse/HUDI-3001 Project: Apache Hudi

[jira] [Updated] (HUDI-2966) Add TaskCompletionListener for HoodieMergeOnReadRDD to close logScanner

2021-12-08 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-2966: --- Priority: Minor (was: Major) > Add TaskCompletionListener for HoodieMergeOnReadRDD to close logScanner >

[jira] [Updated] (HUDI-2966) Add TaskCompletionListener for HoodieMergeOnReadRDD to close logScanner

2021-12-08 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-2966: --- Priority: Major (was: Minor) > Add TaskCompletionListener for HoodieMergeOnReadRDD to close logScanner >

[jira] [Created] (HUDI-2966) Add TaskCompletionListener for HoodieMergeOnReadRDD to close logScanner

2021-12-08 Thread tao meng (Jira)
tao meng created HUDI-2966: -- Summary: Add TaskCompletionListener for HoodieMergeOnReadRDD to close logScanner Key: HUDI-2966 URL: https://issues.apache.org/jira/browse/HUDI-2966 Project: Apache Hudi

[jira] [Created] (HUDI-2958) Automatically set spark.sql.parquet.writelegacyformat. When using bulkinsert to insert data will contains decimal Type.

2021-12-08 Thread tao meng (Jira)
tao meng created HUDI-2958: -- Summary: Automatically set spark.sql.parquet.writelegacyformat. When using bulkinsert to insert data will contains decimal Type. Key: HUDI-2958 URL:

[jira] [Updated] (HUDI-2958) Automatically set spark.sql.parquet.writelegacyformat; When using bulkinsert to insert data which contains decimal Type.

2021-12-08 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-2958: --- Summary: Automatically set spark.sql.parquet.writelegacyformat; When using bulkinsert to insert data which

[jira] [Created] (HUDI-2901) Fixed the bug clustering jobs are not running in parallel

2021-12-01 Thread tao meng (Jira)
tao meng created HUDI-2901: -- Summary: Fixed the bug clustering jobs are not running in parallel Key: HUDI-2901 URL: https://issues.apache.org/jira/browse/HUDI-2901 Project: Apache Hudi Issue Type:

[jira] [Created] (HUDI-2876) hudi should remove the temp file which create by HoodieMergedLogRecordScanner, when we use presto

2021-11-27 Thread tao meng (Jira)
tao meng created HUDI-2876: -- Summary: hudi should remove the temp file which create by HoodieMergedLogRecordScanner, when we use presto Key: HUDI-2876 URL: https://issues.apache.org/jira/browse/HUDI-2876

[jira] [Created] (HUDI-2874) hudi should remove the temp file which create by HoodieMergedLogRecordScanner, when we use hive/presto

2021-11-27 Thread tao meng (Jira)
tao meng created HUDI-2874: -- Summary: hudi should remove the temp file which create by HoodieMergedLogRecordScanner, when we use hive/presto Key: HUDI-2874 URL: https://issues.apache.org/jira/browse/HUDI-2874

[jira] [Created] (HUDI-2873) support optimize data layout by sql and make the build more fast

2021-11-26 Thread tao meng (Jira)
tao meng created HUDI-2873: -- Summary: support optimize data layout by sql and make the build more fast Key: HUDI-2873 URL: https://issues.apache.org/jira/browse/HUDI-2873 Project: Apache Hudi

[jira] [Created] (HUDI-2778) Optimize statistics collection related codes and add more docs for z-order

2021-11-16 Thread tao meng (Jira)
tao meng created HUDI-2778: -- Summary: Optimize statistics collection related codes and add more docs for z-order Key: HUDI-2778 URL: https://issues.apache.org/jira/browse/HUDI-2778 Project: Apache Hudi

[jira] [Created] (HUDI-2758) remove redundant code in the HoodieRealtimeInputFormatUtils.getRealtimeSplits

2021-11-14 Thread tao meng (Jira)
tao meng created HUDI-2758: -- Summary: remove redundant code in the HoodieRealtimeInputFormatUtils.getRealtimeSplits Key: HUDI-2758 URL: https://issues.apache.org/jira/browse/HUDI-2758 Project: Apache Hudi

[jira] [Created] (HUDI-2697) Minor changes about hbase index config.

2021-11-05 Thread tao meng (Jira)
tao meng created HUDI-2697: -- Summary: Minor changes about hbase index config. Key: HUDI-2697 URL: https://issues.apache.org/jira/browse/HUDI-2697 Project: Apache Hudi Issue Type: Bug

[jira] [Created] (HUDI-2676) Hudi should synchronize owner information to hudi _rt/_ro table。

2021-11-02 Thread tao meng (Jira)
tao meng created HUDI-2676: -- Summary: Hudi should synchronize owner information to hudi _rt/_ro table。 Key: HUDI-2676 URL: https://issues.apache.org/jira/browse/HUDI-2676 Project: Apache Hudi

[jira] [Updated] (HUDI-2674) hudi hive reader should not print read values

2021-11-02 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-2674: --- Summary: hudi hive reader should not print read values (was: hudi hive reader should not log read values) >

[jira] [Created] (HUDI-2674) hudi hive reader should not log read values

2021-11-02 Thread tao meng (Jira)
tao meng created HUDI-2674: -- Summary: hudi hive reader should not log read values Key: HUDI-2674 URL: https://issues.apache.org/jira/browse/HUDI-2674 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-2560) Introduce id_based schema to support full schema evolution

2021-10-15 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-2560: --- Description: Introduce id_based schema to support full schema evolution. (was: +Introduce id_based schema to

[jira] [Updated] (HUDI-2560) Introduce id_based schema to support full schema evolution

2021-10-15 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-2560: --- Description: +Introduce id_based schema to support full schema evolution+ > Introduce id_based schema to

[jira] [Created] (HUDI-2560) Introduce id_based schema to support full schema evolution

2021-10-15 Thread tao meng (Jira)
tao meng created HUDI-2560: -- Summary: Introduce id_based schema to support full schema evolution Key: HUDI-2560 URL: https://issues.apache.org/jira/browse/HUDI-2560 Project: Apache Hudi Issue Type:

[jira] [Created] (HUDI-2429) Full schema evolution

2021-09-14 Thread tao meng (Jira)
tao meng created HUDI-2429: -- Summary: Full schema evolution Key: HUDI-2429 URL: https://issues.apache.org/jira/browse/HUDI-2429 Project: Apache Hudi Issue Type: New Feature Components:

[jira] [Updated] (HUDI-2214) residual temporary files after clustering are not cleaned up

2021-07-23 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-2214: --- Description: residual temporary files after clustering are not cleaned up // test step step1: do clustering

[jira] [Created] (HUDI-2214) residual temporary files after clustering are not cleaned up

2021-07-23 Thread tao meng (Jira)
tao meng created HUDI-2214: -- Summary: residual temporary files after clustering are not cleaned up Key: HUDI-2214 URL: https://issues.apache.org/jira/browse/HUDI-2214 Project: Apache Hudi Issue

[jira] [Created] (HUDI-2116) sync 10w partitions to hive by using HiveSyncTool lead to the oom of hive MetaStore

2021-07-01 Thread tao meng (Jira)
tao meng created HUDI-2116: -- Summary: sync 10w partitions to hive by using HiveSyncTool lead to the oom of hive MetaStore Key: HUDI-2116 URL: https://issues.apache.org/jira/browse/HUDI-2116 Project: Apache

[jira] [Created] (HUDI-2102) support hilbert curve for hudi

2021-06-29 Thread tao meng (Jira)
tao meng created HUDI-2102: -- Summary: support hilbert curve for hudi Key: HUDI-2102 URL: https://issues.apache.org/jira/browse/HUDI-2102 Project: Apache Hudi Issue Type: Sub-task

[jira] [Created] (HUDI-2101) support z-order for hudi

2021-06-29 Thread tao meng (Jira)
tao meng created HUDI-2101: -- Summary: support z-order for hudi Key: HUDI-2101 URL: https://issues.apache.org/jira/browse/HUDI-2101 Project: Apache Hudi Issue Type: Sub-task Components:

[jira] [Created] (HUDI-2100) Support Space curve for hudi

2021-06-29 Thread tao meng (Jira)
tao meng created HUDI-2100: -- Summary: Support Space curve for hudi Key: HUDI-2100 URL: https://issues.apache.org/jira/browse/HUDI-2100 Project: Apache Hudi Issue Type: New Feature

[jira] [Created] (HUDI-2099) hive lock which state is WATING should be released, otherwise this hive lock will be locked forever

2021-06-29 Thread tao meng (Jira)
tao meng created HUDI-2099: -- Summary: hive lock which state is WATING should be released, otherwise this hive lock will be locked forever Key: HUDI-2099 URL: https://issues.apache.org/jira/browse/HUDI-2099

[jira] [Created] (HUDI-2098) add Hdfs file lock for HUDI

2021-06-29 Thread tao meng (Jira)
tao meng created HUDI-2098: -- Summary: add Hdfs file lock for HUDI Key: HUDI-2098 URL: https://issues.apache.org/jira/browse/HUDI-2098 Project: Apache Hudi Issue Type: Bug Components:

[jira] [Updated] (HUDI-2098) add Hdfs file lock for HUDI

2021-06-29 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-2098: --- Component/s: (was: Utilities) > add Hdfs file lock for HUDI > --- > >

[jira] [Created] (HUDI-2090) when hudi metadata is enabled, use different user to query table, the query will failed

2021-06-28 Thread tao meng (Jira)
tao meng created HUDI-2090: -- Summary: when hudi metadata is enabled, use different user to query table, the query will failed Key: HUDI-2090 URL: https://issues.apache.org/jira/browse/HUDI-2090 Project:

[jira] [Created] (HUDI-2089) fix the bug that metatable cannot support non_partition table

2021-06-28 Thread tao meng (Jira)
tao meng created HUDI-2089: -- Summary: fix the bug that metatable cannot support non_partition table Key: HUDI-2089 URL: https://issues.apache.org/jira/browse/HUDI-2089 Project: Apache Hudi Issue

[jira] [Created] (HUDI-2086) redo the logical of mor_incremental_view for hive

2021-06-28 Thread tao meng (Jira)
tao meng created HUDI-2086: -- Summary: redo the logical of mor_incremental_view for hive Key: HUDI-2086 URL: https://issues.apache.org/jira/browse/HUDI-2086 Project: Apache Hudi Issue Type: Bug

[jira] [Assigned] (HUDI-2086) redo the logical of mor_incremental_view for hive

2021-06-28 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng reassigned HUDI-2086: -- Assignee: tao meng > redo the logical of mor_incremental_view for hive >

[jira] [Created] (HUDI-2059) When log exists in mor table, clustering is triggered. The query result shows that the update record in log is lost

2021-06-22 Thread tao meng (Jira)
tao meng created HUDI-2059: -- Summary: When log exists in mor table, clustering is triggered. The query result shows that the update record in log is lost Key: HUDI-2059 URL:

[jira] [Created] (HUDI-2058) support incremental query for insert_overwrite_table/insert_overwrite operation on cow table

2021-06-22 Thread tao meng (Jira)
tao meng created HUDI-2058: -- Summary: support incremental query for insert_overwrite_table/insert_overwrite operation on cow table Key: HUDI-2058 URL: https://issues.apache.org/jira/browse/HUDI-2058

[jira] [Closed] (HUDI-1676) Support SQL with spark3

2021-06-08 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng closed HUDI-1676. -- Resolution: Fixed > Support SQL with spark3 > --- > > Key: HUDI-1676 >

[jira] [Assigned] (HUDI-1676) Support SQL with spark3

2021-06-08 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng reassigned HUDI-1676: -- Assignee: tao meng > Support SQL with spark3 > --- > > Key:

[jira] [Commented] (HUDI-1676) Support SQL with spark3

2021-06-08 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359310#comment-17359310 ] tao meng commented on HUDI-1676: [~pzw2018] great works  > Support SQL with spark3 >

[jira] [Created] (HUDI-1817) when query incr view of hudi table by using spark-sql. the result is wrong

2021-04-20 Thread tao meng (Jira)
tao meng created HUDI-1817: -- Summary: when query incr view of hudi table by using spark-sql. the result is wrong Key: HUDI-1817 URL: https://issues.apache.org/jira/browse/HUDI-1817 Project: Apache Hudi

[jira] [Created] (HUDI-1816) when query incr view of hudi table by using spark-sql, the query result is wrong

2021-04-20 Thread tao meng (Jira)
tao meng created HUDI-1816: -- Summary: when query incr view of hudi table by using spark-sql, the query result is wrong Key: HUDI-1816 URL: https://issues.apache.org/jira/browse/HUDI-1816 Project: Apache

[jira] [Created] (HUDI-1783) support Huawei Cloud Object Storage

2021-04-09 Thread tao meng (Jira)
tao meng created HUDI-1783: -- Summary: support Huawei Cloud Object Storage Key: HUDI-1783 URL: https://issues.apache.org/jira/browse/HUDI-1783 Project: Apache Hudi Issue Type: Bug

[jira] [Updated] (HUDI-1722) hive beeline/spark-sql query specified field on mor table occur NPE

2021-03-25 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-1722: --- Description: HUDI-892 introduce this problem。 this pr skip adding projection columns if there are no log

[jira] [Updated] (HUDI-1722) hive beeline/spark-sql query specified field on mor table occur NPE

2021-03-25 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-1722: --- Description: HUDI-892 introduce this problem。 this pr skip adding projection columns if there are no log

[jira] [Updated] (HUDI-1722) hive beeline/spark-sql query specified field on mor table occur NPE

2021-03-25 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-1722: --- Description: HUDI-892 introduce this problem。 this pr skip adding projection columns if there are no log

[jira] [Created] (HUDI-1722) hive beeline/spark-sql query specified field on mor table occur NPE

2021-03-25 Thread tao meng (Jira)
tao meng created HUDI-1722: -- Summary: hive beeline/spark-sql query specified field on mor table occur NPE Key: HUDI-1722 URL: https://issues.apache.org/jira/browse/HUDI-1722 Project: Apache Hudi

[jira] [Updated] (HUDI-1719) hive on spark/mr,Incremental query of the mor table, the partition field is incorrect

2021-03-25 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-1719: --- Description: now hudi use HoodieCombineHiveInputFormat to achieve Incremental query of the mor table. when

[jira] [Updated] (HUDI-1718) when query incr view of mor table which has Multi level partitions, the query failed

2021-03-25 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-1718: --- Description: HoodieCombineHiveInputFormat use "," to join mutil partitions, however hive use "/" to join

[jira] [Created] (HUDI-1720) when query incr view of mor table which has many delete records use sparksql/hive-beeline, StackOverflowError

2021-03-25 Thread tao meng (Jira)
tao meng created HUDI-1720: -- Summary: when query incr view of mor table which has many delete records use sparksql/hive-beeline, StackOverflowError Key: HUDI-1720 URL: https://issues.apache.org/jira/browse/HUDI-1720

[jira] [Created] (HUDI-1719) hive on spark/mr,Incremental query of the mor table, the partition field is incorrect

2021-03-25 Thread tao meng (Jira)
tao meng created HUDI-1719: -- Summary: hive on spark/mr,Incremental query of the mor table, the partition field is incorrect Key: HUDI-1719 URL: https://issues.apache.org/jira/browse/HUDI-1719 Project:

[jira] [Created] (HUDI-1718) when query incr view of mor table which has Multi level partitions, the query failed

2021-03-25 Thread tao meng (Jira)
tao meng created HUDI-1718: -- Summary: when query incr view of mor table which has Multi level partitions, the query failed Key: HUDI-1718 URL: https://issues.apache.org/jira/browse/HUDI-1718 Project:

[jira] [Created] (HUDI-1688) hudi write should uncache rdd, when the write operation is finnished

2021-03-14 Thread tao meng (Jira)
tao meng created HUDI-1688: -- Summary: hudi write should uncache rdd, when the write operation is finnished Key: HUDI-1688 URL: https://issues.apache.org/jira/browse/HUDI-1688 Project: Apache Hudi

[jira] [Updated] (HUDI-1676) Support SQL with spark3

2021-03-09 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-1676: --- Summary: Support SQL with spark3 (was: Support sql with spark3) > Support SQL with spark3 >

[jira] [Created] (HUDI-1677) Support Clustering and Metatable for SQL performance

2021-03-09 Thread tao meng (Jira)
tao meng created HUDI-1677: -- Summary: Support Clustering and Metatable for SQL performance Key: HUDI-1677 URL: https://issues.apache.org/jira/browse/HUDI-1677 Project: Apache Hudi Issue Type:

[jira] [Updated] (HUDI-1676) Support sql with spark3

2021-03-09 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-1676: --- Summary: Support sql with spark3 (was: support sql with spark3) > Support sql with spark3 >

[jira] [Created] (HUDI-1676) support sql with spark3

2021-03-09 Thread tao meng (Jira)
tao meng created HUDI-1676: -- Summary: support sql with spark3 Key: HUDI-1676 URL: https://issues.apache.org/jira/browse/HUDI-1676 Project: Apache Hudi Issue Type: Sub-task Components:

[jira] [Created] (HUDI-1675) Externalize all Hudi configurations

2021-03-09 Thread tao meng (Jira)
tao meng created HUDI-1675: -- Summary: Externalize all Hudi configurations Key: HUDI-1675 URL: https://issues.apache.org/jira/browse/HUDI-1675 Project: Apache Hudi Issue Type: Sub-task

[jira] [Updated] (HUDI-1662) Failed to query real-time view use hive/spark-sql when hudi mor table contains dateType

2021-03-04 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-1662: --- Description: step1: prepare raw DataFrame with DateType, and insert it to HudiMorTable

[jira] [Updated] (HUDI-1662) Failed to query real-time view use hive/spark-sql when hudi mor table contains dateType

2021-03-04 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-1662: --- Description: step1: prepare raw DataFrame with DateType, and insert it to HudiMorTable

[jira] [Updated] (HUDI-1662) Failed to query real-time view use hive/spark-sql when hudi mor table contains dateType

2021-03-04 Thread tao meng (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] tao meng updated HUDI-1662: --- Description: step1: prepare raw DataFrame with DateType, and insert it to HudiMorTable

  1   2   >