[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992206932 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * f5602d4c7e622973626effc61b831b36125234fd

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992204895 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #4295: [HUDI-2995] Enabling metadata table by default

hudi-bot commented on pull request #4295: URL: https://github.com/apache/hudi/pull/4295#issuecomment-992207002 ## CI report: * 774e98c8a46110d3d9a99ae0516bc94996eacefb UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] hudi-bot removed a comment on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot removed a comment on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992225315 ## CI report: * 6aa8c346a73b5d4a81e81cdada171758cdcca2db UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot commented on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992227670 ## CI report: * 6aa8c346a73b5d4a81e81cdada171758cdcca2db Azure:

[GitHub] [hudi] blxyct commented on issue #2104: [SUPPORT] MOR Hive sync - _rt table read issue

blxyct commented on issue #2104: URL: https://github.com/apache/hudi/issues/2104#issuecomment-992256192 @ashishmgofficial Hello, I have encountered the same problem. How did you solve it -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] danny0405 commented on issue #4249: [SUPPORT]FLINK CDC WRITE HUDI, restart job get exception:org.apache.hudi.org.apache.avro.InvalidAvroMagicException: Not an Avro data file

danny0405 commented on issue #4249: URL: https://github.com/apache/hudi/issues/4249#issuecomment-992261280 No, archiving and rollback is different action. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hudi] danny0405 commented on issue #4297: [SUPPORT]Flink SQL No FileSystem for scheme s3

danny0405 commented on issue #4297: URL: https://github.com/apache/hudi/issues/4297#issuecomment-992260632 you need to set up the s3 config in hadoop core-site file ? do you package the s3 package in the bundle jar ? -- This is an automated message from the Apache Git Service. To

[GitHub] [hudi] hudi-bot removed a comment on pull request #4294: [HUDI-2994] Add judgement to existed partitionPath in the catch code block for HU…

hudi-bot removed a comment on pull request #4294: URL: https://github.com/apache/hudi/pull/4294#issuecomment-992183402 ## CI report: * bcc67932c21d90d73c2f85bc4bc08a35411ae6f6 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4294: [HUDI-2994] Add judgement to existed partitionPath in the catch code block for HU…

hudi-bot commented on pull request #4294: URL: https://github.com/apache/hudi/pull/4294#issuecomment-992270158 ## CI report: * bcc67932c21d90d73c2f85bc4bc08a35411ae6f6 Azure:

[GitHub] [hudi] yihua commented on issue #4230: [SUPPORT] org.apache.hudi.exception.HoodieRemoteException: Failed to create marker file

yihua commented on issue #4230: URL: https://github.com/apache/hudi/issues/4230#issuecomment-992211057 @BenjMaq I'm not able to reproduce this problem on Spark2/3 with local FS or S3 on the 0.10.0 release. It would be great if you could run the steps again on 0.10.0 release and check if

[GitHub] [hudi] ngk2009 opened a new issue #4297: Flink SQL No FileSystem for scheme s3

ngk2009 opened a new issue #4297: URL: https://github.com/apache/hudi/issues/4297 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)? yes **Describe the problem you faced** I used the Hudi0.10 version on the

[GitHub] [hudi] zhangyue19921010 edited a comment on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 edited a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-992236967 Hi @yihua Thanks a lot for your attention. I agree with your opinion, indeed deleting archived files will lose historical instants information and affect

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992202872 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * f5602d4c7e622973626effc61b831b36125234fd

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992199532 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN *

[GitHub] [hudi] danny0405 commented on issue #4204: [SUPPORT] FlinkSQL write to Hudi, throw InvalidAvroMagicException: Not an Avro data file

danny0405 commented on issue #4204: URL: https://github.com/apache/hudi/issues/4204#issuecomment-992208857 I would fire a fix then ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot removed a comment on pull request #4295: [HUDI-2995] Enabling metadata table by default

hudi-bot removed a comment on pull request #4295: URL: https://github.com/apache/hudi/pull/4295#issuecomment-992207002 ## CI report: * 774e98c8a46110d3d9a99ae0516bc94996eacefb UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4295: [HUDI-2995] Enabling metadata table by default

hudi-bot commented on pull request #4295: URL: https://github.com/apache/hudi/pull/4295#issuecomment-992209236 ## CI report: * 774e98c8a46110d3d9a99ae0516bc94996eacefb Azure:

[GitHub] [hudi] danny0405 commented on issue #4249: [SUPPORT]FLINK CDC WRITE HUDI, restart job get exception:org.apache.hudi.org.apache.avro.InvalidAvroMagicException: Not an Avro data file

danny0405 commented on issue #4249: URL: https://github.com/apache/hudi/issues/4249#issuecomment-992209900 Then it is not fixed yet. Would fire a fix then ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [hudi] danny0405 opened a new pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

danny0405 opened a new pull request #4296: URL: https://github.com/apache/hudi/pull/4296 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.* ## What is the

[GitHub] [hudi] danny0405 commented on issue #4249: [SUPPORT]FLINK CDC WRITE HUDI, restart job get exception:org.apache.hudi.org.apache.avro.InvalidAvroMagicException: Not an Avro data file

danny0405 commented on issue #4249: URL: https://github.com/apache/hudi/issues/4249#issuecomment-992224166 Fired a fix in https://github.com/apache/hudi/pull/4296 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[jira] [Updated] (HUDI-2997) Skip the corrupt meta file for pending rollback action

[ https://issues.apache.org/jira/browse/HUDI-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2997: - Labels: pull-request-available (was: ) > Skip the corrupt meta file for pending rollback action

[jira] [Updated] (HUDI-2997) Skip the corrupt meta file for pending rollback action

[ https://issues.apache.org/jira/browse/HUDI-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Danny Chen updated HUDI-2997: - Summary: Skip the corrupt meta file for pending rollback action (was: Skip the corrupt meta file for

[GitHub] [hudi] zhangyue19921010 commented on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 commented on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-992236967 Hi @yihua Thanks a lot for your attention. I agree with your opinion, indeed deleting archived files will lose historical instants information and affect some hudi

[GitHub] [hudi] zhangyue19921010 edited a comment on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 edited a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-992236967 Hi @yihua Thanks a lot for your attention. I agree with your opinion, indeed deleting archived files will lose historical instants information and affect

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992206932 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992248491 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * f5602d4c7e622973626effc61b831b36125234fd

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992202872 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN *

[GitHub] [hudi] manojpec opened a new pull request #4295: [HUDI-2995] Enabling metadata table by default

manojpec opened a new pull request #4295: URL: https://github.com/apache/hudi/pull/4295 ## What is the purpose of the pull request Enabling metadata table by default. Note: https://github.com/apache/hudi/pull/4259 is needed before turning on metadata table by default. More

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992204895 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * f5602d4c7e622973626effc61b831b36125234fd

[jira] [Updated] (HUDI-2995) Enable metadata table by default

[ https://issues.apache.org/jira/browse/HUDI-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-2995: - Labels: pull-request-available (was: ) > Enable metadata table by default >

[GitHub] [hudi] manojpec commented on a change in pull request #4259: [HUDI-2962] Local process lock provider to guard single writer process with async table operations

manojpec commented on a change in pull request #4259: URL: https://github.com/apache/hudi/pull/4259#discussion_r767486991 ## File path: hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java ## @@ -44,7 +43,7 @@ // Enable the internal Metadata

[GitHub] [hudi] danny0405 commented on a change in pull request #4294: [HUDI-2994] Add judgement to existed partitionPath in the catch code block for HU…

danny0405 commented on a change in pull request #4294: URL: https://github.com/apache/hudi/pull/4294#discussion_r767493008 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java ## @@ -336,9 +336,10 @@ private void

[GitHub] [hudi] waywtdcc commented on issue #4249: [SUPPORT]FLINK CDC WRITE HUDI, restart job get exception:org.apache.hudi.org.apache.avro.InvalidAvroMagicException: Not an Avro data file

waywtdcc commented on issue #4249: URL: https://github.com/apache/hudi/issues/4249#issuecomment-992234729 > Then it is not fixed yet. Would fire a fix then ~ 'hoodie.fail.on.timeline.archiving': 'false' Can this setting avoid this error -- This is an automated message from the

[GitHub] [hudi] danny0405 commented on a change in pull request #4294: [HUDI-2994] Add judgement to existed partitionPath in the catch code block for HU…

danny0405 commented on a change in pull request #4294: URL: https://github.com/apache/hudi/pull/4294#discussion_r767494176 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java ## @@ -336,9 +336,10 @@ private void

[GitHub] [hudi] hudi-bot removed a comment on pull request #4253: [HUDI-2958] Automatically set spark.sql.parquet.writelegacyformat, when using bulkinsert to insert data which contains decimalType

hudi-bot removed a comment on pull request #4253: URL: https://github.com/apache/hudi/pull/4253#issuecomment-992165710 ## CI report: * 893fe09af34779c0ef98b732a418c9ba941a2bfc Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4253: [HUDI-2958] Automatically set spark.sql.parquet.writelegacyformat, when using bulkinsert to insert data which contains decimalType

hudi-bot commented on pull request #4253: URL: https://github.com/apache/hudi/pull/4253#issuecomment-992217519 ## CI report: * 893fe09af34779c0ef98b732a418c9ba941a2bfc Azure:

[jira] [Created] (HUDI-2997) Skip the corrupt meta file for rollback action

Danny Chen created HUDI-2997: Summary: Skip the corrupt meta file for rollback action Key: HUDI-2997 URL: https://issues.apache.org/jira/browse/HUDI-2997 Project: Apache Hudi Issue Type: Bug

[GitHub] [hudi] JoshuaZhuCN commented on issue #4275: [SUPPORT] How can I control the number of archive files

JoshuaZhuCN commented on issue #4275: URL: https://github.com/apache/hudi/issues/4275#issuecomment-992224816 > Hi @JoshuaZhuCN , I also meet this issue when using S3. Maybe the storage you used is not support append action. So that each time archive will create a new archive file no

[GitHub] [hudi] hudi-bot commented on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot commented on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992225315 ## CI report: * 6aa8c346a73b5d4a81e81cdada171758cdcca2db UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run azure`

[GitHub] [hudi] xiarixiaoyao edited a comment on pull request #4253: [HUDI-2958] Automatically set spark.sql.parquet.writelegacyformat, when using bulkinsert to insert data which contains decimalType

xiarixiaoyao edited a comment on pull request #4253: URL: https://github.com/apache/hudi/pull/4253#issuecomment-992232458 @nsivabalan @codope addressed all comments. could you pls review again， thanks The failure of UT has nothing to do with this pr. -- This is an automated message

[GitHub] [hudi] xiarixiaoyao commented on pull request #4253: [HUDI-2958] Automatically set spark.sql.parquet.writelegacyformat, when using bulkinsert to insert data which contains decimalType

xiarixiaoyao commented on pull request #4253: URL: https://github.com/apache/hudi/pull/4253#issuecomment-992232458 @nsivabalan @codope addressed all comments. The failure of UT has nothing to do with this pr. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] hudi-bot removed a comment on pull request #4295: [HUDI-2995] Enabling metadata table by default

hudi-bot removed a comment on pull request #4295: URL: https://github.com/apache/hudi/pull/4295#issuecomment-992209236 ## CI report: * 774e98c8a46110d3d9a99ae0516bc94996eacefb Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4295: [HUDI-2995] Enabling metadata table by default

hudi-bot commented on pull request #4295: URL: https://github.com/apache/hudi/pull/4295#issuecomment-992246042 ## CI report: * 774e98c8a46110d3d9a99ae0516bc94996eacefb Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot commented on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992276407 ## CI report: * 6aa8c346a73b5d4a81e81cdada171758cdcca2db Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot removed a comment on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992227670 ## CI report: * 6aa8c346a73b5d4a81e81cdada171758cdcca2db Azure:

[jira] [Created] (HUDI-2998) Claim RFC number for RFC for Consistent Hashing Index

Yuwei Xiao created HUDI-2998: Summary: Claim RFC number for RFC for Consistent Hashing Index Key: HUDI-2998 URL: https://issues.apache.org/jira/browse/HUDI-2998 Project: Apache Hudi Issue Type:

[GitHub] [hudi] yanenze commented on issue #4227: [SUPPORT] java.lang.IllegalStateException: Duplicate key Option

yanenze commented on issue #4227: URL: https://github.com/apache/hudi/issues/4227#issuecomment-992290789 > hi @yanenze , i also has this probloms，do you have push patch to github repository ? hi, this problem has been resolved in this PR: #4296 -- This is an automated message

[GitHub] [hudi] danny0405 commented on issue #4297: [SUPPORT]Flink SQL No FileSystem for scheme s3

danny0405 commented on issue #4297: URL: https://github.com/apache/hudi/issues/4297#issuecomment-992312806 Flink use its own plugin to support filesystems other than HDFS. Hudi adapter to different DFS by extending the `FileSystem` interface directly. -- This is an automated message

[GitHub] [hudi] ngk2009 removed a comment on issue #4297: [SUPPORT]Flink SQL No FileSystem for scheme s3

ngk2009 removed a comment on issue #4297: URL: https://github.com/apache/hudi/issues/4297#issuecomment-992271565 Thank you for your reply, I did not find in the guide about flink setting core-site.xml, and I do not plan to use hadoop environment, just build data lake analysis based on S3

[GitHub] [hudi] ngk2009 edited a comment on issue #4297: [SUPPORT]Flink SQL No FileSystem for scheme s3

ngk2009 edited a comment on issue #4297: URL: https://github.com/apache/hudi/issues/4297#issuecomment-992274434 > you need to set up the s3 config in hadoop core-site file ? do you package the s3 package in the bundle jar ? Thank you for your reply, I did not find in the guide about

[GitHub] [hudi] hudi-bot commented on pull request #4294: [HUDI-2994] Add judgement to existed partitionPath in the catch code block for HU…

hudi-bot commented on pull request #4294: URL: https://github.com/apache/hudi/pull/4294#issuecomment-992314782 ## CI report: * fcdfe250d344202f692aa3584c2914d27d3469a2 UNKNOWN * 08a07135cd9bb0eabb732553f53218477e3c9596 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4298: [HUDI-3001]clean up the marker directory when finish bootstrap operat…

hudi-bot removed a comment on pull request #4298: URL: https://github.com/apache/hudi/pull/4298#issuecomment-992384093 ## CI report: * 8dd35055d6206b3a916775f9685b41f11d1c72f0 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4298: [HUDI-3001]clean up the marker directory when finish bootstrap operat…

hudi-bot removed a comment on pull request #4298: URL: https://github.com/apache/hudi/pull/4298#issuecomment-992432083 ## CI report: * 8dd35055d6206b3a916775f9685b41f11d1c72f0 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4298: [HUDI-3001]clean up the marker directory when finish bootstrap operat…

hudi-bot commented on pull request #4298: URL: https://github.com/apache/hudi/pull/4298#issuecomment-992440014 ## CI report: * 8dd35055d6206b3a916775f9685b41f11d1c72f0 Azure:

[GitHub] [hudi] danny0405 commented on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

danny0405 commented on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992440629 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] hudi-bot commented on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot commented on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992283390 ## CI report: * 6aa8c346a73b5d4a81e81cdada171758cdcca2db Azure:

[GitHub] [hudi] parisni commented on issue #3731: [SUPPORT] Concurrent write (OCC) on distinct partitions random errors

parisni commented on issue #3731: URL: https://github.com/apache/hudi/issues/3731#issuecomment-992283484 with the 0.10.0, things looks better: I only get one kind of concurrency error. When running 4 concurrent writers into their own partition, I consistently get the bellow error

[GitHub] [hudi] hudi-bot removed a comment on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot removed a comment on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992288487 ## CI report: * 6c4e1a95ce514e9cda24aedcfb7aa758b37b477c Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot commented on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992334514 ## CI report: * 6c4e1a95ce514e9cda24aedcfb7aa758b37b477c Azure:

[GitHub] [hudi] BenjMaq commented on issue #4208: [SUPPORT] On Hudi 0.9.0 - Alter table throws java.lang.NoSuchMethodException: org.apache.hadoop.hive.ql.metadata.Hive.alterTable(java.lang.String, org

BenjMaq commented on issue #4208: URL: https://github.com/apache/hudi/issues/4208#issuecomment-992339520 Still having this issue in `0.10.0`. I use `hive-exec` `2.3.8`, might be the cause...? Still trying to figure out which part of my environment is causing this problem -- This is an

[jira] [Created] (HUDI-3001) clean up temp marker directory when finish bootstrap operation.

tao meng created HUDI-3001: -- Summary: clean up temp marker directory when finish bootstrap operation. Key: HUDI-3001 URL: https://issues.apache.org/jira/browse/HUDI-3001 Project: Apache Hudi

[GitHub] [hudi] hudi-bot removed a comment on pull request #4298: [HUDI-3001]clean up the marker directory when finish bootstrap operat…

hudi-bot removed a comment on pull request #4298: URL: https://github.com/apache/hudi/pull/4298#issuecomment-992381397 ## CI report: * 8dd35055d6206b3a916775f9685b41f11d1c72f0 UNKNOWN Bot commands @hudi-bot supports the following commands: - `@hudi-bot run

[GitHub] [hudi] hudi-bot commented on pull request #4298: [HUDI-3001]clean up the marker directory when finish bootstrap operat…

hudi-bot commented on pull request #4298: URL: https://github.com/apache/hudi/pull/4298#issuecomment-992384093 ## CI report: * 8dd35055d6206b3a916775f9685b41f11d1c72f0 Azure:

[GitHub] [hudi] forest455 commented on issue #3217: Error in running Deltastreamer in K8s Could not load key generator class org.apache.hudi.hive.NonpartitionedKeyGenerator [SUPPORT]

forest455 commented on issue #3217: URL: https://github.com/apache/hudi/issues/3217#issuecomment-992430616 hi can you tell me how to solve that problem. thanks . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] BenjMaq edited a comment on issue #4208: [SUPPORT] On Hudi 0.9.0 - Alter table throws java.lang.NoSuchMethodException: org.apache.hadoop.hive.ql.metadata.Hive.alterTable(java.lang.Stri

BenjMaq edited a comment on issue #4208: URL: https://github.com/apache/hudi/issues/4208#issuecomment-992339520 @YannByron @xushiyan I was able to go past this error by updating my `hive-exec` dependency to `1.2.1`. Now when I run the `alter table` statement, the table is

[hudi] branch master updated: [HUDI-2990] Sync to HMS when deleting partitions (#4291)

This is an automated email from the ASF dual-hosted git repository. leesf pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new dd96129 [HUDI-2990] Sync to HMS when deleting

[GitHub] [hudi] leesf merged pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

leesf merged pull request #4291: URL: https://github.com/apache/hudi/pull/4291 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [hudi] danny0405 commented on issue #4227: [SUPPORT] java.lang.IllegalStateException: Duplicate key Option

danny0405 commented on issue #4227: URL: https://github.com/apache/hudi/issues/4227#issuecomment-992279389 I have fixed it al-together in this PR: https://github.com/apache/hudi/pull/4296 -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [hudi] hudi-bot removed a comment on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot removed a comment on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992280939 ## CI report: * 6aa8c346a73b5d4a81e81cdada171758cdcca2db Azure:

[jira] [Created] (HUDI-2999) Consistent Hashing Index RFC

Yuwei Xiao created HUDI-2999: Summary: Consistent Hashing Index RFC Key: HUDI-2999 URL: https://issues.apache.org/jira/browse/HUDI-2999 Project: Apache Hudi Issue Type: Task

[GitHub] [hudi] hudi-bot removed a comment on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot removed a comment on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992285898 ## CI report: * 6aa8c346a73b5d4a81e81cdada171758cdcca2db Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992288414 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * f5602d4c7e622973626effc61b831b36125234fd

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992248491 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot commented on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992288487 ## CI report: * 6c4e1a95ce514e9cda24aedcfb7aa758b37b477c Azure:

[GitHub] [hudi] zhangyue19921010 edited a comment on pull request #4078: [HUDI-2833] Clean up unused archive files instead of expanding indefinitely.

zhangyue19921010 edited a comment on pull request #4078: URL: https://github.com/apache/hudi/pull/4078#issuecomment-992236967 Hi @yihua Thanks a lot for your attention. I agree with your opinion, indeed deleting archived files will lose historical instants information and affect

[GitHub] [hudi] BenjMaq edited a comment on issue #4230: [SUPPORT] org.apache.hudi.exception.HoodieRemoteException: Failed to create marker file

BenjMaq edited a comment on issue #4230: URL: https://github.com/apache/hudi/issues/4230#issuecomment-992335137 @yihua , @nsivabalan Using `hoodie.write.markers.type=direct` worked for me. Interestingly, I did not need this config on `0.9.0`. Did the default behavior change in

[GitHub] [hudi] BenjMaq commented on issue #4230: [SUPPORT] org.apache.hudi.exception.HoodieRemoteException: Failed to create marker file

BenjMaq commented on issue #4230: URL: https://github.com/apache/hudi/issues/4230#issuecomment-992335137 @yihua , @nsivabalan Using `hoodie.write.markers.type=direct` worked for me. Interestingly, I did not need this config on 0.9.0. Did the default behavior change in `0.10.0`?

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992299742 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN *

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992338345 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * f5602d4c7e622973626effc61b831b36125234fd

[GitHub] [hudi] xiarixiaoyao opened a new pull request #4298: [HUDI-3001]clean up the marker directory when finish bootstrap operat…

xiarixiaoyao opened a new pull request #4298: URL: https://github.com/apache/hudi/pull/4298 …ion. ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*

[GitHub] [hudi] hudi-bot commented on pull request #4298: [HUDI-3001]clean up the marker directory when finish bootstrap operat…

hudi-bot commented on pull request #4298: URL: https://github.com/apache/hudi/pull/4298#issuecomment-992432083 ## CI report: * 8dd35055d6206b3a916775f9685b41f11d1c72f0 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot removed a comment on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992368106 ## CI report: * e9555d5b58643240e65a6ec3efc3ac3dd6bb88f9 Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot commented on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992442067 ## CI report: * e9555d5b58643240e65a6ec3efc3ac3dd6bb88f9 Azure:

[GitHub] [hudi] york-yu-ctw commented on issue #4297: [SUPPORT]Flink SQL No FileSystem for scheme s3

york-yu-ctw commented on issue #4297: URL: https://github.com/apache/hudi/issues/4297#issuecomment-992454861 I have some tips 1. put `flink-s3-fs-hadoop` into `/opt/flink/lib` 2. add `hadoop-hdfs-client`, `hadoop-aws`, `hadoop-mapreduce-client-core` as well 3. if there is aws

[GitHub] [hudi] ngk2009 commented on issue #4297: [SUPPORT]Flink SQL No FileSystem for scheme s3

ngk2009 commented on issue #4297: URL: https://github.com/apache/hudi/issues/4297#issuecomment-992274434 > you need to set up the s3 config in hadoop core-site file ? do you package the s3 package in the bundle jar ? Thank you for your reply, I did not find in the guide about flink

[GitHub] [hudi] hudi-bot commented on pull request #4294: [HUDI-2994] Add judgement to existed partitionPath in the catch code block for HU…

hudi-bot commented on pull request #4294: URL: https://github.com/apache/hudi/pull/4294#issuecomment-992274266 ## CI report: * bcc67932c21d90d73c2f85bc4bc08a35411ae6f6 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4294: [HUDI-2994] Add judgement to existed partitionPath in the catch code block for HU…

hudi-bot removed a comment on pull request #4294: URL: https://github.com/apache/hudi/pull/4294#issuecomment-992272305 ## CI report: * bcc67932c21d90d73c2f85bc4bc08a35411ae6f6 Azure:

[GitHub] [hudi] hudi-bot removed a comment on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot removed a comment on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992276407 ## CI report: * 6aa8c346a73b5d4a81e81cdada171758cdcca2db Azure:

[GitHub] [hudi] hudi-bot commented on pull request #4296: [HUDI-2997] Skip the corrupt meta file for pending rollback action

hudi-bot commented on pull request #4296: URL: https://github.com/apache/hudi/pull/4296#issuecomment-992280939 ## CI report: * 6aa8c346a73b5d4a81e81cdada171758cdcca2db Azure:

[jira] [Created] (HUDI-3000) [UMBRELLA] Consistent Hashing Index

Yuwei Xiao created HUDI-3000: Summary: [UMBRELLA] Consistent Hashing Index Key: HUDI-3000 URL: https://issues.apache.org/jira/browse/HUDI-3000 Project: Apache Hudi Issue Type: New Feature

[GitHub] [hudi] XuQianJin-Stars removed a comment on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

XuQianJin-Stars removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992291674 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] XuQianJin-Stars commented on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

XuQianJin-Stars commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992291674 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992294965 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * f5602d4c7e622973626effc61b831b36125234fd

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992288414 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN *

[GitHub] [hudi] XuQianJin-Stars commented on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

XuQianJin-Stars commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992294565 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] XuQianJin-Stars removed a comment on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

XuQianJin-Stars removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992294565 @hudi-bot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [hudi] hudi-bot commented on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot commented on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992299742 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN * f5602d4c7e622973626effc61b831b36125234fd

[GitHub] [hudi] hudi-bot removed a comment on pull request #4291: [HUDI-2990] Sync to HMS when deleting partitions

hudi-bot removed a comment on pull request #4291: URL: https://github.com/apache/hudi/pull/4291#issuecomment-992294965 ## CI report: * ac71c00df089f959f3178eeb0c6db689f66c5737 UNKNOWN * cb41d556852651b47c2971a79f26b12e61ebcaed UNKNOWN *

[GitHub] [hudi] Arun-kc edited a comment on issue #4267: [SUPPORT] Hudi partition values not getting reflected in Athena

Arun-kc edited a comment on issue #4267: URL: https://github.com/apache/hudi/issues/4267#issuecomment-992131104 @Carl-Zhou-CN I tried `ALTER TABLE table_name RECOVER PARTITIONS;`, but its not working.

  1   2   3   4   5   6   7   >