[GitHub] [hudi] vinothchandar commented on pull request #1593: [WIP] [HUDI-839] Introducing rollback strategy using marker files

vinothchandar commented on pull request #1593: URL: https://github.com/apache/hudi/pull/1593#issuecomment-650699704 Closing in favor of #1756 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] vinothchandar closed pull request #1593: [WIP] [HUDI-839] Introducing rollback strategy using marker files

vinothchandar closed pull request #1593: URL: https://github.com/apache/hudi/pull/1593 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] vinothchandar commented on pull request #1100: [HUDI-289] Implement a test suite to support long running test for Hudi writing and querying end-end

vinothchandar commented on pull request #1100: URL: https://github.com/apache/hudi/pull/1100#issuecomment-650699629 @n3nash @yanghua do you mind me pushing some changes to this and land this? This is an automated message

[GitHub] [hudi] vinothchandar commented on pull request #1512: [HUDI-763] Add hoodie.table.base.file.format option to hoodie.properties file

vinothchandar commented on pull request #1512: URL: https://github.com/apache/hudi/pull/1512#issuecomment-650699549 @prashantwason @bvaradar IIUC some of this PR overlaps with the changes you are making as well. can you both clarify so we can close or revive this as needed..

[GitHub] [hudi] vinothchandar commented on pull request #1767: [MINOR] Adding test to WriteClient to validate update partition path with global bloom

vinothchandar commented on pull request #1767: URL: https://github.com/apache/hudi/pull/1767#issuecomment-650698829 @nsivabalan this is more than the 50 lines we agreed on for MINOR prefix. can you please file a JIRA and let me know if you think this quallifies for the MINOR prefix.

[jira] [Commented] (HUDI-839) Implement rollbacks using marker files instead of relying on commit metadata

[ https://issues.apache.org/jira/browse/HUDI-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147213#comment-17147213 ] Vinoth Chandar commented on HUDI-839: - yes on it.. will have a review done by monday > Implement

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #322

See Changes: -- [...truncated 2.43 KB...] toolchains.xml /home/jenkins/tools/maven/apache-maven-3.5.4/conf/logging: simplelogger.properties

[jira] [Commented] (HUDI-839) Implement rollbacks using marker files instead of relying on commit metadata

[ https://issues.apache.org/jira/browse/HUDI-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147183#comment-17147183 ] liwei commented on HUDI-839: hello [~vinoth], the pr 1756 have pass all the unit tests. can you help to review 

[GitHub] [hudi] lw309637554 edited a comment on pull request #1756: [HUDI-839] Adding unit test for MarkerFiles,RollbackUtils, RollbackActionExecutor for markers and filelisting

lw309637554 edited a comment on pull request #1756: URL: https://github.com/apache/hudi/pull/1756#issuecomment-650096565 > Took a quick pass at the three test classes you have added.. LGTM . > Will do a detailed pass once you confirm PR is indeed ready.. @vinothchandar hello，i

[GitHub] [hudi] yanghua commented on pull request #1558: [HUDI-796]: added deduping logic for upserts case

yanghua commented on pull request #1558: URL: https://github.com/apache/hudi/pull/1558#issuecomment-650673354 > @yanghua Can we merge this now? Will review soon. This is an automated message from the Apache Git

[jira] [Updated] (HUDI-349) Make cleaner retention based on time period to account for higher deviations in ingestion runs

[ https://issues.apache.org/jira/browse/HUDI-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratyaksh Sharma updated HUDI-349: -- Status: In Progress (was: Open) > Make cleaner retention based on time period to account for

[jira] [Assigned] (HUDI-349) Make cleaner retention based on time period to account for higher deviations in ingestion runs

[ https://issues.apache.org/jira/browse/HUDI-349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratyaksh Sharma reassigned HUDI-349: - Assignee: Pratyaksh Sharma (was: Aravind Suresh) > Make cleaner retention based on time

[GitHub] [hudi] pratyakshsharma commented on a change in pull request #1558: [HUDI-796]: added deduping logic for upserts case

pratyakshsharma commented on a change in pull request #1558: URL: https://github.com/apache/hudi/pull/1558#discussion_r446569482 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/SparkMain.java ## @@ -263,13 +265,26 @@ private static int

[GitHub] [hudi] pratyakshsharma commented on pull request #1562: [HUDI-837]: implemented custom deserializer for AvroKafkaSource

pratyakshsharma commented on pull request #1562: URL: https://github.com/apache/hudi/pull/1562#issuecomment-650631442 @n3nash @vinothchandar I guess we can merge this? :) This is an automated message from the Apache Git

[GitHub] [hudi] pratyakshsharma commented on pull request #1558: [HUDI-796]: added deduping logic for upserts case

pratyakshsharma commented on pull request #1558: URL: https://github.com/apache/hudi/pull/1558#issuecomment-650630601 @yanghua Can we merge this now? This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] pratyakshsharma commented on a change in pull request #1648: [HUDI-916]: added support for multiple input formats in TimestampBasedKeyGenerator

pratyakshsharma commented on a change in pull request #1648: URL: https://github.com/apache/hudi/pull/1648#discussion_r446538524 ## File path: hudi-spark/src/main/java/org/apache/hudi/keygen/parser/HoodieDateTimeParser.java ## @@ -0,0 +1,48 @@ +/* + * Licensed to the Apache

[jira] [Commented] (HUDI-983) Add Metrics section to asf-site

[ https://issues.apache.org/jira/browse/HUDI-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146989#comment-17146989 ] Hong Shen commented on HUDI-983: [~rxu] I have pull a request in [https://github.com/apache/hudi/pull/1769] 

[jira] [Updated] (HUDI-708) Add unit test for TempViewCommand

[ https://issues.apache.org/jira/browse/HUDI-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-708: Labels: pull-request-available (was: ) > Add unit test for TempViewCommand >

[GitHub] [hudi] hddong opened a new pull request #1770: [HUDI-708]Add temps show and unit test for TempViewCommand

hddong opened a new pull request #1770: URL: https://github.com/apache/hudi/pull/1770 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the

[GitHub] [hudi] shenh062326 opened a new pull request #1769: [DOC] Add document for the use of metrics system in Hudi.

shenh062326 opened a new pull request #1769: URL: https://github.com/apache/hudi/pull/1769 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of

[GitHub] [hudi] shenh062326 commented on pull request #1732: [HUDI-1004] Support update metrics in HoodieDeltaStreamerMetrics

shenh062326 commented on pull request #1732: URL: https://github.com/apache/hudi/pull/1732#issuecomment-650534678 > @leesf Given the limited scope of the pr, can we try and avoid copying code from other places Done.

[GitHub] [hudi] shenh062326 commented on a change in pull request #1732: [HUDI-1004] Support update metrics in HoodieDeltaStreamerMetrics

shenh062326 commented on a change in pull request #1732: URL: https://github.com/apache/hudi/pull/1732#discussion_r446508116 ## File path: hudi-client/src/main/java/org/apache/hudi/metrics/HudiGauge.java ## @@ -25,22 +25,21 @@ * Similar to {@link Gauge}, but metric value is

[GitHub] [hudi] leesf commented on a change in pull request #1768: [HUDI-1054][Peformance] Several performance fixes during finalizing writes

leesf commented on a change in pull request #1768: URL: https://github.com/apache/hudi/pull/1768#discussion_r446502715 ## File path: hudi-common/pom.xml ## @@ -147,6 +147,16 @@ test + + + org.apache.spark + spark-core_${scala.binary.version} +

[GitHub] [hudi] leesf commented on pull request #1767: [MINOR] Adding test to WriteClient to validate update partition path with global bloom

leesf commented on pull request #1767: URL: https://github.com/apache/hudi/pull/1767#issuecomment-650524267 @xushiyan Could you please review this PR since you did the related work before. This is an automated message from

[GitHub] [hudi] leesf commented on a change in pull request #1761: [MINOR] Add documentation for using multi-column table keys and for n…

leesf commented on a change in pull request #1761: URL: https://github.com/apache/hudi/pull/1761#discussion_r446499358 ## File path: docs/_docs/2_2_writing_data.md ## @@ -176,15 +176,49 @@ In some cases, you may want to migrate your existing table into Hudi beforehand. ##

[GitHub] [hudi] leesf commented on a change in pull request #1761: [MINOR] Add documentation for using multi-column table keys and for n…

leesf commented on a change in pull request #1761: URL: https://github.com/apache/hudi/pull/1761#discussion_r446499146 ## File path: docs/_docs/2_2_writing_data.md ## @@ -176,15 +176,49 @@ In some cases, you may want to migrate your existing table into Hudi beforehand. ##

[GitHub] [hudi] leesf commented on a change in pull request #1761: [MINOR] Add documentation for using multi-column table keys and for n…

leesf commented on a change in pull request #1761: URL: https://github.com/apache/hudi/pull/1761#discussion_r446498933 ## File path: docs/_docs/2_2_writing_data.md ## @@ -176,15 +176,49 @@ In some cases, you may want to migrate your existing table into Hudi beforehand. ##