[GitHub] [incubator-hudi] vinothchandar commented on issue #528: Does hudi support composite primary key upserts?

vinothchandar commented on issue #528: URL: https://github.com/apache/incubator-hudi/issues/528#issuecomment-621645502 @Murthy-Palla you can configure your key generator and using the `ComplexKeyGenerator` https://github.com/apache/incubator-hudi/blob/master/hudi-spark/src/main/java/or

[GitHub] [incubator-hudi] vinothchandar opened a new pull request #1575: [MINOR] Reorder HoodieTimeline#compareTimestamp arguments for better readability

vinothchandar opened a new pull request #1575: URL: https://github.com/apache/incubator-hudi/pull/1575 - reads nicely as (instantTime1, GREATER_OR_EQUAL, instantTime2) etc ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.or

[jira] [Assigned] (HUDI-850) Avoid unnecessary listings in incremental cleaning mode

[ https://issues.apache.org/jira/browse/HUDI-850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-850: --- Assignee: Balaji Varadarajan (was: Vinoth Chandar) > Avoid unnecessary listings in in

[jira] [Created] (HUDI-850) Avoid unnecessary listings in incremental cleaning mode

Vinoth Chandar created HUDI-850: --- Summary: Avoid unnecessary listings in incremental cleaning mode Key: HUDI-850 URL: https://issues.apache.org/jira/browse/HUDI-850 Project: Apache Hudi (incubating)

[GitHub] [incubator-hudi] hddong commented on pull request #1574: [HUDI-701]Add unit test for HDFSParquetImportCommand

hddong commented on pull request #1574: URL: https://github.com/apache/incubator-hudi/pull/1574#issuecomment-621601455 @yanghua : It's green now, please have a review when free. This is an automated message from the Apache Gi

[GitHub] [incubator-hudi] bvaradar commented on issue #1569: [SUPPORT] Audit Feature In A PartitionPath

bvaradar commented on issue #1569: URL: https://github.com/apache/incubator-hudi/issues/1569#issuecomment-621600324 Great. Thanks for using Hudi. This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [incubator-hudi] codecov-io edited a comment on pull request #1574: [HUDI-701]Add unit test for HDFSParquetImportCommand

codecov-io edited a comment on pull request #1574: URL: https://github.com/apache/incubator-hudi/pull/1574#issuecomment-621599057 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1574?src=pr&el=h1) Report > Merging [#1574](https://codecov.io/gh/apache/incubator-hudi/pull/15

[GitHub] [incubator-hudi] codecov-io commented on pull request #1574: [HUDI-701]Add unit test for HDFSParquetImportCommand

codecov-io commented on pull request #1574: URL: https://github.com/apache/incubator-hudi/pull/1574#issuecomment-621599057 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1574?src=pr&el=h1) Report > Merging [#1574](https://codecov.io/gh/apache/incubator-hudi/pull/1574?src=

[GitHub] [incubator-hudi] codecov-io edited a comment on pull request #1572: [HUDI-836] Implement datadog metric reporter

codecov-io edited a comment on pull request #1572: URL: https://github.com/apache/incubator-hudi/pull/1572#issuecomment-621450664 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1572?src=pr&el=h1) Report > Merging [#1572](https://codecov.io/gh/apache/incubator-hudi/pull/15

[GitHub] [incubator-hudi] codecov-io edited a comment on pull request #1572: [HUDI-836] Implement datadog metric reporter

codecov-io edited a comment on pull request #1572: URL: https://github.com/apache/incubator-hudi/pull/1572#issuecomment-621450664 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1572?src=pr&el=h1) Report > Merging [#1572](https://codecov.io/gh/apache/incubator-hudi/pull/15

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #263

See Changes: -- [...truncated 2.35 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml /home/jenkins/tools/maven/apache-maven-3.5.

[GitHub] [incubator-hudi] yanghua commented on pull request #1574: [HUDI-701]Add unit test for HDFSParquetImportCommand

yanghua commented on pull request #1574: URL: https://github.com/apache/incubator-hudi/pull/1574#issuecomment-621580694 @hddong Travis is red. Pls check it. This is an automated message from the Apache Git Service. To respond

[jira] [Updated] (HUDI-701) Add unit test for HDFSParquetImportCommand

[ https://issues.apache.org/jira/browse/HUDI-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-701: Labels: pull-request-available (was: ) > Add unit test for HDFSParquetImportCommand > --

[GitHub] [incubator-hudi] hddong opened a new pull request #1574: [HUDI-701]Add unit test for HDFSParquetImportCommand

hddong opened a new pull request #1574: URL: https://github.com/apache/incubator-hudi/pull/1574 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose o

[GitHub] [incubator-hudi] yizhu-wish opened a new pull request #1573: Mongo operation class

yizhu-wish opened a new pull request #1573: URL: https://github.com/apache/incubator-hudi/pull/1573 Added Operation utility class for Op-log processing. This is an automated message from the Apache Git Service. To respond to

[jira] [Updated] (HUDI-849) Turn on incremental Syncing by default for DeltaStreamer and spark streaming cases

[ https://issues.apache.org/jira/browse/HUDI-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-849: Status: Open (was: New) > Turn on incremental Syncing by default for DeltaStreamer and spark

[jira] [Assigned] (HUDI-849) Turn on incremental Syncing by default for DeltaStreamer and spark streaming cases

[ https://issues.apache.org/jira/browse/HUDI-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-849: --- Assignee: Balaji Varadarajan > Turn on incremental Syncing by default for DeltaStreame

[jira] [Created] (HUDI-849) Turn on incremental Syncing by default for DeltaStreamer and spark streaming cases

Balaji Varadarajan created HUDI-849: --- Summary: Turn on incremental Syncing by default for DeltaStreamer and spark streaming cases Key: HUDI-849 URL: https://issues.apache.org/jira/browse/HUDI-849 Pr

[jira] [Assigned] (HUDI-848) Turn on embedded timeline server by default for all writes

[ https://issues.apache.org/jira/browse/HUDI-848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-848: --- Assignee: Balaji Varadarajan > Turn on embedded timeline server by default for all wri

[jira] [Created] (HUDI-848) Turn on embedded timeline server by default for all writes

Balaji Varadarajan created HUDI-848: --- Summary: Turn on embedded timeline server by default for all writes Key: HUDI-848 URL: https://issues.apache.org/jira/browse/HUDI-848 Project: Apache Hudi (incub

[jira] [Updated] (HUDI-848) Turn on embedded timeline server by default for all writes

[ https://issues.apache.org/jira/browse/HUDI-848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-848: Status: Open (was: New) > Turn on embedded timeline server by default for all writes > -

[jira] [Updated] (HUDI-846) Turn on incremental cleaning bu default in 0.6.0

[ https://issues.apache.org/jira/browse/HUDI-846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-846: Parent: HUDI-847 Issue Type: Sub-task (was: Improvement) > Turn on incremental clean

[jira] [Created] (HUDI-847) Umbrella ticket for tuning default configs for 0.6.0

Balaji Varadarajan created HUDI-847: --- Summary: Umbrella ticket for tuning default configs for 0.6.0 Key: HUDI-847 URL: https://issues.apache.org/jira/browse/HUDI-847 Project: Apache Hudi (incubating)

[jira] [Assigned] (HUDI-847) Umbrella ticket for tuning default configs for 0.6.0

[ https://issues.apache.org/jira/browse/HUDI-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-847: --- Assignee: Balaji Varadarajan > Umbrella ticket for tuning default configs for 0.6.0 >

[jira] [Updated] (HUDI-847) Umbrella ticket for tuning default configs for 0.6.0

[ https://issues.apache.org/jira/browse/HUDI-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-847: Status: Open (was: New) > Umbrella ticket for tuning default configs for 0.6.0 > ---

[jira] [Assigned] (HUDI-846) Turn on incremental cleaning bu default in 0.6.0

[ https://issues.apache.org/jira/browse/HUDI-846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-846: --- Assignee: Balaji Varadarajan (was: lamber-ken) > Turn on incremental cleaning bu defa

[GitHub] [incubator-hudi] codecov-io commented on pull request #1572: [HUDI-836] Implement datadog metric reporter

codecov-io commented on pull request #1572: URL: https://github.com/apache/incubator-hudi/pull/1572#issuecomment-621450664 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1572?src=pr&el=h1) Report > Merging [#1572](https://codecov.io/gh/apache/incubator-hudi/pull/1572?src=

[jira] [Updated] (HUDI-836) Implement datadog metrics reporter

[ https://issues.apache.org/jira/browse/HUDI-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-836: Status: In Progress (was: Open) > Implement datadog metrics reporter > -- >

[jira] [Updated] (HUDI-836) Implement datadog metrics reporter

[ https://issues.apache.org/jira/browse/HUDI-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raymond Xu updated HUDI-836: Status: Open (was: New) > Implement datadog metrics reporter > -- > >

[GitHub] [incubator-hudi] xushiyan opened a new pull request #1572: [HUDI-836] Implement datadog metric reporter

xushiyan opened a new pull request #1572: URL: https://github.com/apache/incubator-hudi/pull/1572 * Add a new metric reporter type for Datadog * Implement reporting support for Datadog http api ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit

[jira] [Updated] (HUDI-836) Implement datadog metrics reporter

[ https://issues.apache.org/jira/browse/HUDI-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-836: Labels: pull-request-available (was: ) > Implement datadog metrics reporter > --

[jira] [Assigned] (HUDI-846) Turn on incremental cleaning bu default in 0.6.0

[ https://issues.apache.org/jira/browse/HUDI-846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan reassigned HUDI-846: --- Assignee: lamber-ken > Turn on incremental cleaning bu default in 0.6.0 >

[jira] [Created] (HUDI-846) Turn on incremental cleaning bu default in 0.6.0

Balaji Varadarajan created HUDI-846: --- Summary: Turn on incremental cleaning bu default in 0.6.0 Key: HUDI-846 URL: https://issues.apache.org/jira/browse/HUDI-846 Project: Apache Hudi (incubating)

[jira] [Updated] (HUDI-846) Turn on incremental cleaning bu default in 0.6.0

[ https://issues.apache.org/jira/browse/HUDI-846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-846: Status: Open (was: New) > Turn on incremental cleaning bu default in 0.6.0 > ---

[jira] [Commented] (HUDI-677) Abstract/Refactor all transaction management logic into a set of classes

[ https://issues.apache.org/jira/browse/HUDI-677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17095782#comment-17095782 ] Vinoth Chandar commented on HUDI-677: - cc [~pwason] this is also something we need to d

[jira] [Commented] (HUDI-842) Implementation plan for RFC 15 (File Listing and Query Planning Improvements))

[ https://issues.apache.org/jira/browse/HUDI-842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17095723#comment-17095723 ] Vinoth Chandar commented on HUDI-842: - [~pwason] I linked a couple of tickets here, tha

[jira] [Updated] (HUDI-684) Introduce abstraction for writing and reading and compacting from FileGroups

[ https://issues.apache.org/jira/browse/HUDI-684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinoth Chandar updated HUDI-684: Status: Open (was: New) > Introduce abstraction for writing and reading and compacting from FileGrou

[GitHub] [incubator-hudi] dh376 opened a new issue #1571: [SUPPORT] Hudi IllegalArgumentException Wrong FS

dh376 opened a new issue #1571: URL: https://github.com/apache/incubator-hudi/issues/1571 **_Tips before filing an issue_** - Have you gone through our [FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)? - Join the mailing list to engage in conversations and get fast

[GitHub] [incubator-hudi] pratyakshsharma commented on pull request #1558: [HUDI-796]: added deduping logic for upserts case

pratyakshsharma commented on pull request #1558: URL: https://github.com/apache/incubator-hudi/pull/1558#issuecomment-621253677 > @pratyakshsharma : Can we do two kinds of de-duplicate together? First, de-duplicate by different commits, then de-duplicate same commit, they are not incompati

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1532: [HUDI-794]: implemented optional use of --config-folder option in HoodieDeltaStreamer

pratyakshsharma commented on a change in pull request #1532: URL: https://github.com/apache/incubator-hudi/pull/1532#discussion_r417355146 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/DeltaStreamerUtility.java ## @@ -0,0 +1,128 @@ +/* + * Licensed to th

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1402: [HUDI-407] Adding Simple Index

nsivabalan commented on a change in pull request #1402: URL: https://github.com/apache/incubator-hudi/pull/1402#discussion_r417337325 ## File path: hudi-client/src/main/java/org/apache/hudi/index/HoodieGlobalSimpleIndex.java ## @@ -0,0 +1,179 @@ +/* + * Licensed to the Apache

[GitHub] [incubator-hudi] nsivabalan commented on pull request #1402: [HUDI-407] Adding Simple Index

nsivabalan commented on pull request #1402: URL: https://github.com/apache/incubator-hudi/pull/1402#issuecomment-621225622 @vinothchandar : have addressed your comments. This is an automated message from the Apache Git Servic

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1402: [HUDI-407] Adding Simple Index

nsivabalan commented on a change in pull request #1402: URL: https://github.com/apache/incubator-hudi/pull/1402#discussion_r417046732 ## File path: hudi-client/src/main/java/org/apache/hudi/index/HoodieGlobalSimpleIndex.java ## @@ -0,0 +1,157 @@ +/* + * Licensed to the Apache

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1402: [HUDI-407] Adding Simple Index

nsivabalan commented on a change in pull request #1402: URL: https://github.com/apache/incubator-hudi/pull/1402#discussion_r417046732 ## File path: hudi-client/src/main/java/org/apache/hudi/index/HoodieGlobalSimpleIndex.java ## @@ -0,0 +1,157 @@ +/* + * Licensed to the Apache

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1402: [HUDI-407] Adding Simple Index

nsivabalan commented on a change in pull request #1402: URL: https://github.com/apache/incubator-hudi/pull/1402#discussion_r417046515 ## File path: hudi-client/src/main/java/org/apache/hudi/index/HoodieGlobalSimpleIndex.java ## @@ -0,0 +1,157 @@ +/* + * Licensed to the Apache

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1402: [HUDI-407] Adding Simple Index

nsivabalan commented on a change in pull request #1402: URL: https://github.com/apache/incubator-hudi/pull/1402#discussion_r417046515 ## File path: hudi-client/src/main/java/org/apache/hudi/index/HoodieGlobalSimpleIndex.java ## @@ -0,0 +1,157 @@ +/* + * Licensed to the Apache

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1402: [HUDI-407] Adding Simple Index

nsivabalan commented on a change in pull request #1402: URL: https://github.com/apache/incubator-hudi/pull/1402#discussion_r417046515 ## File path: hudi-client/src/main/java/org/apache/hudi/index/HoodieGlobalSimpleIndex.java ## @@ -0,0 +1,157 @@ +/* + * Licensed to the Apache

[GitHub] [incubator-hudi] nsivabalan commented on a change in pull request #1402: [HUDI-407] Adding Simple Index

nsivabalan commented on a change in pull request #1402: URL: https://github.com/apache/incubator-hudi/pull/1402#discussion_r417046515 ## File path: hudi-client/src/main/java/org/apache/hudi/index/HoodieGlobalSimpleIndex.java ## @@ -0,0 +1,157 @@ +/* + * Licensed to the Apache

[GitHub] [incubator-hudi] Murthy-Palla commented on issue #528: Does hudi support composite primary key upserts?

Murthy-Palla commented on issue #528: URL: https://github.com/apache/incubator-hudi/issues/528#issuecomment-621140892 @vinothchandar can you let me know syntax to specify composite keys in RECORDKEY_FIELD_OPT_KEY while creating a hudi dataset using spark datasource api. ---

[jira] [Updated] (HUDI-702) Add unit test for HoodieLogFileCommand

[ https://issues.apache.org/jira/browse/HUDI-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang updated HUDI-702: -- Fix Version/s: 0.6.0 > Add unit test for HoodieLogFileCommand > -- > >

[jira] [Closed] (HUDI-702) Add unit test for HoodieLogFileCommand

[ https://issues.apache.org/jira/browse/HUDI-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang closed HUDI-702. - Resolution: Done Done via master branch: 9059bce977cee98e2d65769622c46a1941c563dd > Add unit test for HoodieLogFil

[jira] [Updated] (HUDI-702) Add unit test for HoodieLogFileCommand

[ https://issues.apache.org/jira/browse/HUDI-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang updated HUDI-702: -- Status: Open (was: New) > Add unit test for HoodieLogFileCommand > -- > >

[incubator-hudi] branch master updated: [HUDI-702] Add test for HoodieLogFileCommand (#1522)

This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 9059bce [HUDI-702] Add test for Hoodi