[GitHub] [hudi] xushiyan edited a comment on pull request #1849: Externalize test classes' configs

xushiyan edited a comment on pull request #1849: URL: https://github.com/apache/hudi/pull/1849#issuecomment-660803628 @vinothchandar The convention proposed here is for functional test requires HoodieConfig, it is to define its own `properties` file and avoid using `getConfigBuilder()`.

[GitHub] [hudi] xushiyan commented on pull request #1849: Externalize test classes' configs

xushiyan commented on pull request #1849: URL: https://github.com/apache/hudi/pull/1849#issuecomment-660803628 @vinothchandar if the approach looks good, i'll apply it to more test classes. thanks. This is an automated

[GitHub] [hudi] xushiyan opened a new pull request #1849: Externalize test classes' configs

xushiyan opened a new pull request #1849: URL: https://github.com/apache/hudi/pull/1849 - Move getConfig() to properties - Use common method to load configs for individual test class ## Committer checklist - [ ] Has a corresponding JIRA in PR title & commit - [ ]

[GitHub] [hudi] bvaradar commented on a change in pull request #1756: [HUDI-839] Introducing support for rollbacks using marker files

bvaradar commented on a change in pull request #1756: URL: https://github.com/apache/hudi/pull/1756#discussion_r457007064 ## File path: hudi-client/src/main/java/org/apache/hudi/config/HoodieWriteConfig.java ## @@ -632,6 +638,10 @@ public FileSystemViewStorageConfig

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #344

See Changes: -- [...truncated 2.35 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml

[GitHub] [hudi] vinothchandar commented on pull request #1756: [HUDI-839] Introducing support for rollbacks using marker files

vinothchandar commented on pull request #1756: URL: https://github.com/apache/hudi/pull/1756#issuecomment-660780134 test is failing on CI (linux?) while it passes locally.. Looking. This is an automated message from the

[hudi] branch hudi_test_suite_refactor updated (b5b2b7a -> 13e3d70)

This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a change to branch hudi_test_suite_refactor in repository https://gitbox.apache.org/repos/asf/hudi.git. discard b5b2b7a [HUDI-394] Provide a basic implementation of test suite add 13e3d70 [HUDI-394]

[GitHub] [hudi] mabin001 commented on pull request #1832: [HUDI-1099]: improve quality of the code calling the method.HiveSyncTool#syncPartitions

mabin001 commented on pull request #1832: URL: https://github.com/apache/hudi/pull/1832#issuecomment-660753692 > > > @mabin001 Thanks for you contributing, `addPartitionsToTable` and `updatePartitionsToTable ` are good to handle empty partitions and it is a more proper way IMHO. > >

[GitHub] [hudi] codecov-commenter commented on pull request #1848: WIP[HUDI-69] Support Spark Datasource for MOR table - RDD approach

codecov-commenter commented on pull request #1848: URL: https://github.com/apache/hudi/pull/1848#issuecomment-660751784 # [Codecov](https://codecov.io/gh/apache/hudi/pull/1848?src=pr=h1) Report > Merging [#1848](https://codecov.io/gh/apache/hudi/pull/1848?src=pr=desc) into

[hudi] branch hudi_test_suite_refactor updated (cc7c314 -> b5b2b7a)

This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a change to branch hudi_test_suite_refactor in repository https://gitbox.apache.org/repos/asf/hudi.git. discard cc7c314 [HUDI-394] Provide a basic implementation of test suite add b5b2b7a [HUDI-394]

[GitHub] [hudi] garyli1019 opened a new pull request #1848: WIP[HUDI-69] Support Spark Datasource for MOR table - RDD approach

garyli1019 opened a new pull request #1848: URL: https://github.com/apache/hudi/pull/1848 ## What is the purpose of the pull request This PR implements Spark Datasource for MOR table in the RDD approach. ParquetFileFormat approach PR: #1722 ## Brief change log -

[GitHub] [hudi] henrywu2019 commented on a change in pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

henrywu2019 commented on a change in pull request #1827: URL: https://github.com/apache/hudi/pull/1827#discussion_r456980751 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/common/HoodieEngineContext.java ## @@ -0,0 +1,48 @@ +/* + * Licensed to the

[GitHub] [hudi] codecov-commenter edited a comment on pull request #1149: [WIP] [HUDI-472] Introduce configurations and new modes of sorting for bulk_insert

codecov-commenter edited a comment on pull request #1149: URL: https://github.com/apache/hudi/pull/1149#issuecomment-652734921 # [Codecov](https://codecov.io/gh/apache/hudi/pull/1149?src=pr=h1) Report > Merging [#1149](https://codecov.io/gh/apache/hudi/pull/1149?src=pr=desc) into

[GitHub] [hudi] nsivabalan edited a comment on pull request #1149: [WIP] [HUDI-472] Introduce configurations and new modes of sorting for bulk_insert

nsivabalan edited a comment on pull request #1149: URL: https://github.com/apache/hudi/pull/1149#issuecomment-660731926 @yihua : I have fixed the flaky test. I assume you have resolved all feedback provided. If not, do let me know. As of now, I am assuming the patch is ready to review

[GitHub] [hudi] nsivabalan commented on pull request #1149: [WIP] [HUDI-472] Introduce configurations and new modes of sorting for bulk_insert

nsivabalan commented on pull request #1149: URL: https://github.com/apache/hudi/pull/1149#issuecomment-660731926 @vinothchandar : yes, the patch is ready for review. This is an automated message from the Apache Git Service.

[GitHub] [hudi] vinothchandar commented on a change in pull request #1756: [HUDI-839] Adding unit test for MarkerFiles,RollbackUtils, RollbackActionExecutor for markers and filelisting

vinothchandar commented on a change in pull request #1756: URL: https://github.com/apache/hudi/pull/1756#discussion_r456965828 ## File path: hudi-client/src/main/java/org/apache/hudi/io/HoodieWriteHandle.java ## @@ -97,28 +98,9 @@ public Path makeNewPath(String partitionPath)

[GitHub] [hudi] nsivabalan closed pull request #1762: [WIP] [HUDI-1013] Bulk insert Dataset

nsivabalan closed pull request #1762: URL: https://github.com/apache/hudi/pull/1762 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] nsivabalan commented on pull request #1762: [WIP] [HUDI-1013] Bulk insert Dataset

nsivabalan commented on pull request #1762: URL: https://github.com/apache/hudi/pull/1762#issuecomment-660721015 Closing this in favor of https://github.com/apache/hudi/pull/1834 This is an automated message from the Apache

[GitHub] [hudi] tooptoop4 commented on issue #1833: [SUPPORT] 100% update on 10mn keys in single partition slow

tooptoop4 commented on issue #1833: URL: https://github.com/apache/hudi/issues/1833#issuecomment-660715533 @bvaradar i noticed "There is insufficient memory for the Java Runtime Environment to continue." error so i reduced SPARK_WORKER_MEMORY (ie leave more room for OS memory). Now the

[GitHub] [hudi] vinothchandar commented on pull request #1762: [WIP] [HUDI-1013] Bulk insert Dataset

vinothchandar commented on pull request #1762: URL: https://github.com/apache/hudi/pull/1762#issuecomment-660684936 @nsivabalan can this be closed in favor of #1834 ? This is an automated message from the Apache Git

[jira] [Commented] (HUDI-1098) Marker file finalizing may block on a data file that was never written

[ https://issues.apache.org/jira/browse/HUDI-1098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17160778#comment-17160778 ] Vinoth Chandar commented on HUDI-1098: -- https://github.com/apache/hudi/issues/1764 has the context >

[GitHub] [hudi] vinothchandar merged pull request #1289: [HUDI-92] Provide reasonable names for Spark DAG stages in Hudi.

vinothchandar merged pull request #1289: URL: https://github.com/apache/hudi/pull/1289 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[hudi] branch master updated: [HUDI-92] Provide reasonable names for Spark DAG stages in HUDI. (#1289)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new b71f25f [HUDI-92] Provide reasonable names for

[jira] [Issue Comment Deleted] (HUDI-490) Add DeltaStream API example to hudi-examples

[ https://issues.apache.org/jira/browse/HUDI-490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roc Marshal updated HUDI-490: - Comment: was deleted (was: Could someone kindly assign this to me? Thank you.) > Add DeltaStream API

[jira] [Closed] (HUDI-1087) Realtime Record Reader needs to handle decimal types

[ https://issues.apache.org/jira/browse/HUDI-1087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-1087. --- > Realtime Record Reader needs to handle decimal types > > >