[GitHub] [hudi] bvaradar commented on a change in pull request #2064: WIP - [HUDI-842] Implementation of HUDI RFC-15.

2020-09-25 Thread GitBox
bvaradar commented on a change in pull request #2064: URL: https://github.com/apache/hudi/pull/2064#discussion_r495419947 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java ## @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache Software

[GitHub] [hudi] liujinhui1994 commented on pull request #2074: [HUDI-1233] Deltastreamer Kafka consumption delay reporting indicators

2020-09-25 Thread GitBox
liujinhui1994 commented on pull request #2074: URL: https://github.com/apache/hudi/pull/2074#issuecomment-699388007 @yanghua please help review This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hudi] bvaradar commented on a change in pull request #2064: WIP - [HUDI-842] Implementation of HUDI RFC-15.

2020-09-25 Thread GitBox
bvaradar commented on a change in pull request #2064: URL: https://github.com/apache/hudi/pull/2064#discussion_r495413600 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java ## @@ -0,0 +1,202 @@ +/* + * Licensed to the Apache Software

[GitHub] [hudi] dugenkui03 opened a new pull request #2115: [MINOR] Mark started and shutdownRequested with volatile.

2020-09-25 Thread GitBox
dugenkui03 opened a new pull request #2115: URL: https://github.com/apache/hudi/pull/2115 Mark `started` and `shutdownRequested` with volatile. Add getter method for `started`. This is an automated message from the

[GitHub] [hudi] dugenkui03 closed issue #2114: [SUPPORT]Replace org.apache.hudi.common.util.StringUtils with JDK and org.apache.commons.lang3.StringUtils

2020-09-25 Thread GitBox
dugenkui03 closed issue #2114: URL: https://github.com/apache/hudi/issues/2114 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] dugenkui03 opened a new issue #2114: [SUPPORT]Replace org.apache.hudi.common.util.StringUtils with JDK and org.apache.commons.lang3.StringUtils

2020-09-25 Thread GitBox
dugenkui03 opened a new issue #2114: URL: https://github.com/apache/hudi/issues/2114 **Describe the problem you faced** I find that most methods in `hudi.StringUtils` is easy to be replaced by JDK and `lang3.StringUtils`. and there is no special operation in `hudi.StringUtils`. I think

[jira] [Updated] (HUDI-1300) Replace org.apache.hudi.common.util.StringUtils with JDK and org.apache.commons.lang3.StringUtils

2020-09-25 Thread dugenkui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dugenkui updated HUDI-1300: --- Description: I find that most methods in hudi.StringUtils is easy to be replaced by JDK and

[jira] [Updated] (HUDI-1300) Replace org.apache.hudi.common.util.StringUtils with JDK and org.apache.commons.lang3.StringUtils

2020-09-25 Thread dugenkui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dugenkui updated HUDI-1300: --- Description: I find that most methods in hudi.StringUtils is easy to be replaced by JDK and

[jira] [Updated] (HUDI-1300) Replace org.apache.hudi.common.util.StringUtils with JDK and org.apache.commons.lang3.StringUtils

2020-09-25 Thread dugenkui (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dugenkui updated HUDI-1300: --- Description: I find that hudi.StringUtils is easy to be replaced by JDK and lang3.StringUtils. and there is

[jira] [Created] (HUDI-1300) Replace org.apache.hudi.common.util.StringUtils with JDK and org.apache.commons.lang3.StringUtils

2020-09-25 Thread dugenkui (Jira)
dugenkui created HUDI-1300: -- Summary: Replace org.apache.hudi.common.util.StringUtils with JDK and org.apache.commons.lang3.StringUtils Key: HUDI-1300 URL: https://issues.apache.org/jira/browse/HUDI-1300

[GitHub] [hudi] wangxianghu commented on pull request #2105: [MINOR] Fix ClassCastException when use QuickstartUtils generate data

2020-09-25 Thread GitBox
wangxianghu commented on pull request #2105: URL: https://github.com/apache/hudi/pull/2105#issuecomment-699251448 > LGTM. I believe this change was introduced a week back or so from this pr - #2071 yes, it's a small oversight. thanks for your review

[GitHub] [hudi] nsivabalan commented on pull request #2092: [HUDI-1285] Fix merge on read DAG to make docker demo pass

2020-09-25 Thread GitBox
nsivabalan commented on pull request #2092: URL: https://github.com/apache/hudi/pull/2092#issuecomment-699219984 @n3nash : I tested this patch and looks good. once you remove the rollback node, we can merge it. This is an

[jira] [Resolved] (HUDI-1213) Set Default for the bootstrap config : hoodie.bootstrap.full.input.provider

2020-09-25 Thread Udit Mehrotra (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Udit Mehrotra resolved HUDI-1213. - Resolution: Fixed > Set Default for the bootstrap config : hoodie.bootstrap.full.input.provider >

[GitHub] [hudi] tooptoop4 commented on issue #2110: [SUPPORT] Executor memory recommendation

2020-09-25 Thread GitBox
tooptoop4 commented on issue #2110: URL: https://github.com/apache/hudi/issues/2110#issuecomment-699071257 @n3nash " if your target file is not very large (<256 MB)" do you mean the size of the incoming CSV or the size of the existing table? Does the size of the existing table matter at

[GitHub] [hudi] satishkotha commented on a change in pull request #2048: [HUDI-1072][WIP] Introduce REPLACE top level action

2020-09-25 Thread GitBox
satishkotha commented on a change in pull request #2048: URL: https://github.com/apache/hudi/pull/2048#discussion_r495137989 ## File path: hudi-client/src/main/java/org/apache/hudi/client/HoodieWriteClient.java ## @@ -586,24 +602,39 @@ public String startCommit() { *

[hudi] branch master updated (2eaba09 -> 1dd6635)

2020-09-25 Thread bhavanisudha
This is an automated email from the ASF dual-hosted git repository. bhavanisudha pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 2eaba09 [HUDI-544] Archived commits command code cleanup (#1242) add 1dd6635 [MINOR] Fix

[GitHub] [hudi] bhasudha merged pull request #2105: [MINOR] Fix ClassCastException when use QuickstartUtils generate data

2020-09-25 Thread GitBox
bhasudha merged pull request #2105: URL: https://github.com/apache/hudi/pull/2105 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] n3nash commented on issue #1958: [SUPPORT] Global Indexes return old partition value when querying Hive tables

2020-09-25 Thread GitBox
n3nash commented on issue #1958: URL: https://github.com/apache/hudi/issues/1958#issuecomment-699031170 We're waiting on the PR to be landed after which we can close this. This is an automated message from the Apache Git

[hudi] branch master updated (6837118 -> 2eaba09)

2020-09-25 Thread nagarwal
This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 6837118 [MINOR] Improve description (#2113) add 2eaba09 [HUDI-544] Archived commits command code cleanup

[GitHub] [hudi] n3nash merged pull request #1242: [HUDI-544] Archived commits command code cleanup

2020-09-25 Thread GitBox
n3nash merged pull request #1242: URL: https://github.com/apache/hudi/pull/1242 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] n3nash commented on pull request #1978: [HUDI-1184] Fix the support of hbase index partition path change

2020-09-25 Thread GitBox
n3nash commented on pull request #1978: URL: https://github.com/apache/hudi/pull/1978#issuecomment-699029537 @hj2016 Can you rebase and squash your commits please ? This is an automated message from the Apache Git Service.

[GitHub] [hudi] n3nash commented on issue #2104: [SUPPORT] MOR Hive sync - _rt table read issue

2020-09-25 Thread GitBox
n3nash commented on issue #2104: URL: https://github.com/apache/hudi/issues/2104#issuecomment-699028582 Okay, which jar have you deployed on the Hive server, is it [hoodie-hadoop-mr-bundle](https://github.com/apache/hudi/tree/master/packaging/hudi-hadoop-mr-bundle) ? For the Hive server

[GitHub] [hudi] n3nash commented on issue #2101: [SUPPORT]Unable to interpret Child JSON fields value as a separate columns rather it is loaded as one single field value. Any way to interpret that.

2020-09-25 Thread GitBox
n3nash commented on issue #2101: URL: https://github.com/apache/hudi/issues/2101#issuecomment-699027752 @getniz Those are good questions, we're all learners here :) You can definitely create a schema in confluent schema registry but the flattening will depend on what is your schema

[GitHub] [hudi] ashishmgofficial commented on issue #2104: [SUPPORT] MOR Hive sync - _rt table read issue

2020-09-25 Thread GitBox
ashishmgofficial commented on issue #2104: URL: https://github.com/apache/hudi/issues/2104#issuecomment-699027470 @n3nash Im using hudi 0.6.0 from maven This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] n3nash commented on issue #2110: [SUPPORT] Executor memory recommendation

2020-09-25 Thread GitBox
n3nash commented on issue #2110: URL: https://github.com/apache/hudi/issues/2110#issuecomment-699025467 @tooptoop4 If you're talking about executor memory, in this particular scenario, you can start with 2GB. In general, the upsert code performs a hash merge for which it uses a spillable

[GitHub] [hudi] n3nash commented on issue #2104: [SUPPORT] MOR Hive sync - _rt table read issue

2020-09-25 Thread GitBox
n3nash commented on issue #2104: URL: https://github.com/apache/hudi/issues/2104#issuecomment-699023400 @ashishmgofficial Looks like a `class not found` issue. Can you tell me which hoodie jar have you deployed on your Hive server ?

[GitHub] [hudi] n3nash commented on issue #2108: [SUPPORT]Submit rollback -->Pending job --> kill YARN --> lost data

2020-09-25 Thread GitBox
n3nash commented on issue #2108: URL: https://github.com/apache/hudi/issues/2108#issuecomment-699022804 @JiaDe-Wu Can you please list the entire `.hoodie` folder and show the contents ? This is an automated message from the

[hudi] branch master updated (83d2e03 -> 6837118)

2020-09-25 Thread leesf
This is an automated email from the ASF dual-hosted git repository. leesf pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git. from 83d2e03 [MINOR] Adding scripts to checkout and push to PRs (#2109) add 6837118 [MINOR] Improve description

[GitHub] [hudi] leesf merged pull request #2113: [MINOR] fix typo

2020-09-25 Thread GitBox
leesf merged pull request #2113: URL: https://github.com/apache/hudi/pull/2113 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] bradleyhurley closed issue #2068: [SUPPORT]Deltastreamer Upsert Very Slow / Never Completes After Initial Data Load

2020-09-25 Thread GitBox
bradleyhurley closed issue #2068: URL: https://github.com/apache/hudi/issues/2068 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] leesf commented on pull request #2111: [HUDI-1234] Insert new records regardless of small file when using insert operation

2020-09-25 Thread GitBox
leesf commented on pull request #2111: URL: https://github.com/apache/hudi/pull/2111#issuecomment-698882595 @SteNicholas would you please also add some tests to the new changes? This is an automated message from the Apache

[GitHub] [hudi] yanghua commented on a change in pull request #2112: [HUDI-995] Migrate HoodieTestUtils APIs to HoodieTestTable

2020-09-25 Thread GitBox
yanghua commented on a change in pull request #2112: URL: https://github.com/apache/hudi/pull/2112#discussion_r494952442 ## File path: hudi-cli/src/test/java/org/apache/hudi/cli/commands/TestCommitsCommand.java ## @@ -168,10 +170,12 @@ public void testShowArchivedCommits()

[GitHub] [hudi] getniz commented on issue #2101: [SUPPORT]Unable to interpret Child JSON fields value as a separate columns rather it is loaded as one single field value. Any way to interpret that.

2020-09-25 Thread GitBox
getniz commented on issue #2101: URL: https://github.com/apache/hudi/issues/2101#issuecomment-698761326 @n3nash thanks for the response with details, 1 & 3 option I may not be able to consider as I need to build this layer as immediate target tables for further consumption in Reporting

[GitHub] [hudi] SteNicholas commented on a change in pull request #2111: [HUDI-1234] Insert new records regardless of small file when using insert operation

2020-09-25 Thread GitBox
SteNicholas commented on a change in pull request #2111: URL: https://github.com/apache/hudi/pull/2111#discussion_r494938396 ## File path: hudi-client/src/main/java/org/apache/hudi/table/WorkloadProfile.java ## @@ -54,13 +55,23 @@ */ private final WorkloadStat

[GitHub] [hudi] leesf commented on a change in pull request #2111: [HUDI-1234] Insert new records regardless of small file when using insert operation

2020-09-25 Thread GitBox
leesf commented on a change in pull request #2111: URL: https://github.com/apache/hudi/pull/2111#discussion_r494930532 ## File path: hudi-client/src/main/java/org/apache/hudi/table/WorkloadProfile.java ## @@ -54,13 +55,23 @@ */ private final WorkloadStat globalStat; +

[GitHub] [hudi] xushiyan commented on a change in pull request #2112: [HUDI-995] Migrate HoodieTestUtils APIs to HoodieTestTable

2020-09-25 Thread GitBox
xushiyan commented on a change in pull request #2112: URL: https://github.com/apache/hudi/pull/2112#discussion_r494635687 ## File path: hudi-cli/src/test/java/org/apache/hudi/cli/testutils/HoodieTestCommitMetadataGenerator.java ## @@ -62,67 +62,53 @@ /** * Create a

[GitHub] [hudi] Karl-WangSK removed a comment on pull request #2106: [HUDI-1284] preCombine all HoodieRecords and update all fields according to orderingVal

2020-09-25 Thread GitBox
Karl-WangSK removed a comment on pull request #2106: URL: https://github.com/apache/hudi/pull/2106#issuecomment-698291855 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] ashishmgofficial edited a comment on issue #2104: [SUPPORT] MOR Hive sync - _rt table read issue

2020-09-25 Thread GitBox
ashishmgofficial edited a comment on issue #2104: URL: https://github.com/apache/hudi/issues/2104#issuecomment-698567986 @n3nash : The following is the stacktrace I got when I queried on Hive CLI `2020-09-24 20:17:49,028 ERROR [39f399ee-de3f-4d33-a1cd-407d2e252f20 main]

[GitHub] [hudi] wangxianghu commented on a change in pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-25 Thread GitBox
wangxianghu commented on a change in pull request #1827: URL: https://github.com/apache/hudi/pull/1827#discussion_r494100797 ## File path: hudi-cli/pom.xml ## @@ -148,7 +148,14 @@ org.apache.hudi - hudi-client + hudi-client-common +

[GitHub] [hudi] bvaradar commented on pull request #1567: [HUDI-840]Clean blank file created by HoodieLogFormatWriter

2020-09-25 Thread GitBox
bvaradar commented on pull request #1567: URL: https://github.com/apache/hudi/pull/1567#issuecomment-698715649 @hddong : I went ahead and redid this change in the interest of time :) Instead of deleting on close, I have made changes to lazily create the log file when appending next

[GitHub] [hudi] Karl-WangSK commented on pull request #2106: [HUDI-1284] preCombine all HoodieRecords and update all fields according to orderingVal

2020-09-25 Thread GitBox
Karl-WangSK commented on pull request #2106: URL: https://github.com/apache/hudi/pull/2106#issuecomment-698291855 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] bvaradar merged pull request #2109: [MINOR] Adding scripts to checkout and push to PRs

2020-09-25 Thread GitBox
bvaradar merged pull request #2109: URL: https://github.com/apache/hudi/pull/2109 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] wangxianghu edited a comment on pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-25 Thread GitBox
wangxianghu edited a comment on pull request #1827: URL: https://github.com/apache/hudi/pull/1827#issuecomment-698161234 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] shenh062326 commented on a change in pull request #2085: [HUDI-1209] Properties File must be optional when running deltastreamer

2020-09-25 Thread GitBox
shenh062326 commented on a change in pull request #2085: URL: https://github.com/apache/hudi/pull/2085#discussion_r494256445 ## File path: hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java ## @@ -112,9 +112,14 @@ public

[GitHub] [hudi] satishkotha commented on a change in pull request #2048: [HUDI-1072][WIP] Introduce REPLACE top level action

2020-09-25 Thread GitBox
satishkotha commented on a change in pull request #2048: URL: https://github.com/apache/hudi/pull/2048#discussion_r494727484 ## File path: hudi-client/src/main/java/org/apache/hudi/client/HoodieWriteClient.java ## @@ -586,24 +602,39 @@ public String startCommit() { *

[GitHub] [hudi] vinothchandar commented on pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-25 Thread GitBox
vinothchandar commented on pull request #1827: URL: https://github.com/apache/hudi/pull/1827#issuecomment-698151969 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] SteNicholas commented on pull request #2111: [HUDI-1234] Insert new records regardless of small file when using insert operation

2020-09-25 Thread GitBox
SteNicholas commented on pull request #2111: URL: https://github.com/apache/hudi/pull/2111#issuecomment-698658606 @leesf @bvaradar Could you please review this pull request? This is an automated message from the Apache Git

[GitHub] [hudi] ashishmgofficial edited a comment on issue #2072: [SUPPORT] Hudi Pyspark Application Example

2020-09-25 Thread GitBox
ashishmgofficial edited a comment on issue #2072: URL: https://github.com/apache/hudi/issues/2072#issuecomment-698472403 @bvaradar @n3nash I can see that the rollback commands are looking for .rollback files in .hoodie folder but all i can see is .restore files The above scenario

[GitHub] [hudi] prashantwason commented on pull request #2064: WIP - [HUDI-842] Implementation of HUDI RFC-15.

2020-09-25 Thread GitBox
prashantwason commented on pull request #2064: URL: https://github.com/apache/hudi/pull/2064#issuecomment-698099305 @umehrot2 Directly using hudi datasource or delta streamer for testing should work too. I haven't testing this yet so please let me know if that doesn't work.

[GitHub] [hudi] wangxianghu edited a comment on pull request #2105: [MINOR] Fix ClassCastException when use QuickstartUtils generate data

2020-09-25 Thread GitBox
wangxianghu edited a comment on pull request #2105: URL: https://github.com/apache/hudi/pull/2105#issuecomment-698074779 @bhasudha This exception occurs because the methods to generate data in `QuickstartUtils` treat `ts` field as `long` type, while the schema provided by

[GitHub] [hudi] ShortFinger commented on issue #143: Tracking ticket for folks to be added to slack group

2020-09-25 Thread GitBox
ShortFinger commented on issue #143: URL: https://github.com/apache/hudi/issues/143#issuecomment-698296175 please add linfour@gmail.com Thanks! This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] wangxianghu commented on pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-25 Thread GitBox
wangxianghu commented on pull request #1827: URL: https://github.com/apache/hudi/pull/1827#issuecomment-698161234 > @wangxianghu Looks like we have much less class splitting now. I want to try and reduce this further if possible. > If its alright with you, I can take over from here,

[GitHub] [hudi] bvaradar commented on a change in pull request #2048: [HUDI-1072][WIP] Introduce REPLACE top level action

2020-09-25 Thread GitBox
bvaradar commented on a change in pull request #2048: URL: https://github.com/apache/hudi/pull/2048#discussion_r493957561 ## File path: hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java ## @@ -554,14 +608,16 @@ protected

[GitHub] [hudi] vinothchandar commented on a change in pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-25 Thread GitBox
vinothchandar commented on a change in pull request #1827: URL: https://github.com/apache/hudi/pull/1827#discussion_r494073875 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieAppendHandle.java ## @@ -134,7 +138,7 @@ private void

[GitHub] [hudi] ashishmgofficial commented on issue #2104: [SUPPORT] MOR Hive sync - _rt table read issue

2020-09-25 Thread GitBox
ashishmgofficial commented on issue #2104: URL: https://github.com/apache/hudi/issues/2104#issuecomment-698567986 @n3nash : The following is the stacktrace I got when I queried on Hive CLI `2020-09-24 20:17:49,028 ERROR [39f399ee-de3f-4d33-a1cd-407d2e252f20 main]

[GitHub] [hudi] ashishmgofficial commented on issue #2072: [SUPPORT] Hudi Pyspark Application Example

2020-09-25 Thread GitBox
ashishmgofficial commented on issue #2072: URL: https://github.com/apache/hudi/issues/2072#issuecomment-698449771 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] yanghua commented on a change in pull request #2112: [HUDI-995] Migrate HoodieTestUtils APIs to HoodieTestTable

2020-09-25 Thread GitBox
yanghua commented on a change in pull request #2112: URL: https://github.com/apache/hudi/pull/2112#discussion_r494952442 ## File path: hudi-cli/src/test/java/org/apache/hudi/cli/commands/TestCommitsCommand.java ## @@ -168,10 +170,12 @@ public void testShowArchivedCommits()

[GitHub] [hudi] dugenkui03 opened a new pull request #2113: [MINOR] fix typo

2020-09-25 Thread GitBox
dugenkui03 opened a new pull request #2113: URL: https://github.com/apache/hudi/pull/2113 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the

[GitHub] [hudi] SteNicholas commented on a change in pull request #2111: [HUDI-1234] Insert new records regardless of small file when using insert operation

2020-09-25 Thread GitBox
SteNicholas commented on a change in pull request #2111: URL: https://github.com/apache/hudi/pull/2111#discussion_r494938396 ## File path: hudi-client/src/main/java/org/apache/hudi/table/WorkloadProfile.java ## @@ -54,13 +55,23 @@ */ private final WorkloadStat

[GitHub] [hudi] leesf commented on pull request #2111: [HUDI-1234] Insert new records regardless of small file when using insert operation

2020-09-25 Thread GitBox
leesf commented on pull request #2111: URL: https://github.com/apache/hudi/pull/2111#issuecomment-698882595 @SteNicholas would you please also add some tests to the new changes? This is an automated message from the Apache

[GitHub] [hudi] leesf commented on a change in pull request #2111: [HUDI-1234] Insert new records regardless of small file when using insert operation

2020-09-25 Thread GitBox
leesf commented on a change in pull request #2111: URL: https://github.com/apache/hudi/pull/2111#discussion_r494930532 ## File path: hudi-client/src/main/java/org/apache/hudi/table/WorkloadProfile.java ## @@ -54,13 +55,23 @@ */ private final WorkloadStat globalStat; +

[jira] [Assigned] (HUDI-1161) Support update partial fields for MoR table

2020-09-25 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf reassigned HUDI-1161: --- Assignee: Nicholas Jiang (was: leesf) > Support update partial fields for MoR table >

[GitHub] [hudi] Karl-WangSK removed a comment on pull request #2106: [HUDI-1284] preCombine all HoodieRecords and update all fields according to orderingVal

2020-09-25 Thread GitBox
Karl-WangSK removed a comment on pull request #2106: URL: https://github.com/apache/hudi/pull/2106#issuecomment-698291855 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] Karl-WangSK commented on pull request #2106: [HUDI-1284] preCombine all HoodieRecords and update all fields according to orderingVal

2020-09-25 Thread GitBox
Karl-WangSK commented on pull request #2106: URL: https://github.com/apache/hudi/pull/2106#issuecomment-698824053 cc @yanghua @leesf This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hudi] getniz commented on issue #2101: [SUPPORT]Unable to interpret Child JSON fields value as a separate columns rather it is loaded as one single field value. Any way to interpret that.

2020-09-25 Thread GitBox
getniz commented on issue #2101: URL: https://github.com/apache/hudi/issues/2101#issuecomment-698761326 @n3nash thanks for the response with details, 1 & 3 option I may not be able to consider as I need to build this layer as immediate target tables for further consumption in Reporting