[GitHub] [hudi] yanghua commented on pull request #2058: [HUDI-1259] Cache some framework binaries to speed up the progress of building docker image in local env

2020-09-12 Thread GitBox
yanghua commented on pull request #2058: URL: https://github.com/apache/hudi/pull/2058#issuecomment-691431626 > Would you mind helping document this if possible ? OK, my pleasure. This is an automated message

[GitHub] [hudi] sathyaprakashg commented on a change in pull request #2012: [HUDI-1129] Deltastreamer Add support for schema evolution

2020-09-12 Thread GitBox
sathyaprakashg commented on a change in pull request #2012: URL: https://github.com/apache/hudi/pull/2012#discussion_r487415553 ## File path: hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java ## @@ -127,12 +128,59 @@ public static GenericRecord

[jira] [Resolved] (HUDI-802) AWSDmsTransformer does not handle insert -> delete of a row in a single batch correctly

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-802. Resolution: Fixed > AWSDmsTransformer does not handle insert -> delete of a row in a single batch > correctly >

[jira] [Closed] (HUDI-802) AWSDmsTransformer does not handle insert -> delete of a row in a single batch correctly

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-802. -- > AWSDmsTransformer does not handle insert -> delete of a row in a single batch > correctly >

[jira] [Resolved] (HUDI-1181) Decimal type display issue for record key field

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-1181. - Fix Version/s: 0.6.1 Resolution: Fixed > Decimal type display issue for record key field >

[jira] [Closed] (HUDI-1181) Decimal type display issue for record key field

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-1181. --- > Decimal type display issue for record key field > --- > >

[jira] [Updated] (HUDI-1181) Decimal type display issue for record key field

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-1181: Status: Open (was: New) > Decimal type display issue for record key field >

[jira] [Updated] (HUDI-1130) Allow for schema evolution within DAG for hudi test suite

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-1130: Status: Open (was: New) > Allow for schema evolution within DAG for hudi test suite >

[jira] [Closed] (HUDI-1254) TypedProperties can not get values by initializing an existing properties

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-1254. --- > TypedProperties can not get values by initializing an existing properties >

[jira] [Resolved] (HUDI-1254) TypedProperties can not get values by initializing an existing properties

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-1254. - Resolution: Fixed > TypedProperties can not get values by initializing an existing properties >

[jira] [Resolved] (HUDI-1130) Allow for schema evolution within DAG for hudi test suite

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-1130. - Fix Version/s: 0.6.1 Resolution: Fixed > Allow for schema evolution within DAG for hudi test suite >

[jira] [Updated] (HUDI-1254) TypedProperties can not get values by initializing an existing properties

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-1254: Status: Open (was: New) > TypedProperties can not get values by initializing an existing properties >

[jira] [Closed] (HUDI-1130) Allow for schema evolution within DAG for hudi test suite

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-1130. --- > Allow for schema evolution within DAG for hudi test suite > - >

[jira] [Closed] (HUDI-1255) Combine and get updateValue in multiFields

2020-09-12 Thread leesf (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf closed HUDI-1255. --- > Combine and get updateValue in multiFields > -- > > Key:

[GitHub] [hudi] vinothchandar commented on pull request #1978: [HUDI-1184] Fix the support of hbase index partition path change

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1978: URL: https://github.com/apache/hudi/pull/1978#issuecomment-691505393 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] hj2016 commented on a change in pull request #2078: [MINOR]Add clinbrain to powered by page

2020-09-12 Thread GitBox
hj2016 commented on a change in pull request #2078: URL: https://github.com/apache/hudi/pull/2078#discussion_r486733426 ## File path: docs/_docs/1_4_powered_by.md ## @@ -28,6 +29,9 @@ offering real-time analysis on hudi dataset. Amazon Web Services is the World's leading

[GitHub] [hudi] leesf commented on a change in pull request #1929: [HUDI-1160] Support update partial fields for CoW table

2020-09-12 Thread GitBox
leesf commented on a change in pull request #1929: URL: https://github.com/apache/hudi/pull/1929#discussion_r487126922 ## File path: hudi-client/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -117,7 +118,17 @@ public boolean commitStats(String

[GitHub] [hudi] xushiyan commented on a change in pull request #2079: [HUDI-995] Use HoodieTestTable in more classes

2020-09-12 Thread GitBox
xushiyan commented on a change in pull request #2079: URL: https://github.com/apache/hudi/pull/2079#discussion_r487433796 ## File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestTable.java ## @@ -66,6 +73,30 @@ public static HoodieTestTable

[GitHub] [hudi] pratyakshsharma commented on pull request #2078: [MINOR]Add clinbrain to powered by page

2020-09-12 Thread GitBox
pratyakshsharma commented on pull request #2078: URL: https://github.com/apache/hudi/pull/2078#issuecomment-691050799 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] sathyaprakashg commented on a change in pull request #2012: [HUDI-1129] Deltastreamer Add support for schema evolution

2020-09-12 Thread GitBox
sathyaprakashg commented on a change in pull request #2012: URL: https://github.com/apache/hudi/pull/2012#discussion_r487415553 ## File path: hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java ## @@ -127,12 +128,59 @@ public static GenericRecord

[GitHub] [hudi] bvaradar commented on issue #2062: [SUPPORT] Duplicates in _ro and _rt table for MOR Table type

2020-09-12 Thread GitBox
bvaradar commented on issue #2062: URL: https://github.com/apache/hudi/issues/2062#issuecomment-691206241 @kpurella : I think you should either try patching the PR I had mentioned or use 0.6.0 This is an automated message

[GitHub] [hudi] vinothchandar commented on pull request #1978: [HUDI-1184] Fix the support of hbase index partition path change

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1978: URL: https://github.com/apache/hudi/pull/1978#issuecomment-691505393 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] leesf commented on pull request #1748: [WIP] [HUDI-1029] Use FastDateFormat for parsing and formating in Timestamp…

2020-09-12 Thread GitBox
leesf commented on pull request #1748: URL: https://github.com/apache/hudi/pull/1748#issuecomment-691043300 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] sathyaprakashg commented on a change in pull request #2012: [HUDI-1129] Deltastreamer Add support for schema evolution

2020-09-12 Thread GitBox
sathyaprakashg commented on a change in pull request #2012: URL: https://github.com/apache/hudi/pull/2012#discussion_r487415553 ## File path: hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java ## @@ -127,12 +128,59 @@ public static GenericRecord

[GitHub] [hudi] abhijeetkushe commented on issue #1737: [SUPPORT]spark streaming create small parquet files

2020-09-12 Thread GitBox
abhijeetkushe commented on issue #1737: URL: https://github.com/apache/hudi/issues/1737#issuecomment-691268965 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] nsivabalan commented on pull request #2084: [HUDI-802] AWSDmsTransformer does not handle insert and delete of a row in a single batch correctly

2020-09-12 Thread GitBox
nsivabalan commented on pull request #2084: URL: https://github.com/apache/hudi/pull/2084#issuecomment-691327993 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] pratyakshsharma commented on pull request #2085: [HUDI-1209] Properties File must be optional when running deltastreamer

2020-09-12 Thread GitBox
pratyakshsharma commented on pull request #2085: URL: https://github.com/apache/hudi/pull/2085#issuecomment-691021969 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] bvaradar closed issue #2062: [SUPPORT] Duplicates in _ro and _rt table for MOR Table type

2020-09-12 Thread GitBox
bvaradar closed issue #2062: URL: https://github.com/apache/hudi/issues/2062 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] vinothchandar commented on pull request #1704: [HUDI-115] Enhance OverwriteWithLatestAvroPayload to also respect ordering value of record in storage

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1704: URL: https://github.com/apache/hudi/pull/1704#issuecomment-691505318 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] tooptoop4 commented on pull request #2084: [HUDI-802] AWSDmsTransformer does not handle insert and delete of a row in a single batch correctly

2020-09-12 Thread GitBox
tooptoop4 commented on pull request #2084: URL: https://github.com/apache/hudi/pull/2084#issuecomment-690925786 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] vinothchandar commented on pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1827: URL: https://github.com/apache/hudi/pull/1827#issuecomment-691505048 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] leesf commented on a change in pull request #1929: [HUDI-1160] Support update partial fields for CoW table

2020-09-12 Thread GitBox
leesf commented on a change in pull request #1929: URL: https://github.com/apache/hudi/pull/1929#discussion_r487126922 ## File path: hudi-client/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -117,7 +118,17 @@ public boolean commitStats(String

[GitHub] [hudi] n3nash commented on pull request #2071: [HUDI-1143] Change timestamp field in HoodieTestDataGenerator from double to long

2020-09-12 Thread GitBox
n3nash commented on pull request #2071: URL: https://github.com/apache/hudi/pull/2071#issuecomment-691200964 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] abhijeetkushe commented on issue #1737: [SUPPORT]spark streaming create small parquet files

2020-09-12 Thread GitBox
abhijeetkushe commented on issue #1737: URL: https://github.com/apache/hudi/issues/1737#issuecomment-691268965 @bvaradar The hudi version we are using 0.5.2-incubating deployed on EMR. Good point on the terminology.I will rephrase my question COW with 'hoodie.cleaner.commits.retained':

[GitHub] [hudi] n3nash commented on pull request #1978: [HUDI-1184] Fix the support of hbase index partition path change

2020-09-12 Thread GitBox
n3nash commented on pull request #1978: URL: https://github.com/apache/hudi/pull/1978#issuecomment-691202637 @hj2016 Please rebase and fix the tests, we can land after that. This is an automated message from the Apache Git

[GitHub] [hudi] hj2016 commented on pull request #1978: [HUDI-1184] Fix the support of hbase index partition path change

2020-09-12 Thread GitBox
hj2016 commented on pull request #1978: URL: https://github.com/apache/hudi/pull/1978#issuecomment-691379071 @n3nash The problem nsivabalan said, I have fixed it a few days ago. I don’t know if you are talking about the same problem. If not, can I elaborate on it?

[GitHub] [hudi] xushiyan commented on a change in pull request #2079: [HUDI-995] Use HoodieTestTable in more classes

2020-09-12 Thread GitBox
xushiyan commented on a change in pull request #2079: URL: https://github.com/apache/hudi/pull/2079#discussion_r487433796 ## File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestTable.java ## @@ -66,6 +73,30 @@ public static HoodieTestTable

[GitHub] [hudi] yanghua closed pull request #2058: [HUDI-1259] Cache some framework binaries to speed up the progress of building docker image in local env

2020-09-12 Thread GitBox
yanghua closed pull request #2058: URL: https://github.com/apache/hudi/pull/2058 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] leesf closed pull request #1748: [WIP] [HUDI-1029] Use FastDateFormat for parsing and formating in Timestamp…

2020-09-12 Thread GitBox
leesf closed pull request #1748: URL: https://github.com/apache/hudi/pull/1748 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] pratyakshsharma merged pull request #2078: [MINOR]Add clinbrain to powered by page

2020-09-12 Thread GitBox
pratyakshsharma merged pull request #2078: URL: https://github.com/apache/hudi/pull/2078 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] pratyakshsharma commented on pull request #2078: [MINOR]Add clinbrain to powered by page

2020-09-12 Thread GitBox
pratyakshsharma commented on pull request #2078: URL: https://github.com/apache/hudi/pull/2078#issuecomment-691050799 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] abhijeetkushe edited a comment on issue #1737: [SUPPORT]spark streaming create small parquet files

2020-09-12 Thread GitBox
abhijeetkushe edited a comment on issue #1737: URL: https://github.com/apache/hudi/issues/1737#issuecomment-690685990 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] vinothchandar commented on a change in pull request #2048: [HUDI-1072][WIP] Introduce REPLACE top level action

2020-09-12 Thread GitBox
vinothchandar commented on a change in pull request #2048: URL: https://github.com/apache/hudi/pull/2048#discussion_r486369440 ## File path: hudi-client/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -87,44 +88,55 @@ protected

[GitHub] [hudi] shenh062326 commented on pull request #2071: [HUDI-1143] Change timestamp field in HoodieTestDataGenerator from double to long

2020-09-12 Thread GitBox
shenh062326 commented on pull request #2071: URL: https://github.com/apache/hudi/pull/2071#issuecomment-690928998 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] hj2016 commented on pull request #1978: [HUDI-1184] Fix the support of hbase index partition path change

2020-09-12 Thread GitBox
hj2016 commented on pull request #1978: URL: https://github.com/apache/hudi/pull/1978#issuecomment-691379071 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] bvaradar commented on pull request #2084: [HUDI-802] AWSDmsTransformer does not handle insert and delete of a row in a single batch correctly

2020-09-12 Thread GitBox
bvaradar commented on pull request #2084: URL: https://github.com/apache/hudi/pull/2084#issuecomment-690863065 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] pratyakshsharma merged pull request #2078: [MINOR]Add clinbrain to powered by page

2020-09-12 Thread GitBox
pratyakshsharma merged pull request #2078: URL: https://github.com/apache/hudi/pull/2078 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hudi] yanghua commented on pull request #2058: [HUDI-1259] Cache some framework binaries to speed up the progress of building docker image in local env

2020-09-12 Thread GitBox
yanghua commented on pull request #2058: URL: https://github.com/apache/hudi/pull/2058#issuecomment-691431626 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] n3nash commented on a change in pull request #2012: [HUDI-1129] Deltastreamer Add support for schema evolution

2020-09-12 Thread GitBox
n3nash commented on a change in pull request #2012: URL: https://github.com/apache/hudi/pull/2012#discussion_r487165310 ## File path: hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java ## @@ -127,12 +128,59 @@ public static GenericRecord bytesToAvro(byte[]

[GitHub] [hudi] bvaradar merged pull request #2084: [HUDI-802] AWSDmsTransformer does not handle insert and delete of a row in a single batch correctly

2020-09-12 Thread GitBox
bvaradar merged pull request #2084: URL: https://github.com/apache/hudi/pull/2084 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] vinothchandar commented on pull request #1242: [HUDI-544] Archived commits command code cleanup

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1242: URL: https://github.com/apache/hudi/pull/1242#issuecomment-691504599 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] shenh062326 commented on pull request #2071: [HUDI-1143] Change timestamp field in HoodieTestDataGenerator from double to long

2020-09-12 Thread GitBox
shenh062326 commented on pull request #2071: URL: https://github.com/apache/hudi/pull/2071#issuecomment-690928998 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Updated] (HUDI-1277) [DOC] Need documentation explaining how to write custom record payload class

2020-09-12 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1277: - Status: Open (was: New) > [DOC] Need documentation explaining how to write custom record

[jira] [Created] (HUDI-1277) [DOC] Need documentation explaining how to write custom record payload class

2020-09-12 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-1277: Summary: [DOC] Need documentation explaining how to write custom record payload class Key: HUDI-1277 URL: https://issues.apache.org/jira/browse/HUDI-1277

[jira] [Updated] (HUDI-1278) Need a generic payload class which can skip late arriving data based on specific fields

2020-09-12 Thread Balaji Varadarajan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balaji Varadarajan updated HUDI-1278: - Status: Open (was: New) > Need a generic payload class which can skip late arriving data

[jira] [Created] (HUDI-1278) Need a generic payload class which can skip late arriving data based on specific fields

2020-09-12 Thread Balaji Varadarajan (Jira)
Balaji Varadarajan created HUDI-1278: Summary: Need a generic payload class which can skip late arriving data based on specific fields Key: HUDI-1278 URL: https://issues.apache.org/jira/browse/HUDI-1278

[GitHub] [hudi] bvaradar commented on issue #2062: [SUPPORT] Duplicates in _ro and _rt table for MOR Table type

2020-09-12 Thread GitBox
bvaradar commented on issue #2062: URL: https://github.com/apache/hudi/issues/2062#issuecomment-691206241 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] n3nash commented on pull request #1242: [HUDI-544] Archived commits command code cleanup

2020-09-12 Thread GitBox
n3nash commented on pull request #1242: URL: https://github.com/apache/hudi/pull/1242#issuecomment-691203252 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] bvaradar commented on pull request #2058: [HUDI-1259] Cache some framework binaries to speed up the progress of building docker image in local env

2020-09-12 Thread GitBox
bvaradar commented on pull request #2058: URL: https://github.com/apache/hudi/pull/2058#issuecomment-691209314 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] yanghua commented on pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-12 Thread GitBox
yanghua commented on pull request #1827: URL: https://github.com/apache/hudi/pull/1827#issuecomment-690922454 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] rajgowtham24 commented on issue #2075: [SUPPORT] hoodie.datasource.write.precombine.field not working as expected

2020-09-12 Thread GitBox
rajgowtham24 commented on issue #2075: URL: https://github.com/apache/hudi/issues/2075#issuecomment-691079649 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] bvaradar commented on issue #1737: [SUPPORT]spark streaming create small parquet files

2020-09-12 Thread GitBox
bvaradar commented on issue #1737: URL: https://github.com/apache/hudi/issues/1737#issuecomment-691198064 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] abhijeetkushe edited a comment on issue #1737: [SUPPORT]spark streaming create small parquet files

2020-09-12 Thread GitBox
abhijeetkushe edited a comment on issue #1737: URL: https://github.com/apache/hudi/issues/1737#issuecomment-690685990 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] pratyakshsharma commented on pull request #1748: [WIP] [HUDI-1029] Use FastDateFormat for parsing and formating in Timestamp…

2020-09-12 Thread GitBox
pratyakshsharma commented on pull request #1748: URL: https://github.com/apache/hudi/pull/1748#issuecomment-690966794 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] pratyakshsharma commented on pull request #1748: [WIP] [HUDI-1029] Use FastDateFormat for parsing and formating in Timestamp…

2020-09-12 Thread GitBox
pratyakshsharma commented on pull request #1748: URL: https://github.com/apache/hudi/pull/1748#issuecomment-690966794 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [hudi] n3nash commented on pull request #1978: [HUDI-1184] Fix the support of hbase index partition path change

2020-09-12 Thread GitBox
n3nash commented on pull request #1978: URL: https://github.com/apache/hudi/pull/1978#issuecomment-691202637 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] yanghua closed pull request #2058: [HUDI-1259] Cache some framework binaries to speed up the progress of building docker image in local env

2020-09-12 Thread GitBox
yanghua closed pull request #2058: URL: https://github.com/apache/hudi/pull/2058 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [hudi] leesf closed pull request #1748: [WIP] [HUDI-1029] Use FastDateFormat for parsing and formating in Timestamp…

2020-09-12 Thread GitBox
leesf closed pull request #1748: URL: https://github.com/apache/hudi/pull/1748 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] vinothchandar commented on pull request #1990: [HUDI-1199]: relocated jetty in hudi-utilities-bundle pom

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1990: URL: https://github.com/apache/hudi/pull/1990#issuecomment-691505608 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] vinothchandar commented on a change in pull request #2048: [HUDI-1072][WIP] Introduce REPLACE top level action

2020-09-12 Thread GitBox
vinothchandar commented on a change in pull request #2048: URL: https://github.com/apache/hudi/pull/2048#discussion_r486369440 ## File path: hudi-client/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -87,44 +88,55 @@ protected

[GitHub] [hudi] leesf commented on pull request #1748: [WIP] [HUDI-1029] Use FastDateFormat for parsing and formating in Timestamp…

2020-09-12 Thread GitBox
leesf commented on pull request #1748: URL: https://github.com/apache/hudi/pull/1748#issuecomment-691043300 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] sathyaprakashg commented on a change in pull request #2012: [HUDI-1129] Deltastreamer Add support for schema evolution

2020-09-12 Thread GitBox
sathyaprakashg commented on a change in pull request #2012: URL: https://github.com/apache/hudi/pull/2012#discussion_r487415553 ## File path: hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java ## @@ -127,12 +128,59 @@ public static GenericRecord

[GitHub] [hudi] nsivabalan commented on pull request #2084: [HUDI-802] AWSDmsTransformer does not handle insert and delete of a row in a single batch correctly

2020-09-12 Thread GitBox
nsivabalan commented on pull request #2084: URL: https://github.com/apache/hudi/pull/2084#issuecomment-691327993 yeah, https://github.com/apache/hudi/pull/1792 fixed the issue for OverwriteWithLatestAvroPayload and not for AWSDmsAvroPayload. thanks @bvaradar.

[GitHub] [hudi] yanghua commented on pull request #2058: [HUDI-1259] Cache some framework binaries to speed up the progress of building docker image in local env

2020-09-12 Thread GitBox
yanghua commented on pull request #2058: URL: https://github.com/apache/hudi/pull/2058#issuecomment-690838645 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] vinothchandar commented on pull request #1242: [HUDI-544] Archived commits command code cleanup

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1242: URL: https://github.com/apache/hudi/pull/1242#issuecomment-691504599 @n3nash if you wish you can also rebase this yourself and push. See https://cwiki.apache.org/confluence/display/HUDI/Resources#Resources-PushingChangesToPRs for a how-to

[GitHub] [hudi] leesf commented on a change in pull request #1929: [HUDI-1160] Support update partial fields for CoW table

2020-09-12 Thread GitBox
leesf commented on a change in pull request #1929: URL: https://github.com/apache/hudi/pull/1929#discussion_r487126922 ## File path: hudi-client/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -117,7 +118,17 @@ public boolean commitStats(String

[GitHub] [hudi] vinothchandar commented on pull request #1990: [HUDI-1199]: relocated jetty in hudi-utilities-bundle pom

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1990: URL: https://github.com/apache/hudi/pull/1990#issuecomment-691505608 @pratyakshsharma please always wait for an explicit LGTM from another committer before merging :) I will take another pass

[GitHub] [hudi] vinothchandar commented on a change in pull request #2048: [HUDI-1072][WIP] Introduce REPLACE top level action

2020-09-12 Thread GitBox
vinothchandar commented on a change in pull request #2048: URL: https://github.com/apache/hudi/pull/2048#discussion_r486369440 ## File path: hudi-client/src/main/java/org/apache/hudi/client/AbstractHoodieWriteClient.java ## @@ -87,44 +88,55 @@ protected

[GitHub] [hudi] abhijeetkushe commented on issue #1737: [SUPPORT]spark streaming create small parquet files

2020-09-12 Thread GitBox
abhijeetkushe commented on issue #1737: URL: https://github.com/apache/hudi/issues/1737#issuecomment-691268965 @bvaradar The hudi version we are using 0.5.2-incubating deployed on EMR. Good point on the terminology.I will rephrase my question COW with 'hoodie.cleaner.commits.retained':

[GitHub] [hudi] abhijeetkushe edited a comment on issue #1737: [SUPPORT]spark streaming create small parquet files

2020-09-12 Thread GitBox
abhijeetkushe edited a comment on issue #1737: URL: https://github.com/apache/hudi/issues/1737#issuecomment-690685990 @vinothchandar I am facing a similar problem.I am doing a POC for Hudi and am using with the same data for both COW and MOR.I see the compaction happening for both

[GitHub] [hudi] bvaradar commented on pull request #2084: [HUDI-802] AWSDmsTransformer does not handle insert and delete of a row in a single batch correctly

2020-09-12 Thread GitBox
bvaradar commented on pull request #2084: URL: https://github.com/apache/hudi/pull/2084#issuecomment-690863065 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] vinothchandar commented on pull request #1242: [HUDI-544] Archived commits command code cleanup

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1242: URL: https://github.com/apache/hudi/pull/1242#issuecomment-691504599 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] shenh062326 commented on pull request #2071: [HUDI-1143] Change timestamp field in HoodieTestDataGenerator from double to long

2020-09-12 Thread GitBox
shenh062326 commented on pull request #2071: URL: https://github.com/apache/hudi/pull/2071#issuecomment-690928998 @n3nash can you take a look at this MR. This is an automated message from the Apache Git Service. To respond

[GitHub] [hudi] bvaradar commented on issue #2062: [SUPPORT] Duplicates in _ro and _rt table for MOR Table type

2020-09-12 Thread GitBox
bvaradar commented on issue #2062: URL: https://github.com/apache/hudi/issues/2062#issuecomment-691206241 @kpurella : I think you should either try patching the PR I had mentioned or use 0.6.0 This is an automated message

[GitHub] [hudi] bvaradar closed issue #2062: [SUPPORT] Duplicates in _ro and _rt table for MOR Table type

2020-09-12 Thread GitBox
bvaradar closed issue #2062: URL: https://github.com/apache/hudi/issues/2062 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hudi] n3nash commented on a change in pull request #2012: [HUDI-1129] Deltastreamer Add support for schema evolution

2020-09-12 Thread GitBox
n3nash commented on a change in pull request #2012: URL: https://github.com/apache/hudi/pull/2012#discussion_r487165310 ## File path: hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java ## @@ -127,12 +128,59 @@ public static GenericRecord bytesToAvro(byte[]

[GitHub] [hudi] bvaradar commented on issue #1737: [SUPPORT]spark streaming create small parquet files

2020-09-12 Thread GitBox
bvaradar commented on issue #1737: URL: https://github.com/apache/hudi/issues/1737#issuecomment-691198064 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] vinothchandar commented on pull request #1704: [HUDI-115] Enhance OverwriteWithLatestAvroPayload to also respect ordering value of record in storage

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1704: URL: https://github.com/apache/hudi/pull/1704#issuecomment-691505318 Looks like this is almost ready . I will rebase, review and try to land this first. thanks for flagging @shenh062326 !

[GitHub] [hudi] vinothchandar commented on pull request #1978: [HUDI-1184] Fix the support of hbase index partition path change

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1978: URL: https://github.com/apache/hudi/pull/1978#issuecomment-691505393 @nsivabalan are you good with the test changes like @hj2016 is asking? please clarify This is an automated

[GitHub] [hudi] pratyakshsharma commented on pull request #2085: [HUDI-1209] Properties File must be optional when running deltastreamer

2020-09-12 Thread GitBox
pratyakshsharma commented on pull request #2085: URL: https://github.com/apache/hudi/pull/2085#issuecomment-691021969 Changes look good to me. Can you see why the build is failing? @shenh062326 This is an automated message

[GitHub] [hudi] vinothchandar commented on pull request #1704: [HUDI-115] Enhance OverwriteWithLatestAvroPayload to also respect ordering value of record in storage

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1704: URL: https://github.com/apache/hudi/pull/1704#issuecomment-691505318 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] xushiyan commented on a change in pull request #2079: [HUDI-995] Use HoodieTestTable in more classes

2020-09-12 Thread GitBox
xushiyan commented on a change in pull request #2079: URL: https://github.com/apache/hudi/pull/2079#discussion_r487433796 ## File path: hudi-common/src/test/java/org/apache/hudi/common/testutils/HoodieTestTable.java ## @@ -66,6 +73,30 @@ public static HoodieTestTable

[GitHub] [hudi] bvaradar commented on pull request #2058: [HUDI-1259] Cache some framework binaries to speed up the progress of building docker image in local env

2020-09-12 Thread GitBox
bvaradar commented on pull request #2058: URL: https://github.com/apache/hudi/pull/2058#issuecomment-691209314 Yes @yanghua : The docker containers are for mainly for internal (testing and demo) consumption but I agree we can document it for engineers to know. Would you mind helping

[GitHub] [hudi] n3nash commented on pull request #1242: [HUDI-544] Archived commits command code cleanup

2020-09-12 Thread GitBox
n3nash commented on pull request #1242: URL: https://github.com/apache/hudi/pull/1242#issuecomment-691203252 @hddong Extremely sorry, this fell through the crack, please rebase and I will merge this right after. This is an

[GitHub] [hudi] yanghua commented on pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-12 Thread GitBox
yanghua commented on pull request #1827: URL: https://github.com/apache/hudi/pull/1827#issuecomment-690922454 @leesf Thanks for your awesome work. Can you squash these commits for the subsequent review? This is an automated

[GitHub] [hudi] n3nash commented on pull request #2071: [HUDI-1143] Change timestamp field in HoodieTestDataGenerator from double to long

2020-09-12 Thread GitBox
n3nash commented on pull request #2071: URL: https://github.com/apache/hudi/pull/2071#issuecomment-691200964 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [hudi] vinothchandar commented on pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1827: URL: https://github.com/apache/hudi/pull/1827#issuecomment-691505048 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [hudi] vinothchandar commented on pull request #1827: [HUDI-1089] Refactor hudi-client to support multi-engine

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1827: URL: https://github.com/apache/hudi/pull/1827#issuecomment-691505048 @wangxianghu That's awesome. Will resume the Review again this weekend This is an automated message from the

[GitHub] [hudi] leesf commented on pull request #1748: [WIP] [HUDI-1029] Use FastDateFormat for parsing and formating in Timestamp…

2020-09-12 Thread GitBox
leesf commented on pull request #1748: URL: https://github.com/apache/hudi/pull/1748#issuecomment-691043300 > @leesf we no longer use SimpleDateFormat in TimestampBasedKeyGenerator. So the issues linked with usage of SimpleDateFormat are also no longer there. Guess we can close this?

[GitHub] [hudi] vinothchandar commented on pull request #1990: [HUDI-1199]: relocated jetty in hudi-utilities-bundle pom

2020-09-12 Thread GitBox
vinothchandar commented on pull request #1990: URL: https://github.com/apache/hudi/pull/1990#issuecomment-691505608 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

  1   2   >