[GitHub] [hudi] wangxianghu opened a new pull request #2800: [MINOR] Remove unused imports and some other checkstyle issues

2021-04-09 Thread GitBox
wangxianghu opened a new pull request #2800: URL: https://github.com/apache/hudi/pull/2800 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the

[GitHub] [hudi] wangxianghu commented on pull request #2452: [HUDI-1531] Introduce HoodiePartitionCleaner to delete specific partition

2021-04-09 Thread GitBox
wangxianghu commented on pull request #2452: URL: https://github.com/apache/hudi/pull/2452#issuecomment-817089012 > can we just have a this implemented as a replace of the partition where all files are replaced by an empty list. cleaner would automatically clean the partition that way. Lov

[GitHub] [hudi] wangxianghu edited a comment on pull request #1946: [HUDI-1176]Upgrade tp log4j2

2021-04-09 Thread GitBox
wangxianghu edited a comment on pull request #1946: URL: https://github.com/apache/hudi/pull/1946#issuecomment-817085349 > @wangxianghu: had upgrade to `2.13.3` and fix the warning. Hi @hddong thanks for your patient. we still need `log4j2.properties` or `log4j2.xml` configs right?

[GitHub] [hudi] wangxianghu commented on pull request #1946: [HUDI-1176]Upgrade tp log4j2

2021-04-09 Thread GitBox
wangxianghu commented on pull request #1946: URL: https://github.com/apache/hudi/pull/1946#issuecomment-817085349 > @wangxianghu: had upgrade to `2.13.3` and fix the warning. Hi @hddong thanks for your patient. we still need `log4j2.properties` configs right? -- This is an autom

[jira] [Commented] (HUDI-1783) Support Huawei Cloud Object Storage

2021-04-09 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17318424#comment-17318424 ] vinoyang commented on HUDI-1783: [~xiaotaotao] I have given you Jira contributor permissio

[jira] [Assigned] (HUDI-1783) Support Huawei Cloud Object Storage

2021-04-09 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang reassigned HUDI-1783: -- Assignee: tao meng > Support Huawei Cloud Object Storage > > >

[jira] [Updated] (HUDI-1783) Support Huawei Cloud Object Storage

2021-04-09 Thread vinoyang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang updated HUDI-1783: --- Summary: Support Huawei Cloud Object Storage (was: support Huawei Cloud Object Storage) > Support Huawei

[hudi] branch master updated: [HUDI-1783] Support Huawei Cloud Object Storage (#2796)

2021-04-09 Thread vinoyang
This is an automated email from the ASF dual-hosted git repository. vinoyang pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/master by this push: new 8d4a7fe [HUDI-1783] Support Huawei Cloud Object

[GitHub] [hudi] yanghua merged pull request #2796: [HUDI-1783] Support Huawei Cloud Object Storage

2021-04-09 Thread GitBox
yanghua merged pull request #2796: URL: https://github.com/apache/hudi/pull/2796 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please co

[GitHub] [hudi] yanghua commented on pull request #2796: [HUDI-1783] Support Huawei Cloud Object Storage

2021-04-09 Thread GitBox
yanghua commented on pull request #2796: URL: https://github.com/apache/hudi/pull/2796#issuecomment-817078875 > already update the code。 @yanghua thanks for you review, sorry for that low level mistake。 It doesn't matter, just relax. -- This is an automated message from the Apache

[GitHub] [hudi] xiarixiaoyao commented on pull request #2722: [HUDI-1722]hive beeline/spark-sql query specified field on mor table occur NPE

2021-04-09 Thread GitBox
xiarixiaoyao commented on pull request #2722: URL: https://github.com/apache/hudi/pull/2722#issuecomment-817071591 @garyli1019 unit test has added, pls review again, thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hudi] xiarixiaoyao commented on pull request #2796: [HUDI-1783] Support Huawei Cloud Object Storage

2021-04-09 Thread GitBox
xiarixiaoyao commented on pull request #2796: URL: https://github.com/apache/hudi/pull/2796#issuecomment-817070783 already update the code。 @yanghua thanks for you review, sorry for that low level mistake。 -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [hudi] codecov-io edited a comment on pull request #2798: [HUDI-1785] Move OperationConverter to hudi-client-common for code reuse

2021-04-09 Thread GitBox
codecov-io edited a comment on pull request #2798: URL: https://github.com/apache/hudi/pull/2798#issuecomment-817064692 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2798?src=pr&el=h1) Report > Merging [#2798](https://codecov.io/gh/apache/hudi/pull/2798?src=pr&el=desc) (81fd3bb) in

[GitHub] [hudi] codecov-io commented on pull request #2798: [HUDI-1785] Move OperationConverter to hudi-client-common for code reuse

2021-04-09 Thread GitBox
codecov-io commented on pull request #2798: URL: https://github.com/apache/hudi/pull/2798#issuecomment-817064692 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2798?src=pr&el=h1) Report > Merging [#2798](https://codecov.io/gh/apache/hudi/pull/2798?src=pr&el=desc) (81fd3bb) into [ma

[GitHub] [hudi] codecov-io commented on pull request #2799: [HUDI-1784] Added print detailed stack log when hbase connection error

2021-04-09 Thread GitBox
codecov-io commented on pull request #2799: URL: https://github.com/apache/hudi/pull/2799#issuecomment-817063994 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2799?src=pr&el=h1) Report > Merging [#2799](https://codecov.io/gh/apache/hudi/pull/2799?src=pr&el=desc) (84cf6fa) into [ma

[GitHub] [hudi] hj2016 commented on issue #2623: org.apache.hudi.exception.HoodieDependentSystemUnavailableException:System HBASE unavailable.

2021-04-09 Thread GitBox
hj2016 commented on issue #2623: URL: https://github.com/apache/hudi/issues/2623#issuecomment-817062980 @n3nash @nsivabalan @root18039532923 I also have the same connection problem. The debug found that it was because of the jar conflict. I submitted a pr [https://github.com/apache/hudi/pu

[jira] [Updated] (HUDI-1784) Added print detailed stack log when hbase connection error

2021-04-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1784: - Labels: pull-request-available (was: ) > Added print detailed stack log when hbase connection err

[GitHub] [hudi] hj2016 opened a new pull request #2799: [HUDI-1784] Added print detailed stack log when hbase connection error

2021-04-09 Thread GitBox
hj2016 opened a new pull request #2799: URL: https://github.com/apache/hudi/pull/2799 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the pull

[jira] [Updated] (HUDI-1785) Move OperationConverter to hudi-client-common for code reuse

2021-04-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1785: - Labels: pull-request-available (was: ) > Move OperationConverter to hudi-client-common for code r

[GitHub] [hudi] wangxianghu opened a new pull request #2798: [HUDI-1785] Move OperationConverter to hudi-client-common for code reuse

2021-04-09 Thread GitBox
wangxianghu opened a new pull request #2798: URL: https://github.com/apache/hudi/pull/2798 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of the

[jira] [Created] (HUDI-1785) Move OperationConverter to hudi-client-common for code reuse

2021-04-09 Thread Xianghu Wang (Jira)
Xianghu Wang created HUDI-1785: -- Summary: Move OperationConverter to hudi-client-common for code reuse Key: HUDI-1785 URL: https://issues.apache.org/jira/browse/HUDI-1785 Project: Apache Hudi I

[jira] [Assigned] (HUDI-1785) Move OperationConverter to hudi-client-common for code reuse

2021-04-09 Thread Xianghu Wang (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianghu Wang reassigned HUDI-1785: -- Assignee: Xianghu Wang > Move OperationConverter to hudi-client-common for code reuse > ---

[jira] [Created] (HUDI-1784) Added print detailed stack log when hbase connection error

2021-04-09 Thread jing (Jira)
jing created HUDI-1784: -- Summary: Added print detailed stack log when hbase connection error Key: HUDI-1784 URL: https://issues.apache.org/jira/browse/HUDI-1784 Project: Apache Hudi Issue Type: Improvem

[GitHub] [hudi] jintaoguan commented on pull request #2773: [HUDI-1764] Add Hudi-CLI support for clustering

2021-04-09 Thread GitBox
jintaoguan commented on pull request #2773: URL: https://github.com/apache/hudi/pull/2773#issuecomment-817008126 Sure. Will do that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hudi] satishkotha edited a comment on pull request #2773: [HUDI-1764] Add Hudi-CLI support for clustering

2021-04-09 Thread GitBox
satishkotha edited a comment on pull request #2773: URL: https://github.com/apache/hudi/pull/2773#issuecomment-817002604 @jintaoguan LGTM. Can you raise PR to update documentation on [CLI page](https://hudi.apache.org/docs/deployment.html#cli) and add example command line screenshots? th

[GitHub] [hudi] satishkotha edited a comment on pull request #2773: [HUDI-1764] Add Hudi-CLI support for clustering

2021-04-09 Thread GitBox
satishkotha edited a comment on pull request #2773: URL: https://github.com/apache/hudi/pull/2773#issuecomment-817002604 @jintaoguan LGTM. Can you raise PR to update documentation on [CLI page ] (https://hudi.apache.org/docs/deployment.html#cli) and add example command line screenshots?

[GitHub] [hudi] satishkotha commented on pull request #2773: [HUDI-1764] Add Hudi-CLI support for clustering

2021-04-09 Thread GitBox
satishkotha commented on pull request #2773: URL: https://github.com/apache/hudi/pull/2773#issuecomment-817002604 @jintaoguan LGTM. Can you raise PR to update documentation on [CLI page ] (https://hudi.apache.org/docs/deployment.html#cli) and add example command line screenshots? the doc

[GitHub] [hudi] satishkotha commented on pull request #2388: [HUDI-1353] add incremental timeline support for pending clustering ops

2021-04-09 Thread GitBox
satishkotha commented on pull request #2388: URL: https://github.com/apache/hudi/pull/2388#issuecomment-816997366 @n3nash i dont have time in next 2-3 weeks to get this done. If you prefer, we can close this one. i can reopen (same PR or a different one) when i'm ready -- This is an auto

[GitHub] [hudi] n3nash commented on a change in pull request #2793: [HUDI-57] Support ORC Storage

2021-04-09 Thread GitBox
n3nash commented on a change in pull request #2793: URL: https://github.com/apache/hudi/pull/2793#discussion_r610913501 ## File path: hudi-common/src/test/java/org/apache/hudi/common/util/TestAvroOrcUtils.java ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] [hudi] n3nash commented on a change in pull request #2793: [HUDI-57] Support ORC Storage

2021-04-09 Thread GitBox
n3nash commented on a change in pull request #2793: URL: https://github.com/apache/hudi/pull/2793#discussion_r610913343 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/OrcReaderIterator.java ## @@ -0,0 +1,118 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [hudi] n3nash commented on a change in pull request #2793: [HUDI-57] Support ORC Storage

2021-04-09 Thread GitBox
n3nash commented on a change in pull request #2793: URL: https://github.com/apache/hudi/pull/2793#discussion_r610913269 ## File path: hudi-common/src/main/java/org/apache/hudi/common/util/OrcUtils.java ## @@ -0,0 +1,172 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] [hudi] n3nash commented on a change in pull request #2793: [HUDI-57] Support ORC Storage

2021-04-09 Thread GitBox
n3nash commented on a change in pull request #2793: URL: https://github.com/apache/hudi/pull/2793#discussion_r610913156 ## File path: hudi-common/src/main/java/org/apache/hudi/io/storage/HoodieOrcReader.java ## @@ -0,0 +1,88 @@ +/* + * Licensed to the Apache Software Foundatio

[GitHub] [hudi] n3nash commented on a change in pull request #2793: [HUDI-57] Support ORC Storage

2021-04-09 Thread GitBox
n3nash commented on a change in pull request #2793: URL: https://github.com/apache/hudi/pull/2793#discussion_r610911298 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/storage/HoodieOrcWriter.java ## @@ -0,0 +1,176 @@ +/* + * Licensed to the Apach

[GitHub] [hudi] n3nash commented on a change in pull request #2793: [HUDI-57] Support ORC Storage

2021-04-09 Thread GitBox
n3nash commented on a change in pull request #2793: URL: https://github.com/apache/hudi/pull/2793#discussion_r610909843 ## File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieStorageConfig.java ## @@ -39,10 +39,19 @@ public static final Stri

[GitHub] [hudi] vansimonsen commented on issue #2294: [SUPPORT] java.lang.IllegalArgumentException: Can not create a Path from an empty string on non partitioned COW table

2021-04-09 Thread GitBox
vansimonsen commented on issue #2294: URL: https://github.com/apache/hudi/issues/2294#issuecomment-816889220 > @vansimonsen : Can you open a new GH issue with the stack trace . @rubenssoto : I believe the PR landed before 0.7.0 was cut. @bvaradar https://github.com/apache/hudi/is

[GitHub] [hudi] vansimonsen opened a new issue #2797: [SUPPORT] Can not create a Path from an empty string on unpartitioned table

2021-04-09 Thread GitBox
vansimonsen opened a new issue #2797: URL: https://github.com/apache/hudi/issues/2797 **Describe the problem you faced** * Issue trying to create unpartitioned tables to hive metastore (in aws glue data catalog) using hudi (Tested on `0.6.0`, `0.7.0` and `0.8.0` ) * Using hudi on

[GitHub] [hudi] codecov-io edited a comment on pull request #2740: [HUDI-1055] Remove hardcoded parquet in tests

2021-04-09 Thread GitBox
codecov-io edited a comment on pull request #2740: URL: https://github.com/apache/hudi/pull/2740#issuecomment-809855336 # [Codecov](https://codecov.io/gh/apache/hudi/pull/2740?src=pr&el=h1) Report > Merging [#2740](https://codecov.io/gh/apache/hudi/pull/2740?src=pr&el=desc) (3e41f49) in

[GitHub] [hudi] TeRS-K commented on a change in pull request #2740: [HUDI-1055] Remove hardcoded parquet in tests

2021-04-09 Thread GitBox
TeRS-K commented on a change in pull request #2740: URL: https://github.com/apache/hudi/pull/2740#discussion_r610789102 ## File path: hudi-cli/src/main/scala/org/apache/hudi/cli/SparkHelpers.scala ## @@ -40,7 +40,7 @@ import scala.collection.mutable._ object SparkHelpers {

[GitHub] [hudi] rubenssoto edited a comment on issue #2509: [SUPPORT]Hudi saves TimestampType as bigInt

2021-04-09 Thread GitBox
rubenssoto edited a comment on issue #2509: URL: https://github.com/apache/hudi/issues/2509#issuecomment-816812025 Hello Guys, @satishkotha @nsivabalan Athena Behavior changes, https://user-images.githubusercontent.com/36298331/114213658-a841c400-9939-11eb-9fc9-a2e51761

[GitHub] [hudi] rubenssoto commented on issue #2509: [SUPPORT]Hudi saves TimestampType as bigInt

2021-04-09 Thread GitBox
rubenssoto commented on issue #2509: URL: https://github.com/apache/hudi/issues/2509#issuecomment-816812025 Hello Guys, Athena Behavior changes, https://user-images.githubusercontent.com/36298331/114213658-a841c400-9939-11eb-9fc9-a2e51761908e.png";> https://user-images.githu

[GitHub] [hudi] TeRS-K closed pull request #2793: [HUDI-57] Support ORC Storage

2021-04-09 Thread GitBox
TeRS-K closed pull request #2793: URL: https://github.com/apache/hudi/pull/2793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please con

[GitHub] [hudi] yanghua commented on a change in pull request #2796: [HUDI-1783]support Huawei Cloud Object Storage

2021-04-09 Thread GitBox
yanghua commented on a change in pull request #2796: URL: https://github.com/apache/hudi/pull/2796#discussion_r610693252 ## File path: hudi-common/src/main/java/org/apache/hudi/common/fs/StorageSchemes.java ## @@ -53,8 +53,9 @@ // Databricks file system DBFS("dbfs", fals

[GitHub] [hudi] kimberlyamandalu commented on issue #2620: [SUPPORT] Performance Tuning: Slow stages (Building Workload Profile & Getting Small files from partitions) during Hudi Writes

2021-04-09 Thread GitBox
kimberlyamandalu commented on issue #2620: URL: https://github.com/apache/hudi/issues/2620#issuecomment-816698689 > @kimberlyamandalu : do you have a support ticket for your question. lets not pollute this issue. we can create a new one for your use-case and can discuss over there h

[GitHub] [hudi] xiarixiaoyao commented on pull request #2796: [HUDI-1783]support Huawei Cloud Object Storage

2021-04-09 Thread GitBox
xiarixiaoyao commented on pull request #2796: URL: https://github.com/apache/hudi/pull/2796#issuecomment-816688293 @nsivabalan could you pls help me to review this pr? thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[jira] [Updated] (HUDI-1783) support Huawei Cloud Object Storage

2021-04-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-1783: - Labels: pull-request-available (was: ) > support Huawei Cloud Object Storage > -

[jira] [Updated] (HUDI-1752) Add HoodieFlinkClient InsertOverwrite

2021-04-09 Thread sivabalan narayanan (Jira)
[ https://issues.apache.org/jira/browse/HUDI-1752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sivabalan narayanan updated HUDI-1752: -- Fix Version/s: (was: 0.8.0) 0.9.0 > Add HoodieFlinkClient InsertO

[GitHub] [hudi] xiarixiaoyao opened a new pull request #2796: [HUDI-1783]support Huawei Cloud Object Storage

2021-04-09 Thread GitBox
xiarixiaoyao opened a new pull request #2796: URL: https://github.com/apache/hudi/pull/2796 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull request.* ## What is the purpose of th

[jira] [Created] (HUDI-1783) support Huawei Cloud Object Storage

2021-04-09 Thread tao meng (Jira)
tao meng created HUDI-1783: -- Summary: support Huawei Cloud Object Storage Key: HUDI-1783 URL: https://issues.apache.org/jira/browse/HUDI-1783 Project: Apache Hudi Issue Type: Bug Componen

[GitHub] [hudi] Magicbeanbuyer commented on issue #2498: [SUPPORT] Hudi MERGE_ON_READ load to dataframe fails for the versions [0.6.0],[0.7.0] and runs for [0.5.3]

2021-04-09 Thread GitBox
Magicbeanbuyer commented on issue #2498: URL: https://github.com/apache/hudi/issues/2498#issuecomment-816555746 Hey @nsivabalan, We have wrapped up our POC, therefore no longer have the setup anymore. Sorry couldn't contribute further to the issue. -- This is an automated messag

[GitHub] [hudi] xiarixiaoyao edited a comment on pull request #2720: [HUDI-1719]hive on spark/mr,Incremental query of the mor table, the partition field is incorrect

2021-04-09 Thread GitBox
xiarixiaoyao edited a comment on pull request #2720: URL: https://github.com/apache/hudi/pull/2720#issuecomment-816553709 @nsivabalan i find that the question: why test in TestHoodieCombineHiveInputFormat is Disabled ? @test @disabled public void testHoodieRealtimeCombineHoodieInp

[GitHub] [hudi] xiarixiaoyao commented on pull request #2720: [HUDI-1719]hive on spark/mr,Incremental query of the mor table, the partition field is incorrect

2021-04-09 Thread GitBox
xiarixiaoyao commented on pull request #2720: URL: https://github.com/apache/hudi/pull/2720#issuecomment-816553709 @nsivabalan i find that the question: why test in TestHoodieCombineHiveInputFormat is Disabled ? @test @disabled public void testHoodieRealtimeCombineHoodieInputForma

[hudi] branch asf-site updated: Travis CI build asf-site

2021-04-09 Thread vinoth
This is an automated email from the ASF dual-hosted git repository. vinoth pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new b353b0b Travis CI build asf-site b353b0b is d

[GitHub] [hudi] yanghua commented on pull request #2793: [HUDI-57] Support ORC Storage

2021-04-09 Thread GitBox
yanghua commented on pull request #2793: URL: https://github.com/apache/hudi/pull/2793#issuecomment-816495293 > How can I trigger a rebuild? option 1: close and reopen the PR; option 2: push an empty commit via git command -- This is an automated message from the Apache

[GitHub] [hudi] n3nash merged pull request #2794: [MINOR] Fix concurrency docs

2021-04-09 Thread GitBox
n3nash merged pull request #2794: URL: https://github.com/apache/hudi/pull/2794 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please con

[hudi] branch asf-site updated: [MINOR] Fix concurrency docs (#2794)

2021-04-09 Thread nagarwal
This is an automated email from the ASF dual-hosted git repository. nagarwal pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/hudi.git The following commit(s) were added to refs/heads/asf-site by this push: new a961350 [MINOR] Fix concurrency docs (#2794

[GitHub] [hudi] garyli1019 commented on pull request #2786: [HUDI-1782] Add more options for HUDI Flink

2021-04-09 Thread GitBox
garyli1019 commented on pull request #2786: URL: https://github.com/apache/hudi/pull/2786#issuecomment-816462388 > > Should we change the 0.8.0 doc as well? It will be merged soon. #2792 > > I think it is not necessary ? People would always see the master document. If the new o