[GitHub] [incubator-hudi] lamber-ken commented on issue #143: Tracking ticket for folks to be added to slack group

lamber-ken commented on issue #143: Tracking ticket for folks to be added to slack group URL: https://github.com/apache/incubator-hudi/issues/143#issuecomment-615390839 @c-f-cooper @superguhua @jenu9417 @dahirainbow welcome and done

[jira] [Created] (HUDI-801) Add a way to postprocess schema after it is loaded from the schema provider

Alexander Filipchik created HUDI-801: Summary: Add a way to postprocess schema after it is loaded from the schema provider Key: HUDI-801 URL: https://issues.apache.org/jira/browse/HUDI-801

[GitHub] [incubator-hudi] vinothchandar commented on issue #1513: [HUDI-793] Adding proper default to hudi metadata fields and proper handling to rewrite routine

vinothchandar commented on issue #1513: [HUDI-793] Adding proper default to hudi metadata fields and proper handling to rewrite routine URL: https://github.com/apache/incubator-hudi/pull/1513#issuecomment-615383799 @pratyakshsharma can you please open a PR with these additional test cases

[jira] [Commented] (HUDI-791) Replace null by Option in Delta Streamer

[ https://issues.apache.org/jira/browse/HUDI-791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085942#comment-17085942 ] Yanjia Gary Li commented on HUDI-791: - [~tison] Thanks for looking into this ticket! The initiative

[jira] [Commented] (HUDI-716) Exception: Not an Avro data file when running HoodieCleanClient.runClean

[ https://issues.apache.org/jira/browse/HUDI-716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085961#comment-17085961 ] Vinoth Chandar commented on HUDI-716: - [~afilipchik] if you are still facing this issue. please share a

[jira] [Created] (HUDI-802) AWSDmsTransformer does not handle insert -> delete of a row in a single batch correctly

Christopher Weaver created HUDI-802: --- Summary: AWSDmsTransformer does not handle insert -> delete of a row in a single batch correctly Key: HUDI-802 URL: https://issues.apache.org/jira/browse/HUDI-802

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1524: [HUDI-801] Adding a way to post process schema after it is fetched

pratyakshsharma commented on a change in pull request #1524: [HUDI-801] Adding a way to post process schema after it is fetched URL: https://github.com/apache/incubator-hudi/pull/1524#discussion_r410440366 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1524: [HUDI-801] Adding a way to post process schema after it is fetched

pratyakshsharma commented on a change in pull request #1524: [HUDI-801] Adding a way to post process schema after it is fetched URL: https://github.com/apache/incubator-hudi/pull/1524#discussion_r410440760 ## File path:

[jira] [Commented] (HUDI-773) Hudi On Azure Data Lake Storage V2

[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086087#comment-17086087 ] Yanjia Gary Li commented on HUDI-773: - Hello [~sasikumar.venkat], I am very new to Azure. How is your

[GitHub] [incubator-hudi] lamber-ken edited a comment on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records

lamber-ken edited a comment on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records URL: https://github.com/apache/incubator-hudi/issues/1491#issuecomment-615473505 Hi @tverdokhlebd, I'm glad to hear that your problem has been solved. > Upsert was hanging without this

[GitHub] [incubator-hudi] lamber-ken edited a comment on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records

lamber-ken edited a comment on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records URL: https://github.com/apache/incubator-hudi/issues/1491#issuecomment-615473505 Hi @tverdokhlebd, glad to hear that your problem has been solved. > Upsert was hanging without this

[GitHub] [incubator-hudi] lamber-ken commented on a change in pull request #1526: [HUDI-1526] Add pyspark example in quickstart

lamber-ken commented on a change in pull request #1526: [HUDI-1526] Add pyspark example in quickstart URL: https://github.com/apache/incubator-hudi/pull/1526#discussion_r410501516 ## File path: docs/_docs/1_1_quick_start_guide.md ## @@ -148,6 +203,31 @@

[jira] [Created] (HUDI-805) Verify which types of Azure storage support Hudi

Yanjia Gary Li created HUDI-805: --- Summary: Verify which types of Azure storage support Hudi Key: HUDI-805 URL: https://issues.apache.org/jira/browse/HUDI-805 Project: Apache Hudi (incubating)

[GitHub] [incubator-hudi] lamber-ken commented on a change in pull request #1526: [HUDI-1526] Add pyspark example in quickstart

lamber-ken commented on a change in pull request #1526: [HUDI-1526] Add pyspark example in quickstart URL: https://github.com/apache/incubator-hudi/pull/1526#discussion_r410501360 ## File path: docs/_docs/1_1_quick_start_guide.md ## @@ -68,6 +81,27 @@

[GitHub] [incubator-hudi] lamber-ken commented on issue #1526: [HUDI-1526] Add pyspark example in quickstart

lamber-ken commented on issue #1526: [HUDI-1526] Add pyspark example in quickstart URL: https://github.com/apache/incubator-hudi/pull/1526#issuecomment-615494434 Thanks for your contribution, left minor comments. Visit https://lamber-ken.github.io/docs/quick-start-guide.html

[GitHub] [incubator-hudi] lamber-ken commented on a change in pull request #1527: [MINOR] fix some places are not elegant, as a newcomer

lamber-ken commented on a change in pull request #1527: [MINOR] fix some places are not elegant, as a newcomer URL: https://github.com/apache/incubator-hudi/pull/1527#discussion_r410505797 ## File path:

[jira] [Updated] (HUDI-803) Improve Unit test coverage of HoodieAvroUtils around default values

[ https://issues.apache.org/jira/browse/HUDI-803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratyaksh Sharma updated HUDI-803: -- Description: Recently there has been lot of work and improvements around schema evolution and

[jira] [Updated] (HUDI-803) Improve Unit test coverage of HoodieAvroUtils around default values

[ https://issues.apache.org/jira/browse/HUDI-803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pratyaksh Sharma updated HUDI-803: -- Status: Open (was: New) > Improve Unit test coverage of HoodieAvroUtils around default values >

[jira] [Created] (HUDI-803) Improve Unit test coverage of HoodieAvroUtils around default values

Pratyaksh Sharma created HUDI-803: - Summary: Improve Unit test coverage of HoodieAvroUtils around default values Key: HUDI-803 URL: https://issues.apache.org/jira/browse/HUDI-803 Project: Apache Hudi

[GitHub] [incubator-hudi] pratyakshsharma commented on issue #1513: [HUDI-793] Adding proper default to hudi metadata fields and proper handling to rewrite routine

pratyakshsharma commented on issue #1513: [HUDI-793] Adding proper default to hudi metadata fields and proper handling to rewrite routine URL: https://github.com/apache/incubator-hudi/pull/1513#issuecomment-615434579 > @pratyakshsharma can you please open a PR with these additional test

[GitHub] [incubator-hudi] lamber-ken edited a comment on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records

lamber-ken edited a comment on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records URL: https://github.com/apache/incubator-hudi/issues/1491#issuecomment-615473505 Hi @tverdokhlebd, glad to hear that your problem has been solved. > Upsert was hanging without this

[jira] [Created] (HUDI-804) Add Azure Support to Hudi Doc

Yanjia Gary Li created HUDI-804: --- Summary: Add Azure Support to Hudi Doc Key: HUDI-804 URL: https://issues.apache.org/jira/browse/HUDI-804 Project: Apache Hudi (incubating) Issue Type:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1524: [HUDI-801] Adding a way to post process schema after it is fetched

pratyakshsharma commented on a change in pull request #1524: [HUDI-801] Adding a way to post process schema after it is fetched URL: https://github.com/apache/incubator-hudi/pull/1524#discussion_r410439724 ## File path:

[GitHub] [incubator-hudi] lamber-ken commented on issue #1526: [HUDI-1526] Add pyspark example in quickstart

lamber-ken commented on issue #1526: [HUDI-1526] Add pyspark example in quickstart URL: https://github.com/apache/incubator-hudi/pull/1526#issuecomment-615497198 hi @bhasudha, do you have any suggestions? This is an

[GitHub] [incubator-hudi] lamber-ken edited a comment on issue #1526: [HUDI-1526] Add pyspark example in quickstart

lamber-ken edited a comment on issue #1526: [HUDI-1526] Add pyspark example in quickstart URL: https://github.com/apache/incubator-hudi/pull/1526#issuecomment-615497198 hi @bhasudha, do you have any suggestions? you can visit https://lamber-ken.github.io/docs/quick-start-guide.html

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #251

See Changes: -- [...truncated 2.32 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml

[incubator-hudi] branch master updated: [MINOR] use Option and fix description in toString method (#1527)

This is an automated email from the ASF dual-hosted git repository. lamberken pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git The following commit(s) were added to refs/heads/master by this push: new 7552365 [MINOR] use Option and fix

[GitHub] [incubator-hudi] lamber-ken merged pull request #1527: [MINOR] use Option and fix description in toString method

lamber-ken merged pull request #1527: [MINOR] use Option and fix description in toString method URL: https://github.com/apache/incubator-hudi/pull/1527 This is an automated message from the Apache Git Service. To respond to

[GitHub] [incubator-hudi] lamber-ken commented on issue #1527: [MINOR] fix some places are not elegant, as a newcomer

lamber-ken commented on issue #1527: [MINOR] fix some places are not elegant, as a newcomer URL: https://github.com/apache/incubator-hudi/pull/1527#issuecomment-615559297 > Please merge @lamber-ken, and how about changing the title to `[MINOR] use Option and fix description in toString

[jira] [Created] (HUDI-807) Support for incremental queries for bootstrapped tables

Udit Mehrotra created HUDI-807: -- Summary: Support for incremental queries for bootstrapped tables Key: HUDI-807 URL: https://issues.apache.org/jira/browse/HUDI-807 Project: Apache Hudi (incubating)

[jira] [Created] (HUDI-806) Implement support for bootstrapping via Spark datasource API

Udit Mehrotra created HUDI-806: -- Summary: Implement support for bootstrapping via Spark datasource API Key: HUDI-806 URL: https://issues.apache.org/jira/browse/HUDI-806 Project: Apache Hudi (incubating)

[GitHub] [incubator-hudi] lamber-ken commented on issue #1527: [MINOR] fix some places are not elegant, as a newcomer

lamber-ken commented on issue #1527: [MINOR] fix some places are not elegant, as a newcomer URL: https://github.com/apache/incubator-hudi/pull/1527#issuecomment-615501557 LGTM, thanks @baobaoyeye This is an automated

[GitHub] [incubator-hudi] leesf commented on issue #1527: [MINOR] fix some places are not elegant, as a newcomer

leesf commented on issue #1527: [MINOR] fix some places are not elegant, as a newcomer URL: https://github.com/apache/incubator-hudi/pull/1527#issuecomment-615522191 Please merge @lamber-ken, and how about changing the title to `[MINOR] use Option and fix description in toString method `?

[jira] [Created] (HUDI-808) Support for cleaning source data

Udit Mehrotra created HUDI-808: -- Summary: Support for cleaning source data Key: HUDI-808 URL: https://issues.apache.org/jira/browse/HUDI-808 Project: Apache Hudi (incubating) Issue Type:

[GitHub] [incubator-hudi] hddong commented on issue #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter

hddong commented on issue #1511: [HUDI-789]Adjust logic of upsert in HDFSParquetImporter URL: https://github.com/apache/incubator-hudi/pull/1511#issuecomment-615161533 @yanghua Thanks for you review, had address them. This

[GitHub] [incubator-hudi] tverdokhlebd commented on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records

tverdokhlebd commented on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records URL: https://github.com/apache/incubator-hudi/issues/1491#issuecomment-615141491 Hi @lamber-ken Yes, I have resolved this problem. Firstly, the problem was in SBT memory that

[jira] [Resolved] (HUDI-777) Update Deltastreamer param description for --target-table

[ https://issues.apache.org/jira/browse/HUDI-777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iftach Schonbaum resolved HUDI-777. --- Resolution: Fixed > Update Deltastreamer param description for --target-table >

[GitHub] [incubator-hudi] jenu9417 commented on issue #143: Tracking ticket for folks to be added to slack group

jenu9417 commented on issue #143: Tracking ticket for folks to be added to slack group URL: https://github.com/apache/incubator-hudi/issues/143#issuecomment-615077522 please add me. jenu9...@gmail.com Thanks. This is an

[jira] [Commented] (HUDI-773) Hudi On Azure Data Lake Storage V2

[ https://issues.apache.org/jira/browse/HUDI-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17085499#comment-17085499 ] Sasikumar Venkatesh commented on HUDI-773: -- Thank you for the prompt reply [~garyli1019]. Let me

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1524: Adding a way to post process schema after it is fetched

pratyakshsharma commented on a change in pull request #1524: Adding a way to post process schema after it is fetched URL: https://github.com/apache/incubator-hudi/pull/1524#discussion_r410180045 ## File path:

[jira] [Updated] (HUDI-800) Metrics getReporter().close() throws NPE when MetricsReporter is InMemoryMetricsReporter

[ https://issues.apache.org/jira/browse/HUDI-800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-800: --- Priority: Minor (was: Major) > Metrics getReporter().close() throws NPE when MetricsReporter is >

[jira] [Created] (HUDI-800) Metrics getReporter().close() throws NPE when MetricsReporter is InMemoryMetricsReporter

leesf created HUDI-800: -- Summary: Metrics getReporter().close() throws NPE when MetricsReporter is InMemoryMetricsReporter Key: HUDI-800 URL: https://issues.apache.org/jira/browse/HUDI-800 Project: Apache Hudi

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1520: [HUDI-797] Small performance improvement for rewriting records.

pratyakshsharma commented on a change in pull request #1520: [HUDI-797] Small performance improvement for rewriting records. URL: https://github.com/apache/incubator-hudi/pull/1520#discussion_r410171475 ## File path:

[GitHub] [incubator-hudi] tverdokhlebd edited a comment on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records

tverdokhlebd edited a comment on issue #1491: [SUPPORT] OutOfMemoryError during upsert 53M records URL: https://github.com/apache/incubator-hudi/issues/1491#issuecomment-615141491 Hi @lamber-ken Yes, I have resolved this problem. Firstly, the problem was in SBT memory that

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1520: [HUDI-797] Small performance improvement for rewriting records.

pratyakshsharma commented on a change in pull request #1520: [HUDI-797] Small performance improvement for rewriting records. URL: https://github.com/apache/incubator-hudi/pull/1520#discussion_r410170533 ## File path:

[GitHub] [incubator-hudi] dahirainbow commented on issue #143: Tracking ticket for folks to be added to slack group

dahirainbow commented on issue #143: Tracking ticket for folks to be added to slack group URL: https://github.com/apache/incubator-hudi/issues/143#issuecomment-615260642 hi,please add me: dahirain...@outlook.com This is an

[GitHub] [incubator-hudi] pratyakshsharma commented on issue #1513: [HUDI-793] Adding proper default to hudi metadata fields and proper handling to rewrite routine

pratyakshsharma commented on issue #1513: [HUDI-793] Adding proper default to hudi metadata fields and proper handling to rewrite routine URL: https://github.com/apache/incubator-hudi/pull/1513#issuecomment-615272042 @vinothchandar I tried few more combinations, looks like only the case