[jira] [Updated] (HUDI-409) Replace Log Magic header with a secure hash to avoid clashes with data

[ https://issues.apache.org/jira/browse/HUDI-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang updated HUDI-409: -- Issue Type: Improvement (was: Bug) > Replace Log Magic header with a secure hash to avoid clashes with data >

[jira] [Assigned] (HUDI-409) Replace Log Magic header with a secure hash to avoid clashes with data

[ https://issues.apache.org/jira/browse/HUDI-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] vinoyang reassigned HUDI-409: - Assignee: Nishith Agarwal > Replace Log Magic header with a secure hash to avoid clashes with data >

[GitHub] [incubator-hudi] tooptoop4 opened a new issue #1342: [SUPPORT] do cow tables need to be converted when changing from hoodie to hudi?

tooptoop4 opened a new issue #1342: [SUPPORT] do cow tables need to be converted when changing from hoodie to hudi? URL: https://github.com/apache/incubator-hudi/issues/1342 this mentions running utility to convert mor files Migration Guide From com.uber.hoodie to org.apache.hudi - HUDI

[jira] [Updated] (HUDI-408) [Umbrella] Refactor/Code clean up hoodie write client

[ https://issues.apache.org/jira/browse/HUDI-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi updated HUDI-408: --- Fix Version/s: (was: 0.5.2) 0.6.0 > [Umbrella] Refactor/Code clean up hoodie

[jira] [Updated] (HUDI-581) NOTICE need more work as it missing content form included 3rd party ALv2 licensed NOTICE files

[ https://issues.apache.org/jira/browse/HUDI-581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi updated HUDI-581: --- Fix Version/s: (was: 0.6.0) 0.5.2 > NOTICE need more work as it missing content

[GitHub] [incubator-hudi] smarthi commented on a change in pull request #1341: Add exportToTable option to CLI

smarthi commented on a change in pull request #1341: Add exportToTable option to CLI URL: https://github.com/apache/incubator-hudi/pull/1341#discussion_r380993198 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/utils/TempTableUtil.java ## @@ -0,0 +1,131 @@ +/* +

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #193

See Changes: -- [...truncated 2.29 KB...] plexus-classworlds-2.5.2.jar /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml

[GitHub] [incubator-hudi] satishkotha commented on issue #1320: [HUDI-571] Add min/max headers on archived files

satishkotha commented on issue #1320: [HUDI-571] Add min/max headers on archived files URL: https://github.com/apache/incubator-hudi/pull/1320#issuecomment-587992448 @n3nash please take one more look. might be easier to discuss this in person.

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1320: [HUDI-571] Add min/max headers on archived files

satishkotha commented on a change in pull request #1320: [HUDI-571] Add min/max headers on archived files URL: https://github.com/apache/incubator-hudi/pull/1320#discussion_r381034791 ## File path:

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1320: [HUDI-571] Add min/max headers on archived files

satishkotha commented on a change in pull request #1320: [HUDI-571] Add min/max headers on archived files URL: https://github.com/apache/incubator-hudi/pull/1320#discussion_r381030813 ## File path: hudi-client/src/main/java/org/apache/hudi/io/HoodieCommitArchiveLog.java

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1320: [HUDI-571] Add min/max headers on archived files

satishkotha commented on a change in pull request #1320: [HUDI-571] Add min/max headers on archived files URL: https://github.com/apache/incubator-hudi/pull/1320#discussion_r381030780 ## File path:

[jira] [Updated] (HUDI-617) Add support for data types convertible to String in TimestampBasedKeyGenerator

[ https://issues.apache.org/jira/browse/HUDI-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Singh updated HUDI-617: Labels: easyfix pull-request-available (was: easyfix pull-request-available pull-requests-available) > Add

[GitHub] [incubator-hudi] smarthi commented on a change in pull request #1341: Add exportToTable option to CLI

smarthi commented on a change in pull request #1341: Add exportToTable option to CLI URL: https://github.com/apache/incubator-hudi/pull/1341#discussion_r380993346 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/utils/TempTableUtil.java ## @@ -0,0 +1,131 @@ +/* +

[GitHub] [incubator-hudi] smarthi commented on a change in pull request #1341: Add exportToTable option to CLI

smarthi commented on a change in pull request #1341: Add exportToTable option to CLI URL: https://github.com/apache/incubator-hudi/pull/1341#discussion_r380993198 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/utils/TempTableUtil.java ## @@ -0,0 +1,131 @@ +/* +

[GitHub] [incubator-hudi] smarthi commented on a change in pull request #1341: Add exportToTable option to CLI

smarthi commented on a change in pull request #1341: Add exportToTable option to CLI URL: https://github.com/apache/incubator-hudi/pull/1341#discussion_r380990171 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/HoodiePrintHelper.java ## @@ -18,13 +18,16 @@

[GitHub] [incubator-hudi] smarthi commented on a change in pull request #1341: Add exportToTable option to CLI

smarthi commented on a change in pull request #1341: Add exportToTable option to CLI URL: https://github.com/apache/incubator-hudi/pull/1341#discussion_r380990440 ## File path: hudi-cli/src/main/java/org/apache/hudi/cli/HoodiePrintHelper.java ## @@ -57,11 +60,38 @@ public

[GitHub] [incubator-hudi] umehrot2 commented on issue #954: org.apache.hudi.org.apache.hadoop_hive.metastore.api.NoSuchObjectException: table not found

umehrot2 commented on issue #954: org.apache.hudi.org.apache.hadoop_hive.metastore.api.NoSuchObjectException: table not found URL: https://github.com/apache/incubator-hudi/issues/954#issuecomment-587942358 @vinothchandar I am happy to take up this integration piece. I have been keeping

[GitHub] [incubator-hudi] umehrot2 commented on issue #954: org.apache.hudi.org.apache.hadoop_hive.metastore.api.NoSuchObjectException: table not found

umehrot2 commented on issue #954: org.apache.hudi.org.apache.hadoop_hive.metastore.api.NoSuchObjectException: table not found URL: https://github.com/apache/incubator-hudi/issues/954#issuecomment-587941167 @jinshuangxian This particular issue has been fixed since our first release of

[GitHub] [incubator-hudi] satishkotha opened a new pull request #1341: Add exportToTable option to CLI

satishkotha opened a new pull request #1341: Add exportToTable option to CLI URL: https://github.com/apache/incubator-hudi/pull/1341 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review https://hudi.apache.org/contributing.html before opening a pull

[GitHub] [incubator-hudi] ramachandranms commented on a change in pull request #1332: [HUDI -409] Match header and footer block length to improve corrupted block detection

ramachandranms commented on a change in pull request #1332: [HUDI -409] Match header and footer block length to improve corrupted block detection URL: https://github.com/apache/incubator-hudi/pull/1332#discussion_r380939677 ## File path:

[GitHub] [incubator-hudi] ramachandranms commented on a change in pull request #1332: [HUDI -409] Match header and footer block length to improve corrupted block detection

ramachandranms commented on a change in pull request #1332: [HUDI -409] Match header and footer block length to improve corrupted block detection URL: https://github.com/apache/incubator-hudi/pull/1332#discussion_r380938785 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on issue #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile

vinothchandar commented on issue #1176: [HUDI-430] Adding InlineFileSystem to support embedding any file format as an InlineFile URL: https://github.com/apache/incubator-hudi/pull/1176#issuecomment-587730864 @nsivabalan awesome. will do..

[GitHub] [incubator-hudi] vinothchandar commented on issue #1289: [HUDI-92] Provide reasonable names for Spark DAG stages in Hudi.

vinothchandar commented on issue #1289: [HUDI-92] Provide reasonable names for Spark DAG stages in Hudi. URL: https://github.com/apache/incubator-hudi/pull/1289#issuecomment-587712891 >So we need to label each stage and they should show up correctly. if we could do that, and if you

[GitHub] [incubator-hudi] prashantwason commented on issue #1289: [HUDI-92] Provide reasonable names for Spark DAG stages in Hudi.

prashantwason commented on issue #1289: [HUDI-92] Provide reasonable names for Spark DAG stages in Hudi. URL: https://github.com/apache/incubator-hudi/pull/1289#issuecomment-587701211 > 1. It seems like you are only covering cases where an RDD is getting created? Is it possible to change

[jira] [Updated] (HUDI-617) Add support for data types convertible to String in TimestampBasedKeyGenerator

[ https://issues.apache.org/jira/browse/HUDI-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-617: Labels: easyfix pull-request-available pull-requests-available (was: easyfix

[incubator-hudi] branch master updated (8c6138c -> c2b08cd)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. from 8c6138c [MINOR] Add javadoc to SchedulerConfGenerator and code clean (#1340) add c2b08cd [HUDI-617]

[GitHub] [incubator-hudi] vinothchandar merged pull request #1339: [HUDI-617] Add support for types implementing CharSequence

vinothchandar merged pull request #1339: [HUDI-617] Add support for types implementing CharSequence URL: https://github.com/apache/incubator-hudi/pull/1339 This is an automated message from the Apache Git Service. To

[incubator-hudi] branch master updated (0049323 -> 8c6138c)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. from 0049323 [HUDI-108] Removing 2GB spark partition limitations in HoodieBloomIndex with spark 2.4.4 (#1315)

[GitHub] [incubator-hudi] vinothchandar merged pull request #1340: [MINOR] Add javadoc to SchedulerConfGenerator and Code clean

vinothchandar merged pull request #1340: [MINOR] Add javadoc to SchedulerConfGenerator and Code clean URL: https://github.com/apache/incubator-hudi/pull/1340 This is an automated message from the Apache Git Service. To

[jira] [Updated] (HUDI-108) Simplify HoodieBloomIndex without the need for 2GB limit handling

[ https://issues.apache.org/jira/browse/HUDI-108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-108: Labels: pull-request-available (was: ) > Simplify HoodieBloomIndex without the need for 2GB limit

[incubator-hudi] branch master updated (b8f9d0e -> 0049323)

This is an automated email from the ASF dual-hosted git repository. vinoth pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-hudi.git. from b8f9d0e [HUDI-615]: Add some methods and test cases for StringUtils. (#1338) add 0049323 [HUDI-108]

[GitHub] [incubator-hudi] vinothchandar merged pull request #1315: [HUDI-108] Removing 2GB spark partition limitations in HoodieBloomIndex with spark 2.4.4

vinothchandar merged pull request #1315: [HUDI-108] Removing 2GB spark partition limitations in HoodieBloomIndex with spark 2.4.4 URL: https://github.com/apache/incubator-hudi/pull/1315 This is an automated message from the

[GitHub] [incubator-hudi] prashantwason commented on a change in pull request #1310: [HUDI-601] Improve unit test coverage for HoodieAvroWriteSupport, HoodieRealtimeRecordReader, RealtimeCompactedReco

prashantwason commented on a change in pull request #1310: [HUDI-601] Improve unit test coverage for HoodieAvroWriteSupport, HoodieRealtimeRecordReader, RealtimeCompactedRecordReader URL: https://github.com/apache/incubator-hudi/pull/1310#discussion_r380870271 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on issue #1328: Hudi upsert hangs

vinothchandar commented on issue #1328: Hudi upsert hangs URL: https://github.com/apache/incubator-hudi/issues/1328#issuecomment-587663160 Started on this.. Was trying to port to scala, since I am not super familiar with pySpark. Will resume today and circle back. :)

[GitHub] [incubator-hudi] vinothchandar commented on issue #954: org.apache.hudi.org.apache.hadoop_hive.metastore.api.NoSuchObjectException: table not found

vinothchandar commented on issue #954: org.apache.hudi.org.apache.hadoop_hive.metastore.api.NoSuchObjectException: table not found URL: https://github.com/apache/incubator-hudi/issues/954#issuecomment-587661582 @umehrot2 Looks like glue integration keeps coming up :).. Do you want to

[GitHub] [incubator-hudi] prashantwason commented on a change in pull request #1310: [HUDI-601] Improve unit test coverage for HoodieAvroWriteSupport, HoodieRealtimeRecordReader, RealtimeCompactedReco

prashantwason commented on a change in pull request #1310: [HUDI-601] Improve unit test coverage for HoodieAvroWriteSupport, HoodieRealtimeRecordReader, RealtimeCompactedRecordReader URL: https://github.com/apache/incubator-hudi/pull/1310#discussion_r380870206 ## File path:

[GitHub] [incubator-hudi] vinothchandar commented on issue #1334: [HUDI-612] Fix return no data when using incremental query

vinothchandar commented on issue #1334: [HUDI-612] Fix return no data when using incremental query URL: https://github.com/apache/incubator-hudi/pull/1334#issuecomment-587659536 @lamber-ken there is `HoodieDataSourceHelpers` has has some helps to figure out the new commits.. what are you

[GitHub] [incubator-hudi] prashantwason commented on a change in pull request #1310: [HUDI-601] Improve unit test coverage for HoodieAvroWriteSupport, HoodieRealtimeRecordReader, RealtimeCompactedReco

prashantwason commented on a change in pull request #1310: [HUDI-601] Improve unit test coverage for HoodieAvroWriteSupport, HoodieRealtimeRecordReader, RealtimeCompactedRecordReader URL: https://github.com/apache/incubator-hudi/pull/1310#discussion_r380867747 ## File path:

[GitHub] [incubator-hudi] prashantwason commented on a change in pull request #1310: [HUDI-601] Improve unit test coverage for HoodieAvroWriteSupport, HoodieRealtimeRecordReader, RealtimeCompactedReco

prashantwason commented on a change in pull request #1310: [HUDI-601] Improve unit test coverage for HoodieAvroWriteSupport, HoodieRealtimeRecordReader, RealtimeCompactedRecordReader URL: https://github.com/apache/incubator-hudi/pull/1310#discussion_r380861934 ## File path:

[GitHub] [incubator-hudi] nsivabalan commented on issue #1315: [HUDI-108] Removing 2GB spark partition limitations in HoodieBloomIndex with spark 2.4.4

nsivabalan commented on issue #1315: [HUDI-108] Removing 2GB spark partition limitations in HoodieBloomIndex with spark 2.4.4 URL: https://github.com/apache/incubator-hudi/pull/1315#issuecomment-587608958 @vinothchandar / @leesf : PR is good to merge.

[jira] [Updated] (HUDI-589) Fix references to Views in some of the pages. Replace with Query instead

[ https://issues.apache.org/jira/browse/HUDI-589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi updated HUDI-589: --- Fix Version/s: 0.5.2 > Fix references to Views in some of the pages. Replace with Query instead >

[jira] [Updated] (HUDI-604) Update docker page

[ https://issues.apache.org/jira/browse/HUDI-604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi updated HUDI-604: --- Fix Version/s: 0.5.2 > Update docker page > -- > > Key: HUDI-604 >

[jira] [Closed] (HUDI-417) Refactor HoodieWriteClient so that commit logic can be shareable by both bootstrap and normal write operations

[ https://issues.apache.org/jira/browse/HUDI-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi closed HUDI-417. -- > Refactor HoodieWriteClient so that commit logic can be shareable by both > bootstrap and normal write

[jira] [Updated] (HUDI-417) Refactor HoodieWriteClient so that commit logic can be shareable by both bootstrap and normal write operations

[ https://issues.apache.org/jira/browse/HUDI-417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi updated HUDI-417: --- Fix Version/s: (was: 0.6.0) 0.5.2 > Refactor HoodieWriteClient so that commit

[jira] [Updated] (HUDI-502) Provide a custom time zone definition for TimestampBasedKeyGenerator

[ https://issues.apache.org/jira/browse/HUDI-502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi updated HUDI-502: --- Fix Version/s: (was: 0.6.0) 0.5.2 > Provide a custom time zone definition for

[jira] [Closed] (HUDI-502) Provide a custom time zone definition for TimestampBasedKeyGenerator

[ https://issues.apache.org/jira/browse/HUDI-502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi closed HUDI-502. -- > Provide a custom time zone definition for TimestampBasedKeyGenerator >

[jira] [Commented] (HUDI-580) Incorrect license header in docker/hoodie/hadoop/base/entrypoint.sh

[ https://issues.apache.org/jira/browse/HUDI-580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17039161#comment-17039161 ] Suneel Marthi commented on HUDI-580: The license header looks OK to me - I would check with Justin once

[GitHub] [incubator-hudi] wangxianghu opened a new pull request #1340: [MINOR] Add javadoc to SchedulerConfGenerator and Code clean

wangxianghu opened a new pull request #1340: [MINOR] Add javadoc to SchedulerConfGenerator and Code clean URL: https://github.com/apache/incubator-hudi/pull/1340 ## *Tips* - *Thank you very much for contributing to Apache Hudi.* - *Please review

[jira] [Updated] (HUDI-615) Add Test cases for StringUtils.

[ https://issues.apache.org/jira/browse/HUDI-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suneel Marthi updated HUDI-615: --- Status: Closed (was: Patch Available) > Add Test cases for StringUtils. >

[GitHub] [incubator-hudi] pratyakshsharma commented on issue #1339: [HUDI-617] Add support for types implementing CharSequence

pratyakshsharma commented on issue #1339: [HUDI-617] Add support for types implementing CharSequence URL: https://github.com/apache/incubator-hudi/pull/1339#issuecomment-587413861 LGTM This is an automated message from the

[jira] [Updated] (HUDI-617) Add support for data ty0es convertible to String in TimestampBasedKeyGenerator

[ https://issues.apache.org/jira/browse/HUDI-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Singh updated HUDI-617: Labels: easyfix pull-requests-available (was: easyfix pull-request-available) > Add support for data types

[jira] [Updated] (HUDI-617) Add support for data types convertible to String in TimestampBasedKeyGenerator

[ https://issues.apache.org/jira/browse/HUDI-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HUDI-617: Labels: easyfix pull-request-available (was: easyfix) > Add support for data types convertible to

[GitHub] [incubator-hudi] amitsingh-10 opened a new pull request #1339: [HUDI-617] Add support for types implementing CharSequence

amitsingh-10 opened a new pull request #1339: [HUDI-617] Add support for types implementing CharSequence URL: https://github.com/apache/incubator-hudi/pull/1339 ## What is the purpose of the pull request Data types extending CharSequence implement a #toString method which provides an

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r380555306 ## File path:

[jira] [Commented] (HUDI-617) Add support for data types convertible to String in TimestampBasedKeyGenerator

[ https://issues.apache.org/jira/browse/HUDI-617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17038864#comment-17038864 ] Amit Singh commented on HUDI-617: - I have attached the sample data and schema to replicate the issue. >

[jira] [Updated] (HUDI-617) Add support for data types convertible to String in TimestampBasedKeyGenerator

[ https://issues.apache.org/jira/browse/HUDI-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Singh updated HUDI-617: Attachment: test_schema.avsc test_data.json > Add support for data types convertible to

[jira] [Updated] (HUDI-617) Add support for data types convertible to String in TimestampBasedKeyGenerator

[ https://issues.apache.org/jira/browse/HUDI-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Singh updated HUDI-617: Fix Version/s: (was: 0.5.1) > Add support for data types convertible to String in

[jira] [Updated] (HUDI-617) Add support for data types convertible to String in TimestampBasedKeyGenerator

[ https://issues.apache.org/jira/browse/HUDI-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Singh updated HUDI-617: Fix Version/s: 0.5.1 > Add support for data types convertible to String in TimestampBasedKeyGenerator >

[jira] [Created] (HUDI-617) Add support for data types convertible to String in TimestampBasedKeyGenerator

Amit Singh created HUDI-617: --- Summary: Add support for data types convertible to String in TimestampBasedKeyGenerator Key: HUDI-617 URL: https://issues.apache.org/jira/browse/HUDI-617 Project: Apache Hudi

[jira] [Updated] (HUDI-617) Add support for data types convertible to String in TimestampBasedKeyGenerator

[ https://issues.apache.org/jira/browse/HUDI-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Singh updated HUDI-617: Description: Currently, TimestampBasedKeyGenerator only supports 4 data types for the partition key. They