[jira] [Created] (HUDI-762) change the pom.xml to supportmaven version to 3.x

yaojingyi created HUDI-762: -- Summary: change the pom.xml to supportmaven version to 3.x Key: HUDI-762 URL: https://issues.apache.org/jira/browse/HUDI-762 Project: Apache Hudi (incubating) Issue

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema

pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r403715629 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema

pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r403715964 ## File path:

[GitHub] [incubator-hudi] jvaesteves opened a new issue #1488: [SUPPORT] Hudi table has only five rows when record key is binary

jvaesteves opened a new issue #1488: [SUPPORT] Hudi table has only five rows when record key is binary URL: https://github.com/apache/incubator-hudi/issues/1488 I was trying Hudi on some ORC backup files from my Kafka broker, to see if it would be a nice deduplication process for the

[GitHub] [incubator-hudi] lamber-ken commented on issue #1488: [SUPPORT] Hudi table has only five rows when record key is binary

lamber-ken commented on issue #1488: [SUPPORT] Hudi table has only five rows when record key is binary URL: https://github.com/apache/incubator-hudi/issues/1488#issuecomment-609446861 hi, try to reproduce it, what the original type of key?

[GitHub] [incubator-hudi] lamber-ken commented on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs

lamber-ken commented on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs URL: https://github.com/apache/incubator-hudi/issues/1487#issuecomment-609472838 Hi, it works fine in my local env. steps: 1.Add `spark-hive` dependency

[GitHub] [incubator-hudi] jvaesteves commented on issue #1488: [SUPPORT] Hudi table has only five rows when record key is binary

jvaesteves commented on issue #1488: [SUPPORT] Hudi table has only five rows when record key is binary URL: https://github.com/apache/incubator-hudi/issues/1488#issuecomment-609461079 Binary (array of bytes) This is an

[GitHub] [incubator-hudi] pratyakshsharma commented on issue #1486: WIP[HUDI-759] Integrate checkpoint privoder with delta streamer

pratyakshsharma commented on issue #1486: WIP[HUDI-759] Integrate checkpoint privoder with delta streamer URL: https://github.com/apache/incubator-hudi/pull/1486#issuecomment-609480755 LGTM This is an automated message from

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r403746299 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r403746251 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r403746266 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema

pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy default values of fields if not present when rewriting incoming record with new schema URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r403739002 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r403740566 ## File path:

[GitHub] [incubator-hudi] xushiyan commented on issue #1480: [SUPPORT] Backwards Incompatible Schema Evolution

xushiyan commented on issue #1480: [SUPPORT] Backwards Incompatible Schema Evolution URL: https://github.com/apache/incubator-hudi/issues/1480#issuecomment-609466504 @vinothchandar Yes the exporter tool can be used for this purpose, with some changes. It currently supports copying Hudi

[GitHub] [incubator-hudi] symfrog commented on issue #1480: [SUPPORT] Backwards Incompatible Schema Evolution

symfrog commented on issue #1480: [SUPPORT] Backwards Incompatible Schema Evolution URL: https://github.com/apache/incubator-hudi/issues/1480#issuecomment-609489074 @xushiyan Yes, thanks, that would work. I am using COW for the tables.

[GitHub] [incubator-hudi] garyli1019 commented on issue #1486: [HUDI-759] Integrate checkpoint privoder with delta streamer

garyli1019 commented on issue #1486: [HUDI-759] Integrate checkpoint privoder with delta streamer URL: https://github.com/apache/incubator-hudi/pull/1486#issuecomment-609493045 Test added. Thanks for the review This is an

[GitHub] [incubator-hudi] malanb5 commented on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs

malanb5 commented on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs URL: https://github.com/apache/incubator-hudi/issues/1487#issuecomment-609493874 I was running this through the JVM not the script spark-submit which loaded in the

[GitHub] [incubator-hudi] malanb5 closed issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs

malanb5 closed issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs URL: https://github.com/apache/incubator-hudi/issues/1487 This is an automated message from the Apache Git

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r403740013 ## File path:

[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment

pratyakshsharma commented on a change in pull request #1150: [HUDI-288]: Add support for ingesting multiple kafka streams in a single DeltaStreamer deployment URL: https://github.com/apache/incubator-hudi/pull/1150#discussion_r403746232 ## File path:

[GitHub] [incubator-hudi] malanb5 commented on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs

malanb5 commented on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs URL: https://github.com/apache/incubator-hudi/issues/1487#issuecomment-609480782 @lamber-ken Thank you for the help. I updated the version of Hive. Now I'm getting

[GitHub] [incubator-hudi] malanb5 opened a new issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs

malanb5 opened a new issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs URL: https://github.com/apache/incubator-hudi/issues/1487 Receiving the following Exception when querying data brought in from a SparkSession from a Hive table, which

[jira] [Updated] (HUDI-724) Parallelize GetSmallFiles For Partitions

[ https://issues.apache.org/jira/browse/HUDI-724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-724: --- Fix Version/s: 0.6.0 > Parallelize GetSmallFiles For Partitions > > >

[jira] [Updated] (HUDI-742) Fix java.lang.NoSuchMethodError: java.lang.Math.floorMod(JI)I

[ https://issues.apache.org/jira/browse/HUDI-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-742: --- Fix Version/s: 0.6.0 > Fix java.lang.NoSuchMethodError: java.lang.Math.floorMod(JI)I >

[jira] [Updated] (HUDI-762) modify the pom.xml to support maven 3.x

[ https://issues.apache.org/jira/browse/HUDI-762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yaojingyi updated HUDI-762: --- Summary: modify the pom.xml to support maven 3.x (was: change the pom.xml to supportmaven version to 3.x) >

[jira] [Updated] (HUDI-713) Datasource Writer throws error on resolving array of struct fields

[ https://issues.apache.org/jira/browse/HUDI-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-713: --- Status: Open (was: New) > Datasource Writer throws error on resolving array of struct fields >

[jira] [Resolved] (HUDI-713) Datasource Writer throws error on resolving array of struct fields

[ https://issues.apache.org/jira/browse/HUDI-713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-713. Fix Version/s: 0.6.0 Resolution: Fixed Fixed via master: ce0a4c64d07d6eea926d1bfb92b69ae387b88f50 >

[jira] [Updated] (HUDI-762) modify the pom.xml to support maven 3.x

[ https://issues.apache.org/jira/browse/HUDI-762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yaojingyi updated HUDI-762: --- Priority: Trivial (was: Major) > modify the pom.xml to support maven 3.x >

[jira] [Updated] (HUDI-762) modify the pom.xml to support maven 3.x

[ https://issues.apache.org/jira/browse/HUDI-762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yaojingyi updated HUDI-762: --- Description: I met [ERROR] [ERROR] Some problems were encountered while processing the POMs: [WARNING]

[jira] [Resolved] (HUDI-756) Organize Cleaning Action execution into a single package in hudi-client

[ https://issues.apache.org/jira/browse/HUDI-756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf resolved HUDI-756. Resolution: Fixed Fixed via master: eaf6cc2d90bf27c0d9414a4ea18dbd1b61f58e50 > Organize Cleaning Action execution

[jira] [Updated] (HUDI-717) Fix HudiHiveClient for Hive 2.x

[ https://issues.apache.org/jira/browse/HUDI-717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] leesf updated HUDI-717: --- Fix Version/s: 0.6.0 > Fix HudiHiveClient for Hive 2.x > --- > > Key:

[GitHub] [incubator-hudi] malanb5 edited a comment on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs

malanb5 edited a comment on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs URL: https://github.com/apache/incubator-hudi/issues/1487#issuecomment-609496962 Posted this on Stack Overflow, hopefully this will help others:

[GitHub] [incubator-hudi] nsivabalan commented on issue #1482: [SUPPORT] Deletion of records through deltaStreamer _hoodie_is_deleted flag does not work as expected

nsivabalan commented on issue #1482: [SUPPORT] Deletion of records through deltaStreamer _hoodie_is_deleted flag does not work as expected URL: https://github.com/apache/incubator-hudi/issues/1482#issuecomment-609498911 @venkee14 : can you try setting a default value for the new field.

[jira] [Assigned] (HUDI-145) Limit the amount of partitions considered for GlobalBloomIndex

[ https://issues.apache.org/jira/browse/HUDI-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jerry reassigned HUDI-145: -- Assignee: jerry > Limit the amount of partitions considered for GlobalBloomIndex >

Build failed in Jenkins: hudi-snapshot-deployment-0.5 #239

See Changes: -- [...truncated 2.35 KB...] /home/jenkins/tools/maven/apache-maven-3.5.4/conf: logging settings.xml toolchains.xml

[jira] [Commented] (HUDI-69) Support realtime view in Spark datasource #136

[ https://issues.apache.org/jira/browse/HUDI-69?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17076023#comment-17076023 ] Yanjia Gary Li commented on HUDI-69: Hello [~bhasudha], I found your commit 

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1486: [HUDI-759] Integrate checkpoint privoder with delta streamer

codecov-io edited a comment on issue #1486: [HUDI-759] Integrate checkpoint privoder with delta streamer URL: https://github.com/apache/incubator-hudi/pull/1486#issuecomment-609364046 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1486?src=pr=h1) Report > Merging

[GitHub] [incubator-hudi] malanb5 commented on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs

malanb5 commented on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs URL: https://github.com/apache/incubator-hudi/issues/1487#issuecomment-609496962 https://stackoverflow.com/a/59823742/8366477

[GitHub] [incubator-hudi] codecov-io edited a comment on issue #1486: [HUDI-759] Integrate checkpoint privoder with delta streamer

codecov-io edited a comment on issue #1486: [HUDI-759] Integrate checkpoint privoder with delta streamer URL: https://github.com/apache/incubator-hudi/pull/1486#issuecomment-609364046 # [Codecov](https://codecov.io/gh/apache/incubator-hudi/pull/1486?src=pr=h1) Report > Merging

[GitHub] [incubator-hudi] lamber-ken commented on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs

lamber-ken commented on issue #1487: [SUPPORT] Exception in thread "main" java.io.IOException: No FileSystem for scheme: hdfs URL: https://github.com/apache/incubator-hudi/issues/1487#issuecomment-609550970 You're always welcome : )

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#discussion_r403839376 ## File path:

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#discussion_r403838755 ## File path:

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#discussion_r403836768 ## File path:

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#discussion_r403838755 ## File path:

[GitHub] [incubator-hudi] satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction

satishkotha commented on a change in pull request #1396: [HUDI-687] Stop incremental reader on RO table before a pending compaction URL: https://github.com/apache/incubator-hudi/pull/1396#discussion_r403836768 ## File path: