[
https://issues.apache.org/jira/browse/HUDI-1747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishith Agarwal updated HUDI-1747:
--
Labels: sev:critical (was: )
> Deltastreamer incremental read is not working on the MOR table
[
https://issues.apache.org/jira/browse/HUDI-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nishith Agarwal updated HUDI-1770:
--
Labels: sev:high (was: )
> Deltastreamer throws errors when not running frequently
>
n3nash commented on issue #2797:
URL: https://github.com/apache/hudi/issues/2797#issuecomment-818459734
@vansimonsen Can you check the issue that @aditiwari01 is pointing to and
check if you are using the correct `KeyGenerators` as well as
`PartitionValueExtractor` (check here ->
n3nash commented on issue #2802:
URL: https://github.com/apache/hudi/issues/2802#issuecomment-818457749
@aditiwari01 What failure do you see ? Generally the following is the
execution path
1. Provide new writer schema to Hudi
2. Write data with new writer schema to hudi log files
rubenssoto closed issue #2588:
URL: https://github.com/apache/hudi/issues/2588
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please
rubenssoto commented on issue #2588:
URL: https://github.com/apache/hudi/issues/2588#issuecomment-818455364
Hello Guys,
I think is something related to EMR and Hive, we made a solution for our own
to only enable hive sync when schema changes.
It could be a good feature to
n3nash commented on issue #2806:
URL: https://github.com/apache/hudi/issues/2806#issuecomment-818455085
@tmac2100 The commands you have provided to reproduce this issue seem to be
related to the internal working of your company. I'm unable to reproduce this
issue with the given command.
rubenssoto commented on issue #2787:
URL: https://github.com/apache/hudi/issues/2787#issuecomment-818453992
Hello,
My Hudi config
`Map(hoodie.datasource.hive_sync.database -> raw_freshdesk,
hoodie.parquet.small.file.limit -> 402653184,
ssdong commented on pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#issuecomment-818401935
> @ssdong possible failures are unrelated. Could you restart the build? I
think you can push an empty commit to restart CI job. (You can also close and
reopen PR)
Yep! Git
satishkotha commented on pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#issuecomment-818389117
> @satishkotha Actually, there is another PR #2809 failing upon
`hudi-utilities` and `hudi-spark-client`at the same spot,
`TestHoodieDeltaStreamer`, in the CI. I wonder if
pengzhiwei2018 commented on issue #2810:
URL: https://github.com/apache/hudi/issues/2810#issuecomment-818387671
Hi @rubenssoto , first you should write some data to the hudi table, then
you can consume the data from that hudi table.
You can refer to the test case in
ssdong commented on pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#issuecomment-818387219
@satishkotha Actually, there is another PR #2809 failing upon
`hudi-utilities` and `hudi-spark-client`at the same spot,
`TestHoodieDeltaStreamer`, in the CI. I wonder if this is a
[
https://issues.apache.org/jira/browse/HUDI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang closed HUDI-1787.
--
Resolution: Done
e16d31dce2386868634963ec7cf79c5dfae03cc7
> Remove the rocksdb jar from hudi-flink-bundle
>
yanghua merged pull request #2807:
URL: https://github.com/apache/hudi/pull/2807
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please
This is an automated email from the ASF dual-hosted git repository.
vinoyang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new e16d31d [HUDI-1787] Remove the rocksdb jar
[
https://issues.apache.org/jira/browse/HUDI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319827#comment-17319827
]
vinoyang commented on HUDI-1787:
[~hiscat] thanks for addressing my concern.
> Remove the rocksdb jar
MyLanPangzi edited a comment on pull request #2807:
URL: https://github.com/apache/hudi/pull/2807#issuecomment-818383636
> > > Hi @MyLanPangzi, we are planning to use `rocksdb` as the default state
backend instead of `hdfs`, would you like to take this ticket
MyLanPangzi commented on pull request #2807:
URL: https://github.com/apache/hudi/pull/2807#issuecomment-818383636
> > > Hi @MyLanPangzi, we are planning to use `rocksdb` as the default state
backend instead of `hdfs`, would you like to take this ticket
[
https://issues.apache.org/jira/browse/HUDI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
谢波 updated HUDI-1787:
-
Description:
Remove the {{RocksDB}} jar from hudi-flink-bundle to avoid conflicts.
hudi-common and hudi-timeline-service
wangxianghu commented on pull request #2807:
URL: https://github.com/apache/hudi/pull/2807#issuecomment-818381188
> > Hi @MyLanPangzi, we are planning to use `rocksdb` as the default state
backend instead of `hdfs`, would you like to take this ticket
[
https://issues.apache.org/jira/browse/HUDI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
谢波 updated HUDI-1787:
-
Description:
Remove the {{RocksDB}} jar from hudi-flink-bundle to avoid conflicts.
hudi-common and hudi-timeline-service
[
https://issues.apache.org/jira/browse/HUDI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
谢波 updated HUDI-1787:
-
Description:
Remove the {{RocksDB}} jar from hudi-flink-bundle to avoid conflicts.
hudi-common and hudi-timeline-service
[
https://issues.apache.org/jira/browse/HUDI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
谢波 updated HUDI-1787:
-
Attachment: image-2021-04-13-10-08-09-099.png
> Remove the rocksdb jar from hudi-flink-bundle
>
[
https://issues.apache.org/jira/browse/HUDI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
谢波 updated HUDI-1787:
-
Description:
Remove the {{RocksDB}} jar from hudi-flink-bundle to avoid conflicts.
hudi-common and hudi-timeline-service
[
https://issues.apache.org/jira/browse/HUDI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
谢波 updated HUDI-1787:
-
Description:
Remove the {{RocksDB}} jar from hudi-flink-bundle to avoid conflicts.
hudi-common and hudi-timeline-service
MyLanPangzi commented on pull request #2807:
URL: https://github.com/apache/hudi/pull/2807#issuecomment-818374939
> Hi @MyLanPangzi, we are planning to use `rocksdb` as the default state
backend instead of `hdfs`, would you like to take this ticket
danny0405 commented on a change in pull request #2808:
URL: https://github.com/apache/hudi/pull/2808#discussion_r612072393
##
File path:
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java
##
@@ -162,17 +162,54 @@ public void
danny0405 commented on a change in pull request #2808:
URL: https://github.com/apache/hudi/pull/2808#discussion_r612072353
##
File path:
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java
##
@@ -162,17 +162,54 @@ public void
danny0405 commented on a change in pull request #2808:
URL: https://github.com/apache/hudi/pull/2808#discussion_r612072088
##
File path:
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java
##
@@ -431,32 +470,49 @@ public String
[
https://issues.apache.org/jira/browse/HUDI-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Govindarajan updated HUDI-1790:
--
Status: In Progress (was: Open)
> Add SqlSource for DeltaStreamer to support backfill
Vinoth Govindarajan created HUDI-1790:
-
Summary: Add SqlSource for DeltaStreamer to support backfill use
cases
Key: HUDI-1790
URL: https://issues.apache.org/jira/browse/HUDI-1790
Project: Apache
nbalajee commented on a change in pull request #2607:
URL: https://github.com/apache/hudi/pull/2607#discussion_r612022146
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/HoodieCreateHandle.java
##
@@ -196,6 +200,10 @@ public IOType getIOType() {
nbalajee commented on pull request #2607:
URL: https://github.com/apache/hudi/pull/2607#issuecomment-818309223
@prashantwason @n3nash - updated the diff addressing the comments. Added
bloomIndex metrics.
--
This is an automated message from the Apache Git Service.
To respond to the
prashantwason commented on a change in pull request #2740:
URL: https://github.com/apache/hudi/pull/2740#discussion_r611995283
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/bootstrap/HoodieSparkBootstrapSchemaProvider.java
##
@@ -47,7 +47,7
prashantwason commented on a change in pull request #2793:
URL: https://github.com/apache/hudi/pull/2793#discussion_r611979346
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/storage/HoodieOrcWriter.java
##
@@ -0,0 +1,176 @@
+/*
+ * Licensed to
TeRS-K commented on a change in pull request #2740:
URL: https://github.com/apache/hudi/pull/2740#discussion_r611989076
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/bootstrap/HoodieSparkBootstrapSchemaProvider.java
##
@@ -47,7 +47,7 @@
prashantwason commented on a change in pull request #2740:
URL: https://github.com/apache/hudi/pull/2740#discussion_r611943226
##
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/bootstrap/HoodieSparkBootstrapSchemaProvider.java
##
@@ -47,7 +47,7
ssdong commented on pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#issuecomment-818047358
hmm, that's strange. I wonder why it would break the CI..
Taking out `new byte[0]` from `createMetaFile` should have 0 impact. It's
supposed to create empty files anyway..
susudong removed a comment on pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#issuecomment-818025806
Weird. Didn’t think the trivial update would break the CI. Taking a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please
susudong commented on pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#issuecomment-818025806
Weird. Didn’t think the trivial update would break the CI. Taking a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
codecov-io commented on pull request #2678:
URL: https://github.com/apache/hudi/pull/2678#issuecomment-817985394
#
[Codecov](https://codecov.io/gh/apache/hudi/pull/2678?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
Report
ssdong commented on pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#issuecomment-817964952
@satishkotha Thank you for your reviewing! I've resolved the comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
ssdong commented on a change in pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#discussion_r611792855
##
File path:
hudi-common/src/test/java/org/apache/hudi/common/testutils/FileCreateUtils.java
##
@@ -157,11 +157,19 @@ public static void
[
https://issues.apache.org/jira/browse/HUDI-1746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1746:
-
Labels: pull-request-available (was: )
> Added support for replace commits in hudi-cli
>
jsbali commented on a change in pull request #2678:
URL: https://github.com/apache/hudi/pull/2678#discussion_r611781544
##
File path:
hudi-cli/src/main/java/org/apache/hudi/cli/commands/CommitsCommand.java
##
@@ -266,12 +267,15 @@ public String showCommitPartitions(
satishkotha commented on a change in pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#discussion_r611774078
##
File path:
hudi-client/hudi-spark-client/src/test/java/org/apache/hudi/io/TestHoodieTimelineArchiveLog.java
##
@@ -248,7 +252,7 @@ public void
rubenssoto opened a new issue #2810:
URL: https://github.com/apache/hudi/issues/2810
Hello Guys,
How hudi with spark structured streaming works?
I'm trying to use no error, but no files are written.
---Function to read Hudi Table
def read_hudi_stream(spark_session,
[
https://issues.apache.org/jira/browse/HUDI-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319485#comment-17319485
]
Jagmeet Bali commented on HUDI-1789:
https://github.com/apache/hudi/pull/2809
> Support snapshot
[
https://issues.apache.org/jira/browse/HUDI-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1789:
-
Labels: pull-request-available (was: )
> Support snapshot queries with maxInstantTimes
>
jsbali opened a new pull request #2809:
URL: https://github.com/apache/hudi/pull/2809
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of the
Jagmeet Bali created HUDI-1789:
--
Summary: Support snapshot queries with maxInstantTimes
Key: HUDI-1789
URL: https://issues.apache.org/jira/browse/HUDI-1789
Project: Apache Hudi
Issue Type: New
leesf commented on a change in pull request #2808:
URL: https://github.com/apache/hudi/pull/2808#discussion_r611611535
##
File path:
hudi-flink/src/main/java/org/apache/hudi/sink/partitioner/BucketAssignFunction.java
##
@@ -190,7 +191,9 @@ public void processElement(I value,
leesf commented on a change in pull request #2808:
URL: https://github.com/apache/hudi/pull/2808#discussion_r611610238
##
File path:
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java
##
@@ -162,17 +162,54 @@ public void
leesf commented on a change in pull request #2808:
URL: https://github.com/apache/hudi/pull/2808#discussion_r611607037
##
File path:
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java
##
@@ -162,17 +162,54 @@ public void
leesf commented on a change in pull request #2808:
URL: https://github.com/apache/hudi/pull/2808#discussion_r611597613
##
File path:
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java
##
@@ -431,32 +470,49 @@ public String
[
https://issues.apache.org/jira/browse/HUDI-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1788:
-
Labels: pull-request-available (was: )
> Insert overwrite (table) for Flink writer
>
danny0405 opened a new pull request #2808:
URL: https://github.com/apache/hudi/pull/2808
Supports `INSERT OVERWRITE` and `INSERT OVERWRITE TABLE` for Flink
writer.
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review
[
https://issues.apache.org/jira/browse/HUDI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319335#comment-17319335
]
vinoyang commented on HUDI-1787:
Hi [~hiscat] Can you share the detailed conflicting information (or stack
liujinhui1994 commented on pull request #2710:
URL: https://github.com/apache/hudi/pull/2710#issuecomment-817658270
@yanghua @xushiyan please help review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
wangxianghu commented on pull request #2807:
URL: https://github.com/apache/hudi/pull/2807#issuecomment-817635339
Hi @MyLanPangzi, we are planning to use `rocksdb` as the default state
backend instead of `hdfs`, would you like to take this ticket
[
https://issues.apache.org/jira/browse/HUDI-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang closed HUDI-1786.
--
Resolution: Done
1ff99ca7d7bbdcb24ebae96c434a35000eded1fc
> Add option for merge max memory
>
This is an automated email from the ASF dual-hosted git repository.
vinoyang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new 1ff99ca [HUDI-1786] Add option for merge max
yanghua merged pull request #2805:
URL: https://github.com/apache/hudi/pull/2805
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please
[
https://issues.apache.org/jira/browse/HUDI-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang closed HUDI-1785.
--
Resolution: Done
040756d8c0db4633ac3cccaa11e4172e8e077e1e
> Move OperationConverter to hudi-client-common for
codecov-io edited a comment on pull request #2805:
URL: https://github.com/apache/hudi/pull/2805#issuecomment-817436943
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2805?src=pr=h1) Report
> Merging
[#2805](https://codecov.io/gh/apache/hudi/pull/2805?src=pr=desc) (e892930)
into
codecov-io commented on pull request #2807:
URL: https://github.com/apache/hudi/pull/2807#issuecomment-817599925
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2807?src=pr=h1) Report
> Merging
[#2807](https://codecov.io/gh/apache/hudi/pull/2807?src=pr=desc) (97b6f4e)
into
[
https://issues.apache.org/jira/browse/HUDI-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang updated HUDI-1785:
---
Fix Version/s: 0.9.0
> Move OperationConverter to hudi-client-common for code reuse
>
This is an automated email from the ASF dual-hosted git repository.
vinoyang pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git.
from 1da16df [HUDI-1784] Added print detailed stack log when hbase
connection error (#2799)
add 040756d
yanghua merged pull request #2798:
URL: https://github.com/apache/hudi/pull/2798
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please
[
https://issues.apache.org/jira/browse/HUDI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1787:
-
Labels: pull-request-available (was: )
> Remove the rocksdb jar from hudi-flink-bundle
>
MyLanPangzi opened a new pull request #2807:
URL: https://github.com/apache/hudi/pull/2807
Remove the RocksDB jar from hudi-flink-bundle to avoid conflicts.
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review
[
https://issues.apache.org/jira/browse/HUDI-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
谢波 reassigned HUDI-1787:
Assignee: 谢波
> Remove the rocksdb jar from hudi-flink-bundle
> -
>
>
[
https://issues.apache.org/jira/browse/HUDI-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17319131#comment-17319131
]
Lutz Weischer commented on HUDI-1657:
-
Thanks, that helped a lot. However the tests fail:
[jw@cn06
danny0405 commented on pull request #2805:
URL: https://github.com/apache/hudi/pull/2805#issuecomment-817577121
Close and re-open to re-trigger the CI tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
danny0405 closed pull request #2805:
URL: https://github.com/apache/hudi/pull/2805
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please
tmac2100 opened a new issue #2806:
URL: https://github.com/apache/hudi/issues/2806
**_Tips before filing an issue_**
- Have you gone through our
[FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)?
- Join the mailing list to engage in conversations and get faster
ssdong commented on a change in pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#discussion_r611372415
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/utils/MetadataConversionUtils.java
##
@@ -76,15 +76,16 @@ public static
aditiwari01 commented on issue #2801:
URL: https://github.com/apache/hudi/issues/2801#issuecomment-817537973
Issue resolved after using correct key generator class. Closing the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
aditiwari01 closed issue #2801:
URL: https://github.com/apache/hudi/issues/2801
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please
This is an automated email from the ASF dual-hosted git repository.
vinoth pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 721e797 Travis CI build asf-site
721e797 is
[
https://issues.apache.org/jira/browse/HUDI-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang closed HUDI-1782.
--
Resolution: Done
eb9b9ca286b638312e6af0294a4b072c62ab0ddf
> Add more options for HUDI Flink
>
[
https://issues.apache.org/jira/browse/HUDI-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang reassigned HUDI-1782:
--
Assignee: Danny Chen
> Add more options for HUDI Flink
> ---
>
>
[
https://issues.apache.org/jira/browse/HUDI-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang updated HUDI-1782:
---
Fix Version/s: 0.9.0
> Add more options for HUDI Flink
> ---
>
>
yanghua merged pull request #2786:
URL: https://github.com/apache/hudi/pull/2786
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please
84 matches
Mail list logo
