nmahmood630 commented on issue #2987:
URL: https://github.com/apache/hudi/issues/2987#issuecomment-848397902
I think the issue I am seeing is similar to:
https://github.com/apache/hudi/issues/2002 where all the records commit time
are getting updated since this table holds aggregation
Xianghu Wang created HUDI-1934:
--
Summary: Update keyGenerator configuration docs
Key: HUDI-1934
URL: https://issues.apache.org/jira/browse/HUDI-1934
Project: Apache Hudi
Issue Type: Sub-task
codecov-commenter edited a comment on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
codecov-commenter edited a comment on pull request #2926:
URL: https://github.com/apache/hudi/pull/2926#issuecomment-835303925
#
Vinay created HUDI-1935:
---
Summary: Update Logger for FlatteningTransformer
Key: HUDI-1935
URL: https://issues.apache.org/jira/browse/HUDI-1935
Project: Apache Hudi
Issue Type: Task
[
https://issues.apache.org/jira/browse/HUDI-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351505#comment-17351505
]
Ethan Guo commented on HUDI-1138:
-
Here is my plan for improving the marker file mechanism:
* Abstraction
pengzhiwei2018 commented on a change in pull request #2926:
URL: https://github.com/apache/hudi/pull/2926#discussion_r639372337
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala
##
@@ -131,15 +133,28 @@ class
veenaypatil commented on pull request #2996:
URL: https://github.com/apache/hudi/pull/2996#issuecomment-848469043
Unit tests for hudi-spark-client failed but this change should not be the
cause of it. @wangxianghu @vinothchandar can you pls merge
--
This is an automated message from
codecov-commenter edited a comment on pull request #2996:
URL: https://github.com/apache/hudi/pull/2996#issuecomment-848443240
#
codecov-commenter edited a comment on pull request #2996:
URL: https://github.com/apache/hudi/pull/2996#issuecomment-848443240
#
codecov-commenter edited a comment on pull request #2996:
URL: https://github.com/apache/hudi/pull/2996#issuecomment-848443240
#
codecov-commenter edited a comment on pull request #2996:
URL: https://github.com/apache/hudi/pull/2996#issuecomment-848443240
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
[
https://issues.apache.org/jira/browse/HUDI-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1935:
-
Labels: pull-request-available (was: )
> Update Logger for FlatteningTransformer
>
veenaypatil opened a new pull request #2996:
URL: https://github.com/apache/hudi/pull/2996
## What is the purpose of the pull request
This PR just updates the Logger statement as it was pointing to different
class
## Brief change log
Modify Logger statement of
codecov-commenter edited a comment on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059
#
codecov-commenter edited a comment on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059
#
danny0405 commented on a change in pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#discussion_r639363503
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/embedded/EmbeddedTimelineServerHelper.java
##
@@ -35,16 +35,22 @@
codecov-commenter commented on pull request #2996:
URL: https://github.com/apache/hudi/pull/2996#issuecomment-848443240
#
[Codecov](https://codecov.io/gh/apache/hudi/pull/2996?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
codecov-commenter edited a comment on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059
#
codecov-commenter edited a comment on pull request #2926:
URL: https://github.com/apache/hudi/pull/2926#issuecomment-835303925
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
codecov-commenter edited a comment on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059
#
codecov-commenter edited a comment on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059
#
codecov-commenter edited a comment on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059
#
pengzhiwei2018 commented on a change in pull request #2926:
URL: https://github.com/apache/hudi/pull/2926#discussion_r639362769
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala
##
@@ -182,4 +197,98 @@ object
pengzhiwei2018 commented on a change in pull request #2925:
URL: https://github.com/apache/hudi/pull/2925#discussion_r639374959
##
File path:
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java
##
@@ -194,11 +197,17 @@ private void syncSchema(String
codecov-commenter edited a comment on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059
#
codecov-commenter edited a comment on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059
#
codecov-commenter edited a comment on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific
nsivabalan merged pull request #2845:
URL: https://github.com/apache/hudi/pull/2845
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service,
This is an automated email from the ASF dual-hosted git repository.
sivabalan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new afa6bc0 [HUDI-1723] Fix path selector listing
sbernauer commented on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-847796132
Sure @nsivabalan i will try out the fix in
https://github.com/apache/hudi/pull/2927 and give feedback.
Thanks for the invitation for slack, i appreciate! My memberId is
loukey_j created HUDI-1931:
--
Summary: BucketAssignFunction use wrong state
Key: HUDI-1931
URL: https://issues.apache.org/jira/browse/HUDI-1931
Project: Apache Hudi
Issue Type: Improvement
nsivabalan edited a comment on issue #2992:
URL: https://github.com/apache/hudi/issues/2992#issuecomment-847899578
@ayush71994 :
1. May I know which config you are referring to here "delete.duplicates"?
Can you point me to full config from here
nsivabalan edited a comment on issue #2992:
URL: https://github.com/apache/hudi/issues/2992#issuecomment-847899578
@ayush71994 :
1. May I know which config you are referring to here "delete.duplicates"?
Can you point me to full config from here
nsivabalan commented on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-847846030
@sbernauer : sorry I might need your email to invite to apache hudi's slack
workspace.
--
This is an automated message from the Apache Git Service.
To respond to the
sbernauer commented on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-847848890
No problem. It is bernaue...@web.de
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
nsivabalan commented on issue #2992:
URL: https://github.com/apache/hudi/issues/2992#issuecomment-847899578
@ayush71994 : May I know which config you are referring to here
"delete.duplicates"? Can you point me to full config from here
https://hudi.apache.org/docs/configurations.html
wangxianghu opened a new pull request #2993:
URL: https://github.com/apache/hudi/pull/2993
…y type
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is
[
https://issues.apache.org/jira/browse/HUDI-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1929:
-
Labels: pull-request-available (was: )
> Make HoodieDeltaStreamer support configure KeyGenerator
nsivabalan commented on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-847855344
@sbernauer @giaosudau @dirksan28 @sathyaprakashg : There are quite a few
flows or use-cases in general wrt schema evolution. Would you mind helping us
explain your use-case.
nsivabalan edited a comment on issue #2992:
URL: https://github.com/apache/hudi/issues/2992#issuecomment-847899578
@ayush71994 : May I know which config you are referring to here
"delete.duplicates"? Can you point me to full config from here
[
https://issues.apache.org/jira/browse/HUDI-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351095#comment-17351095
]
Sugamber commented on HUDI-1668:
[~nishith29] Yes, We can close this.
Thank you!!!
>
nsivabalan edited a comment on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-847855344
@sbernauer @giaosudau @dirksan28 @sathyaprakashg : There are quite a few
flows or use-cases in general wrt schema evolution. Would you mind helping us
explain your
codecov-commenter edited a comment on pull request #2926:
URL: https://github.com/apache/hudi/pull/2926#issuecomment-835303925
#
[
https://issues.apache.org/jira/browse/HUDI-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinay updated HUDI-1935:
Status: In Progress (was: Open)
> Update Logger for FlatteningTransformer
>
Biswajit mohapatra created HUDI-1936:
Summary: Introduce a optional property for conditional upsert
Key: HUDI-1936
URL: https://issues.apache.org/jira/browse/HUDI-1936
Project: Apache Hudi
codecov-commenter edited a comment on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848384059
#
wangxianghu commented on pull request #2993:
URL: https://github.com/apache/hudi/pull/2993#issuecomment-848482449
@yanghua please take a look when free
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
Xianghu Wang created HUDI-1930:
--
Summary: Make Spark DataSource support configure KeyGenerator by
type
Key: HUDI-1930
URL: https://issues.apache.org/jira/browse/HUDI-1930
Project: Apache Hudi
ayush71994 opened a new issue #2992:
URL: https://github.com/apache/hudi/issues/2992
**_Tips before filing an issue_**
- Have you gone through our
[FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)?
- Yes.
- Join the mailing list to engage in conversations and get
yuzhaojing closed pull request #2988:
URL: https://github.com/apache/hudi/pull/2988
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service,
danny0405 commented on pull request #2990:
URL: https://github.com/apache/hudi/pull/2990#issuecomment-847623604
cc @garyli1019, i know that you are using the Flink hudi pipeline, maybe you
should also take a review for this. I got an impression that we should add a
config option for both
codecov-commenter commented on pull request #2991:
URL: https://github.com/apache/hudi/pull/2991#issuecomment-847672177
#
[Codecov](https://codecov.io/gh/apache/hudi/pull/2991?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
Xianghu Wang created HUDI-1929:
--
Summary: Make HoodieDeltaStreamer support configure KeyGenerator
by type
Key: HUDI-1929
URL: https://issues.apache.org/jira/browse/HUDI-1929
Project: Apache Hudi
yuzhaojing opened a new pull request #2988:
URL: https://github.com/apache/hudi/pull/2988
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of the
yuzhaojing opened a new pull request #2989:
URL: https://github.com/apache/hudi/pull/2989
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of the
yuzhaojing closed pull request #2989:
URL: https://github.com/apache/hudi/pull/2989
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service,
[
https://issues.apache.org/jira/browse/HUDI-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yuzhaojing updated HUDI-1923:
-
Description:
In flink, notifyCheckpointComplete not in checkpoint life cycle. If a
checkpoint is success
am-cpp commented on issue #2992:
URL: https://github.com/apache/hudi/issues/2992#issuecomment-847745284
The issue seems to be happening only when the **INSERT_DROP_DUPS_OPT_KEY**
flag is set to **true**. Looks like this config is being used for both:
1. Pre-combining:
jintaoguan opened a new pull request #2991:
URL: https://github.com/apache/hudi/pull/2991
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of the
[
https://issues.apache.org/jira/browse/HUDI-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang updated HUDI-1920:
---
Fix Version/s: 0.9.0
> Set "archived" as the default value of HOODIE_ARCHIVELOG_FOLDER_PROP_NAME
>
This is an automated email from the ASF dual-hosted git repository.
vinoyang pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git.
from aba1ead [HUDI-1919] Type mismatch when streaming read copy_on_write
table using flink (#2986)
add e702074
yanghua merged pull request #2978:
URL: https://github.com/apache/hudi/pull/2978
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please
[
https://issues.apache.org/jira/browse/HUDI-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang updated HUDI-1920:
---
Issue Type: Improvement (was: Task)
> Set "archived" as the default value of
jintaoguan closed pull request #2991:
URL: https://github.com/apache/hudi/pull/2991
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service,
codecov-commenter edited a comment on pull request #2902:
URL: https://github.com/apache/hudi/pull/2902#issuecomment-829931431
#
yuzhaojing opened a new pull request #2990:
URL: https://github.com/apache/hudi/pull/2990
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of the
[
https://issues.apache.org/jira/browse/HUDI-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1923:
-
Labels: pull-request-available (was: )
> Add state in StreamWriteFunction to restore
>
Tandoy commented on issue #143:
URL: https://github.com/apache/hudi/issues/143#issuecomment-847629180
please add me to the slack group
the email: tangzhi8...@gmail.com
thanks a lot
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
[
https://issues.apache.org/jira/browse/HUDI-1920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vinoyang closed HUDI-1920.
--
Resolution: Implemented
e7020748b500e38838a8d84df64267a07b529aa7
> Set "archived" as the default value of
nsivabalan commented on pull request #2923:
URL: https://github.com/apache/hudi/pull/2923#issuecomment-847933890
Can you check CI failure please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
nsivabalan commented on pull request #2310:
URL: https://github.com/apache/hudi/pull/2310#issuecomment-847950023
yeah, looks like it. closing it for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
nsivabalan edited a comment on pull request #2902:
URL: https://github.com/apache/hudi/pull/2902#issuecomment-847918561
Can you check why CI is failing. we can land once fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
vinothchandar commented on pull request #2388:
URL: https://github.com/apache/hudi/pull/2388#issuecomment-848022526
@n3nash @satishkotha Any updates on this? generally love to get these follow
ups from clustering over the fence if we can
--
This is an automated message from the Apache
vinothchandar merged pull request #2981:
URL: https://github.com/apache/hudi/pull/2981
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service,
This is an automated email from the ASF dual-hosted git repository.
vinoth pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git.
from afa6bc0 [HUDI-1723] Fix path selector listing files with the same mod
date (#2845)
add 112732d [HUDI-1922]
nsivabalan commented on a change in pull request #2923:
URL: https://github.com/apache/hudi/pull/2923#discussion_r638867446
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java
##
@@ -62,7 +62,7 @@
} else if
vinothchandar commented on pull request #2378:
URL: https://github.com/apache/hudi/pull/2378#issuecomment-848021692
#2926 overlaps with this? @yui2010 , @pengzhiwei2018 any thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
vinothchandar commented on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-848040019
@danny0405 we left this hanging a bit. Let me re-review this and get it
landing in some form.
--
This is an automated message from the Apache Git Service.
To respond to
nsivabalan commented on pull request #2902:
URL: https://github.com/apache/hudi/pull/2902#issuecomment-847918561
Can you check why CI is failing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
sbernauer edited a comment on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-847940734
Hi @nsivabalan,
we have multiple schema versions of the events we consume. We use kafka and
Confluent Schema Registry. I think all the events in kafka are written
[
https://issues.apache.org/jira/browse/HUDI-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan reopened HUDI-1723:
---
> DFSPathSelector skips files with the same modify date when read up to source
> limit
>
[
https://issues.apache.org/jira/browse/HUDI-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1763:
--
Status: In Progress (was: Open)
> DefaultHoodieRecordPayload does not honor ordering
[
https://issues.apache.org/jira/browse/HUDI-1763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1763:
--
Status: Patch Available (was: In Progress)
> DefaultHoodieRecordPayload does not honor
n3nash commented on issue #2975:
URL: https://github.com/apache/hudi/issues/2975#issuecomment-847968028
@fanaticjo Can you help @calleo since you recently implemented a custom
recordpayload while using pyspark ?
--
This is an automated message from the Apache Git Service.
To respond to
loukey-lj opened a new pull request #2994:
URL: https://github.com/apache/hudi/pull/2994
org.apache.hudi.sink.partitioner.BucketAssignFunction#partitionLoadState
and
org.apache.hudi.sink.partitioner.BucketAssignFunction#indexState
use wrong state, RowDataToHoodieFunction was keyby
vinothchandar commented on a change in pull request #2926:
URL: https://github.com/apache/hudi/pull/2926#discussion_r638929143
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/MergeOnReadSnapshotRelation.scala
##
@@ -131,15 +133,28 @@ class
vinothchandar commented on a change in pull request #2903:
URL: https://github.com/apache/hudi/pull/2903#discussion_r638949244
##
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/hudi/DefaultSource.scala
##
@@ -105,7 +105,9 @@ class DefaultSource extends
Raymond Xu created HUDI-1932:
Summary: Hive Sync should not always update last_commit_time_sync
Key: HUDI-1932
URL: https://issues.apache.org/jira/browse/HUDI-1932
Project: Apache Hudi
Issue
vinothchandar commented on pull request #2496:
URL: https://github.com/apache/hudi/pull/2496#issuecomment-848039122
I have not been able to test this on S3. let me pick it up later next week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please
sbernauer commented on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-847940734
Hi @nsivabalan,
we have multiple schema versions of the events we consume. We use kafka and
Confluent Schema Registry. I think all the events in kafka are written with
[
https://issues.apache.org/jira/browse/HUDI-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-1723:
--
Status: Closed (was: Patch Available)
> DFSPathSelector skips files with the same
[
https://issues.apache.org/jira/browse/HUDI-1723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan resolved HUDI-1723.
---
Resolution: Fixed
> DFSPathSelector skips files with the same modify date when read
codecov-commenter edited a comment on pull request #2977:
URL: https://github.com/apache/hudi/pull/2977#issuecomment-846016572
#
codecov-commenter commented on pull request #2994:
URL: https://github.com/apache/hudi/pull/2994#issuecomment-848024173
#
[Codecov](https://codecov.io/gh/apache/hudi/pull/2994?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=The+Apache+Software+Foundation)
vinothchandar commented on a change in pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#discussion_r638986316
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/embedded/EmbeddedTimelineServerHelper.java
##
@@ -35,16 +35,22 @@
codecov-commenter edited a comment on pull request #2899:
URL: https://github.com/apache/hudi/pull/2899#issuecomment-829195458
#
vaibhav-sinha commented on a change in pull request #2923:
URL: https://github.com/apache/hudi/pull/2923#discussion_r639042519
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/keygen/KeyGenUtils.java
##
@@ -62,7 +62,7 @@
} else if
vaibhav-sinha commented on pull request #2923:
URL: https://github.com/apache/hudi/pull/2923#issuecomment-848098918
The tests were clean except for one test case failing before which I had
fixed. But after merging the latest changes from master, I see a lot of tests
failing and the errors
rshanmugam1 opened a new issue #2609:
URL: https://github.com/apache/hudi/issues/2609
**Describe the problem you faced**
Presto query performance with hudi table takes ~2x extra time when compared
to parquet for simple query . data stored in s3. hudi metadata store enabled.
note,
1 - 100 of 125 matches
Mail list logo
