n3nash edited a comment on pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#issuecomment-740424176
@satishkotha Please create the follow up tickets for performance & address
comment on FileSliceReader and I will approve this PR.
@vinothchandar Please take your pass on
n3nash commented on pull request #2263:
URL: https://github.com/apache/hudi/pull/2263#issuecomment-740424176
@satishkotha Please create the follow up tickets for performance and I will
approve this PR.
@vinothchandar Please take your pass on the PR
codecov-io edited a comment on pull request #2136:
URL: https://github.com/apache/hudi/pull/2136#issuecomment-729377388
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2136?src=pr=h1) Report
> Merging
[#2136](https://codecov.io/gh/apache/hudi/pull/2136?src=pr=desc) (87db045)
into
wosow edited a comment on issue #2282:
URL: https://github.com/apache/hudi/issues/2282#issuecomment-740408194
the entire folder
(hdfs://nameservice/data/wdt/sqoop/cow/inc/stockout_order_20201125) as follows
Permission | Owner | Group | Size | Last Modified | Replication | Block
wosow commented on issue #2282:
URL: https://github.com/apache/hudi/issues/2282#issuecomment-740408194
the entire folder
(hdfs://nameservice/data/wdt/sqoop/cow/inc/stockout_order_20201125) as follows
Permission | Owner | Group | Size | Last Modified | Replication | Block
codecov-io edited a comment on pull request #2136:
URL: https://github.com/apache/hudi/pull/2136#issuecomment-729377388
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2136?src=pr=h1) Report
> Merging
[#2136](https://codecov.io/gh/apache/hudi/pull/2136?src=pr=desc) (87db045)
into
[
https://issues.apache.org/jira/browse/HUDI-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1439:
-
Labels: pull-request-available (was: )
> Remove scala dependency from hudi-client-common
>
codecov-io edited a comment on pull request #2283:
URL: https://github.com/apache/hudi/pull/2283#issuecomment-734137301
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
codecov-io edited a comment on pull request #2283:
URL: https://github.com/apache/hudi/pull/2283#issuecomment-734137301
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2283?src=pr=h1) Report
> Merging
[#2283](https://codecov.io/gh/apache/hudi/pull/2283?src=pr=desc) (5324a66)
into
shenh062326 opened a new pull request #2306:
URL: https://github.com/apache/hudi/pull/2306
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of
vinothchandar opened a new pull request #2305:
URL: https://github.com/apache/hudi/pull/2305
## *Tips*
- *Thank you very much for contributing to Apache Hudi.*
- *Please review https://hudi.apache.org/contributing.html before opening a
pull request.*
## What is the purpose of
[
https://issues.apache.org/jira/browse/HUDI-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245674#comment-17245674
]
Vinoth Chandar commented on HUDI-1401:
--
>Given this, I believe to have this optimization we will
[
https://issues.apache.org/jira/browse/HUDI-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245674#comment-17245674
]
Vinoth Chandar edited comment on HUDI-1401 at 12/8/20, 4:56 AM:
>Given
[
https://issues.apache.org/jira/browse/HUDI-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenh062326 updated HUDI-1439:
--
Status: In Progress (was: Open)
> Remove scala dependency from hudi-client-common
>
[
https://issues.apache.org/jira/browse/HUDI-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shenh062326 updated HUDI-1439:
--
Status: Open (was: New)
> Remove scala dependency from hudi-client-common
>
codecov-io edited a comment on pull request #2136:
URL: https://github.com/apache/hudi/pull/2136#issuecomment-729377388
# [Codecov](https://codecov.io/gh/apache/hudi/pull/2136?src=pr=h1) Report
> Merging
[#2136](https://codecov.io/gh/apache/hudi/pull/2136?src=pr=desc) (a5d8490)
into
[
https://issues.apache.org/jira/browse/HUDI-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangxianghu updated HUDI-1439:
--
Summary: Remove scala dependency from hudi-client-common (was: Remove
scala denpendency from
[
https://issues.apache.org/jira/browse/HUDI-1439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangxianghu reassigned HUDI-1439:
-
Assignee: shenh062326
> Remove scala denpendency from hudi-client-common
>
wangxianghu created HUDI-1439:
-
Summary: Remove scala denpendency from hudi-client-common
Key: HUDI-1439
URL: https://issues.apache.org/jira/browse/HUDI-1439
Project: Apache Hudi
Issue Type:
[
https://issues.apache.org/jira/browse/HUDI-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangxianghu updated HUDI-1438:
--
Summary: Move DataSourceOptions to hudi-client-common to reuse more code
(was: Move DataSourceOptions
[
https://issues.apache.org/jira/browse/HUDI-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangxianghu updated HUDI-1438:
--
Issue Type: Improvement (was: Bug)
> Move DataSourceOptions to hudi-client-common to reuse code
>
wangxianghu created HUDI-1438:
-
Summary: Move DataSourceOptions to hudi-client-common to reuse code
Key: HUDI-1438
URL: https://issues.apache.org/jira/browse/HUDI-1438
Project: Apache Hudi
Issue
bithw1 opened a new issue #2304:
URL: https://github.com/apache/hudi/issues/2304
Hi,
I have following simple code that do upsert 100 times(The code is at the end
of the question description), and I disable the auto clean during writes. When
the writes is done, there are about 100
bithw1 commented on issue #2297:
URL: https://github.com/apache/hudi/issues/2297#issuecomment-740274172
Thanks @bvaradar, I am able to disable clean and run clean manually now.
This is an automated message from the Apache
bithw1 closed issue #2297:
URL: https://github.com/apache/hudi/issues/2297
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
bithw1 closed issue #2276:
URL: https://github.com/apache/hudi/issues/2276
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
bithw1 commented on issue #2276:
URL: https://github.com/apache/hudi/issues/2276#issuecomment-740273830
I am able to run compaction manually.
This is an automated message from the Apache Git Service.
To respond to the
bithw1 closed issue #2299:
URL: https://github.com/apache/hudi/issues/2299
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
n3nash commented on a change in pull request #2301:
URL: https://github.com/apache/hudi/pull/2301#discussion_r537926684
##
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/HoodieTable.java
##
@@ -460,7 +460,7 @@ protected void
vinothchandar commented on issue #2100:
URL: https://github.com/apache/hudi/issues/2100#issuecomment-740207609
@spyzzz thanks for letting us know. This will be in the next release
This is an automated message from the Apache
[
https://issues.apache.org/jira/browse/HUDI-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liwei reassigned HUDI-1437:
---
Assignee: liwei
> some description in spark ui is not reality, Not good for performance
> tracking
>
[
https://issues.apache.org/jira/browse/HUDI-1437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liwei updated HUDI-1437:
Description:
some spark action in hudi ,not set the real description, it is not good for
performance tracking
liwei created HUDI-1437:
---
Summary: some description in spark ui is not reality, Not good
for performance tracking
Key: HUDI-1437
URL: https://issues.apache.org/jira/browse/HUDI-1437
Project: Apache Hudi
trikota-kc commented on issue #2258:
URL: https://github.com/apache/hudi/issues/2258#issuecomment-74850
> I switched to new cluster and I am able to query the data now Thanks
@bhasudha
Would you mind sharing what setting worked for you? And maybe what was the
issue?
I'm
lucprosa opened a new issue #2303:
URL: https://github.com/apache/hudi/issues/2303
Hi all,
We're trying to use the HDFSParquetImporter to import data from parquet
(source) to Hudi (target) on S3 but we're facing some performance problems.
Here's important considerations about our
rahultoall opened a new issue #2302:
URL: https://github.com/apache/hudi/issues/2302
hi,
I am facing issue when i try to sync my hudi table to hive using the spark
DataSource Api.
Spark version - 2.4.7
spark-avro - spark-avro_2.11-2.4.7
hudi-spark -
bithw1 edited a comment on issue #2299:
URL: https://github.com/apache/hudi/issues/2299#issuecomment-739806481
Thanks @bvaradar .
What is the effect of `cleaning older file versions` ? Does this operation
also delete data?
Task the following operations as an example(15
bithw1 commented on issue #2299:
URL: https://github.com/apache/hudi/issues/2299#issuecomment-739806481
Thanks @bvaradar .
What does `cleaning older file versions` mean? Does this operation also
delete data?
Task the following operations as an example(15 commits and each
bvaradar commented on issue #2299:
URL: https://github.com/apache/hudi/issues/2299#issuecomment-739785836
If you want to retain all older version, you can turn off cleaning by
setting "hoodie.clean.automatic=false".
Note that not cleaning older file versions would result in
bithw1 commented on issue #2299:
URL: https://github.com/apache/hudi/issues/2299#issuecomment-739782716
Thanks @bvaradar.
In my case, I would keep all the data since the hoodie table is created.
Should I set `hoodie.cleaner.commits.retained` to a large value, eg,
Integer.MAX_VALUE,
bvaradar commented on issue #2297:
URL: https://github.com/apache/hudi/issues/2297#issuecomment-739777543
For (2), you need to set --hiveconf hive.stats.autogather=false when setting
up your hive session. Otherwise Hive uses its own stats to return the count
query instead of delegating to
bvaradar commented on issue #2299:
URL: https://github.com/apache/hudi/issues/2299#issuecomment-739773157
https://cwiki.apache.org/confluence/display/HUDI/FAQ#FAQ-WhatdoestheHudicleanerdo
Hudi Cleaner removes old file versions that are no longer required based on
the retention
bvaradar commented on issue #2298:
URL: https://github.com/apache/hudi/issues/2298#issuecomment-739766217
I have added a jira to do cleaning periodically
(https://issues.apache.org/jira/browse/HUDI-1436). But, yes you can disable
cleaner and run them through CLI
bvaradar closed issue #2298:
URL: https://github.com/apache/hudi/issues/2298
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
Balaji Varadarajan created HUDI-1436:
Summary: Provide Option to run auto clean every nth commit.
Key: HUDI-1436
URL: https://issues.apache.org/jira/browse/HUDI-1436
Project: Apache Hudi
[
https://issues.apache.org/jira/browse/HUDI-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan updated HUDI-1436:
-
Status: Open (was: New)
> Provide Option to run auto clean every nth commit.
>
[
https://issues.apache.org/jira/browse/HUDI-1436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan reassigned HUDI-1436:
Assignee: Sreeram Ramji
> Provide Option to run auto clean every nth commit.
>
bvaradar commented on issue #2285:
URL: https://github.com/apache/hudi/issues/2285#issuecomment-739754856
cc @umehrot2 : Wondering why there is parquet-hadoop-bundle-1.6.0.jar along
with parquet-hadoop-1.10.1-spark-amzn-1.jar. Wouldn't they cause conflict ?
[
https://issues.apache.org/jira/browse/HUDI-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan updated HUDI-1435:
-
Status: Patch Available (was: In Progress)
> Marker File Reconciliation failing for
[
https://issues.apache.org/jira/browse/HUDI-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan updated HUDI-1435:
-
Status: Open (was: New)
> Marker File Reconciliation failing for Non-Partitioned
[
https://issues.apache.org/jira/browse/HUDI-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan updated HUDI-1435:
-
Status: In Progress (was: Open)
> Marker File Reconciliation failing for Non-Partitioned
[
https://issues.apache.org/jira/browse/HUDI-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan updated HUDI-1435:
-
Status: In Progress (was: Open)
> Marker File Reconciliation failing for Non-Partitioned
bvaradar edited a comment on issue #2294:
URL: https://github.com/apache/hudi/issues/2294#issuecomment-739748464
@bhushanamk : Sorry for the delay. This is a bug and I have opened a PR :
https://github.com/apache/hudi/pull/2301 to fix this. It is an one-line change
and hopefully will fix
bvaradar commented on issue #2294:
URL: https://github.com/apache/hudi/issues/2294#issuecomment-739748464
@bhasudha : Sorry for the delay. This is a bug and I have opened a PR :
https://github.com/apache/hudi/pull/2301 to fix this. It is an one-line change
and hopefully will fix your
[
https://issues.apache.org/jira/browse/HUDI-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1435:
-
Labels: pull-request-available (was: )
> Marker File Reconciliation failing for Non-Partitioned
bvaradar opened a new pull request #2301:
URL: https://github.com/apache/hudi/pull/2301
GH Issue: https://github.com/apache/hudi/issues/2294
This is an automated message from the Apache Git Service.
To respond to the
[
https://issues.apache.org/jira/browse/HUDI-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan updated HUDI-1435:
-
Summary: Marker File Reconciliation failing for Non-Partitioned datasets
when duplicate
[
https://issues.apache.org/jira/browse/HUDI-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan reassigned HUDI-1435:
Assignee: Balaji Varadarajan
> Marker File Reconciliation failing for
Balaji Varadarajan created HUDI-1435:
Summary: Marker File Reconciliation failing for Non-Partitioned
Paths when duplicate marker files present
Key: HUDI-1435
URL:
59 matches
Mail list logo
