[
https://issues.apache.org/jira/browse/HUDI-970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prashant Wason updated HUDI-970:
Status: Open (was: New)
> HoodieTableFileSystem implementation to back API's using consolidated
[
https://issues.apache.org/jira/browse/HUDI-970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prashant Wason closed HUDI-970.
---
Resolution: Fixed
> HoodieTableFileSystem implementation to back API's using consolidated metadata
>
[
https://issues.apache.org/jira/browse/HUDI-969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prashant Wason closed HUDI-969.
---
Resolution: Invalid
> Implement compaction strategies for consolidated metadata table
>
[
https://issues.apache.org/jira/browse/HUDI-968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prashant Wason updated HUDI-968:
Status: Open (was: New)
> Creation of first base/snapshot metadata (similar to
[
https://issues.apache.org/jira/browse/HUDI-969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prashant Wason updated HUDI-969:
Status: Open (was: New)
> Implement compaction strategies for consolidated metadata table
>
[
https://issues.apache.org/jira/browse/HUDI-968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prashant Wason closed HUDI-968.
---
Resolution: Fixed
> Creation of first base/snapshot metadata (similar to onboarding/bootstrapping)
>
prashantwason edited a comment on pull request #2064:
URL: https://github.com/apache/hudi/pull/2064#issuecomment-686688968
Remaining work items:
- [x] 1. Support for rollbacks in MOR Table
- [x] 2. Rollback of metadata if commit eventually fails on dataset
- [x] 3. HUDI-CLI
Prashant Wason created HUDI-1312:
Summary: Query side use of Metadata Table
Key: HUDI-1312
URL: https://issues.apache.org/jira/browse/HUDI-1312
Project: Apache Hudi
Issue Type: New Feature
prashantwason edited a comment on pull request #2064:
URL: https://github.com/apache/hudi/pull/2064#issuecomment-686688968
Remaining work items:
- [x] 1. Support for rollbacks in MOR Table
- [x] 2. Rollback of metadata if commit eventually fails on dataset
- [x] 3. HUDI-CLI
bvaradar commented on issue #2130:
URL: https://github.com/apache/hudi/issues/2130#issuecomment-702062546
@Trevor-zhang : IIUC, this is not specific to Hudi. Did you get the full
exception stack trace from the server logs ? Are you sure this due to
hive.input.format setting ?
leesf commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702125051
Run quickstart demo: found the warn log:
`20/10/01 21:11:18 WARN embedded.EmbeddedTimelineService: Unable to find
driver bind address from spark config`, but not found the warn
Balaji Varadarajan created HUDI-1311:
Summary: Writes creating/updating large number of files seeing
errors when deleting marker files in S3
Key: HUDI-1311
URL: https://issues.apache.org/jira/browse/HUDI-1311
tandonraghavs edited a comment on issue #2131:
URL: https://github.com/apache/hudi/issues/2131#issuecomment-702082811
@bvaradar I am using my Custom class as `PAYLOAD_CLASS_OPT_KEY` key -> But
the problem is **preCombine** doesnt have reference to **Schema** and it is
giving me bytes , so
leesf edited a comment on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702125051
1. Run quickstart demo: found the warn log:
`20/10/01 21:11:18 WARN embedded.EmbeddedTimelineService: Unable to find
driver bind address from spark config`, but works
wangxianghu commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-701951988
> @wangxianghu @yanghua I have rebased this against master. Please take a
look at my changes.
>
> High level, we could re-use more code, but it needs an abstraction that
[
https://issues.apache.org/jira/browse/HUDI-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan reassigned HUDI-1308:
Assignee: Prashant Wason
> Issues found during testing RFC-15
>
[
https://issues.apache.org/jira/browse/HUDI-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan reassigned HUDI-1308:
Assignee: Balaji Varadarajan (was: Prashant Wason)
> Issues found during testing
[
https://issues.apache.org/jira/browse/HUDI-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205435#comment-17205435
]
Balaji Varadarajan commented on HUDI-1308:
--
cc [~vinoth]
> Issues found during testing RFC-15
>
yanghua commented on a change in pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#discussion_r498188986
##
File path: hudi-cli/pom.xml
##
@@ -148,7 +148,14 @@
org.apache.hudi
- hudi-client
+ hudi-client-common
+
yanghua commented on a change in pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#discussion_r498188986
##
File path: hudi-cli/pom.xml
##
@@ -148,7 +148,14 @@
org.apache.hudi
- hudi-client
+ hudi-client-common
+
Balaji Varadarajan created HUDI-1309:
Summary: Listing Metadata unreadable in S3 as the log block is
deemed corrupted
Key: HUDI-1309
URL: https://issues.apache.org/jira/browse/HUDI-1309
Project:
[
https://issues.apache.org/jira/browse/HUDI-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan reassigned HUDI-1309:
Assignee: Prashant Wason
> Listing Metadata unreadable in S3 as the log block is
bvaradar commented on issue #2131:
URL: https://github.com/apache/hudi/issues/2131#issuecomment-702072024
@tandonraghavs : For compaction, the payload class defined in
hoodie.properties is used for pre-combining. Can you check what is the payload
class configured in hoodie.properties ?
Balaji Varadarajan created HUDI-1308:
Summary: Issues found during testing RFC-15
Key: HUDI-1308
URL: https://issues.apache.org/jira/browse/HUDI-1308
Project: Apache Hudi
Issue Type:
[
https://issues.apache.org/jira/browse/HUDI-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balaji Varadarajan updated HUDI-1308:
-
Status: Open (was: New)
> Issues found during testing RFC-15
>
Balaji Varadarajan created HUDI-1310:
Summary: Corruption Block Handling too slow in S3
Key: HUDI-1310
URL: https://issues.apache.org/jira/browse/HUDI-1310
Project: Apache Hudi
Issue
tandonraghavs commented on issue #2131:
URL: https://github.com/apache/hudi/issues/2131#issuecomment-702082811
@bvaradar I am using my Custom class as `PAYLOAD_CLASS_OPT_KEY` key -> But
the problem is **preCombine** doesnt have reference to **Schema** and it is
giving me bytes , so how do
vinothchandar commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-701989092
@wangxianghu Please help test this out if possible. Once the tests pass
again, planning to merge this, morning PST
cc @yanghua
vinothchandar commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-701963298
I actually figured out that we can remove `P` altogether. since
`HoodieIndex#fetchRecordLocation` is not used much outside of internal APIs. So
will push a final change for
Karl-WangSK commented on a change in pull request #2106:
URL: https://github.com/apache/hudi/pull/2106#discussion_r498081682
##
File path:
hudi-client/src/main/java/org/apache/hudi/client/HoodieWriteClient.java
##
@@ -186,11 +186,15 @@ protected void
codecov-commenter commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-701949671
# [Codecov](https://codecov.io/gh/apache/hudi/pull/1827?src=pr=h1) Report
> Merging
[#1827](https://codecov.io/gh/apache/hudi/pull/1827?src=pr=desc) into
wangxianghu commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-70008
@vinothchandar @yanghua @leesf The demo runs well in my local, except the
warning `WARN embedded.EmbeddedTimelineService: Unable to find driver bind
address from spark config`
wangxianghu removed a comment on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702217863
> @leesf do you see the following exception? could not understand how you ll
get the other one even.
>
> ```
> LOG.info("Starting Timeline service !!");
>
wangxianghu commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702217863
> @leesf do you see the following exception? could not understand how you ll
get the other one even.
>
> ```
> LOG.info("Starting Timeline service !!");
>
n3nash commented on issue #2110:
URL: https://github.com/apache/hudi/issues/2110#issuecomment-702239474
@tooptoop4 No, the size of the existing table should not matter.
This is an automated message from the Apache Git
tandonraghavs edited a comment on issue #2131:
URL: https://github.com/apache/hudi/issues/2131#issuecomment-702082811
@bvaradar I am using my Custom class as `PAYLOAD_CLASS_OPT_KEY` key -> But
the problem is **preCombine** doesnt have reference to **Schema** and it is
giving me bytes , so
vinothchandar commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702174407
@leesf do you see the following exception? could not understand how you ll
get the other one even.
```
LOG.info("Starting Timeline service !!");
Option
vinothchandar commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702248781
@wangxianghu can you please test the latest commit. To be clear, you are
saying you don't get the warning on master, but get it on this branch. right?
if this round
satishkotha commented on pull request #2129:
URL: https://github.com/apache/hudi/pull/2129#issuecomment-702275562
@pratyakshsharma will you be able to review this week?
This is an automated message from the Apache Git
tandonraghavs edited a comment on issue #2131:
URL: https://github.com/apache/hudi/issues/2131#issuecomment-702082811
@bvaradar I am using my Custom class as `PAYLOAD_CLASS_OPT_KEY` key -> But
the problem is **preCombine** doesnt have reference to **Schema** and it is
giving me bytes , so
satishkotha commented on pull request #1929:
URL: https://github.com/apache/hudi/pull/1929#issuecomment-702404415
@leesf Any update? Let me know if you need any help here
This is an automated message from the Apache Git
satishkotha commented on pull request #2082:
URL: https://github.com/apache/hudi/pull/2082#issuecomment-702405034
@leesf #2048 is landed. is it possible to merge this and address Balaji's
comments? (I can help if needed)
vinothchandar commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702407275
@wangxianghu Just merged! Thanks again for the herculean effort.
May be some followups could pop up. Would you be interested in taking them
up? if so, I ll mention
vinothchandar merged pull request #1827:
URL: https://github.com/apache/hudi/pull/1827
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
[
https://issues.apache.org/jira/browse/HUDI-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205855#comment-17205855
]
Ryan Pifer commented on HUDI-1289:
--
[~vinoth] I was able to surface the issue. Seems codec package is
wangxianghu edited a comment on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702302083
> @wangxianghu can you please test the latest commit. To be clear, you are
saying you don't get the warning on master, but get it on this branch. right?
>
> if
vinothchandar commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702335492
@wangxianghu duh ofc. I understand now. Thanks for jumping in @wangxianghu !
This is an automated message
wangxianghu commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702420836
> @wangxianghu Just merged! Thanks again for the herculean effort.
>
> May be some followups could pop up. Would you be interested in taking them
up? if so, I ll mention
wangxianghu commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702308385
> > @wangxianghu can you please test the latest commit. To be clear, you are
saying you don't get the warning on master, but get it on this branch. right?
> > if this round
[
https://issues.apache.org/jira/browse/HUDI-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205751#comment-17205751
]
sivabalan narayanan commented on HUDI-89:
-
sorry, I was busy for the last few weeks. Here is my
[
https://issues.apache.org/jira/browse/HUDI-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205751#comment-17205751
]
sivabalan narayanan edited comment on HUDI-89 at 10/1/20, 6:41 PM:
---
sorry,
wangxianghu commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702302083
> @wangxianghu can you please test the latest commit. To be clear, you are
saying you don't get the warning on master, but get it on this branch. right?
>
> if this
wangxianghu edited a comment on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702302083
> @wangxianghu can you please test the latest commit. To be clear, you are
saying you don't get the warning on master, but get it on this branch. right?
>
> if
wangxianghu commented on pull request #1827:
URL: https://github.com/apache/hudi/pull/1827#issuecomment-702318528
@vinothchandar The warn log issue is fixed
This is an automated message from the Apache Git Service.
To
[
https://issues.apache.org/jira/browse/HUDI-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17205913#comment-17205913
]
Vinoth Chandar commented on HUDI-1289:
--
Great! Given how h base and guava are notorious for class
prashantwason edited a comment on pull request #2064:
URL: https://github.com/apache/hudi/pull/2064#issuecomment-686688968
Remaining work items:
- [x] 1. Support for rollbacks in MOR Table
- [ ] 2. Rollback of metadata if commit eventually fails on dataset
- [x] 3. HUDI-CLI
prashantwason commented on a change in pull request #2064:
URL: https://github.com/apache/hudi/pull/2064#discussion_r498587284
##
File path: hudi-client/src/main/resources/metadataSchema.txt
##
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
prashantwason commented on a change in pull request #2064:
URL: https://github.com/apache/hudi/pull/2064#discussion_r498587425
##
File path:
hudi-client/src/main/java/org/apache/hudi/metadata/HoodieMetadataPayload.java
##
@@ -0,0 +1,227 @@
+/*
+ * Licensed to the Apache
bschell commented on a change in pull request #2129:
URL: https://github.com/apache/hudi/pull/2129#discussion_r498571592
##
File path:
hudi-sync/hudi-dla-sync/src/main/java/org/apache/hudi/dla/DLASyncConfig.java
##
@@ -68,6 +68,9 @@
@Parameter(names = {"--help", "-h"},
leesf edited a comment on pull request #2082:
URL: https://github.com/apache/hudi/pull/2082#issuecomment-702494333
> @leesf #2048 is landed. is it possible to merge this and address Balaji's
comments? (I can help if needed)
Sure, considering I am a little busy these days, it is
leesf commented on pull request #2082:
URL: https://github.com/apache/hudi/pull/2082#issuecomment-702494333
> @leesf #2048 is landed. is it possible to merge this and address Balaji's
comments? (I can help if needed)
Sure, considering I am a little busy these days, it is wonderful
leesf commented on pull request #1929:
URL: https://github.com/apache/hudi/pull/1929#issuecomment-702500795
> @leesf Any update? Let me know if you need any help here
ack, will update the PR ASAP
This is an automated
prashantwason edited a comment on pull request #2064:
URL: https://github.com/apache/hudi/pull/2064#issuecomment-686688968
Remaining work items:
- [x] 1. Support for rollbacks in MOR Table
- [x] 2. Rollback of metadata if commit eventually fails on dataset
- [x] 3. HUDI-CLI
63 matches
Mail list logo
