[
https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-376:
--
Attachment: (was: Collibra-DGC-572-Administration-Guide.pdf)
> AWS Glue dependency issue for EMR 5.28.0
>
[
https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-376:
--
Attachment: (was: Collibra-DGC-572-Administration-Guide.pdf)
> AWS Glue dependency issue for EMR 5.28.0
>
[
https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-376:
--
Attachment: Collibra-DGC-572-Administration-Guide.pdf
> AWS Glue dependency issue for EMR 5.28.0
>
[
https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-376:
--
Attachment: Collibra-DGC-572-Administration-Guide.pdf
> AWS Glue dependency issue for EMR 5.28.0
>
[
https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-376:
--
Attachment: (was: Collibra-DGC-572-Administration-Guide.pdf)
> AWS Glue dependency issue for EMR 5.28.0
>
[
https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-376:
--
Attachment: Collibra-DGC-572-Administration-Guide.pdf
> AWS Glue dependency issue for EMR 5.28.0
>
[
https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17008528#comment-17008528
]
Xing Pan commented on HUDI-376:
---
PR: [https://github.com/apache/incubator-hudi/pull/1189]
[~xleesf]
> AWS
[
https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17008482#comment-17008482
]
Xing Pan commented on HUDI-376:
---
[~xleesf] sorry for the delay of response.
I'd like to send a PR, but I
[
https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-376:
--
Description:
Hi hudi team, it's really encouraging that Hudi is finally officially supported
application on AWS
[
https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-376:
--
Issue Type: Improvement (was: Bug)
> AWS Glue dependency issue for EMR 5.28.0
>
[
https://issues.apache.org/jira/browse/HUDI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-376:
--
Description:
Hi hudi team, it's really encouraging that Hudi is finally officially supported
application on AWS
Xing Pan created HUDI-376:
-
Summary: AWS Glue dependency issue for EMR 5.28.0
Key: HUDI-376
URL: https://issues.apache.org/jira/browse/HUDI-376
Project: Apache Hudi (incubating)
Issue Type: Bug
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939797#comment-16939797
]
Xing Pan edited comment on HUDI-269 at 9/28/19 12:23 AM:
-
[~vinoth]
I thought it's
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939797#comment-16939797
]
Xing Pan commented on HUDI-269:
---
[~vinoth]
I thought it's because of that delta streamer too aggressive too,
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938163#comment-16938163
]
Xing Pan commented on HUDI-269:
---
I tried to run the same hudi app via hudi spark datasource writer:
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-269:
--
Attachment: image-2019-09-26-09-02-24-761.png
> Provide ability to throttle DeltaStreamer sync runs
>
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937825#comment-16937825
]
Xing Pan commented on HUDI-269:
---
[~vinoth] , I'm planing to use hudi in our data lake project and happy to
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937358#comment-16937358
]
Xing Pan edited comment on HUDI-269 at 9/25/19 2:56 PM:
[~vbalaji]
yea, these
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937363#comment-16937363
]
Xing Pan commented on HUDI-269:
---
[~vbalaji]
ic, I was just trying to observe the request count change in a
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937358#comment-16937358
]
Xing Pan commented on HUDI-269:
---
[~vbalaji]
yea, these strange 5K requests are mainly head requests, and
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-269:
--
Attachment: hudi_request_test.tar.gz
> Provide ability to throttle DeltaStreamer sync runs
>
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937336#comment-16937336
]
Xing Pan commented on HUDI-269:
---
and I ran delta streamer like:
{code:java}
spark-submit --class
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937334#comment-16937334
]
Xing Pan commented on HUDI-269:
---
[~vbalaji] :
yeah, this is just a single test hudi app in my sandbox emr, so
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937302#comment-16937302
]
Xing Pan edited comment on HUDI-269 at 9/25/19 1:05 AM:
[~vbalaji] [~vinoth]
I
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-269:
--
Attachment: (was: request_histogram.png)
> Provide ability to throttle DeltaStreamer sync runs
>
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937302#comment-16937302
]
Xing Pan commented on HUDI-269:
---
[~vbalaji] [~vinoth]
I just did some simple test on these configs.
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937296#comment-16937296
]
Xing Pan commented on HUDI-269:
---
!image-2019-09-25-08-51-19-686.png!
> Provide ability to throttle
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-269:
--
Attachment: image-2019-09-25-08-51-19-686.png
> Provide ability to throttle DeltaStreamer sync runs
>
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xing Pan updated HUDI-269:
--
Attachment: request_histogram.png
> Provide ability to throttle DeltaStreamer sync runs
>
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936542#comment-16936542
]
Xing Pan edited comment on HUDI-269 at 9/24/19 8:27 AM:
just updated pull request
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936542#comment-16936542
]
Xing Pan commented on HUDI-269:
---
just update pull request
> Provide ability to throttle DeltaStreamer sync
31 matches
Mail list logo
