[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940892#comment-16940892
]
Vinoth Chandar commented on HUDI-269:
-
[~XingXPan] You should nt have to choose based on this..
> if
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939797#comment-16939797
]
Xing Pan commented on HUDI-269:
---
[~vinoth]
I thought it's because of that delta streamer too aggressive too,
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939647#comment-16939647
]
Vinoth Chandar commented on HUDI-269:
-
When you are writing your own spark app, specifying hoodie
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939642#comment-16939642
]
Vinoth Chandar commented on HUDI-269:
-
[~uditme] as fyi, in case you can quickly spot anything.. :)
>
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938543#comment-16938543
]
Balaji Varadarajan commented on HUDI-269:
-
cc [~vinoth]. We will look into this
> Provide ability
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938163#comment-16938163
]
Xing Pan commented on HUDI-269:
---
I tried to run the same hudi app via hudi spark datasource writer:
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937825#comment-16937825
]
Xing Pan commented on HUDI-269:
---
[~vinoth] , I'm planing to use hudi in our data lake project and happy to
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937760#comment-16937760
]
Vinoth Chandar commented on HUDI-269:
-
This is super useful [~XingXPan]. I am currently looking into
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937363#comment-16937363
]
Xing Pan commented on HUDI-269:
---
[~vbalaji]
ic, I was just trying to observe the request count change in a
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937360#comment-16937360
]
BALAJI VARADARAJAN commented on HUDI-269:
-
ok, looks like there is only one file and one partition
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937358#comment-16937358
]
Xing Pan commented on HUDI-269:
---
[~vbalaji]
yea, these strange 5K requests are mainly head requests, and
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937346#comment-16937346
]
BALAJI VARADARAJAN commented on HUDI-269:
-
[~XingXPan]: I cannot understand why we are seeing close
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937336#comment-16937336
]
Xing Pan commented on HUDI-269:
---
and I ran delta streamer like:
{code:java}
spark-submit --class
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937334#comment-16937334
]
Xing Pan commented on HUDI-269:
---
[~vbalaji] :
yeah, this is just a single test hudi app in my sandbox emr, so
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937330#comment-16937330
]
BALAJI VARADARAJAN commented on HUDI-269:
-
[~XingXPan] :
Thank you for sharing the S3 metrics
Can
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937302#comment-16937302
]
Xing Pan commented on HUDI-269:
---
[~vbalaji] [~vinoth]
I just did some simple test on these configs.
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16937296#comment-16937296
]
Xing Pan commented on HUDI-269:
---
!image-2019-09-25-08-51-19-686.png!
> Provide ability to throttle
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936933#comment-16936933
]
BALAJI VARADARAJAN commented on HUDI-269:
-
Agree. [~XingXPan] : You should try the embedded
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936830#comment-16936830
]
Vinoth Chandar commented on HUDI-269:
-
I am thinking if [~XingXPan] can try this out and confirm. we
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936827#comment-16936827
]
Vinoth Chandar commented on HUDI-269:
-
[~XingXPan] can you try turning on the embedded timeline service
[
https://issues.apache.org/jira/browse/HUDI-269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936542#comment-16936542
]
Xing Pan commented on HUDI-269:
---
just update pull request
> Provide ability to throttle DeltaStreamer sync
21 matches
Mail list logo
