[
https://issues.apache.org/jira/browse/TEZ-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated TEZ-2450:
----------------------------------
Attachment: TEZ-2450.2.WIP.patch
Posting the .2 WIP patch addressing review comments.
- Earlier patch was using getInputStream from netty, which internally cached
the downloaded contents. This wouldn't work well for larger payloads. Netty
internally has BodyDeferringAsyncHandler which is specifically used for
downloading larger payloads. Using BodyDeferringAsyncHandler in the latest
patch.
- Exceptions are propagated to the caller as
IOExceptions/InterruptedExceptions. Added a test case in TestFetcher for
setupConnection().
- When common threadpool is available for inputs, it should be passed on to
async client as well. However, currently I found some connect timeout issues
in enabling setAllowPoolingConnection(true) in async client at 10 TB scale.
Trying to debug this before posting the next patch for review.
> support async http clients in ordered & unordered inputs
> --------------------------------------------------------
>
> Key: TEZ-2450
> URL: https://issues.apache.org/jira/browse/TEZ-2450
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-2450.1.patch, TEZ-2450.2.WIP.patch,
> TEZ-2450.WIP.patch
>
>
> It will be helpful to switch between JDK & other async http impls. For LLAP
> scenarios, it would be useful to make http clients interruptible which is
> supported in async libraries.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
