[
https://issues.apache.org/jira/browse/TEZ-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14555089#comment-14555089
]
Siddharth Seth commented on TEZ-2450:
-------------------------------------
* HttpConnnection - connection, input, connectionSucceeded volatile. In
cleanup, connection and input should be set to null after they're cleaned up.
* HttpConnectionParams - Ideally should be final fields with a constructor used
to setup the params in the helper
* AsyncHttpConnection
** final fields where possible, if they're not unset
** initClient may not be safe. A partially constructed connection may be
returned if there's thread contention. See
http://en.wikipedia.org/wiki/Double-checked_locking
** //.setMaximumConnectionsTotal, //.setMaximumConnectionsPerHost commented
out. Should this be set to 1, numFetchers or some other value. Maybe temporary
configurable parameters while this is baked. There's an Exception later on
which refers to setting a parameter which isn't used on a
TooManyConnectionsException
* TezBodyDeferringAsyncHandler - Would be good to file a request with
AsyncClient so that this can be fixed in a later version and we can get rid of
the copy.
* Fetcher - catch(InterruptedException) return null - Does this need to check
the shutdown status ? Also, this likely needs to return a HostFetchResult so
that pending inputs are registered back with the ShuffleManager.
* FetcherOrderedGrouped - same as above. asyncHttp can be final
* ShuffleUtils - some commented out code which can be removed
* TestHttpConnection
** Tests should have a timeout
** NOT_USED_URL - should this be localhost instead ? Not sure what the current
url implies
** testHttpConnection - what is this verifying ?
** testHttpConnection - future.get() and then verify exceptions instead of just
a future.cancel(). A fail() within Worker won't fail the test - it has to be
propagated back to the main thread for it to have any affect.
** testAsyncHttpConnectionInterrupt - Replace the sleep(500) with a
future.get() and verify an InterruptException ?
* TestPipelinedShuffle
** setupTezCluster enables async by default - which means all tests run with
this ?
** last conf.set in baseTest doesn't do anything
** baseTest set async true even though it's already set (both tests will run as
async ?)
> support async http clients in ordered & unordered inputs
> --------------------------------------------------------
>
> Key: TEZ-2450
> URL: https://issues.apache.org/jira/browse/TEZ-2450
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Attachments: TEZ-2450.1.patch, TEZ-2450.2.WIP.patch,
> TEZ-2450.2.patch, TEZ-2450.WIP.patch
>
>
> It will be helpful to switch between JDK & other async http impls. For LLAP
> scenarios, it would be useful to make http clients interruptible which is
> supported in async libraries.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
