[
https://issues.apache.org/jira/browse/IMPALA-10242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214232#comment-17214232
]
Sahil Takiar commented on IMPALA-10242:
---------------------------------------
I'm actually not sure if this will work. Result spooling wasn't really designed
with this use case in mind, although result spooling does make this much easier
to implement. Result spooling is basically backed by a
{{buffered-tuple-stream.h}} which is the same object used for operator
spill-to-disk, I'm not sure if result spooling currently uses a
{{buffered-tuple-stream.h}} in a way that would support this (I think it should
by just pinning the whole stream in memory?), but Tim would know.
There are a few other considerations as well. I don't think we really support
this from a client perspective. To make fetch operations idempotent, Impala
would probably need to support a Fetch Orientation (e.g. TFetchOrientation)
beyond FETCH_NEXT and FETCH_FIRST. Support for something like FETCH_ABSOLUTE
might be necessary.
The issue is that fetch operations are just done using a simple iterator
interface (e.g. FETCH_NEXT). I don't think the impala-shell even tracks at
which point in the result set it has fetched rows up to. It just calls fetch
next in a loop until it returns hasMoreRows = false.
> impala-shell client retry for failed Fetch RPC calls.
> -----------------------------------------------------
>
> Key: IMPALA-10242
> URL: https://issues.apache.org/jira/browse/IMPALA-10242
> Project: IMPALA
> Issue Type: New Feature
> Reporter: Abhishek Rawat
> Priority: Major
>
> impala-shell client can retry failed idempotent rpcs. This work was done as
> part of IMPALA-9466.
> Since Impala also supports result spooling, the impala-shell client could
> also retry failed fetch rpc calls in some scenarios when result spooling is
> enabled.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
