yuzelin commented on code in PR #21133:
URL: https://github.com/apache/flink/pull/21133#discussion_r1024355066
##########
flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/service/SqlGatewayServiceImpl.java:
##########
@@ -311,8 +340,29 @@ public GatewayInfo getGatewayInfo() {
return GatewayInfo.INSTANCE;
}
+ //
--------------------------------------------------------------------------------------------
+
@VisibleForTesting
public Session getSession(SessionHandle sessionHandle) {
return sessionManager.getSession(sessionHandle);
}
+
+ /** Fetch all results for configuring session. */
+ private ResultSet fetchConfigureSessionResult(
+ SessionHandle sessionHandle, OperationHandle operationHandle) {
+ ResultSet firstResult = fetchResults(sessionHandle, operationHandle,
0, Integer.MAX_VALUE);
+ while (firstResult == ResultSet.NOT_READY_RESULTS) {
+ firstResult = fetchResults(sessionHandle, operationHandle, 0,
Integer.MAX_VALUE);
Review Comment:
> Let me ask this question: If one of the operations on the gateway took
some time, say, 5 seconds. How many calls to the gateway would we expect to
make here while waiting for the results to be ready?
I'm afraid that I have no answer to this question. I guess you are worried
about frequent queries will make it a burden to the execution. However, I think
it's not a problem. Let's see how the codes perform the fetching.
Fetching result will finally go to
[fetchResultsInternal](https://github.com/apache/flink/blob/5b32b9defcbb2adf7a0ad0898a67c40e0e012ebb/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/service/operation/OperationManager.java#L331),
here if the operation is not finished, the result will always be `NOT_READY`.
At line 239, we can see the `ResultFetcher` is ready before the status is
turned to finished, and the statement is executed [(in an
`OperationExecutor`)](https://github.com/apache/flink/blob/5b32b9defcbb2adf7a0ad0898a67c40e0e012ebb/flink-table/flink-sql-gateway/src/main/java/org/apache/flink/table/gateway/service/SqlGatewayServiceImpl.java#L178).
Although we still have to get data through `ResultFetcher`, at least the
execution is not bothered. Then the task of fetching the rest results is taken
by `ResultStore`, a
[thread](https://github.com/apache/flink/blob/5b32b9defcbb2adf7a0ad0898a67c40e0e012ebb/flink-table/flink-sql-gateway/src/main/java/org/ap
ache/flink/table/gateway/service/result/ResultStore.java#L149) is fetching
continuously based on the
[`TableResultInternal#collectInternal`](https://github.com/apache/flink/blob/5b32b9defcbb2adf7a0ad0898a67c40e0e012ebb/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableResultInternal.java#L45).
Anyway, the first result is important. The
[implementation](https://github.com/apache/flink/blob/5b32b9defcbb2adf7a0ad0898a67c40e0e012ebb/flink-table/flink-table-api-java/src/main/java/org/apache/flink/table/api/internal/TableResultImpl.java#L105)
of `TableResult#await` is the proof. After getting the first result, we can
control the interval of fetching rest results. For example, in `CliResultView`
of SQL client for print query results, it has set several `REFRESH_INTERVALS`.
However, I see that in remote mode, the cost of network communications in
fetching the first ready result may be large. I think it's worth discussing
when implementing the remote mode. I will also ask @fsk119 for advice.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
