[
https://issues.apache.org/jira/browse/DRILL-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881759#comment-15881759
]
ASF GitHub Bot commented on DRILL-5287:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/758#discussion_r102863394
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java ---
@@ -1010,7 +1010,9 @@ public void addToEventQueue(final QueryState
newState, final Exception exception
private void recordNewState(final QueryState newState) {
state = newState;
- queryManager.updateEphemeralState(newState);
+ if
(queryContext.getOptions().getOption(ExecConstants.ZK_QUERY_STATE_UPDATE)) {
+ queryManager.updateEphemeralState(newState);
+ }
--- End diff --
How does this affect query operation for long-running queries? How does it
impact the query profile? If updates are enabled, do we still do an update at
query completion to finalize the profile? If not, should writing of the profile
be automatically disabled if status updates are disabled?
Do we do any timeout on updates? Will we notice that the query has not been
updated and, say, kill the query due to timeouts?
> Provide option to skip updates of ephemeral state changes in Zookeeper
> ----------------------------------------------------------------------
>
> Key: DRILL-5287
> URL: https://issues.apache.org/jira/browse/DRILL-5287
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.9.0
> Reporter: Padma Penumarthy
> Assignee: Padma Penumarthy
> Fix For: 1.10
>
>
> We put transient profiles in zookeeper and update state as query progresses
> and changes states. It is observed that this adds latency of ~45msec for each
> update in the query execution path. This gets even worse when high number of
> concurrent queries are in progress. For concurrency=100, the average query
> response time even for short queries is 8 sec vs 0.2 sec with these updates
> disabled. For short lived queries in a high-throughput scenario, it is of no
> value to update state changes in zookeeper. We need an option to disable
> these updates for short running operational queries.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
