[
https://issues.apache.org/jira/browse/TEZ-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511820#comment-16511820
]
ASF GitHub Bot commented on TEZ-3904:
-------------------------------------
Github user bjmb commented on a diff in the pull request:
https://github.com/apache/tez/pull/22#discussion_r195270418
--- Diff:
tez-runtime-internals/src/main/java/org/apache/tez/runtime/LogicalIOProcessorRuntimeTask.java
---
@@ -160,14 +161,15 @@
private final boolean initializeProcessorFirst;
private final boolean initializeProcessorIOSerially;
private final TezExecutors sharedExecutor;
+ private final SystemEventHandler systemEventHandler;
public LogicalIOProcessorRuntimeTask(TaskSpec taskSpec, int
appAttemptNumber,
Configuration tezConf, String[] localDirs, TezUmbilical tezUmbilical,
Map<String, ByteBuffer> serviceConsumerMetadata, Map<String, String>
envMap,
Multimap<String, String> startedInputsMap, ObjectRegistry
objectRegistry,
String pid, ExecutionContext ExecutionContext, long memAvailable,
boolean updateSysCounters, HadoopShim hadoopShim,
- TezExecutors sharedExecutor) throws IOException {
+ TezExecutors sharedExecutor, SystemEventHandler systemEventHandler)
throws IOException {
--- End diff --
This API changes, this is bad?
> an API to update tokens for Tez AM and the DAG
> ----------------------------------------------
>
> Key: TEZ-3904
> URL: https://issues.apache.org/jira/browse/TEZ-3904
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Sergey Shelukhin
> Priority: Major
>
> Nothing is permanent in this world, lest of all delegation tokens.
> The current way around token expiration (the one where you cannot keep
> renewing anymore) in Hive when Tez AM is used in session mode is to cycle Tez
> AM. It may happen though that a query is running at that time, and so the AM
> cannot be restarted with new tokens. We let the query run its course and it
> usually dies because it tries to do something with an expired token.
> To get around that, we cycle AMs a few hours before tokens are going to
> expire.
> However, that is still not ideal because it puts an upper bound on safe Hive
> query runtime (a query longer than 3 hours with current config may fail due
> to an expired token if its timing is unlucky), and also precludes setting
> tokens to expire much faster than the standard 7-day time frame.
> There should be a mechanism to replace tokens in the AM, including for a
> running DAG.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
