[jira] [Commented] (TEZ-2022) java.lang.IllegalStateException: Vertex: got invalid start event
[
https://issues.apache.org/jira/browse/TEZ-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057565#comment-15057565
]
Jeff Zhang commented on TEZ-2022:
-
Move to 0.7.2
> java.lang.IllegalStateException: Vertex: got invalid start event
>
>
> Key: TEZ-2022
> URL: https://issues.apache.org/jira/browse/TEZ-2022
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Jeff Zhang
> Attachments: TEZ-2022-1.patch, TEZ-2022-DAG.png,
> TEZ-2022.applog.1.txt, TEZ-2022.applog.txt
>
>
> Latest Tez (with tez-2020 patch) + Pig
> When running pig with rank function, the following exception happens
> consistently
> {code}
> java.lang.IllegalStateException: Vertex: vertex_1422270854961_0113_1_03
> [scope-47] got invalid start event
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3178)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3170)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1547)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:181)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1768)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1754)
> at
> org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
> at
> org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:116)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> {code}
> rawLogs = load '/tmp/logs/root/logs/application_1422270854961_0093/' using
> org.apache.tez.tools.TFileLoader() as (machine:chararray, key:chararray,
> line:chararray);
> raw = FOREACH rawLogs GENERATE TRIM(REGEX_EXTRACT(machine, '(.*)_(\\d+)', 1))
> as machine, key, line;
> machines = FOREACH raw GENERATE machine;
> distinctMachines = DISTINCT machines;
> sortByMachines = ORDER distinctMachines BY machine;
> ranked = RANK sortByMachines;
> dump ranked;
> {code}
> Will attach the DAG and applog asap.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2022) java.lang.IllegalStateException: Vertex: got invalid start event
[
https://issues.apache.org/jira/browse/TEZ-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044410#comment-15044410
]
Jeff Zhang commented on TEZ-2022:
-
[~hitesh] Mind to help review it ?
> java.lang.IllegalStateException: Vertex: got invalid start event
>
>
> Key: TEZ-2022
> URL: https://issues.apache.org/jira/browse/TEZ-2022
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Jeff Zhang
> Attachments: TEZ-2022-1.patch, TEZ-2022-DAG.png,
> TEZ-2022.applog.1.txt, TEZ-2022.applog.txt
>
>
> Latest Tez (with tez-2020 patch) + Pig
> When running pig with rank function, the following exception happens
> consistently
> {code}
> java.lang.IllegalStateException: Vertex: vertex_1422270854961_0113_1_03
> [scope-47] got invalid start event
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3178)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3170)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1547)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:181)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1768)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1754)
> at
> org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
> at
> org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:116)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> {code}
> rawLogs = load '/tmp/logs/root/logs/application_1422270854961_0093/' using
> org.apache.tez.tools.TFileLoader() as (machine:chararray, key:chararray,
> line:chararray);
> raw = FOREACH rawLogs GENERATE TRIM(REGEX_EXTRACT(machine, '(.*)_(\\d+)', 1))
> as machine, key, line;
> machines = FOREACH raw GENERATE machine;
> distinctMachines = DISTINCT machines;
> sortByMachines = ORDER distinctMachines BY machine;
> ranked = RANK sortByMachines;
> dump ranked;
> {code}
> Will attach the DAG and applog asap.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2022) java.lang.IllegalStateException: Vertex: got invalid start event
[
https://issues.apache.org/jira/browse/TEZ-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025111#comment-15025111
]
Hitesh Shah commented on TEZ-2022:
--
Incorrect patch uploaded?
> java.lang.IllegalStateException: Vertex: got invalid start event
>
>
> Key: TEZ-2022
> URL: https://issues.apache.org/jira/browse/TEZ-2022
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Jeff Zhang
> Attachments: TEZ-2022-DAG.png, TEZ-2022.applog.1.txt,
> TEZ-2022.applog.txt, TEZ--1.patch
>
>
> Latest Tez (with tez-2020 patch) + Pig
> When running pig with rank function, the following exception happens
> consistently
> {code}
> java.lang.IllegalStateException: Vertex: vertex_1422270854961_0113_1_03
> [scope-47] got invalid start event
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3178)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3170)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1547)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:181)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1768)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1754)
> at
> org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
> at
> org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:116)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> {code}
> rawLogs = load '/tmp/logs/root/logs/application_1422270854961_0093/' using
> org.apache.tez.tools.TFileLoader() as (machine:chararray, key:chararray,
> line:chararray);
> raw = FOREACH rawLogs GENERATE TRIM(REGEX_EXTRACT(machine, '(.*)_(\\d+)', 1))
> as machine, key, line;
> machines = FOREACH raw GENERATE machine;
> distinctMachines = DISTINCT machines;
> sortByMachines = ORDER distinctMachines BY machine;
> ranked = RANK sortByMachines;
> dump ranked;
> {code}
> Will attach the DAG and applog asap.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2022) java.lang.IllegalStateException: Vertex: got invalid start event
[
https://issues.apache.org/jira/browse/TEZ-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15024208#comment-15024208
]
Jeff Zhang commented on TEZ-2022:
-
Upload one patch to fix it. [~hitesh] Please help review it.
The root cause of this issue is that: the predication in
StartWhileInitializingTransition will also happens in recovery.
{code}
// vertex state machine does not start itself in the initializing state
// this start event can only come directly from the DAG. That means this
// is a top level vertex of the dag
Preconditions.checkState(
(vertex.sourceVertices == null || vertex.sourceVertices.isEmpty()),
"Vertex: " + vertex.logIdentifier + " got invalid start event");
{code}
Here's the code cause this scenario in recovery. RecoveryTransition in
VertexImpl.java
{code}
case NEW:
// Drop all root events if not inited properly
Iterator iterator = vertex.recoveredEvents.iterator();
while (iterator.hasNext()) {
if (iterator.next().getEventType().equals(
EventType.ROOT_INPUT_DATA_INFORMATION_EVENT)) {
iterator.remove();
}
}
// Trigger init if all sources initialized
if (vertex.numInitedSourceVertices == vertex.getInputVerticesCount())
{
vertex.eventHandler.handle(new VertexEvent(vertex.vertexId,
VertexEventType.V_INIT));
}
if (vertex.numStartedSourceVertices ==
vertex.getInputVerticesCount()) {
vertex.eventHandler.handle(new VertexEvent(vertex.vertexId,
VertexEventType.V_START));
}
endState = VertexState.NEW;
{code}
> java.lang.IllegalStateException: Vertex: got invalid start event
>
>
> Key: TEZ-2022
> URL: https://issues.apache.org/jira/browse/TEZ-2022
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Jeff Zhang
> Attachments: TEZ-2022-DAG.png, TEZ-2022.applog.1.txt,
> TEZ-2022.applog.txt, TEZ--1.patch
>
>
> Latest Tez (with tez-2020 patch) + Pig
> When running pig with rank function, the following exception happens
> consistently
> {code}
> java.lang.IllegalStateException: Vertex: vertex_1422270854961_0113_1_03
> [scope-47] got invalid start event
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:145)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3178)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3170)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at
> org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1547)
> at
> org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:181)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1768)
> at
> org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1754)
> at
> org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
> at
> org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:116)
> at java.lang.Thread.run(Thread.java:744)
> {code}
> {code}
> rawLogs = load '/tmp/logs/root/logs/application_1422270854961_0093/' using
> org.apache.tez.tools.TFileLoader() as (machine:chararray, key:chararray,
> line:chararray);
> raw = FOREACH rawLogs GENERATE TRIM(REGEX_EXTRACT(machine, '(.*)_(\\d+)', 1))
> as machine, key, line;
> machines = FOREACH raw GENERATE machine;
> distinctMachines = DISTINCT machines;
> sortByMachines = ORDER distinctMachines BY machine;
> ranked = RANK sortByMachines;
> dump ranked;
> {code}
> Will attach the DAG and applog asap.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2022) java.lang.IllegalStateException: Vertex: got invalid start event
[
https://issues.apache.org/jira/browse/TEZ-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302304#comment-14302304
]
Hitesh Shah commented on TEZ-2022:
--
Seems like multiple bugs, one of which would be a variation of TEZ-1893 where a
user error is not caught and causes the AM to crash.
java.lang.IllegalStateException: Vertex: got invalid start event
Key: TEZ-2022
URL: https://issues.apache.org/jira/browse/TEZ-2022
Project: Apache Tez
Issue Type: Bug
Reporter: Rajesh Balamohan
Attachments: TEZ-2022-DAG.png, TEZ-2022.applog.1.txt,
TEZ-2022.applog.txt
Latest Tez (with tez-2020 patch) + Pig
When running pig with rank function, the following exception happens
consistently
{code}
java.lang.IllegalStateException: Vertex: vertex_1422270854961_0113_1_03
[scope-47] got invalid start event
at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at
org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3178)
at
org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3170)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at
org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
at
org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1547)
at
org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:181)
at
org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1768)
at
org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1754)
at
org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
at
org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:116)
at java.lang.Thread.run(Thread.java:744)
{code}
{code}
rawLogs = load '/tmp/logs/root/logs/application_1422270854961_0093/' using
org.apache.tez.tools.TFileLoader() as (machine:chararray, key:chararray,
line:chararray);
raw = FOREACH rawLogs GENERATE TRIM(REGEX_EXTRACT(machine, '(.*)_(\\d+)', 1))
as machine, key, line;
machines = FOREACH raw GENERATE machine;
distinctMachines = DISTINCT machines;
sortByMachines = ORDER distinctMachines BY machine;
ranked = RANK sortByMachines;
dump ranked;
{code}
Will attach the DAG and applog asap.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2022) java.lang.IllegalStateException: Vertex: got invalid start event
[
https://issues.apache.org/jira/browse/TEZ-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14302314#comment-14302314
]
Bikas Saha commented on TEZ-2022:
-
Lets keep the jira to remove the invalid precondition.
java.lang.IllegalStateException: Vertex: got invalid start event
Key: TEZ-2022
URL: https://issues.apache.org/jira/browse/TEZ-2022
Project: Apache Tez
Issue Type: Bug
Reporter: Rajesh Balamohan
Attachments: TEZ-2022-DAG.png, TEZ-2022.applog.1.txt,
TEZ-2022.applog.txt
Latest Tez (with tez-2020 patch) + Pig
When running pig with rank function, the following exception happens
consistently
{code}
java.lang.IllegalStateException: Vertex: vertex_1422270854961_0113_1_03
[scope-47] got invalid start event
at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at
org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3178)
at
org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3170)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at
org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
at
org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1547)
at
org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:181)
at
org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1768)
at
org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1754)
at
org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
at
org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:116)
at java.lang.Thread.run(Thread.java:744)
{code}
{code}
rawLogs = load '/tmp/logs/root/logs/application_1422270854961_0093/' using
org.apache.tez.tools.TFileLoader() as (machine:chararray, key:chararray,
line:chararray);
raw = FOREACH rawLogs GENERATE TRIM(REGEX_EXTRACT(machine, '(.*)_(\\d+)', 1))
as machine, key, line;
machines = FOREACH raw GENERATE machine;
distinctMachines = DISTINCT machines;
sortByMachines = ORDER distinctMachines BY machine;
ranked = RANK sortByMachines;
dump ranked;
{code}
Will attach the DAG and applog asap.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-2022) java.lang.IllegalStateException: Vertex: got invalid start event
[
https://issues.apache.org/jira/browse/TEZ-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14301713#comment-14301713
]
Bikas Saha commented on TEZ-2022:
-
This Precondition is not valid. A vertex can be in initializing state if the
edges are not defined or the parallelism is not set. This can be at any vertex
in the graph and not necessarily a root vertex reading primary inputs. Even a
root vertex can have another vertex as a source.
For this specific case, it seems like recovery has kicked in for the attached
app log and the vertex managed to get a start event before recovery code moved
the vertex out of initializing. Are the first attempt logs available?
java.lang.IllegalStateException: Vertex: got invalid start event
Key: TEZ-2022
URL: https://issues.apache.org/jira/browse/TEZ-2022
Project: Apache Tez
Issue Type: Bug
Reporter: Rajesh Balamohan
Attachments: TEZ-2022-DAG.png, TEZ-2022.applog.txt
Latest Tez (with tez-2020 patch) + Pig
When running pig with rank function, the following exception happens
consistently
{code}
java.lang.IllegalStateException: Vertex: vertex_1422270854961_0113_1_03
[scope-47] got invalid start event
at
com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at
org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3178)
at
org.apache.tez.dag.app.dag.impl.VertexImpl$StartWhileInitializingTransition.transition(VertexImpl.java:3170)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at
org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
at
org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1547)
at
org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:181)
at
org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1768)
at
org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1754)
at
org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:184)
at
org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:116)
at java.lang.Thread.run(Thread.java:744)
{code}
{code}
rawLogs = load '/tmp/logs/root/logs/application_1422270854961_0093/' using
org.apache.tez.tools.TFileLoader() as (machine:chararray, key:chararray,
line:chararray);
raw = FOREACH rawLogs GENERATE TRIM(REGEX_EXTRACT(machine, '(.*)_(\\d+)', 1))
as machine, key, line;
machines = FOREACH raw GENERATE machine;
distinctMachines = DISTINCT machines;
sortByMachines = ORDER distinctMachines BY machine;
ranked = RANK sortByMachines;
dump ranked;
{code}
Will attach the DAG and applog asap.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
