> On Sept. 30, 2017, 4:41 a.m., Robert Kanter wrote:
> > tools/src/main/java/org/apache/oozie/tools/diag/AppInfoCollector.java
> > Lines 151 (patched)
> > <https://reviews.apache.org/r/62459/diff/7/?file=1837523#file1837523line151>
> >
> > Should we skip control types? That's probably fine for the start
> > action, but what about a decision action? That has some useful info.
> > Maybe we should still print these out, but with fewer fields or something
> > (e.g. no need for a console url).
>
> Robert Kanter wrote:
> Thinking about this more, I think we should include control types. It
> will be helpful for following the flow of a workflow when there's decision
> and fork nodes. Please update the patch to handle this. We don't need to
> print out all of the fields, so you'll have to figure out which ones make
> sense.
>
> Attila Sasvari wrote:
> Sure, I will do so.
>
> Attila Sasvari wrote:
> I believe only the decision node might interesting to see execution path,
> but I am not sure. ``info.txt`` already contains execution / transition info:
> ```
> Action Id : "0000004-171015215418832-oozie-asas-W@ctrl"
> Name : "ctrl"
> Type : "switch"
> Status : "OK"
> Transition : "forking"
> Start Time : "2017-10-15 20:20:36 GMT"
> End Time : "2017-10-15 20:20:36 GMT"
> Error Code : null
> Error Message : null
> Console URL : "-"
> Tracker URI : "-"
> External Child Ids : null
> External Id : "-"
> External Status : "forking"
> Data : null
> Stats : null
> Credentials : null
> Retries : "0"
> User Retry Int : "10"
> User Retry Count : "0"
> User Retry Max : "0"
> ```
>
> If I store resolved action defintion, it would not add too much
> information:
> ```
> # cat 0000004-171015215418832-oozie-asas-W/resolved-actions/ctrl.xml
> <switch xmlns="uri:oozie:workflow:0.2">
> <case to="forking">true</case>
> <default to="end" />
> </switch>%
> ```
>
> I created a workflow with a fork node and also with a decision node. I
> saw that resolved action config xml were empty for control nodes other than
> the decision node.
>
> ```
> 0 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/:start:.xml
> 108 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/ctrl.xml
> 0 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/end.xml
> 0 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/forking.xml
> 0 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/join.xml
> 83834 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/launcher_mr-node.log
> 83843 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/launcher_mr-node2.log
> 47965 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/launcher_shell1.log
> 47965 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/launcher_shell2.log
> 2045 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/mr-node.xml
> 2045 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/mr-node2.xml
> 544 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/shell1.xml
> 544 10-15-17 22:44
> /0000004-171015215418832-oozie-asas-W/resolved-actions/shell2.xml
> ```
> Zero byte xml-s are not interesting. I will remove non-decision action
> types.
>
> Other thing: it would also good to add later an option to download the
> final job graph (thanks to OOZIE-2406 now we can download it as a Dot file
> too). If they have many action, it might help a lot when taking a quick look.
>
> Misc: Would it better to have a different pattern for the filename? For
> example: it might be good to see where the bundle was actually taken (i.e.
> include fqdn-hostname, there are existing diagnostic tools that use similar
> pattern).
update: I talked with Andras Piros and gezapeti and decided to include control
nodes. It makes it easy to see whether the workflow failed or not (you don't
need to open the info.txt).
```
Archive:
/Users/asasvari/workspace/apache/oozie_dup/distro/target/oozie-5.0.0-SNAPSHOT-distro/oozie-5.0.0-SNAPSHOT/./oozie-diag-bundle-1508156773346.zip
Length Date Time Name
-------- ---- ---- ----
7843 10-16-17 14:26 /0000006-171015215418832-oozie-asas-W/info.txt
322 10-16-17 14:26 /0000006-171015215418832-oozie-asas-W/job.properties
20878 10-16-17 14:26 /0000006-171015215418832-oozie-asas-W/log.txt
0 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/
0 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/:start:.xml
108 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/ctrl.xml
0 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/end.xml
0 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/forking.xml
0 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/join.xml
83827 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/launcher_mr-node.log
83836 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/launcher_mr-node2.log
47956 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/launcher_shell1.log
47956 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/launcher_shell2.log
2045 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/mr-node.xml
2045 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/mr-node2.xml
544 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/shell1.xml
543 10-16-17 14:26
/0000006-171015215418832-oozie-asas-W/resolved-actions/shell2.xml
5495 10-16-17 14:26 /0000006-171015215418832-oozie-asas-W/workflow.xml
7268 10-16-17 14:26 /0000007-171015215418832-oozie-asas-W/info.txt
322 10-16-17 14:26 /0000007-171015215418832-oozie-asas-W/job.properties
22429 10-16-17 14:26 /0000007-171015215418832-oozie-asas-W/log.txt
0 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/
0 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/:start:.xml
108 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/ctrl.xml
72 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/fail.xml
0 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/forking.xml
83827 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/launcher_mr-node.log
83836 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/launcher_mr-node2.log
47957 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/launcher_shell1.log
51332 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/launcher_shell2.log
2045 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/mr-node.xml
2045 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/mr-node2.xml
544 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/shell1.xml
543 10-16-17 14:26
/0000007-171015215418832-oozie-asas-W/resolved-actions/shell2.xml
5495 10-16-17 14:26 /0000007-171015215418832-oozie-asas-W/workflow.xml
64567 10-16-17 14:26 /effective-oozie-site.xml
32946 10-16-17 14:26 /instrumentation.txt
32712 10-16-17 14:26 /java-sys-props.txt
3643 10-16-17 14:26 /os-env-vars.txt
279 10-16-17 14:26 /queue-dump.txt
38906 10-16-17 14:26 /sharelib.txt
103433 10-16-17 14:26 /thread-dump.html
```
- Attila
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62459/#review186754
-----------------------------------------------------------
On Oct. 16, 2017, 7:11 a.m., Attila Sasvari wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62459/
> -----------------------------------------------------------
>
> (Updated Oct. 16, 2017, 7:11 a.m.)
>
>
> Review request for oozie.
>
>
> Repository: oozie-git
>
>
> Description
> -------
>
> A diagnostic tool that collects a bunch of job and other information from
> Oozie in a zip file.
>
>
> Diffs
> -----
>
> docs/src/site/twiki/DG_CommandLineTool.twiki
> d4047671876dcc3279a2ec379bc1d003f5e6f1aa
> pom.xml be153c16c45f91a8d57c78d6b7ebe073e7ca1f79
> tools/pom.xml 7306a14e7b237977be00f8fe28e34573540fd508
> tools/src/main/bin/oozie-diag-bundle-collector.sh PRE-CREATION
> tools/src/main/java/org/apache/oozie/tools/diag/AppInfoCollector.java
> PRE-CREATION
> tools/src/main/java/org/apache/oozie/tools/diag/ArgParser.java PRE-CREATION
>
> tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleCollectorDriver.java
> PRE-CREATION
> tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleCompressor.java
> PRE-CREATION
> tools/src/main/java/org/apache/oozie/tools/diag/DiagBundleEntryWriter.java
> PRE-CREATION
> tools/src/main/java/org/apache/oozie/tools/diag/DiagOozieClient.java
> PRE-CREATION
> tools/src/main/java/org/apache/oozie/tools/diag/MetricsCollector.java
> PRE-CREATION
>
> tools/src/main/java/org/apache/oozie/tools/diag/OozieLauncherLogFetcher.java
> PRE-CREATION
> tools/src/main/java/org/apache/oozie/tools/diag/ServerInfoCollector.java
> PRE-CREATION
> tools/src/test/java/org/apache/oozie/tools/diag/TestAppInfoCollector.java
> PRE-CREATION
> tools/src/test/java/org/apache/oozie/tools/diag/TestArgParser.java
> PRE-CREATION
> tools/src/test/java/org/apache/oozie/tools/diag/TestMetricsCollector.java
> PRE-CREATION
>
> tools/src/test/java/org/apache/oozie/tools/diag/TestServerInfoCollector.java
> PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/62459/diff/11/
>
>
> Testing
> -------
>
> - new unit tests: TestOozieDiagBundleCollector
> - started Oozie with a pseudo hadoop cluster, submitted a couple workflows,
> and executed the following commands:
> -- ``bin/oozie-diag-bundle-collector.sh`` (usage info printed),
> -- ``bin/oozie-diag-bundle-collector.sh -numworkflows 2000 -oozie
> http://localhost:11000/oozie -output /tmp``,
> -- ``bin/oozie-diag-bundle-collector.sh -jobs
> 0000001-170918144116149-oozie-asas-W -oozie http://localhost:11000/oozie
> -output .`` (verified zip the tool generated).
>
>
> Thanks,
>
> Attila Sasvari
>
>
