date:20160120

[jira] [Resolved] (TEZ-3050) Tez UI 2: Add counter columns

2016-01-20 Thread Sreenath Somarajapuram (JIRA)


 [ 
https://issues.apache.org/jira/browse/TEZ-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sreenath Somarajapuram resolved TEZ-3050.
-
Resolution: Fixed

Committed to TEZ-2980 branch

> Tez UI 2: Add counter columns
> -
>
> Key: TEZ-3050
> URL: https://issues.apache.org/jira/browse/TEZ-3050
> Project: Apache Tez
>  Issue Type: Sub-task
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-3050.1.patch
>
>
> - A predefined set (Configurable) of counter columns must be available in all 
> column selectors.
> - Columns must be auto populated in entities under a vertex
> - DAGs must have a extra set of default counters available



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (TEZ-3056) Tez UI 2: Remove version changes in TEZ-2980 branch

2016-01-20 Thread Sreenath Somarajapuram (JIRA)

Sreenath Somarajapuram created TEZ-3056:
---

 Summary: Tez UI 2: Remove version changes in TEZ-2980 branch
 Key: TEZ-3056
 URL: https://issues.apache.org/jira/browse/TEZ-3056
 Project: Apache Tez
  Issue Type: Sub-task
Reporter: Sreenath Somarajapuram
Assignee: Sreenath Somarajapuram


- Revert changes that came in with commit b2f0bd50 in TEZ-2980 branch
- This would reset the version in pom files



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: Tez-Build-branch-TEZ-2980 #15

2016-01-20 Thread Apache Jenkins Server

See 

Changes:

[bikas] TEZ-2996. TestAnalyzer fails in trunk after recovery redesign (bikas)

[sree] TEZ-2997. Tez UI: Support searches by CallerContext ID for DAGs (sree)

[zjffdu] TEZ-2976. Recovery fails when InputDescriptor is changed during input

[hitesh] TEZ-2910. Set caller context for tracing ( integrate with HDFS-9184 ).

[rbalamohan] TEZ-3006. Remove unused import in TestHistoryParser (rbalamohan)

[rbalamohan] Updating CHANGES.txt (rbalamohan)

[jeagles] TEZ-3011. Link Vertex Name in Dag Tasks/Task Attempts to Vertex

[jlowe] TEZ-2972. Avoid task rescheduling when a node turns unhealthy (jlowe)

[bikas] TEZ-2972. addendum for findbugs. (Jason Lowe via bikas)

[bikas] TEZ-2914. Ability to limit vertex concurrency (bikas)

[bikas] Fix CHANGES.txt for recent back-ports

[bikas] TEZ-3017. HistoryACLManager does not have a close method for cleanup

[sseth] TEZ-3025. InputInitializer creation should use the dag ugi. (sseth)

[jeagles] TEZ-2129. Task and Attempt views should contain links to the logs

[sseth] TEZ-2978. Add an option to allow the SplitGrouper to generate node local

[sseth] TEZ-2669. Propagation of errors from plugins to the AM for error

[sseth] TEZ-3024. Move TaskCommunicator to correct package. (sseth)

[sseth] TEZ-3033 changes for 0.8.2 release part1. Update version to

[hitesh] TEZ-3032. DAG start time getting logged using system time instead of

[hitesh] Fix CHANGES.txt for TEZ-3032

[hitesh] TEZ-3037. History URL should be set regardless of which history logging

[hitesh] TEZ-3032. Addendum patch. DAG start time getting logged using system

[jeagles] TEZ-2937. Can Processor.close() be called after closing inputs and

[sseth] TEZ-3033 part2. Update documentation for 0.8.2 release.

[rbalamohan] TEZ-2898. tez tools : swimlanes.py is broken in master (rbalamohan)

[sree] TEZ-2982. Tez UI: Create tez-ui2 directory and get a basic dummy page

[sree] TEZ-3016. Tez UI 2: Make bower dependency silent (sree)

[sree] TEZ-2983. Tez UI 2: Get ember initializers functional (sree)

[sree] TEZ-3018. Tez UI 2: Add config.env (sree)

[sree] TEZ-3019. Tez UI 2: Replace BaseURL with Host (sree)

[sree] TEZ-2984. Tez UI 2: Create abstract classes (sree)

[sree] TEZ-3020. Tez UI 2: Add entity blueprint (sree)

[sree] TEZ-2985. Tez UI 2: Create loader and entity classes (sree)

[sree] TEZ-3021. Tez UI 2: Add env service & initializer (sree)

[sree] TEZ-3023. Tez UI 2: Abstract adapter and route (sree)

[sree] TEZ-3022. Tez UI 2: Add serializer & adapter for timeline server (sree)

[sree] Update version in branch to 0.8.2-TEZ-2980-SNAPSHOT to prevent

[sree] TEZ-3026. Tez UI 2: Add adapters for RM & AM (sree)

[sree] TEZ-3027. Tez UI 2: Add header and footer elements (sree)

[sree] TEZ-2986. Tez UI 2: Implement All DAGs page (sree)

[sree] TEZ-3038. Tez UI 2: Create DAG details page (sree)

[sree] TEZ-3039. Tez UI 2: Create all sub-pages for DAG (sree)

[sree] TEZ-3040. Tez UI 2: Create Vertex details page & sub tables (sree)

[sree] TEZ-3041. Tez UI 2: Create Task & Attempt details page with sub tables

[sree] TEZ-3045. Tez UI 2: Create application details page with DAGs tab (sree)

[sree] TEZ-3048. Tez UI 2: Make PhantomJS a local dependency for build tests

[sree] TEZ-3042. Tez UI 2: Create Counters pages (sree)

[sree] TEZ-3043. Tez UI 2: Create configurations page (sree)

[sree] TEZ-3049. Tez UI 2: Add column selector (sree)

[sree] TEZ-3050. Tez UI 2: Add counter columns (sree)

--
[...truncated 4819 lines...]
[INFO] 
[INFO] --- maven-surefire-plugin:2.14.1:test (default-test) @ tez-javadoc-tools 
---
[INFO] No tests to run.
[INFO] 
[INFO] --- buildnumber-maven-plugin:1.1:create (default) @ tez-javadoc-tools ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd 

 && git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildNumber: 8d80272ca0e27287aa2b95eb3b23e75a67b957c7 at 
timestamp: 1453314240776
[INFO] Executing: /bin/sh -c cd 

 && git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ 
tez-javadoc-tools ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ 
tez-javadoc-tools ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO]

[jira] [Created] (TEZ-3061) Tez UI 2: Display in-progress vertex table in DAG details

2016-01-20 Thread Sreenath Somarajapuram (JIRA)

Sreenath Somarajapuram created TEZ-3061:
---

 Summary: Tez UI 2: Display in-progress vertex table in DAG details
 Key: TEZ-3061
 URL: https://issues.apache.org/jira/browse/TEZ-3061
 Project: Apache Tez
  Issue Type: Sub-task
Reporter: Sreenath Somarajapuram






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (TEZ-3062) Tez UI 2: Integrate graphical view

2016-01-20 Thread Sreenath Somarajapuram (JIRA)

Sreenath Somarajapuram created TEZ-3062:
---

 Summary: Tez UI 2: Integrate graphical view
 Key: TEZ-3062
 URL: https://issues.apache.org/jira/browse/TEZ-3062
 Project: Apache Tez
  Issue Type: Sub-task
Reporter: Sreenath Somarajapuram


- Integrate the graphical view component with tez-ui2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: Tez-Build-branch-TEZ-2980 #14

2016-01-20 Thread Apache Jenkins Server

See 

Changes:

[sree] Revert "Update version in branch to 0.8.2-TEZ-2980-SNAPSHOT to prevent"

--
[...truncated 2040 lines...]
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd 

 && git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildNumber: d36f9066b75c451b5d37800eab02641ae0ceded5 at 
timestamp: 1453310459082
[INFO] Executing: /bin/sh -c cd 

 && git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ 
tez-javadoc-tools ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ 
tez-javadoc-tools ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ 
tez-javadoc-tools ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ 
tez-javadoc-tools ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ 
tez-javadoc-tools ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.14.1:test (default-test) @ tez-javadoc-tools 
---
[INFO] No tests to run.
[INFO] Skipping execution of surefire because it has already been run for this 
configuration
[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ tez-javadoc-tools ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-jar-plugin:2.4:test-jar (default) @ tez-javadoc-tools ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: 

[INFO] 
[INFO] 
[INFO] Building hadoop-shim-impls 0.8.2-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hadoop-shim-impls ---
[INFO] Deleting 

[INFO] 
[INFO] --- buildnumber-maven-plugin:1.1:create (default) @ hadoop-shim-impls ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildNumber: d36f9066b75c451b5d37800eab02641ae0ceded5 at 
timestamp: 1453310459145
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ 
hadoop-shim-impls ---
[INFO] 
[INFO] --- buildnumber-maven-plugin:1.1:create (default) @ hadoop-shim-impls ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildNumber: d36f9066b75c451b5d37800eab02641ae0ceded5 at 
timestamp: 1453310459159
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] ---

Jenkins build is back to normal : Tez-Build-branch-0.7 #115

2016-01-20 Thread Apache Jenkins Server

See

[jira] [Created] (TEZ-3060) Tez UI 2: Activate auto-refresh

2016-01-20 Thread Sreenath Somarajapuram (JIRA)

Sreenath Somarajapuram created TEZ-3060:
---

 Summary: Tez UI 2: Activate auto-refresh
 Key: TEZ-3060
 URL: https://issues.apache.org/jira/browse/TEZ-3060
 Project: Apache Tez
  Issue Type: Sub-task
Reporter: Sreenath Somarajapuram


- When auto refresh is selected the following fields must update
-- Status
-- Progress
-- Task counts
-- Counters



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (TEZ-3064) Tez UI 2: Add All DAGs filters

2016-01-20 Thread Sreenath Somarajapuram (JIRA)

Sreenath Somarajapuram created TEZ-3064:
---

 Summary: Tez UI 2: Add All DAGs filters
 Key: TEZ-3064
 URL: https://issues.apache.org/jira/browse/TEZ-3064
 Project: Apache Tez
  Issue Type: Sub-task
Reporter: Sreenath Somarajapuram


Filter for - Name, Id, Submitter, Status, Application and Content ID



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: Tez-Build-branch-TEZ-2980 #13

2016-01-20 Thread Apache Jenkins Server

See 

Changes:

[sree] TEZ-3050. Tez UI 2: Add counter columns (sree)

--
[...truncated 2492 lines...]
[INFO] 
[INFO] --- maven-jar-plugin:2.4:test-jar (default) @ tez-javadoc-tools ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: 

[INFO] 
[INFO] 
[INFO] Building hadoop-shim-impls 0.8.2-TEZ-2980-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hadoop-shim-impls ---
[INFO] Deleting 

[INFO] 
[INFO] --- buildnumber-maven-plugin:1.1:create (default) @ hadoop-shim-impls ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildNumber: d011f7c39de34907d143b85751e15500ee678c90 at 
timestamp: 1453307489733
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ 
hadoop-shim-impls ---
[INFO] 
[INFO] --- buildnumber-maven-plugin:1.1:create (default) @ hadoop-shim-impls ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildNumber: d011f7c39de34907d143b85751e15500ee678c90 at 
timestamp: 1453307489743
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ 
hadoop-shim-impls ---
[INFO] 
[INFO] --- maven-jar-plugin:2.4:test-jar (default) @ hadoop-shim-impls ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] Building jar: 

[INFO] 
[INFO] 
[INFO] Building hadoop-shim-2.6 0.8.2-TEZ-2980-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hadoop-shim-2.6 ---
[INFO] Deleting 

[INFO] 
[INFO] --- buildnumber-maven-plugin:1.1:create (default) @ hadoop-shim-2.6 ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd 

 && git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildNumber: d011f7c39de34907d143b85751e15500ee678c90 at 
timestamp: 1453307489885
[INFO] Executing: /bin/sh -c cd 

 && git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ 
hadoop-shim-2.6 ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ 
hadoop-shim-2.6 ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ 
hadoop-shim-2.6 ---
[INFO] Changes detected - recompiling the

[jira] [Created] (TEZ-3065) Tez UI 2: Implement counter drill down

2016-01-20 Thread Sreenath Somarajapuram (JIRA)

Sreenath Somarajapuram created TEZ-3065:
---

 Summary: Tez UI 2: Implement counter drill down
 Key: TEZ-3065
 URL: https://issues.apache.org/jira/browse/TEZ-3065
 Project: Apache Tez
  Issue Type: Sub-task
Reporter: Sreenath Somarajapuram






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Build failed in Jenkins: Tez-Build #1362

2016-01-20 Thread Apache Jenkins Server

See 

Changes:

[hitesh] TEZ-2594. Fix LICENSE for missing entries for full and minimal 
tarballs.

--
[...truncated 4214 lines...]
[INFO] Working directory: 

[INFO] Storing buildNumber: 73e993cba52e25c0d126a59439a76b43af9506bc at 
timestamp: 1453331256125
[INFO] Executing: /bin/sh -c cd 
 && git rev-parse 
--verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ 
hadoop-shim-impls ---
[INFO] 
[INFO] --- maven-jar-plugin:2.4:test-jar (default) @ hadoop-shim-impls ---
[WARNING] JAR will be empty - no content was marked for inclusion!
[INFO] 
[INFO] 
[INFO] Building hadoop-shim-2.6 0.8.3-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hadoop-shim-2.6 ---
[INFO] 
[INFO] --- buildnumber-maven-plugin:1.1:create (default) @ hadoop-shim-2.6 ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildNumber: 73e993cba52e25c0d126a59439a76b43af9506bc at 
timestamp: 1453331256244
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ 
hadoop-shim-2.6 ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ 
hadoop-shim-2.6 ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ 
hadoop-shim-2.6 ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 2 source files to 

[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ 
hadoop-shim-2.6 ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ 
hadoop-shim-2.6 ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.14.1:test (default-test) @ hadoop-shim-2.6 
---
[INFO] Surefire report directory: 


---
 T E S T S
---

---
 T E S T S
---
Running org.apache.tez.hadoop.shim.TestHadoop25_26_27ShimProvider
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.326 sec

Results :

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- buildnumber-maven-plugin:1.1:create (default) @ hadoop-shim-2.6 ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildNumber: 73e993cba52e25c0d126a59439a76b43af9506bc at 
timestamp: 1453331257506
[INFO] Executing: /bin/sh -c cd 
 
&& git rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ 
hadoop-shim-2.6 ---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ 
hadoop-shim-2.6 ---
[INFO] Using 'UTF-8'

[jira] [Created] (TEZ-3063) Tez UI 2: Display vertex configurations

2016-01-20 Thread Sreenath Somarajapuram (JIRA)

Sreenath Somarajapuram created TEZ-3063:
---

 Summary: Tez UI 2: Display vertex configurations
 Key: TEZ-3063
 URL: https://issues.apache.org/jira/browse/TEZ-3063
 Project: Apache Tez
  Issue Type: Sub-task
Reporter: Sreenath Somarajapuram


Display - source, sink and processor configurations of a vertex



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (TEZ-3056) Tez UI 2: Remove version changes in TEZ-2980 branch

2016-01-20 Thread Sreenath Somarajapuram (JIRA)


 [ 
https://issues.apache.org/jira/browse/TEZ-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sreenath Somarajapuram resolved TEZ-3056.
-
Resolution: Fixed

Used git revert and committed to TEZ-2980 branch as part of commit d36f906

> Tez UI 2: Remove version changes in TEZ-2980 branch
> ---
>
> Key: TEZ-3056
> URL: https://issues.apache.org/jira/browse/TEZ-3056
> Project: Apache Tez
>  Issue Type: Sub-task
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
>
> - Revert changes that came in with commit b2f0bd50 in TEZ-2980 branch
> - This would reset the version in pom files



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (TEZ-3055) I got this jar is by the mvn package and run it on my cluster,occurred this problem.I can run the example which the tez given successful.But when I run this jar which is p

2016-01-20 Thread Hitesh Shah (JIRA)


 [ 
https://issues.apache.org/jira/browse/TEZ-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hitesh Shah resolved TEZ-3055.
--
Resolution: Invalid

Closing this jira out as invalid for now. Please check on the user mailing list 
first and based on discussions there, a jira can then be filed or this one 
re-opened. 

To fix this issue, I suggest adding your jar to tez.aux.uris and retrying. 

> I got this jar is by the mvn package and run it on my cluster,occurred this 
> problem.I can run the example which the tez given successful.But when I run 
> this jar which is packaged by mvn packsge myself is failed.
> ---
>
> Key: TEZ-3055
> URL: https://issues.apache.org/jira/browse/TEZ-3055
> Project: Apache Tez
>  Issue Type: Improvement
> Environment: this is the tez orderedwordcount example,I want it can 
> be run on my cluster,but occurred this problem.This code is packaged by 
> myself.When I run the tez's example jar on my cluster,it's ok.My environment 
> is hadoop2.6.0+tez0.7.0.Can you give me some suggestion? 
>Reporter: Lvpenglin
>  Labels: jar, maven-package, run-example, tez
>
> Vertex failed, vertexName=ASSOCIATE, vertexId=vertex_1452667827400_0022_1_00, 
> diagnostics=
>  Task failed, taskId=task_1452667827400_0022_1_00_00, diagnostics=
>  TaskAttempt 0 failed, info=
>  Error: Failure while running 
> task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class: 
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45)
> at 
> org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96)
> at 
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584)
> at 
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202)
> at 
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
> at 
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at 
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at 
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.ClassNotFoundException: 
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:274)
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:43)
> ... 15 more
> TaskAttempt 1 failed, info=
>  Error: Failure while running 
> task:org.apache.tez.dag.api.TezUncheckedException: Unable to load class: 
> com.teradata.sddg.sgprocessor.TezDataGeneratorTest$DataGenerationProcessor
> at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45)
> at 
> org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:96)
> at 
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.createProcessor(LogicalIOProcessorRuntimeTask.java:584)
> at 
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.initialize(LogicalIOProcessorRuntimeTask.java:202)
> at 
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
> at 
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at 
>

[jira] [Created] (TEZ-3066) TaskAttemptFinishedEvent ConcurrentModificationException if processed by RecoveryService and history logging simultaneously

2016-01-20 Thread Jason Lowe (JIRA)

Jason Lowe created TEZ-3066:
---

 Summary: TaskAttemptFinishedEvent ConcurrentModificationException 
if processed by RecoveryService and history logging simultaneously
 Key: TEZ-3066
 URL: https://issues.apache.org/jira/browse/TEZ-3066
 Project: Apache Tez
  Issue Type: Bug
Affects Versions: 0.7.0
Reporter: Jason Lowe


A ConcurrentModificationException can occur if a TaskAttemptFinishedEvent is 
processed simultaneously by the recovery service and another history logging 
service.  Sample stacktraces to follow.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Link collection page to an article and slides about Tez

2016-01-20 Thread Tsuyoshi Ozawa

Hi Bikas,

Thanks for your reply. Could you give me a privilege to edit the Wiki?
My account name is ozawa.

Thanks,
- Tsuyoshi

On Wed, Jan 20, 2016 at 3:48 AM, Bikas Saha  wrote:
> Great idea. Please feel free to create one on the wiki.
>
>
>
> From: Tsuyoshi Ozawa [mailto:oz...@apache.org]
> Sent: Tuesday, January 19, 2016 12:44 AM
> To: dev ; u...@tez.apache.org
> Subject: Link collection page to an article and slides about Tez
>
>
>
> Hi,
>
>
>
> Are there pages about link collection to slides and articles on Wiki or
> Tez's web site? If not, can we add a page to the wiki?
>
>
>
> I've also been written an about Apache Tez in Japanese. Can we add a link to
> the article from the page?
>
>
>
> Thanks,
>
> - Tsuyoshi

[DISCUSS] Merge OOM improvements

2016-01-20 Thread Jonathan Eagles

A new pattern of pig jobs that succeed with MR and fail with Tez have been
identified. Hoping to brainstorm ideas so we can identify issues and file
target jiras.

Here is a typical stack trace, though sometimes it occurs with final merge
(since in-memory segment overhead > mapout overhead)

Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.apache.hadoop.io.DataInputBuffer.(DataInputBuffer.java:68)
at
org.apache.tez.runtime.library.common.shuffle.orderedgrouped.InMemoryReader.(InMemoryReader.java:42)
at
org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.createInMemorySegments(MergeManager.java:837)
at
org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager.access$200(MergeManager.java:75)
at
org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeManager$InMemoryMerger.merge(MergeManager.java:642)
at
org.apache.tez.runtime.library.common.shuffle.orderedgrouped.MergeThread.run(MergeThread.java:89)

*Details*
  around 1,000,000 spills were fetched committing around 100MB to the
memory budget (500,000 in memory). However, actual memory used for 500,000
segments (50-350 bytes) is 480MB (expected 100-200MB)

*Mapout overhead is not budgeted*
  Each mapoutput needs around 50 bytes in addition to the data

*In Memory Segment overhead is not budgeted*
  Each In memory segment needs around 80 bytes in addition to the data

*Interaction with auto reduce parallelism*
  In this scenario, the upstream vertex was assuming 999 (pig's default
hint to use auto-reduce parallelism) downstream tasks. However, was reduced
to 24 due to auto-reduce parallelism. This is putting 40 times more
segments per downstream task. Should auto-reduce parallelism consider merge
overhead when calculating parallelism?

*Legacy Default Sorter Empty Segment*
  Default sorter does not optimize empty segments like pipeline sorter does
and shows this symptom more.


2016-01-10 11:46:01,208 [INFO] [fetcher {scope_601} #7]
|orderedgrouped.MergeManager|: closeInMemoryFile - map-output of size:
116, inMemoryMapOutputs.size() - 571831, commitMemory - 91503730,
usedMemory -91503846, mapOutput=MapOutput( AttemptIdentifier:
InputAttemptIdentifier [inputIdentifier=Input
Identifier [inputIndex=763962], attemptNumber=0,
pathComponent=attempt_1444791925832_10460712_1_00_017766_0_10003,
spillType=0, spillId=-1], Type: MEMORY)
2016-01-10 11:46:01,208 [INFO] [fetcher {scope_601} #7]
|orderedgrouped.ShuffleScheduler|: Completed fetch for attempt: {763962, 0,
attempt_1444791925832_10460712_1_00_017766_0_10003} to MEMORY, csize=128,
dsize=116, EndTime=1452426361208, TimeTaken=0, Rate=0.00 MB/s
2016-01-10 11:46:01,209 [INFO] [fetcher {scope_601} #7]
|orderedgrouped.ShuffleScheduler|: scope_601: All inputs fetched for input
vertex : scope-601
2016-01-10 11:46:01,209 [INFO] [fetcher {scope_601} #7]
|orderedgrouped.ShuffleScheduler|: copy(1091856 (spillsFetched=1091856) of
1091856. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.68
MB/s)

Build failed in Jenkins: Tez-Build-Hadoop-2.2 #237

2016-01-20 Thread Apache Jenkins Server

See 

Changes:

[sseth] TEZ-3053. Containers timeout if they do not receive a task within the

[hitesh] TEZ-2594. Fix LICENSE for missing entries for full and minimal 
tarballs.

--
[...truncated 1185 lines...]
[4mRunning "concat:dist/scripts/components.js" (concat) task[24m
File "dist/scripts/components.js" created.

[4mRunning "concat:dist/scripts/templates.js" (concat) task[24m
File "dist/scripts/templates.js" created.

[4mRunning "concat:dist/scripts/main.js" (concat) task[24m
File "dist/scripts/main.js" created.

[4mRunning "cssmin:dist" (cssmin) task[24m
File dist/styles/main.css created.

[4mRunning "cssmin:dist/styles/main.css" (cssmin) task[24m
File dist/styles/main.css created.

[4mRunning "copy:dist" (copy) task[24m
Copied [36m27[39m files

[4mRunning "usemin:html" (usemin) task[24m

[1mProcessing as HTML - dist/index.html[22m
Update the HTML to reference our concat/min/revved script files
Update the HTML with the new css filenames
Update the HTML with the new img filenames
Update the HTML with data-main tags
Update the HTML with the data tags
Update the HTML with background imgs, case there is some inline style
Update the HTML with anchors images
Update the HTML with reference in input

[4mRunning "usemin:css" (usemin) task[24m

[1mProcessing as CSS - dist/styles/main.css[22m
Update the CSS with new img filenames

[32mDone, without errors.[39m
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ tez-ui ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ tez-ui ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ 
tez-ui ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ tez-ui 
---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.14.1:test (default-test) @ tez-ui ---
[INFO] No tests to run.
[INFO] 
[INFO] 
[INFO] Building tez-plugins 0.8.3-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tez-plugins ---
[INFO] 
[INFO] --- buildnumber-maven-plugin:1.1:create (default) @ tez-plugins ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd 
 && git 
rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildNumber: 73e993cba52e25c0d126a59439a76b43af9506bc at 
timestamp: 1453336925661
[INFO] Executing: /bin/sh -c cd 
 && git 
rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ 
tez-plugins ---
[INFO] 
[INFO] 
[INFO] Building tez-tools 0.8.3-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ tez-tools ---
[INFO] 
[INFO] --- buildnumber-maven-plugin:1.1:create (default) @ tez-tools ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd 
 && git 
rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildNumber: 73e993cba52e25c0d126a59439a76b43af9506bc at 
timestamp: 1453336925676
[INFO] Executing: /bin/sh -c cd 
 && git 
rev-parse --verify HEAD
[INFO] Working directory: 

[INFO] Storing buildScmBranch: UNKNOWN
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:maven-version (maven-version) @ 
tez-tools ---
[INFO] 
[INFO] 
[INFO] Building tez-perf-analyzer 0.8.3-SNAPSHOT
[INFO]

[jira] [Resolved] (TEZ-3050) Tez UI 2: Add counter columns

[jira] [Created] (TEZ-3056) Tez UI 2: Remove version changes in TEZ-2980 branch

Build failed in Jenkins: Tez-Build-branch-TEZ-2980 #15

[jira] [Created] (TEZ-3061) Tez UI 2: Display in-progress vertex table in DAG details

[jira] [Created] (TEZ-3062) Tez UI 2: Integrate graphical view

Build failed in Jenkins: Tez-Build-branch-TEZ-2980 #14

Jenkins build is back to normal : Tez-Build-branch-0.7 #115

[jira] [Created] (TEZ-3060) Tez UI 2: Activate auto-refresh

[jira] [Created] (TEZ-3064) Tez UI 2: Add All DAGs filters

Build failed in Jenkins: Tez-Build-branch-TEZ-2980 #13

[jira] [Created] (TEZ-3065) Tez UI 2: Implement counter drill down

Build failed in Jenkins: Tez-Build #1362

[jira] [Created] (TEZ-3063) Tez UI 2: Display vertex configurations

[jira] [Resolved] (TEZ-3056) Tez UI 2: Remove version changes in TEZ-2980 branch

[jira] [Resolved] (TEZ-3055) I got this jar is by the mvn package and run it on my cluster,occurred this problem.I can run the example which the tez given successful.But when I run this jar which is p

[jira] [Created] (TEZ-3066) TaskAttemptFinishedEvent ConcurrentModificationException if processed by RecoveryService and history logging simultaneously

Re: Link collection page to an article and slides about Tez

[DISCUSS] Merge OOM improvements

Build failed in Jenkins: Tez-Build-Hadoop-2.2 #237

19 matches

Site Navigation

Mail list logo

Footer information