[jira] [Updated] (TEZ-2182) Support pipelined data transfer for unordered partitioned output
[ https://issues.apache.org/jira/browse/TEZ-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-2182: -- Attachment: TEZ-2182.1.patch Attaching the patch for review. Support pipelined data transfer for unordered partitioned output Key: TEZ-2182 URL: https://issues.apache.org/jira/browse/TEZ-2182 Project: Apache Tez Issue Type: Improvement Reporter: Rajesh Balamohan Assignee: Rajesh Balamohan Attachments: TEZ-2182.1.patch TEZ-2001 covered pipelined data transfer for ordered output. Creating this JIRA to cover pipelined data transfer for unordered partitioned output. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-2182) Support pipelined data transfer for unordered partitioned output
Rajesh Balamohan created TEZ-2182: - Summary: Support pipelined data transfer for unordered partitioned output Key: TEZ-2182 URL: https://issues.apache.org/jira/browse/TEZ-2182 Project: Apache Tez Issue Type: Improvement Reporter: Rajesh Balamohan Assignee: Rajesh Balamohan TEZ-2001 covered pipelined data transfer for ordered output. Creating this JIRA to cover pipelined data transfer for unordered partitioned output. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2182) Support pipelined data transfer for unordered partitioned output
[
https://issues.apache.org/jira/browse/TEZ-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351233#comment-14351233
]
Hadoop QA commented on TEZ-2182:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703171/TEZ-2182.1.patch
against master revision 4a19a43.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/263//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/263//console
This message is automatically generated.
Support pipelined data transfer for unordered partitioned output
Key: TEZ-2182
URL: https://issues.apache.org/jira/browse/TEZ-2182
Project: Apache Tez
Issue Type: Improvement
Reporter: Rajesh Balamohan
Assignee: Rajesh Balamohan
Attachments: TEZ-2182.1.patch
TEZ-2001 covered pipelined data transfer for ordered output. Creating this
JIRA to cover pipelined data transfer for unordered partitioned output.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Success: TEZ-2182 PreCommit Build #263
Jira: https://issues.apache.org/jira/browse/TEZ-2182
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/263/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 2629 lines...]
[INFO] Final Memory: 70M/1028M
[INFO]
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703171/TEZ-2182.1.patch
against master revision 4a19a43.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/263//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/263//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
8619b714f1f72f5d7af2ebfaf1c9daa48921 logged out
==
==
Finished build.
==
==
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #260
Archived 41 artifacts
Archive block size is 32768
Received 4 blocks and 2528516 bytes
Compression is 4.9%
Took 0.76 sec
Description set: TEZ-2182
Recording test results
Email was triggered for: Success
Sending email for trigger: Success
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Assigned] (TEZ-2181) TEZ UI: For a given DAG enable client side sorting and searching.
[ https://issues.apache.org/jira/browse/TEZ-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram reassigned TEZ-2181: --- Assignee: Sreenath Somarajapuram TEZ UI: For a given DAG enable client side sorting and searching. - Key: TEZ-2181 URL: https://issues.apache.org/jira/browse/TEZ-2181 Project: Apache Tez Issue Type: Bug Reporter: Sreenath Somarajapuram Assignee: Sreenath Somarajapuram -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-714) OutputCommitters should not run in the main AM dispatcher thread
[ https://issues.apache.org/jira/browse/TEZ-714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350167#comment-14350167 ] Jeff Zhang commented on TEZ-714: * Vertex Level Commit Add one new state COMMITTING in the state machine of Vertex, and also wrap commit and abort in CallableEvent. No ABORTING state, Vertex would be in TERMINTATING state if it is in aborting. Abort is still inline invoked in InternalErrorTransition Upload a initial patch and the new state machine diagram, [~bikassaha] Please help review. * DAG Commit Add one new date COMMITTING in the state machine of DAG, and also wrap commit and abort in CallableEvent like Vertex. Abort is still inline invoked in InternalErrorTransition * Vertex Group Commit Remember the vertex group commit count, and move to succeeded if all the vertex group commits are done. * Unit test Regarding unit test. Currently I leverage the CallableEvent and make the committer to run in the central dispatcher thread. But this don't consistent with the real behavior and may hide some potential bugs. I am thinking of ways to make it run output committer in separated thread in unit test too. And Will add more unit test in the follow up patch. OutputCommitters should not run in the main AM dispatcher thread Key: TEZ-714 URL: https://issues.apache.org/jira/browse/TEZ-714 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth Assignee: Jeff Zhang Priority: Critical Follow up jira from TEZ-41. 1) If there's multiple OutputCommitters on a Vertex, they can be run in parallel. 2) Running an OutputCommitter in the main thread blocks all other event handling, w.r.t the DAG, and causes the event queue to back up. 3) This should also cover shared commits that happen in the DAG. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-714) OutputCommitters should not run in the main AM dispatcher thread
[
https://issues.apache.org/jira/browse/TEZ-714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350254#comment-14350254
]
Hadoop QA commented on TEZ-714:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703030/Vertex_2.pdf
against master revision 059b91b.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/261//console
This message is automatically generated.
OutputCommitters should not run in the main AM dispatcher thread
Key: TEZ-714
URL: https://issues.apache.org/jira/browse/TEZ-714
Project: Apache Tez
Issue Type: Improvement
Reporter: Siddharth Seth
Assignee: Jeff Zhang
Priority: Critical
Attachments: DAG_2.pdf, TEZ-714-1.patch, Vertex_2.pdf
Follow up jira from TEZ-41.
1) If there's multiple OutputCommitters on a Vertex, they can be run in
parallel.
2) Running an OutputCommitter in the main thread blocks all other event
handling, w.r.t the DAG, and causes the event queue to back up.
3) This should also cover shared commits that happen in the DAG.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1988) Tez UI does not work when using file:// in a browser
[ https://issues.apache.org/jira/browse/TEZ-1988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350193#comment-14350193 ] Sreenath Somarajapuram commented on TEZ-1988: - Support for file:// was scoped out in TEZ-1823. It can be supported with little change. [~pramachandran] [~hshah] Please confirm if we must bring back the behavior or change the readme. Tez UI does not work when using file:// in a browser - Key: TEZ-1988 URL: https://issues.apache.org/jira/browse/TEZ-1988 Project: Apache Tez Issue Type: Bug Components: UI Affects Versions: 0.6.0 Reporter: Hitesh Shah Docs mention that it defaults to using http://localhost for RM and Timeline server but it does not seem to be doing so. It uses file:///:8188 instead. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2105) Totally Sorted Edge with auto-parallelism
[ https://issues.apache.org/jira/browse/TEZ-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350196#comment-14350196 ] shanaka commented on TEZ-2105: -- Thanks Jeff for pointing me a place to start, It's really helpful. Totally Sorted Edge with auto-parallelism - Key: TEZ-2105 URL: https://issues.apache.org/jira/browse/TEZ-2105 Project: Apache Tez Issue Type: New Feature Reporter: Gopal V Labels: gsoc, gsoc2015, hadoop, java, pig, tez Pig-on-Tez supports an edge configuration using a sampled Output along with a vertex manager for automatic parallelism estimation. This is referred to in the Pig-on-Tez Hadoop Summit presentation. http://www.slideshare.net/Hadoop_Summit/pig-on-tez-low-latency-etl-with-big-data/19 Migrating that plan-model into Tez as a native edge type would allow for much more efficient scheduling of the downstream edges and effectively turn the auto-parallelism implementation into a runtime skew-correcting mechanism within this edge. The Tez Edge has enough information to sample, determine partitioning order and correct parallelism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-714) OutputCommitters should not run in the main AM dispatcher thread
[ https://issues.apache.org/jira/browse/TEZ-714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-714: --- Attachment: Vertex_2.pdf DAG_2.pdf TEZ-714-1.patch OutputCommitters should not run in the main AM dispatcher thread Key: TEZ-714 URL: https://issues.apache.org/jira/browse/TEZ-714 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth Assignee: Jeff Zhang Priority: Critical Attachments: DAG_2.pdf, TEZ-714-1.patch, Vertex_2.pdf Follow up jira from TEZ-41. 1) If there's multiple OutputCommitters on a Vertex, they can be run in parallel. 2) Running an OutputCommitter in the main thread blocks all other event handling, w.r.t the DAG, and causes the event queue to back up. 3) This should also cover shared commits that happen in the DAG. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2016) Tez UI Dag View Fit and Finish
[ https://issues.apache.org/jira/browse/TEZ-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2016: Description: TEZ-1973 has added a major contribution to the Tez UI. The core of the functionality is there and is committed to community to reduce effort on keeping patch up-to-date and to allow community exposure to the contributions. A few fit and finish items have been identified and need to be addressed. - Gaussian blur around circles is not a circle (see attachment) - Check in checkbox is not centered (see attachment) - Success icon is not flush (see attachment) - First item in customize vertex should not be selected by default (see attachment) - Edge Id name is not correct (see attachment) - Double click: was never able to see this working. If functionality is there please update the wording (no attachment) -Click and drag on graph should not route tez ui to another page (no attachment) - Prefer to see dag view view height to extend to bottom of screen (no attachment) was: TEZ-1973 has added a major contribution to the Tez UI. The core of the functionality is there and is committed to community to reduce effort on keeping patch up-to-date and to allow community exposure to the contributions. A few fit and finish items have been identified and need to be addressed. - Gaussian blur around circles is not a circle (see attachment) - Check in checkbox is not centered (see attachment) - Success icon is not flush (see attachment) - First item in customize vertex should not be selected by default (see attachment) - Edge Id name is not correct (see attachment) - Double click: was never able to see this working. If functionality is there please update the wording (no attachment) - Click and drag on graph should not route tez ui to another page (no attachment) - Prefer to see dag view view height to extend to bottom of screen (no attachment) Tez UI Dag View Fit and Finish -- Key: TEZ-2016 URL: https://issues.apache.org/jira/browse/TEZ-2016 Project: Apache Tez Issue Type: Improvement Components: UI Reporter: Jonathan Eagles Attachments: Check in checkbox is not centered.png, Customize vetex should not have first item selected by default.png, Edge Id name.png, Gaussian Blur is not Radial.png, Green success icon not flush with edge.png, Wording and functionality.png TEZ-1973 has added a major contribution to the Tez UI. The core of the functionality is there and is committed to community to reduce effort on keeping patch up-to-date and to allow community exposure to the contributions. A few fit and finish items have been identified and need to be addressed. - Gaussian blur around circles is not a circle (see attachment) - Check in checkbox is not centered (see attachment) - Success icon is not flush (see attachment) - First item in customize vertex should not be selected by default (see attachment) - Edge Id name is not correct (see attachment) - Double click: was never able to see this working. If functionality is there please update the wording (no attachment) -Click and drag on graph should not route tez ui to another page (no attachment) - Prefer to see dag view view height to extend to bottom of screen (no attachment) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2016) Tez UI Dag View Fit and Finish
[ https://issues.apache.org/jira/browse/TEZ-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2016: Description: TEZ-1973 has added a major contribution to the Tez UI. The core of the functionality is there and is committed to community to reduce effort on keeping patch up-to-date and to allow community exposure to the contributions. A few fit and finish items have been identified and need to be addressed. - Gaussian blur around circles is not a circle (see attachment) - Check in checkbox is not centered (see attachment) - Success icon is not flush (see attachment) - First item in customize vertex should not be selected by default (see attachment) - Edge Id name is not correct (see attachment) - Double click: was never able to see this working. If functionality is there please update the wording (no attachment) -Click and drag on graph should not route tez ui to another page (no attachment)- - Prefer to see dag view view height to extend to bottom of screen (no attachment) was: TEZ-1973 has added a major contribution to the Tez UI. The core of the functionality is there and is committed to community to reduce effort on keeping patch up-to-date and to allow community exposure to the contributions. A few fit and finish items have been identified and need to be addressed. - Gaussian blur around circles is not a circle (see attachment) - Check in checkbox is not centered (see attachment) - Success icon is not flush (see attachment) - First item in customize vertex should not be selected by default (see attachment) - Edge Id name is not correct (see attachment) - Double click: was never able to see this working. If functionality is there please update the wording (no attachment) -Click and drag on graph should not route tez ui to another page (no attachment) - Prefer to see dag view view height to extend to bottom of screen (no attachment) Tez UI Dag View Fit and Finish -- Key: TEZ-2016 URL: https://issues.apache.org/jira/browse/TEZ-2016 Project: Apache Tez Issue Type: Improvement Components: UI Reporter: Jonathan Eagles Attachments: Check in checkbox is not centered.png, Customize vetex should not have first item selected by default.png, Edge Id name.png, Gaussian Blur is not Radial.png, Green success icon not flush with edge.png, Wording and functionality.png TEZ-1973 has added a major contribution to the Tez UI. The core of the functionality is there and is committed to community to reduce effort on keeping patch up-to-date and to allow community exposure to the contributions. A few fit and finish items have been identified and need to be addressed. - Gaussian blur around circles is not a circle (see attachment) - Check in checkbox is not centered (see attachment) - Success icon is not flush (see attachment) - First item in customize vertex should not be selected by default (see attachment) - Edge Id name is not correct (see attachment) - Double click: was never able to see this working. If functionality is there please update the wording (no attachment) -Click and drag on graph should not route tez ui to another page (no attachment)- - Prefer to see dag view view height to extend to bottom of screen (no attachment) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2091) Add support for hosting TEZ_UI with nodejs
[ https://issues.apache.org/jira/browse/TEZ-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350724#comment-14350724 ] Hitesh Shah commented on TEZ-2091: -- The final patch looks good. Will make some minor doc edits and commit shortly. Add support for hosting TEZ_UI with nodejs -- Key: TEZ-2091 URL: https://issues.apache.org/jira/browse/TEZ-2091 Project: Apache Tez Issue Type: Improvement Components: UI Reporter: Chang Li Assignee: Chang Li Attachments: TEZ_2091_V1.patch, TEZ_2091_V2.patch, TEZ_2091_V3.patch, TEZ_2091_V4.patch, TEZ_2091_V5.patch, TEZ_2091_V6.patch There are certain category of people who would like to host TEZ UI on nodejs, it will be a good feature to be added in -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2179) Timeline relatedentries missing cause exaggerated warning
[
https://issues.apache.org/jira/browse/TEZ-2179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350721#comment-14350721
]
Hitesh Shah commented on TEZ-2179:
--
In that case, how do we get the applicationId? Do we use the tez_app_id entity
and strip out tez_ ?
Timeline relatedentries missing cause exaggerated warning
-
Key: TEZ-2179
URL: https://issues.apache.org/jira/browse/TEZ-2179
Project: Apache Tez
Issue Type: Bug
Affects Versions: 0.6.1
Reporter: Chang Li
Have run into an issue where TimelineClient has posted entities in the
timeline event posted. The related entry is not found (i believe because I
didn't have generic history turned on) and Yarn timeline server assumes the
missing event belongs to the default domain. Regular users don't have
permissions to the default domain and so the update to the related entity
fails. The posting of the event succeeds, just that the cross-link related
entity is not updated. However in the log I see the warning which is going to
scare users
{code}
2015-02-24 20:01:24,526 WARN [HistoryEventHandlingThread]
ats.ATSHistoryLoggingService: Could not post history event to ATS,
atsPutError=6, entityId=tez_application_1423769157162_273679,
eventType=APP_LAUNCHED
2015-02-24 20:01:24,526 WARN [HistoryEventHandlingThread]
ats.ATSHistoryLoggingService: Could not post history event to ATS,
atsPutError=6, entityId=tez_appattempt_1423769157162_273679_01,
eventType=AM_LAUNCHED
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (TEZ-2091) Add support for hosting TEZ_UI with nodejs
[ https://issues.apache.org/jira/browse/TEZ-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-2091: - Attachment: TEZ-2091.V6.edited.patch Add support for hosting TEZ_UI with nodejs -- Key: TEZ-2091 URL: https://issues.apache.org/jira/browse/TEZ-2091 Project: Apache Tez Issue Type: Improvement Components: UI Reporter: Chang Li Assignee: Chang Li Attachments: TEZ-2091.V6.edited.patch, TEZ_2091_V1.patch, TEZ_2091_V2.patch, TEZ_2091_V3.patch, TEZ_2091_V4.patch, TEZ_2091_V5.patch, TEZ_2091_V6.patch There are certain category of people who would like to host TEZ UI on nodejs, it will be a good feature to be added in -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: TEZ-2091 PreCommit Build #262
Jira: https://issues.apache.org/jira/browse/TEZ-2091
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/262/
###
## LAST 60 LINES OF THE CONSOLE
###
[...truncated 2625 lines...]
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703113/TEZ-2091.V6.edited.patch
against master revision 059b91b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/262//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/262//console
This message is automatically generated.
==
==
Adding comment to Jira.
==
==
Comment added.
2543f6ce1096bc51fd34a894f86c09a55fa6f236 logged out
==
==
Finished build.
==
==
Build step 'Execute shell' marked build as failure
Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #260
Archived 41 artifacts
Archive block size is 32768
Received 6 blocks and 2461855 bytes
Compression is 7.4%
Took 0.74 sec
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
###
## FAILED TESTS (if any)
##
All tests passed
[jira] [Commented] (TEZ-2091) Add support for hosting TEZ_UI with nodejs
[
https://issues.apache.org/jira/browse/TEZ-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350827#comment-14350827
]
Hadoop QA commented on TEZ-2091:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12703113/TEZ-2091.V6.edited.patch
against master revision 059b91b.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/262//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/262//console
This message is automatically generated.
Add support for hosting TEZ_UI with nodejs
--
Key: TEZ-2091
URL: https://issues.apache.org/jira/browse/TEZ-2091
Project: Apache Tez
Issue Type: Improvement
Components: UI
Reporter: Chang Li
Assignee: Chang Li
Attachments: TEZ-2091.V6.edited.patch, TEZ_2091_V1.patch,
TEZ_2091_V2.patch, TEZ_2091_V3.patch, TEZ_2091_V4.patch, TEZ_2091_V5.patch,
TEZ_2091_V6.patch
There are certain category of people who would like to host TEZ UI on nodejs,
it will be a good feature to be added in
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1827) MiniTezCluster takes 10 minutes to shut down
[ https://issues.apache.org/jira/browse/TEZ-1827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351000#comment-14351000 ] Hitesh Shah commented on TEZ-1827: -- [~zjffdu] [~bikassaha] review please. MiniTezCluster takes 10 minutes to shut down Key: TEZ-1827 URL: https://issues.apache.org/jira/browse/TEZ-1827 Project: Apache Tez Issue Type: Bug Affects Versions: 0.5.2 Reporter: Ashutosh Chauhan Assignee: Hitesh Shah Priority: Minor Attachments: TEZ-1827.1.patch Always reproducible -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2105) Totally Sorted Edge with auto-parallelism
[ https://issues.apache.org/jira/browse/TEZ-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350112#comment-14350112 ] shanaka commented on TEZ-2105: -- Hi I'm new to Apache Software Foundation and gsoc, so I'm may not me very familiar the way things happen here, but I always wanted to be a part of Apache Software Foundation, I hope to start fresh with this project. This seems like a interesting project to me, as there are many things involved in this like Hadoop, YARN and etc. , so there are many thing I can learn doing this project. I went through your presentation (Pig-on-Tez), but I could not fully understand everything in there. So it would be really helpful if you could help me understanding this project. Thanks. Totally Sorted Edge with auto-parallelism - Key: TEZ-2105 URL: https://issues.apache.org/jira/browse/TEZ-2105 Project: Apache Tez Issue Type: New Feature Reporter: Gopal V Labels: gsoc, gsoc2015, hadoop, java, pig, tez Pig-on-Tez supports an edge configuration using a sampled Output along with a vertex manager for automatic parallelism estimation. This is referred to in the Pig-on-Tez Hadoop Summit presentation. http://www.slideshare.net/Hadoop_Summit/pig-on-tez-low-latency-etl-with-big-data/19 Migrating that plan-model into Tez as a native edge type would allow for much more efficient scheduling of the downstream edges and effectively turn the auto-parallelism implementation into a runtime skew-correcting mechanism within this edge. The Tez Edge has enough information to sample, determine partitioning order and correct parallelism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2105) Totally Sorted Edge with auto-parallelism
[ https://issues.apache.org/jira/browse/TEZ-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350112#comment-14350112 ] shanaka edited comment on TEZ-2105 at 3/6/15 8:37 AM: -- Hi I'm new to Apache Software Foundation and gsoc, so I may not be very familiar the way things happening here, but I always wanted to be a part of Apache Software Foundation, I hope to start fresh with this project. This seems like a interesting project to me, as there are many things involved in this like Hadoop, YARN and etc. , so there are many thing I can learn doing this project. I went through your presentation (Pig-on-Tez), but I could not fully understand everything in there. So it would be really helpful if you could help me understanding this project. Thanks. was (Author: shanaka.kuruwita): Hi I'm new to Apache Software Foundation and gsoc, so I may not me very familiar the way things happen here, but I always wanted to be a part of Apache Software Foundation, I hope to start fresh with this project. This seems like a interesting project to me, as there are many things involved in this like Hadoop, YARN and etc. , so there are many thing I can learn doing this project. I went through your presentation (Pig-on-Tez), but I could not fully understand everything in there. So it would be really helpful if you could help me understanding this project. Thanks. Totally Sorted Edge with auto-parallelism - Key: TEZ-2105 URL: https://issues.apache.org/jira/browse/TEZ-2105 Project: Apache Tez Issue Type: New Feature Reporter: Gopal V Labels: gsoc, gsoc2015, hadoop, java, pig, tez Pig-on-Tez supports an edge configuration using a sampled Output along with a vertex manager for automatic parallelism estimation. This is referred to in the Pig-on-Tez Hadoop Summit presentation. http://www.slideshare.net/Hadoop_Summit/pig-on-tez-low-latency-etl-with-big-data/19 Migrating that plan-model into Tez as a native edge type would allow for much more efficient scheduling of the downstream edges and effectively turn the auto-parallelism implementation into a runtime skew-correcting mechanism within this edge. The Tez Edge has enough information to sample, determine partitioning order and correct parallelism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2105) Totally Sorted Edge with auto-parallelism
[ https://issues.apache.org/jira/browse/TEZ-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350112#comment-14350112 ] shanaka edited comment on TEZ-2105 at 3/6/15 8:36 AM: -- Hi I'm new to Apache Software Foundation and gsoc, so I may not me very familiar the way things happen here, but I always wanted to be a part of Apache Software Foundation, I hope to start fresh with this project. This seems like a interesting project to me, as there are many things involved in this like Hadoop, YARN and etc. , so there are many thing I can learn doing this project. I went through your presentation (Pig-on-Tez), but I could not fully understand everything in there. So it would be really helpful if you could help me understanding this project. Thanks. was (Author: shanaka.kuruwita): Hi I'm new to Apache Software Foundation and gsoc, so I'm may not me very familiar the way things happen here, but I always wanted to be a part of Apache Software Foundation, I hope to start fresh with this project. This seems like a interesting project to me, as there are many things involved in this like Hadoop, YARN and etc. , so there are many thing I can learn doing this project. I went through your presentation (Pig-on-Tez), but I could not fully understand everything in there. So it would be really helpful if you could help me understanding this project. Thanks. Totally Sorted Edge with auto-parallelism - Key: TEZ-2105 URL: https://issues.apache.org/jira/browse/TEZ-2105 Project: Apache Tez Issue Type: New Feature Reporter: Gopal V Labels: gsoc, gsoc2015, hadoop, java, pig, tez Pig-on-Tez supports an edge configuration using a sampled Output along with a vertex manager for automatic parallelism estimation. This is referred to in the Pig-on-Tez Hadoop Summit presentation. http://www.slideshare.net/Hadoop_Summit/pig-on-tez-low-latency-etl-with-big-data/19 Migrating that plan-model into Tez as a native edge type would allow for much more efficient scheduling of the downstream edges and effectively turn the auto-parallelism implementation into a runtime skew-correcting mechanism within this edge. The Tez Edge has enough information to sample, determine partitioning order and correct parallelism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2105) Totally Sorted Edge with auto-parallelism
[ https://issues.apache.org/jira/browse/TEZ-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350128#comment-14350128 ] Jeff Zhang commented on TEZ-2105: - [~shanaka.kuruwita] Glad to know you are interested in this. If you have any question regarding tez or pig on tez, you can send to pig/tez user mail list. Not sure you current knowledge of pig/tez, for this jira I think you should first understand how pig implement order by using tez. You can start with some pig script and check the source code under package org.apache.pig.backend.hadoop.executionengine.tez. Totally Sorted Edge with auto-parallelism - Key: TEZ-2105 URL: https://issues.apache.org/jira/browse/TEZ-2105 Project: Apache Tez Issue Type: New Feature Reporter: Gopal V Labels: gsoc, gsoc2015, hadoop, java, pig, tez Pig-on-Tez supports an edge configuration using a sampled Output along with a vertex manager for automatic parallelism estimation. This is referred to in the Pig-on-Tez Hadoop Summit presentation. http://www.slideshare.net/Hadoop_Summit/pig-on-tez-low-latency-etl-with-big-data/19 Migrating that plan-model into Tez as a native edge type would allow for much more efficient scheduling of the downstream edges and effectively turn the auto-parallelism implementation into a runtime skew-correcting mechanism within this edge. The Tez Edge has enough information to sample, determine partitioning order and correct parallelism. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2168) Fix application dependencies on mutually exclusive artifacts: tez-yarn-timeline-history and tez-yarn-timeline-history-with-acls
[ https://issues.apache.org/jira/browse/TEZ-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14350999#comment-14350999 ] Hitesh Shah commented on TEZ-2168: -- [~jeagles] [~pramachandran] any comments on the final patch? Fix application dependencies on mutually exclusive artifacts: tez-yarn-timeline-history and tez-yarn-timeline-history-with-acls --- Key: TEZ-2168 URL: https://issues.apache.org/jira/browse/TEZ-2168 Project: Apache Tez Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Attachments: TEZ-2168.1.post-script.patch, TEZ-2168.2.post-script.patch, TEZ-2168.combined.patch, TEZ-2168.file-move.sh Feedback from Chris Wensel on dev list related to simplifying the build file for an app dependent on Tez. Currently, based on hadoop versions, one needs to choose between mutually exclusive artifacts: tez-yarn-timeline-history and tez-yarn-timeline-history-with-acls. Suggested options: create a single artifact tez-yarn-timeline-history compiled with a default dep of hadoop 2.6, that includes the Manager. update the TezClient code to gracefully fail if the Manager is not applicable (the runtime env is Hadoop 2.4). or offer tez-yarn-timeline-history-with-acls as an optional artifact for Hadoop 2.6 deployments, with the single Manager class in it, which in turn requires the tez-yarn-timeline-history artifact -- which is sufficient for a 2.4 runtime. if the user provides the additional -with-acls artifact, they are knowingly going to have problems on Hadoop 2.4. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
