[jira] [Commented] (TEZ-3310) Handle splits grouping better when locality information is not available (or only when localhost is available)

2019-05-03 Thread Todd Lipcon (JIRA) 


[ 
https://issues.apache.org/jira/browse/TEZ-3310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832940#comment-16832940
 ] 

Todd Lipcon commented on TEZ-3310:
--

Sure enough, this is causing problems for pseudo-distributed-cluster testing. 
The "min split length" config gets ignored because all of the splits are on 
localhost, and thus queries have different behavior on this cluster than on a 
remote one.

> Handle splits grouping better when locality information is not available (or 
> only when localhost is available)
> --
>
> Key: TEZ-3310
> URL: https://issues.apache.org/jira/browse/TEZ-3310
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Priority: Minor
>
> This is a follow up JIRA to TEZ-3291. TEZ-3291 tries to handle the case when 
> only localhost is specified in the locations. It would be good to improve 
> handling of splits grouping when Tez does not have enough information about 
> the locality.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (TEZ-4066) Upgrade servlet-api from 2.5 to 3.1.0

2019-05-03 Thread TezQA (JIRA) 


[ 
https://issues.apache.org/jira/browse/TEZ-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832876#comment-16832876
 ] 

TezQA commented on TEZ-4066:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  7m 
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  3m 
46s{color} | {color:green} tez-dag in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 59m 
34s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
37s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 89m  9s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/tez:d4a62de |
| JIRA Issue | TEZ-4066 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12967810/TEZ-4066.001.patch |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  xml  compile  |
| uname | Linux c5ef110786fc 4.4.0-143-generic #169~14.04.2-Ubuntu SMP Wed Feb 
13 15:00:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | master / 0f71b0b |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-TEZ-Build/143/testReport/ |
| Max. process+thread count | 1518 (vs. ulimit of 1) |
| modules | C: tez-dag . U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-TEZ-Build/143/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> Upgrade servlet-api from 2.5 to 3.1.0
> -
>
> Key: TEZ-4066
> URL: https://issues.apache.org/jira/browse/TEZ-4066
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
> Attachments: TEZ-4066.001.patch
>
>
> Oozie launcher jobs trying to launch Tez jobs now fail to render Oozie 
> Launcher Job AM due to both 2.5 (from tez) and 3.1.0 (from hadoop) 
> servlet-api both being in the classpath. Tez should sync with servlet api 
> version from tez master branch that only supports hadoop 3+
> {code}
> 2019-04-30 14:53:02,747 WARN [qtp1213419524-119] 
> org.eclipse.jetty.server.HttpChannel:
>

[jira] [Updated] (TEZ-4066) Upgrade servlet-api from 2.5 to 3.1.0

2019-05-03 Thread Jonathan Eagles (JIRA) 


 [ 
https://issues.apache.org/jira/browse/TEZ-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Eagles updated TEZ-4066:
-
Attachment: TEZ-4066.001.patch

> Upgrade servlet-api from 2.5 to 3.1.0
> -
>
> Key: TEZ-4066
> URL: https://issues.apache.org/jira/browse/TEZ-4066
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
> Attachments: TEZ-4066.001.patch
>
>
> Oozie launcher jobs trying to launch Tez jobs now fail to render Oozie 
> Launcher Job AM due to both 2.5 (from tez) and 3.1.0 (from hadoop) 
> servlet-api both being in the classpath. Tez should sync with servlet api 
> version from tez master branch that only supports hadoop 3+
> {code}
> 2019-04-30 14:53:02,747 WARN [qtp1213419524-119] 
> org.eclipse.jetty.server.HttpChannel:
> java.lang.NoSuchMethodError: 
> javax.servlet.http.HttpServletRequest.isAsyncStarted()Z
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:688)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
>   at 
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>   at org.eclipse.jetty.server.Server.handle(Server.java:534)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
>   at 
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
>   at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
>   at 
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
>   at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
>   at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
>   at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
>   at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (TEZ-4066) Upgrade servlet-api from 2.5 to 3.1.0

2019-05-03 Thread Jonathan Eagles (JIRA) 


[ 
https://issues.apache.org/jira/browse/TEZ-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832811#comment-16832811
 ] 

Jonathan Eagles commented on TEZ-4066:
--

More to the point, we can see that hadoop updated jetty and servlet-api at the 
same time while tez only updated jetty and not servlet-api

> Upgrade servlet-api from 2.5 to 3.1.0
> -
>
> Key: TEZ-4066
> URL: https://issues.apache.org/jira/browse/TEZ-4066
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
>
> Oozie launcher jobs trying to launch Tez jobs now fail to render Oozie 
> Launcher Job AM due to both 2.5 (from tez) and 3.1.0 (from hadoop) 
> servlet-api both being in the classpath. Tez should sync with servlet api 
> version from tez master branch that only supports hadoop 3+
> {code}
> 2019-04-30 14:53:02,747 WARN [qtp1213419524-119] 
> org.eclipse.jetty.server.HttpChannel:
> java.lang.NoSuchMethodError: 
> javax.servlet.http.HttpServletRequest.isAsyncStarted()Z
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:688)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
>   at 
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>   at org.eclipse.jetty.server.Server.handle(Server.java:534)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
>   at 
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
>   at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
>   at 
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
>   at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
>   at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
>   at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
>   at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
>   at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (TEZ-4066) Upgrade servlet-api from 2.5 to 3.1.0

2019-05-03 Thread Jonathan Eagles (JIRA) 
Jonathan Eagles created TEZ-4066:


 Summary: Upgrade servlet-api from 2.5 to 3.1.0
 Key: TEZ-4066
 URL: https://issues.apache.org/jira/browse/TEZ-4066
 Project: Apache Tez
  Issue Type: Bug
Reporter: Jonathan Eagles
Assignee: Jonathan Eagles


Oozie launcher jobs trying to launch Tez jobs now fail to render Oozie Launcher 
Job AM due to both 2.5 (from tez) and 3.1.0 (from hadoop) servlet-api both 
being in the classpath. Tez should sync with servlet api version from tez 
master branch that only supports hadoop 3+

{code}
2019-04-30 14:53:02,747 WARN [qtp1213419524-119] 
org.eclipse.jetty.server.HttpChannel:
java.lang.NoSuchMethodError: 
javax.servlet.http.HttpServletRequest.isAsyncStarted()Z
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:688)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)