[
https://issues.apache.org/jira/browse/TAJO-675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13975410#comment-13975410
]
Tajo QA commented on TAJO-675:
------------------------------
{color:red}*-1 overall.*{color} Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641025/TAJO-675.patch
against master revision f1f36ec.
{color:green}+1 @author.{color} The patch does not contain any @author
tags.
{color:red}-1 tests included.{color} The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac.{color} The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc.{color} The applied patch does not increase the
total number of javadoc warnings.
{color:green}+1 checkstyle.{color} The patch generated 0 code style errors.
{color:green}+1 findbugs.{color} The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit.{color} The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests.{color} The patch passed unit tests in tajo-rpc.
Test results:
https://builds.apache.org/job/PreCommit-TAJO-Build/363//testReport/
Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/363//console
This message is automatically generated.
> maximum frame size of frameDecoder should be increased
> ------------------------------------------------------
>
> Key: TAJO-675
> URL: https://issues.apache.org/jira/browse/TAJO-675
> Project: Tajo
> Issue Type: Improvement
> Reporter: Keuntae Park
> Attachments: TAJO-675.patch
>
>
> As Repartitioner::scheduleHashShuffledFetches re-optimizes the number of
> tasks, sometimes one task may fetch results of thousands of tasks.
> In my case, there is a case that one task fetches from 9240 tasks,
> and got following exception
> {noformat}
> 2014-03-07 13:29:24,240 ERROR rpc.AsyncRpcClient
> (AsyncRpcClient.java:exceptionCaught(220)) - skt-rf-01/50.1.103.1:28093,class
> org.apache.tajo.ipc.QueryMasterProtocol,Adjusted frame length exceeds
> 2097152: 7729393 - discarded
> org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted frame
> length exceeds 2097152: 7729393 - discarded
> at
> org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.fail(LengthFieldBasedFrameDecoder.java:417)
> at
> org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.failIfNecessary(LengthFieldBasedFrameDecoder.java:405)
> at
> org.jboss.netty.handler.codec.frame.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:320)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:425)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> 2014-03-07 13:29:24,242 ERROR rpc.AsyncRpcClient
> (AsyncRpcClient.java:exceptionCaught(235)) - RPC
> Exception:org.jboss.netty.handler.codec.frame.TooLongFrameException: Adjusted
> frame length exceeds 2097152: 7729393 - discarded
> {noformat}
> The reason seems that frameDecoder setting in getPipeline() of
> ProtoPipelineFactory.java is 2MB
> {noformat}
> public ChannelPipeline getPipeline() throws Exception {
> ChannelPipeline p = Channels.pipeline();
> p.addLast("frameDecoder", new LengthFieldBasedFrameDecoder(1048576*2, 0,
> 4,
> 0, 4));
> p.addLast("protobufDecoder", new ProtobufDecoder(defaultInstance));
> p.addLast("frameEncoder", new LengthFieldPrepender(4));
> p.addLast("protobufEncoder", new ProtobufEncoder());
> p.addLast("handler", handler);
> return p;
> }
> {noformat}
> I think temporal solution for the issue is just increasing the size.
> However, I'm not sure what the proper size is.
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
