[
https://issues.apache.org/jira/browse/TAJO-1289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287915#comment-14287915
]
ASF GitHub Bot commented on TAJO-1289:
--------------------------------------
Github user hyunsik commented on a diff in the pull request:
https://github.com/apache/tajo/pull/356#discussion_r23396975
--- Diff:
tajo-core/src/main/java/org/apache/tajo/util/history/HistoryWriter.java ---
@@ -49,8 +56,10 @@
public static final String QUERY_LIST = "query-list";
public static final String QUERY_DETAIL = "query-detail";
public static final String HISTORY_FILE_POSTFIX = ".hist";
+ private static short REPLICATION = 1;
--- End diff --
Since history may be not critical data in many cases, one replication would
be enough. But, in some cases, it can be very important. How about improve it
to be some configurable?
> History reader fails to get the query information after a successful query
> execution
> ------------------------------------------------------------------------------------
>
> Key: TAJO-1289
> URL: https://issues.apache.org/jira/browse/TAJO-1289
> Project: Tajo
> Issue Type: Bug
> Components: tajo master
> Reporter: Jihoon Son
> Assignee: Jinho Kim
> Priority: Critical
> Fix For: 0.10
>
>
> When a query execution is successfully finished, its status is written into a
> query history. Once the history is written, the tajo master gets the status
> of the finished query from that written history.
> Here, when the tajo master reads the query history, an I/O exception occurs
> and the tajo cli stops forever. Here is the full stack trace.
> {noformat}
> 2015-01-09 00:27:54,019 INFO
> org.apache.tajo.master.querymaster.QueryInProgress: Stop
> query:q_1420730837752_0001
> 2015-01-09 00:27:54,019 INFO
> org.apache.tajo.master.rm.TajoWorkerResourceManager: Release Resource: 0.0,512
> 2015-01-09 00:27:54,019 INFO
> org.apache.tajo.master.rm.TajoWorkerResourceManager: Released QueryMaster
> (q_1420730837752_0001) resource.
> 2015-01-09 00:27:54,019 INFO
> org.apache.tajo.master.querymaster.QueryInProgress: q_1420730837752_0001
> QueryMaster stopped
> 2015-01-09 00:27:54,032 INFO org.apache.tajo.util.history.HistoryWriter:
> Create query history file:
> hdfs://localhost:7020/tmp/tajo-jihoon/staging/history/20150109/query-list/query-list-002754.hist
> 2015-01-09 00:27:54,899 ERROR org.apache.tajo.util.history.HistoryReader:
> Reading
> error:hdfs://localhost:7020/tmp/tajo-jihoon/staging/history/20150107/query-list/query-list-131932.hist,
> Cannot obtain block length for
> LocatedBlock{BP-1604697128-192.168.0.12-1412676388616:blk_1073741964_1140;
> getBlockSize()=1356; corrupt=false; offset=0; locs=[127.0.0.1:50010]}
> java.io.IOException: Cannot obtain block length for
> LocatedBlock{BP-1604697128-192.168.0.12-1412676388616:blk_1073741964_1140;
> getBlockSize()=1356; corrupt=false; offset=0; locs=[127.0.0.1:50010]}
> at
> org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:350)
> at
> org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:294)
> at
> org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:231)
> at
> org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream.java:224)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1295)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:300)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:296)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:296)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:764)
> at
> org.apache.tajo.util.history.HistoryReader.getQueries(HistoryReader.java:88)
> at
> org.apache.tajo.util.history.HistoryReader.getQueryInfo(HistoryReader.java:294)
> at
> org.apache.tajo.master.querymaster.QueryJobManager.getFinishedQuery(QueryJobManager.java:131)
> at
> org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.getQueryStatus(TajoMasterClientService.java:471)
> at
> org.apache.tajo.ipc.TajoMasterClientProtocol$TajoMasterClientProtocolService$2.callBlockingMethod(TajoMasterClientProtocol.java:551)
> at
> org.apache.tajo.rpc.BlockingRpcServer$ServerHandler.messageReceived(BlockingRpcServer.java:103)
> at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at
> org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:70)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
> at
> org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
> at
> org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
> at
> org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
> at
> org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
> at
> org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
> at
> org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
> at
> org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
> at
> org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
