----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18761/#review36206 -----------------------------------------------------------
Ship it! +1 This patch seems to very useful for 0.8.0 too. Could you commit it to both branch-0.8.0 and master? - Hyunsik Choi On March 5, 2014, 1:27 p.m., Jinho Kim wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/18761/ > ----------------------------------------------------------- > > (Updated March 5, 2014, 1:27 p.m.) > > > Review request for Tajo. > > > Bugs: TAJO-653 > https://issues.apache.org/jira/browse/TAJO-653 > > > Repository: tajo > > > Description > ------- > > I ran a bellow query. This query throws IOException in worker > > {code} > tajo> create table orders_rc ( > o_orderkey INT8, > o_custkey INT8, > o_orderstatus TEXT, > o_totalprice FLOAT8, > o_orderdate TEXT, > o_orderpriority TEXT, > o_clerk TEXT, > o_shippriority INT4, > o_comment TEXT) > using rcfile WITH (compression.codec=org.apache.hadoop.io.compress.GzipCodec) > > tajo> insert overwrite into orders_rc select * from orders; > {code} > > > {code} > 2014-03-05 10:29:18,763 INFO zlib.ZlibFactory > (ZlibFactory.java:<clinit>(48)) - Successfully loaded & initialized > native-zlib library > 2014-03-05 10:29:18,765 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:18,936 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:18,954 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:18,963 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:18,975 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:18,979 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:18,993 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:18,998 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:19,014 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:19,028 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:19,039 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:19,080 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:19,094 INFO compress.CodecPool > (CodecPool.java:getCompressor(108)) - Got brand-new compressor [.gz] > 2014-03-05 10:29:22,484 ERROR worker.Task (Task.java:run(383)) - > java.io.IOException: write beyond end of stream > at > org.apache.hadoop.io.compress.CompressorStream.write(CompressorStream.java:66) > at java.io.BufferedOutputStream.write(BufferedOutputStream.java:122) > at java.io.DataOutputStream.write(DataOutputStream.java:107) > at > org.apache.tajo.storage.rcfile.NonSyncByteArrayOutputStream.writeTo(NonSyncByteArrayOutputStream.java:142) > at > org.apache.tajo.storage.rcfile.RCFile$RCFileAppender.WriteKeyBuffer(RCFile.java:947) > at > org.apache.tajo.storage.rcfile.RCFile$RCFileAppender.writeKey(RCFile.java:1046) > at > org.apache.tajo.storage.rcfile.RCFile$RCFileAppender.flushRecords(RCFile.java:1005) > at > org.apache.tajo.storage.rcfile.RCFile$RCFileAppender.append(RCFile.java:894) > at > org.apache.tajo.storage.rcfile.RCFile$RCFileAppender.addTuple(RCFile.java:851) > at > org.apache.tajo.engine.planner.physical.StoreTableExec.next(StoreTableExec.java:77) > at org.apache.tajo.worker.Task.run(Task.java:374) > at org.apache.tajo.worker.TaskRunner$1.run(TaskRunner.java:391) > at java.lang.Thread.run(Thread.java:744) > {code} > > > Diffs > ----- > > tajo-storage/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java > afbcaa86dc3b69f12d4694e6e3f75b2c71a16afc > > Diff: https://reviews.apache.org/r/18761/diff/ > > > Testing > ------- > > > Thanks, > > Jinho Kim > >
