[
https://issues.apache.org/jira/browse/TAJO-2047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jaehwa Jung resolved TAJO-2047.
-------------------------------
Resolution: Duplicate
> Implement S3TableSpace::commitTable.
> ------------------------------------
>
> Key: TAJO-2047
> URL: https://issues.apache.org/jira/browse/TAJO-2047
> Project: Tajo
> Issue Type: Sub-task
> Components: S3, Storage
> Reporter: Jaehwa Jung
> Assignee: Jaehwa Jung
>
> Query hangs while executing CTAS for partition table on aws as following:
> * Root directory exists on s3
> * Dataset : tpc-h 1G
> * Query
> {code:xml}
> CREATE TABLE LINEITEM_P (L_ORDERKEY bigint, L_PARTKEY bigint, L_SUPPKEY
> bigint, L_LINENUMBER bigint, L_QUANTITY double, L_EXTENDEDPRICE double,
> L_DISCOUNT double, L_TAX double, L_RETURNFLAG text, L_LINESTATUS text,
> L_COMMITDATE text, L_RECEIPTDATE text, L_SHIPINSTRUCT text, L_SHIPMODE text,
> L_COMMENT text)
> USING TEXT WITH ('text.delimiter'='|')
> PARTITION BY COLUMN (L_SHIPDATE text) AS
> SELECT L_ORDERKEY, L_PARTKEY, L_SUPPKEY, L_LINENUMBER, L_QUANTITY,
> L_EXTENDEDPRICE, L_DISCOUNT, L_TAX, L_RETURNFLAG, L_LINESTATUS, L_COMMITDATE,
> L_RECEIPTDATE, L_SHIPINSTRUCT, L_SHIPMODE, L_COMMENT, L_SHIPDATE FROM
> LINEITEM;
> {code}
> * TSql logs
> {code:xml}
> Progress: 99%, response time: 1157.913 sec
> Progress: 99%, response time: 1158.914 sec
> Progress: 100%, response time: 1159.914 sec
> Progress: 100%, response time: 1160.915 sec
> Progress: 100%, response time: 1161.916 sec
> Progress: 100%, response time: 1162.917 sec
> Progress: 100%, response time: 1163.917 sec
> Progress: 100%, response time: 1164.918 sec
> Progress: 100%, response time: 1165.919 sec
> Progress: 100%, response time: 1166.919 sec
> Progress: 100%, response time: 1167.92 sec
> Progress: 100%, response time: 1168.921 sec
> Progress: 100%, response time: 1169.921 sec
> Progress: 100%, response time: 1170.922 sec
> Progress: 100%, response time: 1171.923 sec
> Progress: 100%, response time: 1172.924 sec
> Progress: 100%, response time: 1173.924 sec
> Progress: 100%, response time: 1174.925 sec
> {code}
> * TajoWorker logs
> {code:xml}
> 2016-01-08 07:50:56,158 INFO org.apache.tajo.querymaster.Stage: Stage
> completed - eb_1452238256580_0001_000002 (total=6, success=6, killed=0)
> 2016-01-08 07:50:56,159 INFO org.apache.tajo.querymaster.Query: Processing
> q_1452238256580_0001 of type STAGE_COMPLETED
> 2016-01-08 07:50:56,159 INFO
> org.apache.tajo.engine.planner.global.ParallelExecutionQueue: Next executable
> block eb_1452238256580_0001_000003
> 2016-01-08 07:50:56,159 INFO org.apache.tajo.querymaster.Query: Complete
> Stage[eb_1452238256580_0001_000002], State: SUCCEEDED, 2/2.
> 2016-01-08 07:50:56,159 INFO org.apache.tajo.querymaster.Query: Processing
> q_1452238256580_0001 of type QUERY_COMPLETED
> 2016-01-08 07:50:56,161 INFO org.apache.tajo.worker.ExecutionBlockContext:
> Worker's task counter - total:6, succeeded: 6, killed: 0, failed: 0
> 2016-01-08 07:50:56,194 INFO org.apache.tajo.worker.TaskManager: Stopped
> execution block:eb_1452238256580_0001_000002
> {code}
> I found that query hangs on FileTableSpace::commitTable. We need to implement
> commitTable in S3TableSpace by considering s3 features.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
