[ https://issues.apache.org/jira/browse/TAJO-1952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15023618#comment-15023618 ]
ASF GitHub Bot commented on TAJO-1952: -------------------------------------- Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/846#discussion_r45691039 --- Diff: tajo-plan/src/main/java/org/apache/tajo/plan/partition/PartitionContent.java --- @@ -0,0 +1,69 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.plan.partition; + +import org.apache.hadoop.fs.Path; + +public class PartitionContent { + private Path[] partitionPaths; + private String[] partitionKeys; + private long totalVolume; + + public PartitionContent() { + } + + public PartitionContent(Path[] partitionPaths) { + this.partitionPaths = partitionPaths; + } + + public PartitionContent(Path[] partitionPaths, long totalVolume) { --- End diff -- unused constructor > Implement PartitionedFileFragment > --------------------------------- > > Key: TAJO-1952 > URL: https://issues.apache.org/jira/browse/TAJO-1952 > Project: Tajo > Issue Type: Improvement > Components: Planner/Optimizer, Storage > Reporter: Jaehwa Jung > Assignee: Jaehwa Jung > Fix For: 0.12.0, 0.11.1 > > Attachments: TAJO-1952.patch > > > Currently, PartitionedTableScanNode contains the list of partitions and it > seems to me that the list has some problems as following: > 1. Duplicate Informs: Task contains Fragment which specify target directory > or target file for scanning. A path of partition lists already would write to > Fragment. > 2. Network Resource: When scanning lost of partition, it will occupy network > resource, for example, several hundred kilobytes or more. It looks like an > unnecessary resource because Fragment already has the path of partitions. > I want to improve above problems by implementing new Fragment called > PartitionedFileFragment. Currently, I'm planning the implementation as > following: > * PartitionedFileFragment will borrow FileFragment and it contains the > partition path and the partition key values. > * Remove the path array of partitions from PartitionedTableScanNode. > * Implement a method for getting filtered partition directories in > FileTableSpace. > * Implement a method for making PartitionedFileFragment array. > * Before making splits, call above method and use it for making splits. -- This message was sent by Atlassian JIRA (v6.3.4#6332)