[
https://issues.apache.org/jira/browse/TAJO-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733400#comment-14733400
]
Tajo QA commented on TAJO-1493:
-------------------------------
{color:red}*-1 overall.*{color} Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12754454/TAJO-1493_4.patch
against master revision release-0.9.0-rc0-447-g33b8893.
{color:green}+1 @author.{color} The patch does not contain any @author
tags.
{color:green}+1 tests included.{color} The patch appears to include 5 new
or modified test files.
{color:green}+1 javac.{color} The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc.{color} The applied patch does not increase the
total number of javadoc warnings.
{color:green}+1 checkstyle.{color} The patch generated 0 code style errors.
{color:red}-1 findbugs.{color} The patch appears to cause Findbugs
(version 2.0.3) to fail.
{color:green}+1 release audit.{color} The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests.{color} The patch failed these unit tests in
tajo-catalog/tajo-catalog-client tajo-catalog/tajo-catalog-common
tajo-catalog/tajo-catalog-drivers/tajo-hive tajo-catalog/tajo-catalog-server
tajo-common tajo-core tajo-core-tests tajo-plan:
org.apache.tajo.engine.function.TestBuiltinFunctions
org.apache.tajo.engine.query.TestCreateTable
org.apache.tajo.engine.query.TestInnerJoinQuery
org.apache.tajo.engine.function.TestConditionalExpressions
org.apache.tajo.engine.query.TestInnerJoinWithSubQuery
org.apache.tajo.engine.query.TestInsertQuery
org.apache.tajo.engine.eval.TestSQLExpression
org.apache.tajo.engine.query.TestNullValues
org.apache.tajo.engine.query.TestJsonWithTimezone
org.apache.tajo.engine.query.TestOuterJoinQuery
org.apache.tajo.engine.query.TestCreateDatabase
org.apache.tajo.util.history.TestHistoryWriterReader
org.apache.tajo.engine.function.TestDateTimeFunctions
org.apache.tajo.engine.query.TestWindowQuery
org.apache.tajo.engine.query.TestOuterJoinWithSubQuery
org.apache.tajo.master.TestNonForwardQueryResultSystemScanner
org.apache.tajo.ws.rs.resources.TestQueryResultResource
org.apache.tajo.engine.function.TestStringOperatorsAndFunctions
org.apache.tajo.engine.eval.TestSQLDateTimeTypes
org.apache.tajo.benchmark.TestTPCH
org.apache.tajo.engine.planner.TestPlannerUtil
org.apache.tajo.cli.tsql.commands.TestExecExternalShellCommand
org.apache.tajo.ws.rs.resources.TestClusterResource
org.apache.tajo.client.TestQueryClientExceptions
org.apache.tajo.querymaster.TestQueryState
org.apache.tajo.ws.rs.resources.TestTablesResource
org.apache.tajo.engine.eval.TestPredicates
org.apache.tajo.engine.query.TestAlterTablespace
org.apache.tajo.ws.rs.resources.TestDatabasesResource
org.apache.tajo.cli.tsql.TestTajoCli
org.apache.tajo.engine.query.TestJoinOnPartitionedTables
org.apache.tajo.engine.query.TestTruncateTable
org.apache.tajo.engine.query.TestMultipleJoinTypes
org.apache.tajo.engine.query.TestSelectQuery
org.apache.tajo.engine.function.TestFunctionLoader
org.apache.tajo.engine.codegen.TestEvalCodeGenerator
org.apache.tajo.engine.query.TestInSubquery
org.apache.tajo.cli.tsql.TestDefaultCliOutputFormatter
org.apache.tajo.engine.function.TestMathFunctions
org.apache.tajo.engine.query.TestNetTypes
org.apache.tajo.engine.query.TestTablePartitions
org.apache.tajo.engine.eval.TestIntervalType
org.apache.tajo.master.TestQueryResult
org.apache.tajo.engine.query.TestIndexScan
org.apache.tajo.engine.planner.physical.TestSortExec
org.apache.tajo.engine.query.TestGroupByQuery
org.apache.tajo.engine.query.TestSetSessionQuery
org.apache.tajo.engine.query.TestSelectNestedRecord
org.apache.tajo.engine.function.TestJsonFunctions
org.apache.tajo.cli.tools.TestTajoDump
org.apache.tajo.engine.query.TestCreateIndex
org.apache.tajo.engine.query.TestTableSubQuery
org.apache.tajo.engine.planner.TestQueryValidation
org.apache.tajo.engine.query.TestCommonConditionReduce
org.apache.tajo.engine.query.TestDropTable
org.apache.tajo.cli.tsql.commands.TestHdfsCommand
org.apache.tajo.client.v2.TestTajoClientV2
org.apache.tajo.ws.rs.resources.TestFunctionsResource
org.apache.tajo.engine.query.TestHBaseTable
org.apache.tajo.engine.query.TestCrossJoin
org.apache.tajo.querymaster.TestTaskStatusUpdate
org.apache.tajo.querymaster.TestKillQuery
org.apache.tajo.client.TestTajoClient
org.apache.tajo.client.TestCatalogAdminClientExceptions
org.apache.tajo.engine.query.TestSimpleQuery
org.apache.tajo.ws.rs.resources.TestQueryResource
org.apache.tajo.engine.function.TestPatternMatchingPredicates
org.apache.tajo.engine.query.TestUnionQuery
org.apache.tajo.cli.tsql.TestTajoCliNegatives
org.apache.tajo.engine.function.TestPythonFunctions
org.apache.tajo.querymaster.TestQueryProgress
org.apache.tajo.engine.query.TestAlterTable
org.apache.tajo.engine.query.TestCaseByCases
org.apache.tajo.ws.rs.resources.TestSessionsResource
org.apache.tajo.engine.eval.TestEvalTree
org.apache.tajo.ha.TestHAServiceHDFSImpl
org.apache.tajo.engine.query.TestSortQuery
org.apache.tajo.engine.query.TestCTASQuery
Test results:
https://builds.apache.org/job/PreCommit-TAJO-Build/861//testReport/
Findbugs results:
https://builds.apache.org/job/PreCommit-TAJO-Build/861//findbugsResult
Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/861//console
This message is automatically generated.
> Add a method to get partition directories with filter conditions.
> -----------------------------------------------------------------
>
> Key: TAJO-1493
> URL: https://issues.apache.org/jira/browse/TAJO-1493
> Project: Tajo
> Issue Type: Sub-task
> Components: Catalog
> Reporter: Jaehwa Jung
> Assignee: Jaehwa Jung
> Fix For: 0.11.0, 0.12.0
>
> Attachments: TAJO-1493.patch, TAJO-1493_2.patch, TAJO-1493_3.patch,
> TAJO-1493_4.patch
>
>
> Currently, PartitionedTableRewriter take a look into partition directories
> for rewriting filter conditions. It get all sub directories of table path
> because catalog doesn’t provide partition directories. But if there are lots
> of sub directories on HDFS, such as, more than 10,000 directories, it might
> be cause overload to NameNode. Thus, CatalogStore need to provide partition
> directories for specified filter conditions. I designed new method to
> CatalogStore as follows:
> * method name: getPartitionsWithConditionFilters
> * first parameter: database name
> * second parameter: table name
> * third parameter: where clause (included target column name and partition
> value)
> * return values:
> List<org.apache.tajo.catalog.proto.CatalogProtos.TablePartitionProto>
> * description: It scan right partition directories on CatalogStore with where
> caluse.
> For examples, users set parameters as following:
> ** first parameter: default
> ** second parameter: table1
> ** third parameter: COLUMN_NAME = 'col1' AND PARTITION_VALUE = '3
> In the previous cases, this method will create select clause as follows.
> {code:xml}
> SELECT DISTINCT A.PATH
> FROM PARTITIONS A, (
> SELECT B.PARTITION_ID
> FROM PARTITION_KEYS B
> WHERE B.PARTITION_ID > 0
> AND (
> COLUMN_NAME = 'col1' AND PARTITION_VALUE = '3'
> )
> ) B
> WHERE A.PARTITION_ID > 0
> AND A.TID = ${table_id}
> AND A.PARTITION_ID = B.PARTITION_ID
> {code}
> At the first time, I considered to use EvalNode instead of where clause. But
> I can’t use it because of recursive related problems between tajo-catalog
> module and tajo-plan module. So, I’ll implement utility class to convert
> EvalNode to SQL.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
