[
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16141221#comment-16141221
]
Hadoop QA commented on PHOENIX-418:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12883649/PHOENIX-418-v6.patch
against master branch at commit dd008f4fe4e67836a6fb362843c9618e4e518182.
ATTACHMENT ID: 12883649
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
56 warning messages.
{color:red}-1 release audit{color}. The applied patch generated 2 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+ String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+ String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+ String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+ String query = "SELECT
APPROX_COUNT_DISTINCT(ORGANIZATION_ID||A_STRING||B_STRING) FROM " + tableName;
+ String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+ String query = "SELECT
APPROX_COUNT_DISTINCT(a.ORGANIZATION_ID||a.A_STRING||b.ENTITY_ID) FROM "
+ + tableName + " a, "+ tableName + " b where
a.ORGANIZATION_ID=b.ORGANIZATION_ID and a.ENTITY_ID = b.ENTITY_ID";
+ String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+ String query = "explain SELECT
APPROX_COUNT_DISTINCT(ORGANIZATION_ID||ENTITY_ID) FROM " + tableName;
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.salted.SaltedTableVarLengthRowKeyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TransactionalViewIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.OrderByIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.FirstValuesFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StatementHintsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PowerFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.InListIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.IsNullIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.txn.RollbackIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DynamicUpsertIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryExecWithoutSCNIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SubqueryUsingSortMergeJoinIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.ChildViewsUseParentViewIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.EvaluationOfORIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ConvertTimezoneFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AutoPartitionViewsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StringIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AppendOnlySchemaIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.InstrFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.HashJoinMoreIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ReverseFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ReadOnlyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DisableLocalIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StoreNullsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CountDistinctApproximateHyperLogLogIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UseSchemaIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SkipScanQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MinMaxAggregateFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SignFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.OnDuplicateKeyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SerialIteratorsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.BinaryRowKeyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MD5FunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterSessionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayToStringFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayAppendFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterTableIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PercentileIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.trace.PhoenixTableMetricsWriterIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TenantIdTypeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.IndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.KeyOnlyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.IndexMetadataIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpsertSelectAutoCommitIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.ImmutableIndexWithStatsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SkipScanAfterManualSplitIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.util.IndexScrutinyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterMultiTenantTableWithViewsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.iterate.PhoenixQueryTimeoutIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ColumnProjectionOptimizationIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DynamicFamilyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DeleteIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AutoCommitIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.FirstValueFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpsertSelectIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CastAndCoerceIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MappingTableDataTypeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DecodeFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.tx.TxCheckpointIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.SaltedIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StoreNullsPropIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RegexpSplitFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.ViewIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StringToArrayFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.txn.MutableRollbackIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AbsFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MultiCfQueryExecIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MutableQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PrimitiveTypeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.tx.ParameterizedTransactionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ServerExceptionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryWithTableSampleIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StddevIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ExecuteStatementsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ExpFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.GlobalIndexOptimizationIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ParallelIteratorsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.rpc.UpdateCacheIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DefaultColumnValueIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpperLowerFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UnionAllIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.IndexExpressionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ModulusExpressionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TenantSpecificTablesDDLIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CoalesceFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexSplitForwardScanIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CSVCommonsLoaderIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.OctetLengthFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ConcurrentMutationsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AggregateQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.iterate.RoundRobinResultIteratorIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CbrtFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TimezoneOffsetFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.LnLogFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SpooledTmpFileDeleteIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.GroupByIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RegexpReplaceFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SaltedViewIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TenantSpecificViewIndexSaltedIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryMoreIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterTableWithViewsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ViewIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.FlappingAlterTableIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.coprocessor.StatisticsCollectionRunTrackerIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.NthValueFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.tx.FlappingTransactionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MapReduceIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AggregateIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SortMergeJoinMoreIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayPrependFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArraysWithNullsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ImmutableTablePropertiesIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.salted.SaltedTableUpsertSelectIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ToCharFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.LastValueFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TableSnapshotReadsMapReduceIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ToDateFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.LastValuesFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ExplainPlanWithStatsEnabledIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SortMergeJoinIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PhoenixRuntimeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpsertBigValuesIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.DropColumnIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RoundFloorCeilFuncIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayConcatFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.NamespaceSchemaMappingIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.AsyncIndexDisabledIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ScanQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.trace.PhoenixTracingEndToEndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexSplitReverseScanIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ColumnEncodedBytesPropIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArrayFillFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SqrtFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ArithmeticQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CaseStatementIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RegexpSubstrFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.LikeExpressionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SortOrderIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.tx.TransactionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.NotQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.HashJoinLocalIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.OutOfOrderMutationsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DynamicColumnIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ReverseScanIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.EncodeFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ClientTimeArithmeticQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PointInTimeQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.GetSetByteBitFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TenantSpecificTablesDMLIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SubqueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CursorWithRowValueConstructorIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DistinctPrefixFilterIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.DropMetadataIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TenantSpecificViewIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DateTimeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryWithOffsetIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.UpgradeIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RTrimFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.HashJoinIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.ExplainPlanWithStatsDisabledIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1297//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1297//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1297//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1297//console
This message is automatically generated.
> Support approximate COUNT DISTINCT
> ----------------------------------
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
> Issue Type: Task
> Reporter: James Taylor
> Assignee: Ethan Wang
> Labels: gsoc2016
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch,
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch,
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to
> all distinct values (since this will not scale well when the number of
> distinct values is huge). The Apache Drill folks have had some interesting
> discussions on this
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
> They recommend using [Welford's
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
> I'm open to having a config option that uses exact versus approximate. I
> don't have experience implementing an approximate implementation, so I'm not
> sure how much state is required to keep on the server and return to the
> client (other than realizing it'd be much less that returning all distinct
> values and their counts).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
