[
https://issues.apache.org/jira/browse/UNOMI-1018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18132381#comment-18132381
]
Serge Huber commented on UNOMI-1018:
------------------------------------
Fixed by https://github.com/apache/unomi/pull/926, merged to master as
b5e1bddbc.
*Results (both engines)*
* Aggregate results now include the buckets of every aggregate type Unomi
builds: string, long and double terms, date histograms, and numeric, date and
IP ranges.
* Range buckets were missing on Elasticsearch as well as OpenSearch. OpenSearch
also missed date histograms and long terms, and Elasticsearch reported terms on
a numeric field under a null key.
* IP range requests cannot carry a key on either client, so IP buckets are
reported under the key of the IP range whose bounds they match. IPv6 bounds are
compared as addresses.
*Requests (OpenSearch)*
* The date histogram now accepts the short interval format (1M, the default, or
1d).
* Date ranges without a lower or upper bound no longer fail.
* An IP range's key is no longer sent as its CIDR mask.
* A numeric range without bounds is added, as on Elasticsearch, instead of
being skipped.
*Shared code*
* The rules that turn engine buckets into result keys live in one place,
{{AggregateBuckets}} in persistence-spi, so both engines report buckets the
same way.
* Buckets that share a key are summed. A bucket named {{_all}}, {{_filtered}}
or {{_missing}} is left out so it cannot change those counts.
*Tests*
* PersistenceServiceIT covers date histogram, numeric terms, numeric and date
ranges, and the other-tenant {{_all}} check from UNOMI-1014.
* Unit tests cover the shared key rules and each engine's bucket reader against
response JSON parsed by its client, including double terms and IP ranges, which
no IT can reach.
* CI passed on the merged commit, including both integration test suites.
> OpenSearch aggregate queries miss date and range buckets
> --------------------------------------------------------
>
> Key: UNOMI-1018
> URL: https://issues.apache.org/jira/browse/UNOMI-1018
> Project: Apache Unomi
> Issue Type: Bug
> Affects Versions: unomi-4.0.0
> Reporter: Serge Huber
> Priority: Major
> Fix For: unomi-4.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h3. Summary
> On OpenSearch, an aggregate query that uses a date histogram or a numeric,
> date or IP range returns no value buckets. The same query on Elasticsearch
> returns the expected buckets. Terms aggregates (counts by a string property)
> work on both engines.
> The OpenSearch persistence layer builds these aggregations, but when it reads
> the answer it only handles string-terms buckets. Other bucket types are
> ignored, so the map of results has no entries for the date or range keys.
> The goal of this ticket is that date and range aggregates return their
> buckets on OpenSearch the same way they do on Elasticsearch.
> h3. Ways to explore
> * When reading the aggregation answer, handle the same bucket kinds
> Elasticsearch already handles (for example date histogram, long terms, and
> the range types Unomi supports).
> * Add integration tests that run on both search engines for a date aggregate
> and for a numeric range aggregate, and check the bucket keys and counts.
> h3. How to test
> On Elasticsearch and on OpenSearch, create a few profiles with known dates or
> numeric properties, then send a date-histogram aggregate and a numeric-range
> aggregate. Check that the answer lists the expected buckets and counts on
> both engines. A terms aggregate must still work.
> {*}Correction{*}: numeric, date and IP range aggregates also return no
> buckets on Elasticsearch, because its result reader only handles multi-terms,
> string terms, long terms and date histograms. The range fix is needed on both
> engines. Date histogram and long terms are missing on OpenSearch only.
> Also in scope (from UNOMI-1014): in
> PersistenceServiceIT.testAggregateAndMetricsStayInTheCurrentTenant, add a
> check that the other tenant sees only its own items: inside
> executeAsTenant(otherTenant), aggregateQuery(null, byFirstName,
> Profile.ITEM_TYPE).get("_all") must equal 2.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
