[
https://issues.apache.org/jira/browse/KYLIN-3634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16653068#comment-16653068
]
ASF GitHub Bot commented on KYLIN-3634:
---------------------------------------
codecov-io commented on issue #292: KYLIN-3634 when filter column has null
value may cause incorrect query result
URL: https://github.com/apache/kylin/pull/292#issuecomment-430509449
# [Codecov](https://codecov.io/gh/apache/kylin/pull/292?src=pr&el=h1) Report
> Merging [#292](https://codecov.io/gh/apache/kylin/pull/292?src=pr&el=desc)
into
[master](https://codecov.io/gh/apache/kylin/commit/5f6007ff2b5aead4781c876fa3203dd67188d90b?src=pr&el=desc)
will **decrease** coverage by `<.01%`.
> The diff coverage is `0%`.
[](https://codecov.io/gh/apache/kylin/pull/292?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #292 +/- ##
============================================
- Coverage 21.3% 21.29% -0.01%
Complexity 4444 4444
============================================
Files 1087 1087
Lines 69983 69986 +3
Branches 10109 10109
============================================
Hits 14907 14907
- Misses 53672 53677 +5
+ Partials 1404 1402 -2
```
| [Impacted
Files](https://codecov.io/gh/apache/kylin/pull/292?src=pr&el=tree) | Coverage Δ
| Complexity Δ | |
|---|---|---|---|
|
[...c/main/java/org/apache/kylin/gridtable/GTUtil.java](https://codecov.io/gh/apache/kylin/pull/292/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9ncmlkdGFibGUvR1RVdGlsLmphdmE=)
| `1.35% <0%> (-0.03%)` | `2 <0> (ø)` | |
|
[...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/292/diff?src=pr&el=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh)
| `70.21% <0%> (-0.61%)` | `7% <0%> (ø)` | |
|
[...g/apache/kylin/source/datagen/ColumnGenerator.java](https://codecov.io/gh/apache/kylin/pull/292/diff?src=pr&el=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc291cmNlL2RhdGFnZW4vQ29sdW1uR2VuZXJhdG9yLmphdmE=)
| `72.29% <0%> (+1.35%)` | `8% <0%> (ø)` | :arrow_down: |
------
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/kylin/pull/292?src=pr&el=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/kylin/pull/292?src=pr&el=footer). Last
update
[5f6007f...33181a7](https://codecov.io/gh/apache/kylin/pull/292?src=pr&el=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> when filter column has null value may cause incorrect query result
> ------------------------------------------------------------------
>
> Key: KYLIN-3634
> URL: https://issues.apache.org/jira/browse/KYLIN-3634
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: v2.0.0
> Reporter: WangBo
> Priority: Major
> Attachments:
> 0001-KYLIN-3634-when-filter-column-has-null-value-may-cau.patch
>
>
> h1. Question
> when a column has null value，and using it as a filter column when querying,
> and the filter value is not exist in the table,this may cause incorrect result
> h1. An Example
> h2. Table A
> the table A has three rows,city column of one row has null value
>
> ||day||...||city||price||
> |20180101| |null|10|
> |20180101| |beijing|20|
> |20180101| |shanghai|10|
> h2. Query SQL
> select day,sum(price) from a where city <> 'abc' group by day
> h2. Correct Result
> exclude the row contains null city value
> ||day||col||
> |20180101|30|
> h2. InCorrect Result
> resullt 0 rows
> this happens in our production environment,the kylin version is 2.0.0
> h1. Analysis process
> 1,city column dosen't have a value,so the CompareTupleFilter will turn into
> ConstantTupleFilter(see GTUtil.java)
> 2,if dimensions in the sql dosen't match all the columns using in group
> by,the bytesComparator used in hbase aggregation map will only compare the
> columns using in group by
> 3,when GTAggregateScanner constructs key of aggBufMap,the key may contains
> null value,because the comparator of aggBufMap only compares group by
> columns,so the tuple share same group by columns may also share the same keys
> which contains null value;This may cause kylin server receives tuples
> contains null value;
> 4,when the code which dynamically generated by calcilte deals tuples using
> filter,it first judges whether the column is null.Because filter column in
> the tuple contains null value,so it always return false, no tuples will
> return.
> h1. Solution
> when the filter column value is a invalid means not in the table,turn the
> CompareTupleFiter into IS_NOT_NULL filter,instead of ConstantTupleFilter.TURE
>
> Now I have test the feature in our production environment ;
> test in “mvn test” had passed，but not test in sandbox
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
