[jira] [Updated] (KYLIN-2847) avoid doing useless work by checking query deadline

2017-10-18 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2847: - Attachment: KYLIN-2847.patch.1 patch updated, make "deadline" a local variable of CubeVisitService. > avo

[jira] [Commented] (KYLIN-2847) avoid doing useless work by checking query deadline

2017-10-10 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16199707#comment-16199707 ] Dayue Gao commented on KYLIN-2847: -- Thanks for the input, let me fix it. > avoid doing u

[jira] [Commented] (KYLIN-2847) avoid doing useless work by checking query deadline

2017-09-07 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156762#comment-16156762 ] Dayue Gao commented on KYLIN-2847: -- [~mahongbin] could you take a look and review the pat

[jira] [Updated] (KYLIN-2847) avoid doing useless work by checking query deadline

2017-09-05 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2847: - Attachment: KYLIN-2847.patch patch uploaded, please review > avoid doing useless work by checking query d

[jira] [Created] (KYLIN-2847) avoid doing useless work by checking query deadline

2017-09-05 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2847: Summary: avoid doing useless work by checking query deadline Key: KYLIN-2847 URL: https://issues.apache.org/jira/browse/KYLIN-2847 Project: Kylin Issue Type: Improve

[jira] [Updated] (KYLIN-2844) override max-visit-scanrange and max-fuzzykey-scan at cube level

2017-09-04 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2844: - Attachment: KYLIN-2844.patch patch uploaded, please review > override max-visit-scanrange and max-fuzzyke

[jira] [Created] (KYLIN-2844) override max-visit-scanrange and max-fuzzykey-scan at cube level

2017-09-04 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2844: Summary: override max-visit-scanrange and max-fuzzykey-scan at cube level Key: KYLIN-2844 URL: https://issues.apache.org/jira/browse/KYLIN-2844 Project: Kylin Issue

[jira] [Commented] (KYLIN-2577) Improve hive&mr job configuration

2017-05-03 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996049#comment-15996049 ] Dayue Gao commented on KYLIN-2577: -- As far as I know, SET statements can override configu

[jira] [Resolved] (KYLIN-2165) Use hive table statistics data to get the total count

2017-04-06 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2165. -- Resolution: Fixed https://github.com/apache/kylin/commit/7129ea650e757d6c50a81823760c45734688cfd2 pushe

[jira] [Updated] (KYLIN-2165) Use hive table statistics data to get the total count

2017-04-05 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2165: - Attachment: KYLIN-2165.BUGFIX.patch patch attached, please review > Use hive table statistics data to get

[jira] [Reopened] (KYLIN-2165) Use hive table statistics data to get the total count

2017-04-05 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao reopened KYLIN-2165: -- Hi [~Shaofengshi], After deploying this feature in our production env, I found two bugs # JobEngineConfig

[jira] [Resolved] (KYLIN-2192) More Robust Global Dictionary

2017-03-29 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2192. -- Resolution: Fixed Future works are being done in KYLIN-2506. > More Robust Global Dictionary >

[jira] [Updated] (KYLIN-2501) Stream Aggregate GTRecords at Query Server

2017-03-22 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2501: - Attachment: KYLIN-2501.patch Pass CI,patch uploaded. > Stream Aggregate GTRecords at Query Server > -

[jira] [Commented] (KYLIN-2501) Stream Aggregate GTRecords at Query Server

2017-03-16 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929370#comment-15929370 ] Dayue Gao commented on KYLIN-2501: -- Code pushed to KYLIN-2501 branch, ready to be review.

[jira] [Commented] (KYLIN-2483) SortedIteratorMergerWithLimit could be slower when number of total merge rows is small

2017-03-10 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15906107#comment-15906107 ] Dayue Gao commented on KYLIN-2483: -- Hi [~mahongbin], I think the overhead could come from

[jira] [Updated] (KYLIN-2501) Stream Aggregate GTRecords at Query Server

2017-03-10 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2501: - Description: *Problem* When query server needs to handle millions of records, CubeTupleConverter could be

[jira] [Created] (KYLIN-2501) Stream Aggregate GTRecords at Query Server

2017-03-10 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2501: Summary: Stream Aggregate GTRecords at Query Server Key: KYLIN-2501 URL: https://issues.apache.org/jira/browse/KYLIN-2501 Project: Kylin Issue Type: Improvement

[jira] [Resolved] (KYLIN-2387) A new BitmapCounter with better performance

2017-03-07 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2387. -- Resolution: Fixed Fix Version/s: v2.0.0 > A new BitmapCounter with better performance > -

[jira] [Resolved] (KYLIN-2457) Should copy the latest dictionaries on dimension tables in a batch merge job

2017-02-28 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2457. -- Resolution: Fixed Assignee: zhengdong > Should copy the latest dictionaries on dimension tables in

[jira] [Updated] (KYLIN-2457) Should copy the latest dictionaries on dimension tables in a batch merge job

2017-02-28 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2457: - Fix Version/s: v2.0.0 > Should copy the latest dictionaries on dimension tables in a batch merge job > ---

[jira] [Updated] (KYLIN-2457) Should copy the latest dictionaries on dimension tables in a batch merge job

2017-02-28 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2457: - Affects Version/s: v1.6.0 > Should copy the latest dictionaries on dimension tables in a batch merge job >

[jira] [Commented] (KYLIN-2457) Should copy the latest dictionaries on dimension tables in a batch merge job

2017-02-28 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887881#comment-15887881 ] Dayue Gao commented on KYLIN-2457: -- Merged to master https://github.com/apache/kylin/com

[jira] [Commented] (KYLIN-2457) Should copy the latest dictionaries on dimension tables in a batch merge job

2017-02-22 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878043#comment-15878043 ] Dayue Gao commented on KYLIN-2457: -- +1. Hi [~zhengd], it would be better if you update th

[jira] [Commented] (KYLIN-2451) Set HBASE_RPC_TIMEOUT according to kylin.storage.hbase.coprocessor-timeout-seconds

2017-02-20 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874279#comment-15874279 ] Dayue Gao commented on KYLIN-2451: -- LGTM 👍 > Set HBASE_RPC_TIMEOUT according to > kylin

[jira] [Commented] (KYLIN-2451) Set HBASE_RPC_TIMEOUT according to kylin.storage.hbase.coprocessor-timeout-seconds

2017-02-20 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874211#comment-15874211 ] Dayue Gao commented on KYLIN-2451: -- Hi [~liyang.g...@gmail.com], I see what's the differ

[jira] [Commented] (KYLIN-2451) Set HBASE_RPC_TIMEOUT according to kylin.storage.hbase.coprocessor-timeout-seconds

2017-02-16 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871127#comment-15871127 ] Dayue Gao commented on KYLIN-2451: -- Hi [~liyang.g...@gmail.com], how is this possible? Ha

[jira] [Resolved] (KYLIN-2443) Report coprocessor error information back to client

2017-02-14 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2443. -- Resolution: Fixed Fix Version/s: v2.0.0 > Report coprocessor error information back to client > -

[jira] [Commented] (KYLIN-2443) Report coprocessor error information back to client

2017-02-11 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862616#comment-15862616 ] Dayue Gao commented on KYLIN-2443: -- Commit https://github.com/apache/kylin/commit/43c056

[jira] [Created] (KYLIN-2443) Report coprocessor error information back to client

2017-02-11 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2443: Summary: Report coprocessor error information back to client Key: KYLIN-2443 URL: https://issues.apache.org/jira/browse/KYLIN-2443 Project: Kylin Issue Type: Improve

[jira] [Resolved] (KYLIN-2438) replace scan threshold with max scan bytes

2017-02-10 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2438. -- Resolution: Fixed Fix Version/s: v2.0.0 commit https://github.com/apache/kylin/commit/09a086688a

[jira] [Updated] (KYLIN-2437) collect number of bytes scanned to query metrics

2017-02-09 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2437: - Fix Version/s: v2.0.0 > collect number of bytes scanned to query metrics > ---

[jira] [Resolved] (KYLIN-2437) collect number of bytes scanned to query metrics

2017-02-09 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2437. -- Resolution: Fixed commit https://github.com/apache/kylin/commit/e09338b34c0b07a7167096e45bf9185aa0d0cbd

[jira] [Resolved] (KYLIN-2436) add a configuration knob to disable spilling of aggregation cache

2017-02-09 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2436. -- Resolution: Fixed Fix Version/s: v2.0.0 > add a configuration knob to disable spilling of aggrega

[jira] [Commented] (KYLIN-2436) add a configuration knob to disable spilling of aggregation cache

2017-02-09 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860703#comment-15860703 ] Dayue Gao commented on KYLIN-2436: -- commit https://github.com/apache/kylin/commit/ecf6a6

[jira] [Updated] (KYLIN-2438) replace scan threshold with max scan bytes

2017-02-08 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2438: - Description: In order to guard against bad queries that can consume lots of memory and potentially crash

[jira] [Updated] (KYLIN-2438) replace scan threshold with max scan bytes

2017-02-08 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2438: - Description: In order to guard against bad queries that can consume lots of memory and potentially crash

[jira] [Updated] (KYLIN-2438) replace scan threshold with max scan bytes

2017-02-08 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2438: - Description: In order to guard against bad queries that can consume lots of memory and potentially crash

[jira] [Created] (KYLIN-2438) replace scan threshold with max scan bytes

2017-02-08 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2438: Summary: replace scan threshold with max scan bytes Key: KYLIN-2438 URL: https://issues.apache.org/jira/browse/KYLIN-2438 Project: Kylin Issue Type: Improvement

[jira] [Updated] (KYLIN-2436) add a configuration knob to disable spilling of aggregation cache

2017-02-08 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2436: - Description: Kylin's aggregation operator can spill intermediate results to disk when its estimated memor

[jira] [Created] (KYLIN-2437) collect number of bytes scanned to query metrics

2017-02-08 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2437: Summary: collect number of bytes scanned to query metrics Key: KYLIN-2437 URL: https://issues.apache.org/jira/browse/KYLIN-2437 Project: Kylin Issue Type: Improvemen

[jira] [Created] (KYLIN-2436) add a configuration knob to disable spilling of aggregation cache

2017-02-07 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2436: Summary: add a configuration knob to disable spilling of aggregation cache Key: KYLIN-2436 URL: https://issues.apache.org/jira/browse/KYLIN-2436 Project: Kylin Issu

[jira] [Closed] (KYLIN-1455) HBase ScanMetrics are not properly logged in query log

2017-02-07 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao closed KYLIN-1455. Resolution: Won't Fix > HBase ScanMetrics are not properly logged in query log > ---

[jira] [Resolved] (KYLIN-2058) Make Kylin more resilient to bad queries

2017-02-07 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2058. -- Resolution: Fixed Fix Version/s: v1.6.0 There are still works to be done to defend Kylin against

[jira] [Resolved] (KYLIN-2412) Unclosed DataOutputStream in RoaringBitmapCounter#write()

2017-01-21 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2412. -- Resolution: Fixed commit https://github.com/apache/kylin/commit/d264339b1c16c195ffafc2217b793d81bdbd643

[jira] [Commented] (KYLIN-2412) Unclosed DataOutputStream in RoaringBitmapCounter#write()

2017-01-21 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833305#comment-15833305 ] Dayue Gao commented on KYLIN-2412: -- Hi [~tedyu], since the target of the stream is a Byte

[jira] [Commented] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-18 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15829485#comment-15829485 ] Dayue Gao commented on KYLIN-2387: -- Choosing mutable or immutable bitmaps is implementati

[jira] [Commented] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-17 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15825814#comment-15825814 ] Dayue Gao commented on KYLIN-2387: -- Commit https://github.com/apache/kylin/commit/e89446

[jira] [Commented] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-17 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15825749#comment-15825749 ] Dayue Gao commented on KYLIN-2387: -- OK, I'll remove the usage of ImmutableRoaringBitmap.b

[jira] [Commented] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-17 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15825703#comment-15825703 ] Dayue Gao commented on KYLIN-2387: -- ImmutableRoaringBitmap.bitmapOf is only used in test,

[jira] [Closed] (KYLIN-2398) CubeSegmentScanner generated inaccurate

2017-01-15 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao closed KYLIN-2398. Resolution: Duplicate Fix Version/s: (was: Future) > CubeSegmentScanner generated inaccurate > --

[jira] [Closed] (KYLIN-2399) CubeSegmentScanner generated inaccurate

2017-01-15 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao closed KYLIN-2399. Resolution: Duplicate > CubeSegmentScanner generated inaccurate > --- >

[jira] [Updated] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2387: - Description: We found the old BitmapCounter does not perform very well on very large bitmap. The ineffici

[jira] [Updated] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2387: - Description: We found the old BitmapCounter does not perform very well on very large bitmap. The ineffici

[jira] [Commented] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15821911#comment-15821911 ] Dayue Gao commented on KYLIN-2387: -- commit https://github.com/apache/kylin/commit/d19533

[jira] [Updated] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2387: - Description: We found the old BitmapCounter does not perform very well on very large bitmap. The ineffici

[jira] [Updated] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2387: - Description: We found the old BitmapCounter does not perform very well on very large bitmap. The ineffici

[jira] [Updated] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2387: - Description: We found the old BitmapCounter does not perform very well on very large bitmap. The ineffici

[jira] [Updated] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2387: - Description: We found the old BitmapCounter does not perform very well on very large bitmap. The ineffici

[jira] [Updated] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2387: - Description: We found the old BitmapCounter does not perform very well on very large bitmap. The ineffici

[jira] [Updated] (KYLIN-2387) A new BitmapCounter with better performance

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2387: - Summary: A new BitmapCounter with better performance (was: Improve BitmapCounter performance by avoiding

[jira] [Resolved] (KYLIN-2386) Revert KYLIN-2349 and KYLIN-2353

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2386. -- Resolution: Fixed Fix Version/s: v2.0.0 commit https://github.com/apache/kylin/commit/4b97721518

[jira] [Updated] (KYLIN-2386) Revert KYLIN-2349 and KYLIN-2353

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2386: - Description: In KYLIN-2349 and KYLIN-2353, we changed the storage format of BitmapCounter for better perf

[jira] [Created] (KYLIN-2387) Improve BitmapCounter performance by avoiding expensive deserialize

2017-01-13 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2387: Summary: Improve BitmapCounter performance by avoiding expensive deserialize Key: KYLIN-2387 URL: https://issues.apache.org/jira/browse/KYLIN-2387 Project: Kylin Is

[jira] [Updated] (KYLIN-2386) Revert KYLIN-2349 and KYLIN-2353

2017-01-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2386: - Description: In KYLIN-2349 and KYLIN-2353, we optimized performance of BitmapCounter by changing its stor

[jira] [Created] (KYLIN-2386) Revert KYLIN-2349 and KYLIN-2353

2017-01-13 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2386: Summary: Revert KYLIN-2349 and KYLIN-2353 Key: KYLIN-2386 URL: https://issues.apache.org/jira/browse/KYLIN-2386 Project: Kylin Issue Type: Task Components:

[jira] [Resolved] (KYLIN-2328) Reduce the size of metadata uploaded to distributed cache

2016-12-29 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2328. -- Resolution: Fixed commit to master, thanks for reviewing! https://github.com/apache/kylin/commit/6ea03b

[jira] [Updated] (KYLIN-2328) Reduce the size of metadata uploaded to distributed cache

2016-12-28 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2328: - Attachment: KYLIN-2328.patch Patch uploaded. Have tested batch build and merge. It seems that KafkaFlatT

[jira] [Updated] (KYLIN-2328) Reduce the size of metadata uploaded to distributed cache

2016-12-27 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2328: - Description: Currently, each MR job uploads all the metadata belonging to a cube to distributed cache. Wh

[jira] [Created] (KYLIN-2328) Reduce the size of metadata uploaded to distributed cache

2016-12-27 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2328: Summary: Reduce the size of metadata uploaded to distributed cache Key: KYLIN-2328 URL: https://issues.apache.org/jira/browse/KYLIN-2328 Project: Kylin Issue Type: I

[jira] [Commented] (KYLIN-2266) Reduce memory usage for building global dict

2016-12-11 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15739860#comment-15739860 ] Dayue Gao commented on KYLIN-2266: -- A better way would be refine AppendTrieDictionary to

[jira] [Commented] (KYLIN-2192) More Robust Global Dictionary

2016-12-09 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15735518#comment-15735518 ] Dayue Gao commented on KYLIN-2192: -- Found a bug in AppendTrieDictionary, which cause Kyli

[jira] [Commented] (KYLIN-2253) sql union did not remove duplicated records (distinct)

2016-12-08 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15731699#comment-15731699 ] Dayue Gao commented on KYLIN-2253: -- Hi [~zhoudegao], it's due to a bug described in CALCI

[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"

2016-11-28 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15704027#comment-15704027 ] Dayue Gao commented on KYLIN-2180: -- {quote} Find a project by CubeDesc is already very st

[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"

2016-11-26 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15699198#comment-15699198 ] Dayue Gao commented on KYLIN-2180: -- The problem here is we don't know which project a Cub

[jira] [Commented] (KYLIN-2221) rethink on KYLIN-1684

2016-11-21 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15685855#comment-15685855 ] Dayue Gao commented on KYLIN-2221: -- +1. If we can improve the way how empty segment is d

[jira] [Commented] (KYLIN-2173) push down limit leads to wrong answer when filter is loosened

2016-11-19 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15678999#comment-15678999 ] Dayue Gao commented on KYLIN-2173: -- BTW, could we build a shared CI infrastructure? Runni

[jira] [Commented] (KYLIN-2173) push down limit leads to wrong answer when filter is loosened

2016-11-19 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15678992#comment-15678992 ] Dayue Gao commented on KYLIN-2173: -- {quote} how about the test result? {quote} Problem sh

[jira] [Commented] (KYLIN-2200) CompileException on UNION ALL query when result only contains one column

2016-11-19 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15678953#comment-15678953 ] Dayue Gao commented on KYLIN-2200: -- Thanks Julian. I'll figure out whether it's a calcite

[jira] [Updated] (KYLIN-2200) CompileException on UNION ALL query when result only contains one column

2016-11-17 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao updated KYLIN-2200: - Attachment: KYLIN-2200.patch patch uploaded. There is no ITs for UNION/UNION_ALL right now. Let me add t

[jira] [Commented] (KYLIN-2200) CompileException on UNION ALL query when result only contains one column

2016-11-16 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15673043#comment-15673043 ] Dayue Gao commented on KYLIN-2200: -- Union also fails to remove duplicates. {code:sql} sel

[jira] [Commented] (KYLIN-2200) CompileException on UNION ALL query when result only contains one column

2016-11-16 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15672620#comment-15672620 ] Dayue Gao commented on KYLIN-2200: -- Still got problem on the following query {code:sql} s

[jira] [Commented] (KYLIN-2200) CompileException on UNION ALL query when result only contains one column

2016-11-16 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669874#comment-15669874 ] Dayue Gao commented on KYLIN-2200: -- The root cause is for the second OLAPTableScan, it re

[jira] [Commented] (KYLIN-2200) CompileException on UNION ALL query when result only contains one column

2016-11-16 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669839#comment-15669839 ] Dayue Gao commented on KYLIN-2200: -- I believe it's not a bug in calcite. Because OLAPTabl

[jira] [Commented] (KYLIN-2200) CompileException on UNION ALL query when result only contains one column

2016-11-16 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15669833#comment-15669833 ] Dayue Gao commented on KYLIN-2200: -- Here is calcite generated code {code:java} /* 1 */

[jira] [Created] (KYLIN-2200) CompileException on UNION ALL query when result only contains one column

2016-11-16 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2200: Summary: CompileException on UNION ALL query when result only contains one column Key: KYLIN-2200 URL: https://issues.apache.org/jira/browse/KYLIN-2200 Project: Kylin

[jira] [Commented] (KYLIN-2173) push down limit leads to wrong answer when filter is loosened

2016-11-14 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663899#comment-15663899 ] Dayue Gao commented on KYLIN-2173: -- commit to master and v1.6.0-rc2. UT seems to be broke

[jira] [Commented] (KYLIN-1609) Push down undefined Count Distinct aggregation to storage

2016-11-09 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15650757#comment-15650757 ] Dayue Gao commented on KYLIN-1609: -- Hi [~lidong_sjtu], is there further plan on this? Wha

[jira] [Created] (KYLIN-2173) push down limit leads to wrong answer when filter is loosened

2016-11-09 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2173: Summary: push down limit leads to wrong answer when filter is loosened Key: KYLIN-2173 URL: https://issues.apache.org/jira/browse/KYLIN-2173 Project: Kylin Issue Ty

[jira] [Commented] (KYLIN-2159) Redistribution Hive Table Step always requires row_count filename as 000000_0

2016-11-05 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15639870#comment-15639870 ] Dayue Gao commented on KYLIN-2159: -- We also run into this problem once, should find targe

[jira] [Commented] (KYLIN-2085) PrepareStatement return incorrect result in some cases

2016-11-01 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15625281#comment-15625281 ] Dayue Gao commented on KYLIN-2085: -- Encountered the same problem after upgrading to 1.5.4

[jira] [Resolved] (KYLIN-2079) add explicit configuration knob for coprocessor timeout

2016-10-30 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2079. -- Resolution: Fixed Committed to master and v1.6.0-rc1. The default coprocessor timeout is set to (hbase.

[jira] [Commented] (KYLIN-2079) add explicit configuration knob for coprocessor timeout

2016-10-30 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15619993#comment-15619993 ] Dayue Gao commented on KYLIN-2079: -- [~mahongbin] , the way Kylin avoids retrying coproces

[jira] [Resolved] (KYLIN-2105) add QueryId

2016-10-26 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2105. -- Resolution: Fixed change to use UUID as query id > add QueryId > --- > > Key: K

[jira] [Commented] (KYLIN-2105) add QueryId

2016-10-20 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15591776#comment-15591776 ] Dayue Gao commented on KYLIN-2105: -- [~liyang.g...@gmail.com], thank you for your suggesti

[jira] [Resolved] (KYLIN-2115) some extended column query returns wrong answer

2016-10-20 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2115. -- Resolution: Fixed Fix Version/s: v1.6.0 fixed in https://github.com/apache/kylin/commit/bec4a888

[jira] [Created] (KYLIN-2115) some extended column query returns wrong answer

2016-10-20 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2115: Summary: some extended column query returns wrong answer Key: KYLIN-2115 URL: https://issues.apache.org/jira/browse/KYLIN-2115 Project: Kylin Issue Type: Bug

[jira] [Commented] (KYLIN-2105) add QueryId

2016-10-18 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15587203#comment-15587203 ] Dayue Gao commented on KYLIN-2105: -- commit https://github.com/apache/kylin/commit/db09f5

[jira] [Created] (KYLIN-2105) add QueryId

2016-10-18 Thread Dayue Gao (JIRA)
Dayue Gao created KYLIN-2105: Summary: add QueryId Key: KYLIN-2105 URL: https://issues.apache.org/jira/browse/KYLIN-2105 Project: Kylin Issue Type: Sub-task Components: Query Engine, St

[jira] [Comment Edited] (KYLIN-2012) more robust approach to hive schema changes

2016-10-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571511#comment-15571511 ] Dayue Gao edited comment on KYLIN-2012 at 10/13/16 10:17 AM: -

[jira] [Commented] (KYLIN-2012) more robust approach to hive schema changes

2016-10-13 Thread Dayue Gao (JIRA)
[ https://issues.apache.org/jira/browse/KYLIN-2012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15571511#comment-15571511 ] Dayue Gao commented on KYLIN-2012: -- oops, didn't think about the migration. Please revert

  1   2   >