[jira] [Resolved] (KYLIN-2670) CASE WHEN supporting problem in kylin2.0
[ https://issues.apache.org/jira/browse/KYLIN-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2670. -- Resolution: Fixed Fix Version/s: v2.1.0 > CASE WHEN supporting problem in kylin2.0 > > > Key: KYLIN-2670 > URL: https://issues.apache.org/jira/browse/KYLIN-2670 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.0.0 >Reporter: zhou degao >Assignee: liyang > Labels: focus, scope > Fix For: v2.1.0 > > > Following query failed in kylin 2.0 but succeeded in kylin 1.6 > select "fact_pv_data_alias"."PRODUCT_NAME" as "c0", > "fact_pv_data_alias"."PLATFORM" as "c1" from "CSDNBI"."FACT_PV_DATA" as > "fact_pv_data_alias" group by "fact_pv_data_alias"."PRODUCT_NAME", > "fact_pv_data_alias"."PLATFORM" order by CASE WHEN > "fact_pv_data_alias"."PRODUCT_NAME" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PRODUCT_NAME" ASC, CASE WHEN > "fact_pv_data_alias"."PLATFORM" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PLATFORM" ASC > Reported error in kylin 2.0: > Error while executing SQL "select "fact_pv_data_alias"."PRODUCT_NAME" as > "c0", "fact_pv_data_alias"."PLATFORM" as "c1" from "CSDNBI"."FACT_PV_DATA" as > "fact_pv_data_alias" group by "fact_pv_data_alias"."PRODUCT_NAME", > "fact_pv_data_alias"."PLATFORM" order by CASE WHEN > "fact_pv_data_alias"."PRODUCT_NAME" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PRODUCT_NAME" ASC, CASE WHEN > "fact_pv_data_alias"."PLATFORM" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PLATFORM" ASC LIMIT 5": index (2) must be less than > size (2) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2670) CASE WHEN supporting problem in kylin2.0
[ https://issues.apache.org/jira/browse/KYLIN-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113759#comment-16113759 ] Billy Liu commented on KYLIN-2670: -- Verified on Kylin 2.1 branch. > CASE WHEN supporting problem in kylin2.0 > > > Key: KYLIN-2670 > URL: https://issues.apache.org/jira/browse/KYLIN-2670 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.0.0 >Reporter: zhou degao >Assignee: liyang > Labels: focus, scope > > Following query failed in kylin 2.0 but succeeded in kylin 1.6 > select "fact_pv_data_alias"."PRODUCT_NAME" as "c0", > "fact_pv_data_alias"."PLATFORM" as "c1" from "CSDNBI"."FACT_PV_DATA" as > "fact_pv_data_alias" group by "fact_pv_data_alias"."PRODUCT_NAME", > "fact_pv_data_alias"."PLATFORM" order by CASE WHEN > "fact_pv_data_alias"."PRODUCT_NAME" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PRODUCT_NAME" ASC, CASE WHEN > "fact_pv_data_alias"."PLATFORM" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PLATFORM" ASC > Reported error in kylin 2.0: > Error while executing SQL "select "fact_pv_data_alias"."PRODUCT_NAME" as > "c0", "fact_pv_data_alias"."PLATFORM" as "c1" from "CSDNBI"."FACT_PV_DATA" as > "fact_pv_data_alias" group by "fact_pv_data_alias"."PRODUCT_NAME", > "fact_pv_data_alias"."PLATFORM" order by CASE WHEN > "fact_pv_data_alias"."PRODUCT_NAME" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PRODUCT_NAME" ASC, CASE WHEN > "fact_pv_data_alias"."PLATFORM" IS NULL THEN 1 ELSE 0 END, > "fact_pv_data_alias"."PLATFORM" ASC LIMIT 5": index (2) must be less than > size (2) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2764) Build the dict for UHC column with MR
[ https://issues.apache.org/jira/browse/KYLIN-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112386#comment-16112386 ] liyang commented on KYLIN-2764: --- Could simplify FactDistinctColumns a bit along the way. :-) > Build the dict for UHC column with MR > - > > Key: KYLIN-2764 > URL: https://issues.apache.org/jira/browse/KYLIN-2764 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v2.0.0 >Reporter: kangkaisen >Assignee: kangkaisen > > KYLIN-2217 has built dict for normal column with MR, but the UHC column > still build dict in JobServer. Like KYLIN-2217, we also could use MR build > dict for UHC column. which could thoroughly release the memory pressure and > improve job concurrent for JobServer as well as speed up multi UHC columns > procedure. > The MR input is the output of "Extract Fact Table Distinct Columns", the MR > output is the UHC column dict. Because it is very hard build global dict with > multi reducers, I use one reducer handle one UHC column and allocate enough > memory to the reducer. According to my test, 8G memory is enough. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KYLIN-2777) Should throw error when sql contains non-equi left join
Roger Shi created KYLIN-2777: Summary: Should throw error when sql contains non-equi left join Key: KYLIN-2777 URL: https://issues.apache.org/jira/browse/KYLIN-2777 Project: Kylin Issue Type: Bug Reporter: Roger Shi SQL like: select fact_table.A, lookup_table.B from fact_table fa left join lookup_table lo on fa.C = lo.D and fa.E = 'asdf' The join is non-equi left join. It's not support in Kylin data model, should throw error instead of ignore it. Current implementation leads to wrong result instead of clear error message. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2776) Using dropwizard as default metric framework
[ https://issues.apache.org/jira/browse/KYLIN-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yiming.xu updated KYLIN-2776: - Attachment: KYLIN-2776.patch add patch > Using dropwizard as default metric framework > > > Key: KYLIN-2776 > URL: https://issues.apache.org/jira/browse/KYLIN-2776 > Project: Kylin > Issue Type: New Feature >Affects Versions: v2.0.0 >Reporter: yiming.xu >Assignee: yiming.xu > Attachments: active_calls.png, calls.png, KYLIN-2776.patch, > metric_structure.png, query_count.png, query_duration.png, > query_result_rowcount.png, report.json > > > With https://issues.apache.org/jira/browse/KYLIN-2721.We are plan to release > a new metric framework. > New metric is different hadoop metric and based on dropwizard . which has > the following advantage: > * Well-defined metric model for frequently-needed metrics (ie JVM metrics) > * Well-defined measurements for all metrics (ie max, mean, stddev, > mean_rate, etc), > * Built-in pluggable reporting frameworks like JMX, Console, Log, JSON > We refactored QueryMetric with new metrics, notice the exposed JMX MBeans > have changed a little bit. > A new tool called perflog is also introduced. Perflog traces call duration > time and current active calls by recording them to metric system. > Some snapshots of the new JMX MBeans can be seen in attachments -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2776) Using dropwizard as default metric framework
[ https://issues.apache.org/jira/browse/KYLIN-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-2776: -- Description: With https://issues.apache.org/jira/browse/KYLIN-2721.We are plan to release a new metric framework. New metric is different hadoop metric and based on dropwizard . which has the following advantage: * Well-defined metric model for frequently-needed metrics (ie JVM metrics) * Well-defined measurements for all metrics (ie max, mean, stddev, mean_rate, etc), * Built-in pluggable reporting frameworks like JMX, Console, Log, JSON We refactored QueryMetric with new metrics, notice the exposed JMX MBeans have changed a little bit. A new tool called perflog is also introduced. Perflog traces call duration time and current active calls by recording them to metric system. Some snapshots of the new JMX MBeans can be seen in attachments was: With https://issues.apache.org/jira/browse/KYLIN-2721.We are plan to release a new metric framework. New metric is different hadoop metric and based on dropwizard . which has the following advantage: * Well-defined metric model for frequently-needed metrics (ie JVM metrics) * Well-defined measurements for all metrics (ie max, mean, stddev, mean_rate, etc), * Built-in pluggable reporting frameworks like JMX, Console, Log, JSON We refactor QueryMetric with new metris. New metric add perflog. Perflog trace calls duration time and current active calls by recording them to metric system. Attachment is the difference between the two metric system . > Using dropwizard as default metric framework > > > Key: KYLIN-2776 > URL: https://issues.apache.org/jira/browse/KYLIN-2776 > Project: Kylin > Issue Type: New Feature >Affects Versions: v2.0.0 >Reporter: yiming.xu >Assignee: yiming.xu > Attachments: active_calls.png, calls.png, KYLIN-2776.patch, > metric_structure.png, query_count.png, query_duration.png, > query_result_rowcount.png, report.json > > > With https://issues.apache.org/jira/browse/KYLIN-2721.We are plan to release > a new metric framework. > New metric is different hadoop metric and based on dropwizard . which has > the following advantage: > * Well-defined metric model for frequently-needed metrics (ie JVM metrics) > * Well-defined measurements for all metrics (ie max, mean, stddev, > mean_rate, etc), > * Built-in pluggable reporting frameworks like JMX, Console, Log, JSON > We refactored QueryMetric with new metrics, notice the exposed JMX MBeans > have changed a little bit. > A new tool called perflog is also introduced. Perflog traces call duration > time and current active calls by recording them to metric system. > Some snapshots of the new JMX MBeans can be seen in attachments -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2776) Using dropwizard as default metric framework
[ https://issues.apache.org/jira/browse/KYLIN-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-2776: -- Summary: Using dropwizard as default metric framework (was: New metric framework with kylin) > Using dropwizard as default metric framework > > > Key: KYLIN-2776 > URL: https://issues.apache.org/jira/browse/KYLIN-2776 > Project: Kylin > Issue Type: New Feature >Affects Versions: v2.0.0 >Reporter: yiming.xu >Assignee: yiming.xu > Attachments: active_calls.png, calls.png, metric_structure.png, > query_count.png, query_duration.png, query_result_rowcount.png, report.json > > > With https://issues.apache.org/jira/browse/KYLIN-2721.We are plan to release > a new metric framework. > New metric is different hadoop metric and based on dropwizard . which has > the following advantage: > * Well-defined metric model for frequently-needed metrics (ie JVM metrics) > * Well-defined measurements for all metrics (ie max, mean, stddev, > mean_rate, etc), > * Built-in pluggable reporting frameworks like JMX, Console, Log, JSON > We refactor QueryMetric with new metris. > New metric add perflog. Perflog trace calls duration time and current > active calls by recording them to metric system. > Attachment is the difference between the two metric system . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2776) New metric framework with kylin
[ https://issues.apache.org/jira/browse/KYLIN-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-2776: -- Description: With https://issues.apache.org/jira/browse/KYLIN-2721.We are plan to release a new metric framework. New metric is different hadoop metric and based on dropwizard . which has the following advantage: * Well-defined metric model for frequently-needed metrics (ie JVM metrics) * Well-defined measurements for all metrics (ie max, mean, stddev, mean_rate, etc), * Built-in pluggable reporting frameworks like JMX, Console, Log, JSON We refactor QueryMetric with new metris. New metric add perflog. Perflog trace calls duration time and current active calls by recording them to metric system. Attachment is the difference between the two metric system . was: With https://issues.apache.org/jira/browse/KYLIN-2721.We are plan to release a new metric framework. New metric is different hadoop metric and based on dropwizard . which has the following advantage: * Well-defined metric model for frequently-needed metrics (ie JVM metrics) * Well-defined measurements for all metrics (ie max, mean, stddev, mean_rate, etc), * Built-in pluggable reporting frameworks like JMX, Console, Log, JSON We refactor QueryMetric with new metris. New metric add perflog. Perflog trace calls duration time and current active calls record to metric system. Attachment is the difference between the two metric system . > New metric framework with kylin > --- > > Key: KYLIN-2776 > URL: https://issues.apache.org/jira/browse/KYLIN-2776 > Project: Kylin > Issue Type: New Feature >Affects Versions: v2.0.0 >Reporter: yiming.xu >Assignee: yiming.xu > Attachments: active_calls.png, calls.png, metric_structure.png, > query_count.png, query_duration.png, query_result_rowcount.png, report.json > > > With https://issues.apache.org/jira/browse/KYLIN-2721.We are plan to release > a new metric framework. > New metric is different hadoop metric and based on dropwizard . which has > the following advantage: > * Well-defined metric model for frequently-needed metrics (ie JVM metrics) > * Well-defined measurements for all metrics (ie max, mean, stddev, > mean_rate, etc), > * Built-in pluggable reporting frameworks like JMX, Console, Log, JSON > We refactor QueryMetric with new metris. > New metric add perflog. Perflog trace calls duration time and current > active calls by recording them to metric system. > Attachment is the difference between the two metric system . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KYLIN-2776) new metric framework with kylin
yiming.xu created KYLIN-2776: Summary: new metric framework with kylin Key: KYLIN-2776 URL: https://issues.apache.org/jira/browse/KYLIN-2776 Project: Kylin Issue Type: New Feature Affects Versions: v2.0.0 Reporter: yiming.xu Assignee: yiming.xu Attachments: active_calls.png, calls.png, metric_structure.png, query_count.png, query_duration.png, query_result_rowcount.png, report.json With https://issues.apache.org/jira/browse/KYLIN-2721.We are plan to release a new metric framework. New metric is different hadoop metric and based on dropwizard . which has the following advantage: * Well-defined metric model for frequently-needed metrics (ie JVM metrics) * Well-defined measurements for all metrics (ie max, mean, stddev, mean_rate, etc), * Built-in pluggable reporting frameworks like JMX, Console, Log, JSON We refactor QueryMetric with new metris. New metric add perflog. Perflog trace calls duration time and current active calls record to metric system. Attachment is the difference between the two metric system . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2776) New metric framework with kylin
[ https://issues.apache.org/jira/browse/KYLIN-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yiming.xu updated KYLIN-2776: - Summary: New metric framework with kylin (was: new metric framework with kylin) > New metric framework with kylin > --- > > Key: KYLIN-2776 > URL: https://issues.apache.org/jira/browse/KYLIN-2776 > Project: Kylin > Issue Type: New Feature >Affects Versions: v2.0.0 >Reporter: yiming.xu >Assignee: yiming.xu > Attachments: active_calls.png, calls.png, metric_structure.png, > query_count.png, query_duration.png, query_result_rowcount.png, report.json > > > With https://issues.apache.org/jira/browse/KYLIN-2721.We are plan to release > a new metric framework. > New metric is different hadoop metric and based on dropwizard . which has > the following advantage: > * Well-defined metric model for frequently-needed metrics (ie JVM metrics) > * Well-defined measurements for all metrics (ie max, mean, stddev, > mean_rate, etc), > * Built-in pluggable reporting frameworks like JMX, Console, Log, JSON > We refactor QueryMetric with new metris. > New metric add perflog. Perflog trace calls duration time and current > active calls record to metric system. > Attachment is the difference between the two metric system . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
