[jira] [Resolved] (KYLIN-2202) Fix the conflict between KYLIN-1851 and KYLIN-2135
[ https://issues.apache.org/jira/browse/KYLIN-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XIE FAN resolved KYLIN-2202. Resolution: Fixed > Fix the conflict between KYLIN-1851 and KYLIN-2135 > -- > > Key: KYLIN-2202 > URL: https://issues.apache.org/jira/browse/KYLIN-2202 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.4.1 >Reporter: XIE FAN > Fix For: v1.6.1 > > Attachments: > 0001-KYLIN-2202-fix-the-conflict-between-KYLIN-1851-and-K.patch > > > In KYLIN-1851, we use a new dictionary(TrieDictionaryForest) to reduce the > peek memory usage of the dictionary building procedure. In order to use the > new dictionary, we need to make sure that the input data is sorted. But in > KYLIN-2135, the output data is split into multi files and values in different > files may be out-of-order. So we need to fix the conflict between this two > issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2202) Fix the conflict between KYLIN-1851 and KYLIN-2135
[ https://issues.apache.org/jira/browse/KYLIN-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XIE FAN updated KYLIN-2202: --- Attachment: 0001-KYLIN-2202-fix-the-conflict-between-KYLIN-1851-and-K.patch > Fix the conflict between KYLIN-1851 and KYLIN-2135 > -- > > Key: KYLIN-2202 > URL: https://issues.apache.org/jira/browse/KYLIN-2202 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.4.1 >Reporter: XIE FAN > Fix For: v1.6.1 > > Attachments: > 0001-KYLIN-2202-fix-the-conflict-between-KYLIN-1851-and-K.patch > > > In KYLIN-1851, we use a new dictionary(TrieDictionaryForest) to reduce the > peek memory usage of the dictionary building procedure. In order to use the > new dictionary, we need to make sure that the input data is sorted. But in > KYLIN-2135, the output data is split into multi files and values in different > files may be out-of-order. So we need to fix the conflict between this two > issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682723#comment-15682723 ] liyang commented on KYLIN-2180: --- Created KYLIN-2220 for the ask. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2220) Enforce same name between Cube & CubeDesc
liyang created KYLIN-2220: - Summary: Enforce same name between Cube & CubeDesc Key: KYLIN-2220 URL: https://issues.apache.org/jira/browse/KYLIN-2220 Project: Kylin Issue Type: Improvement Reporter: liyang Cube & CubeDesc have the same lifecycle (created together, drop together), however they are two objects and may have different names (although never happen in normal use case). Let's try to ensure Cube & CubeDesc always share the same name. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682706#comment-15682706
]
kangkaisen commented on KYLIN-2180:
---
I agree with you. I think the {{getCubeNameFromDesc}} method is unnecessary.
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682701#comment-15682701 ] liyang commented on KYLIN-2180: --- The simplest solution is to enforce same name between Cube & CubeDesc. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[
https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682696#comment-15682696
]
liyang commented on KYLIN-2180:
---
Let's fix the navigation from CubeDesc to Project first. Then reusing
{{KylinConfigExt}} will become very straightforward.
> Add project config and make config priority become "cube > project > server"
>
>
> Key: KYLIN-2180
> URL: https://issues.apache.org/jira/browse/KYLIN-2180
> Project: Kylin
> Issue Type: New Feature
> Components: Metadata
>Affects Versions: v1.5.4.1
>Reporter: kangkaisen
>Assignee: kangkaisen
> Attachments: KYLIN-2180-tmp.patch
>
>
> There are cases we want to override global kylin.properties in the scope of a
> project. E.g. the queue name of Hadoop job.
> Finally, the config priority for Kylin should be "cube > project > server". I
> think which is reasonable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-1875) Kylin support SnowFlake schema
[ https://issues.apache.org/jira/browse/KYLIN-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682686#comment-15682686 ] liyang commented on KYLIN-1875: --- Right. The design is a little bit more than the classic "Snowflake". But no harm, right? :-) > Kylin support SnowFlake schema > -- > > Key: KYLIN-1875 > URL: https://issues.apache.org/jira/browse/KYLIN-1875 > Project: Kylin > Issue Type: Improvement >Affects Versions: v1.5.2 >Reporter: Rahul Choubey >Assignee: liyang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2212) 'NOT' operator in filter on derived column may get incorrect result
[ https://issues.apache.org/jira/browse/KYLIN-2212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682628#comment-15682628 ] liyang commented on KYLIN-2212: --- You are right. It should only loosen the whole filter, not part of it. Otherwise, the final result is broken. > 'NOT' operator in filter on derived column may get incorrect result > > > Key: KYLIN-2212 > URL: https://issues.apache.org/jira/browse/KYLIN-2212 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.3, v1.5.4.1 >Reporter: zhengdong >Assignee: zhengdong > > when a compare filter on derived column specified in where clause, it will be > translated into a filter on PK, and further converted to a range filter on PK > ( GT the min, LT the max) if the PK values' s size larger than the > threshold. > However, this loose filter may filter more records out than needed when > decorated with 'NOT' operator -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2211) IDictionaryValueEnumerator should return String instead of byte[]
[ https://issues.apache.org/jira/browse/KYLIN-2211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang resolved KYLIN-2211. --- Resolution: Fixed Assignee: liyang Fix Version/s: v1.6.1 > IDictionaryValueEnumerator should return String instead of byte[] > - > > Key: KYLIN-2211 > URL: https://issues.apache.org/jira/browse/KYLIN-2211 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: liyang > Fix For: v1.6.1 > > > Don't confuse raw value with its converted byte[] form. As the source of > dictionary, the input should be String. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682486#comment-15682486 ] kangkaisen commented on KYLIN-2180: --- This idea has another issue: after migrating the cube form staging env to prod env, the cube should inherit the config of prod env project . So I think we should not depend on the front-end to realize this feature. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2173) push down limit leads to wrong answer when filter is loosened
[
https://issues.apache.org/jira/browse/KYLIN-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682404#comment-15682404
]
Shaofeng SHI commented on KYLIN-2173:
-
Hi Dayue, IT with this change got success in last night.
The IT is running in a hadoop cluster cluster (with hadoop, hive, hbase, kafka
running), hosting it on public cloud like AWS will generate extra cost which we
may not affort for. We tried to get some resource from apache infrastracture
team, but seems they only have shared jenkins and the allocated resource
couldn't fulfill kylin's IT.
I would suggest you setup an IT env with HDP 2.2 sandbox vm + jenkins, running
"mvn verify -fae -Dhdp.version=2.2.4.2-2 -P sandbox" will do all the tests
automatically.
> push down limit leads to wrong answer when filter is loosened
> -
>
> Key: KYLIN-2173
> URL: https://issues.apache.org/jira/browse/KYLIN-2173
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Affects Versions: v1.5.4.1
>Reporter: Dayue Gao
>Assignee: Dayue Gao
> Fix For: v1.6.0
>
>
> To reproduce:
> {noformat}
> select
> test_kylin_fact.cal_dt
> ,sum(test_kylin_fact.price) as GMV
> FROM test_kylin_fact
> left JOIN edw.test_cal_dt as test_cal_dt
> ON test_kylin_fact.cal_dt = test_cal_dt.cal_dt
> where test_cal_dt.week_beg_dt in ('2012-01-01', '2012-01-20')
> group by test_kylin_fact.cal_dt
> limit 12
> {noformat}
> Kylin returns 5 rows, expect 12 rows.
> Root cause: filter condition may be loosened when we translate derived filter
> in DerivedFilterTranslator. If we push down limit, query server won't get
> enough valid records from storage. In the above example, 24 rows returned
> from storage, only 5 are valid.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-1875) Kylin support SnowFlake schema
[ https://issues.apache.org/jira/browse/KYLIN-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682370#comment-15682370 ] Shaofeng SHI commented on KYLIN-1875: - Hi Yang, "Snowflaking is a method of normalising the dimension tables in a star schema", so my understanding is, there is still only one Fact table in snowflake schema. If multiple fact tables there, that should be "fact constellation" schema: https://en.wikipedia.org/wiki/Snowflake_schema > Kylin support SnowFlake schema > -- > > Key: KYLIN-1875 > URL: https://issues.apache.org/jira/browse/KYLIN-1875 > Project: Kylin > Issue Type: Improvement >Affects Versions: v1.5.2 >Reporter: Rahul Choubey >Assignee: liyang > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2219) Tables can not show "extend information" correctly
Wang Cheng created KYLIN-2219: - Summary: Tables can not show "extend information" correctly Key: KYLIN-2219 URL: https://issues.apache.org/jira/browse/KYLIN-2219 Project: Kylin Issue Type: Improvement Reporter: Wang Cheng Assignee: Wang Cheng Priority: Minor When clicking DataSource->Database->Table->Extend Information, it can not show full information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2191) Integer encoding error for width from 5 to 7
[ https://issues.apache.org/jira/browse/KYLIN-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi updated KYLIN-2191: - Attachment: ab6e083517a16bece87db34d9f7f22a5df0662ee.patch > Integer encoding error for width from 5 to 7 > > > Key: KYLIN-2191 > URL: https://issues.apache.org/jira/browse/KYLIN-2191 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v1.5.4, v1.5.4.1 >Reporter: Roger Shi >Assignee: Roger Shi > Fix For: v1.6.1 > > Attachments: ab6e083517a16bece87db34d9f7f22a5df0662ee.patch > > > For integer encoding, if width is 5, 6 and 7. The decoded value is not right. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2191) Integer encoding error for width from 5 to 7
[ https://issues.apache.org/jira/browse/KYLIN-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi resolved KYLIN-2191. -- Resolution: Fixed > Integer encoding error for width from 5 to 7 > > > Key: KYLIN-2191 > URL: https://issues.apache.org/jira/browse/KYLIN-2191 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v1.5.4, v1.5.4.1 >Reporter: Roger Shi >Assignee: Roger Shi > Fix For: v1.6.1 > > Attachments: ab6e083517a16bece87db34d9f7f22a5df0662ee.patch > > > For integer encoding, if width is 5, 6 and 7. The decoded value is not right. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2197) Has only base cuboid for some cube desc
[ https://issues.apache.org/jira/browse/KYLIN-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-2197: Affects Version/s: v1.5.2 v1.5.3 v1.5.4.1 Fix Version/s: v1.6.1 Thanks Roger! > Has only base cuboid for some cube desc > --- > > Key: KYLIN-2197 > URL: https://issues.apache.org/jira/browse/KYLIN-2197 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.5.2, v1.5.3, v1.5.4.1 >Reporter: liyang >Assignee: Roger Shi > Fix For: v1.6.1 > > Attachments: f3d19f6da20fa415eac641b8daf6081b3eaaf9d4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2197) Has only base cuboid for some cube desc
[ https://issues.apache.org/jira/browse/KYLIN-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi resolved KYLIN-2197. -- Resolution: Fixed > Has only base cuboid for some cube desc > --- > > Key: KYLIN-2197 > URL: https://issues.apache.org/jira/browse/KYLIN-2197 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: Roger Shi > Attachments: f3d19f6da20fa415eac641b8daf6081b3eaaf9d4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2201) Cube desc and aggregation group rule combination max check fail
[ https://issues.apache.org/jira/browse/KYLIN-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi updated KYLIN-2201: - Attachment: 297730dc97e705f2c8526dcdc7718cee900e40e9.patch 00ba74327a0322e1ce62d3c1cf7953ab21ae9407.patch > Cube desc and aggregation group rule combination max check fail > --- > > Key: KYLIN-2201 > URL: https://issues.apache.org/jira/browse/KYLIN-2201 > Project: Kylin > Issue Type: Bug >Reporter: Roger Shi > Attachments: 00ba74327a0322e1ce62d3c1cf7953ab21ae9407.patch, > 297730dc97e705f2c8526dcdc7718cee900e40e9.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2201) Cube desc and aggregation group rule combination max check fail
[ https://issues.apache.org/jira/browse/KYLIN-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi resolved KYLIN-2201. -- Resolution: Fixed > Cube desc and aggregation group rule combination max check fail > --- > > Key: KYLIN-2201 > URL: https://issues.apache.org/jira/browse/KYLIN-2201 > Project: Kylin > Issue Type: Bug >Reporter: Roger Shi > Attachments: 00ba74327a0322e1ce62d3c1cf7953ab21ae9407.patch, > 297730dc97e705f2c8526dcdc7718cee900e40e9.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2197) Has only base cuboid for some cube desc
[ https://issues.apache.org/jira/browse/KYLIN-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi updated KYLIN-2197: - Attachment: f3d19f6da20fa415eac641b8daf6081b3eaaf9d4.patch > Has only base cuboid for some cube desc > --- > > Key: KYLIN-2197 > URL: https://issues.apache.org/jira/browse/KYLIN-2197 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: Roger Shi > Attachments: f3d19f6da20fa415eac641b8daf6081b3eaaf9d4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682274#comment-15682274 ] kangkaisen commented on KYLIN-2180: --- This idea is more simpler and the front-end need more work. But whether if user should get the project config firstly when user create the CubeDesc with JSON directly. at the same time, we need to check the cube config contain the project config in back-end. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2217) Reducers build dictionaries locally
XIE FAN created KYLIN-2217: -- Summary: Reducers build dictionaries locally Key: KYLIN-2217 URL: https://issues.apache.org/jira/browse/KYLIN-2217 Project: Kylin Issue Type: Improvement Affects Versions: v1.5.4.1 Reporter: XIE FAN Fix For: Future In KYLIN-1851, we reduce the peek memory usage of the dictionary-building procedure by splitting a single Trie tree structure to Trie forest. But there still exist a bottleneck that all the dictionaries are built in Kylin client. In this issue, we want to use multi reducers to build different dictionaries locally and concurrently，which can further reduce the peek memory usage as well as speed up the dictionary-building procedure. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KYLIN-2209) Potential NPE in StreamingController#deserializeTableDesc()
[
https://issues.apache.org/jira/browse/KYLIN-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kaige Liu reassigned KYLIN-2209:
-
Assignee: Kaige Liu
> Potential NPE in StreamingController#deserializeTableDesc()
> ---
>
> Key: KYLIN-2209
> URL: https://issues.apache.org/jira/browse/KYLIN-2209
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Kaige Liu
>Priority: Minor
>
> {code}
> TableDesc desc = null;
> try {
> logger.debug("Saving TableDesc " +
> streamingRequest.getTableData());
> desc = JsonUtil.readValue(streamingRequest.getTableData(),
> TableDesc.class);
> } catch (JsonParseException e) {
> logger.error("The TableDesc definition is invalid.", e);
> updateRequest(streamingRequest, false, e.getMessage());
> ...
> String[] dbTable = HadoopUtil.parseHiveTableName(desc.getName());
> {code}
> There is a chance that desc is null by the time we get to
> HadoopUtil.parseHiveTableName().
> We should check for null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (KYLIN-2191) Integer encoding error for width from 5 to 7
[ https://issues.apache.org/jira/browse/KYLIN-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy(Yiming) Liu reassigned KYLIN-2191: Assignee: Roger Shi (was: hongbin ma) > Integer encoding error for width from 5 to 7 > > > Key: KYLIN-2191 > URL: https://issues.apache.org/jira/browse/KYLIN-2191 > Project: Kylin > Issue Type: Bug > Components: Job Engine >Affects Versions: v1.5.4, v1.5.4.1 >Reporter: Roger Shi >Assignee: Roger Shi > Fix For: v1.6.1 > > > For integer encoding, if width is 5, 6 and 7. The decoded value is not right. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2197) Has only base cuboid for some cube desc
[ https://issues.apache.org/jira/browse/KYLIN-2197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-2197: -- Assignee: Roger Shi > Has only base cuboid for some cube desc > --- > > Key: KYLIN-2197 > URL: https://issues.apache.org/jira/browse/KYLIN-2197 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: Roger Shi > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682062#comment-15682062 ] liyang commented on KYLIN-2180: --- The navigate from Cube/CubeDesc to Project is not convenient. We can fix that if you need it. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2180) Add project config and make config priority become "cube > project > server"
[ https://issues.apache.org/jira/browse/KYLIN-2180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682056#comment-15682056 ] liyang commented on KYLIN-2180: --- Or another (perhaps even simpler) idea is "project level config" is just a batch operation of cube configs. Let "project level config" be the common config of all cubes in the project. We just provide a GUI that can change config of all cubes at once. > Add project config and make config priority become "cube > project > server" > > > Key: KYLIN-2180 > URL: https://issues.apache.org/jira/browse/KYLIN-2180 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.5.4.1 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2180-tmp.patch > > > There are cases we want to override global kylin.properties in the scope of a > project. E.g. the queue name of Hadoop job. > Finally, the config priority for Kylin should be "cube > project > server". I > think which is reasonable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2216) Potential NPE in model#findTable() call
Ted Yu created KYLIN-2216:
-
Summary: Potential NPE in model#findTable() call
Key: KYLIN-2216
URL: https://issues.apache.org/jira/browse/KYLIN-2216
Project: Kylin
Issue Type: Bug
Reporter: Ted Yu
Priority: Minor
In DimensionDesc :
{code}
if (table != null)
table = table.toUpperCase();
DataModelDesc model = cubeDesc.getModel();
tableRef = model.findTable(this.getTable());
{code}
If table is null, there would be NPE in findTable():
{code}
public TableRef findTable(String table) throws IllegalArgumentException {
TableRef result = tableNameMap.get(table.toUpperCase());
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (KYLIN-1875) Kylin support SnowFlake schema
[
https://issues.apache.org/jira/browse/KYLIN-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680172#comment-15680172
]
liyang edited comment on KYLIN-1875 at 11/20/16 11:29 PM:
--
Initial design, a {{Data Model}} consists of the following:
- one {{Root Fact Table}}
- multiple {{Fact Tables}}
- multiple {{Lookup Tables}} (can fit in memory, typically < 300 MB)
- multiple {{Join Relationships}} that connect tables to the Root Fact Table
directly or indirectly
During cube build
- All tables are joined together and then aggregated as the {{Cube}}
- Lookup tables, due to the small size, are also saved as {{Snapshots}}
Query and limitations
- Lookup tables can be queried independently from the Snapshots, ensures
correct result
- Fact tables can only be queried from cube, which enforces the pre-calculated
joint and aggregated result
-- *No raw records* limitation. This is the same as before.
--- For example, {{select * from F}} actually returns {{select * from F group
by D1, D2, ..., Dn}}, where F is fact table, Dn are dimension columns.
--- To support query of raw records, user can add PK as dimension (at the cost
that the cardinality of PK is very high). Or try the [raw
measure|http://kylin.apache.org/blog/2016/05/29/raw-measure-in-kylin/] feature.
-- *Enforced joint* limitation. Query on any fact table will enforce the joins
of the whole model, since cube only stores pre-calculated joint result. This
limitation also exists in earlier versions.
--- For example, {{select ... from F1}} will be executed as {{select ... from
F1 join F2 ... join L1 join L2...}}, where Fn are fact tables, Ln are lookup
tables.
--- This limitation IS NOT a problem for left join models and inner join models
that has no record loss after inner join.
--- This limitation is a problem for many-to-many relationships. User can work
around by creating multiple models, for example let each fact table has its own
model.
was (Author: liyang.g...@gmail.com):
Initial design, a {{Data Model}} consists of the following:
- one {{Root Fact Table}}
- multiple {{Fact Tables}}
- multiple {{Lookup Tables}} (can fit in memory, typically < 300 MB)
- multiple {{Join Relationships}} that connect tables to the Root Fact Table
directly or indirectly
During cube build
- All tables are joined together and then aggregated as the {{Cube}}
- Lookup tables, due to the small size, are also saved as {{Snapshots}}
Query and limitations
- Lookup tables can be queried independently from the Snapshots, ensures
correct result
- Fact tables can only be queried from cube, which enforces the pre-calculated
joint and aggregated result
-- *No raw records* limitation. This is the same as before.
--- For example, {{select * from F}} actually returns {{select * from F group
by D1, D2, ..., Dn}}, where F is fact table, Dn are dimension columns.
--- To support query of raw records, user can add PK as dimension (at the cost
that the cardinality of PK is very high). Or try the [raw
measure|http://kylin.apache.org/blog/2016/05/29/raw-measure-in-kylin/] feature.
-- *Enforced joint* limitation. Query on any fact table will enforce the joins
of the whole model, since cube only stores pre-calculated joint result. This
limitation also exists in earlier versions.
--- For example, {{select ... from F1}} will be executed as {{select ... from
F1 join F2 ... join L1 join L2...}}, where Fn are fact tables, Ln are lookup
tables.
--- This limitation IS NOT a problem for left join models and inner join models
that has no record loss after inner join.
--- This limitation is a problem for many-to-many relationships. User can work
around by creating multiple models.
> Kylin support SnowFlake schema
> --
>
> Key: KYLIN-1875
> URL: https://issues.apache.org/jira/browse/KYLIN-1875
> Project: Kylin
> Issue Type: Improvement
>Affects Versions: v1.5.2
>Reporter: Rahul Choubey
>Assignee: liyang
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (KYLIN-2215) Refactor DimensionEncoding.encode(byte[]) to encode(String)
[ https://issues.apache.org/jira/browse/KYLIN-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang resolved KYLIN-2215. --- Resolution: Fixed Fix Version/s: v1.6.1 > Refactor DimensionEncoding.encode(byte[]) to encode(String) > --- > > Key: KYLIN-2215 > URL: https://issues.apache.org/jira/browse/KYLIN-2215 > Project: Kylin > Issue Type: Improvement >Reporter: liyang > Fix For: v1.6.1 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KYLIN-2215) Refactor DimensionEncoding.encode(byte[]) to encode(String)
[ https://issues.apache.org/jira/browse/KYLIN-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang reassigned KYLIN-2215: - Assignee: liyang > Refactor DimensionEncoding.encode(byte[]) to encode(String) > --- > > Key: KYLIN-2215 > URL: https://issues.apache.org/jira/browse/KYLIN-2215 > Project: Kylin > Issue Type: Improvement >Reporter: liyang >Assignee: liyang > Fix For: v1.6.1 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1770) Upgrade Calcite dependency (v1.9)
[
https://issues.apache.org/jira/browse/KYLIN-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681144#comment-15681144
]
Billy(Yiming) Liu commented on KYLIN-1770:
--
still failed at IT, query 56:
select test_cal_dt.week_beg_dt, sum(test_kylin_fact.price) as GMV
, count(*) as TRANS_CNT
from test_kylin_fact
inner JOIN edw.test_cal_dt as test_cal_dt
ON test_kylin_fact.cal_dt = test_cal_dt.cal_dt
inner JOIN test_category_groupings
ON test_kylin_fact.leaf_categ_id = test_category_groupings.leaf_categ_id AND
test_kylin_fact.lstg_site_id = test_category_groupings.site_id
inner JOIN edw.test_sites as test_sites
ON test_kylin_fact.lstg_site_id = test_sites.site_id
where
extract(month from test_cal_dt.week_beg_dt) = 12
group by test_cal_dt.week_beg_dt
The return response is empty. The failed statement is the "extract". In Calcite
1.9, CALCITE-1334 has updated the extract function. Will continue investigate
> Upgrade Calcite dependency (v1.9)
> -
>
> Key: KYLIN-1770
> URL: https://issues.apache.org/jira/browse/KYLIN-1770
> Project: Kylin
> Issue Type: Bug
> Components: Driver - JDBC
>Affects Versions: v1.5.1, v1.5.2
>Reporter: Dayue Gao
>Assignee: Billy(Yiming) Liu
> Attachments: KYLIN-1770.patch
>
>
> Sample code to reproduce:
> {code:java}
> final String sql = "select count(*) from kylin_sales where
> LSTG_SITE_ID between ? and ?";
> try (PreparedStatement stmt = conn.prepareStatement(sql)) {
> stmt.setInt(1, 0);
> stmt.setInt(2, 5);
> try (ResultSet rs = stmt.executeQuery()) {
> printResultSet(rs);
> }
> }
> {code}
> Exception stack trace from server log:
> {noformat}
> java.sql.SQLException: Error while preparing statement [select count(*) from
> kylin_sales where LSTG_SITE_ID between ? and ?]
> at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
> at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:203)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement(CalciteConnectionImpl.java:184)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement(CalciteConnectionImpl.java:85)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareStatement(AvaticaConnection.java:153)
> at
> org.apache.kylin.rest.service.QueryService.execute(QueryService.java:353)
> at
> org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:274)
> at
> org.apache.kylin.rest.service.QueryService.query(QueryService.java:120)
> at
> org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
> at
> org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$8610374f.query()
> at
> org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192)
> at
> org.apache.kylin.rest.controller.QueryController.prepareQuery(QueryController.java:101)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
> at
>
