[jira] [Updated] (KYLIN-2556) Introduce Spotbugs
[ https://issues.apache.org/jira/browse/KYLIN-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-2556: Fix Version/s: v2.2.0 I made the same change in Kylin: https://github.com/apache/kylin/commit/f70c18f0b8d5be87f13f1ec65a83b208e6b4367f But I haven't got to know how it works; Will watch it for a while. Thanks Ted's suggestion! > Introduce Spotbugs > -- > > Key: KYLIN-2556 > URL: https://issues.apache.org/jira/browse/KYLIN-2556 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Shaofeng SHI > Fix For: v2.2.0 > > > HADOOP-14316 added Spotbugs which is more powerful than findbugs. > This issue is to introduce Spotbugs to Kylin in a similar manner. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (KYLIN-2720) Should not allow user to access to all tables' metadata of a project
[ https://issues.apache.org/jira/browse/KYLIN-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI reassigned KYLIN-2720: --- Assignee: qiumingming (was: Shaofeng SHI) > Should not allow user to access to all tables' metadata of a project > > > Key: KYLIN-2720 > URL: https://issues.apache.org/jira/browse/KYLIN-2720 > Project: Kylin > Issue Type: Improvement >Reporter: qiumingming >Assignee: qiumingming > Fix For: Future > > Attachments: KYLIN-2720.patch > > > Currently, user can access to all tables and columns metadata of a specific > project as long as he can access to this project, which is not reasonable. > User should just allow to access to tables that he owned cubes dependent to. > However, user can see some other tables in the web UI in current version. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2750) Kylin MapR-DB Storage Connector
[ https://issues.apache.org/jira/browse/KYLIN-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nirav patel updated KYLIN-2750: --- Description: Kylin currently only supports Hbase as a storage engine. MapR-DB is a proprietary version of Hbase. Out of the box kylin cube engine is able to create cubes (hfile, htable) on mapr and mapr can treat as either hbase or maprdb table. However issue arise while trying to query these tables as MapR-DB doesn't support coprocessor service offered by standard hbase. Kylin relies heavily on coprocessor service for aggregation, projection and distributed computation. As a maprdb doesn't supports coprocessor kylin query engine throws following error: "coprocessorService is not supported for MapR" Proposal to fully support MapR-DB on kylin # Create maprdb-storage adapter to store cubes and metadata on maprdb instead of hbase # Run query against maprdb using maprdb-storage adapter and skip coprocessor Options to improve performance of query in lack of coprocessors # Add support for Apache Drill KYLIN-2751 # Add support for Apache Spark DataFrame to query specific cube KYLIN-741 KYLIN-743 was: Kylin currently only supports Hbase as a storage engine. MapR-DB is a proprietary version of Hbase. Out of the box kylin cube engine is able to create cubes (hfile, htable) on mapr and mapr can treat as either hbase or maprdb table. However issue arise while trying to query these tables as MapR-DB doesn't support coprocessor service offered by standard hbase. Kylin relies heavily on coprocessor service for aggregation, projection and distributed computation. As a maprdb doesn't supports coprocessor kylin query engine throws following error: "coprocessorService is not supported for MapR" Proposal to fully support MapR-DB on kylin # Create maprdb-storage adapter to store cubes and metadata on maprdb instead of hbase # Run query against maprdb using maprdb-storage adapter and skip coprocessor Options to improve performance of query in lack of coprocessors # Add support for Drill KYLIN-2751 # Add support for Spark DataFrame to query specific cube KYLIN-741 KYLIN-743 > Kylin MapR-DB Storage Connector > --- > > Key: KYLIN-2750 > URL: https://issues.apache.org/jira/browse/KYLIN-2750 > Project: Kylin > Issue Type: New Feature >Reporter: nirav patel >Assignee: nirav patel > > Kylin currently only supports Hbase as a storage engine. MapR-DB is a > proprietary version of Hbase. Out of the box kylin cube engine is able to > create cubes (hfile, htable) on mapr and mapr can treat as either hbase or > maprdb table. However issue arise while trying to query these tables as > MapR-DB doesn't support coprocessor service offered by standard hbase. Kylin > relies heavily on coprocessor service for aggregation, projection and > distributed computation. > As a maprdb doesn't supports coprocessor kylin query engine throws following > error: > "coprocessorService is not supported for MapR" > Proposal to fully support MapR-DB on kylin > # Create maprdb-storage adapter to store cubes and metadata on maprdb instead > of hbase > # Run query against maprdb using maprdb-storage adapter and skip coprocessor > Options to improve performance of query in lack of coprocessors > # Add support for Apache Drill KYLIN-2751 > # Add support for Apache Spark DataFrame to query specific cube KYLIN-741 > KYLIN-743 > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2750) Kylin MapR-DB Storage Connector
[ https://issues.apache.org/jira/browse/KYLIN-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nirav patel updated KYLIN-2750: --- Description: Kylin currently only supports Hbase as a storage engine. MapR-DB is a proprietary version of Hbase. Out of the box kylin cube engine is able to create cubes (hfile, htable) on mapr and mapr can treat as either hbase or maprdb table. However issue arise while trying to query these tables as MapR-DB doesn't support coprocessor service offered by standard hbase. Kylin relies heavily on coprocessor service for aggregation, projection and distributed computation. As a maprdb doesn't supports coprocessor kylin query engine throws following error: "coprocessorService is not supported for MapR" Proposal to fully support MapR-DB on kylin # Create maprdb-storage adapter to store cubes and metadata on maprdb instead of hbase # Run query against maprdb using maprdb-storage adapter and skip coprocessor Options to improve performance of query in lack of coprocessors # Add support for Drill KYLIN-2751 # Add support for Spark DataFrame to query specific cube KYLIN-741 KYLIN-743 was: Kylin currently only supports Hbase as a storage engine. MapR-DB is a proprietary version of Hbase. Out of the box kylin cube engine is able to create cubes (hfile, htable) on mapr and mapr can treat as either hbase or maprdb table. However issue arise while trying to query these tables as MapR-DB doesn't support coprocessor service offered by standard hbase. Kylin relies heavily on coprocessor service for aggregation, projection and distributed computation. As a maprdb doesn't supports coprocessor kylin query engine throws following error: "coprocessorService is not supported for MapR" Proposal to fully support MapR-DB on kylin # Create maprdb-storage adapter to store cubes and metadata on maprdb instead of hbase # To improve performance of query, add support for Drill KYLIN-2751 # To improve performance of query, add support for Spark DataFrame to query specific cube KYLIN-741 KYLIN-743 > Kylin MapR-DB Storage Connector > --- > > Key: KYLIN-2750 > URL: https://issues.apache.org/jira/browse/KYLIN-2750 > Project: Kylin > Issue Type: New Feature >Reporter: nirav patel >Assignee: nirav patel > > Kylin currently only supports Hbase as a storage engine. MapR-DB is a > proprietary version of Hbase. Out of the box kylin cube engine is able to > create cubes (hfile, htable) on mapr and mapr can treat as either hbase or > maprdb table. However issue arise while trying to query these tables as > MapR-DB doesn't support coprocessor service offered by standard hbase. Kylin > relies heavily on coprocessor service for aggregation, projection and > distributed computation. > As a maprdb doesn't supports coprocessor kylin query engine throws following > error: > "coprocessorService is not supported for MapR" > Proposal to fully support MapR-DB on kylin > # Create maprdb-storage adapter to store cubes and metadata on maprdb instead > of hbase > # Run query against maprdb using maprdb-storage adapter and skip coprocessor > Options to improve performance of query in lack of coprocessors > # Add support for Drill KYLIN-2751 > # Add support for Spark DataFrame to query specific cube KYLIN-741 KYLIN-743 > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2751) Apache Drill as a Query Engine
[ https://issues.apache.org/jira/browse/KYLIN-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nirav patel updated KYLIN-2751: --- Description: Currently to improve query performance kylin relies heavily on hbase coprocessor capabilities. These ties kylin strongly with standard hbase. Also writing coprocessor modules are not trivial tasks. Goal of this story is to support open source distributed query engine like apache drill which can run against varieties of datasources and also provides distributed query capabilities to improve performance. > Apache Drill as a Query Engine > -- > > Key: KYLIN-2751 > URL: https://issues.apache.org/jira/browse/KYLIN-2751 > Project: Kylin > Issue Type: New Feature >Reporter: nirav patel > > Currently to improve query performance kylin relies heavily on hbase > coprocessor capabilities. These ties kylin strongly with standard hbase. Also > writing coprocessor modules are not trivial tasks. Goal of this story is to > support open source distributed query engine like apache drill which can run > against varieties of datasources and also provides distributed query > capabilities to improve performance. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KYLIN-2751) Apache Drill as a Query Engine
nirav patel created KYLIN-2751: -- Summary: Apache Drill as a Query Engine Key: KYLIN-2751 URL: https://issues.apache.org/jira/browse/KYLIN-2751 Project: Kylin Issue Type: New Feature Reporter: nirav patel -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2750) Kylin MapR-DB Storage Connector
[ https://issues.apache.org/jira/browse/KYLIN-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nirav patel updated KYLIN-2750: --- Description: Kylin currently only supports Hbase as a storage engine. MapR-DB is a proprietary version of Hbase. Out of the box kylin cube engine is able to create cubes (hfile, htable) on mapr and mapr can treat as either hbase or maprdb table. However issue arise while trying to query these tables as MapR-DB doesn't support coprocessor service offered by standard hbase. Kylin relies heavily on coprocessor service for aggregation, projection and distributed computation. As a maprdb doesn't supports coprocessor kylin query engine throws following error: "coprocessorService is not supported for MapR" Proposal to fully support MapR-DB on kylin # Create maprdb-storage adapter to store cubes and metadata on maprdb instead of hbase # To improve performance of query, add support for Drill KYLIN-2751 # To improve performance of query, add support for Spark DataFrame to query specific cube KYLIN-741 KYLIN-743 was: Kylin currently only supports Hbase as a storage engine. MapR-DB is a proprietary version of Hbase. Out of the box kylin cube engine is able to create cubes (hfile, htable) on mapr and mapr can treat as either hbase or maprdb table. However issue arise while trying to query these tables as MapR-DB doesn't support coprocessor service offered by standard hbase. Kylin relies heavily on coprocessor service for aggregation, projection and distributed computation. As a maprdb doesn't supports coprocessor kylin query engine throws following error: "coprocessorService is not supported for MapR" Proposal to fully support MapR-DB on kylin # Create maprdb-storage adapter to store cubes and metadata on maprdb instead of hbase # To improve performance of query, add support for Drill or Spark DataFrame to query specific cube > Kylin MapR-DB Storage Connector > --- > > Key: KYLIN-2750 > URL: https://issues.apache.org/jira/browse/KYLIN-2750 > Project: Kylin > Issue Type: New Feature >Reporter: nirav patel >Assignee: nirav patel > > Kylin currently only supports Hbase as a storage engine. MapR-DB is a > proprietary version of Hbase. Out of the box kylin cube engine is able to > create cubes (hfile, htable) on mapr and mapr can treat as either hbase or > maprdb table. However issue arise while trying to query these tables as > MapR-DB doesn't support coprocessor service offered by standard hbase. Kylin > relies heavily on coprocessor service for aggregation, projection and > distributed computation. > As a maprdb doesn't supports coprocessor kylin query engine throws following > error: > "coprocessorService is not supported for MapR" > Proposal to fully support MapR-DB on kylin > # Create maprdb-storage adapter to store cubes and metadata on maprdb instead > of hbase > # To improve performance of query, add support for Drill KYLIN-2751 > # To improve performance of query, add support for Spark DataFrame to query > specific cube KYLIN-741 KYLIN-743 > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-741) Read data from SparkSQL
[ https://issues.apache.org/jira/browse/KYLIN-741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097422#comment-16097422 ] nirav patel commented on KYLIN-741: --- Is this JIRA regarding only building cubes using sparksql or also running queries ? > Read data from SparkSQL > --- > > Key: KYLIN-741 > URL: https://issues.apache.org/jira/browse/KYLIN-741 > Project: Kylin > Issue Type: New Feature > Components: Job Engine, SparkSQL >Reporter: Luke Han >Assignee: Dong Li > Fix For: Backlog > > > Read data from SparkSQL directly. > There are some instances enabled SparkSQL interface for data consuming, it > will be great if Kylin could read data directly from SparkSQL. > This feature does not require Spark Cube Build Engine to be ready. It could > continue to leverage existing MR cube build engine and process data on Hadoop > cluster then persistent cube to HBase. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (KYLIN-2750) Kylin MapR-DB Storage Connector
nirav patel created KYLIN-2750: -- Summary: Kylin MapR-DB Storage Connector Key: KYLIN-2750 URL: https://issues.apache.org/jira/browse/KYLIN-2750 Project: Kylin Issue Type: New Feature Reporter: nirav patel Assignee: nirav patel Kylin currently only supports Hbase as a storage engine. MapR-DB is a proprietary version of Hbase. Out of the box kylin cube engine is able to create cubes (hfile, htable) on mapr and mapr can treat as either hbase or maprdb table. However issue arise while trying to query these tables as MapR-DB doesn't support coprocessor service offered by standard hbase. Kylin relies heavily on coprocessor service for aggregation, projection and distributed computation. As a maprdb doesn't supports coprocessor kylin query engine throws following error: "coprocessorService is not supported for MapR" Proposal to fully support MapR-DB on kylin # Create maprdb-storage adapter to store cubes and metadata on maprdb instead of hbase # To improve performance of query, add support for Drill or Spark DataFrame to query specific cube -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2745) Remove eclipse .settings directory from each module
[ https://issues.apache.org/jira/browse/KYLIN-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097419#comment-16097419 ] nirav patel commented on KYLIN-2745: Hi Dong Li, How about exporting code formatter files and include them in "config" or "extra" directories or make it available for download from http. Or Just have a manual guidelines for intelliJ. It seems like those formatter are fairly static files and need one time setup. That is also a reason I don't see it as git candidate. I have also not seen .settings files being committed for most other projects on github. That directory also stores many other eclipse preferences, plugin preferences etc so highly likely that it is going to vary from person to person depending upon their environment. It comes up in commit all the time and becomes hard to avoid not checking them in accidentally. Thanks > Remove eclipse .settings directory from each module > --- > > Key: KYLIN-2745 > URL: https://issues.apache.org/jira/browse/KYLIN-2745 > Project: Kylin > Issue Type: Task >Affects Versions: Future, v2.0.0 >Reporter: nirav patel >Assignee: nirav patel > Fix For: Future > > Attachments: KYLIN-2745.patch > > > I think this directory should not be part of git repo at all. It should be > listed in .gitignore file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (KYLIN-2604) Use global dict as the default encoding for precise distinct count in web
[ https://issues.apache.org/jira/browse/KYLIN-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen resolved KYLIN-2604. -- Resolution: Fixed Fix Version/s: v2.1.0 > Use global dict as the default encoding for precise distinct count in web > - > > Key: KYLIN-2604 > URL: https://issues.apache.org/jira/browse/KYLIN-2604 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v2.0.0 >Reporter: kangkaisen >Assignee: kangkaisen >Priority: Minor > Fix For: v2.1.0 > > Attachments: KYLIN-2604.patch > > > we should use global dict as the default encoding for precise distinct count > in web, which more easy-to-use for users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2702) Ineffective null check in DataModelDesc#initComputedColumns()
[
https://issues.apache.org/jira/browse/KYLIN-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097345#comment-16097345
]
Shaofeng SHI commented on KYLIN-2702:
-
This has been fixed together with other changes; When I look into, there is no
such issue there, so I would mark it as fixed now.
> Ineffective null check in DataModelDesc#initComputedColumns()
> -
>
> Key: KYLIN-2702
> URL: https://issues.apache.org/jira/browse/KYLIN-2702
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> {code}
> private void initComputedColumns(Map ccInfoMap) {
> if (ccInfoMap == null) {
> logger.error("cc info map is null");
> }
> ...
> CCInfo other = ccInfoMap.get(thisCCName);
> {code}
> Subsequent access to ccInfoMap may produce NPE.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KYLIN-2702) Ineffective null check in DataModelDesc#initComputedColumns()
[
https://issues.apache.org/jira/browse/KYLIN-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-2702.
-
Resolution: Fixed
Fix Version/s: v2.1.0
> Ineffective null check in DataModelDesc#initComputedColumns()
> -
>
> Key: KYLIN-2702
> URL: https://issues.apache.org/jira/browse/KYLIN-2702
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
> Fix For: v2.1.0
>
>
> {code}
> private void initComputedColumns(Map ccInfoMap) {
> if (ccInfoMap == null) {
> logger.error("cc info map is null");
> }
> ...
> CCInfo other = ccInfoMap.get(thisCCName);
> {code}
> Subsequent access to ccInfoMap may produce NPE.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2745) Remove eclipse .settings directory from each module
[ https://issues.apache.org/jira/browse/KYLIN-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097344#comment-16097344 ] Dong Li commented on KYLIN-2745: Hi Shaofeng and Nirav, In fact, these setting files are used for code formatting. In this document, eclipse formatter is suggested for non-eclipse users: http://kylin.apache.org/development/dev_env.html So I would suggest to keep these files, and also remove them from .gitignore. Thanks. > Remove eclipse .settings directory from each module > --- > > Key: KYLIN-2745 > URL: https://issues.apache.org/jira/browse/KYLIN-2745 > Project: Kylin > Issue Type: Task >Affects Versions: Future, v2.0.0 >Reporter: nirav patel >Assignee: nirav patel > Fix For: Future > > Attachments: KYLIN-2745.patch > > > I think this directory should not be part of git repo at all. It should be > listed in .gitignore file. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2604) Use global dict as the default encoding for precise distinct count in web
[ https://issues.apache.org/jira/browse/KYLIN-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097343#comment-16097343 ] Zhixiong Chen commented on KYLIN-2604: -- OK. I have reviewed this patch . I will merge it into Kylin master branch and 2.1 branch. Thanks [~kangkaisen]. > Use global dict as the default encoding for precise distinct count in web > - > > Key: KYLIN-2604 > URL: https://issues.apache.org/jira/browse/KYLIN-2604 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v2.0.0 >Reporter: kangkaisen >Assignee: kangkaisen >Priority: Minor > Attachments: KYLIN-2604.patch > > > we should use global dict as the default encoding for precise distinct count > in web, which more easy-to-use for users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2612) Potential NPE accessing familyMap in AclTableMigrationTool#getAllAceInfo
[
https://issues.apache.org/jira/browse/KYLIN-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-2612:
Fix Version/s: v2.1.0
> Potential NPE accessing familyMap in AclTableMigrationTool#getAllAceInfo
>
>
> Key: KYLIN-2612
> URL: https://issues.apache.org/jira/browse/KYLIN-2612
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Labels: newbie, scope
> Fix For: v2.1.0
>
>
> Here is related code:
> {code}
> NavigableMap familyMap =
> result.getFamilyMap(Bytes.toBytes(AclHBaseStorage.ACL_ACES_FAMILY));
> for (Map.Entry entry : familyMap.entrySet()) {
> {code}
> Good practice is to check whether familyMap is null / empty first.
> See the following class in hbase for example:
> hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaTableUtil.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KYLIN-2612) Potential NPE accessing familyMap in AclTableMigrationTool#getAllAceInfo
[
https://issues.apache.org/jira/browse/KYLIN-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-2612.
-
Resolution: Fixed
> Potential NPE accessing familyMap in AclTableMigrationTool#getAllAceInfo
>
>
> Key: KYLIN-2612
> URL: https://issues.apache.org/jira/browse/KYLIN-2612
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Labels: newbie, scope
> Fix For: v2.1.0
>
>
> Here is related code:
> {code}
> NavigableMap familyMap =
> result.getFamilyMap(Bytes.toBytes(AclHBaseStorage.ACL_ACES_FAMILY));
> for (Map.Entry entry : familyMap.entrySet()) {
> {code}
> Good practice is to check whether familyMap is null / empty first.
> See the following class in hbase for example:
> hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaTableUtil.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2612) Potential NPE accessing familyMap in AclTableMigrationTool#getAllAceInfo
[
https://issues.apache.org/jira/browse/KYLIN-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097341#comment-16097341
]
Shaofeng SHI commented on KYLIN-2612:
-
Fixed it in
https://github.com/apache/kylin/commit/838b047e1201394eeda5e3fcf9dd16cad5e31854
Ted, I didn't see class named "LegacyAclService.java"; I searched the reference
of getFamilyMap method, there is only 1 occurancy, so I assume the
"LegacyAclService.java" might be deleted by someone.
> Potential NPE accessing familyMap in AclTableMigrationTool#getAllAceInfo
>
>
> Key: KYLIN-2612
> URL: https://issues.apache.org/jira/browse/KYLIN-2612
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Labels: newbie, scope
>
> Here is related code:
> {code}
> NavigableMap familyMap =
> result.getFamilyMap(Bytes.toBytes(AclHBaseStorage.ACL_ACES_FAMILY));
> for (Map.Entry entry : familyMap.entrySet()) {
> {code}
> Good practice is to check whether familyMap is null / empty first.
> See the following class in hbase for example:
> hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaTableUtil.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (KYLIN-2612) Potential NPE accessing familyMap in AclTableMigrationTool#getAllAceInfo
[
https://issues.apache.org/jira/browse/KYLIN-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-2612:
---
Assignee: Shaofeng SHI
> Potential NPE accessing familyMap in AclTableMigrationTool#getAllAceInfo
>
>
> Key: KYLIN-2612
> URL: https://issues.apache.org/jira/browse/KYLIN-2612
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Labels: newbie, scope
>
> Here is related code:
> {code}
> NavigableMap familyMap =
> result.getFamilyMap(Bytes.toBytes(AclHBaseStorage.ACL_ACES_FAMILY));
> for (Map.Entry entry : familyMap.entrySet()) {
> {code}
> Good practice is to check whether familyMap is null / empty first.
> See the following class in hbase for example:
> hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaTableUtil.java
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2643) PreparedStatement should be closed in QueryServiceV2#execute()
[
https://issues.apache.org/jira/browse/KYLIN-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097335#comment-16097335
]
Shaofeng SHI commented on KYLIN-2643:
-
The class has been renamed to PushDownRunnerJdbcImpl, there is the unclosed
statements. I fixed it with
https://github.com/apache/kylin/commit/6ca141a9f2e6cd5bc67fc37ec0fba61b1fc2d49c
> PreparedStatement should be closed in QueryServiceV2#execute()
> --
>
> Key: KYLIN-2643
> URL: https://issues.apache.org/jira/browse/KYLIN-2643
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Labels: newbie, scope
>
> {code}
> PreparedStatement preparedState =
> conn.prepareStatement(correctedSql);
> processStatementAttr(preparedState, sqlRequest);
> for (int i = 0; i < ((PrepareSqlRequest)
> sqlRequest).getParams().length; i++) {
> setParam(preparedState, i + 1, ((PrepareSqlRequest)
> sqlRequest).getParams()[i]);
> }
> resultSet = preparedState.executeQuery();
> {code}
> preparedState should be closed upon returning from the method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2643) PreparedStatement should be closed in QueryServiceV2#execute()
[
https://issues.apache.org/jira/browse/KYLIN-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-2643:
Fix Version/s: v2.1.0
Component/s: Query Engine
> PreparedStatement should be closed in QueryServiceV2#execute()
> --
>
> Key: KYLIN-2643
> URL: https://issues.apache.org/jira/browse/KYLIN-2643
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Labels: newbie, scope
> Fix For: v2.1.0
>
>
> {code}
> PreparedStatement preparedState =
> conn.prepareStatement(correctedSql);
> processStatementAttr(preparedState, sqlRequest);
> for (int i = 0; i < ((PrepareSqlRequest)
> sqlRequest).getParams().length; i++) {
> setParam(preparedState, i + 1, ((PrepareSqlRequest)
> sqlRequest).getParams()[i]);
> }
> resultSet = preparedState.executeQuery();
> {code}
> preparedState should be closed upon returning from the method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KYLIN-2643) PreparedStatement should be closed in QueryServiceV2#execute()
[
https://issues.apache.org/jira/browse/KYLIN-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-2643.
-
Resolution: Fixed
> PreparedStatement should be closed in QueryServiceV2#execute()
> --
>
> Key: KYLIN-2643
> URL: https://issues.apache.org/jira/browse/KYLIN-2643
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Labels: newbie, scope
> Fix For: v2.1.0
>
>
> {code}
> PreparedStatement preparedState =
> conn.prepareStatement(correctedSql);
> processStatementAttr(preparedState, sqlRequest);
> for (int i = 0; i < ((PrepareSqlRequest)
> sqlRequest).getParams().length; i++) {
> setParam(preparedState, i + 1, ((PrepareSqlRequest)
> sqlRequest).getParams()[i]);
> }
> resultSet = preparedState.executeQuery();
> {code}
> preparedState should be closed upon returning from the method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (KYLIN-2643) PreparedStatement should be closed in QueryServiceV2#execute()
[
https://issues.apache.org/jira/browse/KYLIN-2643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-2643:
---
Assignee: Shaofeng SHI (was: Yifan Zhang)
> PreparedStatement should be closed in QueryServiceV2#execute()
> --
>
> Key: KYLIN-2643
> URL: https://issues.apache.org/jira/browse/KYLIN-2643
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Labels: newbie, scope
>
> {code}
> PreparedStatement preparedState =
> conn.prepareStatement(correctedSql);
> processStatementAttr(preparedState, sqlRequest);
> for (int i = 0; i < ((PrepareSqlRequest)
> sqlRequest).getParams().length; i++) {
> setParam(preparedState, i + 1, ((PrepareSqlRequest)
> sqlRequest).getParams()[i]);
> }
> resultSet = preparedState.executeQuery();
> {code}
> preparedState should be closed upon returning from the method.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2658) Unclosed ResultSet in JdbcExplorer#loadTableMetadata()
[
https://issues.apache.org/jira/browse/KYLIN-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097331#comment-16097331
]
Shaofeng SHI commented on KYLIN-2658:
-
Fixed in
https://github.com/apache/kylin/commit/a2411fb4b0b9dcc36b1d6d4bb227b5a679d09496
Thanks Ted!
> Unclosed ResultSet in JdbcExplorer#loadTableMetadata()
> --
>
> Key: KYLIN-2658
> URL: https://issues.apache.org/jira/browse/KYLIN-2658
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Fix For: v2.1.0
>
>
> {code}
> ResultSet rs = dbmd.getTables(null, database, table, null);
> ...
> rs = dbmd.getColumns(null, database, table, null);
> {code}
> rs is overwritten without closing the previous instance.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2701) Unclosed PreparedStatement in QueryService#getPrepareOnlySqlResponse
[
https://issues.apache.org/jira/browse/KYLIN-2701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097332#comment-16097332
]
Shaofeng SHI commented on KYLIN-2701:
-
Fixed in
https://github.com/apache/kylin/commit/a2411fb4b0b9dcc36b1d6d4bb227b5a679d09496,
thanks for the reporting!
> Unclosed PreparedStatement in QueryService#getPrepareOnlySqlResponse
>
>
> Key: KYLIN-2701
> URL: https://issues.apache.org/jira/browse/KYLIN-2701
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Fix For: v2.1.0
>
>
> {code}
> try {
> conn.prepareStatement(correctedSql);
> throw new IllegalStateException("Should have thrown
> OnlyPrepareEarlyAbortException");
> {code}
> The return value of prepareStatement() may leak if there is no exception from
> the call.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (KYLIN-2658) Unclosed ResultSet in JdbcExplorer#loadTableMetadata()
[
https://issues.apache.org/jira/browse/KYLIN-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-2658.
-
Resolution: Fixed
> Unclosed ResultSet in JdbcExplorer#loadTableMetadata()
> --
>
> Key: KYLIN-2658
> URL: https://issues.apache.org/jira/browse/KYLIN-2658
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Fix For: v2.1.0
>
>
> {code}
> ResultSet rs = dbmd.getTables(null, database, table, null);
> ...
> rs = dbmd.getColumns(null, database, table, null);
> {code}
> rs is overwritten without closing the previous instance.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (KYLIN-2611) Unclosed HBaseAdmin in AclTableMigrationTool#checkTableExist
[
https://issues.apache.org/jira/browse/KYLIN-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-2611:
---
Assignee: liyang
Fix Version/s: v2.1.0
I see yang has fixed it in commit afaa95a099d69105198b73fbf92622975dbf766c.
Mark the owner to him and status to fixed.
> Unclosed HBaseAdmin in AclTableMigrationTool#checkTableExist
>
>
> Key: KYLIN-2611
> URL: https://issues.apache.org/jira/browse/KYLIN-2611
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: liyang
>Priority: Minor
> Fix For: v2.1.0
>
>
> Here is related code:
> {code}
> Admin hbaseAdmin = new HBaseAdmin(conf);
> return hbaseAdmin.tableExists(TableName.valueOf(tableName));
> {code}
> hbaseAdmin should be closed upon return.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2701) Unclosed PreparedStatement in QueryService#getPrepareOnlySqlResponse
[
https://issues.apache.org/jira/browse/KYLIN-2701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-2701:
Fix Version/s: v2.1.0
> Unclosed PreparedStatement in QueryService#getPrepareOnlySqlResponse
>
>
> Key: KYLIN-2701
> URL: https://issues.apache.org/jira/browse/KYLIN-2701
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Fix For: v2.1.0
>
>
> {code}
> try {
> conn.prepareStatement(correctedSql);
> throw new IllegalStateException("Should have thrown
> OnlyPrepareEarlyAbortException");
> {code}
> The return value of prepareStatement() may leak if there is no exception from
> the call.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (KYLIN-2658) Unclosed ResultSet in JdbcExplorer#loadTableMetadata()
[
https://issues.apache.org/jira/browse/KYLIN-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-2658:
Fix Version/s: v2.1.0
> Unclosed ResultSet in JdbcExplorer#loadTableMetadata()
> --
>
> Key: KYLIN-2658
> URL: https://issues.apache.org/jira/browse/KYLIN-2658
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Fix For: v2.1.0
>
>
> {code}
> ResultSet rs = dbmd.getTables(null, database, table, null);
> ...
> rs = dbmd.getColumns(null, database, table, null);
> {code}
> rs is overwritten without closing the previous instance.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (KYLIN-2658) Unclosed ResultSet in JdbcExplorer#loadTableMetadata()
[
https://issues.apache.org/jira/browse/KYLIN-2658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-2658:
---
Assignee: Shaofeng SHI
> Unclosed ResultSet in JdbcExplorer#loadTableMetadata()
> --
>
> Key: KYLIN-2658
> URL: https://issues.apache.org/jira/browse/KYLIN-2658
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
> Fix For: v2.1.0
>
>
> {code}
> ResultSet rs = dbmd.getTables(null, database, table, null);
> ...
> rs = dbmd.getColumns(null, database, table, null);
> {code}
> rs is overwritten without closing the previous instance.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (KYLIN-2701) Unclosed PreparedStatement in QueryService#getPrepareOnlySqlResponse
[
https://issues.apache.org/jira/browse/KYLIN-2701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-2701:
---
Assignee: Shaofeng SHI
> Unclosed PreparedStatement in QueryService#getPrepareOnlySqlResponse
>
>
> Key: KYLIN-2701
> URL: https://issues.apache.org/jira/browse/KYLIN-2701
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Minor
>
> {code}
> try {
> conn.prepareStatement(correctedSql);
> throw new IllegalStateException("Should have thrown
> OnlyPrepareEarlyAbortException");
> {code}
> The return value of prepareStatement() may leak if there is no exception from
> the call.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2604) Use global dict as the default encoding for precise distinct count in web
[ https://issues.apache.org/jira/browse/KYLIN-2604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097313#comment-16097313 ] Shaofeng SHI commented on KYLIN-2604: - [~Zhixiong Chen] Zhixiong, could you please review and merge this patch? > Use global dict as the default encoding for precise distinct count in web > - > > Key: KYLIN-2604 > URL: https://issues.apache.org/jira/browse/KYLIN-2604 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v2.0.0 >Reporter: kangkaisen >Assignee: kangkaisen >Priority: Minor > Attachments: KYLIN-2604.patch > > > we should use global dict as the default encoding for precise distinct count > in web, which more easy-to-use for users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2711) NPE if job output is lost
[ https://issues.apache.org/jira/browse/KYLIN-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097312#comment-16097312 ] Shaofeng SHI commented on KYLIN-2711: - [~sterligovak] Alexander, patch is recommended; The github repository is a mirror of Apache repository, if you raise a pull request it couldn't be merged directly. > NPE if job output is lost > - > > Key: KYLIN-2711 > URL: https://issues.apache.org/jira/browse/KYLIN-2711 > Project: Kylin > Issue Type: Bug >Affects Versions: all >Reporter: Alexander Sterligov >Assignee: Alexander Sterligov > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2711-avoid-NPE-if-output-is-lost-for-2.0.x.patch, > 0001-KYLIN-2711-avoid-NPE-if-output-is-lost.patch > > > After job crash caused by YARN failure Kylin entered inconsistent state which > causes NullPointerException when trying to get jobs. > 2017-07-05 09:53:04,471 ERROR [http-bio-7070-exec-18] > controller.BasicController:54 : > org.apache.kylin.rest.exception.InternalErrorException: > java.lang.NullPointerException > at > org.apache.kylin.rest.controller.JobController.list(JobController.java:76) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:743) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:672) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:82) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:933) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:867) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:951) > at > org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:842) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:624) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:827) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at >
[jira] [Commented] (KYLIN-2711) NPE if job output is lost
[ https://issues.apache.org/jira/browse/KYLIN-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097280#comment-16097280 ] Alexander Sterligov commented on KYLIN-2711: I've just found that it's not enough. May I create pull request on github or only patch is acceptable? > NPE if job output is lost > - > > Key: KYLIN-2711 > URL: https://issues.apache.org/jira/browse/KYLIN-2711 > Project: Kylin > Issue Type: Bug >Affects Versions: all >Reporter: Alexander Sterligov >Assignee: Alexander Sterligov > Fix For: v2.1.0 > > Attachments: > 0001-KYLIN-2711-avoid-NPE-if-output-is-lost-for-2.0.x.patch, > 0001-KYLIN-2711-avoid-NPE-if-output-is-lost.patch > > > After job crash caused by YARN failure Kylin entered inconsistent state which > causes NullPointerException when trying to get jobs. > 2017-07-05 09:53:04,471 ERROR [http-bio-7070-exec-18] > controller.BasicController:54 : > org.apache.kylin.rest.exception.InternalErrorException: > java.lang.NullPointerException > at > org.apache.kylin.rest.controller.JobController.list(JobController.java:76) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136) > at > org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:743) > at > org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:672) > at > org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:82) > at > org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:933) > at > org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:867) > at > org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:951) > at > org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:842) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:624) > at > org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:827) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:731) > at > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) > at > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) > at > org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at > org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54) > at > org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) > at >
[jira] [Commented] (KYLIN-1926) Loosen the constraint on FK-PK data type matching
[ https://issues.apache.org/jira/browse/KYLIN-1926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097248#comment-16097248 ] liyang commented on KYLIN-1926: --- We could help to debug if there is an easy way to reproduce the problem using sample cube or integration test. > Loosen the constraint on FK-PK data type matching > - > > Key: KYLIN-1926 > URL: https://issues.apache.org/jira/browse/KYLIN-1926 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: all >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Minor > Fix For: v1.5.4 > > Attachments: 0001-KYLIN-1926-FK-PK-data-type-matching.patch > > > If lookup table's PK datatype isn't equal to fact table's FK datatype, Kylin > will report error saying "Primary key are not consistent with Foreign key". > This constraint is too strong. Should allow user to disable this check. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2749) Merge may remove old segments without saving merged segment
[
https://issues.apache.org/jira/browse/KYLIN-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Sterligov updated KYLIN-2749:
---
Description:
Merge started to work on last 7 segments.
During the process hbase had a failure because of spot-instances shutdown in
Amazon. Data was not lost, because it is at S3.
I stopped kylin and did hbase hbck --repair. During the report of repair I
didn't see any information about lost data, just redistribution of regions.
Then after kylin was started I cannot query data from the last 7 segments:
{quote}
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hbase.TableNotFoundException: Table 'KYLIN_7MMHCHKVVB' was
not found, got: KYLIN_7H3WSPX1UJ.
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at
org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:67)
at
org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:31)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators$6.hasNext(Iterators.java:583)
at
org.apache.kylin.storage.gtrecord.SegmentCubeTupleIterator$2.hasNext(SegmentCubeTupleIterator.java:116)
at
org.apache.kylin.storage.gtrecord.SegmentCubeTupleIterator.hasNext(SegmentCubeTupleIterator.java:149)
at com.google.common.collect.Iterators$6.hasNext(Iterators.java:582)
at
org.apache.kylin.storage.gtrecord.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:129)
at
org.apache.kylin.query.enumerator.OLAPEnumerator.moveNext(OLAPEnumerator.java:67)
at Baz$1$1.moveNext(Unknown Source)
at
org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:826)
at
org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:761)
at
org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:302)
at Baz.bind(Unknown Source)
at
org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:331)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:294)
at
org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:553)
at
org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:544)
at
org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:193)
at
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
at
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
at
org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:607)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:600)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:615)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:148)
... 77 more
Caused by: org.apache.hadoop.hbase.TableNotFoundException: Table
'KYLIN_7MMHCHKVVB' was not found, got: KYLIN_7H3WSPX1UJ.
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1310)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1189)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1173)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1130)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:965)
at
org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:83)
at
org.apache.hadoop.hbase.client.HTable.getRegionLocation(HTable.java:505)
at
org.apache.hadoop.hbase.client.HTable.getKeysAndRegionsInRange(HTable.java:721)
at
org.apache.hadoop.hbase.client.HTable.getKeysAndRegionsInRange(HTable.java:691)
at
org.apache.hadoop.hbase.client.HTable.getStartKeysInRange(HTable.java:1796)
at
org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1751)
at
org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1.run(CubeHBaseEndpointRPC.java:182)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
{quote}
[jira] [Updated] (KYLIN-2749) Merge may remove old segments without saving merged segment
[
https://issues.apache.org/jira/browse/KYLIN-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Sterligov updated KYLIN-2749:
---
Description:
Merge started to work on last 7 segments.
During the process hbase had a failure because of spot-instances shutdown in
Amazon. Data was not lost, because it is at S3.
I stopped kylin and did hbase hbck --repair. During the report of repair I
didn't see any information about lost data, just redistribution of regions.
Then after kylin was started I cannot query data from the last 7 segments:
{quote}
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hbase.TableNotFoundException: Table 'KYLIN_7MMHCHKVVB' was
not found, got: KYLIN_7H3WSPX1UJ.
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at
org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:67)
at
org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:31)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators$6.hasNext(Iterators.java:583)
at
org.apache.kylin.storage.gtrecord.SegmentCubeTupleIterator$2.hasNext(SegmentCubeTupleIterator.java:116)
at
org.apache.kylin.storage.gtrecord.SegmentCubeTupleIterator.hasNext(SegmentCubeTupleIterator.java:149)
at com.google.common.collect.Iterators$6.hasNext(Iterators.java:582)
at
org.apache.kylin.storage.gtrecord.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:129)
at
org.apache.kylin.query.enumerator.OLAPEnumerator.moveNext(OLAPEnumerator.java:67)
at Baz$1$1.moveNext(Unknown Source)
at
org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:826)
at
org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:761)
at
org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:302)
at Baz.bind(Unknown Source)
at
org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:331)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:294)
at
org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:553)
at
org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:544)
at
org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:193)
at
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
at
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
at
org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:607)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:600)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:615)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:148)
... 77 more
Caused by: org.apache.hadoop.hbase.TableNotFoundException: Table
'KYLIN_7MMHCHKVVB' was not found, got: KYLIN_7H3WSPX1UJ.
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1310)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1189)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1173)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1130)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:965)
at
org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:83)
at
org.apache.hadoop.hbase.client.HTable.getRegionLocation(HTable.java:505)
at
org.apache.hadoop.hbase.client.HTable.getKeysAndRegionsInRange(HTable.java:721)
at
org.apache.hadoop.hbase.client.HTable.getKeysAndRegionsInRange(HTable.java:691)
at
org.apache.hadoop.hbase.client.HTable.getStartKeysInRange(HTable.java:1796)
at
org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1751)
at
org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1.run(CubeHBaseEndpointRPC.java:182)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
{quote}
[jira] [Updated] (KYLIN-2749) Merge may remove old segments without saving merged segment
[
https://issues.apache.org/jira/browse/KYLIN-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Sterligov updated KYLIN-2749:
---
Summary: Merge may remove old segments without saving merged segment (was:
Merge may remove merged segments before proper register of merged segment)
> Merge may remove old segments without saving merged segment
> ---
>
> Key: KYLIN-2749
> URL: https://issues.apache.org/jira/browse/KYLIN-2749
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.0.0
>Reporter: Alexander Sterligov
>
> Merge started to work on last 7 segments.
> During the process hbase had a failure because of spot-instances shutdown in
> Amazon. Data was not lost, because it is at S3.
> I stopped kylin and did hbase hbck --repair. During the report of repair I
> didn't see any information about lost data, just redistribution of regions.
> Then after kylin was started I cannot query data from the last 7 query
> segments with error:
> {quote}
> Caused by: java.lang.RuntimeException:
> org.apache.hadoop.hbase.TableNotFoundException: Table 'KYLIN_7MMHCHKVVB' was
> not found, got: KYLIN_7H3WSPX1UJ.
> at com.google.common.base.Throwables.propagate(Throwables.java:160)
> at
> org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:67)
> at
> org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:31)
> at
> com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
> at com.google.common.collect.Iterators$6.hasNext(Iterators.java:583)
> at
> org.apache.kylin.storage.gtrecord.SegmentCubeTupleIterator$2.hasNext(SegmentCubeTupleIterator.java:116)
> at
> org.apache.kylin.storage.gtrecord.SegmentCubeTupleIterator.hasNext(SegmentCubeTupleIterator.java:149)
> at com.google.common.collect.Iterators$6.hasNext(Iterators.java:582)
> at
> org.apache.kylin.storage.gtrecord.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:129)
> at
> org.apache.kylin.query.enumerator.OLAPEnumerator.moveNext(OLAPEnumerator.java:67)
> at Baz$1$1.moveNext(Unknown Source)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:826)
> at
> org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:761)
> at
> org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:302)
> at Baz.bind(Unknown Source)
> at
> org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:331)
> at
> org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:294)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:553)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:544)
> at
> org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:193)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
> at
> org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
> at
> org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:607)
> at
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:600)
> at
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:615)
> at
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:148)
> ... 77 more
> Caused by: org.apache.hadoop.hbase.TableNotFoundException: Table
> 'KYLIN_7MMHCHKVVB' was not found, got: KYLIN_7H3WSPX1UJ.
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1310)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1189)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1173)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1130)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:965)
> at
> org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:83)
> at
> org.apache.hadoop.hbase.client.HTable.getRegionLocation(HTable.java:505)
> at
> org.apache.hadoop.hbase.client.HTable.getKeysAndRegionsInRange(HTable.java:721)
> at
>
[jira] [Created] (KYLIN-2749) Merge may remove merged segments before proper register of merged segment
Alexander Sterligov created KYLIN-2749:
--
Summary: Merge may remove merged segments before proper register
of merged segment
Key: KYLIN-2749
URL: https://issues.apache.org/jira/browse/KYLIN-2749
Project: Kylin
Issue Type: Bug
Affects Versions: v2.0.0
Reporter: Alexander Sterligov
Merge started to work on last 7 segments.
During the process hbase had a failure because of spot-instances shutdown in
Amazon. Data was not lost, because it is at S3.
I stopped kylin and did hbase hbck --repair. During the report of repair I
didn't see any information about lost data, just redistribution of regions.
Then after kylin was started I cannot query data from the last 7 query segments
with error:
{quote}
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hbase.TableNotFoundException: Table 'KYLIN_7MMHCHKVVB' was
not found, got: KYLIN_7H3WSPX1UJ.
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at
org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:67)
at
org.apache.kylin.storage.hbase.cube.v2.ExpectedSizeIterator.next(ExpectedSizeIterator.java:31)
at
com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
at com.google.common.collect.Iterators$6.hasNext(Iterators.java:583)
at
org.apache.kylin.storage.gtrecord.SegmentCubeTupleIterator$2.hasNext(SegmentCubeTupleIterator.java:116)
at
org.apache.kylin.storage.gtrecord.SegmentCubeTupleIterator.hasNext(SegmentCubeTupleIterator.java:149)
at com.google.common.collect.Iterators$6.hasNext(Iterators.java:582)
at
org.apache.kylin.storage.gtrecord.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:129)
at
org.apache.kylin.query.enumerator.OLAPEnumerator.moveNext(OLAPEnumerator.java:67)
at Baz$1$1.moveNext(Unknown Source)
at
org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:826)
at
org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:761)
at
org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:302)
at Baz.bind(Unknown Source)
at
org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:331)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:294)
at
org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:553)
at
org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:544)
at
org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:193)
at
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
at
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
at
org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:607)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:600)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:615)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:148)
... 77 more
Caused by: org.apache.hadoop.hbase.TableNotFoundException: Table
'KYLIN_7MMHCHKVVB' was not found, got: KYLIN_7H3WSPX1UJ.
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1310)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1189)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1173)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1130)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:965)
at
org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:83)
at
org.apache.hadoop.hbase.client.HTable.getRegionLocation(HTable.java:505)
at
org.apache.hadoop.hbase.client.HTable.getKeysAndRegionsInRange(HTable.java:721)
at
org.apache.hadoop.hbase.client.HTable.getKeysAndRegionsInRange(HTable.java:691)
at
org.apache.hadoop.hbase.client.HTable.getStartKeysInRange(HTable.java:1796)
at
org.apache.hadoop.hbase.client.HTable.coprocessorService(HTable.java:1751)
at
org.apache.kylin.storage.hbase.cube.v2.CubeHBaseEndpointRPC$1.run(CubeHBaseEndpointRPC.java:182)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
[jira] [Commented] (KYLIN-2727) Introduce cube planner able to select cost-effective cuboids to be built by cost-based algorithms
[ https://issues.apache.org/jira/browse/KYLIN-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097235#comment-16097235 ] liyang commented on KYLIN-2727: --- That's good information. Thanks Yanghong! > Introduce cube planner able to select cost-effective cuboids to be built by > cost-based algorithms > - > > Key: KYLIN-2727 > URL: https://issues.apache.org/jira/browse/KYLIN-2727 > Project: Kylin > Issue Type: New Feature >Affects Versions: v2.0.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > There're several disadvantages to create partial cubes only based on static > rules: > * To learn the concept of these rules will bring extra burden to cube admins > * To achieve a goal cuboid set may need a very complicated set of static rules > * Cube designers may be not familiar with business related query patterns, > resulting in few static rules able to applied > * Static rules created at first may not correct or user query patterns are > changing dynamically > The goal of cube planner is to reduce effort for cube admins to design an > effective partial cube. It owns the following advantages: > * No sophisticated combination of static rules is needed > * Allow more dimensions to be included with few static rules > * Able to adjust a non-cost-effective cuboid set to a cost-effective one > based on historical queries -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (KYLIN-2747) Fail to start kylin if current work directory contains file or directory named "hadoop"
[ https://issues.apache.org/jira/browse/KYLIN-2747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liyang updated KYLIN-2747: -- Labels: scope (was: ) > Fail to start kylin if current work directory contains file or directory > named "hadoop" > --- > > Key: KYLIN-2747 > URL: https://issues.apache.org/jira/browse/KYLIN-2747 > Project: Kylin > Issue Type: Bug >Reporter: Kaige Liu >Assignee: Kaige Liu > Labels: scope > > OS: ubuntu 14.04 > Reproduce steps: > 1. touch hadoop > 2. bin/kylin.sh start > root@hn0-ambari:~/kap-2.4.0-GA-hbase1.x# touch hadoop > root@hn0-ambari:~/kap-2.4.0-GA-hbase1.x# bin/kylin.sh start > Retrieving hive dependency... > Retrieving Spark dependency... > Retrieving hbase dependency... > Exception in thread "main" java.io.FileNotFoundException: > /tmp/kylin-env-diff-7420732488608534086.sh.props (No such file or directory) > at java.io.FileInputStream.open0(Native Method) > at java.io.FileInputStream.open(FileInputStream.java:195) > at java.io.FileInputStream.(FileInputStream.java:138) > at java.io.FileInputStream.(FileInputStream.java:93) > at > io.kyligence.kap.engine.mr.tool.DumpHadoopSystemProps.readAndDelete(SourceFile:131) > at > io.kyligence.kap.engine.mr.tool.DumpHadoopSystemProps.diffSystemProps(SourceFile:109) > at > io.kyligence.kap.engine.mr.tool.DumpHadoopSystemProps.main(SourceFile:69) > Faild to run io.kyligence.kap.engine.mr.tool.DumpHadoopSystemProps -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2739) Add a property, time zone, for the partition column of model
[ https://issues.apache.org/jira/browse/KYLIN-2739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097218#comment-16097218 ] liyang commented on KYLIN-2739: --- Can the so-called system cube stores UTC time just like other normal cubes? That will greatly simplify the storage. There is no good reason to distinguish timezone at storage level, but complexity. We could always convert PST into UTC and the store it. Right? > Add a property, time zone, for the partition column of model > - > > Key: KYLIN-2739 > URL: https://issues.apache.org/jira/browse/KYLIN-2739 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: v2.0.0 >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2672) Only clean necessary cache for CubeMigrationCLI
[ https://issues.apache.org/jira/browse/KYLIN-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097175#comment-16097175 ] kangkaisen commented on KYLIN-2672: --- KYLIN-2717 is great. After KYLIN-2717, we could only reload the related tables. I will update this patch when you finish KYLIN-2717. Thanks you. > Only clean necessary cache for CubeMigrationCLI > --- > > Key: KYLIN-2672 > URL: https://issues.apache.org/jira/browse/KYLIN-2672 > Project: Kylin > Issue Type: Improvement > Components: Tools, Build and Test >Affects Versions: v2.0.0 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2672.patch > > > Currently, we simply clear ALL cache in CubeMigrationCLI. which will make a > few of queries slower in prod env when we have many tables, models, cubes and > migrate cube often. > So, we could only clean necessary cache for CubeMigrationCLI. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2744) Should return correct type for SUM measure in web
[ https://issues.apache.org/jira/browse/KYLIN-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097168#comment-16097168 ] kangkaisen commented on KYLIN-2744: --- Thanks liyang and Billy for your comment. I think the column type and sum measure type should be same in web, and back-end use current Ingester and Aggregator handle sum measure, and the final result will consistent with Presto and Hive. which is reasonable, user-friendly and easy. As for the sum for double type is not fully precise，If user want to get fully precise result, user should use decimal type in Hive. > Should return correct type for SUM measure in web > - > > Key: KYLIN-2744 > URL: https://issues.apache.org/jira/browse/KYLIN-2744 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v2.0.0 >Reporter: kangkaisen >Assignee: kangkaisen > Attachments: KYLIN-2744.patch > > > Currently, Kylin return decimal type for the sum measure of double type, > which will result in wrong result. So, We should return correct type for SUM > measure in web. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KYLIN-2653) Spark cubing support HBase cluster with kerberos
[
https://issues.apache.org/jira/browse/KYLIN-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097154#comment-16097154
]
kangkaisen commented on KYLIN-2653:
---
liyang. Thanks for your review.
I agree with you.
1 There no doubt that {{KylinConfig}} and {{KylinConfigBase}} should be
real-only. I have rolled back the signature for {{getAllProperties}} and
{{reloadKylinConfig}}. But I changed the {{getAllProperties}} in KylinConfigExt
to public. I think which is reasonable because we need at least one way to get
the all properties and this operation is read-only. what do you think of ?
2 This commit didn't invoke {KylinConfigBase.setProperty()}}.
> Spark cubing support HBase cluster with kerberos
>
>
> Key: KYLIN-2653
> URL: https://issues.apache.org/jira/browse/KYLIN-2653
> Project: Kylin
> Issue Type: Bug
> Components: Spark Engine
>Affects Versions: v2.0.0
>Reporter: kangkaisen
>Assignee: kangkaisen
>
> Currently, Spark cubing doesn't support HBase cluster with kerberos.
> Temporarily，we could support HBase cluster with kerberos on Yarn client mode,
> because which is easy.
> In the long term，we should avoid access HBase in Spark cubing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
