[jira] [Commented] (KYLIN-4188) Parquet as Cube storage V2
[ https://issues.apache.org/jira/browse/KYLIN-4188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16965647#comment-16965647 ] Jiatao Tao commented on KYLIN-4188: --- +1 > Parquet as Cube storage V2 > -- > > Key: KYLIN-4188 > URL: https://issues.apache.org/jira/browse/KYLIN-4188 > Project: Kylin > Issue Type: New Feature >Reporter: nichunen >Assignee: yiming.xu >Priority: Major > Fix For: Future > > > Branch "kylin-on-parquet" has implemented and it's ready for public review > and evaluation, the issue is https://issues.apache.org/jira/browse/KYLIN-3621. > However, as the discussion in > https://sematext.com/opensee/m/Kylin/8WImho86LC1RxadI?subj=Evaluate+Kylin+on+Parquet, > to make it more stable and with higher performance, some tasks of > improvement should be done. > The key tasks may be: > 1. Calculation of queries on Kylin should be all distributed > 2. Remove dictionaries for dimensions > 3. Store value with its data type(not binary) > 4. The new build engine with Spark-SQL -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KYLIN-3960) Only update user when login in LDAP environment
[ https://issues.apache.org/jira/browse/KYLIN-3960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819771#comment-16819771 ] Jiatao Tao commented on KYLIN-3960: --- We have the logic that updates user when users successful login and this may cause problems, We cannot remove the logic directly cuz it sticks with LDAP. So before the updating, we check whether it is LDAP or not first. > Only update user when login in LDAP environment > --- > > Key: KYLIN-3960 > URL: https://issues.apache.org/jira/browse/KYLIN-3960 > Project: Kylin > Issue Type: Improvement > Components: Security >Reporter: Jiatao Tao >Assignee: Jiatao Tao >Priority: Minor > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3960) Only update user when login in LDAP environment
Jiatao Tao created KYLIN-3960: - Summary: Only update user when login in LDAP environment Key: KYLIN-3960 URL: https://issues.apache.org/jira/browse/KYLIN-3960 Project: Kylin Issue Type: Improvement Components: Security Reporter: Jiatao Tao Assignee: Jiatao Tao -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3948) org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V
[ https://issues.apache.org/jira/browse/KYLIN-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao reassigned KYLIN-3948: - Assignee: (was: Jiatao Tao) > org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V > --- > > Key: KYLIN-3948 > URL: https://issues.apache.org/jira/browse/KYLIN-3948 > Project: Kylin > Issue Type: Test > Components: Environment >Affects Versions: v2.5.2 > Environment: hadoop-3.1.1 > apache-hive-2.3.4-bin > hbase-2.1.1 > apache-kylin-2.5.2-bin-hadoop3 >Reporter: praveenece >Priority: Critical > > hadoop-3.1.1 > apache-hive-2.3.4-bin > hbase-2.1.1 > apache-kylin-2.5.2-bin-hadoop3 > Now I tried with above environment, kylin is stated but i unable to load > metadata from hbase to kylin getting this below error in kylin UI. > org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V > 2019-04-09 15:02:44,979 DEBUG [http-nio-7070-exec-9] common.KylinConfig:327 : > KYLIN_CONF property was not set, will seek KYLIN_HOME env variable > 2019-04-09 15:02:50,071 ERROR [http-nio-7070-exec-6] > controller.TableController:190 : > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/conf/Configuration;)V > java.lang.NoSuchMethodError: > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/conf/Configuration;)V > at > org.apache.kylin.source.hive.CLIHiveClient.getMetaStoreClient(CLIHiveClient.java:132) > at > org.apache.kylin.source.hive.CLIHiveClient.getHiveDbNames(CLIHiveClient.java:116) > at > org.apache.kylin.source.hive.HiveMetadataExplorer.listDatabases(HiveMetadataExplorer.java:47) > at > org.apache.kylin.rest.service.TableService.getSourceDbNames(TableService.java:277) > at > org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:188) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3948) org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V
[ https://issues.apache.org/jira/browse/KYLIN-3948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao reassigned KYLIN-3948: - Assignee: Jiatao Tao > org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V > --- > > Key: KYLIN-3948 > URL: https://issues.apache.org/jira/browse/KYLIN-3948 > Project: Kylin > Issue Type: Test > Components: Environment >Affects Versions: v2.5.2 > Environment: hadoop-3.1.1 > apache-hive-2.3.4-bin > hbase-2.1.1 > apache-kylin-2.5.2-bin-hadoop3 >Reporter: praveenece >Assignee: Jiatao Tao >Priority: Critical > > hadoop-3.1.1 > apache-hive-2.3.4-bin > hbase-2.1.1 > apache-kylin-2.5.2-bin-hadoop3 > Now I tried with above environment, kylin is stated but i unable to load > metadata from hbase to kylin getting this below error in kylin UI. > org.apache.hadoop.hive.metastore.HiveMetaStoreClint.(Lorg/apache/hadoop/conf/configuration;)V > 2019-04-09 15:02:44,979 DEBUG [http-nio-7070-exec-9] common.KylinConfig:327 : > KYLIN_CONF property was not set, will seek KYLIN_HOME env variable > 2019-04-09 15:02:50,071 ERROR [http-nio-7070-exec-6] > controller.TableController:190 : > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/conf/Configuration;)V > java.lang.NoSuchMethodError: > org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(Lorg/apache/hadoop/conf/Configuration;)V > at > org.apache.kylin.source.hive.CLIHiveClient.getMetaStoreClient(CLIHiveClient.java:132) > at > org.apache.kylin.source.hive.CLIHiveClient.getHiveDbNames(CLIHiveClient.java:116) > at > org.apache.kylin.source.hive.HiveMetadataExplorer.listDatabases(HiveMetadataExplorer.java:47) > at > org.apache.kylin.rest.service.TableService.getSourceDbNames(TableService.java:277) > at > org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:188) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) > at > org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3562) TS conflict when kylin update metadata in HBase
[
https://issues.apache.org/jira/browse/KYLIN-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819678#comment-16819678
]
Jiatao Tao commented on KYLIN-3562:
---
Hi [~zhao jintao]
As I commented in 25/九月/18 11:09, I also think "The updating operation is
pretty confusing and useless", I wanted to delete this at first but then I was
told that may be used by LDAP, so I changed the update logic, now it will only
update user when the user is changed, you can check out this in the latest code.
And the perfect solution, I think, is separate LDAP, and make a new
implementation.
> TS conflict when kylin update metadata in HBase
> ---
>
> Key: KYLIN-3562
> URL: https://issues.apache.org/jira/browse/KYLIN-3562
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Lingang Deng
>Assignee: Jiatao Tao
>Priority: Major
> Fix For: v2.4.2, v2.5.1
>
> Attachments: image-2018-09-17-16-40-56-212.png,
> image-2018-09-25-15-03-51-009.png, image-2018-09-25-16-43-50-277.png
>
>
> Error log was as follows,
> {code:java}
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting
> conflict /user/admin, expect old TS 1536928877043, but it is 1536928907207
> at
> org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)
> at
> org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:318)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:303)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:282)
> at
> org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192){code}
>
> what disturbs me the most was that the error was happened several hours,
> then my all build job and query job failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3834) Add monitor for curator-based scheduler
Jiatao Tao created KYLIN-3834: - Summary: Add monitor for curator-based scheduler Key: KYLIN-3834 URL: https://issues.apache.org/jira/browse/KYLIN-3834 Project: Kylin Issue Type: Improvement Components: Job Engine Reporter: Jiatao Tao Assignee: Jiatao Tao Fix For: v3.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[ https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733993#comment-16733993 ] Jiatao Tao edited comment on KYLIN-3474 at 1/4/19 9:49 AM: --- Hi [~temple.zhou] In my opinion, the true problem is the wrong update user logic when authenticating. In the past, Kylin will update the user per authentication, and after KYLIN-3562, it has been removed, so there won't be so many updates. And I think it's reasonable to clear cache when the user has been updated. was (Author: aron.tao): Hi [~temple.zhou] In my opinion, the true problem is the wrong update user logic when authenticating. In the past, Kylin will update the user per authentication, and after KYLIN-3562, it has been removed, so there won't be so many updates.(https://issues.apache.org/jira/browse/KYLIN-3562) And I think it's reasonable to clear cache when the user has been updated. > Tableau 10.5 get malformed token (multi-query instance) > --- > > Key: KYLIN-3474 > URL: https://issues.apache.org/jira/browse/KYLIN-3474 > Project: Kylin > Issue Type: Bug > Components: Query Engine, Security >Affects Versions: v2.3.0, v2.4.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > Attachments: KYLIN-3474.master.001.patch, kylin.log, > tableau-malformed-token.png > > > I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the > Tableau Server will get the malformed token error when do query via restful > API. > However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the > error will be gone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3474) Tableau 10.5 get malformed token (multi-query instance)
[ https://issues.apache.org/jira/browse/KYLIN-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733993#comment-16733993 ] Jiatao Tao commented on KYLIN-3474: --- Hi [~temple.zhou] In my opinion, the true problem is the wrong update user logic when authenticating. In the past, Kylin will update the user per authentication, and after KYLIN-3562, it has been removed, so there won't be so many updates.(https://issues.apache.org/jira/browse/KYLIN-3562) And I think it's reasonable to clear cache when the user has been updated. > Tableau 10.5 get malformed token (multi-query instance) > --- > > Key: KYLIN-3474 > URL: https://issues.apache.org/jira/browse/KYLIN-3474 > Project: Kylin > Issue Type: Bug > Components: Query Engine, Security >Affects Versions: v2.3.0, v2.4.0 >Reporter: Temple Zhou >Assignee: Temple Zhou >Priority: Major > Attachments: KYLIN-3474.master.001.patch, kylin.log, > tableau-malformed-token.png > > > I found that when I use Tableau 10.5 with Kylin 2.4.0(multi-query) , the > Tableau Server will get the malformed token error when do query via restful > API. > However, when I use Tableau 10.5 with Kylin 2.4.0(single query instance), the > error will be gone. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3719) 使用superset的时候执行查询语句失败
[ https://issues.apache.org/jira/browse/KYLIN-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720238#comment-16720238 ] Jiatao Tao commented on KYLIN-3719: --- And do remember to keep the case, cuz the content wrapped by "" means it's case sensitivity. > 使用superset的时候执行查询语句失败 > - > > Key: KYLIN-3719 > URL: https://issues.apache.org/jira/browse/KYLIN-3719 > Project: Kylin > Issue Type: Bug >Affects Versions: v2.5.2 >Reporter: 王振强 >Priority: Major > Attachments: menu.saveimg.savepath20181213211434.jpg > > > 使用superset加载kylin 查询时superset发出的sql的目标table是带""的，kylin执行会报错 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3669) Add log to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiatao Tao updated KYLIN-3669:
--
Summary: Add log to GTStreamAggregateScanner (was: Add logs to
GTStreamAggregateScanner)
> Add log to GTStreamAggregateScanner
> ---
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3669) Add logs to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiatao Tao updated KYLIN-3669:
--
Description:
There's no log printing in GTStreamAggregateScanner, and you know nothing about
whether you go to this path or not, it's pretty unfriendly for trouble shooting.
Code in _SegmentCubeTupleIterator_.
{code:java}
boolean hasMultiplePartitions = records instanceof
SortMergedPartitionResultIterator;
if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
...
// input records are ordered, leverage stream aggregator to produce
possibly fewer records
try (GTStreamAggregateScanner aggregator = new
GTStreamAggregateScanner(inputScanner, scanRequest)) {
result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
} catch (IOException ioe) {
// implementation of close method of anonymous IGTScanner is empty, no
way throw exception
throw new IllegalStateException("IOException is not expected here.",
ioe);
}
return result;
}
{code}
was:
There's no log printing in GTStreamAggregateScanner, and you know nothing about
whether you go to this path or not, it's pretty unfriendly for trouble shooting.
Code in _SegmentCubeTupleIterator_.
{code:java}
boolean hasMultiplePartitions = records instanceof
SortMergedPartitionResultIterator;
if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
// input records are ordered, leverage stream aggregator to produce
possibly fewer records
IGTScanner inputScanner = new IGTScanner() {
public GTInfo getInfo() {
return scanRequest.getInfo();
}
public void close() {
// Underlying resource is hold by scanner and it will be
closed at
// SegmentCubeTupleIterator#close, caller is
SequentialCubeTupleIterator
}
public Iterator iterator() {
return records;
}
};
Iterator result;
try (GTStreamAggregateScanner aggregator = new
GTStreamAggregateScanner(inputScanner, scanRequest)) {
result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
} catch (IOException ioe) {
// implementation of close method of anonymous IGTScanner is
empty, no way throw exception
throw new IllegalStateException("IOException is not expected
here.", ioe);
}
return result;
}
{code}
> Add logs to GTStreamAggregateScanner
>
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> ...
> // input records are ordered, leverage stream aggregator to produce
> possibly fewer records
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is empty,
> no way throw exception
> throw new IllegalStateException("IOException is not expected here.",
> ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3669) Add logs to GTStreamAggregateScanner
[
https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiatao Tao updated KYLIN-3669:
--
Description:
There's no log printing in GTStreamAggregateScanner, and you know nothing about
whether you go to this path or not, it's pretty unfriendly for trouble shooting.
Code in _SegmentCubeTupleIterator_.
{code:java}
boolean hasMultiplePartitions = records instanceof
SortMergedPartitionResultIterator;
if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
// input records are ordered, leverage stream aggregator to produce
possibly fewer records
IGTScanner inputScanner = new IGTScanner() {
public GTInfo getInfo() {
return scanRequest.getInfo();
}
public void close() {
// Underlying resource is hold by scanner and it will be
closed at
// SegmentCubeTupleIterator#close, caller is
SequentialCubeTupleIterator
}
public Iterator iterator() {
return records;
}
};
Iterator result;
try (GTStreamAggregateScanner aggregator = new
GTStreamAggregateScanner(inputScanner, scanRequest)) {
result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
} catch (IOException ioe) {
// implementation of close method of anonymous IGTScanner is
empty, no way throw exception
throw new IllegalStateException("IOException is not expected
here.", ioe);
}
return result;
}
{code}
was:
There's no log printing in GTStreamAggregateScanner, and you cannot know that
you go to that path, it's pretty confusing.
> Add logs to GTStreamAggregateScanner
>
>
> Key: KYLIN-3669
> URL: https://issues.apache.org/jira/browse/KYLIN-3669
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
>Reporter: Jiatao Tao
>Assignee: Jiatao Tao
>Priority: Trivial
>
> There's no log printing in GTStreamAggregateScanner, and you know nothing
> about whether you go to this path or not, it's pretty unfriendly for trouble
> shooting.
> Code in _SegmentCubeTupleIterator_.
> {code:java}
> boolean hasMultiplePartitions = records instanceof
> SortMergedPartitionResultIterator;
> if (hasMultiplePartitions && context.isStreamAggregateEnabled()) {
> // input records are ordered, leverage stream aggregator to
> produce possibly fewer records
> IGTScanner inputScanner = new IGTScanner() {
> public GTInfo getInfo() {
> return scanRequest.getInfo();
> }
> public void close() {
> // Underlying resource is hold by scanner and it will be
> closed at
> // SegmentCubeTupleIterator#close, caller is
> SequentialCubeTupleIterator
> }
> public Iterator iterator() {
> return records;
> }
> };
> Iterator result;
> try (GTStreamAggregateScanner aggregator = new
> GTStreamAggregateScanner(inputScanner, scanRequest)) {
> result = aggregator.valuesIterator(gtDimsIdx, gtMetricsIdx);
> } catch (IOException ioe) {
> // implementation of close method of anonymous IGTScanner is
> empty, no way throw exception
> throw new IllegalStateException("IOException is not expected
> here.", ioe);
> }
> return result;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3669) Add logs to GTStreamAggregateScanner
[ https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao updated KYLIN-3669: -- Description: There's no log printing in GTStreamAggregateScanner, and you cannot know that you go to that path, it's pretty confusing. was: There's no log printing in GTStreamAggregateScanner, and you cannot know that you go to that path, it's pretty confusing. !image-2018-11-06-23-00-57-775.png! > Add logs to GTStreamAggregateScanner > > > Key: KYLIN-3669 > URL: https://issues.apache.org/jira/browse/KYLIN-3669 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Reporter: Jiatao Tao >Assignee: Jiatao Tao >Priority: Trivial > > There's no log printing in GTStreamAggregateScanner, and you cannot know that > you go to that path, it's pretty confusing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3669) Add logs to GTStreamAggregateScanner
[ https://issues.apache.org/jira/browse/KYLIN-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiatao Tao updated KYLIN-3669: -- Attachment: (was: image-2018-11-06-23-00-57-775.png) > Add logs to GTStreamAggregateScanner > > > Key: KYLIN-3669 > URL: https://issues.apache.org/jira/browse/KYLIN-3669 > Project: Kylin > Issue Type: Improvement > Components: Query Engine >Reporter: Jiatao Tao >Assignee: Jiatao Tao >Priority: Trivial > > There's no log printing in GTStreamAggregateScanner, and you cannot know that > you go to that path, it's pretty confusing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3669) Add logs to GTStreamAggregateScanner
Jiatao Tao created KYLIN-3669: - Summary: Add logs to GTStreamAggregateScanner Key: KYLIN-3669 URL: https://issues.apache.org/jira/browse/KYLIN-3669 Project: Kylin Issue Type: Improvement Components: Query Engine Reporter: Jiatao Tao Assignee: Jiatao Tao Attachments: image-2018-11-06-23-00-57-775.png There's no log printing in GTStreamAggregateScanner, and you cannot know that you go to that path, it's pretty confusing. !image-2018-11-06-23-00-57-775.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-2898) Introduce memcached as a distributed cache for queries
[ https://issues.apache.org/jira/browse/KYLIN-2898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674747#comment-16674747 ] Jiatao Tao edited comment on KYLIN-2898 at 11/5/18 7:06 AM: Hi [~Wang Ken], there's a critical bug reported by sonar, and I offer a fix, can you take a look and confirm the change? Thanks. https://github.com/apache/kylin/pull/326/files was (Author: aron.tao): Hi [~Wang Ken], there's a problem reported by sonar, and I offer a fix, can you take a look and confirm the change? Thanks. https://github.com/apache/kylin/pull/326/files > Introduce memcached as a distributed cache for queries > -- > > Key: KYLIN-2898 > URL: https://issues.apache.org/jira/browse/KYLIN-2898 > Project: Kylin > Issue Type: Sub-task > Components: Query Engine >Reporter: Zhong Yanghong >Assignee: Wang Ken >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2898) Introduce memcached as a distributed cache for queries
[ https://issues.apache.org/jira/browse/KYLIN-2898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16674747#comment-16674747 ] Jiatao Tao commented on KYLIN-2898: --- Hi [~Wang Ken], there's a problem reported by sonar, and I offer a fix, can you take a look and confirm the change? Thanks. https://github.com/apache/kylin/pull/326/files > Introduce memcached as a distributed cache for queries > -- > > Key: KYLIN-2898 > URL: https://issues.apache.org/jira/browse/KYLIN-2898 > Project: Kylin > Issue Type: Sub-task > Components: Query Engine >Reporter: Zhong Yanghong >Assignee: Wang Ken >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673899#comment-16673899 ] Jiatao Tao commented on KYLIN-3597: --- A new CRITICAL bug report by sonar: Make sure this expression can't be zero before doing this division. I'll fix this. > Fix sonar reported static code issues > - > > Key: KYLIN-3597 > URL: https://issues.apache.org/jira/browse/KYLIN-3597 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shaofeng SHI >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3657) sql length over 64 KB cause query fail
[ https://issues.apache.org/jira/browse/KYLIN-3657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671031#comment-16671031 ] Jiatao Tao commented on KYLIN-3657: --- Janino is the lib that Calcite used for compiling the generated code when processing the query. It's Calcite's limit, maybe we can seek help from them. > sql length over 64 KB cause query fail > -- > > Key: KYLIN-3657 > URL: https://issues.apache.org/jira/browse/KYLIN-3657 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v2.4.0 >Reporter: wangxianbin >Priority: Major > > at org.apache.calcite.avatica.Helper.wrap(Helper.java:37) > at > org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:108) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1281) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:331) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:228) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:801) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:656) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:618) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.prepareStatement_(CalciteConnectionImpl.java:196) > ... 84 more > Caused by: org.codehaus.janino.JaninoRuntimeException: Code attribute in > class "Baz$1$1" grows beyond 64 KB > at org.codehaus.janino.CodeContext.makeSpace(CodeContext.java:936) > at org.codehaus.janino.CodeContext.write(CodeContext.java:830) > at org.codehaus.janino.UnitCompiler.writeOpcode(UnitCompiler.java:10179) > at org.codehaus.janino.UnitCompiler.referenceThis(UnitCompiler.java:8228) > at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:3328) > at org.codehaus.janino.UnitCompiler.access$7800(UnitCompiler.java:183) > at > org.codehaus.janino.UnitCompiler$10.visitQualifiedThisReference(UnitCompiler.java:3265) > at org.codehaus.janino.Java$QualifiedThisReference.accept(Java.java:3336) > at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:3278) > at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:4345) > at org.codehaus.janino.UnitCompiler.compileContext2(UnitCompiler.java:3178) > at org.codehaus.janino.UnitCompiler.access$5600(UnitCompiler.java:183) > at > org.codehaus.janino.UnitCompiler$9.visitFieldAccess(UnitCompiler.java:3140) > at org.codehaus.janino.Java$FieldAccess.accept(Java.java:3232) > at org.codehaus.janino.UnitCompiler.compileContext(UnitCompiler.java:3148) > at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:4344) > at org.codehaus.janino.UnitCompiler.compileGet2(UnitCompiler.java:3904) > at org.codehaus.janino.UnitCompiler.access$6900(UnitCompiler.java:183) > at > org.codehaus.janino.UnitCompiler$10.visitMethodInvocation(UnitCompiler.java:3251) > at org.codehaus.janino.Java$MethodInvocation.accept(Java.java:3974) > at org.codehaus.janino.UnitCompiler.compileGet(UnitCompiler.java:3278) > at org.codehaus.janino.UnitCompiler.compileGetValue(UnitCompiler.java:4345) > at org.codehaus.janino.UnitCompiler.compileBoolean2(UnitCompiler.java:2969) > at org.codehaus.janino.UnitCompiler.access$5000(UnitCompiler.java:183) > at > org.codehaus.janino.UnitCompiler$8.visitBinaryOperation(UnitCompiler.java:2797) > at org.codehaus.janino.Java$BinaryOperation.accept(Java.java:3768) > at org.codehaus.janino.UnitCompiler.compileBoolean(UnitCompiler.java:2830) > at org.codehaus.janino.UnitCompiler.compileBoolean2(UnitCompiler.java:2923) > at org.codehaus.janino.UnitCompiler.access$5000(UnitCompiler.java:183) > at > org.codehaus.janino.UnitCompiler$8.visitBinaryOperation(UnitCompiler.java:2797) > at org.codehaus.janino.Java$BinaryOperation.accept(Java.java:3768) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (KYLIN-3562) TS conflict when kylin update metadata in HBase
[
https://issues.apache.org/jira/browse/KYLIN-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1824#comment-1824
]
Jiatao Tao edited comment on KYLIN-3562 at 10/29/18 7:33 AM:
-
We cannot remove the logic that updates user when user login in directly cuz it
sticks with LDAP.
In this commit:
https://github.com/apache/kylin/commit/3bcbaa88b97cf868ddf8f51a54b015ef3b7cc068,
I refine the logic that updates the user, so in most case, it won't update the
user.
The next step may separate LDAP so the design can be cleaner.
was (Author: aron.tao):
We can not remove the logic that updates user when user login in directly cuz
it sticks with LDAP.
In this commit:
https://github.com/apache/kylin/commit/3bcbaa88b97cf868ddf8f51a54b015ef3b7cc068,
I refine the logic that updates the user, so in most case, it won't update the
user.
The next step may separate LDAP so the design can be cleaner.
> TS conflict when kylin update metadata in HBase
> ---
>
> Key: KYLIN-3562
> URL: https://issues.apache.org/jira/browse/KYLIN-3562
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Lingang Deng
>Assignee: Jiatao Tao
>Priority: Major
> Fix For: Future, v2.4.2, v2.5.1
>
> Attachments: image-2018-09-17-16-40-56-212.png,
> image-2018-09-25-15-03-51-009.png, image-2018-09-25-16-43-50-277.png
>
>
> Error log was as follows,
> {code:java}
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting
> conflict /user/admin, expect old TS 1536928877043, but it is 1536928907207
> at
> org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)
> at
> org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:318)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:303)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:282)
> at
> org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192){code}
>
> what disturbs me the most was that the error was happened several hours,
> then my all build job and query job failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3562) TS conflict when kylin update metadata in HBase
[
https://issues.apache.org/jira/browse/KYLIN-3562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1824#comment-1824
]
Jiatao Tao commented on KYLIN-3562:
---
We can not remove the logic that updates user when user login in directly cuz
it sticks with LDAP.
In this commit:
https://github.com/apache/kylin/commit/3bcbaa88b97cf868ddf8f51a54b015ef3b7cc068,
I refine the logic that updates the user, so in most case, it won't update the
user.
The next step may separate LDAP so the design can be cleaner.
> TS conflict when kylin update metadata in HBase
> ---
>
> Key: KYLIN-3562
> URL: https://issues.apache.org/jira/browse/KYLIN-3562
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.4.0
>Reporter: Lingang Deng
>Assignee: Jiatao Tao
>Priority: Major
> Fix For: Future, v2.4.2, v2.5.1
>
> Attachments: image-2018-09-17-16-40-56-212.png,
> image-2018-09-25-15-03-51-009.png, image-2018-09-25-16-43-50-277.png
>
>
> Error log was as follows,
> {code:java}
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting
> conflict /user/admin, expect old TS 1536928877043, but it is 1536928907207
> at
> org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)
> at
> org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:318)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:303)
> at
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:282)
> at
> org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192){code}
>
> what disturbs me the most was that the error was happened several hours,
> then my all build job and query job failed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
