[jira] [Commented] (KYLIN-3417) Consider replacing ReentrantReadWriteLock with StampedLock
[
https://issues.apache.org/jira/browse/KYLIN-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16624443#comment-16624443
]
Ted Yu commented on KYLIN-3417:
---
For phase I, we don't need to use Optimistic read lock
> Consider replacing ReentrantReadWriteLock with StampedLock
> --
>
> Key: KYLIN-3417
> URL: https://issues.apache.org/jira/browse/KYLIN-3417
> Project: Kylin
> Issue Type: Improvement
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Major
> Fix For: Backlog
>
>
> ReentrantReadWriteLock's are only the right solution when there is long hold
> time due to expensive I/O.
> It is expensive for readers.
> We should see if the lighter {{StampedLock}} can be used instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3447) Upgrade zookeeper to 3.4.13
[ https://issues.apache.org/jira/browse/KYLIN-3447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-3447: -- Description: zookeeper 3.4.13 is being released with the following fixes: ZOOKEEPER-2959 fixes data loss when observer is used ZOOKEEPER-2184 allows ZooKeeper Java clients to work in dynamic IP (container / cloud) environment was: zookeeper 3.4.13 is being released with the following fixes: ZOOKEEPER-2959 fixes data loss when observer is used ZOOKEEPER-2184 allows ZooKeeper Java clients to work in dynamic IP (container / cloud) environment > Upgrade zookeeper to 3.4.13 > --- > > Key: KYLIN-3447 > URL: https://issues.apache.org/jira/browse/KYLIN-3447 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Priority: Major > > zookeeper 3.4.13 is being released with the following fixes: > ZOOKEEPER-2959 fixes data loss when observer is used > ZOOKEEPER-2184 allows ZooKeeper Java clients to work in dynamic IP (container > / cloud) > environment -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3484) Update Hadoop version to 2.7.7
[ https://issues.apache.org/jira/browse/KYLIN-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-3484: -- Description: We should upgrade the Hadoop 2.7 dependency to 2.7.7, to pick up bug and security fixes . (was: We should upgrade the Hadoop 2.7 dependency to 2.7.7, to pick up bug and security fixes.) > Update Hadoop version to 2.7.7 > -- > > Key: KYLIN-3484 > URL: https://issues.apache.org/jira/browse/KYLIN-3484 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Priority: Minor > > We should upgrade the Hadoop 2.7 dependency to 2.7.7, to pick up bug and > security fixes . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3519) Upgrade Jacoco version to 0.8.2
[ https://issues.apache.org/jira/browse/KYLIN-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-3519: -- Description: Jacoco 0.8.2 adds Java 11 support: https://github.com/jacoco/jacoco/releases/tag/v0.8.2 Java 11 RC1 is out. We should consider upgrading Jacoco. was: Jacoco 0.8.2 adds Java 11 support: https://github.com/jacoco/jacoco/releases/tag/v0.8.2 Java 11 RC1 is out. We should consider upgrading Jacoco. > Upgrade Jacoco version to 0.8.2 > --- > > Key: KYLIN-3519 > URL: https://issues.apache.org/jira/browse/KYLIN-3519 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Priority: Minor > > Jacoco 0.8.2 adds Java 11 support: >https://github.com/jacoco/jacoco/releases/tag/v0.8.2 > Java 11 RC1 is out. > We should consider upgrading Jacoco. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3559) Use Splitter for splitting String
[ https://issues.apache.org/jira/browse/KYLIN-3559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-3559: -- Description: See http://errorprone.info/bugpattern/StringSplitter for why Splitter is preferred . (was: See http://errorprone.info/bugpattern/StringSplitter for why Splitter is preferred.) > Use Splitter for splitting String > - > > Key: KYLIN-3559 > URL: https://issues.apache.org/jira/browse/KYLIN-3559 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Priority: Major > Fix For: v2.6.0 > > > See http://errorprone.info/bugpattern/StringSplitter for why Splitter is > preferred . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3575) Unclosed Connection in DriverTest
Ted Yu created KYLIN-3575:
-
Summary: Unclosed Connection in DriverTest
Key: KYLIN-3575
URL: https://issues.apache.org/jira/browse/KYLIN-3575
Project: Kylin
Issue Type: Bug
Reporter: Ted Yu
In testCalciteProps :
{code}
KylinConnection conn = (KylinConnection)
driver.connect("jdbc:kylin:test_url/test_db", props);
{code}
The connection should be closed upon return.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3574) Unclosed lookup table in DictionaryGeneratorCLI#processSegment
Ted Yu created KYLIN-3574:
-
Summary: Unclosed lookup table in
DictionaryGeneratorCLI#processSegment
Key: KYLIN-3574
URL: https://issues.apache.org/jira/browse/KYLIN-3574
Project: Kylin
Issue Type: Bug
Reporter: Ted Yu
Here is related code:
{code}
JoinDesc join =
cubeSeg.getModel().getJoinsTree().getJoinByPKSide(lookup);
cubeMgr.getLookupTable(cubeSeg, join);
{code}
The returned lookup table should be closed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3546) Kylin doesn't persist FK to intermediate table if FK is not a dimension
[
https://issues.apache.org/jira/browse/KYLIN-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623349#comment-16623349
]
ASF GitHub Bot commented on KYLIN-3546:
---
hit-lacus commented on a change in pull request #241: KYLIN-3546 Add column
which belongs to root fact table in join relation but lost
URL: https://github.com/apache/kylin/pull/241#discussion_r219443869
##
File path:
source-kafka/src/main/java/org/apache/kylin/source/kafka/KafkaInputBase.java
##
@@ -89,7 +95,28 @@ public DataModelDesc getDataModel() {
@Override
public List getAllColumns() {
-return flatDesc.getFactColumns();
+final Set factTableColumnSet = Sets.newHashSet();
+TableRef rootFactTable = getDataModel().getRootFactTable();
+for (TblColRef col : flatDesc.getAllColumns()) {
+if (col.getTableRef().equals(rootFactTable)) {
+factTableColumnSet.add(col);
+}
+}
+// Add column which belongs to root fact table in join
relation but lost
+for (JoinTableDesc joinTableDesc :
getDataModel().getJoinTables()) {
+JoinDesc jd = joinTableDesc.getJoin();
+for (TblColRef colRef : jd.getForeignKeyColumns()) {
+if (colRef.getTableRef().equals(rootFactTable)) {
+factTableColumnSet.add(colRef);
+}
+}
+for (TblColRef colRef : jd.getPrimaryKeyColumns()) {
Review comment:
OK, I will fix it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Kylin doesn't persist FK to intermediate table if FK is not a dimension
> ---
>
> Key: KYLIN-3546
> URL: https://issues.apache.org/jira/browse/KYLIN-3546
> Project: Kylin
> Issue Type: Bug
> Components: Streaming
>Affects Versions: v2.4.0
>Reporter: wlxie
>Assignee: XiaoXiang Yu
>Priority: Major
> Fix For: v2.4.2, v2.5.1
>
> Attachments: base_2018_09_18_09_23_53.zip, kylin_kafka.txt
>
>
> 各位老师好，
> 在使用kafka的streaming
> table作为事实表创建cube时候，如果维度和指标都不选事实表中的关联键的话，在跑cube作业的时候，创建保存kafka数据的临时表会没有这个关联键，导致后面创建关联表的临时表会报错，具体报错信息请查看附件，谢谢。
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3546) Kylin doesn't persist FK to intermediate table if FK is not a dimension
[
https://issues.apache.org/jira/browse/KYLIN-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623345#comment-16623345
]
ASF GitHub Bot commented on KYLIN-3546:
---
shaofengshi commented on a change in pull request #241: KYLIN-3546 Add column
which belongs to root fact table in join relation but lost
URL: https://github.com/apache/kylin/pull/241#discussion_r219434659
##
File path:
source-kafka/src/main/java/org/apache/kylin/source/kafka/KafkaInputBase.java
##
@@ -89,7 +95,28 @@ public DataModelDesc getDataModel() {
@Override
public List getAllColumns() {
-return flatDesc.getFactColumns();
+final Set factTableColumnSet = Sets.newHashSet();
+TableRef rootFactTable = getDataModel().getRootFactTable();
+for (TblColRef col : flatDesc.getAllColumns()) {
+if (col.getTableRef().equals(rootFactTable)) {
+factTableColumnSet.add(col);
+}
+}
+// Add column which belongs to root fact table in join
relation but lost
+for (JoinTableDesc joinTableDesc :
getDataModel().getJoinTables()) {
+JoinDesc jd = joinTableDesc.getJoin();
+for (TblColRef colRef : jd.getForeignKeyColumns()) {
+if (colRef.getTableRef().equals(rootFactTable)) {
+factTableColumnSet.add(colRef);
+}
+}
+for (TblColRef colRef : jd.getPrimaryKeyColumns()) {
Review comment:
PK columns are on lookup tables, they won't be fact table, so this loop has
no necessary to be here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Kylin doesn't persist FK to intermediate table if FK is not a dimension
> ---
>
> Key: KYLIN-3546
> URL: https://issues.apache.org/jira/browse/KYLIN-3546
> Project: Kylin
> Issue Type: Bug
> Components: Streaming
>Affects Versions: v2.4.0
>Reporter: wlxie
>Assignee: XiaoXiang Yu
>Priority: Major
> Fix For: v2.4.2, v2.5.1
>
> Attachments: base_2018_09_18_09_23_53.zip, kylin_kafka.txt
>
>
> 各位老师好，
> 在使用kafka的streaming
> table作为事实表创建cube时候，如果维度和指标都不选事实表中的关联键的话，在跑cube作业的时候，创建保存kafka数据的临时表会没有这个关联键，导致后面创建关联表的临时表会报错，具体报错信息请查看附件，谢谢。
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KYLIN-3497) Make JDBC Module more testable
[ https://issues.apache.org/jira/browse/KYLIN-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-3497. - Resolution: Fixed > Make JDBC Module more testable > -- > > Key: KYLIN-3497 > URL: https://issues.apache.org/jira/browse/KYLIN-3497 > Project: Kylin > Issue Type: Improvement > Components: Driver - JDBC >Reporter: Ian Hu >Assignee: Ian Hu >Priority: Minor > Fix For: v2.6.0 > > > While I am trying my work about KYLIN-3496, I found it is difficult to test. > I would offer a work to make it more testable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3497) Make JDBC Module more testable
[
https://issues.apache.org/jira/browse/KYLIN-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623341#comment-16623341
]
ASF GitHub Bot commented on KYLIN-3497:
---
shaofengshi closed pull request #199: KYLIN-3497 Make JDBC Module more testable
URL: https://github.com/apache/kylin/pull/199
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/jdbc/pom.xml b/jdbc/pom.xml
index 661356c412..a04d81b4c5 100644
--- a/jdbc/pom.xml
+++ b/jdbc/pom.xml
@@ -52,6 +52,13 @@
test
+
+org.mockito
+mockito-core
+${mockito.version}
+test
+
+
diff --git a/jdbc/src/main/java/org/apache/kylin/jdbc/IRemoteClient.java
b/jdbc/src/main/java/org/apache/kylin/jdbc/IRemoteClient.java
index dfd8d76719..630eedd1fc 100644
--- a/jdbc/src/main/java/org/apache/kylin/jdbc/IRemoteClient.java
+++ b/jdbc/src/main/java/org/apache/kylin/jdbc/IRemoteClient.java
@@ -23,13 +23,12 @@
import java.util.List;
import java.util.Map;
-import org.apache.calcite.avatica.AvaticaParameter;
import org.apache.calcite.avatica.ColumnMetaData;
import org.apache.kylin.jdbc.KylinMeta.KMetaProject;
public interface IRemoteClient extends Closeable {
-public static class QueryResult {
+class QueryResult {
public final List columnMeta;
public final Iterable iterable;
@@ -42,16 +41,16 @@ public QueryResult(List columnMeta,
Iterable iterable) {
/**
* Connect to Kylin restful service. IOException will be thrown if
authentication failed.
*/
-public void connect() throws IOException;
+void connect() throws IOException;
/**
* Retrieve meta data of given project.
*/
-public KMetaProject retrieveMetaData(String project) throws IOException;
+KMetaProject retrieveMetaData(String project) throws IOException;
/**
* Execute query remotely and get back result.
*/
-public QueryResult executeQuery(String sql, List params,
List paramValues, Map queryToggles) throws IOException;
+QueryResult executeQuery(String sql, List paramValues, Map queryToggles) throws IOException;
}
diff --git a/jdbc/src/main/java/org/apache/kylin/jdbc/JdbcFactory.java
b/jdbc/src/main/java/org/apache/kylin/jdbc/JdbcFactory.java
new file mode 100644
index 00..f95a30125a
--- /dev/null
+++ b/jdbc/src/main/java/org/apache/kylin/jdbc/JdbcFactory.java
@@ -0,0 +1,24 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.jdbc;
+
+import org.apache.calcite.avatica.AvaticaFactory;
+
+public interface JdbcFactory extends RemoteClientFactory, AvaticaFactory {
+}
diff --git a/jdbc/src/main/java/org/apache/kylin/jdbc/KylinClient.java
b/jdbc/src/main/java/org/apache/kylin/jdbc/KylinClient.java
index 9f8c7dd24e..b98b2d108c 100644
--- a/jdbc/src/main/java/org/apache/kylin/jdbc/KylinClient.java
+++ b/jdbc/src/main/java/org/apache/kylin/jdbc/KylinClient.java
@@ -34,11 +34,11 @@
import javax.xml.bind.DatatypeConverter;
-import org.apache.calcite.avatica.AvaticaParameter;
import org.apache.calcite.avatica.ColumnMetaData;
import org.apache.calcite.avatica.ColumnMetaData.Rep;
import org.apache.calcite.avatica.ColumnMetaData.ScalarType;
import org.apache.http.HttpResponse;
+import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.methods.HttpRequestBase;
@@ -64,19 +64,20 @@
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.annotations.VisibleForTesting;
public class KylinClient implements IRemoteClient {
private static final Logger logger =
LoggerFactory.getLogger(KylinClient.class);
-private final KylinConnection conn;
+private final KylinConnectionInfo connInfo;
private final
[jira] [Commented] (KYLIN-3497) Make JDBC Module more testable
[ https://issues.apache.org/jira/browse/KYLIN-3497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623342#comment-16623342 ] ASF subversion and git services commented on KYLIN-3497: Commit 715f819f7f8013578cbe392c7f3e2ace1bf97b38 in kylin's branch refs/heads/master from hujixu [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=715f819 ] KYLIN-3497 Make JDBC Module more testable De-coupling IRemoteClient with KylinConnection. Add dependencies mocking to improve test coverage. > Make JDBC Module more testable > -- > > Key: KYLIN-3497 > URL: https://issues.apache.org/jira/browse/KYLIN-3497 > Project: Kylin > Issue Type: Improvement > Components: Driver - JDBC >Reporter: Ian Hu >Assignee: Ian Hu >Priority: Minor > Fix For: v2.6.0 > > > While I am trying my work about KYLIN-3496, I found it is difficult to test. > I would offer a work to make it more testable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3496) Make calcite extras props available in JDBC Driver
[ https://issues.apache.org/jira/browse/KYLIN-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3496: Affects Version/s: (was: v2.4.0) Fix Version/s: v2.6.0 > Make calcite extras props available in JDBC Driver > -- > > Key: KYLIN-3496 > URL: https://issues.apache.org/jira/browse/KYLIN-3496 > Project: Kylin > Issue Type: Improvement > Components: Driver - JDBC, Query Engine >Reporter: Ian Hu >Assignee: Ian Hu >Priority: Minor > Labels: features > Fix For: v2.6.0 > > > Like #KYLIN-3475 , calcite can be configured in the server by > `kylin.properties`, while the JDBC Driver is closer with the real query sql > generation, and if calcite can be configured by jdbc client there would be > more flexable to various situation. Like quoting as mysql or use > mysql-specified SQL grammer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KYLIN-3515) Cubing jobs may interfere with each other if use same hive view
[
https://issues.apache.org/jira/browse/KYLIN-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-3515.
-
Resolution: Fixed
> Cubing jobs may interfere with each other if use same hive view
>
>
> Key: KYLIN-3515
> URL: https://issues.apache.org/jira/browse/KYLIN-3515
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.4.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Major
> Fix For: v2.6.0
>
> Attachments: Getting started
>
>
> The root cause is for hive view, during cubing, kylin will materialize the
> view by creating an intermediate table(drop intermediate table first). The
> intermediate tables' name is like kylin_intermediate_\{view_name}, that means
> jobs will create tables with the same name if the same view is referenced. So
> one job's intermediate table may be dropped by another job, in such cases,
> error like "table not found" will happen
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3515) Cubing jobs may interfere with each other if use same hive view
[
https://issues.apache.org/jira/browse/KYLIN-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623320#comment-16623320
]
ASF subversion and git services commented on KYLIN-3515:
Commit cf171da0ef30fec80b21cd1cb2247d0c0bd91716 in kylin's branch
refs/heads/master from [~nichunen]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=cf171da ]
KYLIN-3515 Add uuid for materialized table of hive view
> Cubing jobs may interfere with each other if use same hive view
>
>
> Key: KYLIN-3515
> URL: https://issues.apache.org/jira/browse/KYLIN-3515
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
>Affects Versions: v2.4.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Major
> Fix For: v2.6.0
>
> Attachments: Getting started
>
>
> The root cause is for hive view, during cubing, kylin will materialize the
> view by creating an intermediate table(drop intermediate table first). The
> intermediate tables' name is like kylin_intermediate_\{view_name}, that means
> jobs will create tables with the same name if the same view is referenced. So
> one job's intermediate table may be dropped by another job, in such cases,
> error like "table not found" will happen
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3515) Cubing jobs may interfere with each other if use same hive view
[
https://issues.apache.org/jira/browse/KYLIN-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623319#comment-16623319
]
ASF GitHub Bot commented on KYLIN-3515:
---
shaofengshi closed pull request #219: KYLIN-3515 Add uuid for materialized
table of hive view
URL: https://github.com/apache/kylin/pull/219
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java
b/core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java
index 23dcaf37eb..a774c0d333 100755
--- a/core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java
+++ b/core-cube/src/main/java/org/apache/kylin/cube/CubeManager.java
@@ -1023,8 +1023,8 @@ public DictionaryInfo saveDictionary(CubeSegment cubeSeg,
TblColRef col, IReadab
return dictAssist.getDictionary(cubeSeg, col);
}
-public SnapshotTable buildSnapshotTable(CubeSegment cubeSeg, String
lookupTable) throws IOException {
-return dictAssist.buildSnapshotTable(cubeSeg, lookupTable);
+public SnapshotTable buildSnapshotTable(CubeSegment cubeSeg, String
lookupTable, String uuid) throws IOException {
+return dictAssist.buildSnapshotTable(cubeSeg, lookupTable, uuid);
}
private TableMetadataManager getMetadataManager() {
@@ -1098,7 +1098,7 @@ private void saveDictionaryInfo(CubeSegment cubeSeg,
TblColRef col, DictionaryIn
return (Dictionary) info.getDictionaryObject();
}
-public SnapshotTable buildSnapshotTable(CubeSegment cubeSeg, String
lookupTable) throws IOException {
+public SnapshotTable buildSnapshotTable(CubeSegment cubeSeg, String
lookupTable, String uuid) throws IOException {
// work on copy instead of cached objects
CubeInstance cubeCopy =
cubeSeg.getCubeInstance().latestCopyForWrite(); // get a latest copy
CubeSegment segCopy = cubeCopy.getSegmentById(cubeSeg.getUuid());
@@ -1107,7 +1107,7 @@ public SnapshotTable buildSnapshotTable(CubeSegment
cubeSeg, String lookupTable)
SnapshotManager snapshotMgr = getSnapshotManager();
TableDesc tableDesc = new
TableDesc(metaMgr.getTableDesc(lookupTable, segCopy.getProject()));
-IReadableTable hiveTable =
SourceManager.createReadableTable(tableDesc);
+IReadableTable hiveTable =
SourceManager.createReadableTable(tableDesc, uuid);
SnapshotTable snapshot = snapshotMgr.buildSnapshot(hiveTable,
tableDesc, cubeSeg.getConfig());
CubeDesc cubeDesc = cubeSeg.getCubeDesc();
diff --git
a/core-cube/src/main/java/org/apache/kylin/cube/cli/DictionaryGeneratorCLI.java
b/core-cube/src/main/java/org/apache/kylin/cube/cli/DictionaryGeneratorCLI.java
index 7fcf3208d1..6de42ac3bd 100644
---
a/core-cube/src/main/java/org/apache/kylin/cube/cli/DictionaryGeneratorCLI.java
+++
b/core-cube/src/main/java/org/apache/kylin/cube/cli/DictionaryGeneratorCLI.java
@@ -42,26 +42,28 @@
private static final Logger logger =
LoggerFactory.getLogger(DictionaryGeneratorCLI.class);
-public static void processSegment(KylinConfig config, String cubeName,
String segmentID, DistinctColumnValuesProvider factTableValueProvider,
DictionaryProvider dictProvider) throws IOException {
+public static void processSegment(KylinConfig config, String cubeName,
String segmentID, String uuid,
+DistinctColumnValuesProvider factTableValueProvider,
DictionaryProvider dictProvider) throws IOException {
CubeInstance cube = CubeManager.getInstance(config).getCube(cubeName);
CubeSegment segment = cube.getSegmentById(segmentID);
-processSegment(config, segment, factTableValueProvider, dictProvider);
+processSegment(config, segment, uuid, factTableValueProvider,
dictProvider);
}
-private static void processSegment(KylinConfig config, CubeSegment
cubeSeg, DistinctColumnValuesProvider factTableValueProvider,
DictionaryProvider dictProvider) throws IOException {
+private static void processSegment(KylinConfig config, CubeSegment
cubeSeg, String uuid,
+DistinctColumnValuesProvider factTableValueProvider,
DictionaryProvider dictProvider) throws IOException {
CubeManager cubeMgr = CubeManager.getInstance(config);
// dictionary
for (TblColRef col :
cubeSeg.getCubeDesc().getAllColumnsNeedDictionaryBuilt()) {
logger.info("Building dictionary for " + col);
IReadableTable inpTable =
factTableValueProvider.getDistinctValuesFor(col);
-
+
Dictionary preBuiltDict = null;
if (dictProvider != null) {
preBuiltDict =
[jira] [Commented] (KYLIN-3546) Kylin doesn't persist FK to intermediate table if FK is not a dimension
[ https://issues.apache.org/jira/browse/KYLIN-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623130#comment-16623130 ] ASF GitHub Bot commented on KYLIN-3546: --- asfgit commented on issue #241: KYLIN-3546 Add column which belongs to root fact table in join relation but lost URL: https://github.com/apache/kylin/pull/241#issuecomment-423427022 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Kylin doesn't persist FK to intermediate table if FK is not a dimension > --- > > Key: KYLIN-3546 > URL: https://issues.apache.org/jira/browse/KYLIN-3546 > Project: Kylin > Issue Type: Bug > Components: Streaming >Affects Versions: v2.4.0 >Reporter: wlxie >Assignee: XiaoXiang Yu >Priority: Major > Fix For: v2.4.2, v2.5.1 > > Attachments: base_2018_09_18_09_23_53.zip, kylin_kafka.txt > > > 各位老师好， > 在使用kafka的streaming > table作为事实表创建cube时候，如果维度和指标都不选事实表中的关联键的话，在跑cube作业的时候，创建保存kafka数据的临时表会没有这个关联键，导致后面创建关联表的临时表会报错，具体报错信息请查看附件，谢谢。 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3546) Kylin doesn't persist FK to intermediate table if FK is not a dimension
[ https://issues.apache.org/jira/browse/KYLIN-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623129#comment-16623129 ] ASF GitHub Bot commented on KYLIN-3546: --- hit-lacus opened a new pull request #241: KYLIN-3546 Add column which belongs to root fact table in join relation but lost URL: https://github.com/apache/kylin/pull/241 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Kylin doesn't persist FK to intermediate table if FK is not a dimension > --- > > Key: KYLIN-3546 > URL: https://issues.apache.org/jira/browse/KYLIN-3546 > Project: Kylin > Issue Type: Bug > Components: Streaming >Affects Versions: v2.4.0 >Reporter: wlxie >Assignee: XiaoXiang Yu >Priority: Major > Fix For: v2.4.2, v2.5.1 > > Attachments: base_2018_09_18_09_23_53.zip, kylin_kafka.txt > > > 各位老师好， > 在使用kafka的streaming > table作为事实表创建cube时候，如果维度和指标都不选事实表中的关联键的话，在跑cube作业的时候，创建保存kafka数据的临时表会没有这个关联键，导致后面创建关联表的临时表会报错，具体报错信息请查看附件，谢谢。 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3546) Kylin doesn't persist FK to intermediate table if FK is not a dimension
[ https://issues.apache.org/jira/browse/KYLIN-3546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623131#comment-16623131 ] ASF GitHub Bot commented on KYLIN-3546: --- asfgit commented on issue #241: KYLIN-3546 Add column which belongs to root fact table in join relation but lost URL: https://github.com/apache/kylin/pull/241#issuecomment-423427023 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Kylin doesn't persist FK to intermediate table if FK is not a dimension > --- > > Key: KYLIN-3546 > URL: https://issues.apache.org/jira/browse/KYLIN-3546 > Project: Kylin > Issue Type: Bug > Components: Streaming >Affects Versions: v2.4.0 >Reporter: wlxie >Assignee: XiaoXiang Yu >Priority: Major > Fix For: v2.4.2, v2.5.1 > > Attachments: base_2018_09_18_09_23_53.zip, kylin_kafka.txt > > > 各位老师好， > 在使用kafka的streaming > table作为事实表创建cube时候，如果维度和指标都不选事实表中的关联键的话，在跑cube作业的时候，创建保存kafka数据的临时表会没有这个关联键，导致后面创建关联表的临时表会报错，具体报错信息请查看附件，谢谢。 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3556) Interned string should not be used as lock object
[
https://issues.apache.org/jira/browse/KYLIN-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kaige Liu reassigned KYLIN-3556:
-
Assignee: Kaige Liu
> Interned string should not be used as lock object
> -
>
> Key: KYLIN-3556
> URL: https://issues.apache.org/jira/browse/KYLIN-3556
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
>Affects Versions: v2.5.0
>Reporter: Ted Yu
>Assignee: Kaige Liu
>Priority: Minor
> Fix For: v2.5.1
>
>
> In JDBCResourceDAO :
> {code}
> public void execute(Connection connection) throws SQLException {
> synchronized (resPath.intern()) {
> {code}
> Locking on an interned string can cause unexpected locking collisions with
> other part of code.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
