[jira] [Created] (KYLIN-3407) java.io.FileNotFoundException: File does not exist: hdfs://localhost:9000/**/hive/lib/hive-catalog-core not found
Rahul Midha created KYLIN-3407: -- Summary: java.io.FileNotFoundException: File does not exist: hdfs://localhost:9000/**/hive/lib/hive-catalog-core not found Key: KYLIN-3407 URL: https://issues.apache.org/jira/browse/KYLIN-3407 Project: Kylin Issue Type: Bug Components: Job Engine Reporter: Rahul Midha While creating a cube in kylin i am getting error "java.io.FileNotFoundException: File does not exist: hdfs://localhost:9000/**/hive/lib/hive-catalog-core.jar not found" while file is there in the path, also when i remove that file from the path, the error come for some other jar file. my hadoop version is hadoop2.7.3 ,hive2.3.3, hbase1.1.1, kylin2.3.1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
add na new dimension to the cube
I have a cube that builds data for two months. Now I want to add a new dimension to the cube. Do I have to clear the already created segments? If I don't need to clear the already constructed segments, what should I do?
[jira] [Created] (KYLIN-3406) When the /execute_output/ metadata file sinking to HDFS is deleted, the monitoring page does not display any tasks.
yujie.sun created KYLIN-3406:
Summary: When the /execute_output/ metadata file sinking to HDFS
is deleted, the monitoring page does not display any tasks.
Key: KYLIN-3406
URL: https://issues.apache.org/jira/browse/KYLIN-3406
Project: Kylin
Issue Type: Bug
Components: Metadata
Affects Versions: v2.3.1
Reporter: yujie.sun
Fix For: v2.4.0
*{color:#24292e}Issue description:{color}*
When a single file of Hbase metadata is greater than a certain value, the file
is sunk into HDFS for storage. When files in execute_output/ are deleted in
HDFS, the monitoring page will not display any tasks.
*{color:#24292e}Reproduction step:{color}*
{color:#24292e}1.Use Hbase as metadata storage{color}
{color:#24292e}2.Modify the hbase configuration ,configure
"hbase.client.keyvalue.maxsize" into a small value, such as 524288.{color}
{color:#24292e}3.Create a new build task, after the build task finished,find
the /execute_output hbase metadata path on HDFS,
/\{working_dir}/\{metadata_url}/resources/execute_output/..., and delete one of
the files.{color}
{color:#24292e}4.At this time,{color:#33}the monitoring page will not
display any tasks.{color}{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (KYLIN-3405) kylin mandatory dimension
Lemont created KYLIN-3405: - Summary: kylin mandatory dimension Key: KYLIN-3405 URL: https://issues.apache.org/jira/browse/KYLIN-3405 Project: Kylin Issue Type: Bug Reporter: Lemont I created a simple cube on kylin 2.3.1，the table has three dimensions:pt,biz_time and periods_time ，one measure :online_rider Something trouble me is that when i set biz_time to the mandatory dimension,the sql will execute fail. The sql is : select floor((1524931200 - biz_time)/(30*86400)), count(DISTINCT online_rider) onlineRiders from rider_view where pt = '20180530' and periods_time in(0) and biz_time >1524931200 group by floor((1524931200 - biz_time)*1.0/(30*86400)) . The error is : null while executing sql. But when I remove the condition of periods_time in(0) or floor function the sql will execute success. I don't know what's the relationship between the two. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] shaofengshi closed pull request #154: KYLIN-3388 remove create redistribute flat hive table step if not set…
shaofengshi closed pull request #154: KYLIN-3388 remove create redistribute
flat hive table step if not set…
URL: https://github.com/apache/kylin/pull/154
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
b/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
index 57e1c40f60..a6c6daad71 100644
--- a/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
+++ b/core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java
@@ -189,11 +189,7 @@ public static void
appendJoinStatement(IJoinedFlatTableDesc flatDesc, StringBuil
}
private static void appendDistributeStatement(StringBuilder sql, TblColRef
redistCol) {
-if (redistCol != null) {
-sql.append(" DISTRIBUTE BY ").append(colName(redistCol,
true)).append(";\n");
-} else {
-sql.append(" DISTRIBUTE BY RAND()").append(";\n");
-}
+sql.append(" DISTRIBUTE BY ").append(colName(redistCol,
true)).append(";\n");
}
private static void appendClusterStatement(StringBuilder sql, TblColRef
clusterCol) {
diff --git
a/source-hive/src/main/java/org/apache/kylin/source/hive/HiveMRInput.java
b/source-hive/src/main/java/org/apache/kylin/source/hive/HiveMRInput.java
index 0e791ebe86..8653500b98 100644
--- a/source-hive/src/main/java/org/apache/kylin/source/hive/HiveMRInput.java
+++ b/source-hive/src/main/java/org/apache/kylin/source/hive/HiveMRInput.java
@@ -23,7 +23,6 @@
import java.util.List;
import java.util.Set;
-import com.google.common.collect.Lists;
import org.apache.commons.lang.StringUtils;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
@@ -59,6 +58,7 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import com.google.common.collect.Lists;
import com.google.common.collect.Sets;
public class HiveMRInput implements IMRInput {
@@ -153,7 +153,9 @@ public void
addStepPhase1_CreateFlatTable(DefaultChainedExecutable jobFlow) {
// then count and redistribute
if (cubeConfig.isHiveRedistributeEnabled()) {
-
jobFlow.addTask(createRedistributeFlatHiveTableStep(hiveInitStatements,
cubeName));
+if (flatDesc.getClusterBy() != null ||
flatDesc.getDistributedBy() != null) {
+
jobFlow.addTask(createRedistributeFlatHiveTableStep(hiveInitStatements,
cubeName));
+}
}
// special for hive
@@ -449,8 +451,7 @@ private String cleanUpIntermediateFlatTable(KylinConfig
config) throws IOExcepti
}
config.getCliCommandExecutor().execute(hiveCmdBuilder.build());
rmdirOnHDFS(getExternalDataPaths());
-output.append(
-"Path " + getExternalDataPaths() + " is deleted. \n");
+output.append("Path " + getExternalDataPaths() + " is deleted.
\n");
return output.toString();
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (KYLIN-3404) IllegalArgumentException throw during cube optimization
Chuqian Yu created KYLIN-3404:
-
Summary: IllegalArgumentException throw during cube optimization
Key: KYLIN-3404
URL: https://issues.apache.org/jira/browse/KYLIN-3404
Project: Kylin
Issue Type: Bug
Reporter: Chuqian Yu
{code:java}
2018-06-12 20:53:37,062 ERROR [Scheduler 2088456043 Job
bea409fd-bf7d-49d6-a52a-7fed0919cf96-376] execution.AbstractExecutable:100 :
1th retries for onExecuteFinished fails due to {}
java.lang.IllegalArgumentException
at org.apache.kylin.cube.CubeUpdate.setCubeInstance(CubeUpdate.java:50)
at org.apache.kylin.cube.CubeUpdate.(CubeUpdate.java:41)
at
org.apache.kylin.job.execution.CheckpointExecutable.onExecuteFinished(CheckpointExecutable.java:88)
at
org.apache.kylin.job.execution.AbstractExecutable.onExecuteFinishedWithRetry(AbstractExecutable.java:98)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:175)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2018-06-12 20:53:37,063 ERROR [pool-9-thread-7] threadpool.DefaultScheduler:302
: ExecuteException job:bea409fd-bf7d-49d6-a52a-7fed0919cf96
org.apache.kylin.job.exception.ExecuteException:
java.lang.IllegalArgumentException
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException
at org.apache.kylin.cube.CubeUpdate.setCubeInstance(CubeUpdate.java:50)
at org.apache.kylin.cube.CubeUpdate.(CubeUpdate.java:41)
at
org.apache.kylin.job.execution.CheckpointExecutable.onExecuteFinished(CheckpointExecutable.java:88)
at
org.apache.kylin.job.execution.AbstractExecutable.onExecuteFinishedWithRetry(AbstractExecutable.java:98)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:175)
... 4 more
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] coveralls commented on issue #154: KYLIN-3388 remove create redistribute flat hive table step if not set…
coveralls commented on issue #154: KYLIN-3388 remove create redistribute flat hive table step if not set… URL: https://github.com/apache/kylin/pull/154#issuecomment-396561500 ## Pull Request Test Coverage Report for [Build 3226](https://coveralls.io/builds/17443154) * **0** of **4** **(0.0%)** changed or added relevant lines in **2** files are covered. * **1** unchanged line in **1** file lost coverage. * Overall coverage increased (+**0.002%**) to **24.181%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-job/src/main/java/org/apache/kylin/job/JoinedFlatTable.java](https://coveralls.io/builds/17443154/source?filename=core-job%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fjob%2FJoinedFlatTable.java#L192) | 0 | 1 | 0.0% | [source-hive/src/main/java/org/apache/kylin/source/hive/HiveMRInput.java](https://coveralls.io/builds/17443154/source?filename=source-hive%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fsource%2Fhive%2FHiveMRInput.java#L156) | 0 | 3 | 0.0% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/17443154/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L553) | 1 | 78.12% | | Totals | [](https://coveralls.io/builds/17443154) | | :-- | --: | | Change from base [Build 3225](https://coveralls.io/builds/17438083): | 0.002% | | Covered Lines: | 14800 | | Relevant Lines: | 61205 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #154: KYLIN-3388 remove create redistribute flat hive table step if not set…
codecov-io commented on issue #154: KYLIN-3388 remove create redistribute flat hive table step if not set… URL: https://github.com/apache/kylin/pull/154#issuecomment-396535440 # [Codecov](https://codecov.io/gh/apache/kylin/pull/154?src=pr=h1) Report > Merging [#154](https://codecov.io/gh/apache/kylin/pull/154?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/fd7cc177f78e4b037d57e7a28b749c747485040e?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `0%`. [](https://codecov.io/gh/apache/kylin/pull/154?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #154 +/- ## + Coverage 22.15% 22.15% +<.01% Complexity 4021 4021 Files 1012 1012 Lines 6120761205 -2 Branches 8768 8769 +1 + Hits 1355813559 +1 + Misses4641046407 -3 Partials 1239 1239 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/154?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ain/java/org/apache/kylin/job/JoinedFlatTable.java](https://codecov.io/gh/apache/kylin/pull/154/diff?src=pr=tree#diff-Y29yZS1qb2Ivc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2pvYi9Kb2luZWRGbGF0VGFibGUuamF2YQ==) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...java/org/apache/kylin/source/hive/HiveMRInput.java](https://codecov.io/gh/apache/kylin/pull/154/diff?src=pr=tree#diff-c291cmNlLWhpdmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3NvdXJjZS9oaXZlL0hpdmVNUklucHV0LmphdmE=) | `10.34% <0%> (ø)` | `1 <0> (ø)` | :arrow_down: | | [...a/org/apache/kylin/dict/Number2BytesConverter.java](https://codecov.io/gh/apache/kylin/pull/154/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L051bWJlcjJCeXRlc0NvbnZlcnRlci5qYXZh) | `81.74% <0%> (-0.8%)` | `17% <0%> (-1%)` | | | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/154/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `69.6% <0%> (-0.61%)` | `7% <0%> (ø)` | | | [...g/apache/kylin/source/datagen/ColumnGenerator.java](https://codecov.io/gh/apache/kylin/pull/154/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc291cmNlL2RhdGFnZW4vQ29sdW1uR2VuZXJhdG9yLmphdmE=) | `72.29% <0%> (+1.35%)` | `8% <0%> (ø)` | :arrow_down: | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/154/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `78.37% <0%> (+5.4%)` | `6% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/154?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/154?src=pr=footer). Last update [fd7cc17...5c83e21](https://codecov.io/gh/apache/kylin/pull/154?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Wayne1c opened a new pull request #154: KYLIN-3388 remove create redistribute flat hive table step if not set…
Wayne1c opened a new pull request #154: KYLIN-3388 remove create redistribute flat hive table step if not set… URL: https://github.com/apache/kylin/pull/154 … distribute column This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] asfgit commented on issue #154: KYLIN-3388 remove create redistribute flat hive table step if not set…
asfgit commented on issue #154: KYLIN-3388 remove create redistribute flat hive table step if not set… URL: https://github.com/apache/kylin/pull/154#issuecomment-396530155 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (KYLIN-3403) Querying sample cube with filter "KYLIN_CAL_DT.WEEK_BEG_DT >= CAST('2001-09-09' AS DATE)" returns unexpected empty result set
Yifan Zhang created KYLIN-3403:
--
Summary: Querying sample cube with filter
"KYLIN_CAL_DT.WEEK_BEG_DT >= CAST('2001-09-09' AS DATE)" returns unexpected
empty result set
Key: KYLIN-3403
URL: https://issues.apache.org/jira/browse/KYLIN-3403
Project: Kylin
Issue Type: Bug
Components: Query Engine
Reporter: Yifan Zhang
Assignee: Yifan Zhang
2018-05-16 13:30:17,824 DEBUG [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
cachesync.CachedCrudAssist : Loaded 3 ColumnACL(s) out of 3 resource
2018-05-16 13:30:17,840 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
routing.QueryRouter : Find candidates by table DEFAULT.KYLIN_SALES and
project=learn_kylin :
INVERTED_INDEX[name=kylin_sales_cube],CUBE[name=kylin_sales_cube]
2018-05-16 13:30:17,841 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
routing.QueryRouter : Applying rule: class
org.apache.kylin.query.routing.rules.RemoveBlackoutRealizationsRule,
realizations before:
[INVERTED_INDEX[name=kylin_sales_cube],CUBE[name=kylin_sales_cube]],
realizations after:
[INVERTED_INDEX[name=kylin_sales_cube],CUBE[name=kylin_sales_cube]]
2018-05-16 13:30:17,843 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
routing.QueryRouter : Applying rule: class
org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule,
realizations before:
[INVERTED_INDEX[name=kylin_sales_cube],CUBE[name=kylin_sales_cube]],
realizations after:
[INVERTED_INDEX[name=kylin_sales_cube],CUBE[name=kylin_sales_cube]]
2018-05-16 13:30:17,844 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
rules.RealizationSortRule : INVERTED_INDEX[name=kylin_sales_cube] priority 1
cost 8360. CUBE[name=kylin_sales_cube] priority 1 cost 836.
2018-05-16 13:30:17,844 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
routing.QueryRouter : Applying rule: class
org.apache.kylin.query.routing.rules.RealizationSortRule, realizations before:
[INVERTED_INDEX[name=kylin_sales_cube],CUBE[name=kylin_sales_cube]],
realizations after:
[CUBE[name=kylin_sales_cube],INVERTED_INDEX[name=kylin_sales_cube]]
2018-05-16 13:30:17,844 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
routing.QueryRouter : The realizations remaining:
[CUBE[name=kylin_sales_cube],INVERTED_INDEX[name=kylin_sales_cube]],and the
final chosen one for current olap context 0 is CUBE[name=kylin_sales_cube]
2018-05-16 13:30:18,042 DEBUG [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
enumerator.OLAPEnumerator : query storage...
2018-05-16 13:30:18,082 DEBUG [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
gtrecord.GTCubeStorageQueryBase : Need storage aggregation
2018-05-16 13:30:18,083 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
gtrecord.GTCubeStorageQueryBase : exactAggregation is false because need
storage aggregation
2018-05-16 13:30:18,091 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
lookup.SnapshotManager : Loading snapshotTable from
/table_snapshot/DEFAULT.KYLIN_CAL_DT/2d3d125a-30f7-438f-bede-7212c5666f4a.snapshot,
with loadData: true
2018-05-16 13:30:18,197 DEBUG [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
lookup.SnapshotManager : Loaded snapshot at
/table_snapshot/DEFAULT.KYLIN_CAL_DT/2d3d125a-30f7-438f-bede-7212c5666f4a.snapshot
2018-05-16 13:30:18,426 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
gtrecord.GTCubeStorageQueryBase : Filter column set for query: []
2018-05-16 13:30:18,426 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
gtrecord.GTCubeStorageQueryBase : Filter mask is: 0
2018-05-16 13:30:18,426 DEBUG [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
gtrecord.GTCubeStorageQueryBase : storageLimitLevel set to NO_LIMIT because
derived column require post aggregation: [DEFAULT.KYLIN_SALES.PART_DT]
2018-05-16 13:30:18,426 DEBUG [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
gtrecord.GTCubeStorageQueryBase : storageLimitLevel set to NO_LIMIT because the
query has order by
2018-05-16 13:30:18,427 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
gtrecord.GTCubeStorageQueryBase : Cuboid identified: cube=kylin_sales_cube,
cuboidId=16384, groupsD=[], filterD=[], limitPushdown=2147483647,
limitLevel=NO_LIMIT, storageAggr=true
2018-05-16 13:30:18,429 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
gtrecord.CubeSegmentScanner : Init CubeSegmentScanner for segment
2012010100_20180403152544
2018-05-16 13:30:18,476 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
gtrecord.ScannerWorker : Segment
kylin_sales_cube[2012010100_20180403152544] will be skipped
2018-05-16 13:30:18,477 DEBUG [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
enumerator.OLAPEnumerator : return TupleIterator...
2018-05-16 13:30:18,481 INFO [Query aa20e5fd-8b86-4359-a87a-2a2e7b8c100a-162]
service.QueryService : Processed rows for each storageContext: 0
2018-05-16 13:30:18,481 INFO [Query
[KYLIN-3388] Hive data may become inconsistent after redistribution
Hello Kylin users, Recently Yanghong Zhong from eBay team reported that the source data may become inconsistent after the "Redistribute flat hive table" step. This is caused by a bug in Hive for "distribute by rand()" statement. While Kylin depends on this to make the data distribution more even. For more information, please check: https://issues.apache.org/jira/browse/KYLIN-3388 Before a hot-fix is released, we recommend you disable the redistribution feature to ensure data's accuracy, by setting: kylin.source.hive.redistribute-flat-table=false in conf/kylin.properties. A restart is needed to take effect. Thanks for the attention. -- Best regards, Shaofeng Shi 史少锋
