[jira] [Created] (KYLIN-1196) Prepare release v1.2
Shaofeng SHI created KYLIN-1196: --- Summary: Prepare release v1.2 Key: KYLIN-1196 URL: https://issues.apache.org/jira/browse/KYLIN-1196 Project: Kylin Issue Type: Bug Components: General Reporter: Shaofeng SHI Assignee: Shaofeng SHI -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1206) Support "UNION" in query
Shaofeng SHI created KYLIN-1206: --- Summary: Support "UNION" in query Key: KYLIN-1206 URL: https://issues.apache.org/jira/browse/KYLIN-1206 Project: Kylin Issue Type: Wish Components: Query Engine Affects Versions: v1.1.1, v0.7.1, v0.7.2 Reporter: Shaofeng SHI Assignee: liyang Fix For: Backlog Kylin doesn't support UNION in SQL query now, this should be implemented. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1219) Kylin support SSO with Spring SAML
Shaofeng SHI created KYLIN-1219: --- Summary: Kylin support SSO with Spring SAML Key: KYLIN-1219 URL: https://issues.apache.org/jira/browse/KYLIN-1219 Project: Kylin Issue Type: New Feature Components: REST Service Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.1 Besides LDAP, SSO is widely adopted by enterprises for user authentication; As Kylin rest service is implemented with Spring security framework, it is natural to implement SSO with SAML 2.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1221) Allow user to kill Hadoop job in the job list page
Shaofeng SHI created KYLIN-1221: --- Summary: Allow user to kill Hadoop job in the job list page Key: KYLIN-1221 URL: https://issues.apache.org/jira/browse/KYLIN-1221 Project: Kylin Issue Type: Wish Components: Job Engine Affects Versions: v0.7.1, v0.7.2, v1.0, v1.1 Reporter: Shaofeng SHI Assignee: ZhouQianhao Sometimes user expects to abondon the current running hadoop job, he can choose to kill the related job directly from the Kylin job page, instead of manually run yarn or mapred CLI to kill it. Besides, when user discards a Kylin job, the underlying running hadoop job should also be killed automatically, to release the computing resources in time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1229) CubeMigrationCLI should not need set KYLIN_HOME
Shaofeng SHI created KYLIN-1229:
---
Summary: CubeMigrationCLI should not need set KYLIN_HOME
Key: KYLIN-1229
URL: https://issues.apache.org/jira/browse/KYLIN-1229
Project: Kylin
Issue Type: Bug
Affects Versions: v1.0, v1.1, v1.2
Reporter: Shaofeng SHI
Assignee: Shaofeng SHI
Priority: Minor
Fix For: v1.3
When release a cube from one Kylin server to another Kylin server, if didn't
set KYLIN_HOME environment variable, it will throw exception:
{code}
hbase -Dlog4j.configuration=/conf/log4j.properties
org.apache.hadoop.util.RunJar lib/kylin-job-1.2-SNAPSHOT.jar
org.apache.kylin.job.tools.CubeMigrationCLI kylin-qa:80 kylin-prod:80
CUBE_NAME PROJ_NAME true true false
L4J [2015-12-14
00:44:53,578][ERROR][org.apache.kylin.common.persistence.ResourceStore] -
Create new store instance failed
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:76)
at
org.apache.kylin.job.tools.CubeMigrationCLI.moveCube(CubeMigrationCLI.java:94)
at
org.apache.kylin.job.tools.CubeMigrationCLI.moveCube(CubeMigrationCLI.java:138)
at
org.apache.kylin.job.tools.CubeMigrationCLI.main(CubeMigrationCLI.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.RuntimeException: Didn't find KYLIN_CONF or KYLIN_HOME,
please set one of them
at
org.apache.kylin.common.KylinConfig.getKylinProperties(KylinConfig.java:600)
at
org.apache.kylin.common.KylinConfig.getKylinPropertiesAsInputSteam(KylinConfig.java:608)
at
org.apache.kylin.common.KylinConfig.loadKylinConfig(KylinConfig.java:249)
at
org.apache.kylin.common.KylinConfig.getInstanceFromEnv(KylinConfig.java:145)
at
org.apache.kylin.common.util.HadoopUtil.getCurrentHBaseConfiguration(HadoopUtil.java:56)
at
org.apache.kylin.common.persistence.HBaseConnection.get(HBaseConnection.java:68)
at
org.apache.kylin.common.persistence.HBaseResourceStore.getConnection(HBaseResourceStore.java:81)
at
org.apache.kylin.common.persistence.HBaseResourceStore.createHTableIfNeeded(HBaseResourceStore.java:100)
at
org.apache.kylin.common.persistence.HBaseResourceStore.(HBaseResourceStore.java:95)
... 13 more
Exception in thread "main" java.lang.IllegalArgumentException: Failed to find
metadata store by url: kylin_metadata_qa@hbase
at
org.apache.kylin.common.persistence.ResourceStore.getStore(ResourceStore.java:91)
at
org.apache.kylin.job.tools.CubeMigrationCLI.moveCube(CubeMigrationCLI.java:94)
at
org.apache.kylin.job.tools.CubeMigrationCLI.moveCube(CubeMigrationCLI.java:138)
at
org.apache.kylin.job.tools.CubeMigrationCLI.main(CubeMigrationCLI.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
{code}
The problem is between HBaseResourceStore.getConnection() and
HBaseConnection.get(url), which depends on an implicit KylinConfig, insteading
using the given KylinConfig.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1230) When CubeMigrationCLI copied ACL from one env to another, it may not work
Shaofeng SHI created KYLIN-1230: --- Summary: When CubeMigrationCLI copied ACL from one env to another, it may not work Key: KYLIN-1230 URL: https://issues.apache.org/jira/browse/KYLIN-1230 Project: Kylin Issue Type: Bug Reporter: Shaofeng SHI Assignee: Shaofeng SHI Found an issue when using CubeMigrationCLI to migrate a cube from QA to production if selecting “copyACL” as true (this is a newly added feature). The root cause is the project in production has a different uuid as it in QA. The cube’s ACL refers to its parent by uuid. Then if a cube’s ACL is copied from QA, it couldn't find the parent, then an error is thrown. To fix the issue, it should copy the project (if it doesn't exist in the target env), or give up copy ACL when detecting the project uuid is different (and remind user to manually update ACL). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1250) Got NPE when discarding a job
Shaofeng SHI created KYLIN-1250:
---
Summary: Got NPE when discarding a job
Key: KYLIN-1250
URL: https://issues.apache.org/jira/browse/KYLIN-1250
Project: Kylin
Issue Type: Bug
Components: General
Affects Versions: v1.2
Reporter: Shaofeng SHI
Assignee: Shaofeng SHI
Fix For: v1.3
User reported that he couldn't rebuild a cube, Kylin reports "The cube
has running job () please discard it and try again."
Then he tried to discard the job, but got an NullPointerException;
{code}
Caused by: java.lang.NullPointerException
at org.apache.kylin.rest.service.JobService.cancelJob(JobService.java:321)
at
org.apache.kylin.rest.service.JobService$$FastClassByCGLIB$$83a44b2a.invoke()
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:64)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInv
{code}
User said "We haven’t done the purge action. It’s possible that the bug was
triggered when we have made the changes to filtering while the cube was still
in the build process?"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1252) Use org.apache.httpcomponents.httpclient to replace commons-httpclient
Shaofeng SHI created KYLIN-1252: --- Summary: Use org.apache.httpcomponents.httpclient to replace commons-httpclient Key: KYLIN-1252 URL: https://issues.apache.org/jira/browse/KYLIN-1252 Project: Kylin Issue Type: Improvement Components: General Reporter: Shaofeng SHI Assignee: Shaofeng SHI Priority: Minor Fix For: v1.3 After merge the patch of KYLIN-1014, there are two http client library in kylin's pom.xml: org.apache.httpcomponents.httpclient and commons-httpclient; As the development of commons-httpclient is ended, we should replace it with the httpcomponents. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1278) Query engine optimization: skip the condition checking on partition column if a segment's time is totaly included in the query range
Shaofeng SHI created KYLIN-1278: --- Summary: Query engine optimization: skip the condition checking on partition column if a segment's time is totaly included in the query range Key: KYLIN-1278 URL: https://issues.apache.org/jira/browse/KYLIN-1278 Project: Kylin Issue Type: Improvement Components: Query Engine Reporter: Shaofeng SHI Assignee: liyang Fix For: Backlog See an example: a cube (which is partitioned by date column "CAL_DT") has three segments: S1, S2, S3; Now a query like "select from fact_table where CAL_DT > '-mm-dd' group by dimA, dimB..." comes, and the "CAL_DT" is only used for filtering, not in group by; By checking the startTime/endTime, Kylin found S2 and S3 need be scanned, and S3 is totally match the condition "CAL_DT > -mm-dd". Then when scan S3, Kylin can ignore the condition on "CAL_DT", selecting a smaller cuboid, this should gain better performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1281) Add "endDate" for cube descriptor
Shaofeng SHI created KYLIN-1281: --- Summary: Add "endDate" for cube descriptor Key: KYLIN-1281 URL: https://issues.apache.org/jira/browse/KYLIN-1281 Project: Kylin Issue Type: Improvement Components: Metadata Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.0 For some legacy cube, we expect set an "endDate" for it and don't allow building beyond that date. So we can create new cube from that date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1292) Create a doc for all properties in conf/kylin.properties
Shaofeng SHI created KYLIN-1292: --- Summary: Create a doc for all properties in conf/kylin.properties Key: KYLIN-1292 URL: https://issues.apache.org/jira/browse/KYLIN-1292 Project: Kylin Issue Type: Improvement Components: Documentation Reporter: Shaofeng SHI Need a central place to introduce the variables in kylin.properties, list the information about: name, value options, default value, impact, since version, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1311) Stream cubing auto assignment and load balance
Shaofeng SHI created KYLIN-1311: --- Summary: Stream cubing auto assignment and load balance Key: KYLIN-1311 URL: https://issues.apache.org/jira/browse/KYLIN-1311 Project: Kylin Issue Type: Sub-task Components: Job Engine Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.1 This is a sub task of KYLIN-1117, need assign the stream cubing job to cluster for load balance and fail over. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1322) Wrong package name for TimedJsonStreamParser
Shaofeng SHI created KYLIN-1322: --- Summary: Wrong package name for TimedJsonStreamParser Key: KYLIN-1322 URL: https://issues.apache.org/jira/browse/KYLIN-1322 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Zhong,Jason Get this error when trigger a streaming job with 2.x-staging: 2016-01-15 09:28:58,567 ERROR [main StreamingCLI:97]: error start streaming java.lang.RuntimeException: failed to create instance of StreamingParser at org.apache.kylin.source.kafka.KafkaStreamingInput.getBatchWithTimeWindow(KafkaStreamingInput.java:101) at org.apache.kylin.engine.streaming.OneOffStreamingBuilder$1.run(OneOffStreamingBuilder.java:72) at org.apache.kylin.engine.streaming.cli.StreamingCLI.startOneOffCubeStreaming(StreamingCLI.java:104) at org.apache.kylin.engine.streaming.cli.StreamingCLI.main(StreamingCLI.java:91) Caused by: java.lang.ClassNotFoundException: org.apache.kylin.streaming.TimedJsonStreamParser at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:190) at org.apache.kylin.source.kafka.StreamingParser.getStreamingParser(StreamingParser.java:82) at org.apache.kylin.source.kafka.KafkaStreamingInput.getBatchWithTimeWindow(KafkaStreamingInput.java:74) ... 3 more And then I found the class has been refactored to package org.apache.kylin.source.kafka, while the generated JSON is still using the old package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1328) DataModel cache should be flushed as other types
Shaofeng SHI created KYLIN-1328: --- Summary: DataModel cache should be flushed as other types Key: KYLIN-1328 URL: https://issues.apache.org/jira/browse/KYLIN-1328 Project: Kylin Issue Type: Bug Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.0 When remove a data model, Kylin will trigger an cache event whose type is "DATA_MODEL", the CacheService.removeCache method is invoked, and then an "UnsupportedOperationException" is thrown, this is not expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1351) Support common RDBMS as data source in Kylin
Shaofeng SHI created KYLIN-1351: --- Summary: Support common RDBMS as data source in Kylin Key: KYLIN-1351 URL: https://issues.apache.org/jira/browse/KYLIN-1351 Project: Kylin Issue Type: New Feature Reporter: Shaofeng SHI >From v2.0, Kylin's plug-in architecture makes it possible to have multiple >data sources, cube engines and storages. Some users ever aksed that whether >Kylin support source data feeded from RDBMS like Oracle, MySQL, now it is >possible to do that. Some tools like Apache Sqoop can easily export data from >RDBMS to HDFS, that would help Kylin get the data and then build that into >cubes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1354) Couldn't edit a cube if it has no "partition date" set
Shaofeng SHI created KYLIN-1354: --- Summary: Couldn't edit a cube if it has no "partition date" set Key: KYLIN-1354 URL: https://issues.apache.org/jira/browse/KYLIN-1354 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v1.2 Reporter: Shaofeng SHI Assignee: Zhong,Jason Create a cube which has no partition date column, save. Edit this cube, in the "Refresh setting" step, the "Partition Date Column" wasn't set (this is correct), the "Start Date" is "1969-12-31" (this is confusing), the button "next" is disabled without error/warning message. After select a partition column it becomes enabled, while this incorrect as this cube isn't partitioned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1355) Add script to build a java-hotfix package, instead of a full package
Shaofeng SHI created KYLIN-1355: --- Summary: Add script to build a java-hotfix package, instead of a full package Key: KYLIN-1355 URL: https://issues.apache.org/jira/browse/KYLIN-1355 Project: Kylin Issue Type: Improvement Components: Environment Reporter: Shaofeng SHI Assignee: hongbin ma Sometimes we need deploy only java code change, but he ./scripts/package.sh will make a full binary package which cost extra time on building the web resources. It would be more efficient if we have a way to package only jar files into a tar ball. Then in the server side, we only replace the jars to take effective. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1367) Use by-layer cubing algorithm if there is memory hungry measure
Shaofeng SHI created KYLIN-1367: --- Summary: Use by-layer cubing algorithm if there is memory hungry measure Key: KYLIN-1367 URL: https://issues.apache.org/jira/browse/KYLIN-1367 Project: Kylin Issue Type: Improvement Components: Job Engine Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1375) Update tutorials for Kylin v2.0
Shaofeng SHI created KYLIN-1375: --- Summary: Update tutorials for Kylin v2.0 Key: KYLIN-1375 URL: https://issues.apache.org/jira/browse/KYLIN-1375 Project: Kylin Issue Type: Task Components: Documentation Affects Versions: v2.0 Reporter: Shaofeng SHI V2.0 has many UI changes, the tutorials need be updated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1377) TopN measure should support more expressions
Shaofeng SHI created KYLIN-1377: --- Summary: TopN measure should support more expressions Key: KYLIN-1377 URL: https://issues.apache.org/jira/browse/KYLIN-1377 Project: Kylin Issue Type: New Feature Reporter: Shaofeng SHI Fix For: v2.1 TopN should support not only SUM, but also MAX, MIN as the expression. A possible case is, find out the sellers which sold the top expensive items: select seller_id, max(price) from sals_records where region = 'US' and year = '2015' order by max(price) desc limit 100; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1378) Add UI for TopN measure
Shaofeng SHI created KYLIN-1378:
---
Summary: Add UI for TopN measure
Key: KYLIN-1378
URL: https://issues.apache.org/jira/browse/KYLIN-1378
Project: Kylin
Issue Type: Sub-task
Components: Web
Reporter: Shaofeng SHI
Assignee: Zhong,Jason
Need the user interface for user to define the TopN. User need to selecting: 1)
the literal column; 2) the metrics column; 3) the expression (default SUM); 4)
the soring order (default Desc).
A sample is:
{
"name" : "TOP_SELLER",
"function" : {
"expression" : "TOP_N",
"parameter" : {
"name": "counter",
"type" : "column",
"value" : "PRICE",
"next_parameter" : {
"name": "literal",
"type" : "column",
"value" : "SELLER_ID",
"next_parameter" : {
"name": "expression",
"type" : "",
"value" : "SUM",
"next_parameter" : {
"name": "order",
"type" : "",
"value" : "DESC"
}
}
},
"returntype" : "topn(100)"
},
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1382) CubeMigrationCLI reports error when migrate cube
Shaofeng SHI created KYLIN-1382:
---
Summary: CubeMigrationCLI reports error when migrate cube
Key: KYLIN-1382
URL: https://issues.apache.org/jira/browse/KYLIN-1382
Project: Kylin
Issue Type: Bug
Components: Client - CLI
Affects Versions: v2.0
Reporter: Shaofeng SHI
Assignee: Shaofeng SHI
Fix For: v2.0
With 2.0-rc code, when migrate a cube, it reports error:
{code}
2016-01-29 00:20:34,451 INFO [main CubeMigrationCLI:282]: Operation index :14
2016-01-29 00:20:34,451 INFO [main CubeMigrationCLI:303]: Executing operation:
COPY_ACL:5c658fb6-e2cf-470d-af35-2b88b80ec694,
fb366d08-d084-4b3e-84f3-d9a3c64e7031, project,
2016-01-29 00:20:34,535 ERROR [main CubeMigrationCLI:286]: error met
java.lang.IllegalArgumentException: to use hbase storage, pls set
'kylin.storage.url=hbase' in kylin.properties
at
org.apache.kylin.storage.hbase.HBaseConnection.newHBaseConfiguration(HBaseConnection.java:88)
at
org.apache.kylin.storage.hbase.HBaseConnection.get(HBaseConnection.java:130)
at
org.apache.kylin.storage.hbase.util.CubeMigrationCLI.doOpt(CubeMigrationCLI.java:425)
at
org.apache.kylin.storage.hbase.util.CubeMigrationCLI.doOpts(CubeMigrationCLI.java:283)
at
org.apache.kylin.storage.hbase.util.CubeMigrationCLI.moveCube(CubeMigrationCLI.java:142)
at
org.apache.kylin.storage.hbase.util.CubeMigrationCLI.moveCube(CubeMigrationCLI.java:150)
at
org.apache.kylin.storage.hbase.util.CubeMigrationCLI.main(CubeMigrationCLI.java:91)
2016-01-29 00:20:34,536 INFO [main CubeMigrationCLI:287]: Try undoing previous
changes
2016-01-29 00:20:34,536 INFO [main CubeMigrationCLI:467]: Undo operation:
COPY_ACL:5c658fb6-e2cf-470d-af35-2b88b80ec694,
fb366d08-d084-4b3e-84f3-d9a3c64e7031, Tracking,
2016-01-29 00:20:34,568 ERROR [main CubeMigrationCLI:293]: error met
java.lang.IllegalArgumentException: to use hbase storage, pls set
'kylin.storage.url=hbase' in kylin.properties
at
org.apache.kylin.storage.hbase.HBaseConnection.newHBaseConfiguration(HBaseConnection.java:88)
at
org.apache.kylin.storage.hbase.HBaseConnection.get(HBaseConnection.java:130)
at
org.apache.kylin.storage.hbase.util.CubeMigrationCLI.doOpt(CubeMigrationCLI.java:425)
at
org.apache.kylin.storage.hbase.util.CubeMigrationCLI.doOpts(CubeMigrationCLI.java:283)
at
org.apache.kylin.storage.hbase.util.CubeMigrationCLI.moveCube(CubeMigrationCLI.java:142)
at
org.apache.kylin.storage.hbase.util.CubeMigrationCLI.moveCube(CubeMigrationCLI.java:150)
at
org.apache.kylin.storage.hbase.util.CubeMigrationCLI.main(CubeMigrationCLI.java:91)
2016-01-29 00:20:34,568 INFO [main CubeMigrationCLI:294]: Continue undoing...
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1383) Evaluate "deploy.env" in kylin.properties
Shaofeng SHI created KYLIN-1383: --- Summary: Evaluate "deploy.env" in kylin.properties Key: KYLIN-1383 URL: https://issues.apache.org/jira/browse/KYLIN-1383 Project: Kylin Issue Type: Improvement Components: General Reporter: Shaofeng SHI In kylin.properties, there is a config "deploy.env", which has three options: DEV|QA|PROD, but there is no explaination on what behaviors/impaction of each. Need re-evaluate this property and decide whether keep it there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1385) Enhance FactDistinctColumnsJob to use multiple reducers
Shaofeng SHI created KYLIN-1385: --- Summary: Enhance FactDistinctColumnsJob to use multiple reducers Key: KYLIN-1385 URL: https://issues.apache.org/jira/browse/KYLIN-1385 Project: Kylin Issue Type: Improvement Components: Job Engine Affects Versions: v1.2, v2.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.1 So far the FactDistinctColumnsJob is using 1 reducer to merge the distinct values for all dimension columns on fact table. When there are 1 or multiple ultra high cardinality columns, it couldn't gain concurrency; It should dispatch the workload to multiple reducers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1387) Streaming cubing doesn't generate cuboids files on HDFS, cause cube merge failure
Shaofeng SHI created KYLIN-1387: --- Summary: Streaming cubing doesn't generate cuboids files on HDFS, cause cube merge failure Key: KYLIN-1387 URL: https://issues.apache.org/jira/browse/KYLIN-1387 Project: Kylin Issue Type: Bug Components: streaming Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.1 In an early version of Kylin streaming, the HBaseMROutput2.java will use HBase Scan API to do the merge. Later Yang refactored that into HBaseMROutput2Transition.java, while keep using the cuboids files on HDFS to do the merge. But the StreamingBatchBuilder didn't generate the cuboids, which causes the merge failed to run. Solution: as a workaround, the StreamingBatchBuilder should leave cuboids files when building micro segments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1392) Support fixedLength encoding for TopN measure
Shaofeng SHI created KYLIN-1392: --- Summary: Support fixedLength encoding for TopN measure Key: KYLIN-1392 URL: https://issues.apache.org/jira/browse/KYLIN-1392 Project: Kylin Issue Type: Sub-task Components: Job Engine, Metadata, Query Engine Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Dong Li Currently TopN measure uses only dictionary encoding for the literal column; It should be made as configurable, so user can select to use fixedLength to encode that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1393) Streaming: add a parameter for "max-gap-to-fill" at cube level
Shaofeng SHI created KYLIN-1393: --- Summary: Streaming: add a parameter for "max-gap-to-fill" at cube level Key: KYLIN-1393 URL: https://issues.apache.org/jira/browse/KYLIN-1393 Project: Kylin Issue Type: Improvement Components: streaming Affects Versions: v2.0 Reporter: Shaofeng SHI Fix For: v2.1 When there is a gap (or say hole) be found in a streaming cube, kylin will try to build a segment for that gap at one time; If the gap is big, it may not be able to finish the building in-memory. So kylin should provide a configurable max-gap-to-fill at the cube level, user can customize that by evaluating the speed that data comes. Then a big gap will be splitted as several small gaps which can be built in sequence. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1407) Update the UI page for calculate table cardinality
Shaofeng SHI created KYLIN-1407: --- Summary: Update the UI page for calculate table cardinality Key: KYLIN-1407 URL: https://issues.apache.org/jira/browse/KYLIN-1407 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v1.0, v1.1, v1.2 Reporter: Shaofeng SHI Assignee: Zhong,Jason There are a couple issues in current "Calculate Cardinality" page: 1. the "Delimiter" and "Format" is not needed since kylin change to use hcatalog to read hive table; they should be removed from UI 2. the page title is inaccurate 3. should allow user to select (or enter) a partition information, so only need scan specific partition to get the rough cardinality info. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1412) Widget width is too small to select
Shaofeng SHI created KYLIN-1412: --- Summary: Widget width is too small to select Key: KYLIN-1412 URL: https://issues.apache.org/jira/browse/KYLIN-1412 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Zhong,Jason -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1413) Row key column's sequence is wrong after saving the cube
Shaofeng SHI created KYLIN-1413: --- Summary: Row key column's sequence is wrong after saving the cube Key: KYLIN-1413 URL: https://issues.apache.org/jira/browse/KYLIN-1413 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v2.1 Reporter: Shaofeng SHI Assignee: Zhong,Jason With the latest build on 2.x-staging, I create a cube, and adjusted the rowkey sequence with my preference, while after saving the cube, I found the sequence was wrong. Even re-edit the cube, still got the issue. This need be double checked against the 2.0-rc to ensure it doesn't exist in 2.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1414) Couldn't drag and drop rowkey, js error is thrown in browser console
Shaofeng SHI created KYLIN-1414:
---
Summary: Couldn't drag and drop rowkey, js error is thrown in
browser console
Key: KYLIN-1414
URL: https://issues.apache.org/jira/browse/KYLIN-1414
Project: Kylin
Issue Type: Bug
Components: Web
Affects Versions: v2.0
Reporter: Shaofeng SHI
Assignee: Zhong,Jason
Fix For: v2.0
Create a new cube with the web wizard, in the "Advanced setting" step, try to
drag and drop a column in the "Rowkeys" section, but found it couldn't be d&d;
In the browser console (Chrome), there is JS error:
{code}
TypeError: element.sortable is not a function
at link (http://kylin-qa/kylin/js/scripts.min.0.js:32925:19)
at nodeLinkFn (http://kylin-qa/kylin/js/scripts.min.0.js:6780:13)
at compositeLinkFn (http://kylin-qa/kylin/js/scripts.min.0.js:6167:13)
at compositeLinkFn (http://kylin-qa/kylin/js/scripts.min.0.js:6170:13)
at publicLinkFn (http://kylin-qa/kylin/js/scripts.min.0.js:6063:30)
at boundTranscludeFn (http://kylin-qa/kylin/js/scripts.min.0.js:6187:21)
at controllersBoundTransclude
(http://kylin-qa/kylin/js/scripts.min.0.js:6801:18)
at Object.ngIfWatchAction [as fn]
(http://kylin-qa/kylin/js/scripts.min.0.js:19799:15)
at Scope.$digest (http://kylin-qa/kylin/js/scripts.min.0.js:12702:29)
at Scope.$apply (http://kylin-qa/kylin/js/scripts.min.0.js:12967:24)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1415) Cube parallel merge
Shaofeng SHI created KYLIN-1415: --- Summary: Cube parallel merge Key: KYLIN-1415 URL: https://issues.apache.org/jira/browse/KYLIN-1415 Project: Kylin Issue Type: Improvement Components: Job Engine, Metadata, streaming Reporter: Shaofeng SHI Assignee: Shaofeng SHI Investigate to remove the restriction on cube merge: only 1 merge job is allowed for a cube at one time. This is important especially for streaming cubing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1417) TimedJsonStreamParser is case sensitive for message's property name
Shaofeng SHI created KYLIN-1417: --- Summary: TimedJsonStreamParser is case sensitive for message's property name Key: KYLIN-1417 URL: https://issues.apache.org/jira/browse/KYLIN-1417 Project: Kylin Issue Type: Bug Components: streaming Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.0 TimedJsonStreamParser is case sensitive for message's property name, this is wrong; It should be case insensitive. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1420) Query returns empty result on partition column's boundary condition
Shaofeng SHI created KYLIN-1420: --- Summary: Query returns empty result on partition column's boundary condition Key: KYLIN-1420 URL: https://issues.apache.org/jira/browse/KYLIN-1420 Project: Kylin Issue Type: Bug Components: Query Engine Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.0 This issue is reported by an user. The cube has a "month_end_dt" column as the partition column. In the first segment, it built the data from 2015-01-01 to 2016-01-01; The second segment it built the data after 2016-01-01. The queries which doesn't have filtering condition works as expected, such as: select month_end_dt,count(*) from fact_table group by month_end_dt order by month_end_dt; The result set includes the records for each month; But when filter on the month_end_date='2015-12-31' it is returning 0 records: select month_end_dt, count(*) from fact_table where month_end_dt='2015-12-31' group by month_end_dt After doing some investigation, identify it is a bug in v2 query engine: In CubeSegmentScanner.getSegmentStartAndEnd(), it uses -1 as 'roundingFlag' for cubeSeg.getDateRangeEnd(), which may get a smaller value than the cubeSegment's end date; While in GTScanRangePlanner.newScanRange(), it still treat the value as a closed condition, that caused the query on boundary condition returns empty directly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1421) Cube "source record" is always zero for streaming
Shaofeng SHI created KYLIN-1421: --- Summary: Cube "source record" is always zero for streaming Key: KYLIN-1421 URL: https://issues.apache.org/jira/browse/KYLIN-1421 Project: Kylin Issue Type: Bug Components: streaming Affects Versions: v2.0 Reporter: Shaofeng SHI The streaming cube engine doesn't collect the source record number, which cause the total record number for the cube is always be zero on UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1423) HBase size precision issue
Shaofeng SHI created KYLIN-1423:
---
Summary: HBase size precision issue
Key: KYLIN-1423
URL: https://issues.apache.org/jira/browse/KYLIN-1423
Project: Kylin
Issue Type: Bug
Components: Web
Affects Versions: v2.0
Reporter: Shaofeng SHI
Assignee: Zhong,Jason
With v2.0, in a cube's "HBase" tab, it shows:
{code}
HTable: KYLIN_KDKGMXLZRJ
Region Count: 2
Size: 260.0 MB
Start Time: 2016-02-16 16:40:00
End Time: 2016-02-16 20:40:00
HTable: KYLIN_FR2EIUHZ28
Region Count: 2
Size: 0 bytes
Start Time: 2016-02-16 20:40:00
End Time: 2016-02-17 00:40:00
{code}
Two issues about the "Size" display:
1) the size has 5 precision is not accurate at all because the nubmer after .
are all 0. Just "260 MB" is good.
2) the second table shows "0 bytes" seems like an empty table, while it has
many records; if the size is small or failed to get the size, show "N/A" would
be better than show "0 bytes"
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1424) Should support multiple selection in picking up dimension/measure column step in data model wizard
Shaofeng SHI created KYLIN-1424: --- Summary: Should support multiple selection in picking up dimension/measure column step in data model wizard Key: KYLIN-1424 URL: https://issues.apache.org/jira/browse/KYLIN-1424 Project: Kylin Issue Type: Improvement Components: Web Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Zhong,Jason I like the UI of "Auto Generator" in "Dimesion" step of the cube wizard, which allows me to select multiple colums at once; hope the UI in data model wizard can follow the same style to give better user experience. Besides, maybe the "Auto Generator" is not the best name, it can be another name like "Add in batch". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1429) First segment couldn't be detected/shown, only after second is built it appears
Shaofeng SHI created KYLIN-1429: --- Summary: First segment couldn't be detected/shown, only after second is built it appears Key: KYLIN-1429 URL: https://issues.apache.org/jira/browse/KYLIN-1429 Project: Kylin Issue Type: Bug Components: streaming Affects Versions: v2.0 Reporter: Shaofeng SHI Create a new streaming cube, start the micro batch build, after the first build finish, on web UI it still shows no segment; After finish the second micro batch, on web it will show the two segments at once; The problem only exists in the first segment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1430) Not add "STREAMING_" prefix when import a streaming table
Shaofeng SHI created KYLIN-1430: --- Summary: Not add "STREAMING_" prefix when import a streaming table Key: KYLIN-1430 URL: https://issues.apache.org/jira/browse/KYLIN-1430 Project: Kylin Issue Type: Bug Components: Metadata, Web Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.0 Now when add a streaming table, when enter a table name, Kylin will automatically add a prefix on saving; This might not be expected, as the query may expect to use the original table name, user need keep control on it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1431) Define stream config at table level, instead of on cube level
Shaofeng SHI created KYLIN-1431: --- Summary: Define stream config at table level, instead of on cube level Key: KYLIN-1431 URL: https://issues.apache.org/jira/browse/KYLIN-1431 Project: Kylin Issue Type: Improvement Components: Metadata, streaming, Web Affects Versions: v2.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI In 2.0 streaming, user need enter the kafka information when create the cube, like the topic, the broker list, etc; while these info should be independent with cube, and can be reused across cubes which share the same table. The expected design is, define kafka config when adding the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1456) Shouldn't use "1970-01-01" as the default end date
Shaofeng SHI created KYLIN-1456: --- Summary: Shouldn't use "1970-01-01" as the default end date Key: KYLIN-1456 URL: https://issues.apache.org/jira/browse/KYLIN-1456 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v1.3 Reporter: Shaofeng SHI Assignee: Zhong,Jason Fix For: v1.3 Attachments: Screen Shot 2016-02-28 at 9.13.44 AM.png Create a new cube, set 2016-01-01 as start date; Build the first segment, kylin shows the dialogue asking for selecting the end date, while the default end date is 1970-01-01. Please see the attachement. Expected behavior: leave blank for the end date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1477) TopN measure should support ascending order
Shaofeng SHI created KYLIN-1477: --- Summary: TopN measure should support ascending order Key: KYLIN-1477 URL: https://issues.apache.org/jira/browse/KYLIN-1477 Project: Kylin Issue Type: Wish Affects Versions: v2.0 Reporter: Shaofeng SHI Fix For: Backlog So far the TopN measure only support in descending order; it should support ascending order, for the case that need to find the lowest metrics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1478) TopN measure should support non-dictionary encoding for ultra high cardinality
Shaofeng SHI created KYLIN-1478: --- Summary: TopN measure should support non-dictionary encoding for ultra high cardinality Key: KYLIN-1478 URL: https://issues.apache.org/jira/browse/KYLIN-1478 Project: Kylin Issue Type: Wish Affects Versions: v2.0 Reporter: Shaofeng SHI Fix For: Future TopN measure uses dictionary to encode the literal column, that may not work when the cardinality is ultra high. Need support other encoding like fixedLength. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1499) Couldn't save query, error in backend
Shaofeng SHI created KYLIN-1499:
---
Summary: Couldn't save query, error in backend
Key: KYLIN-1499
URL: https://issues.apache.org/jira/browse/KYLIN-1499
Project: Kylin
Issue Type: Bug
Components: Web
Affects Versions: v1.5.0
Reporter: Shaofeng SHI
Assignee: Zhong,Jason
Run kylin in sandbox, spring profile "testing"; after run a SQL query, click
"Save" to save the query, there is no response in UI, in kylin.log there is
error:
{code}
2016-03-16 14:11:52,071 ERROR [http-bio-7070-exec-9]
controller.BasicController:44 :
org.apache.hadoop.hbase.TableNotFoundException: kylin_metadata_user
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1352)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1167)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1150)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1107)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:941)
at
org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:78)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:125)
at org.apache.hadoop.hbase.client.HTable.get(HTable.java:854)
at
org.apache.kylin.rest.service.QueryService.getQueries(QueryService.java:188)
at
org.apache.kylin.rest.service.QueryService.saveQuery(QueryService.java:126)
at
org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke()
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
at
org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$fbf31bec.saveQuery()
at
org.apache.kylin.rest.controller.QueryController.saveQuery(QueryController.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1507) Couldn't find hive dependency jar on some platform like CDH
Shaofeng SHI created KYLIN-1507: --- Summary: Couldn't find hive dependency jar on some platform like CDH Key: KYLIN-1507 URL: https://issues.apache.org/jira/browse/KYLIN-1507 Project: Kylin Issue Type: Bug Components: General Affects Versions: v1.5.0 Reporter: Shaofeng SHI Reported by user ianzeng in u...@kylin.apache.org mailing list: I has installed kylin 1.5 on redhead 6.3. I try build sample cube. But got error msg as follow: 2016-03-18 18:18:43,084 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.server.conf; Ignoring. 2016-03-18 18:18:43,093 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 2016-03-18 18:18:43,509 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id 2016-03-18 18:18:43,921 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 1 2016-03-18 18:18:43,933 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ] 2016-03-18 18:18:44,120 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2047) at org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobContextImpl.java:184) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:746) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) Caused by: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.mapreduce.HCatInputFormat not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1953) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2045) ... 8 more And -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1521) Couldn't refresh a cube segment whose start time is before 1970-01-01
Shaofeng SHI created KYLIN-1521: --- Summary: Couldn't refresh a cube segment whose start time is before 1970-01-01 Key: KYLIN-1521 URL: https://issues.apache.org/jira/browse/KYLIN-1521 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v1.3.0, v1.5.0 Reporter: Shaofeng SHI Assignee: Zhong,Jason Fix For: v1.3.1, v1.5.1 When refresh a segment whose start date is before 1970-01-01, the "Submit" button is disabled, user couldn't proceed. The root cause is, in refresh page it checks whether the start time is > 0, if not the button is disabled; This check should be removed to support earlier dates. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1524) Get "java.lang.Double cannot be cast to java.lang.Long" error when Top-N metris data type is BigInt
Shaofeng SHI created KYLIN-1524: --- Summary: Get "java.lang.Double cannot be cast to java.lang.Long" error when Top-N metris data type is BigInt Key: KYLIN-1524 URL: https://issues.apache.org/jira/browse/KYLIN-1524 Project: Kylin Issue Type: Bug Affects Versions: v1.5.0 Reporter: Shaofeng SHI Fix For: v1.5.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1528) Create a branch for v1.5 with HBase 1.1 API
Shaofeng SHI created KYLIN-1528: --- Summary: Create a branch for v1.5 with HBase 1.1 API Key: KYLIN-1528 URL: https://issues.apache.org/jira/browse/KYLIN-1528 Project: Kylin Issue Type: Improvement Components: Storage - HBase Reporter: Shaofeng SHI Assignee: Shaofeng SHI Create a new branch for Kylin v1.5 compile with HBase v1.1 API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1537) Failed to create flat hive table, when name is too long
Shaofeng SHI created KYLIN-1537:
---
Summary: Failed to create flat hive table, when name is too long
Key: KYLIN-1537
URL: https://issues.apache.org/jira/browse/KYLIN-1537
Project: Kylin
Issue Type: Bug
Components: Job Engine
Affects Versions: v1.5.0
Reporter: Shaofeng SHI
Assignee: Shaofeng SHI
Fix For: v1.5.1
Run Kylin regression test on HDP 2.4, Hive version 1.2.1.2.4; A cube couldn't
build, but others can; Got error in the first step:
{code}
OS command error exit with 12 -- hive -e "USE default;
DROP TABLE IF EXISTS
kylin_intermediate_test_kylin_cube_with_slr_desc_1970010100_2015010100;
CREATE EXTERNAL TABLE IF NOT EXISTS
kylin_intermediate_test_kylin_cube_with_slr_desc_1970010100_2015010100
(
DEFAULT_TEST_KYLIN_FACT_CAL_DT date
,DEFAULT_TEST_KYLIN_FACT_LEAF_CATEG_ID bigint
,DEFAULT_TEST_KYLIN_FACT_LSTG_SITE_ID int
,DEFAULT_TEST_CATEGORY_GROUPINGS_META_CATEG_NAME string
,DEFAULT_TEST_CATEGORY_GROUPINGS_CATEG_LVL2_NAME string
,DEFAULT_TEST_CATEGORY_GROUPINGS_CATEG_LVL3_NAME string
,DEFAULT_TEST_KYLIN_FACT_LSTG_FORMAT_NAME string
,DEFAULT_TEST_KYLIN_FACT_SLR_SEGMENT_CD smallint
,DEFAULT_TEST_KYLIN_FACT_SELLER_ID bigint
,DEFAULT_TEST_KYLIN_FACT_PRICE decimal(19,4)
,DEFAULT_TEST_KYLIN_FACT_ITEM_COUNT int
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\177'
STORED AS SEQUENCEFILE
LOCATION
'/kylin/kylin_metadata/kylin-a28a48e4-83a2-4b71-b06d-9a281540507d/kylin_intermediate_test_kylin_cube_with_slr_desc_1970010100_2015010100';
SET dfs.replication=2;
SET hive.exec.compress.output=true;
SET hive.auto.convert.join.noconditionaltask=true;
SET hive.auto.convert.join.noconditionaltask.size=3;
SET hive.merge.mapfiles=true;
SET hive.merge.mapredfiles=true;
SET hive.merge.size.per.task=6400;
INSERT OVERWRITE TABLE
kylin_intermediate_test_kylin_cube_with_slr_desc_1970010100_2015010100
SELECT
TEST_KYLIN_FACT.CAL_DT
,TEST_KYLIN_FACT.LEAF_CATEG_ID
,TEST_KYLIN_FACT.LSTG_SITE_ID
,TEST_CATEGORY_GROUPINGS.META_CATEG_NAME
,TEST_CATEGORY_GROUPINGS.CATEG_LVL2_NAME
,TEST_CATEGORY_GROUPINGS.CATEG_LVL3_NAME
,TEST_KYLIN_FACT.LSTG_FORMAT_NAME
,TEST_KYLIN_FACT.SLR_SEGMENT_CD
,TEST_KYLIN_FACT.SELLER_ID
,TEST_KYLIN_FACT.PRICE
,TEST_KYLIN_FACT.ITEM_COUNT
FROM DEFAULT.TEST_KYLIN_FACT as TEST_KYLIN_FACT
INNER JOIN EDW.TEST_CAL_DT as TEST_CAL_DT
ON TEST_KYLIN_FACT.CAL_DT = TEST_CAL_DT.CAL_DT
INNER JOIN DEFAULT.TEST_CATEGORY_GROUPINGS as TEST_CATEGORY_GROUPINGS
ON TEST_KYLIN_FACT.LEAF_CATEG_ID = TEST_CATEGORY_GROUPINGS.LEAF_CATEG_ID AND
TEST_KYLIN_FACT.LSTG_SITE_ID = TEST_CATEGORY_GROUPINGS.SITE_ID
INNER JOIN EDW.TEST_SITES as TEST_SITES
ON TEST_KYLIN_FACT.LSTG_SITE_ID = TEST_SITES.SITE_ID
INNER JOIN EDW.TEST_SELLER_TYPE_DIM as TEST_SELLER_TYPE_DIM
ON TEST_KYLIN_FACT.SLR_SEGMENT_CD = TEST_SELLER_TYPE_DIM.SELLER_TYPE_CD
WHERE (TEST_KYLIN_FACT.CAL_DT < '2015-01-01')
;
"
WARNING: Use "yarn jar" to launch YARN applications.
Logging initialized using configuration in
file:/etc/hive/2.4.0.0-169/0/hive-log4j.properties
OK
Time taken: 1.812 seconds
OK
Time taken: 5.622 seconds
FAILED: Hive Internal Error:
org.codehaus.jettison.json.JSONException(JSONArray[0] is not a JSONObject.)
org.codehaus.jettison.json.JSONException: JSONArray[0] is not a JSONObject.
at
org.codehaus.jettison.json.JSONArray.getJSONObject(JSONArray.java:258)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getEntityReferenceFromDSL(HiveMetaStoreBridge.java:193)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getTableReference(HiveMetaStoreBridge.java:221)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerTable(HiveMetaStoreBridge.java:276)
at
org.apache.atlas.hive.hook.HiveHook.handleCreateTable(HiveHook.java:271)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:205)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:314)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:711)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodA
[jira] [Created] (KYLIN-1538) DoubleDeltaSerializer case obvious error after deserialize and serialize
Shaofeng SHI created KYLIN-1538: --- Summary: DoubleDeltaSerializer case obvious error after deserialize and serialize Key: KYLIN-1538 URL: https://issues.apache.org/jira/browse/KYLIN-1538 Project: Kylin Issue Type: Bug Components: General Affects Versions: v1.5.0 Reporter: Shaofeng SHI Assignee: liyang Fix For: v1.5.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1540) REST API for deleting segment
Shaofeng SHI created KYLIN-1540: --- Summary: REST API for deleting segment Key: KYLIN-1540 URL: https://issues.apache.org/jira/browse/KYLIN-1540 Project: Kylin Issue Type: New Feature Components: REST Service Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v1.5.1 To allow user have more flexibility in managing the cube segments, Kylin need a REST API to delete segment. But the deletion can only happen on head or tail segment, one segment one time; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1551) Should check and ensure TopN measure has two parameters specified
Shaofeng SHI created KYLIN-1551:
---
Summary: Should check and ensure TopN measure has two parameters
specified
Key: KYLIN-1551
URL: https://issues.apache.org/jira/browse/KYLIN-1551
Project: Kylin
Issue Type: Bug
Components: Web
Affects Versions: v1.5.0
Reporter: Shaofeng SHI
Assignee: Zhong,Jason
Fix For: v1.5.1
User from mailing list reported an error when using TopN:
{code}
when I use TOP-N, exception occured
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.rangeCheck(ArrayList.java:635)
at java.util.ArrayList.get(ArrayList.java:411)
at
org.apache.kylin.measure.topn.TopNMeasureType.getColumnsNeedDictionary(TopNMeasureType.java:170)
at
org.apache.kylin.cube.model.CubeDesc.getAllColumnsNeedDictionary(CubeDesc.java:975)
at
org.apache.kylin.cube.CubeManager.getAllDictColumnsOnFact(CubeManager.java:904)
at
org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:80)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:118)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}
The reason should be: no second parameter be specified for TopN. I can
reproduce it on v1.5.0; The cube GUI should check and report to user when find
such a case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1553) Cannot find rowkey column "COL_NAME" in cube CubeDesc
Shaofeng SHI created KYLIN-1553: --- Summary: Cannot find rowkey column "COL_NAME" in cube CubeDesc Key: KYLIN-1553 URL: https://issues.apache.org/jira/browse/KYLIN-1553 Project: Kylin Issue Type: Bug Components: Metadata, Web Affects Versions: v1.5.0 Reporter: Shaofeng SHI Assignee: Zhong,Jason Fix For: v1.5.1 Create a cube, select a lookup table's PK as dimension column, save the cube Kylin reports error "Cannot find rowkey column SITE_ID in cube CubeDesc [name=my_cube]" Check the JSON of the cube definition, the PK wasn't replaced with the FK on fact table (expected behavior). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1557) Enhance the check on aggregation group dimension number
Shaofeng SHI created KYLIN-1557: --- Summary: Enhance the check on aggregation group dimension number Key: KYLIN-1557 URL: https://issues.apache.org/jira/browse/KYLIN-1557 Project: Kylin Issue Type: Improvement Components: Metadata Affects Versions: v1.5.0 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v1.5.1 If the dimension number in an aggregation group exceeds the number of "kylin.cube.aggrgroup.max.size" (default 12), Kylin will throw an error saying "Aggregation group has too many dimensions"; User has to remove some dimensions to proceed. This check adds up the number of mandantory dimensions, hierarchy, joint and normal dimensions; This is too strong. The check should be optimized, all mandantory + joint dimensions should be treated as 1; hierarchy dimensions should be treated as N; nomal dimensions be 2^N; Control total combination numbers at a certain level. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1560) Make BatchCubingJobBuilder2 easier to add additional step
Shaofeng SHI created KYLIN-1560: --- Summary: Make BatchCubingJobBuilder2 easier to add additional step Key: KYLIN-1560 URL: https://issues.apache.org/jira/browse/KYLIN-1560 Project: Kylin Issue Type: Improvement Components: Job Engine Reporter: Shaofeng SHI Assignee: Shaofeng SHI Priority: Minor Add one or two empty methods in BatchCubingJobBuilder2 to make subclasses can easily insert steps before/after cubing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1566) Add and use a separate kylin_job_conf.xml for in-mem cubing
Shaofeng SHI created KYLIN-1566: --- Summary: Add and use a separate kylin_job_conf.xml for in-mem cubing Key: KYLIN-1566 URL: https://issues.apache.org/jira/browse/KYLIN-1566 Project: Kylin Issue Type: Improvement Components: Job Engine Affects Versions: v1.5.0 Reporter: Shaofeng SHI Assignee: Dong Li >From v1.5.0, kylin has two cubing algorithm: by-layer and by-split(in-mem). The in-mem cubing require more memory resource for the mapper; so far it is configured by parameter "kylin.job.cubing.inmem.mrjob_conf_override" with default value "mapreduce.map.java.opts=-Xmx2700m; mapreduce.map.memory.mb=3072; mapreduce.task.io.sort.mb=200"; This is a little confusing as user can also configure such parameters in kylin_job_conf.xml A better way is adding a new config file for in-mem cubing job, moving the in-mem specific mapreduce parameters there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1567) Front-end work for KYLIN-1557
Shaofeng SHI created KYLIN-1567: --- Summary: Front-end work for KYLIN-1557 Key: KYLIN-1567 URL: https://issues.apache.org/jira/browse/KYLIN-1567 Project: Kylin Issue Type: Improvement Components: Web Affects Versions: v1.5.1 Reporter: Shaofeng SHI Assignee: Zhong,Jason Fix For: v1.5.2 KYLIN-1557 will add check on the aggregation group combinations; If an agg group's combination number exceeds a threshold, it will report error. So the cube desc generated by JS should better be good by default; A simple implementation would be: if user selects <= 12 dimensions, it is okay to put them all in one group's "include" field; If > 12, keep 11 in the "include" field, and move all left to the "joint" list; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1580) Use 1 byte instead of 8 bytes as column indicator in fact distinct MR job
Shaofeng SHI created KYLIN-1580: --- Summary: Use 1 byte instead of 8 bytes as column indicator in fact distinct MR job Key: KYLIN-1580 URL: https://issues.apache.org/jira/browse/KYLIN-1580 Project: Kylin Issue Type: Improvement Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v1.5.2 One small enhancement in fact distinct job; today it uses 8 bytes in the head of the key to indicates the column index as well as cuboid id; actually it can be optimized to 1 byte for column index, and next 8 for cuboid id if the first is a special value; this will reduce the network IO between mappers and reducers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1581) NPE in Job engine when execute MR job
Shaofeng SHI created KYLIN-1581:
---
Summary: NPE in Job engine when execute MR job
Key: KYLIN-1581
URL: https://issues.apache.org/jira/browse/KYLIN-1581
Project: Kylin
Issue Type: Bug
Affects Versions: v1.5.1, v1.5.0
Reporter: Shaofeng SHI
An NPE throws when executing a job step:
{code}
org.apache.kylin.job.exception.ExecuteException: java.lang.NullPointerException
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124)
at
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at
org.apache.kylin.engine.mr.common.MapReduceExecutable.onExecuteStart(MapReduceExecutable.java:77)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:105)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114)
... 4 more
{code}
It looks like that, this step had been executed before, maybe after a restart
it wants to check the job's status from hadoop, but the recorded job id wasn't
found in hadoop, caused this NPE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1582) Add a tutorial for building a streaming cube step by step
Shaofeng SHI created KYLIN-1582: --- Summary: Add a tutorial for building a streaming cube step by step Key: KYLIN-1582 URL: https://issues.apache.org/jira/browse/KYLIN-1582 Project: Kylin Issue Type: Task Reporter: Shaofeng SHI -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1623) Make the hll precision for data samping configurable
Shaofeng SHI created KYLIN-1623: --- Summary: Make the hll precision for data samping configurable Key: KYLIN-1623 URL: https://issues.apache.org/jira/browse/KYLIN-1623 Project: Kylin Issue Type: New Feature Components: Job Engine Reporter: Shaofeng SHI Assignee: Shaofeng SHI now kylin uses hll(14) for sampling; it should be configurable at cube level for small or large cubes. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1631) Web UI of TopN, "group by" column only need fact column
Shaofeng SHI created KYLIN-1631: --- Summary: Web UI of TopN, "group by" column only need fact column Key: KYLIN-1631 URL: https://issues.apache.org/jira/browse/KYLIN-1631 Project: Kylin Issue Type: Bug Components: Web Reporter: Shaofeng SHI Assignee: Zhong,Jason Fix For: v1.5.2 As Top-N is for the long-tail issue, the "group by" column should be from fact table. So far the web ui shows all columns (fact + lookup), that is more than needed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1643) Kylin's hll counter count "NULL" as a value
Shaofeng SHI created KYLIN-1643: --- Summary: Kylin's hll counter count "NULL" as a value Key: KYLIN-1643 URL: https://issues.apache.org/jira/browse/KYLIN-1643 Project: Kylin Issue Type: Bug Components: General Reporter: Shaofeng SHI Assignee: Shaofeng SHI Reported by Xie Linjiong, he found Kylin's HyperLogLog count NULL as 1, but Hive count it as 0; Below is a sample: A. source table: hive> select * from temp.temp_kylin_null_test_20160429; OK 10003 NULL 10002 NULL 10001 male B. hive's query result: hive> select count(distinct gender)],count(*) from temp.temp_kylin_null_test_20160429 where user_id in (10002,10003); OK 0 2 C. Kylin's query result: 1 2 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1646) Column appeared twice if it was declared as both dimension and measure
Shaofeng SHI created KYLIN-1646: --- Summary: Column appeared twice if it was declared as both dimension and measure Key: KYLIN-1646 URL: https://issues.apache.org/jira/browse/KYLIN-1646 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v1.5.0 Reporter: Shaofeng SHI Assignee: Zhong,Jason Attachments: Screen Shot 2016-05-03 at 9.50.50 AM.png Create a data model, define a column as both dimension and measure (this is reasonable as the model may have several cubes). When create the cube measure, it shows the column name twice, please see the screen shot. Expected behavior, the list option should not have duplication. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1647) Purge a cube, and then build again, the start date is not updated
Shaofeng SHI created KYLIN-1647: --- Summary: Purge a cube, and then build again, the start date is not updated Key: KYLIN-1647 URL: https://issues.apache.org/jira/browse/KYLIN-1647 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v1.5.1 Reporter: Shaofeng SHI Assignee: Zhong,Jason Have a cube which has been built; disable it and then click "Purge", the purge got succeed; then I click "Build" to rebuild it, the web UI shows a dialogue to enter the end date; but the start date is not the very begining of this cube, it is the end date of the segment that I just purged. Although refresh the web page can flush the cache, expect it's behavior is correct without extra action. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1660) Streaming/kafka config not match with table name
Shaofeng SHI created KYLIN-1660: --- Summary: Streaming/kafka config not match with table name Key: KYLIN-1660 URL: https://issues.apache.org/jira/browse/KYLIN-1660 Project: Kylin Issue Type: Bug Components: streaming, Web Affects Versions: v1.5.1 Reporter: Shaofeng SHI Assignee: Zhong,Jason Fix For: v1.5.2 Define a streaming table, in the "Table name", if enter a name without db schema, like "STREAMING_TBL", the save can get success; but when trigger the streaming cube build, an exception "Table DEFAULT.STREAMING_TBL is not a streaming table." will be thrown. In the metastore, I see the streaming conf and kafka conf are named with the origin table name ( "STREAMING_TBL" in this case), while the table desc, its name is " "DEFAULT.STREAMING_TBL" Expected behavior: automatically append db schema for the table before saving the stream config. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1666) Should check project selection before add a stream table
Shaofeng SHI created KYLIN-1666:
---
Summary: Should check project selection before add a stream table
Key: KYLIN-1666
URL: https://issues.apache.org/jira/browse/KYLIN-1666
Project: Kylin
Issue Type: Bug
Components: Web
Affects Versions: v1.5.1, v1.5.0
Reporter: Shaofeng SHI
Assignee: Zhong,Jason
In Kylin web, if not select a "project", the page will display "Please select
your project first", but I still be able to click "Add streaming table"; when
save the table info, a NPE was thrown:
{code}
java.lang.NullPointerException
at
java.util.concurrent.ConcurrentSkipListMap.comparable(ConcurrentSkipListMap.java:659)
at
java.util.concurrent.ConcurrentSkipListMap.doGet(ConcurrentSkipListMap.java:821)
at
java.util.concurrent.ConcurrentSkipListMap.get(ConcurrentSkipListMap.java:1626)
at
org.apache.kylin.common.restclient.SingleValueCache.get(SingleValueCache.java:85)
at
org.apache.kylin.metadata.project.ProjectManager.getProject(ProjectManager.java:138)
at
org.apache.kylin.metadata.project.ProjectManager.addTableDescToProject(ProjectManager.java:280)
at
org.apache.kylin.rest.service.CubeService.syncTableToProject(CubeService.java:580)
at
org.apache.kylin.rest.service.CubeService$$FastClassByCGLIB$$17a07c0e.invoke()
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
{code}
Suggestion: don't allow user to take create actions if no project is selected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1667) Streaming table name should allow enter "DB.TABLE" format
Shaofeng SHI created KYLIN-1667:
---
Summary: Streaming table name should allow enter "DB.TABLE" format
Key: KYLIN-1667
URL: https://issues.apache.org/jira/browse/KYLIN-1667
Project: Kylin
Issue Type: Bug
Components: streaming, Web
Affects Versions: v1.5.1, v1.5.0
Reporter: Shaofeng SHI
Assignee: Zhong,Jason
Add a streaming table, in the "Table Name" field, if I enter a name
"DEFAULT.TBL", an error will be thrown in backend:
{code}
java.lang.IllegalStateException: Cannot find table 'null' in metadata manager
at
org.apache.kylin.metadata.project.ProjectManager.addTableDescToProject(ProjectManager.java:284)
at
org.apache.kylin.rest.service.CubeService.syncTableToProject(CubeService.java:580)
at
org.apache.kylin.rest.service.CubeService$$FastClassByCGLIB$$17a07c0e.invoke()
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
{code}
Kylin UI should support custom DB name when creating a streaming table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1668) Rowkey column shouldn't allow delete and add
Shaofeng SHI created KYLIN-1668: --- Summary: Rowkey column shouldn't allow delete and add Key: KYLIN-1668 URL: https://issues.apache.org/jira/browse/KYLIN-1668 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v1.5.1, v1.3.0, v1.5.0 Reporter: Shaofeng SHI Assignee: Zhong,Jason Every column in rowkey are required, couldn't be deleted. And no need to add other columns. The "-" icon and "New Rowkey Column" button should be removed from the UI. Besides, please add some text to let user know the column sequence is adjustable with drag&drop. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1669) Sunset the "Capacity" field from DataModel
Shaofeng SHI created KYLIN-1669: --- Summary: Sunset the "Capacity" field from DataModel Key: KYLIN-1669 URL: https://issues.apache.org/jira/browse/KYLIN-1669 Project: Kylin Issue Type: Improvement Components: Metadata, Web Reporter: Shaofeng SHI Assignee: Shaofeng SHI Kylin used to use the "Capacity" on model as the hint to build the cube; While as only three options (SMALL, MEDIUM, LARGE), it couldn't give much flexibility, and may confuse user. As today kylin allows cube level to overwrite the config in kylin.properties, many parameters can be specified on cube, that would give user much flexibility; So the "Capacity" concept is redundant, can sunset it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1677) Distribute source data by certain columns when creating flat table
Shaofeng SHI created KYLIN-1677: --- Summary: Distribute source data by certain columns when creating flat table Key: KYLIN-1677 URL: https://issues.apache.org/jira/browse/KYLIN-1677 Project: Kylin Issue Type: Improvement Components: Job Engine Reporter: Shaofeng SHI Assignee: Shaofeng SHI Inspired by KYLIN-1656, Kylin can distribute the source data by certain columns when creating the flat hive table; Then the data assigned to a mapper will have more similarity, more aggregation can happen at mapper side, and then less shuffle and reduce is needed. Columns can be used for the distribution includes: ultra high cardinality column, mandantory column, partition date/time column, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1686) Get "Access is denied" error on clicking "Admin" tab
Shaofeng SHI created KYLIN-1686:
---
Summary: Get "Access is denied" error on clicking "Admin" tab
Key: KYLIN-1686
URL: https://issues.apache.org/jira/browse/KYLIN-1686
Project: Kylin
Issue Type: Bug
Components: Web
Reporter: Shaofeng SHI
Assignee: Zhong,Jason
Enable LDAP authentication, login with an analyst account, when clicking the
"Admin" tab, the page shows an exception:
{code}
org.springframework.security.access.AccessDeniedException: Access is denied
at
org.springframework.security.access.vote.AffirmativeBased.decide(AffirmativeBased.java:83)
at
org.springframework.security.access.intercept.AbstractSecurityInterceptor.beforeInvocation(AbstractSecurityInterceptor.java:206)
at
org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:60)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622)
at
org.apache.kylin.rest.service.DiagnosisService$$EnhancerByCGLIB$$6cb9c74a.getProjectBadQueryHistory()
at
io.kyligence.kap.rest.controller.BadQueryController.getBadQuerySql(BadQueryController.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1687) Error to select cuboid
Shaofeng SHI created KYLIN-1687:
---
Summary: Error to select cuboid
Key: KYLIN-1687
URL: https://issues.apache.org/jira/browse/KYLIN-1687
Project: Kylin
Issue Type: Bug
Components: Metadata
Reporter: Shaofeng SHI
Assignee: hongbin ma
Reported by user lancelot chen from mailing list:
{code}
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:794)
at java.util.Collections.min(Collections.java:665)
at
org.apache.kylin.cube.cuboid.Cuboid.translateToValidCuboid(Cuboid.java:201)
at
org.apache.kylin.cube.cuboid.Cuboid.translateToValidCuboid(Cuboid.java:125)
at org.apache.kylin.cube.cuboid.Cuboid.findById(Cuboid.java:67)
at
org.apache.kylin.storage.hbase.cube.v2.CubeStorageQuery.identifyCuboid(CubeStorageQuery.java:183)
at
org.apache.kylin.storage.hbase.cube.v2.CubeStorageQuery.search(CubeStorageQuery.java:96)
at
org.apache.kylin.query.enumerator.OLAPEnumerator.queryStorage(OLAPEnumerator.java:125)
at
org.apache.kylin.query.enumerator.OLAPEnumerator.moveNext(OLAPEnumerator.java:71)
at Baz$1$1.moveNext(Unknown Source)
at
org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:819)
at
org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:754)
at
org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:302)
at Baz.bind(Unknown Source)
at
org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:326)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:281)
at
org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:545)
at
org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:536)
at
org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:187)
at
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:65)
at
org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
at
org.apache.calcite.avatica.AvaticaConnection$1.execute(AvaticaConnection.java:566)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:578)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:571)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:135)
... 80 more
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1717) Make job engine scheduler configurable
Shaofeng SHI created KYLIN-1717: --- Summary: Make job engine scheduler configurable Key: KYLIN-1717 URL: https://issues.apache.org/jira/browse/KYLIN-1717 Project: Kylin Issue Type: Improvement Components: Job Engine Reporter: Shaofeng SHI Assignee: Shaofeng SHI Today the job engine scheduler is a simple implementation; need abstract it to decouple the implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1747) Calculate all 0 (exception mandatory) cuboids
Shaofeng SHI created KYLIN-1747: --- Summary: Calculate all 0 (exception mandatory) cuboids Key: KYLIN-1747 URL: https://issues.apache.org/jira/browse/KYLIN-1747 Project: Kylin Issue Type: Improvement Components: Job Engine, Query Engine Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v1.5.3 Kylin doesn't calculate all 0 combinations (exception mandatory dimensions) today; There wasn't specific consideration, but this may lead some query be slow. Investigate to remove that design, and make sure all functions (including query, merge cuboid statistics, etc) won't be broken. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1763) Release v1.5.2.1
Shaofeng SHI created KYLIN-1763: --- Summary: Release v1.5.2.1 Key: KYLIN-1763 URL: https://issues.apache.org/jira/browse/KYLIN-1763 Project: Kylin Issue Type: Task Reporter: Shaofeng SHI Assignee: Shaofeng SHI -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1767) Update UI for TopN: specify encoding and multiple "group by"
Shaofeng SHI created KYLIN-1767: --- Summary: Update UI for TopN: specify encoding and multiple "group by" Key: KYLIN-1767 URL: https://issues.apache.org/jira/browse/KYLIN-1767 Project: Kylin Issue Type: Task Components: Web Reporter: Shaofeng SHI Assignee: Zhong,Jason Fix For: v1.5.3 KYLIN-1478 and KYLIN-1693 need changes from the UI part. The sample json can be found in these two JIRAs; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1774) Update Kylin's tomcat version to 7.0.69
Shaofeng SHI created KYLIN-1774: --- Summary: Update Kylin's tomcat version to 7.0.69 Key: KYLIN-1774 URL: https://issues.apache.org/jira/browse/KYLIN-1774 Project: Kylin Issue Type: Task Components: REST Service Reporter: Shaofeng SHI Assignee: Shaofeng SHI Priority: Minor Fix For: v1.5.3 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1777) Streaming cube build shouldn't check working segment
Shaofeng SHI created KYLIN-1777:
---
Summary: Streaming cube build shouldn't check working segment
Key: KYLIN-1777
URL: https://issues.apache.org/jira/browse/KYLIN-1777
Project: Kylin
Issue Type: Bug
Components: streaming
Affects Versions: v1.5.3
Reporter: Shaofeng SHI
Fix For: v1.5.3
A bug newly introduced with the changes of KYLIN-1744, reported by felixcui01
in mailing list:
{code}
2016-06-07 17:55:51,024 ERROR [main StreamingCLI:106]: error start streaming
java.lang.IllegalStateException: There is already a building segment!
at
org.apache.kylin.cube.CubeManager.checkNoBuildingSegment(CubeManager.java:567)
at org.apache.kylin.cube.CubeManager.appendSegment(CubeManager.java:400)
at
org.apache.kylin.engine.streaming.cube.StreamingCubeBuilder.createBuildable(StreamingCubeBuilder.java:118)
at
org.apache.kylin.engine.streaming.OneOffStreamingBuilder$1.run(OneOffStreamingBuilder.java:76)
at
org.apache.kylin.engine.streaming.cli.StreamingCLI.startOneOffCubeStreaming(StreamingCLI.java:126)
at
org.apache.kylin.engine.streaming.cli.StreamingCLI.main(StreamingCLI.java:100)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1780) Potential issue in SnapshotTable.equas()
Shaofeng SHI created KYLIN-1780: --- Summary: Potential issue in SnapshotTable.equas() Key: KYLIN-1780 URL: https://issues.apache.org/jira/browse/KYLIN-1780 Project: Kylin Issue Type: Bug Reporter: Shaofeng SHI Assignee: Shaofeng SHI SnapshotTable.equas() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1789) Couldn't use View as Lookup when join type is "inner"
Shaofeng SHI created KYLIN-1789: --- Summary: Couldn't use View as Lookup when join type is "inner" Key: KYLIN-1789 URL: https://issues.apache.org/jira/browse/KYLIN-1789 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: v1.5.2, v1.5.2.1 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v1.5.3 Reported by Bhanu Mohanty in user mailing list: I am using kylin-1.5.2.1 Added hive view as a look up table Getting error at Build Dimension Dictionary DEFAULT.kylin_intermediate_DEFAULT_* If the join is "inner" It works when I changed the join to "left" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1793) Job couldn't stop when hive commands got error with beeline
Shaofeng SHI created KYLIN-1793: --- Summary: Job couldn't stop when hive commands got error with beeline Key: KYLIN-1793 URL: https://issues.apache.org/jira/browse/KYLIN-1793 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: v1.5.2, v1.5.1, v1.5.0 Reporter: Shaofeng SHI Assignee: Dong Li Configure Kylin to use beeline as the hive command line; submit a cube build job, the job moves to 100% with success, while I found there was error in the hive related steps, but the error wasn't captured by Kylin; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1795) bin/sample.sh doesn't work when configured hive client is beeline
Shaofeng SHI created KYLIN-1795: --- Summary: bin/sample.sh doesn't work when configured hive client is beeline Key: KYLIN-1795 URL: https://issues.apache.org/jira/browse/KYLIN-1795 Project: Kylin Issue Type: Bug Components: Client - CLI Affects Versions: v1.5.0 Reporter: Shaofeng SHI Priority: Minor If configure to use beeline as hive client in kylin.properties, then run bin/sample.sh, it will fail to load data into hive, like: 0: jdbc:hive2://hiveserver:10> LOAD DATA LOCAL INPATH 'DEFAULT.KYLIN_SALES.csv' OVERWRITE INTO TABLE DEFAULT.KYLIN_SALES; Error: Error while compiling statement: FAILED: SemanticException Line 1:23 Invalid path ''DEFAULT.KYLIN_SALES.csv'': No files matching path file:/var/run/cloudera-scm-agent/process/1358-hive-HIVESERVER2/DEFAULT.KYLIN_SALES.csv (state=42000,code=4) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1797) bin/find-hive-dependency.sh may not get correct local classpath when configured hive client is beeline
Shaofeng SHI created KYLIN-1797: --- Summary: bin/find-hive-dependency.sh may not get correct local classpath when configured hive client is beeline Key: KYLIN-1797 URL: https://issues.apache.org/jira/browse/KYLIN-1797 Project: Kylin Issue Type: Bug Components: Client - CLI Affects Versions: v1.3.0, v1.5.0 Reporter: Shaofeng SHI -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1818) change kafka dependency to provided
Shaofeng SHI created KYLIN-1818: --- Summary: change kafka dependency to provided Key: KYLIN-1818 URL: https://issues.apache.org/jira/browse/KYLIN-1818 Project: Kylin Issue Type: Improvement Components: streaming Affects Versions: v1.5.2 Reporter: Shaofeng SHI Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1831) Couldn't run "count (distinct pk)" on a lookup table
Shaofeng SHI created KYLIN-1831:
---
Summary: Couldn't run "count (distinct pk)" on a lookup table
Key: KYLIN-1831
URL: https://issues.apache.org/jira/browse/KYLIN-1831
Project: Kylin
Issue Type: Bug
Components: Query Engine
Affects Versions: v1.5.2
Reporter: Shaofeng SHI
Assignee: Dong Li
I have a SSB testing cube, in which ssb.customer is a lookup table, the cube
has been built successfully (table snapshot exists); As KYLIN-1016 mentioned,
count distinct should work even if there is no pre-defined measure; The SQL is:
"select count (distinct c_custkey) from ssb.customer", the error log is:
{code}
Caused by: java.lang.IllegalStateException: Can't create EnumerableAggregate!
at
org.apache.kylin.query.relnode.OLAPAggregateRel.implementEnumerable(OLAPAggregateRel.java:373)
at
org.apache.kylin.query.relnode.OLAPRel$JavaImplementor.createEnumerable(OLAPRel.java:158)
at
org.apache.kylin.query.relnode.OLAPRel$JavaImplementor.createEnumerable(OLAPRel.java:154)
at
org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:99)
at
org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:102)
at
org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)
at
org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1171)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:297)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:196)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:721)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:588)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:558)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:573)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:571)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:135)
... 80 more
Caused by: org.apache.calcite.rel.InvalidRelException: distinct aggregation not
supported
at
org.apache.calcite.adapter.enumerable.EnumerableAggregate.(EnumerableAggregate.java:71)
at
org.apache.kylin.query.relnode.OLAPAggregateRel.implementEnumerable(OLAPAggregateRel.java:370)
... 95 more
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1838) Fix sample cube definition
Shaofeng SHI created KYLIN-1838: --- Summary: Fix sample cube definition Key: KYLIN-1838 URL: https://issues.apache.org/jira/browse/KYLIN-1838 Project: Kylin Issue Type: Bug Components: Documentation Reporter: Shaofeng SHI Priority: Minor Fix For: v1.5.3 Problem reported by Alberto Ramón in user mailing list: In example "learn_kylin", table name "DEFAULT.KYLIN_CATEGORY_GROUPINGS" appears 2 times: -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1859) Use segment "uuid" instead of "name" to seek a segment across the system
Shaofeng SHI created KYLIN-1859: --- Summary: Use segment "uuid" instead of "name" to seek a segment across the system Key: KYLIN-1859 URL: https://issues.apache.org/jira/browse/KYLIN-1859 Project: Kylin Issue Type: Bug Reporter: Shaofeng SHI Assignee: Shaofeng SHI Priority: Minor The "uuid" is unique, while "name" isn't (name is composed by startTime, endTime). It's better to use "uuid" in the job chains. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1860) "Models" is very slow to show up
Shaofeng SHI created KYLIN-1860: --- Summary: "Models" is very slow to show up Key: KYLIN-1860 URL: https://issues.apache.org/jira/browse/KYLIN-1860 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v1.5.2, v1.5.3 Reporter: Shaofeng SHI Assignee: Zhong,Jason Fix For: v1.5.3 There is an "loading" icon, after about 1 minute it shows up, this is too slow; there must be something wrong. Pls see the screenshot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1861) Hive may fail to create flat table with "GC overhead error"
Shaofeng SHI created KYLIN-1861: --- Summary: Hive may fail to create flat table with "GC overhead error" Key: KYLIN-1861 URL: https://issues.apache.org/jira/browse/KYLIN-1861 Project: Kylin Issue Type: Test Reporter: Shaofeng SHI in conf/kylin_hive_conf.xml, it enables "hive.auto.convert.join.noconditionaltask", which is the optimization about converting common join into mapjoin based on the input file size, and give a big value (3, 300MB) to "hive.auto.convert.join.noconditionaltask.size". So if the memory of the mapper isn't enough, it will cause the error. To fix the error, user can reduce the "hive.auto.convert.join.noconditionaltask.size" in conf/kylin_hive_conf.xml, and then retry the job. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1862) "table not found" in "Build Dimension Dictionary" step
Shaofeng SHI created KYLIN-1862: --- Summary: "table not found" in "Build Dimension Dictionary" step Key: KYLIN-1862 URL: https://issues.apache.org/jira/browse/KYLIN-1862 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: v1.5.3 Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v1.5.3 In a cube, I use a hive view as lookup table; The view name is "ssb.v_customer"; In the "Materialize Hive View in Lookup Tables" step, the log said: Table default.kylin_intermediate_ssb_v_customer stats: [numFiles=2, numRows=180, totalSize=82661469, rawDataSize=169673761] In the "Build Dimension Dictionary" step, it failed with message: "DEFAULT.kylin_intermediate_DEFAULT_V_CUSTOMER table not found" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1868) Page is blank with JS error
Shaofeng SHI created KYLIN-1868:
---
Summary: Page is blank with JS error
Key: KYLIN-1868
URL: https://issues.apache.org/jira/browse/KYLIN-1868
Project: Kylin
Issue Type: Bug
Components: Web
Affects Versions: v1.5.3
Reporter: Shaofeng SHI
Assignee: Zhong,Jason
Fix For: v1.5.3
Attachments: Screen Shot 2016-07-11 at 11.08.21 AM.png
Try to create a new cube, in the "dimension" tab, the page is blank (see
attachement) and there is JS error in console:
{code}
scripts.min.0.js:10159 TypeError: Cannot read property 'fact_table' of undefined
at Scope.$scope.initColumns
(http://host:7070/kylin/js/scripts.min.0.js:42705:47)
at new (http://host:7070/kylin/js/scripts.min.0.js:42770:16)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1871) Couldn't generate sample data for stream cube
Shaofeng SHI created KYLIN-1871: --- Summary: Couldn't generate sample data for stream cube Key: KYLIN-1871 URL: https://issues.apache.org/jira/browse/KYLIN-1871 Project: Kylin Issue Type: Bug Components: General Affects Versions: v1.5.3 Reporter: Shaofeng SHI Priority: Minor Fix For: v1.5.3 Trying to build sample streaming cube with the doc https://kylin.apache.org/docs15/tutorial/cube_streaming.html , but got error when generating the sample data: ./bin/kylin.sh org.apache.kylin.source.kafka.util.KafkaSampleProducer --topic kylin_demo --broker sandbox:6667 —delay 0 Exception in thread "main" java.lang.NoClassDefFoundError: kafka/producer/ProducerConfig at org.apache.kylin.source.kafka.util.KafkaSampleProducer.main(KafkaSampleProducer.java:102) Caused by: java.lang.ClassNotFoundException: kafka.producer.ProducerConfig at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 1 more it should be related with KYLIN-1818 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1902) 404 error on get /kylin/models
Shaofeng SHI created KYLIN-1902: --- Summary: 404 error on get /kylin/models Key: KYLIN-1902 URL: https://issues.apache.org/jira/browse/KYLIN-1902 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v1.5.2 Reporter: Shaofeng SHI Assignee: Zhong,Jason Fix For: v1.5.3 Request URL:http://localhost:7070/kylin/models Request Method:GET Status Code:404 Not Found Remote Address:[::1]:7070 I think this issue has existed for some time; Although it doesn't impact on user experience, an 404 error in web console is confusing. @Jason Can we fix it in v1.5.3? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1911) NPE when extended column has NULL value
Shaofeng SHI created KYLIN-1911:
---
Summary: NPE when extended column has NULL value
Key: KYLIN-1911
URL: https://issues.apache.org/jira/browse/KYLIN-1911
Project: Kylin
Issue Type: Bug
Components: Query Engine
Affects Versions: v1.5.2, v1.5.3, v1.5.2.1
Reporter: Shaofeng SHI
Assignee: hongbin ma
{code}
Caused by: java.lang.NullPointerException
at java.lang.String.(String.java:505)
at
org.apache.kylin.measure.extendedcolumn.ExtendedColumnMeasureType$2.reload(ExtendedColumnMeasureType.java:152)
at
org.apache.kylin.storage.hbase.cube.v2.CubeTupleConverter.translateResult(CubeTupleConverter.java:175)
at
org.apache.kylin.storage.hbase.cube.v2.SequentialCubeTupleIterator.hasNext(SequentialCubeTupleIterator.java:116)
at
org.apache.kylin.query.enumerator.OLAPEnumerator.moveNext(OLAPEnumerator.java:74)
at Baz$1$1.moveNext(Unknown Source)
at
org.apache.calcite.linq4j.EnumerableDefaults.groupBy_(EnumerableDefaults.java:819)
at
org.apache.calcite.linq4j.EnumerableDefaults.groupBy(EnumerableDefaults.java:754)
at
org.apache.calcite.linq4j.DefaultEnumerable.groupBy(DefaultEnumerable.java:302)
at Baz.bind(Unknown Source)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (KYLIN-1917) TopN counter merge performance improvement
Shaofeng SHI created KYLIN-1917: --- Summary: TopN counter merge performance improvement Key: KYLIN-1917 URL: https://issues.apache.org/jira/browse/KYLIN-1917 Project: Kylin Issue Type: Improvement Components: Job Engine Affects Versions: v1.5.2, v1.5.3 Reporter: Shaofeng SHI Fix For: Backlog Observed the reduce phase of cube build is slow when there is TopN counter. There should be room for performance improvement. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1926) Loosen the constraint on FK-PK data type matching
Shaofeng SHI created KYLIN-1926: --- Summary: Loosen the constraint on FK-PK data type matching Key: KYLIN-1926 URL: https://issues.apache.org/jira/browse/KYLIN-1926 Project: Kylin Issue Type: Improvement Components: Metadata Affects Versions: all Reporter: Shaofeng SHI Assignee: Shaofeng SHI Priority: Minor Fix For: v1.5.4 If lookup table's PK datatype isn't equal to fact table's FK datatype, Kylin will report error saying "Primary key are not consistent with Foreign key". This constraint is too strong. Should allow user to disable this check. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1929) Error to load slow query in "Monitor" page for non-admin user
Shaofeng SHI created KYLIN-1929: --- Summary: Error to load slow query in "Monitor" page for non-admin user Key: KYLIN-1929 URL: https://issues.apache.org/jira/browse/KYLIN-1929 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v1.5.3 Reporter: Shaofeng SHI Assignee: Dong Li Fix For: v1.5.4 Attachments: Screen Shot 2016-07-29 at 4.47.11 PM.png Login Kylin with a non-admin account, click to "Monitor" tab, an error dialog shows "Failed to load query" (see screenshot) In browser console, a 403 error: GET http://host:7070/kylin/api/diag/project/sql 403 (Forbidden) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1933) Deploy in cluster mode, the "query" node report "scheduler has not been started" every second
Shaofeng SHI created KYLIN-1933: --- Summary: Deploy in cluster mode, the "query" node report "scheduler has not been started" every second Key: KYLIN-1933 URL: https://issues.apache.org/jira/browse/KYLIN-1933 Project: Kylin Issue Type: Bug Components: Job Engine Affects Versions: v1.5.3 Reporter: Shaofeng SHI Assignee: Shaofeng SHI -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1978) kylin.sh compatible issue on Ubuntu
Shaofeng SHI created KYLIN-1978:
---
Summary: kylin.sh compatible issue on Ubuntu
Key: KYLIN-1978
URL: https://issues.apache.org/jira/browse/KYLIN-1978
Project: Kylin
Issue Type: Bug
Components: Environment
Affects Versions: v1.5.3
Reporter: Shaofeng SHI
Assignee: hongbin ma
Reported by Marcelo(marcelo.n...@quantium.com.au) in the mailing list:
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.4 LTS"
NAME="Ubuntu"
VERSION="14.04.4 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.4 LTS"
VERSION_ID="14.04"
HOME_URL="http://www.ubuntu.com/";
SUPPORT_URL="http://help.ubuntu.com/";
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/";
mapr@qtausc-vpcsdev04:~/kylin/apache-kylin-1.5.3-HBase1.x-bin/bin$ ./kylin.sh
start
*KYLIN_HOME is set to /home/mapr/kylin/apache-kylin-1.5.3-HBase1.x-bin*
cat: invalid option -- '1'
Try 'cat --help' for more information.
-mkdir: Not enough arguments: expected 1 but got 0
Usage: hadoop fs [generic options] -mkdir [-p] ...
failed to create , Please make sure the user has right to access
That is what is happening when I try to start kylin.
I traced the error and the first one come from get-properties.sh at these
line
for i in `cat ${KYLIN_HOME}/conf/kylin.properties | grep -w "^$1" | grep -v
'^#' | awk -F= '{ n = index($0,"="); print substr($0,n+1)}' | cut -c 1-`
and as you can see kylin home is set.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
