[jira] [Updated] (KYLIN-2491) Cube with error job can be dropped
[ https://issues.apache.org/jira/browse/KYLIN-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma updated KYLIN-2491: -- Affects Version/s: (was: v1.5.4.1) v1.6.0 > Cube with error job can be dropped > -- > > Key: KYLIN-2491 > URL: https://issues.apache.org/jira/browse/KYLIN-2491 > Project: Kylin > Issue Type: Bug > Components: REST Service >Affects Versions: v1.6.0 >Reporter: nichunen >Assignee: nichunen > Fix For: v2.0.0 > > Attachments: KYLIN-2491.patch > > > If a cube build failed, it can be dropped and left a error job, the job can > be resumed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (KYLIN-2491) Cube with error job can be dropped
[ https://issues.apache.org/jira/browse/KYLIN-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hongbin ma resolved KYLIN-2491. --- Resolution: Fixed > Cube with error job can be dropped > -- > > Key: KYLIN-2491 > URL: https://issues.apache.org/jira/browse/KYLIN-2491 > Project: Kylin > Issue Type: Bug > Components: REST Service >Affects Versions: v1.6.0 >Reporter: nichunen >Assignee: nichunen > Fix For: v2.0.0 > > Attachments: KYLIN-2491.patch > > > If a cube build failed, it can be dropped and left a error job, the job can > be resumed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KYLIN-2491) Cube with error job can be dropped
[ https://issues.apache.org/jira/browse/KYLIN-2491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nichunen updated KYLIN-2491: Attachment: KYLIN-2491.patch Attached is the patch to fix. > Cube with error job can be dropped > -- > > Key: KYLIN-2491 > URL: https://issues.apache.org/jira/browse/KYLIN-2491 > Project: Kylin > Issue Type: Bug > Components: REST Service >Affects Versions: v1.5.4.1 >Reporter: nichunen >Assignee: nichunen > Fix For: v2.0.0 > > Attachments: KYLIN-2491.patch > > > If a cube build failed, it can be dropped and left a error job, the job can > be resumed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (KYLIN-2491) Cube with error job can be dropped
nichunen created KYLIN-2491: --- Summary: Cube with error job can be dropped Key: KYLIN-2491 URL: https://issues.apache.org/jira/browse/KYLIN-2491 Project: Kylin Issue Type: Bug Components: REST Service Affects Versions: v1.5.4.1 Reporter: nichunen Assignee: nichunen Fix For: v2.0.0 If a cube build failed, it can be dropped and left a error job, the job can be resumed. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (KYLIN-2490) Couldn't get cube size on Azure HDInsight
Shaofeng SHI created KYLIN-2490: --- Summary: Couldn't get cube size on Azure HDInsight Key: KYLIN-2490 URL: https://issues.apache.org/jira/browse/KYLIN-2490 Project: Kylin Issue Type: Bug Components: Job Engine Reporter: Shaofeng SHI Assignee: Shaofeng SHI Fix For: v2.0.0 Kylin uses "HDFS_BYTES_WRITTEN" counter to get the cube size from Hadoop; While some Cloud-based hadoop platform are using blob storage as the file system, where the counter isn't available, then Kylin fails to get the counter, always show 0 as the cube size. On Azure, the counter name is "WASB_BYTES_WRITTEN" -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (KYLIN-186) System Health Indictor for end user
[ https://issues.apache.org/jira/browse/KYLIN-186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi closed KYLIN-186. --- Resolution: Duplicate > System Health Indictor for end user > --- > > Key: KYLIN-186 > URL: https://issues.apache.org/jira/browse/KYLIN-186 > Project: Kylin > Issue Type: Wish > Components: Web >Reporter: Luke Han >Assignee: Kejia Wang >Priority: Trivial > Labels: github-import > Fix For: Backlog > > > As an end user, I would like to know is the entire system available or with > any issue, and what's kind of the root cause: Hadoop Cluster? Map Reduce Job > and so on. > Also I would like to have a more detail view of each status. > Imported from GitHub > Url: https://github.com/KylinOLAP/Kylin/issues/319 > Created by: [lukehan|https://github.com/lukehan] > Labels: newfeature, > Milestone: Backlog > Created at: Fri Dec 26 15:22:18 CST 2014 > State: open -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KYLIN-188) REST server expose metrics for job engine & query engine
[ https://issues.apache.org/jira/browse/KYLIN-188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Shi updated KYLIN-188: Labels: bigtask github-import (was: github-import) > REST server expose metrics for job engine & query engine > > > Key: KYLIN-188 > URL: https://issues.apache.org/jira/browse/KYLIN-188 > Project: Kylin > Issue Type: Wish > Components: REST Service >Reporter: Luke Han >Priority: Minor > Labels: bigtask, github-import > Fix For: Backlog > > > 1. Rest server need to expose the metrics for job engine & query engine > 2. Job Engine expose metrics for job: job queue length, job latency, job > count ... > 3. Query engine expose metrics for query: query latency, query scan count, > qps ... > Imported from GitHub > Url: https://github.com/KylinOLAP/Kylin/issues/317 > Created by: [lukehan|https://github.com/lukehan] > Labels: newfeature, > Milestone: Backlog > Created at: Fri Dec 26 15:16:53 CST 2014 > State: open -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (KYLIN-2489) Upgrade zookeeper dependency to 3.4.8
Ted Yu created KYLIN-2489: - Summary: Upgrade zookeeper dependency to 3.4.8 Key: KYLIN-2489 URL: https://issues.apache.org/jira/browse/KYLIN-2489 Project: Kylin Issue Type: Improvement Reporter: Ted Yu zookeeper 3.4.8 has been released. Among the fixes the following are desirable: ZOOKEEPER-706 large numbers of watches can cause session re-establishment to fail ZOOKEEPER-1797 PurgeTxnLog may delete data logs during roll This issue upgrades zookeeper dependency to 3.4.8 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (KYLIN-2387) A new BitmapCounter with better performance
[ https://issues.apache.org/jira/browse/KYLIN-2387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dayue Gao resolved KYLIN-2387. -- Resolution: Fixed Fix Version/s: v2.0.0 > A new BitmapCounter with better performance > --- > > Key: KYLIN-2387 > URL: https://issues.apache.org/jira/browse/KYLIN-2387 > Project: Kylin > Issue Type: Improvement > Components: Metadata, Query Engine, Storage - HBase >Affects Versions: v2.0.0 >Reporter: Dayue Gao >Assignee: Dayue Gao > Fix For: v2.0.0 > > > We found the old BitmapCounter does not perform very well on very large > bitmap. The inefficiency comes from > * Poor serialize implementation: instead of serialize bitmap directly to > ByteBuffer, it uses ByteArrayOutputStream as a temporal storage, which causes > superfluous memory allocations > * Poor peekLength implementation: the whole bitmap is deserialized in order > to retrieve its serialized size > * Extra deserialize cost: even if only cardinality info is needed to answer > query, the whole bitmap is deserialize into MutableRoaringBitmap > A new BitmapCounter is designed to solve these problems > * It comes in tow flavors, mutable and immutable, which is based on > Mutable/Immutable RoaringBitmap correspondingly > * ImmutableBitmapCounter has lower deserialize cost, as it just maps to a > copied buffer. So we always deserialize to ImmutableBitmapCounter at first, > and convert it to MutableBitmapCounter only when necessary > * peekLength is implemented using > ImmutableRoaringBitmap.serializedSizeInBytes, which is very fast since only > the header of roaring format is examined > * It can directly serializes to ByteBuffer, no intermediate buffer is > allocated > * The wire format is the same as before > ([RoaringFormatSpec|https://github.com/RoaringBitmap/RoaringFormatSpec/]). > Therefore no cube rebuild is needed -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (KYLIN-2488) query failed because scan threshold
zhou degao created KYLIN-2488: - Summary: query failed because scan threshold Key: KYLIN-2488 URL: https://issues.apache.org/jira/browse/KYLIN-2488 Project: Kylin Issue Type: Bug Components: Query Engine Affects Versions: v1.6.0 Reporter: zhou degao Assignee: liyang Attachments: cube.PNG When i submit this query: select "time_by_day_alias"."THE_YEAR" as "c0", "git_log_alias"."PROJECT_NAME" as "c1", "git_log_alias"."BRANCH_NAME" as "c2", count(distinct "git_log_alias"."COMMIT_ID") as "m0" from "CODE_VIEW_178E383E_623C_45E3_BA1B_1333DFE2240B" as "git_log_alias" join "TIME_BY_DAY" as "time_by_day_alias" on "git_log_alias"."FORMATED_DATE" = "time_by_day_alias"."THE_DATE" group by "time_by_day_alias"."THE_YEAR", "git_log_alias"."PROJECT_NAME", "git_log_alias"."BRANCH_NAME" I got following error even though i set kylin.query.scan.threshold=1 2017-03-07 14:27:43,770 INFO [Query c422b09c-b1a5-4dcc-afac-7c9a25cff3fe-66] endpoint.CubeVisitService: The cube visit did not finish normally because scan num exceeds threshold org.apache.kylin.gridtable.GTScanExceedThresholdException: Exceed scan threshold at 49145 at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService$1.hasNext(CubeVisitService.java:258) at org.apache.kylin.storage.hbase.cube.v2.HBaseReadonlyStore$1$1.hasNext(HBaseReadonlyStore.java:111) at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.CubeVisitService.visitCube(CubeVisitService.java:290) at org.apache.kylin.storage.hbase.cube.v2.coprocessor.endpoint.generated.CubeVisitProtos$CubeVisitService.callMethod(CubeVisitProtos.java:4117) at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7864) at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1991) at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1973) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2180) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112) at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133) at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108) at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
