[jira] [Updated] (KYLIN-1848) Can't sort cubes by any field in Web UI
[ https://issues.apache.org/jira/browse/KYLIN-1848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated KYLIN-1848: - Attachment: 0001-KYLIN-1848-web-fix-sort-cubes-issue.patch I have fixed this and created a patch, please help review, thanks! > Can't sort cubes by any field in Web UI > --- > > Key: KYLIN-1848 > URL: https://issues.apache.org/jira/browse/KYLIN-1848 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v1.5.2 >Reporter: Dayue Gao >Assignee: Zhong,Jason > Attachments: 0001-KYLIN-1848-web-fix-sort-cubes-issue.patch > > > In a project containing dozens of cubes, it's helpful to sort cubes by fields > like "Create Time", "Status", and so on. I tried it today but found it's not > working, could we fix it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-1903) add authentication for CubeMigrationCLI
Zhong,Jason created KYLIN-1903: -- Summary: add authentication for CubeMigrationCLI Key: KYLIN-1903 URL: https://issues.apache.org/jira/browse/KYLIN-1903 Project: Kylin Issue Type: Improvement Components: Client - CLI Affects Versions: v1.5.2 Reporter: Zhong,Jason Assignee: hongbin ma CubeMigrationCLI will consume Rest API '/api/admin/config', but it's not authenticated, need to add security check for the too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1883) Consensus Problem when running the tool, MetadataCleanupJob
[ https://issues.apache.org/jira/browse/KYLIN-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383469#comment-15383469 ] Zhong Yanghong commented on KYLIN-1883: --- No, it's still not working for our environment. Since the TIME_THREADSHOLD is for the time the DICT_A created, this case always happen for streaming with a retention days setting. The segment and the related DICT are created before more than 10 days. By changing the workflow, we can reduce the chance of this kind of consensus problem happening. > Consensus Problem when running the tool, MetadataCleanupJob > --- > > Key: KYLIN-1883 > URL: https://issues.apache.org/jira/browse/KYLIN-1883 > Project: Kylin > Issue Type: Bug >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: > better_solution_for_consensus_issue_of_MetadataCleanupJob.patch > > > When do the cleanup, current strategy is as follows: > 1. firstly create an referenceSet > 2. then add items not belonging to the referenceSet to the toDeleteSet > 3. finally delete those items in the toDeleteSet > Consensus issue will occur since we cannot make sure that all of the items in > toDeleteSet are not referenced in case that referenceSet changes during the > process. > For example, before the cleanup, SEGMENT_A is deleted and leave a DICT_A > created at the building step. Then the referenceSet will not include DICT_A. > After creating the reference set, SEGMENT_B is starting to build. Since > DICT_A still exists, it can still be referenced by SEGMENT_B. Then DICT_A > will still be included in the toDeleteSet and will be deleted later. Finally > SEGMENT_B only owns a reference with no data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1855) Should exclude those joins in whose related lookup tables no dimensions are used in cube
[ https://issues.apache.org/jira/browse/KYLIN-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383457#comment-15383457 ] Zhong Yanghong commented on KYLIN-1855: --- Hi mahong, thanks for your clear explanation. That's exactly what I want to mean. > Should exclude those joins in whose related lookup tables no dimensions are > used in cube > > > Key: KYLIN-1855 > URL: https://issues.apache.org/jira/browse/KYLIN-1855 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: exclude_unused_joins.patch > > > A cube is based on a model in which a star schema is defined. In some cases, > the cube utilizes only a few lookup tables rather than all. In this case, > when creating the sql for the flat table, those lookup tables should not be > included. Otherwise, it will confuse users when query. If users do query > according to the definition of the flat table, error of no realization will > occur due to lack of the related join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1854) Allow deleting cube instance when its underlying cubedesc went wrong
[ https://issues.apache.org/jira/browse/KYLIN-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15383433#comment-15383433 ] kangkaisen commented on KYLIN-1854: --- OK，thanks. > Allow deleting cube instance when its underlying cubedesc went wrong > > > Key: KYLIN-1854 > URL: https://issues.apache.org/jira/browse/KYLIN-1854 > Project: Kylin > Issue Type: Improvement >Reporter: hongbin ma >Assignee: hongbin ma > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1894) GlobalDictionary may corrupt when server suddenly crash
[ https://issues.apache.org/jira/browse/KYLIN-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382480#comment-15382480 ] Yerui Sun commented on KYLIN-1894: -- hongbin, Global Dict is used for the full-type precise distinct count, you're right. I've written a blog about how to use it, and will post it after v1.5.3 is released officially. > GlobalDictionary may corrupt when server suddenly crash > --- > > Key: KYLIN-1894 > URL: https://issues.apache.org/jira/browse/KYLIN-1894 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: v1.5.3 >Reporter: Yerui Sun >Assignee: Yerui Sun > Fix For: v1.5.3 > > Attachments: KYLIN-1894.2.patch, KYLIN-1894.3.patch > > > Global Dictionary store data on hdfs directly, and overwrite directly when > data file updated. If the server crashed suddenly during writing file, the > data file may be corrupt and can't be recovered. > To resolve this problem, copy the data file into a tmp directory and copy > back after the file is updated successfully. > I'll post a patch later with this solution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1890) support hbase table prefix configurable
[ https://issues.apache.org/jira/browse/KYLIN-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382414#comment-15382414 ] Shaofeng SHI commented on KYLIN-1890: - Considering that Cubes can be migrated from one Kylin env to another, using different table prefixs may not solve the problem. Of course, if the table name can be changed automatically during the migration, that is acceptable. > support hbase table prefix configurable > --- > > Key: KYLIN-1890 > URL: https://issues.apache.org/jira/browse/KYLIN-1890 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.5.2 >Reporter: fengYu >Assignee: fengYu > Attachments: > 0001-KYLIN-1890-support-hbase-table-prefix-configurable.patch > > > some times we need deploy two kylin env based on same hbase, I want to change > hbase table name prefix based two reasons: > 1、different kylin env will generate the same table name > 2、while clean invalid htable for one env will cause delete all tables belong > to another env. > different kylin env use different namespace is acceptable either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1892) merge interval support
[ https://issues.apache.org/jira/browse/KYLIN-1892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382377#comment-15382377 ] hongbin ma commented on KYLIN-1892: --- This is exactly I want, I called it "volatile periods". Auto merging will never attempt to merge segments in "volatile periods". are you interested in committing a patch? > merge interval support > -- > > Key: KYLIN-1892 > URL: https://issues.apache.org/jira/browse/KYLIN-1892 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Affects Versions: v1.5.2 >Reporter: fengYu >Assignee: fengYu > > We always has some data need to be amended some days later > in current kylin, once I set Auto Merge Thresholds, the segment newly build > will merge if reach Thresholds, the next day refresh will refresh merged > segemnt, which is unnecessary. > So I want to add a interval configuration means auto merge will merge > segments outside of the interval. > for example, if interval = 2, Auto Merge Thresholds=7, if 07-01 to 07-07 is > built, auto merge will not trigger, when 07-09 built success, auto merge will > trigger and merge segments from 07-01 to 07-07. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1883) Consensus Problem when running the tool, MetadataCleanupJob
[ https://issues.apache.org/jira/browse/KYLIN-1883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382374#comment-15382374 ] hongbin ma commented on KYLIN-1883: --- Hi yanghong It seems to me that even if we change the workflow order, it still cannot completely avoid the describe issue. To enforce optimal consistency we'll need more complex locking, which is obviously too much. currently on master branch, MetadataCleanupJob will not remove entries younger than TIME_THREADSHOLD (2 days), is it not working for your environment? > Consensus Problem when running the tool, MetadataCleanupJob > --- > > Key: KYLIN-1883 > URL: https://issues.apache.org/jira/browse/KYLIN-1883 > Project: Kylin > Issue Type: Bug >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: > better_solution_for_consensus_issue_of_MetadataCleanupJob.patch > > > When do the cleanup, current strategy is as follows: > 1. firstly create an referenceSet > 2. then add items not belonging to the referenceSet to the toDeleteSet > 3. finally delete those items in the toDeleteSet > Consensus issue will occur since we cannot make sure that all of the items in > toDeleteSet are not referenced in case that referenceSet changes during the > process. > For example, before the cleanup, SEGMENT_A is deleted and leave a DICT_A > created at the building step. Then the referenceSet will not include DICT_A. > After creating the reference set, SEGMENT_B is starting to build. Since > DICT_A still exists, it can still be referenced by SEGMENT_B. Then DICT_A > will still be included in the toDeleteSet and will be deleted later. Finally > SEGMENT_B only owns a reference with no data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1889) Didn't deal with the failure of renaming folder in hdfs when running the tool CubeMigrationCLI
[ https://issues.apache.org/jira/browse/KYLIN-1889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382359#comment-15382359 ] hongbin ma commented on KYLIN-1889: --- the patch looks good. which branches are your submitting to ? > Didn't deal with the failure of renaming folder in hdfs when running the tool > CubeMigrationCLI > -- > > Key: KYLIN-1889 > URL: https://issues.apache.org/jira/browse/KYLIN-1889 > Project: Kylin > Issue Type: Bug > Components: Tools, Build and Test >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: > retry_when_fail_to_rename_hdfs_folder_and_throw_exception_if_necessary.patch > > > In some cases, the invoking of the "rename" method of FileSystem may fail and > return false without throwing any Exception. Currently, such case is regarded > as renaming successfully, which is not true. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1855) Should exclude those joins in whose related lookup tables no dimensions are used in cube
[ https://issues.apache.org/jira/browse/KYLIN-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382355#comment-15382355 ] hongbin ma commented on KYLIN-1855: --- Hi yanghong correct me if I'm wrong. Suppose we have a fact table F and two lookup table X,Y suppose F has 1 rows, "F inner join X" produces 9000, and "F inner join X inner join Y" produces 8000 Now we have a model M, the join relationship is "F inner join X inner join Y". Next we have a cube C, it contains only F and X. When we build the cube C, currently the hive sql will contain ""F inner join X inner join Y", which is trying to keep all cubes in M having the same "star schema". so the "real fact table" for cube C will contain 8000 rows. Now come the issue, when users queries "select count(*) from F inner join X", it will return 8000, which is wrong from a DBMS perspective. A smarter user will query "select count(*) from F inner join X inner join Y", however kylin will throw "no realization found" because currently the CubeCapabilityChecker sees that cube C does not include table Y. that's the issue. If we want to enforce all cubes under the same model to have uniformed star schema, i.e, each cube fully respect all the inner joins, we should relax the CubeCapabilityChecker. Otherwise, if we allow some cubes seeing a "real fact table" of 8000, and some seeing of 9000, then we need to merge yanghong's patch. To me, the first makes more sense. > Should exclude those joins in whose related lookup tables no dimensions are > used in cube > > > Key: KYLIN-1855 > URL: https://issues.apache.org/jira/browse/KYLIN-1855 > Project: Kylin > Issue Type: Improvement >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > Attachments: exclude_unused_joins.patch > > > A cube is based on a model in which a star schema is defined. In some cases, > the cube utilizes only a few lookup tables rather than all. In this case, > when creating the sql for the flat table, those lookup tables should not be > included. Otherwise, it will confuse users when query. If users do query > according to the definition of the flat table, error of no realization will > occur due to lack of the related join. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1894) GlobalDictionary may corrupt when server suddenly crash
[ https://issues.apache.org/jira/browse/KYLIN-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382328#comment-15382328 ] Shaofeng SHI commented on KYLIN-1894: - Looks good; patch merged and this time 'mvn test' is passed; thanks > GlobalDictionary may corrupt when server suddenly crash > --- > > Key: KYLIN-1894 > URL: https://issues.apache.org/jira/browse/KYLIN-1894 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: v1.5.3 >Reporter: Yerui Sun >Assignee: Yerui Sun > Fix For: v1.5.3 > > Attachments: KYLIN-1894.2.patch, KYLIN-1894.3.patch > > > Global Dictionary store data on hdfs directly, and overwrite directly when > data file updated. If the server crashed suddenly during writing file, the > data file may be corrupt and can't be recovered. > To resolve this problem, copy the data file into a tmp directory and copy > back after the file is updated successfully. > I'll post a patch later with this solution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1890) support hbase table prefix configurable
[ https://issues.apache.org/jira/browse/KYLIN-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382310#comment-15382310 ] hongbin ma commented on KYLIN-1890: --- it's a good idea. besides this solution, kylin will also attach a tag to each created htable. (check the htable's HTableDescriptor, entry "KYLIN_HOST") The tag will be the metadata store name: /** * For each cube htable, we leverage htable's metadata to keep track of * which kylin server(represented by its kylin_metadata prefix) owns this htable */ public final static String HTableTag = "KYLIN_HOST"; > support hbase table prefix configurable > --- > > Key: KYLIN-1890 > URL: https://issues.apache.org/jira/browse/KYLIN-1890 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.5.2 >Reporter: fengYu >Assignee: fengYu > Attachments: > 0001-KYLIN-1890-support-hbase-table-prefix-configurable.patch > > > some times we need deploy two kylin env based on same hbase, I want to change > hbase table name prefix based two reasons: > 1、different kylin env will generate the same table name > 2、while clean invalid htable for one env will cause delete all tables belong > to another env. > different kylin env use different namespace is acceptable either. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1888) support forward cube building
[ https://issues.apache.org/jira/browse/KYLIN-1888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382304#comment-15382304 ] hongbin ma commented on KYLIN-1888: --- The term forward is not suitable here, are you indicating "backward"? > support forward cube building > - > > Key: KYLIN-1888 > URL: https://issues.apache.org/jira/browse/KYLIN-1888 > Project: Kylin > Issue Type: Improvement >Affects Versions: v1.5.2 >Reporter: fengYu >Assignee: fengYu > Attachments: 0001-forward-build-job.patch > > > This is used when user want to see data from last some days, and then fill up > history data from cube start date. > FORWARD just like reverse side of BUILD -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1867) Upgrade dependency libraries
[ https://issues.apache.org/jira/browse/KYLIN-1867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382294#comment-15382294 ] hongbin ma commented on KYLIN-1867: --- Hi billy Is the patch already merged in master? Suggest to dry run the kylin server in sandbox to make sure everything is okay. UT and IT and carefully protected by maven. > Upgrade dependency libraries > > > Key: KYLIN-1867 > URL: https://issues.apache.org/jira/browse/KYLIN-1867 > Project: Kylin > Issue Type: Improvement > Components: 3rd Party, Tools, Build and Test >Affects Versions: v1.5.2 >Reporter: Billy(Yiming) Liu >Assignee: hongbin ma >Priority: Minor > Attachments: KYLIN-1867.patch > > > Currently, Kylin has 167 unique dependencies, but 109 are outdated, even some > has security vulnerabilities. The detail report could be found at > https://www.versioneye.com/user/projects/577fcf5a5bb139003969db09?child=summary > Without changing the APIs and Hadoop ecosystem version compatible, it's > better if Kylin could catch up to the newest release dependencies. > I could work on this item, and check the compatible and pass the tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1480) NPE throws while execute sql with more than two join.
[ https://issues.apache.org/jira/browse/KYLIN-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382282#comment-15382282 ] hongbin ma commented on KYLIN-1480: --- As I checked at least it's not applied on the master branch > NPE throws while execute sql with more than two join. > - > > Key: KYLIN-1480 > URL: https://issues.apache.org/jira/browse/KYLIN-1480 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v1.4.0, v1.2, v1.1, v1.0 >Reporter: fengYu >Assignee: fengYu > Attachments: 0001-unit-test-case-for-KYLIN-1480.patch, > NPE-in-more-joins.patch > > > Hi, I encounter NPE while execute sql more than two join, for example : > select A.type, A.cmd, count(1) from fact as A inner join (select type, > count(1) from fact group by type having count(1) > 2) as B on A.type = B.type > inner join (select cmd, count(1) from fact group by cmd having count(1) > 2) > as C on A.cmd = C.cmd group by A.type, A.cmd; > the fact table is create like this : > CREATE TABLE `fact`( > `fname` string, > `lname` string, > `dt` date, > `cost` int, > `type` string, > `cmd` string); > Kylin throws exception like this : > Caused by: java.lang.NullPointerException > at > org.apache.kylin.query.relnode.OLAPProjectRel.implementOLAP(OLAPProjectRel.java:103) > at > org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:81) > at > org.apache.kylin.query.relnode.OLAPAggregateRel.implementOLAP(OLAPAggregateRel.java:132) > at > org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:81) > at > org.apache.kylin.query.relnode.OLAPLimitRel.implementOLAP(OLAPLimitRel.java:73) > at > org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:81) > at > org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:67) > at > org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:99) > at > org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1050) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:293) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:188) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:671) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:572) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:541) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:173) > at > org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:561) > at > org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:477) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:109) > I try it in kylin-1.0 and kylin-2.x-staging, the same exception throws. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-1884) Reload metadata automatically after migrating cube
[ https://issues.apache.org/jira/browse/KYLIN-1884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-1884. - Resolution: Fixed Looks good; merged in master and v1.5.3-release branch. thanks Kaisen! > Reload metadata automatically after migrating cube > -- > > Key: KYLIN-1884 > URL: https://issues.apache.org/jira/browse/KYLIN-1884 > Project: Kylin > Issue Type: Improvement > Components: Tools, Build and Test >Affects Versions: v1.5.2 >Reporter: kangkaisen >Assignee: kangkaisen > Fix For: v1.5.3 > > Attachments: KYLIN-1884.patch > > > in the current version of Kylin, after migrating cube we need reload metadata > manually. > in our production environment, we have many restServers. > so, we hope to reload metadata automatically after migrating cube. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1480) NPE throws while execute sql with more than two join.
[ https://issues.apache.org/jira/browse/KYLIN-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382277#comment-15382277 ] hongbin ma commented on KYLIN-1480: --- what is the fixed version for the JIRA? The comments are in a mess > NPE throws while execute sql with more than two join. > - > > Key: KYLIN-1480 > URL: https://issues.apache.org/jira/browse/KYLIN-1480 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v1.4.0, v1.2, v1.1, v1.0 >Reporter: fengYu >Assignee: fengYu > Attachments: 0001-unit-test-case-for-KYLIN-1480.patch, > NPE-in-more-joins.patch > > > Hi, I encounter NPE while execute sql more than two join, for example : > select A.type, A.cmd, count(1) from fact as A inner join (select type, > count(1) from fact group by type having count(1) > 2) as B on A.type = B.type > inner join (select cmd, count(1) from fact group by cmd having count(1) > 2) > as C on A.cmd = C.cmd group by A.type, A.cmd; > the fact table is create like this : > CREATE TABLE `fact`( > `fname` string, > `lname` string, > `dt` date, > `cost` int, > `type` string, > `cmd` string); > Kylin throws exception like this : > Caused by: java.lang.NullPointerException > at > org.apache.kylin.query.relnode.OLAPProjectRel.implementOLAP(OLAPProjectRel.java:103) > at > org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:81) > at > org.apache.kylin.query.relnode.OLAPAggregateRel.implementOLAP(OLAPAggregateRel.java:132) > at > org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:81) > at > org.apache.kylin.query.relnode.OLAPLimitRel.implementOLAP(OLAPLimitRel.java:73) > at > org.apache.kylin.query.relnode.OLAPRel$OLAPImplementor.visitChild(OLAPRel.java:81) > at > org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:67) > at > org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:99) > at > org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92) > at > org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1050) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:293) > at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:188) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:671) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:572) > at > org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:541) > at > org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:173) > at > org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:561) > at > org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:477) > at > org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:109) > I try it in kylin-1.0 and kylin-2.x-staging, the same exception throws. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1854) Allow deleting cube instance when its underlying cubedesc went wrong
[ https://issues.apache.org/jira/browse/KYLIN-1854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382265#comment-15382265 ] hongbin ma commented on KYLIN-1854: --- it won't the jira is meant to make web GUI more robust when something like cube description went inconsistent,etc. > Allow deleting cube instance when its underlying cubedesc went wrong > > > Key: KYLIN-1854 > URL: https://issues.apache.org/jira/browse/KYLIN-1854 > Project: Kylin > Issue Type: Improvement >Reporter: hongbin ma >Assignee: hongbin ma > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1899) Sample Cube data could not be initialized, I am confused
[ https://issues.apache.org/jira/browse/KYLIN-1899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382260#comment-15382260 ] hongbin ma commented on KYLIN-1899: --- if it's compression related issue, refer to http://kylin.apache.org/docs15/install/advance_settings.html > Sample Cube data could not be initialized, I am confused > > > Key: KYLIN-1899 > URL: https://issues.apache.org/jira/browse/KYLIN-1899 > Project: Kylin > Issue Type: Test > Components: Environment >Affects Versions: v1.5.2 > Environment: hadoop-2.6.4, > hbase-1.1.4,hive-1.2.1,apache-kylin-1.5.2.1-bin >Reporter: guoguo >Assignee: hongbin ma > Labels: starter > > When I ran sample.sh，it says the scripts were completed. Then I open the Web > UI and selected the "learn_kylin" project and clicked the action->>build > button，and I waited the progress in the Monitor Tag. However the progress > always ended with errors, and the error info shows: > MapredLocal task succeeded > Launching Job 1 out of 3 > Number of reduce tasks is set to 0 since there's no reduce operator > Starting Job = job_1468687641453_0002, Tracking URL = > http://master:/proxy/application_1468687641453_0002/ > Kill Command = /home/hadoop/hadoop-2.6.4/bin/hadoop job -kill > job_1468687641453_0002 > Hadoop job information for Stage-11: number of mappers: 1; number of > reducers: 0 > 2016-07-17 00:56:30,157 Stage-11 map = 0%, reduce = 0% > 2016-07-17 00:56:32,260 Stage-11 map = 100%, reduce = 0% > Ended Job = job_1468687641453_0002 with errors > Error during job, obtaining debugging information... > Examining task ID: task_1468687641453_0002_m_00 (and more) from job > job_1468687641453_0002 > Task with the most failures(1): > - > Task ID: > task_1468687641453_0002_m_00 > URL: > > http://master:8088/taskdetails.jsp?jobid=job_1468687641453_0002=task_1468687641453_0002_m_00 > - > Diagnostic Messages for this Task: > Error: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z > FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.mr.MapRedTask > MapReduce Jobs Launched: > Stage-Stage-11: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL > Total MapReduce CPU Time Spent: 0 msec > I don't know what's wrong and how to fix the problem? Thanks in advance! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1869) When building snapshot for lookup tables, should we build those dimensions used by model or the whole
[ https://issues.apache.org/jira/browse/KYLIN-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382233#comment-15382233 ] hongbin ma commented on KYLIN-1869: --- Sorry, https://issues.apache.org/jira/browse/KYLIN-1313 might not be able to overcome snapshot's size limit > When building snapshot for lookup tables, should we build those dimensions > used by model or the whole > - > > Key: KYLIN-1869 > URL: https://issues.apache.org/jira/browse/KYLIN-1869 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > Currently when building a snapshot for a lookup table, the input is the whole > value set of the lookup table, which may be not so reasonable. In some cases, > a lookup table owns tens columns. However, the columns used by a model or a > cube is only a few, 1 to 5. Those unused columns will make the snapshot too > large, which will bring burdens for both storing and loading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1869) When building snapshot for lookup tables, should we build those dimensions used by model or the whole
[ https://issues.apache.org/jira/browse/KYLIN-1869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382230#comment-15382230 ] hongbin ma commented on KYLIN-1869: --- will https://issues.apache.org/jira/browse/KYLIN-1313 possibly help? haven't tried it for columns on lookup table, but I assume it will work? > When building snapshot for lookup tables, should we build those dimensions > used by model or the whole > - > > Key: KYLIN-1869 > URL: https://issues.apache.org/jira/browse/KYLIN-1869 > Project: Kylin > Issue Type: Improvement > Components: Job Engine >Reporter: Zhong Yanghong >Assignee: Zhong Yanghong > > Currently when building a snapshot for a lookup table, the input is the whole > value set of the lookup table, which may be not so reasonable. In some cases, > a lookup table owns tens columns. However, the columns used by a model or a > cube is only a few, 1 to 5. Those unused columns will make the snapshot too > large, which will bring burdens for both storing and loading. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (KYLIN-1785) NoSuchElementException when Mandatory Dimensions contains all Dimensions
[
https://issues.apache.org/jira/browse/KYLIN-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hongbin ma reopened KYLIN-1785:
---
> NoSuchElementException when Mandatory Dimensions contains all Dimensions
>
>
> Key: KYLIN-1785
> URL: https://issues.apache.org/jira/browse/KYLIN-1785
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v1.5.1, v1.5.2
>Reporter: guohuili
>Assignee: hongbin ma
> Fix For: v1.5.3
>
>
> When {{Mandatory Dimensions}} included all dimensions in the Aggregation
> Groups, {{NoSuchElementException}} will thrown in {{Build N-Dimension Cuboid
> Data}} step(or {{Build Cube}} steps if in-mem cubing):
> {code}
> 2016-06-13 11:46:13,842 INFO [main] org.apache.kylin.dict.DictionaryManager:
> DictionaryManager(1419528284) loading DictionaryInfo(loadDictObj:true) at
> /dict/FUJIAN.HTTP_10T_PARTITION/MODEL_NAME/a1437b13-e7f6-49dc-bad7-232f80535f9a.dict
> 2016-06-13 11:46:13,847 INFO [main] org.apache.hadoop.mapred.MapTask:
> Starting flush of map output
> 2016-06-13 11:46:13,918 INFO [main] org.apache.hadoop.io.compress.CodecPool:
> Got brand-new compressor [.snappy]
> 2016-06-13 11:46:14,895 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.util.NoSuchElementException
> at java.util.ArrayList$Itr.next(ArrayList.java:834)
> at java.util.Collections.min(Collections.java:665)
> at
> org.apache.kylin.cube.cuboid.Cuboid.translateToValidCuboid(Cuboid.java:201)
> at
> org.apache.kylin.cube.cuboid.Cuboid.translateToValidCuboid(Cuboid.java:125)
> at org.apache.kylin.cube.cuboid.Cuboid.findById(Cuboid.java:67)
> at
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.map(NDCuboidMapper.java:148)
> at
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.map(NDCuboidMapper.java:49)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1793)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> 2016-06-13 11:46:15,934 INFO [main] org.apache.hadoop.mapred.Task: Runnning
> cleanup for the task
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KYLIN-1894) GlobalDictionary may corrupt when server suddenly crash
[ https://issues.apache.org/jira/browse/KYLIN-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yerui Sun updated KYLIN-1894: - Attachment: KYLIN-1894.3.patch [~Shaofengshi], sorry for my mistake, I missed one test before post the patch. The new patch is attached, based on the previous one. The tmp files also been cleaned as your guide line. Thanks for your reviewing again~ > GlobalDictionary may corrupt when server suddenly crash > --- > > Key: KYLIN-1894 > URL: https://issues.apache.org/jira/browse/KYLIN-1894 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: v1.5.3 >Reporter: Yerui Sun >Assignee: Yerui Sun > Fix For: v1.5.3 > > Attachments: KYLIN-1894.2.patch, KYLIN-1894.3.patch > > > Global Dictionary store data on hdfs directly, and overwrite directly when > data file updated. If the server crashed suddenly during writing file, the > data file may be corrupt and can't be recovered. > To resolve this problem, copy the data file into a tmp directory and copy > back after the file is updated successfully. > I'll post a patch later with this solution. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (KYLIN-1860) "Models" is slow to show up
[
https://issues.apache.org/jira/browse/KYLIN-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382006#comment-15382006
]
Shaofeng SHI edited comment on KYLIN-1860 at 7/18/16 9:49 AM:
--
in MetadataManager.getModels() method, there is such a code snippet; it was for
the metadata compitability when upgrading from 0.6 I remember (may have cube
but no model); Now it is time to remove this redundant code
{code}
//TODO, list model from realization,compatible with old meta data,will
remove
RealizationRegistry registry = RealizationRegistry.getInstance(config);
for (RealizationEntry realization :
projectInstance.getRealizationEntries()) {
IRealization rel = registry.getRealization(realization.getType(),
realization.getRealization());
if (rel != null) {
DataModelDesc modelDesc = rel.getDataModelDesc();
if (modelDesc != null && !ret.contains(modelDesc)) {
ProjectManager.getInstance(config).updateModelToProject(modelDesc.getName(),
projectName);
ret.add(modelDesc);
}
} else {
logger.warn("Realization '" + realization + "' defined under
project '" + projectInstance + "' is not found");
}
}
{code}
was (Author: shaofengshi):
in MetadataManager.getModels() method, there is such a code snippet:
{code}
//TODO, list model from realization,compatible with old meta data,will
remove
RealizationRegistry registry = RealizationRegistry.getInstance(config);
for (RealizationEntry realization :
projectInstance.getRealizationEntries()) {
IRealization rel = registry.getRealization(realization.getType(),
realization.getRealization());
if (rel != null) {
DataModelDesc modelDesc = rel.getDataModelDesc();
if (modelDesc != null && !ret.contains(modelDesc)) {
ProjectManager.getInstance(config).updateModelToProject(modelDesc.getName(),
projectName);
ret.add(modelDesc);
}
} else {
logger.warn("Realization '" + realization + "' defined under
project '" + projectInstance + "' is not found");
}
}
{code}
> "Models" is slow to show up
> ---
>
> Key: KYLIN-1860
> URL: https://issues.apache.org/jira/browse/KYLIN-1860
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v1.5.2, v1.5.3
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
> Fix For: v1.5.3
>
> Attachments: Screen Shot 2016-07-08 at 5.10.41 PM.png
>
>
> There is an "loading" icon, after about 1 minute it shows up, this is too
> slow; there must be something wrong. Pls see the screenshot
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-1860) "Models" is slow to show up
[
https://issues.apache.org/jira/browse/KYLIN-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15382006#comment-15382006
]
Shaofeng SHI commented on KYLIN-1860:
-
in MetadataManager.getModels() method, there is such a code snippet:
{code}
//TODO, list model from realization,compatible with old meta data,will
remove
RealizationRegistry registry = RealizationRegistry.getInstance(config);
for (RealizationEntry realization :
projectInstance.getRealizationEntries()) {
IRealization rel = registry.getRealization(realization.getType(),
realization.getRealization());
if (rel != null) {
DataModelDesc modelDesc = rel.getDataModelDesc();
if (modelDesc != null && !ret.contains(modelDesc)) {
ProjectManager.getInstance(config).updateModelToProject(modelDesc.getName(),
projectName);
ret.add(modelDesc);
}
} else {
logger.warn("Realization '" + realization + "' defined under
project '" + projectInstance + "' is not found");
}
}
{code}
> "Models" is slow to show up
> ---
>
> Key: KYLIN-1860
> URL: https://issues.apache.org/jira/browse/KYLIN-1860
> Project: Kylin
> Issue Type: Bug
> Components: Web
>Affects Versions: v1.5.2, v1.5.3
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
> Fix For: v1.5.3
>
> Attachments: Screen Shot 2016-07-08 at 5.10.41 PM.png
>
>
> There is an "loading" icon, after about 1 minute it shows up, this is too
> slow; there must be something wrong. Pls see the screenshot
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KYLIN-1860) "Models" is slow to show up
[ https://issues.apache.org/jira/browse/KYLIN-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-1860: Summary: "Models" is slow to show up (was: "Models" is very slow to show up) > "Models" is slow to show up > --- > > Key: KYLIN-1860 > URL: https://issues.apache.org/jira/browse/KYLIN-1860 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v1.5.2, v1.5.3 >Reporter: Shaofeng SHI >Assignee: Zhong,Jason > Fix For: v1.5.3 > > Attachments: Screen Shot 2016-07-08 at 5.10.41 PM.png > > > There is an "loading" icon, after about 1 minute it shows up, this is too > slow; there must be something wrong. Pls see the screenshot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1901) Diagnosis package run command failed when using beeline
[
https://issues.apache.org/jira/browse/KYLIN-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381984#comment-15381984
]
Billy(Yiming) Liu commented on KYLIN-1901:
--
No blocking.
> Diagnosis package run command failed when using beeline
> ---
>
> Key: KYLIN-1901
> URL: https://issues.apache.org/jira/browse/KYLIN-1901
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v1.5.2
>Reporter: Billy(Yiming) Liu
>Assignee: Dong Li
>Priority: Minor
>
> It seems passed the wrong parameters for Beeline case.
> {quote}
> 2016-07-18 15:15:13,863 INFO [http-bio-7070-exec-2]
> tool.JobDiagnosisInfoCLI:187 : Queue:default
> 2016-07-18 15:15:13,871 INFO [http-bio-7070-exec-2]
> tool.JobDiagnosisInfoCLI:130 : ClientEnvExtractor args: [-destDir,
> /opt/client/kylin/apache-kylin-1.5.2.1-bin/bin/../tomcat/temp/1468826111567-0/job_2016_07_18_00_15_11/client,
> -compress, false, -submodule, true]
> 2016-07-18 15:15:21,324 WARN [http-bio-7070-exec-2]
> tool.ClientEnvExtractor:98 : Failed to run command: beeline --version.
> java.io.IOException: OS command error exit with 255 -- beeline --version
> WARNING: Use "yarn jar" to launch YARN applications.
> Not a valid JAR: /opt/client/Hive/Beeline/lib/-u
> at
> org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:92)
> at
> org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:80)
> at
> org.apache.kylin.tool.ClientEnvExtractor.addShellOutput(ClientEnvExtractor.java:94)
> at
> org.apache.kylin.tool.ClientEnvExtractor.executeExtract(ClientEnvExtractor.java:65)
> at
> org.apache.kylin.tool.AbstractInfoExtractor.execute(AbstractInfoExtractor.java:99)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:39)
> at
> org.apache.kylin.tool.JobDiagnosisInfoCLI.executeExtract(JobDiagnosisInfoCLI.java:131)
> at
> org.apache.kylin.tool.AbstractInfoExtractor.execute(AbstractInfoExtractor.java:99)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:39)
> at
> org.apache.kylin.rest.service.DiagnosisService.dumpJobDiagnosisInfo(DiagnosisService.java:64)
> at
> org.apache.kylin.rest.service.DiagnosisService$$FastClassByCGLIB$$d802fbf6.invoke()
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:64)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622)
> at
> org.apache.kylin.rest.service.DiagnosisService$$EnhancerByCGLIB$$8319c027.dumpJobDiagnosisInfo()
> at
> org.apache.kylin.rest.controller.DiagnosisController.dumpJobDiagnosisInfo(DiagnosisController.java:99)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
> at
>
[jira] [Commented] (KYLIN-1901) Diagnosis package run command failed when using beeline
[
https://issues.apache.org/jira/browse/KYLIN-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381977#comment-15381977
]
Dong Li commented on KYLIN-1901:
Is this message blocking package generation?
> Diagnosis package run command failed when using beeline
> ---
>
> Key: KYLIN-1901
> URL: https://issues.apache.org/jira/browse/KYLIN-1901
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v1.5.2
>Reporter: Billy(Yiming) Liu
>Assignee: Dong Li
>Priority: Minor
>
> It seems passed the wrong parameters for Beeline case.
> {quote}
> 2016-07-18 15:15:13,863 INFO [http-bio-7070-exec-2]
> tool.JobDiagnosisInfoCLI:187 : Queue:default
> 2016-07-18 15:15:13,871 INFO [http-bio-7070-exec-2]
> tool.JobDiagnosisInfoCLI:130 : ClientEnvExtractor args: [-destDir,
> /opt/client/kylin/apache-kylin-1.5.2.1-bin/bin/../tomcat/temp/1468826111567-0/job_2016_07_18_00_15_11/client,
> -compress, false, -submodule, true]
> 2016-07-18 15:15:21,324 WARN [http-bio-7070-exec-2]
> tool.ClientEnvExtractor:98 : Failed to run command: beeline --version.
> java.io.IOException: OS command error exit with 255 -- beeline --version
> WARNING: Use "yarn jar" to launch YARN applications.
> Not a valid JAR: /opt/client/Hive/Beeline/lib/-u
> at
> org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:92)
> at
> org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:80)
> at
> org.apache.kylin.tool.ClientEnvExtractor.addShellOutput(ClientEnvExtractor.java:94)
> at
> org.apache.kylin.tool.ClientEnvExtractor.executeExtract(ClientEnvExtractor.java:65)
> at
> org.apache.kylin.tool.AbstractInfoExtractor.execute(AbstractInfoExtractor.java:99)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:39)
> at
> org.apache.kylin.tool.JobDiagnosisInfoCLI.executeExtract(JobDiagnosisInfoCLI.java:131)
> at
> org.apache.kylin.tool.AbstractInfoExtractor.execute(AbstractInfoExtractor.java:99)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:39)
> at
> org.apache.kylin.rest.service.DiagnosisService.dumpJobDiagnosisInfo(DiagnosisService.java:64)
> at
> org.apache.kylin.rest.service.DiagnosisService$$FastClassByCGLIB$$d802fbf6.invoke()
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:64)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622)
> at
> org.apache.kylin.rest.service.DiagnosisService$$EnhancerByCGLIB$$8319c027.dumpJobDiagnosisInfo()
> at
> org.apache.kylin.rest.controller.DiagnosisController.dumpJobDiagnosisInfo(DiagnosisController.java:99)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
> at
>
[jira] [Assigned] (KYLIN-1901) Diagnosis package run command failed when using beeline
[
https://issues.apache.org/jira/browse/KYLIN-1901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dong Li reassigned KYLIN-1901:
--
Assignee: Dong Li (was: hongbin ma)
> Diagnosis package run command failed when using beeline
> ---
>
> Key: KYLIN-1901
> URL: https://issues.apache.org/jira/browse/KYLIN-1901
> Project: Kylin
> Issue Type: Bug
> Components: Tools, Build and Test
>Affects Versions: v1.5.2
>Reporter: Billy(Yiming) Liu
>Assignee: Dong Li
>Priority: Minor
>
> It seems passed the wrong parameters for Beeline case.
> {quote}
> 2016-07-18 15:15:13,863 INFO [http-bio-7070-exec-2]
> tool.JobDiagnosisInfoCLI:187 : Queue:default
> 2016-07-18 15:15:13,871 INFO [http-bio-7070-exec-2]
> tool.JobDiagnosisInfoCLI:130 : ClientEnvExtractor args: [-destDir,
> /opt/client/kylin/apache-kylin-1.5.2.1-bin/bin/../tomcat/temp/1468826111567-0/job_2016_07_18_00_15_11/client,
> -compress, false, -submodule, true]
> 2016-07-18 15:15:21,324 WARN [http-bio-7070-exec-2]
> tool.ClientEnvExtractor:98 : Failed to run command: beeline --version.
> java.io.IOException: OS command error exit with 255 -- beeline --version
> WARNING: Use "yarn jar" to launch YARN applications.
> Not a valid JAR: /opt/client/Hive/Beeline/lib/-u
> at
> org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:92)
> at
> org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:80)
> at
> org.apache.kylin.tool.ClientEnvExtractor.addShellOutput(ClientEnvExtractor.java:94)
> at
> org.apache.kylin.tool.ClientEnvExtractor.executeExtract(ClientEnvExtractor.java:65)
> at
> org.apache.kylin.tool.AbstractInfoExtractor.execute(AbstractInfoExtractor.java:99)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:39)
> at
> org.apache.kylin.tool.JobDiagnosisInfoCLI.executeExtract(JobDiagnosisInfoCLI.java:131)
> at
> org.apache.kylin.tool.AbstractInfoExtractor.execute(AbstractInfoExtractor.java:99)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:39)
> at
> org.apache.kylin.rest.service.DiagnosisService.dumpJobDiagnosisInfo(DiagnosisService.java:64)
> at
> org.apache.kylin.rest.service.DiagnosisService$$FastClassByCGLIB$$d802fbf6.invoke()
> at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:64)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622)
> at
> org.apache.kylin.rest.service.DiagnosisService$$EnhancerByCGLIB$$8319c027.dumpJobDiagnosisInfo()
> at
> org.apache.kylin.rest.controller.DiagnosisController.dumpJobDiagnosisInfo(DiagnosisController.java:99)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
> at
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
> at
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
> at
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
> at
> org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
> at
> org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
> at
> org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
> at
> org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
> at
>
[jira] [Created] (KYLIN-1902) 404 error on get /kylin/models
Shaofeng SHI created KYLIN-1902: --- Summary: 404 error on get /kylin/models Key: KYLIN-1902 URL: https://issues.apache.org/jira/browse/KYLIN-1902 Project: Kylin Issue Type: Bug Components: Web Affects Versions: v1.5.2 Reporter: Shaofeng SHI Assignee: Zhong,Jason Fix For: v1.5.3 Request URL:http://localhost:7070/kylin/models Request Method:GET Status Code:404 Not Found Remote Address:[::1]:7070 I think this issue has existed for some time; Although it doesn't impact on user experience, an 404 error in web console is confusing. @Jason Can we fix it in v1.5.3? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-1894) GlobalDictionary may corrupt when server suddenly crash
[
https://issues.apache.org/jira/browse/KYLIN-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381919#comment-15381919
]
Shaofeng SHI commented on KYLIN-1894:
-
Hi Yerui, I applied the patch in the "v1.5.3-release" branch, and then run "mvn
test", it failed with NPE:
{code}
2016-07-18 16:42:54,917 ERROR [main TrieDictionary:182]: Not a valid value:
2016-07-18 16:42:54,917 ERROR [main TrieDictionary:182]: Not a valid value: p
2016-07-18 16:42:54,917 ERROR [main TrieDictionary:182]: Not a valid value: pa
2016-07-18 16:42:54,917 ERROR [main TrieDictionary:182]: Not a valid value: pb
2016-07-18 16:42:54,917 ERROR [main TrieDictionary:182]: Not a valid value:
parti
2016-07-18 16:42:54,917 ERROR [main TrieDictionary:182]: Not a valid value:
partz
2016-07-18 16:42:54,917 ERROR [main TrieDictionary:182]: Not a valid value:
partyz
Total 2 values
0 (0):
01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz
1 (1):
01234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789xyz
The length of the long string is 1136
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.735 sec - in
org.apache.kylin.dict.TrieDictionaryTest
Results :
Tests in error:
AppendTrieDictionaryTest.englishWordsTest:94->testStringDictAppend:166 »
NullPointer
AppendTrieDictionaryTest.testStringRepeatly:86->testStringDictAppend:166 »
NullPointer
AppendTrieDictionaryTest.categoryNamesTest:101->testStringDictAppend:166 »
NullPointer
Tests run: 30, Failures: 0, Errors: 3, Skipped: 1
{code}
Do I miss any commits in this branch?
> GlobalDictionary may corrupt when server suddenly crash
> ---
>
> Key: KYLIN-1894
> URL: https://issues.apache.org/jira/browse/KYLIN-1894
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v1.5.3
>Reporter: Yerui Sun
>Assignee: Yerui Sun
> Fix For: v1.5.3
>
> Attachments: KYLIN-1894.2.patch
>
>
> Global Dictionary store data on hdfs directly, and overwrite directly when
> data file updated. If the server crashed suddenly during writing file, the
> data file may be corrupt and can't be recovered.
> To resolve this problem, copy the data file into a tmp directory and copy
> back after the file is updated successfully.
> I'll post a patch later with this solution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-1785) NoSuchElementException when Mandatory Dimensions contains all Dimensions
[
https://issues.apache.org/jira/browse/KYLIN-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381896#comment-15381896
]
Wang Cheng commented on KYLIN-1785:
---
I couldn't recur this bug with following environments:
project: kylin default project
data model: test_kylin_inner_join_model_desc
cube description:
{
"uuid": "5758cc32-2383-446e-b4da-8c4171794677",
"version": "1.5.3",
"name": "test_kylin_cube_innerjoin_jira_desc",
"description": "",
"dimensions": [
{
"name": "DEFAULT.TEST_KYLIN_FACT.TRANS_ID",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "TRANS_ID",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.CAL_DT",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "CAL_DT",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.LSTG_FORMAT_NAME",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "LSTG_FORMAT_NAME",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.LEAF_CATEG_ID",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "LEAF_CATEG_ID",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.LSTG_SITE_ID",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "LSTG_SITE_ID",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.SLR_SEGMENT_CD",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "SLR_SEGMENT_CD",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.SELLER_ID",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "SELLER_ID",
"derived": null
},
{
"name": "EDW.TEST_CAL_DT_DERIVED",
"table": "EDW.TEST_CAL_DT",
"column": null,
"derived": [
"CAL_DT",
"WEEK_BEG_DT"
]
}
],
"measures": [
{
"name": "_COUNT_",
"function": {
"expression": "COUNT",
"parameter": {
"type": "constant",
"value": "1",
"next_parameter": null
},
"returntype": "bigint"
},
"dependent_measure_ref": null
}
],
"rowkey": {
"rowkey_columns": [
{
"column": "TRANS_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "CAL_DT",
"encoding": "date",
"isShardBy": false
},
{
"column": "LSTG_FORMAT_NAME",
"encoding": "dict",
"isShardBy": false
},
{
"column": "LEAF_CATEG_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "LSTG_SITE_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "SLR_SEGMENT_CD",
"encoding": "dict",
"isShardBy": false
},
{
"column": "SELLER_ID",
"encoding": "dict",
"isShardBy": false
}
]
},
"signature": "uOeyO87BMBQOb7jqzXhMmQ==",
"last_modified": 1468828440396,
"model_name": "test_kylin_inner_join_model_desc",
"null_string": null,
"hbase_mapping": {
"column_family": [
{
"name": "F1",
"columns": [
{
"qualifier": "M",
"measure_refs": [
"_COUNT_"
]
}
]
}
]
},
"aggregation_groups": [
{
"includes": [
"TRANS_ID",
"CAL_DT",
"LSTG_FORMAT_NAME",
"LEAF_CATEG_ID",
"LSTG_SITE_ID",
"SLR_SEGMENT_CD",
"SELLER_ID"
],
"select_rule": {
"hierarchy_dims": [],
"mandatory_dims": [
"TRANS_ID",
"CAL_DT",
"LSTG_FORMAT_NAME",
"LEAF_CATEG_ID",
"LSTG_SITE_ID",
"SLR_SEGMENT_CD",
"SELLER_ID"
],
"joint_dims": []
}
}
],
"notify_list": [],
"status_need_notify": [
"ERROR",
"DISCARDED",
"SUCCEED"
],
"partition_date_start": 0,
"partition_date_end": 31536,
"auto_merge_time_ranges": [
60480,
241920
],
"retention_range": 0,
"engine_type": 2,
"storage_type": 2,
"override_kylin_properties": {}
}
In "aggregation_groups" field, I filled all dims of "includes" into
"mandatory_dims" and it passed the build without any exceptions.
Maybe I misunderstood somewhere, could you pls provide your cube json or more
information.
> NoSuchElementException when Mandatory Dimensions contains all Dimensions
>
>
> Key: KYLIN-1785
> URL: https://issues.apache.org/jira/browse/KYLIN-1785
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v1.5.1, v1.5.2
>Reporter: guohuili
>Assignee: hongbin ma
> Fix For: v1.5.3
>
>
> When {{Mandatory Dimensions}} included all dimensions in the Aggregation
> Groups, {{NoSuchElementException}}
[jira] [Comment Edited] (KYLIN-1785) NoSuchElementException when Mandatory Dimensions contains all Dimensions
[
https://issues.apache.org/jira/browse/KYLIN-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15381896#comment-15381896
]
Wang Cheng edited comment on KYLIN-1785 at 7/18/16 8:36 AM:
I couldn't recur this bug with following environment:
project: kylin default project
data model: test_kylin_inner_join_model_desc
cube description:
{
"uuid": "5758cc32-2383-446e-b4da-8c4171794677",
"version": "1.5.3",
"name": "test_kylin_cube_innerjoin_jira_desc",
"description": "",
"dimensions": [
{
"name": "DEFAULT.TEST_KYLIN_FACT.TRANS_ID",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "TRANS_ID",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.CAL_DT",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "CAL_DT",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.LSTG_FORMAT_NAME",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "LSTG_FORMAT_NAME",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.LEAF_CATEG_ID",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "LEAF_CATEG_ID",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.LSTG_SITE_ID",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "LSTG_SITE_ID",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.SLR_SEGMENT_CD",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "SLR_SEGMENT_CD",
"derived": null
},
{
"name": "DEFAULT.TEST_KYLIN_FACT.SELLER_ID",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "SELLER_ID",
"derived": null
},
{
"name": "EDW.TEST_CAL_DT_DERIVED",
"table": "EDW.TEST_CAL_DT",
"column": null,
"derived": [
"CAL_DT",
"WEEK_BEG_DT"
]
}
],
"measures": [
{
"name": "_COUNT_",
"function": {
"expression": "COUNT",
"parameter": {
"type": "constant",
"value": "1",
"next_parameter": null
},
"returntype": "bigint"
},
"dependent_measure_ref": null
}
],
"rowkey": {
"rowkey_columns": [
{
"column": "TRANS_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "CAL_DT",
"encoding": "date",
"isShardBy": false
},
{
"column": "LSTG_FORMAT_NAME",
"encoding": "dict",
"isShardBy": false
},
{
"column": "LEAF_CATEG_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "LSTG_SITE_ID",
"encoding": "dict",
"isShardBy": false
},
{
"column": "SLR_SEGMENT_CD",
"encoding": "dict",
"isShardBy": false
},
{
"column": "SELLER_ID",
"encoding": "dict",
"isShardBy": false
}
]
},
"signature": "uOeyO87BMBQOb7jqzXhMmQ==",
"last_modified": 1468828440396,
"model_name": "test_kylin_inner_join_model_desc",
"null_string": null,
"hbase_mapping": {
"column_family": [
{
"name": "F1",
"columns": [
{
"qualifier": "M",
"measure_refs": [
"_COUNT_"
]
}
]
}
]
},
"aggregation_groups": [
{
"includes": [
"TRANS_ID",
"CAL_DT",
"LSTG_FORMAT_NAME",
"LEAF_CATEG_ID",
"LSTG_SITE_ID",
"SLR_SEGMENT_CD",
"SELLER_ID"
],
"select_rule": {
"hierarchy_dims": [],
"mandatory_dims": [
"TRANS_ID",
"CAL_DT",
"LSTG_FORMAT_NAME",
"LEAF_CATEG_ID",
"LSTG_SITE_ID",
"SLR_SEGMENT_CD",
"SELLER_ID"
],
"joint_dims": []
}
}
],
"notify_list": [],
"status_need_notify": [
"ERROR",
"DISCARDED",
"SUCCEED"
],
"partition_date_start": 0,
"partition_date_end": 31536,
"auto_merge_time_ranges": [
60480,
241920
],
"retention_range": 0,
"engine_type": 2,
"storage_type": 2,
"override_kylin_properties": {}
}
In "aggregation_groups" field, I filled all dims of "includes" into
"mandatory_dims" and it passed the build without any exceptions.
Maybe I misunderstood somewhere, could you pls provide your cube json or more
information.
was (Author: wormholer):
I couldn't recur this bug with following environments:
project: kylin default project
data model: test_kylin_inner_join_model_desc
cube description:
{
"uuid": "5758cc32-2383-446e-b4da-8c4171794677",
"version": "1.5.3",
"name": "test_kylin_cube_innerjoin_jira_desc",
"description": "",
"dimensions": [
{
"name": "DEFAULT.TEST_KYLIN_FACT.TRANS_ID",
"table": "DEFAULT.TEST_KYLIN_FACT",
"column": "TRANS_ID",
"derived": null
},
{
"name":
[jira] [Created] (KYLIN-1901) Diagnosis package run command failed when using beeline
Billy(Yiming) Liu created KYLIN-1901:
Summary: Diagnosis package run command failed when using beeline
Key: KYLIN-1901
URL: https://issues.apache.org/jira/browse/KYLIN-1901
Project: Kylin
Issue Type: Bug
Components: Tools, Build and Test
Affects Versions: v1.5.2
Reporter: Billy(Yiming) Liu
Assignee: hongbin ma
Priority: Minor
It seems passed the wrong parameters for Beeline case.
{quote}
2016-07-18 15:15:13,863 INFO [http-bio-7070-exec-2]
tool.JobDiagnosisInfoCLI:187 : Queue:default
2016-07-18 15:15:13,871 INFO [http-bio-7070-exec-2]
tool.JobDiagnosisInfoCLI:130 : ClientEnvExtractor args: [-destDir,
/opt/client/kylin/apache-kylin-1.5.2.1-bin/bin/../tomcat/temp/1468826111567-0/job_2016_07_18_00_15_11/client,
-compress, false, -submodule, true]
2016-07-18 15:15:21,324 WARN [http-bio-7070-exec-2] tool.ClientEnvExtractor:98
: Failed to run command: beeline --version.
java.io.IOException: OS command error exit with 255 -- beeline --version
WARNING: Use "yarn jar" to launch YARN applications.
Not a valid JAR: /opt/client/Hive/Beeline/lib/-u
at
org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:92)
at
org.apache.kylin.common.util.CliCommandExecutor.execute(CliCommandExecutor.java:80)
at
org.apache.kylin.tool.ClientEnvExtractor.addShellOutput(ClientEnvExtractor.java:94)
at
org.apache.kylin.tool.ClientEnvExtractor.executeExtract(ClientEnvExtractor.java:65)
at
org.apache.kylin.tool.AbstractInfoExtractor.execute(AbstractInfoExtractor.java:99)
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:39)
at
org.apache.kylin.tool.JobDiagnosisInfoCLI.executeExtract(JobDiagnosisInfoCLI.java:131)
at
org.apache.kylin.tool.AbstractInfoExtractor.execute(AbstractInfoExtractor.java:99)
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:39)
at
org.apache.kylin.rest.service.DiagnosisService.dumpJobDiagnosisInfo(DiagnosisService.java:64)
at
org.apache.kylin.rest.service.DiagnosisService$$FastClassByCGLIB$$d802fbf6.invoke()
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:689)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:64)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:622)
at
org.apache.kylin.rest.service.DiagnosisService$$EnhancerByCGLIB$$8319c027.dumpJobDiagnosisInfo()
at
org.apache.kylin.rest.controller.DiagnosisController.dumpJobDiagnosisInfo(DiagnosisController.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
at
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
at
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
at
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
at
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
at
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
at
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:778)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at
[jira] [Resolved] (KYLIN-1785) NoSuchElementException when Mandatory Dimensions contains all Dimensions
[
https://issues.apache.org/jira/browse/KYLIN-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-1785.
-
Resolution: Fixed
> NoSuchElementException when Mandatory Dimensions contains all Dimensions
>
>
> Key: KYLIN-1785
> URL: https://issues.apache.org/jira/browse/KYLIN-1785
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v1.5.1, v1.5.2
>Reporter: guohuili
>Assignee: hongbin ma
> Fix For: v1.5.3
>
>
> When {{Mandatory Dimensions}} included all dimensions in the Aggregation
> Groups, {{NoSuchElementException}} will thrown in {{Build N-Dimension Cuboid
> Data}} step(or {{Build Cube}} steps if in-mem cubing):
> {code}
> 2016-06-13 11:46:13,842 INFO [main] org.apache.kylin.dict.DictionaryManager:
> DictionaryManager(1419528284) loading DictionaryInfo(loadDictObj:true) at
> /dict/FUJIAN.HTTP_10T_PARTITION/MODEL_NAME/a1437b13-e7f6-49dc-bad7-232f80535f9a.dict
> 2016-06-13 11:46:13,847 INFO [main] org.apache.hadoop.mapred.MapTask:
> Starting flush of map output
> 2016-06-13 11:46:13,918 INFO [main] org.apache.hadoop.io.compress.CodecPool:
> Got brand-new compressor [.snappy]
> 2016-06-13 11:46:14,895 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.util.NoSuchElementException
> at java.util.ArrayList$Itr.next(ArrayList.java:834)
> at java.util.Collections.min(Collections.java:665)
> at
> org.apache.kylin.cube.cuboid.Cuboid.translateToValidCuboid(Cuboid.java:201)
> at
> org.apache.kylin.cube.cuboid.Cuboid.translateToValidCuboid(Cuboid.java:125)
> at org.apache.kylin.cube.cuboid.Cuboid.findById(Cuboid.java:67)
> at
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.map(NDCuboidMapper.java:148)
> at
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.map(NDCuboidMapper.java:49)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1793)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> 2016-06-13 11:46:15,934 INFO [main] org.apache.hadoop.mapred.Task: Runnning
> cleanup for the task
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (KYLIN-1785) NoSuchElementException when Mandatory Dimensions contains all Dimensions
[
https://issues.apache.org/jira/browse/KYLIN-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-1785:
Affects Version/s: v1.5.2
Fix Version/s: v1.5.3
> NoSuchElementException when Mandatory Dimensions contains all Dimensions
>
>
> Key: KYLIN-1785
> URL: https://issues.apache.org/jira/browse/KYLIN-1785
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v1.5.1, v1.5.2
>Reporter: guohuili
>Assignee: hongbin ma
> Fix For: v1.5.3
>
>
> When {{Mandatory Dimensions}} included all dimensions in the Aggregation
> Groups, {{NoSuchElementException}} will thrown in {{Build N-Dimension Cuboid
> Data}} step(or {{Build Cube}} steps if in-mem cubing):
> {code}
> 2016-06-13 11:46:13,842 INFO [main] org.apache.kylin.dict.DictionaryManager:
> DictionaryManager(1419528284) loading DictionaryInfo(loadDictObj:true) at
> /dict/FUJIAN.HTTP_10T_PARTITION/MODEL_NAME/a1437b13-e7f6-49dc-bad7-232f80535f9a.dict
> 2016-06-13 11:46:13,847 INFO [main] org.apache.hadoop.mapred.MapTask:
> Starting flush of map output
> 2016-06-13 11:46:13,918 INFO [main] org.apache.hadoop.io.compress.CodecPool:
> Got brand-new compressor [.snappy]
> 2016-06-13 11:46:14,895 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.util.NoSuchElementException
> at java.util.ArrayList$Itr.next(ArrayList.java:834)
> at java.util.Collections.min(Collections.java:665)
> at
> org.apache.kylin.cube.cuboid.Cuboid.translateToValidCuboid(Cuboid.java:201)
> at
> org.apache.kylin.cube.cuboid.Cuboid.translateToValidCuboid(Cuboid.java:125)
> at org.apache.kylin.cube.cuboid.Cuboid.findById(Cuboid.java:67)
> at
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.map(NDCuboidMapper.java:148)
> at
> org.apache.kylin.engine.mr.steps.NDCuboidMapper.map(NDCuboidMapper.java:49)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1793)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> 2016-06-13 11:46:15,934 INFO [main] org.apache.hadoop.mapred.Task: Runnning
> cleanup for the task
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
