[jira] [Commented] (KYLIN-2361) Upgrade to Tomcat 8.X
[ https://issues.apache.org/jira/browse/KYLIN-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852531#comment-15852531 ] Billy Liu commented on KYLIN-2361: -- https://github.com/openwide-java/tomcat-classloader-ordered provides the patch for ordered class loading. > Upgrade to Tomcat 8.X > - > > Key: KYLIN-2361 > URL: https://issues.apache.org/jira/browse/KYLIN-2361 > Project: Kylin > Issue Type: Task > Components: Web >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v2.0.0 > > > Apache Tomcat 8.5.x supports the same Servlet, JSP, EL, and WebSocket > Specification versions as Apache Tomcat 8.0.x. In addition to that, it also > implements the JASPIC 1.1 specification. There are significant changes in > many areas under the hood, resulting in improved performance, stability, and > total cost of ownership. Please refer to the Apache Tomcat 8.5 Changelog for > details. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Reopened] (KYLIN-2361) Upgrade to Tomcat 8.X
[ https://issues.apache.org/jira/browse/KYLIN-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu reopened KYLIN-2361: -- Tomcat 8 classloader could not guarantee the alphabetical order as it was the case with Tomcat 7.x. But Kylin depends on the loader order (atopcalcite and caicite), so the upgrade is pending. See the discussion in https://bz.apache.org/bugzilla/show_bug.cgi?id=57129 for more information. > Upgrade to Tomcat 8.X > - > > Key: KYLIN-2361 > URL: https://issues.apache.org/jira/browse/KYLIN-2361 > Project: Kylin > Issue Type: Task > Components: Web >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v2.0.0 > > > Apache Tomcat 8.5.x supports the same Servlet, JSP, EL, and WebSocket > Specification versions as Apache Tomcat 8.0.x. In addition to that, it also > implements the JASPIC 1.1 specification. There are significant changes in > many areas under the hood, resulting in improved performance, stability, and > total cost of ownership. Please refer to the Apache Tomcat 8.5 Changelog for > details. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
[ https://issues.apache.org/jira/browse/KYLIN-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15855491#comment-15855491 ] Billy Liu commented on KYLIN-2428: -- [INFO] Apache Kylin ... SUCCESS [ 3.045 s] [INFO] Apache Kylin - Calcite Overrides ... SUCCESS [ 2.161 s] [INFO] Apache Kylin - Core Common . SUCCESS [ 5.515 s] [INFO] Apache Kylin - Core Metadata ... SUCCESS [ 25.081 s] [INFO] Apache Kylin - Core Dictionary . SUCCESS [01:14 min] [INFO] Apache Kylin - Core Cube ... SUCCESS [01:44 min] [INFO] Apache Kylin - Core Job SUCCESS [01:58 min] [INFO] Apache Kylin - Core Storage SUCCESS [ 34.999 s] [INFO] Apache Kylin - MapReduce Engine SUCCESS [ 20.623 s] [INFO] Apache Kylin - HBase Storage ... SUCCESS [ 12.564 s] [INFO] Apache Kylin - Spark Engine SUCCESS [ 13.664 s] [INFO] Apache Kylin - Hive Source . SUCCESS [ 4.314 s] [INFO] Apache Kylin - Kafka Source SUCCESS [ 3.250 s] [INFO] Apache Kylin - Query ... SUCCESS [ 2.485 s] [INFO] Apache Kylin - Tool SUCCESS [ 4.199 s] [INFO] Apache Kylin - REST Server Base SUCCESS [ 6.802 s] [INFO] Apache Kylin - REST Server . SUCCESS [ 37.196 s] [INFO] Apache Kylin - JDBC Driver . SUCCESS [ 5.646 s] [INFO] Apache Kylin - Assembly SUCCESS [ 5.158 s] [INFO] Apache Kylin - Integration Test SUCCESS [ 01:08 h] [INFO] Apache Kylin - Tomcat Extension SUCCESS [ 2.810 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:16 h [INFO] Finished at: 2017-02-07T15:27:23+00:00 [INFO] Final Memory: 75M/935M > Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server > > > Key: KYLIN-2428 > URL: https://issues.apache.org/jira/browse/KYLIN-2428 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2428.patch > > > Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and > one web application: server. > Currently, all libraries have shaded some used third party libraries into the > package. For example, guava, curator, commons, kyro in kylin-job. The > duplicate libraries in runtime classpath may have potential class loading > conflicts and waste computing resource. We should leverage the hadoop > provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
Billy Liu created KYLIN-2428: Summary: Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server Key: KYLIN-2428 URL: https://issues.apache.org/jira/browse/KYLIN-2428 Project: Kylin Issue Type: Improvement Components: General Affects Versions: v1.6.0 Reporter: Billy Liu Assignee: Billy Liu Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and one web application: server. Currently, all libraries have shaded some used third party libraries into the package. For example, guava, curator, commons, kyro in kylin-job. The duplicate libraries in runtime classpath may have potential class loading conflicts and waste computing resource. We should leverage the hadoop provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
[ https://issues.apache.org/jira/browse/KYLIN-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15854147#comment-15854147 ] Billy Liu commented on KYLIN-2428: -- Compared with kylin 1.6: kylin-coprocessor is reduced from 2.4M to 1.9M, by removing extendedset, commons-lang3, but also introduced tduning.math kylin-jdbc increased from 7.6M to 8.5M since Avatica 1.9 upgrade. kylin-job is reduced from 9.5M to 5.8M, by removing esotericsoftware, guava, jsch, extendedset, jsr305, commons-cli, commons-io, commons-lang, curator and objenesis. Web application has reduced from 94 libraries to 80. > Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server > > > Key: KYLIN-2428 > URL: https://issues.apache.org/jira/browse/KYLIN-2428 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2428.patch > > > Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and > one web application: server. > Currently, all libraries have shaded some used third party libraries into the > package. For example, guava, curator, commons, kyro in kylin-job. The > duplicate libraries in runtime classpath may have potential class loading > conflicts and waste computing resource. We should leverage the hadoop > provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
[ https://issues.apache.org/jira/browse/KYLIN-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2428: - Attachment: KYLIN-2428.patch > Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server > > > Key: KYLIN-2428 > URL: https://issues.apache.org/jira/browse/KYLIN-2428 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2428.patch > > > Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and > one web application: server. > Currently, all libraries have shaded some used third party libraries into the > package. For example, guava, curator, commons, kyro in kylin-job. The > duplicate libraries in runtime classpath may have potential class loading > conflicts and waste computing resource. We should leverage the hadoop > provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
[ https://issues.apache.org/jira/browse/KYLIN-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15854211#comment-15854211 ] Billy Liu commented on KYLIN-2428: -- By refine the JDBC shade setting, the package has reduced from 7.6M to 5.4M, no duplicated shaded any more. > Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server > > > Key: KYLIN-2428 > URL: https://issues.apache.org/jira/browse/KYLIN-2428 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2428.patch > > > Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and > one web application: server. > Currently, all libraries have shaded some used third party libraries into the > package. For example, guava, curator, commons, kyro in kylin-job. The > duplicate libraries in runtime classpath may have potential class loading > conflicts and waste computing resource. We should leverage the hadoop > provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
[ https://issues.apache.org/jira/browse/KYLIN-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2428: - Attachment: (was: KYLIN-2428.patch) > Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server > > > Key: KYLIN-2428 > URL: https://issues.apache.org/jira/browse/KYLIN-2428 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2428.patch > > > Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and > one web application: server. > Currently, all libraries have shaded some used third party libraries into the > package. For example, guava, curator, commons, kyro in kylin-job. The > duplicate libraries in runtime classpath may have potential class loading > conflicts and waste computing resource. We should leverage the hadoop > provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
[ https://issues.apache.org/jira/browse/KYLIN-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2428: - Attachment: KYLIN-2428.patch > Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server > > > Key: KYLIN-2428 > URL: https://issues.apache.org/jira/browse/KYLIN-2428 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2428.patch > > > Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and > one web application: server. > Currently, all libraries have shaded some used third party libraries into the > package. For example, guava, curator, commons, kyro in kylin-job. The > duplicate libraries in runtime classpath may have potential class loading > conflicts and waste computing resource. We should leverage the hadoop > provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2424) Optimize the integration test's performance
[ https://issues.apache.org/jira/browse/KYLIN-2424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857252#comment-15857252 ] Billy Liu commented on KYLIN-2424: -- Thanks [~Shaofengshi], it reduces my IT from 3 hours to 1 hour. > Optimize the integration test's performance > --- > > Key: KYLIN-2424 > URL: https://issues.apache.org/jira/browse/KYLIN-2424 > Project: Kylin > Issue Type: Improvement > Components: Tools, Build and Test >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI > Fix For: v2.0.0 > > > Kylin's integration test is slow, especially the ITCombinationTest. Most of > time are spent on H2 to execute the test queries. In a latest integration > test, this test case take 90 minutes to finish. > By checking H2's document, I think the main problem is the absence of index > on the tables, while index is very important for a relational database's > query performance. So when Kylin create the tables in H2, shoud create index > on the columns that will be used in the queries, like the pk/fk, the > filtering columns etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2431) StorageCleanupJob will remove intermediate tables created by other kylin instances
[ https://issues.apache.org/jira/browse/KYLIN-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15857231#comment-15857231 ] Billy Liu commented on KYLIN-2431: -- +1 > StorageCleanupJob will remove intermediate tables created by other kylin > instances > -- > > Key: KYLIN-2431 > URL: https://issues.apache.org/jira/browse/KYLIN-2431 > Project: Kylin > Issue Type: Improvement >Reporter: Dong Li >Assignee: Dong Li >Priority: Minor > > If QA and PROD instances are in same hive environment, running > StorageCleanupJob on QA will remove intermediate tables created by PROD, > which might fail Kylin jobs of PROD. > A solution is to add metastore name to hive table prefix, then filter table > names with metastore name during cleanup job. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2424) Optimize the integration test's performance
[ https://issues.apache.org/jira/browse/KYLIN-2424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852812#comment-15852812 ] Billy Liu commented on KYLIN-2424: -- Hi [~Shaofengshi], I found you added new optional property index with default value 'T', is 'T' a kind of index? > Optimize the integration test's performance > --- > > Key: KYLIN-2424 > URL: https://issues.apache.org/jira/browse/KYLIN-2424 > Project: Kylin > Issue Type: Improvement > Components: Tools, Build and Test >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI > Fix For: v2.0.0 > > > Kylin's integration test is slow, especially the ITCombinationTest. Most of > time are spent on H2 to execute the test queries. In a latest integration > test, this test case take 90 minutes to finish. > By checking H2's document, I think the main problem is the absence of index > on the tables, while index is very important for a relational database's > query performance. So when Kylin create the tables in H2, shoud create index > on the columns that will be used in the queries, like the pk/fk, the > filtering columns etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2361) Upgrade to Tomcat 8.X
[ https://issues.apache.org/jira/browse/KYLIN-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15852697#comment-15852697 ] Billy Liu commented on KYLIN-2361: -- [INFO] Apache Kylin ... SUCCESS [ 3.211 s] [INFO] Apache Kylin - Calcite Overrides ... SUCCESS [ 2.743 s] [INFO] Apache Kylin - Core Common . SUCCESS [ 5.418 s] [INFO] Apache Kylin - Core Metadata ... SUCCESS [ 27.684 s] [INFO] Apache Kylin - Core Dictionary . SUCCESS [01:22 min] [INFO] Apache Kylin - Core Cube ... SUCCESS [01:54 min] [INFO] Apache Kylin - Core Job SUCCESS [01:58 min] [INFO] Apache Kylin - Core Storage SUCCESS [ 39.640 s] [INFO] Apache Kylin - MapReduce Engine SUCCESS [ 22.566 s] [INFO] Apache Kylin - HBase Storage ... SUCCESS [ 13.671 s] [INFO] Apache Kylin - Spark Engine SUCCESS [ 15.224 s] [INFO] Apache Kylin - Hive Source . SUCCESS [ 4.526 s] [INFO] Apache Kylin - Kafka Source SUCCESS [ 3.537 s] [INFO] Apache Kylin - Query ... SUCCESS [ 2.860 s] [INFO] Apache Kylin - Tool SUCCESS [ 4.780 s] [INFO] Apache Kylin - REST Server Base SUCCESS [ 8.026 s] [INFO] Apache Kylin - REST Server . SUCCESS [ 40.812 s] [INFO] Apache Kylin - JDBC Driver . SUCCESS [ 8.145 s] [INFO] Apache Kylin - Assembly SUCCESS [ 6.611 s] [INFO] Apache Kylin - Integration Test SUCCESS [ 03:02 h] [INFO] Apache Kylin - Tomcat Extension SUCCESS [ 1.025 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 03:10 h [INFO] Finished at: 2017-02-04T16:15:54+00:00 [INFO] Final Memory: 76M/926M [INFO] > Upgrade to Tomcat 8.X > - > > Key: KYLIN-2361 > URL: https://issues.apache.org/jira/browse/KYLIN-2361 > Project: Kylin > Issue Type: Task > Components: Web >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v2.0.0 > > > Apache Tomcat 8.5.x supports the same Servlet, JSP, EL, and WebSocket > Specification versions as Apache Tomcat 8.0.x. In addition to that, it also > implements the JASPIC 1.1 specification. There are significant changes in > many areas under the hood, resulting in improved performance, stability, and > total cost of ownership. Please refer to the Apache Tomcat 8.5 Changelog for > details. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (KYLIN-2361) Upgrade to Tomcat 8.X
[ https://issues.apache.org/jira/browse/KYLIN-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2361. -- Resolution: Fixed > Upgrade to Tomcat 8.X > - > > Key: KYLIN-2361 > URL: https://issues.apache.org/jira/browse/KYLIN-2361 > Project: Kylin > Issue Type: Task > Components: Web >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v2.0.0 > > > Apache Tomcat 8.5.x supports the same Servlet, JSP, EL, and WebSocket > Specification versions as Apache Tomcat 8.0.x. In addition to that, it also > implements the JASPIC 1.1 specification. There are significant changes in > many areas under the hood, resulting in improved performance, stability, and > total cost of ownership. Please refer to the Apache Tomcat 8.5 Changelog for > details. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2361) Upgrade to Tomcat 8.X
[ https://issues.apache.org/jira/browse/KYLIN-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15853606#comment-15853606 ] Billy Liu commented on KYLIN-2361: -- Code merged. > Upgrade to Tomcat 8.X > - > > Key: KYLIN-2361 > URL: https://issues.apache.org/jira/browse/KYLIN-2361 > Project: Kylin > Issue Type: Task > Components: Web >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v2.0.0 > > > Apache Tomcat 8.5.x supports the same Servlet, JSP, EL, and WebSocket > Specification versions as Apache Tomcat 8.0.x. In addition to that, it also > implements the JASPIC 1.1 specification. There are significant changes in > many areas under the hood, resulting in improved performance, stability, and > total cost of ownership. Please refer to the Apache Tomcat 8.5 Changelog for > details. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2394) Upgrade Calcite to 1.11
[ https://issues.apache.org/jira/browse/KYLIN-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832581#comment-15832581 ] Billy Liu commented on KYLIN-2394: -- The upgrade has passed IT. [INFO] Reactor Summary: [INFO] [INFO] Apache Kylin ... SUCCESS [ 4.838 s] [INFO] Apache Kylin - Calcite Overrides ... SUCCESS [ 3.787 s] [INFO] Apache Kylin - Core Common . SUCCESS [ 6.676 s] [INFO] Apache Kylin - Core Metadata ... SUCCESS [ 38.910 s] [INFO] Apache Kylin - Core Dictionary . SUCCESS [01:31 min] [INFO] Apache Kylin - Core Cube ... SUCCESS [01:48 min] [INFO] Apache Kylin - Core Job SUCCESS [04:10 min] [INFO] Apache Kylin - Core Storage SUCCESS [ 48.719 s] [INFO] Apache Kylin - MapReduce Engine SUCCESS [ 38.682 s] [INFO] Apache Kylin - HBase Storage ... SUCCESS [ 16.423 s] [INFO] Apache Kylin - Spark Engine SUCCESS [ 21.081 s] [INFO] Apache Kylin - Hive Source . SUCCESS [ 11.339 s] [INFO] Apache Kylin - Kafka Source SUCCESS [ 5.936 s] [INFO] Apache Kylin - Query ... SUCCESS [ 4.079 s] [INFO] Apache Kylin - Tool SUCCESS [ 6.418 s] [INFO] Apache Kylin - REST Server Base SUCCESS [ 10.294 s] [INFO] Apache Kylin - REST Server . SUCCESS [ 53.092 s] [INFO] Apache Kylin - JDBC Driver . SUCCESS [ 10.688 s] [INFO] Apache Kylin - Assembly SUCCESS [ 10.025 s] [INFO] Apache Kylin - Integration Test SUCCESS [ 03:10 h] [INFO] Apache Kylin - Tomcat Extension SUCCESS [ 4.546 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 03:22 h [INFO] Finished at: 2017-01-21T00:41:16+00:00 [INFO] Final Memory: 70M/796M [INFO] > Upgrade Calcite to 1.11 > --- > > Key: KYLIN-2394 > URL: https://issues.apache.org/jira/browse/KYLIN-2394 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > > Calcite has release 1.11.0 recently. In this JIRA, Kylin will upgrade the > calcite to 1.11.0 and avatica to 1.9.0. > Some potential refactor may be required, since protobuf 3.0(upgrade from 2.5) > is needed by Calcite. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2394) Upgrade Calcite to 1.11
[ https://issues.apache.org/jira/browse/KYLIN-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2394: - Attachment: 0002-KYLIN-2394-merge-CALCITE-1592.patch 0001-KYLIN-2394-Upgrade-Calcite-to-1.11-and-Avatica-to-1..patch > Upgrade Calcite to 1.11 > --- > > Key: KYLIN-2394 > URL: https://issues.apache.org/jira/browse/KYLIN-2394 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: > 0001-KYLIN-2394-Upgrade-Calcite-to-1.11-and-Avatica-to-1..patch, > 0002-KYLIN-2394-merge-CALCITE-1592.patch > > > Calcite has release 1.11.0 recently. In this JIRA, Kylin will upgrade the > calcite to 1.11.0 and avatica to 1.9.0. > Some potential refactor may be required, since protobuf 3.0(upgrade from 2.5) > is needed by Calcite. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2394) Upgrade Calcite to 1.11 and Avatica to 1.9
[ https://issues.apache.org/jira/browse/KYLIN-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2394: - Summary: Upgrade Calcite to 1.11 and Avatica to 1.9 (was: Upgrade Calcite to 1.11) > Upgrade Calcite to 1.11 and Avatica to 1.9 > -- > > Key: KYLIN-2394 > URL: https://issues.apache.org/jira/browse/KYLIN-2394 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Fix For: v2.0.0 > > Attachments: > 0001-KYLIN-2394-Upgrade-Calcite-to-1.11-and-Avatica-to-1..patch, > 0002-KYLIN-2394-merge-CALCITE-1592.patch > > > Calcite has release 1.11.0 recently. In this JIRA, Kylin will upgrade the > calcite to 1.11.0 and avatica to 1.9.0. > Some potential refactor may be required, since protobuf 3.0(upgrade from 2.5) > is needed by Calcite. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2394) Upgrade Calcite to 1.11
[ https://issues.apache.org/jira/browse/KYLIN-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2394. -- Resolution: Fixed Fix Version/s: v2.0.0 https://github.com/apache/kylin/commit/2c4591a609c8d8141c880722a3b8b282b785133e > Upgrade Calcite to 1.11 > --- > > Key: KYLIN-2394 > URL: https://issues.apache.org/jira/browse/KYLIN-2394 > Project: Kylin > Issue Type: New Feature > Components: Query Engine >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Fix For: v2.0.0 > > Attachments: > 0001-KYLIN-2394-Upgrade-Calcite-to-1.11-and-Avatica-to-1..patch, > 0002-KYLIN-2394-merge-CALCITE-1592.patch > > > Calcite has release 1.11.0 recently. In this JIRA, Kylin will upgrade the > calcite to 1.11.0 and avatica to 1.9.0. > Some potential refactor may be required, since protobuf 3.0(upgrade from 2.5) > is needed by Calcite. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2450) sample.sh should generate sample data on kylin.source.hive.database-for-flat-table, not hard code DEFAULT
Billy Liu created KYLIN-2450: Summary: sample.sh should generate sample data on kylin.source.hive.database-for-flat-table, not hard code DEFAULT Key: KYLIN-2450 URL: https://issues.apache.org/jira/browse/KYLIN-2450 Project: Kylin Issue Type: Improvement Components: General Affects Versions: v1.6.0 Reporter: Billy Liu Assignee: Billy Liu Priority: Minor Currently, the sample.sh will generate five sample tables into Hive DEFAULT database, and upload the sample cube, with hard-coded schema to DEFAULT also. But in some user cases, the DEFAULT database is not allowed to write. Kylin has property kylin.source.hive.database-for-flat-table, sample.sh should respect this variable and generate the samples into customized database. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (KYLIN-2448) Cloning a Model with a '-' in the name
[ https://issues.apache.org/jira/browse/KYLIN-2448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2448. -- Resolution: Fixed Fix Version/s: v2.0.0 This issue has been fixed at https://github.com/apache/kylin/commit/bc1a49ec57987cf52d2596ac4d7a439b7596eb0d > Cloning a Model with a '-' in the name > -- > > Key: KYLIN-2448 > URL: https://issues.apache.org/jira/browse/KYLIN-2448 > Project: Kylin > Issue Type: Bug >Affects Versions: v1.6.0 >Reporter: Berry Österlund >Priority: Trivial > Fix For: v2.0.0 > > > When I clone a model, and I put a '-' in the name of the new model, the model > gets created fine. I cant edit it, as that is not a allowed character. But > the model is actually there with the wrong name > It should be restricted to only clone a model with the allowed characters in > the name. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2363) support limit of dimensions in a cuboid
[ https://issues.apache.org/jira/browse/KYLIN-2363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15877465#comment-15877465 ] Billy Liu commented on KYLIN-2363: -- Thanks [~feng_xiao_yu], this is very useful. > support limit of dimensions in a cuboid > --- > > Key: KYLIN-2363 > URL: https://issues.apache.org/jira/browse/KYLIN-2363 > Project: Kylin > Issue Type: Improvement >Reporter: fengYu > > the scene like this: > I have 20+ dimensions, However the query will only use at most 5 dimensions > in all dimensions, so cuboid that contains 5+ dimensions(except base cuboid) > is useless. > I think we can add a configuration in cube, which limit the max dimensions > that cuboid includes. > What's more, we can config which level(number of dimension) need to > calculate. in above scene, we only calculate leve 1,2,3,4,5. and skip level 5+ -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2458) change a build job to DISCARDED status, the value in Duration column increase
[ https://issues.apache.org/jira/browse/KYLIN-2458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878457#comment-15878457 ] Billy Liu commented on KYLIN-2458: -- Thanks for reporting. Could you please describe how to reproduce it? > change a build job to DISCARDED status, the value in Duration column increase > -- > > Key: KYLIN-2458 > URL: https://issues.apache.org/jira/browse/KYLIN-2458 > Project: Kylin > Issue Type: Bug > Components: Web >Affects Versions: v1.6.0 >Reporter: 巧克力黑 >Assignee: Zhong,Jason > Attachments: discard.png, duration.png > > > When i discard a job to DISCARDED status, the job can change to a DISCARDED > status ,but the Duration is increasing all the time, and also > I discard the job when the kylin in a hive beeline process, i found the > hive's beeline process in kylin client is not killed in some situations. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2457) Should copy the latest dictionaries on dimension tables in a batch merge job
[ https://issues.apache.org/jira/browse/KYLIN-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878475#comment-15878475 ] Billy Liu commented on KYLIN-2457: -- +1 > Should copy the latest dictionaries on dimension tables in a batch merge job > > > Key: KYLIN-2457 > URL: https://issues.apache.org/jira/browse/KYLIN-2457 > Project: Kylin > Issue Type: Bug >Reporter: zhengdong >Priority: Critical > Attachments: > KYLIN-2457-Should-copy-the-latest-dictionaries-on-di.patch > > > In a batch merge job, we need to create dictionaries for all dimensions for > the new segment. For those dictionaries on dimension table, we currently just > copy them from the earliest segment of the merging segments. > However, we should select the newest dictionary for the new segment, since > the incremental dimension table is allowed. The older dictionary can't find > the records corresponding to the new key added to a dimension table and lead > wrong query result. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2417) Compare the performance between HDFSMetaStore and HBaseMetaStore
[ https://issues.apache.org/jira/browse/KYLIN-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878486#comment-15878486 ] Billy Liu commented on KYLIN-2417: -- Hi [~xiefan46], could you share the test result at here? > Compare the performance between HDFSMetaStore and HBaseMetaStore > > > Key: KYLIN-2417 > URL: https://issues.apache.org/jira/browse/KYLIN-2417 > Project: Kylin > Issue Type: Test >Reporter: XIE FAN >Assignee: XIE FAN > Fix For: v2.0.0 > > > We add a new metadata store HDFSMetaStore in KYLIN-2374. The new meta store > has passed the functional tests. But the performance of HDFSMetaStore is > uncertain. So we need to add some performance tests on it to ensure it's > quality. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2455) Unsupported columntypes in model
[ https://issues.apache.org/jira/browse/KYLIN-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878481#comment-15878481 ] Billy Liu commented on KYLIN-2455: -- Thanks Berry for reporting. This could be a part of model validation. > Unsupported columntypes in model > > > Key: KYLIN-2455 > URL: https://issues.apache.org/jira/browse/KYLIN-2455 > Project: Kylin > Issue Type: Bug >Reporter: Berry Österlund >Priority: Trivial > Labels: newbie > > It's possible to create a model and build a cube on it with unsupported > column types. The error will be shown first when you try to access that > table. During my tests, I added a BINARY column to one of the lookup tables. > I got an error when accessing the lookup table, not before. > It should be impossible to add unsupporter column types during model > creation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2384) Refactor and refine Number Dictionary
[ https://issues.apache.org/jira/browse/KYLIN-2384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878490#comment-15878490 ] Billy Liu commented on KYLIN-2384: -- Hi [~xiefan46], so what's the refactor result, how many Number Dictionary left? Does it has compatible issue? > Refactor and refine Number Dictionary > - > > Key: KYLIN-2384 > URL: https://issues.apache.org/jira/browse/KYLIN-2384 > Project: Kylin > Issue Type: Improvement >Reporter: XIE FAN >Assignee: XIE FAN >Priority: Minor > > There are three kings of number dictionary: NumberDictionary.class > NumberDictionary2.class and NumberDictionaryForest.class, which is too > confused -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2374) Allow kylin to store metadata in HDFS instead of HBase
[ https://issues.apache.org/jira/browse/KYLIN-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878492#comment-15878492 ] Billy Liu commented on KYLIN-2374: -- Hi [~xiefan46], how to enable this feature? Any configuration? > Allow kylin to store metadata in HDFS instead of HBase > -- > > Key: KYLIN-2374 > URL: https://issues.apache.org/jira/browse/KYLIN-2374 > Project: Kylin > Issue Type: Improvement >Reporter: XIE FAN >Assignee: XIE FAN > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2325) kylin sql optimization is very slow about 100ms or so, and consumes a lot of cpu
[ https://issues.apache.org/jira/browse/KYLIN-2325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15878494#comment-15878494 ] Billy Liu commented on KYLIN-2325: -- Hi [~elasticlog], are you talking about this https://issues.apache.org/jira/browse/KYLIN-2001 > kylin sql optimization is very slow about 100ms or so, and consumes a lot of > cpu > > > Key: KYLIN-2325 > URL: https://issues.apache.org/jira/browse/KYLIN-2325 > Project: Kylin > Issue Type: Improvement >Reporter: WangTaiZe > Labels: performance > Fix For: v2.0.0 > > Original Estimate: 12h > Remaining Estimate: 12h > > kylin sql optimization is too slow, and consumes too much cpu. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (KYLIN-2102) use ·`bash` in script，not just sh
[ https://issues.apache.org/jira/browse/KYLIN-2102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2102. -- Resolution: Fixed Assignee: Kaige Liu (was: Zhong,Jason) Fixed at KYLIN-2362 > use ·`bash` in script，not just sh > - > > Key: KYLIN-2102 > URL: https://issues.apache.org/jira/browse/KYLIN-2102 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: all >Reporter: chen >Assignee: Kaige Liu > > Sometimes the default shell is not bash.But the scripts ,for example > "get-properties.sh"、“kylin.sh”,use `sh xxx`.It may cause some strange mistack. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2470) cube build failed it there is no hive data partitions from start time to end time
[ https://issues.apache.org/jira/browse/KYLIN-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15884150#comment-15884150 ] Billy Liu commented on KYLIN-2470: -- What's your expectation behavior? > cube build failed it there is no hive data partitions from start time to end > time > -- > > Key: KYLIN-2470 > URL: https://issues.apache.org/jira/browse/KYLIN-2470 > Project: Kylin > Issue Type: Wish > Environment: hadoop2.7.3 hive2.1.1 hbase1.2.4 > apache-kylin-1.6.0-hbase1.x-bin >Reporter: zhou degao > > I hope the build will be successful then there is no hive data partitions. > ERRORS are: > java.lang.IllegalArgumentException: Map input splits are 0 bytes, something > is wrong! > at > org.apache.kylin.engine.mr.common.AbstractHadoopJob.getTotalMapInputMB(AbstractHadoopJob.java:555) > at > org.apache.kylin.engine.mr.steps.CuboidJob.setReduceTaskNum(CuboidJob.java:175) > at org.apache.kylin.engine.mr.steps.CuboidJob.run(CuboidJob.java:138) > at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92) > at > org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > result code:2 > In another environment > (hadoop2.6.4,hive1.2.1,hbase1.1.3,apache-kylin-1.6.0-hbase1.x-bin) the cube > build is successful if there is no hive data partitions from start time to > end time . -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2461) An error when Cube runs the action "build" :java.lang.NoSuchMethodError: org.apache.hadoop.hive.shims.HadoopShims.getUGIForConf(Lorg/apache/hadoop/conf/Configuration;)L
[ https://issues.apache.org/jira/browse/KYLIN-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15880522#comment-15880522 ] Billy Liu commented on KYLIN-2461: -- JIRA is for confirmed bug or feature proposal. Please do not ask question by filing JIRA. Could you forward your question to kylin user mailer? > An error when Cube runs the action "build" :java.lang.NoSuchMethodError: > org.apache.hadoop.hive.shims.HadoopShims.getUGIForConf(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/security/UserGroupInformation > --- > > Key: KYLIN-2461 > URL: https://issues.apache.org/jira/browse/KYLIN-2461 > Project: Kylin > Issue Type: Test >Affects Versions: v1.5.1 > Environment: Hadoop 2.6.0-cdh5.6.0 > Hive 2.0.0 > HBase 1.1.8 > apache-kylin-1.5.1-bin >Reporter: chunyanglu > Labels: build, security > Fix For: 1.x-HBase1.1.3 > > Attachments: 9dd804fc-f924-11e6-8d2f-49ca0548d5ea.png > > > I an login in the web https://bihdp06:7070/kylin with ADMIN/KYLIN > After i select action Build > step 1 "Create Intermediate Flat Hive Table" is working well > but when it comes step 2: Extract Fact Table Distinct Columns ,it throws an > ERROR: > java.lang.NoSuchMethodError: > org.apache.hadoop.hive.shims.HadoopShims.getUGIForConf(Lorg/apache/hadoop/conf/Configuration;)Lorg/apache/hadoop/security/UserGroupInformation; > at > org.apache.hive.hcatalog.common.HiveClientCache$HiveClientCacheKey.(HiveClientCache.java:250) > at > org.apache.hive.hcatalog.common.HiveClientCache$HiveClientCacheKey.fromHiveConf(HiveClientCache.java:256) > at > org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:187) > at org.apache.hive.hcatalog.common.HCatUtil.getHiveClient(HCatUtil.java:569) > at > org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:104) > at > org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86) > at > org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95) > at > org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51) > at > org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:81) > at > org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:111) > at > org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:91) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at > org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:118) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114) > at > org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50) > at > org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:114) > at > org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2417) Compare the performance between HDFSMetaStore and HBaseMetaStore
[ https://issues.apache.org/jira/browse/KYLIN-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879902#comment-15879902 ] Billy Liu commented on KYLIN-2417: -- Thank you for leaving the comments. I suppose the "Fixed" issue always has result. Otherwise, it's an ongoing task. > Compare the performance between HDFSMetaStore and HBaseMetaStore > > > Key: KYLIN-2417 > URL: https://issues.apache.org/jira/browse/KYLIN-2417 > Project: Kylin > Issue Type: Test >Reporter: XIE FAN >Assignee: XIE FAN > Fix For: v2.0.0 > > > We add a new metadata store HDFSMetaStore in KYLIN-2374. The new meta store > has passed the functional tests. But the performance of HDFSMetaStore is > uncertain. So we need to add some performance tests on it to ensure it's > quality. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Issue Comment Deleted] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
[ https://issues.apache.org/jira/browse/KYLIN-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2428: - Comment: was deleted (was: [INFO] Apache Kylin ... SUCCESS [ 3.045 s] [INFO] Apache Kylin - Calcite Overrides ... SUCCESS [ 2.161 s] [INFO] Apache Kylin - Core Common . SUCCESS [ 5.515 s] [INFO] Apache Kylin - Core Metadata ... SUCCESS [ 25.081 s] [INFO] Apache Kylin - Core Dictionary . SUCCESS [01:14 min] [INFO] Apache Kylin - Core Cube ... SUCCESS [01:44 min] [INFO] Apache Kylin - Core Job SUCCESS [01:58 min] [INFO] Apache Kylin - Core Storage SUCCESS [ 34.999 s] [INFO] Apache Kylin - MapReduce Engine SUCCESS [ 20.623 s] [INFO] Apache Kylin - HBase Storage ... SUCCESS [ 12.564 s] [INFO] Apache Kylin - Spark Engine SUCCESS [ 13.664 s] [INFO] Apache Kylin - Hive Source . SUCCESS [ 4.314 s] [INFO] Apache Kylin - Kafka Source SUCCESS [ 3.250 s] [INFO] Apache Kylin - Query ... SUCCESS [ 2.485 s] [INFO] Apache Kylin - Tool SUCCESS [ 4.199 s] [INFO] Apache Kylin - REST Server Base SUCCESS [ 6.802 s] [INFO] Apache Kylin - REST Server . SUCCESS [ 37.196 s] [INFO] Apache Kylin - JDBC Driver . SUCCESS [ 5.646 s] [INFO] Apache Kylin - Assembly SUCCESS [ 5.158 s] [INFO] Apache Kylin - Integration Test SUCCESS [ 01:08 h] [INFO] Apache Kylin - Tomcat Extension SUCCESS [ 2.810 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:16 h [INFO] Finished at: 2017-02-07T15:27:23+00:00 [INFO] Final Memory: 75M/935M) > Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server > > > Key: KYLIN-2428 > URL: https://issues.apache.org/jira/browse/KYLIN-2428 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2428.patch > > > Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and > one web application: server. > Currently, all libraries have shaded some used third party libraries into the > package. For example, guava, curator, commons, kyro in kylin-job. The > duplicate libraries in runtime classpath may have potential class loading > conflicts and waste computing resource. We should leverage the hadoop > provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
[ https://issues.apache.org/jira/browse/KYLIN-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865161#comment-15865161 ] Billy Liu commented on KYLIN-2428: -- [INFO] Apache Kylin ... SUCCESS [ 3.483 s] [INFO] Apache Kylin - Calcite Overrides ... SUCCESS [ 2.295 s] [INFO] Apache Kylin - Core Common . SUCCESS [ 5.559 s] [INFO] Apache Kylin - Core Metadata ... SUCCESS [ 26.885 s] [INFO] Apache Kylin - Core Dictionary . SUCCESS [01:18 min] [INFO] Apache Kylin - Core Cube ... SUCCESS [01:49 min] [INFO] Apache Kylin - Core Job SUCCESS [01:58 min] [INFO] Apache Kylin - Core Storage SUCCESS [ 35.183 s] [INFO] Apache Kylin - MapReduce Engine SUCCESS [ 22.122 s] [INFO] Apache Kylin - HBase Storage ... SUCCESS [ 14.430 s] [INFO] Apache Kylin - Spark Engine SUCCESS [ 9.362 s] [INFO] Apache Kylin - Hive Source . SUCCESS [ 4.243 s] [INFO] Apache Kylin - Kafka Source SUCCESS [ 3.351 s] [INFO] Apache Kylin - Query ... SUCCESS [ 2.604 s] [INFO] Apache Kylin - Tool SUCCESS [ 4.809 s] [INFO] Apache Kylin - REST Server Base SUCCESS [ 7.992 s] [INFO] Apache Kylin - REST Server . SUCCESS [ 39.903 s] [INFO] Apache Kylin - JDBC Driver . SUCCESS [ 4.899 s] [INFO] Apache Kylin - Assembly SUCCESS [ 6.212 s] [INFO] Apache Kylin - Tool Assembly ... SUCCESS [ 2.187 s] [INFO] Apache Kylin - Integration Test SUCCESS [ 01:03 h] [INFO] Apache Kylin - Tomcat Extension SUCCESS [ 0.633 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 01:12 h [INFO] Finished at: 2017-02-14T14:25:15+00:00 [INFO] Final Memory: 76M/856M > Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server > > > Key: KYLIN-2428 > URL: https://issues.apache.org/jira/browse/KYLIN-2428 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2428.patch > > > Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and > one web application: server. > Currently, all libraries have shaded some used third party libraries into the > package. For example, guava, curator, commons, kyro in kylin-job. The > duplicate libraries in runtime classpath may have potential class loading > conflicts and waste computing resource. We should leverage the hadoop > provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
[ https://issues.apache.org/jira/browse/KYLIN-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2428: - Attachment: (was: KYLIN-2428.patch) > Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server > > > Key: KYLIN-2428 > URL: https://issues.apache.org/jira/browse/KYLIN-2428 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2428.patch > > > Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and > one web application: server. > Currently, all libraries have shaded some used third party libraries into the > package. For example, guava, curator, commons, kyro in kylin-job. The > duplicate libraries in runtime classpath may have potential class loading > conflicts and waste computing resource. We should leverage the hadoop > provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
[ https://issues.apache.org/jira/browse/KYLIN-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2428: - Attachment: KYLIN-2428.patch > Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server > > > Key: KYLIN-2428 > URL: https://issues.apache.org/jira/browse/KYLIN-2428 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2428.patch > > > Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and > one web application: server. > Currently, all libraries have shaded some used third party libraries into the > package. For example, guava, curator, commons, kyro in kylin-job. The > duplicate libraries in runtime classpath may have potential class loading > conflicts and waste computing resource. We should leverage the hadoop > provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2428) Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server
[ https://issues.apache.org/jira/browse/KYLIN-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15865163#comment-15865163 ] Billy Liu commented on KYLIN-2428: -- Hi [~Shaofengshi], the patch has rebased with the latest master with support on Spark IT. It passed IT test. Could you help review this patch? or check the code on the dev branch https://github.com/apache/kylin/commit/f10d9d9ba95878a97f70975547763083c000bcfe > Cleanup unnecessary shaded libraries for job/coprocessor/jdbc/server > > > Key: KYLIN-2428 > URL: https://issues.apache.org/jira/browse/KYLIN-2428 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2428.patch > > > Kylin releases three libraries: kylin-coprocessor, kylin-jdbc, kylin-job and > one web application: server. > Currently, all libraries have shaded some used third party libraries into the > package. For example, guava, curator, commons, kyro in kylin-job. The > duplicate libraries in runtime classpath may have potential class loading > conflicts and waste computing resource. We should leverage the hadoop > provided libraries at runtime instead of the shaded one. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2439) add table use rest api
[
https://issues.apache.org/jira/browse/KYLIN-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859226#comment-15859226
]
Billy Liu commented on KYLIN-2439:
--
http://kylin.apache.org/docs16/howto/howto_use_restapi.html#load-hive-tables
> add table use rest api
> --
>
> Key: KYLIN-2439
> URL: https://issues.apache.org/jira/browse/KYLIN-2439
> Project: Kylin
> Issue Type: Bug
> Components: REST Service
>Affects Versions: v1.5.1
>Reporter: xufan
>Assignee: Zhong,Jason
> Fix For: v1.5.1
>
>
> when i add table use rest api /tables/addStreamingSrc, an exception found in
> server:
> java.lang.IllegalStateException: Cannot find table 'null' in metadata manager
> at
> org.apache.kylin.metadata.project.ProjectManager.addTableDescToProject(ProjectManager.java:284)
> at
> org.apache.kylin.rest.service.CubeService.syncTableToProject(CubeService.java:580)
> i think this code in TableController::addStreamingTable maybe wrong
> cubeMgmtService.syncTableToProject(new String[]{desc.getName()}, project);
> it should be :
> cubeMgmtService.syncTableToProject(new String[]{desc.getIdentity()},
> project);
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KYLIN-2439) add table use rest api
[
https://issues.apache.org/jira/browse/KYLIN-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859245#comment-15859245
]
Billy Liu commented on KYLIN-2439:
--
The better is to define new Hive view with right column type, and reload into
Kylin.
> add table use rest api
> --
>
> Key: KYLIN-2439
> URL: https://issues.apache.org/jira/browse/KYLIN-2439
> Project: Kylin
> Issue Type: Bug
> Components: REST Service
>Affects Versions: v1.5.1
>Reporter: xufan
>Assignee: Zhong,Jason
> Fix For: v1.5.1
>
>
> when i add table use rest api /tables/addStreamingSrc, an exception found in
> server:
> java.lang.IllegalStateException: Cannot find table 'null' in metadata manager
> at
> org.apache.kylin.metadata.project.ProjectManager.addTableDescToProject(ProjectManager.java:284)
> at
> org.apache.kylin.rest.service.CubeService.syncTableToProject(CubeService.java:580)
> i think this code in TableController::addStreamingTable maybe wrong
> cubeMgmtService.syncTableToProject(new String[]{desc.getName()}, project);
> it should be :
> cubeMgmtService.syncTableToProject(new String[]{desc.getIdentity()},
> project);
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KYLIN-2439) add table use rest api
[
https://issues.apache.org/jira/browse/KYLIN-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859340#comment-15859340
]
Billy Liu commented on KYLIN-2439:
--
Could you identify the code line number？Any patch is welcomed.
> add table use rest api
> --
>
> Key: KYLIN-2439
> URL: https://issues.apache.org/jira/browse/KYLIN-2439
> Project: Kylin
> Issue Type: Bug
> Components: REST Service
>Affects Versions: v1.5.1
>Reporter: xufan
>Assignee: Zhong,Jason
> Fix For: v1.5.1
>
>
> when i add table use rest api /tables/addStreamingSrc, an exception found in
> server:
> java.lang.IllegalStateException: Cannot find table 'null' in metadata manager
> at
> org.apache.kylin.metadata.project.ProjectManager.addTableDescToProject(ProjectManager.java:284)
> at
> org.apache.kylin.rest.service.CubeService.syncTableToProject(CubeService.java:580)
> i think this code in TableController::addStreamingTable maybe wrong
> cubeMgmtService.syncTableToProject(new String[]{desc.getName()}, project);
> it should be :
> cubeMgmtService.syncTableToProject(new String[]{desc.getIdentity()},
> project);
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KYLIN-2439) add table use rest api
[
https://issues.apache.org/jira/browse/KYLIN-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859425#comment-15859425
]
Billy Liu commented on KYLIN-2439:
--
The issue has been fixed at https://issues.apache.org/jira/browse/KYLIN-1660 by
commit:
https://github.com/apache/kylin/commit/3542dd29147ed20a409c84212235dc6c3c5f0ad9
> add table use rest api
> --
>
> Key: KYLIN-2439
> URL: https://issues.apache.org/jira/browse/KYLIN-2439
> Project: Kylin
> Issue Type: Bug
> Components: REST Service
>Affects Versions: v1.5.1
>Reporter: xufan
>Assignee: Zhong,Jason
> Fix For: v1.5.1
>
>
> when i add table use rest api /tables/addStreamingSrc, an exception found in
> server:
> java.lang.IllegalStateException: Cannot find table 'null' in metadata manager
> at
> org.apache.kylin.metadata.project.ProjectManager.addTableDescToProject(ProjectManager.java:284)
> at
> org.apache.kylin.rest.service.CubeService.syncTableToProject(CubeService.java:580)
> i think this code in TableController::addStreamingTable maybe wrong
> cubeMgmtService.syncTableToProject(new String[]{desc.getName()}, project);
> it should be :
> cubeMgmtService.syncTableToProject(new String[]{desc.getIdentity()},
> project);
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (KYLIN-2450) sample.sh should generate sample data on kylin.source.hive.database-for-flat-table, not hard code DEFAULT
[ https://issues.apache.org/jira/browse/KYLIN-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2450. -- Resolution: Fixed https://github.com/apache/kylin/commit/a60f893ff0c6d8f1e34ef9bbdd3cab7cc875aa99 The new script will respect kylin.source.hive.database-for-flat-table, the related sample cube is also updated. > sample.sh should generate sample data on > kylin.source.hive.database-for-flat-table, not hard code DEFAULT > - > > Key: KYLIN-2450 > URL: https://issues.apache.org/jira/browse/KYLIN-2450 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > > Currently, the sample.sh will generate five sample tables into Hive DEFAULT > database, and upload the sample cube, with hard-coded schema to DEFAULT also. > But in some user cases, the DEFAULT database is not allowed to write. Kylin > has property kylin.source.hive.database-for-flat-table, sample.sh should > respect this variable and generate the samples into customized database. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (KYLIN-2439) add table use rest api
[
https://issues.apache.org/jira/browse/KYLIN-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15859098#comment-15859098
]
Billy Liu commented on KYLIN-2439:
--
The addStreamingSrc API is no longer available. For creating streaming source,
please checkout POST to /streaming/
> add table use rest api
> --
>
> Key: KYLIN-2439
> URL: https://issues.apache.org/jira/browse/KYLIN-2439
> Project: Kylin
> Issue Type: Bug
> Components: REST Service
>Affects Versions: v1.5.1
>Reporter: xufan
>Assignee: Zhong,Jason
> Fix For: v1.5.1
>
>
> when i add table use rest api /tables/addStreamingSrc, an exception found in
> server:
> java.lang.IllegalStateException: Cannot find table 'null' in metadata manager
> at
> org.apache.kylin.metadata.project.ProjectManager.addTableDescToProject(ProjectManager.java:284)
> at
> org.apache.kylin.rest.service.CubeService.syncTableToProject(CubeService.java:580)
> i think this code in TableController::addStreamingTable maybe wrong
> cubeMgmtService.syncTableToProject(new String[]{desc.getName()}, project);
> it should be :
> cubeMgmtService.syncTableToProject(new String[]{desc.getIdentity()},
> project);
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (KYLIN-2182) Aggregation Group Joint Dimension generation default rule update
[ https://issues.apache.org/jira/browse/KYLIN-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15725634#comment-15725634 ] Billy Liu commented on KYLIN-2182: -- Current rule is very simple, if too many dimensions defined(more than 10), then the last few dimensions will be added as joint dimensions, that's why we say it not meaningful. Base on the dimension cardinality is better. Could you continue this improvement? > Aggregation Group Joint Dimension generation default rule update > > > Key: KYLIN-2182 > URL: https://issues.apache.org/jira/browse/KYLIN-2182 > Project: Kylin > Issue Type: Improvement >Affects Versions: v1.5.4.1 >Reporter: Zhong,Jason > Fix For: v1.6.1 > > > when we create cube, if we have multiple dimensions, we will split dimensions > into different Joint, this is not so meaningful -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2251) JDBC Driver httpcore dependency conflict
[ https://issues.apache.org/jira/browse/KYLIN-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15737820#comment-15737820 ] Billy Liu commented on KYLIN-2251: -- I know it's not a common fix by depending on the maven loading order. I did it because I don't think here was a real conflict case. No exception, no wrong result, all IT passed. So I did a very lightweight hack for this update. > JDBC Driver httpcore dependency conflict > > > Key: KYLIN-2251 > URL: https://issues.apache.org/jira/browse/KYLIN-2251 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v1.6.1 > > > Report by xwhfcenter from github: > "There is a conflict in dependency of httpcore in module JDBC Driver" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2265) Adjust default config for resource-limit sandbox
Billy Liu created KYLIN-2265: Summary: Adjust default config for resource-limit sandbox Key: KYLIN-2265 URL: https://issues.apache.org/jira/browse/KYLIN-2265 Project: Kylin Issue Type: Improvement Components: General Affects Versions: v1.6.0 Reporter: Billy Liu Assignee: Billy Liu Most users deploy Kylin on their own sandbox for the first trial. Most sandbox has most 8G memory. The most used sandboxes are HDP sandbox and CDH sandbox. We'd better make the default kylin configuration convenient for these sandbox environment. The suggestion includes reducing region-cut-gb, hfile-size-gb, max-region-count, reduce-input-mb, max-reducer-number,mapreduce.map.memory.mb,mapreduce.map.java.opts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KYLIN-2264) Date error when use new streaming cube in Kylin1.6.0
[ https://issues.apache.org/jira/browse/KYLIN-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu reassigned KYLIN-2264: Assignee: Shaofeng SHI (was: Zhong,Jason) > Date error when use new streaming cube in Kylin1.6.0 > > > Key: KYLIN-2264 > URL: https://issues.apache.org/jira/browse/KYLIN-2264 > Project: Kylin > Issue Type: Bug > Components: streaming, Web >Affects Versions: v1.6.0 > Environment: Debian 3.2.54-2 x86_64 GNU/Linux >Reporter: WangSheng >Assignee: Shaofeng SHI > > I installed Kylin1.6.0 and built streaming cube successgfully.But I found two > problems which I didn't met in Kylin1.5.*. > First, segments' start/end time displayed on Kylin Web are earlier 8 hours > than my PC date, but streaming cube's Last Build Time and Create Time > displayed on Kylin Web are same with my PC date. Maybe something wrong when > Kylin Web transform the segments' start/end timestamp into date, but I'm not > sure. > Second, I did sql query from streaming cube, but the records' time related > columns like "HOUR_START" and "MINUTE_START" are all earlier 8 hours than my > PC time. I found that these time related columns' timestamp from HBase are > correct by remote debug, so I guess something wrong when Kylin server > transform these timestamp into date. > By the way, I only changed the "kylin.rest.timezone=GM+8" in file > "kylin.properties", and my PC date is same with my server date. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (KYLIN-2261) Cleanup Hbase Storage issue
[ https://issues.apache.org/jira/browse/KYLIN-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734223#comment-15734223 ] Billy Liu edited comment on KYLIN-2261 at 12/9/16 5:57 AM: --- For Kylin 1.6, it's in the tool/kylin-tool-1.6.0.jar. was (Author: yimingliu): For Kylin 1.6, it's in the tool/kylin-tool-1.6.0.jar. Could you try to cp this library into lib. > Cleanup Hbase Storage issue > --- > > Key: KYLIN-2261 > URL: https://issues.apache.org/jira/browse/KYLIN-2261 > Project: Kylin > Issue Type: Test > Components: Client - CLI >Affects Versions: v1.6.0 > Environment: CDH-5.7.2 > Hbase1.2 > Kylin1.6 >Reporter: QiLiFei >Priority: Critical > Labels: test > Original Estimate: 2h > Remaining Estimate: 2h > > When I try to run below command according doc > (http://kylin.apache.org/docs16/howto/howto_cleanup_storage.html), it will > always raise error "Error: Could not find or load main class > org.apache.kylin.tool.StorageCleanupJob" > Command : > /opt/kylin/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete > false > > Is the class in the 'kylin-storage-hbase.jar ' ? > And it should be put into $KYLIN_HOME/lib/ , Right ? > I've put jar file in $KYLIN_HOME/lib & $HBase_Home/lib/ and set the 777 > authority . However it's still cannot working!! > If I'm wrong , please correct me ! Thanks. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2263) Display reasonable exception message if could not find kafka dependency for streaming build
Billy Liu created KYLIN-2263: Summary: Display reasonable exception message if could not find kafka dependency for streaming build Key: KYLIN-2263 URL: https://issues.apache.org/jira/browse/KYLIN-2263 Project: Kylin Issue Type: Improvement Components: Web Affects Versions: v1.6.0 Reporter: Billy Liu Assignee: Billy Liu Priority: Minor Kafka is optional dependency for Kylin install. But is mandatory for streaming build. Currently, if no KAFKA_HOME exported, the build will show "Error", but without any more detail message. It's not convenient for new user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2261) Cleanup Hbase Storage issue
[ https://issues.apache.org/jira/browse/KYLIN-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734443#comment-15734443 ] Billy Liu commented on KYLIN-2261: -- Maybe something wrong in your HBASE_CLASSPATH, which is used to loading Kylin dependencies. I could not reproduce your issue. > Cleanup Hbase Storage issue > --- > > Key: KYLIN-2261 > URL: https://issues.apache.org/jira/browse/KYLIN-2261 > Project: Kylin > Issue Type: Test > Components: Client - CLI >Affects Versions: v1.6.0 > Environment: CDH-5.7.2 > Hbase1.2 > Kylin1.6 >Reporter: QiLiFei >Priority: Critical > Labels: test > Original Estimate: 2h > Remaining Estimate: 2h > > When I try to run below command according doc > (http://kylin.apache.org/docs16/howto/howto_cleanup_storage.html), it will > always raise error "Error: Could not find or load main class > org.apache.kylin.tool.StorageCleanupJob" > Command : > /opt/kylin/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete > false > > Is the class in the 'kylin-storage-hbase.jar ' ? > And it should be put into $KYLIN_HOME/lib/ , Right ? > I've put jar file in $KYLIN_HOME/lib & $HBase_Home/lib/ and set the 777 > authority . However it's still cannot working!! > If I'm wrong , please correct me ! Thanks. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2265) Adjust default config for resource-limit sandbox
[ https://issues.apache.org/jira/browse/KYLIN-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2265: - Attachment: KYLIN-2265.patch Hi [~Shaofengshi], Could you help review this update patch? In this update, reduce region-cut-gb to 1 reduce hflie-size-gb to 1 reduce max-region-count to 100 reduce reduce-input-mb to 100 reduce max-reducer-number to 100 reduce mapper-input-rows to 20 reduce mapreduce.map.memory.mb to 1024 reduce mapreduce.map.java.opts to -Xmx700m reduce mapreduce.task.io.sort.mb to 100 > Adjust default config for resource-limit sandbox > > > Key: KYLIN-2265 > URL: https://issues.apache.org/jira/browse/KYLIN-2265 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2265.patch > > > Most users deploy Kylin on their own sandbox for the first trial. Most > sandbox has most 8G memory. The most used sandboxes are HDP sandbox and CDH > sandbox. We'd better make the default kylin configuration convenient for > these sandbox environment. The suggestion includes reducing region-cut-gb, > hfile-size-gb, max-region-count, reduce-input-mb, > max-reducer-number,mapreduce.map.memory.mb,mapreduce.map.java.opts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2263) Display reasonable exception message if could not find kafka dependency for streaming build
[ https://issues.apache.org/jira/browse/KYLIN-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2263. -- Resolution: Fixed Fix Version/s: v1.6.1 will show "Could not find Kafka dependency" if that happends > Display reasonable exception message if could not find kafka dependency for > streaming build > > > Key: KYLIN-2263 > URL: https://issues.apache.org/jira/browse/KYLIN-2263 > Project: Kylin > Issue Type: Improvement > Components: Web >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v1.6.1 > > > Kafka is optional dependency for Kylin install. But is mandatory for > streaming build. Currently, if no KAFKA_HOME exported, the build will show > "Error", but without any more detail message. It's not convenient for new > user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2261) Cleanup Hbase Storage issue
[
https://issues.apache.org/jira/browse/KYLIN-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734594#comment-15734594
]
Billy Liu commented on KYLIN-2261:
--
In kylin.sh retrieveDependency method, you will find how the dependency works.
The dependency is defined as HBASE_CLASSPATH_PREFIX, it will be loaded by HBase.
In your shell, try run:
`export
HBASE_CLASSPATH_PREFIX=${KYLIN_HOME}/conf:${KYLIN_HOME}/lib/*:${KYLIN_HOME}/tool/*:${KYLIN_HOME}/ext/*:${HBASE_CLASSPATH_PREFIX}
hbase classpath`
My HDP sandbox will show
"/opt/apache-kylin-1.6.1-SNAPSHOT-bin/conf:/opt/apache-kylin-1.6.1-SNAPSHOT-bin/lib/*:/opt/apache-kylin-1.6.1-SNAPSHOT-bin/tool/*:/opt/apache-kylin-1.6.1-SNAPSHOT-bin/ext/*::/usr/hdp/2.2.4.2-2/hbase/conf"
You will notice the kylin libraries in the classpath.
> Cleanup Hbase Storage issue
> ---
>
> Key: KYLIN-2261
> URL: https://issues.apache.org/jira/browse/KYLIN-2261
> Project: Kylin
> Issue Type: Test
> Components: Client - CLI
>Affects Versions: v1.6.0
> Environment: CDH-5.7.2
> Hbase1.2
> Kylin1.6
>Reporter: QiLiFei
>Priority: Critical
> Labels: test
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> When I try to run below command according doc
> (http://kylin.apache.org/docs16/howto/howto_cleanup_storage.html), it will
> always raise error "Error: Could not find or load main class
> org.apache.kylin.tool.StorageCleanupJob"
> Command :
> /opt/kylin/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete
> false
>
> Is the class in the 'kylin-storage-hbase.jar ' ?
> And it should be put into $KYLIN_HOME/lib/ , Right ?
> I've put jar file in $KYLIN_HOME/lib & $HBase_Home/lib/ and set the 777
> authority . However it's still cannot working!!
> If I'm wrong , please correct me ! Thanks.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2261) Cleanup Hbase Storage issue
[ https://issues.apache.org/jira/browse/KYLIN-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735639#comment-15735639 ] Billy Liu commented on KYLIN-2261: -- Thanks for pointing out. You are correct. The StorageCleanupJob is still under hbase component for 1.6.0. https://issues.apache.org/jira/browse/KYLIN-2144 did the refactor, but will be effective in next release 1.6.1. But the document was updated too early. > Cleanup Hbase Storage issue > --- > > Key: KYLIN-2261 > URL: https://issues.apache.org/jira/browse/KYLIN-2261 > Project: Kylin > Issue Type: Test > Components: Client - CLI >Affects Versions: v1.6.0 > Environment: CDH-5.7.2 > Hbase1.2 > Kylin1.6 >Reporter: QiLiFei >Priority: Critical > Labels: test > Original Estimate: 2h > Remaining Estimate: 2h > > When I try to run below command according doc > (http://kylin.apache.org/docs16/howto/howto_cleanup_storage.html), it will > always raise error "Error: Could not find or load main class > org.apache.kylin.tool.StorageCleanupJob" > Command : > /opt/kylin/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete > false > > Is the class in the 'kylin-storage-hbase.jar ' ? > And it should be put into $KYLIN_HOME/lib/ , Right ? > I've put jar file in $KYLIN_HOME/lib & $HBase_Home/lib/ and set the 777 > authority . However it's still cannot working!! > If I'm wrong , please correct me ! Thanks. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2261) Cleanup Hbase Storage issue
[ https://issues.apache.org/jira/browse/KYLIN-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735631#comment-15735631 ] Billy Liu commented on KYLIN-2261: -- QiLiFei commented on KYLIN-2261: I downloaded the kylin source code and find that the org.apache.kylin.tool.StorageCleanupJob is not exist. It should be the one org.apache.kylin.storage.hbase.util.StorageCleanupJob . And finally the below command can work correctly in the CLI. I considered there will be a P2 bug assigned to the doc (http://kylin.apache.org/docs16/howto/howto_cleanup_storage.html)!!! Correct : ./kylin.sh org.apache.kylin.storage.hbase.util.StorageCleanupJob --delete false Invalid : ./kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete false > Cleanup Hbase Storage issue > --- > > Key: KYLIN-2261 > URL: https://issues.apache.org/jira/browse/KYLIN-2261 > Project: Kylin > Issue Type: Test > Components: Client - CLI >Affects Versions: v1.6.0 > Environment: CDH-5.7.2 > Hbase1.2 > Kylin1.6 >Reporter: QiLiFei >Priority: Critical > Labels: test > Original Estimate: 2h > Remaining Estimate: 2h > > When I try to run below command according doc > (http://kylin.apache.org/docs16/howto/howto_cleanup_storage.html), it will > always raise error "Error: Could not find or load main class > org.apache.kylin.tool.StorageCleanupJob" > Command : > /opt/kylin/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete > false > > Is the class in the 'kylin-storage-hbase.jar ' ? > And it should be put into $KYLIN_HOME/lib/ , Right ? > I've put jar file in $KYLIN_HOME/lib & $HBase_Home/lib/ and set the 777 > authority . However it's still cannot working!! > If I'm wrong , please correct me ! Thanks. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2265) Adjust default config for resource-limit sandbox
[ https://issues.apache.org/jira/browse/KYLIN-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735613#comment-15735613 ] Billy Liu commented on KYLIN-2265: -- I'd like to prepare a document introducing how to config Kylin, including sandbox and production environment. Some of them are already there in http://kylin.apache.org/docs16/install/advance_settings.html. But most of the config items are still not explained well. > Adjust default config for resource-limit sandbox > > > Key: KYLIN-2265 > URL: https://issues.apache.org/jira/browse/KYLIN-2265 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2265.patch > > > Most users deploy Kylin on their own sandbox for the first trial. Most > sandbox has most 8G memory. The most used sandboxes are HDP sandbox and CDH > sandbox. We'd better make the default kylin configuration convenient for > these sandbox environment. The suggestion includes reducing region-cut-gb, > hfile-size-gb, max-region-count, reduce-input-mb, > max-reducer-number,mapreduce.map.memory.mb,mapreduce.map.java.opts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2274) Dundas ODBC Error: SQLColAttribute unknown attr,
[
https://issues.apache.org/jira/browse/KYLIN-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15743713#comment-15743713
]
Billy Liu commented on KYLIN-2274:
--
What's the result if you run the second query in Kylin insight directly?
> Dundas ODBC Error: SQLColAttribute unknown attr,
>
>
> Key: KYLIN-2274
> URL: https://issues.apache.org/jira/browse/KYLIN-2274
> Project: Kylin
> Issue Type: Bug
> Components: Driver - ODBC
>Affects Versions: v1.6.0
> Environment: Kylin 1.6.0
> ODBC 1.6 64 bits
> Dundas BI: 3.0.2 Windows7 64 bits
>Reporter: albertoramon
> Attachments: dundas.png, odbc-2016-12-06-19-01-51.log
>
>
> I tested [Dundas Software|http://www.dundas.com/dundas-bi]
> (I used Kylin Sample Cube)
> You can read list of tables :)
> !dundas.png!
> But when you try to load data of these, there is two types of errors:
> *Error 1:* (This query works OK on Kylin UI)
> {quote}
> [INFO ] [2016-12-06.19:05:30]SELECT * FROM "DEFAULT"."KYLIN_CAL_DT" WHERE 1 =
> 2
> [INFO ] [2016-12-06.19:05:30]Successfully done executing the query
> [FATAL] [2016-12-06.19:05:30]SQLColAttribute unknown attr, ColNum: 1, FldID:
> 1211
> {quote}
> *Error 2:*
> {quote}
> [ERROR][2016-12-06.19:30:27]The REST query request failed, the error message
> is: Error while executing SQL "SELECT T."USER_DEFINED_FIELD1" AS "TE0",
> T."SITE_ID" AS "TE1" FROM "DEFAULT"."KYLIN_CAL_DT" T GROUP BY
> T."USER_DEFINED_FIELD1", T."SITE_ID" ORDER BY T."USER_DEFINED_FIELD1",
> T."SITE_ID"": From line 1, column 106 to line 1, column 126: Column
> 'USER_DEFINED_FIELD1' not found in table 'T'
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KYLIN-2265) Adjust default config for resource-limit sandbox
[ https://issues.apache.org/jira/browse/KYLIN-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15744357#comment-15744357 ] Billy Liu commented on KYLIN-2265: -- I see. Most user use Hadoop cluster instead of sandbox. No config change for this JIRA, I will update the doc later. > Adjust default config for resource-limit sandbox > > > Key: KYLIN-2265 > URL: https://issues.apache.org/jira/browse/KYLIN-2265 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2265.patch > > > Most users deploy Kylin on their own sandbox for the first trial. Most > sandbox has most 8G memory. The most used sandboxes are HDP sandbox and CDH > sandbox. We'd better make the default kylin configuration convenient for > these sandbox environment. The suggestion includes reducing region-cut-gb, > hfile-size-gb, max-region-count, reduce-input-mb, > max-reducer-number,mapreduce.map.memory.mb,mapreduce.map.java.opts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2131) Load Kafka client configuration from properties files
[ https://issues.apache.org/jira/browse/KYLIN-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2131: - Attachment: KYLIN-2131.patch Better refactor done. > Load Kafka client configuration from properties files > - > > Key: KYLIN-2131 > URL: https://issues.apache.org/jira/browse/KYLIN-2131 > Project: Kylin > Issue Type: Improvement > Components: streaming >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Attachments: KYLIN-2131.patch > > > The latest kafka source hardcode the connection properties(such as > "timeout.ms" in source files, it could be better if could refactor these > properties into files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2131) Load Kafka client configuration from properties files
[ https://issues.apache.org/jira/browse/KYLIN-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2131: - Attachment: (was: KYLIN-2131.patch) > Load Kafka client configuration from properties files > - > > Key: KYLIN-2131 > URL: https://issues.apache.org/jira/browse/KYLIN-2131 > Project: Kylin > Issue Type: Improvement > Components: streaming >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > > The latest kafka source hardcode the connection properties(such as > "timeout.ms" in source files, it could be better if could refactor these > properties into files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-2285) ffdfds
[ https://issues.apache.org/jira/browse/KYLIN-2285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu closed KYLIN-2285. Resolution: Invalid > ffdfds > -- > > Key: KYLIN-2285 > URL: https://issues.apache.org/jira/browse/KYLIN-2285 > Project: Kylin > Issue Type: Bug > Components: Web >Reporter: paldheeru39 >Assignee: Zhong,Jason > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2252) Enhance project/model/cube name check
Billy Liu created KYLIN-2252: Summary: Enhance project/model/cube name check Key: KYLIN-2252 URL: https://issues.apache.org/jira/browse/KYLIN-2252 Project: Kylin Issue Type: Improvement Components: REST Service Affects Versions: v1.6.0 Reporter: Billy Liu Assignee: Billy Liu Priority: Minor Enhance the project/model/cube name check in Controller side, only letters, numbers and underline are valid. Others will throw out BadRequestException. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2252) Enhance project/model/cube name check
[ https://issues.apache.org/jira/browse/KYLIN-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2252. -- Resolution: Fixed Fix Version/s: v1.6.1 commit id: 1e787165edee713a59db8627e70f79edabfd7d9d > Enhance project/model/cube name check > - > > Key: KYLIN-2252 > URL: https://issues.apache.org/jira/browse/KYLIN-2252 > Project: Kylin > Issue Type: Improvement > Components: REST Service >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v1.6.1 > > > Enhance the project/model/cube name check in Controller side, only letters, > numbers and underline are valid. Others will throw out BadRequestException. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2251) JDBC Driver httpcore dependency conflict
[ https://issues.apache.org/jira/browse/KYLIN-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2251. -- Resolution: Fixed Fix Version/s: v1.6.1 > JDBC Driver httpcore dependency conflict > > > Key: KYLIN-2251 > URL: https://issues.apache.org/jira/browse/KYLIN-2251 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v1.6.1 > > > Report by xwhfcenter from github: > "There is a conflict in dependency of httpcore in module JDBC Driver" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2251) JDBC Driver httpcore dependency conflict
[ https://issues.apache.org/jira/browse/KYLIN-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727269#comment-15727269 ] Billy Liu commented on KYLIN-2251: -- Will keep the httpcore same version with httpclient, which is lower than the dependency of avatica. > JDBC Driver httpcore dependency conflict > > > Key: KYLIN-2251 > URL: https://issues.apache.org/jira/browse/KYLIN-2251 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > > Report by xwhfcenter from github: > "There is a conflict in dependency of httpcore in module JDBC Driver" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2251) JDBC Driver httpcore dependency conflict
Billy Liu created KYLIN-2251: Summary: JDBC Driver httpcore dependency conflict Key: KYLIN-2251 URL: https://issues.apache.org/jira/browse/KYLIN-2251 Project: Kylin Issue Type: Bug Components: Driver - JDBC Affects Versions: v1.6.0 Reporter: Billy Liu Assignee: Billy Liu Priority: Minor Report by xwhfcenter from github: "There is a conflict in dependency of httpcore in module JDBC Driver" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2251) JDBC Driver httpcore dependency conflict
[ https://issues.apache.org/jira/browse/KYLIN-2251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15727275#comment-15727275 ] Billy Liu commented on KYLIN-2251: -- commit: https://github.com/apache/kylin/commit/ea13af4d8aa6a5b25fc328ee1adaf2b454eddea0 > JDBC Driver httpcore dependency conflict > > > Key: KYLIN-2251 > URL: https://issues.apache.org/jira/browse/KYLIN-2251 > Project: Kylin > Issue Type: Bug > Components: Driver - JDBC >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > > Report by xwhfcenter from github: > "There is a conflict in dependency of httpcore in module JDBC Driver" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2293) Refactor KylinConfig to separate test case into KylinTestConfig
Billy Liu created KYLIN-2293: Summary: Refactor KylinConfig to separate test case into KylinTestConfig Key: KYLIN-2293 URL: https://issues.apache.org/jira/browse/KYLIN-2293 Project: Kylin Issue Type: Task Components: Metadata Affects Versions: v1.6.0 Reporter: Billy Liu Assignee: Billy Liu Priority: Minor Currently, here are some "For test" method exposed in KylinConfig, the refactor would move all these "For test" methods into another Test related class, to reduce the unnecessary APIs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2293) Refactor KylinConfig to separate test case into KylinTestConfig
[ https://issues.apache.org/jira/browse/KYLIN-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2293: - Attachment: KYLIN-2293.patch Has moved all test hack code into KylinTestConfig > Refactor KylinConfig to separate test case into KylinTestConfig > > > Key: KYLIN-2293 > URL: https://issues.apache.org/jira/browse/KYLIN-2293 > Project: Kylin > Issue Type: Task > Components: Metadata >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Attachments: KYLIN-2293.patch > > > Currently, here are some "For test" method exposed in KylinConfig, the > refactor would move all these "For test" methods into another Test related > class, to reduce the unnecessary APIs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2131) Load Kafka client configuration from properties files
[ https://issues.apache.org/jira/browse/KYLIN-2131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2131: - Attachment: KYLIN-2131.patch > Load Kafka client configuration from properties files > - > > Key: KYLIN-2131 > URL: https://issues.apache.org/jira/browse/KYLIN-2131 > Project: Kylin > Issue Type: Improvement > Components: streaming >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Attachments: KYLIN-2131.patch > > > The latest kafka source hardcode the connection properties(such as > "timeout.ms" in source files, it could be better if could refactor these > properties into files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2031) New Fix_length_Hex encoding to support hash value and better Integer encoding to support negative value
[ https://issues.apache.org/jira/browse/KYLIN-2031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2031: - Summary: New Fix_length_Hex encoding to support hash value and better Integer encoding to support negative value (was: some more DimensionEncoding) > New Fix_length_Hex encoding to support hash value and better Integer encoding > to support negative value > --- > > Key: KYLIN-2031 > URL: https://issues.apache.org/jira/browse/KYLIN-2031 > Project: Kylin > Issue Type: New Feature >Reporter: hongbin ma >Assignee: hongbin ma > Fix For: v1.6.1 > > > 1. for some use cases string value represent hash code is used. The string > only consist of [0~9A~F] (hex values), so two characters can be squashed into > one byte > 2. The current IntegerDimEnc does not support negative values, need another > IntegerDimEnc that supports negative values -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2265) Adjust default config for resource-limit sandbox
[ https://issues.apache.org/jira/browse/KYLIN-2265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15739885#comment-15739885 ] Billy Liu commented on KYLIN-2265: -- Hi Shaofeng, do you have more suggestions on what the default would be. I try to propose the sandbox(10G memory, 2 core) as default, because it has the least limitations to the hadoop environment. The default configuration should build sample cube smoothly. For production deployment, the administrator should reconsider the cluster size, and source data size carefully. The configuration document is mandatory for reference. > Adjust default config for resource-limit sandbox > > > Key: KYLIN-2265 > URL: https://issues.apache.org/jira/browse/KYLIN-2265 > Project: Kylin > Issue Type: Improvement > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2265.patch > > > Most users deploy Kylin on their own sandbox for the first trial. Most > sandbox has most 8G memory. The most used sandboxes are HDP sandbox and CDH > sandbox. We'd better make the default kylin configuration convenient for > these sandbox environment. The suggestion includes reducing region-cut-gb, > hfile-size-gb, max-region-count, reduce-input-mb, > max-reducer-number,mapreduce.map.memory.mb,mapreduce.map.java.opts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2021) Cognos Issues
[ https://issues.apache.org/jira/browse/KYLIN-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15739838#comment-15739838 ] Billy Liu commented on KYLIN-2021: -- Hi [~mahongbin], could you explain more what the issues are? Especially the issue topic name. Then we could send out more clear message to Cognos community how Kylin could support, and what not support yet. > Cognos Issues > - > > Key: KYLIN-2021 > URL: https://issues.apache.org/jira/browse/KYLIN-2021 > Project: Kylin > Issue Type: Improvement >Reporter: hongbin ma >Assignee: hongbin ma > Fix For: v1.6.0 > > > cognos will generate some queries that kylin does not support yet -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2261) Cleanup Hbase Storage issue
[ https://issues.apache.org/jira/browse/KYLIN-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734223#comment-15734223 ] Billy Liu commented on KYLIN-2261: -- For Kylin 1.6, it's in the tool/kylin-tool-1.6.0.jar. Could you try to cp this library into lib. > Cleanup Hbase Storage issue > --- > > Key: KYLIN-2261 > URL: https://issues.apache.org/jira/browse/KYLIN-2261 > Project: Kylin > Issue Type: Test > Components: Client - CLI >Affects Versions: v1.6.0 > Environment: CDH-5.7.2 > Hbase1.2 > Kylin1.6 >Reporter: QiLiFei >Priority: Critical > Labels: test > Original Estimate: 2h > Remaining Estimate: 2h > > When I try to run below command according doc > (http://kylin.apache.org/docs16/howto/howto_cleanup_storage.html), it will > always raise error "Error: Could not find or load main class > org.apache.kylin.tool.StorageCleanupJob" > Command : > /opt/kylin/bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete > false > > Is the class in the 'kylin-storage-hbase.jar ' ? > And it should be put into $KYLIN_HOME/lib/ , Right ? > I've put jar file in $KYLIN_HOME/lib & $HBase_Home/lib/ and set the 777 > authority . However it's still cannot working!! > If I'm wrong , please correct me ! Thanks. > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2373) kyin1.5.3, the exposed tables often disappear from the kylin Insight page.
[ https://issues.apache.org/jira/browse/KYLIN-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15815193#comment-15815193 ] Billy Liu commented on KYLIN-2373: -- Interesting found. That should be a bug. The patch is welcomed. > kyin1.5.3, the exposed tables often disappear from the kylin Insight page. > -- > > Key: KYLIN-2373 > URL: https://issues.apache.org/jira/browse/KYLIN-2373 > Project: Kylin > Issue Type: Bug > Components: REST Service >Affects Versions: v1.5.3 >Reporter: chelubaiq >Assignee: Zhong,Jason > > 1 environment: > kylin1.5.3 > tow nodes: "query" server a and "all" server b, > with config: kylin.rest.servers=a_ip,b_ip > 2 problem: > the exposed tables often disappear from the kylin Insight page. > the log says: > ERROR [http-bio-7070-exec-9] project.ProjectL2Cache:240 : Realization > 'CUBE[name=custom_out_sales4]' reports column 'PROJECT1.TABLE1.COLUMN1', but > it is not equal to 'ColumnDesc [name=COLUMN1,table=PROJECT1.TABLE1]' > according to MetadataManager > 3 one way to reproduce: > in project1, cube1 is ready; > in server a, choose project1, and reload one table from the DataSource of > Model page, succeed; the table exist in Insight page. > in server b, choose project1, refresh Insight page, no tables found: "No > Result." > if click "reload metadata" in the system page, tables will show up. > 4 > reason of table not found in server b: > in ProjectL2Cache, ColumnDesc from project realization is not equal to > ColumnDesc from MetadataManager, > because table.equals(other.table) is false in the ColumnDesc.equals() > method, > because table's lastModified not equal. > and table's lastModified from MetadataManager > from project realization > the bug may exist in CacheService.rebuildCache(): > case TABLE: > getMetadataManager().reloadTableCache(cacheKey); > CubeDescManager.clearCache(); > break; > MetadataManager.reloadAllDataModel() may be also needed here. > or MetadataManager.reloadDataModelDesc(forEachModelNameOfTheProject) > or MetadataManager.clearCache() > 5 > why table exist in server a's Insight page? > according to 4, server a should also have same equality problem and should > also disappear? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2383) count distinct should not include NULL
[ https://issues.apache.org/jira/browse/KYLIN-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15821062#comment-15821062 ] Billy Liu commented on KYLIN-2383: -- https://github.com/apache/kylin/commit/c6108a78eeca8e4b906aeba4f94e28addd974676 > count distinct should not include NULL > -- > > Key: KYLIN-2383 > URL: https://issues.apache.org/jira/browse/KYLIN-2383 > Project: Kylin > Issue Type: Bug >Reporter: liyang >Assignee: liyang > Fix For: v2.0.0 > > > Count distinct of (NULL, 1, 2, 3) should be 3 not 4. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2385) org.apache.hadoop.hbase.DoNotRetryIOException:
[ https://issues.apache.org/jira/browse/KYLIN-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2385. -- > org.apache.hadoop.hbase.DoNotRetryIOException: > -- > > Key: KYLIN-2385 > URL: https://issues.apache.org/jira/browse/KYLIN-2385 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v1.6.0 > Environment: CDH 5.4.3 > hbase-common-1.0.0-cdh5.4.3.jar > hive-1.1.0 >Reporter: lx >Assignee: liyang > > my KYLIN cube build sucessful,but when i execute the sql in kylin insight > .point out error as follow: > Error while executing SQL "select * from table limit 5000": > org.apache.hadoop.hbase.DoNotRetryIOException: > org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoSuchMethodError: > org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase/regionserver/Region; > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2066) at > org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107) at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) at > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) at > java.lang.Thread.run(Thread.java:744) Caused by: java.lang.NoSuchMethodError: > org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment.getRegion()Lorg/apache/hadoop/hbase -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (KYLIN-2228) Creating a Custom Interceptor in Apache DS 2.0.0-M23
[ https://issues.apache.org/jira/browse/KYLIN-2228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu closed KYLIN-2228. Resolution: Invalid > Creating a Custom Interceptor in Apache DS 2.0.0-M23 > > > Key: KYLIN-2228 > URL: https://issues.apache.org/jira/browse/KYLIN-2228 > Project: Kylin > Issue Type: Improvement >Reporter: Gautham Gurudatta Shet >Priority: Blocker > > The link > "http://directory.apache.org/apacheds/advanced-ug/6-implementing-interceptor.html; > does not have info on how to create a custom interceptor for Apache DS > 2.0.0-M23. The info is present for Apache DS 1.5.5. Can you please provide > this info? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2361) Upgrade to Tomcat 8.X
Billy Liu created KYLIN-2361: Summary: Upgrade to Tomcat 8.X Key: KYLIN-2361 URL: https://issues.apache.org/jira/browse/KYLIN-2361 Project: Kylin Issue Type: Task Components: Web Affects Versions: v1.6.0 Reporter: Billy Liu Assignee: Billy Liu Priority: Minor Apache Tomcat 8.5.x supports the same Servlet, JSP, EL, and WebSocket Specification versions as Apache Tomcat 8.0.x. In addition to that, it also implements the JASPIC 1.1 specification. There are significant changes in many areas under the hood, resulting in improved performance, stability, and total cost of ownership. Please refer to the Apache Tomcat 8.5 Changelog for details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2361) Upgrade to Tomcat 8.X
[ https://issues.apache.org/jira/browse/KYLIN-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803305#comment-15803305 ] Billy Liu commented on KYLIN-2361: -- https://github.com/apache/kylin/commit/f267ba9dcb3db317d07eb367ddff668e03291519 > Upgrade to Tomcat 8.X > - > > Key: KYLIN-2361 > URL: https://issues.apache.org/jira/browse/KYLIN-2361 > Project: Kylin > Issue Type: Task > Components: Web >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v2.0.0 > > > Apache Tomcat 8.5.x supports the same Servlet, JSP, EL, and WebSocket > Specification versions as Apache Tomcat 8.0.x. In addition to that, it also > implements the JASPIC 1.1 specification. There are significant changes in > many areas under the hood, resulting in improved performance, stability, and > total cost of ownership. Please refer to the Apache Tomcat 8.5 Changelog for > details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2361) Upgrade to Tomcat 8.X
[ https://issues.apache.org/jira/browse/KYLIN-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2361. -- Resolution: Fixed Fix Version/s: v2.0.0 > Upgrade to Tomcat 8.X > - > > Key: KYLIN-2361 > URL: https://issues.apache.org/jira/browse/KYLIN-2361 > Project: Kylin > Issue Type: Task > Components: Web >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v2.0.0 > > > Apache Tomcat 8.5.x supports the same Servlet, JSP, EL, and WebSocket > Specification versions as Apache Tomcat 8.0.x. In addition to that, it also > implements the JASPIC 1.1 specification. There are significant changes in > many areas under the hood, resulting in improved performance, stability, and > total cost of ownership. Please refer to the Apache Tomcat 8.5 Changelog for > details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2360) Fix potential bugs reported by Sonar
[ https://issues.apache.org/jira/browse/KYLIN-2360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15800912#comment-15800912 ] Billy Liu commented on KYLIN-2360: -- https://github.com/apache/kylin/commit/c64f55b2e34089d10c8a55301b62f96aa597d8f1 > Fix potential bugs reported by Sonar > > > Key: KYLIN-2360 > URL: https://issues.apache.org/jira/browse/KYLIN-2360 > Project: Kylin > Issue Type: Improvement >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > > https://builds.apache.org/analysis/component_issues?id=org.apache.kylin%3Akylin#resolved=false|types=BUG > reports some potential bugs, I will try to fix some of them to improve the > code quality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2356) Incorrect result when filter on numeric columns
[ https://issues.apache.org/jira/browse/KYLIN-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15800762#comment-15800762 ] Billy Liu commented on KYLIN-2356: -- Patch is good. But if the anonymous class name could be refactored to IntegerCodeSystem, DecimalCodeSystem, it would be easier for reading. > Incorrect result when filter on numeric columns > --- > > Key: KYLIN-2356 > URL: https://issues.apache.org/jira/browse/KYLIN-2356 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v1.6.0, v1.5.4.1 >Reporter: zhengdong >Assignee: zhengdong > Attachments: > 0001-KYLIN-2356-Incorrect-result-when-filter-on-numeric-c.patch > > > For now, we always use StringCodeSystem.INSTANCE to evaluate tuple when > translate derived column filter. This may lead to incorrect result for filter > on numeric column -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2360) Fix potential bugs reported by Sonar
Billy Liu created KYLIN-2360: Summary: Fix potential bugs reported by Sonar Key: KYLIN-2360 URL: https://issues.apache.org/jira/browse/KYLIN-2360 Project: Kylin Issue Type: Improvement Reporter: Billy Liu Assignee: Billy Liu Priority: Minor https://builds.apache.org/analysis/component_issues?id=org.apache.kylin%3Akylin#resolved=false|types=BUG reports some potential bugs, I will try to fix some of them to improve the code quality. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2356) Incorrect result when filter on numeric columns
[ https://issues.apache.org/jira/browse/KYLIN-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15800941#comment-15800941 ] Billy Liu commented on KYLIN-2356: -- Thanks [~zhengd], the code has merged into master. https://github.com/apache/kylin/commit/980b03b1abb8af82a9bceeb497847fe17c19c3e3 It would be better if more test cases could be added also. > Incorrect result when filter on numeric columns > --- > > Key: KYLIN-2356 > URL: https://issues.apache.org/jira/browse/KYLIN-2356 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v1.6.0, v1.5.4.1 >Reporter: zhengdong >Assignee: zhengdong > Fix For: v2.0.0 > > Attachments: > 0001-KYLIN-2356-Incorrect-result-when-filter-on-numeric-c.patch, > KYLIN-2356-Incorrect-result-when-filter-on-numeric-c.patch > > > For now, we always use StringCodeSystem.INSTANCE to evaluate tuple when > translate derived column filter. This may lead to incorrect result for filter > on numeric column -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2356) Incorrect result when filter on numeric columns
[ https://issues.apache.org/jira/browse/KYLIN-2356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2356. -- Resolution: Fixed Fix Version/s: v2.0.0 > Incorrect result when filter on numeric columns > --- > > Key: KYLIN-2356 > URL: https://issues.apache.org/jira/browse/KYLIN-2356 > Project: Kylin > Issue Type: Bug > Components: Query Engine >Affects Versions: v1.6.0, v1.5.4.1 >Reporter: zhengdong >Assignee: zhengdong > Fix For: v2.0.0 > > Attachments: > 0001-KYLIN-2356-Incorrect-result-when-filter-on-numeric-c.patch, > KYLIN-2356-Incorrect-result-when-filter-on-numeric-c.patch > > > For now, we always use StringCodeSystem.INSTANCE to evaluate tuple when > translate derived column filter. This may lead to incorrect result for filter > on numeric column -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2364) Output table name to error info in LookupTable
[ https://issues.apache.org/jira/browse/KYLIN-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15808675#comment-15808675 ] Billy Liu commented on KYLIN-2364: -- Thanks, [~kangkaisen], patch merged. https://github.com/apache/kylin/commit/e46d699e05100db084db354d7efb3786575d5c54 > Output table name to error info in LookupTable > -- > > Key: KYLIN-2364 > URL: https://issues.apache.org/jira/browse/KYLIN-2364 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: v1.6.0 >Reporter: kangkaisen >Assignee: kangkaisen >Priority: Minor > Attachments: KYLIN-2364.patch > > > We should output table name so that the user know which LookupTable is broken. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2364) Output table name to error info in LookupTable
[ https://issues.apache.org/jira/browse/KYLIN-2364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2364. -- Resolution: Fixed Fix Version/s: v2.0.0 > Output table name to error info in LookupTable > -- > > Key: KYLIN-2364 > URL: https://issues.apache.org/jira/browse/KYLIN-2364 > Project: Kylin > Issue Type: Improvement > Components: Metadata >Affects Versions: v1.6.0 >Reporter: kangkaisen >Assignee: kangkaisen >Priority: Minor > Fix For: v2.0.0 > > Attachments: KYLIN-2364.patch > > > We should output table name so that the user know which LookupTable is broken. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2317) Hybrid Cube CLI Tools
[ https://issues.apache.org/jira/browse/KYLIN-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15785080#comment-15785080 ] Billy Liu commented on KYLIN-2317: -- [INFO] Reactor Summary: [INFO] [INFO] Apache Kylin ... SUCCESS [ 4.894 s] [INFO] Apache Kylin - Calcite Overrides ... SUCCESS [ 4.568 s] [INFO] Apache Kylin - Core Common . SUCCESS [ 6.527 s] [INFO] Apache Kylin - Core Metadata ... SUCCESS [ 32.379 s] [INFO] Apache Kylin - Core Dictionary . SUCCESS [02:45 min] [INFO] Apache Kylin - Core Cube ... SUCCESS [01:18 min] [INFO] Apache Kylin - Core Job SUCCESS [04:09 min] [INFO] Apache Kylin - Core Storage SUCCESS [ 39.226 s] [INFO] Apache Kylin - MapReduce Engine SUCCESS [ 22.737 s] [INFO] Apache Kylin - HBase Storage ... SUCCESS [ 29.333 s] [INFO] Apache Kylin - Spark Engine SUCCESS [ 16.995 s] [INFO] Apache Kylin - Hive Source . SUCCESS [ 3.335 s] [INFO] Apache Kylin - Kafka Source SUCCESS [ 2.317 s] [INFO] Apache Kylin - Query ... SUCCESS [ 2.946 s] [INFO] Apache Kylin - Tool SUCCESS [ 3.098 s] [INFO] Apache Kylin - REST Server Base SUCCESS [ 7.406 s] [INFO] Apache Kylin - REST Server . SUCCESS [ 49.415 s] [INFO] Apache Kylin - JDBC Driver . SUCCESS [ 8.223 s] [INFO] Apache Kylin - Assembly SUCCESS [ 11.712 s] [INFO] Apache Kylin - Integration Test SUCCESS [ 03:20 h] [INFO] Apache Kylin - Tomcat Extension SUCCESS [ 5.789 s] > Hybrid Cube CLI Tools > - > > Key: KYLIN-2317 > URL: https://issues.apache.org/jira/browse/KYLIN-2317 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Attachments: KYLIN-2317.patch > > > Provide CLI tools for Hybrid Cube feature > 1. Create new Hybrid Cube by merging existing cubes > 2. Check cubes no overlap > 3. Merge existing Cube into existing Hybrid Cube -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2346) OLAP TableScan MUST NOT be shared by more than one prent
[ https://issues.apache.org/jira/browse/KYLIN-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15792989#comment-15792989 ] Billy Liu commented on KYLIN-2346: -- Sorry, could not catch up what you mean. What is "prent"? What is your expected result? To find the SQL reference, please check http://calcite.apache.org/docs/reference.html first. Thank you. And one more thing, please do not ask question in JIRA. JIRA is designed for confirmed bug tracking or feature proposal. > OLAP TableScan MUST NOT be shared by more than one prent > > > Key: KYLIN-2346 > URL: https://issues.apache.org/jira/browse/KYLIN-2346 > Project: Kylin > Issue Type: Bug >Reporter: james >Priority: Blocker > > i am a new fresher of the apache-kylin, > i build cube,success. > i want to run a sql sentence > the result of the relation-database is right..what i want.. > the result of the cube modile is wrong..the results is as the titles > the sql is > select * from tradectr.dbo.g_og_corpinfo t1 left join > tradectr.dbo.g_og_corpinfo t2 on t1.parentsid = t2.corpsid where > t2.levelscode = 'G011' AND t2.corpname like '%湖北%' > it is a simple sql,just to find the company of the hubei province... > also somebody could tell me the type of the sql which apache-kylin > supports,thanks very much。 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2345) Not able to open web interface for Kylin
[ https://issues.apache.org/jira/browse/KYLIN-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15792999#comment-15792999 ] Billy Liu commented on KYLIN-2345: -- Please do not open JIRA for this kind of question. Please raise your question with more detail exception log in kylin user mailer. Most of these cases are classpath issue or misconfigured. > Not able to open web interface for Kylin > > > Key: KYLIN-2345 > URL: https://issues.apache.org/jira/browse/KYLIN-2345 > Project: Kylin > Issue Type: Bug >Reporter: Anurag >Priority: Blocker > > Not able to open web interface for Kylin > 1. hadoop version is hadoop-2.7.3 > 2. apache kylin version apache-kylin-1.6.0-hbase1.x-bin > 3 .hbase version hbase-1.1.7 > 4. hive version is apache-hive-2.1.0-bin > issues in log > :org.springframework.beans.factory.BeanCurrentlyInCreationException: Error > creating bean with name 'mvcContentNegotiationManager': Requested bean is > currently in creation: Is there an unresolvable circular reference? > at > org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.beforeSingletonCreation(DefaultSingletonBeanRegistry.java:327) > at > org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:217) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2323) Refine Table load/unload error message
[ https://issues.apache.org/jira/browse/KYLIN-2323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2323: - Attachment: KYLIN-2323-2.patch Hi [~zhongjian], there is more refactor work in this update. Including: display table type in ext info, remove unused table APIs, unify the ext data structure. Could you have a check? > Refine Table load/unload error message > -- > > Key: KYLIN-2323 > URL: https://issues.apache.org/jira/browse/KYLIN-2323 > Project: Kylin > Issue Type: Improvement > Components: REST Service >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v2.0.0 > > Attachments: KYLIN-2323-2.patch, KYLIN-2323.patch > > > There is no exception handling in TableController, so most of exceptions will > not be found in kylin.log, but kylin.out. The TableController should provide > more useful messages, and be stable when exception happens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (KYLIN-2343) When syn hive table, got error but actually the table is synced
[ https://issues.apache.org/jira/browse/KYLIN-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu reassigned KYLIN-2343: Assignee: Billy Liu > When syn hive table, got error but actually the table is synced > --- > > Key: KYLIN-2343 > URL: https://issues.apache.org/jira/browse/KYLIN-2343 > Project: Kylin > Issue Type: Bug > Components: REST Service >Affects Versions: v2.0.0 >Reporter: Shaofeng SHI >Assignee: Billy Liu > Fix For: v2.0.0 > > Attachments: Screen Shot 2016-12-30 at 11.34.24 AM.png > > > See the screen shot. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2343) When syn hive table, got error but actually the table is synced
[ https://issues.apache.org/jira/browse/KYLIN-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2343. -- Resolution: Fixed https://github.com/apache/kylin/commit/e7e29f1fee854f64f18bb898033f09e32996f16a Fixed. > When syn hive table, got error but actually the table is synced > --- > > Key: KYLIN-2343 > URL: https://issues.apache.org/jira/browse/KYLIN-2343 > Project: Kylin > Issue Type: Bug > Components: REST Service >Affects Versions: v2.0.0 >Reporter: Shaofeng SHI >Assignee: Billy Liu > Fix For: v2.0.0 > > Attachments: Screen Shot 2016-12-30 at 11.34.24 AM.png > > > See the screen shot. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KYLIN-2317) Hybrid Cube CLI Tools
[ https://issues.apache.org/jira/browse/KYLIN-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15794146#comment-15794146 ] Billy Liu commented on KYLIN-2317: -- Thanks [~Shaofengshi]. Patch merged. https://github.com/apache/kylin/commit/d7196d20ed7a70d11192d6f36eb802ee21f5539a I will prepare some documents later and continue maintain this tool. > Hybrid Cube CLI Tools > - > > Key: KYLIN-2317 > URL: https://issues.apache.org/jira/browse/KYLIN-2317 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Fix For: v2.0.0 > > Attachments: KYLIN-2317.patch > > > Provide CLI tools for Hybrid Cube feature > 1. Create new Hybrid Cube by merging existing cubes > 2. Check cubes no overlap > 3. Merge existing Cube into existing Hybrid Cube -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2317) Hybrid Cube CLI Tools
[ https://issues.apache.org/jira/browse/KYLIN-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2317. -- Resolution: Fixed Fix Version/s: v2.0.0 > Hybrid Cube CLI Tools > - > > Key: KYLIN-2317 > URL: https://issues.apache.org/jira/browse/KYLIN-2317 > Project: Kylin > Issue Type: New Feature > Components: Metadata >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu > Fix For: v2.0.0 > > Attachments: KYLIN-2317.patch > > > Provide CLI tools for Hybrid Cube feature > 1. Create new Hybrid Cube by merging existing cubes > 2. Check cubes no overlap > 3. Merge existing Cube into existing Hybrid Cube -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KYLIN-2368) Enable Findbugs plugin
Billy Liu created KYLIN-2368: Summary: Enable Findbugs plugin Key: KYLIN-2368 URL: https://issues.apache.org/jira/browse/KYLIN-2368 Project: Kylin Issue Type: Task Components: General Affects Versions: v1.6.0 Reporter: Billy Liu Assignee: Billy Liu Priority: Minor FindBugs looks for bugs in Java programs. It is based on the concept of bug patterns. A bug pattern is a code idiom that is often an error. This task will enable FindBugs for Kylin report. To check code by FindBugs: mvn findbugs:findbugs or mvn site -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (KYLIN-2368) Enable Findbugs plugin
[ https://issues.apache.org/jira/browse/KYLIN-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu resolved KYLIN-2368. -- Resolution: Fixed https://github.com/apache/kylin/commit/75625ae512b3d6cc080053e4e8871f3fff816ef0 > Enable Findbugs plugin > -- > > Key: KYLIN-2368 > URL: https://issues.apache.org/jira/browse/KYLIN-2368 > Project: Kylin > Issue Type: Task > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > > FindBugs looks for bugs in Java programs. It is based on the concept of bug > patterns. A bug pattern is a code idiom that is often an error. > This task will enable FindBugs for Kylin report. > To check code by FindBugs: > mvn findbugs:findbugs > or mvn site -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KYLIN-2368) Enable Findbugs plugin
[ https://issues.apache.org/jira/browse/KYLIN-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billy Liu updated KYLIN-2368: - Fix Version/s: v2.0.0 > Enable Findbugs plugin > -- > > Key: KYLIN-2368 > URL: https://issues.apache.org/jira/browse/KYLIN-2368 > Project: Kylin > Issue Type: Task > Components: General >Affects Versions: v1.6.0 >Reporter: Billy Liu >Assignee: Billy Liu >Priority: Minor > Fix For: v2.0.0 > > > FindBugs looks for bugs in Java programs. It is based on the concept of bug > patterns. A bug pattern is a code idiom that is often an error. > This task will enable FindBugs for Kylin report. > To check code by FindBugs: > mvn findbugs:findbugs > or mvn site -- This message was sent by Atlassian JIRA (v6.3.4#6332)
