[jira] [Commented] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633248#comment-16633248
]
ASF GitHub Bot commented on KYLIN-3605:
---
shaofengshi closed pull request #276: KYLIN-3605 upgrade hadoop-common and
zookeeper version to fix securit…
URL: https://github.com/apache/kylin/pull/276
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/LICENSE b/LICENSE
index a237c61e69..26e015cf79 100644
--- a/LICENSE
+++ b/LICENSE
@@ -265,5 +265,8 @@ RocksDB is dual-licensed under both the GPLv2 and Apache
2.0 License.You may
select, at your option, one of the above-listed licenses.
https://github.com/facebook/rocksdb/blob/master/LICENSE.Apache
-
-
+==
+For HadoopMetrics2Reporter:
+==
+HadoopMetrics2Reporter is licensed under Apache 2.0 License.
+https://github.com/joshelser/dropwizard-hadoop-metrics2
diff --git a/core-common/pom.xml b/core-common/pom.xml
index 3b5f7fc16b..594e39b39b 100644
--- a/core-common/pom.xml
+++ b/core-common/pom.xml
@@ -86,11 +86,6 @@
${dropwizard.version}
-
-com.github.joshelser
-
dropwizard-metrics-hadoop-metrics2-reporter
-0.1.2
-
commons-dbcp
commons-dbcp
@@ -101,5 +96,12 @@
mysql-connector-java
provided
+
+
+org.mockito
+mockito-core
+${mockito.version}
+test
+
diff --git
a/core-common/src/main/java/org/apache/kylin/common/metrics/metrics2/HadoopMetrics2Reporter.java
b/core-common/src/main/java/org/apache/kylin/common/metrics/metrics2/HadoopMetrics2Reporter.java
new file mode 100644
index 00..c0e48a6b7e
--- /dev/null
+++
b/core-common/src/main/java/org/apache/kylin/common/metrics/metrics2/HadoopMetrics2Reporter.java
@@ -0,0 +1,495 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.kylin.common.metrics.metrics2;
+
+import java.text.SimpleDateFormat;
+import java.util.Collections;
+import java.util.Date;
+import java.util.Iterator;
+import java.util.Locale;
+import java.util.Map.Entry;
+import java.util.Objects;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.hadoop.metrics2.MetricsCollector;
+import org.apache.hadoop.metrics2.MetricsInfo;
+import org.apache.hadoop.metrics2.MetricsRecordBuilder;
+import org.apache.hadoop.metrics2.MetricsSource;
+import org.apache.hadoop.metrics2.MetricsSystem;
+import org.apache.hadoop.metrics2.lib.Interns;
+import org.apache.hadoop.metrics2.lib.MetricsRegistry;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.codahale.metrics.Counter;
+import com.codahale.metrics.Gauge;
+import com.codahale.metrics.Histogram;
+import com.codahale.metrics.Meter;
+import com.codahale.metrics.MetricFilter;
+import com.codahale.metrics.MetricRegistry;
+import com.codahale.metrics.ScheduledReporter;
+import com.codahale.metrics.Snapshot;
+import com.codahale.metrics.Timer;
+
+/**
+ * Modified from https://github.com/joshelser/dropwizard-hadoop-metrics2,
Copyright by Josh Elser
+ *
+ * A {@link com.codahale.metrics.Reporter} which also acts as a Hadoop Metrics2
+ * {@link MetricsSource}. Configure it like other Reporters.
+ *
+ *
+ * final HadoopMetrics2Reporter metrics2Reporter =
HadoopMetrics2Reporter.forRegistry(metrics)
+ * .build(DefaultMetricsSystem.initialize("Phoenix"), // The
application-level name
+ *"QueryServer", // Component name
+ *"Phoenix Query Server", // Component description
+ *"General"); // Name for each metric record
+ *
+ * metrics2Reporter.start(30, TimeUnit.SECONDS);
+ *
+ */
+public class HadoopMetrics2Reporter
[jira] [Commented] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633247#comment-16633247
]
ASF GitHub Bot commented on KYLIN-3605:
---
shaofengshi commented on issue #276: KYLIN-3605 upgrade hadoop-common and
zookeeper version to fix securit…
URL: https://github.com/apache/kylin/pull/276#issuecomment-425696064
Merged.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Fix security issues reported by snyk.io
> ---
>
> Key: KYLIN-3605
> URL: https://issues.apache.org/jira/browse/KYLIN-3605
> Project: Kylin
> Issue Type: Improvement
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.6.0
>
>
> HIGH SEVERITY
> h1. Arbitrary Code Execution
> * Vulnerable module: commons-beanutils:commons-beanutils
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> * *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT
> › org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›
> commons-configuration:commons-configuration@1.6 ›
> commons-digester:commons-digester@1.8 ›
> commons-beanutils:commons-beanutils@1.7.0
> *Remediation:* No remediation path available.
> h2. Overview
> [{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
> Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
> property, which allows remote attackers to manipulate the {{ClassLoader}}and
> execute arbitrary code via the class parameter, as demonstrated by the
> passing of this parameter to the {{getClass}} method of the
> \{{ActionForm}}object in Struts 1.
>
> HIGH SEVERITY
> h1. Arbitrary Command Execution
> * Vulnerable module: org.mortbay.jetty:jetty
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> * *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT
> › org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
> *Remediation:* No remediation path available.
> h2. Overview
> [org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
> is an open-source project providing a HTTP server, HTTP client and
> javax.servlet container.
> Affected versions of this package are vulnerable to Arbitrary Command
> Execution. It writes backtrace data without sanitizing non-printable
> characters, which might allow remote attackers to modify a window's title, or
> possibly execute arbitrary commands or overwrite files, via an HTTP request
> containing an escape sequence for a terminal emulator, related to (1) a
> string value in the Age parameter to the default URI for the Cookie Dump
> Servlet in test-jetty-webapp/src/main/java/com/acme/CookieDump.java under
> cookie/, (2) an alphabetic value in the A parameter to jsp/expr.jsp, or (3)
> an alphabetic value in the Content-Length HTTP header to an arbitrary
> application.
> HIGH SEVERITY
> h1. Information Exposure
> * Vulnerable module: org.apache.hadoop:hadoop-common
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> * *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT
> › org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1
> *Remediation:* No remediation path available.
> h2. Overview
> [{{org.apache.hadoop:hadoop-common}}|https://hadoop.apache.org/] is a
> framework that allows for the distributed processing of large data sets
> across clusters of computers using simple programming models.
> Affected versions of the package are vulnerable to Information Exposure.
> If you use the CredentialProvider feature to encrypt passwords used in
> NodeManager configs, it may be possible for any Container launched by that
> NodeManager to gain access to the encryption password. The other passwords
> themselves are not directly exposed.
>
> HIGH SEVERITY
> h1. Access Restriction Bypass
> * Vulnerable module: org.springframework.security:spring-security-core
> * Introduced through: org.apache.kylin:kylin-server-base@2.6.0-SNAPSHOT
[jira] [Commented] (KYLIN-3560) Should not depend on personal repository
[
https://issues.apache.org/jira/browse/KYLIN-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633245#comment-16633245
]
ASF subversion and git services commented on KYLIN-3560:
Commit 8e98b8b8807929cb7bbdf8f52bbb98383c254a33 in kylin's branch
refs/heads/master from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=8e98b8b ]
KYLIN-3560 Should not depend on personal repository
> Should not depend on personal repository
>
>
> Key: KYLIN-3560
> URL: https://issues.apache.org/jira/browse/KYLIN-3560
> Project: Kylin
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.6.0
>
>
> In core-common/pom.xml:
> {code}
> com.github.joshelser
> {code}
> We shouldn't depend on personal repository.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633244#comment-16633244
]
ASF subversion and git services commented on KYLIN-3605:
Commit 55a085cffa14a0e24b2f8d716d3e4925faf3c40d in kylin's branch
refs/heads/master from shaofengshi
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=55a085c ]
KYLIN-3605 upgrade zookeeper version to fix security issues
> Fix security issues reported by snyk.io
> ---
>
> Key: KYLIN-3605
> URL: https://issues.apache.org/jira/browse/KYLIN-3605
> Project: Kylin
> Issue Type: Improvement
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.6.0
>
>
> HIGH SEVERITY
> h1. Arbitrary Code Execution
> * Vulnerable module: commons-beanutils:commons-beanutils
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> * *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT
> › org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›
> commons-configuration:commons-configuration@1.6 ›
> commons-digester:commons-digester@1.8 ›
> commons-beanutils:commons-beanutils@1.7.0
> *Remediation:* No remediation path available.
> h2. Overview
> [{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
> Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
> property, which allows remote attackers to manipulate the {{ClassLoader}}and
> execute arbitrary code via the class parameter, as demonstrated by the
> passing of this parameter to the {{getClass}} method of the
> \{{ActionForm}}object in Struts 1.
>
> HIGH SEVERITY
> h1. Arbitrary Command Execution
> * Vulnerable module: org.mortbay.jetty:jetty
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> * *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT
> › org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
> *Remediation:* No remediation path available.
> h2. Overview
> [org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
> is an open-source project providing a HTTP server, HTTP client and
> javax.servlet container.
> Affected versions of this package are vulnerable to Arbitrary Command
> Execution. It writes backtrace data without sanitizing non-printable
> characters, which might allow remote attackers to modify a window's title, or
> possibly execute arbitrary commands or overwrite files, via an HTTP request
> containing an escape sequence for a terminal emulator, related to (1) a
> string value in the Age parameter to the default URI for the Cookie Dump
> Servlet in test-jetty-webapp/src/main/java/com/acme/CookieDump.java under
> cookie/, (2) an alphabetic value in the A parameter to jsp/expr.jsp, or (3)
> an alphabetic value in the Content-Length HTTP header to an arbitrary
> application.
> HIGH SEVERITY
> h1. Information Exposure
> * Vulnerable module: org.apache.hadoop:hadoop-common
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> * *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT
> › org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1
> *Remediation:* No remediation path available.
> h2. Overview
> [{{org.apache.hadoop:hadoop-common}}|https://hadoop.apache.org/] is a
> framework that allows for the distributed processing of large data sets
> across clusters of computers using simple programming models.
> Affected versions of the package are vulnerable to Information Exposure.
> If you use the CredentialProvider feature to encrypt passwords used in
> NodeManager configs, it may be possible for any Container launched by that
> NodeManager to gain access to the encryption password. The other passwords
> themselves are not directly exposed.
>
> HIGH SEVERITY
> h1. Access Restriction Bypass
> * Vulnerable module: org.springframework.security:spring-security-core
> * Introduced through: org.apache.kylin:kylin-server-base@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> * *Introduced through*: org.apache.kylin:kylin-server@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-server-base@2.6.0-SNAPSHOT
> ›org.springframework.security.extensions:spring-security-saml2-core@1.0.2.RELEASE
[jira] [Commented] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633234#comment-16633234
]
ASF GitHub Bot commented on KYLIN-3605:
---
coveralls commented on issue #276: KYLIN-3605 upgrade hadoop-common and
zookeeper version to fix securit…
URL: https://github.com/apache/kylin/pull/276#issuecomment-425694314
## Pull Request Test Coverage Report for [Build
3724](https://coveralls.io/builds/19266739)
* **135** of **163** **(82.82%)** changed or added relevant lines in
**1** file are covered.
* **24** unchanged lines in **3** files lost coverage.
* Overall coverage increased (+**0.1%**) to **23.289%**
---
| Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
| :-|--||---: |
|
[core-common/src/main/java/org/apache/kylin/common/metrics/metrics2/HadoopMetrics2Reporter.java](https://coveralls.io/builds/19266739/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fmetrics%2Fmetrics2%2FHadoopMetrics2Reporter.java#L143)
| 135 | 163 | 82.82%
| Files with Coverage Reduction | New Missed Lines | % |
| :-|--|--: |
|
[core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/InMemCubeBuilder.java](https://coveralls.io/builds/19266739/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FInMemCubeBuilder.java#L527)
| 5 | 0.0% |
|
[query/src/main/java/org/apache/kylin/query/routing/rules/RemoveBlackoutRealizationsRule.java](https://coveralls.io/builds/19266739/source?filename=query%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fquery%2Frouting%2Frules%2FRemoveBlackoutRealizationsRule.java#L62)
| 6 | 0.0% |
|
[core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/DoggedCubeBuilder.java](https://coveralls.io/builds/19266739/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FDoggedCubeBuilder.java#L355)
| 13 | 0.0% |
| Totals | [](https://coveralls.io/builds/19266739)
|
| :-- | --: |
| Change from base [Build 3723](https://coveralls.io/builds/19266461): |
0.1% |
| Covered Lines: | 16297 |
| Relevant Lines: | 69976 |
---
# - [Coveralls](https://coveralls.io)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Fix security issues reported by snyk.io
> ---
>
> Key: KYLIN-3605
> URL: https://issues.apache.org/jira/browse/KYLIN-3605
> Project: Kylin
> Issue Type: Improvement
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.6.0
>
>
> HIGH SEVERITY
> h1. Arbitrary Code Execution
> * Vulnerable module: commons-beanutils:commons-beanutils
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> * *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT
> › org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›
> commons-configuration:commons-configuration@1.6 ›
> commons-digester:commons-digester@1.8 ›
> commons-beanutils:commons-beanutils@1.7.0
> *Remediation:* No remediation path available.
> h2. Overview
> [{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
> Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
> property, which allows remote attackers to manipulate the {{ClassLoader}}and
> execute arbitrary code via the class parameter, as demonstrated by the
> passing of this parameter to the {{getClass}} method of the
> \{{ActionForm}}object in Struts 1.
>
> HIGH SEVERITY
> h1. Arbitrary Command Execution
> * Vulnerable module: org.mortbay.jetty:jetty
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> * *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT
> › org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
> *Remediation:* No remediation path available.
> h2. Overview
> [org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
> is an open-source project providing a HTTP server, HTTP client and
>
[jira] [Commented] (KYLIN-3605) Fix security issues reported by snyk.io
[ https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633233#comment-16633233 ] ASF GitHub Bot commented on KYLIN-3605: --- codecov-io edited a comment on issue #276: KYLIN-3605 upgrade hadoop-common and zookeeper version to fix securit… URL: https://github.com/apache/kylin/pull/276#issuecomment-425686030 # [Codecov](https://codecov.io/gh/apache/kylin/pull/276?src=pr=h1) Report > Merging [#276](https://codecov.io/gh/apache/kylin/pull/276?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/8db033097833be5118655a6d88a5b96e862055ae?src=pr=desc) will **increase** coverage by `0.13%`. > The diff coverage is `80.98%`. [](https://codecov.io/gh/apache/kylin/pull/276?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #276 +/- ## + Coverage 21.14% 21.28% +0.13% - Complexity 4405 4435 +30 Files 1086 1087 +1 Lines 6980169976 +175 Branches 1011010126 +16 + Hits 1476014891 +131 - Misses5364153681 +40 - Partials 1400 1404 +4 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/276?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...ylin/common/metrics/metrics2/Metrics2Reporter.java](https://codecov.io/gh/apache/kylin/pull/276/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9tZXRyaWNzL21ldHJpY3MyL01ldHJpY3MyUmVwb3J0ZXIuamF2YQ==) | `0% <ø> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...ommon/metrics/metrics2/HadoopMetrics2Reporter.java](https://codecov.io/gh/apache/kylin/pull/276/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9tZXRyaWNzL21ldHJpY3MyL0hhZG9vcE1ldHJpY3MyUmVwb3J0ZXIuamF2YQ==) | `80.98% <80.98%> (ø)` | `30 <30> (?)` | | | [...lin/dict/lookup/cache/RocksDBLookupTableCache.java](https://codecov.io/gh/apache/kylin/pull/276/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGVDYWNoZS5qYXZh) | `76.16% <0%> (-0.52%)` | `27% <0%> (ø)` | | | [...ache/kylin/cube/inmemcubing/DoggedCubeBuilder.java](https://codecov.io/gh/apache/kylin/pull/276/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL0RvZ2dlZEN1YmVCdWlsZGVyLmphdmE=) | `0% <0%> (ø)` | `0% <0%> (ø)` | :arrow_down: | | [.../routing/rules/RemoveBlackoutRealizationsRule.java](https://codecov.io/gh/apache/kylin/pull/276/diff?src=pr=tree#diff-cXVlcnkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3F1ZXJ5L3JvdXRpbmcvcnVsZXMvUmVtb3ZlQmxhY2tvdXRSZWFsaXphdGlvbnNSdWxlLmphdmE=) | `0% <0%> (ø)` | `0% <0%> (ø)` | :arrow_down: | | [...pache/kylin/cube/inmemcubing/InMemCubeBuilder.java](https://codecov.io/gh/apache/kylin/pull/276/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL0luTWVtQ3ViZUJ1aWxkZXIuamF2YQ==) | `0% <0%> (ø)` | `0% <0%> (ø)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/276?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/276?src=pr=footer). Last update [8db0330...8bf7b3d](https://codecov.io/gh/apache/kylin/pull/276?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix security issues reported by snyk.io > --- > > Key: KYLIN-3605 > URL: https://issues.apache.org/jira/browse/KYLIN-3605 > Project: Kylin > Issue Type: Improvement >Reporter: Shaofeng SHI >Assignee: Shaofeng SHI >Priority: Major > Fix For: v2.6.0 > > > HIGH SEVERITY > h1. Arbitrary Code Execution > * Vulnerable module: commons-beanutils:commons-beanutils > * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT > h2. Detailed paths and
[jira] [Updated] (KYLIN-3606) Unstable KylinUserManagerTest
[
https://issues.apache.org/jira/browse/KYLIN-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3606:
Attachment: 435155092log.txt
> Unstable KylinUserManagerTest
> -
>
> Key: KYLIN-3606
> URL: https://issues.apache.org/jira/browse/KYLIN-3606
> Project: Kylin
> Issue Type: Bug
>Affects Versions: v2.6.0
>Reporter: Shaofeng SHI
>Priority: Major
> Attachments: 435155092log.txt
>
>
> {code:java}
> [[1;34mINFO[m] [1;32mTests run: [0;1;32m3[m, Failures: 0, Errors: 0,
> Skipped: 0, Time elapsed: 4.187 s - in
> org.apache.kylin.rest.security.[1mQueryWithTableACLTest[m
> [[1;34mINFO[m] Running
> org.apache.kylin.rest.security.[1mKylinUserManagerTest[m
> 2018-09-30 04:01:45,114 INFO [main] common.KylinConfig:113 : Destroy
> KylinConfig
> 2018-09-30 04:01:45,160 INFO [main] common.KylinConfig:332 : Loading
> kylin-defaults.properties from
> /home/travis/build/apache/kylin/core-common/target/classes/kylin-defaults.properties
> 2018-09-30 04:01:45,169 INFO [main] common.KylinConfig:287 : Use
> KYLIN_CONF=../examples/test_metadata/
> 2018-09-30 04:01:45,170 INFO [main] common.KylinConfig:99 : Initialized a
> new KylinConfig from getInstanceFromEnv : 1932094362
> 2018-09-30 04:01:45,170 INFO [main] common.KylinConfigBase:198 : Kylin
> Config was updated with kylin.metadata.url : ../examples/test_metadata/
> 2018-09-30 04:01:45,170 INFO [main] common.KylinConfigBase:198 : Kylin
> Config was updated with kylin.env.hdfs-working-dir : file:///tmp/kylin
> 2018-09-30 04:01:45,193 INFO [main] util.MultiNodeManagerTestBase:50 :
> Chosen port for CacheServiceTest is 45238
> 2018-09-30 04:01:45,194 INFO [main] common.KylinConfigBase:198 : Kylin
> Config was updated with kylin.server.cluster-servers : localhost:45238
> 2018-09-30 04:01:45,194 INFO [main] common.KylinConfigBase:198 : Kylin
> Config was updated with kylin.server.cluster-servers : localhost:45238
> 2018-09-30 04:01:45,194 INFO [main] common.KylinConfigBase:198 : Kylin
> Config was updated with kylin.metadata.url : ../examples/test_metadata
> 2018-09-30 04:01:45,273 INFO [Thread-2-SendThread(localhost:2181)]
> zookeeper.ClientCnxn:1029 : Opening socket connection to server
> localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL
> (unknown error)
> 2018-09-30 04:01:45,274 INFO [Thread-2-SendThread(localhost:2181)]
> zookeeper.ClientCnxn:1166 : Socket error occurred: localhost/127.0.0.1:2181:
> Connection refused
> 2018-09-30 04:01:45,275 INFO [main] util.log:192 : Logging initialized
> @92503ms
> 2018-09-30 04:01:46,021 INFO [main] server.Server:346 :
> jetty-9.3.22.v20171030
> 2018-09-30 04:01:46,225 INFO [main] handler.ContextHandler:781 : Started
> o.e.j.s.ServletContextHandler@545311c{/,null,AVAILABLE}
> 2018-09-30 04:01:46,254 INFO [main] server.AbstractConnector:278 : Started
> ServerConnector@4bdfeead{HTTP/1.1,[http/1.1]}{0.0.0.0:45238}
> 2018-09-30 04:01:46,254 INFO [main] server.Server:414 : Started @93482ms
> 2018-09-30 04:01:46,255 INFO [main] security.KylinUserManager:61 :
> Initializing KylinUserManager with config ../examples/test_metadata/
> 2018-09-30 04:01:46,256 INFO [main] persistence.ResourceStore:88 : Using
> metadata url ../examples/test_metadata/ for resource store
> 2018-09-30 04:01:46,256 DEBUG [main] cachesync.CachedCrudAssist:118 :
> Reloading ManagedUser from
> /home/travis/build/apache/kylin/server/../examples/test_metadata/user
> 2018-09-30 04:01:46,257 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded
> 0 ManagedUser(s) out of 0 resource
> 2018-09-30 04:01:46,257 INFO [main] common.KylinConfig:432 : Creating new
> manager instance of class org.apache.kylin.metadata.cachesync.Broadcaster
> 2018-09-30 04:01:46,258 DEBUG [main] cachesync.Broadcaster:101 : 1 nodes in
> the cluster: [localhost:45238]
> 2018-09-30 04:01:46,259 INFO [main] security.KylinUserManager:61 :
> Initializing KylinUserManager with config ../examples/test_metadata
> 2018-09-30 04:01:46,259 INFO [main] persistence.ResourceStore:88 : Using
> metadata url ../examples/test_metadata for resource store
> 2018-09-30 04:01:46,259 DEBUG [main] cachesync.CachedCrudAssist:118 :
> Reloading ManagedUser from
> /home/travis/build/apache/kylin/server/../examples/test_metadata/user
> 2018-09-30 04:01:46,260 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded
> 0 ManagedUser(s) out of 0 resource
> 2018-09-30 04:01:46,260 INFO [main] common.KylinConfig:432 : Creating new
> manager instance of class org.apache.kylin.metadata.cachesync.Broadcaster
> 2018-09-30 04:01:46,260 DEBUG [main] cachesync.Broadcaster:101 : 1 nodes in
> the cluster: [localhost:45238]
> 2018-09-30 04:01:46,261 DEBUG [main] cachesync.CachedCrudAssist:190 : Saving
> ManagedUser at /user/U1
> 2018-09-30 04:01:46,263
[jira] [Created] (KYLIN-3606) Unstable KylinUserManagerTest
Shaofeng SHI created KYLIN-3606:
---
Summary: Unstable KylinUserManagerTest
Key: KYLIN-3606
URL: https://issues.apache.org/jira/browse/KYLIN-3606
Project: Kylin
Issue Type: Bug
Affects Versions: v2.6.0
Reporter: Shaofeng SHI
{code:java}
[[1;34mINFO[m] [1;32mTests run: [0;1;32m3[m, Failures: 0, Errors: 0,
Skipped: 0, Time elapsed: 4.187 s - in
org.apache.kylin.rest.security.[1mQueryWithTableACLTest[m
[[1;34mINFO[m] Running
org.apache.kylin.rest.security.[1mKylinUserManagerTest[m
2018-09-30 04:01:45,114 INFO [main] common.KylinConfig:113 : Destroy
KylinConfig
2018-09-30 04:01:45,160 INFO [main] common.KylinConfig:332 : Loading
kylin-defaults.properties from
/home/travis/build/apache/kylin/core-common/target/classes/kylin-defaults.properties
2018-09-30 04:01:45,169 INFO [main] common.KylinConfig:287 : Use
KYLIN_CONF=../examples/test_metadata/
2018-09-30 04:01:45,170 INFO [main] common.KylinConfig:99 : Initialized a new
KylinConfig from getInstanceFromEnv : 1932094362
2018-09-30 04:01:45,170 INFO [main] common.KylinConfigBase:198 : Kylin Config
was updated with kylin.metadata.url : ../examples/test_metadata/
2018-09-30 04:01:45,170 INFO [main] common.KylinConfigBase:198 : Kylin Config
was updated with kylin.env.hdfs-working-dir : file:///tmp/kylin
2018-09-30 04:01:45,193 INFO [main] util.MultiNodeManagerTestBase:50 : Chosen
port for CacheServiceTest is 45238
2018-09-30 04:01:45,194 INFO [main] common.KylinConfigBase:198 : Kylin Config
was updated with kylin.server.cluster-servers : localhost:45238
2018-09-30 04:01:45,194 INFO [main] common.KylinConfigBase:198 : Kylin Config
was updated with kylin.server.cluster-servers : localhost:45238
2018-09-30 04:01:45,194 INFO [main] common.KylinConfigBase:198 : Kylin Config
was updated with kylin.metadata.url : ../examples/test_metadata
2018-09-30 04:01:45,273 INFO [Thread-2-SendThread(localhost:2181)]
zookeeper.ClientCnxn:1029 : Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown
error)
2018-09-30 04:01:45,274 INFO [Thread-2-SendThread(localhost:2181)]
zookeeper.ClientCnxn:1166 : Socket error occurred: localhost/127.0.0.1:2181:
Connection refused
2018-09-30 04:01:45,275 INFO [main] util.log:192 : Logging initialized @92503ms
2018-09-30 04:01:46,021 INFO [main] server.Server:346 : jetty-9.3.22.v20171030
2018-09-30 04:01:46,225 INFO [main] handler.ContextHandler:781 : Started
o.e.j.s.ServletContextHandler@545311c{/,null,AVAILABLE}
2018-09-30 04:01:46,254 INFO [main] server.AbstractConnector:278 : Started
ServerConnector@4bdfeead{HTTP/1.1,[http/1.1]}{0.0.0.0:45238}
2018-09-30 04:01:46,254 INFO [main] server.Server:414 : Started @93482ms
2018-09-30 04:01:46,255 INFO [main] security.KylinUserManager:61 :
Initializing KylinUserManager with config ../examples/test_metadata/
2018-09-30 04:01:46,256 INFO [main] persistence.ResourceStore:88 : Using
metadata url ../examples/test_metadata/ for resource store
2018-09-30 04:01:46,256 DEBUG [main] cachesync.CachedCrudAssist:118 : Reloading
ManagedUser from
/home/travis/build/apache/kylin/server/../examples/test_metadata/user
2018-09-30 04:01:46,257 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded 0
ManagedUser(s) out of 0 resource
2018-09-30 04:01:46,257 INFO [main] common.KylinConfig:432 : Creating new
manager instance of class org.apache.kylin.metadata.cachesync.Broadcaster
2018-09-30 04:01:46,258 DEBUG [main] cachesync.Broadcaster:101 : 1 nodes in the
cluster: [localhost:45238]
2018-09-30 04:01:46,259 INFO [main] security.KylinUserManager:61 :
Initializing KylinUserManager with config ../examples/test_metadata
2018-09-30 04:01:46,259 INFO [main] persistence.ResourceStore:88 : Using
metadata url ../examples/test_metadata for resource store
2018-09-30 04:01:46,259 DEBUG [main] cachesync.CachedCrudAssist:118 : Reloading
ManagedUser from
/home/travis/build/apache/kylin/server/../examples/test_metadata/user
2018-09-30 04:01:46,260 DEBUG [main] cachesync.CachedCrudAssist:127 : Loaded 0
ManagedUser(s) out of 0 resource
2018-09-30 04:01:46,260 INFO [main] common.KylinConfig:432 : Creating new
manager instance of class org.apache.kylin.metadata.cachesync.Broadcaster
2018-09-30 04:01:46,260 DEBUG [main] cachesync.Broadcaster:101 : 1 nodes in the
cluster: [localhost:45238]
2018-09-30 04:01:46,261 DEBUG [main] cachesync.CachedCrudAssist:190 : Saving
ManagedUser at /user/U1
2018-09-30 04:01:46,263 DEBUG [pool-173-thread-1] cachesync.Broadcaster:113 :
Servers in the cluster: [localhost:45238]
2018-09-30 04:01:46,375 INFO [Thread-2-SendThread(localhost:2181)]
zookeeper.ClientCnxn:1029 : Opening socket connection to server
localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown
error)
2018-09-30 04:01:46,376 INFO [Thread-2-SendThread(localhost:2181)]
[jira] [Resolved] (KYLIN-3588) Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
[
https://issues.apache.org/jira/browse/KYLIN-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI resolved KYLIN-3588.
-
Resolution: Fixed
> Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
> ---
>
> Key: KYLIN-3588
> URL: https://issues.apache.org/jira/browse/KYLIN-3588
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
> Fix For: v2.6.0
>
>
> Here is related code:
> {code}
> IRealizationFilter filter = filters.get(conf);
> if (filter == null) {
> try {
> Class clz =
> ClassUtil.forName(conf.getQueryRealizationFilter(), IRealizationFilter.class);
> filter =
> clz.getConstructor(KylinConfig.class).newInstance(conf);
> } catch (Exception e) {
> throw new RuntimeException(e);
> }
> filters.put(conf, filter);
> {code}
> Two threads may find out there is no filter for conf, they would proceed with
> creation of IRealizationFilter instances and try to put into the map.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3588) Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
[
https://issues.apache.org/jira/browse/KYLIN-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633214#comment-16633214
]
ASF GitHub Bot commented on KYLIN-3588:
---
shaofengshi closed pull request #248: KYLIN-3588, fix potentially duplicate
putting in RemoveBlackoutRealiz…
URL: https://github.com/apache/kylin/pull/248
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/query/src/main/java/org/apache/kylin/query/routing/rules/RemoveBlackoutRealizationsRule.java
b/query/src/main/java/org/apache/kylin/query/routing/rules/RemoveBlackoutRealizationsRule.java
index c64f6dae73..278bb46452 100644
---
a/query/src/main/java/org/apache/kylin/query/routing/rules/RemoveBlackoutRealizationsRule.java
+++
b/query/src/main/java/org/apache/kylin/query/routing/rules/RemoveBlackoutRealizationsRule.java
@@ -59,13 +59,16 @@ public static boolean accept(IRealization real) {
private static IRealizationFilter getFilterImpl(KylinConfig conf) {
IRealizationFilter filter = filters.get(conf);
if (filter == null) {
-try {
-Class clz =
ClassUtil.forName(conf.getQueryRealizationFilter(), IRealizationFilter.class);
-filter =
clz.getConstructor(KylinConfig.class).newInstance(conf);
-} catch (Exception e) {
-throw new RuntimeException(e);
+synchronized (RemoveBlackoutRealizationsRule.class) {
+try {
+Class clz =
ClassUtil.forName(conf.getQueryRealizationFilter(),
+IRealizationFilter.class);
+filter =
clz.getConstructor(KylinConfig.class).newInstance(conf);
+} catch (Exception e) {
+throw new RuntimeException(e);
+}
+filters.put(conf, filter);
}
-filters.put(conf, filter);
}
return filter;
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
> ---
>
> Key: KYLIN-3588
> URL: https://issues.apache.org/jira/browse/KYLIN-3588
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
> Fix For: v2.6.0
>
>
> Here is related code:
> {code}
> IRealizationFilter filter = filters.get(conf);
> if (filter == null) {
> try {
> Class clz =
> ClassUtil.forName(conf.getQueryRealizationFilter(), IRealizationFilter.class);
> filter =
> clz.getConstructor(KylinConfig.class).newInstance(conf);
> } catch (Exception e) {
> throw new RuntimeException(e);
> }
> filters.put(conf, filter);
> {code}
> Two threads may find out there is no filter for conf, they would proceed with
> creation of IRealizationFilter instances and try to put into the map.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3588) Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
[
https://issues.apache.org/jira/browse/KYLIN-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3588:
Fix Version/s: v2.6.0
> Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
> ---
>
> Key: KYLIN-3588
> URL: https://issues.apache.org/jira/browse/KYLIN-3588
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
> Fix For: v2.6.0
>
>
> Here is related code:
> {code}
> IRealizationFilter filter = filters.get(conf);
> if (filter == null) {
> try {
> Class clz =
> ClassUtil.forName(conf.getQueryRealizationFilter(), IRealizationFilter.class);
> filter =
> clz.getConstructor(KylinConfig.class).newInstance(conf);
> } catch (Exception e) {
> throw new RuntimeException(e);
> }
> filters.put(conf, filter);
> {code}
> Two threads may find out there is no filter for conf, they would proceed with
> creation of IRealizationFilter instances and try to put into the map.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3588) Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
[
https://issues.apache.org/jira/browse/KYLIN-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633215#comment-16633215
]
ASF subversion and git services commented on KYLIN-3588:
Commit 974bf6f3a971d256e2a93a11942a87143e61db21 in kylin's branch
refs/heads/master from tttMelody
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=974bf6f ]
KYLIN-3588, fix potentially duplicate putting in
RemoveBlackoutRealizationsRule#getFilterImpl
> Potentially duplicate put to RemoveBlackoutRealizationsRule#filters map
> ---
>
> Key: KYLIN-3588
> URL: https://issues.apache.org/jira/browse/KYLIN-3588
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: jiatao.tao
>Priority: Minor
> Fix For: v2.6.0
>
>
> Here is related code:
> {code}
> IRealizationFilter filter = filters.get(conf);
> if (filter == null) {
> try {
> Class clz =
> ClassUtil.forName(conf.getQueryRealizationFilter(), IRealizationFilter.class);
> filter =
> clz.getConstructor(KylinConfig.class).newInstance(conf);
> } catch (Exception e) {
> throw new RuntimeException(e);
> }
> filters.put(conf, filter);
> {code}
> Two threads may find out there is no filter for conf, they would proceed with
> creation of IRealizationFilter instances and try to put into the map.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KYLIN-3581) compareTo method should be paired with equals method
[ https://issues.apache.org/jira/browse/KYLIN-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI resolved KYLIN-3581. - Resolution: Fixed > compareTo method should be paired with equals method > > > Key: KYLIN-3581 > URL: https://issues.apache.org/jira/browse/KYLIN-3581 > Project: Kylin > Issue Type: Bug >Reporter: Ted Yu >Assignee: XiaoXiang Yu >Priority: Minor > Fix For: v2.6.0 > > > DoggedCubeBuilder$MergeSlot and InMemCubeBuilder$CuboidTask define compareTo > method. > They should define equals method as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3581) compareTo method should be paired with equals method
[ https://issues.apache.org/jira/browse/KYLIN-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3581: Fix Version/s: (was: v2.5.1) (was: v2.4.2) v2.6.0 > compareTo method should be paired with equals method > > > Key: KYLIN-3581 > URL: https://issues.apache.org/jira/browse/KYLIN-3581 > Project: Kylin > Issue Type: Bug >Reporter: Ted Yu >Assignee: XiaoXiang Yu >Priority: Minor > Fix For: v2.6.0 > > > DoggedCubeBuilder$MergeSlot and InMemCubeBuilder$CuboidTask define compareTo > method. > They should define equals method as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3581) compareTo method should be paired with equals method
[
https://issues.apache.org/jira/browse/KYLIN-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633211#comment-16633211
]
ASF GitHub Bot commented on KYLIN-3581:
---
shaofengshi closed pull request #249: KYLIN-3581 Obey the contract of
'x.equals(y) should be consistent with x.compareTo(y)'
URL: https://github.com/apache/kylin/pull/249
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/DoggedCubeBuilder.java
b/core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/DoggedCubeBuilder.java
index d761505b2c..06e4a5d582 100644
---
a/core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/DoggedCubeBuilder.java
+++
b/core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/DoggedCubeBuilder.java
@@ -347,6 +347,19 @@ public boolean fetchNext() throws IOException {
}
}
+@Override
+public boolean equals(Object o) {
+if (this == o) return true;
+if (o == null || getClass() != o.getClass()) return false;
+MergeSlot mergeSlot = (MergeSlot) o;
+return compareTo(mergeSlot) == 0;
+}
+
+@Override
+public int hashCode() {
+return Long.hashCode(currentCuboidId);
+}
+
@Override
public int compareTo(MergeSlot o) {
long cuboidComp = this.currentCuboidId - o.currentCuboidId;
diff --git
a/core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/InMemCubeBuilder.java
b/core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/InMemCubeBuilder.java
index db3eb5dd80..ef61ce9a5a 100644
---
a/core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/InMemCubeBuilder.java
+++
b/core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/InMemCubeBuilder.java
@@ -521,5 +521,18 @@ public int compareTo(CuboidTask o) {
long comp = this.childCuboidId - o.childCuboidId;
return comp < 0 ? -1 : (comp > 0 ? 1 : 0);
}
+
+@Override
+public boolean equals(Object o) {
+if (this == o) return true;
+if (o == null || getClass() != o.getClass()) return false;
+CuboidTask that = (CuboidTask) o;
+return compareTo(that) == 0;
+}
+
+@Override
+public int hashCode() {
+return Long.hashCode(childCuboidId);
+}
}
}
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> compareTo method should be paired with equals method
>
>
> Key: KYLIN-3581
> URL: https://issues.apache.org/jira/browse/KYLIN-3581
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: XiaoXiang Yu
>Priority: Minor
> Fix For: v2.6.0
>
>
> DoggedCubeBuilder$MergeSlot and InMemCubeBuilder$CuboidTask define compareTo
> method.
> They should define equals method as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3581) compareTo method should be paired with equals method
[ https://issues.apache.org/jira/browse/KYLIN-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633213#comment-16633213 ] ASF subversion and git services commented on KYLIN-3581: Commit 3342d08ef8f4684cac6fe0bfc7484fbba6aa8fb9 in kylin's branch refs/heads/master from hit-lacus [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=3342d08 ] KYLIN-3581 Obey the contract of 'x.equals(y) should be consistent with x.compareTo(y)' https://issues.apache.org/jira/browse/KYLIN-3581 > compareTo method should be paired with equals method > > > Key: KYLIN-3581 > URL: https://issues.apache.org/jira/browse/KYLIN-3581 > Project: Kylin > Issue Type: Bug >Reporter: Ted Yu >Assignee: XiaoXiang Yu >Priority: Minor > Fix For: v2.6.0 > > > DoggedCubeBuilder$MergeSlot and InMemCubeBuilder$CuboidTask define compareTo > method. > They should define equals method as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3581) compareTo method should be paired with equals method
[ https://issues.apache.org/jira/browse/KYLIN-3581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633212#comment-16633212 ] ASF subversion and git services commented on KYLIN-3581: Commit 3342d08ef8f4684cac6fe0bfc7484fbba6aa8fb9 in kylin's branch refs/heads/master from hit-lacus [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=3342d08 ] KYLIN-3581 Obey the contract of 'x.equals(y) should be consistent with x.compareTo(y)' https://issues.apache.org/jira/browse/KYLIN-3581 > compareTo method should be paired with equals method > > > Key: KYLIN-3581 > URL: https://issues.apache.org/jira/browse/KYLIN-3581 > Project: Kylin > Issue Type: Bug >Reporter: Ted Yu >Assignee: XiaoXiang Yu >Priority: Minor > Fix For: v2.6.0 > > > DoggedCubeBuilder$MergeSlot and InMemCubeBuilder$CuboidTask define compareTo > method. > They should define equals method as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3605:
Description:
HIGH SEVERITY
h1. Arbitrary Code Execution
* Vulnerable module: commons-beanutils:commons-beanutils
* Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
h2. Detailed paths and remediation
* *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
org.apache.hadoop:hadoop-common@2.7.1 ›
commons-configuration:commons-configuration@1.6 ›
commons-digester:commons-digester@1.8 ›
commons-beanutils:commons-beanutils@1.7.0
*Remediation:* No remediation path available.
h2. Overview
[{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
property, which allows remote attackers to manipulate the {{ClassLoader}}and
execute arbitrary code via the class parameter, as demonstrated by the passing
of this parameter to the {{getClass}} method of the \{{ActionForm}}object in
Struts 1.
HIGH SEVERITY
h1. Arbitrary Command Execution
* Vulnerable module: org.mortbay.jetty:jetty
* Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
h2. Detailed paths and remediation
* *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
*Remediation:* No remediation path available.
h2. Overview
[org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
is an open-source project providing a HTTP server, HTTP client and
javax.servlet container.
Affected versions of this package are vulnerable to Arbitrary Command
Execution. It writes backtrace data without sanitizing non-printable
characters, which might allow remote attackers to modify a window's title, or
possibly execute arbitrary commands or overwrite files, via an HTTP request
containing an escape sequence for a terminal emulator, related to (1) a string
value in the Age parameter to the default URI for the Cookie Dump Servlet in
test-jetty-webapp/src/main/java/com/acme/CookieDump.java under cookie/, (2) an
alphabetic value in the A parameter to jsp/expr.jsp, or (3) an alphabetic value
in the Content-Length HTTP header to an arbitrary application.
HIGH SEVERITY
h1. Information Exposure
* Vulnerable module: org.apache.hadoop:hadoop-common
* Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
h2. Detailed paths and remediation
* *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
org.apache.hadoop:hadoop-common@2.7.1
*Remediation:* No remediation path available.
h2. Overview
[{{org.apache.hadoop:hadoop-common}}|https://hadoop.apache.org/] is a framework
that allows for the distributed processing of large data sets across clusters
of computers using simple programming models.
Affected versions of the package are vulnerable to Information Exposure.
If you use the CredentialProvider feature to encrypt passwords used in
NodeManager configs, it may be possible for any Container launched by that
NodeManager to gain access to the encryption password. The other passwords
themselves are not directly exposed.
HIGH SEVERITY
h1. Access Restriction Bypass
* Vulnerable module: org.springframework.security:spring-security-core
* Introduced through: org.apache.kylin:kylin-server-base@2.6.0-SNAPSHOT
h2. Detailed paths and remediation
* *Introduced through*: org.apache.kylin:kylin-server@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-server-base@2.6.0-SNAPSHOT
›org.springframework.security.extensions:spring-security-saml2-core@1.0.2.RELEASE
› org.springframework.security:spring-security-web@4.2.3.RELEASE ›
org.springframework.security:spring-security-core@4.2.3.RELEASE
*Remediation:* No remediation path available.
* *Introduced through*: org.apache.kylin:kylin-server@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-server-base@2.6.0-SNAPSHOT
›org.springframework.security.extensions:spring-security-saml2-core@1.0.2.RELEASE
› org.springframework.security:spring-security-config@4.2.3.RELEASE ›
org.springframework.security:spring-security-core@4.2.3.RELEASE
*Remediation:* No remediation path available.
* *Introduced through*: org.apache.kylin:kylin-server@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-server-base@2.6.0-SNAPSHOT
›org.springframework.security:spring-security-ldap@4.2.3.RELEASE ›
[jira] [Commented] (KYLIN-3583) Integer multiplication is performed where Long result is expected
[
https://issues.apache.org/jira/browse/KYLIN-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633203#comment-16633203
]
ASF GitHub Bot commented on KYLIN-3583:
---
shaofengshi closed pull request #251: KYLIN-3583, KYLIN-3579
URL: https://github.com/apache/kylin/pull/251
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-cube/src/main/java/org/apache/kylin/cube/cuboid/algorithm/BPUSCalculator.java
b/core-cube/src/main/java/org/apache/kylin/cube/cuboid/algorithm/BPUSCalculator.java
index ea91c6cb81..6316858d58 100755
---
a/core-cube/src/main/java/org/apache/kylin/cube/cuboid/algorithm/BPUSCalculator.java
+++
b/core-cube/src/main/java/org/apache/kylin/cube/cuboid/algorithm/BPUSCalculator.java
@@ -60,15 +60,15 @@ protected BPUSCalculator(CuboidStats cuboidStats,
ImmutableMap initC
cuboidAggCostMap.put(cuboid, getCuboidCost(cuboid));
}
}
-Set mandatoryCuboidSetWithStats = cuboidAggCostMap.keySet();
+
//Initialize stats for selection cuboids
long baseCuboidCost = getCuboidCost(cuboidStats.getBaseCuboid());
for (Long cuboid : cuboidStats.getAllCuboidsForSelection()) {
long leastCost = baseCuboidCost;
-for (Long cuboidTarget : mandatoryCuboidSetWithStats) {
-if ((cuboid | cuboidTarget) == cuboidTarget) {
-if (leastCost > cuboidAggCostMap.get(cuboidTarget)) {
-leastCost = cuboidAggCostMap.get(cuboidTarget);
+for (Map.Entry cuboidTargetEntry :
cuboidAggCostMap.entrySet()) {
+if ((cuboid | cuboidTargetEntry.getKey()) ==
cuboidTargetEntry.getKey()) {
+if (leastCost > cuboidTargetEntry.getValue()) {
+leastCost = cuboidTargetEntry.getValue();
}
}
}
@@ -106,9 +106,9 @@ protected BPUSCalculator(CuboidStats cuboidStats,
ImmutableMap initC
}
double totalCostSaving = 0;
int benefitCount = 0;
-for (Long cuboid : cuboidAggCostMapCopy.keySet()) {
-if (cuboidAggCostMapCopy.get(cuboid) <
processCuboidAggCostMap.get(cuboid)) {
-totalCostSaving += processCuboidAggCostMap.get(cuboid) -
cuboidAggCostMapCopy.get(cuboid);
+for (Map.Entry entry : cuboidAggCostMapCopy.entrySet()) {
+if (entry.getValue() <
processCuboidAggCostMap.get(entry.getKey())) {
+totalCostSaving += processCuboidAggCostMap.get(entry.getKey())
- entry.getValue();
benefitCount++;
}
}
diff --git
a/core-cube/src/main/java/org/apache/kylin/cube/gridtable/CubeCodeSystem.java
b/core-cube/src/main/java/org/apache/kylin/cube/gridtable/CubeCodeSystem.java
index 357747663d..4c71fea011 100644
---
a/core-cube/src/main/java/org/apache/kylin/cube/gridtable/CubeCodeSystem.java
+++
b/core-cube/src/main/java/org/apache/kylin/cube/gridtable/CubeCodeSystem.java
@@ -168,13 +168,13 @@ public Object decodeColumnValue(int col, ByteBuffer buf) {
// deal with holistic distinct count
if (dependentMetricsMap != null) {
-for (Integer child : dependentMetricsMap.keySet()) {
-if (columns.get(child)) {
-Integer parent = dependentMetricsMap.get(child);
+for (Map.Entry childEntry :
dependentMetricsMap.entrySet()) {
+if (columns.get(childEntry.getKey())) {
+Integer parent = childEntry.getValue();
if (columns.get(parent) == false)
throw new IllegalStateException();
-int childIdx = columns.trueBitIndexOf(child);
+int childIdx = columns.trueBitIndexOf(childEntry.getKey());
int parentIdx = columns.trueBitIndexOf(parent);
result[childIdx].setDependentAggregator(result[parentIdx]);
}
diff --git
a/core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/RecordEventTimeDetail.java
b/core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/RecordEventTimeDetail.java
index c611d0f943..827b5a10e1 100644
---
a/core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/RecordEventTimeDetail.java
+++
b/core-metrics/src/main/java/org/apache/kylin/metrics/lib/impl/RecordEventTimeDetail.java
@@ -71,7 +71,7 @@ public RecordEventTimeDetail(long timeStamp) {
this.second = calendar.get(Calendar.SECOND);
long timeStampForWeekBegin = timeStamp;
-timeStampForWeekBegin -= 360 * 24 *
(calendar.get(Calendar.DAY_OF_WEEK) - 1);
+timeStampForWeekBegin -= 360L * 24 *
[jira] [Commented] (KYLIN-3583) Integer multiplication is performed where Long result is expected
[
https://issues.apache.org/jira/browse/KYLIN-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633204#comment-16633204
]
ASF subversion and git services commented on KYLIN-3583:
Commit aefaf9cccef7a21d96912ac55cf622d452d4d05c in kylin's branch
refs/heads/master from chao long
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=aefaf9c ]
KYLIN-3583 Cast to long before multiplication
> Integer multiplication is performed where Long result is expected
> -
>
> Key: KYLIN-3583
> URL: https://issues.apache.org/jira/browse/KYLIN-3583
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> In RecordEventTimeDetail :
> {code}
> long timeStampForWeekBegin = timeStamp;
> timeStampForWeekBegin -= 360 * 24 *
> (calendar.get(Calendar.DAY_OF_WEEK) - 1);
> {code}
> Casting to long should be done first before multiplication.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3583) Integer multiplication is performed where Long result is expected
[
https://issues.apache.org/jira/browse/KYLIN-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633199#comment-16633199
]
ASF GitHub Bot commented on KYLIN-3583:
---
codecov-io commented on issue #251: KYLIN-3583, KYLIN-3579
URL: https://github.com/apache/kylin/pull/251#issuecomment-425690030
# [Codecov](https://codecov.io/gh/apache/kylin/pull/251?src=pr=h1) Report
> :exclamation: No coverage uploaded for pull request base
(`master@f42e937`). [Click here to learn what that
means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `0%`.
[](https://codecov.io/gh/apache/kylin/pull/251?src=pr=tree)
```diff
@@Coverage Diff@@
## master #251 +/- ##
=
Coverage ? 21.15%
Complexity? 4404
=
Files ? 1086
Lines ?69801
Branches ?10110
=
Hits ?14763
Misses?53638
Partials ? 1400
```
| [Impacted
Files](https://codecov.io/gh/apache/kylin/pull/251?src=pr=tree) | Coverage Δ
| Complexity Δ | |
|---|---|---|---|
|
[.../kylin/metrics/lib/impl/RecordEventTimeDetail.java](https://codecov.io/gh/apache/kylin/pull/251/diff?src=pr=tree#diff-Y29yZS1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9tZXRyaWNzL2xpYi9pbXBsL1JlY29yZEV2ZW50VGltZURldGFpbC5qYXZh)
| `0% <0%> (ø)` | `0 <0> (?)` | |
|
[...rg/apache/kylin/cube/gridtable/CubeCodeSystem.java](https://codecov.io/gh/apache/kylin/pull/251/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2dyaWR0YWJsZS9DdWJlQ29kZVN5c3RlbS5qYXZh)
| `25.37% <0%> (ø)` | `8 <0> (?)` | |
|
[...he/kylin/cube/cuboid/algorithm/BPUSCalculator.java](https://codecov.io/gh/apache/kylin/pull/251/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9hbGdvcml0aG0vQlBVU0NhbGN1bGF0b3IuamF2YQ==)
| `0% <0%> (ø)` | `0 <0> (?)` | |
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/kylin/pull/251?src=pr=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/kylin/pull/251?src=pr=footer). Last
update
[f42e937...6bdbe40](https://codecov.io/gh/apache/kylin/pull/251?src=pr=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Integer multiplication is performed where Long result is expected
> -
>
> Key: KYLIN-3583
> URL: https://issues.apache.org/jira/browse/KYLIN-3583
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> In RecordEventTimeDetail :
> {code}
> long timeStampForWeekBegin = timeStamp;
> timeStampForWeekBegin -= 360 * 24 *
> (calendar.get(Calendar.DAY_OF_WEEK) - 1);
> {code}
> Casting to long should be done first before multiplication.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3605:
Description:
HIGH SEVERITY
h1. Arbitrary Code Execution
* Vulnerable module: commons-beanutils:commons-beanutils
* Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
h2. Detailed paths and remediation
* *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
org.apache.hadoop:hadoop-common@2.7.1 ›
commons-configuration:commons-configuration@1.6 ›
commons-digester:commons-digester@1.8 ›
commons-beanutils:commons-beanutils@1.7.0
*Remediation:* No remediation path available.
h2. Overview
[{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
property, which allows remote attackers to manipulate the {{ClassLoader}}and
execute arbitrary code via the class parameter, as demonstrated by the passing
of this parameter to the {{getClass}} method of the \{{ActionForm}}object in
Struts 1.
HIGH SEVERITY
h1. Arbitrary Command Execution
* Vulnerable module: org.mortbay.jetty:jetty
* Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
h2. Detailed paths and remediation
* *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
*Remediation:* No remediation path available.
h2. Overview
[org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
is an open-source project providing a HTTP server, HTTP client and
javax.servlet container.
Affected versions of this package are vulnerable to Arbitrary Command
Execution. It writes backtrace data without sanitizing non-printable
characters, which might allow remote attackers to modify a window's title, or
possibly execute arbitrary commands or overwrite files, via an HTTP request
containing an escape sequence for a terminal emulator, related to (1) a string
value in the Age parameter to the default URI for the Cookie Dump Servlet in
test-jetty-webapp/src/main/java/com/acme/CookieDump.java under cookie/, (2) an
alphabetic value in the A parameter to jsp/expr.jsp, or (3) an alphabetic value
in the Content-Length HTTP header to an arbitrary application.
HIGH SEVERITY
h1. Information Exposure
* Vulnerable module: org.apache.hadoop:hadoop-common
* Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
h2. Detailed paths and remediation
* *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
org.apache.hadoop:hadoop-common@2.7.1
*Remediation:* No remediation path available.
h2. Overview
[{{org.apache.hadoop:hadoop-common}}|https://hadoop.apache.org/] is a framework
that allows for the distributed processing of large data sets across clusters
of computers using simple programming models.
Affected versions of the package are vulnerable to Information Exposure.
If you use the CredentialProvider feature to encrypt passwords used in
NodeManager configs, it may be possible for any Container launched by that
NodeManager to gain access to the encryption password. The other passwords
themselves are not directly exposed.
HIGH SEVERITY
h1. Access Restriction Bypass
* Vulnerable module: org.springframework.security:spring-security-core
* Introduced through: org.apache.kylin:kylin-server-base@2.6.0-SNAPSHOT
h2. Detailed paths and remediation
*
*Introduced through*: org.apache.kylin:kylin-server@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-server-base@2.6.0-SNAPSHOT
›org.springframework.security.extensions:spring-security-saml2-core@1.0.2.RELEASE
› org.springframework.security:spring-security-web@4.2.3.RELEASE ›
org.springframework.security:spring-security-core@4.2.3.RELEASE
*Remediation:* No remediation path available.
*
*Introduced through*: org.apache.kylin:kylin-server@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-server-base@2.6.0-SNAPSHOT
›org.springframework.security.extensions:spring-security-saml2-core@1.0.2.RELEASE
› org.springframework.security:spring-security-config@4.2.3.RELEASE ›
org.springframework.security:spring-security-core@4.2.3.RELEASE
*Remediation:* No remediation path available.
*
*Introduced through*: org.apache.kylin:kylin-server@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-server-base@2.6.0-SNAPSHOT
›org.springframework.security:spring-security-ldap@4.2.3.RELEASE ›
[jira] [Issue Comment Deleted] (KYLIN-3590) Missing argument to exception message String in ModelController
[
https://issues.apache.org/jira/browse/KYLIN-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yichen Zhou updated KYLIN-3590:
---
Comment: was deleted
(was: I will fix this.)
> Missing argument to exception message String in ModelController
> ---
>
> Key: KYLIN-3590
> URL: https://issues.apache.org/jira/browse/KYLIN-3590
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Yiming Liu
>Priority: Major
> Fix For: v2.5.1
>
>
> Here is related code:
> {code}
> String.format(Locale.ROOT,
> "Invalid model name %s, only letters, numbers and
> underscore " + "supported."),
> {code}
> There is no argument for '%s'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KYLIN-3590) Missing argument to exception message String in ModelController
[
https://issues.apache.org/jira/browse/KYLIN-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yichen Zhou resolved KYLIN-3590.
Resolution: Fixed
Assignee: Yiming Liu (was: Yichen Zhou)
Fix Version/s: v2.5.1
Fixed by [https://github.com/apache/kylin/pull/270.] Already merged.
> Missing argument to exception message String in ModelController
> ---
>
> Key: KYLIN-3590
> URL: https://issues.apache.org/jira/browse/KYLIN-3590
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Yiming Liu
>Priority: Major
> Fix For: v2.5.1
>
>
> Here is related code:
> {code}
> String.format(Locale.ROOT,
> "Invalid model name %s, only letters, numbers and
> underscore " + "supported."),
> {code}
> There is no argument for '%s'
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (KYLIN-3589) Different type is used to check presence in Collection in AclPermissionUtil
[
https://issues.apache.org/jira/browse/KYLIN-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yichen Zhou resolved KYLIN-3589.
Resolution: Fixed
Assignee: Yiming Liu (was: Yichen Zhou)
Fix Version/s: v2.6.0
> Different type is used to check presence in Collection in AclPermissionUtil
> ---
>
> Key: KYLIN-3589
> URL: https://issues.apache.org/jira/browse/KYLIN-3589
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Yiming Liu
>Priority: Major
> Fix For: v2.6.0
>
>
> Here is related code:
> {code}
> public static List transformAuthorities(Collection GrantedAuthority> authorities) {
> List ret = new ArrayList();
> for (GrantedAuthority auth : authorities) {
> if (!authorities.contains(auth.getAuthority())) {
> {code}
> auth.getAuthority() shouldn't be used to check presence in the Collection
> since it is not of GrantedAuthority type.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3589) Different type is used to check presence in Collection in AclPermissionUtil
[
https://issues.apache.org/jira/browse/KYLIN-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633191#comment-16633191
]
Yichen Zhou commented on KYLIN-3589:
Fixed by [https://github.com/apache/kylin/pull/270.] Already merged.
> Different type is used to check presence in Collection in AclPermissionUtil
> ---
>
> Key: KYLIN-3589
> URL: https://issues.apache.org/jira/browse/KYLIN-3589
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Yichen Zhou
>Priority: Major
>
> Here is related code:
> {code}
> public static List transformAuthorities(Collection GrantedAuthority> authorities) {
> List ret = new ArrayList();
> for (GrantedAuthority auth : authorities) {
> if (!authorities.contains(auth.getAuthority())) {
> {code}
> auth.getAuthority() shouldn't be used to check presence in the Collection
> since it is not of GrantedAuthority type.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (KYLIN-3589) Different type is used to check presence in Collection in AclPermissionUtil
[
https://issues.apache.org/jira/browse/KYLIN-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yichen Zhou updated KYLIN-3589:
---
Comment: was deleted
(was: I will fix this.)
> Different type is used to check presence in Collection in AclPermissionUtil
> ---
>
> Key: KYLIN-3589
> URL: https://issues.apache.org/jira/browse/KYLIN-3589
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Yichen Zhou
>Priority: Major
>
> Here is related code:
> {code}
> public static List transformAuthorities(Collection GrantedAuthority> authorities) {
> List ret = new ArrayList();
> for (GrantedAuthority auth : authorities) {
> if (!authorities.contains(auth.getAuthority())) {
> {code}
> auth.getAuthority() shouldn't be used to check presence in the Collection
> since it is not of GrantedAuthority type.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-2690) Upgrade calcite to release 1.18
[ https://issues.apache.org/jira/browse/KYLIN-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-2690: -- Summary: Upgrade calcite to release 1.18 (was: Upgrade calcite to release 1.17) > Upgrade calcite to release 1.18 > --- > > Key: KYLIN-2690 > URL: https://issues.apache.org/jira/browse/KYLIN-2690 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Priority: Major > > Calcite 1.16 has been released. > We should consider upgrading to this release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-2690) Upgrade calcite to release 1.18
[ https://issues.apache.org/jira/browse/KYLIN-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-2690: -- Description: Calcite 1.18 has been released. We should consider upgrading to this release. was: Calcite 1.16 has been released. We should consider upgrading to this release. > Upgrade calcite to release 1.18 > --- > > Key: KYLIN-2690 > URL: https://issues.apache.org/jira/browse/KYLIN-2690 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Priority: Major > > Calcite 1.18 has been released. > We should consider upgrading to this release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3272) Upgrade Spark dependency to 2.3.2
[ https://issues.apache.org/jira/browse/KYLIN-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-3272: -- Description: Currently Spark 2.1.2 is used. Spark 2.3.2 was released. We should upgrade the dependency to 2.3.2 https://sematext.com/opensee/m/Spark/q3RTtb4Q9LI4F6F?subj=+ANNOUNCE+Announcing+Apache+Spark+2+3+2 was: Currently Spark 2.1.2 is used. Spark 2.3.1 was released. We should upgrade the dependency to 2.3.1 > Upgrade Spark dependency to 2.3.2 > - > > Key: KYLIN-3272 > URL: https://issues.apache.org/jira/browse/KYLIN-3272 > Project: Kylin > Issue Type: Improvement > Components: Spark Engine >Reporter: Ted Yu >Priority: Minor > > Currently Spark 2.1.2 is used. > Spark 2.3.2 was released. > We should upgrade the dependency to 2.3.2 > https://sematext.com/opensee/m/Spark/q3RTtb4Q9LI4F6F?subj=+ANNOUNCE+Announcing+Apache+Spark+2+3+2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633174#comment-16633174
]
ASF GitHub Bot commented on KYLIN-3605:
---
codecov-io commented on issue #276: KYLIN-3605 upgrade hadoop-common and
zookeeper version to fix securit…
URL: https://github.com/apache/kylin/pull/276#issuecomment-425686030
# [Codecov](https://codecov.io/gh/apache/kylin/pull/276?src=pr=h1) Report
> :exclamation: No coverage uploaded for pull request base
(`master@f42e937`). [Click here to learn what that
means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `0%`.
[](https://codecov.io/gh/apache/kylin/pull/276?src=pr=tree)
```diff
@@Coverage Diff@@
## master #276 +/- ##
=
Coverage ? 21.09%
Complexity? 4405
=
Files ? 1087
Lines ?69965
Branches ?10120
=
Hits ?14761
Misses?53804
Partials ? 1400
```
| [Impacted
Files](https://codecov.io/gh/apache/kylin/pull/276?src=pr=tree) | Coverage Δ
| Complexity Δ | |
|---|---|---|---|
|
[...ylin/common/metrics/metrics2/Metrics2Reporter.java](https://codecov.io/gh/apache/kylin/pull/276/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9tZXRyaWNzL21ldHJpY3MyL01ldHJpY3MyUmVwb3J0ZXIuamF2YQ==)
| `0% <ø> (ø)` | `0 <0> (?)` | |
|
[...ommon/metrics/metrics2/HadoopMetrics2Reporter.java](https://codecov.io/gh/apache/kylin/pull/276/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9tZXRyaWNzL21ldHJpY3MyL0hhZG9vcE1ldHJpY3MyUmVwb3J0ZXIuamF2YQ==)
| `0% <0%> (ø)` | `0 <0> (?)` | |
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/kylin/pull/276?src=pr=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/kylin/pull/276?src=pr=footer). Last
update
[f42e937...0ae4686](https://codecov.io/gh/apache/kylin/pull/276?src=pr=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Fix security issues reported by snyk.io
> ---
>
> Key: KYLIN-3605
> URL: https://issues.apache.org/jira/browse/KYLIN-3605
> Project: Kylin
> Issue Type: Improvement
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.6.0
>
>
> HIGH SEVERITY
> h1. Arbitrary Code Execution
> * Vulnerable module: commons-beanutils:commons-beanutils
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›
> commons-configuration:commons-configuration@1.6 ›
> commons-digester:commons-digester@1.8 ›
> commons-beanutils:commons-beanutils@1.7.0
> *Remediation:* No remediation path available.
> h2. Overview
> [{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
> Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
> property, which allows remote attackers to manipulate the {{ClassLoader}}and
> execute arbitrary code via the class parameter, as demonstrated by the
> passing of this parameter to the {{getClass}} method of the
> {{ActionForm}}object in Struts 1.
>
> HIGH SEVERITY
> h1. Arbitrary Command Execution
> * Vulnerable module: org.mortbay.jetty:jetty
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
>
[jira] [Assigned] (KYLIN-3560) Should not depend on personal repository
[
https://issues.apache.org/jira/browse/KYLIN-3560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-3560:
---
Assignee: Shaofeng SHI (was: Chao Long)
> Should not depend on personal repository
>
>
> Key: KYLIN-3560
> URL: https://issues.apache.org/jira/browse/KYLIN-3560
> Project: Kylin
> Issue Type: Task
>Reporter: Ted Yu
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.6.0
>
>
> In core-common/pom.xml:
> {code}
> com.github.joshelser
> {code}
> We shouldn't depend on personal repository.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3599) Bulk Add Measures
[ https://issues.apache.org/jira/browse/KYLIN-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pan, Julian updated KYLIN-3599: --- Description: If user want to add hundreds measure, user need add them one by one in currently website. Add a bulk add button to resolve this issues in measure step. P.S just include common measure (sum, max, min, raw, percentile), due to count_distinct, topN and extended_column need more setting. Image attached and the default name generate automatically by pattern [measure_columnName]. was: If user want to add hundreds measure, user need add them one by one in currently website. Add a bulk add button to resolve this issues in measure step. P.S just include common measure (sum, max, min, raw, percentile), due to count_distinct, topN and extended_column need more setting. Image attached and the default name generate automatically by pattern [measure_] > Bulk Add Measures > - > > Key: KYLIN-3599 > URL: https://issues.apache.org/jira/browse/KYLIN-3599 > Project: Kylin > Issue Type: Improvement > Components: Web , Website >Reporter: Pan, Julian >Assignee: Pan, Julian >Priority: Major > Fix For: v2.5.1 > > Attachments: bluk add button.png, select all or clean.png > > > If user want to add hundreds measure, user need add them one by one in > currently website. > Add a bulk add button to resolve this issues in measure step. > P.S just include common measure (sum, max, min, raw, percentile), due to > count_distinct, topN and extended_column need more setting. Image attached > and the default name generate automatically by pattern [measure_columnName]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3599) Bulk Add Measures
[ https://issues.apache.org/jira/browse/KYLIN-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pan, Julian updated KYLIN-3599: --- Description: If user want to add hundreds measure, user need add them one by one in currently website. Add a bulk add button to resolve this issues in measure step. P.S just include common measure (sum, max, min, raw, percentile), due to count_distinct, topN and extended_column need more setting. Image attached and the default name generate automatically by pattern [measure_] was: If user want to add hundreds measure, user need add them one by one in currently website. Add a bulk add button to resolve this issues in measure step. P.S just include common measure (sum, max, min, raw, percentile), due to count_distinct, topN and extended_column need more setting. > Bulk Add Measures > - > > Key: KYLIN-3599 > URL: https://issues.apache.org/jira/browse/KYLIN-3599 > Project: Kylin > Issue Type: Improvement > Components: Web , Website >Reporter: Pan, Julian >Assignee: Pan, Julian >Priority: Major > Fix For: v2.5.1 > > Attachments: bluk add button.png, select all or clean.png > > > If user want to add hundreds measure, user need add them one by one in > currently website. > Add a bulk add button to resolve this issues in measure step. > P.S just include common measure (sum, max, min, raw, percentile), due to > count_distinct, topN and extended_column need more setting. Image attached > and the default name generate automatically by pattern [measure_] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633162#comment-16633162
]
Shaofeng SHI commented on KYLIN-3605:
-
If upgrade hadoop version from 2.7.1 to 2.7.7, the UT will fail:
[ERROR] Failures:
[ERROR] QueryMetricsTest.testQueryMetrics:80 expected:<1> but was:<0>
[INFO]
> Fix security issues reported by snyk.io
> ---
>
> Key: KYLIN-3605
> URL: https://issues.apache.org/jira/browse/KYLIN-3605
> Project: Kylin
> Issue Type: Improvement
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.6.0
>
>
> HIGH SEVERITY
> h1. Arbitrary Code Execution
> * Vulnerable module: commons-beanutils:commons-beanutils
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›
> commons-configuration:commons-configuration@1.6 ›
> commons-digester:commons-digester@1.8 ›
> commons-beanutils:commons-beanutils@1.7.0
> *Remediation:* No remediation path available.
> h2. Overview
> [{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
> Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
> property, which allows remote attackers to manipulate the {{ClassLoader}}and
> execute arbitrary code via the class parameter, as demonstrated by the
> passing of this parameter to the {{getClass}} method of the
> {{ActionForm}}object in Struts 1.
>
> HIGH SEVERITY
> h1. Arbitrary Command Execution
> * Vulnerable module: org.mortbay.jetty:jetty
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
> *Remediation:* No remediation path available.
> h2. Overview
> [org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
> is an open-source project providing a HTTP server, HTTP client and
> javax.servlet container.
> Affected versions of this package are vulnerable to Arbitrary Command
> Execution. It writes backtrace data without sanitizing non-printable
> characters, which might allow remote attackers to modify a window's title, or
> possibly execute arbitrary commands or overwrite files, via an HTTP request
> containing an escape sequence for a terminal emulator, related to (1) a
> string value in the Age parameter to the default URI for the Cookie Dump
> Servlet in test-jetty-webapp/src/main/java/com/acme/CookieDump.java under
> cookie/, (2) an alphabetic value in the A parameter to jsp/expr.jsp, or (3)
> an alphabetic value in the Content-Length HTTP header to an arbitrary
> application.
> HIGH SEVERITY
> h1. Information Exposure
> * Vulnerable module: org.apache.hadoop:hadoop-common
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1
> *Remediation:* No remediation path available.
> h2. Overview
> [{{org.apache.hadoop:hadoop-common}}|https://hadoop.apache.org/] is a
> framework that allows for the distributed processing of large data sets
> across clusters of computers using simple programming models.
> Affected versions of the package are vulnerable to Information Exposure.
> If you use the CredentialProvider feature to encrypt passwords used in
> NodeManager configs, it may be possible for any Container launched by that
> NodeManager to gain access to the encryption password. The other passwords
> themselves are not directly exposed.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3519) Upgrade Jacoco version to 0.8.2
[ https://issues.apache.org/jira/browse/KYLIN-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-3519: -- Description: Jacoco 0.8.2 adds Java 11 support: https://github.com/jacoco/jacoco/releases/tag/v0.8.2 Java 11 RC1 is out. We should consider upgrading Jacoco. was: Jacoco 0.8.2 adds Java 11 support: https://github.com/jacoco/jacoco/releases/tag/v0.8.2 Java 11 RC1 is out. We should consider upgrading Jacoco. > Upgrade Jacoco version to 0.8.2 > --- > > Key: KYLIN-3519 > URL: https://issues.apache.org/jira/browse/KYLIN-3519 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Priority: Minor > > Jacoco 0.8.2 adds Java 11 support: >https://github.com/jacoco/jacoco/releases/tag/v0.8.2 > Java 11 RC1 is out. > We should consider upgrading Jacoco. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3572) Upgrade commons-dbcp to latest commons-dbcp2
[ https://issues.apache.org/jira/browse/KYLIN-3572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-3572: -- Description: There have been many improvements and a major release in commons-dbcp. The latest version is 2.5.0. This Jira proposes to upgrade the version of commons-dbcp to commons-dbcp2 version 2.5.0 in order to benefit from the improvements/additions. This will involve some code changes in addition to the updated dependencies, as the API has changed slightly to use different terminology was: There have been many improvements and a major release in commons-dbcp. The latest version is 2.5.0. This Jira proposes to upgrade the version of commons-dbcp to commons-dbcp2 version 2.5.0 in order to benefit from the improvements/additions. This will involve some code changes in addition to the updated dependencies, as the API has changed slightly to use different terminology > Upgrade commons-dbcp to latest commons-dbcp2 > > > Key: KYLIN-3572 > URL: https://issues.apache.org/jira/browse/KYLIN-3572 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Priority: Major > > There have been many improvements and a major release in commons-dbcp. The > latest version is 2.5.0. > This Jira proposes to upgrade the version of commons-dbcp to commons-dbcp2 > version 2.5.0 in order to benefit from the improvements/additions. This will > involve some code changes in addition to the updated dependencies, as the API > has changed slightly to use different terminology -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3573) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/KYLIN-3573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-3573: -- Description: Apache parent POM 21 has just been released. It generates .sha512 checksums during release with Maven Release Plugin. We should consider upgrading parent POM. was: Apache parent POM 21 has just been released. It generates .sha512 checksums during release with Maven Release Plugin. We should consider upgrading parent POM. > Upgrade Apache parent POM to version 21 > --- > > Key: KYLIN-3573 > URL: https://issues.apache.org/jira/browse/KYLIN-3573 > Project: Kylin > Issue Type: Task >Reporter: Ted Yu >Priority: Major > > Apache parent POM 21 has just been released. It generates .sha512 checksums > during release with Maven Release Plugin. > We should consider upgrading parent POM. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3523) Upgrade hadoop and hbase dependencies in master-hadoop3.1 branch
[ https://issues.apache.org/jira/browse/KYLIN-3523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated KYLIN-3523: -- Description: Hadoop 3.1.1 has been released as stable. hbase 2.0.2 vote has passed. We should upgrade to these two versions in master-hadoop3.1 branch. was: Hadoop 3.1.1 has been released as stable. hbase 2.0.2 vote seems to pass. We should upgrade to these two versions in master-hadoop3.1 branch. > Upgrade hadoop and hbase dependencies in master-hadoop3.1 branch > > > Key: KYLIN-3523 > URL: https://issues.apache.org/jira/browse/KYLIN-3523 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Priority: Major > > Hadoop 3.1.1 has been released as stable. > hbase 2.0.2 vote has passed. > We should upgrade to these two versions in master-hadoop3.1 branch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3602) Enable more checkstyle rules
[ https://issues.apache.org/jira/browse/KYLIN-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633159#comment-16633159 ] ASF GitHub Bot commented on KYLIN-3602: --- shaofengshi closed pull request #277: KYLIN-3602 Enable more checkstyle rules URL: https://github.com/apache/kylin/pull/277 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/dev-support/checkstyle-suppressions.xml b/dev-support/checkstyle-suppressions.xml index 06de62554f..f110b7c3ef 100644 --- a/dev-support/checkstyle-suppressions.xml +++ b/dev-support/checkstyle-suppressions.xml @@ -25,4 +25,40 @@ + + + + + + + + + + + + + + + + + + + + + + + diff --git a/dev-support/checkstyle.xml b/dev-support/checkstyle.xml index 45325bf950..d8eb73fc89 100644 --- a/dev-support/checkstyle.xml +++ b/dev-support/checkstyle.xml @@ -28,11 +28,31 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Enable more checkstyle rules > > > Key: KYLIN-3602 > URL: https://issues.apache.org/jira/browse/KYLIN-3602 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Yichen Zhou >Priority: Major > Fix For: v2.6.0 > > Attachments: checkstyle-aggregate.html > > > The checkstyle rules of kylin are too weak. We need to reinfore them to > achieve better code quality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3602) Enable more checkstyle rules
[ https://issues.apache.org/jira/browse/KYLIN-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633160#comment-16633160 ] ASF subversion and git services commented on KYLIN-3602: Commit f42e937a2e968f94d8484f7bdafa7a86903eaa58 in kylin's branch refs/heads/master from [~yichen.zhou] [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=f42e937 ] KYLIN-3602 Enable more checkstyle rules > Enable more checkstyle rules > > > Key: KYLIN-3602 > URL: https://issues.apache.org/jira/browse/KYLIN-3602 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Yichen Zhou >Priority: Major > Fix For: v2.6.0 > > Attachments: checkstyle-aggregate.html > > > The checkstyle rules of kylin are too weak. We need to reinfore them to > achieve better code quality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3602) Enable more checkstyle rules
[ https://issues.apache.org/jira/browse/KYLIN-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633007#comment-16633007 ] ASF GitHub Bot commented on KYLIN-3602: --- codecov-io commented on issue #277: KYLIN-3602 Enable more checkstyle rules URL: https://github.com/apache/kylin/pull/277#issuecomment-425644108 # [Codecov](https://codecov.io/gh/apache/kylin/pull/277?src=pr=h1) Report > Merging [#277](https://codecov.io/gh/apache/kylin/pull/277?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/bd8961c884cdc8ec5a9903a1d5a317bb626a2dff?src=pr=desc) will **increase** coverage by `0.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/kylin/pull/277?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #277 +/- ## + Coverage 21.14% 21.15% +0.01% - Complexity 4405 4406 +1 Files 1086 1086 Lines 6980269802 Branches 1011010110 + Hits 1475914768 +9 + Misses5364253637 -5 + Partials 1401 1397 -4 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/277?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/277/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh) | `70.21% <0%> (+0.6%)` | `7% <0%> (ø)` | :arrow_down: | | [...g/apache/kylin/source/datagen/ColumnGenerator.java](https://codecov.io/gh/apache/kylin/pull/277/diff?src=pr=tree#diff-Y29yZS1tZXRhZGF0YS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUva3lsaW4vc291cmNlL2RhdGFnZW4vQ29sdW1uR2VuZXJhdG9yLmphdmE=) | `72.29% <0%> (+1.35%)` | `8% <0%> (ø)` | :arrow_down: | | [.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/277/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=) | `66.15% <0%> (+2.3%)` | `0% <0%> (ø)` | :arrow_down: | | [...he/kylin/dict/lookup/cache/RocksDBLookupTable.java](https://codecov.io/gh/apache/kylin/pull/277/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGUuamF2YQ==) | `78.37% <0%> (+5.4%)` | `6% <0%> (+1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/277?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/277?src=pr=footer). Last update [bd8961c...fefc0ec](https://codecov.io/gh/apache/kylin/pull/277?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Enable more checkstyle rules > > > Key: KYLIN-3602 > URL: https://issues.apache.org/jira/browse/KYLIN-3602 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Yichen Zhou >Priority: Major > Fix For: v2.6.0 > > Attachments: checkstyle-aggregate.html > > > The checkstyle rules of kylin are too weak. We need to reinfore them to > achieve better code quality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3602) Enable more checkstyle rules
[ https://issues.apache.org/jira/browse/KYLIN-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633006#comment-16633006 ] ASF GitHub Bot commented on KYLIN-3602: --- coveralls commented on issue #277: KYLIN-3602 Enable more checkstyle rules URL: https://github.com/apache/kylin/pull/277#issuecomment-425644083 ## Pull Request Test Coverage Report for [Build 3719](https://coveralls.io/builds/19261940) * **0** of **0** changed or added relevant lines in **0** files are covered. * No unchanged relevant lines lost coverage. * Overall coverage increased (+**0.007%**) to **23.161%** --- | Totals | [](https://coveralls.io/builds/19261940) | | :-- | --: | | Change from base [Build 3717](https://coveralls.io/builds/19261333): | 0.007% | | Covered Lines: | 16167 | | Relevant Lines: | 69802 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Enable more checkstyle rules > > > Key: KYLIN-3602 > URL: https://issues.apache.org/jira/browse/KYLIN-3602 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Yichen Zhou >Priority: Major > Fix For: v2.6.0 > > Attachments: checkstyle-aggregate.html > > > The checkstyle rules of kylin are too weak. We need to reinfore them to > achieve better code quality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2892) Make Kylin compile with Java 9
[
https://issues.apache.org/jira/browse/KYLIN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16633000#comment-16633000
]
Billy Liu commented on KYLIN-2892:
--
Since Hadoop 2.7.x is not compatible with Java 9+, Kylin has dependency with
Hadoop 2.7.x. So have to postpone the JDK upgrade to 9.
> Make Kylin compile with Java 9
> --
>
> Key: KYLIN-2892
> URL: https://issues.apache.org/jira/browse/KYLIN-2892
> Project: Kylin
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Major
>
> When I attempted building with Java 9, I got:
> {code}
> [ERROR] Failed to execute goal on project kylin-engine-mr: Could not resolve
> dependencies for project org.apache.kylin:kylin-engine-mr:jar:2.2.0-SNAPSHOT:
> Could not find artifact jdk.tools:jdk.tools:jar:1.7 at specified path
> /jdk-9/../lib/tools.jar -> [Help 1]
> {code}
> The dependency seems to come from kylin-engine-mr module.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3602) Enable more checkstyle rules
[ https://issues.apache.org/jira/browse/KYLIN-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632993#comment-16632993 ] ASF GitHub Bot commented on KYLIN-3602: --- ZhouYC627 opened a new pull request #277: KYLIN-3602 Enable more checkstyle rules URL: https://github.com/apache/kylin/pull/277 Enable more checkstyle rules and suppress the checkstyle errors of old files in _dev-support/checkstyle-suppressions.xml_ https://issues.apache.org/jira/browse/KYLIN-3602 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Enable more checkstyle rules > > > Key: KYLIN-3602 > URL: https://issues.apache.org/jira/browse/KYLIN-3602 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Yichen Zhou >Priority: Major > Fix For: v2.6.0 > > Attachments: checkstyle-aggregate.html > > > The checkstyle rules of kylin are too weak. We need to reinfore them to > achieve better code quality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3602) Enable more checkstyle rules
[ https://issues.apache.org/jira/browse/KYLIN-3602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632994#comment-16632994 ] ASF GitHub Bot commented on KYLIN-3602: --- asfgit commented on issue #277: KYLIN-3602 Enable more checkstyle rules URL: https://github.com/apache/kylin/pull/277#issuecomment-425642269 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Enable more checkstyle rules > > > Key: KYLIN-3602 > URL: https://issues.apache.org/jira/browse/KYLIN-3602 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Yichen Zhou >Priority: Major > Fix For: v2.6.0 > > Attachments: checkstyle-aggregate.html > > > The checkstyle rules of kylin are too weak. We need to reinfore them to > achieve better code quality. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3334) Prepare for Java 10
[
https://issues.apache.org/jira/browse/KYLIN-3334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632980#comment-16632980
]
Billy Liu commented on KYLIN-3334:
--
The compile fails since JDK 9, the same error message.
> Prepare for Java 10
> ---
>
> Key: KYLIN-3334
> URL: https://issues.apache.org/jira/browse/KYLIN-3334
> Project: Kylin
> Issue Type: Task
>Reporter: Ted Yu
>Priority: Major
>
> When compiling with Java 10 , MapReduce Engine module fails with
> {code}
> [ERROR] Failed to execute goal on project kylin-engine-mr: Could not resolve
> dependencies for project org.apache.kylin:kylin-engine-mr:jar:2.4.0-SNAPSHOT:
> Could not find artifact jdk.tools:jdk.tools:jar:1.7 at specified path
> /a/jdk-10/../lib/tools.jar -> [Help 1]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (KYLIN-3604) Can't build cube with spark in HBase standalone mode
[
https://issues.apache.org/jira/browse/KYLIN-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-3604:
---
Assignee: Yichen Zhou
> Can't build cube with spark in HBase standalone mode
>
>
> Key: KYLIN-3604
> URL: https://issues.apache.org/jira/browse/KYLIN-3604
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Affects Versions: v2.5.0
>Reporter: Colin Ma
>Assignee: Yichen Zhou
>Priority: Blocker
> Fix For: v2.5.1
>
> Attachments: KYLIN-3604.001.patch
>
>
> With Hbase standalone mode, Cube can't be built on step 8 Convert Cuboid Data
> to HFile, the following is the related exception:
> 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Input path:
> {color:#d04437}hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/cuboid/{color}
> 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Output path:
> {color:#14892c}hdfs://nameservice3/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/hfile{color}
> 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Loading HBase configuration
> {color:#d04437}from:hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml{color}
> 18/09/29 11:13:21 ERROR yarn.ApplicationMaster: User class threw exception:
> java.lang.RuntimeException: error execute
> org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS:
> hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
> expected: hdfs://nameservice3 java.lang.RuntimeException: error execute
> org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS:
> hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
> expected: hdfs://nameservice3 at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
> at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606) at
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:637)
> Caused by: java.lang.IllegalArgumentException: Wrong FS:
> hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
> expected: hdfs://nameservice3 at
> org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645) at
> org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:193)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:302)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:298)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:298)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:766) at
> org.apache.kylin.storage.hbase.steps.SparkCubeHFile.execute(SparkCubeHFile.java:183)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 6 more 18/09/29 11:13:21 INFO yarn.ApplicationMaster: Final app status:
> FAILED, exitCode: 15, (reason: User class threw exception:
> java.lang.RuntimeException: error execute
> org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS:
> hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
> expected: hdfs://nameservice3) 18/09/29 11:13:21 INFO spark.SparkContext:
> Invoking stop() from shutdown hook 18/09/29 11:13:21 INFO
> server.ServerConnector: Stopped Spark@1785d078
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3604) Can't build cube with spark in HBase standalone mode
[
https://issues.apache.org/jira/browse/KYLIN-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3604:
Fix Version/s: v2.5.1
> Can't build cube with spark in HBase standalone mode
>
>
> Key: KYLIN-3604
> URL: https://issues.apache.org/jira/browse/KYLIN-3604
> Project: Kylin
> Issue Type: Bug
> Components: Storage - HBase
>Affects Versions: v2.5.0
>Reporter: Colin Ma
>Assignee: Yichen Zhou
>Priority: Blocker
> Fix For: v2.5.1
>
> Attachments: KYLIN-3604.001.patch
>
>
> With Hbase standalone mode, Cube can't be built on step 8 Convert Cuboid Data
> to HFile, the following is the related exception:
> 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Input path:
> {color:#d04437}hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/cuboid/{color}
> 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Output path:
> {color:#14892c}hdfs://nameservice3/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/hfile{color}
> 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Loading HBase configuration
> {color:#d04437}from:hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml{color}
> 18/09/29 11:13:21 ERROR yarn.ApplicationMaster: User class threw exception:
> java.lang.RuntimeException: error execute
> org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS:
> hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
> expected: hdfs://nameservice3 java.lang.RuntimeException: error execute
> org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS:
> hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
> expected: hdfs://nameservice3 at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
> at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606) at
> org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:637)
> Caused by: java.lang.IllegalArgumentException: Wrong FS:
> hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
> expected: hdfs://nameservice3 at
> org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645) at
> org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:193)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:302)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:298)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:298)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:766) at
> org.apache.kylin.storage.hbase.steps.SparkCubeHFile.execute(SparkCubeHFile.java:183)
> at
> org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
> ... 6 more 18/09/29 11:13:21 INFO yarn.ApplicationMaster: Final app status:
> FAILED, exitCode: 15, (reason: User class threw exception:
> java.lang.RuntimeException: error execute
> org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS:
> hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
> expected: hdfs://nameservice3) 18/09/29 11:13:21 INFO spark.SparkContext:
> Invoking stop() from shutdown hook 18/09/29 11:13:21 INFO
> server.ServerConnector: Stopped Spark@1785d078
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shaofeng SHI updated KYLIN-3597: Fix Version/s: v2.6.0 > Fix sonar reported static code issues > - > > Key: KYLIN-3597 > URL: https://issues.apache.org/jira/browse/KYLIN-3597 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shaofeng SHI >Priority: Major > Fix For: v2.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2924) Utilize error-prone to discover common coding mistakes
[ https://issues.apache.org/jira/browse/KYLIN-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632934#comment-16632934 ] ASF GitHub Bot commented on KYLIN-2924: --- shaofengshi closed pull request #270: KYLIN-2924 enable google error-prone in compile phase URL: https://github.com/apache/kylin/pull/270 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Utilize error-prone to discover common coding mistakes > -- > > Key: KYLIN-2924 > URL: https://issues.apache.org/jira/browse/KYLIN-2924 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Billy Liu >Priority: Major > Fix For: v2.6.0 > > > http://errorprone.info/ is a tool which detects common coding mistakes. > We should incorporate into Kylin build. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-2924) Utilize error-prone to discover common coding mistakes
[ https://issues.apache.org/jira/browse/KYLIN-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632935#comment-16632935 ] ASF subversion and git services commented on KYLIN-2924: Commit bd8961c884cdc8ec5a9903a1d5a317bb626a2dff in kylin's branch refs/heads/master from [~yimingliu] [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=bd8961c ] KYLIN-2924 enable google error-prone in compile phase > Utilize error-prone to discover common coding mistakes > -- > > Key: KYLIN-2924 > URL: https://issues.apache.org/jira/browse/KYLIN-2924 > Project: Kylin > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Billy Liu >Priority: Major > Fix For: v2.6.0 > > > http://errorprone.info/ is a tool which detects common coding mistakes. > We should incorporate into Kylin build. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI reassigned KYLIN-3605:
---
Assignee: Shaofeng SHI
> Fix security issues reported by snyk.io
> ---
>
> Key: KYLIN-3605
> URL: https://issues.apache.org/jira/browse/KYLIN-3605
> Project: Kylin
> Issue Type: Improvement
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.6.0
>
>
> HIGH SEVERITY
> h1. Arbitrary Code Execution
> * Vulnerable module: commons-beanutils:commons-beanutils
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›
> commons-configuration:commons-configuration@1.6 ›
> commons-digester:commons-digester@1.8 ›
> commons-beanutils:commons-beanutils@1.7.0
> *Remediation:* No remediation path available.
> h2. Overview
> [{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
> Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
> property, which allows remote attackers to manipulate the {{ClassLoader}}and
> execute arbitrary code via the class parameter, as demonstrated by the
> passing of this parameter to the {{getClass}} method of the
> {{ActionForm}}object in Struts 1.
>
> HIGH SEVERITY
> h1. Arbitrary Command Execution
> * Vulnerable module: org.mortbay.jetty:jetty
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
> *Remediation:* No remediation path available.
> h2. Overview
> [org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
> is an open-source project providing a HTTP server, HTTP client and
> javax.servlet container.
> Affected versions of this package are vulnerable to Arbitrary Command
> Execution. It writes backtrace data without sanitizing non-printable
> characters, which might allow remote attackers to modify a window's title, or
> possibly execute arbitrary commands or overwrite files, via an HTTP request
> containing an escape sequence for a terminal emulator, related to (1) a
> string value in the Age parameter to the default URI for the Cookie Dump
> Servlet in test-jetty-webapp/src/main/java/com/acme/CookieDump.java under
> cookie/, (2) an alphabetic value in the A parameter to jsp/expr.jsp, or (3)
> an alphabetic value in the Content-Length HTTP header to an arbitrary
> application.
> HIGH SEVERITY
> h1. Information Exposure
> * Vulnerable module: org.apache.hadoop:hadoop-common
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1
> *Remediation:* No remediation path available.
> h2. Overview
> [{{org.apache.hadoop:hadoop-common}}|https://hadoop.apache.org/] is a
> framework that allows for the distributed processing of large data sets
> across clusters of computers using simple programming models.
> Affected versions of the package are vulnerable to Information Exposure.
> If you use the CredentialProvider feature to encrypt passwords used in
> NodeManager configs, it may be possible for any Container launched by that
> NodeManager to gain access to the encryption password. The other passwords
> themselves are not directly exposed.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3605:
Fix Version/s: v2.6.0
> Fix security issues reported by snyk.io
> ---
>
> Key: KYLIN-3605
> URL: https://issues.apache.org/jira/browse/KYLIN-3605
> Project: Kylin
> Issue Type: Improvement
>Reporter: Shaofeng SHI
>Assignee: Shaofeng SHI
>Priority: Major
> Fix For: v2.6.0
>
>
> HIGH SEVERITY
> h1. Arbitrary Code Execution
> * Vulnerable module: commons-beanutils:commons-beanutils
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›
> commons-configuration:commons-configuration@1.6 ›
> commons-digester:commons-digester@1.8 ›
> commons-beanutils:commons-beanutils@1.7.0
> *Remediation:* No remediation path available.
> h2. Overview
> [{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
> Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
> property, which allows remote attackers to manipulate the {{ClassLoader}}and
> execute arbitrary code via the class parameter, as demonstrated by the
> passing of this parameter to the {{getClass}} method of the
> {{ActionForm}}object in Struts 1.
>
> HIGH SEVERITY
> h1. Arbitrary Command Execution
> * Vulnerable module: org.mortbay.jetty:jetty
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
> *Remediation:* No remediation path available.
> h2. Overview
> [org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
> is an open-source project providing a HTTP server, HTTP client and
> javax.servlet container.
> Affected versions of this package are vulnerable to Arbitrary Command
> Execution. It writes backtrace data without sanitizing non-printable
> characters, which might allow remote attackers to modify a window's title, or
> possibly execute arbitrary commands or overwrite files, via an HTTP request
> containing an escape sequence for a terminal emulator, related to (1) a
> string value in the Age parameter to the default URI for the Cookie Dump
> Servlet in test-jetty-webapp/src/main/java/com/acme/CookieDump.java under
> cookie/, (2) an alphabetic value in the A parameter to jsp/expr.jsp, or (3)
> an alphabetic value in the Content-Length HTTP header to an arbitrary
> application.
> HIGH SEVERITY
> h1. Information Exposure
> * Vulnerable module: org.apache.hadoop:hadoop-common
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1
> *Remediation:* No remediation path available.
> h2. Overview
> [{{org.apache.hadoop:hadoop-common}}|https://hadoop.apache.org/] is a
> framework that allows for the distributed processing of large data sets
> across clusters of computers using simple programming models.
> Affected versions of the package are vulnerable to Information Exposure.
> If you use the CredentialProvider feature to encrypt passwords used in
> NodeManager configs, it may be possible for any Container launched by that
> NodeManager to gain access to the encryption password. The other passwords
> themselves are not directly exposed.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632930#comment-16632930
]
ASF GitHub Bot commented on KYLIN-3605:
---
shaofengshi opened a new pull request #276: KYLIN-3605 upgrade hadoop-common
and zookeeper version to fix securit…
URL: https://github.com/apache/kylin/pull/276
…y issues
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Fix security issues reported by snyk.io
> ---
>
> Key: KYLIN-3605
> URL: https://issues.apache.org/jira/browse/KYLIN-3605
> Project: Kylin
> Issue Type: Improvement
>Reporter: Shaofeng SHI
>Priority: Major
>
> HIGH SEVERITY
> h1. Arbitrary Code Execution
> * Vulnerable module: commons-beanutils:commons-beanutils
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›
> commons-configuration:commons-configuration@1.6 ›
> commons-digester:commons-digester@1.8 ›
> commons-beanutils:commons-beanutils@1.7.0
> *Remediation:* No remediation path available.
> h2. Overview
> [{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
> Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
> property, which allows remote attackers to manipulate the {{ClassLoader}}and
> execute arbitrary code via the class parameter, as demonstrated by the
> passing of this parameter to the {{getClass}} method of the
> {{ActionForm}}object in Struts 1.
>
> HIGH SEVERITY
> h1. Arbitrary Command Execution
> * Vulnerable module: org.mortbay.jetty:jetty
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
> *Remediation:* No remediation path available.
> h2. Overview
> [org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
> is an open-source project providing a HTTP server, HTTP client and
> javax.servlet container.
> Affected versions of this package are vulnerable to Arbitrary Command
> Execution. It writes backtrace data without sanitizing non-printable
> characters, which might allow remote attackers to modify a window's title, or
> possibly execute arbitrary commands or overwrite files, via an HTTP request
> containing an escape sequence for a terminal emulator, related to (1) a
> string value in the Age parameter to the default URI for the Cookie Dump
> Servlet in test-jetty-webapp/src/main/java/com/acme/CookieDump.java under
> cookie/, (2) an alphabetic value in the A parameter to jsp/expr.jsp, or (3)
> an alphabetic value in the Content-Length HTTP header to an arbitrary
> application.
> HIGH SEVERITY
> h1. Information Exposure
> * Vulnerable module: org.apache.hadoop:hadoop-common
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1
> *Remediation:* No remediation path available.
> h2. Overview
> [{{org.apache.hadoop:hadoop-common}}|https://hadoop.apache.org/] is a
> framework that allows for the distributed processing of large data sets
> across clusters of computers using simple programming models.
> Affected versions of the package are vulnerable to Information Exposure.
> If you use the CredentialProvider feature to encrypt passwords used in
> NodeManager configs, it may be possible for any Container launched by that
> NodeManager to gain access to the encryption password. The other passwords
> themselves are not directly exposed.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632932#comment-16632932
]
ASF GitHub Bot commented on KYLIN-3605:
---
asfgit commented on issue #276: KYLIN-3605 upgrade hadoop-common and zookeeper
version to fix securit…
URL: https://github.com/apache/kylin/pull/276#issuecomment-425635060
Can one of the admins verify this patch?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Fix security issues reported by snyk.io
> ---
>
> Key: KYLIN-3605
> URL: https://issues.apache.org/jira/browse/KYLIN-3605
> Project: Kylin
> Issue Type: Improvement
>Reporter: Shaofeng SHI
>Priority: Major
>
> HIGH SEVERITY
> h1. Arbitrary Code Execution
> * Vulnerable module: commons-beanutils:commons-beanutils
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›
> commons-configuration:commons-configuration@1.6 ›
> commons-digester:commons-digester@1.8 ›
> commons-beanutils:commons-beanutils@1.7.0
> *Remediation:* No remediation path available.
> h2. Overview
> [{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
> Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
> property, which allows remote attackers to manipulate the {{ClassLoader}}and
> execute arbitrary code via the class parameter, as demonstrated by the
> passing of this parameter to the {{getClass}} method of the
> {{ActionForm}}object in Struts 1.
>
> HIGH SEVERITY
> h1. Arbitrary Command Execution
> * Vulnerable module: org.mortbay.jetty:jetty
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
> *Remediation:* No remediation path available.
> h2. Overview
> [org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
> is an open-source project providing a HTTP server, HTTP client and
> javax.servlet container.
> Affected versions of this package are vulnerable to Arbitrary Command
> Execution. It writes backtrace data without sanitizing non-printable
> characters, which might allow remote attackers to modify a window's title, or
> possibly execute arbitrary commands or overwrite files, via an HTTP request
> containing an escape sequence for a terminal emulator, related to (1) a
> string value in the Age parameter to the default URI for the Cookie Dump
> Servlet in test-jetty-webapp/src/main/java/com/acme/CookieDump.java under
> cookie/, (2) an alphabetic value in the A parameter to jsp/expr.jsp, or (3)
> an alphabetic value in the Content-Length HTTP header to an arbitrary
> application.
> HIGH SEVERITY
> h1. Information Exposure
> * Vulnerable module: org.apache.hadoop:hadoop-common
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1
> *Remediation:* No remediation path available.
> h2. Overview
> [{{org.apache.hadoop:hadoop-common}}|https://hadoop.apache.org/] is a
> framework that allows for the distributed processing of large data sets
> across clusters of computers using simple programming models.
> Affected versions of the package are vulnerable to Information Exposure.
> If you use the CredentialProvider feature to encrypt passwords used in
> NodeManager configs, it may be possible for any Container launched by that
> NodeManager to gain access to the encryption password. The other passwords
> themselves are not directly exposed.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632931#comment-16632931
]
ASF GitHub Bot commented on KYLIN-3605:
---
asfgit commented on issue #276: KYLIN-3605 upgrade hadoop-common and zookeeper
version to fix securit…
URL: https://github.com/apache/kylin/pull/276#issuecomment-425635058
Can one of the admins verify this patch?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Fix security issues reported by snyk.io
> ---
>
> Key: KYLIN-3605
> URL: https://issues.apache.org/jira/browse/KYLIN-3605
> Project: Kylin
> Issue Type: Improvement
>Reporter: Shaofeng SHI
>Priority: Major
>
> HIGH SEVERITY
> h1. Arbitrary Code Execution
> * Vulnerable module: commons-beanutils:commons-beanutils
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›
> commons-configuration:commons-configuration@1.6 ›
> commons-digester:commons-digester@1.8 ›
> commons-beanutils:commons-beanutils@1.7.0
> *Remediation:* No remediation path available.
> h2. Overview
> [{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
> Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
> property, which allows remote attackers to manipulate the {{ClassLoader}}and
> execute arbitrary code via the class parameter, as demonstrated by the
> passing of this parameter to the {{getClass}} method of the
> {{ActionForm}}object in Struts 1.
>
> HIGH SEVERITY
> h1. Arbitrary Command Execution
> * Vulnerable module: org.mortbay.jetty:jetty
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
> *Remediation:* No remediation path available.
> h2. Overview
> [org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
> is an open-source project providing a HTTP server, HTTP client and
> javax.servlet container.
> Affected versions of this package are vulnerable to Arbitrary Command
> Execution. It writes backtrace data without sanitizing non-printable
> characters, which might allow remote attackers to modify a window's title, or
> possibly execute arbitrary commands or overwrite files, via an HTTP request
> containing an escape sequence for a terminal emulator, related to (1) a
> string value in the Age parameter to the default URI for the Cookie Dump
> Servlet in test-jetty-webapp/src/main/java/com/acme/CookieDump.java under
> cookie/, (2) an alphabetic value in the A parameter to jsp/expr.jsp, or (3)
> an alphabetic value in the Content-Length HTTP header to an arbitrary
> application.
> HIGH SEVERITY
> h1. Information Exposure
> * Vulnerable module: org.apache.hadoop:hadoop-common
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1
> *Remediation:* No remediation path available.
> h2. Overview
> [{{org.apache.hadoop:hadoop-common}}|https://hadoop.apache.org/] is a
> framework that allows for the distributed processing of large data sets
> across clusters of computers using simple programming models.
> Affected versions of the package are vulnerable to Information Exposure.
> If you use the CredentialProvider feature to encrypt passwords used in
> NodeManager configs, it may be possible for any Container launched by that
> NodeManager to gain access to the encryption password. The other passwords
> themselves are not directly exposed.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KYLIN-3605) Fix security issues reported by snyk.io
[
https://issues.apache.org/jira/browse/KYLIN-3605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shaofeng SHI updated KYLIN-3605:
Description:
HIGH SEVERITY
h1. Arbitrary Code Execution
* Vulnerable module: commons-beanutils:commons-beanutils
* Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
h2. Detailed paths and remediation
*
*Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
org.apache.hadoop:hadoop-common@2.7.1 ›
commons-configuration:commons-configuration@1.6 ›
commons-digester:commons-digester@1.8 ›
commons-beanutils:commons-beanutils@1.7.0
*Remediation:* No remediation path available.
h2. Overview
[{{commons-beanutils:commons-beanutils}}|http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22commons-beanutils%22]
Vulnerable versions of _Apache Commons BeanUtils_, do not suppress the class
property, which allows remote attackers to manipulate the {{ClassLoader}}and
execute arbitrary code via the class parameter, as demonstrated by the passing
of this parameter to the {{getClass}} method of the {{ActionForm}}object in
Struts 1.
HIGH SEVERITY
h1. Arbitrary Command Execution
* Vulnerable module: org.mortbay.jetty:jetty
* Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
h2. Detailed paths and remediation
*
*Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
org.apache.hadoop:hadoop-common@2.7.1 ›org.mortbay.jetty:jetty@6.1.26
*Remediation:* No remediation path available.
h2. Overview
[org.mortbay.jetty:jetty|https://mvnrepository.com/artifact/org.mortbay.jetty]
is an open-source project providing a HTTP server, HTTP client and
javax.servlet container.
Affected versions of this package are vulnerable to Arbitrary Command
Execution. It writes backtrace data without sanitizing non-printable
characters, which might allow remote attackers to modify a window's title, or
possibly execute arbitrary commands or overwrite files, via an HTTP request
containing an escape sequence for a terminal emulator, related to (1) a string
value in the Age parameter to the default URI for the Cookie Dump Servlet in
test-jetty-webapp/src/main/java/com/acme/CookieDump.java under cookie/, (2) an
alphabetic value in the A parameter to jsp/expr.jsp, or (3) an alphabetic value
in the Content-Length HTTP header to an arbitrary application.
HIGH SEVERITY
h1. Information Exposure
* Vulnerable module: org.apache.hadoop:hadoop-common
* Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
h2. Detailed paths and remediation
*
*Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
org.apache.hadoop:hadoop-common@2.7.1
*Remediation:* No remediation path available.
h2. Overview
[{{org.apache.hadoop:hadoop-common}}|https://hadoop.apache.org/] is a framework
that allows for the distributed processing of large data sets across clusters
of computers using simple programming models.
Affected versions of the package are vulnerable to Information Exposure.
If you use the CredentialProvider feature to encrypt passwords used in
NodeManager configs, it may be possible for any Container launched by that
NodeManager to gain access to the encryption password. The other passwords
themselves are not directly exposed.
was:
[org.apache.commons:commons-compress|https://github.com/apache/commons-compress]
defines an API for working with compression and archive formats.
Affected versions of this package are vulnerable to Directory Traversal.
> Fix security issues reported by snyk.io
> ---
>
> Key: KYLIN-3605
> URL: https://issues.apache.org/jira/browse/KYLIN-3605
> Project: Kylin
> Issue Type: Improvement
>Reporter: Shaofeng SHI
>Priority: Major
>
> HIGH SEVERITY
> h1. Arbitrary Code Execution
> * Vulnerable module: commons-beanutils:commons-beanutils
> * Introduced through: org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> h2. Detailed paths and remediation
> *
> *Introduced through*: org.apache.kylin:kylin-core-metadata@2.6.0-SNAPSHOT ›
> org.apache.kylin:kylin-core-common@2.6.0-SNAPSHOT
> ›com.github.joshelser:dropwizard-metrics-hadoop-metrics2-reporter@0.1.2 ›
> org.apache.hadoop:hadoop-common@2.7.1 ›
> commons-configuration:commons-configuration@1.6 ›
> commons-digester:commons-digester@1.8 ›
> commons-beanutils:commons-beanutils@1.7.0
> *Remediation:* No remediation path available.
> h2. Overview
>
[jira] [Created] (KYLIN-3605) Fix security issues reported by snyk.io
Shaofeng SHI created KYLIN-3605: --- Summary: Fix security issues reported by snyk.io Key: KYLIN-3605 URL: https://issues.apache.org/jira/browse/KYLIN-3605 Project: Kylin Issue Type: Improvement Reporter: Shaofeng SHI [org.apache.commons:commons-compress|https://github.com/apache/commons-compress] defines an API for working with compression and archive formats. Affected versions of this package are vulnerable to Directory Traversal. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3604) Can't build cube with spark in HBase standalone mode
[
https://issues.apache.org/jira/browse/KYLIN-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Colin Ma updated KYLIN-3604:
Description:
With Hbase standalone mode, Cube can't be built on step 8 Convert Cuboid Data
to HFile, the following is the related exception:
18/09/29 11:13:21 INFO steps.SparkCubeHFile: Input path:
{color:#d04437}hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/cuboid/{color}
18/09/29 11:13:21 INFO steps.SparkCubeHFile: Output path:
{color:#14892c}hdfs://nameservice3/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/hfile{color}
18/09/29 11:13:21 INFO steps.SparkCubeHFile: Loading HBase configuration
{color:#d04437}from:hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml{color}
18/09/29 11:13:21 ERROR yarn.ApplicationMaster: User class threw exception:
java.lang.RuntimeException: error execute
org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS:
hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
expected: hdfs://nameservice3 java.lang.RuntimeException: error execute
org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS:
hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
expected: hdfs://nameservice3 at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606) at
org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:637)
Caused by: java.lang.IllegalArgumentException: Wrong FS:
hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
expected: hdfs://nameservice3 at
org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645) at
org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:193)
at
org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105)
at
org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:302)
at
org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:298)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:298)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:766) at
org.apache.kylin.storage.hbase.steps.SparkCubeHFile.execute(SparkCubeHFile.java:183)
at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
... 6 more 18/09/29 11:13:21 INFO yarn.ApplicationMaster: Final app status:
FAILED, exitCode: 15, (reason: User class threw exception:
java.lang.RuntimeException: error execute
org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS:
hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
expected: hdfs://nameservice3) 18/09/29 11:13:21 INFO spark.SparkContext:
Invoking stop() from shutdown hook 18/09/29 11:13:21 INFO
server.ServerConnector: Stopped Spark@1785d078
was:
With Hbase standalone mode, Cube can't be built on step 8 Convert Cuboid Data
to HFile, the following is the related exception:
18/09/29 11:13:21 INFO steps.SparkCubeHFile: Input path:
hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/cuboid/
18/09/29 11:13:21 INFO steps.SparkCubeHFile: Output path:
hdfs://nameservice3/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/hfile
18/09/29 11:13:21 INFO steps.SparkCubeHFile: Loading HBase configuration
from:hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml
18/09/29 11:13:21 ERROR yarn.ApplicationMaster: User class threw exception:
java.lang.RuntimeException: error execute
org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS:
hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
expected: hdfs://nameservice3 java.lang.RuntimeException: error execute
org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS:
hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml,
expected: hdfs://nameservice3 at
org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
at
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632906#comment-16632906 ] ASF subversion and git services commented on KYLIN-3597: Commit 796fede6740dd90e624a3d79791625176b504b0a in kylin's branch refs/heads/master from hit-lacus [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=796fede ] KYLIN-3597 Improve code smell > Fix sonar reported static code issues > - > > Key: KYLIN-3597 > URL: https://issues.apache.org/jira/browse/KYLIN-3597 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shaofeng SHI >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[
https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632905#comment-16632905
]
ASF GitHub Bot commented on KYLIN-3597:
---
shaofengshi closed pull request #274: KYLIN-3597 Improve code smell
URL: https://github.com/apache/kylin/pull/274
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/core-common/src/main/java/org/apache/kylin/common/persistence/IdentifierFileResourceStore.java
b/core-common/src/main/java/org/apache/kylin/common/persistence/IdentifierFileResourceStore.java
index e516bd1149..9475e44189 100644
---
a/core-common/src/main/java/org/apache/kylin/common/persistence/IdentifierFileResourceStore.java
+++
b/core-common/src/main/java/org/apache/kylin/common/persistence/IdentifierFileResourceStore.java
@@ -17,8 +17,6 @@
*/
package org.apache.kylin.common.persistence;
-import java.io.File;
-
import org.apache.kylin.common.KylinConfig;
import org.apache.kylin.common.StorageURL;
import org.slf4j.Logger;
@@ -38,8 +36,6 @@
private static final String IFILE_SCHEME = "ifile";
-private File root;
-
public IdentifierFileResourceStore(KylinConfig kylinConfig) throws
Exception {
super(kylinConfig);
}
diff --git
a/core-cube/src/main/java/org/apache/kylin/gridtable/GTFilterScanner.java
b/core-cube/src/main/java/org/apache/kylin/gridtable/GTFilterScanner.java
index 12074bd850..89d29e3627 100644
--- a/core-cube/src/main/java/org/apache/kylin/gridtable/GTFilterScanner.java
+++ b/core-cube/src/main/java/org/apache/kylin/gridtable/GTFilterScanner.java
@@ -72,75 +72,74 @@ public long getInputRowCount() {
@Override
public Iterator iterator() {
-return new Iterator() {
-
-private Iterator inputIterator = delegated.iterator();
-private FilterResultCache resultCache = new
FilterResultCache(getInfo(), filter);
-
-@Override
-public boolean hasNext() {
-if (next != null)
-return true;
-
-while (inputIterator.hasNext()) {
-next = inputIterator.next();
-inputRowCount++;
-if (!evaluate()) {
-continue;
-}
-return true;
-}
-next = null;
-return false;
-}
+return new GTFilterScannerIterator();
+}
-private boolean evaluate() {
-if (checker != null && checker.shouldBypass(next)) {
-return false;
-}
+private class GTFilterScannerIterator implements Iterator {
+private Iterator inputIterator = delegated.iterator();
+private FilterResultCache resultCache = new
FilterResultCache(getInfo(), filter);
-if (filter == null)
-return true;
+@Override
+public boolean hasNext() {
+if (next != null)
+return true;
-// 'next' and 'oneTuple' are referring to the same record
-boolean[] cachedResult = resultCache.checkCache(next);
-if (cachedResult != null)
-return cachedResult[0];
+while (inputIterator.hasNext()) {
+next = inputIterator.next();
+inputRowCount++;
+if (!evaluate()) {
+continue;
+}
+return true;
+}
+next = null;
+return false;
+}
-boolean result = filter.evaluate(oneTuple, filterCodeSystem);
-resultCache.setLastResult(result);
-return result;
+private boolean evaluate() {
+if (checker != null && checker.shouldBypass(next)) {
+return false;
}
-@Override
-public GTRecord next() {
-// fetch next record
-if (next == null) {
-hasNext();
-if (next == null)
-throw new NoSuchElementException();
-}
+if (filter == null)
+return true;
-GTRecord result = next;
-next = null;
-return result;
-}
+// 'next' and 'oneTuple' are referring to the same record
+boolean[] cachedResult = resultCache.checkCache(next);
+if (cachedResult != null)
+return cachedResult[0];
+
+boolean result = filter.evaluate(oneTuple, filterCodeSystem);
+resultCache.setLastResult(result);
+
[jira] [Commented] (KYLIN-3232) Need document for ops tools
[ https://issues.apache.org/jira/browse/KYLIN-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632904#comment-16632904 ] ASF subversion and git services commented on KYLIN-3232: Commit fa2994c75c312ff3d37484661c6797c52853d081 in kylin's branch refs/heads/document from GinaZhai [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=fa2994c ] KYLIN-3232 Add document for ops tools > Need document for ops tools > --- > > Key: KYLIN-3232 > URL: https://issues.apache.org/jira/browse/KYLIN-3232 > Project: Kylin > Issue Type: Task > Components: Documentation >Reporter: Shaofeng SHI >Assignee: Na Zhai >Priority: Major > Fix For: v2.6.0 > > > Kylin has many good ops tool, like CubeMigrationCLI, CubeMetaIngester, etc; > They can greatly improve administrators productivity. But now there is no > document about them (you need check the code to know what they're doing) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3232) Need document for ops tools
[
https://issues.apache.org/jira/browse/KYLIN-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632903#comment-16632903
]
ASF GitHub Bot commented on KYLIN-3232:
---
shaofengshi closed pull request #272: KYLIN-3232 Add document for ops tools
URL: https://github.com/apache/kylin/pull/272
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/website/_data/docs-cn.yml b/website/_data/docs-cn.yml
index f2b1bb35a6..93f29ecd76 100644
--- a/website/_data/docs-cn.yml
+++ b/website/_data/docs-cn.yml
@@ -53,6 +53,7 @@
- tutorial/squirrel
- tutorial/Qlik
- tutorial/superset
+ - tutorial/tools
- title: 帮助
diff --git a/website/_data/docs.yml b/website/_data/docs.yml
index 9146cfbbf8..ead957ec9e 100644
--- a/website/_data/docs.yml
+++ b/website/_data/docs.yml
@@ -65,6 +65,7 @@
- tutorial/hue
- tutorial/Qlik
- tutorial/superset
+ - tutorial/tools
- title: How To
docs:
diff --git a/website/_docs/tutorial/tools.cn.md
b/website/_docs/tutorial/tools.cn.md
new file mode 100644
index 00..20cb65bfca
--- /dev/null
+++ b/website/_docs/tutorial/tools.cn.md
@@ -0,0 +1,156 @@
+---
+layout: docs-cn
+title: "Kylin 中的工具类"
+categories: tutorial
+permalink: /cn/docs/tutorial/tools.html
+---
+Kylin
有很多好的工具类。这篇文档会介绍以下几个工具类：KylinConfigCLI.java，CubeMetaExtractor.java，CubeMetaIngester.java，CubeMigrationCLI.java
和 CubeMigrationCheckCLI.java。在使用这些工具类前，首先要切换到 KYLIN_HOME 目录下。
+
+## KylinConfigCLI.java
+
+### 作用
+KylinConfigCLI 工具类会将您输入的 Kylin 参数的值输出。
+
+### 如何使用
+类名后只能写一个参数，conf_name 即您想要知道其值的参数名称。
+{% highlight Groff markup %}
+./bin/kylin.sh org.apache.kylin.tool.KylinConfigCLI
+{% endhighlight %}
+例如：
+{% highlight Groff markup %}
+./bin/kylin.sh org.apache.kylin.tool.KylinConfigCLI kylin.server.mode
+{% endhighlight %}
+结果：
+{% highlight Groff markup %}
+all
+{% endhighlight %}
+如果您不知道参数的准确名称，您可以使用以下命令，然后所有以该前缀为前缀的参数的值都会被列出。
+{% highlight Groff markup %}
+./bin/kylin.sh org.apache.kylin.tool.KylinConfigCLI .
+{% endhighlight %}
+例如：
+{% highlight Groff markup %}
+./bin/kylin.sh org.apache.kylin.tool.KylinConfigCLI kylin.job.
+{% endhighlight %}
+结果：
+{% highlight Groff markup %}
+max-concurrent-jobs=10
+retry=3
+sampling-percentage=100
+{% endhighlight %}
+
+## CubeMetaExtractor.java
+
+### 作用
+CubeMetaExtractor.java 用于提取与 Cube 相关的信息以达到调试/分发的目的。
+
+### 如何使用
+类名后至少写两个参数。
+{% highlight Groff markup %}
+./bin/kylin.sh org.apache.kylin.tool.CubeMetaExtractor -
-destDir
+{% endhighlight %}
+例如：
+{% highlight Groff markup %}
+./bin/kylin.sh org.apache.kylin.tool.CubeMetaExtractor -cube querycube
-destDir /root/newconfigdir1
+{% endhighlight %}
+结果：
+命令执行成功后，您想要抽取的 Cube / project / hybrid 将会存在于您指定的 destDir 目录中。
+
+下面会列出所有支持的参数：
+
+| Parameter | Description
|
+| - |
:--
|
+| allProjects | Specify realizations
in all projects to extract |
+| compress| Specify whether to
compress the output with zip. Default true.
|
+| cube| Specify which Cube
to extract
|
+| destDir | (Required) Specify
the dest dir to save the related information
|
+| engineType| Specify the engine
type to overwrite. Default is empty, keep origin.
|
+| hybrid| Specify which hybrid
to extract |
+| includeJobs | Set this to true if
want to extract job info/outputs too. Default false
|
+| includeSegmentDetails | Set this to true if
want to extract segment details too, such as dict, tablesnapshot. Default false
|
+| includeSegments | Set this to true if
want extract the segments info. Default true
|
+| onlyOutput| When include jobs,
only extract output of job. Default true
|
+| packagetype | Specify the package
type
|
+|
[jira] [Commented] (KYLIN-3604) Can't build cube with spark in HBase standalone mode
[ https://issues.apache.org/jira/browse/KYLIN-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632900#comment-16632900 ] Colin Ma commented on KYLIN-3604: - The problem is caused by the different FileSystem in HBase standalone mode. Output path has the different FileSystem with Input path and HBase configuration. Upload an initial patch for review. > Can't build cube with spark in HBase standalone mode > > > Key: KYLIN-3604 > URL: https://issues.apache.org/jira/browse/KYLIN-3604 > Project: Kylin > Issue Type: Bug > Components: Storage - HBase >Affects Versions: v2.5.0 >Reporter: Colin Ma >Priority: Blocker > Attachments: KYLIN-3604.001.patch > > > With Hbase standalone mode, Cube can't be built on step 8 Convert Cuboid Data > to HFile, the following is the related exception: > 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Input path: > hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/cuboid/ > 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Output path: > hdfs://nameservice3/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/hfile > 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Loading HBase configuration > from:hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml > 18/09/29 11:13:21 ERROR yarn.ApplicationMaster: User class threw exception: > java.lang.RuntimeException: error execute > org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS: > hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, > expected: hdfs://nameservice3 java.lang.RuntimeException: error execute > org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS: > hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, > expected: hdfs://nameservice3 at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) at > org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:637) > Caused by: java.lang.IllegalArgumentException: Wrong FS: > hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, > expected: hdfs://nameservice3 at > org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645) at > org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:193) > at > org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105) > at > org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:302) > at > org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:298) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:298) > at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:766) at > org.apache.kylin.storage.hbase.steps.SparkCubeHFile.execute(SparkCubeHFile.java:183) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 6 more 18/09/29 11:13:21 INFO yarn.ApplicationMaster: Final app status: > FAILED, exitCode: 15, (reason: User class threw exception: > java.lang.RuntimeException: error execute > org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS: > hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, > expected: hdfs://nameservice3) 18/09/29 11:13:21 INFO spark.SparkContext: > Invoking stop() from shutdown hook 18/09/29 11:13:21 INFO > server.ServerConnector: Stopped Spark@1785d078 > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KYLIN-3604) Can't build cube with spark in HBase standalone mode
[ https://issues.apache.org/jira/browse/KYLIN-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma updated KYLIN-3604: Attachment: KYLIN-3604.001.patch > Can't build cube with spark in HBase standalone mode > > > Key: KYLIN-3604 > URL: https://issues.apache.org/jira/browse/KYLIN-3604 > Project: Kylin > Issue Type: Bug > Components: Storage - HBase >Affects Versions: v2.5.0 >Reporter: Colin Ma >Priority: Blocker > Attachments: KYLIN-3604.001.patch > > > With Hbase standalone mode, Cube can't be built on step 8 Convert Cuboid Data > to HFile, the following is the related exception: > 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Input path: > hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/cuboid/ > 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Output path: > hdfs://nameservice3/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/hfile > 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Loading HBase configuration > from:hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml > 18/09/29 11:13:21 ERROR yarn.ApplicationMaster: User class threw exception: > java.lang.RuntimeException: error execute > org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS: > hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, > expected: hdfs://nameservice3 java.lang.RuntimeException: error execute > org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS: > hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, > expected: hdfs://nameservice3 at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) at > org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:637) > Caused by: java.lang.IllegalArgumentException: Wrong FS: > hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, > expected: hdfs://nameservice3 at > org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645) at > org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:193) > at > org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105) > at > org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:302) > at > org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:298) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:298) > at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:766) at > org.apache.kylin.storage.hbase.steps.SparkCubeHFile.execute(SparkCubeHFile.java:183) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 6 more 18/09/29 11:13:21 INFO yarn.ApplicationMaster: Final app status: > FAILED, exitCode: 15, (reason: User class threw exception: > java.lang.RuntimeException: error execute > org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS: > hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, > expected: hdfs://nameservice3) 18/09/29 11:13:21 INFO spark.SparkContext: > Invoking stop() from shutdown hook 18/09/29 11:13:21 INFO > server.ServerConnector: Stopped Spark@1785d078 > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KYLIN-3604) Can't build cube with spark in HBase standalone mode
Colin Ma created KYLIN-3604: --- Summary: Can't build cube with spark in HBase standalone mode Key: KYLIN-3604 URL: https://issues.apache.org/jira/browse/KYLIN-3604 Project: Kylin Issue Type: Bug Components: Storage - HBase Affects Versions: v2.5.0 Reporter: Colin Ma With Hbase standalone mode, Cube can't be built on step 8 Convert Cuboid Data to HFile, the following is the related exception: 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Input path: hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/cuboid/ 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Output path: hdfs://nameservice3/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/test_spark_cube/hfile 18/09/29 11:13:21 INFO steps.SparkCubeHFile: Loading HBase configuration from:hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml 18/09/29 11:13:21 ERROR yarn.ApplicationMaster: User class threw exception: java.lang.RuntimeException: error execute org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS: hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, expected: hdfs://nameservice3 java.lang.RuntimeException: error execute org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS: hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, expected: hdfs://nameservice3 at org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:637) Caused by: java.lang.IllegalArgumentException: Wrong FS: hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, expected: hdfs://nameservice3 at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645) at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:193) at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105) at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:302) at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:298) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:298) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:766) at org.apache.kylin.storage.hbase.steps.SparkCubeHFile.execute(SparkCubeHFile.java:183) at org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) ... 6 more 18/09/29 11:13:21 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.lang.RuntimeException: error execute org.apache.kylin.storage.hbase.steps.SparkCubeHFile. Root cause: Wrong FS: hdfs://nameservice1/kylin/kylin_metadata/kylin-b65c0e62-69e9-bb11-9d7d-e6e5abc7ef8e/hbase-conf.xml, expected: hdfs://nameservice3) 18/09/29 11:13:21 INFO spark.SparkContext: Invoking stop() from shutdown hook 18/09/29 11:13:21 INFO server.ServerConnector: Stopped Spark@1785d078 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3594) Select with Catalog fails
[
https://issues.apache.org/jira/browse/KYLIN-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632872#comment-16632872
]
ASF GitHub Bot commented on KYLIN-3594:
---
coveralls commented on issue #268: KYLIN-3594 Obey JDBC standard
URL: https://github.com/apache/kylin/pull/268#issuecomment-425625274
## Pull Request Test Coverage Report for [Build
3714](https://coveralls.io/builds/19260478)
* **4** of **7** **(57.14%)** changed or added relevant lines in **2**
files are covered.
* **3** unchanged lines in **2** files lost coverage.
* Overall coverage decreased (**-0.002%**) to **23.17%**
---
| Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
| :-|--||---: |
|
[query/src/main/java/org/apache/kylin/query/util/QueryUtil.java](https://coveralls.io/builds/19260478/source?filename=query%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fquery%2Futil%2FQueryUtil.java#L103)
| 4 | 5 | 80.0%
|
[server-base/src/main/java/org/apache/kylin/rest/service/QueryService.java](https://coveralls.io/builds/19260478/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fservice%2FQueryService.java#L561)
| 0 | 2 | 0.0%
| Files with Coverage Reduction | New Missed Lines | % |
| :-|--|--: |
|
[core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://coveralls.io/builds/19260478/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Finmemcubing%2FMemDiskStore.java#L553)
| 1 | 78.12% |
|
[core-cube/src/main/java/org/apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://coveralls.io/builds/19260478/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcube%2Fcuboid%2FTreeCuboidScheduler.java#L124)
| 2 | 68.46% |
| Totals | [](https://coveralls.io/builds/19260478)
|
| :-- | --: |
| Change from base [Build 3711](https://coveralls.io/builds/19259592): |
-0.002% |
| Covered Lines: | 16161 |
| Relevant Lines: | 69750 |
---
# - [Coveralls](https://coveralls.io)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Select with Catalog fails
> -
>
> Key: KYLIN-3594
> URL: https://issues.apache.org/jira/browse/KYLIN-3594
> Project: Kylin
> Issue Type: Bug
>Reporter: Hosur Narahari
>Assignee: XiaoXiang Yu
>Priority: Major
>
> By using DatabaseMetaData if we get catalog using getCatalogs() method, it
> return value "defaultCatalog". It returns actual hive schema when we execute
> getSchemas().
> According to JDBC contract, catalog.schema.table should be valid from clause
> and many query layers use that. But kylin fails when we execute that query.
> I've tried to write sample code piece for that below.
>
> _DatabaseMetaData db = conn.getMetaData();_
> _ResultSet catalogSet = db.getCatalogs();_
> _String catalog = "";_
> _if(catalogSet.next()) {_
> _catalog = catalogSet.getString("TABLE_CAT");_
> _}_
> _ResultSet schemaSet = db.getSchemas();_
> _String schema = "";_
> _if(schemaSet.next()) {_
> _schema = schemaSet.getString("TABLE_SCHEM");_
> _}_
> _StringBuilder sb = new StringBuilder("SELECT * FROM ");_
> _if(!catalog.isEmpty()) {_
> _sb.append(catalog + ".");_
> _}_
> _if(!schema.isEmpty()) {_
> _sb.append(schema + ".");_
> _}_
> _sb.append("kylin_sales limit 10");_
> _String query = sb.toString();_
> _Statement stat = conn.createStatement();_
> _ResultSet rs = stat.executeQuery(query);_
> _while(rs.next()) {_
> _System.out.println(rs.getObject("trans_id"));_
> _}_
> In short, the above snippet is executing the query,
> _select * from defaultCatalog.DEFAULT.kylin_sales._
>
> Same thing happens even with different schemas if we have like,
> _select * from defaultCatalog.test.kylin_sales_ also fails.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3594) Select with Catalog fails
[
https://issues.apache.org/jira/browse/KYLIN-3594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632871#comment-16632871
]
ASF GitHub Bot commented on KYLIN-3594:
---
codecov-io commented on issue #268: KYLIN-3594 Obey JDBC standard
URL: https://github.com/apache/kylin/pull/268#issuecomment-425625134
# [Codecov](https://codecov.io/gh/apache/kylin/pull/268?src=pr=h1) Report
> Merging [#268](https://codecov.io/gh/apache/kylin/pull/268?src=pr=desc)
into
[master](https://codecov.io/gh/apache/kylin/commit/10587a65fe0552179a5c8a6e1151686ce1c8a135?src=pr=desc)
will **decrease** coverage by `<.01%`.
> The diff coverage is `42.85%`.
[](https://codecov.io/gh/apache/kylin/pull/268?src=pr=tree)
```diff
@@ Coverage Diff @@
## master #268 +/- ##
- Coverage 21.16% 21.15% -0.01%
- Complexity 4405 4406 +1
Files 1086 1086
Lines 6974569750 +5
Branches 1008810088
- Hits 1476114758 -3
- Misses5358653591 +5
- Partials 1398 1401 +3
```
| [Impacted
Files](https://codecov.io/gh/apache/kylin/pull/268?src=pr=tree) | Coverage Δ
| Complexity Δ | |
|---|---|---|---|
|
[...va/org/apache/kylin/rest/service/QueryService.java](https://codecov.io/gh/apache/kylin/pull/268/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3Qvc2VydmljZS9RdWVyeVNlcnZpY2UuamF2YQ==)
| `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: |
|
[...in/java/org/apache/kylin/query/util/QueryUtil.java](https://codecov.io/gh/apache/kylin/pull/268/diff?src=pr=tree#diff-cXVlcnkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3F1ZXJ5L3V0aWwvUXVlcnlVdGlsLmphdmE=)
| `61.33% <60%> (-0.64%)` | `19 <1> (+1)` | |
|
[.../apache/kylin/cube/cuboid/TreeCuboidScheduler.java](https://codecov.io/gh/apache/kylin/pull/268/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2N1Ym9pZC9UcmVlQ3Vib2lkU2NoZWR1bGVyLmphdmE=)
| `63.84% <0%> (-2.31%)` | `0% <0%> (ø)` | |
|
[...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java](https://codecov.io/gh/apache/kylin/pull/268/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9jdWJlL2lubWVtY3ViaW5nL01lbURpc2tTdG9yZS5qYXZh)
| `69.6% <0%> (-0.61%)` | `7% <0%> (ø)` | |
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/kylin/pull/268?src=pr=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/kylin/pull/268?src=pr=footer). Last
update
[10587a6...1708a23](https://codecov.io/gh/apache/kylin/pull/268?src=pr=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Select with Catalog fails
> -
>
> Key: KYLIN-3594
> URL: https://issues.apache.org/jira/browse/KYLIN-3594
> Project: Kylin
> Issue Type: Bug
>Reporter: Hosur Narahari
>Assignee: XiaoXiang Yu
>Priority: Major
>
> By using DatabaseMetaData if we get catalog using getCatalogs() method, it
> return value "defaultCatalog". It returns actual hive schema when we execute
> getSchemas().
> According to JDBC contract, catalog.schema.table should be valid from clause
> and many query layers use that. But kylin fails when we execute that query.
> I've tried to write sample code piece for that below.
>
> _DatabaseMetaData db = conn.getMetaData();_
> _ResultSet catalogSet = db.getCatalogs();_
> _String catalog = "";_
> _if(catalogSet.next()) {_
> _catalog = catalogSet.getString("TABLE_CAT");_
> _}_
> _ResultSet schemaSet = db.getSchemas();_
> _String schema = "";_
> _if(schemaSet.next()) {_
> _schema = schemaSet.getString("TABLE_SCHEM");_
> _}_
> _StringBuilder sb = new StringBuilder("SELECT * FROM ");_
> _if(!catalog.isEmpty()) {_
> _sb.append(catalog + ".");_
> _}_
>
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632839#comment-16632839 ] ASF GitHub Bot commented on KYLIN-3597: --- coveralls commented on issue #274: KYLIN-3597 Improve code smell URL: https://github.com/apache/kylin/pull/274#issuecomment-425620750 ## Pull Request Test Coverage Report for [Build 3713](https://coveralls.io/builds/19260169) * **21** of **38** **(55.26%)** changed or added relevant lines in **3** files are covered. * **2** unchanged lines in **2** files lost coverage. * Overall coverage increased (+**0.002%**) to **23.174%** --- | Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | | :-|--||---: | | [core-cube/src/main/java/org/apache/kylin/gridtable/GTUtil.java](https://coveralls.io/builds/19260169/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fgridtable%2FGTUtil.java#L197) | 0 | 2 | 0.0% | [server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java](https://coveralls.io/builds/19260169/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FCubeController.java#L370) | 0 | 2 | 0.0% | [core-cube/src/main/java/org/apache/kylin/gridtable/GTFilterScanner.java](https://coveralls.io/builds/19260169/source?filename=core-cube%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fgridtable%2FGTFilterScanner.java#L85) | 21 | 34 | 61.76% | Files with Coverage Reduction | New Missed Lines | % | | :-|--|--: | | [server-base/src/main/java/org/apache/kylin/rest/controller/CubeController.java](https://coveralls.io/builds/19260169/source?filename=server-base%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Frest%2Fcontroller%2FCubeController.java#L369) | 1 | 0.0% | | [core-common/src/main/java/org/apache/kylin/common/persistence/IdentifierFileResourceStore.java](https://coveralls.io/builds/19260169/source?filename=core-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fkylin%2Fcommon%2Fpersistence%2FIdentifierFileResourceStore.java#L40) | 1 | 0.0% | | Totals | [](https://coveralls.io/builds/19260169) | | :-- | --: | | Change from base [Build 3711](https://coveralls.io/builds/19259592): | 0.002% | | Covered Lines: | 16162 | | Relevant Lines: | 69742 | --- # - [Coveralls](https://coveralls.io) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix sonar reported static code issues > - > > Key: KYLIN-3597 > URL: https://issues.apache.org/jira/browse/KYLIN-3597 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shaofeng SHI >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632837#comment-16632837 ] ASF GitHub Bot commented on KYLIN-3597: --- codecov-io commented on issue #274: KYLIN-3597 Improve code smell URL: https://github.com/apache/kylin/pull/274#issuecomment-425620454 # [Codecov](https://codecov.io/gh/apache/kylin/pull/274?src=pr=h1) Report > Merging [#274](https://codecov.io/gh/apache/kylin/pull/274?src=pr=desc) into [master](https://codecov.io/gh/apache/kylin/commit/10587a65fe0552179a5c8a6e1151686ce1c8a135?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `42.1%`. [](https://codecov.io/gh/apache/kylin/pull/274?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #274 +/- ## + Coverage 21.16% 21.16% +<.01% Complexity 4405 4405 Files 1086 1086 Lines 6974569742 -3 Branches 1008810087 -1 Hits 1476114761 + Misses5358653582 -4 - Partials 1398 1399 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/kylin/pull/274?src=pr=tree) | Coverage Δ | Complexity Δ | | |---|---|---|---| | [...torage/gtrecord/SortedIteratorMergerWithLimit.java](https://codecov.io/gh/apache/kylin/pull/274/diff?src=pr=tree#diff-Y29yZS1zdG9yYWdlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9zdG9yYWdlL2d0cmVjb3JkL1NvcnRlZEl0ZXJhdG9yTWVyZ2VyV2l0aExpbWl0LmphdmE=) | `78.72% <ø> (ø)` | `2 <0> (ø)` | :arrow_down: | | [...ommon/persistence/IdentifierFileResourceStore.java](https://codecov.io/gh/apache/kylin/pull/274/diff?src=pr=tree#diff-Y29yZS1jb21tb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL2NvbW1vbi9wZXJzaXN0ZW5jZS9JZGVudGlmaWVyRmlsZVJlc291cmNlU3RvcmUuamF2YQ==) | `0% <ø> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...c/main/java/org/apache/kylin/gridtable/GTUtil.java](https://codecov.io/gh/apache/kylin/pull/274/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9ncmlkdGFibGUvR1RVdGlsLmphdmE=) | `1.37% <0%> (ø)` | `2 <0> (ø)` | :arrow_down: | | [...g/apache/kylin/rest/controller/CubeController.java](https://codecov.io/gh/apache/kylin/pull/274/diff?src=pr=tree#diff-c2VydmVyLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2t5bGluL3Jlc3QvY29udHJvbGxlci9DdWJlQ29udHJvbGxlci5qYXZh) | `0% <0%> (ø)` | `0 <0> (ø)` | :arrow_down: | | [...va/org/apache/kylin/gridtable/GTFilterScanner.java](https://codecov.io/gh/apache/kylin/pull/274/diff?src=pr=tree#diff-Y29yZS1jdWJlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9ncmlkdGFibGUvR1RGaWx0ZXJTY2FubmVyLmphdmE=) | `37.34% <47.05%> (+0.76%)` | `2 <1> (ø)` | :arrow_down: | | [...lin/dict/lookup/cache/RocksDBLookupTableCache.java](https://codecov.io/gh/apache/kylin/pull/274/diff?src=pr=tree#diff-Y29yZS1kaWN0aW9uYXJ5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9reWxpbi9kaWN0L2xvb2t1cC9jYWNoZS9Sb2Nrc0RCTG9va3VwVGFibGVDYWNoZS5qYXZh) | `76.16% <0%> (-0.52%)` | `27% <0%> (ø)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/kylin/pull/274?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/kylin/pull/274?src=pr=footer). Last update [10587a6...fbb2f66](https://codecov.io/gh/apache/kylin/pull/274?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix sonar reported static code issues > - > > Key: KYLIN-3597 > URL: https://issues.apache.org/jira/browse/KYLIN-3597 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shaofeng SHI >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632821#comment-16632821 ] ASF GitHub Bot commented on KYLIN-3597: --- asfgit commented on issue #274: KYLIN-3597 Improve code smell URL: https://github.com/apache/kylin/pull/274#issuecomment-425619077 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix sonar reported static code issues > - > > Key: KYLIN-3597 > URL: https://issues.apache.org/jira/browse/KYLIN-3597 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shaofeng SHI >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632820#comment-16632820 ] ASF GitHub Bot commented on KYLIN-3597: --- asfgit commented on issue #274: KYLIN-3597 Improve code smell URL: https://github.com/apache/kylin/pull/274#issuecomment-425619076 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix sonar reported static code issues > - > > Key: KYLIN-3597 > URL: https://issues.apache.org/jira/browse/KYLIN-3597 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shaofeng SHI >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KYLIN-3597) Fix sonar reported static code issues
[ https://issues.apache.org/jira/browse/KYLIN-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16632819#comment-16632819 ] ASF GitHub Bot commented on KYLIN-3597: --- hit-lacus opened a new pull request #274: KYLIN-3597 Improve code smell URL: https://github.com/apache/kylin/pull/274 https://issues.apache.org/jira/projects/KYLIN/issues/KYLIN-3597 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix sonar reported static code issues > - > > Key: KYLIN-3597 > URL: https://issues.apache.org/jira/browse/KYLIN-3597 > Project: Kylin > Issue Type: Improvement > Components: Others >Reporter: Shaofeng SHI >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
