[jira] [Commented] (KYLIN-5114) Build cube error:java.lang.NoSuchMethodError
[
https://issues.apache.org/jira/browse/KYLIN-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581608#comment-17581608
]
dingpeng commented on KYLIN-5114:
-
yarn上是没有job信息的，是不是还是哪个包没配好
> Build cube error:java.lang.NoSuchMethodError
>
>
> Key: KYLIN-5114
> URL: https://issues.apache.org/jira/browse/KYLIN-5114
> Project: Kylin
> Issue Type: Task
> Components: Environment , Spark Engine
>Affects Versions: v4.0.0
> Environment: centos7
> Hive 3.1.2
> hadoop 3.1.3
> spark 2.4.5
>Reporter: qingquanzhang
>Assignee: mukvin
>Priority: Major
> Labels: Question
> Fix For: Future
>
> Attachments: image-2021-11-04-16-36-25-537.png,
> image-2021-11-04-17-34-11-875.png, kylin.log
>
>
> Build cube error:
> {code:java}
> Caused by: java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V{code}
> So I tryed move $HIVE_HOME/lib/guava-27.0-jre.jar into $KYLIN_HOME/tools/ and
> $KYLIN_HOME/tomcat/lib/ and restart kylin,but still the same error.
>
> !image-2021-11-04-16-36-25-537.png!
> 虽然报错了,但是该segment却有了记录,再次构建cube就会提示segment已存在.如果删除这个segment,还会报错,报错信息如下
> !image-2021-11-04-17-34-11-875.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (KYLIN-5221) Prepare general document for package and testing
[ https://issues.apache.org/jira/browse/KYLIN-5221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu updated KYLIN-5221: Fix Version/s: 5.0-alpha > Prepare general document for package and testing > - > > Key: KYLIN-5221 > URL: https://issues.apache.org/jira/browse/KYLIN-5221 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > Fix For: 5.0-alpha > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KYLIN-5220) New integration testing module
[ https://issues.apache.org/jira/browse/KYLIN-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu updated KYLIN-5220: Fix Version/s: 5.0-alpha > New integration testing module > -- > > Key: KYLIN-5220 > URL: https://issues.apache.org/jira/browse/KYLIN-5220 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > Fix For: 5.0-alpha > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KYLIN-5219) Refine package name
[ https://issues.apache.org/jira/browse/KYLIN-5219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu updated KYLIN-5219: Fix Version/s: 5.0-alpha > Refine package name > --- > > Key: KYLIN-5219 > URL: https://issues.apache.org/jira/browse/KYLIN-5219 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Assignee: mukvin >Priority: Major > Fix For: 5.0-alpha > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KYLIN-5222) Overall introduction article of Kylin 5.0
[ https://issues.apache.org/jira/browse/KYLIN-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu updated KYLIN-5222: Fix Version/s: 5.0-alpha > Overall introduction article of Kylin 5.0 > - > > Key: KYLIN-5222 > URL: https://issues.apache.org/jira/browse/KYLIN-5222 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Assignee: Xiaoxiang Yu >Priority: Major > Fix For: 5.0-alpha > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KYLIN-5218) Remove wrong license header
[ https://issues.apache.org/jira/browse/KYLIN-5218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu updated KYLIN-5218: Fix Version/s: 5.0-alpha > Remove wrong license header > --- > > Key: KYLIN-5218 > URL: https://issues.apache.org/jira/browse/KYLIN-5218 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Assignee: mukvin >Priority: Major > Fix For: 5.0-alpha > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KYLIN-5226) Fix wrong name and classpath in shell in kylin5.0
[ https://issues.apache.org/jira/browse/KYLIN-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu resolved KYLIN-5226. - Resolution: Fixed > Fix wrong name and classpath in shell in kylin5.0 > - > > Key: KYLIN-5226 > URL: https://issues.apache.org/jira/browse/KYLIN-5226 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-alpha >Reporter: Zhenning Zhang >Assignee: Zhenning Zhang >Priority: Blocker > Fix For: 5.0-alpha > > > There are some wrong classpath in .sh file at build/bin of kylin5.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (KYLIN-5226) Fix wrong name and classpath in shell in kylin5.0
[ https://issues.apache.org/jira/browse/KYLIN-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu updated KYLIN-5226: Fix Version/s: 5.0-alpha > Fix wrong name and classpath in shell in kylin5.0 > - > > Key: KYLIN-5226 > URL: https://issues.apache.org/jira/browse/KYLIN-5226 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-alpha >Reporter: Zhenning Zhang >Assignee: Zhenning Zhang >Priority: Blocker > Fix For: 5.0-alpha > > > There are some wrong classpath in .sh file at build/bin of kylin5.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (KYLIN-5226) Fix wrong name and classpath in shell in kylin5.0
[ https://issues.apache.org/jira/browse/KYLIN-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu reassigned KYLIN-5226: --- Assignee: Zhenning Zhang > Fix wrong name and classpath in shell in kylin5.0 > - > > Key: KYLIN-5226 > URL: https://issues.apache.org/jira/browse/KYLIN-5226 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-alpha >Reporter: Zhenning Zhang >Assignee: Zhenning Zhang >Priority: Blocker > > There are some wrong classpath in .sh file at build/bin of kylin5.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5226) Fix wrong name and classpath in shell in kylin5.0
[ https://issues.apache.org/jira/browse/KYLIN-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581593#comment-17581593 ] Xiaoxiang Yu commented on KYLIN-5226: - Looks like Zhenning has use wrong Jira ID, the correct commit should be [https://github.com/apache/kylin/commit/06ba9dc5d62a4802b325e2250178331230c5a058] . > Fix wrong name and classpath in shell in kylin5.0 > - > > Key: KYLIN-5226 > URL: https://issues.apache.org/jira/browse/KYLIN-5226 > Project: Kylin > Issue Type: Bug >Affects Versions: 5.0-alpha >Reporter: Zhenning Zhang >Priority: Blocker > > There are some wrong classpath in .sh file at build/bin of kylin5.0. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (KYLIN-5229) PropertiesDelegate introduce lock to KylinConfig
[
https://issues.apache.org/jira/browse/KYLIN-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoxiang Yu reassigned KYLIN-5229:
---
Assignee: mukvin
> PropertiesDelegate introduce lock to KylinConfig
>
>
> Key: KYLIN-5229
> URL: https://issues.apache.org/jira/browse/KYLIN-5229
> Project: Kylin
> Issue Type: Sub-task
> Components: Metadata
>Reporter: Xiaoxiang Yu
>Assignee: mukvin
>Priority: Major
> Fix For: 5.0-alpha
>
> Attachments: image-2022-08-17-13-17-40-751.png
>
>
> h2. Background
> Kyligence team plans to add new feature which supports reading config entry
> from external Config Server(such as Nacos) other than local kylin.properties.
> So we add {{PropertiesDelegate}} and {{IExternalConfigLoader}} to support
> this request.
> h2. Issue
> {{{}PropertiesDelegate introduces performance downgrade because it add lock
> to KylinConfigBase#getOptional{}}}, impact query concurrency badly, from
> Yanghong Team's load testing result, query concurrency change from 60 to 3.
> h2. Root Cause
>
> {code:java}
> KylinConfigBase
> protected String getOptional(String prop, String dft) {
> final String property = System.getProperty(prop);
> return property != null ? getSubstitutor().replace(property)
> : getSubstitutor().replace(properties.getProperty(prop, dft)); //
> Step 1
> }
>
> protected StrSubstitutor getSubstitutor() {
> // overrides > env > properties
> final Map all = Maps.newHashMap();
> all.putAll((Map) properties); // Step 2
> all.putAll(STATIC_SYSTEM_ENV);
> all.putAll(overrides);
> return new StrSubstitutor(all);
> }
> PropertiesDelegate
> @Override
> public Set> entrySet() {
> return getAllProperties().entrySet(); // Step 3
> }
> private synchronized Properties getAllProperties() { // Step 4, synchronized
>
> Properties propertiesView = new Properties();
> if (this.configLoader != null) {
> propertiesView.putAll(this.configLoader.getProperties()); // Step 5
> }
> propertiesView.putAll(this.properties);
> return propertiesView;
> }
> Hashtable
> public synchronized void putAll(Map t) {
> for (Map.Entry e : t.entrySet())
> put(e.getKey(), e.getValue()); // Step 7
> }
> public synchronized V put(K key, V value) { // Step 8
> // Make sure the value is not null
> if (value == null) {
> throw new NullPointerException();
> }
> // Makes sure the key is not already in the hashtable.
> Entry tab[] = table;
> int hash = key.hashCode();
> int index = (hash & 0x7FFF) % tab.length;
> @SuppressWarnings("unchecked")
> Entry entry = (Entry)tab[index];
> for(; entry != null ; entry = entry.next) {
> if ((entry.hash == hash) && entry.key.equals(key)) {
> V old = entry.value;
> entry.value = value;
> return old;
> }
> }
> addEntry(hash, key, value, index);
> return null;
> }
> {code}
>
> h2. Screenshots
> Java Stack of Query Thread:
> !image-2022-08-17-13-17-40-751.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (KYLIN-5231) RuntimeException in cube build with Spark engine
[ https://issues.apache.org/jira/browse/KYLIN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581591#comment-17581591 ] Xiaoxiang Yu edited comment on KYLIN-5231 at 8/19/22 1:55 AM: -- [~sandeepputrevu] Hi, Kylin 3.X doesn't support Spark 3.X(only Spark 2.4.x or earlier), and Kylin 4.X doesn't support Spark 3.2.X or higher version(only Spark 2.4.x or Spark 3.1.x). Kylin 5.0 alpha, which is now under development, will support Spark 3.2+. was (Author: xxyu): [~sandeepputrevu] Hi, Kylin 3.X doesn't support Spark 3.X(only Spark 2.4.x), and Kylin 4.X doesn't support Spark 3.2.X or higher version(only Spark 2.4.x or Spark 3.1.x). > RuntimeException in cube build with Spark engine > > > Key: KYLIN-5231 > URL: https://issues.apache.org/jira/browse/KYLIN-5231 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.2 >Reporter: Sandeep Putrevu >Assignee: Xiaoxiang Yu >Priority: Major > > Cube build is failing at *Step #7 : Build Cube with Spark* > > Our configuration: > Spark version : 3.2.1 > Hadoop version : 3.1.1.3.1.4.0 > Kylin version : 3.1.2 > Error : > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause: > scala.reflect.ClassTag$$anon$1 > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955) > at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) > at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) > at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) > at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: java.lang.ClassNotFoundException: scala.reflect.ClassTag$$anon$1 > at java.net.URLClassLoader.findClass(URLClassLoader.java:382) > at java.lang.ClassLoader.loadClass(ClassLoader.java:418) > at java.lang.ClassLoader.loadClass(ClassLoader.java:351) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:142) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 13 more -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (KYLIN-5228) There is length limitation during loading properties to Kylin config in Kylin5.0.
[ https://issues.apache.org/jira/browse/KYLIN-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu resolved KYLIN-5228. - Fix Version/s: 5.0-alpha Resolution: Fixed > There is length limitation during loading properties to Kylin config in > Kylin5.0. > -- > > Key: KYLIN-5228 > URL: https://issues.apache.org/jira/browse/KYLIN-5228 > Project: Kylin > Issue Type: Bug >Reporter: Zhenning Zhang >Assignee: Zhenning Zhang >Priority: Blocker > Fix For: 5.0-alpha > > > Currently we use properties.list() to get kylin config from properties to > string, but this method results in a maximum length limit (which is set as 37 > )for each properties. We can leverage the properties.store() method -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (KYLIN-5228) There is length limitation during loading properties to Kylin config in Kylin5.0.
[ https://issues.apache.org/jira/browse/KYLIN-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu reassigned KYLIN-5228: --- Assignee: Zhenning Zhang > There is length limitation during loading properties to Kylin config in > Kylin5.0. > -- > > Key: KYLIN-5228 > URL: https://issues.apache.org/jira/browse/KYLIN-5228 > Project: Kylin > Issue Type: Bug >Reporter: Zhenning Zhang >Assignee: Zhenning Zhang >Priority: Blocker > > Currently we use properties.list() to get kylin config from properties to > string, but this method results in a maximum length limit (which is set as 37 > )for each properties. We can leverage the properties.store() method -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (KYLIN-5231) RuntimeException in cube build with Spark engine
[ https://issues.apache.org/jira/browse/KYLIN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoxiang Yu reassigned KYLIN-5231: --- Assignee: Xiaoxiang Yu > RuntimeException in cube build with Spark engine > > > Key: KYLIN-5231 > URL: https://issues.apache.org/jira/browse/KYLIN-5231 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.2 >Reporter: Sandeep Putrevu >Assignee: Xiaoxiang Yu >Priority: Major > > Cube build is failing at *Step #7 : Build Cube with Spark* > > Our configuration: > Spark version : 3.2.1 > Hadoop version : 3.1.1.3.1.4.0 > Kylin version : 3.1.2 > Error : > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause: > scala.reflect.ClassTag$$anon$1 > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955) > at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) > at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) > at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) > at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: java.lang.ClassNotFoundException: scala.reflect.ClassTag$$anon$1 > at java.net.URLClassLoader.findClass(URLClassLoader.java:382) > at java.lang.ClassLoader.loadClass(ClassLoader.java:418) > at java.lang.ClassLoader.loadClass(ClassLoader.java:351) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:142) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 13 more -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5231) RuntimeException in cube build with Spark engine
[ https://issues.apache.org/jira/browse/KYLIN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581591#comment-17581591 ] Xiaoxiang Yu commented on KYLIN-5231: - [~sandeepputrevu] Hi, Kylin 3.X doesn't support Spark 3.X(only Spark 2.4.x), and Kylin 4.X doesn't support Spark 3.2.X or higher version(only Spark 2.4.x or Spark 3.1.x). > RuntimeException in cube build with Spark engine > > > Key: KYLIN-5231 > URL: https://issues.apache.org/jira/browse/KYLIN-5231 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.2 >Reporter: Sandeep Putrevu >Priority: Major > > Cube build is failing at *Step #7 : Build Cube with Spark* > > Our configuration: > Spark version : 3.2.1 > Hadoop version : 3.1.1.3.1.4.0 > Kylin version : 3.1.2 > Error : > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause: > scala.reflect.ClassTag$$anon$1 > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955) > at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) > at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) > at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) > at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: java.lang.ClassNotFoundException: scala.reflect.ClassTag$$anon$1 > at java.net.URLClassLoader.findClass(URLClassLoader.java:382) > at java.lang.ClassLoader.loadClass(ClassLoader.java:418) > at java.lang.ClassLoader.loadClass(ClassLoader.java:351) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:142) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 13 more -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5230) The bug from change password
[ https://issues.apache.org/jira/browse/KYLIN-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581445#comment-17581445 ] liyang commented on KYLIN-5230: --- Nice suggestion and thanks for the report! > The bug from change password > > > Key: KYLIN-5230 > URL: https://issues.apache.org/jira/browse/KYLIN-5230 > Project: Kylin > Issue Type: Improvement > Components: Client - CLI >Affects Versions: v4.0.1 >Reporter: Li Can >Priority: Major > Attachments: image.png > > > 1.after login, you can change password by input any letter in current > password field that may not right current password with new password. > 2.after changed password, old password can still work for a period of time if > the system hava cache function, and the new password dosen't work util cached > old password expired. > 3.when the new password can work, if you don't restart the system, you can > only login but cann't access any service.(refer to the attachment). > > The suggestion is to disable the change password function, because the > Authentication may integrated from external system, such as keystone. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5231) RuntimeException in cube build with Spark engine
[ https://issues.apache.org/jira/browse/KYLIN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581444#comment-17581444 ] Sandeep Putrevu commented on KYLIN-5231: This is our first attempt after upgrading the Spark to 3.2.1 , earlier it used to work fine with Spark 2.4 > RuntimeException in cube build with Spark engine > > > Key: KYLIN-5231 > URL: https://issues.apache.org/jira/browse/KYLIN-5231 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.2 >Reporter: Sandeep Putrevu >Priority: Major > > Cube build is failing at *Step #7 : Build Cube with Spark* > > Our configuration: > Spark version : 3.2.1 > Hadoop version : 3.1.1.3.1.4.0 > Kylin version : 3.1.2 > Error : > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause: > scala.reflect.ClassTag$$anon$1 > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955) > at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) > at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) > at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) > at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: java.lang.ClassNotFoundException: scala.reflect.ClassTag$$anon$1 > at java.net.URLClassLoader.findClass(URLClassLoader.java:382) > at java.lang.ClassLoader.loadClass(ClassLoader.java:418) > at java.lang.ClassLoader.loadClass(ClassLoader.java:351) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:142) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 13 more -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5114) Build cube error:java.lang.NoSuchMethodError
[
https://issues.apache.org/jira/browse/KYLIN-5114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581443#comment-17581443
]
liyang commented on KYLIN-5114:
---
通常是数据量太大了，超过了集群现有的资源负载范围，建议用小数据量先试试。
对数据量和所需的资源有数了以后，也可以用人工指定 spark 资源（通过参数配置），来替代 Detect Resource。
> Build cube error:java.lang.NoSuchMethodError
>
>
> Key: KYLIN-5114
> URL: https://issues.apache.org/jira/browse/KYLIN-5114
> Project: Kylin
> Issue Type: Task
> Components: Environment , Spark Engine
>Affects Versions: v4.0.0
> Environment: centos7
> Hive 3.1.2
> hadoop 3.1.3
> spark 2.4.5
>Reporter: qingquanzhang
>Assignee: mukvin
>Priority: Major
> Labels: Question
> Fix For: Future
>
> Attachments: image-2021-11-04-16-36-25-537.png,
> image-2021-11-04-17-34-11-875.png, kylin.log
>
>
> Build cube error:
> {code:java}
> Caused by: java.lang.NoSuchMethodError:
> com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V{code}
> So I tryed move $HIVE_HOME/lib/guava-27.0-jre.jar into $KYLIN_HOME/tools/ and
> $KYLIN_HOME/tomcat/lib/ and restart kylin,but still the same error.
>
> !image-2021-11-04-16-36-25-537.png!
> 虽然报错了,但是该segment却有了记录,再次构建cube就会提示segment已存在.如果删除这个segment,还会报错,报错信息如下
> !image-2021-11-04-17-34-11-875.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (KYLIN-5231) RuntimeException in cube build with Spark engine
[ https://issues.apache.org/jira/browse/KYLIN-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581442#comment-17581442 ] liyang commented on KYLIN-5231: --- Most likely a env problem. Is this set of env used to work? Or it is a first attempt? - Spark version : 3.2.1 - Hadoop version : 3.1.1.3.1.4.0 - Kylin version : 3.1.2 > RuntimeException in cube build with Spark engine > > > Key: KYLIN-5231 > URL: https://issues.apache.org/jira/browse/KYLIN-5231 > Project: Kylin > Issue Type: Bug >Affects Versions: v3.1.2 >Reporter: Sandeep Putrevu >Priority: Major > > Cube build is failing at *Step #7 : Build Cube with Spark* > > Our configuration: > Spark version : 3.2.1 > Hadoop version : 3.1.1.3.1.4.0 > Kylin version : 3.1.2 > Error : > Exception in thread "main" java.lang.RuntimeException: error execute > org.apache.kylin.engine.spark.SparkCubingByLayer. Root cause: > scala.reflect.ClassTag$$anon$1 > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42) > at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) > at > org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955) > at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) > at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) > at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) > at > org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043) > at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052) > at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) > Caused by: java.lang.ClassNotFoundException: scala.reflect.ClassTag$$anon$1 > at java.net.URLClassLoader.findClass(URLClassLoader.java:382) > at java.lang.ClassLoader.loadClass(ClassLoader.java:418) > at java.lang.ClassLoader.loadClass(ClassLoader.java:351) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.kylin.engine.spark.SparkCubingByLayer.execute(SparkCubingByLayer.java:142) > at > org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) > ... 13 more -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5232) Fix checkstyle issue
[ https://issues.apache.org/jira/browse/KYLIN-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581292#comment-17581292 ] ASF subversion and git services commented on KYLIN-5232: Commit cfc55f348bfb1b87533c8ed8637e19f91a26b4e0 in kylin's branch refs/heads/kylin5 from Mukvin [ https://gitbox.apache.org/repos/asf?p=kylin.git;h=cfc55f348b ] KYLIN-5232 fix checkstyle > Fix checkstyle issue > > > Key: KYLIN-5232 > URL: https://issues.apache.org/jira/browse/KYLIN-5232 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > Fix For: 5.0-alpha > > > Fix problems reported by maven checkstyle plugins in kylin5 branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5232) Fix checkstyle issue
[ https://issues.apache.org/jira/browse/KYLIN-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581291#comment-17581291 ] ASF GitHub Bot commented on KYLIN-5232: --- hit-lacus merged PR #1941: URL: https://github.com/apache/kylin/pull/1941 > Fix checkstyle issue > > > Key: KYLIN-5232 > URL: https://issues.apache.org/jira/browse/KYLIN-5232 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > Fix For: 5.0-alpha > > > Fix problems reported by maven checkstyle plugins in kylin5 branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [kylin] hit-lacus merged pull request #1941: KYLIN-5232 fix checkstyle
hit-lacus merged PR #1941: URL: https://github.com/apache/kylin/pull/1941 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@kylin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (KYLIN-5232) Fix checkstyle issue
[ https://issues.apache.org/jira/browse/KYLIN-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581280#comment-17581280 ] ASF GitHub Bot commented on KYLIN-5232: --- Mukvin opened a new pull request, #1941: URL: https://github.com/apache/kylin/pull/1941 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Github Branch As most of the development works are on Kylin 4, we need to switch it as main branch. Apache Kylin community changes the branch settings on Github since 2021-08-04 : 1. The default branch _main_ is for **Kylin 4.x** (Parquet storage); 2. The original branch _master_ for **Kylin 3.x** (HBase storage) has been renamed to **kylin3** ; Please check [Intro to Kylin 4 architecture](https://kylin.apache.org/blog/2021/07/02/Apache-Kylin4-A-new-storage-and-compute-architecture/) and [INFRA-22166](https://issues.apache.org/jira/browse/INFRA-22166) if you are interested. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... > Fix checkstyle issue > > > Key: KYLIN-5232 > URL: https://issues.apache.org/jira/browse/KYLIN-5232 > Project: Kylin > Issue Type: Sub-task >Reporter: Xiaoxiang Yu >Priority: Major > Fix For: 5.0-alpha > > > Fix problems reported by maven checkstyle plugins in kylin5 branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[GitHub] [kylin] Mukvin opened a new pull request, #1941: KYLIN-5232 fix checkstyle
Mukvin opened a new pull request, #1941: URL: https://github.com/apache/kylin/pull/1941 ## Proposed changes Describe the big picture of your changes here to communicate to the maintainers why we should accept this pull request. If it fixes a bug or resolves a feature request, be sure to link to that issue. ## Github Branch As most of the development works are on Kylin 4, we need to switch it as main branch. Apache Kylin community changes the branch settings on Github since 2021-08-04 : 1. The default branch _main_ is for **Kylin 4.x** (Parquet storage); 2. The original branch _master_ for **Kylin 3.x** (HBase storage) has been renamed to **kylin3** ; Please check [Intro to Kylin 4 architecture](https://kylin.apache.org/blog/2021/07/02/Apache-Kylin4-A-new-storage-and-compute-architecture/) and [INFRA-22166](https://issues.apache.org/jira/browse/INFRA-22166) if you are interested. ## Types of changes What types of changes does your code introduce to Kylin? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have create an issue on [Kylin's jira](https://issues.apache.org/jira/browse/KYLIN), and have described the bug/feature there in detail - [ ] Commit messages in my PR start with the related jira ID, like "KYLIN- Make Kylin project open-source" - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if appropriate) - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at u...@kylin.apache.org or d...@kylin.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@kylin.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (KYLIN-5232) Fix checkstyle issue
Xiaoxiang Yu created KYLIN-5232: --- Summary: Fix checkstyle issue Key: KYLIN-5232 URL: https://issues.apache.org/jira/browse/KYLIN-5232 Project: Kylin Issue Type: Sub-task Reporter: Xiaoxiang Yu Fix For: 5.0-alpha Fix problems reported by maven checkstyle plugins in kylin5 branch. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (KYLIN-5229) PropertiesDelegate introduce lock to KylinConfig
[
https://issues.apache.org/jira/browse/KYLIN-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17581221#comment-17581221
]
Xiaoxiang Yu commented on KYLIN-5229:
-
https://github.com/apache/kylin/pull/1940
> PropertiesDelegate introduce lock to KylinConfig
>
>
> Key: KYLIN-5229
> URL: https://issues.apache.org/jira/browse/KYLIN-5229
> Project: Kylin
> Issue Type: Sub-task
> Components: Metadata
>Reporter: Xiaoxiang Yu
>Priority: Major
> Fix For: 5.0-alpha
>
> Attachments: image-2022-08-17-13-17-40-751.png
>
>
> h2. Background
> Kyligence team plans to add new feature which supports reading config entry
> from external Config Server(such as Nacos) other than local kylin.properties.
> So we add {{PropertiesDelegate}} and {{IExternalConfigLoader}} to support
> this request.
> h2. Issue
> {{{}PropertiesDelegate introduces performance downgrade because it add lock
> to KylinConfigBase#getOptional{}}}, impact query concurrency badly, from
> Yanghong Team's load testing result, query concurrency change from 60 to 3.
> h2. Root Cause
>
> {code:java}
> KylinConfigBase
> protected String getOptional(String prop, String dft) {
> final String property = System.getProperty(prop);
> return property != null ? getSubstitutor().replace(property)
> : getSubstitutor().replace(properties.getProperty(prop, dft)); //
> Step 1
> }
>
> protected StrSubstitutor getSubstitutor() {
> // overrides > env > properties
> final Map all = Maps.newHashMap();
> all.putAll((Map) properties); // Step 2
> all.putAll(STATIC_SYSTEM_ENV);
> all.putAll(overrides);
> return new StrSubstitutor(all);
> }
> PropertiesDelegate
> @Override
> public Set> entrySet() {
> return getAllProperties().entrySet(); // Step 3
> }
> private synchronized Properties getAllProperties() { // Step 4, synchronized
>
> Properties propertiesView = new Properties();
> if (this.configLoader != null) {
> propertiesView.putAll(this.configLoader.getProperties()); // Step 5
> }
> propertiesView.putAll(this.properties);
> return propertiesView;
> }
> Hashtable
> public synchronized void putAll(Map t) {
> for (Map.Entry e : t.entrySet())
> put(e.getKey(), e.getValue()); // Step 7
> }
> public synchronized V put(K key, V value) { // Step 8
> // Make sure the value is not null
> if (value == null) {
> throw new NullPointerException();
> }
> // Makes sure the key is not already in the hashtable.
> Entry tab[] = table;
> int hash = key.hashCode();
> int index = (hash & 0x7FFF) % tab.length;
> @SuppressWarnings("unchecked")
> Entry entry = (Entry)tab[index];
> for(; entry != null ; entry = entry.next) {
> if ((entry.hash == hash) && entry.key.equals(key)) {
> V old = entry.value;
> entry.value = value;
> return old;
> }
> }
> addEntry(hash, key, value, index);
> return null;
> }
> {code}
>
> h2. Screenshots
> Java Stack of Query Thread:
> !image-2022-08-17-13-17-40-751.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
