blocked because of many connection errors from mysql

[liuzhixin]

HI, kylin team:

There is something wrong with the connection from mysql.
#
Kylin version: 2.5.0-hadoop3.1 for hbase 2.0
#
NestedThrowables:
java.sql.SQLException: Unable to open a test connection to the given database. 
JDBC url = 
jdbc:mysql://metadb-forhivereplica.c5yzcdreb1xr.us-east-1.rds.amazonaws.com:3306/hive_emr_test?characterEncoding=UTF-8=true,
 username = root. Terminating connection pool (set lazyInit to true if you 
expect to start your database after your app). Original Exception: --
java.sql.SQLException: null,  message from server: "Host '172.31.26.19' is 
blocked because of many connection errors; unblock with 'mysqladmin 
flush-hosts'"
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1055)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1095)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2031)
at com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:718)
at com.mysql.jdbc.JDBC4Connection.(JDBC4Connection.java:46)
at sun.reflect.GeneratedConstructorAccessor91.newInstance(Unknown 
Source)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:302)
at 
com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:282)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
at com.jolbox.bonecp.BoneCP.(BoneCP.java:416)
at 
com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
at 
org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:483)
at 
org.datanucleus.store.rdbms.RDBMSStoreManager.(RDBMSStoreManager.java:297)
at sun.reflect.GeneratedConstructorAccessor125.newInstance(Unknown 
Source)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at 
org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:606)
at 
org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
at 
org.datanucleus.NucleusContextHelper.createStoreManagerForProperties(NucleusContextHelper.java:133)
at 
org.datanucleus.PersistenceNucleusContextImpl.initialise(PersistenceNucleusContextImpl.java:422)
at 
org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:817)
at 
org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:334)
at 
org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:213)
at sun.reflect.GeneratedMethodAccessor147.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1975)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1970)
at 
javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1177)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:814)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:702)
at 
org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:519)
at 
org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:548)
at 
org.apache.hadoop.hive.metastore.ObjectStore.initializeHelper(ObjectStore.java:403)
at 
org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:340)
at 
org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:301)
at 
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
at 
org.apache.hadoop.hive.metastore.RawStoreProxy.(RawStoreProxy.java:58)
at 
org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67)
at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStoreForConf(HiveMetaStore.java:624)
at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMSForConf(HiveMetaStore.java:590)
  

Re: Re: Re: Slow Query Performance With 'WHERE' Clause

[Shrikant Bang]

Thank you Ma, for response and making me correct about query filters( WHERE
clause) in Kylin.

I got a an idea that Kylin is using HBase FuzzyFilters

while
scanning rows from HBase at coprocessor end.

This also clarified me  importance of order of the dimensions in the
rowkey.

Regards,
Shrikant Bang.


On Wed, Oct 24, 2018 at 7:56 AM Ma Gang  wrote:

> Hi Shrikant,
> Which Kylin version you used? By default, most of the filters will be
> pushed down to coprocessor, you may grep "Total filtered row:" in your
> log to see how many rows are filtered from coprocessor side.
>
> If you want to change code to add some log there, you can add log at
> method: CubeHBaseEndpointRPC.getGTScanner(final GTScanRequest scanRequest)
> the parameter GTScanRequest has a field:filterPushDown, is the filter
> push down to coprocessor, you may log some information there, but the
> filter is not the original filter you input but the converted filter that
> has constant values encoded.
>
>
> At 2018-10-24 01:00:15, "Shrikant Bang"  wrote:
>
> Hi Team,
>
>   I tried debugging code to understand query execution (on Kylin v2.5
> hbase 1.x) . As I understood query is formed and executed using Calcite
> APIs.
>
>   Could you please help me to get answer of below?
>
>   1.  With my understanding default implementation of filter operation
> (WHERE clause) is running on Kylin’s node. Is it correct?
>
>   2.  How to make use of CoprocessorFilter
> 
> for running filters on HBase side (if it is not enabled by default)?
>
>
> Thank You,
> Shrikant Bang
>
> On Tue, Oct 23, 2018 at 4:33 PM Shrikant Bang 
> wrote:
>
>> Thank you Ma for response.
>>
>> I didn't find any logs about applying filters in co-processors. Also I
>> don't see logs for logical/physical query plans in kylin.log. Am I missing
>> anything?
>>
>> I would like to put logger statements in code and build specific
>> module(s) as required for tracing latencies -- would you please suggest any
>> classes which can provide me these details?
>>
>> Regards,
>> Shrikant Bang.
>>
>> On Tue, Oct 23, 2018 at 3:09 PM Ma Gang  wrote:
>>
>>> You may post your query related log here, there should be some query log
>>> that indicated whether the filter is push down or not, at least in the
>>> returned response stats, there's some log show how many rows are filtered
>>> in the coprocessor side.
>>>
>>> At 2018-10-23 16:58:36, "Sachin Aggarwal"  
>>> wrote:
>>> >Hi Team,
>>> >
>>> > Could you please confirm if filtering of rows (WHERE clause) is done
>>> >in CoProcessor side?
>>> >
>>> > Is there any APIs/logging to get physical plan of query? It will help
>>> >us in optimising the cube.
>>> >
>>> >
>>> >
>>> >On Mon, Oct 22, 2018 at 8:58 PM Shrikant Bang 
>>> >wrote:
>>> >
>>> >> Thanks ShaoFeng for response. I will try this and will update the results
>>> >> of my queries.
>>> >>
>>> >> I would like to learn, how to identify the bottleneck in query 
>>> >> executions.
>>> >> Can we trace the query execution in each stage with timestamp?
>>> >>
>>> >> Also is there way we can get physical plan of query? This could help me 
>>> >> to
>>> >> design/tune my cube/queries for better response time.
>>> >>
>>> >> Regards,
>>> >> Shrikant Bang
>>> >>
>>> >> On Mon, Oct 22, 2018 at 8:01 PM ShaoFeng Shi 
>>> >> wrote:
>>> >>
>>> >>> Hi Shrikant,
>>> >>>
>>> >>> What's the order of the dimensions in the rowkey? In this case, you need
>>> >>> to put "d3" at the heading position of the rowkey.
>>> >>>
>>> >>> Here is a good reference on how to design a cube, maybe we need add that
>>> >>> into the FAQ or as a part of document:
>>> >>> https://www.slideshare.net/YangLi43/design-cube-in-apache-kylin
>>> >>>
>>> >>> Shrikant Bang  于2018年10月22日周一 下午3:51写道：
>>> >>>
>>>  Hi Team,
>>> 
>>>  We are working on benchmark test for Kylin v2.5-Hbase-1.x as part of 
>>>  PoC.
>>> 
>>>  Here is my cube (pseudo) :
>>> 
>>>  *Dimension Table* : D1
>>>  *Fact Table* : F1, F2
>>> 
>>>  *Metrics* : SUM(D1.m1), SUM(D2.m2)
>>>  *Dimension Columns* -- Normal (D1.d1, D1.d2, D1.d3, F1.a1, F2.b1 )
>>> 
>>>  JOIN (D1.d1 = F1.a1 AND D2.d2 = F2.b1)
>>> 
>>>  When I run a query matching to the cuboids it runs very fast :
>>>  pseudo example query:
>>> 
>>>  SELECT SUM(D1.m1), SUM(D2.m2), d1, d2, d3
>>>  FROM D1
>>>  JOIN F1
>>>  ON D1.d1 = F1.a1
>>>  JOIN F2
>>>  ON D1.d2 = F2.b1
>>>  GROUP BY d1, d2, d3
>>> 
>>> 
>>>  But when I add where clause to query it become very slow in response
>>>  pseudo example query:
>>> 
>>>  SELECT SUM(D1.m1), SUM(D2.m2), d1, d2, d3
>>>  FROM D1
>>>  JOIN F1
>>>  ON D1.d1 = F1.a1

[jira] [Created] (KYLIN-3647) Fix inconsistent states of job and its sub-task

[XiaoXiang Yu (JIRA)]

XiaoXiang Yu created KYLIN-3647:
---

 Summary: Fix inconsistent states of job and its sub-task
 Key: KYLIN-3647
 URL: https://issues.apache.org/jira/browse/KYLIN-3647
 Project: Kylin
  Issue Type: Bug
  Components: Job Engine
Affects Versions: v2.5.0
Reporter: XiaoXiang Yu






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

user ADMIN can't login

[Fei Yi]

User ADMIN can't log in. After restarting kylin, it will be normal, but
can't log in the next day, ANALYST user has been normal.

HTTP Status 500 %E2%80%93 Internal Server ErrorHTTP Status 500 %E2%80%93
Internal Server Error
--

*Type* Exception Report

*Message* Overwriting conflict /user/ADMIN, expect old TS 1540406402017,
but it is 1540406402299

*Description* The server encountered an unexpected condition that prevented
it from fulfilling the request.

*Exception*

org.apache.kylin.common.persistence.WriteConflictException:
Overwriting conflict /user/ADMIN, expect old TS 1540406402017, but it
is 
1540406402299%0A%09org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)%0A%09org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:323)%0A%09org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:308)%0A%09org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:287)%0A%09org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192)%0A%09org.apache.kylin.rest.security.KylinUserManager.update(KylinUserManager.java:122)%0A%09org.apache.kylin.rest.service.KylinUserService.updateUser(KylinUserService.java:85)%0A%09org.apache.kylin.rest.security.KylinAuthenticationProvider.authenticate(KylinAuthenticationProvider.java:117)%0A%09org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:174)%0A%09org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:199)%0A%09org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:180)%0A%09org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)%0A%09org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)%0A%09org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)%0A%09org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)%0A%09org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)%0A%09org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)%0A%09com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209)%0A%09com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244)%0A

*Note* The full stack trace of the root cause is available in the server
logs.
--
Apache Tomcat/7.0.90 HTTP Status 500 – Internal Server Error
--

*Type* Exception Report

*Message* Overwriting conflict /user/ADMIN, expect old TS 1540406402017,
but it is 1540406402299

*Description* The server encountered an unexpected condition that prevented
it from fulfilling the request.

*Exception*

org.apache.kylin.common.persistence.WriteConflictException:
Overwriting conflict /user/ADMIN, expect old TS 1540406402017, but it
is 1540406402299

org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)

org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:323)

org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:308)

org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:287)

[jira] [Created] (KYLIN-3648) Kylin may not add "kylin.storage.hbase.cluster-hdfs-config-file" to configuration properly

[Shaofeng SHI (JIRA)]

Shaofeng SHI created KYLIN-3648:
---

 Summary: Kylin may not add 
"kylin.storage.hbase.cluster-hdfs-config-file" to configuration properly
 Key: KYLIN-3648
 URL: https://issues.apache.org/jira/browse/KYLIN-3648
 Project: Kylin
  Issue Type: Bug
  Components: Storage - HBase
Reporter: Shaofeng SHI


From this blog,  we know that Configuration.addResource should better use a 
Path as the input instead of a string:
conf.addResource(new Path("/home/hadoop/conf/core-site.xml"));
[https://dzone.com/articles/debugging-%E2%80%9Cwrong-fs-expected]

 

But in Kylin's HBaseConnection, it add the hbase cluster fs configure file as 
string:
{code:java}
String hdfsConfigFile = 
KylinConfig.getInstanceFromEnv().getHBaseClusterHDFSConfigFile();
if (hdfsConfigFile == null || hdfsConfigFile.isEmpty()) {
return;
}
Configuration hdfsConf = new Configuration(false);
hdfsConf.addResource(hdfsConfigFile);
{code}
This may make it doesn't work.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: user ADMIN can't login

[倪春恩]

Hi Fei Yi,

Is your kylin in a cluster mode (multiple instances)?

Best regards,
 
Chun’en Ni(George)

- 原始邮件 -
发件人: "Fei Yi" 
收件人: dev@kylin.apache.org, u...@kylin.apache.org
发送时间: 星期四, 2018年 10 月 25日 上午 11:20:21
主题: user ADMIN can't login

User ADMIN can't log in. After restarting kylin, it will be normal, but
can't log in the next day, ANALYST user has been normal.

HTTP Status 500 %E2%80%93 Internal Server ErrorHTTP Status 500 %E2%80%93
Internal Server Error
--

*Type* Exception Report

*Message* Overwriting conflict /user/ADMIN, expect old TS 1540406402017,
but it is 1540406402299

*Description* The server encountered an unexpected condition that prevented
it from fulfilling the request.

*Exception*

org.apache.kylin.common.persistence.WriteConflictException:
Overwriting conflict /user/ADMIN, expect old TS 1540406402017, but it
is 
1540406402299%0A%09org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)%0A%09org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:323)%0A%09org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:308)%0A%09org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:287)%0A%09org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192)%0A%09org.apache.kylin.rest.security.KylinUserManager.update(KylinUserManager.java:122)%0A%09org.apache.kylin.rest.service.KylinUserService.updateUser(KylinUserService.java:85)%0A%09org.apache.kylin.rest.security.KylinAuthenticationProvider.authenticate(KylinAuthenticationProvider.java:117)%0A%09org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:174)%0A%09org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:199)%0A%09org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:180)%0A%09org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)%0A%09org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)%0A%09org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)%0A%09org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)%0A%09org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)%0A%09org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)%0A%09com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209)%0A%09com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244)%0A

*Note* The full stack trace of the root cause is available in the server
logs.
--
Apache Tomcat/7.0.90 HTTP Status 500 – Internal Server Error
--

*Type* Exception Report

*Message* Overwriting conflict /user/ADMIN, expect old TS 1540406402017,
but it is 1540406402299

*Description* The server encountered an unexpected condition that prevented
it from fulfilling the request.

*Exception*

org.apache.kylin.common.persistence.WriteConflictException:
Overwriting conflict /user/ADMIN, expect old TS 1540406402017, but it
is 1540406402299

org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)

org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:323)
 

Re: user ADMIN can't login

[Fei Yi]

env:kylin2.5.0-cdh57 and  cdh.5.14.4

Fei Yi  于2018年10月25日周四 上午11:20写道：

> User ADMIN can't log in. After restarting kylin, it will be normal, but
> can't log in the next day, ANALYST user has been normal.
>
> HTTP Status 500 %E2%80%93 Internal Server ErrorHTTP Status 500 %E2%80%93
> Internal Server Error
> --
>
> *Type* Exception Report
>
> *Message* Overwriting conflict /user/ADMIN, expect old TS 1540406402017,
> but it is 1540406402299
>
> *Description* The server encountered an unexpected condition that
> prevented it from fulfilling the request.
>
> *Exception*
>
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting 
> conflict /user/ADMIN, expect old TS 1540406402017, but it is 
> 1540406402299%0A%09org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)%0A%09org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:323)%0A%09org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:308)%0A%09org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:287)%0A%09org.apache.kylin.metadata.cachesync.CachedCrudAssist.save(CachedCrudAssist.java:192)%0A%09org.apache.kylin.rest.security.KylinUserManager.update(KylinUserManager.java:122)%0A%09org.apache.kylin.rest.service.KylinUserService.updateUser(KylinUserService.java:85)%0A%09org.apache.kylin.rest.security.KylinAuthenticationProvider.authenticate(KylinAuthenticationProvider.java:117)%0A%09org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:174)%0A%09org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:199)%0A%09org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:180)%0A%09org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)%0A%09org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)%0A%09org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)%0A%09org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)%0A%09org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)%0A%09org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)%0A%09org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)%0A%09org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)%0A%09com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:209)%0A%09com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:244)%0A
>
> *Note* The full stack trace of the root cause is available in the server
> logs.
> --
> Apache Tomcat/7.0.90 HTTP Status 500 – Internal Server Error
> --
>
> *Type* Exception Report
>
> *Message* Overwriting conflict /user/ADMIN, expect old TS 1540406402017,
> but it is 1540406402299
>
> *Description* The server encountered an unexpected condition that
> prevented it from fulfilling the request.
>
> *Exception*
>
> org.apache.kylin.common.persistence.WriteConflictException: Overwriting 
> conflict /user/ADMIN, expect old TS 1540406402017, but it is 1540406402299
>   
> org.apache.kylin.storage.hbase.HBaseResourceStore.checkAndPutResourceImpl(HBaseResourceStore.java:325)
>   
> org.apache.kylin.common.persistence.ResourceStore.checkAndPutResourceCheckpoint(ResourceStore.java:323)
>   
> org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:308)
>   
> 

Streaming Table和Kafka遇到的Timeout expired while fetching topic metadata问题

[elbert....@qq.com]

大家好，我按照http://kylin.apache.org/cn/docs/tutorial/cube_streaming.html这个教程一路配置下来，做好model和cube，到build的时候报错，如下：

生产的命令：
/opt/apache-kylin-2.5.0-bin-cdh57/bin/kylin.sh 
org.apache.kylin.source.kafka.util.KafkaSampleProducer --topic kylin_demo_1  
--broker 192.168.1.14:9092
还有部分截图
kylin.log的日志如下：
2018-10-19 15:25:31,868 ERROR [http-bio-7070-exec-11] 
controller.CubeController:401 : Timeout expired while fetching topic metadata
org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching 
topic metadata
2018-10-19 15:25:31,873 ERROR [http-bio-7070-exec-11] 
controller.BasicController:62 :
org.apache.kylin.rest.exception.InternalErrorException: Timeout expired while 
fetching topic metadata
at 
org.apache.kylin.rest.controller.CubeController.buildInternal(CubeController.java:402)
at 
org.apache.kylin.rest.controller.CubeController.rebuild2(CubeController.java:385)
at 
org.apache.kylin.rest.controller.CubeController.build2(CubeController.java:376)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
at 
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
at 
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at 
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at 
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
at 
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at 
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at 
org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:653)
at 
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at 
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at 
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at 
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at 
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at 
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at 

Re: Streaming Table和Kafka遇到的Timeout expired while fetching topic metadata问题

[elbert.lau]

https://www.oschina.net/question/3821020_2288164?sort=default

--
Sent from: http://apache-kylin.74782.x6.nabble.com/

[jira] [Created] (KYLIN-3646) Add instruction about rowkey sequence in "Advanced setting" page

[Shaofeng SHI (JIRA)]

Shaofeng SHI created KYLIN-3646:
---

 Summary: Add instruction about rowkey sequence in "Advanced 
setting" page
 Key: KYLIN-3646
 URL: https://issues.apache.org/jira/browse/KYLIN-3646
 Project: Kylin
  Issue Type: Improvement
  Components: Web 
Reporter: Shaofeng SHI


The position of dimension in rowkey is important for performance (both build 
and query); But this message is not clear for a normal user. We'd better add 
instruction on the web GUI, so that user knows to adjust it by some rule.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)