[ 
https://issues.apache.org/jira/browse/KYLIN-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15405617#comment-15405617
 ] 

jiangshouzhuang edited comment on KYLIN-1760 at 8/3/16 9:54 AM:
----------------------------------------------------------------

I'm very happy that you resolve the bug which cannot save queries by Web GUI.
But when I query the htable 'kylin_metadata_user' in release 1.5.3, I find that 
'kylin_metadata_user' column family is 'a', but it need 'q'.

hbase(main):004:0> desc 'kylin_metadata_user'
Table kylin_metadata_user is ENABLED                                            
                                                                                
       
kylin_metadata_user, {TABLE_ATTRIBUTES => {METADATA => {'UUID' => 
'41e483de-7e3e-4374-9b0f-2f361841aba2'}}                                        
                     
COLUMN FAMILIES DESCRIPTION                                                     
                                                                                
       
{NAME => 'a', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', 
REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSIONS 
=> '0', TTL => 'FOREV
ER', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true', 
BLOCKCACHE => 'true'}                                                           
        
1 row(s) in 0.0520 seconds

hbase(main):005:0> alter 'kylin_metadata_user' ,{ NAME => 'q', VERSIONS => '1' }
Updating all regions with the new schema...
0/1 regions updated.
1/1 regions updated.
Done.
0 row(s) in 3.0320 seconds

When I add then column family 'q', I can use 
"http://ipaddress:7070/kylin/query"; to save queries.

hbase(main):006:0> scan 'kylin_metadata_user'
ROW                                        COLUMN+CELL                          
                                                                                
       
 ADMIN                                     column=q:c, timestamp=1470211633112, 
value=[{"name":"s_1","project":"myproject_pvuv","sql":"select * from 
STREAMING_SALES_TA
                                           BLE 
","description":null,"id":"-932856725"}]                                        
                                        
1 row(s) in 0.0480 seconds

The source code is location in 
'apache-kylin-1.5.3\server-base\src\main\java\org\apache\kylin\rest\service\QueryService.java',as
 following:

public static final String USER_QUERY_FAMILY = "q";

public void saveQuery(final String creator, final Query query) throws 
IOException {
        List<Query> queries = getQueries(creator);
        queries.add(query);
        Query[] queryArray = new Query[queries.size()];

        byte[] bytes = querySerializer.serialize(queries.toArray(queryArray));
        HTableInterface htable = null;
        try {
            htable = HBaseConnection.get(hbaseUrl).getTable(userTableName);
            Put put = new Put(Bytes.toBytes(creator));
            put.add(Bytes.toBytes(USER_QUERY_FAMILY), 
Bytes.toBytes(USER_QUERY_COLUMN), bytes);

            htable.put(put);
            htable.flushCommits();
        } finally {
            IOUtils.closeQuietly(htable);
        }
    }
        

I hope that you can take a little time to solve the bug. Thank you very much.

Error log as following :
2016-08-03 17:50:59,656 DEBUG [http-bio-7070-exec-1] hbase.HBaseConnection:238 
: HTable 'kylin_metadata_user' already exists
2016-08-03 17:50:59,661 ERROR [http-bio-7070-exec-1] 
controller.BasicController:44 : 
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: 
org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column family 
q does not exist in region 
kylin_metadata_user,,1470217040618.dfee2dff003b16dfd569b9318e09ddeb. in table 
'kylin_metadata_user', {TABLE_ATTRIBUTES => {METADATA => {'UUID' => 
'a05e588b-8c62-4d81-8b24-60fa66e2fc43'}}, {NAME => 'a', DATA_BLOCK_ENCODING => 
'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => '0', COMPRESSION => 'NONE', 
VERSIONS => '1', TTL => 'FOREVER', MIN_VERSIONS => '0', KEEP_DELETED_CELLS => 
'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true', BLOCKCACHE => 'true'}
        at 
org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:7371)
        at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:6531)
        at 
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1936)
        at 
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32381)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2127)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
        at 
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
        at java.lang.Thread.run(Thread.java:745)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at 
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
        at 
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
        at 
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:326)
        at org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:861)
        at org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:848)
        at 
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:136)
        at org.apache.hadoop.hbase.client.HTable.get(HTable.java:865)
        at org.apache.hadoop.hbase.client.HTable.get(HTable.java:831)
        at 
org.apache.kylin.rest.service.QueryService.getQueries(QueryService.java:192)
        at 
org.apache.kylin.rest.service.QueryService.saveQuery(QueryService.java:129)
        at 
org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke(<generated>)
        at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
        at 
org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
        at 
org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$f2ce01fc.saveQuery(<generated>)
        at 
org.apache.kylin.rest.controller.QueryController.saveQuery(QueryController.java:110)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
        at 
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
        at 
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
        at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
        at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
        at 
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
        at 
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
        at 
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
        at 
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
        at 
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
        at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at 
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
        at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
        at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at 
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at 
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at 
org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at 
org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at 
org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at 
org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at 
org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:91)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at 
org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at 
org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at 
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
        at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at 
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
        at 
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
        at 
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
        at 
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
        at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at 
com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:195)
        at 
com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:266)
        at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
        at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
        at 
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
        at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
        at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
        at 
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)
        at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
        at 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
        at 
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
        at 
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at 
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:745)





was (Author: jiangshouzhuang):
I'm very happy that you resolve the bug which cannot save queries by Web GUI.
But when I query the htable 'kylin_metadata_user' in release 1.5.3, I find that 
'kylin_metadata_user' column family is 'a', but it need 'q'.

hbase(main):004:0> desc 'kylin_metadata_user'
Table kylin_metadata_user is ENABLED                                            
                                                                                
       
kylin_metadata_user, {TABLE_ATTRIBUTES => {METADATA => {'UUID' => 
'41e483de-7e3e-4374-9b0f-2f361841aba2'}}                                        
                     
COLUMN FAMILIES DESCRIPTION                                                     
                                                                                
       
{NAME => 'a', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', 
REPLICATION_SCOPE => '0', VERSIONS => '1', COMPRESSION => 'NONE', MIN_VERSIONS 
=> '0', TTL => 'FOREV
ER', KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'true', 
BLOCKCACHE => 'true'}                                                           
        
1 row(s) in 0.0520 seconds

hbase(main):005:0> alter 'kylin_metadata_user' ,{ NAME => 'q', VERSIONS => '1' }
Updating all regions with the new schema...
0/1 regions updated.
1/1 regions updated.
Done.
0 row(s) in 3.0320 seconds

When I add then column family 'q', I can use 
"http://ipaddress:7070/kylin/query"; to save queries.

hbase(main):006:0> scan 'kylin_metadata_user'
ROW                                        COLUMN+CELL                          
                                                                                
       
 ADMIN                                     column=q:c, timestamp=1470211633112, 
value=[{"name":"s_1","project":"myproject_pvuv","sql":"select * from 
STREAMING_SALES_TA
                                           BLE 
","description":null,"id":"-932856725"}]                                        
                                        
1 row(s) in 0.0480 seconds

The source code is location in 
'apache-kylin-1.5.3\server-base\src\main\java\org\apache\kylin\rest\service\QueryService.java',as
 following:

public static final String USER_QUERY_FAMILY = "q";

public void saveQuery(final String creator, final Query query) throws 
IOException {
        List<Query> queries = getQueries(creator);
        queries.add(query);
        Query[] queryArray = new Query[queries.size()];

        byte[] bytes = querySerializer.serialize(queries.toArray(queryArray));
        HTableInterface htable = null;
        try {
            htable = HBaseConnection.get(hbaseUrl).getTable(userTableName);
            Put put = new Put(Bytes.toBytes(creator));
            put.add(Bytes.toBytes(USER_QUERY_FAMILY), 
Bytes.toBytes(USER_QUERY_COLUMN), bytes);

            htable.put(put);
            htable.flushCommits();
        } finally {
            IOUtils.closeQuietly(htable);
        }
    }
        

I hope that you can take a little time to solve the bug. Thank you very much.

> Save query hits org.apache.hadoop.hbase.TableNotFoundException: 
> kylin_metadata_user
> -----------------------------------------------------------------------------------
>
>                 Key: KYLIN-1760
>                 URL: https://issues.apache.org/jira/browse/KYLIN-1760
>             Project: Kylin
>          Issue Type: Bug
>          Components: Query Engine
>    Affects Versions: v1.5.1
>            Reporter: jiangshouzhuang
>            Assignee: liyang
>             Fix For: v1.5.3
>
>
> When I use "http://ipaddress:7070/kylin/query"; to save queries, I notice the 
> log file kylin.log which report some errors :
> org.apache.hadoop.hbase.TableNotFoundException: Table 'kylin_metadata_user' 
> was not found, got: kylin_metadata.
>       at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1275)
>       at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1156)
>       at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1140)
>       at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1097)
>       at 
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:932)
>       at 
> org.apache.hadoop.hbase.client.HRegionLocator.getRegionLocation(HRegionLocator.java:83)
>       at 
> org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:79)
>       at 
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
>       at org.apache.hadoop.hbase.client.HTable.get(HTable.java:889)
>       at org.apache.hadoop.hbase.client.HTable.get(HTable.java:855)
>       at 
> org.apache.kylin.rest.service.QueryService.getQueries(QueryService.java:189)
>       at 
> org.apache.kylin.rest.service.QueryService.saveQuery(QueryService.java:129)
>       at 
> org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke(<generated>)
>       at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
>       at 
> org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
>       at 
> org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$67382972.saveQuery(<generated>)
>       at 
> org.apache.kylin.rest.controller.QueryController.saveQuery(QueryController.java:110)
>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>       at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>       at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>       at java.lang.reflect.Method.invoke(Method.java:606)
>       at 
> org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
>       at 
> org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
>       at 
> org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
>       at 
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
>       at 
> org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
> By anaylzing the source codes of Kylin, I see the file 
> apache-kylin-1.5.2\server\src\main\java\org\apache\kylin\rest\service\QueryService.java,
>  so I create the table "kylin_metadata_user" in the HBase.
> hbase(main):001:0> create 'kylin_metadata_user','q'
> 0 row(s) in 1.5880 seconds
> => Hbase::Table - kylin_metadata_user
> But I think that it maybe a bug. Thank you.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to