date:20190121

?????? ??????cube????????????

2019-01-21 Thread ????????

Hi
here are results:


kylin: 
select count(*) from ICSTOCKBILL_1W   
result:10366


hive:
select count(*) from ICSTOCKBILL_1W 
result:10411




--  --
??: "JiaTao Tao";
: 2019??1??22??(??) 3:04
??: "dev";

: Re: ??cube



Hi
Can you try "select count(*)" and compare the result with hive?

FYI: http://kylin.apache.org/docs/gettingstarted/faq.html (Why I got an
error when running a ??select * ?? query?)


 <3513797...@qq.com> ??2019??1??22?? 5:21??

> group by ??
> ?? goodsid 1137??
> select * from ICSTOCKBILL_1W where goodsid = '1137'
> ??
>
>
> cube
> 
>
>
> hive
> ??cube
>
>
> --  --
> *??:* "Chao Long";
> *:* 2019??1??22??(??) 12:11
> *??:* "dev";
> *:* cube
>
> Hi,
>   The cube only has aggregated data, so your queries should include "group
> by" clause.
>   You can check the faq:
> http://kylin.apache.org/docs/gettingstarted/faq.html(# Why I got an error
> when running a ??select * ?? query)
>
>
> --
> Best Regards,
> Chao Long
>
>
> --  --
> ??: ""<3513797...@qq.com>;
> : 2019??1??22??(??) 11:06
> ??: "dev";
>
> : ??cube
>
>
>
> Hi,
>
>
> ??hivehivecube
>  
> ?? select * from ICSTOCKBILL_1W where goodsid = '643'??
> ICSTOCKBILL_1W ??
> 
> hive
> cube
>
>
> ??goodsid 
>
>
> env:
> ubuntu 16.04 hadoop 3??
> apache-kylin-2.5.2-bin-hadoop3  or apache-kylin-2.6.0-bin-hadoop3 ??kylin
> ??
> hive:2.3.4
> hbase:1.3.3
> zookeeper:3.4.13
>
>
>
> ??
>
>
> Best regards
>


-- 


Regards!

Aron Tao

Re: 生成的cube部分数据缺失

2019-01-21 Thread JiaTao Tao

Hi
Can you try "select count(*)" and compare the result with hive?

FYI: http://kylin.apache.org/docs/gettingstarted/faq.html (Why I got an
error when running a “select * “ query?)


奥威软件 <3513797...@qq.com> 于2019年1月22日周二 上午5:21写道：

> 没有group by 也一样能查到数据的
> 例如把 goodsid 改为1137，
> select * from ICSTOCKBILL_1W where goodsid = '1137'
> 结果：
>
>
> 现在的问题是cube的数据有部分缺失了，查不到数据，
> 造成汇总数据错误
>
>
> hive表里的数据是完整的，
> 生成的cube缺丢失了部分数据
>
>
> -- 原始邮件 --
> *发件人:* "Chao Long";
> *发送时间:* 2019年1月22日(星期二) 中午12:11
> *收件人:* "dev";
> *主题:* 回复：生成的cube部分数据缺失
>
> Hi,
>   The cube only has aggregated data, so your queries should include "group
> by" clause.
>   You can check the faq:
> http://kylin.apache.org/docs/gettingstarted/faq.html(# Why I got an error
> when running a “select * “ query)
>
>
> --
> Best Regards,
> Chao Long
>
>
> -- 原始邮件 --
> 发件人: "奥威软件"<3513797...@qq.com>;
> 发送时间: 2019年1月22日(星期二) 中午11:06
> 收件人: "dev";
>
> 主题: 生成的cube部分数据缺失
>
>
>
> Hi,
>
>
> 已确认，在hive表里，数据完整，但同样的查询语句查询事实表，hive有数据且完整，但查询cube 就没有数据，
> 查询语句如 select * from ICSTOCKBILL_1W where goodsid = '643'，
> ICSTOCKBILL_1W 为事实表，
> 通过查询语句
> hive表有数据
> 查询cube没有数据
>
>
> 经常检查有个位数的goodsid 缺失了，且找不到规律和疑点。
>
>
> env:
> ubuntu 16.04 hadoop集群 3台
> apache-kylin-2.5.2-bin-hadoop3  or apache-kylin-2.6.0-bin-hadoop3 （kylin
> 单机）
> hive:2.3.4
> hbase:1.3.3
> zookeeper:3.4.13
>
>
>
> 请帮忙看下怎么解决，谢谢！
>
>
> Best regards
>


-- 


Regards!

Aron Tao

????????????cube????????????

2019-01-21 Thread ????????

group by ??
?? goodsid 1137??
select * from ICSTOCKBILL_1W where goodsid = '1137' 
??





cube





hive
??cube




--  --
??: "Chao Long";
: 2019??1??22??(??) 12:11
??: "dev";

: cube



Hi,
  The cube only has aggregated data, so your queries should include "group by" 
clause.
  You can check the faq: http://kylin.apache.org/docs/gettingstarted/faq.html(# 
Why I got an error when running a ??select * ?? query)


--
Best Regards,
Chao Long


--  --
??: ""<3513797...@qq.com>;
: 2019??1??22??(??) 11:06
??: "dev";

: ??cube



Hi,


??hivehivecube
 
?? select * from ICSTOCKBILL_1W where goodsid = '643'??
ICSTOCKBILL_1W ??
 
hive
cube


??goodsid 


env:
ubuntu 16.04 hadoop 3?? 
apache-kylin-2.5.2-bin-hadoop3  or apache-kylin-2.6.0-bin-hadoop3 ??kylin ??
hive:2.3.4
hbase:1.3.3
zookeeper:3.4.13 



??


Best regards

[jira] [Created] (KYLIN-3780) Add built instance in Job info

2019-01-21 Thread Liu Shaohui (JIRA)

Liu Shaohui created KYLIN-3780:
--

 Summary: Add built instance in Job info
 Key: KYLIN-3780
 URL: https://issues.apache.org/jira/browse/KYLIN-3780
 Project: Kylin
  Issue Type: New Feature
Reporter: Liu Shaohui
Assignee: Liu Shaohui


In DistributedScheduler, it's hard to known which machine the kylin job is 
running on.

But this info is helpful to debug the failed jobs.

So we add the  job built instance info in job info and kylin web ui.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

????????????cube????????????

2019-01-21 Thread Chao Long

Hi,
  The cube only has aggregated data, so your queries should include "group by" 
clause.
  You can check the faq: http://kylin.apache.org/docs/gettingstarted/faq.html(# 
Why I got an error when running a ??select * ?? query)


--
Best Regards,
Chao Long


--  --
??: ""<3513797...@qq.com>;
: 2019??1??22??(??) 11:06
??: "dev";

: ??cube



Hi,


??hivehivecube
 
?? select * from ICSTOCKBILL_1W where goodsid = '643'??
ICSTOCKBILL_1W ??
 
hive
cube


??goodsid 


env:
ubuntu 16.04 hadoop 3?? 
apache-kylin-2.5.2-bin-hadoop3  or apache-kylin-2.6.0-bin-hadoop3 ??kylin ??
hive:2.3.4
hbase:1.3.3
zookeeper:3.4.13 



??


Best regards

??????cube????????????

2019-01-21 Thread ????????

Hi,


??hivehivecube
 
?? select * from ICSTOCKBILL_1W where goodsid = '643'??
ICSTOCKBILL_1W ??
 
hive
cube


??goodsid 


env:
ubuntu 16.04 hadoop 3?? 
apache-kylin-2.5.2-bin-hadoop3  or apache-kylin-2.6.0-bin-hadoop3 ??kylin ??
hive:2.3.4
hbase:1.3.3
zookeeper:3.4.13 



??


Best regards

Re: Using mysql as adata source instead of Kylin

2019-01-21 Thread Rongchuan Jin

Hi hubert:
Thanks to @youcheng.zhang (JIRA) who has fixed this bug. You can 
cherry-pick this 
commit(https://github.com/apache/kylin/commit/b4dace4db8304a53082265e614384132316801bb)
 now or wait for 
the next release. 

Best Regards
Rongchuan.Jin

 

在 2019/1/16 下午6:15，“Rongchuan Jin” 写入:

Hi hubert:
I don’t think root cause is conflict between pushdown property.

Best Regards
Rongchuan.Jin

 

在 2019/1/16 下午5:50，“hubert stefani” 写入:

 Ok. I will create the JIRA in a few minutes. Any chance there is a 
conflict in the configuration with the  kylin.query.pushdown.jdbc. properties  ?

Le mercredi 16 janvier 2019 à 10:46:59 UTC+1, Rongchuan Jin 
 a écrit :  
 
 Hi hubert:
  Your configuration seems okay. I reproduced it in my environment, it 
seems like a bug within ClassLoader. Welcome to create a JIRA ticket.

Best Regards
Rongchuan.Jin

 

在 2019/1/16 下午5:36，“hubert stefani” 写入:

Please find hereunder the kylin properties lines related to mysql 
jdbc connection:  (it works properly through spark shell script)

# NOVAGEN -- CUSTOM CONFIGURATION FOR MYSQL AS SOURCE

kylin.source.jdbc.connection-url=jdbc:mysql://ip-172-31-36-94.eu-west-1.compute.internal:3306/decat
kylin.source.jdbc.driver=com.mysql.jdbc.Driver
kylin.source.jdbc.dialect=mysql
kylin.source.jdbc.user=root
kylin.source.jdbc.pass=

kylin.query.pushdown.runner-class-name=org.apache.kylin.query.pushdown.PushdownRunnerSDKImpl
kylin.source.default=16
kylin.source.jdbc.sqoop-home=/usr/bin

kylin.source.jdbc.adaptor=org.apache.kylin.sdk.datasource.adaptor.MysqlAdaptor



Le mercredi 16 janvier 2019 à 05:54:51 UTC+1, Rongchuan Jin 
 a écrit :  

Hi Hubert,
Could you please provide your kylin.properties with prefix 
“kylin.source.jdbc ” or just show your project configurations. I think it’s 
easier to find the root cause.

Best Regards
Rongchuan.Jin


在 2019/1/14 下午9:24，“hubert 
stefani” 写入:

Hi,
  we are currently trying to test the KYLIN-3552 feature to 
rely directly on Mysql for data source instead of Hive. 

(we tried as much as possible to add correct info in 
kylin.properties, add mysql jdbc drivers, create mysql.xml in 
./conf/datasources ... as mentioned in 
https://github.com/apache/kylin/pull/327/files )



the error we entountered is ( excerpt from kylin.log ) :

2019-01-14 09:48:16,223 WARN  [http-bio-7070-exec-2] 
extensible.JdbcSource:50 : DataSource cannot be connected. This may not be 
required in a MapReduce job.
java.lang.LinkageError: loader constraint violation: loader 
(instance of sun/misc/Launcher$AppClassLoader) previously initiated loading for 
a different type with name "org/apache/kylin/common/KylinConfig"
at java.lang.ClassLoader.defineClass1(Native Method)
at 
java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at 
java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at 
java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at 
java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at 
java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native 
Method)
at 
java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at 
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at 
org.apache.kylin.sdk.datasource.framework.SourceConnectorFactory.getJdbcConnector(SourceConnectorFactory.java:27)


with further trace : 

2019-01-14 09:48:16,229 ERROR [http-bio-7070-exec-2] 
controller.TableController:197 : 
java.lang.NullPointerException
at 
org.apache.kylin.source.jdbc.extensible.JdbcExplorer.listDatabases(JdbcExplorer.java:58)
at 
org.apache.kylin.rest.service.TableService.getSourceDbNames(TableService.java:280)
at 
org.apache.kylin.rest.cont

Re: ERROR context.ContextLoader:350 : Context initialization failed

2019-01-21 Thread ShaoFeng Shi

It seems you're running Kylin from IDE, please make sure the hbase-site.xml
under "/home/hadoop/Desktop/kylin-2.3.x
(2)/server/../examples/test_case_data/sandbox" is valid for your
environment. If you look at that file, it uses "sandbox.hortonworks.com" as
the zk host name, making sure this host is valid in your machine.

Best regards,

Shaofeng Shi 史少锋
Apache Kylin PMC
Work email: shaofeng@kyligence.io
Kyligence Inc: https://kyligence.io/

Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscr...@kylin.apache.org
Join Kylin dev mail group: dev-subscr...@kylin.apache.org




Na Zhai  于2019年1月17日周四 下午5:33写道：

> Hi, Jock.
>
>Do you follow the instructions on this page?
> http://kylin.apache.org/development/dev_env.html. And I find “ClusterId
> read in ZooKeeper is null” in the log that you provide, so I think the
> environment for you developing has the problem.
>
>
>
> 发送自 Windows 10 版邮件应用
>
>
>
> 
> 发件人: Jock 
> 发送时间: Wednesday, January 16, 2019 5:17:39 PM
> 收件人: dev@kylin.apache.org
> 主题: ERROR context.ContextLoader:350 : Context initialization failed
>
> Hello,
>
>
> I am getting following error when I try to run the source code of  kylin
> 2.3 in IntelliJ IDEA. I have no idea  about it. Could you give some
> suggests about it?
>
>
> Really appreciated.
>
>
> 2019-01-16 16:34:01,778 INFO  [main] common.KylinConfig:378 : Setting
> sandbox env, KYLIN_CONF=/home/hadoop/Desktop/kylin-2.3.x
> (2)/server/../examples/test_case_data/sandbox
> 2019-01-16 16:34:01,801 INFO  [main] util.ClassUtil:40 : Adding path
> /home/hadoop/Desktop/kylin-2.3.x
> (2)/server/../examples/test_case_data/sandbox to class path
> 2019-01-16 16:34:01,807 INFO  [main] common.KylinConfig:319 : Loading
> kylin-defaults.properties from
> /home/hadoop/Desktop/kylin-2.3.x%20(2)/core-common/target/classes/kylin-defaults.properties
> 2019-01-16 16:34:01,891 INFO  [main] common.KylinConfig:274 : Use
> KYLIN_CONF=/home/hadoop/Desktop/kylin-2.3.x
> (2)/server/../examples/test_case_data/sandbox
> 2019-01-16 16:34:01,896 INFO  [main] common.KylinConfig:99 : Initialized a
> new KylinConfig from getInstanceFromEnv : 1308927845
> 2019-01-16 16:34:01,897 INFO  [main] common.KylinConfigBase:1074 :
> override kylin.engine.mr.job-jar to /home/hadoop/Desktop/kylin-2.3.x
> (2)/server/../assembly/target/kylin-assembly-2.3.2-SNAPSHOT-job.jar
> 2019-01-16 16:34:01,901 INFO  [main] common.KylinConfigBase:919 : override
> kylin.storage.hbase.coprocessor-local-jar to
> /home/hadoop/Desktop/kylin-2.3.x
> (2)/server/../storage-hbase/target/kylin-storage-hbase-2.3.2-SNAPSHOT-coprocessor.jar
> 一月 16, 2019 4:34:03 下午 org.apache.catalina.core.AprLifecycleListener
> lifecycleEvent
> 信息: The APR based Apache Tomcat Native library which allows optimal
> performance in production environments was not found on the
> java.library.path:
> /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 一月 16, 2019 4:34:05 下午 org.apache.coyote.AbstractProtocol init
> 信息: Initializing ProtocolHandler ["http-bio-7070"]
> 一月 16, 2019 4:34:05 下午 org.apache.catalina.core.StandardService
> startInternal
> 信息: Starting service Tomcat
> 一月 16, 2019 4:34:05 下午 org.apache.catalina.core.StandardEngine
> startInternal
> 信息: Starting Servlet Engine: Apache Tomcat/7.0.85
> 一月 16, 2019 4:34:06 下午 org.apache.catalina.startup.ContextConfig
> getDefaultWebXmlFragment
> 信息: No global web.xml found
> 一月 16, 2019 4:34:33 下午 org.apache.catalina.startup.TldConfig execute
> 信息: At least one JAR was scanned for TLDs yet contained no TLDs. Enable
> debug logging for this logger for a complete list of JARs that were scanned
> but no TLDs were found in them. Skipping unneeded JARs during scanning can
> improve startup time and JSP compilation time.
> 一月 16, 2019 4:34:33 下午 org.apache.catalina.core.ApplicationContext log
> 信息: No Spring WebApplicationInitializer types detected on classpath
> 一月 16, 2019 4:34:33 下午 org.apache.catalina.core.ApplicationContext log
> 信息: Initializing Spring root WebApplicationContext
> 2019-01-16 16:34:36,912 DEBUG [localhost-startStop-1]
> security.PasswordPlaceholderConfigurer:174 : Loading properties file from
> InputStream resource [resource loaded through InputStream]
> 2019-01-16 16:34:38,213 INFO  [localhost-startStop-1]
> metrics.MetricsManager:135 : Kylin metrics monitor is not enabled!!!
> 2019-01-16 16:34:39,843 INFO  [localhost-startStop-1]
> init.InitialTaskManager:38 : Kylin service is starting.
> 2019-01-16 16:34:40,114 INFO  [localhost-startStop-1]
> persistence.ResourceStore:86 : Using metadata url kylin_metadata@hbase
> for resource store
> 2019-01-16 16:34:40,753 WARN  [localhost-startStop-1]
> util.NativeCodeLoader:62 : Unable to load native-hadoop library for your
> platform... using builtin-java classes where applicable
> 2019-01-16 16:34:42,391 WARN  [localhost-startStop-1]
> shortcircuit.DomainSocketFact

[jira] [Created] (KYLIN-3779) JDBC sources with Case sensitive behaviour

2019-01-21 Thread Hubert STEFANI (JIRA)

Hubert STEFANI created KYLIN-3779:
-

 Summary: JDBC sources with Case sensitive behaviour
 Key: KYLIN-3779
 URL: https://issues.apache.org/jira/browse/KYLIN-3779
 Project: Kylin
  Issue Type: Bug
  Components: RDBMS Source
Affects Versions: v2.6.0
Reporter: Hubert STEFANI


When using a Mysql instance with names (database, tables, columns) in lower 
case, Kylin converts names to upper case in metadata tables.

Hence, when trying to build  the cube, the upper case names are used (through 
SQOOP) to load data and the sqoop step fails.

could the names be kept in original case ?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

?????? ??????cube????????????

Re: 生成的cube部分数据缺失

????????????cube????????????

[jira] [Created] (KYLIN-3780) Add built instance in Job info

????????????cube????????????

??????cube????????????

Re: Using mysql as adata source instead of Kylin

Re: ERROR context.ContextLoader:350 : Context initialization failed

[jira] [Created] (KYLIN-3779) JDBC sources with Case sensitive behaviour

9 matches

Site Navigation

Mail list logo

Footer information