Re: [VOTE] Release apache-kylin-2.1.0 (RC1)
Yeah I already did that. 2017-08-08 14:25 GMT+08:00 Luke Han : > Hi Shaofeng, when canceling one vote, please also update in the original > thread > > > Best Regards! > - > > Luke Han > > On Mon, Aug 7, 2017 at 9:52 AM, 杨浩 wrote: > > > +1 > > > > 2017-08-06 20:39 GMT+08:00 Li Yang : > > > > > +1 > > > > > > mvn test passed > > > > > > java version "1.7.0_95" > > > OpenJDK Runtime Environment (rhel-2.6.4.0.el6_7-x86_64 u95-b00) > > > OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode) > > > > > > > > > > > > On Sat, Aug 5, 2017 at 6:37 PM, ShaoFeng Shi > > > wrote: > > > > > > > Hi all, > > > > > > > > I have created a build for Apache Kylin 2.1.0, release candidate 1. > > > > > > > > Changes highlights: > > > > KYLIN-2506 - Refactor global dictionary > > > > KYLIN-2575 - Experimental feature: Computed Column > > > > KYLIN-2579 KYLIN-2580 - Improvement on subqueries > > > > KYLIN-2633 - Upgrade Spark to 2.1 > > > > KYLIN-2646 - Project level query authorization > > > > > > > > And more than 60 bug fixes. > > > > > > > > Thanks to everyone who has contributed to this release. > > > > Here’s release notes: > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa? > > > > projectId=12316121&version=12340443 > > > > > > > > The commit to be voted upon: > > > > > > > > https://github.com/apache/kylin/commit/ > 47b5a0ded63e721736dbe6c4ecf1f0 > > > > 2b0b97ba43 > > > > > > > > Its hash is 47b5a0ded63e721736dbe6c4ecf1f02b0b97ba43. > > > > > > > > The artifacts to be voted on are located here: > > > > https://dist.apache.org/repos/dist/dev/kylin/apache-kylin-2.1.0-rc1/ > > > > > > > > (The binary packages for HBase 1.x and CDH 5.7are also provided for > > > > testing) > > > > > > > > The hashes of the artifacts are as follows: > > > > apache-kylin-2.1.0-src.tar.gz.md5 bb0458bff380f0670ccea57773d809d9 > > > > apache-kylin-2.1.0-src.tar.gz.sha1 5e5eebacdd72ded25508c4947565a1 > > > > ef48784ddb > > > > > > > > A staged Maven repository is available for review at: > > > > https://repository.apache.org/content/repositories/ > > orgapachekylin-1042/ > > > > > > > > Release artifacts are signed with the following key: > > > > https://people.apache.org/keys/committer/shaofengshi.asc > > > > > > > > Please vote on releasing this package as Apache Kylin 2.1.0. > > > > > > > > The vote is open for the next 72 hours and passes if a majority of > > > > at least three +1 PPMC votes are cast. > > > > > > > > [ ] +1 Release this package as Apache Kylin 2.1.0 > > > > [ ] 0 I don't feel strongly about it, but I'm okay with the release > > > > [ ] -1 Do not release this package because... > > > > > > > > Here is my vote: > > > > > > > > +1 (binding) > > > > > > > > -- > > > > Best regards, > > > > > > > > Shaofeng Shi 史少锋 > > > > > > > > > > -- Best regards, Shaofeng Shi 史少锋
The REST query request failed
Hello!
I use tableau as kylin's visualization tool. And I want to know can I
create calculated field in tableau to select?
" if [还款逾期天数] >90 then [本息余额_1] ELSE 0 END " this is my calculated field
but there is a error when I used the calculated field.
the error is :
[Kylin][ODBC 1.0(w) Driver]The REST query request failed, the error message
is: Error while executing SQL "SELECT SUM((CASE WHEN
("BASEINFO_REPAY_HIS_FACT_PARTATION"."CUR_LATEDAYS" > 90) THEN
"BASEINFO_REPAY_HIS_FACT_PARTATION"."REMAIN_CAPITAL_INTEREST" ELSE 0 END)) AS
"sum_Calculation_FCFAIAHBFAGFBCEH_ok" FROM
"TN_QUERY"."BASEINFO_REPAY_HIS_FACT_PARTATION"
"BASEINFO_REPAY_HIS_FACT_PARTATION" LEFT JOIN "TN_QUERY"."DIM_BP_PRODUCT"
"DIM_BP_PRODUCT" ON ("BASEINFO_REPAY_HIS_FACT_PARTATION"."PRODUCT_CODE" =
"DIM_BP_PRODUCT"."PRODUCT_CODE") LEFT JOIN "TN_QUERY"."DIM_BP_SALES_CHANNEL"
"DIM_BP_SALES_CHANNEL" ON ("BASEINFO_REPAY_HIS_FACT_PARTATION"."BUSINESS_TYPE"
= "DIM_BP_SALES_CHANNEL"."CHANNEL_CODE") LEFT JOIN
"TN_QUERY"."DIM_FUND_CHANNEL" "DIM_FUND_CHANNEL" ON
("BASEINFO_REPAY_HIS_FACT_PARTATION"."ROOT_CODE" =
"DIM_FUND_CHANNEL"."FUND_CHANNEL_CODE") LEFT JOIN "TN_QUERY"."DIM_ORG"
"DIM_ORG" ON ("BASEINFO_REPAY_HIS_FACT_PARTATION"."BRANCH_CODE" =
"DIM_ORG"."BRANCH_CODE") ": Can't find any realization. Please confirm with
providers. SQL digest: fact table
TN_QUERY.BASEINFO_REPAY_HIS_FACT_PARTATION,group by [],filter on [],with
aggregates[FunctionDesc [expression=SUM, parameter=null, returnType=null]].
Re: sql语句中当in的列表超过一定数据量的时候性能突然变差，如何解决？
Found by looking at the source code by kylin.query.scan.fuzzykey.max attribute control, the default is 200, when conditions combined with more than 200, it will range scan, if less than this value will be according to the rowkey to obtain, in kylin. If you want to modify the properties of adding kylin.query.scan.fuzzykey.max attribute -- View this message in context: http://apache-kylin.74782.x6.nabble.com/sql-in-tp8630p8665.html Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: How to migrate cube from one Kylin to another (for example, Kylin QA to Kylin Production) within the different Hadoop Cluster
Not to do the whole data migration, it is iterative development, the demand upgrade, the part of the metadata that we want to migrate, what could be the solution? shaofengshi wrote > Kylin metadata migration is very simple, just follow the "backup/restore > metadata" guide. > > But if you want to have the Cube data also be moved to new cluster, you > need migrate: > > 1) Hive tables (metadata and files); > 2) Files in Kylin HDFS working dir; > 3) HBase tables (KYLIN_.*) > > There is no a tool to automate these steps. You need do that by your own. > > 2017-08-03 15:06 GMT+08:00 wangzy24 < > wangzy24@ > >: > >> What would you do if you wanted to do metadata migration between >> completely >> independent environments within the different Hadoop Cluster? >> >> -- >> View this message in context: http://apache-kylin.74782.x6. >> nabble.com/How-to-migrate-cube-from-one-Kylin-to- >> another-for-example-Kylin-QA-to-Kylin-Production-within- >> the-dir-tp8628.html >> Sent from the Apache Kylin mailing list archive at Nabble.com. >> > > > > -- > Best regards, > > Shaofeng Shi 史少锋 -- View this message in context: http://apache-kylin.74782.x6.nabble.com/How-to-migrate-cube-from-one-Kylin-to-another-for-example-Kylin-QA-to-Kylin-Production-within-the-dir-tp8628p8666.html Sent from the Apache Kylin mailing list archive at Nabble.com.
[jira] [Created] (KYLIN-2779) Segment size estimation is wrong for bitmap measure
Alexander Sterligov created KYLIN-2779: -- Summary: Segment size estimation is wrong for bitmap measure Key: KYLIN-2779 URL: https://issues.apache.org/jira/browse/KYLIN-2779 Project: Kylin Issue Type: Bug Affects Versions: v2.0.0 Reporter: Alexander Sterligov If bitmap is used for distinct count, then size estimation is wrong for about 4 times. The problem is fixed by kylin.cube.size-estimate-countdistinct-ratio=0.2. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[Draft][REPORT] Apache Kylin - August 2017
Hi all, Here's Kylin board report for this cycle, please help to review and leave your feedback here. Thanks. ## Description: Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets. ## Issues: - there are no issues requiring board attention at this time ## Activity: - Hadoop & Kylin Meetup hosted in Beijing on 2017-07-14 - Alberto Ramón presented Kylin topic at Docker Meetup on 2017-06-15 - Yang Li presented Keynote about Apache Kylin at Tableau User Conf Shenzhen on 2017-07-12 - Luke Han presented Keynote about Apache Kylin at Strata Hadoop World Beijing on 2017-07-15 - Dong Li presented Apache Kylin 2.x at Strata Hadoop World Beijing on 2017-07-15 - Luke Han presented Keynote about Apache Kylin at Strata Hadoop World Beijing on 2017-07-15 - Kaige Liu presented Kylin on AWS at AWS Summit China in Beijing on 2017-07-26 - Dayue Gao presented Kylin topic at Druid Meetup Beijing on 2017-08-05 ## PMC changes: - Currently 18 PMC members. - No new PMC members added in the last 3 months ## Committer base changes: - Currently 28 committers - No new committers in the last 3 months. - Last committer addition: Fri Apr 28 2017 (Alberto Ramón) ## Releases: - Last release was 2.0.0, released on Sun Apr 30 2017 ## Mailing list activity: - dev@kylin.apache.org: - 370 subscribers (up 33 in the last 3 months) - 778 emails sent in the past 3 months, 794 in the previous cycle - iss...@kylin.apache.org: - 71 subscribers (up 9 in the last 3 months) - 1258 emails sent in the past 3 months, 921 in the previous cycle - u...@kylin.apache.org: - 286 subscribers (up 10 in the last 3 months) - 327 emails sent in the past 3 months, 232 in the previous cycle ## JIRA activity: - 180 JIRA tickets created in the last 3 months - 124 JIRA tickets closed/resolved in the last 3 months
Use AWS Redshift as Data Source for Apache Kylin
Hi We are looking at option of Using Redshift as data source to create OLAP in Apache Kylin. Is this possible? if Yes Can we install the apache Kylin on the Redshift cluster server or Is it mandatory that we need to have hadoop structure as base on which Apache Kyling can be installed. If we have to use Redshift as data source what is the approach we need to take. Please guide -- View this message in context: http://apache-kylin.74782.x6.nabble.com/Use-AWS-Redshift-as-Data-Source-for-Apache-Kylin-tp8669.html Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: Use AWS Redshift as Data Source for Apache Kylin
Hi there, Today, Kylin only supports ready data from Hive or Kafka. It doable if you could find a way to expose your data through Hive interface. Or, you could customize a little bit to read data from Redshift. And, one idea we have (actually there are many other users are doing today) is to setup EMR + Kylin, then upload data from Redshift or other data source to EMR/S3, then Kylin could easily to consume through Hive. BTW, why you are trying to read data from Redshift and build cube in Kylin? What's challenge you are facing now? Thanks. Luke Best Regards! - Luke Han On Tue, Aug 8, 2017 at 11:31 PM, bsmithab wrote: > Hi > > We are looking at option of Using Redshift as data source to create OLAP in > Apache Kylin. Is this possible? if Yes Can we install the apache Kylin on > the Redshift cluster server or Is it mandatory that we need to have hadoop > structure as base on which Apache Kyling can be installed. If we have to > use > Redshift as data source what is the approach we need to take. Please guide > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/Use-AWS-Redshift-as-Data-Source-for-Apache-Kylin-tp8669.html > Sent from the Apache Kylin mailing list archive at Nabble.com. >
Re: Kylin 主从同步经常失败
I am sure it's not the network problem. Because some changes can successfully send to other nodes. So, I think it's the program sometimes can not reponse status with 200, perhaps 500. But there are no more specific error message to verify that. So I suggest if you can add retry logic and narrow the try catch scope to log more detailed message -- View this message in context: http://apache-kylin.74782.x6.nabble.com/Kylin-tp8640p8663.html Sent from the Apache Kylin mailing list archive at Nabble.com.
[jira] [Created] (KYLIN-2780) QueryController.getMetadata and CacheController.wipeCache may be deadlock
Fangyuan Deng created KYLIN-2780: Summary: QueryController.getMetadata and CacheController.wipeCache may be deadlock Key: KYLIN-2780 URL: https://issues.apache.org/jira/browse/KYLIN-2780 Project: Kylin Issue Type: Bug Components: Query Engine Affects Versions: v2.0.0, v1.5.4.1 Reporter: Fangyuan Deng Assignee: liyang When we send update metadata request to the query sever manually, while some getMetadata requests come, dead lock may hadoop. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: Timeout when refreshing a cube
Hi Shaofeng Shi,
Looking for some information on Hbase UI I realized that the node informed
in the log was stopped during the process. So this explaind the timeout,
correct?
{"url":"http://:7070/kylin/api/jobs/349d62e3-7748-46ee-8009-3085726b5933","exception":"java.lang.RuntimeException:
org.apache.kylin.job.exception.PersistentException:
java.net.SocketTimeoutException: callTimeout=6, callDuration=63012: row
'/execute/349d62e3-7748-46ee-8009-3085726b5933' on table 'kylin_metadata' at
region=kylin_metadata,/dict/VIEW_CREDITO/COD_PROMOTOR/a61c83e7-01e7-4971-9539-dd5c7ed07dff.dict,1500376865680.301b695304237ff0d1a7774bbff83b67.,
/hostname=,/16020,1501960877256, seqNum=163046"}
We are running a Cluster on AWS EMR (hbase in S3) and using the tasknodes
with autoscalling feature turned on, I´m understandig that this node was
decomissioned due this configuration.
Can you see any other problem we migth face?
Thanks for answering my post.
--
View this message in context:
http://apache-kylin.74782.x6.nabble.com/Timeout-when-refreshing-a-cube-tp8658p8671.html
Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: Use AWS Redshift as Data Source for Apache Kylin
Can we use ODBC/JDBC connection to directly pull data from Redshift Data warehouse? -- View this message in context: http://apache-kylin.74782.x6.nabble.com/Use-AWS-Redshift-as-Data-Source-for-Apache-Kylin-tp8669p8673.html Sent from the Apache Kylin mailing list archive at Nabble.com.
Re: Use AWS Redshift as Data Source for Apache Kylin
JDBC is good idea which actually already have some investment here. Please stay tuned for the progress. Thanks. Best Regards! - Luke Han On Wed, Aug 9, 2017 at 12:50 PM, bsmithab wrote: > Can we use ODBC/JDBC connection to directly pull data from Redshift Data > warehouse? > > -- > View this message in context: http://apache-kylin.74782.x6. > nabble.com/Use-AWS-Redshift-as-Data-Source-for-Apache- > Kylin-tp8669p8673.html > Sent from the Apache Kylin mailing list archive at Nabble.com. >
??????[Draft][REPORT] Apache Kylin - August 2017
+1. -- -- ??: "Luke Han";; : 2017??8??8??(??) 7:29 ??: "private"; "dev"; : [Draft][REPORT] Apache Kylin - August 2017 Hi all, Here's Kylin board report for this cycle, please help to review and leave your feedback here. Thanks. ## Description: Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets. ## Issues: - there are no issues requiring board attention at this time ## Activity: - Hadoop & Kylin Meetup hosted in Beijing on 2017-07-14 - Alberto Ram??n presented Kylin topic at Docker Meetup on 2017-06-15 - Yang Li presented Keynote about Apache Kylin at Tableau User Conf Shenzhen on 2017-07-12 - Luke Han presented Keynote about Apache Kylin at Strata Hadoop World Beijing on 2017-07-15 - Dong Li presented Apache Kylin 2.x at Strata Hadoop World Beijing on 2017-07-15 - Luke Han presented Keynote about Apache Kylin at Strata Hadoop World Beijing on 2017-07-15 - Kaige Liu presented Kylin on AWS at AWS Summit China in Beijing on 2017-07-26 - Dayue Gao presented Kylin topic at Druid Meetup Beijing on 2017-08-05 ## PMC changes: - Currently 18 PMC members. - No new PMC members added in the last 3 months ## Committer base changes: - Currently 28 committers - No new committers in the last 3 months. - Last committer addition: Fri Apr 28 2017 (Alberto Ram??n) ## Releases: - Last release was 2.0.0, released on Sun Apr 30 2017 ## Mailing list activity: - dev@kylin.apache.org: - 370 subscribers (up 33 in the last 3 months) - 778 emails sent in the past 3 months, 794 in the previous cycle - iss...@kylin.apache.org: - 71 subscribers (up 9 in the last 3 months) - 1258 emails sent in the past 3 months, 921 in the previous cycle - u...@kylin.apache.org: - 286 subscribers (up 10 in the last 3 months) - 327 emails sent in the past 3 months, 232 in the previous cycle ## JIRA activity: - 180 JIRA tickets created in the last 3 months - 124 JIRA tickets closed/resolved in the last 3 months
