We have contacted kyligence but haven't got any response yet. If you or someone else part of kyligence let me know. My engineering director and engineers would be interested in talking about it.
Thanks, Nirav On Mon, Jul 17, 2017 at 1:03 PM, Li Yang <[email protected]> wrote: > I knew some commercial version of Kylin could store metadata and cube > outside of HBase. > > On Tue, Jul 11, 2017 at 5:19 AM, Nirav Patel <[email protected]> > wrote: > > > "Please also note that Kylin's metadata is persisted to HBase by default, > > you need have another implementation for ResourceStore." > > > > I found almost 600 matches for string "hbase" in modules other then > > hbase-storage. Hbase seems tightly integrated into modules where it > should > > not be. Even if it is used for metadata it should been pluggable and not > > directly called from other modules. I can't even run DebugTomcat.java > > because it relies on classes like AclHBaseStorage (part of server-base) > and > > calls methods of Hbase storage directly. I think we should make Hbase > > dependency injection little bit more pluggable then what it is now. Or > use > > embedded database like derby or rdbms like mysql which are not so costly > to > > deploy like hbase. > > > > Thanks, > > Nirav > > > > On Mon, Jul 10, 2017 at 11:03 AM, Nirav Patel <[email protected]> > > wrote: > > > > > No equivalent. Instead they came up with Drill - MaprDB connector for > > > performance and flexibility. So like earlier version of Kylin I should > > be > > > able to make drill calls but still do filtering and aggregation on > > cluster. > > > Only thing is we need to run separate drill-bit service under Yarn for > > > this to work. > > > > > > Thanks, > > > Nirav > > > > > > On Sun, Jul 9, 2017 at 5:06 PM, ShaoFeng Shi <[email protected]> > > > wrote: > > > > > >> Is there some coprocessor-equivalent mechanism in maprdb? If yes, you > > can > > >> try to implement the logic into it. Otherwise, the performance might > be > > >> unacceptable when there is bunch of data in each region. > > >> > > >> I rememer in very early version, Kylin doesn't have coprocessor, all > > data > > >> be fetched to Kylin server and then do filtering and aggregation by > > >> Calcite. Later coprocessor was added and improved continiously. Now > > there > > >> is no option to disable it. You have to modify the code if don't want > to > > >> trigger it. > > >> > > >> > > >> > > >> 2017-07-09 1:34 GMT+08:00 Nirav Patel <[email protected]>: > > >> > > >> > > > >> > Thanks Shao for some pointers. I would like some more advice on how > I > > >> can > > >> > replace maprdb as datasource. > > >> > > > >> > Currently, you can create cubes on maprdb using hbase-storage > module > > as > > >> > maprdb does support hbase client/admin APIs. Only problem is while > > >> querying > > >> > it throws an error due to Lack of support of Co-processor. > > >> > > > >> > My initial goal is to get past coprocessor error and return query > > >> result. > > >> > Then find a better way to implement that strategy - may be a > separate > > >> > maprdb-storage module. Then look into how to improve performance > > without > > >> > coprocessors. Any advice on these approaches ? > > >> > > > >> > Thanks, > > >> > Nirav > > >> > > > >> > On Sat, Jul 8, 2017 at 7:54 AM, ShaoFeng Shi < > [email protected]> > > >> > wrote: > > >> > > > >> >> The core modules like core-cube, core-storage are totally > independent > > >> of > > >> >> HBase; while some others like engine-mr, engine-spark has > > dependencies > > >> on > > >> >> HBase. If you want to replace it, you need implement new cubing > > engine > > >> as > > >> >> well. Please also note that Kylin's metadata is persisted to HBase > by > > >> >> default, you need have another implementation for ResourceStore. > > >> >> > > >> >> In a short, the plug-in architecture works (we have verfied that), > > >> while > > >> >> changing the storage is a complex task which take some time to be > > >> function > > >> >> complete and performance stable. > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> > > >> >> 2017-07-08 5:19 GMT+08:00 Nirav Patel <[email protected]>: > > >> >> > > >> >>> Yes, MapR-DB doesn't support coprocessors. > > >> >>> > > >> >>> Here's the thing though - based on kylin plugin architecture it > > >> >>> shouldn't be a problem ideally. Aggregation as well as other > DML/DDL > > >> >>> operation on datasources should be done transparently. i.e. using > > >> >>> kylin-hbase adapter written with calcite or something. It's upto > the > > >> writer > > >> >>> of those adapters to implement aggregates however they want. i.e. > > >> either > > >> >>> using coprocessors, or in-memory on application server side or > using > > >> spark. > > >> >>> http://kylin.apache.org/development/plugin_arch.html > > >> >>> > > >> >>> I think for mapr-db we can leverage mapr-drill with secondary > > indexes > > >> >>> for faster filtering and drill does parallel aggregation as well. > > >> Other > > >> >>> option is to use mapr-spark with mapr-db which can does the same. > > >> >>> > > >> >>> Do you know how tightly hbase is coupled with other modules of > kylin > > >> >>> source other then hbase-storage. > > >> >>> > > >> >>> > > >> >>> > > >> >>> > > >> >>> > > >> >>> > > >> >>> On Thu, Jul 6, 2017 at 6:53 PM, ShaoFeng Shi < > > [email protected]> > > >> >>> wrote: > > >> >>> > > >> >>>> Hi Nirav, > > >> >>>> > > >> >>>> I googled that "HBase coprocessors are not present in MapR DB", > is > > >> >>>> this true? You know Kylin relies on HBase coprocessor to do > > filering > > >> and > > >> >>>> aggregation in each region local; If coprocessor is not > available, > > >> the > > >> >>>> performance will be a problem. > > >> >>>> > > >> >>>> 2017-07-07 1:29 GMT+08:00 Nirav Patel <[email protected]>: > > >> >>>> > > >> >>>>> Hi, We are a mapr users. You will need to deploy separate Hbase > > >> >>>>> Cluster alongside your mapr Cluster or on top of it. You won't > be > > >> able to > > >> >>>>> use Mapr-DB with Kylin. > > >> >>>>> I am looking into writing Kyling - MaprDB adapter. > > >> >>>>> > > >> >>>>> On Tue, Jul 4, 2017 at 8:23 AM, <[email protected]> wrote: > > >> >>>>> > > >> >>>>>> V5.2, > > >> >>>>>> > > >> >>>>>> I’m trying to figure out how to deploy Kylin on a MapR cluster, > > >> >>>>>> whether Kylin has to be installed on the same cluster of the > MapR > > >> cluster > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> *From:* Luke Han [mailto:[email protected]] > > >> >>>>>> *Sent:* Tuesday, July 04, 2017 10:04 PM > > >> >>>>>> *To:* user > > >> >>>>>> *Subject:* Re: Kylin for MapR > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> Kylin support MapR, which MapR version you are using now? > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> Best Regards! > > >> >>>>>> --------------------- > > >> >>>>>> > > >> >>>>>> Luke Han > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> On Tue, Jul 4, 2017 at 4:19 PM, <[email protected]> wrote: > > >> >>>>>> > > >> >>>>>> Hi > > >> >>>>>> > > >> >>>>>> Does kylin support MapR version of the Hadoop? > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> This e-mail (including any attachments) is private and > > >> confidential, > > >> >>>>>> may contain proprietary or privileged information and is > intended > > >> for the > > >> >>>>>> named recipient(s) only. Unintended recipients are strictly > > >> prohibited from > > >> >>>>>> taking action on the basis of information in this e-mail and > must > > >> contact > > >> >>>>>> the sender immediately, delete this e-mail (and all > attachments) > > >> and > > >> >>>>>> destroy any hard copies. Nomura will not accept responsibility > or > > >> liability > > >> >>>>>> for the accuracy or completeness of, or the presence of any > virus > > >> or > > >> >>>>>> disabling code in, this e-mail. If verification is sought > please > > >> request a > > >> >>>>>> hard copy. Any reference to the terms of executed transactions > > >> should be > > >> >>>>>> treated as preliminary only and subject to formal written > > >> confirmation by > > >> >>>>>> Nomura. Nomura reserves the right to retain, monitor and > > intercept > > >> e-mail > > >> >>>>>> communications through its networks (subject to and in > accordance > > >> with > > >> >>>>>> applicable laws). No confidentiality or privilege is waived or > > >> lost by > > >> >>>>>> Nomura by any mistransmission of this e-mail. Any reference to > > >> "Nomura" is > > >> >>>>>> a reference to any entity in the Nomura Holdings, Inc. group. > > >> Please read > > >> >>>>>> our Electronic Communications Legal Notice which forms part of > > >> this e-mail: > > >> >>>>>> http://www.Nomura.com/email_disclaimer.htm > > >> >>>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www. > > >> Nomura.com_email-5Fdisclaimer.htm&d=DwMFaQ&c=dCBwIlVXJsYZrY6 > > >> gpNt0LA&r=nFqGYx5SOTBIei-Hwk8uz_JfhmSu2UlelQfdmGX2k5I& > > >> m=s6jR2dvhFFTZIuHhCE8WU8EYNQ7M5u-9SggFAoyn5-M&s=iXZr3lmnuJ4X > > >> k2-qORnbYZQALda0jRLZ3kzGy816bco&e=> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> > > >> >>>>>> This e-mail (including any attachments) is private and > > >> confidential, > > >> >>>>>> may contain proprietary or privileged information and is > intended > > >> for the > > >> >>>>>> named recipient(s) only. Unintended recipients are strictly > > >> prohibited from > > >> >>>>>> taking action on the basis of information in this e-mail and > must > > >> contact > > >> >>>>>> the sender immediately, delete this e-mail (and all > attachments) > > >> and > > >> >>>>>> destroy any hard copies. Nomura will not accept responsibility > or > > >> liability > > >> >>>>>> for the accuracy or completeness of, or the presence of any > virus > > >> or > > >> >>>>>> disabling code in, this e-mail. If verification is sought > please > > >> request a > > >> >>>>>> hard copy. Any reference to the terms of executed transactions > > >> should be > > >> >>>>>> treated as preliminary only and subject to formal written > > >> confirmation by > > >> >>>>>> Nomura. Nomura reserves the right to retain, monitor and > > intercept > > >> e-mail > > >> >>>>>> communications through its networks (subject to and in > accordance > > >> with > > >> >>>>>> applicable laws). No confidentiality or privilege is waived or > > >> lost by > > >> >>>>>> Nomura by any mistransmission of this e-mail. Any reference to > > >> "Nomura" is > > >> >>>>>> a reference to any entity in the Nomura Holdings, Inc. group. > > >> Please read > > >> >>>>>> our Electronic Communications Legal Notice which forms part of > > >> this e-mail: > > >> >>>>>> http://www.Nomura.com/email_disclaimer.htm > > >> >>>>>> > > >> >>>>> > > >> >>>>> > > >> >>>>> > > >> >>>>> > > >> >>>>> [image: What's New with Xactly] > > >> >>>>> <http://www.xactlycorp.com/email-click/> > > >> >>>>> > > >> >>>>> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] > > >> >>>>> <https://www.linkedin.com/company/xactly-corporation> [image: > > >> >>>>> Twitter] <https://twitter.com/Xactly> [image: Facebook] > > >> >>>>> <https://www.facebook.com/XactlyCorp> [image: YouTube] > > >> >>>>> <http://www.youtube.com/xactlycorporation> > > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> -- > > >> >>>> Best regards, > > >> >>>> > > >> >>>> Shaofeng Shi 史少锋 > > >> >>>> > > >> >>>> > > >> >>> > > >> >>> > > >> >>> > > >> >>> [image: What's New with Xactly] <http://www.xactlycorp.com/ema > > >> il-click/> > > >> >>> > > >> >>> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] > > >> >>> <https://www.linkedin.com/company/xactly-corporation> [image: > > >> Twitter] > > >> >>> <https://twitter.com/Xactly> [image: Facebook] > > >> >>> <https://www.facebook.com/XactlyCorp> [image: YouTube] > > >> >>> <http://www.youtube.com/xactlycorporation> > > >> >>> > > >> >> > > >> >> > > >> >> > > >> >> -- > > >> >> Best regards, > > >> >> > > >> >> Shaofeng Shi 史少锋 > > >> >> > > >> >> > > >> > > > >> > > > >> > > > >> > [image: What's New with Xactly] <http://www.xactlycorp.com/ > > email-click/ > > >> > > > >> > > > >> > <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] > > >> > <https://www.linkedin.com/company/xactly-corporation> [image: > > Twitter] > > >> > <https://twitter.com/Xactly> [image: Facebook] > > >> > <https://www.facebook.com/XactlyCorp> [image: YouTube] > > >> > <http://www.youtube.com/xactlycorporation> > > >> > > > >> > > >> > > >> > > >> -- > > >> Best regards, > > >> > > >> Shaofeng Shi 史少锋 > > >> > > > > > > > > > > -- > > > > > > [image: What's New with Xactly] <http://www.xactlycorp.com/email-click/> > > > > <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] > > <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] > > <https://twitter.com/Xactly> [image: Facebook] > > <https://www.facebook.com/XactlyCorp> [image: YouTube] > > <http://www.youtube.com/xactlycorporation> > > > -- [image: What's New with Xactly] <http://www.xactlycorp.com/email-click/> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] <https://twitter.com/Xactly> [image: Facebook] <https://www.facebook.com/XactlyCorp> [image: YouTube] <http://www.youtube.com/xactlycorporation>
