No equivalent. Instead they came up with Drill - MaprDB connector for performance and flexibility. So like earlier version of Kylin I should be able to make drill calls but still do filtering and aggregation on cluster. Only thing is we need to run separate drill-bit service under Yarn for this to work.
Thanks, Nirav On Sun, Jul 9, 2017 at 5:06 PM, ShaoFeng Shi <[email protected]> wrote: > Is there some coprocessor-equivalent mechanism in maprdb? If yes, you can > try to implement the logic into it. Otherwise, the performance might be > unacceptable when there is bunch of data in each region. > > I rememer in very early version, Kylin doesn't have coprocessor, all data > be fetched to Kylin server and then do filtering and aggregation by > Calcite. Later coprocessor was added and improved continiously. Now there > is no option to disable it. You have to modify the code if don't want to > trigger it. > > > > 2017-07-09 1:34 GMT+08:00 Nirav Patel <[email protected]>: > > > > > Thanks Shao for some pointers. I would like some more advice on how I can > > replace maprdb as datasource. > > > > Currently, you can create cubes on maprdb using hbase-storage module as > > maprdb does support hbase client/admin APIs. Only problem is while > querying > > it throws an error due to Lack of support of Co-processor. > > > > My initial goal is to get past coprocessor error and return query result. > > Then find a better way to implement that strategy - may be a separate > > maprdb-storage module. Then look into how to improve performance without > > coprocessors. Any advice on these approaches ? > > > > Thanks, > > Nirav > > > > On Sat, Jul 8, 2017 at 7:54 AM, ShaoFeng Shi <[email protected]> > > wrote: > > > >> The core modules like core-cube, core-storage are totally independent of > >> HBase; while some others like engine-mr, engine-spark has dependencies > on > >> HBase. If you want to replace it, you need implement new cubing engine > as > >> well. Please also note that Kylin's metadata is persisted to HBase by > >> default, you need have another implementation for ResourceStore. > >> > >> In a short, the plug-in architecture works (we have verfied that), while > >> changing the storage is a complex task which take some time to be > function > >> complete and performance stable. > >> > >> > >> > >> > >> > >> > >> 2017-07-08 5:19 GMT+08:00 Nirav Patel <[email protected]>: > >> > >>> Yes, MapR-DB doesn't support coprocessors. > >>> > >>> Here's the thing though - based on kylin plugin architecture it > >>> shouldn't be a problem ideally. Aggregation as well as other DML/DDL > >>> operation on datasources should be done transparently. i.e. using > >>> kylin-hbase adapter written with calcite or something. It's upto the > writer > >>> of those adapters to implement aggregates however they want. i.e. > either > >>> using coprocessors, or in-memory on application server side or using > spark. > >>> http://kylin.apache.org/development/plugin_arch.html > >>> > >>> I think for mapr-db we can leverage mapr-drill with secondary indexes > >>> for faster filtering and drill does parallel aggregation as well. Other > >>> option is to use mapr-spark with mapr-db which can does the same. > >>> > >>> Do you know how tightly hbase is coupled with other modules of kylin > >>> source other then hbase-storage. > >>> > >>> > >>> > >>> > >>> > >>> > >>> On Thu, Jul 6, 2017 at 6:53 PM, ShaoFeng Shi <[email protected]> > >>> wrote: > >>> > >>>> Hi Nirav, > >>>> > >>>> I googled that "HBase coprocessors are not present in MapR DB", is > >>>> this true? You know Kylin relies on HBase coprocessor to do filering > and > >>>> aggregation in each region local; If coprocessor is not available, the > >>>> performance will be a problem. > >>>> > >>>> 2017-07-07 1:29 GMT+08:00 Nirav Patel <[email protected]>: > >>>> > >>>>> Hi, We are a mapr users. You will need to deploy separate Hbase > >>>>> Cluster alongside your mapr Cluster or on top of it. You won't be > able to > >>>>> use Mapr-DB with Kylin. > >>>>> I am looking into writing Kyling - MaprDB adapter. > >>>>> > >>>>> On Tue, Jul 4, 2017 at 8:23 AM, <[email protected]> wrote: > >>>>> > >>>>>> V5.2, > >>>>>> > >>>>>> I’m trying to figure out how to deploy Kylin on a MapR cluster, > >>>>>> whether Kylin has to be installed on the same cluster of the MapR > cluster > >>>>>> > >>>>>> > >>>>>> > >>>>>> *From:* Luke Han [mailto:[email protected]] > >>>>>> *Sent:* Tuesday, July 04, 2017 10:04 PM > >>>>>> *To:* user > >>>>>> *Subject:* Re: Kylin for MapR > >>>>>> > >>>>>> > >>>>>> > >>>>>> Kylin support MapR, which MapR version you are using now? > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> Best Regards! > >>>>>> --------------------- > >>>>>> > >>>>>> Luke Han > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Tue, Jul 4, 2017 at 4:19 PM, <[email protected]> wrote: > >>>>>> > >>>>>> Hi > >>>>>> > >>>>>> Does kylin support MapR version of the Hadoop? > >>>>>> > >>>>>> > >>>>>> > >>>>>> This e-mail (including any attachments) is private and confidential, > >>>>>> may contain proprietary or privileged information and is intended > for the > >>>>>> named recipient(s) only. Unintended recipients are strictly > prohibited from > >>>>>> taking action on the basis of information in this e-mail and must > contact > >>>>>> the sender immediately, delete this e-mail (and all attachments) and > >>>>>> destroy any hard copies. Nomura will not accept responsibility or > liability > >>>>>> for the accuracy or completeness of, or the presence of any virus or > >>>>>> disabling code in, this e-mail. If verification is sought please > request a > >>>>>> hard copy. Any reference to the terms of executed transactions > should be > >>>>>> treated as preliminary only and subject to formal written > confirmation by > >>>>>> Nomura. Nomura reserves the right to retain, monitor and intercept > e-mail > >>>>>> communications through its networks (subject to and in accordance > with > >>>>>> applicable laws). No confidentiality or privilege is waived or lost > by > >>>>>> Nomura by any mistransmission of this e-mail. Any reference to > "Nomura" is > >>>>>> a reference to any entity in the Nomura Holdings, Inc. group. > Please read > >>>>>> our Electronic Communications Legal Notice which forms part of this > e-mail: > >>>>>> http://www.Nomura.com/email_disclaimer.htm > >>>>>> <https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.Nomura.com_email-5Fdisclaimer.htm&d=DwMFaQ&c= > dCBwIlVXJsYZrY6gpNt0LA&r=nFqGYx5SOTBIei-Hwk8uz_JfhmSu2UlelQfdmGX2k5I&m= > s6jR2dvhFFTZIuHhCE8WU8EYNQ7M5u-9SggFAoyn5-M&s=iXZr3lmnuJ4Xk2- > qORnbYZQALda0jRLZ3kzGy816bco&e=> > >>>>>> > >>>>>> > >>>>>> > >>>>>> This e-mail (including any attachments) is private and confidential, > >>>>>> may contain proprietary or privileged information and is intended > for the > >>>>>> named recipient(s) only. Unintended recipients are strictly > prohibited from > >>>>>> taking action on the basis of information in this e-mail and must > contact > >>>>>> the sender immediately, delete this e-mail (and all attachments) and > >>>>>> destroy any hard copies. Nomura will not accept responsibility or > liability > >>>>>> for the accuracy or completeness of, or the presence of any virus or > >>>>>> disabling code in, this e-mail. If verification is sought please > request a > >>>>>> hard copy. Any reference to the terms of executed transactions > should be > >>>>>> treated as preliminary only and subject to formal written > confirmation by > >>>>>> Nomura. Nomura reserves the right to retain, monitor and intercept > e-mail > >>>>>> communications through its networks (subject to and in accordance > with > >>>>>> applicable laws). No confidentiality or privilege is waived or lost > by > >>>>>> Nomura by any mistransmission of this e-mail. Any reference to > "Nomura" is > >>>>>> a reference to any entity in the Nomura Holdings, Inc. group. > Please read > >>>>>> our Electronic Communications Legal Notice which forms part of this > e-mail: > >>>>>> http://www.Nomura.com/email_disclaimer.htm > >>>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> [image: What's New with Xactly] > >>>>> <http://www.xactlycorp.com/email-click/> > >>>>> > >>>>> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] > >>>>> <https://www.linkedin.com/company/xactly-corporation> [image: > >>>>> Twitter] <https://twitter.com/Xactly> [image: Facebook] > >>>>> <https://www.facebook.com/XactlyCorp> [image: YouTube] > >>>>> <http://www.youtube.com/xactlycorporation> > >>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Best regards, > >>>> > >>>> Shaofeng Shi 史少锋 > >>>> > >>>> > >>> > >>> > >>> > >>> [image: What's New with Xactly] <http://www.xactlycorp.com/ > email-click/> > >>> > >>> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] > >>> <https://www.linkedin.com/company/xactly-corporation> [image: > Twitter] > >>> <https://twitter.com/Xactly> [image: Facebook] > >>> <https://www.facebook.com/XactlyCorp> [image: YouTube] > >>> <http://www.youtube.com/xactlycorporation> > >>> > >> > >> > >> > >> -- > >> Best regards, > >> > >> Shaofeng Shi 史少锋 > >> > >> > > > > > > > > [image: What's New with Xactly] <http://www.xactlycorp.com/email-click/> > > > > <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] > > <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] > > <https://twitter.com/Xactly> [image: Facebook] > > <https://www.facebook.com/XactlyCorp> [image: YouTube] > > <http://www.youtube.com/xactlycorporation> > > > > > > -- > Best regards, > > Shaofeng Shi 史少锋 > -- [image: What's New with Xactly] <http://www.xactlycorp.com/email-click/> <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] <https://twitter.com/Xactly> [image: Facebook] <https://www.facebook.com/XactlyCorp> [image: YouTube] <http://www.youtube.com/xactlycorporation>
