Re: Kylin defaut storage system is HDFS?

[marc nicole]

Hi Yu,

This link *https://kylin.apache.org/docs31/tutorial/setup_jdbc_datasource.html
*
suggests that "*Since v2.3.0 Apache Kylin starts to support JDBC as the
third type of data source (after Hive, Kafka)"*

So My question of if I can have MySQL as an  alternative to Hive has
positive answer according to the link above, or am I wrong?

Le lun. 28 août 2023 à 04:48, Xiaoxiang Yu  a écrit :

> Hi,
> For Kylin 5, you have to use a distributed storage, and the default
> choice is HDFS,
> and alternative choice is Cloud Storage(like S3), you can NOT deploy and
> run Kylin
> without a distributed storage.
> Besides, you need a RDBMS as a metastore, Zookeeper as service
> discovery,
> a Spark cluster as compute service, a Hive Metastore for seeking
> databases and tables.
> Finally, HBae is totally NOT necessary for Kylin 4.0 or higher.
>
> For the question 'Could I use Kylin with just MySQL + Sqoop? (no Hive)
> ', the
> answer is no, you need to install and deployed Zookeeper, a distributed
> storage
> (HDFS or cloud storage), a Spark cluster and a Hive metastore. Here is a
> diagram
> maybe helpful:
> https://kylin.apache.org/images/blog/kylin4_on_cloud/3_kylin_cluster.jpg
>
> Here are some links:
> - https://kylin.apache.org/blog/2022/04/20/kylin4-on-cloud-part1/
> -
> https://kylin.apache.org/5.0/docs/deployment/on-premises/installation/platform/install_on_apache_hadoop
>
>
> 
> With warm regard
> Xiaoxiang Yu
>
>
>
> On Sat, Aug 26, 2023 at 8:03 PM marc nicole  wrote:
>
>> Hello,
>>
>> I have few questions regarding storage mean for Kylin:
>>
>> I was wondering if Kylin would work normally if I don't configure it to
>> work with any storage tool (as MySQL with Sqoop Or with Hive)? It would
>> then automatically use HDFS ?
>>
>> Also is configuring HBASE necessary?
>>
>> Could I use Kylin with just MySQL + Sqoop? (no Hive)
>> What the use of HBase if the normal used storage is Hive?
>>
>> Thanks. Regards
>>
>

Re: Kylin defaut storage system is HDFS?

[Xiaoxiang Yu]

Hi,
For Kylin 5, you have to use a distributed storage, and the default
choice is HDFS,
and alternative choice is Cloud Storage(like S3), you can NOT deploy and
run Kylin
without a distributed storage.
Besides, you need a RDBMS as a metastore, Zookeeper as service
discovery,
a Spark cluster as compute service, a Hive Metastore for seeking
databases and tables.
Finally, HBae is totally NOT necessary for Kylin 4.0 or higher.

For the question 'Could I use Kylin with just MySQL + Sqoop? (no Hive)
', the
answer is no, you need to install and deployed Zookeeper, a distributed
storage
(HDFS or cloud storage), a Spark cluster and a Hive metastore. Here is a
diagram
maybe helpful:
https://kylin.apache.org/images/blog/kylin4_on_cloud/3_kylin_cluster.jpg

Here are some links:
- https://kylin.apache.org/blog/2022/04/20/kylin4-on-cloud-part1/
-
https://kylin.apache.org/5.0/docs/deployment/on-premises/installation/platform/install_on_apache_hadoop



With warm regard
Xiaoxiang Yu



On Sat, Aug 26, 2023 at 8:03 PM marc nicole  wrote:

> Hello,
>
> I have few questions regarding storage mean for Kylin:
>
> I was wondering if Kylin would work normally if I don't configure it to
> work with any storage tool (as MySQL with Sqoop Or with Hive)? It would
> then automatically use HDFS ?
>
> Also is configuring HBASE necessary?
>
> Could I use Kylin with just MySQL + Sqoop? (no Hive)
> What the use of HBase if the normal used storage is Hive?
>
> Thanks. Regards
>

Kylin defaut storage system is HDFS?

[marc nicole]

Hello,

I have few questions regarding storage mean for Kylin:

I was wondering if Kylin would work normally if I don't configure it to
work with any storage tool (as MySQL with Sqoop Or with Hive)? It would
then automatically use HDFS ?

Also is configuring HBASE necessary?

Could I use Kylin with just MySQL + Sqoop? (no Hive)
What the use of HBase if the normal used storage is Hive?

Thanks. Regards