Hello,
1. I don't know what error did you met, "I have an error message : <
INSTANCE > when loading new data sources." is not enough for me to analyse the
problem. Could you please show with me the exception(stacktrace)?
2. I have tested Kylin 4.0 to load Hive metadata using beeline and succeed.
3. My env : Hadoop 2.6, and Hive 1.1.0.
Here is my kylin conf in kylin.properties:
kylin.source.hive.client=beeline
kylin.source.hive.beeline-params=-u jdbc:hive2://cdh-master:10000 -n root
Here is connection with beeline:
[root@cdh-master apache-kylin-4.0.0-SNAPSHOT-bin]# beeline -u
jdbc:hive2://cdh-master:10000 -n root
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M;
support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated
and will likely be removed in a future release
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M;
support was removed in 8.0
scan complete in 9ms
Connecting to jdbc:hive2://cdh-master:10000
Connected to: Apache Hive (version 1.1.0-cdh5.7.6)
Driver: Hive JDBC (version 1.1.0-cdh5.7.6)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.1.0-cdh5.7.6 by Apache Hive
0: jdbc:hive2://cdh-master:10000> show databases;
INFO : Compiling
command(queryId=hive_20210916192525_0729cad4-0143-407a-9b4a-af1026912612): show
databases
INFO : Semantic Analysis Completed
INFO : Returning Hive schema:
Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from
deserializer)], properties:null)
INFO : Completed compiling
command(queryId=hive_20210916192525_0729cad4-0143-407a-9b4a-af1026912612); Time
taken: 0.002 seconds
INFO : Executing
command(queryId=hive_20210916192525_0729cad4-0143-407a-9b4a-af1026912612): show
databases
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=hive_20210916192525_0729cad4-0143-407a-9b4a-af1026912612); Time
taken: 0.123 seconds
INFO : OK
+-----------------------+--+
| database_name |
+-----------------------+--+
| apache |
| citibike |
| cube_planner_demo |
Here is verification using arthas watch
cmd(https://arthas.aliyun.com/doc/en/watch.html):
[arthas@15680]$ watch org.apache.kylin.source.hive.BeelineHiveClient init -x 2
Press Q or Ctrl+C to abort.
Affect(class count: 1 , method count: 1) cost in 126 ms, listenerId: 2
method=org.apache.kylin.source.hive.BeelineHiveClient.init location=AtExit
ts=2021-09-16 19:20:49; [cost=43.493591ms] result=@ArrayList[
@Object[][
@String[jdbc:hive2://cdh-master:10000],
@Properties[isEmpty=false;size=11],
],
@BeelineHiveClient[
HIVE_AUTH_USER=@String[user],
HIVE_AUTH_PASSWD=@String[password],
cnct=@HiveConnection[org.apache.hive.jdbc.HiveConnection@5d596e96],
stmt=@HiveStatement[org.apache.hive.jdbc.HiveStatement@4d2a9dec],
metaData=@HiveDatabaseMetaData[org.apache.hive.jdbc.HiveDatabaseMetaData@413d7dcd],
],
null,
--
Best wishes to you !
From ：Xiaoxiang Yu
At 2021-09-16 17:41:37, "MAMERI Jamal - externe"
<[email protected]> wrote:
>Hi Yaqian Zhang,
>
>
>
>By default Kylin uses Hive CLI to synchronize Hive tables.
>
>My need is using Beeline, instead of Hive CLI, to add data source tables in a
>project.
>
>When I param : kylin.source.hive.client=beeline, I have an error message : <
>INSTANCE > when loading new data sources.
>
>
>
>Regards,
>
>
>
>Jamal Mameri
>
>
>
>On 2021/09/16 09:02:14, Yaqian Zhang <[email protected]<mailto:[email protected]>> wrote:
>
>> Hi:>
>
>>
>
>> Kylin4 use spark engine to build cube, and spark application are submitted
>> through spark-submit, which has nothing to do with hive cli or beeline.. >
>
>>
>
>> Therefore, this configuration of “kylin.source.hive.client" has no effect
>> in kylin 4.>
>
>>
>
>> > 在 2021年9月16日，下午4:19，MAMERI Jamal - externe
>> > <[email protected]<mailto:[email protected]>> 写道：>
>
>> > >
>
>> > Hello,>
>
>> > >
>
>> > I hope my email finds you well.>
>
>> > >
>
>> > I have installed the Apache Kylin 4.0.0 version.>
>
>> > >
>
>> > I want to use Beeline as Hive client but the following properties aren't
>> > existing in the file : KYLIN_HOME/conf/kylin.properties :>
>
>> > >
>
>> > >
>
>> > ## Hive client, valid value [cli, beeline]>
>
>> > kylin.source.hive.client=beeline>
>
>> > #>
>
>> > ## Absolute path to beeline shell, can be set to spark beeline instead of
>> > the default hive beeline on PATH>
>
>> > kylin.source.hive.beeline-shell=/apache/hive/bin/beeline>
>
>> > #>
>
>> > ## Parameters for beeline client, only necessary if hive client is beeline>
>
>> > kylin.source.hive.beeline-params=>
>
>> > >
>
>> > >
>
>> > Is it just an oversight?>
>
>> > Can we use the Beeline client for Hive with the new version 4.0.0 ?>
>
>> > Or only Hive client (cli) is available for this version ?>
>
>> > >
>
>> > Thanks,>
>
>> > >
>
>> > Best Regards,>
>
>> > >
>
>> > Jamal Mameri>
>
>>
>
>>
