Difference between hive-exec-log4j.properties & hive-log4j.properties

[Akash Mishra]

Hi *,

I am unable to understand the difference between two logs configuration
file present in the /conf folder. Can someone explain the difference?

According to documentation,

Starting with release 0.6 – Hive uses the hive-exec-log4j.properties (falling
back to hive-log4j.properties only if it's missing) to determine where
these logs are delivered by default.


Does that mean logs configuration in hive-log4j.properties will be
overridden by hive-exec-log4j.properties?

-- 

Regards,
Akash Mishra.


"It's not our abilities that make us, but our decisions."--Albus Dumbledore

Killing JVM on Metastore on OnOutOfMemory error

[Akash Mishra]

Hi *,

I am trying to set up an OnOutOfMemory kill on hive-env.sh so that JVM
should shutdown on OOM error in Hive Metastore.

Code is :

if [ "$SERVICE" = "metastore" ]; then
  export HADOOP_OPTS="$HADOOP_OPTS -XX:OnOutOfMemoryError=\"kill -9 %p\" "
fi


On starting Hive Metastore is, I am getting,


h...@hadoopdev7.mlan:~> hive --service metastore
Unrecognized option: -9
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Unable to determine Hadoop version information.
'hadoop version' returned:
Unrecognized option: -9
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.



Any ideas?





-- 

Regards,
Akash Mishra.


"It's not our abilities that make us, but our decisions."--Albus Dumbledore

Re: Why Hive uses MetaStore？

[Akash Mishra]

Hive need a place to store the Metadata. e.g What tables have data where,
how many columns e.t.c. This is where MetaStore gets its name from. It
helps user store the information about metadata.

HiveMetaStore provides following:

1. Separations of concerns: All Storage related information is abstracted.
You can store metadata in MYSQL, Postgres, or you inhouse DB.
2. Gives a clean API all hive query runner, e.g: Hive JDBC, Spark and hive
cli.

any many more reason.



On Wed, Jan 15, 2020 at 3:17 PM qq <987626...@qq.com> wrote:

> Thank you very much for your answer. But I ’m very sorry, I still do n’t
> understand. Can you elaborate more?
>
>
> -- 原始邮件 --
> *发件人:* "David Mollitor";
> *发送时间:* 2020年1月15日(星期三) 晚上11:01
> *收件人:* "user";
> *主题:* Re: Why Hive uses MetaStore？
>
> In the beginning, hive was a command line tool.  All the heavy lifting
> happened on the user's local box.  If a user wanted to execute hive from
> their laptop, or a server, it always needs access to the list of available
> tables (and their schemas and their locations), otherwise every SQL script
> would have to start with the table definition of every table involved in
> the query. Today, it's primarily HiveServer2 and there can be many
> instances in the same environment.  They all need to share table meta data.
>
> It is also helpful to keep statistics on every table.  Table statistics
> are key in generating efficient query plans.  If yue statistics are not
> persisted anywhere, it would be lost after every query.
>
> Since Hive2, the role of the metastore has only expanded to include, for
> example, a global locking system to protect tables from concurrent access.
>
> On Wed, Jan 15, 2020, 9:51 AM qq <987626...@qq.com> wrote:
>
>> Hello:
>>   Why Hive uses MetaStore？Is there a related JIRA?
>>
>>  thinks.
>>  I am looking forward to your reply！
>>
>

-- 

Regards,
Akash Mishra.


"It's not our abilities that make us, but our decisions."--Albus Dumbledore