Dynamic vs Static partition

[Sachit Murarka]

Hi,

I am unclear about Dynamic and static partition.

   1. Why static partition is good for loading large files and why can’t we
   use dynamic partition for the same?
   2. Why does dynamic partition take more time in loading data than static
   partitions?

   Also please explain when to use strict and nonstrict mode.

Kind Regards,
Sachit Murarka

Re: 答复: Re: Does Hive SQL support reading data without locking?

[Furcy Pin]

You can disable locking by setting

SET hive.support.concurrency=true ;

It can be used in a single session before a query.

However locking exists for a reason: what do you think will happen if you
overwrite files from a table while Tableau is reading them?
It really depends on your use case, but if possible, partitioning your
table would be better.

For instance, if the table is not too big, and if you bulk-import your
table every day
you could have one partition per day and a view that exposes the last
partition.
Some tools like Airflow support the easy retrieval of the latest available
partition.






2018-01-02 10:28 GMT+01:00 :

> Bulk import and no partition
>
>
> 发件人: Jörn Franke 
> 收件人: user@hive.apache.org
> 日期: 2018/01/02 14:56
> 主题: Re: Does Hive SQL support reading data without locking?
> --
>
>
>
> How do you import data ? Bulk import?
>
> What about using partitions ( or is the data too small for daily
> partitions?)
>
> On 2. Jan 2018, at 04:59, *l...@china-inv.cn*  wrote:
>
> Hi, All,
>
> We are using Hive to persist our data and we run cron jobs to import new
> data into Hive daily.
>
> At the same time, our users may query data from Hive at the same time by
> using a third party software like Tableau
>
> We found an issue that importing job will fail while the table is queried
> by user because of failing to get an exclusive lock
>
> Does Hive SQL support reading data without locking (read lock) just like
> 'with (nolock)' provided by SQL server?
>
> Thanks
>
> Boying
>
> --
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外披露或复制本邮件的任何内容，
> 或者打开本邮件的任何附件。请即回复邮件告知发件人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
>
> This email message may contain confidential and/or privileged information.
> If you are not the intended recipient, please do not read, save, forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender immediately
> by replying this email, and delete this email and any attachment or links
> to this email completely and immediately from your computer system.
>
> --
>
>
>
>
> --
>
>
> 本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外披露或复制本邮件的任何内容，
> 或者打开本邮件的任何附件。请即回复邮件告知发件人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。
>
>
>
> This email message may contain confidential and/or privileged information.
> If you are not the intended recipient, please do not read, save, forward,
> disclose or copy the contents of this email or open any file attached to
> this email. We will be grateful if you could advise the sender immediately
> by replying this email, and delete this email and any attachment or links
> to this email completely and immediately from your computer system.
>
>
> --
>
>

答复: Re: Does Hive SQL support reading data without locking?

[luby]

Bulk import and no partition



发件人: 
Jörn Franke 
收件人:
user@hive.apache.org
日期:
2018/01/02 14:56
主题:
Re: Does Hive SQL support reading data without locking?



How do you import data ? Bulk import?

What about using partitions ( or is the data too small for daily 
partitions?)

On 2. Jan 2018, at 04:59, l...@china-inv.cn wrote:

Hi, All, 

We are using Hive to persist our data and we run cron jobs to import new 
data into Hive daily. 

At the same time, our users may query data from Hive at the same time by 
using a third party software like Tableau 

We found an issue that importing job will fail while the table is queried 
by user because of failing to get an exclusive lock 

Does Hive SQL support reading data without locking (read lock) just like 
'with (nolock)' provided by SQL server? 

Thanks 

Boying 


 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。 

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system. 






 
本邮件内容包含保密信息。如阁下并非拟发送的收件人，请您不要阅读、保存、对外
披露或复制本邮件的任何内容，或者打开本邮件的任何附件。请即回复邮件告知发件
人，并立刻将该邮件及其附件从您的电脑系统中全部删除，不胜感激。

 
This email message may contain confidential and/or privileged information. 
If you are not the intended recipient, please do not read, save, forward, 
disclose or copy the contents of this email or open any file attached to 
this email. We will be grateful if you could advise the sender immediately 
by replying this email, and delete this email and any attachment or links 
to this email completely and immediately from your computer system.