Re: behavior or insert overwrite with dynamic partitions

[Eugene Koifman]

I think ³it should overwrite the entire table² makes more sense.
One can reasonably expect that
³insert overwrite table t partition (pk)
   select 1 as val, key from src where key > 20;²  should actually be
written as

³insert overwrite table t
   select 1 as val, key from src where key > 20;²

and for Hive to know partition columns.
With this syntax the ³overwrite the entire table² is more intuitive.



On 10/17/16, 7:29 PM, "Sushanth Sowmyan"  wrote:

>I expect the following because it follows per-ptn if-write-then-overwrite
>semantics:
>
>0,10
>1,25
>1,50
>
>There can be a case to be made that it should overwrite the entire table,
>and that would make sense too(probably more sense than this one), but not
>one I'd think we should switch behavior to(backward compatibility).
>
>On Oct 17, 2016 18:10, "Sergey Shelukhin"  wrote:
>
>> What do you think this SHOULD do?
>>
>> > select key from src;
>> 10
>> 25
>> 50
>>
>> > create table t(val int) partitioned by (pk int);
>> > insert overwrite table t partition (pk)
>>   select 0 as val, key from src where key < 30;
>> > insert overwrite table t partition (pk)
>>   select 1 as val, key from src where key > 20;
>>
>>
>> > select val, pk from t;
>> ?
>>
>>

Re: behavior or insert overwrite with dynamic partitions

[Sushanth Sowmyan]

I expect the following because it follows per-ptn if-write-then-overwrite
semantics:

0,10
1,25
1,50

There can be a case to be made that it should overwrite the entire table,
and that would make sense too(probably more sense than this one), but not
one I'd think we should switch behavior to(backward compatibility).

On Oct 17, 2016 18:10, "Sergey Shelukhin"  wrote:

> What do you think this SHOULD do?
>
> > select key from src;
> 10
> 25
> 50
>
> > create table t(val int) partitioned by (pk int);
> > insert overwrite table t partition (pk)
>   select 0 as val, key from src where key < 30;
> > insert overwrite table t partition (pk)
>   select 1 as val, key from src where key > 20;
>
>
> > select val, pk from t;
> ?
>
>

Re: behavior or insert overwrite with dynamic partitions

[Xuefu Zhang]

It seems right to me that an existing partition should be overwritten if
that partition gets any data while older, untouched partition should stay.
After all, we are overwriting certain partitions, not the whole table.

--Xuefu

On Mon, Oct 17, 2016 at 6:10 PM, Sergey Shelukhin 
wrote:

> What do you think this SHOULD do?
>
> > select key from src;
> 10
> 25
> 50
>
> > create table t(val int) partitioned by (pk int);
> > insert overwrite table t partition (pk)
>   select 0 as val, key from src where key < 30;
> > insert overwrite table t partition (pk)
>   select 1 as val, key from src where key > 20;
>
>
> > select val, pk from t;
> ?
>
>

behavior or insert overwrite with dynamic partitions

[Sergey Shelukhin]

What do you think this SHOULD do?

> select key from src;
10
25
50

> create table t(val int) partitioned by (pk int);
> insert overwrite table t partition (pk)
  select 0 as val, key from src where key < 30;
> insert overwrite table t partition (pk)
  select 1 as val, key from src where key > 20;


> select val, pk from t;
?