Hi Pablo,
Sorry for the delay. I think many list members (myself included) have been
on vacation the last couple weeks for the holidays.
I checked this on the latest development versions of Impala and Kudu
(corresponding to upcoming releases in January) and with the new syntax as
follows it seems to be working correctly:
create table todd_test (
a bigint,
primary key (a)
) partition by range(a) (
partition values < 1480550400000,
partition 1480550400000 <= values < 1483228800000,
partition 1483228800000 <= values < 1485907200000,
partition 1485907200000 <= values)
stored as kudu;
I didn't get a chance to try the latest released version of Impala-Kudu,
but I'm guessing you might be on a slightly older one. Can you check the
installed Parcel version and see if it matches
IMPALA_KUDU-2.7.0-3.cdh5.9.0.p0.10? This version is available from
http://archive.cloudera.com/beta/impala-kudu/parcels/latest/
-Todd
On Tue, Dec 27, 2016 at 1:44 PM, Pablo Vazquez <pablo.vazq...@globant.com>
wrote:
> Hi Kudu team.
>
> Just wondering if you have any comment related to the issue with the
> BIGINT partitions.
>
> Best Regards.
>
> On Fri, Dec 23, 2016 at 4:05 PM, Pablo Vazquez <pablo.vazq...@globant.com>
> wrote:
>
>> Hi Kudu fellowship!
>>
>> Happy holidays for all you guys.
>>
>> Recently I tried to create a simple 4-column table using a BIGINT as part
>> of the PK and use the same column for partitioning by RANGE.
>>
>> This is the script:
>>
>> *CREATE EXTERNAL TABLE `kd_clicks` (*
>> *`channel_id` BIGINT,*
>> *`timestamp` BIGINT,*
>> *`channel_name` STRING,*
>> *`cookie_hash` STRING*
>> *)*
>> *DISTRIBUTE BY RANGE (`timestamp`)*
>> *SPLIT ROWS ( *
>> *(1480550400000),*
>> *(1483228800000),*
>> *(1485907200000)*
>> *)*
>> *TBLPROPERTIES(*
>> * 'storage_handler' = 'com.cloudera.kudu.hive.KuduStorageHandler',*
>> * 'kudu.table_name' = 'kd_clicks',*
>> * 'kudu.master_addresses' = 'XXXXXX',*
>> * 'kudu.key_columns' = 'channel_id, timestamp'*
>> *)*
>> *;*
>>
>> However, in the Kudu UI the partitions are not created properly:
>>
>> Partition schema & range bounds
>>
>> Range component:
>> (timestamp int64)
>> Range bounds:
>> [<start>, (int64 timestamp=-1213317120))
>> [(int64 timestamp=-1213317120), (int64 timestamp=-151484416))
>> [(int64 timestamp=-151484416), (int64 timestamp=1465082880))
>> [(int64 timestamp=1465082880), <end>)
>>
>>
>> We made some tests and we found the issue appears when the partition is
>> defined above 2,147,483,647 (upper limit for Int-32). It seems the range
>> partition only accept INT32 boundaries although the column is INT64.
>>
>> I'd appreciate your comments. We're not sure if we are missing something
>> during table creation.
>>
>> Regards.
>> --
>> *Pablo Quetzalcóatl Vázquez*| Software Designer
>> *GLOBANT*
>>
>> [image: Facebook]
>> <http://s.wisestamp.com/links?url=https%3A%2F%2Fwww.facebook.com%2FGlobant&sn=b3psb3BlekB5YWhvby5jb20%3D>
>> [image:
>> Twitter]
>> <http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.twitter.com%2Fglobant&sn=b3psb3BlekB5YWhvby5jb20%3D>
>> [image:
>> Youtube]
>> <http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.youtube.com%2FGlobant&sn=b3psb3BlekB5YWhvby5jb20%3D>
>> [image:
>> Linkedin]
>> <http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fglobant&sn=b3psb3BlekB5YWhvby5jb20%3D>
>> [image:
>> Pinterest]
>> <http://s.wisestamp.com/links?url=http%3A%2F%2Fpinterest.com%2Fglobant%2F&sn=b3psb3BlekB5YWhvby5jb20%3D>
>> [image:
>> Globant]
>> <http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.globant.com%2F&sn=b3psb3BlekB5YWhvby5jb20%3D>
>>
>
>
>
> --
> *Pablo Quetzalcóatl Vázquez*| Software Designer
> *GLOBANT*
>
> [image: Facebook]
> <http://s.wisestamp.com/links?url=https%3A%2F%2Fwww.facebook.com%2FGlobant&sn=b3psb3BlekB5YWhvby5jb20%3D>
> [image:
> Twitter]
> <http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.twitter.com%2Fglobant&sn=b3psb3BlekB5YWhvby5jb20%3D>
> [image:
> Youtube]
> <http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.youtube.com%2FGlobant&sn=b3psb3BlekB5YWhvby5jb20%3D>
> [image:
> Linkedin]
> <http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fglobant&sn=b3psb3BlekB5YWhvby5jb20%3D>
> [image:
> Pinterest]
> <http://s.wisestamp.com/links?url=http%3A%2F%2Fpinterest.com%2Fglobant%2F&sn=b3psb3BlekB5YWhvby5jb20%3D>
> [image:
> Globant]
> <http://s.wisestamp.com/links?url=http%3A%2F%2Fwww.globant.com%2F&sn=b3psb3BlekB5YWhvby5jb20%3D>
>
> The information contained in this e-mail may be confidential. It has been
> sent for the sole use of the intended recipient(s). If the reader of this
> message is not an intended recipient, you are hereby notified that any
> unauthorized review, use, disclosure, dissemination, distribution or
> copying of this communication, or any of its contents,
> is strictly prohibited. If you have received it by mistake please let us
> know by e-mail immediately and delete it from your system. Many thanks.
>
>
>
> La información contenida en este mensaje puede ser confidencial. Ha sido
> enviada para el uso exclusivo del destinatario(s) previsto. Si el lector de
> este mensaje no fuera el destinatario previsto, por el presente queda Ud.
> notificado que cualquier lectura, uso, publicación, diseminación,
> distribución o copiado de esta comunicación o su contenido está
> estrictamente prohibido. En caso de que Ud. hubiera recibido este mensaje
> por error le agradeceremos notificarnos por e-mail inmediatamente y
> eliminarlo de su sistema. Muchas gracias.
>
>
--
Todd Lipcon
Software Engineer, Cloudera
