Re: Any plans for supporting schemas (namespaces of tables)?

[William Berkeley]

Hi Martin. I don't see any conflicts between that and any current or
near-term work I know of happening in Kudu. There are a couple of related
JIRAs for database support: KUDU-2063
 and KUDU-2362
.

In fact, Impala does something similar right now. Kudu tables managed by
Impala are named like impala::database_name.table_name. It might also be a
good idea to add some kind of identifier like the initial "impala::" for
presto-managed or -created tables, since many other integrations may use a
database_name.table_name convention.

-Will

On Tue, Apr 24, 2018 at 7:01 PM, Martin Weindel 
wrote:

> Hi,
>
> are there any plans for supporting schemas in the sense of relational
> databases, i.e. namespaces of tables?
>
> I'm asking, as I have a discussion in the Presto project for the proposed
> Kudu connector. (see https://github.com/prestodb/presto/pull/10388)
>
> As Presto supports catalog > schema > table, I thought it a good idea to
> make use of these namespaces of tables also in the Kudu connector.
>
> This happens by using a dot in the Kudu table name. E.g. the Kudu table
> name "mytable" is  mapped to kudu.default.mytable in Presto and a Kudu
> table named "myschema.mytable2" to kudu.myschema.mytable2. There is nothing
> in Kudu which prevents using table names with dots, so it does not seem to
> be a problem at least at the moment.
>
> Can someone of the Kudu developer team can give me a hint, if you see any
> conflicts with the Kudu roadmap?
>
> Thanks,
>
> Martin Weindel
>
>

Any plans for supporting schemas (namespaces of tables)?

[Martin Weindel]

Hi,

are there any plans for supporting schemas in the sense of relational 
databases, i.e. namespaces of tables?


I'm asking, as I have a discussion in the Presto project for the 
proposed Kudu connector. (see https://github.com/prestodb/presto/pull/10388)


As Presto supports catalog > schema > table, I thought it a good idea to 
make use of these namespaces of tables also in the Kudu connector.


This happens by using a dot in the Kudu table name. E.g. the Kudu table 
name "mytable" is  mapped to kudu.default.mytable in Presto and a Kudu 
table named "myschema.mytable2" to kudu.myschema.mytable2. There is 
nothing in Kudu which prevents using table names with dots, so it does 
not seem to be a problem at least at the moment.


Can someone of the Kudu developer team can give me a hint, if you see 
any conflicts with the Kudu roadmap?


Thanks,

Martin Weindel

Re: Reverse sort on Primary Key

[Dan Burkert]

The comparator for ScanTokens is based on the partition key, which isn't
(necessarily) the same as primary key order.  If you have any kind of hash
partitioning you will need to do a merge, and if the range partitioning
doesn't match the PK then it will be completely different.  So, if you set
up the table's partitioning very carefully you could do the reverse sorting
tablet-by-tablet instead of globally.

- Dan


On Mon, Apr 23, 2018 at 9:32 PM, Scott Reynolds 
wrote:

> Thanks guys thought it should be easy. I am looking to paginate through
> the records, not return the most recent value.
>
> If I need to toggle -- which I might -- I was planning on using token api.
> The KuduScanToken actually implements Comparable and it's implementation is
> around the tablet. I think this means you can sort the list of
> KuduScanTokens and call out to each one sequentially. Does that sound crazy?
>
> On Mon, Apr 23, 2018 at 3:23 PM Dan Burkert  wrote:
>
>> Hey Scott,
>>
>> Patrick's answer is spot on.  I'm curious, though, is your usecase to
>> find the latest value?  Effectively a 'SORT BY DESC date LIMIT 1', or are
>> you looking for the last n values, or all values?  I ask because we
>> frequently get the 'last value' question, and the solution for that might
>> be more specific (and simpler) than a generalized reverse sort + limit.
>>
>> - Dan
>>
>> On Mon, Apr 23, 2018 at 1:25 PM, Patrick Angeles 
>> wrote:
>>
>>> The common technique is to use (MAX_LONG - timestamp). Unfortunately
>>> this won't let you toggle the sort order back-and-forth on the same table.
>>> You could have a duplicate table with the inverse key, effectively using it
>>> as a secondary index.
>>>
>>> As of version 0.98, HBase supports a reverse scan without a 'secondary
>>> index' table (HBASE-4811), so with a bit of work Kudu may be able to
>>> provide something similar.
>>>
>>>
>>> Patrick Angeles
>>> Chief Architect Financial Services
>>> 151 West 26th Street Suite 1002 |
>>> New
>>> York, NY 10001
>>> 
>>> +1 (917) 633-4524 <(917)%20633-4524>
>>>
>>> On Sat, Apr 21, 2018 at 10:36 PM, Scott Reynolds 
>>> wrote:
>>>
 Today we are using Kudu  to store timeseries information and would like
 the ability to toggle the sort direction. It is unclear to me at the moment
 how to achieve this efficiently. I naively assumed Kudu could read the
 primary key in reverse but there doesn't appear to be the case ATM.

 If you were tasked with implementing a reverse sort on the primary key
 (Date Desc) how would you go about implementing it ?

 Thanks!

>>>
>>>
>>