Re: UDF get_splits()
Hello Eric,
Now I understand why get_splits() returns LlapInputSplit[]. It seems that
get_splits() is not part of public API as there is no documentation about
it (and the user cannot do anything useful using the result of get_splits()
anyway).
Thank you very much for your email and saving me a lot of time!
--- Sungwoo
On Mon, Apr 6, 2020 at 4:38 AM Eric Wohlstadter wrote:
> Hi Sungoo,
> get_splits() is used by the HiveWarehouseConnector, you can see an
> example here:
> https://github.com/hortonworks/hive-warehouse-connector/blob/HDP-3.1.5.20-4-tag/src/main/java/com/hortonworks/spark/sql/hive/llap/HiveWarehouseDataSourceReader.java
>
> That being said, I'm not sure if this UDF is technically supported as a
> public API by the Hive community, so you may want to check about that.
>
> Eric
>
> On Sun, Apr 5, 2020 at 11:52 AM Sungwoo Park wrote:
>
>> Hello,
>>
>> I would like to learn the use of UDF get_splits(). I tried such queries
>> as:
>>
>> select get_splits("select * from web_returns", 1) ;
>> select get_splits("select count(*) from web_returns", 1);
>>
>> These queries just return InputSplit objects, and I would like to see an
>> example that uses the result of calling get_splits(). If someone could
>> share a practical example using get_splits(), I would appreciate it very
>> much.
>>
>> Thanks,
>>
>> --- Sungwoo
>>
>
Re: UDF get_splits()
Hi Sungoo,
get_splits() is used by the HiveWarehouseConnector, you can see an example
here:
https://github.com/hortonworks/hive-warehouse-connector/blob/HDP-3.1.5.20-4-tag/src/main/java/com/hortonworks/spark/sql/hive/llap/HiveWarehouseDataSourceReader.java
That being said, I'm not sure if this UDF is technically supported as a
public API by the Hive community, so you may want to check about that.
Eric
On Sun, Apr 5, 2020 at 11:52 AM Sungwoo Park wrote:
> Hello,
>
> I would like to learn the use of UDF get_splits(). I tried such queries as:
>
> select get_splits("select * from web_returns", 1) ;
> select get_splits("select count(*) from web_returns", 1);
>
> These queries just return InputSplit objects, and I would like to see an
> example that uses the result of calling get_splits(). If someone could
> share a practical example using get_splits(), I would appreciate it very
> much.
>
> Thanks,
>
> --- Sungwoo
>
UDF get_splits()
Hello,
I would like to learn the use of UDF get_splits(). I tried such queries as:
select get_splits("select * from web_returns", 1) ;
select get_splits("select count(*) from web_returns", 1);
These queries just return InputSplit objects, and I would like to see an
example that uses the result of calling get_splits(). If someone could
share a practical example using get_splits(), I would appreciate it very
much.
Thanks,
--- Sungwoo
