the short answer is there is no native hive UDF that solves your unique
case.  That means you have to solve it.


i searched for something like you were looking for myself and found this
general recipe:

http://www.onlinemathlearning.com/median-frequency-table.html

off the top of my head i'm not sure how easy this would be in SQL but i
imagine using ROW_NUMBER() function a clever person could do it. And if not
SQL then perhaps a custom UDF but ultimately you have to do the work and
now you have a potential recipe to follow. :)




On Wed, Mar 19, 2014 at 9:37 PM, Seema Datar <[email protected]> wrote:

>  Not really. If it was a single column with no counters, Hive provides an
> option to use percentile. So basically if the data was like -
>
>  100
> 100
> 200
> 200
> 200
> 200
> 300
>
>  But if we have 2 columns, one that maintain the value and the other that
> maintains the count, how can Hive be used to derive the percentile?
>
>  Value     Count
> 100          2
> 200          4
> 300          1
>
>  Thanks,
> Seema
>
>   From: Stephen Sprague <[email protected]>
> Reply-To: "[email protected]" <[email protected]>
> Date: Thursday, March 20, 2014 5:28 AM
> To: "[email protected]" <[email protected]>
> Subject: Re: computing median and percentiles
>
>   not a hive question is it?   its more like a math question.
>
>
>
> On Wed, Mar 19, 2014 at 1:30 PM, Seema Datar <[email protected]> wrote:
>
>>
>>
>>   I understand the percentile function is supported in Hive in the
>> latest versions. However, how does once calculate percentiles when the data
>> is across two columns. So say -
>>
>>  Value  Count
>>
>>  100 2   ( so basically 100 occurred twice)
>> 200 4
>> 300 1
>> 400 6
>> 500 3
>>
>>
>>  I want to find out the 0.25 percentile for the value distribution. How
>> can I do it using the Hive percentile function?
>>
>>
>>
>

Reply via email to