I don't want to groupBy since i want the rows separate for the subsequent
transformations. But i want to groupBy (i am using partitionBy here) using
many attributes while counting the frequency for each different group of
records (with respect to the the attributes first mentioned)
Le mar. 19 avr. 2022 à 14:06, Sean Owen <sro...@gmail.com> a écrit :
> Just .groupBy(...).count() ?
>
> On Tue, Apr 19, 2022 at 6:24 AM marc nicole <mk1853...@gmail.com> wrote:
>
>> Hello guys,
>>
>> I want to group by certain column attributes (e.g.,List<Column >
>> groupByQidAttributes) a dataset (initDataset) and then count the
>> occurrences of associated grouped rows, how do i achieve that neatly?
>> I tried through the following code:
>> Dataset<Row> groupedRowsDF = initDataset.withColumn("qidsFreqs", count(
>> "*").over(Window.partitionBy(groupByQidAttributes.toArray(new Column[
>> groupByQidAttributes.size()])))); Is that OK to use for the purpose?
>>
>>
