Re: Basic Grouping Question

Hi Once you specify the aggregates on group By function (I am assuming you mean dataframe here?), grouping and aggregate both works in distributed fashion (you may want to look into how reduceByKey and/or aggregateBykey work). On Mon, Feb 20, 2017 at 10:23 PM, Marco Mans

Basic Grouping Question

Hi! I'm new to Spark and trying to write my first spark job on some data I have. The data is in this (parquet) format: Code,timestamp, value A, 2017-01-01, 123 A, 2017-01-02, 124 A, 2017-01-03, 126 B, 2017-01-01, 127 B, 2017-01-02, 126 B, 2017-01-03, 123 I want to write a little map-reduce