ok found my own answer via:
https://www.ericlin.me/2016/03/hive-dynamic-insert-query-only-uses-1-reducer-out-of-thousands-of-reducers/
This setting gets rid of the last reduce phase in my insert:
set hive.optimize.sort.dynamic.partition=false;
Now I get as many files in my partition as I have
Hi all,
I got a query doing an insert overwrite like this:
WITH tbl1 AS (
SELECT
col0, col1, local_date, local_hour
FROM tbl1
WHERE
),
tbl2 AS (
SELECT col0, col1, local_date, local_hour
FROM tbl2
WHERE
)
INSERT OVERWRITE TABLE
tbl3
PARTITION (local_date, local_hour)
