Re: hive window function can only calculate the main table?

2017-10-09 Thread Gopal Vijayaraghavan
> ) t_result where formable = ’t1' … > This sql using 29+ hours in 11 computers cluster within 600G memory. > In my opinion, the time wasting in the `order by sampledate` and `calculate > the table B’s record`. Is there a setting to avoid `table B`’s record not to > get ‘avg_wfoy_b2’ column,

Re: does anyone care about list bucketing stored as directories?

2017-10-09 Thread Sergey Shelukhin
Ok, here’s synopsis that is hopefully clearer. 1) LB, when stored as directories, adds a lot of low-level complexity to Hive tables that has to be accounted for in many places in the code where the files are written or modified - from FSOP to ACID/replication/export. 2) While working on some

hive window function can only calculate the main table?

2017-10-09 Thread newbegin2020
Hi all, I am using hive(1.2.0) on spark(1.6.0). These days I met a problem that is window function waste too much time: I have table `A` has 3 columns 5 million lines and table `B` has 10 columns and 3 billion lines 嵐, my hql like: ``` select *, … ( select id, sampledate, max(b1) over wf7 as