Hmm ... any indications of when 0.6 will officially release? This will go to a large-scale production system so we'd like to have a stable official release if possible.
Thanks, -L On Mon, Aug 16, 2010 at 11:47 AM, Ning Zhang <[email protected]> wrote: > Dynamic partition is supported in hive 0.6+. 0.6 is not officially released > yet, but you can find the branch at > http://svn.apache.org/repos/asf/hadoop/hive/branches/branch-0.6. > > Ning > > On Aug 16, 2010, at 6:39 AM, Luke Crouch wrote: > > Do I need a certain version of Hive? I'm tried this: > > > set hive.exec.dynamic.partition=true; > > set hive.exec.dynamic.partition.mode=nostrict; > > And still get: > > 10/08/16 13:37:36 INFO parse.ParseDriver: Parsing command: FROM ( FROM ( > SELECT project, file, os, country, dt FROM downloads WHERE dt='2010-07-01' > CLUSTER BY project, file, dt ) a SELECT TRANSFORM(file, os, country, dt, > project) USING '/var/local/sfpy/dstat/dstat/transform/reduce.py ' AS > (project, file, downloads, os, country) ) b INSERT OVERWRITE TABLE > daily_file_totals PARTITION (dt='2010-07-01', project) SELECT file, > downloads, os, country, project > FAILED: Parse Error: line 1:351 mismatched input ')' expecting = in > destination specification > > (i.e., I changed project to the last field of the final SELECT) > > Thanks again, > -L > > On Sat, Aug 14, 2010 at 12:38 PM, Namit Jain <[email protected]> wrote: > >> Project needs to be the last column >> In the select list >> >> Also, you need to set a variable to >> Enable dynamic prtn inserts >> >> >> Sent from my iPhone >> >> On Aug 14, 2010, at 9:46 AM, "Luke Crouch" <[email protected]> wrote: >> >> > I'm trying to do a dynamic partition insert like so: >> > >> > FROM ( >> > FROM ( >> > SELECT project, file, os, country, dt >> > FROM test_downloads WHERE dt='2010-06-30' >> > CLUSTER BY project, file, dt >> > ) a >> > SELECT TRANSFORM(project, file, os, country, dt) >> > USING '/var/local/sfpy/dstat/dstat/transform/reduce.py -- >> > test' >> > AS (project, file, downloads, os, country) >> > ) b >> > INSERT OVERWRITE TABLE test_daily_file_totals PARTITION >> (dt='2010-06-30 >> > ', project) >> > SELECT project, file, downloads, os, country >> > >> > The test_daily_file_totals table is created like so: >> > >> > CREATE TABLE IF NOT EXISTS {{ table_name }} ( >> > file STRING, >> > downloads INT, >> > os STRING, >> > country STRING >> > ) >> > PARTITIONED BY (dt STRING, project STRING) >> > ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' >> > STORED AS TEXTFILE >> > >> > But I get an error with the dynamic partition syntax: >> > >> > FAILED: Parse Error: line 12:89 mismatched input ')' expecting = in >> > destination specification >> > >> > Can someone see something I'm doing wrong in the sql? I have set >> > both hive.merge.mapfiles and hive.merge.mapredfiles to false in hive >> > configuration. >> > >> > Thanks, >> > -L >> > > >
