I have a PutHDFS processor drop a file, I then have a long chain of ReplaceText -> PutHiveQL processors that runs a series of steps. The below ~4 steps allow me to take the file generated by NiFi in one format and move it into the final table, which is ORC with several Timestamp columns (thus why I'm not using AvroToORC, since I'd lose my Timestamps.
The exact HQL, all in one block, is roughly: DROP TABLE `db.tbl_${filename}`; CREATE TABLE ` db.tbl _${filename}`( Some list of columns goes here that exactly matches the schema of `prod_db.tbl` ) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\001' STORED AS TEXTFILE; LOAD DATA INPATH '${absolute.hdfs.path}/${filename}' INTO TABLE ` db.tbl _${filename}`; INSERT INTO `prod_db.tbl` SELECT * FROM ` db.tbl _${filename}`; DROP TABLE ` db.tbl _${filename}`; Right now I'm having to split this into 5 separate ReplaceText steps, each one followed by a PutHiveQL. Is there a way I can push a multi-statement, order dependent, script like this to Hive in a simpler way? Thanks, Peter