I'm building a rest service with the HTTP Request and Response Processors to support data extracts from Hive. Since some of the extracts can be quiet large using the SelectHiveQL Processor isn't a performant option and instead I'm trying to use on demand Hive Temporary Tables to do the heavy lifting via CTAS(Create Table as Select). Since GetHDFS doesn't support an incoming connection I'm trying to figure out another way to fetch the files Hive creates and return them as a download in the web service. Has anyone else worked out a good solution for fetching the contents of a directory from HDFS as a part of larger flow?
Thanks Shawn
