Re: Reading and Writing with Hive 0.13 from a Yarn application

2014-09-03 Thread Ashutosh Chauhan
Hi Nathan, This was done in https://issues.apache.org/jira/browse/HIVE-6248 Reasoning was to minimize api surface area to users so that they are immune of incompatible changes in internal classes and thus making it easier for them to consume this and not get worried about version upgrade. Seems

RE: Reading and Writing with Hive 0.13 from a Yarn application

2014-09-03 Thread Nathan Bamford
?Hi Ashutosh, Thanks for the reply! Well, we are a yarn app that is essentially doing the same things mapreduce does. For regular files in Hadoop, we get the block locations and sizes and perform some internal sorting and load balancing on the master which then creates the slave yarn apps

Re: Reading and Writing with Hive 0.13 from a Yarn application

2014-09-03 Thread Ashutosh Chauhan
This api is designed for use cases like yours only. So, I will say api is failing if it cannot service what you are trying to do with it. So, I will encourage you to use this api and consider current shortcoming as missing feature in it. Feel free to file a jira requesting addition of these