ACID transaction over ORC

2023-03-22 Thread Nirav Patel
Hi, Are there any companies using hive ACID capabilities ? I am trying to understand how stable/reliable is ACID transaction on ORC in terms of ingesting real time data over 10 mins window and writing those data potentially on same ORC partition (merge into). Nirav

Re: Specifying YARN Node (Label) for LLAP AM

2023-03-22 Thread Aaron Grubb
Hi Sungwoo, Thanks for your reply but I was referring exclusively to the LLAP application master which is not Tez-related. Thanks, Aaron On Wed, 2023-03-22 at 20:02 +0900, Sungwoo Park wrote: Hello, A similar issue was discussed in the Tez mailing list a long time ago:

Re: Specifying YARN Node (Label) for LLAP AM

2023-03-22 Thread Sungwoo Park
Hello, A similar issue was discussed in the Tez mailing list a long time ago: https://lists.apache.org/thread/0vjor12lpcncg43rn6vddw8yc1k62c81 Tez still does not support specifying node labels for AMs, but as explained in the response, this is quite easy to implement if you can re-compile Tez.

Specifying YARN Node (Label) for LLAP AM

2023-03-22 Thread Aaron Grubb
Hi all, I have a Hadoop cluster (3.3.4) with 6 nodes of equal resource size that run HDFS and YARN and 1 node with lower resources which only runs YARN that I use for Hive AMs, the LLAP AM, Spark AMs and Hive file merge containers. The HDFS nodes are set up such that the queue for LLAP on the