I am pleased to announce the release of MR3 0.11. The new feature in MR3 0.11 is the support for autoscaling. Autoscaling works on both Hadoop and Kubernetes.
I have added a quick start guide on running Hive on MR3 on Amazon EKS with autoscaling. Autoscaling on Amazon EKS (which uses Kubernetes Autoscaler) is much faster than on Amazon EMR. For example, on EKS, a scale-out operation takes about 2 minutes to attach a new node, and a scale-in operation takes slightly longer than 1 minute to retire an existing node. In contrast, on EMR, the user should wait at least 5 minutes to trigger scale-in/out. So if you would like to try autoscaling with Hive on MR3, we suggest EKS instead of EMR. https://mr3.postech.ac.kr/quickstart/aws/run-eks-autoscaling/ You can download MR3 0.11 at: https://mr3.postech.ac.kr/download/home/ Cheers, --- Sungwoo Park