Hey all, recently joined and wanted to share some success I am having with Myriad on my test cluster. Obviously some of the issues that have been talked about here and on the git issues I've run into, but all in all it's been a great experience. (I had help).
A few notes: My cluster is a Mesos based cluster running on top of a MapR filesystem (4.0.2). It's working pretty well for things like Spark and Docker, MRv1 is a hacked setup that I wouldn't recommend to anyone, but it was sorta working. I do multiple things with this cluster, but one is a crude packet capture process that really works well from an "edge case" point of view due to the use of a hive transform and other crazy stuff. 1. Hive is working great. No issues there, tweaked some mapreduce settings, added some profiles that fit my cluster and things seems to be humming along well. 2. The API was confusing until it was explained to me. Basically, coming from a marathon world, I saw the instances setting as the "number" of instances I want running, rather than go up or down by x instances. I see why this the API is setup like this, but perhaps some consideration to make it more intuitive? Like an option to specify what you want to running addition to the flex up and flex down. Also, on the flex down, is there an option to specify which instances you want to flex down? On flex up, I can setup 1 large, then run 2 medium, and then have 2 small running on the cluster, but on the way down, it appears it's only the number of instances I want flexed. 3. If I shut down the resource manager, (on purpose) there should be a way to have that auto kill nodemanagers. Right? As of now if I want to reset things, I need to scale down in marathon, then run a script on each node that kills processes. 4. The myriad-config-default.yml needs to be moved outside the bundled jar so we can update our clusters without rebuilding. I know this is alpha and it's probably on a list, but I figured I'd mention it. (Perhaps checking the location of executor, then class path etc). 5. I'd be happy to run though any tests or check any bugs people may want a confirmation on with my cluster. It's not "production" but it is doing work so I have some flexibility in changing things up. I wish I could do more on the coding side, but I am more of a hacker/scripter than a java dev, and would hate for any of my bad code to make it into a project like this with so much potential. All is in all, I am quite impressed. It seems more stable than my MRv1 on Mesos/MapR so that's nice. Still playing with settings and other things, and wanted to share some successes instead of just issues, thanks for all the hard work here. John Omernik
