[jira] [Commented] (HIVE-15688) LlapServiceDriver - an option to get rid of run.sh
[ https://issues.apache.org/jira/browse/HIVE-15688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15850372#comment-15850372 ] Sergey Shelukhin commented on HIVE-15688: - run.sh is still generated, and could be used to restart the cluster. I can change the default behavior though. Having an extra JVM still has a lot of overhead given how big Hive classpath, configs, etc. is, and most of the time people want to start the cluster immediately after generating the package. > LlapServiceDriver - an option to get rid of run.sh > -- > > Key: HIVE-15688 > URL: https://issues.apache.org/jira/browse/HIVE-15688 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15688.patch > > > run.sh is very slow because it's 4 calls to slider, which means 4 JVMs, 4 > connections to RM and other crap, for 2-5sec. of overhead per call, > depending on the machine/cluster. > What we need is a mode for llapservicedriver that would not generate run.sh, > but would rather run the cluster immediately by calling the corresponding 4 > slider APIs. Should probably be the default, too. For compat with scripts we > might generate blank run.sh for now. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15688) LlapServiceDriver - an option to get rid of run.sh
[ https://issues.apache.org/jira/browse/HIVE-15688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849355#comment-15849355 ] Siddharth Seth commented on HIVE-15688: --- I don't think start should be the default. That breaks existing behaviour which a lot of people are likely used to. Thought we'd be doing this by continuing to generate a run.sh - which invokes a single java binary (instead of the 4 slider commands), accepting parameters for whether a destroy is required etc. That still saves most of the time, and retains compatibility. Having a package sit around which can be invoked multiple times is really useful while debugging. Also serves as a history for what was started (think this may still be the case - except without the run.sh). Most of the code would remain unchanged - except for the run.sh generation which points to the slider command or the new Java launch. > LlapServiceDriver - an option to get rid of run.sh > -- > > Key: HIVE-15688 > URL: https://issues.apache.org/jira/browse/HIVE-15688 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-15688.patch > > > run.sh is very slow because it's 4 calls to slider, which means 4 JVMs, 4 > connections to RM and other crap, for 2-5sec. of overhead per call, > depending on the machine/cluster. > What we need is a mode for llapservicedriver that would not generate run.sh, > but would rather run the cluster immediately by calling the corresponding 4 > slider APIs. Should probably be the default, too. For compat with scripts we > might generate blank run.sh for now. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-15688) LlapServiceDriver - an option to get rid of run.sh
[ https://issues.apache.org/jira/browse/HIVE-15688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15832779#comment-15832779 ] Sergey Shelukhin commented on HIVE-15688: - [~gopalv] [~hagleitn] fyi > LlapServiceDriver - an option to get rid of run.sh > -- > > Key: HIVE-15688 > URL: https://issues.apache.org/jira/browse/HIVE-15688 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin > > run.sh is very slow because it's 4 calls to slider, which means 4 JVMs, 4 > connections to RM and other crap, for 2-5sec. of overhead per call, > depending on the machine/cluster. > What we need is a mode for llapservicedriver that would not generate run.sh, > but would rather run the cluster immediately by calling the corresponding 4 > slider APIs. Should probably be the default, too. For compat with scripts we > might generate blank run.sh for now. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
