[
https://issues.apache.org/jira/browse/GIRAPH-312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Reisman updated GIRAPH-312:
-------------------------------
Attachment: GIRAPH-312-5.patch
This one is just right, should be good to go. You run it with:
bin/giraph-admin -Dgiraph.zk.whatever=... [-Dgiraph....] -cleanZk
where your -D options are the normal giraph ZK options (CSV hostname list of ZK
quorum servers, ZK port, ZK_BASE_NODE, etc.) or else it has defaults. If you
don't give it a server list, it quits since this is really meant for ZK quorums
that stay up indefinitely on your typical cluster. It pulls this from a
Configuration it makes and recursively deletes from the BspService.BASE_DIR
(which is "_hadoopBsp") under whatever base node you gave it (if any.)
It reports progress as it goes. This patch also splits off all the shell-level
config stuff from bin/giraph into its own bin/giraph-env script which is then
sourced in bin/giraph and bin/giraph-admin.
Passes mvn verify and field tests on ZK quorum. The GiraphZooKeeperAdmin class
should also provide an easy template for how to add functionality to the
giraph-admin script in the future.
> Giraph needs an admin script
> ----------------------------
>
> Key: GIRAPH-312
> URL: https://issues.apache.org/jira/browse/GIRAPH-312
> Project: Giraph
> Issue Type: New Feature
> Components: conf and scripts, zookeeper
> Affects Versions: 0.2.0
> Reporter: Eli Reisman
> Assignee: Eli Reisman
> Priority: Minor
> Fix For: 0.2.0
>
> Attachments: GIRAPH-312-1.patch, GIRAPH-312-2.patch,
> GIRAPH-312-3.patch, GIRAPH-312-3.patch, GIRAPH-312-4.patch, GIRAPH-312-5.patch
>
>
> Our zookeeper instances have very long uptimes on our cluster, and failed job
> trees are never cleaned from memory. There is a separate shell script to do
> this, but its not picky about which node trees it erases, and on some systems
> some Giraph users may not have access to it.
> This patch will add a shell script to activate a new class which will use
> Giraph conf file options or our normal -Dgiraph.XYZ command-line opts to get
> the ZK quorum info, and clean out the remnants from its memory of old failed
> and killed jobs. They do pile up over time.
> This led to the larger idea that Giraph needs a general giraph-admin shell
> script as a home for stuff like this. Jakob suggested it would be a good idea
> to put this into such a script since then admin groups can be created so that
> not every Giraph client can run it. This script currently only has code to
> start up the zk cleaner, but can have more options added to it as JIRA's to
> add features crop up.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
