Looks pretty cool!
On Fri, Jun 7, 2013 at 2:25 PM, Justin Israel <[email protected]>wrote: > After realizing that I wrote this following line in haste, I figured I > would rephrase it: > *"While there are a few open source projects out there with a similar > theme, they are either not very good, or not specifically focused on VFX > workloads."* > * > * > Plow aims to satisfy as much of these areas as possible (but not limited > to): > > - Scalability for very large farms > - Scheduling, dispatching, and constructs that make sense for VFX (to > handle wildly varying resource, run time, dependency, and priority > requirements for tasks) > - Flexibility of integration to existing pipelines (environment > management, statistics reporting, submission / job description, tracking / > asset management) > - Decent client tools > - Decent configuration options > > There... I feel better about that replacement description :-) > > > > On Fri, Jun 7, 2013 at 2:20 PM, Justin Israel <[email protected]>wrote: > >> Hi Everyone! >> >> I wanted to announce a project I have been working on for a while now. It >> is still in dev status, but has made enough progress towards being >> production ready to reach out to enthusiastic developers. >> >> *Plow* - A render farm management software specifically designed for VFX >> workloads. >> >> While there are a few open source projects out there with a similar >> theme, they are either not very good, or not specifically focused on VFX >> workloads. >> >> Members of the Plow project have previously written 1+ proprietary Render >> Queues still in active production by one of the largest VFX studios in the >> world. >> >> The software consists of: >> >> 1. Plow server, written in Java, using a PostgreSQL database >> 2. rndaemon - Render Node Daemon, a process manager that runs on each >> node which should be made available to run tasks. Reports stats of both >> the >> system health, and all running tasks (written in python) >> 3. python client - Plow uses thrift, so generating client bindings >> for any language is quite easy. We use python as the official client API. >> One could choose to write client code for Javascript, C++, Objective-C, >> etc. >> 4. plow-wrangler - A PySide client application for managing and >> monitoring the queue (still work in progress) >> >> We have separated our job description library into a project called >> Blueprint. Via Blueprint, you can define job "template" modules, which make >> submitting to Plow really easy, via blueprint files. >> Currently we have Plow running jobs such as Blender, Maya, arbitrary >> commands, and command ranges. With Blueprint, its a minimal amount of code >> to create a new module to support custom required/optional args, and how to >> ultimately end up with a Job that can be submitted. >> >> Some of the features: >> >> - Project / Folder structure - Supports multiple "projects", each >> with various "folders" to filter jobs into, with individual resource and >> priority control. >> - Job/Layer/Task inter-dependencies (i.e. have a task wait on a >> layer, or a job wait on a job) >> - Resource and Historical stats-based scheduling - Server-smarts in >> how it decides which task to dispatch to which node based on a number of >> strategies: Available resources, past runs of previous version of the same >> task, service and tag based, ... i.e. fit as many tasks onto as many cpus >> as efficiently and comfortably as possible. >> - Clusters - organize render nodes into logical clusters, and >> associate resource quotas between projects and clusters. >> - Service-based layers - A Job consists of 1 or more layers which >> consists of 1 or more tasks. Layers of a job can be associated with >> service >> definitions, such as Maya for example. A service can dictate how few or >> many resources its tasks can get, and what kind of hardware then end up >> running on. >> >> I'm mentioning the project here because I know there are a lot of smart >> developers in this group. If you are interested in getting involved with a >> seriously awesome system, there is plenty of cool stuff to do ranging from >> server => client => gui => integration with different applications (Maya, >> Nuke, Houdini, ...). >> >> To develop or use, make sure to build from source. Features and fixes are >> happening so rapidly that we are not really at the stage to be providing >> regularly update binary released. But it is the plan to do that eventually. >> >> Links: >> >> - Plow Repo - https://github.com/sqlboy/plow >> - Blueprint Repo - https://github.com/sqlboy/blueprint >> - Pretty empty placeholder site - http://www.plowrender.com >> - Plow User Group - http://groups.google.com/group/plow-render >> - Dev Tips Wiki - http://code.google.com/p/plow/wiki/dev_tips >> >> If you want to get involved, you can fork the repos on github and submit >> pull requests. Definitely hit me up with any "getting started" issues. Hope >> this intrigues some of you out there! >> >> -- justin >> >> > -- > You received this message because you are subscribed to the Google Groups > "Python Programming for Autodesk Maya" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
