Hi Everyone! I wanted to announce a project I have been working on for a while now. It is still in dev status, but has made enough progress towards being production ready to reach out to enthusiastic developers.
*Plow* - A render farm management software specifically designed for VFX workloads. While there are a few open source projects out there with a similar theme, they are either not very good, or not specifically focused on VFX workloads. Members of the Plow project have previously written 1+ proprietary Render Queues still in active production by one of the largest VFX studios in the world. The software consists of: 1. Plow server, written in Java, using a PostgreSQL database 2. rndaemon - Render Node Daemon, a process manager that runs on each node which should be made available to run tasks. Reports stats of both the system health, and all running tasks (written in python) 3. python client - Plow uses thrift, so generating client bindings for any language is quite easy. We use python as the official client API. One could choose to write client code for Javascript, C++, Objective-C, etc. 4. plow-wrangler - A PySide client application for managing and monitoring the queue (still work in progress) We have separated our job description library into a project called Blueprint. Via Blueprint, you can define job "template" modules, which make submitting to Plow really easy, via blueprint files. Currently we have Plow running jobs such as Blender, Maya, arbitrary commands, and command ranges. With Blueprint, its a minimal amount of code to create a new module to support custom required/optional args, and how to ultimately end up with a Job that can be submitted. Some of the features: - Project / Folder structure - Supports multiple "projects", each with various "folders" to filter jobs into, with individual resource and priority control. - Job/Layer/Task inter-dependencies (i.e. have a task wait on a layer, or a job wait on a job) - Resource and Historical stats-based scheduling - Server-smarts in how it decides which task to dispatch to which node based on a number of strategies: Available resources, past runs of previous version of the same task, service and tag based, ... i.e. fit as many tasks onto as many cpus as efficiently and comfortably as possible. - Clusters - organize render nodes into logical clusters, and associate resource quotas between projects and clusters. - Service-based layers - A Job consists of 1 or more layers which consists of 1 or more tasks. Layers of a job can be associated with service definitions, such as Maya for example. A service can dictate how few or many resources its tasks can get, and what kind of hardware then end up running on. I'm mentioning the project here because I know there are a lot of smart developers in this group. If you are interested in getting involved with a seriously awesome system, there is plenty of cool stuff to do ranging from server => client => gui => integration with different applications (Maya, Nuke, Houdini, ...). To develop or use, make sure to build from source. Features and fixes are happening so rapidly that we are not really at the stage to be providing regularly update binary released. But it is the plan to do that eventually. Links: - Plow Repo - https://github.com/sqlboy/plow - Blueprint Repo - https://github.com/sqlboy/blueprint - Pretty empty placeholder site - http://www.plowrender.com - Plow User Group - http://groups.google.com/group/plow-render - Dev Tips Wiki - http://code.google.com/p/plow/wiki/dev_tips If you want to get involved, you can fork the repos on github and submit pull requests. Definitely hit me up with any "getting started" issues. Hope this intrigues some of you out there! -- justin -- You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
