Hi diskimage-builder has been working on a feature/v2 branch incorporating some largely internal changes to the way it finds and calls elements, enhancements to partitioning and removal of some long-deprecated elements. We have just tagged 2.0.0rc1 and are requesting testing by interested parties.
You can view the branch at https://git.openstack.org/cgit/openstack/diskimage-builder/log/?h=feature/v2 For any issues it is probably best to file a bug. Developers are standing by in #openstack-dib. === User facing changes If you use dib exclusively via the command-line disk-image-create installed from a package or via pypi you are unlikely to notice any difference (if you run it directly from a git-tree checkout, you're in development territory so read on). v2 includes a new framework for partitioning contributed by Andreas Florath. This should allow for creating multiple partitions, images with encryption, LVM support and flexibility for multiple-devices, all of which are currently not supported. Please check the v2 documentation, specs and reach out if these features interest you (some parts still in review). Element override is now supported. If you have an element of the same name earlier in the ELEMENTS_PATH, it will override later instances (previously, the behaviour was undefined). A number of long-deprecated elements have been removed in v2, which are to the best of our knowledge unused. - partitioning-sfdisk - deploy-ironic-element - ironc-discovered-ramdisk - serial-console-element - map-services === Developer changes Developers, or those that are doing more "interesting" things in deployment or CI, might like to read below to follow the history of the branch and hopefully do some testing with the feature/v2 branch ASAP so we can address any concerns. dib started out mostly as a shell script collection that used setup.py to install itself. Over time it has made more and more use of Python. The current instructions say to checkout the source tree and run "./bin/disk-image-create"; this causes a number of issues for the internal Python components. We carry hacks to try and figure out of we're being called uninstalled from a source tree, or installed in pip develop mode, or installed on the system. For purposes of both users and development we want dib to be as "pythonic" as possible and behave like all other projects. Two major visible changes are: - command-line scripts are entry points (i.e. need to be installed) - elements have moved under diskimage_create module The result of the first is that "./bin/disk-image-create" from the source tree is no longer there. Like all other projects, you should install dib into a virtualenv (if you're developing, use pip -e) and "disk-image-create" will "just work". FYI, on the back-end, this entry-point essentially exec()s the old shell-script driver; but we reserve the right to move more of the logic into python over time. Also, instead of having to os.system(/path/disk-image-create), we can allow direct calls like a real library. A side-effect of this is that we have removed and deprecated the dib-utils package. This was intended to be a more generic repository of tools that might be useful outside dib, but that did not eventuate and it has been folded back into dib for simplicity. The second change, moving the inbuilt elements under the diskimage_create module, is a simplification so we always have a canonical path to our elements. Currently the elements are installed via "data_files" in setup.cfg, which means setup.py puts the elements in /usr/share... when installing, or leaves them in the tree when installed via pip -e. Again we have path guessing hacks trying to guess what situation we're in. Elements have moved from elements/ to diskimage_builder/elements in the source tree (we did try to minimise this with symlinks, but setuptools doesn't like it, it complicates things when git merging across the branches and is legacy cruft to go wrong in the future. It was better to make a clean break). Since we always know where elements are relative to the imported diskimage_builder module we can drop all the path guessing complexity. This has other good flow-on effects such as testr being able to find unit-tests for elements in the normal fashion and having imports work as usual. We are aware there are a number of tools that like to take dib elements and do things with them. This is fine, I guess, although of course if we don't know about how you use elements we're liable to break them. Thus your use may be confused that the elements have moved. Reading some of the dib source you may find there is a canonical way to find out the included dib elements path -- ask dib itself, something like: DIB_ELEMENTS=$(python -c ' import diskimage_builder.paths; diskimage_builder.paths.show_path("elements")') Note you probably do not want this. As mentioned, another feature of v2 is override elements -- an element that appears first in the element path-list will override any built-in one (just like $PATH). There is a function, diskimage_builder.get_elements(), which will correctly process the element path, calculate overrides and return a canonical list of elements, their dependencies and correct paths. THAT said, you probably do not want this either! There are a number of elements that do things on behalf of other elements -- they look for a file in the included elements, say, and use that as a manifest for something. Previously, these would just have to make up their own element processing via inspection of the command-line arguments. dib now exports pre-computed variables that an element can walk for all the current build elements -- a YAML list for easy python decoding and a function that builds an array for Bash elements. === Merge plans Barring any new developments, I would think that 1.28.0 would be the last release before the v2 branch is merged back to master. I imagine this would be in about one to two weeks. The future is obviously dependent on what contributions appear. It seems the main area of interest currently is in creation of containers; if this interests you keep an eye on the review queue. Thanks, -i __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
