Karen Tung wrote: > Jean McCormack wrote: >> Dave Miner wrote: >>> Jean McCormack wrote: >>>> Great. Here's what I'm working doing. >>>> >>>> The user specifies the "boot_area" in the manifest. Let's say it's >>>> rpool/dc-builds. >>>> >>>> We create pkg_image and bootroot as zfs dataset/mountpoints >>>> underneath it. >>>> Also we create logs, media, and tmp as just plain old directories >>>> under it. They >>>> don't need to be datasets and it could cause loss of info they want >>>> like an old logfile >>>> if they rollback. So I'm not doing that. >>>> >>> >>> That would seemingly only be a concern if snapshots were taken of >>> those datasets, and would still be a concern with this design if you >>> actually snapshot the top-level dataset (which I'm not clear on), so >>> I guess I don't see how this is a reason not to make them datasets. >>> By making them datasets, you would provide for independent rollback >>> of each of them if desired, as well as all the other goodness that >>> separate datasets allow, which is not possible in the scheme you're >>> suggesting. >>> >>> Dave >> True. It's not a major change and easily made. And I'm not >> snapshotting the top level. >> >> Jean >> > Following the discussion so far, it sounds like Dave suggests that we > make each of the directory datasets, and snapshot each of > them individually. Jean suggests that we only make pkg_img and > bootroot datasets. > > I think both approach have problems in the sense that things might get > out of sync, and it might also be wasteful of space. > > From the DC point of view, there's no way to determine which of the > scripts registered in the finalizer are for constructing the > bootroot. So, if we want to snapshot the pkg_image and bootroot > datasets separately, how do we know when to > start snapshot the bootroot dataset? My original message mentioned that I believe we need to add this to the manifest. This way users can also have flexibility in what they want snapshotted if they add their own script. And that area could conceivable not be pkg_image or bootroot. > To make it work, do we need to snapshot the bootroot dataset at step > 0, even though > nothing will be in it for a while? No. There's nothing there so why snapshot it? > When we snapshot it separately, we also need to make sure that the > snapshots are in sync. > I know the risk of them getting out of sync should be minimal as the > DC app will be doing the checkpointing, but since they > are separate, there's still a chance that things might get out of > sync. What about the case that somebody rollback one of the > datasets to look at something? I don't understand your rollback concern. If the script depends upon another area it should snapshot that area. Then the build rolls it back (doesn't matter if someone else already has) and you get what you want. > > In my opinion, there's 2 types of data during the process of > constructing the image. There's the data, and metadata that's used > during the build process, and all of them have to be kept consistent > at all times. These include the files > from the pkg_img, bootroot, and tmp directory. > There's the output data, these actually are best kept in places that > don't get snapshot so we can preserve information from one run of the > DC to the next. These include the log files and > the output iso/usb images. We could timestamp or do some other things > these files to make them unique and not overwrite each other > from one run to the next. Yes. But if DC doesn't really use the snapshotting or rollback capabilities for these the info would be safe. When I thought about Dave's proposal I actually could see a good use for getting these under zfs. For people like me who get tired in the evening and blow away entire directories they need. A simple rollback and you have your old log files and outputs. So we would be creating these datasets as a service to the user who may or may not decide to use the zfs capabilities that come with them.
> > The achieve the above, I suggest to put pkg_img, bootroot and the tmp > directory into one dataset, and snapshot it as we checkpoint. How do you envision this looking? In order to do this you'd need pkg_image, bootroot and tmp to be under yet another subdirectory. Seems like it could be confusing to the user which is what we're trying to get away from. Jean > > Thanks, > > --Karen > > >
