hi folks I've been looking into a candidate solution for supporting the ordering of property values in the SMF repository. The subject of the ordering of property values has come up here a few times before, for example see:
http://www.opensolaris.org/jive/thread.jspa?messageID=53690톺 A key advantage of preserving ordering would be that it would no longer be necessary to store order-sensitive multiply-valued values in a single astring property using a separator to delimit the individual values. The issue is that at present, SMF properties with multiple values are not stored in the repository in a manner which allows the values to be retrieved in the same order in which they were added. To support such ordering, a few things need to be done. Firstly, libscf, more specifically, scf_entry_add_value() - the function which collates values for commitment to the repository - needs to collect values in the order in which they were added. At present it adds values at the head of the list for commitment, so that we commit values in reverse order. This part is easily fixed by adding new values at the tail of the list. From there, the repository protocol needs to do the right thing with respect to ordering, which I believe it does. Finally, at the backend, in the sqlite .db files which comprise the repository, we need to store values in such a way that they can be retrieved in order. At present, property values are stored as multiple rows in a table (value_tbl), and are associated with a particular property via an identifier they share (value_id). When these are retrieved, we carry out a SELECT operation which selects all values with the required value_id, specifying no particular order (though it seems like we get the right order due to sqlite implementation details which we shouldn't rely on). To support ordering at the repository level, I suggest adding an additional column to the value_tbl, named value_order. When we add property values, we would need to also specify ordering (0, 1, 2, ..) for this column. On retrieval then, we can modify the backend SELECT statement to ORDER BY the value_order column and thus ordering is preserved. A major concern with such a change of course is handling upgrade to a new repository format, while also supporting downgrade from the new format to the previous format. To ensure svc.configd can deal with the repository format, the repository carries a schema version number which must match the version encoded in svc.configd in order for the repository to be usable. My suggestion is that we do not bump this version number for this change. A downgraded svc.configd can actually handle a repository with the extra value_order column perfectly (it merely ignores it). It might make sense introduce a minor version number into the schema_version table, to reflect that the schema has changed, but not in an incompatible manner. This leaves the issue of upgrading to the new repository format. Ideally, we would like to add the new column as soon as the repository is writable on upgrade, but we have to deal with a few complications: - during boot, we need to read property values prior to the point when we can do the repository upgrade (i.e. prior to the repository being writable) - the version of sqlite used does not support the ADD COLUMN command, so adding a new column to the value_tbl is not straightforward. To address the first issue, we need to keep track of whether upgrade has been carried out for a particular backend. This is important, because during first boot post-upgrade, we will need to read values before we can write to (and upgrade) the repository. This means we need to modify the SELECT statement that retrieves values based on whether the upgrade has been done (since prior to upgrade it cannot use the ORDER BY value_order clause in the SELECT statement that retrieves values). To address the second issue, we just need to do a more complex dance of creating a temporary table which includes the new column, populate it with the value_tbl values, delete the original value_tbl, create the new value_tbl (with the additional column), populate it with the values from the temporary value table, and finally delete the temporary value table. Testing indicates this just takes a few seconds. I've posted a webrev which contains what I have so far: http://cr.opensolaris.org/~amaguire/smf_values/ I've confirmed this approach works, and supports upgrade/downgrade, but I think it needs some additional investigation to be determined sufficiently robust (I still see one complaint about the lack of a value_order column during first boot post-bfu). I'm assuming we'd also need to assess the impact on performance (if any) that adding an additional column to the value table would have, but I thought it'd be useful to have a rough candidate solution to start discussing how/if the ordering issue can be resolved. Comments are appreciated! Alan
