Here's my current lens for managing the /etc/conf.d/net file for gentoo; it uses the @syntax to map between baselayout 1 and 2.
The big thing is the difference in handling quoting; the old lens would allow you to switch between quoting styles within a single variable (since it was a real array) but now it's forced to one quoting style per-array. I can think of situations where this might bite you (including literal $'s) but I can't think of a good way to handle it. -Doug On 04/25/2012 06:09 PM, Raphaël Pinson wrote: > On Wed, Apr 25, 2012 at 10:09 PM, Doug Warner <d...@warner.fm> wrote: >> Sorry for the long delay here; starting to come back around to this problem >> :) >> >> I really like your idea for supporting multiple syntaxes here and marking >> them >> with this type of "version" string (you could label each syntax for each >> matching version to make it easier to understand what is going on). > > > You need to leave one syntax without a "@syntax" node (probably the > most recent syntax). The reason for this is that otherwise, this > parameter would become necessary for the tree to be valid, and users > would have to specify which syntax they want to use. It's much easier > if users don't care about it and just get the newest syntax by > default, without having to even know that "@syntax" nodes exist. > > >> So, related to this; would it be possible to include an external lens to >> handle the parsing for the "old version" and write the new version inside the >> current file? I was thinking of just continuing to use the ShellVars lens to >> parse the line for the "old version" and tag it the appropriate syntax >> version >> and then write the new one as the default. > > > Sure, that's possible, but there are conditions. The example I gave > was quite simple (even simplistic). You have to avoid ambiguity in the > Augeas bidirectional transformations. That means: > > * No ambiguity on keys (same key mapping different syntaxes with > different lenses); > * No ambiguity on syntax (overlapping regexes for several syntaxes). > > > In the example I gave, I handled these two cases by: > > * Using a single label (in that case "key") declaration to avoid the > ambiguity on the keys, and handling the syntax cases inside this > single lens (declaring 3 lenses and using a union of them wouldn't > work, it would raise an ambiguity since the 3 keys would match the > same regex); > * Making sure syntaxes are non-overlapping (in this case, using > colons, commas or equal sign as separators ensure they cannot be > overlapping). > > The lens declaration could very well use more complex lenses, and even > lenses from other modules (such as Shellvars.lns), so long as it > follows these two rules. > > If you have a specific case, that could be interesting. > > Raphaël > > > >> On 02/11/2012 05:07 PM, Raphaël Pinson wrote: >>> After thinking about it some more, I think it could be useful to get >>> to a standard using subnodes. >>> >>> Here is an example, unrelated to the suggested problem: >>> >>> >>> module Migration = >>> >>> (* There are 3 possible syntaxes for a line >>> The last one doesn't have a "@syntax" subnode, it's the default - >>> newest - syntax *) >>> let line = [ key Rx.word . >>> ( (Sep.colon . store Rx.word . [ label "@syntax" . value "1" ]) >>> | (Sep.space . store Rx.word . [ label "@syntax" . value "2" ]) >>> | (Sep.equal . store Rx.word) ) >>> . Util.eol ] >>> >>> let lns = line* >>> >>> (* Now for some tests *) >>> let full = "foo:bar\nfoo=baz\nfoo baa\n" >>> >>> (* Get all kind of syntaxes *) >>> test lns get full = >>> { "foo" = "bar" >>> { "@syntax" = "1" } >>> } >>> { "foo" = "baz" } >>> { "foo" = "baa" >>> { "@syntax" = "2" } >>> } >>> >>> (* Migrate all to new syntax *) >>> test lns put full after >>> rm "//@syntax" = >>> "foo=bar\nfoo=baz\nfoo=baa\n" >>> >>> (* Force syntax 1 for last entry *) >>> test lns put full after >>> set "/foo[3]/@syntax" "1" = >>> "foo:bar\nfoo=baz\nfoo:baa\n" >>> >>> >>> >>> Could that be an interesting approach in general? >>> >>> Raphaël >>> >>> >>> >>> 2012/1/26 Raphaël Pinson <raph...@gmail.com>: >>>> Hi Doug, >>>> >>>> On Thu, Jan 26, 2012 at 4:07 PM, Doug Warner <d...@warner.fm> wrote: >>>>> I'm working on a lens to work with Gentoo's new /etc/conf.d/net format >>>>> and I >>>>> realized I need to convert their old format to the new format. >>>>> >>>>> What's the best way to migrate from one format to another? I have my lens >>>>> mostly working for the new format now, but I realized I might need to >>>>> parse >>>>> both versions; but then I thought that it might not update when augeas >>>>> reads/writes the config file. >>>>> >>>>> I am using puppet to manage these files; so the values are typically set >>>>> each >>>>> run; I'm hoping that would cause augeas to write down the new format >>>>> rather >>>>> than leave it the old version. >>>>> >>>>> I guess the last option is to wipe out the old file when I upgrade and let >>>>> puppet/augeas write down the new file, but that seems painful as well. >>>>> >>>>> Any recommendations on what to do here? >>>>> >>>> >>>> >>>> That's a very interesting question. I see two ways of doing this: >>>> * write a single lens which will map both syntaxes to different nodes >>>> in the tree, and use the "move" command to migrate; >>>> * or write two lenses bound to two different files (file.old and >>>> file.new or something similar) and use the "move" command to migrate. >>>> >>>> >>>> Here is a (minimalistic) proof of concept using the first option: >>>> >>>> $ cat myconfig.aug >>>> module MyConfig = >>>> >>>> autoload xfm >>>> >>>> let old_record = >>>> let entry = [ Util.del_str "\"" >>>> . label "entry" >>>> . store /[0-9.\/]+/ >>>> . Util.del_str "\"" ] >>>> in [ label "old_record" . store Rx.word >>>> . Util.del_str "=(" >>>> . Build.opt_list entry Sep.space >>>> . Util.del_str ")" . Util.eol ] >>>> >>>> let new_record = >>>> let entry = [ label "entry" >>>> . store /[0-9.\/]+/ ] >>>> in [ label "new_record" . store Rx.word >>>> . Util.del_str "=\"" >>>> . Build.opt_list entry Util.eol >>>> . Util.del_str "\"" . Util.eol ] >>>> >>>> >>>> let lns = old_record | new_record >>>> >>>> let filter = incl "/file" >>>> >>>> let xfm = transform lns filter >>>> >>>> $ cat fakeroot/file >>>> config_eth0=("192.168.0.1/24" "192.168.0.2/24") >>>> >>>> $ augtool -I . -r fakeroot >>>> /files >>>> /files/file >>>> /files/file/old_record = "config_eth0" >>>> /files/file/old_record/entry[1] = "192.168.0.1/24" >>>> /files/file/old_record/entry[2] = "192.168.0.2/24" >>>> rpinson@rpinson:~/bas/augeas_migration$ augtool -I . -r fakeroot/ >>>> augtool> print /files/ >>>> /files >>>> /files/file >>>> /files/file/old_record = "config_eth0" >>>> /files/file/old_record/entry[1] = "192.168.0.1/24" >>>> /files/file/old_record/entry[2] = "192.168.0.2/24" >>>> augtool> mv /files/file/old_record /files/file/new_record >>>> augtool> print /files/ >>>> /files >>>> /files/file >>>> /files/file/new_record = "config_eth0" >>>> /files/file/new_record/entry[1] = "192.168.0.1/24" >>>> /files/file/new_record/entry[2] = "192.168.0.2/24" >>>> augtool> save >>>> Saved 1 file(s) >>>> >>>> $ cat fakeroot/file >>>> config_eth0="192.168.0.1/24 >>>> 192.168.0.2/24" >>>> >>>> >>>> Note that you can combine this with the -b flag to keep the old version at >>>> hand. >>>> >>>> >>>> Raphaël >> >>
(* Lens to match the crazy newline-split variables that Gentoo *) (* uses in it's openrc-based /etc/conf.d/net file *) module GentooConf_net= autoload xfm let nl = Util.del_str "\n" let eol = Util.eol let key_re = /[A-Za-z0-9_]+/ let eq = Util.del_str "=" let comment = Util.comment let empty = Util.empty let indent = Util.indent let char = /[^;#() '"\t\n]|\\\\"/ let dquot = /"([^"\\\n]|\\\\.)*"/ (* " Emacs, relax *) let sqword = /[^('\n]+/ let dqword = /[^("\n]+/ let uqword = /[^("'\n]+/ let semicol_eol = del /[ \t]*[;\n]/ "\n" let comment_eol = Util.comment_eol let comment_or_eol = comment_eol | semicol_eol (* handle old style bash arrays in new syntax *) let v1_list = let v1_list_value = store /[^;#()'"\t\n]+/ in [ label "quote" . store /['"]/ ] . [ label "value" . v1_list_value ] . del /['"]/ "\"" . [ Util.del_ws_spc . del /['"]/ "\"" . label "value" . v1_list_value . del /['"]/ "\"" ]* let v1_arr = del /\([ \t]*/ "(" . v1_list . del /[ \t]*\)/ ")" let v1_simple = del /\(/ "(" . [ label "value" . empty ] . del /\)/ ")" (* handle lists of arrays *) let list(word:regexp) = let list_value = store word in [ label "value" . list_value ] . [ nl . label "value" . list_value ]* (* handle single quoted lists *) let squote_arr = [ label "quote" . store /'/ ] . (list sqword)? . del /'/ "'" (* similarly handle double qouted lists *) let dquote_arr = [ label "quote" . store /"/ ] . (list dqword)? . del /"/ "\"" (* handle unquoted single value *) let unquot_val = [ label "quote" . store "" ] . [label "value" . store uqword+]? (* the source/. inclusion method of bash *) let source = [ Util.indent . del /\.|source/ "." . label ".source" . Util.del_ws_spc . store /[^;=# \t\n]+/ . comment_or_eol ] (* lens for key value pairs *) let kv = [ key key_re . eq . ( ( v1_arr . [ label "@syntax" . value "1" ] ) | ( v1_simple . [ label "@syntax" . value "1" ] ) | squote_arr | dquote_arr | unquot_val ) . nl ] | source let lns = ( comment | empty | kv )* let filter = incl "/etc/conf.d/net*" . Util.stdexcl let xfm = transform lns filter
(* Local Variables: *) (* mode: caml *) (* End: *) module Test_GentooConf_Net= let complicated_bridge_vlan_old_arrays = "config_eth0=(\"null\") config_eth1=(\"null\") vconfig_eth1=(\"set_name_type VLAN_PLUS_VID_NO_PAD\") vlans_eth1=\"10 11 15\" config_vlan10=(\"null\") bridge_br10=(\"vlan10\") routes_br10=(\"default via 10.128.0.254\") config_br10=(\"10.128.1.11/22\") RC_NEED_br10=\"net.eth1\" brctl_br10=(\"setfd 0\") bridge_br11=(\"vlan11\") config_vlan11=(\"null\") brctl_br11=(\"setfd 0\") RC_NEED_br11=\"net.eth1\" config_br11=(\"null\") RC_NEED_br15=\"net.eth1\" brctl_br15=(\"setfd 0\") config_br15=(\"null\") bridge_br15=(\"vlan15\") config_vlan15=(\"null\") vlan_start_eth1=\"no\" RC_NEED_vlan11=\"net.eth1\" RC_NEED_vlan10=\"net.eth1\" RC_NEED_vlan15=\"net.eth1\" " let complicated_bridge_vlan = "config_eth0=\"null\" config_eth1=\"null\" vconfig_eth1=\"set_name_type VLAN_PLUS_VID_NO_PAD\" vlans_eth1=\"10 11 12\" config_vlan10=\"null\" bridge_br10=\"vlan10\" routes_br10=\"default via 192.168.0.254\" config_br10=\"192.168.0.1/22\" RC_NEED_br10=\"net.eth1\" brctl_br10=\"setfd 0\" bridge_br11=\"vlan11\" config_vlan11=\"null\" brctl_br11=\"setfd 0\" RC_NEED_br11=\"net.eth1\" config_br11=\"null\" RC_NEED_br12=\"net.eth1\" brctl_br12=\"setfd 0\" config_br12=\"null\" bridge_br12=\"vlan12\" config_vlan12=\"null\" vlan_start_eth1=\"no\" RC_NEED_vlan11=\"net.eth1\" RC_NEED_vlan10=\"net.eth1\" RC_NEED_vlan12=\"net.eth1\" " (* FIXME need to parse the old bash arrays *) test GentooConf_Net.lns get complicated_bridge_vlan_old_arrays = { "config_eth0" { "quote" = "\"" } { "value" = "null" } { "@syntax" = "1" } } { "config_eth1" { "quote" = "\"" } { "value" = "null" } { "@syntax" = "1" } } { "vconfig_eth1" { "quote" = "\"" } { "value" = "set_name_type VLAN_PLUS_VID_NO_PAD" } { "@syntax" = "1" } } { "vlans_eth1" { "quote" = "\"" } { "value" = "10 11 15" } } { "config_vlan10" { "quote" = "\"" } { "value" = "null" } { "@syntax" = "1" } } { "bridge_br10" { "quote" = "\"" } { "value" = "vlan10" } { "@syntax" = "1" } } { "routes_br10" { "quote" = "\"" } { "value" = "default via 10.128.0.254" } { "@syntax" = "1" } } { "config_br10" { "quote" = "\"" } { "value" = "10.128.1.11/22" } { "@syntax" = "1" } } { "RC_NEED_br10" { "quote" = "\"" } { "value" = "net.eth1" } } { "brctl_br10" { "quote" = "\"" } { "value" = "setfd 0" } { "@syntax" = "1" } } { "bridge_br11" { "quote" = "\"" } { "value" = "vlan11" } { "@syntax" = "1" } } { "config_vlan11" { "quote" = "\"" } { "value" = "null" } { "@syntax" = "1" } } { "brctl_br11" { "quote" = "\"" } { "value" = "setfd 0" } { "@syntax" = "1" } } { "RC_NEED_br11" { "quote" = "\"" } { "value" = "net.eth1" } } { "config_br11" { "quote" = "\"" } { "value" = "null" } { "@syntax" = "1" } } { "RC_NEED_br15" { "quote" = "\"" } { "value" = "net.eth1" } } { "brctl_br15" { "quote" = "\"" } { "value" = "setfd 0" } { "@syntax" = "1" } } { "config_br15" { "quote" = "\"" } { "value" = "null" } { "@syntax" = "1" } } { "bridge_br15" { "quote" = "\"" } { "value" = "vlan15" } { "@syntax" = "1" } } { "config_vlan15" { "quote" = "\"" } { "value" = "null" } { "@syntax" = "1" } } { "vlan_start_eth1" { "quote" = "\"" } { "value" = "no" } } { "RC_NEED_vlan11" { "quote" = "\"" } { "value" = "net.eth1" } } { "RC_NEED_vlan10" { "quote" = "\"" } { "value" = "net.eth1" } } { "RC_NEED_vlan15" { "quote" = "\"" } { "value" = "net.eth1" } } test GentooConf_Net.lns get complicated_bridge_vlan = { "config_eth0" { "quote" = "\"" } { "value" = "null" } } { "config_eth1" { "quote" = "\"" } { "value" = "null" } } { "vconfig_eth1" { "quote" = "\"" } { "value" = "set_name_type VLAN_PLUS_VID_NO_PAD" } } { "vlans_eth1" { "quote" = "\"" } { "value" = "10 11 12" } } { "config_vlan10" { "quote" = "\"" } { "value" = "null" } } { "bridge_br10" { "quote" = "\"" } { "value" = "vlan10" } } { "routes_br10" { "quote" = "\"" } { "value" = "default via 192.168.0.254" } } { "config_br10" { "quote" = "\"" } { "value" = "192.168.0.1/22" } } { "RC_NEED_br10" { "quote" = "\"" } { "value" = "net.eth1" } } { "brctl_br10" { "quote" = "\"" } { "value" = "setfd 0" } } { "bridge_br11" { "quote" = "\"" } { "value" = "vlan11" } } { "config_vlan11" { "quote" = "\"" } { "value" = "null" } } { "brctl_br11" { "quote" = "\"" } { "value" = "setfd 0" } } { "RC_NEED_br11" { "quote" = "\"" } { "value" = "net.eth1" } } { "config_br11" { "quote" = "\"" } { "value" = "null" } } { "RC_NEED_br12" { "quote" = "\"" } { "value" = "net.eth1" } } { "brctl_br12" { "quote" = "\"" } { "value" = "setfd 0" } } { "config_br12" { "quote" = "\"" } { "value" = "null" } } { "bridge_br12" { "quote" = "\"" } { "value" = "vlan12" } } { "config_vlan12" { "quote" = "\"" } { "value" = "null" } } { "vlan_start_eth1" { "quote" = "\"" } { "value" = "no" } } { "RC_NEED_vlan11" { "quote" = "\"" } { "value" = "net.eth1" } } { "RC_NEED_vlan10" { "quote" = "\"" } { "value" = "net.eth1" } } { "RC_NEED_vlan12" { "quote" = "\"" } { "value" = "net.eth1" } } test GentooConf_Net.lns get "config_eth0=\"null\"\n" = { "config_eth0" { "quote" = "\"" } { "value" = "null" } } test GentooConf_Net.lns get "config_eth0=(\"null\")\n" = { "config_eth0" { "quote" = "\"" } { "value" = "null" } { "@syntax" = "1" } } test GentooConf_Net.lns get "config_eth0=(\"192.168.0.1/24\" \"192.168.0.2/24\")\n" = { "config_eth0" { "quote" = "\"" } { "value" = "192.168.0.1/24" } { "value" = "192.168.0.2/24" } { "@syntax" = "1" } } test GentooConf_Net.lns put "" after set "config_eth0/quote" "\""; set "config_eth0/value" "192.168.0.1/24"; set "config_eth0/@syntax" "1" = "config_eth0=(\"192.168.0.1/24\")\n" test GentooConf_Net.lns put "" after set "config_eth0/quote" "\""; set "config_eth0/value[1]" "192.168.0.1/24"; set "config_eth0/value[2]" "192.168.0.2/24"; set "config_eth0/@syntax" "1" = "config_eth0=(\"192.168.0.1/24\" \"192.168.0.2/24\")\n" test GentooConf_Net.lns get "config_eth0=( \"null\" )\n" = { "config_eth0" { "quote" = "\"" } { "value" = "null" } { "@syntax" = "1" } } test GentooConf_Net.lns put "" after set "config_eth0/quote" "\""; set "config_eth0/value[1]" "null" = "config_eth0=\"null\"\n" (* empty space should get deleted *) test GentooConf_Net.lns get "config_eth0=\" \"\n" = { "config_eth0" { "quote" = "\"" } { "value" = " " } } (* empty space should get deleted *) test GentooConf_Net.lns get "config_eth0=\" \"\n" = { "config_eth0" { "quote" = "\"" } { "value" = " " } } test GentooConf_Net.lns get "vlans_eth1=\"10\"\n" = { "vlans_eth1" { "quote" = "\"" } { "value" = "10" } } test GentooConf_Net.lns put "" after set "vlans_eth1/quote" "\""; set "vlans_eth1/value[1]" "10" = "vlans_eth1=\"10\"\n" (* leading space should get deleted *) test GentooConf_Net.lns get "vlans_eth1=\" 10\"\n" = { "vlans_eth1" { "quote" = "\"" } { "value" = " 10" } } (* trailing space should get deleted *) test GentooConf_Net.lns get "vlans_eth1=\"10 \"\n" = { "vlans_eth1" { "quote" = "\"" } { "value" = "10 " } } test GentooConf_Net.lns get "vlans_eth1=\"10 11\"\n" = { "vlans_eth1" { "quote" = "\"" } { "value" = "10 11" } } test GentooConf_Net.lns put "" after set "vlans_eth1/quote" "\""; set "vlans_eth1/value[1]" "10 11" = "vlans_eth1=\"10 11\"\n" test GentooConf_Net.lns get "config_eth0=\"192.168.0.1/24\"\n" = { "config_eth0" { "quote" = "\"" } { "value" = "192.168.0.1/24" } } test GentooConf_Net.lns put "" after set "config_eth0/quote" "\""; set "config_eth0/value[1]" "192.168.0.1/24" = "config_eth0=\"192.168.0.1/24\"\n" test GentooConf_Net.lns get "config_eth0=\"192.168.0.1/24\n" . "192.168.0.2/24\"\n" = { "config_eth0" { "quote" = "\"" } { "value" = "192.168.0.1/24" } { "value" = "192.168.0.2/24" } } (* FIXME doesn't parse at all b/c of trailing newline *) (* hard to fix; this might just be invalid *) (* test GentooConf_Net.lns get "config_eth0=\"192.168.0.1/24\n" . "192.168.0.2/24\n\"\n" = { "config_eth0" { "quote" = "\"" } { "value" = "192.168.0.1/24" } { "value" = "192.168.0.2/24" } } *) test GentooConf_Net.lns put "" after set "config_eth0/quote" "\""; set "config_eth0/value[1]" "192.168.0.1/24"; set "config_eth0/value[2]" "192.168.0.2/24" = "config_eth0=\"192.168.0.1/24\n192.168.0.2/24\"\n" test GentooConf_Net.lns get "config_eth0=\"192.168.0.2/24 scope host\"\n" = { "config_eth0" { "quote" = "\"" } { "value" = "192.168.0.2/24 scope host" } } test GentooConf_Net.lns get "config_eth0=\"192.168.0.2/24 scope host\n" . "4321:0:1:2:3:4:567:89ab/64 nodad home preferred_lft 0\"\n" = { "config_eth0" { "quote" = "\"" } { "value" = "192.168.0.2/24 scope host" } { "value" = "4321:0:1:2:3:4:567:89ab/64 nodad home preferred_lft 0" } } test GentooConf_Net.lns put "config_eth0=(\"192.168.0.1/24\")\n" after rm "config_eth0/@syntax" = "config_eth0=\"192.168.0.1/24\"\n" test GentooConf_Net.lns put "config_eth0=(\"192.168.0.1/24\" \"192.168.0.2/24\")\n" after rm "config_eth0/@syntax" = "config_eth0=\"192.168.0.1/24\n192.168.0.2/24\"\n" test GentooConf_Net.lns get "config_eth0=\"192.168.0.1/24\"\n" . ". /etc/conf.d/net.extra\n" = { "config_eth0" { "quote" = "\"" } { "value" = "192.168.0.1/24" } } { ".source" = "/etc/conf.d/net.extra" } (* config_eth0="null" config_eth1="null" vlans_eth1="10 11" vlan10_name="vlan10" config_vlan10="192.168.0.1/24" routes_vlan10="10.0.0.0/8 via 192.168.0.2" vlan11_name="vlan11" config_vlan11="172.16.0.1/25" routes_vlan11="default via 172.16.0.2" #vlan_start_eth1="no" #rc_need_vlan10="net.eth1" #rc_need_vlan11="net.eth1" *)
signature.asc
Description: OpenPGP digital signature
_______________________________________________ augeas-devel mailing list augeas-devel@redhat.com https://www.redhat.com/mailman/listinfo/augeas-devel