Re: Massive libxo-zation that breaks everything
On Mar 1, 2015, at 11:11, David Chisnall thera...@freebsd.org wrote: How would it be in a port? It involves modifying core utilities (some of which, like ifconfig, rely on kernel APIs that change between releases) to emit structured output. Maintaining two copies of each utility, one in the base system with plain-text output only and another in ports with XML/JSON output would be very painful. It would work fine if we had *libraries* for ifconfig/netstat/route/etc. Obviously that's not the case and no one has stepped up to implement them. I've also seen FreeBSD committers expressing their distaste for libraries for trivial command line utilities, which implies they are unaware of another world beyond the CLI. :-) -- Rui Paulo ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
On 03/01/15 13:25, Craig Rodrigues wrote: On Sun, Mar 1, 2015 at 10:49 AM, Harrison Grundy harrison.gru...@astrodoggroup.com wrote: Thanks! That does seem useful, but I'm not sure I see the reasoning behind putting into base, over a port or package , since processing XML in base is a pain, and it can't serve up JSON or HTML without additional utilities anyway. I think if you take another pass at reading the entire thread of responses to see the discussion for the motivation behind libxo, you will get the idea: https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html There are many people who are building products on top of FreeBSD. For these people, it is super useful for the base utilities in FreeBSD to emit output in properly formatted XML or JSON. That way, they do not need to write scripts to take the output of say, netstat, and use awk/sed/whatever scripts to take the human readable netstat output and convert it to a form which can be used in a script. There are many, many parsers for XML and JSON not in the base system. For people building products on top of FreeBSD, they don't care if these parsers are not in the base, since they can add these parsers on top of base FreeBSD. For example, languages like Python and Ruby have excellent parsers for JSON and XML. Many people build products using these languages. There are JSON and XML parsers in C, C++, and other languages as well. In addition to people building products, the other audience of people who benefit from libxo are devops people. These days, devops folks have no problem using Python, Ruby, Perl, whatever to write scripts to interact with Unix boxes and pull information off of it. Having the base utilities emit info in native JSON or XML greatly facilitates this. I talked to one person who is improving FreeBSD support for Saltstack (a devops framework). He told me that if more base utilities such as sysctl, could use libxo to emit output in JSON, that would greatly facilitate improving FreeBSD support for these devops frameworks. That is because Saltstack would require less FreeBSD-specific parsing code for getting info from base utilities. I like the idea behind this... where I'm running into difficulty is why these bits of functionality need to be combined. What someone does with ifconfig on the command line, versus what someone wants to know about their network interfaces in an XML dump may be very distinct bits of information. (For instance, an XML dump of network information may want to incorporate data from netstat, route, and ifconfig, while ifconfig is really only designed to deal with the interfaces themselves.) I know it involves some notable reworking, but would there be any interest in seeing what libifying a handful of binaries and splitting the libxo and CLI functionality looks like? It seems like this may give FreeBSD the best of both worlds, so the XML output isn't logically limited by how the CLI needs to work, and the CLI won't have to deal with the complexity XML output can require. I'd be happy to get this set up for a few binaries so people can see how it might work and what it could look like if there's any interest in going that route. --- Harrison ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sa(4) driver changes available for test
On Mar 1, 2015, at 7:36 PM, Kenneth D. Merry k...@freebsd.org wrote: On Sun, Mar 01, 2015 at 19:28:37 -0500, Dan Langille wrote: On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry k...@freebsd.org wrote: On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry k...@freebsd.org wrote: I have a fairly large set of changes to the sa(4) driver and mt(1) driver that I'm planning to commit in the near future. A description of the changes is here and below in this message. If you have tape hardware and the inclination, I'd appreciate testing and feedback. Rough draft commit message: http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt The patches against FreeBSD/head as of SVN revision 278706: http://people.freebsd.org/~ken/sa_changes.20150213.3.txt And (untested) patches against FreeBSD stable/10 as of SVN revision 278721. http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt The intent is to get the tape infrastructure more up to date, so we can support LTFS and more modern tape drives: http://www.ibm.com/systems/storage/tape/ltfs/ I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port depends on the patches linked above. It isn't fully cleaned up and ready for redistribution. If you're interested, though, let me know and I'll tell you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 or TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and older drives don't have the necessary features to support LTFS. The commit message below outlines most of the changes. A few comments: 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. 2. The XML output is similar to what GEOM and CTL do. It would be nice to figure out how to put a standard schema on it so that standard tools could read it. I don't know how feasible that is, since I haven't time to dig into it. If anyone has suggestions on whether that is feasible or advisable, I'd appreciate feedback. 3. I have tested with a reasonable amount of tape hardware (see below for a list), but more testing and feedback would be good. 4. Standard 'mt status' output looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP 5. 'mt status -v' looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP - Tape I/O parameters: Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes Maximum block size supported by tape drive and media (max_blk): 8388608 bytes Minimum block size supported by tape drive and media (min_blk): 1 bytes Block granularity supported by tape drive and media (blk_gran): 0 bytes Maximum possible I/O size (max_effective_iosize): 1081344 bytes # mtx -f /dev/pass0 status Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) Data Transfer Element 0:Empty Data Transfer Element 1:Empty Storage Element 1:Empty Storage Element 2:Empty Storage Element 3:Empty Storage Element 4:Full :VolumeTag=FAI260 Storage Element 5:Full :VolumeTag=FAI261 Storage Element 6:Full :VolumeTag=FAI262 Storage Element 7:Full :VolumeTag=FAI263 Storage Element 8:Empty Storage Element 9:Empty Storage Element 10:Empty It was at this point I spent the next 90 minute trying to get the tape drive out of the tape library to free a stuck tape. Some of this was spent attempting, and failing, to undo a stripped screw. I stopped the attempt when I noticed the screw did need to be removed. :/ Thanks for all of the effort! Looks like it is paying off! :) When I do this command, I hear the drive move a bit, to read the tape: # mt -f /dev/nsa1 status Drive: sa1: DEC TZ89 (C) DEC 2561 Serial Number: CXA09S1340
Re: sa(4) driver changes available for test
On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry k...@freebsd.org wrote:
On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry k...@freebsd.org wrote:
I have a fairly large set of changes to the sa(4) driver and mt(1) driver
that I'm planning to commit in the near future.
A description of the changes is here and below in this message.
If you have tape hardware and the inclination, I'd appreciate testing and
feedback.
I have a DLT 8000 and an SDLT 220.
I don't have anything running current, but I have a spare machine which I
could use for testing.
Do you see any value is tests with that hardware? I'd be testing it via
Bacula.
disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer.
Actually, yes. Bacula is a bit tricky to configure, so your trying it out
would be helpful if you have the time.
In looking at the manuals for both the SDLT 220 and the DLT 8000, they both
claim to support long position information for the SCSI READ POSITION
command.
You can see what I'm talking about by doing:
mt eod
mt status
On my DDS-4 tape drive, this shows:
# mt -f /dev/nsa3 status
Drive: sa3: SEAGATE DAT06240-XXX 8071 Serial Number: HJ00YWY
-
Mode Density Blocksize bpi Compression
Current: 0x26:DDS-4 1024 bytes 97000enabled (DCLZ)
-
Current Driver State: at rest.
-
Partition: 0 Calc File Number: -1 Calc Record Number: -1
Residual:0 Reported File Number: -1 Reported Record Number: -1
Flags: None
But on an LTO-5, which will give long position information, I get:
[root@doc ~]# mt status
Drive: sa0: IBM ULTRIUM-HH5 E4J1
-
Mode Density Blocksize bpi Compression
Current: 0x58:LTO-5 variable 384607 enabled (0x1)
-
Current Driver State: at rest.
-
Partition: 0 Calc File Number: 2 Calc Record Number: -1
Residual:0 Reported File Number: 2 Reported Record Number: 32373
Flags: None
That, in combination with the changes I made to the position information
code in the driver, mean that even the old MTIOCGET ioctl should return an
accurate file number at end of data. e.g., on the LTO-5:
[root@doc ~]# mt ostatus
Mode Density Blocksize bpi Compression
Current: 0x58:LTO-5 variable 384607 0x1
-available modes-
0:0x58:LTO-5 variable 384607 0x1
1:0x58:LTO-5 variable 384607 0x1
2:0x58:LTO-5 variable 384607 0x1
3:0x58:LTO-5 variable 384607 0x1
-
Current Driver State: at rest.
-
File Number: 2 Record Number: -1 Residual Count -1
So the thing to try, in addition to just making sure that Bacula continues
to work properly, is to try setting this for the tape drive in
bacula-sd.conf:
Hardware End of Medium = yes
It looks like the Bacula tape program (btape) has a test mode, and it would
be good to run through the tests on one of the tape drives and see whether
they work, and whether the results are different before and after the
changes. I'm not sure how to enable the test mode.
I have this in /usr/local/etc/bacula/bacula-sd.conf
Device {
Name= DLT
Description = QUANTUM DLT7000 1624
Media Type = DLT
Archive Device = /dev/nsa1
Autochanger = YES
Drive Index = 0
Offline On Unmount = no
Hardware End of Medium = yes
BSF at EOM = yes
Backward Space Record = no
Fast Forward Space File = no
TWO EOF = yes
}
FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape
test on this same model.
Here's the test I ran tonight:
[root@cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf
/dev/nsa1
Tape block granularity is 1024 bytes.
btape: butil.c:287-0 Using device: /dev/nsa1 for writing.
btape: btape.c:469-0 open device DLT (/dev/nsa1): OK
*test
=== Write, rewind, and re-read test ===
I'm going to write 1 records and an EOF
then write 1 records and an EOF, then rewind,
and re-read the data to verify that it is correct.
This is an *essential* feature ...
btape: btape.c:1152-0 Wrote 1 blocks of 64412 bytes.
btape: btape.c:604-0 Wrote 1 EOF to DLT
Re: Massive libxo-zation that breaks everything
On Sun, Mar 1, 2015 at 10:49 AM, Harrison Grundy harrison.gru...@astrodoggroup.com wrote: Thanks! That does seem useful, but I'm not sure I see the reasoning behind putting into base, over a port or package , since processing XML in base is a pain, and it can't serve up JSON or HTML without additional utilities anyway. I think if you take another pass at reading the entire thread of responses to see the discussion for the motivation behind libxo, you will get the idea: https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html There are many people who are building products on top of FreeBSD. For these people, it is super useful for the base utilities in FreeBSD to emit output in properly formatted XML or JSON. That way, they do not need to write scripts to take the output of say, netstat, and use awk/sed/whatever scripts to take the human readable netstat output and convert it to a form which can be used in a script. There are many, many parsers for XML and JSON not in the base system. For people building products on top of FreeBSD, they don't care if these parsers are not in the base, since they can add these parsers on top of base FreeBSD. For example, languages like Python and Ruby have excellent parsers for JSON and XML. Many people build products using these languages. There are JSON and XML parsers in C, C++, and other languages as well. In addition to people building products, the other audience of people who benefit from libxo are devops people. These days, devops folks have no problem using Python, Ruby, Perl, whatever to write scripts to interact with Unix boxes and pull information off of it. Having the base utilities emit info in native JSON or XML greatly facilitates this. I talked to one person who is improving FreeBSD support for Saltstack (a devops framework). He told me that if more base utilities such as sysctl, could use libxo to emit output in JSON, that would greatly facilitate improving FreeBSD support for these devops frameworks. That is because Saltstack would require less FreeBSD-specific parsing code for getting info from base utilities. -- Craig ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sa(4) driver changes available for test
On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry k...@freebsd.org wrote: I have a fairly large set of changes to the sa(4) driver and mt(1) driver that I'm planning to commit in the near future. A description of the changes is here and below in this message. If you have tape hardware and the inclination, I'd appreciate testing and feedback. Rough draft commit message: http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt The patches against FreeBSD/head as of SVN revision 278706: http://people.freebsd.org/~ken/sa_changes.20150213.3.txt And (untested) patches against FreeBSD stable/10 as of SVN revision 278721. http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt The intent is to get the tape infrastructure more up to date, so we can support LTFS and more modern tape drives: http://www.ibm.com/systems/storage/tape/ltfs/ I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port depends on the patches linked above. It isn't fully cleaned up and ready for redistribution. If you're interested, though, let me know and I'll tell you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 or TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and older drives don't have the necessary features to support LTFS. The commit message below outlines most of the changes. A few comments: 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. 2. The XML output is similar to what GEOM and CTL do. It would be nice to figure out how to put a standard schema on it so that standard tools could read it. I don't know how feasible that is, since I haven't time to dig into it. If anyone has suggestions on whether that is feasible or advisable, I'd appreciate feedback. 3. I have tested with a reasonable amount of tape hardware (see below for a list), but more testing and feedback would be good. 4. Standard 'mt status' output looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP 5. 'mt status -v' looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP - Tape I/O parameters: Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes Maximum block size supported by tape drive and media (max_blk): 8388608 bytes Minimum block size supported by tape drive and media (min_blk): 1 bytes Block granularity supported by tape drive and media (blk_gran): 0 bytes Maximum possible I/O size (max_effective_iosize): 1081344 bytes # mtx -f /dev/pass0 status Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) Data Transfer Element 0:Empty Data Transfer Element 1:Empty Storage Element 1:Empty Storage Element 2:Empty Storage Element 3:Empty Storage Element 4:Full :VolumeTag=FAI260 Storage Element 5:Full :VolumeTag=FAI261 Storage Element 6:Full :VolumeTag=FAI262 Storage Element 7:Full :VolumeTag=FAI263 Storage Element 8:Empty Storage Element 9:Empty Storage Element 10:Empty It was at this point I spent the next 90 minute trying to get the tape drive out of the tape library to free a stuck tape. Some of this was spent attempting, and failing, to undo a stripped screw. I stopped the attempt when I noticed the screw did need to be removed. :/ When I do this command, I hear the drive move a bit, to read the tape: # mt -f /dev/nsa1 status Drive: sa1: DEC TZ89 (C) DEC 2561 Serial Number: CXA09S1340 - Mode DensityBlocksize bpi Compression Current: 0x1b:DLTapeIV(35GB)variable 85937enabled (IDRC) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record
Re: sa(4) driver changes available for test
On Sun, Mar 01, 2015 at 19:40:40 -0500, Dan Langille wrote: On Mar 1, 2015, at 7:36 PM, Kenneth D. Merry k...@freebsd.org wrote: On Sun, Mar 01, 2015 at 19:28:37 -0500, Dan Langille wrote: On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry k...@freebsd.org wrote: On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry k...@freebsd.org wrote: I have a fairly large set of changes to the sa(4) driver and mt(1) driver that I'm planning to commit in the near future. A description of the changes is here and below in this message. If you have tape hardware and the inclination, I'd appreciate testing and feedback. Rough draft commit message: http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt The patches against FreeBSD/head as of SVN revision 278706: http://people.freebsd.org/~ken/sa_changes.20150213.3.txt And (untested) patches against FreeBSD stable/10 as of SVN revision 278721. http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt The intent is to get the tape infrastructure more up to date, so we can support LTFS and more modern tape drives: http://www.ibm.com/systems/storage/tape/ltfs/ I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port depends on the patches linked above. It isn't fully cleaned up and ready for redistribution. If you're interested, though, let me know and I'll tell you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 or TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and older drives don't have the necessary features to support LTFS. The commit message below outlines most of the changes. A few comments: 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. 2. The XML output is similar to what GEOM and CTL do. It would be nice to figure out how to put a standard schema on it so that standard tools could read it. I don't know how feasible that is, since I haven't time to dig into it. If anyone has suggestions on whether that is feasible or advisable, I'd appreciate feedback. 3. I have tested with a reasonable amount of tape hardware (see below for a list), but more testing and feedback would be good. 4. Standard 'mt status' output looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP 5. 'mt status -v' looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP - Tape I/O parameters: Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes Maximum block size supported by tape drive and media (max_blk): 8388608 bytes Minimum block size supported by tape drive and media (min_blk): 1 bytes Block granularity supported by tape drive and media (blk_gran): 0 bytes Maximum possible I/O size (max_effective_iosize): 1081344 bytes # mtx -f /dev/pass0 status Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) Data Transfer Element 0:Empty Data Transfer Element 1:Empty Storage Element 1:Empty Storage Element 2:Empty Storage Element 3:Empty Storage Element 4:Full :VolumeTag=FAI260 Storage Element 5:Full :VolumeTag=FAI261 Storage Element 6:Full :VolumeTag=FAI262 Storage Element 7:Full :VolumeTag=FAI263 Storage Element 8:Empty Storage Element 9:Empty Storage Element 10:Empty It was at this point I spent the next 90 minute trying to get the tape drive out of the tape library to free a stuck tape. Some of this was spent attempting, and failing, to undo a stripped screw. I stopped the attempt when I noticed the screw did need to be removed. :/ Thanks for all of the effort!
Re: sa(4) driver changes available for test
On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry k...@freebsd.org wrote:
On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry k...@freebsd.org wrote:
I have a fairly large set of changes to the sa(4) driver and mt(1) driver
that I'm planning to commit in the near future.
A description of the changes is here and below in this message.
If you have tape hardware and the inclination, I'd appreciate testing and
feedback.
I have a DLT 8000 and an SDLT 220.
I don't have anything running current, but I have a spare machine which I
could use for testing.
Do you see any value is tests with that hardware? I'd be testing it via
Bacula.
disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer.
Actually, yes. Bacula is a bit tricky to configure, so your trying it out
would be helpful if you have the time.
In looking at the manuals for both the SDLT 220 and the DLT 8000, they both
claim to support long position information for the SCSI READ POSITION
command.
You can see what I'm talking about by doing:
mt eod
mt status
On my DDS-4 tape drive, this shows:
# mt -f /dev/nsa3 status
Drive: sa3: SEAGATE DAT06240-XXX 8071 Serial Number: HJ00YWY
-
Mode Density Blocksize bpi Compression
Current: 0x26:DDS-4 1024 bytes 97000enabled (DCLZ)
-
Current Driver State: at rest.
-
Partition: 0 Calc File Number: -1 Calc Record Number: -1
Residual:0 Reported File Number: -1 Reported Record Number: -1
Flags: None
But on an LTO-5, which will give long position information, I get:
[root@doc ~]# mt status
Drive: sa0: IBM ULTRIUM-HH5 E4J1
-
Mode Density Blocksize bpi Compression
Current: 0x58:LTO-5 variable 384607 enabled (0x1)
-
Current Driver State: at rest.
-
Partition: 0 Calc File Number: 2 Calc Record Number: -1
Residual:0 Reported File Number: 2 Reported Record Number: 32373
Flags: None
That, in combination with the changes I made to the position information
code in the driver, mean that even the old MTIOCGET ioctl should return an
accurate file number at end of data. e.g., on the LTO-5:
[root@doc ~]# mt ostatus
Mode Density Blocksize bpi Compression
Current: 0x58:LTO-5 variable 384607 0x1
-available modes-
0:0x58:LTO-5 variable 384607 0x1
1:0x58:LTO-5 variable 384607 0x1
2:0x58:LTO-5 variable 384607 0x1
3:0x58:LTO-5 variable 384607 0x1
-
Current Driver State: at rest.
-
File Number: 2 Record Number: -1 Residual Count -1
So the thing to try, in addition to just making sure that Bacula continues
to work properly, is to try setting this for the tape drive in
bacula-sd.conf:
Hardware End of Medium = yes
It looks like the Bacula tape program (btape) has a test mode, and it would
be good to run through the tests on one of the tape drives and see whether
they work, and whether the results are different before and after the
changes. I'm not sure how to enable the test mode.
I have this in /usr/local/etc/bacula/bacula-sd.conf
Device {
Name= DLT
Description = QUANTUM DLT7000 1624
Media Type = DLT
Archive Device = /dev/nsa1
Autochanger = YES
Drive Index = 0
Offline On Unmount = no
Hardware End of Medium = yes
BSF at EOM = yes
Backward Space Record = no
Fast Forward Space File = no
TWO EOF = yes
}
FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape test
on this same model.
Here's the test I ran tonight:
[root@cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf
/dev/nsa1
Tape block granularity is 1024 bytes.
btape: butil.c:287-0 Using device: /dev/nsa1 for writing.
btape: btape.c:469-0 open device DLT (/dev/nsa1): OK
*test
=== Write, rewind, and re-read test ===
I'm going to write 1 records and an EOF
then write 1 records and an EOF, then rewind,
and re-read the data to verify that it is correct.
This is an *essential* feature ...
btape: btape.c:1152-0 Wrote 1 blocks of 64412 bytes.
btape: btape.c:604-0 Wrote 1 EOF to DLT (/dev/nsa1)
btape: btape.c:1168-0 Wrote 1 blocks of 64412 bytes.
btape: btape.c:604-0 Wrote 1 EOF to DLT (/dev/nsa1)
btape: btape.c:604-0 Wrote 1 EOF to DLT (/dev/nsa1)
btape: btape.c:1210-0
Re: sa(4) driver changes available for test
On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry k...@freebsd.org wrote: On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry k...@freebsd.org wrote: I have a fairly large set of changes to the sa(4) driver and mt(1) driver that I'm planning to commit in the near future. A description of the changes is here and below in this message. If you have tape hardware and the inclination, I'd appreciate testing and feedback. Rough draft commit message: http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt The patches against FreeBSD/head as of SVN revision 278706: http://people.freebsd.org/~ken/sa_changes.20150213.3.txt And (untested) patches against FreeBSD stable/10 as of SVN revision 278721. http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt The intent is to get the tape infrastructure more up to date, so we can support LTFS and more modern tape drives: http://www.ibm.com/systems/storage/tape/ltfs/ I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port depends on the patches linked above. It isn't fully cleaned up and ready for redistribution. If you're interested, though, let me know and I'll tell you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 or TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and older drives don't have the necessary features to support LTFS. The commit message below outlines most of the changes. A few comments: 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. 2. The XML output is similar to what GEOM and CTL do. It would be nice to figure out how to put a standard schema on it so that standard tools could read it. I don't know how feasible that is, since I haven't time to dig into it. If anyone has suggestions on whether that is feasible or advisable, I'd appreciate feedback. 3. I have tested with a reasonable amount of tape hardware (see below for a list), but more testing and feedback would be good. 4. Standard 'mt status' output looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP 5. 'mt status -v' looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP - Tape I/O parameters: Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes Maximum block size supported by tape drive and media (max_blk): 8388608 bytes Minimum block size supported by tape drive and media (min_blk): 1 bytes Block granularity supported by tape drive and media (blk_gran): 0 bytes Maximum possible I/O size (max_effective_iosize): 1081344 bytes # mtx -f /dev/pass0 status Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) Data Transfer Element 0:Empty Data Transfer Element 1:Empty Storage Element 1:Empty Storage Element 2:Empty Storage Element 3:Empty Storage Element 4:Full :VolumeTag=FAI260 Storage Element 5:Full :VolumeTag=FAI261 Storage Element 6:Full :VolumeTag=FAI262 Storage Element 7:Full :VolumeTag=FAI263 Storage Element 8:Empty Storage Element 9:Empty Storage Element 10:Empty It was at this point I spent the next 90 minute trying to get the tape drive out of the tape library to free a stuck tape. Some of this was spent attempting, and failing, to undo a stripped screw. I stopped the attempt when I noticed the screw did need to be removed. :/ Thanks for all of the effort! Looks like it is paying off! :) When I do this command, I hear the drive move a bit, to read the tape: # mt -f /dev/nsa1 status Drive: sa1: DEC TZ89 (C) DEC 2561 Serial Number: CXA09S1340 - Mode DensityBlocksize bpi Compression Current:
Re: sa(4) driver changes available for test
On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry k...@freebsd.org wrote:
On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry k...@freebsd.org wrote:
On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry k...@freebsd.org wrote:
I have a fairly large set of changes to the sa(4) driver and mt(1) driver
that I'm planning to commit in the near future.
A description of the changes is here and below in this message.
If you have tape hardware and the inclination, I'd appreciate testing and
feedback.
I have a DLT 8000 and an SDLT 220.
I don't have anything running current, but I have a spare machine which I
could use for testing.
Do you see any value is tests with that hardware? I'd be testing it via
Bacula.
disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer.
Actually, yes. Bacula is a bit tricky to configure, so your trying it out
would be helpful if you have the time.
In looking at the manuals for both the SDLT 220 and the DLT 8000, they both
claim to support long position information for the SCSI READ POSITION
command.
You can see what I'm talking about by doing:
mt eod
mt status
On my DDS-4 tape drive, this shows:
# mt -f /dev/nsa3 status
Drive: sa3: SEAGATE DAT06240-XXX 8071 Serial Number: HJ00YWY
-
Mode Density Blocksize bpi Compression
Current: 0x26:DDS-4 1024 bytes 97000enabled (DCLZ)
-
Current Driver State: at rest.
-
Partition: 0 Calc File Number: -1 Calc Record Number: -1
Residual:0 Reported File Number: -1 Reported Record Number: -1
Flags: None
But on an LTO-5, which will give long position information, I get:
[root@doc ~]# mt status
Drive: sa0: IBM ULTRIUM-HH5 E4J1
-
Mode Density Blocksize bpi Compression
Current: 0x58:LTO-5 variable 384607 enabled (0x1)
-
Current Driver State: at rest.
-
Partition: 0 Calc File Number: 2 Calc Record Number: -1
Residual:0 Reported File Number: 2 Reported Record Number: 32373
Flags: None
That, in combination with the changes I made to the position information
code in the driver, mean that even the old MTIOCGET ioctl should return an
accurate file number at end of data. e.g., on the LTO-5:
[root@doc ~]# mt ostatus
Mode Density Blocksize bpi Compression
Current: 0x58:LTO-5 variable 384607 0x1
-available modes-
0:0x58:LTO-5 variable 384607 0x1
1:0x58:LTO-5 variable 384607 0x1
2:0x58:LTO-5 variable 384607 0x1
3:0x58:LTO-5 variable 384607 0x1
-
Current Driver State: at rest.
-
File Number: 2 Record Number: -1 Residual Count -1
So the thing to try, in addition to just making sure that Bacula continues
to work properly, is to try setting this for the tape drive in
bacula-sd.conf:
Hardware End of Medium = yes
It looks like the Bacula tape program (btape) has a test mode, and it would
be good to run through the tests on one of the tape drives and see whether
they work, and whether the results are different before and after the
changes. I'm not sure how to enable the test mode.
I have this in /usr/local/etc/bacula/bacula-sd.conf
Device {
Name= DLT
Description = QUANTUM DLT7000 1624
Media Type = DLT
Archive Device = /dev/nsa1
Autochanger = YES
Drive Index = 0
Offline On Unmount = no
Hardware End of Medium = yes
BSF at EOM = yes
Backward Space Record = no
Fast Forward Space File = no
TWO EOF = yes
}
FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape
test on this same model.
Here's the test I ran tonight:
[root@cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf
/dev/nsa1
Tape block granularity is 1024 bytes.
btape: butil.c:287-0 Using device: /dev/nsa1 for writing.
btape: btape.c:469-0 open device DLT (/dev/nsa1): OK
*test
=== Write, rewind, and re-read test ===
I'm going to write 1 records and an EOF
then write 1 records and an EOF, then rewind,
and re-read the data to verify that it is correct.
This is an *essential* feature ...
btape: btape.c:1152-0 Wrote 1 blocks of 64412 bytes.
btape: btape.c:604-0 Wrote 1 EOF to DLT (/dev/nsa1)
btape:
Re: sa(4) driver changes available for test
On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry k...@freebsd.org wrote: I have a fairly large set of changes to the sa(4) driver and mt(1) driver that I'm planning to commit in the near future. A description of the changes is here and below in this message. If you have tape hardware and the inclination, I'd appreciate testing and feedback. Rough draft commit message: http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt The patches against FreeBSD/head as of SVN revision 278706: http://people.freebsd.org/~ken/sa_changes.20150213.3.txt And (untested) patches against FreeBSD stable/10 as of SVN revision 278721. http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt The intent is to get the tape infrastructure more up to date, so we can support LTFS and more modern tape drives: http://www.ibm.com/systems/storage/tape/ltfs/ I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port depends on the patches linked above. It isn't fully cleaned up and ready for redistribution. If you're interested, though, let me know and I'll tell you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 or TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and older drives don't have the necessary features to support LTFS. The commit message below outlines most of the changes. A few comments: 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. 2. The XML output is similar to what GEOM and CTL do. It would be nice to figure out how to put a standard schema on it so that standard tools could read it. I don't know how feasible that is, since I haven't time to dig into it. If anyone has suggestions on whether that is feasible or advisable, I'd appreciate feedback. 3. I have tested with a reasonable amount of tape hardware (see below for a list), but more testing and feedback would be good. 4. Standard 'mt status' output looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP 5. 'mt status -v' looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP - Tape I/O parameters: Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes Maximum block size supported by tape drive and media (max_blk): 8388608 bytes Minimum block size supported by tape drive and media (min_blk): 1 bytes Block granularity supported by tape drive and media (blk_gran): 0 bytes Maximum possible I/O size (max_effective_iosize): 1081344 bytes # mtx -f /dev/pass0 status Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) Data Transfer Element 0:Empty Data Transfer Element 1:Empty Storage Element 1:Empty Storage Element 2:Empty Storage Element 3:Empty Storage Element 4:Full :VolumeTag=FAI260 Storage Element 5:Full :VolumeTag=FAI261 Storage Element 6:Full :VolumeTag=FAI262 Storage Element 7:Full :VolumeTag=FAI263 Storage Element 8:Empty Storage Element 9:Empty Storage Element 10:Empty It was at this point I spent the next 90 minute trying to get the tape drive out of the tape library to free a stuck tape. Some of this was spent attempting, and failing, to undo a stripped screw. I stopped the attempt when I noticed the screw did need to be removed. :/ Thanks for all of the effort! Looks like it is paying off! :) When I do this command, I hear the drive move a bit, to read the tape: # mt -f /dev/nsa1 status Drive: sa1: DEC TZ89 (C) DEC 2561 Serial Number: CXA09S1340 - Mode DensityBlocksize bpi
Re: sa(4) driver changes available for test
On Sun, Mar 01, 2015 at 19:28:37 -0500, Dan Langille wrote: On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry k...@freebsd.org wrote: On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote: On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry k...@freebsd.org wrote: I have a fairly large set of changes to the sa(4) driver and mt(1) driver that I'm planning to commit in the near future. A description of the changes is here and below in this message. If you have tape hardware and the inclination, I'd appreciate testing and feedback. Rough draft commit message: http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt The patches against FreeBSD/head as of SVN revision 278706: http://people.freebsd.org/~ken/sa_changes.20150213.3.txt And (untested) patches against FreeBSD stable/10 as of SVN revision 278721. http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt The intent is to get the tape infrastructure more up to date, so we can support LTFS and more modern tape drives: http://www.ibm.com/systems/storage/tape/ltfs/ I have ported IBM's LTFS Single Drive Edition to FreeBSD. The port depends on the patches linked above. It isn't fully cleaned up and ready for redistribution. If you're interested, though, let me know and I'll tell you when it is ready to go out. You need an IBM LTO-5, LTO-6, TS1140 or TS1150 tape drive. HP drives aren't supported by IBM's LTFS, and older drives don't have the necessary features to support LTFS. The commit message below outlines most of the changes. A few comments: 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately. 2. The XML output is similar to what GEOM and CTL do. It would be nice to figure out how to put a standard schema on it so that standard tools could read it. I don't know how feasible that is, since I haven't time to dig into it. If anyone has suggestions on whether that is feasible or advisable, I'd appreciate feedback. 3. I have tested with a reasonable amount of tape hardware (see below for a list), but more testing and feedback would be good. 4. Standard 'mt status' output looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP 5. 'mt status -v' looks like this: # mt -f /dev/nsa3 status -v Drive: sa3: IBM ULTRIUM-HH6 E4J1 Serial Number: 101500520A - Mode Density Blocksize bpi Compression Current: 0x5a:LTO-6 variable 384607 enabled (0xff) - Current Driver State: at rest. - Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual:0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP - Tape I/O parameters: Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes Maximum block size supported by tape drive and media (max_blk): 8388608 bytes Minimum block size supported by tape drive and media (min_blk): 1 bytes Block granularity supported by tape drive and media (blk_gran): 0 bytes Maximum possible I/O size (max_effective_iosize): 1081344 bytes # mtx -f /dev/pass0 status Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export ) Data Transfer Element 0:Empty Data Transfer Element 1:Empty Storage Element 1:Empty Storage Element 2:Empty Storage Element 3:Empty Storage Element 4:Full :VolumeTag=FAI260 Storage Element 5:Full :VolumeTag=FAI261 Storage Element 6:Full :VolumeTag=FAI262 Storage Element 7:Full :VolumeTag=FAI263 Storage Element 8:Empty Storage Element 9:Empty Storage Element 10:Empty It was at this point I spent the next 90 minute trying to get the tape drive out of the tape library to free a stuck tape. Some of this was spent attempting, and failing, to undo a stripped screw. I stopped the attempt when I noticed the screw did need to be removed. :/ Thanks for all of the effort! Looks like it is paying off! :) When I do this command, I hear the drive move a bit, to read the tape: # mt -f /dev/nsa1 status
Re: Massive libxo-zation that breaks everything
On 2015-03-01 19:20, Arseny Nasokin wrote: On 1 March 2015 at 22:10, Allan Jude allanj...@freebsd.org wrote: On 2015-03-01 13:49, Harrison Grundy wrote: Thanks! That does seem useful, but I'm not sure I see the reasoning behind putting into base, over a port or package, since processing XML in base is a pain, and it can't serve up JSON or HTML without additional utilities anyway. (If I'm reviving a long-settled thing, let me know and I'll drop it. I'm trying to understand the use case for this.) --- Harrison On 03/01/15 10:31, Craig Rodrigues wrote: On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy harrison.gru...@astrodoggroup.com wrote: If someone could summarize what this is, I'd greatly appreciate it. https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org I think you're missing the important bit here. This isn't about adding a parser for anything, this is about making the tools in base, like netstat, wc, uptime, etc, output in JSON or XML, so you can use the data programmatically. Your scripts no longer have to rely on awk/sed/grep magic to get a specific bit of information out of the uptime command, the command can just output the data in a structured machine readable format. I am not sure how you can put netstat into the ports tree. -- Allan Jude Hi, Do we have command-line tools in base which work with XML/JSON from stdin or file? -- Eir Nym ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org Not really, although there are libraries for such (bsdxml, libucl which can read JSON) in base. I am working on a tool that can do some of this: https://github.com/allanjude/uclcmd/ -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: HEADS UP: PCI SR-IOV infrastructure has been committed to head
Awesome news my friend. I got distracted with some fire drills this week, but I'll make it a priority to get the ixl changes committed next week. Thanks to you and everyone involved for getting this done. Jack On Sat, Feb 28, 2015 at 6:01 PM, Ryan Stone ryst...@gmail.com wrote: I've just finished committing support for PCI Single Root I/O Virtualization in the pci subsystem to head. This should be a no-op for everyone right now, but there were some minor refactorings in the pci code that could have a lingering bug. I did make sure to test that it boots on a variety of systems (but only i386/amd64, as that's all that I have access to). What's been committed to head is only the pci subsystem side of things, along with the userland tools to configure SR-IOV (along with, I'm happy to say, a full set of man pages). What's not in head yet are any drivers making use of the infrastructure. Full support for ixl(4) is complete and I've sent the patch to jfv@; I hope to see the driver support committed soon. I don't have any word on timelines for getting support in other drivers. Unfortunately adding SR-IOV support to a driver is not trivial as the standard leaves a lot of the details up to particular implementations (in the same way the the PCIe standard does not define how to send a packet from a NIC; instead defining how the PCIe device will expose its registers and whatnot, and its up to the PCIe device and driver to understand how to poke at the registers to send a packet). I have heard anecdotally that a number of driver maintainers have been very interested in this work so I hope that to see more drivers supported SR-IOV in the near future. I encourage all driver maintainers to read over the new manpages and contact me if they have any questions about the new infrastructure. Anybody interested in using SR-IOV should try to attend BSDCan 2015, as I will be giving a talk on the subject. I intend to focus more on the system administration side of configuring and using SR-IOV rather than the details of implementing an SR-IOV driver. If anybody did an svn up half-way through my muddled series of commits, sorry about the temporary breakage. My buildworld/buildkernel on r279466 just completed successfully so please make sure that you have at least that revision. If you still have problems, please let me know. I do want to thank John Baldwin for advice about the PCI Subsystem and newbus and Jack Vogel for his help with the Fortville NIC, including getting me early access to the VF driver for testing purposes. Thanks to everybody who reviewed the changes. Specially thanks to Mark Johnston and Sean Mahood, who literally spent hours with me in a meeting room reviewing the entire patch series last summer (thankfully, those hours at least weren't consecutive). Above all, thanks to Sandvine Inc. for sponsoring this work. This is definitely the biggest contribution we've ever made to FreeBSD and I hope to see this kind of thing continue. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Jenkins build is back to normal : FreeBSD_HEAD #2474
See https://jenkins.freebsd.org/job/FreeBSD_HEAD/2474/changes ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Massive libxo-zation that breaks everything
Hello. First, I would be happy to have JSON and XML output about filesystems, users, routes... but I don't like how it makes code of df, w, netstat hard to read/maintain and often broken. I don't think it would be good to continue with this. Maybe the effort should be put to creating new layer/library and then something on top of it that actually outputs JSON and XML. Or, if that's too difficult... maybe just regular df/w/netstat could be copied to somewhere else and made code libxo-output-only. And original df/w/netstat changes reverted and left alone. Then, maybe later, df/w/netstat/... could be updated to this new layer/library. Or maybe this should be just left as it is. That would mean having two netstat's in system, which could be both good (separation) and bad (maintaining). Just some ideas... I don't know how to solve this issue fully. I'm also not likely the one who would write code for all this. Hell, those aren't even all my ideas here. I just worry that system drop-in xo-zation is bad for overall health of base. Oh and, it makes rescue larger and more complex, too? On that, there was suggestion to maybe create separate first aid kit and emergency room types of system rescue utils/methods. Thanks. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
How about we allow JSON input on those utils too... Then we get into full-blown hell faster. Hmm... I would like to talk with system using JSON. JSON would be in utils that are or at least function similarly to rm, mv, ls, find, mount, zpool, zfs, geom, mdconfig, tar, df, netstat, ifconfig... (or maybe even talk JSON directly to the kernel?!). How to have all that goodness, while at same time not having extra library dependency? Without that library the system wouldn't work at all (and there is no fallback mode for manual operation). Fragile library that requires you to have worse-than-bad-Perl-looking (funnily I write lot of Perl myself) parts everywhere in your code. And requires huge set of tests to verify correct operation. Hope that you never see things like this: df Shared object libxo.so.0 not found, required by df Or this: netstat -nhbdWi Name Mtu Network Address Ipkts Ierrs Idrop IbytesOpkts Oerrs Obytes Coll Drop smc0 %6s 1.5K Link#1 52:54:00:12:34:56 %8s 4.4M %5s 0 %5s 0 %10s 290M %8s 3.3M %5s 0 %10s 756M %5s 0 %5s 11 smc0- fe80::5054:ff fe80::5054:ff:fe1 %8s 8.7K - - %10s 592K %8s 17K - %10s 1.0M - - smc0- 2001:ad0:91f: 2001:ad0:91f:0:50 %8s 4.4M - - %10s 225M %8s 3.3M - %10s 709M - - smc0- 10.0.0.0/24 10.0.0.48 %8s 4.0K - - %10s 504K %8s 3.2K - %10s 233K - - lo0%6s16K Link#2%8s 272 %5s 0 %5s 0 %10s24K %8s 272 %5s 0 %10s24K %5s 0 %5s 0 lo0 - ::1/128 ::1 %8s 136 - - %10s16K %8s 136 - %10s16K - - lo0 - fe80::1%lo0/6 fe80::1%lo0 %8s0 - - %10s 0 %8s7 - %10s963 - - lo0 - 127.0.0.0/8 127.0.0.1 %8s 136 - - %10s 8.6K %8s 136 - %10s 8.6K - - Or, at least you only see it (occasionally) in CURRENT. ( And, all current libxo issues seem to be fixed, for now... ) ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
On 3/1/15 4:29 PM, Rui Paulo wrote: On Mar 1, 2015, at 11:11, David Chisnall thera...@freebsd.org wrote: How would it be in a port? It involves modifying core utilities (some of which, like ifconfig, rely on kernel APIs that change between releases) to emit structured output. Maintaining two copies of each utility, one in the base system with plain-text output only and another in ports with XML/JSON output would be very painful. It would work fine if we had *libraries* for ifconfig/netstat/route/etc. Obviously that's not the case and no one has stepped up to implement them. I've also seen FreeBSD committers expressing their distaste for libraries for trivial command line utilities, which implies they are unaware of another world beyond the CLI. :-) +1. Next step is making those utils into libraries. Should be easy, just make all globals into TLS and don't munge stdin/stdout in dumb ways and we're there. Glad you are a big thinker Rui. :) -Alfred ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
On 3/1/15 4:57 PM, Harrison Grundy wrote: I like the idea behind this... where I'm running into difficulty is why these bits of functionality need to be combined. What someone does with ifconfig on the command line, versus what someone wants to know about their network interfaces in an XML dump may be very distinct bits of information. (For instance, an XML dump of network information may want to incorporate data from netstat, route, and ifconfig, while ifconfig is really only designed to deal with the interfaces themselves.) Part of this is putting the burden of machine readable output into the utilities themselves. Meaning that people have complained against libxo because well you can get the same information by combining data points X, Y and Z from sysctl whereas ifconfig just does that for you. So if we split the code then those interfaces would wither and die from neglect because honestly very few core freebsd devs care about XML and JSON. So in order to prevent bitotting and make this a feature of FreeBSD, we decided to put it into the base utils so that we would be sure to carry the burden and keep it working. I know it involves some notable reworking, but would there be any interest in seeing what libifying a handful of binaries and splitting the libxo and CLI functionality looks like? It seems like this may give FreeBSD the best of both worlds, so the XML output isn't logically limited by how the CLI needs to work, and the CLI won't have to deal with the complexity XML output can require. I'd be happy to get this set up for a few binaries so people can see how it might work and what it could look like if there's any interest in going that route. Please do. That would be great. Please make sure they are thread safe. :) -Alfred ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sa(4) driver changes available for test
On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote:
On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry k...@freebsd.org wrote:
On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry k...@freebsd.org wrote:
On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry k...@freebsd.org wrote:
I have a fairly large set of changes to the sa(4) driver and mt(1)
driver
that I'm planning to commit in the near future.
A description of the changes is here and below in this message.
If you have tape hardware and the inclination, I'd appreciate testing
and
feedback.
I have a DLT 8000 and an SDLT 220.
I don't have anything running current, but I have a spare machine which
I could use for testing.
Do you see any value is tests with that hardware? I'd be testing it via
Bacula.
disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer.
Actually, yes. Bacula is a bit tricky to configure, so your trying it out
would be helpful if you have the time.
In looking at the manuals for both the SDLT 220 and the DLT 8000, they
both
claim to support long position information for the SCSI READ POSITION
command.
You can see what I'm talking about by doing:
mt eod
mt status
On my DDS-4 tape drive, this shows:
# mt -f /dev/nsa3 status
Drive: sa3: SEAGATE DAT06240-XXX 8071 Serial Number: HJ00YWY
-
Mode Density Blocksize bpi Compression
Current: 0x26:DDS-4 1024 bytes 97000enabled (DCLZ)
-
Current Driver State: at rest.
-
Partition: 0 Calc File Number: -1 Calc Record Number: -1
Residual:0 Reported File Number: -1 Reported Record Number: -1
Flags: None
But on an LTO-5, which will give long position information, I get:
[root@doc ~]# mt status
Drive: sa0: IBM ULTRIUM-HH5 E4J1
-
Mode Density Blocksize bpi Compression
Current: 0x58:LTO-5 variable 384607 enabled (0x1)
-
Current Driver State: at rest.
-
Partition: 0 Calc File Number: 2 Calc Record Number: -1
Residual:0 Reported File Number: 2 Reported Record Number: 32373
Flags: None
That, in combination with the changes I made to the position information
code in the driver, mean that even the old MTIOCGET ioctl should return an
accurate file number at end of data. e.g., on the LTO-5:
[root@doc ~]# mt ostatus
Mode Density Blocksize bpi Compression
Current: 0x58:LTO-5 variable 384607 0x1
-available modes-
0:0x58:LTO-5 variable 384607 0x1
1:0x58:LTO-5 variable 384607 0x1
2:0x58:LTO-5 variable 384607 0x1
3:0x58:LTO-5 variable 384607 0x1
-
Current Driver State: at rest.
-
File Number: 2 Record Number: -1 Residual Count -1
So the thing to try, in addition to just making sure that Bacula continues
to work properly, is to try setting this for the tape drive in
bacula-sd.conf:
Hardware End of Medium = yes
It looks like the Bacula tape program (btape) has a test mode, and it
would
be good to run through the tests on one of the tape drives and see whether
they work, and whether the results are different before and after the
changes. I'm not sure how to enable the test mode.
I have this in /usr/local/etc/bacula/bacula-sd.conf
Device {
Name= DLT
Description = QUANTUM DLT7000 1624
Media Type = DLT
Archive Device = /dev/nsa1
Autochanger = YES
Drive Index = 0
Offline On Unmount = no
Hardware End of Medium = yes
BSF at EOM = yes
Backward Space Record = no
Fast Forward Space File = no
TWO EOF = yes
}
FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape
test on this same model.
Here's the test I ran tonight:
[root@cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf
/dev/nsa1
Tape block granularity is 1024 bytes.
btape: butil.c:287-0 Using device: /dev/nsa1 for writing.
btape: btape.c:469-0 open device DLT (/dev/nsa1): OK
*test
=== Write, rewind, and re-read test ===
I'm going to write 1 records and an EOF
then write 1 records and an EOF, then rewind,
and re-read
Re: Massive libxo-zation that breaks everything
On 2015-03-01 13:49, Harrison Grundy wrote: Thanks! That does seem useful, but I'm not sure I see the reasoning behind putting into base, over a port or package, since processing XML in base is a pain, and it can't serve up JSON or HTML without additional utilities anyway. (If I'm reviving a long-settled thing, let me know and I'll drop it. I'm trying to understand the use case for this.) --- Harrison On 03/01/15 10:31, Craig Rodrigues wrote: On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy harrison.gru...@astrodoggroup.com wrote: If someone could summarize what this is, I'd greatly appreciate it. https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org I think you're missing the important bit here. This isn't about adding a parser for anything, this is about making the tools in base, like netstat, wc, uptime, etc, output in JSON or XML, so you can use the data programmatically. Your scripts no longer have to rely on awk/sed/grep magic to get a specific bit of information out of the uptime command, the command can just output the data in a structured machine readable format. I am not sure how you can put netstat into the ports tree. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: Massive libxo-zation that breaks everything
On 1 Mar 2015, at 18:49, Harrison Grundy harrison.gru...@astrodoggroup.com wrote: That does seem useful, but I'm not sure I see the reasoning behind putting into base, over a port or package, since processing XML in base is a pain, and it can't serve up JSON or HTML without additional utilities anyway. How would it be in a port? It involves modifying core utilities (some of which, like ifconfig, rely on kernel APIs that change between releases) to emit structured output. Maintaining two copies of each utility, one in the base system with plain-text output only and another in ports with XML/JSON output would be very painful. The goal of having machine-readable output in the base system is that people building systems atop FreeBSD will be able to expect a stable, machine-parsable, extensible, output from these tools. If you're building a web admin GUI or automated administration tool for FreeBSD 11, or improving integration in your favourite DE, then you should be able to rely on the output from base system tools, without having to depend on anything external. My only concern with libxo at present is that many of the modified tools are not emitting self-describing output (e.g. not specifying units for things), but that's something that we have a year to shake out before 11. David ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: sa(4) driver changes available for test
On Feb 28, 2015, at 6:42 PM, Dan Langille d...@langille.org wrote: On Feb 28, 2015, at 6:15 PM, Kenneth D. Merry k...@freebsd.org wrote: On Sat, Feb 28, 2015 at 17:29:48 -0500, Dan Langille wrote: On Feb 18, 2015, at 7:13 PM, Kenneth D. Merry k...@freebsd.org wrote: I have updated the patches. I have removed the XPT_DEV_ADVINFO changes from the patches to head, since I committed those separately. I have (hopefully) fixed the build for the stable/10 patches by MFCing dependencies. (One of them mav did for me, thanks!) Rough draft commit message: http://people.freebsd.org/~ken/sa_changes_commitmsg.20150218.1.txt I have current installed and running with Bacula, but I have not tried the tape drive yet. Thanks for all your work on this! It seems like your changes are in there from about 5 days ago. Yes, that is correct. Having solved my server hardware issues, I'm now having issues with the autochanger mechanism of the tape library. Does it work with chio(1)? Does it look like hardware or software? (If it is software, I can help with that.) Hardware. The screw drive for the tape robot is sticky. It can be moved by hand, but the motor in this DL 891 cannot. It might need lube. It was suspect last time I used it. Worst case, I will remove ins of the two DLT 8000 tape drives and load tapes manually. A bit of lube, and we have POST: https://twitter.com/DLangille/status/572117570997919744 More, soon. — Dan Langille http://langille.org/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
On Sun, 01 Mar 2015 11:26:01 -0500 Allan Jude allanj...@freebsd.org wrote: On 2015-03-01 08:33, Sulev-Madis Silber (ketas) wrote: Hello. First, I would be happy to have JSON and XML output about filesystems, users, routes... but I don't like how it makes code of df, w, netstat hard to read/maintain and often broken. I don't think it would be good to continue with this. Maybe the effort should be put to creating new layer/library and then something on top of it that actually outputs JSON and XML. Or, if that's too difficult... maybe just regular df/w/netstat could be copied to somewhere else and made code libxo-output-only. And original df/w/netstat changes reverted and left alone. Then, maybe later, df/w/netstat/... could be updated to this new layer/library. Or maybe this should be just left as it is. That would mean having two netstat's in system, which could be both good (separation) and bad (maintaining). Just some ideas... I don't know how to solve this issue fully. I'm also not likely the one who would write code for all this. Hell, those aren't even all my ideas here. I just worry that system drop-in xo-zation is bad for overall health of base. Oh and, it makes rescue larger and more complex, too? On that, there was suggestion to maybe create separate first aid kit and emergency room types of system rescue utils/methods. Thanks. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org Is there a specific bug you are running in to? So far the only bugs I've seen with the xo-ification have been ones where the JSON output was not always well formed. There was one where uptime(1) wouldn't emit any output when called in a pipeline (missing call to xo_finish, that wouldn't affect the output on a tty due to implicit flush on newline). -- Michael Gmelin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
On 2015-03-01 08:33, Sulev-Madis Silber (ketas) wrote: Hello. First, I would be happy to have JSON and XML output about filesystems, users, routes... but I don't like how it makes code of df, w, netstat hard to read/maintain and often broken. I don't think it would be good to continue with this. Maybe the effort should be put to creating new layer/library and then something on top of it that actually outputs JSON and XML. Or, if that's too difficult... maybe just regular df/w/netstat could be copied to somewhere else and made code libxo-output-only. And original df/w/netstat changes reverted and left alone. Then, maybe later, df/w/netstat/... could be updated to this new layer/library. Or maybe this should be just left as it is. That would mean having two netstat's in system, which could be both good (separation) and bad (maintaining). Just some ideas... I don't know how to solve this issue fully. I'm also not likely the one who would write code for all this. Hell, those aren't even all my ideas here. I just worry that system drop-in xo-zation is bad for overall health of base. Oh and, it makes rescue larger and more complex, too? On that, there was suggestion to maybe create separate first aid kit and emergency room types of system rescue utils/methods. Thanks. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org Is there a specific bug you are running in to? So far the only bugs I've seen with the xo-ification have been ones where the JSON output was not always well formed. -- Allan Jude signature.asc Description: OpenPGP digital signature
Re: Massive libxo-zation that breaks everything
On Sun, Mar 01, 2015 at 11:26:01AM -0500, Allan Jude wrote: On 2015-03-01 08:33, Sulev-Madis Silber (ketas) wrote: Is there a specific bug you are running in to? So far the only bugs I've seen with the xo-ification have been ones where the JSON output was not always well formed. Given the poor quality of the manpages, it does lead one to wonder about the quality of the library code. -- Steve ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
On 03/01/15 05:33, Sulev-Madis Silber (ketas) wrote: Hello. First, I would be happy to have JSON and XML output about filesystems, users, routes... but I don't like how it makes code of df, w, netstat hard to read/maintain and often broken. I don't think it would be good to continue with this. Maybe the effort should be put to creating new layer/library and then something on top of it that actually outputs JSON and XML. Or, if that's too difficult... maybe just regular df/w/netstat could be copied to somewhere else and made code libxo-output-only. And original df/w/netstat changes reverted and left alone. Then, maybe later, df/w/netstat/... could be updated to this new layer/library. Or maybe this should be just left as it is. That would mean having two netstat's in system, which could be both good (separation) and bad (maintaining). Just some ideas... I don't know how to solve this issue fully. I'm also not likely the one who would write code for all this. Hell, those aren't even all my ideas here. I just worry that system drop-in xo-zation is bad for overall health of base. Oh and, it makes rescue larger and more complex, too? On that, there was suggestion to maybe create separate first aid kit and emergency room types of system rescue utils/methods. Thanks. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org Forgive my ignorance about exactly what this is, and possibly reviving a long-settled debate, but this sounds like something that would be a great deal more useful as a port/package, rather than in base. Due to the lack of XML parsing code in -base, the difficulty in maintaining yet another interface, and the overhead involved in doing it, I don't quite see where one would really want XML output *and* be entirely opposed to ports or packages. If someone could summarize what this is, I'd greatly appreciate it. --- Harrison ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
On Mar 1, 2015, at 09:25, Harrison Grundy harrison.gru...@astrodoggroup.com wrote: Due to the lack of XML parsing code in -base, the difficulty in maintaining yet another interface, and the overhead involved in doing it, I don't quite see where one would really want XML output *and* be entirely opposed to ports or packages. You're already too late. Everything's already in base: man libbsdxml man libxo -- Rui Paulo ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy harrison.gru...@astrodoggroup.com wrote: If someone could summarize what this is, I'd greatly appreciate it. https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
Thanks! That does seem useful, but I'm not sure I see the reasoning behind putting into base, over a port or package, since processing XML in base is a pain, and it can't serve up JSON or HTML without additional utilities anyway. (If I'm reviving a long-settled thing, let me know and I'll drop it. I'm trying to understand the use case for this.) --- Harrison On 03/01/15 10:31, Craig Rodrigues wrote: On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy harrison.gru...@astrodoggroup.com wrote: If someone could summarize what this is, I'd greatly appreciate it. https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
On 03/01/15 11:11, David Chisnall wrote: On 1 Mar 2015, at 18:49, Harrison Grundy harrison.gru...@astrodoggroup.com wrote: That does seem useful, but I'm not sure I see the reasoning behind putting into base, over a port or package, since processing XML in base is a pain, and it can't serve up JSON or HTML without additional utilities anyway. How would it be in a port? It involves modifying core utilities (some of which, like ifconfig, rely on kernel APIs that change between releases) to emit structured output. Maintaining two copies of each utility, one in the base system with plain-text output only and another in ports with XML/JSON output would be very painful. The goal of having machine-readable output in the base system is that people building systems atop FreeBSD will be able to expect a stable, machine-parsable, extensible, output from these tools. If you're building a web admin GUI or automated administration tool for FreeBSD 11, or improving integration in your favourite DE, then you should be able to rely on the output from base system tools, without having to depend on anything external. My only concern with libxo at present is that many of the modified tools are not emitting self-describing output (e.g. not specifying units for things), but that's something that we have a year to shake out before 11. David Thinking about the kernel interfaces, I can see the need for it in base... but then it boils down to why, say, ifconfig in particular needs to support this behavior. It seems easier/safer to implement this independently of the existent commands using the various APIs. By putting this in the actual base utilities, an API that I would assume needs to remain relatively static is being introduced that may have some fairly large consequences for how those utilities need to operate. It just seems odd to me to have the same command one would use to, say, join a wireless network and set an IP also spit out HTML reports on interfaces. It seems like the needs of those programs may diverge over time, or that functionality needed for some new ifconfig thing would then require picking (and being stuck with) an XML schema for that information at the same time. --- Harrison ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Massive libxo-zation that breaks everything
On Mar 1, 2015, at 5:33, Sulev-Madis Silber (ketas) madis...@hot.ee wrote: Hello. First, I would be happy to have JSON and XML output about filesystems, users, routes... but I don't like how it makes code of df, w, netstat hard to read/maintain and often broken. I don't think it would be good to continue with this. Maybe the effort should be put to creating new layer/library and then something on top of it that actually outputs JSON and XML. Or, if that's too difficult... maybe just regular df/w/netstat could be copied to somewhere else and made code libxo-output-only. And original df/w/netstat changes reverted and left alone. Then, maybe later, df/w/netstat/... could be updated to this new layer/library. Or maybe this should be just left as it is. That would mean having two netstat's in system, which could be both good (separation) and bad (maintaining). Just some ideas... I don't know how to solve this issue fully. I'm also not likely the one who would write code for all this. Hell, those aren't even all my ideas here. I just worry that system drop-in xo-zation is bad for overall health of base. Oh and, it makes rescue larger and more complex, too? On that, there was suggestion to maybe create separate first aid kit and emergency room types of system rescue utils/methods. Hi Sulev, Could you please cite instances where things were broken with converting utilities (netstat seems to be a sticking point — do you have data to support this?) over to libxo? The big problem (I think) is lack of tests with these utilities to ensure that output doesn’t break unknowingly. I definitely see the wisdom in having /rescue be de-libxoified. It seems to go against the purpose of having a small “rescue image” binary — whereas the bsdxml integration into geom(3)/geom(4) is a necessary evil. Thanks! signature.asc Description: Message signed with OpenPGP using GPGMail
