Jenkins build is back to stable : FreeBSD_HEAD-tests2 #799

[jenkins-admin]

See 

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

nvidia+HEAD amd64 trouble

[Sergey V. Dyatko]

Hi,

I have Lenovo IdeaPad z400 touch laptop. It have GeForce GT 635M graphics card.
Also it have integrated Intel card
agp0:  on vgapci0
agp0: aperture size is 256M, detected 65532k stolen memory

when I enable nvidia on laptop bios and trying to use it  I got following
panic: 

NVRM: failed to copy vbios to system memory.
NVRM: RmInitAdapter failed! (0x30:0x:800)
nvidia0: NVRM: rm_init_adapter() failed!

full core.txt file located here: http://svn.freebsd.by/files/coredump-nvidia.txt
tell me what other information I need to provide

It is HEAD@r279306 amd64

[tiger@laptop]:~/tmpdir>pkg info -x nvidia
nvidia-driver-346.47
nvidia-settings-340.24_1

[tiger@laptop]:~/tmpdir>pkg info -x ^xorg
xorg-7.7_1
xorg-apps-7.7_2
xorg-cf-files-1.0.5_1
xorg-docs-1.7,1
xorg-drivers-7.7_2
xorg-fonts-7.7
xorg-fonts-100dpi-7.7
xorg-fonts-75dpi-7.7
xorg-fonts-cyrillic-7.7
xorg-fonts-miscbitmaps-7.7
xorg-fonts-truetype-7.7_1
xorg-fonts-type1-7.7
xorg-libraries-7.7_2
xorg-macros-1.19.0
xorg-server-1.14.7_2,1

I would be very appreciated for any help

--
wbr, tiger

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Jenkins build became unstable: FreeBSD_HEAD-tests2 #798

[jenkins-admin]

See 

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Alfred Perlstein]

> On Mar 2, 2015, at 7:14 PM, Julian Elischer  wrote:
> 
>> On 3/2/15 5:30 PM, Alfred Perlstein wrote:
>> 
 On Mar 2, 2015, at 4:22 PM, Andrey Chernov  wrote:
 
> On 02.03.2015 22:55, Julian Elischer wrote:
> On 3/2/15 5:27 AM, Alfred Perlstein wrote:
> 
> 
>>> On 3/2/15 4:14 AM, Julian Elischer wrote:
>>> On 3/1/15 10:49 AM, Harrison Grundy wrote:
>>> Thanks!
>>> 
>>> That does seem useful, but I'm not sure I see the reasoning behind
>>> putting into base, over a port or package, since processing XML in base
>>> is a pain, and it can't serve up JSON or HTML without additional
>>> utilities anyway.
>>> 
>>> (If I'm reviving a long-settled thing, let me know and I'll drop it.
>>> I'm
>>> trying to understand the use case for this.)
>> To me it would almost seem more useful to have a programmable filter
>> for which you could produce
>> parse grammars to parse the output of various programs..
>> thus
>> 
>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
>> with a set of grammars in /usr/share/xmlize/
>> then we could use it for out-of-tree programs as well if we wrote
>> grammars for them..
>> 
>> The sentiment of machine-readable output is nice, but I think it's
>> slightly off target.
>> we shouldn't have to change all out utilities, and it isn't going to
>> help at all with 3rd party apps,
>> e.g. samba stuff. A generally easy to program output grammar parser
>> would be truely useful.
>> and not just for FreeBSD.
>> 
>> I've been watching with an uncomfortable feeling, but it's taken me a
>> while to put my
>> finger on what it was..
> Are you sure it's not the hairs on the back of your neck standing up
> due to NIH?
> 
> Juniper has been doing this for years and it's very useful for them.
 I'm not saying the ability to generate machine readable output is wrong,
 but that the 'unix way' would be to make a filter for it. It seems that
 the noisy people don't
 agree with me so I will not stand in the way of progress..
>>> I agree. Even if someone starts with json and xml only, it will need
>>> some 3rd format soon, and adding any new format have real possibility to
>>> break all already existent (like adding json+xml breaks plain text in
>>> pipes). Moreover, it violates Unix principle 'one tool == one general
>>> function' and lots of other rules like Eric Raymond ones, making each
>>> program looks like systemd. It makes harder to merge changes from other
>>> BSDs too.
>>> Proper way to do this thing is to back out all changes and write
>>> completely separate templates-based parser - xml/json writer.
>> 
>> Read the library. It doesn't care what output format it needs. It is up to 
>> the translation layer to do it. You could even do a csv format or most any 
>> other structured output format without changing the userland utils.
> As far as I can see that's not an argument either way.
> I just think it makes more sense to spend more time writing one generic 
> converter and grammar files than to mess up the insides of every utility in 
> the system. If we had a tool, we could have grammar templates for 3rd party 
> tools easily.. do YOU want to make libxo changes to 3rd party ports? of 
> course not. so you are going off here solving a half of the problem.

Actually I want to shame third party ports into adopting libxo (or at least 
providing machine readable output). 

I know it's scary to try to lead the pack, after all we could be wrong, but 
maybe it's time to try something new and see what happens. 

And no, your idea doesn't make sense it just will lead to those files bit 
rotting.  

Bedsides that you don't even have a real spec other than "it should be done 
differently". 

Again, show the code. 


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Julian Elischer]

On 3/2/15 5:30 PM, Alfred Perlstein wrote:



On Mar 2, 2015, at 4:22 PM, Andrey Chernov  wrote:


On 02.03.2015 22:55, Julian Elischer wrote:

On 3/2/15 5:27 AM, Alfred Perlstein wrote:



On 3/2/15 4:14 AM, Julian Elischer wrote:

On 3/1/15 10:49 AM, Harrison Grundy wrote:
Thanks!

That does seem useful, but I'm not sure I see the reasoning behind
putting into base, over a port or package, since processing XML in base
is a pain, and it can't serve up JSON or HTML without additional
utilities anyway.

(If I'm reviving a long-settled thing, let me know and I'll drop it.
I'm
trying to understand the use case for this.)

To me it would almost seem more useful to have a programmable filter
for which you could produce
parse grammars to parse the output of various programs..
thus

ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
with a set of grammars in /usr/share/xmlize/
then we could use it for out-of-tree programs as well if we wrote
grammars for them..

The sentiment of machine-readable output is nice, but I think it's
slightly off target.
we shouldn't have to change all out utilities, and it isn't going to
help at all with 3rd party apps,
e.g. samba stuff. A generally easy to program output grammar parser
would be truely useful.
and not just for FreeBSD.

I've been watching with an uncomfortable feeling, but it's taken me a
while to put my
finger on what it was..

Are you sure it's not the hairs on the back of your neck standing up
due to NIH?

Juniper has been doing this for years and it's very useful for them.

I'm not saying the ability to generate machine readable output is wrong,
but that the 'unix way' would be to make a filter for it. It seems that
the noisy people don't
agree with me so I will not stand in the way of progress..

I agree. Even if someone starts with json and xml only, it will need
some 3rd format soon, and adding any new format have real possibility to
break all already existent (like adding json+xml breaks plain text in
pipes). Moreover, it violates Unix principle 'one tool == one general
function' and lots of other rules like Eric Raymond ones, making each
program looks like systemd. It makes harder to merge changes from other
BSDs too.
Proper way to do this thing is to back out all changes and write
completely separate templates-based parser - xml/json writer.


Read the library. It doesn't care what output format it needs. It is up to the 
translation layer to do it. You could even do a csv format or most any other 
structured output format without changing the userland utils.

As far as I can see that's not an argument either way.
I just think it makes more sense to spend more time writing one 
generic converter and grammar files than to mess up the insides of 
every utility in the system. If we had a tool, we could have grammar 
templates for 3rd party tools easily.. do YOU want to make libxo 
changes to 3rd party ports? of course not. so you are going off here 
solving a half of the problem.










--
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"






___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Alfred Perlstein]

> On Mar 2, 2015, at 6:27 PM, Andrey Chernov  wrote:
>> 
>> 
>> The responsibility is on you to provide something better, both the 
>> architecture AND code. So if you want it backed out, then write something 
>> better. Otherwise step back and let progress happen.
> 
> As it seems you know a lot about my responsibility and duty, I am very
> surprised by your broad interests. If you let me to speak for myself a
> bit, I can tell what I feel. In this particular case my responsibility
> is just to give good advice at the road fork with one way leads to
> obvious hell, nothing more. I already express my opinion from the
> technical point of view and don't want to participate in the flame war,
> so continue this thread without me, please.
> 

Other bad ideas (in the past) were support for threads, kernel threads, SMP, 
etc in the project's past. So maybe trying something new might be a good idea. 
At the end of the day, if in a year we think it's really terrible then it can 
be replaced, ripped out, or better yet can be fixed.  

What's worse than a bad idea is not moving on any ideas. 

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Steve Kargl]

On Mon, Mar 02, 2015 at 05:30:31PM -0800, Alfred Perlstein wrote:
> 
> Read the library. It doesn't care what output format it needs.
> It is up to the translation layer to do it. You could even do
> a csv format or most any other structured output format without
> changing the userland utils. 
> 

Yeah, read the library code because the manpages are atrocious.
There are 12 manpages apparently documenting xo_emit().

-- 
Steve

Index: xo_attr.3
===
--- xo_attr.3   (revision 279526)
+++ xo_attr.3   (working copy)
@@ -11,8 +11,8 @@
 .Dt LIBXO 3
 .Os
 .Sh NAME
-.Nm xo_emit
-.Nd emit formatted output based on format string and arguments
+.Nm xo_attr
+.Nd FUBAR
 .Sh LIBRARY
 .Lb libxo
 .Sh SYNOPSIS
Index: xo_create.3
===
--- xo_create.3 (revision 279526)
+++ xo_create.3 (working copy)
@@ -11,8 +11,8 @@
 .Dt LIBXO 3
 .Os
 .Sh NAME
-.Nm xo_emit
-.Nd emit formatted output based on format string and arguments
+.Nm xo_create
+.Nd FUBAR
 .Sh LIBRARY
 .Lb libxo
 .Sh SYNOPSIS
Index: xo_finish.3
===
--- xo_finish.3 (revision 279526)
+++ xo_finish.3 (working copy)
@@ -11,8 +11,8 @@
 .Dt LIBXO 3
 .Os
 .Sh NAME
-.Nm xo_emit
-.Nd emit formatted output based on format string and arguments
+.Nm xo_finish
+.Nd FUBAR
 .Sh LIBRARY
 .Lb libxo
 .Sh SYNOPSIS
Index: xo_flush.3
===
--- xo_flush.3  (revision 279526)
+++ xo_flush.3  (working copy)
@@ -11,8 +11,8 @@
 .Dt LIBXO 3
 .Os
 .Sh NAME
-.Nm xo_emit
-.Nd emit formatted output based on format string and arguments
+.Nm xo_flush
+.Nd FUBAR
 .Sh LIBRARY
 .Lb libxo
 .Sh SYNOPSIS
Index: xo_open_container.3
===
--- xo_open_container.3 (revision 279526)
+++ xo_open_container.3 (working copy)
@@ -11,8 +11,8 @@
 .Dt LIBXO 3
 .Os
 .Sh NAME
-.Nm xo_emit
-.Nd emit formatted output based on format string and arguments
+.Nm xo_open_container
+.Nd FUBAR
 .Sh LIBRARY
 .Lb libxo
 .Sh SYNOPSIS
Index: xo_open_list.3
===
--- xo_open_list.3  (revision 279526)
+++ xo_open_list.3  (working copy)
@@ -11,8 +11,8 @@
 .Dt LIBXO 3
 .Os
 .Sh NAME
-.Nm xo_emit
-.Nd emit formatted output based on format string and arguments
+.Nm xo_open_list
+.Nd FUBAR
 .Sh LIBRARY
 .Lb libxo
 .Sh SYNOPSIS
Index: xo_set_allocator.3
===
--- xo_set_allocator.3  (revision 279526)
+++ xo_set_allocator.3  (working copy)
@@ -11,8 +11,8 @@
 .Dt LIBXO 3
 .Os
 .Sh NAME
-.Nm xo_emit
-.Nd emit formatted output based on format string and arguments
+.Nm xo_set_allocator
+.Nd FUBAR
 .Sh LIBRARY
 .Lb libxo
 .Sh SYNOPSIS
Index: xo_set_flags.3
===
--- xo_set_flags.3  (revision 279526)
+++ xo_set_flags.3  (working copy)
@@ -11,8 +11,8 @@
 .Dt LIBXO 3
 .Os
 .Sh NAME
-.Nm xo_emit
-.Nd emit formatted output based on format string and arguments
+.Nm xo_set_flags
+.Nd FUBAR
 .Sh LIBRARY
 .Lb libxo
 .Sh SYNOPSIS
Index: xo_set_info.3
===
--- xo_set_info.3   (revision 279526)
+++ xo_set_info.3   (working copy)
@@ -11,8 +11,8 @@
 .Dt LIBXO 3
 .Os
 .Sh NAME
-.Nm xo_emit
-.Nd emit formatted output based on format string and arguments
+.Nm xo_set_info
+.Nd FUBAR
 .Sh LIBRARY
 .Lb libxo
 .Sh SYNOPSIS
Index: xo_set_style.3
===
--- xo_set_style.3  (revision 279526)
+++ xo_set_style.3  (working copy)
@@ -11,8 +11,8 @@
 .Dt LIBXO 3
 .Os
 .Sh NAME
-.Nm xo_emit
-.Nd emit formatted output based on format string and arguments
+.Nm xo_set_style
+.Nd FUBAR
 .Sh LIBRARY
 .Lb libxo
 .Sh SYNOPSIS
Index: xo_set_writer.3
===
--- xo_set_writer.3 (revision 279526)
+++ xo_set_writer.3 (working copy)
@@ -11,8 +11,8 @@
 .Dt LIBXO 3
 .Os
 .Sh NAME
-.Nm xo_emit
-.Nd emit formatted output based on format string and arguments
+.Nm xo_set_writer
+.Nd FUBAR
 .Sh LIBRARY
 .Lb libxo
 .Sh SYNOPSIS
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Andrey Chernov]

On 03.03.2015 4:45, Alfred Perlstein wrote:
> 
> 
>> On Mar 2, 2015, at 5:37 PM, Andrey Chernov  wrote:
>>
>>> On 03.03.2015 4:30, Alfred Perlstein wrote:
>>>
>>>
> On Mar 2, 2015, at 4:22 PM, Andrey Chernov  wrote:
>
>> On 02.03.2015 22:55, Julian Elischer wrote:
>> On 3/2/15 5:27 AM, Alfred Perlstein wrote:
>>
>>
 On 3/2/15 4:14 AM, Julian Elischer wrote:
 On 3/1/15 10:49 AM, Harrison Grundy wrote:
 Thanks!

 That does seem useful, but I'm not sure I see the reasoning behind
 putting into base, over a port or package, since processing XML in base
 is a pain, and it can't serve up JSON or HTML without additional
 utilities anyway.

 (If I'm reviving a long-settled thing, let me know and I'll drop it.
 I'm
 trying to understand the use case for this.)
>>>
>>> To me it would almost seem more useful to have a programmable filter
>>> for which you could produce
>>> parse grammars to parse the output of various programs..
>>> thus
>>>
>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
>>> with a set of grammars in /usr/share/xmlize/
>>> then we could use it for out-of-tree programs as well if we wrote
>>> grammars for them..
>>>
>>> The sentiment of machine-readable output is nice, but I think it's
>>> slightly off target.
>>> we shouldn't have to change all out utilities, and it isn't going to
>>> help at all with 3rd party apps,
>>> e.g. samba stuff. A generally easy to program output grammar parser
>>> would be truely useful.
>>> and not just for FreeBSD.
>>>
>>> I've been watching with an uncomfortable feeling, but it's taken me a
>>> while to put my
>>> finger on what it was..
>> Are you sure it's not the hairs on the back of your neck standing up
>> due to NIH?
>>
>> Juniper has been doing this for years and it's very useful for them.
> I'm not saying the ability to generate machine readable output is wrong,
> but that the 'unix way' would be to make a filter for it. It seems that
> the noisy people don't
> agree with me so I will not stand in the way of progress..

 I agree. Even if someone starts with json and xml only, it will need
 some 3rd format soon, and adding any new format have real possibility to
 break all already existent (like adding json+xml breaks plain text in
 pipes). Moreover, it violates Unix principle 'one tool == one general
 function' and lots of other rules like Eric Raymond ones, making each
 program looks like systemd. It makes harder to merge changes from other
 BSDs too.
 Proper way to do this thing is to back out all changes and write
 completely separate templates-based parser - xml/json writer.
>>>
>>>
>>> Read the library. It doesn't care what output format it needs. It is up to 
>>> the translation layer to do it. You could even do a csv format or most any 
>>> other structured output format without changing the userland utils.
>>
>> I am happy the library can do it. So please stop to change userland
>> utils and back out all libxo changes from them. My concern is userland
>> utils, feel free to implement anything you need/want without changing
>> them in this ugly way.
> 
> 
> The responsibility is on you to provide something better, both the 
> architecture AND code. So if you want it backed out, then write something 
> better. Otherwise step back and let progress happen. 
> 

As it seems you know a lot about my responsibility and duty, I am very
surprised by your broad interests. If you let me to speak for myself a
bit, I can tell what I feel. In this particular case my responsibility
is just to give good advice at the road fork with one way leads to
obvious hell, nothing more. I already express my opinion from the
technical point of view and don't want to participate in the flame war,
so continue this thread without me, please.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Alfred Perlstein]

> On Mar 2, 2015, at 5:37 PM, Andrey Chernov  wrote:
> 
>> On 03.03.2015 4:30, Alfred Perlstein wrote:
>> 
>> 
 On Mar 2, 2015, at 4:22 PM, Andrey Chernov  wrote:
 
> On 02.03.2015 22:55, Julian Elischer wrote:
> On 3/2/15 5:27 AM, Alfred Perlstein wrote:
> 
> 
>>> On 3/2/15 4:14 AM, Julian Elischer wrote:
>>> On 3/1/15 10:49 AM, Harrison Grundy wrote:
>>> Thanks!
>>> 
>>> That does seem useful, but I'm not sure I see the reasoning behind
>>> putting into base, over a port or package, since processing XML in base
>>> is a pain, and it can't serve up JSON or HTML without additional
>>> utilities anyway.
>>> 
>>> (If I'm reviving a long-settled thing, let me know and I'll drop it.
>>> I'm
>>> trying to understand the use case for this.)
>> 
>> To me it would almost seem more useful to have a programmable filter
>> for which you could produce
>> parse grammars to parse the output of various programs..
>> thus
>> 
>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
>> with a set of grammars in /usr/share/xmlize/
>> then we could use it for out-of-tree programs as well if we wrote
>> grammars for them..
>> 
>> The sentiment of machine-readable output is nice, but I think it's
>> slightly off target.
>> we shouldn't have to change all out utilities, and it isn't going to
>> help at all with 3rd party apps,
>> e.g. samba stuff. A generally easy to program output grammar parser
>> would be truely useful.
>> and not just for FreeBSD.
>> 
>> I've been watching with an uncomfortable feeling, but it's taken me a
>> while to put my
>> finger on what it was..
> Are you sure it's not the hairs on the back of your neck standing up
> due to NIH?
> 
> Juniper has been doing this for years and it's very useful for them.
 I'm not saying the ability to generate machine readable output is wrong,
 but that the 'unix way' would be to make a filter for it. It seems that
 the noisy people don't
 agree with me so I will not stand in the way of progress..
>>> 
>>> I agree. Even if someone starts with json and xml only, it will need
>>> some 3rd format soon, and adding any new format have real possibility to
>>> break all already existent (like adding json+xml breaks plain text in
>>> pipes). Moreover, it violates Unix principle 'one tool == one general
>>> function' and lots of other rules like Eric Raymond ones, making each
>>> program looks like systemd. It makes harder to merge changes from other
>>> BSDs too.
>>> Proper way to do this thing is to back out all changes and write
>>> completely separate templates-based parser - xml/json writer.
>> 
>> 
>> Read the library. It doesn't care what output format it needs. It is up to 
>> the translation layer to do it. You could even do a csv format or most any 
>> other structured output format without changing the userland utils.
> 
> I am happy the library can do it. So please stop to change userland
> utils and back out all libxo changes from them. My concern is userland
> utils, feel free to implement anything you need/want without changing
> them in this ugly way.


The responsibility is on you to provide something better, both the architecture 
AND code. So if you want it backed out, then write something better. Otherwise 
step back and let progress happen. 

-Alfred. 

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Andrey Chernov]

On 03.03.2015 3:48, Allan Jude wrote:
> On 2015-03-02 19:22, Andrey Chernov wrote:
>> On 02.03.2015 22:55, Julian Elischer wrote:
>>> On 3/2/15 5:27 AM, Alfred Perlstein wrote:


 On 3/2/15 4:14 AM, Julian Elischer wrote:
> On 3/1/15 10:49 AM, Harrison Grundy wrote:
>> Thanks!
>>
>> That does seem useful, but I'm not sure I see the reasoning behind
>> putting into base, over a port or package, since processing XML in base
>> is a pain, and it can't serve up JSON or HTML without additional
>> utilities anyway.
>>
>> (If I'm reviving a long-settled thing, let me know and I'll drop it.
>> I'm
>> trying to understand the use case for this.)
>
> To me it would almost seem more useful to have a programmable filter
> for which you could produce
> parse grammars to parse the output of various programs..
> thus
>
> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
> with a set of grammars in /usr/share/xmlize/
> then we could use it for out-of-tree programs as well if we wrote
> grammars for them..
>
> The sentiment of machine-readable output is nice, but I think it's
> slightly off target.
> we shouldn't have to change all out utilities, and it isn't going to
> help at all with 3rd party apps,
> e.g. samba stuff. A generally easy to program output grammar parser
> would be truely useful.
> and not just for FreeBSD.
>
> I've been watching with an uncomfortable feeling, but it's taken me a
> while to put my
> finger on what it was..
>
>
 Are you sure it's not the hairs on the back of your neck standing up
 due to NIH?

 Juniper has been doing this for years and it's very useful for them.
>>> I'm not saying the ability to generate machine readable output is wrong,
>>> but that the 'unix way' would be to make a filter for it. It seems that
>>> the noisy people don't
>>> agree with me so I will not stand in the way of progress..
>>
>> I agree. Even if someone starts with json and xml only, it will need
>> some 3rd format soon, and adding any new format have real possibility to
>> break all already existent (like adding json+xml breaks plain text in
>> pipes). Moreover, it violates Unix principle 'one tool == one general
>> function' and lots of other rules like Eric Raymond ones, making each
>> program looks like systemd. It makes harder to merge changes from other
>> BSDs too.
>> Proper way to do this thing is to back out all changes and write
>> completely separate templates-based parser - xml/json writer.
>>
> 
> Have you actually looked at libxo? It isn't really specific to xmj/json,
> and could handle adding another format without the need to modify wc
> 
> xo_open_container("wc");
> xo_open_list("file");
>   xo_open_instance("file");
> if (cnt(*argv) != 0)
>   ++errors;
>   xo_close_instance("file");
> xo_close_list("file");
> xo_close_container("wc");
> xo_finish();
> 
> 
> with cnt() being the existing functions in wc, but with the printf
> replaced with an xo wrapper. There is nothing specific to json or xml,
> that is all handled in libxo.

So, why you ever need to modify wc? Just load wc inside your
json/xml/etc writer, replacing its printf at the ld-elf.so level.

If you can't get json/xml/whatever from wc directly, it is not wc
problem, so don't touch it. The problem is on your side, feel free to
implement anything you need without affecting source code of others.

> Obviously, more care needs to be taken when converting the utilities to
> ensure that the existing output is not changed, but it is not quite as
> invasive as some people seem to think.

I think yes, it is very invasive and leads to dead end as architectural
design.

-- 
http://ache.vniz.net/



signature.asc
Description: OpenPGP digital signature

Re: Massive libxo-zation that breaks everything

[Andrey Chernov]

On 03.03.2015 4:30, Alfred Perlstein wrote:
> 
> 
>> On Mar 2, 2015, at 4:22 PM, Andrey Chernov  wrote:
>>
>>> On 02.03.2015 22:55, Julian Elischer wrote:
 On 3/2/15 5:27 AM, Alfred Perlstein wrote:


> On 3/2/15 4:14 AM, Julian Elischer wrote:
>> On 3/1/15 10:49 AM, Harrison Grundy wrote:
>> Thanks!
>>
>> That does seem useful, but I'm not sure I see the reasoning behind
>> putting into base, over a port or package, since processing XML in base
>> is a pain, and it can't serve up JSON or HTML without additional
>> utilities anyway.
>>
>> (If I'm reviving a long-settled thing, let me know and I'll drop it.
>> I'm
>> trying to understand the use case for this.)
>
> To me it would almost seem more useful to have a programmable filter
> for which you could produce
> parse grammars to parse the output of various programs..
> thus
>
> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
> with a set of grammars in /usr/share/xmlize/
> then we could use it for out-of-tree programs as well if we wrote
> grammars for them..
>
> The sentiment of machine-readable output is nice, but I think it's
> slightly off target.
> we shouldn't have to change all out utilities, and it isn't going to
> help at all with 3rd party apps,
> e.g. samba stuff. A generally easy to program output grammar parser
> would be truely useful.
> and not just for FreeBSD.
>
> I've been watching with an uncomfortable feeling, but it's taken me a
> while to put my
> finger on what it was..
 Are you sure it's not the hairs on the back of your neck standing up
 due to NIH?

 Juniper has been doing this for years and it's very useful for them.
>>> I'm not saying the ability to generate machine readable output is wrong,
>>> but that the 'unix way' would be to make a filter for it. It seems that
>>> the noisy people don't
>>> agree with me so I will not stand in the way of progress..
>>
>> I agree. Even if someone starts with json and xml only, it will need
>> some 3rd format soon, and adding any new format have real possibility to
>> break all already existent (like adding json+xml breaks plain text in
>> pipes). Moreover, it violates Unix principle 'one tool == one general
>> function' and lots of other rules like Eric Raymond ones, making each
>> program looks like systemd. It makes harder to merge changes from other
>> BSDs too.
>> Proper way to do this thing is to back out all changes and write
>> completely separate templates-based parser - xml/json writer.
> 
> 
> Read the library. It doesn't care what output format it needs. It is up to 
> the translation layer to do it. You could even do a csv format or most any 
> other structured output format without changing the userland utils. 

I am happy the library can do it. So please stop to change userland
utils and back out all libxo changes from them. My concern is userland
utils, feel free to implement anything you need/want without changing
them in this ugly way.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Alfred Perlstein]

> On Mar 2, 2015, at 4:22 PM, Andrey Chernov  wrote:
> 
>> On 02.03.2015 22:55, Julian Elischer wrote:
>>> On 3/2/15 5:27 AM, Alfred Perlstein wrote:
>>> 
>>> 
 On 3/2/15 4:14 AM, Julian Elischer wrote:
> On 3/1/15 10:49 AM, Harrison Grundy wrote:
> Thanks!
> 
> That does seem useful, but I'm not sure I see the reasoning behind
> putting into base, over a port or package, since processing XML in base
> is a pain, and it can't serve up JSON or HTML without additional
> utilities anyway.
> 
> (If I'm reviving a long-settled thing, let me know and I'll drop it.
> I'm
> trying to understand the use case for this.)
 
 To me it would almost seem more useful to have a programmable filter
 for which you could produce
 parse grammars to parse the output of various programs..
 thus
 
 ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
 with a set of grammars in /usr/share/xmlize/
 then we could use it for out-of-tree programs as well if we wrote
 grammars for them..
 
 The sentiment of machine-readable output is nice, but I think it's
 slightly off target.
 we shouldn't have to change all out utilities, and it isn't going to
 help at all with 3rd party apps,
 e.g. samba stuff. A generally easy to program output grammar parser
 would be truely useful.
 and not just for FreeBSD.
 
 I've been watching with an uncomfortable feeling, but it's taken me a
 while to put my
 finger on what it was..
>>> Are you sure it's not the hairs on the back of your neck standing up
>>> due to NIH?
>>> 
>>> Juniper has been doing this for years and it's very useful for them.
>> I'm not saying the ability to generate machine readable output is wrong,
>> but that the 'unix way' would be to make a filter for it. It seems that
>> the noisy people don't
>> agree with me so I will not stand in the way of progress..
> 
> I agree. Even if someone starts with json and xml only, it will need
> some 3rd format soon, and adding any new format have real possibility to
> break all already existent (like adding json+xml breaks plain text in
> pipes). Moreover, it violates Unix principle 'one tool == one general
> function' and lots of other rules like Eric Raymond ones, making each
> program looks like systemd. It makes harder to merge changes from other
> BSDs too.
> Proper way to do this thing is to back out all changes and write
> completely separate templates-based parser - xml/json writer.


Read the library. It doesn't care what output format it needs. It is up to the 
translation layer to do it. You could even do a csv format or most any other 
structured output format without changing the userland utils. 





> 
> -- 
> http://ache.vniz.net/
> ___
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
> 
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Mehmet Erol Sanliturk]

On Mon, Mar 2, 2015 at 3:45 PM, Alfred Perlstein  wrote:

>
>
> On 3/2/15 4:57 PM, Adrian Chadd wrote:
>
>> .. we can/should do both.
>>
>> Just make sure the json/html/xml output is versioned, otherwise you're
>> going to end up with /exactly the same problems/ you have with the
>> current format.
>>
>>  +1
>
>
> -Alfred
>
> ___
>
>


To process the generated XML or any other structured file automatically ,
at least the following three items should be present in a suitable enclosed
owner item :

generated_by_the_program = "..."
version_of_the_file = "..."
kind_of_the_file = { "Data" | "Result" | "Variables" | ... }


where "Data" for statistical analyses ,
 "Result" as "Success" or "Failure"
 "Variables" as Internal values of some variables .


By using these values of items , the rest of the file will be parsed
accordingly  .

These item values may be contained in the generated file name , but this
requires another routine to process file names to extract these information
, which is very unnecessary and does not save anything other than cause
much trouble .


Thank you very much .


Mehmet Erol Sanliturk
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Allan Jude]

On 2015-03-02 19:22, Andrey Chernov wrote:
> On 02.03.2015 22:55, Julian Elischer wrote:
>> On 3/2/15 5:27 AM, Alfred Perlstein wrote:
>>>
>>>
>>> On 3/2/15 4:14 AM, Julian Elischer wrote:
 On 3/1/15 10:49 AM, Harrison Grundy wrote:
> Thanks!
>
> That does seem useful, but I'm not sure I see the reasoning behind
> putting into base, over a port or package, since processing XML in base
> is a pain, and it can't serve up JSON or HTML without additional
> utilities anyway.
>
> (If I'm reviving a long-settled thing, let me know and I'll drop it.
> I'm
> trying to understand the use case for this.)

 To me it would almost seem more useful to have a programmable filter
 for which you could produce
 parse grammars to parse the output of various programs..
 thus

 ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
 with a set of grammars in /usr/share/xmlize/
 then we could use it for out-of-tree programs as well if we wrote
 grammars for them..

 The sentiment of machine-readable output is nice, but I think it's
 slightly off target.
 we shouldn't have to change all out utilities, and it isn't going to
 help at all with 3rd party apps,
 e.g. samba stuff. A generally easy to program output grammar parser
 would be truely useful.
 and not just for FreeBSD.

 I've been watching with an uncomfortable feeling, but it's taken me a
 while to put my
 finger on what it was..


>>> Are you sure it's not the hairs on the back of your neck standing up
>>> due to NIH?
>>>
>>> Juniper has been doing this for years and it's very useful for them.
>> I'm not saying the ability to generate machine readable output is wrong,
>> but that the 'unix way' would be to make a filter for it. It seems that
>> the noisy people don't
>> agree with me so I will not stand in the way of progress..
> 
> I agree. Even if someone starts with json and xml only, it will need
> some 3rd format soon, and adding any new format have real possibility to
> break all already existent (like adding json+xml breaks plain text in
> pipes). Moreover, it violates Unix principle 'one tool == one general
> function' and lots of other rules like Eric Raymond ones, making each
> program looks like systemd. It makes harder to merge changes from other
> BSDs too.
> Proper way to do this thing is to back out all changes and write
> completely separate templates-based parser - xml/json writer.
> 

Have you actually looked at libxo? It isn't really specific to xmj/json,
and could handle adding another format without the need to modify wc

xo_open_container("wc");
xo_open_list("file");
  xo_open_instance("file");
if (cnt(*argv) != 0)
  ++errors;
  xo_close_instance("file");
xo_close_list("file");
xo_close_container("wc");
xo_finish();


with cnt() being the existing functions in wc, but with the printf
replaced with an xo wrapper. There is nothing specific to json or xml,
that is all handled in libxo.

Obviously, more care needs to be taken when converting the utilities to
ensure that the existing output is not changed, but it is not quite as
invasive as some people seem to think.

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature

Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?

[Adrian Chadd]

On 2 March 2015 at 16:25, Konstantin Belousov  wrote:

>> Ok, but is there a specific time length that this should be?
> Difference between hold and wire is effectively that held pages are
> still kept on the page queues, providing potentially uneccessary work
> for pagedaemon to find them and skip. Wired pages are removed from the
> queues.
>
> This means that holding a page is much cheaper, by the cost of leaving
> slightly more work to the system. Also, holding a page only requires the
> page lock, while wiring contend on the page queue lock, in addition to
> the page lock.

Thanks for the description - that makes things a lot clearer!

>>
>> A DMA operation to a slow device could be up to hundreds of
>> milliseconds; or seconds if things are really backed up.
>>
>> Using wire instead of hold definitely made things work without having
>> the page disappear from underneath it. Oleksander knows more about the
>> details of that.
>
> Page cannot 'disappear'. The only thing which could happen with the
> memory page is reuse, when the page is removed from the previous object
> and re-purposed for some other object, loosing old content.
>
> Your terminology suggests that something unrelated happen.

Yup, and that's what I'm worried about :(




-adrian
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?

[Konstantin Belousov]

On Mon, Mar 02, 2015 at 04:17:03PM -0800, Adrian Chadd wrote:
> On 2 March 2015 at 16:12, K. Macy  wrote:
> >>> Right above vm_page_hold():
> >>> /*
> >>>  * Keep page from being freed by the page daemon
> >>>  * much of the same effect as wiring, except much lower
> >>>  * overhead and should be used only for *very* temporary
> >>>  * holding ("wiring").
> >>>  */
> >>
> >> What's the definition of "very temporary holding" ? What's the
> >> behavioural difference?
> >
> > Long enough to complete a DMA operation versus the lifetime of  an
> > executing program.
> 
> Ok, but is there a specific time length that this should be?
Difference between hold and wire is effectively that held pages are
still kept on the page queues, providing potentially uneccessary work
for pagedaemon to find them and skip. Wired pages are removed from the
queues.

This means that holding a page is much cheaper, by the cost of leaving
slightly more work to the system. Also, holding a page only requires the
page lock, while wiring contend on the page queue lock, in addition to
the page lock.

> 
> A DMA operation to a slow device could be up to hundreds of
> milliseconds; or seconds if things are really backed up.
> 
> Using wire instead of hold definitely made things work without having
> the page disappear from underneath it. Oleksander knows more about the
> details of that.

Page cannot 'disappear'. The only thing which could happen with the
memory page is reuse, when the page is removed from the previous object
and re-purposed for some other object, loosing old content.

Your terminology suggests that something unrelated happen.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Andrey Chernov]

On 02.03.2015 22:55, Julian Elischer wrote:
> On 3/2/15 5:27 AM, Alfred Perlstein wrote:
>>
>>
>> On 3/2/15 4:14 AM, Julian Elischer wrote:
>>> On 3/1/15 10:49 AM, Harrison Grundy wrote:
 Thanks!

 That does seem useful, but I'm not sure I see the reasoning behind
 putting into base, over a port or package, since processing XML in base
 is a pain, and it can't serve up JSON or HTML without additional
 utilities anyway.

 (If I'm reviving a long-settled thing, let me know and I'll drop it.
 I'm
 trying to understand the use case for this.)
>>>
>>> To me it would almost seem more useful to have a programmable filter
>>> for which you could produce
>>> parse grammars to parse the output of various programs..
>>> thus
>>>
>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
>>> with a set of grammars in /usr/share/xmlize/
>>> then we could use it for out-of-tree programs as well if we wrote
>>> grammars for them..
>>>
>>> The sentiment of machine-readable output is nice, but I think it's
>>> slightly off target.
>>> we shouldn't have to change all out utilities, and it isn't going to
>>> help at all with 3rd party apps,
>>> e.g. samba stuff. A generally easy to program output grammar parser
>>> would be truely useful.
>>> and not just for FreeBSD.
>>>
>>> I've been watching with an uncomfortable feeling, but it's taken me a
>>> while to put my
>>> finger on what it was..
>>>
>>>
>> Are you sure it's not the hairs on the back of your neck standing up
>> due to NIH?
>>
>> Juniper has been doing this for years and it's very useful for them.
> I'm not saying the ability to generate machine readable output is wrong,
> but that the 'unix way' would be to make a filter for it. It seems that
> the noisy people don't
> agree with me so I will not stand in the way of progress..

I agree. Even if someone starts with json and xml only, it will need
some 3rd format soon, and adding any new format have real possibility to
break all already existent (like adding json+xml breaks plain text in
pipes). Moreover, it violates Unix principle 'one tool == one general
function' and lots of other rules like Eric Raymond ones, making each
program looks like systemd. It makes harder to merge changes from other
BSDs too.
Proper way to do this thing is to back out all changes and write
completely separate templates-based parser - xml/json writer.

-- 
http://ache.vniz.net/
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?

[Adrian Chadd]

On 2 March 2015 at 16:12, K. Macy  wrote:
>>> Right above vm_page_hold():
>>> /*
>>>  * Keep page from being freed by the page daemon
>>>  * much of the same effect as wiring, except much lower
>>>  * overhead and should be used only for *very* temporary
>>>  * holding ("wiring").
>>>  */
>>
>> What's the definition of "very temporary holding" ? What's the
>> behavioural difference?
>
> Long enough to complete a DMA operation versus the lifetime of  an
> executing program.

Ok, but is there a specific time length that this should be?

A DMA operation to a slow device could be up to hundreds of
milliseconds; or seconds if things are really backed up.

Using wire instead of hold definitely made things work without having
the page disappear from underneath it. Oleksander knows more about the
details of that.



-adrian
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?

[K. Macy]

>> Right above vm_page_hold():
>> /*
>>  * Keep page from being freed by the page daemon
>>  * much of the same effect as wiring, except much lower
>>  * overhead and should be used only for *very* temporary
>>  * holding ("wiring").
>>  */
>
> What's the definition of "very temporary holding" ? What's the
> behavioural difference?


Long enough to complete a DMA operation versus the lifetime of  an
executing program.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?

[Adrian Chadd]

On 2 March 2015 at 15:05, K. Macy  wrote:
> On Mon, Mar 2, 2015 at 12:42 PM, Adrian Chadd  wrote:
>> Hi,
>>
>> gonzo@ committed a fix (r278615) to the videocore driver for the
>> raspberry pi. The fix involved doing an explicit wire of pages that
>> were about to be passed down to the hardware to send to the
>> videobuffer hardware.
>>
>> It turns out that doing vm_fault_quick_hold_pages() wasn't enough -
>> the pages weren't being wired, and they may disappear before the
>> hardware gets to them.
>
>
> Then your code is buggy or you've hit a bug in the VM. Holding a page
> is a short-term wiring.

Well, you can look at what's going on in the vchiq code. gonzo made it
an explicit wire in order to avoid issues.

> Right above vm_page_hold():
> /*
>  * Keep page from being freed by the page daemon
>  * much of the same effect as wiring, except much lower
>  * overhead and should be used only for *very* temporary
>  * holding ("wiring").
>  */

What's the definition of "very temporary holding" ? What's the
behavioural difference?



-adrian
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Alfred Perlstein]

On 3/2/15 4:57 PM, Adrian Chadd wrote:

.. we can/should do both.

Just make sure the json/html/xml output is versioned, otherwise you're
going to end up with /exactly the same problems/ you have with the
current format.


+1


-Alfred

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?

[K. Macy]

On Mon, Mar 2, 2015 at 12:42 PM, Adrian Chadd  wrote:
> Hi,
>
> gonzo@ committed a fix (r278615) to the videocore driver for the
> raspberry pi. The fix involved doing an explicit wire of pages that
> were about to be passed down to the hardware to send to the
> videobuffer hardware.
>
> It turns out that doing vm_fault_quick_hold_pages() wasn't enough -
> the pages weren't being wired, and they may disappear before the
> hardware gets to them.


Then your code is buggy or you've hit a bug in the VM. Holding a page
is a short-term wiring.

Right above vm_page_hold():
/*
 * Keep page from being freed by the page daemon
 * much of the same effect as wiring, except much lower
 * overhead and should be used only for *very* temporary
 * holding ("wiring").
 */


> I looked at vmapbuf() and how it uses vm_fault_quick_hold_pages(), but
> I can't find anything that wires the pages down before it hands the
> addresses to the hardware.
>
> So, am I missing something about how/where that's done?
Yes.

> Thanks,
>
>
> -adrian
> ___
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: sa(4) driver changes available for test

[Dan Langille]

> On Mar 2, 2015, at 4:42 PM, Kenneth D. Merry  wrote:
> 
> On Mon, Mar 02, 2015 at 16:34:34 -0500, Dan Langille wrote:
>> 
>>> On Mar 2, 2015, at 2:47 PM, Dan Langille  wrote:
>>> 
>>> 
 On Mar 2, 2015, at 2:07 PM, Dan Langille  wrote:
 
 
> On Mar 2, 2015, at 12:28 PM, Kenneth D. Merry  wrote:
> 
> On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote:
>> 
>>> On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry  wrote:
>>> 
>>> On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote:
 
> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry  wrote:
> 
> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote:
>> 
>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry  
>>> wrote:
>>> 
>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
 
> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry  
> wrote:
> 
> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
>> 
>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry 
>>>  wrote:
>>> 
>>> 
>>> I have a fairly large set of changes to the sa(4) driver and 
>>> mt(1) driver
>>> that I'm planning to commit in the near future.
>>> 
>>> A description of the changes is here and below in this message.
>>> 
>>> If you have tape hardware and the inclination, I'd appreciate 
>>> testing and
>>> feedback.
>> 
>> I have a DLT 8000 and an SDLT 220.
>> 
>> I don't have anything running current, but I have a spare 
>> machine which I could use for testing.
>> 
>> Do you see any value is tests with that hardware? I'd be testing 
>> it via Bacula.
>> 
>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula 
>> committer.
>> 
> 
> Actually, yes.  Bacula is a bit tricky to configure, so your 
> trying it out
> would be helpful if you have the time.
> 
> In looking at the manuals for both the SDLT 220 and the DLT 8000, 
> they both
> claim to support long position information for the SCSI READ 
> POSITION
> command.
> 
> You can see what I'm talking about by doing:
> 
> mt eod
> mt status
> 
> On my DDS-4 tape drive, this shows:
> 
> # mt -f /dev/nsa3 status
> Drive: sa3:  Serial Number: HJ00YWY
> -
> Mode  Density  Blocksize  bpi  Compression
> Current:  0x26:DDS-4   1024 bytes 97000enabled 
> (DCLZ)
> -
> Current Driver State: at rest.
> -
> Partition:   0  Calc File Number:  -1 Calc Record Number: 
> -1
> Residual:0  Reported File Number:  -1 Reported Record Number: 
> -1
> Flags: None
> 
> But on an LTO-5, which will give long position information, I get:
> 
> [root@doc ~]# mt status
> Drive: sa0: 
> -
> Mode  Density  Blocksize  bpi  Compression
> Current:  0x58:LTO-5   variable   384607   enabled 
> (0x1)
> -
> Current Driver State: at rest.
> -
> Partition:   0  Calc File Number:   2 Calc Record Number: 
> -1
> Residual:0  Reported File Number:   2 Reported Record Number: 
> 32373
> Flags: None
> 
> That, in combination with the changes I made to the position 
> information
> code in the driver, mean that even the old MTIOCGET ioctl should 
> return an
> accurate file number at end of data.  e.g., on the LTO-5:
> 
> [root@doc ~]# mt ostatus
> Mode  Density  Blocksize  bpi  Compression
> Current:  0x58:LTO-5   variable   384607   0x1
> -available modes-
> 0:0x58:LTO-5   variable   384607   0x1
> 1:0x58:LTO-5   variable   384607   0x1
> 2:0x58:LTO-5   variable   384607   0x1
> 3:0x58:LTO-5   variable   384607   0x1

Re: Massive libxo-zation that breaks everything

[Adrian Chadd]

.. we can/should do both.

Just make sure the json/html/xml output is versioned, otherwise you're
going to end up with /exactly the same problems/ you have with the
current format.



-adrian
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: sa(4) driver changes available for test

[Kenneth D. Merry]

On Mon, Mar 02, 2015 at 16:34:34 -0500, Dan Langille wrote:
> 
> > On Mar 2, 2015, at 2:47 PM, Dan Langille  wrote:
> > 
> > 
> >> On Mar 2, 2015, at 2:07 PM, Dan Langille  wrote:
> >> 
> >> 
> >>> On Mar 2, 2015, at 12:28 PM, Kenneth D. Merry  wrote:
> >>> 
> >>> On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote:
>  
> > On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry  wrote:
> > 
> > On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote:
> >> 
> >>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry  wrote:
> >>> 
> >>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote:
>  
> > On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry  
> > wrote:
> > 
> > On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
> >> 
> >>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry  
> >>> wrote:
> >>> 
> >>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
>  
> > On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry 
> >  wrote:
> > 
> > 
> > I have a fairly large set of changes to the sa(4) driver and 
> > mt(1) driver
> > that I'm planning to commit in the near future.
> > 
> > A description of the changes is here and below in this message.
> > 
> > If you have tape hardware and the inclination, I'd appreciate 
> > testing and
> > feedback.
>  
>  I have a DLT 8000 and an SDLT 220.
>  
>  I don't have anything running current, but I have a spare 
>  machine which I could use for testing.
>  
>  Do you see any value is tests with that hardware? I'd be testing 
>  it via Bacula.
>  
>  disclosure: I'm the sysutils/bacula-* maintainer and a Bacula 
>  committer.
>  
> >>> 
> >>> Actually, yes.  Bacula is a bit tricky to configure, so your 
> >>> trying it out
> >>> would be helpful if you have the time.
> >>> 
> >>> In looking at the manuals for both the SDLT 220 and the DLT 8000, 
> >>> they both
> >>> claim to support long position information for the SCSI READ 
> >>> POSITION
> >>> command.
> >>> 
> >>> You can see what I'm talking about by doing:
> >>> 
> >>> mt eod
> >>> mt status
> >>> 
> >>> On my DDS-4 tape drive, this shows:
> >>> 
> >>> # mt -f /dev/nsa3 status
> >>> Drive: sa3:  Serial Number: HJ00YWY
> >>> -
> >>> Mode  Density  Blocksize  bpi  Compression
> >>> Current:  0x26:DDS-4   1024 bytes 97000enabled 
> >>> (DCLZ)
> >>> -
> >>> Current Driver State: at rest.
> >>> -
> >>> Partition:   0  Calc File Number:  -1 Calc Record Number: 
> >>> -1
> >>> Residual:0  Reported File Number:  -1 Reported Record Number: 
> >>> -1
> >>> Flags: None
> >>> 
> >>> But on an LTO-5, which will give long position information, I get:
> >>> 
> >>> [root@doc ~]# mt status
> >>> Drive: sa0: 
> >>> -
> >>> Mode  Density  Blocksize  bpi  Compression
> >>> Current:  0x58:LTO-5   variable   384607   enabled 
> >>> (0x1)
> >>> -
> >>> Current Driver State: at rest.
> >>> -
> >>> Partition:   0  Calc File Number:   2 Calc Record Number: 
> >>> -1
> >>> Residual:0  Reported File Number:   2 Reported Record Number: 
> >>> 32373
> >>> Flags: None
> >>> 
> >>> That, in combination with the changes I made to the position 
> >>> information
> >>> code in the driver, mean that even the old MTIOCGET ioctl should 
> >>> return an
> >>> accurate file number at end of data.  e.g., on the LTO-5:
> >>> 
> >>> [root@doc ~]# mt ostatus
> >>> Mode  Density  Blocksize  bpi  Compression
> >>> Current:  0x58:LTO-5   variable   384607   0x1
> >>> -available modes-
> >>> 0:0x58:LTO-5   variable   384607   0x1
> >>> 1:0x58:LTO-5   variable   384607   0x1
> >>> 2:0x58:LTO-5   variable   384607   0x1
> >>> 3:0x58:LTO-5   variable   384607   0x1
> >>> -
> >>> Curr

Re: sa(4) driver changes available for test

[Dan Langille]

> On Mar 2, 2015, at 2:47 PM, Dan Langille  wrote:
> 
> 
>> On Mar 2, 2015, at 2:07 PM, Dan Langille  wrote:
>> 
>> 
>>> On Mar 2, 2015, at 12:28 PM, Kenneth D. Merry  wrote:
>>> 
>>> On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote:
 
> On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry  wrote:
> 
> On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote:
>> 
>>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry  wrote:
>>> 
>>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote:
 
> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry  wrote:
> 
> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
>> 
>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry  
>>> wrote:
>>> 
>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
 
> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry  
> wrote:
> 
> 
> I have a fairly large set of changes to the sa(4) driver and 
> mt(1) driver
> that I'm planning to commit in the near future.
> 
> A description of the changes is here and below in this message.
> 
> If you have tape hardware and the inclination, I'd appreciate 
> testing and
> feedback.
 
 I have a DLT 8000 and an SDLT 220.
 
 I don't have anything running current, but I have a spare machine 
 which I could use for testing.
 
 Do you see any value is tests with that hardware? I'd be testing 
 it via Bacula.
 
 disclosure: I'm the sysutils/bacula-* maintainer and a Bacula 
 committer.
 
>>> 
>>> Actually, yes.  Bacula is a bit tricky to configure, so your trying 
>>> it out
>>> would be helpful if you have the time.
>>> 
>>> In looking at the manuals for both the SDLT 220 and the DLT 8000, 
>>> they both
>>> claim to support long position information for the SCSI READ 
>>> POSITION
>>> command.
>>> 
>>> You can see what I'm talking about by doing:
>>> 
>>> mt eod
>>> mt status
>>> 
>>> On my DDS-4 tape drive, this shows:
>>> 
>>> # mt -f /dev/nsa3 status
>>> Drive: sa3:  Serial Number: HJ00YWY
>>> -
>>> Mode  Density  Blocksize  bpi  Compression
>>> Current:  0x26:DDS-4   1024 bytes 97000enabled 
>>> (DCLZ)
>>> -
>>> Current Driver State: at rest.
>>> -
>>> Partition:   0  Calc File Number:  -1 Calc Record Number: -1
>>> Residual:0  Reported File Number:  -1 Reported Record Number: -1
>>> Flags: None
>>> 
>>> But on an LTO-5, which will give long position information, I get:
>>> 
>>> [root@doc ~]# mt status
>>> Drive: sa0: 
>>> -
>>> Mode  Density  Blocksize  bpi  Compression
>>> Current:  0x58:LTO-5   variable   384607   enabled (0x1)
>>> -
>>> Current Driver State: at rest.
>>> -
>>> Partition:   0  Calc File Number:   2 Calc Record Number: -1
>>> Residual:0  Reported File Number:   2 Reported Record Number: 
>>> 32373
>>> Flags: None
>>> 
>>> That, in combination with the changes I made to the position 
>>> information
>>> code in the driver, mean that even the old MTIOCGET ioctl should 
>>> return an
>>> accurate file number at end of data.  e.g., on the LTO-5:
>>> 
>>> [root@doc ~]# mt ostatus
>>> Mode  Density  Blocksize  bpi  Compression
>>> Current:  0x58:LTO-5   variable   384607   0x1
>>> -available modes-
>>> 0:0x58:LTO-5   variable   384607   0x1
>>> 1:0x58:LTO-5   variable   384607   0x1
>>> 2:0x58:LTO-5   variable   384607   0x1
>>> 3:0x58:LTO-5   variable   384607   0x1
>>> -
>>> Current Driver State: at rest.
>>> -
>>> File Number: 2  Record Number: -1   Residual Count -1
>>> 
>>> So the thing to try, in addition to just making sure that Bacula 
>>> continues
>>> to work properly, is to try setting this for the tape drive in
>>> bacula-sd.conf:

Re: Massive libxo-zation that breaks everything

[Harrison Grundy]

On 03/02/15 11:55, Julian Elischer wrote:
> On 3/2/15 5:27 AM, Alfred Perlstein wrote:
>>
>>
>> On 3/2/15 4:14 AM, Julian Elischer wrote:
>>> On 3/1/15 10:49 AM, Harrison Grundy wrote:
 Thanks!

 That does seem useful, but I'm not sure I see the reasoning behind
 putting into base, over a port or package, since processing XML in base
 is a pain, and it can't serve up JSON or HTML without additional
 utilities anyway.

 (If I'm reviving a long-settled thing, let me know and I'll drop it.
 I'm
 trying to understand the use case for this.)
>>>
>>> To me it would almost seem more useful to have a programmable filter
>>> for which you could produce
>>> parse grammars to parse the output of various programs..
>>> thus
>>>
>>> ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
>>> with a set of grammars in /usr/share/xmlize/
>>> then we could use it for out-of-tree programs as well if we wrote
>>> grammars for them..
>>>
>>> The sentiment of machine-readable output is nice, but I think it's
>>> slightly off target.
>>> we shouldn't have to change all out utilities, and it isn't going to
>>> help at all with 3rd party apps,
>>> e.g. samba stuff. A generally easy to program output grammar parser
>>> would be truely useful.
>>> and not just for FreeBSD.
>>>
>>> I've been watching with an uncomfortable feeling, but it's taken me a
>>> while to put my
>>> finger on what it was..
>>>
>>>
>> Are you sure it's not the hairs on the back of your neck standing up
>> due to NIH?
>>
>> Juniper has been doing this for years and it's very useful for them.
> I'm not saying the ability to generate machine readable output is wrong,
> but that the 'unix way' would be to make a filter for it. It seems that
> the noisy people don't
> agree with me so I will not stand in the way of progress..
> 

Julian and Alfred,

Does the "lib-ification" idea address your differing concerns?

The idea is to split base binaries into a library that does the "work",
and a binary that does the UI bits.

This way, "df" becomes a CLI interface connected to "libdf" that handles
the CLI output. "xodf" (or whatever we'd like to name it) becomes a
libxo interface on top of libdf.

This allows the CLI code to remain largely unchanged, and permits the
XML output code to link across multiple utilities at the same time. (So
that things like, say, ifconfig information can be incorporated into
route and netstat results by linking to "libifconfig" "libroute" and
"libnetstat", respectively.

If we're going to have to touch every base binary anyway, it seems like
this might be a useful abstraction to get at the same time.

--- Harrison
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Doing zero-copy stuff in drivers, or "is vm_fault_quick_hold_pages() enough" ?

[Adrian Chadd]

Hi,

gonzo@ committed a fix (r278615) to the videocore driver for the
raspberry pi. The fix involved doing an explicit wire of pages that
were about to be passed down to the hardware to send to the
videobuffer hardware.

It turns out that doing vm_fault_quick_hold_pages() wasn't enough -
the pages weren't being wired, and they may disappear before the
hardware gets to them.

I looked at vmapbuf() and how it uses vm_fault_quick_hold_pages(), but
I can't find anything that wires the pages down before it hands the
addresses to the hardware.

So, am I missing something about how/where that's done?

Thanks,


-adrian
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Alfred Perlstein]

On 3/2/15 2:53 PM, Julian Elischer wrote:

On 3/2/15 5:25 AM, Alfred Perlstein wrote:



On 3/2/15 4:25 AM, David Chisnall wrote:

On 2 Mar 2015, at 09:16, Julian Elischer  wrote:
if we develop a suitable post processor with pluggable grammars, we 
save a lot of work.
given enough examples you could almost have automatically generated 
grammars.
This decoupled approach is problematic.  A large part of the point 
of libxo is to allow changing the human-readable output without 
breaking tools that consume the output.  Now I need to keep the tool 
that consumes it and the tool that produces it in sync, so that's an 
extra set of moving parts.  When you throw jails with multiple 
versions of world into the mix, it becomes a recipe for disaster.

why? the jail has it own /usr/share?




+1


 I think the risk is exactly opposite.  That the human readable output 
will change subtly with bugs in the xo implementation.
and people will not update the two output paths in exactly the same 
way, leading bugs. I'm not going to fight on it, but I am 
uncomfortable with it. 
So you mean that we're going to have to act like mature software devs 
and have regression tests (atf) and such?  I welcome such a change.



You are increasing the complexity of every program you touch.

And its utility as well.  Worth it.

-Alfred


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Julian Elischer]

On 3/2/15 5:27 AM, Alfred Perlstein wrote:



On 3/2/15 4:14 AM, Julian Elischer wrote:

On 3/1/15 10:49 AM, Harrison Grundy wrote:

Thanks!

That does seem useful, but I'm not sure I see the reasoning behind
putting into base, over a port or package, since processing XML in 
base

is a pain, and it can't serve up JSON or HTML without additional
utilities anyway.

(If I'm reviving a long-settled thing, let me know and I'll drop 
it. I'm

trying to understand the use case for this.)


To me it would almost seem more useful to have a programmable 
filter for which you could produce

parse grammars to parse the output of various programs..
thus

ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
with a set of grammars in /usr/share/xmlize/
then we could use it for out-of-tree programs as well if we wrote 
grammars for them..


The sentiment of machine-readable output is nice, but I think it's 
slightly off target.
we shouldn't have to change all out utilities, and it isn't going 
to help at all with 3rd party apps,
e.g. samba stuff. A generally easy to program output grammar parser 
would be truely useful.

and not just for FreeBSD.

I've been watching with an uncomfortable feeling, but it's taken me 
a while to put my

finger on what it was..


Are you sure it's not the hairs on the back of your neck standing up 
due to NIH?


Juniper has been doing this for years and it's very useful for them.

I'm not saying the ability to generate machine readable output is wrong,
but that the 'unix way' would be to make a filter for it. It seems 
that the noisy people don't

agree with me so I will not stand in the way of progress..




-Alfred




___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Julian Elischer]

On 3/2/15 5:25 AM, Alfred Perlstein wrote:



On 3/2/15 4:25 AM, David Chisnall wrote:

On 2 Mar 2015, at 09:16, Julian Elischer  wrote:
if we develop a suitable post processor with pluggable grammars, 
we save a lot of work.
given enough examples you could almost have automatically 
generated grammars.
This decoupled approach is problematic.  A large part of the point 
of libxo is to allow changing the human-readable output without 
breaking tools that consume the output.  Now I need to keep the 
tool that consumes it and the tool that produces it in sync, so 
that's an extra set of moving parts.  When you throw jails with 
multiple versions of world into the mix, it becomes a recipe for 
disaster.

why? the jail has it own /usr/share?




+1


 I think the risk is exactly opposite.  That the human readable 
output will change subtly with bugs in the xo implementation.
and people will not update the two output paths in exactly the same 
way, leading bugs. I'm not going to fight on it, but I am 
uncomfortable with it.  You are increasing the complexity of every 
program you touch.



___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"




___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: sa(4) driver changes available for test

[Dan Langille]

> On Mar 2, 2015, at 2:07 PM, Dan Langille  wrote:
> 
> 
>> On Mar 2, 2015, at 12:28 PM, Kenneth D. Merry  wrote:
>> 
>> On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote:
>>> 
 On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry  wrote:
 
 On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote:
> 
>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry  wrote:
>> 
>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote:
>>> 
 On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry  wrote:
 
 On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
> 
>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry  
>> wrote:
>> 
>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
>>> 
 On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry  
 wrote:
 
 
 I have a fairly large set of changes to the sa(4) driver and mt(1) 
 driver
 that I'm planning to commit in the near future.
 
 A description of the changes is here and below in this message.
 
 If you have tape hardware and the inclination, I'd appreciate 
 testing and
 feedback.
>>> 
>>> I have a DLT 8000 and an SDLT 220.
>>> 
>>> I don't have anything running current, but I have a spare machine 
>>> which I could use for testing.
>>> 
>>> Do you see any value is tests with that hardware? I'd be testing it 
>>> via Bacula.
>>> 
>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula 
>>> committer.
>>> 
>> 
>> Actually, yes.  Bacula is a bit tricky to configure, so your trying 
>> it out
>> would be helpful if you have the time.
>> 
>> In looking at the manuals for both the SDLT 220 and the DLT 8000, 
>> they both
>> claim to support long position information for the SCSI READ POSITION
>> command.
>> 
>> You can see what I'm talking about by doing:
>> 
>> mt eod
>> mt status
>> 
>> On my DDS-4 tape drive, this shows:
>> 
>> # mt -f /dev/nsa3 status
>> Drive: sa3:  Serial Number: HJ00YWY
>> -
>> Mode  Density  Blocksize  bpi  Compression
>> Current:  0x26:DDS-4   1024 bytes 97000enabled (DCLZ)
>> -
>> Current Driver State: at rest.
>> -
>> Partition:   0  Calc File Number:  -1 Calc Record Number: -1
>> Residual:0  Reported File Number:  -1 Reported Record Number: -1
>> Flags: None
>> 
>> But on an LTO-5, which will give long position information, I get:
>> 
>> [root@doc ~]# mt status
>> Drive: sa0: 
>> -
>> Mode  Density  Blocksize  bpi  Compression
>> Current:  0x58:LTO-5   variable   384607   enabled (0x1)
>> -
>> Current Driver State: at rest.
>> -
>> Partition:   0  Calc File Number:   2 Calc Record Number: -1
>> Residual:0  Reported File Number:   2 Reported Record Number: 
>> 32373
>> Flags: None
>> 
>> That, in combination with the changes I made to the position 
>> information
>> code in the driver, mean that even the old MTIOCGET ioctl should 
>> return an
>> accurate file number at end of data.  e.g., on the LTO-5:
>> 
>> [root@doc ~]# mt ostatus
>> Mode  Density  Blocksize  bpi  Compression
>> Current:  0x58:LTO-5   variable   384607   0x1
>> -available modes-
>> 0:0x58:LTO-5   variable   384607   0x1
>> 1:0x58:LTO-5   variable   384607   0x1
>> 2:0x58:LTO-5   variable   384607   0x1
>> 3:0x58:LTO-5   variable   384607   0x1
>> -
>> Current Driver State: at rest.
>> -
>> File Number: 2  Record Number: -1   Residual Count -1
>> 
>> So the thing to try, in addition to just making sure that Bacula 
>> continues
>> to work properly, is to try setting this for the tape drive in
>> bacula-sd.conf:
>> 
>> Hardware End of Medium = yes
>> 
>> It looks like the Bacula tape program (btape) has a test mode, and 
>> it would
>> be good to run through the te

Re: sa(4) driver changes available for test

[Dan Langille]

> On Mar 2, 2015, at 12:28 PM, Kenneth D. Merry  wrote:
> 
> On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote:
>> 
>>> On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry  wrote:
>>> 
>>> On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote:
 
> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry  wrote:
> 
> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote:
>> 
>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry  wrote:
>>> 
>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
 
> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry  
> wrote:
> 
> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
>> 
>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry  
>>> wrote:
>>> 
>>> 
>>> I have a fairly large set of changes to the sa(4) driver and mt(1) 
>>> driver
>>> that I'm planning to commit in the near future.
>>> 
>>> A description of the changes is here and below in this message.
>>> 
>>> If you have tape hardware and the inclination, I'd appreciate 
>>> testing and
>>> feedback.
>> 
>> I have a DLT 8000 and an SDLT 220.
>> 
>> I don't have anything running current, but I have a spare machine 
>> which I could use for testing.
>> 
>> Do you see any value is tests with that hardware? I'd be testing it 
>> via Bacula.
>> 
>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula 
>> committer.
>> 
> 
> Actually, yes.  Bacula is a bit tricky to configure, so your trying 
> it out
> would be helpful if you have the time.
> 
> In looking at the manuals for both the SDLT 220 and the DLT 8000, 
> they both
> claim to support long position information for the SCSI READ POSITION
> command.
> 
> You can see what I'm talking about by doing:
> 
> mt eod
> mt status
> 
> On my DDS-4 tape drive, this shows:
> 
> # mt -f /dev/nsa3 status
> Drive: sa3:  Serial Number: HJ00YWY
> -
> Mode  Density  Blocksize  bpi  Compression
> Current:  0x26:DDS-4   1024 bytes 97000enabled (DCLZ)
> -
> Current Driver State: at rest.
> -
> Partition:   0  Calc File Number:  -1 Calc Record Number: -1
> Residual:0  Reported File Number:  -1 Reported Record Number: -1
> Flags: None
> 
> But on an LTO-5, which will give long position information, I get:
> 
> [root@doc ~]# mt status
> Drive: sa0: 
> -
> Mode  Density  Blocksize  bpi  Compression
> Current:  0x58:LTO-5   variable   384607   enabled (0x1)
> -
> Current Driver State: at rest.
> -
> Partition:   0  Calc File Number:   2 Calc Record Number: -1
> Residual:0  Reported File Number:   2 Reported Record Number: 
> 32373
> Flags: None
> 
> That, in combination with the changes I made to the position 
> information
> code in the driver, mean that even the old MTIOCGET ioctl should 
> return an
> accurate file number at end of data.  e.g., on the LTO-5:
> 
> [root@doc ~]# mt ostatus
> Mode  Density  Blocksize  bpi  Compression
> Current:  0x58:LTO-5   variable   384607   0x1
> -available modes-
> 0:0x58:LTO-5   variable   384607   0x1
> 1:0x58:LTO-5   variable   384607   0x1
> 2:0x58:LTO-5   variable   384607   0x1
> 3:0x58:LTO-5   variable   384607   0x1
> -
> Current Driver State: at rest.
> -
> File Number: 2  Record Number: -1   Residual Count -1
> 
> So the thing to try, in addition to just making sure that Bacula 
> continues
> to work properly, is to try setting this for the tape drive in
> bacula-sd.conf:
> 
> Hardware End of Medium = yes
> 
> It looks like the Bacula tape program (btape) has a test mode, and it 
> would
> be good to run through the tests on one of the tape drives and see 
> whether
> they work, and whether the results are different before and after the
> changes.  I'm not sure how to

Re: sa(4) driver changes available for test

[Kenneth D. Merry]

On Mon, Mar 02, 2015 at 11:45:59 -0500, Dan Langille wrote:
> 
> > On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry  wrote:
> > 
> > 
> > I have a fairly large set of changes to the sa(4) driver and mt(1) driver
> > that I'm planning to commit in the near future.
> > 
> > A description of the changes is here and below in this message.
> > 
> > If you have tape hardware and the inclination, I'd appreciate testing and
> > feedback.
> 
> This came to me today via the Bacula mailing lists.
> 
> http://marc.info/?l=bacula-users&m=142531236722693&w=2
> 
> > As far as I can tell ltfs support on linux sits on top of the standard 
> > mt-st stuff \
> > as a userspace (fuse) filesystem 
> > I'd hope it's much the same with BSD. Removing the standard interface would 
> > be \
> > counterproductive overall
> 
> Can you answer that and I'll relay please?

Sure.  In short, the current interface will stay in place.  I have added
additional ioctls that provide more features and information, but I don't
see any issue with leaving the current ioctls in place.

The MTIOCGET ioctl even gets an improvement in behavior when the tape drive
supports long position information -- it will report the file number after
a 'mt eod'.

IBM's LTFS sits on top of their own Linux tape driver, and operates with
a combination of tape driver ioctls (e.g. the standard MTIOTCOP ioctls)
and SCSI passthrough.

When I ported it to FreeBSD, I ran into several areas where we needed
more information out of the tape driver.  So that was the primary
motivation behind adding the additional features.  (Other features I
implemented using SCSI passthrough.)

He is correct that it runs with FUSE, although it can be linked into an
application as a library as well.

Ken
-- 
Kenneth Merry
k...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: sa(4) driver changes available for test

[Kenneth D. Merry]

On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote:
> 
> > On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry  wrote:
> > 
> > On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote:
> >> 
> >>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry  wrote:
> >>> 
> >>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote:
>  
> > On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry  wrote:
> > 
> > On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
> >> 
> >>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry  
> >>> wrote:
> >>> 
> >>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
>  
> > On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry  
> > wrote:
> > 
> > 
> > I have a fairly large set of changes to the sa(4) driver and mt(1) 
> > driver
> > that I'm planning to commit in the near future.
> > 
> > A description of the changes is here and below in this message.
> > 
> > If you have tape hardware and the inclination, I'd appreciate 
> > testing and
> > feedback.
>  
>  I have a DLT 8000 and an SDLT 220.
>  
>  I don't have anything running current, but I have a spare machine 
>  which I could use for testing.
>  
>  Do you see any value is tests with that hardware? I'd be testing it 
>  via Bacula.
>  
>  disclosure: I'm the sysutils/bacula-* maintainer and a Bacula 
>  committer.
>  
> >>> 
> >>> Actually, yes.  Bacula is a bit tricky to configure, so your trying 
> >>> it out
> >>> would be helpful if you have the time.
> >>> 
> >>> In looking at the manuals for both the SDLT 220 and the DLT 8000, 
> >>> they both
> >>> claim to support long position information for the SCSI READ POSITION
> >>> command.
> >>> 
> >>> You can see what I'm talking about by doing:
> >>> 
> >>> mt eod
> >>> mt status
> >>> 
> >>> On my DDS-4 tape drive, this shows:
> >>> 
> >>> # mt -f /dev/nsa3 status
> >>> Drive: sa3:  Serial Number: HJ00YWY
> >>> -
> >>> Mode  Density  Blocksize  bpi  Compression
> >>> Current:  0x26:DDS-4   1024 bytes 97000enabled (DCLZ)
> >>> -
> >>> Current Driver State: at rest.
> >>> -
> >>> Partition:   0  Calc File Number:  -1 Calc Record Number: -1
> >>> Residual:0  Reported File Number:  -1 Reported Record Number: -1
> >>> Flags: None
> >>> 
> >>> But on an LTO-5, which will give long position information, I get:
> >>> 
> >>> [root@doc ~]# mt status
> >>> Drive: sa0: 
> >>> -
> >>> Mode  Density  Blocksize  bpi  Compression
> >>> Current:  0x58:LTO-5   variable   384607   enabled (0x1)
> >>> -
> >>> Current Driver State: at rest.
> >>> -
> >>> Partition:   0  Calc File Number:   2 Calc Record Number: -1
> >>> Residual:0  Reported File Number:   2 Reported Record Number: 
> >>> 32373
> >>> Flags: None
> >>> 
> >>> That, in combination with the changes I made to the position 
> >>> information
> >>> code in the driver, mean that even the old MTIOCGET ioctl should 
> >>> return an
> >>> accurate file number at end of data.  e.g., on the LTO-5:
> >>> 
> >>> [root@doc ~]# mt ostatus
> >>> Mode  Density  Blocksize  bpi  Compression
> >>> Current:  0x58:LTO-5   variable   384607   0x1
> >>> -available modes-
> >>> 0:0x58:LTO-5   variable   384607   0x1
> >>> 1:0x58:LTO-5   variable   384607   0x1
> >>> 2:0x58:LTO-5   variable   384607   0x1
> >>> 3:0x58:LTO-5   variable   384607   0x1
> >>> -
> >>> Current Driver State: at rest.
> >>> -
> >>> File Number: 2  Record Number: -1   Residual Count -1
> >>> 
> >>> So the thing to try, in addition to just making sure that Bacula 
> >>> continues
> >>> to work properly, is to try setting this for the tape drive in
> >>> bacula-sd.conf:
> >>> 
> >>> Hardware End of Medium = yes
> >>> 
> >>> It looks like the Bacula tape program (btape) has a test mode, and it 
> >>> would
> >>> be good to run through the tests on one of the tape drives and see 
> >>> whether
> >>> they work, and whether the results are different before and after the
> >>> changes.  I'm not sure how to enable the test mode.
> >> 
> >> I have this in /usr/l

Re: sa(4) driver changes available for test

[Kenneth D. Merry]

On Mon, Mar 02, 2015 at 11:43:15 -0500, Dan Langille wrote:
> 
> > On Mar 1, 2015, at 9:06 PM, Kenneth D. Merry  wrote:
> > 
> > On Sun, Mar 01, 2015 at 19:40:40 -0500, Dan Langille wrote:
> >> 
> >>> On Mar 1, 2015, at 7:36 PM, Kenneth D. Merry  wrote:
> >>> 
> >>> On Sun, Mar 01, 2015 at 19:28:37 -0500, Dan Langille wrote:
>  
> > On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry  wrote:
> > 
> > On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote:
> >> 
> >>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry  
> >>> wrote:
> >>> 
> >>> 
> >>> I have a fairly large set of changes to the sa(4) driver and mt(1) 
> >>> driver
> >>> that I'm planning to commit in the near future.
> >>> 
> >>> A description of the changes is here and below in this message.
> >>> 
> >>> If you have tape hardware and the inclination, I'd appreciate testing 
> >>> and
> >>> feedback.
> >>> 
> >>> 
> >>> Rough draft commit message:
> >>> 
> >>> http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt
> >>> 
> >>> The patches against FreeBSD/head as of SVN revision 278706:
> >>> 
> >>> http://people.freebsd.org/~ken/sa_changes.20150213.3.txt
> >>> 
> >>> And (untested) patches against FreeBSD stable/10 as of SVN revision 
> >>> 278721.
> >>> 
> >>> http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt
> >>> 
> >>> 
> >>> The intent is to get the tape infrastructure more up to date, so we 
> >>> can
> >>> support LTFS and more modern tape drives:
> >>> 
> >>> http://www.ibm.com/systems/storage/tape/ltfs/
> >>> 
> >>> I have ported IBM's LTFS Single Drive Edition to FreeBSD.  The port 
> >>> depends
> >>> on the patches linked above.  It isn't fully cleaned up and ready for
> >>> redistribution.  If you're interested, though, let me know and I'll 
> >>> tell
> >>> you when it is ready to go out.  You need an IBM LTO-5, LTO-6, TS1140 
> >>> or
> >>> TS1150 tape drive.  HP drives aren't supported by IBM's LTFS, and 
> >>> older
> >>> drives don't have the necessary features to support LTFS.
> >>> 
> >>> The commit message below outlines most of the changes.
> >>> 
> >>> A few comments:
> >>> 
> >>> 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately.
> >>> 
> >>> 2. The XML output is similar to what GEOM and CTL do.  It would be 
> >>> nice to
> >>> figure out how to put a standard schema on it so that standard tools
> >>> could read it.  I don't know how feasible that is, since I haven't
> >>> time to dig into it.  If anyone has suggestions on whether that is
> >>> feasible or advisable, I'd appreciate feedback.
> >>> 
> >>> 3. I have tested with a reasonable amount of tape hardware (see below 
> >>> for a
> >>> list), but more testing and feedback would be good.
> >>> 
> >>> 4. Standard 'mt status' output looks like this:
> >>> 
> >>> # mt -f /dev/nsa3 status  -v
> >>> Drive: sa3:  Serial Number: 101500520A
> >>> -
> >>> Mode  Density  Blocksize  bpi  Compression
> >>> Current:  0x5a:LTO-6   variable   384607   enabled (0xff)
> >>> -
> >>> Current Driver State: at rest.
> >>> -
> >>> Partition:   0  Calc File Number:   0 Calc Record Number: 0
> >>> Residual:0  Reported File Number:   0 Reported Record Number: 0
> >>> Flags: BOP
> >>> 
> >>> 5. 'mt status -v' looks like this:
> >>> 
> >>> # mt -f /dev/nsa3 status  -v
> >>> Drive: sa3:  Serial Number: 101500520A
> >>> -
> >>> Mode  Density  Blocksize  bpi  Compression
> >>> Current:  0x5a:LTO-6   variable   384607   enabled (0xff)
> >>> -
> >>> Current Driver State: at rest.
> >>> -
> >>> Partition:   0  Calc File Number:   0 Calc Record Number: 0
> >>> Residual:0  Reported File Number:   0 Reported Record Number: 0
> >>> Flags: BOP
> >>> -
> >>> Tape I/O parameters:
> >>> Maximum I/O size allowed by driver and controller (maxio): 1081344 
> >>> bytes
> >>> Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes
> >>> Maximum block size supported by tape drive and media (max_blk): 
> >>> 8388608 bytes
> >>> Minimum block size supported by tape drive and media (min_blk): 1 
> >>> bytes
> >>> Block granularity supported by tape drive and media (blk_gran): 0 
> >>> bytes
> >>> Maximum possible I/O size (max_effective_iosize): 1081344 bytes
> >> 
> >> 
> >> # mtx -f /dev/pass

Re: sa(4) driver changes available for test

[Dan Langille]

> On Mar 1, 2015, at 9:06 PM, Kenneth D. Merry  wrote:
> 
> On Sun, Mar 01, 2015 at 19:40:40 -0500, Dan Langille wrote:
>> 
>>> On Mar 1, 2015, at 7:36 PM, Kenneth D. Merry  wrote:
>>> 
>>> On Sun, Mar 01, 2015 at 19:28:37 -0500, Dan Langille wrote:
 
> On Mar 1, 2015, at 7:18 PM, Kenneth D. Merry  wrote:
> 
> On Sun, Mar 01, 2015 at 17:06:24 -0500, Dan Langille wrote:
>> 
>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry  wrote:
>>> 
>>> 
>>> I have a fairly large set of changes to the sa(4) driver and mt(1) 
>>> driver
>>> that I'm planning to commit in the near future.
>>> 
>>> A description of the changes is here and below in this message.
>>> 
>>> If you have tape hardware and the inclination, I'd appreciate testing 
>>> and
>>> feedback.
>>> 
>>> 
>>> Rough draft commit message:
>>> 
>>> http://people.freebsd.org/~ken/sa_changes_commitmsg.20150213.3.txt
>>> 
>>> The patches against FreeBSD/head as of SVN revision 278706:
>>> 
>>> http://people.freebsd.org/~ken/sa_changes.20150213.3.txt
>>> 
>>> And (untested) patches against FreeBSD stable/10 as of SVN revision 
>>> 278721.
>>> 
>>> http://people.freebsd.org/~ken/sa_changes.stable_10.20150213.3.txt
>>> 
>>> 
>>> The intent is to get the tape infrastructure more up to date, so we can
>>> support LTFS and more modern tape drives:
>>> 
>>> http://www.ibm.com/systems/storage/tape/ltfs/
>>> 
>>> I have ported IBM's LTFS Single Drive Edition to FreeBSD.  The port 
>>> depends
>>> on the patches linked above.  It isn't fully cleaned up and ready for
>>> redistribution.  If you're interested, though, let me know and I'll tell
>>> you when it is ready to go out.  You need an IBM LTO-5, LTO-6, TS1140 or
>>> TS1150 tape drive.  HP drives aren't supported by IBM's LTFS, and older
>>> drives don't have the necessary features to support LTFS.
>>> 
>>> The commit message below outlines most of the changes.
>>> 
>>> A few comments:
>>> 
>>> 1. I'm planning to commit the XPT_DEV_ADVINFO changes separately.
>>> 
>>> 2. The XML output is similar to what GEOM and CTL do.  It would be nice 
>>> to
>>> figure out how to put a standard schema on it so that standard tools
>>> could read it.  I don't know how feasible that is, since I haven't
>>> time to dig into it.  If anyone has suggestions on whether that is
>>> feasible or advisable, I'd appreciate feedback.
>>> 
>>> 3. I have tested with a reasonable amount of tape hardware (see below 
>>> for a
>>> list), but more testing and feedback would be good.
>>> 
>>> 4. Standard 'mt status' output looks like this:
>>> 
>>> # mt -f /dev/nsa3 status  -v
>>> Drive: sa3:  Serial Number: 101500520A
>>> -
>>> Mode  Density  Blocksize  bpi  Compression
>>> Current:  0x5a:LTO-6   variable   384607   enabled (0xff)
>>> -
>>> Current Driver State: at rest.
>>> -
>>> Partition:   0  Calc File Number:   0 Calc Record Number: 0
>>> Residual:0  Reported File Number:   0 Reported Record Number: 0
>>> Flags: BOP
>>> 
>>> 5. 'mt status -v' looks like this:
>>> 
>>> # mt -f /dev/nsa3 status  -v
>>> Drive: sa3:  Serial Number: 101500520A
>>> -
>>> Mode  Density  Blocksize  bpi  Compression
>>> Current:  0x5a:LTO-6   variable   384607   enabled (0xff)
>>> -
>>> Current Driver State: at rest.
>>> -
>>> Partition:   0  Calc File Number:   0 Calc Record Number: 0
>>> Residual:0  Reported File Number:   0 Reported Record Number: 0
>>> Flags: BOP
>>> -
>>> Tape I/O parameters:
>>> Maximum I/O size allowed by driver and controller (maxio): 1081344 bytes
>>> Maximum I/O size reported by controller (cpi_maxio): 5197824 bytes
>>> Maximum block size supported by tape drive and media (max_blk): 8388608 
>>> bytes
>>> Minimum block size supported by tape drive and media (min_blk): 1 bytes
>>> Block granularity supported by tape drive and media (blk_gran): 0 bytes
>>> Maximum possible I/O size (max_effective_iosize): 1081344 bytes
>> 
>> 
>> # mtx -f /dev/pass0 status
>> Storage Changer /dev/pass0:2 Drives, 10 Slots ( 0 Import/Export )
>> Data Transfer Element 0:Empty
>> Data Transfer Element 1:Empty
>>Storage Element 1:Empty
>>Storage Element 2:Empty
>>Storage Element 3:Empty
>>Storage Element 4:Full :VolumeTag=FAI260  
>>Storage Element 5:Full 

Re: sa(4) driver changes available for test

[Dan Langille]

> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry  wrote:
> 
> 
> I have a fairly large set of changes to the sa(4) driver and mt(1) driver
> that I'm planning to commit in the near future.
> 
> A description of the changes is here and below in this message.
> 
> If you have tape hardware and the inclination, I'd appreciate testing and
> feedback.

This came to me today via the Bacula mailing lists.

http://marc.info/?l=bacula-users&m=142531236722693&w=2

> As far as I can tell ltfs support on linux sits on top of the standard mt-st 
> stuff \
> as a userspace (fuse) filesystem 
> I'd hope it's much the same with BSD. Removing the standard interface would 
> be \
> counterproductive overall

Can you answer that and I'll relay please?

— 
Dan Langille
http://langille.org/





___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: sa(4) driver changes available for test

[Dan Langille]

> On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry  wrote:
> 
> On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote:
>> 
>>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry  wrote:
>>> 
>>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote:
 
> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry  wrote:
> 
> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
>> 
>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry  wrote:
>>> 
>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
 
> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry  
> wrote:
> 
> 
> I have a fairly large set of changes to the sa(4) driver and mt(1) 
> driver
> that I'm planning to commit in the near future.
> 
> A description of the changes is here and below in this message.
> 
> If you have tape hardware and the inclination, I'd appreciate testing 
> and
> feedback.
 
 I have a DLT 8000 and an SDLT 220.
 
 I don't have anything running current, but I have a spare machine 
 which I could use for testing.
 
 Do you see any value is tests with that hardware? I'd be testing it 
 via Bacula.
 
 disclosure: I'm the sysutils/bacula-* maintainer and a Bacula 
 committer.
 
>>> 
>>> Actually, yes.  Bacula is a bit tricky to configure, so your trying it 
>>> out
>>> would be helpful if you have the time.
>>> 
>>> In looking at the manuals for both the SDLT 220 and the DLT 8000, they 
>>> both
>>> claim to support long position information for the SCSI READ POSITION
>>> command.
>>> 
>>> You can see what I'm talking about by doing:
>>> 
>>> mt eod
>>> mt status
>>> 
>>> On my DDS-4 tape drive, this shows:
>>> 
>>> # mt -f /dev/nsa3 status
>>> Drive: sa3:  Serial Number: HJ00YWY
>>> -
>>> Mode  Density  Blocksize  bpi  Compression
>>> Current:  0x26:DDS-4   1024 bytes 97000enabled (DCLZ)
>>> -
>>> Current Driver State: at rest.
>>> -
>>> Partition:   0  Calc File Number:  -1 Calc Record Number: -1
>>> Residual:0  Reported File Number:  -1 Reported Record Number: -1
>>> Flags: None
>>> 
>>> But on an LTO-5, which will give long position information, I get:
>>> 
>>> [root@doc ~]# mt status
>>> Drive: sa0: 
>>> -
>>> Mode  Density  Blocksize  bpi  Compression
>>> Current:  0x58:LTO-5   variable   384607   enabled (0x1)
>>> -
>>> Current Driver State: at rest.
>>> -
>>> Partition:   0  Calc File Number:   2 Calc Record Number: -1
>>> Residual:0  Reported File Number:   2 Reported Record Number: 32373
>>> Flags: None
>>> 
>>> That, in combination with the changes I made to the position information
>>> code in the driver, mean that even the old MTIOCGET ioctl should return 
>>> an
>>> accurate file number at end of data.  e.g., on the LTO-5:
>>> 
>>> [root@doc ~]# mt ostatus
>>> Mode  Density  Blocksize  bpi  Compression
>>> Current:  0x58:LTO-5   variable   384607   0x1
>>> -available modes-
>>> 0:0x58:LTO-5   variable   384607   0x1
>>> 1:0x58:LTO-5   variable   384607   0x1
>>> 2:0x58:LTO-5   variable   384607   0x1
>>> 3:0x58:LTO-5   variable   384607   0x1
>>> -
>>> Current Driver State: at rest.
>>> -
>>> File Number: 2  Record Number: -1   Residual Count -1
>>> 
>>> So the thing to try, in addition to just making sure that Bacula 
>>> continues
>>> to work properly, is to try setting this for the tape drive in
>>> bacula-sd.conf:
>>> 
>>> Hardware End of Medium = yes
>>> 
>>> It looks like the Bacula tape program (btape) has a test mode, and it 
>>> would
>>> be good to run through the tests on one of the tape drives and see 
>>> whether
>>> they work, and whether the results are different before and after the
>>> changes.  I'm not sure how to enable the test mode.
>> 
>> I have this in /usr/local/etc/bacula/bacula-sd.conf
>> 
>> Device {
>> Name= DLT
>> Description = "QUANTUM DLT7000 1624"
>> Media Type  = DLT
>> Archive Device  = /dev/nsa1
>> 
>> Autochanger = YES
>> Drive Index = 0
>> 
>> Offlin

Re: sa(4) driver changes available for test

[Kenneth D. Merry]

On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote:
> 
> > On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry  wrote:
> > 
> > On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote:
> >> 
> >>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry  wrote:
> >>> 
> >>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
>  
> > On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry  wrote:
> > 
> > On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
> >> 
> >>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry  
> >>> wrote:
> >>> 
> >>> 
> >>> I have a fairly large set of changes to the sa(4) driver and mt(1) 
> >>> driver
> >>> that I'm planning to commit in the near future.
> >>> 
> >>> A description of the changes is here and below in this message.
> >>> 
> >>> If you have tape hardware and the inclination, I'd appreciate testing 
> >>> and
> >>> feedback.
> >> 
> >> I have a DLT 8000 and an SDLT 220.
> >> 
> >> I don't have anything running current, but I have a spare machine 
> >> which I could use for testing.
> >> 
> >> Do you see any value is tests with that hardware? I'd be testing it 
> >> via Bacula.
> >> 
> >> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula 
> >> committer.
> >> 
> > 
> > Actually, yes.  Bacula is a bit tricky to configure, so your trying it 
> > out
> > would be helpful if you have the time.
> > 
> > In looking at the manuals for both the SDLT 220 and the DLT 8000, they 
> > both
> > claim to support long position information for the SCSI READ POSITION
> > command.
> > 
> > You can see what I'm talking about by doing:
> > 
> > mt eod
> > mt status
> > 
> > On my DDS-4 tape drive, this shows:
> > 
> > # mt -f /dev/nsa3 status
> > Drive: sa3:  Serial Number: HJ00YWY
> > -
> > Mode  Density  Blocksize  bpi  Compression
> > Current:  0x26:DDS-4   1024 bytes 97000enabled (DCLZ)
> > -
> > Current Driver State: at rest.
> > -
> > Partition:   0  Calc File Number:  -1 Calc Record Number: -1
> > Residual:0  Reported File Number:  -1 Reported Record Number: -1
> > Flags: None
> > 
> > But on an LTO-5, which will give long position information, I get:
> > 
> > [root@doc ~]# mt status
> > Drive: sa0: 
> > -
> > Mode  Density  Blocksize  bpi  Compression
> > Current:  0x58:LTO-5   variable   384607   enabled (0x1)
> > -
> > Current Driver State: at rest.
> > -
> > Partition:   0  Calc File Number:   2 Calc Record Number: -1
> > Residual:0  Reported File Number:   2 Reported Record Number: 32373
> > Flags: None
> > 
> > That, in combination with the changes I made to the position information
> > code in the driver, mean that even the old MTIOCGET ioctl should return 
> > an
> > accurate file number at end of data.  e.g., on the LTO-5:
> > 
> > [root@doc ~]# mt ostatus
> > Mode  Density  Blocksize  bpi  Compression
> > Current:  0x58:LTO-5   variable   384607   0x1
> > -available modes-
> > 0:0x58:LTO-5   variable   384607   0x1
> > 1:0x58:LTO-5   variable   384607   0x1
> > 2:0x58:LTO-5   variable   384607   0x1
> > 3:0x58:LTO-5   variable   384607   0x1
> > -
> > Current Driver State: at rest.
> > -
> > File Number: 2  Record Number: -1   Residual Count -1
> > 
> > So the thing to try, in addition to just making sure that Bacula 
> > continues
> > to work properly, is to try setting this for the tape drive in
> > bacula-sd.conf:
> > 
> > Hardware End of Medium = yes
> > 
> > It looks like the Bacula tape program (btape) has a test mode, and it 
> > would
> > be good to run through the tests on one of the tape drives and see 
> > whether
> > they work, and whether the results are different before and after the
> > changes.  I'm not sure how to enable the test mode.
>  
>  I have this in /usr/local/etc/bacula/bacula-sd.conf
>  
>  Device {
>  Name= DLT
>  Description = "QUANTUM DLT7000 1624"
>  Media Type  = DLT
>  Archive Device  = /dev/nsa1
>  
>  Autochanger = YES
>  Drive Index = 0
>  
>  Offline On Unmount  = no
>  Hardware End of Medium  = yes
> >

Re: sa(4) driver changes available for test

[Dan Langille]

> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry  wrote:
> 
> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote:
>> 
>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry  wrote:
>>> 
>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
 
> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry  wrote:
> 
> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
>> 
>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry  wrote:
>>> 
>>> 
>>> I have a fairly large set of changes to the sa(4) driver and mt(1) 
>>> driver
>>> that I'm planning to commit in the near future.
>>> 
>>> A description of the changes is here and below in this message.
>>> 
>>> If you have tape hardware and the inclination, I'd appreciate testing 
>>> and
>>> feedback.
>> 
>> I have a DLT 8000 and an SDLT 220.
>> 
>> I don't have anything running current, but I have a spare machine which 
>> I could use for testing.
>> 
>> Do you see any value is tests with that hardware? I'd be testing it via 
>> Bacula.
>> 
>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer.
>> 
> 
> Actually, yes.  Bacula is a bit tricky to configure, so your trying it out
> would be helpful if you have the time.
> 
> In looking at the manuals for both the SDLT 220 and the DLT 8000, they 
> both
> claim to support long position information for the SCSI READ POSITION
> command.
> 
> You can see what I'm talking about by doing:
> 
> mt eod
> mt status
> 
> On my DDS-4 tape drive, this shows:
> 
> # mt -f /dev/nsa3 status
> Drive: sa3:  Serial Number: HJ00YWY
> -
> Mode  Density  Blocksize  bpi  Compression
> Current:  0x26:DDS-4   1024 bytes 97000enabled (DCLZ)
> -
> Current Driver State: at rest.
> -
> Partition:   0  Calc File Number:  -1 Calc Record Number: -1
> Residual:0  Reported File Number:  -1 Reported Record Number: -1
> Flags: None
> 
> But on an LTO-5, which will give long position information, I get:
> 
> [root@doc ~]# mt status
> Drive: sa0: 
> -
> Mode  Density  Blocksize  bpi  Compression
> Current:  0x58:LTO-5   variable   384607   enabled (0x1)
> -
> Current Driver State: at rest.
> -
> Partition:   0  Calc File Number:   2 Calc Record Number: -1
> Residual:0  Reported File Number:   2 Reported Record Number: 32373
> Flags: None
> 
> That, in combination with the changes I made to the position information
> code in the driver, mean that even the old MTIOCGET ioctl should return an
> accurate file number at end of data.  e.g., on the LTO-5:
> 
> [root@doc ~]# mt ostatus
> Mode  Density  Blocksize  bpi  Compression
> Current:  0x58:LTO-5   variable   384607   0x1
> -available modes-
> 0:0x58:LTO-5   variable   384607   0x1
> 1:0x58:LTO-5   variable   384607   0x1
> 2:0x58:LTO-5   variable   384607   0x1
> 3:0x58:LTO-5   variable   384607   0x1
> -
> Current Driver State: at rest.
> -
> File Number: 2  Record Number: -1   Residual Count -1
> 
> So the thing to try, in addition to just making sure that Bacula continues
> to work properly, is to try setting this for the tape drive in
> bacula-sd.conf:
> 
> Hardware End of Medium = yes
> 
> It looks like the Bacula tape program (btape) has a test mode, and it 
> would
> be good to run through the tests on one of the tape drives and see whether
> they work, and whether the results are different before and after the
> changes.  I'm not sure how to enable the test mode.
 
 I have this in /usr/local/etc/bacula/bacula-sd.conf
 
 Device {
 Name= DLT
 Description = "QUANTUM DLT7000 1624"
 Media Type  = DLT
 Archive Device  = /dev/nsa1
 
 Autochanger = YES
 Drive Index = 0
 
 Offline On Unmount  = no
 Hardware End of Medium  = yes
 BSF at EOM  = yes
 Backward Space Record   = no
 Fast Forward Space File = no
 TWO EOF = yes
 }
 
 FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape 
 test on this same model.
 
 Here's the test I ran tonight:
 
 [root@cuppy:/usr/home/dan] # btape -c /usr/lo

Re: Massive libxo-zation that breaks everything

[Chris H]

On Mon, 2 Mar 2015 09:23:55 + David Chisnall  wrote

> On 1 Mar 2015, at 21:29, Rui Paulo  wrote:
> > 
> > On Mar 1, 2015, at 11:11, David Chisnall  wrote:
> >> How would it be in a port?  It involves modifying core utilities (some of
> >> which, like ifconfig, rely on kernel APIs that change between releases) to
> >> emit structured output. Maintaining two copies of each utility, one in the
> >> base system with plain-text output only and another in ports with XML/JSON
> >> output would be very painful. > 
> > It would work fine if we had *libraries* for ifconfig/netstat/route/etc. 
> > Obviously that's not the case and no one has stepped up to implement them. 
> > I've also seen FreeBSD committers expressing their distaste for libraries
for
> > "trivial" command line utilities, which implies they are unaware of another
> > world beyond the CLI.  :-) 
>
> I am completely in favour of libraries for the underlying functionality of
> these commands and would love to see all of the system management commands
> become thin wrappers around a library, though it's a lot of engineering work.
>  In particular, these libraries will need to have stable APIs that we can
> support across multiple major releases, and getting those right is difficult.
>  We really don't want to be stuck in 10 years maintaining a hastily designed
> API for a library. 
>
> I see one use of the libxo output as helping to design those APIs.  People
> are going to wrap various tools in libraries for their favourite scripting
> languages and this will give us a corpus for experimenting. 
>
> It's also worth noting that often invoking a tool and consuming its output is
> the easiest way to get a stable API and ABI where performance is not a
> primary concern (i.e. most management interfaces). 
>
> As to a world beyond the CLI, I saw a nice demo a few years ago of a terminal
> emulator that used WebKit and came with a hacked-up set of parsers for common
> tools.  I'd love to have something simpler (no need for a full WebKit -
> simple outline and table views would be enough and could be done with curses
> for ssh) for FreeBSD where I could type ls in the CLI and get a table view
> that I could then sort and filter by selecting column headings.  Those of us
> that have used Lisp and Smalltalk environments know that a CLI doesn't have
> to be a teletype emulator. 
Apologies in advance, if I'm way off base, or simply reiterating here;
But what about something like kernshell(1)?
Every database has it's own "shell", the system has it's own contributed
shells. Couldn't a shell be adapted with a language/dialect to make use
of already existing parsers/languages/3rd party utilities?
Then it would simply be one item to maintain, and the language could
be modified/adapted, as needed.

Again, apologies, if I haven't thought this through enough. But in
an effort to simplify everything, for all concerned. This seemed
like a possible direction. So thought I'd mention it.

--Chris
>
> David
> 
> ___
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous

[Ryan Stone]

Can you post the contents of your make.conf and src.conf?  I didn't
see this in any of my "make tinderbox" runs
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Alfred Perlstein]

On 3/2/15 4:14 AM, Julian Elischer wrote:

On 3/1/15 10:49 AM, Harrison Grundy wrote:

Thanks!

That does seem useful, but I'm not sure I see the reasoning behind
putting into base, over a port or package, since processing XML in base
is a pain, and it can't serve up JSON or HTML without additional
utilities anyway.

(If I'm reviving a long-settled thing, let me know and I'll drop it. I'm
trying to understand the use case for this.)


To me it would almost seem more useful to have a programmable filter 
for which you could produce

parse grammars to parse the output of various programs..
thus

ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
with a set of grammars in /usr/share/xmlize/
then we could use it for out-of-tree programs as well if we wrote 
grammars for them..


The sentiment of machine-readable output is nice, but I think it's 
slightly off target.
we shouldn't have to change all out utilities, and it isn't going to 
help at all with 3rd party apps,
e.g. samba stuff. A generally easy to program output grammar parser 
would be truely useful.

and not just for FreeBSD.

I've been watching with an uncomfortable feeling, but it's taken me a 
while to put my

finger on what it was..


Are you sure it's not the hairs on the back of your neck standing up due 
to NIH?


Juniper has been doing this for years and it's very useful for them.

-Alfred

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Alfred Perlstein]

On 3/2/15 4:25 AM, David Chisnall wrote:

On 2 Mar 2015, at 09:16, Julian Elischer  wrote:

if we develop a suitable post processor with pluggable grammars, we save a lot 
of work.
given enough examples you could almost have automatically generated grammars.

This decoupled approach is problematic.  A large part of the point of libxo is 
to allow changing the human-readable output without breaking tools that consume 
the output.  Now I need to keep the tool that consumes it and the tool that 
produces it in sync, so that's an extra set of moving parts.  When you throw 
jails with multiple versions of world into the mix, it becomes a recipe for 
disaster.


+1
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[David Chisnall]

On 2 Mar 2015, at 09:24, Harrison Grundy  
wrote:
> 
> It would seem like the libxo stuff runs the risk of becoming this same
> API.

Why?  The 'API' in the case of an libxo-ised program is a stream on stdout that 
is then consumed by a JSON or XML parser. XML and JSON are intrinsically 
extensible formats.  This is *the entire point* of libxo: that we can extend 
the output from these tools without breaking things that wish to consume them 
and which currently rely on fragile parsers.

David

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Harrison Grundy]

On 03/02/15 01:23, David Chisnall wrote:
> On 1 Mar 2015, at 21:29, Rui Paulo  wrote:
>> 
>> On Mar 1, 2015, at 11:11, David Chisnall 
>> wrote:
>>> How would it be in a port?  It involves modifying core
>>> utilities (some of which, like ifconfig, rely on kernel APIs
>>> that change between releases) to emit structured output.
>>> Maintaining two copies of each utility, one in the base system
>>> with plain-text output only and another in ports with XML/JSON
>>> output would be very painful.
>> 
>> It would work fine if we had *libraries* for
>> ifconfig/netstat/route/etc.  Obviously that's not the case and no
>> one has stepped up to implement them.  I've also seen FreeBSD
>> committers expressing their distaste for libraries for "trivial"
>> command line utilities, which implies they are unaware of another
>> world beyond the CLI.  :-)
> 
> I am completely in favour of libraries for the underlying
> functionality of these commands and would love to see all of the
> system management commands become thin wrappers around a library,
> though it's a lot of engineering work.  In particular, these
> libraries will need to have stable APIs that we can support across
> multiple major releases, and getting those right is difficult.  We
> really don't want to be stuck in 10 years maintaining a hastily
> designed API for a library.
> 
> I see one use of the libxo output as helping to design those APIs.
> People are going to wrap various tools in libraries for their
> favourite scripting languages and this will give us a corpus for
> experimenting.
> 
> It's also worth noting that often invoking a tool and consuming its
> output is the easiest way to get a stable API and ABI where
> performance is not a primary concern (i.e. most management
> interfaces).
> 
> As to a world beyond the CLI, I saw a nice demo a few years ago of
> a terminal emulator that used WebKit and came with a hacked-up set
> of parsers for common tools.  I'd love to have something simpler
> (no need for a full WebKit - simple outline and table views would
> be enough and could be done with curses for ssh) for FreeBSD where
> I could type ls in the CLI and get a table view that I could then
> sort and filter by selecting column headings.  Those of us that
> have used Lisp and Smalltalk environments know that a CLI doesn't
> have to be a teletype emulator.
> 
> David

It would seem like the libxo stuff runs the risk of becoming this same
API.

--- Harrison
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

r279514: buildworld failure: /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded operator '<<' is ambiguous

[O. Hartmann]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Buildworld fails in CURRENT r279514 with the following error:

[...]

- --- sbin.all__D ---
- --- camcontrol.o ---
cc  -O2 -pipe -O3 -O3 -pipe -march=native  -std=gnu99 -fstack-protector
- -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter
- -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type
- -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wunused-parameter -Wcast-align
- -Wchar-subscripts -Winline -Wnested-externs -Wredundant-decls
- -Wold-style-definition -Wno-pointer-sign -Wmissing-variable-declarations
- -Wthread-safety -Wno-empty-body -Wno-string-plus-int 
-Wno-unused-const-variable
- -Qunused-arguments -c /usr/src/sbin/camcontrol/camcontrol.c --- gnu.all__D
- --- /usr/src/gnu/usr.bin/rcs/lib/rcsrev.c:750:8: warning: add explicit braces
to avoid dangling else [-Wdangling-else] else { ^ --- lib.all__D
- --- /usr/src/lib/libnv/tests/dnv_tests.cc:453:2: error: use of overloaded
operator '<<' is ambiguous (with operand types 'basic_ostream >' and 'nullptr_t') ATF_REQUIRE_EQ(actual_val,
NULL); --- kerberos5.all__D
- --- 
/usr/src/kerberos5/usr.bin/ksu/../../../crypto/heimdal/appl/su/su.c:437:42:
warning: data argument not used by format string [-Wformat-extra-args] ---
lib.all__D ---
^~~~ 
/usr/obj/usr/src/tmp/usr/include/atf-c++/macros.hpp:114:53:
note: expanded from macro 'ATF_REQUIRE_EQ' << " (" << (expected) << " != " <<
(actual) << ")"; \ ~~~ ^  
-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJU9ECRAAoJEOgBcD7A/5N826IH/3i9l8z5ER6jqo34eco4Mh9M
FwtLr4vKMMNDIDE4DRF+3+71K+PpX0pOi1eVugDpR71Y7oTK6Ja8vKL2t7GzpA1J
yzkQNyzMgzrEDR8B4fqRCqBlssg+ZkZGVCQHUxr0iTJyTu3lcf4scU1ghn1rt954
Kk+PSNowyRAakCA1l3m9Q15IDdQTL6LBGX76OzQAJIECmCowKnKFnOw6vk14fdOG
EiDe9X0z74j79CUfAKWZgfOt4FqqVXKo+4SZmyF6R9oxcAqtqlwixV5nk8iFY47c
lv5WBPJXpNYZ5+0lj28BLojeY4dV32lq8BZepbvSZG013VvZDTZLIwlWB9wZEDc=
=5+8f
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[David Chisnall]

On 2 Mar 2015, at 09:16, Julian Elischer  wrote:
> 
> if we develop a suitable post processor with pluggable grammars, we save a 
> lot of work.
> given enough examples you could almost have automatically generated grammars.

This decoupled approach is problematic.  A large part of the point of libxo is 
to allow changing the human-readable output without breaking tools that consume 
the output.  Now I need to keep the tool that consumes it and the tool that 
produces it in sync, so that's an extra set of moving parts.  When you throw 
jails with multiple versions of world into the mix, it becomes a recipe for 
disaster.

David

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[David Chisnall]

On 1 Mar 2015, at 21:29, Rui Paulo  wrote:
> 
> On Mar 1, 2015, at 11:11, David Chisnall  wrote:
>> How would it be in a port?  It involves modifying core utilities (some of 
>> which, like ifconfig, rely on kernel APIs that change between releases) to 
>> emit structured output. Maintaining two copies of each utility, one in the 
>> base system with plain-text output only and another in ports with XML/JSON 
>> output would be very painful.
> 
> It would work fine if we had *libraries* for ifconfig/netstat/route/etc.  
> Obviously that's not the case and no one has stepped up to implement them.  
> I've also seen FreeBSD committers expressing their distaste for libraries for 
> "trivial" command line utilities, which implies they are unaware of another 
> world beyond the CLI.  :-)

I am completely in favour of libraries for the underlying functionality of 
these commands and would love to see all of the system management commands 
become thin wrappers around a library, though it's a lot of engineering work.  
In particular, these libraries will need to have stable APIs that we can 
support across multiple major releases, and getting those right is difficult.  
We really don't want to be stuck in 10 years maintaining a hastily designed API 
for a library.

I see one use of the libxo output as helping to design those APIs.  People are 
going to wrap various tools in libraries for their favourite scripting 
languages and this will give us a corpus for experimenting.

It's also worth noting that often invoking a tool and consuming its output is 
the easiest way to get a stable API and ABI where performance is not a primary 
concern (i.e. most management interfaces).

As to a world beyond the CLI, I saw a nice demo a few years ago of a terminal 
emulator that used WebKit and came with a hacked-up set of parsers for common 
tools.  I'd love to have something simpler (no need for a full WebKit - simple 
outline and table views would be enough and could be done with curses for ssh) 
for FreeBSD where I could type ls in the CLI and get a table view that I could 
then sort and filter by selecting column headings.  Those of us that have used 
Lisp and Smalltalk environments know that a CLI doesn't have to be a teletype 
emulator.

David

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Julian Elischer]

On 3/1/15 11:10 AM, Allan Jude wrote:

On 2015-03-01 13:49, Harrison Grundy wrote:

Thanks!

That does seem useful, but I'm not sure I see the reasoning behind
putting into base, over a port or package, since processing XML in base
is a pain, and it can't serve up JSON or HTML without additional
utilities anyway.

(If I'm reviving a long-settled thing, let me know and I'll drop it. I'm
trying to understand the use case for this.)

--- Harrison

On 03/01/15 10:31, Craig Rodrigues wrote:

On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy <
harrison.gru...@astrodoggroup.com> wrote:



If someone could summarize what this is, I'd greatly appreciate it.


https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


I think you're missing the important bit here.

This isn't about adding a parser for anything, this is about making the
tools in base, like netstat, wc, uptime, etc, output in JSON or XML, so
you can use the data programmatically.

exactly. I think that's the wrong path to take.
we have to change EVERY PROGRAM IN THE WORLD.
if we develop a suitable post processor with pluggable grammars, we 
save a lot of work.
given enough examples you could almost have automatically generated 
grammars.





Your scripts no longer have to rely on awk/sed/grep magic to get a
specific bit of information out of the uptime command, the command can
just output the data in a structured machine readable format.

I am not sure how you can put netstat into the ports tree.




___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Julian Elischer]

On 3/1/15 10:49 AM, Harrison Grundy wrote:

Thanks!

That does seem useful, but I'm not sure I see the reasoning behind
putting into base, over a port or package, since processing XML in base
is a pain, and it can't serve up JSON or HTML without additional
utilities anyway.

(If I'm reviving a long-settled thing, let me know and I'll drop it. I'm
trying to understand the use case for this.)


To me it would almost seem more useful to have a programmable filter 
for which you could produce

parse grammars to parse the output of various programs..
thus

ifconfig -a | xmlize -g ifconfig | your-favourite-xml-parser
with a set of grammars in /usr/share/xmlize/
then we could use it for out-of-tree programs as well if we wrote 
grammars for them..


The sentiment of machine-readable output is nice, but I think it's 
slightly off target.
we shouldn't have to change all out utilities, and it isn't going to 
help at all with 3rd party apps,
e.g. samba stuff. A generally easy to program output grammar parser 
would be truely useful.

and not just for FreeBSD.

I've been watching with an uncomfortable feeling, but it's taken me a 
while to put my

finger on what it was..






--- Harrison





___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Massive libxo-zation that breaks everything

[Arseny Nasokin]

On 2 March 2015 at 04:47, Allan Jude  wrote:

> On 2015-03-01 19:20, Arseny Nasokin wrote:
> > On 1 March 2015 at 22:10, Allan Jude  wrote:
> >
> >> On 2015-03-01 13:49, Harrison Grundy wrote:
> >>> Thanks!
> >>>
> >>> That does seem useful, but I'm not sure I see the reasoning behind
> >>> putting into base, over a port or package, since processing XML in base
> >>> is a pain, and it can't serve up JSON or HTML without additional
> >>> utilities anyway.
> >>>
> >>> (If I'm reviving a long-settled thing, let me know and I'll drop it.
> I'm
> >>> trying to understand the use case for this.)
> >>>
> >>> --- Harrison
> >>>
> >>> On 03/01/15 10:31, Craig Rodrigues wrote:
>  On Sun, Mar 1, 2015 at 9:25 AM, Harrison Grundy <
>  harrison.gru...@astrodoggroup.com> wrote:
> 
> >
> >
> > If someone could summarize what this is, I'd greatly appreciate it.
> >
> 
> 
> https://lists.freebsd.org/pipermail/freebsd-arch/2014-July/015633.html
>  ___
>  freebsd-current@freebsd.org mailing list
>  http://lists.freebsd.org/mailman/listinfo/freebsd-current
>  To unsubscribe, send any mail to "
> >> freebsd-current-unsubscr...@freebsd.org"
> 
> >>> ___
> >>> freebsd-current@freebsd.org mailing list
> >>> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> >>> To unsubscribe, send any mail to "
> >> freebsd-current-unsubscr...@freebsd.org"
> >>>
> >>
> >> I think you're missing the important bit here.
> >>
> >> This isn't about adding a parser for anything, this is about making the
> >> tools in base, like netstat, wc, uptime, etc, output in JSON or XML, so
> >> you can use the data programmatically.
> >>
> >> Your scripts no longer have to rely on awk/sed/grep magic to get a
> >> specific bit of information out of the uptime command, the command can
> >> just output the data in a structured machine readable format.
> >>
> >> I am not sure how you can put netstat into the ports tree.
> >>
> >>
> >> --
> >> Allan Jude
> >>
> >>
> > Hi,
> >
> > Do we have command-line tools in base which work with XML/JSON from stdin
> > or file?
> >
> > -- Eir Nym
> > ___
> > freebsd-current@freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "
> freebsd-current-unsubscr...@freebsd.org"
> >
>
> Not really, although there are libraries for such (bsdxml, libucl which
> can read JSON) in base.
>
> I am working on a tool that can do some of this:
>
> https://github.com/allanjude/uclcmd/
>
> --
> Allan Jude
>
>
Allan,

What do you think about tools like plutil(1) or defaults(1) from Mac OS X.
Plists are dict-like structures (XML and JSON too).


-- Eir Nym
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: FreeBSD FUSE calls truncate() on read-only files

[Julian Elischer]

On 2/27/15 7:32 AM, Benjamin Kaduk wrote:

On Fri, 27 Feb 2015, Julian Elischer wrote:


for example it caches information when it shouldn't, even from 'dynamic' file
systems
We had to change the code to disable  it as our data is synthetic and might
change between reads.
fstat info is also cached and confused our apps mightily.

You are of course planning to file bug reports about these issues, I
presume?

-Ben



First I hope to actually understand the problems.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"