Re: svn commit: r339898 - head/lib/libc/amd64/sys

2018-11-04 Thread Julian Elischer

what's an ifunc?


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: vnet & firewalls in 12.0

2018-10-18 Thread Julian Elischer

I will only discuss ipfw.. I dont' use pf.


On 18/10/18 11:33 am, Ernie Luzar wrote:
Wanting to get a head start on using 12.0 and vnet jails with in 
jail firewall.


1. Will Vimage be compiled as a module in the 12.0 kernel and be 
included in the base system release?

it's in base.. not  a module


1.a. Has the boot time console log message about vimage being 
"highly experimental" been removed?


2. Has the pf firewall been fixed so it can now run in a vnet jail 
or multiple vnet jails with out concern for which firewall is 
running on the host?


2.a. Is each vnet/pf log only viewable from it's vnet jail console?

2.b. Will pf/kernel module auto load on first call from a vnet jail?

2.c. Does vnet/pf NAT work?



3. Does the ipfw firewall still have the 11.x release mandatory 
requirements that the host must also be running ipfw for the vnet 
jailed ipfw to work?

never heard about that..
effectively each network stack can have its own firewall. The ipfw 
module must be loaded so it will be 'hooked into' each stack.

whether you use it or not is up to you.


3.a. Are all vnet/ipfw log messages still intermixed with the host's 
ipfw log messages?
that is probably the case.  there is no per-jail kernel logging 
facility. (Sounds like a good idea!  send patches!)


3.b. Does vnet/ipfw NAT work?

last I checked it did.



4. Has any work been done to ipf (ipfilter) so it will function when 
used in a vnet jail?

I don't know how many people are using that... not a lot.


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


a little secret info on AZURE VMs..

2018-09-10 Thread Julian Elischer

Apparently if you publish an Azure image to their marketplace, they
blindly store billing information at location 65536 of the VHD file..
so you need to ensure that your first partitions start after that.
if you use a layout with your first partition starting at 64 sectors
in, this location falls in the middle of your boot code.
So it fails to boot.

I haven't found any documentation of this yet..


Julian

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: gcc/clang interoperability problem with a custom "samba" build in recent -current.

2018-07-26 Thread Julian Elischer

On 25/7/18 12:40 am, Julian Elischer wrote:

On 22/7/18 4:32 am, Dimitry Andric wrote:

On 21 Jul 2018, at 21:11, Yuri Pankov  wrote:

Yuri Pankov wrote:

Julian Elischer wrote:

...

anyone know if there is a clang equivalent of -Wp, -E,-lang-asm?

In later GCC versions the cpp's -lang-asm seems to be deprecated in
favor of -x assembler-with-cpp as it conflicts with -l option.
Could you try changing the -Wp,-E,-lang-asm to 
-Wp,-E,-xassembler-with-cpp?
I did this but if failed.. I had to reread this email  a few times to 
think of replacing the whole

-Wp,-E,-lang-asm  with just  -xassembler-with-cpp!


Just tried it myself, and if you indeed mean the 
third_party/aesni-intel/aesni-intel_asm.c, the following seems to 
work for me:


clang -xassembler-with-cpp -c 
third_party/aesni-intel/aesni-intel_asm.c

Yes, that is exactly what I suggested to Julian on IRC.  The point is
that the ".c" extension is misleading, it should more likely be a ".S"
extension.  But maybe this source file is used for multiple purposes.

Note that -x assembler-with-cpp should also work fine for gcc.

Ah..  the trick is to remove the -Wp,-E as well...


-Dimitry


thanks

I tried that but the version of the file we have has several lines 
that caused problems...


a lot of the assembler has assembler comments (starting with '#') 
which clang complained about and died..


it also had #.align 4

which I HOPE is just a commented out line..
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: gcc/clang interoperability problem with a custom "samba" build in recent -current.

2018-07-25 Thread Julian Elischer

On 22/7/18 3:11 am, Yuri Pankov wrote:

Yuri Pankov wrote:

Julian Elischer wrote:

I would really like ot get some pointers as to who are our tools
committers at the moment, in particular who might know about these 
issues.

The main issue for me at the moment is the ability to compile the
aesni code in Samba from clang..

Julian


On 20/7/18 7:32 pm, Julian Elischer wrote:

compiling our samba with gcc 4.2.1 in 12 gave us some off behaviour
when lld became the linker I think..

1/ linking needed some directories added to some of the build
scripts because previously apparently it looked in $SYSROOT/usr/lib
by default and now it doesn't.

2/ compiling our samba produces a libtdb.so that has various symbols
in it, (according to nm(1) ), but when we try link against it we get
complaints about those symbols not being defined.

3/ an attempt to switch to using clang to compile everything 
leads to:



"--aes-accel=intelaesni selected and compiler rejects 
-Wp,-E,-lang-asm.


One wonders whether there is a clang equivalent of 
"-Wp,-E,-lang-asm"


The AES acceleration is a configure option for the samba package.

Apparently turning it on requires -Wp,-E,-lang-asm.

which apparently gcc 4.2.1 has, but clang doesn't have.

     FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) 
(based

     on LLVM 6.0.1)
     Target: x86_64-unknown-freebsd12.0
     Thread model: posix
     InstalledDir: /usr/bin

anyone know if there is a clang equivalent of -Wp, -E,-lang-asm?


In later GCC versions the cpp's -lang-asm seems to be deprecated in
favor of -x assembler-with-cpp as it conflicts with -l option.

Could you try changing the -Wp,-E,-lang-asm to 
-Wp,-E,-xassembler-with-cpp?


Just tried it myself, and if you indeed mean the 
third_party/aesni-intel/aesni-intel_asm.c, the following seems to 
work for me:


clang -xassembler-with-cpp -c third_party/aesni-intel/aesni-intel_asm.c


when I try it I get lots of errors due to hte fact that there are 
assembled comments that start with #

and they are all cited as errors by clang.
I had to put '//' before about 80 lines line that.





possible work arrounds include:

1/ Get gcc/lld to produce a library from which lld can find the 
symbols


2/ find a way to compile this with clang but everything else with 
gcc?


3/ find a way to allow clang to use
-Wp,-E,-lang-asm

whatever that means


Thoughts from any tools people?





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: gcc/clang interoperability problem with a custom "samba" build in recent -current.

2018-07-24 Thread Julian Elischer

On 22/7/18 4:32 am, Dimitry Andric wrote:

On 21 Jul 2018, at 21:11, Yuri Pankov  wrote:

Yuri Pankov wrote:

Julian Elischer wrote:

...

anyone know if there is a clang equivalent of -Wp, -E,-lang-asm?

In later GCC versions the cpp's -lang-asm seems to be deprecated in
favor of -x assembler-with-cpp as it conflicts with -l option.
Could you try changing the -Wp,-E,-lang-asm to -Wp,-E,-xassembler-with-cpp?

Just tried it myself, and if you indeed mean the 
third_party/aesni-intel/aesni-intel_asm.c, the following seems to work for me:

clang -xassembler-with-cpp -c third_party/aesni-intel/aesni-intel_asm.c

Yes, that is exactly what I suggested to Julian on IRC.  The point is
that the ".c" extension is misleading, it should more likely be a ".S"
extension.  But maybe this source file is used for multiple purposes.

Note that -x assembler-with-cpp should also work fine for gcc.

-Dimitry


thanks

I tried that but the version of the file we have has several lines 
that caused problems...


a lot of the assembler has assembler comments (starting with '#') 
which clang complained about and died..


it also had #.align 4

which I HOPE is just a commented out line..
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: gcc/clang interoperability problem with a custom "samba" build in recent -current.

2018-07-21 Thread Julian Elischer
I would really like ot get some pointers as to who are our tools 
committers at the moment, in particular who might know about these issues.
The main issue for me at the moment is the ability to compile the 
aesni code in Samba from clang..


Julian


On 20/7/18 7:32 pm, Julian Elischer wrote:
compiling our samba with gcc 4.2.1 in 12 gave us some off behaviour 
when lld became the linker I think..


1/ linking needed some directories added to some of the build 
scripts because previously apparently it looked in $SYSROOT/usr/lib 
by default and now it doesn't.


2/ compiling our samba produces a libtdb.so that has various symbols 
in it, (according to nm(1) ), but when we try link against it we get 
complaints about those symbols not being defined.


3/ an attempt to switch to using clang to compile everything leads to:


"--aes-accel=intelaesni selected and compiler rejects -Wp,-E,-lang-asm.

One wonders whether there is a clang equivalent of "-Wp,-E,-lang-asm"

The AES acceleration is a configure option for the samba package.

Apparently turning it on requires -Wp,-E,-lang-asm.

which apparently gcc 4.2.1 has, but clang doesn't have.

   FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based
   on LLVM 6.0.1)
   Target: x86_64-unknown-freebsd12.0
   Thread model: posix
   InstalledDir: /usr/bin

anyone know if there is a clang equivalent of -Wp, -E,-lang-asm?

possible work arrounds include:

1/ Get gcc/lld to produce a library from which lld can find the symbols

2/ find a way to compile this with clang but everything else with gcc?

3/ find a way to allow clang to use
-Wp,-E,-lang-asm

whatever that means


Thoughts from any tools people?

Julian

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


gcc/clang interoperability problem with a custom "samba" build in recent -current.

2018-07-20 Thread Julian Elischer
compiling our samba with gcc 4.2.1 in 12 gave us some off behaviour 
when lld became the linker I think..


1/ linking needed some directories added to some of the build scripts 
because previously apparently it looked in $SYSROOT/usr/lib by default 
and now it doesn't.


2/ compiling our samba produces a libtdb.so that has various symbols 
in it, (according to nm(1) ), but when we try link against it we get 
complaints about those symbols not being defined.


3/ an attempt to switch to using clang to compile everything leads to:


"--aes-accel=intelaesni selected and compiler rejects -Wp,-E,-lang-asm.

One wonders whether there is a clang equivalent of "-Wp,-E,-lang-asm"

The AES acceleration is a configure option for the samba package.

Apparently turning it on requires -Wp,-E,-lang-asm.

which apparently gcc 4.2.1 has, but clang doesn't have.

   FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based
   on LLVM 6.0.1)
   Target: x86_64-unknown-freebsd12.0
   Thread model: posix
   InstalledDir: /usr/bin

anyone know if there is a clang equivalent of -Wp, -E,-lang-asm?

possible work arrounds include:

1/ Get gcc/lld to produce a library from which lld can find the symbols

2/ find a way to compile this with clang but everything else with gcc?

3/ find a way to allow clang to use
-Wp,-E,-lang-asm

whatever that means


Thoughts from any tools people?

Julian

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: GELI attach issue from r326073 -> r334996

2018-06-17 Thread Julian Elischer

On 14/6/18 4:46 am, Michael Jung wrote:

On 2018-06-13 15:27, Ian Lepore wrote:

On Wed, 2018-06-13 at 14:29 -0400, Michael Jung wrote:

Hi!

I just tried updating current from r326073 -> r334996 and when
I try 'geli attach' I get the following error:

# geli attach -p -k mykey.key /dev/gpt/da14
geli: Missing keyno argument
#

If I boot the old kernel GELI attaches just fine.

I ran into this once before but can not find the thread.  I recall
it being a bug... with no resolution.

I mount zfs partitions over GELI - my painful solution was to
nuke each GPT partition in the zpool, resilver, repeat and
rinse until everything was non-encrypted and repeat the cycle
to re-encrypt.  NOT FUN.

Looking for suggestions to supply additional information to debug
and resolve.

dmesg attached from working kernel.

Thanks!


r333439 added a '-n keyno' parameter to geli attach, but it's supposed
to default to trying all keys if you don't use -n. Is it possible
you're running a newer kernel (or geom_eli module) than your userland
or vice versa?

-- Ian


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"


Ian:

Yes you are correct Maybe not the best method but I normally 
installkernel -
boot into single user mode - GELI attach, zfs mount -av, then 
installworld.


Yes that is the prescribed method. if it doesn't work then whoever 
changed the

geli code should fix it to handle this.
You should be able to run older code on newer kernels with very few 
exceptions.

I don't know if this also affects upgrades form 11. Have you tested that?


My boot volume is UFS, but many of the mount points are on zpools.

What would be the best way to test a new kernel without a full 
installworld

with new userland geli bits?

I currently don't have a way to backup my 35TB of data :-( and don't 
want to
lose anything. and I need a back out method should a full 
installworld fail.


Thanks for you quick reply.

--mikej
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: head@r334204, Bad link elm in callout_process()

2018-05-29 Thread Julian Elischer

On 29/5/18 9:53 pm, Andriy Gapon wrote:

On 29/05/2018 14:53, Hans Petter Selasky wrote:

On 05/29/18 13:20, Andriy Gapon wrote:

(kgdb) p *$4.lh_first->c_links.le.le_next
$6 = {
    c_links = {
  le = {
    le_next = 0x0,
    le_prev = 0xfe0003999f98

Where does the le_prev point?

Typically happens when callouts are not properly drained before freeing memory.

Yeah, this could have been a pilot error.
I added some debugging code and that code could do callout_init + callout_reset
on an active or pending callout.  I am not seeing the problem without the
debugging code.
Sorry for the noise!

did you add code to that machine?? (bob.$work)






___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


crash on process exit.. current at about r332467

2018-04-23 Thread Julian Elischer

back trace at:  http://www.freebsd.org/~julian/bob-crash.png

If anyone wants to take a look..

In the exit syscall, while deallocating a vm object.

I haven't see references to a similar crash in the last 10 days or 
so.. But if it rings any bells...




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: SCHED_ULE makes 256Mbyte i386 unusable

2018-04-23 Thread Julian Elischer

On 22/4/18 9:43 pm, Rick Macklem wrote:

Konstantin Belousov wrote:

On Sat, Apr 21, 2018 at 11:30:55PM +, Rick Macklem wrote:

Konstantin Belousov wrote:

On Sat, Apr 21, 2018 at 07:21:58PM +, Rick Macklem wrote:

I decided to start a new thread on current related to SCHED_ULE, since I see
more than just performance degradation and on a recent current kernel.
(I cc'd a couple of the people discussing performance problems in freebsd-stable
  recently under a subject line of "Re: kern.sched.quantum: Creepy, sadistic 
scheduler".

When testing a pNFS server on a single core i386 with 256Mbytes using a Dec. 
2017
current/head kernel, I would see about a 30% performance degradation (elapsed
run time for a kernel build over NFSv4.1) when the server kernel was built with
options SCHED_ULE
instead of
options SCHED_4BSD

So, now that I have decreased the number of nfsd kernel threads to 32, it works
with both schedulers and with essentially the same performance. (ie. The 30%
performance degradation has disappeared.)


Now, with a kernel from a couple of days ago, the
options SCHED_ULE
kernel becomes unusable shortly after starting testing.
I have seen two variants of this:
- Became essentially hung. All I could do was ping the machine from the network.
- Reported "vm_thread_new: kstack allocation failed
   and then any attempt to do anything gets "No more processes".

This is strange.  It usually means that you get KVA either exhausted or
severly fragmented.

Yes. I reduced the number of nfsd threads from 256->32 and the SCHED_ULE
kernel is working ok now. I haven't done enough to compare performance yet.
Maybe I'll post again when I have some numbers.


Enter ddb, it should be operational since pings are replied.  Try to see
where the threads are stuck.

I didn't do this, since reducing the number of kernel threads seems to have 
fixed
the problem. For the pNFS server, the nfsd threads will spawn additional kernel
threads to do proxies to the mirrored DS servers.


with the only difference being a kernel built with
options SCHED_4BSD
everything works and performs the same as the Dec 2017 kernel.

I can try rolling back through the revisions, but it would be nice if someone
could suggest where to start, because it takes a couple of hours to build a
kernel on this system.

So, something has made things worse for a head/current kernel this winter, rick

There are at least two potentially relevant changes.

First is r326758 Dec 11 which bumped KSTACK_PAGES on i386 to 4.

I've been running this machine with KSTACK_PAGES=4 for some time, so no change.

W.r.t. Rodney Grimes comments about this (which didn't end up in this messages
in the thread):
I didn't see any instability when using KSTACK_PAGES=4 for this until this 
cropped
up and seemed to be scheduler related (but not really, it seems).
I bumped it to KSTACK_PAGES=4 because I needed that for the pNFS Metadata
Server code.

Yes, NFS does use quite a bit of kernel stack. Unfortunately, it isn't one big
item getting allocated on the stack, but many moderate sized ones.
(A part of it is multiple instances of "struct vattr", some buried in "struct 
nfsvattr",
  that NFS needs to use. I don't think these are large enough to justify 
malloc/free,
  but it has to use several of them.)

One case I did try fixing was about 6 cases where "struct nfsstate" ended up on
the stack. I changes the code to malloc/free them and then when testing, to
my surprise I had a 20% performance hit and shelved the patch.


you might try using uma. especially setting up a non-freeing zone, 
where he system allocates what it needs and then just recycles them.

(man uma)

Now that I know that the server was running near its limit, I might try this one
again, to see if the performance hit doesn't occur when the machine has adequate
memory. If the performance hit goes away, I could commit this, but it wouldn't 
have that much effect on the kstack usage. (It's interesting how this patch 
ended
up related to the issue this thread discussed.)


Second is r332489 Apr 13, which introduced 4/4G KVA/UVA split.

Could this change have resulted in the system being able to allocate fewer
kernel threads/stacks for some reason?

Well, it could, as anything can be buggy. But the intent of the change
was to give 4G KVA, and it did.

Righto. No concern here. I suspect the Dec. 2017 kernel was close to the limit
(see performance issue that went away, noted above) and any change could
have pushed it across the line, I think.


Consequences of the first one are obvious, it is much harder to find
the place to map the stack.  Second change, on the other hand, provides
almost full 4G for KVA and should have mostly compensate for the negative
effects of the first.

And, I cannot see how changing the scheduler would fix or even affect that
behaviour.

My hunch is that the system was running near its limit for kernel 
threads/stacks.
Then, somehow, the timing SCHED_ULE caused resulted in the nfsd trying to get

Re: SCHED_ULE makes 256Mbyte i386 unusable

2018-04-23 Thread Julian Elischer

On 22/4/18 10:36 pm, Rodney W. Grimes wrote:

Konstantin Belousov wrote:

On Sat, Apr 21, 2018 at 11:30:55PM +, Rick Macklem wrote:

Konstantin Belousov wrote:

On Sat, Apr 21, 2018 at 07:21:58PM +, Rick Macklem wrote:

I decided to start a new thread on current related to SCHED_ULE, since I see
more than just performance degradation and on a recent current kernel.
(I cc'd a couple of the people discussing performance problems in freebsd-stable
  recently under a subject line of "Re: kern.sched.quantum: Creepy, sadistic 
scheduler".

When testing a pNFS server on a single core i386 with 256Mbytes using a Dec. 
2017
current/head kernel, I would see about a 30% performance degradation (elapsed
run time for a kernel build over NFSv4.1) when the server kernel was built with
options SCHED_ULE
instead of
options SCHED_4BSD

So, now that I have decreased the number of nfsd kernel threads to 32, it works
with both schedulers and with essentially the same performance. (ie. The 30%
performance degradation has disappeared.)


Now, with a kernel from a couple of days ago, the
options SCHED_ULE
kernel becomes unusable shortly after starting testing.
I have seen two variants of this:
- Became essentially hung. All I could do was ping the machine from the network.
- Reported "vm_thread_new: kstack allocation failed
   and then any attempt to do anything gets "No more processes".

This is strange.  It usually means that you get KVA either exhausted or
severly fragmented.

Yes. I reduced the number of nfsd threads from 256->32 and the SCHED_ULE
kernel is working ok now. I haven't done enough to compare performance yet.
Maybe I'll post again when I have some numbers.


Enter ddb, it should be operational since pings are replied.  Try to see
where the threads are stuck.

I didn't do this, since reducing the number of kernel threads seems to have 
fixed
the problem. For the pNFS server, the nfsd threads will spawn additional kernel
threads to do proxies to the mirrored DS servers.


with the only difference being a kernel built with
options SCHED_4BSD
everything works and performs the same as the Dec 2017 kernel.

I can try rolling back through the revisions, but it would be nice if someone
could suggest where to start, because it takes a couple of hours to build a
kernel on this system.

So, something has made things worse for a head/current kernel this winter, rick

There are at least two potentially relevant changes.

First is r326758 Dec 11 which bumped KSTACK_PAGES on i386 to 4.

I've been running this machine with KSTACK_PAGES=4 for some time, so no change.

W.r.t. Rodney Grimes comments about this (which didn't end up in this messages
in the thread):
I didn't see any instability when using KSTACK_PAGES=4 for this until this 
cropped
up and seemed to be scheduler related (but not really, it seems).
I bumped it to KSTACK_PAGES=4 because I needed that for the pNFS Metadata
Server code.

Yes, NFS does use quite a bit of kernel stack. Unfortunately, it isn't one big
item getting allocated on the stack, but many moderate sized ones.
(A part of it is multiple instances of "struct vattr", some buried in "struct 
nfsvattr",
  that NFS needs to use. I don't think these are large enough to justify 
malloc/free,
  but it has to use several of them.)

One case I did try fixing was about 6 cases where "struct nfsstate" ended up on
the stack. I changes the code to malloc/free them and then when testing, to
my surprise I had a 20% performance hit and shelved the patch.
Now that I know that the server was running near its limit, I might try this one
again, to see if the performance hit doesn't occur when the machine has adequate
memory. If the performance hit goes away, I could commit this, but it wouldn't
have that much effect on the kstack usage. (It's interesting how this patch 
ended
up related to the issue this thread discussed.)

Anything we can do to help relieve KSTACK usage, especially on i386
is helpfull.  These is a thread back quite some time where someone
came up with a compile time static "this functions uses X bytes of
local stack" and a bit of clean up was done.  We should persue
this issue further.


that was me.

use
|-Wframe-larger-than||=|¶ 
 
and set it to something like 512 bytes




My experiece with the i386/KSTACK issues was attempting to do installs
from snapshot .iso's, I usually had to change to a custom kernel without
INVARIANTS and WITNESS, or reduce KSTACK to 2 and suffer the small stack
problem (ie, dont use NFS during install).  Neither was very pleasant.

I have found it in practical to run the 4 page KSTACK in production
VM's using i386 due to memory requirements.  I run many very lean
i386 VM's with 64MB of memory.  I suspect our user base also has
many people doing this, and it would be to our advantage to try
and reduce our kernel stack needs.



Second is r332489 Apr 13, which introduced 4/4G 

Re: SCHED_ULE makes 256Mbyte i386 unusable

2018-04-23 Thread Julian Elischer

On 22/4/18 10:36 pm, Rodney W. Grimes wrote:

Konstantin Belousov wrote:

On Sat, Apr 21, 2018 at 11:30:55PM +, Rick Macklem wrote:

Konstantin Belousov wrote:

On Sat, Apr 21, 2018 at 07:21:58PM +, Rick Macklem wrote:

I decided to start a new thread on current related to SCHED_ULE, since I see
more than just performance degradation and on a recent current kernel.
(I cc'd a couple of the people discussing performance problems in freebsd-stable
  recently under a subject line of "Re: kern.sched.quantum: Creepy, sadistic 
scheduler".

When testing a pNFS server on a single core i386 with 256Mbytes using a Dec. 
2017
current/head kernel, I would see about a 30% performance degradation (elapsed
run time for a kernel build over NFSv4.1) when the server kernel was built with
options SCHED_ULE
instead of
options SCHED_4BSD

So, now that I have decreased the number of nfsd kernel threads to 32, it works
with both schedulers and with essentially the same performance. (ie. The 30%
performance degradation has disappeared.)


Now, with a kernel from a couple of days ago, the
options SCHED_ULE
kernel becomes unusable shortly after starting testing.
I have seen two variants of this:
- Became essentially hung. All I could do was ping the machine from the network.
- Reported "vm_thread_new: kstack allocation failed
   and then any attempt to do anything gets "No more processes".

This is strange.  It usually means that you get KVA either exhausted or
severly fragmented.

Yes. I reduced the number of nfsd threads from 256->32 and the SCHED_ULE
kernel is working ok now. I haven't done enough to compare performance yet.
Maybe I'll post again when I have some numbers.


Enter ddb, it should be operational since pings are replied.  Try to see
where the threads are stuck.

I didn't do this, since reducing the number of kernel threads seems to have 
fixed
the problem. For the pNFS server, the nfsd threads will spawn additional kernel
threads to do proxies to the mirrored DS servers.


with the only difference being a kernel built with
options SCHED_4BSD
everything works and performs the same as the Dec 2017 kernel.

I can try rolling back through the revisions, but it would be nice if someone
could suggest where to start, because it takes a couple of hours to build a
kernel on this system.

So, something has made things worse for a head/current kernel this winter, rick

There are at least two potentially relevant changes.

First is r326758 Dec 11 which bumped KSTACK_PAGES on i386 to 4.

I've been running this machine with KSTACK_PAGES=4 for some time, so no change.

W.r.t. Rodney Grimes comments about this (which didn't end up in this messages
in the thread):
I didn't see any instability when using KSTACK_PAGES=4 for this until this 
cropped
up and seemed to be scheduler related (but not really, it seems).
I bumped it to KSTACK_PAGES=4 because I needed that for the pNFS Metadata
Server code.

Yes, NFS does use quite a bit of kernel stack. Unfortunately, it isn't one big
item getting allocated on the stack, but many moderate sized ones.
(A part of it is multiple instances of "struct vattr", some buried in "struct 
nfsvattr",
  that NFS needs to use. I don't think these are large enough to justify 
malloc/free,
  but it has to use several of them.)

One case I did try fixing was about 6 cases where "struct nfsstate" ended up on
the stack. I changes the code to malloc/free them and then when testing, to
my surprise I had a 20% performance hit and shelved the patch.
Now that I know that the server was running near its limit, I might try this one
again, to see if the performance hit doesn't occur when the machine has adequate
memory. If the performance hit goes away, I could commit this, but it wouldn't
have that much effect on the kstack usage. (It's interesting how this patch 
ended
up related to the issue this thread discussed.)

Anything we can do to help relieve KSTACK usage, especially on i386
is helpfull.  These is a thread back quite some time where someone
came up with a compile time static "this functions uses X bytes of
local stack" and a bit of clean up was done.  We should persue
this issue further.


that was me.

use
|-Wframe-larger-than||=|¶ 
 
and set it to something like 512 bytes (obviously you have to make 
warnings non fatal as well).






My experiece with the i386/KSTACK issues was attempting to do installs
from snapshot .iso's, I usually had to change to a custom kernel without
INVARIANTS and WITNESS, or reduce KSTACK to 2 and suffer the small stack
problem (ie, dont use NFS during install).  Neither was very pleasant.

I have found it in practical to run the 4 page KSTACK in production
VM's using i386 due to memory requirements.  I run many very lean
i386 VM's with 64MB of memory.  I suspect our user base also has
many people doing this, and it would be to our advantage to try
and reduce our kernel stack 

Re: anyone running with ngroups increased from 16?

2018-04-16 Thread Julian Elischer

On 16/4/18 6:37 pm, Julian Elischer wrote:
Windows users seem to have an almost unlimited number of groups and 
soem places seem to use them a LOT.
This gives Posix systems problems with deciding how to handle them 
all. Especially when getting

user credentials from winbindd (samba).

Does anyone know of any work done to either bypass this limit or to 
at least expand it?


I mean with the other applications such NFS usages etc.
I know mountd explodes with > 16..  has anyone done a cleaning pass?



Thanks

Julian

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Module compiles looking in /usr/src when alternate src tree is in use

2018-04-16 Thread Julian Elischer

On 11/4/18 8:29 am, Rodney W. Grimes wrote:

Rodney W. Grimes  wrote:


I am having a compile time issue for a patched that compiled fine on my
r329294 system, but now failes to compile with what looks like a wrong
header being included.

You may find it helpful to do something like:

make -dv -C sys/modules/vmm -V CFLAGS > /tmp/dvo 2>&1
egrep ':.PARSE|/usr/src/sys' /tmp/dvo | grep -B1 usr/src | more

The arg to -V doesn't matter btw.
You could narrow that down if you know what var -I/usr/src/sys is in
(probably CFLAGS but you never know)
the above should help find the makefile that is introducing the bogus -I


Thank you, that does help narrow it down:  (I backed up a vew lines
from the first place I saw src/.)

...
Global:.PARSEFILE = bsd.kmod.mk
Global:.PARSEDIR = /usr/src-topo/share/mk
Global:.PARSEFILE = bsd.kmod.mk
Result[] of :U is "/usr/src/sys"
Result[] of :U is "/usr/src/sys"
Global:SYSDIR = ${:U/usr/src/sys:tA}
Global:.PARSEDIR = /usr/src-topo/share/mk
Global:.PARSEFILE = bsd.kmod.mk
Result[] of :U is "/usr/src/sys"
Applying[] :t to "/usr/src/sys"
Result[] of :t is "/usr/src/sys"
Result[] of :U is "/usr/src/sys"
Applying[] :t to "/usr/src/sys"
Result[] of :t is "/usr/src/sys"
Result[] of :U is "/usr/src/sys"
Applying[] :t to "/usr/src/sys"
Result[] of :t is "/usr/src/sys"
Global:.MAKE.MAKEFILES = /usr/src-topo/share/mk/sys.mk 
/usr/src-topo/share/mk/local.sys.env.mk /usr/src-topo/share/mk/src.sys.env.mk 
/usr/src-topo/share/mk/bsd.mkopt.m
k /usr/src-topo/share/mk/src.sys.obj.mk /usr/src-topo/share/mk/auto.obj.mk 
/usr/src-topo/share/mk/bsd.suffixes.mk /usr/src-topo/share/mk/local.sys.mk 
/usr/src-topo/sha
re/mk/src.sys.mk /usr/src-topo/sys/modules/vmm/Makefile 
/usr/src-topo/share/mk/bsd.kmod.mk /usr/src/sys/conf/kmod.mk

^
Thats gona bust something some day

Global:.PARSEDIR = /usr/src/sys/conf
Global:.PARSEFILE = kmod.mk
Global:.INCLUDEDFROMDIR = /usr/src/sys/conf
Oh my!  Ug


So something in bsd.kmod.mk is going very wrong... it looks like it
starts to pull all sorts of stuff from /usr/src/sys!


I seem to remember that there is code in the Makefiles that looks for 
the sys directory.
I believe it can be directed to use a directory but in its absence it 
looks at some well
known locations, which would probably fail if there is already a 
DIFFERENT tree in /usr/src.



I have wrapped the long line so I can point to a difference between
r329294 and r332262 make log of this file.

r329294 make output:

cc  -O2 -pipe -DVMM_KEEP_STATS -DSMP  -fno-strict-aliasing -Werror -D_KERNEL \
-DKLD_MODULE -nostdinc  -I/usr/src-topo/sys/amd64/vmm \
-I/usr/src-topo/sys/amd64/vmm/io -I/usr/src-topo/sys/amd64/vmm/intel \
-I/usr/src-topo/sys/amd64/vmm/amd -I. -I/usr/src-topo/sys -fno-common  \
^ this is what I would 
expect




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


anyone running with ngroups increased from 16?

2018-04-16 Thread Julian Elischer
Windows users seem to have an almost unlimited number of groups and 
soem places seem to use them a LOT.
This gives Posix systems problems with deciding how to handle them 
all. Especially when getting

user credentials from winbindd (samba).

Does anyone know of any work done to either bypass this limit or to at 
least expand it?


Thanks

Julian

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: g_handleattr: md0 bio_length 24 len 31 -> EFAULT

2018-04-09 Thread Julian Elischer

On 8/4/18 10:48 pm, Kyle Evans wrote:

On Sun, Apr 8, 2018 at 5:53 AM, Peter Holm  wrote:

On Sun, Apr 08, 2018 at 02:36:08AM -0700, Michael Dexter wrote:

On 3/24/18 2:35 AM, O. Hartmann wrote:

Writing out memory (md) backed images of UFS2 filesystems (NanoBSD images, 
created via
the classical manual way, no makefs), my recent CURRENT system dumps the
console full of these error messages:

g_handleattr: md0 bio_length 24 len 31 -> EFAULT

I do not know what they are supposed to mean and I'd like to ask whether 
someone could
shed some light on this.

I am seeing this on the the latest snapshot when attempting to run
option_survey.sh which creates an md-attached disk image.

Anyone else seeing this?

Michael

Yes, I have:

[pho@freefall ~/public_html/stress/log]$ grep -a g_handleattr: `ls -rt`
numa025.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
numa025.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
numa025.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
numa025.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
numa025.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
numa025.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
numa025.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
numa025.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
numa025.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
kostik1104.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
kostik1104.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
kostik1104.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
kostik1104.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
kostik1104.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
kostik1104.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
kostik1104.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
kostik1104.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
kostik1104.txt:g_handleattr: md10 bio_length 24 len 31 -> EFAULT
[pho@freefall ~/public_html/stress/log]$


FYI- this should be fixed by r332070.

Thanks,

Kyle Evans
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


so I think current has other issues I want to avoid

can I just pull that commit and have have svn know how to upgrade past 
it later?



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


deadlock detected in -current (new).. NFS? Anyone with 'amd' knowledge?

2018-04-07 Thread Julian Elischer

Anyone seen these recently?



I just got the following:

panic: deadlkres: possible deadlock detected for (address)  blocked 
for (big number) ticks


anyone seen similar on very new (yesterday) -current GENERIC. ?
there was a prior message that may or may not be related:

newnfs: server pid741@gamma  error: fileid changed. fsid 0:0: expected 
fileid 0x1, got 0x2

(BROKEN NFS SERVER OR MIDDLEWARE)

(amd screwed up).

I suspect that amd is broken on -current
using known good config files results in a hung nfs mount
and even after killing amd, any attempt to touch the mountpoint 
results in a hung (but interruptable) proccess.
For example as /net  was an amd mountpoint,   ls -l /    results in a 
hung copy of ls.

(but you can get it back with ^C)

I suspect the two are related.

Julian


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


zfsloader: messy output on Dell iDrac serial console

2018-04-06 Thread Julian Elischer
running on *some* Dell servers the serial console looks messed up at 
the time of the FreeBSD menu.
it looks like it's dropping (or adding? but that's less likely) 
characters.


 |  5. [K]ernel: kernel (1 of 2)   |  .- .    -.
 |  6. Configure Boot [O]ptions... |2  --| y/   -.   
-/`   -o/
 |137. Select Boot [E]nvironment...    | `:`  
`:` ::/sy++:.
 | H| .-- `--. --  
/6H`:  :`

 | | .---..

IS there an RTS/CTS input we should be attending to?

I will add that some of the other modules (e.g. the mfi raid setup 
section) also get affected,

so it's not just a bug in our code.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


tzset (and friends) patch to watch /etc/localtime.. comments?

2018-03-07 Thread Julian Elischer
anyone interested in ctime, tzset, and localtime.c  please look 
athttps://reviews.freebsd.org/D14608


comments and improvements welcome.


Julian

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: A small procedural request

2018-02-21 Thread Julian Elischer

On 21/2/18 7:14 pm, Tomoaki AOKI wrote:

Hi.

+1. But have one suggestion for format.
Something like

  Broken by: rXXX
  Broken by: Unknown (Bugfix but the revision introduced it is unknown)

and optionally

  Broken by: No (To emphasize it's NOT a bugfix.)


I think that is probably too much, but the    Broken by:  would be 
good.


would be better for scripts already handling "MFC after: " or
"X-MFC-With: " etc. to support this.

If put on the top with "MFC rXX: Comments", it can be

  FIX rXX: Comments


possibly..
that Would allow some sort of collection of the data to  suggest good 
places to

retrospectively base your head following (but not too closely) branches.
but may be more work that people are willing to do..
For myself, just a hint of where the bug was introduced would help a lot.
further more if you have a branch/product based at some point in time, 
this would help

you to know when a patch needs to be cherry picked back to your code.


or for multiple revisions,

  FIX rXX rYY rZZ: Comments for multiple individuals
  FIX rXX-rYY: Comments for massive continuous range

would be better.

Regards.


On Wed, 21 Feb 2018 12:01:33 +0800
Julian Elischer <jul...@freebsd.org> wrote:


Hi,〓 I have a very small request to those committing into head.

If you commit a fix, then if it is possible to easily do so, can you
give the revision number in which the regression was introduced?

like "this was〓 broken in r329xxx"

this allows people who are looking for specific problems to say "Ok
that bug was introduced after the snapshot I'm working on and can't be
my issue".

(we are not always working on the very tip).


thanks

Julian


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"






___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


A small procedural request

2018-02-20 Thread Julian Elischer

Hi,  I have a very small request to those committing into head.

If you commit a fix, then if it is possible to easily do so, can you 
give the revision number in which the regression was introduced?


like "this was  broken in r329xxx"

this allows people who are looking for specific problems to say "Ok 
that bug was introduced after the snapshot I'm working on and can't be 
my issue".


(we are not always working on the very tip).


thanks

Julian


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: 12.0-CURRENT missing timezones

2018-02-18 Thread Julian Elischer

On 10/2/18 2:48 am, Warner Losh wrote:

OK. That makes sense again.


I've pushed two changes (r329064 and r329075) which should correct this.



thanks.. I was just about to go investigate this as I noticed my build 
last week had no timezone info.




Warner

On Fri, Feb 9, 2018 at 10:28 AM, David Boyd  wrote:


On Fri, 2018-02-09 at 10:16 -0700, Warner Losh wrote:



On Fri, Feb 9, 2018 at 10:13 AM, Warner Losh  wrote:



On Fri, Feb 9, 2018 at 9:56 AM, David Boyd  wrote:

In the most recent images for 12.0-CURRENT

 FreeBSD-12.0-CURRENT-amd64-20180208-r329009.vmdk.xz

 FreeBSD-12.0-CURRENT-amd64-20180208-r329009-disc1.iso

 FreeBSD-12.0-CURRENT-amd64-20180131-r328637-memstick.img

the /usr/share/zoneinfo directory is sparsely populated with
directories. The only file present is "zone.tab".


I think that may be my fault for changing find -s to find | sort, but I
think I neglected to add sort to ITOOLS to fix it. Building a test now.


I am surprised the 0131 is empty, though My change was after that.

Warner


Oops. My mistake (cut and paste error). All dates should be 20180208.


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Since last week (today) current on my Ryzen box is unstable

2018-02-18 Thread Julian Elischer

On 19/2/18 4:33 am, Gleb Smirnoff wrote:

On Sun, Feb 18, 2018 at 10:15:24PM +0200, Andriy Gapon wrote:
A> On 18/02/2018 15:26, Gleb Smirnoff wrote:
A> > My only point is that it is a performance improvement. IMHO that's enough 
:)
A>
A> I don't think that passing an invalid argument to a documented KPI is 
"enough"
A> for any optimization.

I don't see a sense in making this KPI so sacred. This is something used 
internally
in kernel, and not used outside. The KPI has changed several times in the past.

A> > If you can't suggest a more elegant way of doing that improvement, then all
A> > I can suggest is to document it and add its support to ZFS.
A>
A> In return I can only suggest that (1) you run your suggestion by arch@ -- 
unless
A> that's already been done and you can point me to the discussion,  (2) 
document
A> it and (3) double-check that all implementations confirm to it.

I can provide a patch for ZFS.



If any module outside of the code that implements it needs to know 
about it,

then it is in the KPI and should be documented in the KPI documentation
(e.g. man 9)


Since the Filesystems need to know about this, it must be an externally
visible feature and therefore needs to be documented.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: /usr/obj is 11GB huge on FreeBSD 12-current

2017-12-27 Thread Julian Elischer

On 16/12/17 2:39 am, Konstantin Belousov wrote:



Put the following into /etc/src.conf:


This brings up two questions:
when to use make.conf and when to use src.conf,

and..


WITHOUT_PROFILE=yes
WITHOUT_DEBUG_FILES=yes
WITHOUT_TESTS=yes

which of the following is correct and why?

WITH_DEBUG_FILES=no
WITHOUT_DEBUG_FILES=yes

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: RFC how to use kernel procs/threads efficiently

2017-10-13 Thread Julian Elischer

On 10/10/17 8:33 pm, Rick Macklem wrote:

Julian Elischer wrote:
[stuff snipped]

On 10/10/17 4:25 am, Rick Macklem wrote:

--> As such, having a fixed reasonable # of threads is probably the best
that can be done.
- The current patch has the # of threads as a sysctl with a default of 
32.

why not set it to ncpu or something?

Well, each of these threads will do an RPC, which means a couple of short
bursts of CPU and then sleep the rest of the time waiting for the RPC reply
to come back from the Data Server.
As such, it would seem to me that you would want a lot more threads than
CPUs on the machine?
However, setting the default to "N * ncpu" seems better than just a fixed "32"
to me. (For nfsd, the current default is 8 * ncpu, so maybe that is a good
default for this too?)
yeah I really just meant "some function of ncpu"..  not specifically 
"ncpu x 1"

What do you think?

Thanks for the comment, rick





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: anyone know how to get rid of this error? (10.4/Gcc vs binutils port)

2017-10-03 Thread Julian Elischer

On 4/10/17 12:56 am, Ian Lepore wrote:

On Wed, 2017-10-04 at 00:18 +0800, Julian Elischer wrote:

Our 10.4 system is using gcc (for now).

when we compile the devel/binutils port, we get a failure with a
bunch
of these errors:


`_ZTSN12_GLOBAL__N_110Stub_tableILi64ELb1EEE' referenced in section
`.rodata' of aarch64.o: defined in discarded section
`.rodata._ZTSN12_GLOBAL__N_110Stub_tableILi64ELb1EEE[_ZTSN12_GLOBAL__
N_110Stub_tableILi64ELb1EEE]'
of aarch64.o
`_ZTSN12_GLOBAL__N_110Stub_tableILi64ELb0EEE' referenced in section
`.rodata' of aarch64.o: defined in discarded section
`.rodata._ZTSN12_GLOBAL__N_110Stub_tableILi64ELb0EEE[_ZTSN12_GLOBAL__
N_110Stub_tableILi64ELb0EEE]'
of aarch64.o


I managed to defeat these one before but I forget how.

possibly the answer is to use clang/clang++ for this item but I
tried
defining CC and CXX  to clang/clang++ in the Makefile but that
didn't
seem to help

there's probably a USE_CLANG option or something that I haven't seen.

We ran into the same thing recently at $work.  The root cause for us
involved having same-named classes in anonymous namespaces in different
compilation units.  The classes made reference to something that was
declared extern "C", bringing into play some rules about how C things
in anon namespaces really refer to the same global C object outside of
any namespace.  Then link-time optimization led to discarding the
object from one compilation unit, and that somehow resulted in
discarding the referenced extern "C" thing completely, even though it
was still referenced from the non-discarded instance of an object from
a different compilation unit.  Phew.

The same code has no problems with clang on freebsd 11, just with gcc
on 10.3.

So, for us the fix was a bit heavy-handed: we just renamed one of the
classes involved in the problem so that there were no longer any same-
named classes in anon namespaces in separate compilation units.
  Probably not a good option for you.  A fix involving compile options
might result in not discarding unreferenced segments at all, and with
templated code that might result in huge binaries.

Mixing clang-compiled and gcc-compiled c++ may give you grief with
exceptions and other things (it would on ARM on 10.x, but maybe not on
x86 arches).

-- Ian



thanks

the issue comes up in the binutils port which is a dependency of gdb.

I don't think we actually need to install the binutils port on our 
appliance, (and we only install gdb to generate backtraces on debug 
reports)


I just added CC=clang and CXX=clang++ in the makefile that called it 
and the problem seemed to go away.



All i wanted to do is get gdb compiled and I end up with gcc6, llvm 
and binutils (plus a whole lot more) as a bonus (plus an extra 30 
minutes of compile time)




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

anyone know how to get rid of this error? (10.4/Gcc vs binutils port)

2017-10-03 Thread Julian Elischer

Our 10.4 system is using gcc (for now).

when we compile the devel/binutils port, we get a failure with a bunch 
of these errors:



`_ZTSN12_GLOBAL__N_110Stub_tableILi64ELb1EEE' referenced in section 
`.rodata' of aarch64.o: defined in discarded section 
`.rodata._ZTSN12_GLOBAL__N_110Stub_tableILi64ELb1EEE[_ZTSN12_GLOBAL__N_110Stub_tableILi64ELb1EEE]' 
of aarch64.o
`_ZTSN12_GLOBAL__N_110Stub_tableILi64ELb0EEE' referenced in section 
`.rodata' of aarch64.o: defined in discarded section 
`.rodata._ZTSN12_GLOBAL__N_110Stub_tableILi64ELb0EEE[_ZTSN12_GLOBAL__N_110Stub_tableILi64ELb0EEE]' 
of aarch64.o



I managed to defeat these one before but I forget how.

possibly the answer is to use clang/clang++ for this item but I tried 
defining CC and CXX  to clang/clang++ in the Makefile but that didn't 
seem to help


there's probably a USE_CLANG option or something that I haven't seen.


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: extending the maximum filename length (pointer to patch)[request for input]

2017-09-12 Thread Julian Elischer

On 13/9/17 1:16 am, Jonathan Anderson wrote:

On 12 Sep 2017, at 14:38, Julian Elischer wrote:

“这是一个测试多字符文件名长度的文件目的是命名一个文件用中文或者日文或者韩文字符并且要求字符长度超过八十五个字符然后拷贝该文件到我们的共享文件夹看看是否能够拷贝该文件到我.txt” 

(I have no idea what that says but apparently it's a real filename 
from a windows machine that blew up when written via samba.)


Google Translate says, amusingly:
"This is a test file for the length of the file name. The purpose is 
to name a file in Chinese or Japanese or Korean characters and 
require the character to be longer than 85 characters and then copy 
the file into our shared folder to see if it can copy the file To 
me" (.txt, I guess)


No matter what number you choose for a path length, you're never 
going to win against that specific user. :)

ha!
it was give as an example to show us the limit.
Apparently however our company has difficulty selling in China due to 
this limit because Microsoft has a much larger limit and people use it.





Jon
--
Jonathan Anderson
jonat...@freebsd.org




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: extending the maximum filename length (pointer to patch)[request for input]

2017-09-12 Thread Julian Elischer

On 12/9/17 2:17 pm, Conrad Meyer wrote:

On Sat, Sep 9, 2017 at 9:09 AM, Julian Elischer <jul...@freebsd.org> wrote:

maybe we could get it into -current.
It'd be silly to have to have people re-inventing hte wheel all the time.
How about you put those changes into the reviews.freebsd.org and we can get
some general consensus on them.
We'll have to do similar for the Asian customers and anyone who uses UTF-8.
So it
would be silly to have to develop it all again (but subtly different of
course).

The key issue is how many system calls and other APIs would be broken,
and how many would be broken in a non backwards compatible way?

We would need it in a stable/10 and 11 branch but if the patch is isolated
enough we could carry it forward until we get to 12.

One has to allow people to do whatever they are used to with Windows.
And in this case the issue is serving files over samba to windows machines.

Hey Julian,

I've thrown the patch up at https://reviews.freebsd.org/D12330 .  I
haven't actually tested it on FreeBSD, but it does compile.  We also
have some patches against contrib/pjdfstest to fix those tests against
long file names, but I think we can hold off on those changes until
we've nailed down what the architectural change will be (if any).


thanks!

that looks a lot like a proof -of concept patch we derived a while 
back but never really tested.
The issue for us is that using UTF-8 the filenames become too short 
for common usage in China and Japan.

Apparently they routinely nae files with the contents of a small novella.

e.g.

“这是一个测试多字符文件名长度的文件目的是命名一个文件用中文或者日文或者韩文字符并且要求字符长度超过八十五个字符然后拷贝该文件到我们的共享文件夹看看是否能够拷贝该文件到我.txt”
(I have no idea what that says but apparently it's a real filename from a 
windows machine that blew up when written via samba.)

Does anyone else have any thoughts about whether FreeBSD 12 might grow longer 
path/filename support?
 (I'm told Linux uses 1K and 4096 for filenames and path length.)

Julian



It's quite possible this accidentally breaks even more APIs than
expected and we should do some fine tuning to reduce the damage.  Our
$WORK product mostly doesn't care about ABI, so we may not have
noticed some ABI breakage.

If anyone else is interested, please subscribe or add yourself as a
reviewer on the phabricator revision.

Best,
Conrad



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: extending the maximum filename length

2017-09-09 Thread Julian Elischer

On 9/9/17 1:28 am, Conrad Meyer wrote:

On Fri, Sep 8, 2017 at 10:15 AM, Julian Elischer <jul...@freebsd.org> wrote:

Has anyone using freeBSD ever increased NAME_MAX (filename maximum length)
and have any experience with it?

We ($JOB) would recompile the entire system so intra-system compatibility
would probably be ok, and we have our own filesystem which would support it.

But I wonder if anyone has tried it and hit unexpected problems.


reason:  Chinese and Japanese people who have gotten into the habit of
having a filename of 256 CHINESE/JAPANESE characters in Microsoft and want
to store their files on a FreeBSD based server witout having to rename
everything. (3 bytes for each of those characters giving a limit of 83
characters).

(though since each character is a word the names if I could read them must
be amazing)


NFS behaviour is one issue I don't know but would be interested in..  could
we SERVE such files?

Hey Julian,

We've done exactly this at Isilon.

Basically we bumped NAME_MAX and related constants out by 4x (maximum
length of a UTF-8 encoded Unicode character, in bytes).  So NAME_MAX
is 1023, PATH_MAX is 4096, and a bunch of follow-on equivalent/related
constants are bumped similarly.

To avoid breaking too many ABIs, we've retained a SHORT_NAME_MAX
constant of 255 which several non-filesystem NAME_MAX users were
converted to.  Stuff like module name APIs, procstat, etc.

Individual filesystems are free to implement and restrict maximum
component lengths in VOP_LOOKUP.  For us, we retained 255 bytes for
basically all filesystems (see e.g. r316509, r313475) aside from IFS
(the OneFS filesystem).  For IFS we check that component names are 255
("MAXNAMCHARS") unicode codepoints or fewer (not all names shorter
than NAME_MAX bytes are valid).

NFS commonly does not support >255 byte names, so we have a hack there
to export longer names as some shorter name plus a hash (total length
of 255 bytes or fewer) to uniquely identify the file.  SMB exports
longer names just fine.

Anyway, we'd love to shift some of these patches upstream, if there is
interest in supporting this kind of thing.

Sure, there are a few snags here and there and some ABIs change.  But
overall it seems to work pretty well.

Best,
Conrad


maybe we could get it into -current.
It'd be silly to have to have people re-inventing hte wheel all the time.
How about you put those changes into the reviews.freebsd.org and we 
can get some general consensus on them.
We'll have to do similar for the Asian customers and anyone who uses 
UTF-8. So it
would be silly to have to develop it all again (but subtly different 
of course).


The key issue is how many system calls and other APIs would be broken,
and how many would be broken in a non backwards compatible way?

We would need it in a stable/10 and 11 branch but if the patch is isolated
enough we could carry it forward until we get to 12.

One has to allow people to do whatever they are used to with Windows.
And in this case the issue is serving files over samba to windows 
machines.


Julian
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


extending the maximum filename length

2017-09-08 Thread Julian Elischer
Has anyone using freeBSD ever increased NAME_MAX (filename maximum 
length) and have any experience with it?


We ($JOB) would recompile the entire system so intra-system 
compatibility would probably be ok, and we have our own filesystem 
which would support it.


But I wonder if anyone has tried it and hit unexpected problems.


reason:  Chinese and Japanese people who have gotten into the habit of 
having a filename of 256 CHINESE/JAPANESE characters in Microsoft and 
want to store their files on a FreeBSD based server witout having to 
rename everything. (3 bytes for each of those characters giving a 
limit of 83 characters).


(though since each character is a word the names if I could read them 
must be amazing)



NFS behaviour is one issue I don't know but would be interested in..  
could we SERVE such files?



Julian


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: anyone had experience expanding uid_t and gid_t?

2017-08-21 Thread Julian Elischer

On 19/8/17 11:15 am, Julian Elischer wrote:

at $JOB there are clients where 32bits is starting to chafe.

Has anyone expanded them?

Other than a few offline comments I haven't heard anyone directly 
respond to this.

Does anyone have any comments on feasibility or suggestions?
NFSV3 will definitely be screwed..



This is starting to become a serious limitation in some places.

Especially large institutions with Samba active.

Samba uses a map between SIDs (session IDs) and UIDS, but it's a 
sparse map and due to various issues the mapping is not able to 
re-use numbers well.







___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


assigning priorities to swap partitions?

2017-08-21 Thread Julian Elischer

On an AZURE system there is a "local" device that is useful as swap.

It is, I believe, faster than regular network based storage, but it is 
ephemeral, and may go away during a shutdown.
It is in some machines a bit small so we'd like to add a bit more for 
safety.
But we would like the ephemeral local storage to be used first. Can 
this be done at all? I can't see anything


The last time I checked all swap was used in some balanced way. (the 
manual says so)
One solution is to have a small cron job that only creates the added 
swap partition when the first one is (say) 70% full,
but it'd be nice if there were some less hackish way. Another would be 
to occasionally wake up, and if the swap in use would all fit into the 
device we would like to be used, we do a swapoff on the other and 
force everything to be put on the fast drive..  but that sort of 
defeats the purpose as it's doing extra work..


Has anyone done any work on adding priority to swap?

Julian

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

mapping a device from device-id to /dev

2017-08-21 Thread Julian Elischer


I have the following (Azure) device (disk) id:

dev.storvsc..%pnpinfo: classid=32412632-86cb-44a2-9b5c-50d1417354f5 
deviceid=-0001-8899--

the question is:  "how can I map that do /dev/da1"..
I know that for my device it IS /dev/da1
but how can I prove it? there are so many mappings from one space to another 
I've totally lost track.
 there are acpi mappings, pnpmappngs, /dev/ mappings, PCI space mappings, 
things in sysctl, things in their own spaces, etc. etc.
In some architectures htere are fdt mappings and in some it's still pretty 
random from what I see.

Is there a document that covers all of these?



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


anyone had experience expanding uid_t and gid_t?

2017-08-18 Thread Julian Elischer

at $JOB there are clients where 32bits is starting to chafe.

Has anyone expanded them?


This is starting to become a serious limitation in some places.

Especially large institutions with Samba active.

Samba uses a map between SIDs (session IDs) and UIDS, but it's a 
sparse map and due to various issues the mapping is not able to re-use 
numbers well.





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [Help] Linux low level data structures < - > FreeBSD low level data structures

2017-06-04 Thread Julian Elischer

On 4/6/17 7:07 pm, blubee blubeeme wrote:

Hello

Is there anyone on either of these lists that have experience with both
linux low level data structures and their equivalents on FreeBSD?

For instance the linux header file:


which includes the header file:


Then looking at that file:







You are going to have to be a lot more specific about this.
I have worked in several places where they use s shim layer to make 
Linux basic services work on freeBSD.

usually  a mix of functions, macros and inlines.
However you need to narrow down your questions a bit as the POSSIBLE 
scope of your question is too large for anyone to attempt an answer.


Remember that both systems are POSIX inspired so outside the kernel 
there are many more simlarities than one might be led to expect,

 but you need to be way more specific.
It's even possible to write kernel code to run on both, but it is 
usually domain specific.





I'll be doing a lot of work trying to find these FreeBSD equivalent of
these types of files to port some code.

Does anyone here have experience with something like this? Is there any
other projects that maps these low level data structures from
Linux <-> FreeBSD, etc?

Best,
Owen
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Time to increase MAXPHYS?

2017-06-03 Thread Julian Elischer

On 4/6/17 4:59 am, Colin Percival wrote:

On January 24, 1998, in what was later renumbered to SVN r32724, dyson@
wrote:

Add better support for larger I/O clusters, including larger physical
I/O.  The support is not mature yet, and some of the underlying implementation
needs help.  However, support does exist for IDE devices now.

and increased MAXPHYS from 64 kB to 128 kB.  Is it time to increase it again,
or do we need to wait at least two decades between changes?

This is hurting performance on some systems; in particular, EC2 "io1" disks
are optimized for 256 kB I/Os, EC2 "st1" (throughput optimized spinning rust)
disks are optimized for 1 MB I/Os, and Amazon's NFS service (EFS) recommends
using a maximum I/O size of 1 MB (and despite NFS not being *physical* I/O it
seems to still be limited by MAXPHYS).


We increase it in freebsd 8 and 10.3 on our systems,  Only good results.

sys/sys/param.h:#define MAXPHYS (1024 * 1024)   /* max raw I/O 
transfer size */


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Ssh.. can we please have HPN back?

2017-05-22 Thread Julian Elischer

On 23/5/17 12:13 am, Allan Jude wrote:

On 2017-05-22 03:50, Julian Elischer wrote:

On 22/5/17 2:20 pm, Allan Jude wrote:

On 2017-05-18 22:28, Julian Elischer wrote:

So after stripping out the HPN version of ssh from our product becasue
"it was no longer needed" we dicovered that we were premature in
doing so.
Apparently ssh still really needs HPN to get any throughput at all when
there are latencies involved.


For example, with HPN we get 13MB/sec between the Azure US west
Data center and the Azure East data center.But the standard ssh in 10.3
(with HPN stripped out) can barely manage 2MB/sec transfers.

I did ask at the time whether it was proved that the new ssh didn't
require the HPN changes,
and was assured, "no" but it would appear that the picture isn't as
clear.
tht seems silly to have to import the port when we have what would
otherwise be a
perfectly good ssh as part of hte system, and it's really annoying
having to specify
/usr/local/bin/scp  or /usr/local/bin/ssh in every script.

So can we please have the latest version of the HPN changes back in the
default system please?
It seem rather odd that the upstream openssh has had this problem for SO
LONG and not fixed it.

Julian


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to
"freebsd-current-unsubscr...@freebsd.org"

I have this stand-alone patch ready now:

https://github.com/openssh/openssh-portable/compare/master...allanjude:V_7_5_dynamic_window


In my benchmarks with 100ms of latency (from dummynet) is increases SSH
send throughput from 1 megabyte/sec to 225 megabytes/sec provided a
large enough socket buffer.

Still seeing lesser performance on the recv case, working on it.

We have two patches of our own that upstream have ignored   an option to
set NoDelay   and a pushwindow setting option
(though I'm not sure the second is operational in the current code, it
does apply with no errors...)

The NoDelay options massively speeds up the case where you have chatty
back and forth traffic (client/server) through a ssh tunnel.

Are your changes against the sources in current?  what about stable/10?


That change is against upstream's latest release, but should apply
cleanly to any version of OpenSSH from the past 10 years that does not
have the HPN patches applied already.

However, the function channel_check_window() in stable/10 is currently
the same as the latest upstream since the commit that removed HPN. The
function is actually unchanged from upstream if you go back far enough
to before we added HPN as well.

If you revert both HPN commits, the most recent change to that function
is Aug 2008, when upstream introduced the 'move the window forward every
3 packets' thing that seams to be the main cause of the window scaling
problem in the first place. I think the nodelay option might be what
mitigates that, as it causes rapid back and forth and never allows the
window to grow to even the 2mb allowed by SSH since version 4.7



great

I'll look at applying it at $JOB and let you know what happens .. 
assuming I can get the patch downloaded and applied.



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: The futur of the roff toolchain

2017-05-22 Thread Julian Elischer

On 21/5/17 8:57 pm, Baptiste Daroussin wrote:

Hi all,


[...]


No the problem left is documentations available in share/doc.

I would like to push them elsewhere. Those documents are mostly useful for
historical reason (hence we want to keep them) but not really for daily use of
modern FreeBSD.
Another issue with those documentation, they are installed as text/ascii version
in base, which makes most of them not really readable (as the documents has not
be written for a ascii/text target but more for a PDF/html view - using pic(1)
for example)


I was surprised that they stayed in src when we split it up.
And really they should be supplemented by every FreeBSD based paper 
presented at a conference :-)




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Ssh.. can we please have HPN back?

2017-05-22 Thread Julian Elischer

On 22/5/17 2:20 pm, Allan Jude wrote:

On 2017-05-18 22:28, Julian Elischer wrote:

So after stripping out the HPN version of ssh from our product becasue
"it was no longer needed" we dicovered that we were premature in doing so.
Apparently ssh still really needs HPN to get any throughput at all when
there are latencies involved.


For example, with HPN we get 13MB/sec between the Azure US west
Data center and the Azure East data center.But the standard ssh in 10.3
(with HPN stripped out) can barely manage 2MB/sec transfers.

I did ask at the time whether it was proved that the new ssh didn't
require the HPN changes,
and was assured, "no" but it would appear that the picture isn't as clear.
tht seems silly to have to import the port when we have what would
otherwise be a
perfectly good ssh as part of hte system, and it's really annoying
having to specify
/usr/local/bin/scp  or /usr/local/bin/ssh in every script.

So can we please have the latest version of the HPN changes back in the
default system please?
It seem rather odd that the upstream openssh has had this problem for SO
LONG and not fixed it.

Julian


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

I have this stand-alone patch ready now:

https://github.com/openssh/openssh-portable/compare/master...allanjude:V_7_5_dynamic_window

In my benchmarks with 100ms of latency (from dummynet) is increases SSH
send throughput from 1 megabyte/sec to 225 megabytes/sec provided a
large enough socket buffer.

Still seeing lesser performance on the recv case, working on it.


We have two patches of our own that upstream have ignored   an option 
to set NoDelay   and a pushwindow setting option
(though I'm not sure the second is operational in the current code, it 
does apply with no errors...)


The NoDelay options massively speeds up the case where you have chatty 
back and forth traffic (client/server) through a ssh tunnel.


Are your changes against the sources in current?  what about stable/10?





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Ssh.. can we please have HPN back?

2017-05-18 Thread Julian Elischer

So after stripping out the HPN version of ssh from our product becasue
"it was no longer needed" we dicovered that we were premature in doing so.
Apparently ssh still really needs HPN to get any throughput at all when
there are latencies involved.


For example, with HPN we get 13MB/sec between the Azure US west
Data center and the Azure East data center.But the standard ssh in 10.3
(with HPN stripped out) can barely manage 2MB/sec transfers.

I did ask at the time whether it was proved that the new ssh didn't 
require the HPN changes,

and was assured, "no" but it would appear that the picture isn't as clear.
tht seems silly to have to import the port when we have what would 
otherwise be a
perfectly good ssh as part of hte system, and it's really annoying 
having to specify

/usr/local/bin/scp  or /usr/local/bin/ssh in every script.

So can we please have the latest version of the HPN changes back in 
the default system please?
It seem rather odd that the upstream openssh has had this problem for 
SO LONG and not fixed it.


Julian


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: bootcode capable of booting both UFS and ZFS? (Amazon/ec2)

2017-05-07 Thread Julian Elischer

On 7/5/17 1:45 pm, Warner Losh wrote:

On Sat, May 6, 2017 at 10:03 PM, Julian Elischer <jul...@freebsd.org> wrote:

On 6/5/17 4:01 am, Toomas Soome wrote:



On 5. mai 2017, at 22:07, Julian Elischer <jul...@freebsd.org
<mailto:jul...@freebsd.org>> wrote:

Subject says it all really, is this an option at this time?

we'd like to try boot the main zfs root partition and then fall back to a
small UFS based recovery partition.. is that possible?

I know we could use grub but I'd prefer keep it in the family.





it is, sure. but there is an compromise to be made for it.

Lets start with what I have done in illumos port, as the idea there is
exactly about having as “universal” binaries as possible (just the binaries
are listed below to get the size):

-r-xr-xr-x   1 root sys   171008 apr 30 19:55 bootia32.efi
-r-xr-xr-x   1 root sys   148992 apr 30 19:55 bootx64.efi
-r--r--r--   1 root sys 1255 okt 25  2015 cdboot
-r--r--r--   1 root sys   154112 apr 30 19:55 gptzfsboot
-r-xr-xr-x   1 root sys   482293 mai  2 21:10 loader32.efi
-r-xr-xr-x   1 root sys   499218 mai  2 21:10 loader64.efi
-r--r--r--   1 root sys  512 okt 15  2015 pmbr
-r--r--r--   1 root sys   377344 mai  2 21:10 pxeboot
-r--r--r--   1 root sys   376832 mai  2 21:10 zfsloader

the loader (bios/efi) is built with full complement - zfs, ufs, dosfs,
cd9660, nfs, tftp + gzipfs. The cdboot is starting zfsloader (thats trivial
string change).

The gptzfsboot in illumos case is only built with zfs, dosfs and ufs - as
it has to support only disk based media to read out the loader. Also I am
building gptzfsboot with libstand and libi386 to get as much shared code as
possible - which has both good and bad sides, as usual;)

The gptzfsboot size means that with ufs the dedicated boot partition is
needed (freebsd-boot), with zfs the illumos port is always using the 3.5MB
boot area after first 2 labels (as there is no geli, the illumos does not
need dedicated boot partition with zfs).

As the freebsd-boot is currently created 512k, the size is not an issue.
Also using common code does allow the generic partition code to be used, so
GPT/MBR/BSD (VTOC in illumos case) labels are not problem.


So, even just with cd boot (iso), starting zfsloader (which in fbsd has
built in ufs, zfs etc), you already can get rescue capability.

Now, even with just adding ufs reader to gptzfsboot, we can use gpt +
freebsd-boot and ufs root but loading zfsloader on usb image, so it can be
used for both live/install and rescue, because zfsloader itself has support
for all file systems + partition types.

I have kept myself a bit off from freebsd gptzfsboot because of simple
reason - the older setups have smaller size for freebsd boot, and not
everyone is necessarily happy about size changes:D also in freebsd case
there is another factor called geli - it most certainly does contribute some
bits, but also needs to be properly addressed on IO call stack (as we have
seen with zfsbootcfg bits). But then again, here also the shared code can
help to reduce the complexity.

Yea, the zfsloader/loader*.efi in that listing above is actually built
with framebuffer code and compiled in 8x16 default font (lz4 compressed
ascii+boxdrawing basically - because zfs has lz4, the decompressor is always
there), and ficl 4.1, so thats a bit of difference from fbsd loader.

Also note that we can still build the smaller dedicated blocks like boot2,
just that we can not use those blocks for more universal cases and
eventually those special cases will diminish.


thanks for that..

  so, here's my exact problem I need to solve.
FreeBSD 10 (or newer) on Amazon EC2.
We need to have a plan for recovering the scenario where somethign goes
wrong (e.g. during an upgrade) and we are left with a system where the
default zpool rootfs points to a dataset that doesn't boot. It is possible
that mabe the entire pool is unbootable into multi-user..  Maybe somehow it
filled up? who knows. It's hard to predict future problems.
There is no console access at all so there is no possibility of human
intervention. So all recovery paths that start "enter single user mode
and" are unusable.

The customers who own the amazon account are not crazy about giving us the
keys to the kingdom as far as all their EC2 instances, so taking a root
drive off a 'sick' VM and grafting it onto a freebsd instance to 'repair' it
becomes a task we don't want to really have to ask them to do. They may not
have the in-house expertise to do it. confidently.

This leaves us with automatic recovery, or at least automatic methods of
getting access to that drive from the network.
Since the regular root is zfs, my gut feeling is that to deduce the chances
of confusion during recovery, I'd like the (recovery) system itself to be
running off a UFS partition, and potentially, with a memory root filesystem.
As long as it can be reached over the n

Re: bootcode capable of booting both UFS and ZFS? (Amazon/ec2)

2017-05-06 Thread Julian Elischer

On 6/5/17 4:01 am, Toomas Soome wrote:


On 5. mai 2017, at 22:07, Julian Elischer <jul...@freebsd.org 
<mailto:jul...@freebsd.org>> wrote:


Subject says it all really, is this an option at this time?

we'd like to try boot the main zfs root partition and then fall 
back to a small UFS based recovery partition.. is that possible?


I know we could use grub but I'd prefer keep it in the family.






it is, sure. but there is an compromise to be made for it.

Lets start with what I have done in illumos port, as the idea there 
is exactly about having as “universal” binaries as possible (just 
the binaries are listed below to get the size):


-r-xr-xr-x   1 root sys   171008 apr 30 19:55 bootia32.efi
-r-xr-xr-x   1 root sys   148992 apr 30 19:55 bootx64.efi
-r--r--r--   1 root sys 1255 okt 25  2015 cdboot
-r--r--r--   1 root sys   154112 apr 30 19:55 gptzfsboot
-r-xr-xr-x   1 root sys   482293 mai  2 21:10 loader32.efi
-r-xr-xr-x   1 root sys   499218 mai  2 21:10 loader64.efi
-r--r--r--   1 root sys  512 okt 15  2015 pmbr
-r--r--r--   1 root sys   377344 mai  2 21:10 pxeboot
-r--r--r--   1 root sys   376832 mai  2 21:10 zfsloader

the loader (bios/efi) is built with full complement - zfs, ufs, 
dosfs, cd9660, nfs, tftp + gzipfs. The cdboot is starting zfsloader 
(thats trivial string change).


The gptzfsboot in illumos case is only built with zfs, dosfs and ufs 
- as it has to support only disk based media to read out the loader. 
Also I am building gptzfsboot with libstand and libi386 to get as 
much shared code as possible - which has both good and bad sides, as 
usual;)


The gptzfsboot size means that with ufs the dedicated boot partition 
is needed (freebsd-boot), with zfs the illumos port is always using 
the 3.5MB boot area after first 2 labels (as there is no geli, the 
illumos does not need dedicated boot partition with zfs).


As the freebsd-boot is currently created 512k, the size is not an 
issue. Also using common code does allow the generic partition code 
to be used, so GPT/MBR/BSD (VTOC in illumos case) labels are not 
problem.



So, even just with cd boot (iso), starting zfsloader (which in fbsd 
has built in ufs, zfs etc), you already can get rescue capability.


Now, even with just adding ufs reader to gptzfsboot, we can use gpt 
+ freebsd-boot and ufs root but loading zfsloader on usb image, so 
it can be used for both live/install and rescue, because zfsloader 
itself has support for all file systems + partition types.


I have kept myself a bit off from freebsd gptzfsboot because of 
simple reason - the older setups have smaller size for freebsd boot, 
and not everyone is necessarily happy about size changes:D also in 
freebsd case there is another factor called geli - it most certainly 
does contribute some bits, but also needs to be properly addressed 
on IO call stack (as we have seen with zfsbootcfg bits). But then 
again, here also the shared code can help to reduce the complexity.


Yea, the zfsloader/loader*.efi in that listing above is actually 
built with framebuffer code and compiled in 8x16 default font (lz4 
compressed ascii+boxdrawing basically - because zfs has lz4, the 
decompressor is always there), and ficl 4.1, so thats a bit of 
difference from fbsd loader.


Also note that we can still build the smaller dedicated blocks like 
boot2, just that we can not use those blocks for more universal 
cases and eventually those special cases will diminish.


thanks for that..

 so, here's my exact problem I need to solve.
FreeBSD 10 (or newer) on Amazon EC2.
We need to have a plan for recovering the scenario where somethign 
goes wrong (e.g. during an upgrade) and we are left with a system 
where the default zpool rootfs points to a dataset that doesn't boot. 
It is possible that mabe the entire pool is unbootable into 
multi-user..  Maybe somehow it filled up? who knows. It's hard to 
predict future problems.
There is no console access at all so there is no possibility of human 
intervention. So all recovery paths that start "enter single user mode 
and" are unusable.


The customers who own the amazon account are not crazy about giving us 
the keys to the kingdom as far as all their EC2 instances, so taking a 
root drive off a 'sick' VM and grafting it onto a freebsd instance to 
'repair' it becomes a task we don't want to really have to ask them to 
do. They may not have the in-house expertise to do it. confidently.


This leaves us with automatic recovery, or at least automatic methods 
of getting access to that drive from the network.
Since the regular root is zfs, my gut feeling is that to deduce the 
chances of confusion during recovery, I'd like the (recovery) system 
itself to be running off a UFS partition, and potentially, with a 
memory root filesystem. As long as it can be reached over the network 
we can then take over.


we'd also like to have the bo

bootcode capable of booting both UFS and ZFS?

2017-05-05 Thread Julian Elischer

Subject says it all really, is this an option at this time?

we'd like to try boot the main zfs root partition and then fall back 
to a small UFS based recovery partition.. is that possible?


I know we could use grub but I'd prefer keep it in the family.



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: process killed: text file modification

2017-04-16 Thread Julian Elischer

On 13/4/17 5:45 am, Rick Macklem wrote:

I have just committed a patch to head (r316745) which should fix this.
(It includes code to handle the recent change to head to make the pageouts
  write through the buffer cache.)

It will be MFC'd and should be in 11.1.


is there any relevance of this change to stable/10?



Thanks everyone, for your help with this, rick

From: owner-freebsd-curr...@freebsd.org  on behalf 
of Rick Macklem 
Sent: Friday, March 24, 2017 4:14:45 PM
To: Konstantin Belousov
Cc: Gergely Czuczy; Dimitry Andric; Ian Lepore; FreeBSD Current
Subject: Re: process killed: text file modification

I can't do commits until I get home in mid-April.

That's why it will be waiting until then.

It should make it into stable/11 in plenty of time for 11.1.

Thanks for your help with this, rick

From: owner-freebsd-curr...@freebsd.org  on behalf 
of Konstantin Belousov 
Sent: Friday, March 24, 2017 3:01:41 AM
To: Rick Macklem
Cc: Gergely Czuczy; Dimitry Andric; Ian Lepore; FreeBSD Current
Subject: Re: process killed: text file modification

On Thu, Mar 23, 2017 at 09:39:00PM +, Rick Macklem wrote:

Try whatever you like. However, if the case that failed before doesn't fail now,
I'd call the test a success.

Thanks, rick
ps: It looks like Kostik is going to work on converting the NFS vop_putpages() 
to
   using the buffer cache. However, if this isn't ready for head/current by 
mid-April,
   I will commit this patch to help fix things in the meantime.

I do not see a reason to wait for my work before committing your patch.
IMO, instead, it should be committed ASAP and merged into stable/11 for
upcoming 11.1.



I will do required adjustments if/when _putpages() patch progresses enough.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: effect of strip(1) on du(1)

2017-03-21 Thread Julian Elischer

On 3/3/17 8:31 am, Rodney W. Grimes wrote:

On Fri, Mar 3, 2017 at 2:04 AM, Peter Jeremy  wrote:

On 2017-Mar-02 22:29:46 +0300, Subbsd  wrote:

During some interval after strip call, du will show 512B for any file.
If execute du(1) after strip(1) without delay, this behavior is reproduced 100%:

What filesystem are you using?  strip(1) rewrites the target file and du(1)
reports the number of blocks reported by stat(2).  It seems that you are
hitting a situation where the file metadata isn't immediately updated.

--
Peter Jeremy


Got it. My filesystem is ZFS. Looks like when ZFS open and write data
to file, we get wrong number of blocks during a small interval after
writing. Thanks for pointing this out!

Even if that is the case file system cache effects should NOT be
visible to a userland process.   This is NOT as if your running
2 different processing beating on a file.  Your test cases are
serialially syncronous shell invoked commands seperated with
&& the results should be exact and predictable.

When strip returns the operation from the userland perspecive
is completed and any and all processeses started after that
should have the view of the completed strip command.

This IS a bug.


actually it's all in how you look at it.
Due to the way ZFS is doing the work and the metadata transitions, 
that amount of storage is actually directly attributable to that 
file's existence.

so from that perspective the du is correct.


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Fwd: Re: I/O semantics of pipe and FIFO.

2017-03-04 Thread Julian Elischer


an interesting point to discuss? is our behaviour in this test right?
   from: "austin-group mailng list (posix standard discussion)"

-- rest of email is quoted ---
On 5/3/17 5:48 am, Stephane Chazelas wrote:

2017-03-04 13:14:08 +, Danny Niu:

Hi all.

I couldn't remember where I saw it saying, that when reading
from a pipe or a FIFO, the read syscall returns the content of
at most one write call. It's a bit similar to the
message-nondiscard semantics of dear old STREAM.

Currently, I'm reading through the text to find out a bit
more, and I appreciate a bit of pointer on this.

[...]

(echo x; echo y) | (sleep 1; dd count=1 2> /dev/null)

outputs both x and y in all of Linux, FreeBSD and Solaris in my
tests.

That a read wouldn't read what's currently in the pipe would be
quite surprising.

I also wouldn't expect pipes to store the writes as individual
separate message but use one buffer.

In:

(
  dd bs=4 count=1 if=/dev/zero 2> /dev/null
  echo first through >&2
  dd bs=4 count=1 if=/dev/zero 2> /dev/null
  echo second through >&2
) | (sleep 1; dd bs=10 count=1 2> /dev/null) | wc -c

That is where the second write blocks because the pipe is full,
the reading dd still reads both writes in Linux and Solaris in
my tests (on Solaris (10 on amd64 at least), reduce to 2
instead of 4 or both writes would block).

On FreeBSD, I get only the first write (using 8000 followed by
1 for instance).

FreeBSD is also the only one of the three where

dd bs=100 count=1 if=/dev/zero | dd bs=100 count=1 | wc -c

Doesn't output 100. The others schedule both processes back
and forth during their write() and read() system call while the
pipe is being filled and emptied several times.

--
Stephane

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: CFLAGS for certain ports

2017-03-03 Thread Julian Elischer

On 2/3/17 8:58 pm, Dimitry Andric wrote:

On 2 Mar 2017, at 12:02, Mingo Rrubioer  wrote:

I would like to see how well FreeBSD does as a workstation OS in the
HPC world due to its stability and reliability, as well as LLVM/clang.
I would like to know if FreeBSD has something similar to Gentoo's
/etc/portage/make.conf file and /etc/portage/package.use/* files in
order to compile certain ports with certain compiler flags.

It doesn't, though it would certainly be nice to have something like it
at some point.  The current idiom is to put something similar to the
following in your /etc/make.conf:

.if ${.CURDIR:M/usr/ports/foo/bar}
CFLAGS+= [... flags for the foo/bar port ...]
.endif

.if ${.CURDIR:M/usr/ports/what/ever}
CFLAGS+= [... flags for the what/ever port ...]
.endif



Regarding LLVM/clang, I've been reading the documentation and found
these flags: -arch=, -march=, -mcpu=,
--target=, target-cpu . I'm not quite sure which
one would be the one to use. In case someone wants to know, my initial
play/test machine has this processor:

CPU: Intel(R) Core(TM) i7-3820 CPU @ 3.60GHz (3600.11-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x206d7  Family=0x6  Model=0x2d  Stepping=7

And I'm currently running: 11.0-RELEASE-p8.

So I imagine I should use something like CFLAGS+= -march=corei7-avx
-march=sandybridge -target-cpu. Is that correct?

Don't specify -march or -mcpu directly, but add the following line to
/etc/make.conf:

CPUTYPE?= native

This will take care of everything automatically.  See also make.conf(5).

-Dimitry

Many many 3rd party packages use a consistent set of variables fed 
into the automumble tools.
Only today I was pleasantly surprised to see the lftp port pick up my 
CFLAGS and LDFLAGS changes without me having to change anything in the 
ports themselves..


we should investigate what these defacto stndards are and document them


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: confusing KTR_SCHED traces

2017-03-01 Thread Julian Elischer

On 18/2/17 2:48 am, Andriy Gapon wrote:

First, an example, three consecutive entries for the same thread (from top to
bottom):
KTRGRAPH group:"thread", id:"zio_write_intr_3 tid 100260", state:"sleep",
attributes: prio:84, wmesg:"-", lockname:"(null)"
KTRGRAPH group:"thread", id:"zio_write_intr_3 tid 100260", state:"spinning",
attributes: lockname:"sched lock 1"
KTRGRAPH group:"thread", id:"zio_write_intr_3 tid 100260", state:"running",
attributes: none

Any automatic analysis tool including schedgraph.py will assume that the thread
ends up in the running state.  In reality, of course, the thread is in the
sleeping state.
The confusing trace is a result of logging the thread's intention to switch out
in mi_switch() before calling sched_switch().  In ULE's sched_switch() we
acquire the "TDQ_LOCK" which could be contested.  In that case the thread spins
waiting for the lock to be released.  This is reported as "spinning" and then
"running" states.

I would like to fix that, but not sure how to do that best.
One idea is to move the mi_switch() trace closer to the cpu_switch() call
similarly to DTrace sched:cpu-off and sched:cpu-on probes.


I think that is the way to fix it


Any suggestions are welcome.
Thanks!



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [IGNORE] ldd linker script /usr/lib/libc.so fail [IGNORE]

2017-01-29 Thread Julian Elischer
Tracked this down to a rogue copy of libc.so in an unexpected place 
which was being found earlier than the real one.



On 30/1/17 1:13 am, Julian Elischer wrote:

Hi

the linker script /usr/lib/libc.so fails when you are using the 
--sysroot options because it


contains absolute paths.


Does anyone know if there is a way to add the sysroot to the script?

currently teh on ein our sysroot looks like:

$ cat /usr/build/buildroot/tools/x86_FBSD1X_gcc4.2.4/usr/lib/libc.so
/* $FreeBSD$ */
GROUP ( /lib/libc.so.7 /usr/lib/libc_nonshared.a 
/usr/lib/libssp_nonshared.a )


but I'd like to do something like:

GROUP ( ${sysroot}/lib/libc.so.7 ${sysroot}/usr/lib/libc_nonshared.a 
${sysroot}/usr/lib/libssp_nonshared.a )


but don't think I can do that

from what I see below however it shouldn't be needed.

Is this a bug in our version of ld? or am I misreading it?


I quote from one such source :

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Using_ld_the_GNU_Linker/simple-commands.html 




INPUT(file, file, …), INPUT(file file …)

   The INPUT command directs the linker to include the named files in
   the link, as though they were named on the command line.

   For example, if you always want to include subr.o any time you do
   a link, but you can't be bothered to put it on every link command
   line, then you can put INPUT (subr.o) in your linker script.

   In fact, if you like, you can list all of your input files in the
   linker script, and then invoke the linker with nothing but a -T
   option.

   In case a /sysroot prefix/ is configured, and the filename starts
   with the / character, and the script being processed was located
   inside the /sysroot prefix/, the filename will be looked for in
   the /sysroot prefix/. Otherwise, the linker will try to open the
   file in the current directory. If it is not found, the linker will
   search through the archive library search path. See the
   description of -L in Section 3.1 /Command Line Options/
<https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Using_ld_the_GNU_Linker/invocation.html#OPTIONS>.


   If you use INPUT (-lfile), ld will transform the name to
   libfile.a, as with the command line argument -l.

   When you use the INPUT command in an implicit linker script, the
   files will be included in the link at the point at which the
   linker script file is included. This can affect archive searching.

GROUP(file, file, …), GROUP(file file …)

   The GROUP command is like INPUT, except that the named files
   should all be archives, and they are searched repeatedly until no
   new undefined references are created.

   =


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

ldd linker script /usr/lib/libc.so fail

2017-01-29 Thread Julian Elischer

Hi

the linker script /usr/lib/libc.so fails when you are using the 
--sysroot options because it


contains absolute paths.


Does anyone know if there is a way to add the sysroot to the script?

currently teh on ein our sysroot looks like:

$ cat /usr/build/buildroot/tools/x86_FBSD1X_gcc4.2.4/usr/lib/libc.so
/* $FreeBSD$ */
GROUP ( /lib/libc.so.7 /usr/lib/libc_nonshared.a 
/usr/lib/libssp_nonshared.a )


but I'd like to do something like:

GROUP ( ${sysroot}/lib/libc.so.7 ${sysroot}/usr/lib/libc_nonshared.a 
${sysroot}/usr/lib/libssp_nonshared.a )


but don't think I can do that

from what I see below however it shouldn't be needed.

Is this a bug in our version of ld? or am I misreading it?


I quote from one such source :

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Using_ld_the_GNU_Linker/simple-commands.html


INPUT(file, file, …), INPUT(file file …)

   The INPUT command directs the linker to include the named files in
   the link, as though they were named on the command line.

   For example, if you always want to include subr.o any time you do
   a link, but you can't be bothered to put it on every link command
   line, then you can put INPUT (subr.o) in your linker script.

   In fact, if you like, you can list all of your input files in the
   linker script, and then invoke the linker with nothing but a -T
   option.

   In case a /sysroot prefix/ is configured, and the filename starts
   with the / character, and the script being processed was located
   inside the /sysroot prefix/, the filename will be looked for in
   the /sysroot prefix/. Otherwise, the linker will try to open the
   file in the current directory. If it is not found, the linker will
   search through the archive library search path. See the
   description of -L in Section 3.1 /Command Line Options/
   
.


   If you use INPUT (-lfile), ld will transform the name to
   libfile.a, as with the command line argument -l.

   When you use the INPUT command in an implicit linker script, the
   files will be included in the link at the point at which the
   linker script file is included. This can affect archive searching.

GROUP(file, file, …), GROUP(file file …)

   The GROUP command is like INPUT, except that the named files
   should all be archives, and they are searched repeatedly until no
   new undefined references are created.

   =


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: gptzfsboot grew a lot after skein support was added; need knob to control bloat

2017-01-28 Thread Julian Elischer

On 28/1/17 1:35 am, Allan Jude wrote:

On 2017-01-27 12:33, Shawn Webb wrote:

On Fri, Jan 27, 2017 at 12:30:17PM -0500, Allan Jude wrote:

On 2017-01-27 12:05, Warner Losh wrote:

On Fri, Jan 27, 2017 at 12:34 AM, Toomas Soome  wrote:

On 27. jaan 2017, at 1:40, Ngie Cooper (yaneurabeya)  
wrote:

Hi,
   I tried upgrading one of my workstations and unfortunately the 
freebsd-boot partition is too small (I follow manpage directions, exactly, and 
those seem to be too small as of 10.3-RELEASE timeframe), and I don???t have 
enough space or ability to resize the partition and make it bigger. So, I???m 
in need of a build knob to control the bloat, and/or having an alternative boot 
loader without geli/skein/crypto support compiled in. Would you be opposed to 
the work?
Thanks,
-Ngie


I do agree that since the geli knob is already there, it may do. Of course we 
also can think of additional knobs, but there is an issue - it wont help just 
to exclude some files, the additional features also do sit in the code, so the 
replacement stubs will be needed, also testing them all over will take some 
time. And the preprocessor spaghetti really is nasty thing to deal with;)

And then there is another issue (partly why I did the feature support in first 
place) - as the kernel does not block user from enabling the features, the user 
can end up facing non-bootable setup which is also not good, as user is using 
perfectly legal options, and still the whole thing is just rendered unusable???

I'm curious why you can't find the space for a bigger partition?
Almost all drives these days are partitioned with a little wasted
space, and that wasted space should be more than enough to cover us
here. Also, most drives have a swap partition that can be shrunk a
trivial amount to get space for this...

Warner


I need to do some testing to make a recipe that works for it, but the
other option is to use the ZFS bootcode area.

ZFS it self, reserves something like 3.5 mb of space in the ZFS
partition, for boot code. This is how we boot ZFS on MBR.

It should be possible to use this on GPT as well, we just don't.

In the future, maybe it'd be a good idea for the installer to leave
more space (a few MB, perhaps?) between the freebsd-boot and
freebsd-swap partitions? At least, for ZFS installs.

Thanks,


The PMBR code has a limitation for 536kb, and it all has to fit under
the 640k barrier, so the current 512kb size is plenty. The issue is some
people are upgrading from systems that were isntalled long ago, when
64kb or less was the default.

with 512K we can append a copy of FreeBSD1.0 on the end of the 
bootblock and leave us the option of bringing up a networked NFS based 
system for debugging.


maybe we could fall back to that after a crash...



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: gptzfsboot grew a lot after skein support was added; need knob to control bloat

2017-01-28 Thread Julian Elischer

On 28/1/17 4:16 am, Ngie Cooper wrote:

On Jan 27, 2017, at 09:05, Warner Losh  wrote:

...


I'm curious why you can't find the space for a bigger partition?
Almost all drives these days are partitioned with a little wasted
space, and that wasted space should be more than enough to cover us
here. Also, most drives have a swap partition that can be shrunk a
trivial amount to get space for this...

Unfortunately, in my infinite wisdom (IIRC) I put the zfs partition before the 
swap partition.

We have a similar problem at work with sys/boot unfortunately, but that's a 
side discussion for another time/place.

Thank you for the idea though -- I'll check when I get back to work.


at $JOB we are just testing a script that expands the root zfs 
partition on in-field appliances by shaving a bit off swap and 
cannibalising a small data partition we don't really use. I see we 
only left 64K for the boot part. It's big enough for us for now, but 
possibly we should fix that as well.
We have a mirror setup for system disks so we have the ability to take 
each system drive offline one at a time and rearrange it and then 
re-add the root partition to the mirror.
What are the chances a regular gpt+ZFS (no encrypt) bootblock will 
grow over 64K?





-Ngie
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: r312602: panic: Panic String: __lockmgr_args: unknown lockmgr request 0x0

2017-01-22 Thread Julian Elischer

On 22/01/2017 4:51 AM, O. Hartmann wrote:

Am Sat, 21 Jan 2017 21:13:49 +0100
Mateusz Guzik  schrieb:


On Sat, Jan 21, 2017 at 08:45:55PM +0100, O. Hartmann wrote:

The most recent CURRENT panics spontanously and crashes with the error message:

  
Panic String: __lockmgr_args: unknown lockmgr request 0x0
   

That's a braino in r312600, will be fixed shortly.


??? What is a "braino"?


it's like a typo but more in the head than in the fingers.

The sort of error you make when your wife/girlfriend/boss/kids 
distract you right in the middle of some bit of code and you forget 
that you just decided to reverse the sense of the conditional, and 
complete the rest of the code backwards.


Some of us older folks don't need an external source of confusion.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: recent change to vim defaults?

2017-01-18 Thread Julian Elischer

On 19/01/2017 11:18 AM, Julian Elischer wrote:

On 19/01/2017 1:37 AM, Adam Weinberger wrote:
On 18 Jan, 2017, at 10:35, Julian Elischer <jul...@freebsd.org> 
wrote:


On 17/01/2017 1:23 AM, Adam Weinberger wrote:
On 16 Jan, 2017, at 9:25, Baptiste Daroussin <b...@freebsd.org> 
wrote:


On Mon, Jan 16, 2017 at 12:03:08AM +0800, Julian Elischer wrote:
I noticed that suddenly vim is grabbing mouse movements, which 
makes life

really hard.

Was there a specific revision that brought in this change, and 
can it be

removed?
This change appeared in one of the last patchset of vim 7.4 and 
was one of the

"features" of the vim 8.0 release.

I do agree this is just totally painful :(

Best regards,
Bapt
One of the things that I inherited with the Vim port was the 
DEFAULT_VIMRC option (which installs 
/usr/ports/editors/vim/files/vimrc), and I haven't touched it.


I have moused disabled in all my boxes so I have no idea about 
bad mouse behaviour in Vim. If there is a bad default that is 
causing grief, let's just fix it in that default vimrc.


I'm not really understanding what the unexpected behaviour is so 
I can't make an intelligent recommendation myself, but I'll go 
with whatever you folks suggest.


# Adam

I'm in iterm on my mac.
I ssh to a freebsd machine
I use vim on a file.
I used to be able to use the mouse on my mac to copy a few lines 
into the cut buffer.. slide-shift-click etc..
now suddenly if I try highlight some code in vim to copy it vim 
drags stuff around, scrolls up and down, deletes stuff and 
generally makes a mess.

if click, instead of starting a copy zone it grabs some of the text.

basically it makes hte mouse useless.
I can;t copy and paste from a file I'm ediitng. I end up having to 
exit vim and do it in vi.
There have been a number of recommendations in this thread for you, 
Julian, including "set mouse=a" and "set mouse=v". Test some of 
them out and let me know what works for you.


actually I never saw one about mouse=a however I did see and try 
mouse=v which didn't work for me.


I have now tried mouse=a and am happy to say that that does what I 
need.

thanks!


actually no, 'set mouse='
seems to be what I want.. not sure why I thought =a worked:




# Adam




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: recent change to vim defaults?

2017-01-18 Thread Julian Elischer

On 19/01/2017 1:37 AM, Adam Weinberger wrote:

On 18 Jan, 2017, at 10:35, Julian Elischer <jul...@freebsd.org> wrote:

On 17/01/2017 1:23 AM, Adam Weinberger wrote:

On 16 Jan, 2017, at 9:25, Baptiste Daroussin <b...@freebsd.org> wrote:

On Mon, Jan 16, 2017 at 12:03:08AM +0800, Julian Elischer wrote:

I noticed that suddenly vim is grabbing mouse movements, which makes life
really hard.

Was there a specific revision that brought in this change, and can it be
removed?

This change appeared in one of the last patchset of vim 7.4 and was one of the
"features" of the vim 8.0 release.

I do agree this is just totally painful :(

Best regards,
Bapt

One of the things that I inherited with the Vim port was the DEFAULT_VIMRC 
option (which installs /usr/ports/editors/vim/files/vimrc), and I haven't 
touched it.

I have moused disabled in all my boxes so I have no idea about bad mouse 
behaviour in Vim. If there is a bad default that is causing grief, let's just 
fix it in that default vimrc.

I'm not really understanding what the unexpected behaviour is so I can't make 
an intelligent recommendation myself, but I'll go with whatever you folks 
suggest.

# Adam

I'm in iterm on my mac.
I ssh to a freebsd machine
I use vim on a file.
I used to be able to use the mouse on my mac to copy a few lines into the cut 
buffer.. slide-shift-click etc..
now suddenly if I try highlight some code in vim to copy it vim drags stuff 
around, scrolls up and down, deletes stuff and generally makes a mess.
if click, instead of starting a copy zone it grabs some of the text.

basically it makes hte mouse useless.
I can;t copy and paste from a file I'm ediitng. I end up having to exit vim and 
do it in vi.

There have been a number of recommendations in this thread for you, Julian, including "set 
mouse=a" and "set mouse=v". Test some of them out and let me know what works for you.


actually I never saw one about mouse=a however I did see and try 
mouse=v which didn't work for me.


I have now tried mouse=a and am happy to say that that does what I need.
thanks!



# Adam




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: recent change to vim defaults?

2017-01-18 Thread Julian Elischer

On 18/01/2017 5:03 PM, Raimund Sacherer wrote:

I have to put mouse=v to get the behavior I am used to.

Best


doesn't really work for me.

vim is still taking mouse events


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: recent change to vim defaults?

2017-01-18 Thread Julian Elischer

On 17/01/2017 1:23 AM, Adam Weinberger wrote:

On 16 Jan, 2017, at 9:25, Baptiste Daroussin <b...@freebsd.org> wrote:

On Mon, Jan 16, 2017 at 12:03:08AM +0800, Julian Elischer wrote:

I noticed that suddenly vim is grabbing mouse movements, which makes life
really hard.

Was there a specific revision that brought in this change, and can it be
removed?

This change appeared in one of the last patchset of vim 7.4 and was one of the
"features" of the vim 8.0 release.

I do agree this is just totally painful :(

Best regards,
Bapt

One of the things that I inherited with the Vim port was the DEFAULT_VIMRC 
option (which installs /usr/ports/editors/vim/files/vimrc), and I haven't 
touched it.

I have moused disabled in all my boxes so I have no idea about bad mouse 
behaviour in Vim. If there is a bad default that is causing grief, let's just 
fix it in that default vimrc.

I'm not really understanding what the unexpected behaviour is so I can't make 
an intelligent recommendation myself, but I'll go with whatever you folks 
suggest.

# Adam

I'm in iterm on my mac.
I ssh to a freebsd machine
I use vim on a file.
I used to be able to use the mouse on my mac to copy a few lines into 
the cut buffer..  slide-shift-click etc..
now suddenly if I try highlight some code in vim to copy it vim drags 
stuff around, scrolls up and down, deletes stuff and generally makes a 
mess.

if click, instead of starting a copy zone it grabs some of the text.

basically it makes hte mouse useless.
I can;t copy and paste from a file I'm ediitng. I end up having to 
exit vim and do it in vi.







___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: recent change to vim defaults?

2017-01-17 Thread Julian Elischer

On 17/01/2017 12:07 AM, ohauer wrote:

I suspect you mean the /usr/local/etc/vim/vimrc and gvimrc files.
That was the first place I've tried to overwrite it, but without 
luck (even with set mouse=) but it works in ~/.vimrc


what to put IN the file?


--
olli
--
send with broken GMX mailer client, sorry for tofu and html scrap
On 15/01/2017, 22:48 Benjamin Kaduk <ka...@mit.edu> wrote:

On Mon, Jan 16, 2017 at 12:03:08AM +0800, Julian Elischer wrote:
> I noticed that suddenly vim is grabbing mouse movements, which
makes
> life really hard.
>
> Was there a specific revision that brought in this change, and
can it
> be removed?

I remember seeing something go by during an upgrade somewhat
recently
about there now being a defaults file that gets used when a user
does
not specify a .vimrc. Unfortunately, I don't remember whether I saw
that notice on a FreeBSD machine or a Debian one, and haven't
been able
to find the notice I remember through searching some likely places.

Just to check: do you have a .vimrc file in place already?


not yet.
when I work out what to put into it I will make it.



-Ben
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to
"freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


recent change to vim defaults?

2017-01-15 Thread Julian Elischer
I noticed that suddenly vim is grabbing mouse movements, which makes 
life really hard.


Was there a specific revision that brought in this change, and can it 
be removed?


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: TSC as timecounter makes system lag [-> jhb]

2017-01-14 Thread Julian Elischer

On 15/01/2017 10:11 AM, Jia-Shiun Li wrote:

On Fri, Jan 13, 2017 at 8:26 AM, Jia-Shiun Li  wrote:


Hi all,

since 2 or 3 weeks ago, I noticed that my old Penryn-based Intel Pentium
T4200 notebook lagged a lot. System time was running a lot slower,
sometimes even looked like it freezed. Keystroke repeat rate was slow too.

Since system time is slow, I tried to change timecounter from default TSC
to HPET. And it resumed normal immediately.



Did a binary search. Turns out it was caused by r310177 "Enable
EARLY_AP_STARTUP on amd64 and i386 kernels by default." r310175 does not
have this issue. Removing this option from kernel config also solves it.


making sure jhb notices this.

-Jia-Shiun.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


FreeBSD system profiling and tuning for 10, 11 and 12

2016-12-17 Thread Julian Elischer

Hi

I'm looking for recent information regarding profiling and tuning in 
FreeBSD.


Google has turned up some links but I think that the best leads are 
still hiding..
 for example I only found 
https://wiki.freebsd.org/NetworkPerformanceTuning recently.


(BTW Anyone who has a moment is encouraged to check if they have 
anything to add to it.)


I am sure there is better information around for profiling the kernel 
and modules,
but I am not seeing "the definitive profiler's guide" out there. I do 
know several people

have blogs that cover this sort of thing. If you have one or know of good
profiling resources we should gather them.. send them to me and I'll 
try make

sure everything is up to date, and put them together in a wiki page.

there is already some stuff there,
(see 
https://wiki.freebsd.org/NetworkPerformanceTuning?action=fullsearch=180=profiling=Text 
)

but it could do with gathering together.


Julian



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: tput failing on 10 and 12 systems?

2016-11-21 Thread Julian Elischer

On 22/11/2016 12:49 AM, Adrian Chadd wrote:

Hiya,

Just to follow through - part of tput is to decode command args, but
it then just requests it from the ncurses tputs() call.

So it could be tput, but it could also be the BSD ncurses work.


https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=214709



-adrian


On 21 November 2016 at 07:54, Julian Elischer <jul...@freebsd.org> wrote:

example on freefall (FreeBSD 12.0-CURRENT #0 r306376)

julian@freefall:tput setaf 4|od -c
000  033   [   m
003

so nothing happens, and tput returns 1.

but on a FreeBSD 8 system:

[jelischer@alpha ~]$ tput setaf 4|od -c
000  033   [   3   4   m
005

which make the text change color.


similarly all the related tput color commands I can think of fail on
10,11,12

but programs such as vim can use color just fine. So I suspect tput itself.


anyone have experience with this? seems easy to reproduce.







___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: tput failing on 10 and 12 systems?

2016-11-21 Thread Julian Elischer

On 22/11/2016 12:49 AM, Adrian Chadd wrote:

Hiya,

Just to follow through - part of tput is to decode command args, but
it then just requests it from the ncurses tputs() call.

So it could be tput, but it could also be the BSD ncurses work.

but the ncurses stuff works for things like vim



-adrian


On 21 November 2016 at 07:54, Julian Elischer <jul...@freebsd.org> wrote:

example on freefall (FreeBSD 12.0-CURRENT #0 r306376)

julian@freefall:tput setaf 4|od -c
000  033   [   m
003

so nothing happens, and tput returns 1.

but on a FreeBSD 8 system:

[jelischer@alpha ~]$ tput setaf 4|od -c
000  033   [   3   4   m
005

which make the text change color.


similarly all the related tput color commands I can think of fail on
10,11,12

but programs such as vim can use color just fine. So I suspect tput itself.


anyone have experience with this? seems easy to reproduce.







___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


tput failing on 10 and 12 systems?

2016-11-21 Thread Julian Elischer

example on freefall (FreeBSD 12.0-CURRENT #0 r306376)

julian@freefall:tput setaf 4|od -c
000  033   [   m
003

so nothing happens, and tput returns 1.

but on a FreeBSD 8 system:

[jelischer@alpha ~]$ tput setaf 4|od -c
000  033   [   3   4   m
005

which make the text change color.


similarly all the related tput color commands I can think of fail on 
10,11,12


but programs such as vim can use color just fine. So I suspect tput 
itself.



anyone have experience with this? seems easy to reproduce.







___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


problem with mpt driver. anyone seen this or similar? (10.3)

2016-11-08 Thread Julian Elischer

Does this ring any bells?
even a theory would be a big improvement.

memcpy+0xc
mpt_read_cfg_page+0xcc
mpt_cation+0x148e
xpt_action_default+0x7e
cam_periph_runccb+0x7c
passdoioctl+0x719
passioctl+0x30
devfs_ioctl_f+0x7c
kern_ioctl+0x1a8
sys_ioctl+0x11f
amd64_syscall+0x3f9
xfast_syscall+0xf7

we see a memory access fault at line 1821..

1786 int
1787 mpt_read_cfg_page(struct mpt_softc *mpt, int Action, uint32_t PageAddress,
1788   CONFIG_PAGE_HEADER *hdr, size_t len, int sleep_ok,
1789   int timeout_ms)
1790 {
1791 request_t*req;
1792 cfgparms_tparams;
1793 int   error;
1794
1795 req = mpt_get_request(mpt, sleep_ok);
1796 if (req == NULL) {
1797 mpt_prt(mpt, "mpt_read_cfg_page: Get request failed!\n");
1798 return (-1);
1799 }
1800
1801 params.Action = Action;
1802 params.PageVersion = hdr->PageVersion;
1803 params.PageLength = hdr->PageLength;
1804 params.PageNumber = hdr->PageNumber;
1805 params.PageType = hdr->PageType & MPI_CONFIG_PAGETYPE_MASK;
1806 params.PageAddress = PageAddress;
1807 error = mpt_issue_cfg_req(mpt, req, ,
1808   req->req_pbuf + MPT_RQSL(mpt),
1809   len, sleep_ok, timeout_ms);
1810 if (error != 0) {
1811 mpt_prt(mpt, "read_cfg_page(%d) timed out\n", Action);
1812 return (-1);
1813 }
1814
1815 if ((req->IOCStatus & MPI_IOCSTATUS_MASK) != 
MPI_IOCSTATUS_SUCCESS) {
1816 mpt_prt(mpt, "mpt_read_cfg_page: Config Info Status %x\n",
1817 req->IOCStatus);
1818 mpt_free_request(mpt, req);
1819 return (-1);
1820 }
1821 memcpy(hdr, ((uint8_t *)req->req_vbuf)+MPT_RQSL(mpt), len);   
<--
1822 mpt_free_request(mpt, req);
1823 return (0);
1824 }
1825
1826 int
1827 mpt_write_cfg_page(struct mpt_softc *mpt, int Action, uint32_t PageAddress,
"mpt/mpt.c" [readonly] 3146 lines --58%--

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Fwd: Re: how to reduce the size of /usr/share/i18n data?

2016-11-05 Thread Julian Elischer

On 3/11/2016 7:46 PM, Tomoaki AOKI wrote:

On Thu, 3 Nov 2016 06:46:39 -0400
Mark Heily  wrote:


On Nov 3, 2016 5:30 AM, "Kurt Jaeger"  wrote:

Hi!


So I am to take it that no-one has any idea how this stuff works and
how to stub it out?

Or had time to write about it.

--
p...@opsec.eu+49 171 3101372 4 years to

go !
Maybe you could use 'svn blame' to research who has commited those files,
and contact them directly?

Not shure but maybe WITHOUT_ICONV knob would eliminate them from build.
Beware! It should completely eliminate iconv features.

See commit message for Revision 219019 below.

   https://svnweb.freebsd.org/base/head/share/i18n/Makefile?view=log

Sorry, I don't know how to safely delete some (not all) part of
conversions to shrink the contents there, keeping limited iconv
features to work.

there are some 3rd party apps that want them so I'd rather just know 
how to make a really small subset.


there is a database that is describes the data there but I have no 
clue how to generate a new db.





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [RFC] remove GNU rcs from FreeBSD 12

2016-11-03 Thread Julian Elischer

On 3/11/2016 10:45 AM, Pete Wright wrote:



On 11/02/2016 19:17, Julian Elischer wrote:

On 26/10/2016 2:27 AM, Eivind Nicolay Evensen wrote:

On Sun, Sep 11, 2016 at 03:38:04PM +0200, Baptiste Daroussin wrote:

hi,

For long we are planning to remove GNU rcs from base, after a 
failed attempt
before FreeBSD 10.0. Let see where we are to be able to remove it 
from FreeBSD

12.


why should we remove it?
What will replace it?  it's an integral part of many people's systems.

Is there a non gnu RCS with the same features?



surprised to hear so many people are dependent upon having rcs in 
their base system.  there are options though - for example OpenBSD 
uses OpenRCS in their base:

I don't see what is surprising about it.
It's been common "best practice" for decades to keep all your files in 
/etc under source control. RCS fits he bill well and many people have 
it in their muscle memory.




http://man.openbsd.org/rcs.1

its not strictly GNU compliant as I believe it adheres to the 
original implementation (which frankly is probably a good thing 
imho).  GNU RCS is also available via ports/pkgs as well.


If people are adamant about preserving a rcs binary in base though 
this seems like a great opportunity to step up and help 
import/support OpenRCS.


that's ok by me as long as:
1/ it can continue to read all the old data
2/ it works the same (for scripts etc)



just my two bits - hope it helps.

-pete


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Fwd: Re: how to reduce the size of /usr/share/i18n data?

2016-11-03 Thread Julian Elischer


So I am to take it that no-one has any idea how this stuff works and 
how to stub it out?


On 1/11/2016 7:11 PM, Julian Elischer wrote:


01.11.2016 17:53, Julian Elischer пишет:
there are a number of packages that want to link with or use that 
data, and you can't always disable it, but it's very very big 
(38MB?), especially in the context of an appliance that doesn't 
really need it at all.



If anyone has a procedure to follow to put that onto a diet, maybe 
just as a stub then I'm all ears.


+1

Introduction of such large part of base system is kind of 
catastrophe for embedded systems
that need only ASCII and may be additionally one of "good old" 8-bit 
locales.


FreeBSD 11 got pretty large and embedded-unfriendly without clear 
way to exclude such unneeded parts.



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [RFC] remove GNU rcs from FreeBSD 12

2016-11-02 Thread Julian Elischer

On 26/10/2016 2:27 AM, Eivind Nicolay Evensen wrote:

On Sun, Sep 11, 2016 at 03:38:04PM +0200, Baptiste Daroussin wrote:

hi,

For long we are planning to remove GNU rcs from base, after a failed attempt
before FreeBSD 10.0. Let see where we are to be able to remove it from FreeBSD
12.


why should we remove it?
What will replace it?  it's an integral part of many people's systems.

Is there a non gnu RCS with the same features?


Whatever the outcome may be and for whatever my opinion is worth, I hope
rcs will stay in base. I don't care about the licensing. I don't
care if a switch to openrcs happens either, as long as it works.

For years, one has been able to rely upon this operating system having
certain pieces of software available. Losing that makes it a worse choice
than before.

I've already had to readd cvs to my freebsd tree since that was removed,
but if it keeps getting worse and worse and there's soon a
freebsd kernel and some random bits of freebsd userland available
through ports, there's not much reason to keep using it as an operating
system, because then it is not an operating system anymore, rather an
emulation of another "system" built around that concept.



Eivind N. E.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Fwd: Re: how to reduce the size of /usr/share/i18n data?

2016-11-01 Thread Julian Elischer




 Forwarded Message 
From:   25 2016 <>
X-Mozilla-Status:   0013
X-Mozilla-Status2:  
X-Mozilla-Keys: 
Return-Path:<eu...@grosbein.net>
Received: 	from sea.spamstick.net (sea.spamstick.net [204.109.57.196]) 
by vps1.elischer.org (8.15.2/8.15.2) with ESMTPS id uA1AxGgp045127 
(version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) 
for <jul...@elischer.org>; Tue, 1 Nov 2016 03:59:20 -0700 (PDT) 
(envelope-from eu...@grosbein.net)
Received: 	from mx2.freebsd.org ([8.8.178.116]) by sea.spamstick.net 
with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.86) 
(envelope-from <eu...@grosbein.net>) id 1c1Wmg-0008PF-00 for 
jul...@elischer.org; Tue, 01 Nov 2016 06:59:16 -0400
Received: 	from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using 
TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client 
certificate requested) by mx2.freebsd.org (Postfix) with ESMTPS id 
9F2AE6694B for <jul...@elischer.org>; Tue, 1 Nov 2016 10:59:09 + 
(UTC) (envelope-from eu...@grosbein.net)
Received: 	from freefall.freebsd.org (freefall.freebsd.org 
[IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with 
ESMTP id 85EE31320 for <jul...@elischer.org>; Tue, 1 Nov 2016 10:59:09 
+ (UTC) (envelope-from eu...@grosbein.net)
Received: 	by freefall.freebsd.org (Postfix) id 847141D68; Tue, 1 Nov 
2016 10:59:09 + (UTC)

Delivered-To:   jul...@localmail.freebsd.org
Received: 	from mx1.freebsd.org (mx1.freebsd.org 
[IPv6:2001:1900:2254:206a::19:1]) by freefall.freebsd.org (Postfix) 
with ESMTP id 839321D67 for <jul...@localmail.freebsd.org>; Tue, 1 Nov 
2016 10:59:09 + (UTC) (envelope-from eu...@grosbein.net)
Received: 	from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) 
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN 
"hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by 
mx1.freebsd.org (Postfix) with ESMTPS id 1CDD4131E; Tue, 1 Nov 2016 
10:59:08 + (UTC) (envelope-from eu...@grosbein.net)
Received: 	from eg.sd.rdtc.ru (r...@eg.sd.rdtc.ru [62.231.161.221]) by 
hz.grosbein.net (8.14.9/8.14.9) with ESMTP id uA1Ax3QX049994 
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); 
Tue, 1 Nov 2016 11:59:04 +0100 (CET) (envelope-from eu...@grosbein.net)

X-Envelope-From:eu...@grosbein.net
X-Envelope-To:  jul...@freebsd.org
Received: 	from [10.58.0.10] (dadv@dadvw [10.58.0.10]) by 
eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id uA1AwxFL006726 
(version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 
1 Nov 2016 17:59:00 +0700 (KRAT) (envelope-from eu...@grosbein.net)

Subject:    Re: how to reduce the size of /usr/share/i18n data?
To: 	Julian Elischer <jul...@freebsd.org>, freebsd 
<freebsd-hack...@freebsd.org>

References: <7b036323-aa77-6d41-36b0-439a12a36...@freebsd.org>
From:   Eugene Grosbein <eu...@grosbein.net>
Message-ID: <58187573.7020...@grosbein.net>
Date:   Tue, 1 Nov 2016 17:58:59 +0700
User-Agent: 	Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) 
Gecko/20100101 Thunderbird/38.7.2

MIME-Version:   1.0
In-Reply-To:<7b036323-aa77-6d41-36b0-439a12a36...@freebsd.org>
Content-Type:   text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding:  8bit
X-Spam-Status: 	No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM 
autolearn=no version=3.3.2
X-Spam-Report: 	* -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 
1% * [score: 0.] * 2.6 LOCAL_FROM From my domains
X-Spam-Checker-Version: 	SpamAssassin 3.3.2 (2011-06-06) on 
hz.grosbein.net
X-Filter-ID: 
s0sct1PQhAABKnZB5plbIXW6wgLqzC+TM2pKklekeEcWCsCPLMUAyrpAA5pkQvo1v0WM/M+C7g6f 
jq7KivljIgACc3eCAL7Sz/73rBvo2zdf4seDSE2pIFs49EdaF4r0GlH4z9w5rjAiHtPWW3keAShN 
xLoqnsw2BxcNT/6ha/Mu906Tg7P5f/6ZDJb4IgmNpVaLEMBpkLjCNy1XrjsuhMD0qhcz494kqrQX 
F25rLf9/U/jsH4EXGgfUT7LIXyPe+DsKzbLMLhst3n8kX6ngqWiWloQ6ztDmkvSWg7Jumw5Pg6dw 
4lDcu51VWwE6bi+ZrE6kdjyQEObSFjdQy/q2Lhnis/L5RJfxkhIgp8XOgK0jJHKF9QPFEIASHMiN 
C2cwmeLy/f6zOMlHyzALZS3+BqGp3RSwCXW+mzuiSaXGa7JQvpJ0u5JNUTFfeWDaD3S/lnEaTdft 
T8Kul6ZpNgSqNORxWsgqcF2qRmccXjejs6+dtOiBbpDQ+zv6ENK2PSDvNSjVRJ+igYTdhxxwxq0Y 
S6Qp9bdL8oYeEB2/gTYIJTIxL7hrJSk60SF3F6RYOYr2

X-Report-Abuse-To:  s...@sea.spamstick.net
X-SpamStick-Class:  ham
X-SpamStick-Evidence:   SB/spamstick_net (0.0487503235617)
X-Recommended-Action:   accept



01.11.2016 17:53, Julian Elischer пишет:

there are a number of packages that want to link with or use that data, and you 
can't always disable it, but it's very very big (38MB?), especially in the 
context of an appliance that doesn't really need it at all.


If anyone has a procedure to follow to put that onto a diet, maybe just as a 
stub then I'm all ears.


+1

Introduction of such large part of base system is kind of catastrophe for 
embedded systems
that need only ASCII and may be additionally one of "good old" 8-bit loc

Re: zfs crash on FreeBSD 10.3

2016-10-18 Thread Julian Elischer

On 14/10/2016 6:31 PM, Trond Endrestøl wrote:

On Fri, 14 Oct 2016 00:19-0700, Julian Elischer wrote:


I attempted to add a second partition to an existing FS pool in FreeBSD 10.3
and the result was a crash..

is there anyone out there with a scratch system (10.3) (or two spare drives)
who can show me this working?

Does it look familiar to anyone?

The drive 'boot0' is being used as the root drive, but we are in single user
mode, so its' read-only at this stage.

=== cut-n-paste =

[boot -s]

[...]

Trying to mount root from zfs:p8/image1 []...
Enter full pathname of shell or RETURN for /bin/sh:
PS1="# "
#
#
# ls /dev/gpt
boot0boot1
# zpool attach -f p8 gpt/boot0 gpt/boot1

Do you really need to force zpool to attach the second partition?
What happens if you omit the -f flag?

from memory, it does nothing
I can't test it now but I know the example I saw on the internet gave 
'-f , and when I tried to leave it out,
it didn't work. I vaguely remember that it complained about the 
filesystem being in use.


I will try it again some time when I have the setup running and get 
back to you.



Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address= 0x50
fault code= supervisor read data, page not present
instruction pointer= 0x20:0x81717063
stack pointer= 0x28:0xfe0169bfc640
frame pointer= 0x28:0xfe0169bfc9a0
code segment= base 0x0, limit 0xf, type 0x1b
 = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags= interrupt enabled, resume, IOPL = 0
current process= 3 (txg_thread_enter)
trap number= 12
Panic:Thought about setting the watchdog to 10 Minutes
panic: page fault
cpuid = 1

KDB: stack backtrace:
  stack1 db_trace_self_wrapper+0x2a
  kdb_backtrace+0x37 vpanic+0xf7
  panic+0x67 trap_fatal+0x264
  trap_pfault+0x1c2
  trap+0x38c
  calltrap+0x8
  dsl_scan_sync+0x2f3
  spa_sync+0x328
  txg_sync_thread+0x140
  fork_exit+0x135
  fork_trampoline+0xe

KDB: enter: panic
[ thread pid 3 tid 100328 ]
Stopped at  kdb_enter+0x50: movq$0,0x6bd5cd(%rip)
db> reboot

I will add that after this, every boot hits this problem. (same stack trace).
the box is effectively bricked
(or would be if it weren't just a bhyve image)



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


zfs crash on FreeBSD 10.3

2016-10-14 Thread Julian Elischer
I attempted to add a second partition to an existing FS pool in 
FreeBSD 10.3 and the result was a crash..


is there anyone out there with a scratch system (10.3) (or two spare 
drives) who can show me this working?


Does it look familiar to anyone?

The drive 'boot0' is being used as the root drive, but we are in 
single user mode, so its' read-only at this stage.


=== cut-n-paste =

[boot -s]

[...]

Trying to mount root from zfs:p8/image1 []...
Enter full pathname of shell or RETURN for /bin/sh:
PS1="# "
#
#
# ls /dev/gpt
boot0boot1
# zpool attach -f p8 gpt/boot0 gpt/boot1

Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address= 0x50
fault code= supervisor read data, page not present
instruction pointer= 0x20:0x81717063
stack pointer= 0x28:0xfe0169bfc640
frame pointer= 0x28:0xfe0169bfc9a0
code segment= base 0x0, limit 0xf, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags= interrupt enabled, resume, IOPL = 0
current process= 3 (txg_thread_enter)
trap number= 12
Panic:Thought about setting the watchdog to 10 Minutes
panic: page fault
cpuid = 1

KDB: stack backtrace:
 stack1 db_trace_self_wrapper+0x2a
 kdb_backtrace+0x37 vpanic+0xf7
 panic+0x67 trap_fatal+0x264
 trap_pfault+0x1c2
 trap+0x38c
 calltrap+0x8
 dsl_scan_sync+0x2f3
 spa_sync+0x328
 txg_sync_thread+0x140
 fork_exit+0x135
 fork_trampoline+0xe

KDB: enter: panic
[ thread pid 3 tid 100328 ]
Stopped at  kdb_enter+0x50: movq$0,0x6bd5cd(%rip)
db> reboot

I will add that after this, every boot hits this problem. (same stack 
trace). the box is effectively bricked

(or would be if it weren't just a bhyve image)


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Maximum memory for various freebsd releases

2016-09-19 Thread Julian Elischer
Looking through the various release notes I see that one thing thing 
that is rarely mentioned is the amount of physical memory supported, 
or any tuning that may be required to run more than some amoutn 
(should it be necessary to change some table sizes etc.).


If anyone has that information I'm looking to know if it is possible 
to run 768GM of ram on an 8.0 machine and a 10.0 machine, and if not 
in the default configuration, whether there are any changes to the 
configuration that would allow this.


Julian


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Passwordless accounts vi ports!

2016-08-11 Thread Julian Elischer

On 11/08/2016 1:16 PM, Ngie Cooper wrote:

On Aug 10, 2016, at 22:05, O. Hartmann  wrote:

I just checked the security scanning outputs of FreeBSD and found this
surprising result:

[...]
Checking for passwordless accounts:
polkitd::565:565::0:0:Polkit Daemon User:/var/empty:/usr/sbin/nologin
pulse::563:563::0:0:PulseAudio System User:/nonexistent:/usr/sbin/nologin
saned::194:194::0:0:SANE Scanner Daemon:/nonexistent:/bin/sh
clamav::106:106::0:0:Clamav Antivirus:/nonexistent:/usr/sbin/nologin
bacula::910:910::0:0:Bacula Daemon:/var/db/bacula:/usr/sbin/nologin
[...]

Obviously, some ports install accounts but do not secure them as there is an
empty password.

I consider this not a feature, but a bug.

saned is the only one that might concern me because the login shell isn't 
nologin(1).


but other tools use the password database.. e.g. ftp



Cheers,
-Ngie
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: date(1) default format changed between 10.3 and 11.0-BETA3

2016-08-06 Thread Julian Elischer

On 6/08/2016 11:09 PM, Benjamin Kaduk wrote:

On Sat, 6 Aug 2016, Baptiste Daroussin wrote:


On Sat, Aug 06, 2016 at 02:15:36PM +1000, Greg 'groggy' Lehey wrote:

On Friday,  5 August 2016 at 18:56:33 +0300, Andrey A. Chernov wrote:

On 05.08.2016 18:44, Mark Martinec wrote:

On 2016-08-05 17:23, Andrey Chernov wrote:

POSIX does say that the default format should be the same
as with "+%a %b %e %H:%M:%S %Z %Y".
It also says that %a and %b are locale's abbreviated names.

It is true for _POSIX_ locale only, as I already say. en_US.* is not
POSIX or C locale.

It still violates POLA.


I really do not think that it violates POLA fiven that the behaviour you are
expecting is still available in the default configurtion that is still POSIX.

Regardless, at a new major release is precisely when it is permissible to
break POLA.
switching from short form to long form is more than a POLA..  short 
form has a specific fixed layout

and feeding a long form string into it will break things.



Set LC_TIME to C and then you are back on your behaviour (and this is the
default when you install FreeBSD).

locales should be seen as tzdata for exemple, they are a moving target
complicated to handle for every locale we do support: 78 for 11.0-RELEASE and
193 if we do count the encoding variants. locales are updated very often (for
exemple cldr unicode make a new release of the data every 8 month or so)

As I understand it, your change will improve the maintainability of
locales in FreeBSD in the future, which justifies a POLA break at the
release boundary.

-Ben
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: date(1) default format changed between 10.3 and 11.0-BETA3

2016-08-04 Thread Julian Elischer

On 5/08/2016 5:44 AM, Mark Martinec wrote:

Should I open a bug report, or has the problem been noted?
it's not clear without reading the standard whether the bug is in the 
old or new version.

have you tried other systems? In particular I'd check OSX

sh-3.2$ export LC_CTYPE="en_US.UTF-8"
sh-3.2$ export LC_TIME="en_US.UTF-8"
sh-3.2$ export LC_ALL="en_US.UTF-8"
sh-3.2$ export LC_NUMERIC="en_US.UTF-8"
sh-3.2$ date
Fri Aug  5 12:57:47 AWST 2016

if it IS a bug then yes, file a report with full reproduction steps.



  Mark


On 2016-08-04 04:32, Julian Elischer wrote:

On 4/08/2016 7:24 AM, Mark Martinec wrote:

Is it normal/expected/documented that the date(1) command in 11.0
now produces a timestamp in substantially different format
in an "en_US.UTF-8" locale (long names, commas, 12 vs. 24h hour 
time):


  Thursday, August  4, 2016 at 12:50:43 AM CEST
vs:
  Thu Aug  4 00:52:29 CEST 2016


one of those is a bug.  the formats are defined in posix I believe.



Setting LC_TIME does not help:

  $LC_TIME="C" date
  Thursday, August  4, 2016 at 01:13:37 AM CEST

although LC_ALL="C" _does_ help.


This is funny too, especially regarding commas:
  $ LC_ALL="en_GB.UTF-8" date
  Thursday  4 August 2016 at 01:16:45 CEST
  $ LC_ALL="en_US.UTF-8" date
  Thursday, August  4, 2016 at 01:16:54 AM CEST


The date(1) man page states:
  The date utility is expected to be compatible with IEEE Std 1003.2
  (“POSIX.2”).
What does POSIX.2 say about date(1) following a locale?



==
11.0-BETA3:

$ date
Thursday, August  4, 2016 at 12:50:43 AM CEST

$ uname -a
FreeBSD xxx.ijs.si 11.0-BETA3 FreeBSD 11.0-BETA3 #0 r303469: Fri 
Jul 29 02:27:28 UTC 2016 
r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64


$ locale
LANG=
LC_CTYPE="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=en_US.UTF-8

==
10.3-RELEASE-p6 :

$ date
Thu Aug  4 00:52:29 CEST 2016

$ freebsd-version
10.3-RELEASE-p6

$ uname -a
FreeBSD yyy.ijs.si 10.3-RELEASE-p4 FreeBSD 10.3-RELEASE-p4 #0: Sat 
May 28 12:23:44 UTC 2016 
r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64


$ locale
LANG=
LC_CTYPE="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=en_US.UTF-8



  Mark

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: Socket sendmsg() porting question

2016-08-04 Thread Julian Elischer

On 4/08/2016 1:18 AM, Lundberg, Johannes wrote:

​Hi Alan

Thanks for the reply.

Can I still use the same receiving function for sendmsg/send and tell what
kind of message is coming?
How would I tell if there is an fd attached or not?

Even if I set cmsg_level and cmsg_type it won't let me send it. The problem
is having a zero length attachment on freebsd
I can't send -1 as fd because that errors to invalid file descriptor.


On Wed, Aug 3, 2016 at 10:12 AM, Alan Somers  wrote:


On Wed, Aug 3, 2016 at 10:54 AM, Lundberg, Johannes
 wrote:

Hi

I'm porting a project to fbsd and I have problem with this part that

works

in linux but not fbsd when fd = -1.

https://github.com/Cloudef/wlc/blob/master/src/session/fd.c#L80-L108

I get "invalid argument" from sendmsg() when setting CMSG_LEN(0).

Anyone have a clue how to correctly do this on fbsd?

Thanks!

Johannes


It sounds like you're trying to send an empty cmsg.  The error may
happen because your msg_controllen field is inconsistent with your
cmsg_len field.  You're setting msg_controllen as if there were a full
cmsg, but  then cmsg_len says that there is no cmsg.  Or maybe the
error is because (just guessing) FreeBSD doesn't allow sending empty
or undefined cmsgs.  Notice that cmsg_level and cmsg_type are
undefined in the case where fd == -1.  POSIX doesn't say whether
sendmsg supports empty cmsgs, but why bother?  You could just use send
instead of sendmsg if you're not sending a file descriptor.

-Alan


I think it's a standards interpretation thing.

what data do you send WITH the message? I assume you have some in-band 
data as well.


if you have no FD, just use "sendto()."

the other end will still be able to do a recvmesg() but will discover 
no added info.



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: date(1) default format changed between 10.3 and 11.0-BETA3

2016-08-03 Thread Julian Elischer

On 4/08/2016 7:24 AM, Mark Martinec wrote:

Is it normal/expected/documented that the date(1) command in 11.0
now produces a timestamp in substantially different format
in an "en_US.UTF-8" locale (long names, commas, 12 vs. 24h hour time):

  Thursday, August  4, 2016 at 12:50:43 AM CEST
vs:
  Thu Aug  4 00:52:29 CEST 2016


one of those is a bug.  the formats are defined in posix I believe.



Setting LC_TIME does not help:

  $ LC_TIME="C" date
  Thursday, August  4, 2016 at 01:13:37 AM CEST

although LC_ALL="C" _does_ help.


This is funny too, especially regarding commas:
  $ LC_ALL="en_GB.UTF-8" date
  Thursday  4 August 2016 at 01:16:45 CEST
  $ LC_ALL="en_US.UTF-8" date
  Thursday, August  4, 2016 at 01:16:54 AM CEST


The date(1) man page states:
  The date utility is expected to be compatible with IEEE Std 1003.2
  (“POSIX.2”).
What does POSIX.2 say about date(1) following a locale?



==
11.0-BETA3:

$ date
Thursday, August  4, 2016 at 12:50:43 AM CEST

$ uname -a
FreeBSD xxx.ijs.si 11.0-BETA3 FreeBSD 11.0-BETA3 #0 r303469: Fri Jul 
29 02:27:28 UTC 2016 
r...@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64


$ locale
LANG=
LC_CTYPE="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=en_US.UTF-8

==
10.3-RELEASE-p6 :

$ date
Thu Aug  4 00:52:29 CEST 2016

$ freebsd-version
10.3-RELEASE-p6

$ uname -a
FreeBSD yyy.ijs.si 10.3-RELEASE-p4 FreeBSD 10.3-RELEASE-p4 #0: Sat 
May 28 12:23:44 UTC 2016 
r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64


$ locale
LANG=
LC_CTYPE="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=en_US.UTF-8



  Mark
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Xen networking problems in -current with xn driver?

2016-08-02 Thread Julian Elischer
I upgraded my VPS machine to today's current, and on reboot I couldn't 
get into it by network.


A quick switch to the VNC console showed that it was up but that it 
couldn't get out.



The xn interfaces said they were UP but attempts to get out were met 
with "network is down".


if I did 'tcpdump -n -i xn0' (and xn1) hten all was fine again.

tcpdump saw packets, and in fact ipfw saw some packets coming in even 
before that but it was not possible to send.



Has anyone seen similar?

some relevant parts of the dmesg output.:


T(vga): text 80x25
XEN: Hypervisor version 3.4 detected.
CPU: Intel(R) Xeon(R) CPU   E5620  @ 2.40GHz (2400.05-MHz 
686-class CPU)

  Origin="GenuineIntel"  Id=0x206c2  Family=0x6  Model=0x2c Stepping=2
Features=0x1781fbff
Features2=0x80982201
  AMD Features=0x2010
  AMD Features2=0x1
Hypervisor: Origin = "XenVMMXenVMM"
real memory  = 536870912 (512 MB)
avail memory = 503783424 (480 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: 
WARNING: L1 data cache covers less APIC IDs than a core
0 < 1
WARNING: L2 data cache covers less APIC IDs than a core
0 < 1
WARNING: L3 data cache covers less APIC IDs than a core
0 < 1

ipfw2 (+ipv6) initialized, divert loadable, nat enabled, default to 
deny, logging disabled

xs_dev0:  on xenstore0
xenbusb_front0:  on xenstore0
xn0:  at device/vif/0 on xenbusb_front0
xn0: Ethernet address: 00:16:3e:01:99:54
xn1:  at device/vif/1 on xenbusb_front0
xn1: Ethernet address: 00:16:3e:01:9a:54
xenbusb_back0:  on xenstore0
xenballoon0:  on xenstore0
xctrl0:  on xenstore0
xn0: backend features: feature-sg feature-gso-tcp4
xn1: backend features: feature-sg feature-gso-tcp4
xbd0: 20480MB  at device/vbd/768 on xenbusb_front0
xbd0: attaching as ada0
xbd0: features: write_barrier
xbd0: synchronize cache commands enabled.

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: dtrace and kernel modules

2016-07-12 Thread Julian Elischer

On 13/07/2016 1:14 AM, Mark Johnston wrote:

On Thu, Jul 07, 2016 at 12:51:52PM +0800, Julian Elischer wrote:

I'm specifically interested in the case of kernel modules that
instantiate new syscalls.

How much support do we have for that?  In the one example in our
sources of a kld with a syscall (kgssapi.ko) dtrace seems to find
regular function entrypoints but not the syscall.


root@porridge:/usr/src # dtrace -n ":kgssapi::entry {}"
dtrace: description ':kgssapi::entry ' matched 138 probes
^C

root@porridge:/usr/src # dtrace -n "syscall:kgssapi::entry {}"
dtrace: invalid probe specifier syscall:kgssapi::entry {}: probe
description syscall:kgssapi::entry does not match any probes
root@porridge:/usr/src #


Do we have plans to support dynamic syscall support?

I don't know of any plans to add support. It would be fairly
straightforward to dynamically create syscall probes using a hook or
eventhandler in syscall_register(), but getting argument type info would
be more difficult. Right now, argument types are specified by code
generated by makesyscalls.sh using the types in syscalls.master. I'm not
sure how one might obtain these types for dynamically-registered
syscalls.


yes that is the tricky part for sure.

for now function calls is probably enough because every syscall ends 
up calling a function somewhere :-)




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


dtrace and kernel modules

2016-07-06 Thread Julian Elischer
I'm specifically interested in the case of kernel modules that 
instantiate new syscalls.


How much support do we have for that?  In the one example in our 
sources of a kld with a syscall (kgssapi.ko) dtrace seems to find 
regular function entrypoints but not the syscall.



root@porridge:/usr/src # dtrace -n ":kgssapi::entry {}"
dtrace: description ':kgssapi::entry ' matched 138 probes
^C

root@porridge:/usr/src # dtrace -n "syscall:kgssapi::entry {}"
dtrace: invalid probe specifier syscall:kgssapi::entry {}: probe 
description syscall:kgssapi::entry does not match any probes

root@porridge:/usr/src #


Do we have plans to support dynamic syscall support?

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: Virtualbox kernel module on 11-CURRENT

2016-06-20 Thread Julian Elischer

On 8/06/2016 5:13 AM, Kevin Oberman wrote:

On Tue, Jun 7, 2016 at 1:04 AM, Guido Falsi  wrote:


On 06/07/16 02:23, Rafael Rodrigues Nakano wrote:

Hello,

I tried installing virtualbox from packages, building it from sources,
trying the GENERIC kernel but everytime I can't start the kernel module
'vboxdrv', it says:
"KLD vboxdrv.ko: depends on kernel - not available or version mismatch.
linker_load_file: Unsupported file type"


The virtualbox module needs to be in full sync with the kernel. Most
probably the sources being used by the cluster for building packages on
head are a little different from yours, so the kernel module is not in
sync.

You will need to build the kernel module yourself to actually match your
kernel sources.

It's not really a problem or a bug, it's how it works. On head there is
no warranty about the KBI. This cannot happen on releases and stable
because the KBI is not going to change there.

--
Guido Falsi 


I don't think this is true. While shareable libraries have fixed ABIs, I
believe the KBI can change even in STABLE branches.  If a security fix
requires it, it might even change in a RELEASE. I my be wrong about this,
but I recall having to re-build the VB kmod port even withing a minor
version (i.e. STABLE).


We try hard NOT to change the KBI within a single stable branch.
we do things like add spare fields before we make a new stable branch 
to help with this.



In any case, I do strongly recommend the use of PORTS_MODULES in
/etc/src.conf to assure that the kernel modules always get re-built when
the kernel is re-built. so that the sources, the kernel, and the module are
in sync. The PORTS_MODULES are re-installed as a part of the "make
installkernel", so things are almost safe, but beware of "make
reinstallkernel" as it does not do the right thing. (See
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=201779)
--
Kevin Oberman, Part time kid herder and retired Network Engineer
E-mail: rkober...@gmail.com
PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


question on packaging base.

2016-05-31 Thread Julian Elischer
Given the work going on for incremental builds, together with the work 
in packaging the base, is it planned that we will be able to rebuild 
jut one package at a time?


For example will be be able to rebuild just he openssl-base (or 
whatever it is called) package without recompiling everything else, 
even if we have not previously recompiled everything else?





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: pkg chroot issues?

2016-05-29 Thread Julian Elischer

On 30/05/2016 12:33 AM, Tim Kientzle wrote:

On May 29, 2016, at 8:55 AM, Julian Elischer <jul...@freebsd.org> wrote:

I was thinking that in order to do this properly the chrooted child should do 
all it's fetch requests etc via the non-chrooted parent, but that would have 
probably been a bit too complicated. (including add requests)

How complex would it be to perform all of the fetch requests before chroot?
I don't know but you'd have to read the DB in the chroot before 
knowing what dependencies you need to fix.




Tim





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: pkg chroot issues?

2016-05-29 Thread Julian Elischer

On 23/05/2016 4:32 AM, b...@freebsd.org wrote:

On Sun, May 22, 2016 at 10:31:08PM +0200, b...@freebsd.org wrote:

On Sun, May 22, 2016 at 01:24:12PM -0700, Tim Kientzle wrote:

Crochet has some experimental hooks to install packages onto the system being 
built, but this seems to be hitting problems due to limitations in 'pkg -c'.  
In particular, it seems that pkg performs the chroot before it does any network 
lookups.  This is a problem if the chroot is not a complete system environment 
(which it cannot be when you're building an image for another system).

There's some further discussion on github:

   https://github.com/freebsd/crochet/issues/141

Any suggestions?


I'll reply directly to github thanks for pointing me to the ticket

Best regards,
Bapt

As people might only follow this thread and not the ticket here is what I
answered:

pkg supports an option for that which is pkg -o NAMESERVER= to avoid having to
copy resolv.conf it was broken in 1.8 but I fixed it in pkg 1.8.0 which has been
released today.

the problem with pkg -c is that it calls chroot very early. To avoid that
problem we have added pkg -r which does not perform any chroot at all therefore
having not network issue, but the ports tree are is not yet entirely aware of it
and some scripts (preinstall/postinstall) might cause some issues.
at least creating users from the script is safe in that regard. for most simple
case that should work.


As Bapt knows (I've been harassing him) pkg needs an "install all this 
over there" mode.
The answer is "-r" but it does suffer from the fact that it requires 
postinstall scripts to cooperate
by knowing to look for the appropriate environment value (I forget its 
name).


I've used three different methods to get around this..
1/ use -c and prepopulate it with a copy of /rescue, and copying all 
the pkg files I need in first into /pkg, which I delete afterwards.
I'd like clarification in the Man page about setting a good $PATH in 
the chrooted part of pkg currently I just copy /rescue into /bin but 
pkg seems to sometimes use full paths for things so that meant also 
/usr/bin and usr/sbin. (or maybe that was an install script.).
(in my application I could just link all 4 locations to be the same 
place).


2/ use   PKG_DBDIR=$(TARGET_DIR)/var/db/pkg pkg add --relocate 
$(TARGET_DIR) $(PKGFILE)

Almost the same as -r from my perspective.

3/ because I need to work as non-root, and pkg doesn't have a -u and 
-g option like chroot does, there are times when I need to do:


sudo chroot -u $ME -g $MYGROUP sh -c "INSTALL_AS_USER pkg add -f -M 
$(PKGFILE)"


instead of using pkg -c.

I worked up a patch for -u and -g but lost it.. it was only about 20 
lines including the usage() change.  I'm sure bapt could redo it much 
better.

using -u automatically enabled install_as_user.

I was thinking that in order to do this properly the chrooted child 
should do all it's fetch requests etc via the non-chrooted parent, but 
that would have probably been a bit too complicated. (including add 
requests)







Best regards,
Bapt



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


documentation looking for a home. (kernel modules and Vimage)

2016-05-13 Thread Julian Elischer
Sorry for double posting.. this email addresses two separate groups of 
people.


Some time ago I wrote the following.

http://p4web.freebsd.org/@md=d=//depot/projects/vimage/=//depot/projects/vimage/porting_to_vimage.txt=s0G@//depot/projects/vimage/porting_to_vimage.txt?ac=64=18

It really should get split into bits and put into the handbook or 
something..


It includes a section that is all about the module load inits that is 
not specific to vimage, and there is also  a good description of 
vimage in the that the handbook could use somewhere, maybe in the 
developer's hand book.



Firstly are there any doc people that can take a look at it and see if 
it would fit in somewhere?  (Do we have an editor in chief?)  and also 
maybe some kernel people who can go over the part that describes how 
modules are loaded and check that it is still up to date.. I last 
changed it in 2010 so it could be in need of updating.


If you know about kerle module loading please take a look and see if 
anything stands out to you.. It is in the second half of the doc.


Also anyone lookign at vimage might find it worth a look.


Julian


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


RFC on a published change to FreeBSD 11 kqueue file ops.

2016-04-26 Thread Julian Elischer
the following change is sitting out at github, to add kqueue support 
for more file operations:


https://github.com/dmatveev/libinotify-kqueue/blob/master/patches

does anyone have reasons why we shouldn't import this change.

libinotify is now a port and could use these.


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [CFT] packaging the base system with pkg(8)

2016-04-20 Thread Julian Elischer

On 20/04/2016 2:25 PM, Daniel Eischen wrote:

On Wed, 20 Apr 2016, Allan Jude wrote:


On 2016-04-20 01:12, Daniel Eischen wrote:


For one of our Solaris 11 boxes, which also serves as a VNC
thin client server and NFS server, we have:

  [sol11] $ pkg list | wc -l
 968

That server includes the gnome desktop, firefox, thunderbird,
perl, python, wireshark, CDR tools, etc.  So arguably, it is
comparable to my FreeBSD desktop at home with KDE, firefox,
thunderbird, and similar tools.  For that FreeBSD box, and
just for ports packages (since I don't have base pkg'd):

  [freebsd11] $ pkg info | wc -l
 865

[And it really bothers me that FreeBSD 'pkg list' behaves
 like 'pkg files' or similar should.  It seems intuitive
 that 'pkg list' should list the packages, not all the files
 in all the packages.]

If you add in 750+ FreeBSD base packages (1600+), that seems
like a very large number of packages.  And upgrading ports
packages is not always painless.  For the 865 FreeBSD packages
I have installed, only 27 of them are explicit - the rest are
dependencies.  I do not look forward to updating my packages,
even with poudriere.  There is usually manual intervention
required.  So it is with this experience that I do sort of
cringe at having 750+ FreeBSD base packages.

I do like maintaining Solaris 11 boxes much better with their
pkg management, much better than the old patchadm.



does 'pkg prime-list' give you watch you are looking for? (pkgs you
explicitly installed)


pkg prime-list does show the explicitly installed packages,
not sure how one would know to use 'prime-list' since it
doesn't appear in any of the man pages (looking at FreeBSD
10-stable right now).  And it doesn't show version information,
nor other option arguments that I can tell.  pkg help prime-list
says it's just an alias for "query -e '%a = 0' '%n'".  Seems
like "query -e '%a = 0' '%n\t%v\t%o'" is a little nicer,
though I'm not sure how to get all the columns to line up
nicely.

equally missing from help is 'pkg leaf'

These show that information needed is available.. it just needs to be 
presented right.





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [CFT] packaging the base system with pkg(8)

2016-04-19 Thread Julian Elischer

On 20/04/2016 11:41 AM, Nathan Whitehorn wrote:



On 04/19/16 20:15, Warner Losh wrote:
On Apr 19, 2016, at 4:12 PM, Matthew Grooms  
wrote:


On 4/19/2016 3:09 PM, Poul-Henning Kamp wrote:

As far as I know, nobody is taking the source code or the Makefiles
away, so if somebody doesn't like the system being distributed with
pkg, they can very well roll their own.

It's nice to see the level of enthusiasm the FreeBSD project can
muster, I just wish it wasn't always enthusiasm for stopping 
progress.


Maybe I missed an email in this thread, but I don't recall anyone 
completely rejecting the idea of packaging the base system. What I 
see is a discussion related to doing it in the best way possible.
Sadly the tenor and tone of the discussion isn’t one where progress 
is made. The tone has been a bit toxic and demanding, which grinds 
people into dust, rather than motivating them to fix things. You 
might call it a discussion, but it reads to me more as a bunch of 
angry villagers storming the castle. No good can come from that. 
Tone down the outrage by a factor of 100 and try to have the 
conversation again.


Warner


Yes, and that tone raises everyone's defensive hackles, which is 
never good. I'm going to make a patch for a reduced package count 
and hopefully we can restart this discussion then.


It would also be nice to get a statement of what the intended scope 
of these patches is from some of the people involved in the project. 
It's a major change to the system and it would be nice to have some 
kind of architectural document about what is happening. I'm not 
sure, for instance, what the release for 11 looks like with these 
changes, what changes need to be made to the installer (something of 
particular interest to me), how we build install media now that base 
is no longer self-contained (due to lack of pkg), what specific 
problems were intended to be solved, how package dependencies work, 
etc. Something like a few-page white paper would be *really* helpful 
for those of us who weren't at the BSDcan where this was discussed. 
Even a wiki page would help a lot.

-Nathan



my problem with 400 packages is that is is hard to decide what you are 
actually running.. or is it FreeBSD 11? is it FreeBSD 10.95342453?
you have no way to tell exactly what you have without comparing all 
the packages to a known list.
uname doesn't mean much, nor does "__FreeBSD_version" if everything 
comes with its own versions.


the 'leaf' concept in pkg helps with this a bit, but we've always 
considered FreeBSD bas as a sort of monalithic entity that moves 
forward together.


you are running 10.1p8 pr 10.2p1  that tells you all you need to know.
If you now need to take into account 400 different dimensions you have 
a much harder way to describe what you have..


I mentioned this before  but I think hte answer is to make a change on 
the way that "meta packages" are displayed by default in pkg.



If I install the meta package, I really don't want to see all the sub 
packages tat are unchanged unless I add '-v'.  On the other hand if I 
upgrade a sub package I want to see that in the context of the 
metapackage. Similarly if I uninstall of the subpackages.


so something like this would remove most of my objections:

# pkg info
= system packages
FreeBSD-networking-11.0.2_1FreeBSD networking subsystem and 
commands
 - ipfw-11.0.2-1   ipfw tools (uninstalled)
 - fbsd-tcpdump-11.0.2-1   Built in tcpdump tools (uninstalled)
 * openssl-11.0.2-2Openssl support (upgraded CVE-123456
FreeBSD-base-base-11.0.2-1 The absolute minimum booting base 
system
[...]
 external packages ==
apache22-2.2.31Version 2.2.x of Apache web server with prefork 
MPM.
apr-1.5.2.1.5.4Apache Portability Library
autoconf-2.69  Automatically configure source code on many Un*x 
platforms
autoconf-wrapper-20131203  Wrapper script for GNU autoconf
[...]


Maybe I uninstalled ipfw because I use pf and I install the ports 
tcpdump so I can remove the built in one.

I have installed a new openssl due to a bugfix..

This gives me a real instant feel for what I'm running..
if I add -v  then I see all 400 packages, but I really don't want to 
see them 99.99% of the time


I believe the "leaf" method gives close to this but if we could get 
the above I'd have absolutely no objections.




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"





___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: [CFT] packaging the base system with pkg(8)

2016-04-19 Thread Julian Elischer

On 19/04/2016 5:29 AM, Alfred Perlstein wrote:
Guys please stop arguing about the number of packages.  The high 
granularity is VERY useful!



it's going to make us a laughing stock
"look FreeBSD just split into 1.43 million packages" (effectively the 
same number.. it's bigger than 10)



Managing large groups of small packages is much easier than just 
having large packages.
err, Alfred, what planet do you live on? when they get out of sync it 
becomes a nightmare.
If you also had a packaging system that was smart enough to manage a 
hierarchy of packages then "maybe"..




All this can be done by meta-packages which depend on larger package 
groups.
Currently Metapackage is a way to make 10 packages look like 11 
packages.  The framework needs to understand to hide the 10 internal 
packages if they are part of a metapackage.


Later pkg can be augmented to "remove packages not explicitly 
installed" which would remove leaf packages.


Example: you installed "base-debug" which pulls in let's say 50 
small packages, later you want all of those removed, you can do 
something like:  "pkg delete --leafs base-debug" which should delete 
"base-debug" and any dangling packages it pulled in not required by 
other pkgs.


Huge thanks to the team that implemented this!


I'm sure the work was large and will be useful in the future but we 
are not ready to have the system installed like this.
no-one wants to see 750 packages show up when you do an enquiry on a 
newly installed system.

I could live with:

base-utils11.1
 - ktrace  uninstalled
 - tcpdump uninstalled
 + dd  11.1.1   (CVE-123412 fix)



but not
{700 packages )
dd  11.1.1 dd with CVE fix
{29 more packages}
[tcpdump line is not present but you don't notice that]
{10 more packages}
[ktrace line would be here but you don't notice that either]
{15 more packages}


In other words, I have no objection to all the utilities coming in the form of 
little packages.
but I have a major objection if that fact is at all obvious to the end user,
and certainly if we need to wade through 750 packages to see what's going on.



thanks.
-Alfred

On 4/18/16 1:07 PM, Lev Serebryakov wrote:

On 18.04.2016 22:40, Glen Barber wrote:


This granularity allows easy removal of things that may not be wanted
(such as *-debug*, *-profile*, etc.) on systems with little 
storage.  On

one of my testing systems, I removed the tests packages and all debug
and profiling, and the number of base system packages is 383.
  IMHO, granularity like "all base debug", "all base profile" is 
enough
for this. Really, I hardly could imagine why I will need only 1 
debug or

profile package, say, for csh. On resource-constrained systems NanoBSD
is much better anyway (for example, my typical NanoBSD installation is
37MB base system, 12MB /boot and 10 packages), and on developer system
where you need profiled libraries it is Ok to install all of them and
don't think about 100 packages for them.

  Idea of "Roles" from old FreeBSD installers looks much better. 
Again,

here are some "contrib" software which have one-to-one replacements in
ports, like sendmail, ssh/sshd, ntpd, but split all other
FreeBSD-specific code? Yes, debug. Yes, profile. Yes, static 
libraries.

Yes, lib32 on 64 bit system.

   It seems that it is ideological ("holy war") discussion more than
technical one...



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
"freebsd-current-unsubscr...@freebsd.org"




___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: [CFT] packaging the base system with pkg(8)

2016-04-19 Thread Julian Elischer

On 19/04/2016 3:14 AM, Glen Barber wrote:

On Mon, Apr 18, 2016 at 12:01:46PM -0700, Sean Fagan wrote:

On Apr 18, 2016, at 11:52 AM, Lev Serebryakov  wrote:

I understand, that maybe it is too late, but ARE YOU KIDDING?! 755
packages?! WHY?! What are reasons and goals to split base in such
enormous number of packages?

Just a guess, having done the same thing myself:  it means that updates can be
more targeted.


This is exactly the reason, which has been answered numerous times.
But I would have thought that to the logical mind, obviously the simle 
statement "755 packages" is the wrong answer..

more than 10 should be obviously wrong.
The only POSSIBLE thing that would make this an OK thing would be to 
follow that statement with "But to the external user it's really 4 
packages unless you elect to split one of them."


It would require that all 755 do *NOT show up* in the (standard) list 
of installed packages.
Maybe they could show up in some other special list if you asked for 
fine grained information.


We've managed to keep this disease out of BSD since I started to do it 
in 1990. First we laughed/fumed at Sun's Solaris when they unbundled 
the compiler. then we fumed at xorg when hey took a useful package and 
made 190 odd packages out of it. Please don't force this on us!



Glen



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: ng_ether(4) performance implications

2016-02-23 Thread Julian Elischer

On 23/02/2016 7:09 AM, Franco Fichtner wrote:

Hi all,

I'm working on FreeBSD-based configuration code dating back more
than 5 years.  Although this code uses NETGRAPH compiled into the
kernel, it also makes use of NGM_ETHER_DETACH and a self-rolled
NGM_ETHER_ATTACH to avoid having netgraph-attached interfaces when
mpd isn't needed.

In 2016, how is the state of ng_ether(4) performance to assert
whether this approach is actually useful or not.

the performance is much as it always was..
ng_ether passes packets to the next ng module as fast as they come.
netgraph does sacrifice some speed for generality, but I think it's 
not too much.


Seeing that NGM_ETHER_ATTACH is not available and should usefulness
be implicated, would code for NGM_ETHER_ATTACH be merged into
FreeBSD?


sure.. diffs always appreciated.




Thanks,
Franco
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: IoT OS

2016-01-22 Thread Julian Elischer

On 22/01/2016 2:08 AM, Mathieu Prevot wrote:

2016-01-21 17:38 GMT+01:00 NGie Cooper :


On Jan 21, 2016, at 08:34, Jan Bramkamp  wrote:


On 21/01/16 17:19, Mathieu Prevot wrote:
Dear all,

I would like to connect several connected object (with homogeneous or
heterogenous hardare: intel edison,  samsung artik, apple AX, intel

core,

etc) so the calculation needs, the storage/memory, the connection, etc

are

decoupled; hence we can reach an ecosystem with several clouds.

How do you recommend to reach that ? from the kernel, a module, or
eventually a software ?

Your message contains neither enough information nor a precise enough

question for anyone to provide you a helpful answer.

Please describe your problem in sufficient detail and reformulate your

question. If you still think these mailing lists (current@ and hackers@)
are a good audience for your question afterward ask them again.

It depends on your workload and hardware requirements (there isn't a
simple answer to your question because you didn't describe what you needed
with concrete requirements).

I would talk to cem@. He's working on ioat(4) on head for us ($work).
Thanks,
-NGie


Say all objects are connected peer to peer with wifi, some of them are
connected to internet through gsm network or wifi to a box. These object
are moving in space, and for some reasons, connections are dynamical and
can be severely impaired or lost.

They have incoming local streams of data (eg HD videos, accelerometer, GPS,
other wifi and gsm signals, etc).

I would like to abstract the CPU layer, storage layer, and internet
connection so that in realtime results of one of my objects are saved if
this object dies, so that if one of the object giving internet access to
the group loose its connection, the redundancy allows the group of object
not to lose internet connection.

Can I consider these as different load balancing layers ? Do you recommend
to implement this at the kernel layer or at an API layer ? Can I see that
as a lightweight cluster ?

I think the API is more flexible, especially if I have an heterogeneous (by
CPU, OS) set of connected object. However, working at the kernel level
allows existing programs not to be rewritten. What are your thoughts ?

Do you recommend another list ?


This is still very hard to understand.
Are you planning to work with some API that is described in a document 
somewhere?

if so please give links.



Thanks
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: HPN and None options in OpenSSH

2016-01-22 Thread Julian Elischer

On 22/01/2016 10:31 PM, Dag-Erling Smørgrav wrote:

The HPN and None cipher patches have been removed from FreeBSD-CURRENT.
I intend to remove them from FreeBSD-STABLE this weekend.

The HPN patches were of limited usefulness and required a great deal of
effort to maintain in our tree.  The None cipher patch was less onerous,
but it was a terrible idea with a very small user base since it was a
compile-time option and off by default.

The HPN-related configuration variables have been marked deprecated,
while those related to the None cipher have been marked unsupported.
This means that the former will be accepted with a warning, whereas the
latter will result in an error.

Most users will not be affected by this change.  Those who are should
switch to the openssh-portable port, which still offers both patches,
with HPN enabled by default.

It is expected that FreeBSD 10.3 will ship with OpenSSH 7.1p2, with a
number of modifications intended to reduce the impact of upstream
changes on existing systems.

what is the internal window size in the new ssh?


DES


___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: kernel && poudriere jails

2015-12-27 Thread Julian Elischer

On 28/12/2015 1:48 AM, Matthias Apitz wrote:

Hello,

I have on a Dell M5500 my poudriere jails for amd64; the host system
is at the moment r276659 (January 2015) and the jails are:

r276659 (January 2015) + ports r392920 (July 2015)
r276659 (January 2015) + ports r403255 (December 2015)

I'm right now updating the host to r292778 (as of today) and will run
new jails:

r285885 (August 2015)   + ports r403255 (December 2015)
r292778 (December 2015) + ports xxx (January 2016)

My question is: will the two old jails based on r276659 still work on a
host r292778?


My experience is that you are more likely to have problems in your 
existing setup because you should always try keep the kernel newer 
than the newest jail.

your new stup shoud just work.

In the one time I had problems, I fixed it by putting a copy of libc 
from the base into /lib/base/libc.so and using libmap.conf to make 
sure everyting used it.  I was an exceptional case however as the jail 
had some private changes so that doesn't really count.




Thanks

matthias



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


ssh and HPN

2015-12-20 Thread Julian Elischer
So was there ever a result as to whether HPN still plays a useful role 
in the current openSSH?
I haven't seen any note in the openssh docs about adding bigger window 
support and that is a definite requirement when you are on the end of 
a 500mSec RTT like I am,
but the last time we discussed it, there was talk of some changes in 
openssh making the HPN changes redundant.
If anyone has more information about that claim, I'd like some more 
definite information.



___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


Re: My build work and goals

2015-12-10 Thread Julian Elischer

On 11/12/2015 3:56 AM, Bryan Drewery wrote:

I think it is fair to
say that most people don't understand how any of this works.

[...]

- No one really was trying to improve it head-on and focused on
FreeBSD's general audience needs.



The maintenance problems come down to expertise.  Most developers are
not experts in the build and don't have time or interest to become so.

 These three points are all pretty much the same thing.
I agree that it definitely needs to have a fairy godfather.




Some improvements I have made recently:
- WITH_FAST_DEPEND: Replacing the antiquated 'make depend'/'mkdep' with
compiler dependency flags to generate the .depend files as a side-effect
of compiling.  This saves tremendous time in buildworld and buildkernel.


Mach used to have this through their entire (bsd 4.3) tree. I forget 
how they bootstrapped it.
Gcc would generate added dep files which were included into the static 
files.

Without the dep files, every thing was assumed to be dirty.
The new dep files only kicked in on the second build.
It worked very very well.









___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"


  1   2   3   4   5   6   7   8   9   10   >