[Bug 230620] "install -d" issue

2018-08-14 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230620

--- Comment #2 from Eugene Grosbein  ---
(In reply to Ian Lepore from comment #1)

Thank you for explanation. This is not obvious from the synopsis. Perhaps, it
should be made more clear by replacing "directory ..." with "directory1 ...
directoryN" just like previous lines already have "file1 ... file N directory".

Also, one can consider this as PR feature request then: teach install(1) to
create missing directories while copying files. It could use new "-F" flag for
that to force such creation.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Jails on ZFS yielding 100% load on gstat

2018-08-14 Thread Marco Steinbach
On Sun, 12 Aug 2018 20:50:47 +0200
Marco Steinbach  wrote:

> Hi there.
> 
[...]

Thanks for all the answers and suggestions -- one of the MySQL
databases contained a defective table, repairing it brought the storage
load down to normal levels.

I've had defective tables on UFS in the past, and can't remember them
bogging down the system as they do, when running on ZFS.

I am aware of the blocksize issue, though.


Again, thanks for all the suggestions.

MfG CoCo
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


[Bug 230620] "install -d" issue

2018-08-14 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230620

Ian Lepore  changed:

   What|Removed |Added

 CC||i...@freebsd.org

--- Comment #1 from Ian Lepore  ---
Install -d ONLY creates directories.  You are not allowed to specify both
directories and filenames on the command line when using -d.  This is
documented by having a different synopsis line in the manpage for install -d.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


[Bug 230620] "install -d" issue

2018-08-14 Thread bugzilla-noreply
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230620

Bug ID: 230620
   Summary: "install -d" issue
   Product: Base System
   Version: 11.1-STABLE
  Hardware: Any
OS: Any
Status: New
  Severity: Affects Some People
  Priority: ---
 Component: bin
  Assignee: sta...@freebsd.org
  Reporter: eu...@freebsd.org
 Flags: mfc-stable11?

install(1) manual page tells that the install utility can create missing parent
directories as required if -d options is specified. This mode seems to work
half-way only, as it really creates needed directories but fails to copy
specified files there:

$ cd /tmp
$ touch file1 file2
$ install -d dir file1 file2
install: file1 exists but is not a directory
$ ls -lR dir
total 0

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Jails on ZFS yielding 100% load on gstat

2018-08-14 Thread Mike Tancsa
On 8/14/2018 5:57 AM, Marco Steinbach wrote:
> 
> All my machines have at least 32GB, I've limited the maximum ARC size
> to 4GB, though, since last time I checked ZFS took quite a large piece
> of the cake, and didn't give it back, when not limited by setting
> vfs.zfs.arc_max in loader.conf :)

BTW for recent RELENG_11 boxes this is now a run time tunable.

> 
> I've since found the culprit, I think. Please see my other post in this
> thread.

What was the issue ?

---Mike


-- 
---
Mike Tancsa, tel +1 519 651 3400 x203
Sentex Communications, m...@sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Jails on ZFS yielding 100% load on gstat

2018-08-14 Thread Marco Steinbach
On Mon, 13 Aug 2018 10:56:01 -0400
"Kevin P. Neal"  wrote:

> On Sun, Aug 12, 2018 at 08:50:47PM +0200, Marco Steinbach wrote:
> > Hi there.
> > 
> > % zpool list
> > NAMESIZE  ALLOC   FREE  EXPANDSZ   FRAGCAP  DEDUP  HEALTH
> > ALTROOT zroot  5.41T   670G  4.75T -13%12%  1.00x
> > ONLINE  -
> > 
> > % uname -a
> > FreeBSD XXX 11.1-STABLE FreeBSD 11.1-STABLE #0 r322984 [...] amd64
> > 
> > 
> > I'm running multiple jails on ZFS, using ezjail to manage them,
> > including a websever and a mailserver. The mailserver is using a
> > MySQL database, otherwise depending on dovecot and postfix. Very
> > low volume, just a few polls / logins per minute.
> > 
> > I am experiencing very high loads as per gstat:
> > 
> > dT: 1.021s  w: 1.000s
> >  L(q)  ops/sr/s   kBps   ms/rw/s   kBps   ms/w   %busy Name
> > 4181  0  00.0169873   20.1   98.2| ada0
> > 2111  0  00.01005407.3   90.6| ada1
> > 0 88  0  00.0 764581.4   43.3| ada2
> > 0  0  0  00.0  0  00.00.0|
> > ada0p1 3150  0  00.0150603   20.2   95.1|
> > ada0p2 1 31  0  00.0 20270   19.2  117.0|
> > ada0p3 0  0  0  00.0  0  00.00.0|
> > gpt/gptboot0 0  0  0  00.0  0  00.0
> > 0.0| ada1p1 1 85  0  00.0 853418.4
> > 68.9| ada1p2 1 25  0  00.0 152000.9
> > 75.0| ada1p3 0  0  0  00.0  0  00.0
> > 0.0| ada2p1 0 62  0  00.0 622511.6
> > 9.9| ada2p2 0 26  0  00.0 152080.5
> > 42.0| ada2p3 0  0  0  00.0  0  00.0
> > 0.0| gpt/gptboot1 0  0  0  00.0  0  0
> > 0.00.0| gpt/gptboot2  
> 
> Say, these don't look very evenly spread.
> 
> What's ada0p3?

I posted 'list', instead of 'zpool status' :/

It's the third drive in my raidz1.

> > These loads lead to the system suffering from very much delayed
> > responses to even the basic task of echoing characters entered on
> > the console, consequently rendering the services offered unusable
> > to the users because of the delays.  
> 
> How much memory does your machine have, and how much is being used by
> ZFS? You can get that from the default top display if you want.
> 
> You can also use 'zpool iostat ' to see how much traffic is
> going to specifically ZFS.
> 

I see a lot of imbalanced IO on my zpools, although I am using the
exact same model and firmware level drives in my raidz1 setups.

All my machines have at least 32GB, I've limited the maximum ARC size
to 4GB, though, since last time I checked ZFS took quite a large piece
of the cake, and didn't give it back, when not limited by setting
vfs.zfs.arc_max in loader.conf :)

I've since found the culprit, I think. Please see my other post in this
thread.

MfG CoCo


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Jails on ZFS yielding 100% load on gstat

2018-08-14 Thread Marco Steinbach
On Mon, 13 Aug 2018 10:12:26 -0400
Mike Tancsa  wrote:

> On 8/12/2018 2:50 PM, Marco Steinbach wrote:
> > 
> > These loads lead to the system suffering from very much delayed
> > responses to even the basic task of echoing characters entered on
> > the console, consequently rendering the services offered unusable
> > to the users because of the delays.  
> 
> 
> Do you have a LOT of files and or metadata ? Have a look at the cache
> stats to see if you are perhaps grinding away on big directory
> lookups? Install sysutils/zfs-stats and post
> zfs-stats -a
> 
> Also, does
> top -mio -I
> shed any light as to whats taking up the disk io ?
> 
>   ---Mike

I've had these kinds of sudden load spikes on my raidz1 setups in the
past, which I can't remember seeing, before all machines were migrated
from UFS to ZFS.

Stats during these spikes, that is, if the machine in question would
still respond to commands, did show 100% load on gstat, with ZFS having 
low throughput (typically less than a megabyte per second).

Top display varied wildly, up to the point where each and any process
accessing storage would put loads > 80% on IO.  Which was kind of to be
expected, since something was clogging the queue.

In this particular case, as noted in another post, a customers php
script ran into a defective MySQL table, and instead of simply
erroring out kept hammering it. Leading to what I assume to be a myriad
of small IO operations -- probably amplified by the blocksize
clash between MySQL and ZFS, which otherwise never impacted performance
in my use-cases m|


I'll gather more detailed stats the next time I see ZFS storage
bogging down.

MfG CoCo
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: FreeBSD blocks on BOCHS serial port

2018-08-14 Thread Warner Losh
On Tue, Aug 14, 2018 at 2:17 AM, Eugene Grosbein  wrote:

> 14.08.2018 9:47, Kurt Jaeger wrote:
>
> >> 14.08.2018 3:15, Alexander Lochmann wrote:
> >>
>  You should not rely on defaults and make sure you disable modem
> control/CD
>  either explicitly (using stty(1) etc.) or implicitly by switching to
> /dev/cuau0
>  instead of /dev/ttyu0. Flow control settings should match too, for
> both sides
>  of virtual port.
> >>> Thx. I cannot even run 'stty < /dev/ttyu1' to see the current settings.
> >>> It simply blocks...
> >>
> >> Use /dev/ttyu1.init to see defaults and /dev/ttyu1.lock to set/show
> >> locked defaults that cannot be changed without disabling a lock first.
> >
> > Thanks for this pointer! Is that behaviour written down/explained
> > somewhere in the man pages ?
>
> Sort of. Default serial driver for modern FreeBSD is uart(4) that documents
> these file names extremely concisely but does not explain their meaning.
> Legacy serial driver sio(4) has manual page describing them in detail.


I've copied the sio.4 text into uart.4 since it was good at explaining.

Warner
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Jails on ZFS yielding 100% load on gstat

2018-08-14 Thread Marco Steinbach
On Mon, 13 Aug 2018 08:09:00 -0600
Alan Somers  wrote:

> Jails probably aren't the source of your problem.  You need to find
> out what process or processes are responsible for all this activity.
> Since the write bandwidth is fairly low, you might have a process
> that's sync(2)ing or fsync(2)ing. too often.  "gstat -o" will show if
> that's the case.  You can also try running "top -mio" to see which
> processes are doing the most I/O.
> 
> -Alan

[...]

Thank you for hinting me to the -o flag of gstat, very useful.


MfG CoCo
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: All the memory eaten away by ZFS 'solaris' malloc - on 11.1-R amd64

2018-08-14 Thread Andriy Gapon
On 07/08/2018 15:58, Mark Martinec wrote:
> Collected, here it is:
> 
>   https://www.ijs.si/usr/mark/tmp/dtrace-cmd.out.bz2


I see one memory leak, not sure if it's the only one.
It looks like vdev_geom_read_config() leaks all parsed vdev nvlist-s but
the last.  The problems seems to come from r316760.  Before that commit
the function would return upon finding the first valid config, but now
it keeps iterating.

The memory leak should not be a problem when vdev-s are probed
sufficiently rarely, but it appears that with an unhealthy pool the
probing can happen much more frequently (e.g., every time pools are listed).

-- 
Andriy Gapon
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: FreeBSD blocks on BOCHS serial port

2018-08-14 Thread Eugene Grosbein
14.08.2018 9:47, Kurt Jaeger wrote:

>> 14.08.2018 3:15, Alexander Lochmann wrote:
>>
 You should not rely on defaults and make sure you disable modem control/CD
 either explicitly (using stty(1) etc.) or implicitly by switching to 
 /dev/cuau0
 instead of /dev/ttyu0. Flow control settings should match too, for both 
 sides
 of virtual port.
>>> Thx. I cannot even run 'stty < /dev/ttyu1' to see the current settings.
>>> It simply blocks...
>>
>> Use /dev/ttyu1.init to see defaults and /dev/ttyu1.lock to set/show
>> locked defaults that cannot be changed without disabling a lock first.
> 
> Thanks for this pointer! Is that behaviour written down/explained
> somewhere in the man pages ?

Sort of. Default serial driver for modern FreeBSD is uart(4) that documents
these file names extremely concisely but does not explain their meaning.
Legacy serial driver sio(4) has manual page describing them in detail.


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"