Re: Something in r291926 (and earlier) causes reboots during periodic daily (14 jails on system)

[Alexander Leidinger]

On Wed, 09 Dec 2015 09:38:07 +0100
Miroslav Lachman <000.f...@quip.cz> wrote:

> Alexander Leidinger wrote on 12/09/2015 09:00:

> > Does this ring a bell for someone or any ideas before I try to hunt
> > this down?
> 
> The same problem was reported yesterday on Stable
> 
> Periodic jobs triggering panics in 10.1 and 10.2
> https://lists.freebsd.org/pipermail/freebsd-stable/2015-December/083807.html
> 
> Also ZFS with jails.

The above mail has a backtrace involving ZFS.

I'm running the periodic scripts serially now, 8 out of 14 already
finished without issues. Smells like a concurrency issue. I would
assume it's something introduced between 5 and 1 month ago and MFCed
back to 10.

Does this ring a bell for someone on fs@?

Bye,
Alexander.

-- 
http://www.Leidinger.net alexan...@leidinger.net: PGP 0xC773696B3BAC17DC
http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0xC773696B3BAC17DC
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Periodic jobs triggering panics in 10.1 and 10.2

[Jan Bramkamp]

On 09/12/15 01:04, Michael B. Eichorn wrote:

On Tue, 2015-12-08 at 16:31 -0600, Dustin Wenz wrote:

I suspect this is a zfs bug that is triggered by the access patterns
in the periodic scripts. There is significant load on the system when
the scheduled processes start, because all jails execute the same
scripts at the same time.

I've been able to alleviate this problem by disabling the security
scans within the jails, but leave it enabled on the root host.


To avoid the problem of jails all starting things at the same time, use
the cron(8) flags -j and -J to set a 'jitter' which will cause cron to
sleep for a random period of specified duration (60 sec max). Cron
flags can be set using the rc.conf variable 'cron_flags'.


While jitter would reduce the resource contention a thundering herd of 
cronjobs shouldn't cause the kernel to divide by zero. Spreading the 
load by introducing jitter to cronjobs might hide the problem, but it 
still needs further analysis.


@Dustin Wenz: Can you reproduce the problem and file a PR to track this?
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Periodic jobs triggering panics in 10.1 and 10.2

[Jan Bramkamp]

On 09/12/15 13:45, Michelle Sullivan wrote:

Michael B. Eichorn wrote:

On Tue, 2015-12-08 at 16:31 -0600, Dustin Wenz wrote:


I suspect this is a zfs bug that is triggered by the access patterns
in the periodic scripts. There is significant load on the system when
the scheduled processes start, because all jails execute the same
scripts at the same time.

I've been able to alleviate this problem by disabling the security
scans within the jails, but leave it enabled on the root host.



To avoid the problem of jails all starting things at the same time, use
the cron(8) flags -j and -J to set a 'jitter' which will cause cron to
sleep for a random period of specified duration (60 sec max). Cron
flags can be set using the rc.conf variable 'cron_flags'.
___



No that will just hide it (if successful at all) and it won't work in
all cases.

... i386 is even worse for similar (not the same) instability triggered
by the same scripts ... because zfs should not be used with the stock
i386 kernel (which means if you're using it the whole patching process
with freebsd-update won't work or will 'undo' your kernel config.)


Do you have a good idea how to prevent users from shooting themselves in 
the foot by running ZFS on 32 Bit kernels?



Personally I think zfs should be optional only for 'advanced' users and
come with a whole host of warnings about what it is not suitable for
however, it seems to be treated as a magic bullet for data corruption
issues yet all I have seen is an ever growing list of where it causes
problems.. when did UFS become an unreliable FS that is susceptible to
chronic data corruption?


As storage capacity grew a lot faster than reliability.

UFS is a good file system for its time, but it trusts hardware 
absolutely. Modern hardware doesn't deserve this level of trust. ZFS 
detects and recovers without dataloss from most errors caused by the 
limited hardware reliability.


ZFS isn't just a tool to deal with hardware limitations it's also a 
convenience I no longer want to give up. Snapshots and replication 
streams simplify backups and a background scrub once a week (or month) 
sure beats waiting for fsck.

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Periodic jobs triggering panics in 10.1 and 10.2

[Michelle Sullivan]

Michael B. Eichorn wrote:
> On Tue, 2015-12-08 at 16:31 -0600, Dustin Wenz wrote:
>   
>> I suspect this is a zfs bug that is triggered by the access patterns
>> in the periodic scripts. There is significant load on the system when
>> the scheduled processes start, because all jails execute the same
>> scripts at the same time.
>>
>> I've been able to alleviate this problem by disabling the security
>> scans within the jails, but leave it enabled on the root host.
>> 
>
> To avoid the problem of jails all starting things at the same time, use
> the cron(8) flags -j and -J to set a 'jitter' which will cause cron to
> sleep for a random period of specified duration (60 sec max). Cron
> flags can be set using the rc.conf variable 'cron_flags'.
> ___
>   

No that will just hide it (if successful at all) and it won't work in
all cases.

... i386 is even worse for similar (not the same) instability triggered
by the same scripts ... because zfs should not be used with the stock
i386 kernel (which means if you're using it the whole patching process
with freebsd-update won't work or will 'undo' your kernel config.)

Personally I think zfs should be optional only for 'advanced' users and
come with a whole host of warnings about what it is not suitable for
however, it seems to be treated as a magic bullet for data corruption
issues yet all I have seen is an ever growing list of where it causes
problems.. when did UFS become an unreliable FS that is susceptible to
chronic data corruption?

-- 
Michelle Sullivan
http://www.mhix.org/

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Periodic jobs triggering panics in 10.1 and 10.2

[Michael B. Eichorn]

On Wed, 2015-12-09 at 11:24 +0100, Jan Bramkamp wrote:
> 
> On 09/12/15 01:04, Michael B. Eichorn wrote:
> > On Tue, 2015-12-08 at 16:31 -0600, Dustin Wenz wrote:
> > > I suspect this is a zfs bug that is triggered by the access
> > > patterns
> > > in the periodic scripts. There is significant load on the system
> > > when
> > > the scheduled processes start, because all jails execute the same
> > > scripts at the same time.
> > > 
> > > I've been able to alleviate this problem by disabling the
> > > security
> > > scans within the jails, but leave it enabled on the root host.
> > 
> > To avoid the problem of jails all starting things at the same time,
> > use
> > the cron(8) flags -j and -J to set a 'jitter' which will cause cron
> > to
> > sleep for a random period of specified duration (60 sec max). Cron
> > flags can be set using the rc.conf variable 'cron_flags'.
> 
> While jitter would reduce the resource contention a thundering herd
> of 
> cronjobs shouldn't cause the kernel to divide by zero. Spreading the 
> load by introducing jitter to cronjobs might hide the problem, but it
> still needs further analysis.

I concur, used the word 'avoid' in the sense that this was an
improvement to his work-around (which is all that I quoted). I had no
intent to imply that it was the solution to the bug, which of course
deserves solving. Sorry for any confusion, I will try to be more clear
in the future.

smime.p7s
Description: S/MIME cryptographic signature

intel_do_flush_locked failed: Input/output error

[Joseph Olatt]

Firefox keeps crashing on my trusty Thinkpad T60 with the following
error:

  joji@peace> firefox
  GLib-GIO-Message: Using the 'memory' GSettings backend.  Your settings
  will not be saved or shared with other applications.
  ATTENTION: default value of option force_s3tc_enable overridden by
  environment.
  intel_do_flush_locked failed: Input/output error



>From searching the Internet, I couldn't find any recent FreeBSD users
complaining about similar issues. Got me wondering if I had something 
set up incorrectly at my end. Almost always, firefox crashes when the
page contains videos (Flash not installed on this laptop).

Anybody else having similar issues?

Some pertinent info:
uname -a:
  FreeBSD peace 10.2-STABLE FreeBSD 10.2-STABLE #10 r291993: Tue Dec  8
  12:56:35 CST 2015 root@peace:/usr/obj/usr/src/sys/PEACE  i386

pciconf -lv:
  joji@peace> pciconf -lv 
  hostb0@pci0:0:0:0:  class=0x06 card=0x201717aa chip=0x27a08086 
rev=0x03 hdr=0x00
  vendor = 'Intel Corporation'
  device = 'Mobile 945GM/PM/GMS, 943/940GML and 945GT Express Memory 
Controller Hub'
  class  = bridge
  subclass   = HOST-PCI
  vgapci0@pci0:0:2:0: class=0x03 card=0x201a17aa chip=0x27a28086 
rev=0x03 hdr=0x00
  vendor = 'Intel Corporation'
  device = 'Mobile 945GM/GMS, 943/940GML Express Integrated Graphics 
Controller'
  class  = display
  subclass   = VGA
  vgapci1@pci0:0:2:1: class=0x038000 card=0x201a17aa chip=0x27a68086 
rev=0x03 hdr=0x00
  vendor = 'Intel Corporation'
  device = 'Mobile 945GM/GMS/GME, 943/940GML Express Integrated 
Graphics Controller'
  class  = display


kldstat:
  Id Refs AddressSize Name
   1   21 0xc040 c35428   kernel
   21 0xc6417000 74000i915kms.ko
   31 0xc648b000 45000drm2.ko
   44 0xc5cc9000 4000 iicbus.ko
   51 0xc64f9000 3000 iic.ko
   61 0xc64fd000 4000 iicbb.ko

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Something in r291926 (and earlier) causes reboots during periodic daily (14 jails on system)

[Karl Denninger]

I doubt very much that's involved; I have multiple production systems
with dynamic_write_buffer turned on and I've yet to see a panic from
periodic activity or anything like it.

The only panic I've got open right now that's related to ZFS happens
during a send/receive backup and it's able to be reproduced only with
difficulty (it looks to be a problem in snapshot management as I've
isolated it fairly well but haven't yet found the root cause) -- and it
can be coaxed out of a bare "clean" -STABLE install with no ZFS patches
in at all.  (The codepath that results in the panic doesn't have a
logical connection to that patch, but I tested without it anyway just to
make sure.)

On 12/9/2015 15:15, Dustin Wenz wrote:
> Are you by chance using the ARC patch from this PR?
>
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594
>
> Do you have a vfs.zfs.dynamic_write_buffer tunable defined, if so, what is it 
> set to?
>
>   - .Dustin
>
>
>> On Dec 9, 2015, at 3:51 AM, Alexander Leidinger  
>> wrote:
>>
>> On Wed, 09 Dec 2015 09:38:07 +0100
>> Miroslav Lachman <000.f...@quip.cz> wrote:
>>
>>> Alexander Leidinger wrote on 12/09/2015 09:00:
 Does this ring a bell for someone or any ideas before I try to hunt
 this down?
>>> The same problem was reported yesterday on Stable
>>>
>>> Periodic jobs triggering panics in 10.1 and 10.2
>>> https://lists.freebsd.org/pipermail/freebsd-stable/2015-December/083807.html
>>>
>>> Also ZFS with jails.
>> The above mail has a backtrace involving ZFS.
>>
>> I'm running the periodic scripts serially now, 8 out of 14 already
>> finished without issues. Smells like a concurrency issue. I would
>> assume it's something introduced between 5 and 1 month ago and MFCed
>> back to 10.
>>
>> Does this ring a bell for someone on fs@?
>>
>> Bye,
>> Alexander.
>>
>> -- 
>> http://www.Leidinger.net alexan...@leidinger.net: PGP 0xC773696B3BAC17DC
>> http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0xC773696B3BAC17DC
>> ___
>> freebsd-stable@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
>> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

-- 
Karl Denninger
k...@denninger.net 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: Periodic jobs triggering panics in 10.1 and 10.2

[Michelle Sullivan]

Jan Bramkamp wrote:
> On 09/12/15 13:45, Michelle Sullivan wrote:
>>
>> No that will just hide it (if successful at all) and it won't work in
>> all cases.
>>
>> ... i386 is even worse for similar (not the same) instability triggered
>> by the same scripts ... because zfs should not be used with the stock
>> i386 kernel (which means if you're using it the whole patching process
>> with freebsd-update won't work or will 'undo' your kernel config.)
>
> Do you have a good idea how to prevent users from shooting themselves
> in the foot by running ZFS on 32 Bit kernels?

Yes default not to having zfs available on any platform and allow people
that know what they are doing to turn it on  I mean "prevent users
from shooting themselves in the foot" - how about by not having an
option to install a zfs root on the default install disks?

>
>> Personally I think zfs should be optional only for 'advanced' users and
>> come with a whole host of warnings about what it is not suitable for
>> however, it seems to be treated as a magic bullet for data corruption
>> issues yet all I have seen is an ever growing list of where it causes
>> problems.. when did UFS become an unreliable FS that is susceptible to
>> chronic data corruption?
>
> As storage capacity grew a lot faster than reliability.

Yeah, that's why we have these multi-tes-of-terrabyte laptops that must
have a zfs root install...

>
> UFS is a good file system for its time, but it trusts hardware
> absolutely. Modern hardware doesn't deserve this level of trust.

Ok at this point we have to question things...

Does your average home machine need zfs?  (because windows doesn't) ...
does your average laptop require zfs (or even benefit) ...?   In fact
when I look at it, I'm running  70+ servers and a few desktops and I'm
running 5 of them with zfs...  2 of them absolutely need it, 2 of them
are solaris (which probably doesn't count and certainly doesn't have
relevance to FreeBSD) the other is a 2005 P4 based server that is
completely unusable because zfs on i386 doesn't work with the stock
kernel  and guess what ... it has 73G 15k SCSI Server drives in it
so it probably has reliable hardware that doesn't suffer from "Modern
hardware doesn't deserve this level of trust"

> ZFS detects and recovers without dataloss from most errors caused by
> the limited hardware reliability.

Currently I've had more problems with the reliability of zfs in FreeBSD
than reliability of hardware..  I do get your point though...

>
> ZFS isn't just a tool to deal with hardware limitations it's also a
> convenience I no longer want to give up. Snapshots and replication
> streams simplify backups and a background scrub once a week (or month)
> sure beats waiting for fsck.
Now this is the one set of reasons I can really appreciate and had it
been the opening argument I'd have understood your position, but it
seems this is a side note to the above and the above is where I see it's
completely useless...  When ZFS was first developed a friend and I in
Sun had lots of fun setting up servers where we just chucked any old
drives we could lay our hands on into a pool ... this we found very cool
and this was where 'unreliable' hardware was an understatement - the
drives were pulled from machines because SMART (and other tools) were
reporting the drive(s) failing. but it was a work around for bad
sectors etc...

Seriously though the default to install with zfs and root on zfs is a
really bad idea - the people who know how not to shoot themselves in the
foot are those people that don't need a selectable option in the install
because they know how to configure it... they're the people who will
probably be in every manual and advanced option they can find anyhow (or
just using boot servers and predefined install scripts)!!

Regards,

-- 
Michelle Sullivan
http://www.mhix.org/

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Something in r291926 (and earlier) causes reboots during periodic daily (14 jails on system)

[Dustin Wenz]

Are you by chance using the ARC patch from this PR?

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594

Do you have a vfs.zfs.dynamic_write_buffer tunable defined, if so, what is it 
set to?

- .Dustin


> On Dec 9, 2015, at 3:51 AM, Alexander Leidinger  
> wrote:
> 
> On Wed, 09 Dec 2015 09:38:07 +0100
> Miroslav Lachman <000.f...@quip.cz> wrote:
> 
>> Alexander Leidinger wrote on 12/09/2015 09:00:
> 
>>> Does this ring a bell for someone or any ideas before I try to hunt
>>> this down?
>> 
>> The same problem was reported yesterday on Stable
>> 
>> Periodic jobs triggering panics in 10.1 and 10.2
>> https://lists.freebsd.org/pipermail/freebsd-stable/2015-December/083807.html
>> 
>> Also ZFS with jails.
> 
> The above mail has a backtrace involving ZFS.
> 
> I'm running the periodic scripts serially now, 8 out of 14 already
> finished without issues. Smells like a concurrency issue. I would
> assume it's something introduced between 5 and 1 month ago and MFCed
> back to 10.
> 
> Does this ring a bell for someone on fs@?
> 
> Bye,
> Alexander.
> 
> -- 
> http://www.Leidinger.net alexan...@leidinger.net: PGP 0xC773696B3BAC17DC
> http://www.FreeBSD.orgnetch...@freebsd.org  : PGP 0xC773696B3BAC17DC
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Periodic jobs triggering panics in 10.1 and 10.2

[Michael B. Eichorn]

On Wed, 2015-12-09 at 23:26 +0100, Michelle Sullivan wrote:
> Jan Bramkamp wrote:
> > On 09/12/15 13:45, Michelle Sullivan wrote:
> > > 
> > > No that will just hide it (if successful at all) and it won't
> > > work in
> > > all cases.
> > > 
> > > ... i386 is even worse for similar (not the same) instability
> > > triggered
> > > by the same scripts ... because zfs should not be used with the
> > > stock
> > > i386 kernel (which means if you're using it the whole patching
> > > process
> > > with freebsd-update won't work or will 'undo' your kernel
> > > config.)
> > 
> > Do you have a good idea how to prevent users from shooting
> > themselves
> > in the foot by running ZFS on 32 Bit kernels?
> 
> Yes default not to having zfs available on any platform and allow
> people
> that know what they are doing to turn it on  I mean "prevent
> users
> from shooting themselves in the foot" - how about by not having an
> option to install a zfs root on the default install disks?
> 
> > 
> > > Personally I think zfs should be optional only for 'advanced'
> > > users and
> > > come with a whole host of warnings about what it is not suitable
> > > for
> > > however, it seems to be treated as a magic bullet for data
> > > corruption
> > > issues yet all I have seen is an ever growing list of where it
> > > causes
> > > problems.. when did UFS become an unreliable FS that is
> > > susceptible to
> > > chronic data corruption?
> > 
> > As storage capacity grew a lot faster than reliability.
> 
> Yeah, that's why we have these multi-tes-of-terrabyte laptops that
> must
> have a zfs root install...
> 
> > 
> > UFS is a good file system for its time, but it trusts hardware
> > absolutely. Modern hardware doesn't deserve this level of trust.
> 
> Ok at this point we have to question things...
> 
> Does your average home machine need zfs?  (because windows doesn't)
> ...
> does your average laptop require zfs (or even benefit) ...?   In fact
> when I look at it, I'm running  70+ servers and a few desktops and
> I'm
> running 5 of them with zfs...  2 of them absolutely need it, 2 of
> them
> are solaris (which probably doesn't count and certainly doesn't have
> relevance to FreeBSD) the other is a 2005 P4 based server that is
> completely unusable because zfs on i386 doesn't work with the stock
> kernel  and guess what ... it has 73G 15k SCSI Server drives in
> it
> so it probably has reliable hardware that doesn't suffer from "Modern
> hardware doesn't deserve this level of trust"
> 
> > ZFS detects and recovers without dataloss from most errors caused
> > by
> > the limited hardware reliability.
> 
> Currently I've had more problems with the reliability of zfs in
> FreeBSD
> than reliability of hardware..  I do get your point though...
> 
> > 
> > ZFS isn't just a tool to deal with hardware limitations it's also a
> > convenience I no longer want to give up. Snapshots and replication
> > streams simplify backups and a background scrub once a week (or
> > month)
> > sure beats waiting for fsck.
> Now this is the one set of reasons I can really appreciate and had it
> been the opening argument I'd have understood your position, but it
> seems this is a side note to the above and the above is where I see
> it's
> completely useless...  When ZFS was first developed a friend and I in
> Sun had lots of fun setting up servers where we just chucked any old
> drives we could lay our hands on into a pool ... this we found very
> cool
> and this was where 'unreliable' hardware was an understatement - the
> drives were pulled from machines because SMART (and other tools) were
> reporting the drive(s) failing. but it was a work around for bad
> sectors etc...
> 
> Seriously though the default to install with zfs and root on zfs is a
> really bad idea - the people who know how not to shoot themselves in
> the
> foot are those people that don't need a selectable option in the
> install
> because they know how to configure it... they're the people who will
> probably be in every manual and advanced option they can find anyhow
> (or
> just using boot servers and predefined install scripts)!!
> 
> Regards,
> 

I sorry, but I really don't get your point, PCBSD has shown a great
reason why zfs on root and on laptops/desktops is a good idea... boot
environments. They have pretty much figured out how to use snapshots to
go from A-B ping-pong installations to A-B-C-D-E installations. I
am even aware of people using it to run Release and Current on the same
machine. Unfortunately at the moment the system requires GRUB, but
there is ongoing work to add the ability to the FreeBSD bootloader.

Further IIRC zfs send-receive has a history involving a developer who
wanted a better rsync for transfering his work to a laptop. In addition
we have pretty much Moore's Lawed our way to the point where a new
laptop today can out spec a typical server from when ZFS was first
implemented.

Hiding features because you 'can' shoot your foot 

Re: Periodic jobs triggering panics in 10.1 and 10.2

[Freddie Cash]

On Dec 9, 2015 7:24 PM, "Karl Denninger"  wrote:
>
> On 12/9/2015 17:29, Michael B. Eichorn wrote:
> > I sorry, but I really don't get your point, PCBSD has shown a great
> > reason why zfs on root and on laptops/desktops is a good idea... boot
> > environments. They have pretty much figured out how to use snapshots
> > to go from A-B ping-pong installations to A-B-C-D-E installations.
> > I am even aware of people using it to run Release and Current on the
> > same machine. Unfortunately at the moment the system requires GRUB,
> > but there is ongoing work to add the ability to the FreeBSD
> > bootloader. Further IIRC zfs send-receive has a history involving a
> > developer who wanted a better rsync for transfering his work to a
> > laptop. In addition we have pretty much Moore's Lawed our way to the
> > point where a new laptop today can out spec a typical server from when
> > ZFS was first implemented. Hiding features because you 'can' shoot
> > your foot off is hardly a typical UNIXy way of thinking anyway.
>
> Boot environments work fine on FreeBSD.  Look at "beadm" :-)
>
> What are you trying to do that you need GRUB?

GRUB supports listing and selecting boot environments as part of the boot
process.

FreeBSD 8.x and 9.x boot loaders don't support listing or selecting BEs.
Neither does 10.0; not sure about 10.1.

Devin Teske and company did a lot of work on this area, and I believe 10.2,
certainly -CURRENT, supports this.

Because of BEs, PC-BSD flip-flopped between the FreeBSD loader and GRUB in
the 9.x and 10.x releases. I think they support both now.

Typos courtesy of my phone's autocorrect.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: Periodic jobs triggering panics in 10.1 and 10.2

[Karl Denninger]

On 12/9/2015 17:29, Michael B. Eichorn wrote:
> I sorry, but I really don't get your point, PCBSD has shown a great
> reason why zfs on root and on laptops/desktops is a good idea... boot
> environments. They have pretty much figured out how to use snapshots
> to go from A-B ping-pong installations to A-B-C-D-E installations.
> I am even aware of people using it to run Release and Current on the
> same machine. Unfortunately at the moment the system requires GRUB,
> but there is ongoing work to add the ability to the FreeBSD
> bootloader. Further IIRC zfs send-receive has a history involving a
> developer who wanted a better rsync for transfering his work to a
> laptop. In addition we have pretty much Moore's Lawed our way to the
> point where a new laptop today can out spec a typical server from when
> ZFS was first implemented. Hiding features because you 'can' shoot
> your foot off is hardly a typical UNIXy way of thinking anyway. 

Boot environments work fine on FreeBSD.  Look at "beadm" :-)

What are you trying to do that you need GRUB?

-- 
Karl Denninger
k...@denninger.net 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: Periodic jobs triggering panics in 10.1 and 10.2

[Karl Denninger]

On 12/9/2015 21:37, Freddie Cash wrote:
>
>
> On Dec 9, 2015 7:24 PM, "Karl Denninger"  > wrote:
> >
> > On 12/9/2015 17:29, Michael B. Eichorn wrote:
> > > I sorry, but I really don't get your point, PCBSD has shown a great
> > > reason why zfs on root and on laptops/desktops is a good idea... boot
> > > environments. They have pretty much figured out how to use snapshots
> > > to go from A-B ping-pong installations to A-B-C-D-E installations.
> > > I am even aware of people using it to run Release and Current on the
> > > same machine. Unfortunately at the moment the system requires GRUB,
> > > but there is ongoing work to add the ability to the FreeBSD
> > > bootloader. Further IIRC zfs send-receive has a history involving a
> > > developer who wanted a better rsync for transfering his work to a
> > > laptop. In addition we have pretty much Moore's Lawed our way to the
> > > point where a new laptop today can out spec a typical server from when
> > > ZFS was first implemented. Hiding features because you 'can' shoot
> > > your foot off is hardly a typical UNIXy way of thinking anyway.
> >
> > Boot environments work fine on FreeBSD.  Look at "beadm" :-)
> >
> > What are you trying to do that you need GRUB?
>
> GRUB supports listing and selecting boot environments as part of the
> boot process.
>
> FreeBSD 8.x and 9.x boot loaders don't support listing or selecting
> BEs. Neither does 10.0; not sure about 10.1.
>
> Devin Teske and company did a lot of work on this area, and I believe
> 10.2, certainly -CURRENT, supports this.
>
> Because of BEs, PC-BSD flip-flopped between the FreeBSD loader and
> GRUB in the 9.x and 10.x releases. I think they support both now.
>
> Typos courtesy of my phone's autocorrect.
>

Beadm allows you to set the "next boot" and works as expected.

You can't select from the boot loader but you can certainly ping-pong
between a working and test environment easily (I do this all the time)

-- 
Karl Denninger
k...@denninger.net 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: ICH5 ATA DMA timeouts

[Perry Hutchison]

Interesting.  The ICH5 is part of the processor chipset, and it
could be "anywhere" on the system board, so finding/identifying
it (so as to point a fan at it, or figure out a way to add a
heatsink) may be less than straightforward -- but it's worth
keeping in mind as something to look into if problems arise.

Any idea where one finds cheap (but reliable) PCI SATA cards
these days?  It would have to be PCI -- this box does not have
PCI-E slots.

Florian Ermisch  wrote:

> I had some onboard (S)ATA controllers becoming unreliable in the
> past. Two of them due to chipsets overheating under load which
> were fixable with additional cooling. In the third one we just
> added a cheap SATA card and spread the redundant disks between
> onboard and add-on controller to make a temporary failure a minor
> issue.
>
> Regards, Florian
>
> Am 6. Dezember 2015 22:21:35 MEZ, schrieb Steven Hartland 
> :
> > Check cables and devices are in good condition. These things are
> > usually a connectivity issue or device failing
> > 
> > On 06/12/2015 03:06, Perry Hutchison wrote:
> > > Does anyone know the condition of the ICH5 ATA support in FreeBSD
> > > 10?
> > >
> > > In preparing to repurpose an elderly Dell Dimension 4600 from
> > > Windows to FreeBSD, and needing to decide what to do about
> > > drives, I found several mentions in the archives* of ICH5 ATA
> > > DMA timeouts -- mostly affecting the SATA ports, but the
> > > prevalence of SATA reports may just indicate which ports were
> > > getting the most use:  a couple of the reports involved the
> > > PATA ports.
> > >
> > > While there have been commits to the ATA code since then, I didn't
> > > find any definitive statement that the DMA timeouts had been fixed.
> > > Did I miss something, or would I be better off using a separate SATA
> > > or PATA PCI card instead of the ICH5's built-in ports?
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: ICH5 ATA DMA timeouts

[Perry Hutchison]

Er, there's nothing to check, yet.  I'm trying to *avoid* trouble
by finding out in advance about any dodgy hardware support.

In several of the cited threads, it was pretty clear that the
hardware was not faulty, since the identical hardware had run
properly on 4.x but started generating DMA timeouts on 5.x; also
diagnostics were run (=> drives OK), cables replaced, etc.  In at
least one case the failure was reproduced on a different box than
where it had originally been observed.

It was suspected that there might have been some kind of interaction
between geom and the ATA code, e.g. a timing problem, since many of
the failures involved software mirroring (vinum or gmirror).

Steven Hartland  wrote:

> Check cables and devices are in good condition. These things are
> usually a connectivity issue or device failing
>
> On 06/12/2015 03:06, Perry Hutchison wrote:
> > Does anyone know the condition of the ICH5 ATA support in FreeBSD 10?
> >
> > In preparing to repurpose an elderly Dell Dimension 4600 from Windows
> > to FreeBSD, and needing to decide what to do about drives, I found
> > several mentions in the archives* of ICH5 ATA DMA timeouts -- mostly
> > affecting the SATA ports, but the prevalence of SATA reports may
> > just indicate which ports were getting the most use:  a couple of
> > the reports involved the PATA ports.
> >
> > While there have been commits to the ATA code since then, I didn't
> > find any definitive statement that the DMA timeouts had been fixed.
> > Did I miss something, or would I be better off using a separate SATA
> > or PATA PCI card instead of the ICH5's built-in ports?
> >
> > Relevant parts of dmesg (with no hard drives attached):
> >
> > FreeBSD 10.2-RELEASE #0 r28: Wed Aug 12 19:31:38 UTC 2015
> >  r...@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC i386
> > CPU: Intel(R) Pentium(R) 4 CPU 2.80GHz (2793.06-MHz 686-class CPU)
> >Origin="GenuineIntel"  Id=0xf34  Family=0xf  Model=0x3  Stepping=4
> >
> > Features=0xbfebfbff
> >Features2=0x441d
> >TSC: P-state invariant
> > uhci0:  port 0xff80-0xff9f irq 
> > 16 at device 29.0 on pci0
> > usbus0 on uhci0
> > uhci1:  port 0xff60-0xff7f irq 
> > 19 at device 29.1 on pci0
> > usbus1 on uhci1
> > uhci2:  port 0xff40-0xff5f irq 
> > 18 at device 29.2 on pci0
> > usbus2 on uhci2
> > uhci3:  port 0xff20-0xff3f irq 
> > 16 at device 29.3 on pci0
> > usbus3 on uhci3
> > ehci0:  mem 
> > 0xffa80800-0xffa80bff irq 23 at device 29.7 on pci0
> > usbus4: EHCI version 1.0
> > usbus4 on ehci0
> > atapci0:  port 
> > 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf mem 0xfeb7fc00-0xfeb7 
> > irq 18 at device 31.1 on pci0
> > ata0:  at channel 0 on atapci0
> > ata1:  at channel 1 on atapci0
> > atapci1:  port 
> > 0xfe00-0xfe07,0xfe10-0xfe13,0xfe20-0xfe27,0xfe30-0xfe33,0xfea0-0xfeaf irq 
> > 18 at device 31.2 on pci0
> > ata2:  at channel 0 on atapci1
> > ata3:  at channel 1 on atapci1
> > pci0:  at device 31.3 (no driver attached)
> > pcm0:  port 0xee00-0xeeff,0xedc0-0xedff mem 
> > 0xfeb7fa00-0xfeb7fbff,0xfeb7f900-0xfeb7f9ff irq 17 at device 31.5 on pci0
> > pcm0: primary codec not ready!
> > pcm0: 
> > ata0: reset tp1 mask=00 ostat0=ff ostat1=ff
> > ata1: reset tp1 mask=03 ostat0=00 ostat1=00
> > ata1: stat0=0x00 err=0x01 lsb=0x14 msb=0xeb
> > ata1: stat1=0x00 err=0x01 lsb=0x14 msb=0xeb
> > ata1: reset tp2 stat0=00 stat1=00 devices=0x3
> > ata2: SATA reset: ports status=0x00
> > ata2: p0: SATA connect timeout status=0004
> > ata3: SATA reset: ports status=0x00
> > ata3: p0: SATA connect timeout status=0004
> > pass0 at ata1 bus 0 scbus1 target 0 lun 0
> > pass0:  Removable CD-ROM SCSI device
> > pass0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
> > pass1 at ata1 bus 0 scbus1 target 1 lun 0
> > pass1:  Removable CD-ROM SCSI device
> > pass1: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
> > cd0 at ata1 bus 0 scbus1 target 0 lun 0
> > cd0:  Removable CD-ROM SCSI device
> > cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
> > cd0: Attempt to query device size failed: NOT READY, Medium not present
> > cd1 at ata1 bus 0 scbus1 target 1 lun 0
> > cd1:  Removable CD-ROM SCSI device
> > cd1: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes)
> > cd1: Attempt to query device size failed: NOT READY, Medium not present - 
> > tray closed
> > GEOM: new disk cd0
> > GEOM: new disk cd1
> >
> > * Archive mentions, in http://lists.freebsd.org/pipermail/...
> >
> >freebsd-hardware/2004-September/thread.html#1924
> >freebsd-current/2005-February/thread.html#46719
> >freebsd-current/2005-February/thread.html#46737
> >freebsd-stable/2005-March/thread.html#13265
> >freebsd-stable/2007-May/thread.html#35061
> >freebsd-stable/2007-July/thread.html#36308
> 

Re: Periodic jobs triggering panics in 10.1 and 10.2

[Dustin Wenz]

PF filed:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=205163

Please let me know if there is any more useful information I can include.

- .Dustin Wenz


> On Dec 9, 2015, at 4:24 AM, Jan Bramkamp  wrote:
> 
> 
> 
> On 09/12/15 01:04, Michael B. Eichorn wrote:
>> On Tue, 2015-12-08 at 16:31 -0600, Dustin Wenz wrote:
>>> I suspect this is a zfs bug that is triggered by the access patterns
>>> in the periodic scripts. There is significant load on the system when
>>> the scheduled processes start, because all jails execute the same
>>> scripts at the same time.
>>> 
>>> I've been able to alleviate this problem by disabling the security
>>> scans within the jails, but leave it enabled on the root host.
>> 
>> To avoid the problem of jails all starting things at the same time, use
>> the cron(8) flags -j and -J to set a 'jitter' which will cause cron to
>> sleep for a random period of specified duration (60 sec max). Cron
>> flags can be set using the rc.conf variable 'cron_flags'.
> 
> While jitter would reduce the resource contention a thundering herd of 
> cronjobs shouldn't cause the kernel to divide by zero. Spreading the load by 
> introducing jitter to cronjobs might hide the problem, but it still needs 
> further analysis.
> 
> @Dustin Wenz: Can you reproduce the problem and file a PR to track this?
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"