Re: AIO not working on NFS share - ? Regression

[Kevin Oberman]

On Mon, Sep 5, 2016 at 1:41 PM, Jim Ohlstein  wrote:

> Hello,
>
> On 09/05/2016 04:34 PM, Konstantin Belousov wrote:
>
>> On Mon, Sep 05, 2016 at 02:26:30PM -0600, Alan Somers wrote:
>>
>>> Just to get your webserver out of the picture, could you please try
>>> running the aio test suite on an NFS share?  Do it like this:
>>> $ cd /usr/tests/sys/aio
>>> $ export TMPFS=/path/to/some/nfs/share
>>> $ kyua test
>>>
>> By default, unsafe aio is disabled, i.e. aio where the operation is not
>> really async and can usurp kernel thread for indefinite amount of time.
>> Non-local filesystems accesses are classified as unsafe, since their
>> completion is controlled by remote host.
>>
>
> Setting vfs.aio.enable_unsafe to 1 solved the problem, though I don't
> recall this being necessary in the past.
>
> Thanks for the quick responses and fix.
>
>
>
>>
>>> -Alan
>>>
>>> On Mon, Sep 5, 2016 at 2:14 PM, Jim Ohlstein  wrote:
>>>
 Hello,

 For sometime now we've used a backend (non-internet accessible) server
 to
 generate static content that is shared via NFS to a front-end server for
 internet access. Both machines use ZFS exclusively.

 The webserver is nginx and we had AIO set to on, with the AIO module
 built
 into the kernel.In 10.x it "just worked".

 About two weeks ago I upgraded that backend machine to 11-STABLE in
 order to
 start building packages for 11. I've just updated it again to r305417. I
 don't recall it not working as expected.

 More recently I upgraded the front-end machine to 11-STABLE @r305311.
 Since
 then, AIO is not working and I get a critical error in the nginx logs
 like
 this:

 2016/09/05 15:58:02 [crit] 21145#0: *649 aio_read("/path/to/file")
 failed
 (45: Operation not supported) while sending response to client ...

 In 11 AIO is built into the kernel by default, and I have confirmed
 that it
 works as expected in files from a local file system. Since the webserver
 isn't aware that the file system in question is shared via NFS, it seems
 this is an operating system issue.

>>>
> --
> Jim Ohlstein
>

Could this also be related to AIO hanging VBox VMs? See
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=212128
--
Kevin Oberman, Retired Network Engineer
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: AIO not working on NFS share - ? Regression

[Jim Ohlstein]

Hello,

On 09/05/2016 04:34 PM, Konstantin Belousov wrote:

On Mon, Sep 05, 2016 at 02:26:30PM -0600, Alan Somers wrote:

Just to get your webserver out of the picture, could you please try
running the aio test suite on an NFS share?  Do it like this:
$ cd /usr/tests/sys/aio
$ export TMPFS=/path/to/some/nfs/share
$ kyua test

By default, unsafe aio is disabled, i.e. aio where the operation is not
really async and can usurp kernel thread for indefinite amount of time.
Non-local filesystems accesses are classified as unsafe, since their
completion is controlled by remote host.


Setting vfs.aio.enable_unsafe to 1 solved the problem, though I don't 
recall this being necessary in the past.


Thanks for the quick responses and fix.





-Alan

On Mon, Sep 5, 2016 at 2:14 PM, Jim Ohlstein  wrote:

Hello,

For sometime now we've used a backend (non-internet accessible) server to
generate static content that is shared via NFS to a front-end server for
internet access. Both machines use ZFS exclusively.

The webserver is nginx and we had AIO set to on, with the AIO module built
into the kernel.In 10.x it "just worked".

About two weeks ago I upgraded that backend machine to 11-STABLE in order to
start building packages for 11. I've just updated it again to r305417. I
don't recall it not working as expected.

More recently I upgraded the front-end machine to 11-STABLE @r305311. Since
then, AIO is not working and I get a critical error in the nginx logs like
this:

2016/09/05 15:58:02 [crit] 21145#0: *649 aio_read("/path/to/file") failed
(45: Operation not supported) while sending response to client ...

In 11 AIO is built into the kernel by default, and I have confirmed that it
works as expected in files from a local file system. Since the webserver
isn't aware that the file system in question is shared via NFS, it seems
this is an operating system issue.


--
Jim Ohlstein
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: AIO not working on NFS share - ? Regression

[Konstantin Belousov]

On Mon, Sep 05, 2016 at 02:26:30PM -0600, Alan Somers wrote:
> Just to get your webserver out of the picture, could you please try
> running the aio test suite on an NFS share?  Do it like this:
> $ cd /usr/tests/sys/aio
> $ export TMPFS=/path/to/some/nfs/share
> $ kyua test
By default, unsafe aio is disabled, i.e. aio where the operation is not
really async and can usurp kernel thread for indefinite amount of time.
Non-local filesystems accesses are classified as unsafe, since their
completion is controlled by remote host.

> 
> -Alan
> 
> On Mon, Sep 5, 2016 at 2:14 PM, Jim Ohlstein  wrote:
> > Hello,
> >
> > For sometime now we've used a backend (non-internet accessible) server to
> > generate static content that is shared via NFS to a front-end server for
> > internet access. Both machines use ZFS exclusively.
> >
> > The webserver is nginx and we had AIO set to on, with the AIO module built
> > into the kernel.In 10.x it "just worked".
> >
> > About two weeks ago I upgraded that backend machine to 11-STABLE in order to
> > start building packages for 11. I've just updated it again to r305417. I
> > don't recall it not working as expected.
> >
> > More recently I upgraded the front-end machine to 11-STABLE @r305311. Since
> > then, AIO is not working and I get a critical error in the nginx logs like
> > this:
> >
> > 2016/09/05 15:58:02 [crit] 21145#0: *649 aio_read("/path/to/file") failed
> > (45: Operation not supported) while sending response to client ...
> >
> > In 11 AIO is built into the kernel by default, and I have confirmed that it
> > works as expected in files from a local file system. Since the webserver
> > isn't aware that the file system in question is shared via NFS, it seems
> > this is an operating system issue.
> >
> > --
> > Jim Ohlstein
> > ___
> > freebsd-stable@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: AIO not working on NFS share - ? Regression

[Alan Somers]

Just to get your webserver out of the picture, could you please try
running the aio test suite on an NFS share?  Do it like this:
$ cd /usr/tests/sys/aio
$ export TMPFS=/path/to/some/nfs/share
$ kyua test

-Alan

On Mon, Sep 5, 2016 at 2:14 PM, Jim Ohlstein  wrote:
> Hello,
>
> For sometime now we've used a backend (non-internet accessible) server to
> generate static content that is shared via NFS to a front-end server for
> internet access. Both machines use ZFS exclusively.
>
> The webserver is nginx and we had AIO set to on, with the AIO module built
> into the kernel.In 10.x it "just worked".
>
> About two weeks ago I upgraded that backend machine to 11-STABLE in order to
> start building packages for 11. I've just updated it again to r305417. I
> don't recall it not working as expected.
>
> More recently I upgraded the front-end machine to 11-STABLE @r305311. Since
> then, AIO is not working and I get a critical error in the nginx logs like
> this:
>
> 2016/09/05 15:58:02 [crit] 21145#0: *649 aio_read("/path/to/file") failed
> (45: Operation not supported) while sending response to client ...
>
> In 11 AIO is built into the kernel by default, and I have confirmed that it
> works as expected in files from a local file system. Since the webserver
> isn't aware that the file system in question is shared via NFS, it seems
> this is an operating system issue.
>
> --
> Jim Ohlstein
> ___
> freebsd-stable@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

AIO not working on NFS share - ? Regression

[Jim Ohlstein]

Hello,

For sometime now we've used a backend (non-internet accessible) server 
to generate static content that is shared via NFS to a front-end server 
for internet access. Both machines use ZFS exclusively.


The webserver is nginx and we had AIO set to on, with the AIO module 
built into the kernel.In 10.x it "just worked".


About two weeks ago I upgraded that backend machine to 11-STABLE in 
order to start building packages for 11. I've just updated it again to 
r305417. I don't recall it not working as expected.


More recently I upgraded the front-end machine to 11-STABLE @r305311. 
Since then, AIO is not working and I get a critical error in the nginx 
logs like this:


2016/09/05 15:58:02 [crit] 21145#0: *649 aio_read("/path/to/file") 
failed (45: Operation not supported) while sending response to client ...


In 11 AIO is built into the kernel by default, and I have confirmed that 
it works as expected in files from a local file system. Since the 
webserver isn't aware that the file system in question is shared via 
NFS, it seems this is an operating system issue.


--
Jim Ohlstein
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"