The branch, master has been updated via 1ca4df5 Use sys_pread/sys_pwrite to cope correctly with 64-bit sizes. As in the default VFS case fall back from pread -> read and pwrite -> write on an ESPIPE error in the worker thread. via d8c6991 Add man page for vfs_aio_pthread module. from 851b2c1 s3: Fix a panic in aio_pthread
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit 1ca4df58c76e98d963d90c092532ca584606ccff Author: Jeremy Allison <j...@samba.org> Date: Wed Jan 25 15:30:56 2012 -0800 Use sys_pread/sys_pwrite to cope correctly with 64-bit sizes. As in the default VFS case fall back from pread -> read and pwrite -> write on an ESPIPE error in the worker thread. Autobuild-User: Jeremy Allison <j...@samba.org> Autobuild-Date: Thu Jan 26 02:11:28 CET 2012 on sn-devel-104 commit d8c699190d2cc0ce64395c7b2b10bb25c98a2943 Author: Jeremy Allison <j...@samba.org> Date: Wed Jan 25 14:11:12 2012 -0800 Add man page for vfs_aio_pthread module. ----------------------------------------------------------------------- Summary of changes: docs-xml/manpages-3/vfs_aio_pthread.8.xml | 118 +++++++++++++++++++++++++++++ source3/modules/vfs_aio_pthread.c | 18 ++++- 2 files changed, 134 insertions(+), 2 deletions(-) create mode 100644 docs-xml/manpages-3/vfs_aio_pthread.8.xml Changeset truncated at 500 lines: diff --git a/docs-xml/manpages-3/vfs_aio_pthread.8.xml b/docs-xml/manpages-3/vfs_aio_pthread.8.xml new file mode 100644 index 0000000..625d6f3 --- /dev/null +++ b/docs-xml/manpages-3/vfs_aio_pthread.8.xml @@ -0,0 +1,118 @@ +<?xml version="1.0" encoding="iso-8859-1"?> +<!DOCTYPE refentry PUBLIC "-//Samba-Team//DTD DocBook V4.2-Based Variant V1.0//EN" "http://www.samba.org/samba/DTD/samba-doc"> +<refentry id="vfs_aio_pthread.8"> + +<refmeta> + <refentrytitle>vfs_aio_pthread</refentrytitle> + <manvolnum>8</manvolnum> + <refmiscinfo class="source">Samba</refmiscinfo> + <refmiscinfo class="manual">System Administration tools</refmiscinfo> + <refmiscinfo class="version">3.6</refmiscinfo> +</refmeta> + + +<refnamediv> + <refname>vfs_aio_pthread</refname> + <refpurpose>implement async I/O in Samba vfs using a pthread pool</refpurpose> +</refnamediv> + +<refsynopsisdiv> + <cmdsynopsis> + <command>vfs objects = aio_pthread</command> + </cmdsynopsis> +</refsynopsisdiv> + +<refsect1> + <title>DESCRIPTION</title> + + <para>This VFS module is part of the + <citerefentry><refentrytitle>samba</refentrytitle> + <manvolnum>7</manvolnum></citerefentry> suite.</para> + + <para>The <command>aio_pthread</command> VFS module enables asynchronous + I/O for Samba on platforms which have the pthreads API available, + without using the Posix AIO interface. Posix AIO can suffer from severe + limitations. For example, on some Linux versions the + real-time signals that it uses are broken under heavy load. + Other systems only allow AIO when special kernel modules are + loaded or only allow a certain system-wide amount of async + requests being scheduled. Systems based on glibc (most Linux + systems) only allow a single outstanding request per file + descriptor which essentially makes Posix AIO useless on systems + using the glibc implementation.</para> + + <para>To work around all these limitations, the aio_pthread module + was written. It uses a pthread pool instead of the + internal Posix AIO interface to allow read and write calls + to be process asynchronously. A pthread pool is created + which expands dynamically by creating new threads as work is + given to it to a maximum of 100 threads per smbd process. + To change this limit see the "aio num threads" parameter + below. New threads are not created if idle threads are + available when a new read or write request is received, + the new work is given to an existing idle thread. Threads + terminate themselves if idle for one second. + </para> + + <para> + Note that the smb.conf parameters <command>aio read size</command> + and <command>aio write size</command> must also be set appropriately + for this module to be active. + </para> + + <para>This module MUST be listed last in any module stack as + the Samba VFS pread/pwrite interface is not thread-safe. This + module makes direct pread and pwrite system calls and does + NOT call the Samba VFS pread and pwrite interfaces.</para> + +</refsect1> + + +<refsect1> + <title>EXAMPLES</title> + + <para>Straight forward use:</para> + +<programlisting> + <smbconfsection name="[cooldata]"/> + <smbconfoption name="path">/data/ice</smbconfoption> + <smbconfoption name="vfs objects">aio_fork</smbconfoption> +</programlisting> + +</refsect1> + +<refsect1> + <title>OPTIONS</title> + + <variablelist> + + <varlistentry> + <term>aio_pthread:aio num threads = INTEGER</term> + <listitem> + <para>Limit the maximum number of threads per smbd that + will be created in the thread pool to service IO requests. + </para> + <para>By default this is set to 100.</para> + </listitem> + </varlistentry> + + </variablelist> +</refsect1> +<refsect1> + <title>VERSION</title> + + <para>This man page is correct for version 3.6.3 of the Samba suite. + </para> +</refsect1> + +<refsect1> + <title>AUTHOR</title> + + <para>The original Samba software and related utilities + were created by Andrew Tridgell. Samba is now developed + by the Samba Team as an Open Source project similar + to the way the Linux kernel is developed.</para> + +</refsect1> + +</refentry> diff --git a/source3/modules/vfs_aio_pthread.c b/source3/modules/vfs_aio_pthread.c index e1cc492..c172ff0 100644 --- a/source3/modules/vfs_aio_pthread.c +++ b/source3/modules/vfs_aio_pthread.c @@ -115,15 +115,29 @@ static void aio_worker(void *private_data) (struct aio_private_data *)private_data; if (pd->write_command) { - pd->ret_size = pwrite(pd->aiocb->aio_fildes, + pd->ret_size = sys_pwrite(pd->aiocb->aio_fildes, (const void *)pd->aiocb->aio_buf, pd->aiocb->aio_nbytes, pd->aiocb->aio_offset); + if (pd->ret_size == -1 && errno == ESPIPE) { + /* Maintain the fiction that pipes can + be seeked (sought?) on. */ + pd->ret_size = sys_write(pd->aiocb->aio_fildes, + (const void *)pd->aiocb->aio_buf, + pd->aiocb->aio_nbytes); + } } else { - pd->ret_size = pread(pd->aiocb->aio_fildes, + pd->ret_size = sys_pread(pd->aiocb->aio_fildes, (void *)pd->aiocb->aio_buf, pd->aiocb->aio_nbytes, pd->aiocb->aio_offset); + if (pd->ret_size == -1 && errno == ESPIPE) { + /* Maintain the fiction that pipes can + be seeked (sought?) on. */ + pd->ret_size = sys_read(pd->aiocb->aio_fildes, + (void *)pd->aiocb->aio_buf, + pd->aiocb->aio_nbytes); + } } if (pd->ret_size == -1) { pd->ret_errno = errno; -- Samba Shared Repository