The branch, v4-20-test has been updated via 0bd26677e65 s3:smb2_server: return NT_STATUS_NETWORK_SESSION_EXPIRED for compound requests via 6a416edfa21 s4:torture/smb2: let smb2.session.expire2* also check compound requests via bb8d642a140 docs-xml/manpages: 'ceph_new' prefix for config-param of vfs_ceph_new via a8c3db7e21e vfs_ceph_new: use 'ceph_new' for config-param prefix via ce958aeef1a vfs_ceph_new: handle errno properly for 'readdir' via 07f156d8436 vfs_ceph{_new}: do not set errno upon successful call to libcephfs via 0042d902498 vfs_ceph_new: Unconditionally use ceph_select_filesystem via a78c9b0a505 docs-xml/manpages: add entry for vfs_ceph_new via f399191fba5 vfs_ceph_new: common prefix to debug-log messages via c569dfcb8bc vfs_ceph_new: debug-log upon libcephfs low-level calls via 07579685b91 vfs_ceph_new: use low-level APIs for xattr ops via 34f9de8bbe3 vfs_ceph_new: use low-level APIs for mknodat via 0ab6fa78a4b vfs_ceph_new: use low-level APIs for renameat via 405a93f07ce vfs_ceph_new: use low-level APIs for linkat via 2583b0e8829 vfs_ceph_new: use low-level APIs for ftruncate/fallocate via dda8b674320 vfs_ceph_new: use low-level APIs for fsync via 61d7c591dee vfs_ceph_new: use low-level APIs for lseek via 9d51b996bfe vfs_ceph_new: use low-level APIs for read/write via 4b53a5e3910 vfs_ceph_new: use low-level APIs for symlink/readlink via 6b79716d2f6 vfs_ceph_new: use low-level APIs for unlinkat via b9b5dab06c5 vfs_ceph_new: use low-level APIs for fntimes via 946921a5191 vfs_ceph_new: use low-level APIs for fchown/fchmod via 62758ba3d48 vfs_ceph_new: proper error handling to readdir via 6403fb8d5ac vfs_ceph_new: use low-level APIs for readdir ops via 7c1f160f838 vfs_ceph_new: use low-level APIs for mkdirat via cf7fd4172ab vfs_ceph_new: use low-level APIs for fdopendir via dff808b32a4 vfs_ceph_new: use low-level APIs for fstatat via 73ef8906b3f vfs_ceph_new: use low-level APIs for fstat via 76dd52023b0 vfs_ceph_new: use low-level APIs for open/close via d81835ab8ea vfs_ceph_new: ref cephmount_cached entry in handle->data via 8d2255e5898 vfs_ceph_new: use low-level APIs for lchown via 1a9d6754c25 vfs_ceph_new: use low-level APIs for statfs via 8aa247461bb vfs_ceph_new: use low-level APIs for lstat via 30d2e62c399 vfs_ceph_new: use low-level APIs for stat via 94aa465513e vfs_ceph_new: use low-level APIs for disk_free via 565fe2e2f4c vfs_ceph_new: next iteration of samba-to-cephfs bridge from ebca31b6971 s3:ntlm_auth: make logs more consistent with length check
https://git.samba.org/?p=samba.git;a=shortlog;h=v4-20-test - Log ----------------------------------------------------------------- commit 0bd26677e65b700c23d07e524c1b884ee0bfe6fc Author: Stefan Metzmacher <me...@samba.org> Date: Tue Aug 13 14:07:06 2024 +0200 s3:smb2_server: return NT_STATUS_NETWORK_SESSION_EXPIRED for compound requests BUG: https://bugzilla.samba.org/show_bug.cgi?id=15696 Signed-off-by: Stefan Metzmacher <me...@samba.org> Reviewed-by: Jeremy Allison <j...@samba.org> Autobuild-User(master): Jeremy Allison <j...@samba.org> Autobuild-Date(master): Tue Aug 13 22:29:28 UTC 2024 on atb-devel-224 (cherry picked from commit 4df1bfd07012dd3d2d2921281e6d6e309303b88d) Autobuild-User(v4-20-test): Jule Anger <jan...@samba.org> Autobuild-Date(v4-20-test): Tue Aug 20 12:47:54 UTC 2024 on atb-devel-224 commit 6a416edfa214c9807a65d6d62cbe2e30f836d701 Author: Stefan Metzmacher <me...@samba.org> Date: Tue Aug 13 12:47:59 2024 +0200 s4:torture/smb2: let smb2.session.expire2* also check compound requests This shows that all compound related requests should get NT_STATUS_NETWORK_SESSION_EXPIRED. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15696 Signed-off-by: Stefan Metzmacher <me...@samba.org> Reviewed-by: Jeremy Allison <j...@samba.org> (cherry picked from commit f6009aa73b9234df1e6ab689de322487ad1394ed) commit bb8d642a1402c9d625e02d610d973a34b83ef3a2 Author: Shachar Sharon <ssha...@redhat.com> Date: Mon Aug 5 19:12:29 2024 +0300 docs-xml/manpages: 'ceph_new' prefix for config-param of vfs_ceph_new With 'ceph_new' prefix used by vfs_ceph_new for config parameters, update the relevant man-page accordingly. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Anoop C S <anoo...@samba.org> Reviewed-by: Guenther Deschner <g...@samba.org> Autobuild-User(master): Günther Deschner <g...@samba.org> Autobuild-Date(master): Thu Aug 8 13:54:34 UTC 2024 on atb-devel-224 (cherry picked from commit 68f0835c8e1c5029cd831c267b75c02185b206c7) commit a8c3db7e21e0e0696e15166a5ed683cdb3d44034 Author: Shachar Sharon <ssha...@redhat.com> Date: Mon Aug 5 16:21:10 2024 +0300 vfs_ceph_new: use 'ceph_new' for config-param prefix Use explicit 'ceph_new' prefix to each of the ceph specific config parameters to avoid confusion with legacy 'vfs_ceph' module. Hence, users will have in their smb.conf a format similar to: ... [smbshare] vfs objects = ceph_new ceph_new: config_file = /etc/ceph/ceph.conf ceph_new: user_id = user1 ceph_new: filesystem = fs1 ... BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Anoop C S <anoo...@samba.org> Reviewed-by: Guenther Deschner <g...@samba.org> (cherry picked from commit aca4cf8327dcaef782dedd98a63a020469c45cdb) commit ce958aeef1afe8e54c82383ac81ab9765c78e040 Author: Shachar Sharon <ssha...@redhat.com> Date: Tue Jul 30 17:36:09 2024 +0300 vfs_ceph_new: handle errno properly for 'readdir' Take special care for readdir errno setting: in case of error, update errno by libcephfs (and protect from possible over-write by debug logging); in the case of successful result or end-of-stream restore errno to its previous value before calling the readdir_fn VFS hook. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> Autobuild-User(master): Günther Deschner <g...@samba.org> Autobuild-Date(master): Wed Aug 7 14:20:02 UTC 2024 on atb-devel-224 (cherry picked from commit aa043a5808b73fc272de585c1446372fa3f21d08) commit 07f156d8436ac5ecbfd99e88231625351b54e16e Author: Shachar Sharon <ssha...@redhat.com> Date: Tue Jul 30 09:55:44 2024 +0300 vfs_ceph{_new}: do not set errno upon successful call to libcephfs There is code in Samba that expects errno from a previous system call to be preserved through a subsequent system call. Thus, avoid setting "errno = 0" in status_code() and lstatus_code() upon successful return from libcephfs API call. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit a7f4e2bd47c7f4728f3ac8d90af693156a69c557) commit 0042d902498d2640c0f36b22f4b4c8fee63b24b4 Author: Anoop C S <anoo...@samba.org> Date: Fri Aug 2 11:10:28 2024 +0530 vfs_ceph_new: Unconditionally use ceph_select_filesystem Currently we don't have an explicit check for the presence of ceph_select_filesystem() libcephfs API as it is always found to be present with the minimum ceph version that is supported with Samba right now. Therefore under this assumption directly call ceph_select_filesystem() without any #ifdefs. Please note that this change is already part of vfs_ceph via ef0068cd. ref: https://gitlab.com/samba-team/samba/-/merge_requests/3715 BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Anoop C S <anoo...@samba.org> Reviewed-by: Guenther Deschner <g...@samba.org> Autobuild-User(master): Anoop C S <anoo...@samba.org> Autobuild-Date(master): Mon Aug 5 16:06:47 UTC 2024 on atb-devel-224 (cherry picked from commit de2f76fa47e6e672ce353ea9d3dc4019965c6491) commit a78c9b0a5053fbe3363aedbd131de04780162909 Author: Shachar Sharon <ssha...@redhat.com> Date: Tue Feb 20 19:37:45 2024 +0200 docs-xml/manpages: add entry for vfs_ceph_new Create man entry for the newly added vfs_ceph_new module: almost identical to existing vfs_ceph, except to the configuration entry: [sharename] vfs objects = ceph_new ... Adds a bit of info for the motivation behind this new module. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit d8c84a2993b84ebb69011c33c1b5d44801c15363) commit f399191fba551f429874315b938b83a6ed1d040c Author: Shachar Sharon <ssha...@redhat.com> Date: Tue Jul 16 14:33:16 2024 +0300 vfs_ceph_new: common prefix to debug-log messages Keep logging consistent: add "[CEPH] " prefix to DBG_DEBUG log messages where missing. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> Autobuild-User(master): Günther Deschner <g...@samba.org> Autobuild-Date(master): Mon Jul 29 15:58:15 UTC 2024 on atb-devel-224 (cherry picked from commit 3bb6d441bf047bef6d95675057cecd3865a25540) commit c569dfcb8bcc89037e39869ecf90655d4f7bdf83 Author: Shachar Sharon <ssha...@redhat.com> Date: Wed Jun 26 17:24:37 2024 +0300 vfs_ceph_new: debug-log upon libcephfs low-level calls Add developer's debug-logging upon each call to libcephfs' low-level APIs. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit d00f20f30f4e77463e82d202099682b7ef68260f) commit 07579685b91c70f743121bc6e1b3bcf07ab738e7 Author: Shachar Sharon <ssha...@redhat.com> Date: Sun Jun 23 14:57:10 2024 +0300 vfs_ceph_new: use low-level APIs for xattr ops Implement extended-attributes operations using libcephfs' low-level APIs. Whenever possible, use the open file-handle from fsp-extension to resolve inode-reference and user-permissions. Otherwise, resolve both on-the-fly. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 0a8445e891c64d703d44258b2eef85296265c55f) commit 34f9de8bbe3142dd12825ec4407b24b41a381bb4 Author: Shachar Sharon <ssha...@redhat.com> Date: Sun Jun 23 13:08:25 2024 +0300 vfs_ceph_new: use low-level APIs for mknodat Implement mknodat operations using libcephfs' low-level APIs. Requires parent directory to have valid inode-ref associated with its fsp extension. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit e714e5ddc50a771e743f5e63f686c106abe33b75) commit 0ab6fa78a4b7abda52616f65120335270ae11c4d Author: Shachar Sharon <ssha...@redhat.com> Date: Sun Jun 23 12:47:19 2024 +0300 vfs_ceph_new: use low-level APIs for renameat Implement renameat operations using libcephfs' low-level APIs. Requires both directories to have valid inode-ref associated with their fsp extension. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 83011357fb834e92505f17d6f65d5f32e3d37ec0) commit 405a93f07cea4aa31a15de457e49abbc539058cc Author: Shachar Sharon <ssha...@redhat.com> Date: Thu Jun 20 22:46:52 2024 +0300 vfs_ceph_new: use low-level APIs for linkat Implement link operations using libcephfs' low-level APIs. Requires two phase operation: resolve (by-lookup) reference to inode and then do the actual (hard) link operation using parent dir-inode reference to the locally-cached inode. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 9a70bd606728110ad91cab547a4e31350010bb68) commit 2583b0e8829d584c5040edfb8ecc720e332c3995 Author: Shachar Sharon <ssha...@redhat.com> Date: Thu Jun 20 14:58:34 2024 +0300 vfs_ceph_new: use low-level APIs for ftruncate/fallocate Implement ftruncate/fallocate operations using libcephfs' low-level APIs. Requires open ceph Fh* associated with fsp (extension). BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit b536bf1fa87fb794e2992ab5368f41fdba80e3ad) commit dda8b67432005ed11401773bda6b384b28b1391f Author: Shachar Sharon <ssha...@redhat.com> Date: Thu Jun 20 12:43:39 2024 +0300 vfs_ceph_new: use low-level APIs for fsync Implement fsync operation using libcephfs' low-level APIs. Requires open ceph Fh* associated with fsp (extension). BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit e15586fc6097565208011c556282d83eeec2230b) commit 61d7c591deedb9d61e57c06aa28decc356bf4b42 Author: Shachar Sharon <ssha...@redhat.com> Date: Thu Jun 20 12:23:03 2024 +0300 vfs_ceph_new: use low-level APIs for lseek Implement lseek operation using libcephfs' low-level APIs. Requires open ceph Fh* associated with fsp (extension). BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 30c1a613fee3f625c0559e49e037af9fad04c3b8) commit 9d51b996bfe889653349bc172d55bc6fedcc8c84 Author: Shachar Sharon <ssha...@redhat.com> Date: Wed Jun 19 17:52:45 2024 +0300 vfs_ceph_new: use low-level APIs for read/write Implement read/write IO operations using libcephfs' low-level APIs. Requires open ceph Fh* associated with fsp (extension) to complete both pread/pwrite as well as async I/O operations. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 29bbe0f52d4ffae9dbb070ffc525acf99203444b) commit 4b53a5e3910a094956f5bff86620b01e1f70c680 Author: Shachar Sharon <ssha...@redhat.com> Date: Wed Jun 26 13:46:54 2024 +0300 vfs_ceph_new: use low-level APIs for symlink/readlink Implement unlinkat using libcephfs low-level APIs. For readlink operation need to resolve child inode by-lookup and then used the inode reference for the actual low-level readlink. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 53c9269b219a54236500d22d8a4c7f2ed582faaf) commit 6b79716d2f605d0fbd200f2e5c3f3005e4417eba Author: Shachar Sharon <ssha...@redhat.com> Date: Wed Jun 19 14:27:24 2024 +0300 vfs_ceph_new: use low-level APIs for unlinkat Implement unlinkat using libcephfs low-level APIs. Operate using parent directory's open file-handle. When flags has AT_REMOVEDIR bit set call low-level rmdir; otherwise, do normal unlink. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 362a7cf8664270145bff815347e447797cc1a643) commit b9b5dab06c5e6c529327d2c1fb343996a941807c Author: Shachar Sharon <ssha...@redhat.com> Date: Wed Jun 19 13:59:53 2024 +0300 vfs_ceph_new: use low-level APIs for fntimes Implement fntimes hook using libcephfs' low-level APIs. Convert smb_file_time to ceph_statx plus proper field mask on-the-fly upon issuing low-level call to libcephfs. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 20b7d2bfe06beefb5e7f091eb317ad18cb53f8a9) commit 946921a5191131a439b4116aef8ed5c542edd376 Author: Shachar Sharon <ssha...@redhat.com> Date: Wed Jun 19 12:48:14 2024 +0300 vfs_ceph_new: use low-level APIs for fchown/fchmod Use libcephfs' low-level APIs to implement 'fchown' and 'fchmod' using open file-handle. If fsp does not have an open cephfs Fh reference, set errno to EBADF and return -1 to VFS. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit cb14d3630d8c110405c2a43bef15aa31ec4a0fba) commit 62758ba3d4897d317b8ee1167d9cec80dfecdf09 Author: Shachar Sharon <ssha...@redhat.com> Date: Wed Jul 17 11:41:13 2024 +0300 vfs_ceph_new: proper error handling to readdir Error handling in the case of 'ceph_readdir' is done by setting 'errno' deep within libcephfs code. In case of error, emit proper debug message and re-update errno to avoid possible over-write by logging mechanism. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 24a3423949e127177c019a0d126c6f7523e61984) commit 6403fb8d5acd84c3ff84a14e9ab4cebeded8ecde Author: Shachar Sharon <ssha...@redhat.com> Date: Mon Jun 24 13:33:05 2024 +0300 vfs_ceph_new: use low-level APIs for readdir ops Implement readdir and rewinddir operations using libcephfs' low-level APIs. Casts the opaque DIR pointer into struct vfs_ceph_dirp (the first member of struct vfs_ceph_fh) to resolve the ceph_dir_result pointer which libcephfs expects for readdir operations. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 99c7179e5da6d201f03b1a04dbe2a6722090783d) commit 7c1f160f83899a37e618c83f0bc86dce889ec3e2 Author: Shachar Sharon <ssha...@redhat.com> Date: Wed Jun 19 11:55:27 2024 +0300 vfs_ceph_new: use low-level APIs for mkdirat Implement 'mkdirat' hook using libcephfs' low-level APIs, via the open file-handle reference to parent directory. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit bd955af86e71fa6c87648e578890ea6f4d490d4b) commit cf7fd4172ab1fbb694c8894a66a1731ab22c2180 Author: Shachar Sharon <ssha...@redhat.com> Date: Tue Jun 18 17:20:59 2024 +0300 vfs_ceph_new: use low-level APIs for fdopendir Implement fdopendir using libcephfs low-level API and cached (via fsp) open file-handle. Embed the result within cached vfs_ceph_fh so it may be used properly by closedir. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit a8a7339c6b7a6866399fd6c409228267a585740f) commit dff808b32a4502fae2103031f47674c1ec240db9 Author: Shachar Sharon <ssha...@redhat.com> Date: Wed Jun 19 12:35:11 2024 +0300 vfs_ceph_new: use low-level APIs for fstatat Use libcephfs' low-level APIs to do lookup-by-name via parent's open reference followed by getattr on the inode itself. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 13671cefffb268d84c973583669681318a2ce3bb) commit 73ef8906b3fefc105be9fd520ccfe857b6c1af09 Author: Shachar Sharon <ssha...@redhat.com> Date: Mon Jun 24 11:39:43 2024 +0300 vfs_ceph_new: use low-level APIs for fstat Use libcephfs' low-level APIs and apply the same logic as stat, but via explicit inode-reference. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit f16183f90abba3c2d3d26262926f1454275a9d3f) commit 76dd52023b0a349ead38e8a81cf8a3a5eb1adce0 Author: Shachar Sharon <ssha...@redhat.com> Date: Tue Jun 18 15:02:52 2024 +0300 vfs_ceph_new: use low-level APIs for open/close Implement openat, close and closedir and hooks using libcephfs' low-level APIs. Cache the open Fh* from libcephfs and its related meta-data using VFS fsp-extension mechanism. Upon open-create of new vfs_ceph_fh store the caller credentials (ceph's UserPerm*) within the same context object for subsequent calls. In addition, provide a "pseudo" fd numbering which is reported back to VFS layer and used as debugging hints. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 491676846458980944b76d1693726627a9a32503) commit d81835ab8ea47cce1efac5b621bfba5b91e6217a Author: Shachar Sharon <ssha...@redhat.com> Date: Tue Jun 18 12:58:52 2024 +0300 vfs_ceph_new: ref cephmount_cached entry in handle->data Allow direct access to ceph-mount cached-entry via 'handle->data' private pointer. Required in order to allow more complex cached-state with each cephfs mount. Users should now use the local-helper function 'cmount_of' to access the underlying ceph_mount_info. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 31085c7efc3572bd6200d3d8e49c1e554cdbfbcc) commit 8d2255e5898b03ac1475746ed80425a842021056 Author: Shachar Sharon <ssha...@redhat.com> Date: Mon Jun 17 16:59:05 2024 +0300 vfs_ceph_new: use low-level APIs for lchown Use libcephfs' low-level API ceph_ll_setattr to implement VFS lchown_fn hook. Use to standard pattern of iget/iput to allow operation by Inode reference. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit beb21324c9a554f50d8d99af2a1b7fe8a17c8ebb) commit 1a9d6754c25ef311a0d71990f1408752edeef4ca Author: Shachar Sharon <ssha...@redhat.com> Date: Mon Jun 17 18:02:07 2024 +0300 vfs_ceph_new: use low-level APIs for statfs BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 47224fbdeb55100cf8a7ee75e13b954ab71fc158) commit 8aa247461bb23a98a1e0ca4c6f6ccbe434bdc51f Author: Shachar Sharon <ssha...@redhat.com> Date: Mon Jun 17 15:57:42 2024 +0300 vfs_ceph_new: use low-level APIs for lstat Use libcephfs' low-level APIs and apply the same logic as stat, but using AT_SYMLINK_NOFOLLOW flags. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 93d786b14358db5664e13b1aa43f3f03e7cf0be3) commit 30d2e62c399736c505305c24ba737763337f31cd Author: Shachar Sharon <ssha...@redhat.com> Date: Mon Jun 17 12:11:18 2024 +0300 vfs_ceph_new: use low-level APIs for stat Start migrating to libcephfs' low-level APIs, using explicit Inode* reference. Implement the VFS 'stat' hook using a ceph_ll_getattr function, encapsulated with a pair of iget/iput to hold a pinned-to-cache Inode* instance. Upon calling to libcephfs this new code crates and destroys on-the-fly a Ceph UserPerm instance based on the uig, gid and groups from 'handle->conn->session_info->unix_token'. This logic ensures that the correct caller-credentials are passed-on to cephfs (instead of those set upon connection-creation in legacy 'vfs_ceph.c'). BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 1b78d79663c48aa4b6810a875427de85ae49a2e8) commit 94aa465513e54167b3d39180aa4e0e70f93fcc26 Author: Shachar Sharon <ssha...@redhat.com> Date: Sun Jun 16 14:50:08 2024 +0300 vfs_ceph_new: use low-level APIs for disk_free Start using libcephfs low-level APIs: get reference to root inode and use it to query statfs. Requires an explicit put-inode to avoid resource leakage by libcephfs. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 192b0cf8717d79197b985539c9db8ca07a89c570) commit 565fe2e2f4c23644a3909b40ffa0c9917e9c6de1 Author: Shachar Sharon <ssha...@redhat.com> Date: Thu Jun 13 15:54:48 2024 +0300 vfs_ceph_new: next iteration of samba-to-cephfs bridge Defined new module 'vfs_ceph_new.c' which serves as a place holder for the next development phase of the bridge between samba's VFS layer and libcephfs. Begin with a module which is almost identical to existing 'vfs_ceph.c', except for hooks-names prefix which is 'vfs_ceph_' in order to make clear distinction from existing code base. Following commits will also switch to low-level APIs. BUG: https://bugzilla.samba.org/show_bug.cgi?id=15686 Signed-off-by: Shachar Sharon <ssha...@redhat.com> Reviewed-by: Guenther Deschner <g...@samba.org> Reviewed-by: Anoop C S <anoo...@samba.org> (cherry picked from commit 3720452720b4760509875f0d2a8ed0d104bb1844) ----------------------------------------------------------------------- Summary of changes: .../{vfs_ceph.8.xml => vfs_ceph_new.8.xml} | 60 +- docs-xml/wscript_build | 1 + source3/modules/vfs_ceph_new.c | 3100 ++++++++++++++++++++ source3/modules/wscript_build | 10 + source3/smbd/smb2_server.c | 16 +- source3/wscript | 1 + source4/torture/smb2/session.c | 56 + 7 files changed, 3218 insertions(+), 26 deletions(-) copy docs-xml/manpages/{vfs_ceph.8.xml => vfs_ceph_new.8.xml} (66%) create mode 100644 source3/modules/vfs_ceph_new.c Changeset truncated at 500 lines: diff --git a/docs-xml/manpages/vfs_ceph.8.xml b/docs-xml/manpages/vfs_ceph_new.8.xml similarity index 66% copy from docs-xml/manpages/vfs_ceph.8.xml copy to docs-xml/manpages/vfs_ceph_new.8.xml index 47b5523b9a1..b0640a591a5 100644 --- a/docs-xml/manpages/vfs_ceph.8.xml +++ b/docs-xml/manpages/vfs_ceph_new.8.xml @@ -1,9 +1,9 @@ <?xml version="1.0" encoding="iso-8859-1"?> <!DOCTYPE refentry PUBLIC "-//Samba-Team//DTD DocBook V4.2-Based Variant V1.0//EN" "http://www.samba.org/samba/DTD/samba-doc"> -<refentry id="vfs_ceph.8"> +<refentry id="vfs_ceph_new.8"> <refmeta> - <refentrytitle>vfs_ceph</refentrytitle> + <refentrytitle>vfs_ceph_new</refentrytitle> <manvolnum>8</manvolnum> <refmiscinfo class="source">Samba</refmiscinfo> <refmiscinfo class="manual">System Administration tools</refmiscinfo> @@ -12,15 +12,15 @@ <refnamediv> - <refname>vfs_ceph</refname> + <refname>vfs_ceph_new</refname> <refpurpose> - Utilize features provided by CephFS + Utilize features provided by libcephfs low-level APIs </refpurpose> </refnamediv> <refsynopsisdiv> <cmdsynopsis> - <command>vfs objects = ceph</command> + <command>vfs objects = ceph_new</command> </cmdsynopsis> </refsynopsisdiv> @@ -32,7 +32,7 @@ <manvolnum>8</manvolnum></citerefentry> suite.</para> <para> - The <command>vfs_ceph</command> VFS module exposes + The <command>vfs_ceph_new</command> VFS module exposes CephFS specific features for use by Samba. </para> @@ -51,37 +51,47 @@ </para> <para> - <command>vfs_ceph</command> performs mapping between Windows and - POSIX Access Control Lists (ACLs). To ensure correct processing - and enforcement of POSIX ACLs, the following Ceph configuration - parameters are automatically applied: + <command>vfs_ceph_new</command> performs mapping between Windows + and POSIX Access Control Lists (ACLs). To ensure correct + processing and enforcement of POSIX ACLs, the following Ceph + configuration parameters are automatically applied: </para> <programlisting> <command>client acl type = posix_acl</command> <command>fuse default permissions = false</command> </programlisting> + + <para> + <emphasis role="strong">NOTE</emphasis>: + This is a second implementation of a ceph module which uses libcephfs + low-level APIs (compared to the original + <citerefentry><refentrytitle>vfs_ceph</refentrytitle> + <manvolnum>8</manvolnum></citerefentry> module which uses path-based + APIs). Using the low-level API allows more optimized and fine-grained + access to the Ceph storage layer. + </para> </refsect1> <refsect1> <title>CONFIGURATION</title> <para> - <command>vfs_ceph</command> requires that the underlying share - path is a Ceph filesystem. + <command>vfs_ceph_new</command> requires that the underlying + share path is a Ceph filesystem. </para> <programlisting> <smbconfsection name="[share]"/> - <smbconfoption name="vfs objects">ceph</smbconfoption> + <smbconfoption name="vfs objects">ceph_new</smbconfoption> <smbconfoption name="path">/non-mounted/cephfs/path</smbconfoption> <smbconfoption name="kernel share modes">no</smbconfoption> </programlisting> <para> - Since <command>vfs_ceph</command> does not require a filesystem - mount, the share <command>path</command> is treated differently: - it is interpreted as an absolute path within the Ceph filesystem - on the attached Ceph cluster. + Since <command>vfs_ceph_new</command> does not require a + filesystem mount, the share <command>path</command> is treated + differently: it is interpreted as an absolute path within the + Ceph filesystem on the attached Ceph cluster. In a ctdb cluster environment where ctdb manages Samba, <command>CTDB_SAMBA_SKIP_SHARE_CHECK=yes</command> must be configured to disable local share path checks, otherwise ctdb @@ -101,20 +111,20 @@ <variablelist> <varlistentry> - <term>ceph:config_file = path</term> + <term>ceph_new:config_file = path</term> <listitem> <para> Allows one to define a ceph configfile to use. Empty by default. </para> <para> - Example: ceph:config_file = + Example: ceph_new:config_file = /etc/ceph/ceph.conf </para> </listitem> </varlistentry> <varlistentry> - <term>ceph:user_id = name</term> + <term>ceph_new:user_id = name</term> <listitem> <para> Allows one to explicitly set the client ID used for the @@ -122,22 +132,22 @@ client default). </para> <para> - Example: ceph:user_id = samba + Example: ceph_new:user_id = samba </para> </listitem> </varlistentry> <varlistentry> - <term>ceph:filesystem = fs_name</term> + <term>ceph_new:filesystem = fs_name</term> <listitem> <para> Allows one to explicitly select the CephFS file system to use when the Ceph cluster supports more than one - file system. Empty by default (use the default file system - of the Ceph cluster). + file system. Empty by default (use the default file + system of the Ceph cluster). </para> <para> - Example: ceph:filesystem = myfs2 + Example: ceph_new:filesystem = myfs2 </para> </listitem> </varlistentry> diff --git a/docs-xml/wscript_build b/docs-xml/wscript_build index 434afacaf1e..967e18a6596 100644 --- a/docs-xml/wscript_build +++ b/docs-xml/wscript_build @@ -78,6 +78,7 @@ vfs_module_manpages = ['vfs_acl_tdb', 'vfs_cap', 'vfs_catia', 'vfs_ceph', + 'vfs_ceph_new', 'vfs_ceph_snapshots', 'vfs_commit', 'vfs_crossrename', diff --git a/source3/modules/vfs_ceph_new.c b/source3/modules/vfs_ceph_new.c new file mode 100644 index 00000000000..25e78444fb5 --- /dev/null +++ b/source3/modules/vfs_ceph_new.c @@ -0,0 +1,3100 @@ +/* + Unix SMB/CIFS implementation. + Wrap disk only vfs functions to sidestep dodgy compilers. + Copyright (C) Tim Potter 1998 + Copyright (C) Jeremy Allison 2007 + Copyright (C) Brian Chrisman 2011 <bchris...@gmail.com> + Copyright (C) Richard Sharpe 2011 <realrichardsha...@gmail.com> + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see <http://www.gnu.org/licenses/>. +*/ + +/* + * This VFS only works with the libcephfs.so user-space client. It is not needed + * if you are using the kernel client or the FUSE client. + * + * Add the following smb.conf parameter to each share that will be hosted on + * Ceph: + * + * vfs objects = [any others you need go here] ceph_new + */ + +#include "includes.h" +#include "smbd/smbd.h" +#include "system/filesys.h" +#include <dirent.h> +#include <sys/statvfs.h> +#include "cephfs/libcephfs.h" +#include "smbprofile.h" +#include "modules/posixacl_xattr.h" +#include "lib/util/tevent_unix.h" + +#undef DBGC_CLASS +#define DBGC_CLASS DBGC_VFS + +#ifndef LIBCEPHFS_VERSION +#define LIBCEPHFS_VERSION(maj, min, extra) ((maj << 16) + (min << 8) + extra) +#define LIBCEPHFS_VERSION_CODE LIBCEPHFS_VERSION(0, 0, 0) +#endif + +/* + * Use %llu whenever we have a 64bit unsigned int, and cast to (long long + * unsigned) + */ +#define llu(_var) ((long long unsigned)_var) + +/* + * Note, libcephfs's return code model is to return -errno. Thus we have to + * convert to what Samba expects: set errno to non-negative value and return -1. + * + * Using convenience helper functions to avoid non-hygienic macro. + */ +static int status_code(int ret) +{ + if (ret < 0) { + errno = -ret; + return -1; + } + return ret; +} + +static ssize_t lstatus_code(intmax_t ret) +{ + if (ret < 0) { + errno = -((int)ret); + return -1; + } + return (ssize_t)ret; +} + +/* + * Track unique connections, as virtual mounts, to cephfs file systems. + * Individual mount-entries will be set on the handle->data attribute, but + * the mounts themselves will be shared so as not to spawn extra mounts + * to the same cephfs. + * + * Individual mounts are IDed by a 'cookie' value that is a string built + * from identifying parameters found in smb.conf. + */ + +static struct cephmount_cached { + char *cookie; + uint32_t count; + struct ceph_mount_info *mount; + struct cephmount_cached *next, *prev; + uint64_t fd_index; +} *cephmount_cached; + +static int cephmount_cache_add(const char *cookie, + struct ceph_mount_info *mount, + struct cephmount_cached **out_entry) +{ + struct cephmount_cached *entry = NULL; + + entry = talloc_zero(NULL, struct cephmount_cached); + if (entry == NULL) { + errno = ENOMEM; + return -1; + } + + entry->cookie = talloc_strdup(entry, cookie); + if (entry->cookie == NULL) { + talloc_free(entry); + errno = ENOMEM; + return -1; + } + + entry->mount = mount; + entry->count = 1; + + DBG_DEBUG("[CEPH] adding mount cache entry for %s\n", entry->cookie); + DLIST_ADD(cephmount_cached, entry); + + *out_entry = entry; + return 0; +} + +static struct cephmount_cached *cephmount_cache_update(const char *cookie) +{ + struct cephmount_cached *entry = NULL; + + for (entry = cephmount_cached; entry; entry = entry->next) { + if (strcmp(entry->cookie, cookie) == 0) { + entry->count++; + DBG_DEBUG("[CEPH] updated mount cache: count is [%" + PRIu32 "]\n", entry->count); + return entry; + } + } + + errno = ENOENT; + return NULL; +} + +static int cephmount_cache_remove(struct cephmount_cached *entry) +{ + if (--entry->count) { + DBG_DEBUG("[CEPH] updated mount cache: count is [%" + PRIu32 "]\n", entry->count); + return entry->count; + } + + DBG_DEBUG("[CEPH] removing mount cache entry for %s\n", entry->cookie); + DLIST_REMOVE(cephmount_cached, entry); + talloc_free(entry); + return 0; +} + +static char *cephmount_get_cookie(TALLOC_CTX * mem_ctx, const int snum) +{ + const char *conf_file = + lp_parm_const_string(snum, "ceph_new", "config_file", "."); + const char *user_id = + lp_parm_const_string(snum, "ceph_new", "user_id", ""); + const char *fsname = + lp_parm_const_string(snum, "ceph_new", "filesystem", ""); + return talloc_asprintf(mem_ctx, "(%s/%s/%s)", conf_file, user_id, + fsname); +} + +static struct ceph_mount_info *cephmount_mount_fs(const int snum) +{ + int ret; + char buf[256]; + struct ceph_mount_info *mnt = NULL; + /* if config_file and/or user_id are NULL, ceph will use defaults */ + const char *conf_file = + lp_parm_const_string(snum, "ceph_new", "config_file", NULL); + const char *user_id = + lp_parm_const_string(snum, "ceph_new", "user_id", NULL); + const char *fsname = + lp_parm_const_string(snum, "ceph_new", "filesystem", NULL); + + DBG_DEBUG("[CEPH] calling: ceph_create\n"); + ret = ceph_create(&mnt, user_id); + if (ret) { + errno = -ret; + return NULL; + } + + DBG_DEBUG("[CEPH] calling: ceph_conf_read_file with %s\n", + (conf_file == NULL ? "default path" : conf_file)); + ret = ceph_conf_read_file(mnt, conf_file); + if (ret) { + goto err_cm_release; + } + + DBG_DEBUG("[CEPH] calling: ceph_conf_get\n"); + ret = ceph_conf_get(mnt, "log file", buf, sizeof(buf)); + if (ret < 0) { + goto err_cm_release; + } + + /* libcephfs disables POSIX ACL support by default, enable it... */ + ret = ceph_conf_set(mnt, "client_acl_type", "posix_acl"); + if (ret < 0) { + goto err_cm_release; + } + /* tell libcephfs to perform local permission checks */ + ret = ceph_conf_set(mnt, "fuse_default_permissions", "false"); + if (ret < 0) { + goto err_cm_release; + } + /* + * select a cephfs file system to use: + * In ceph, multiple file system support has been stable since + * 'pacific'. Permit different shares to access different file systems. + */ + if (fsname != NULL) { + ret = ceph_select_filesystem(mnt, fsname); + if (ret < 0) { + goto err_cm_release; + } + } + + DBG_DEBUG("[CEPH] calling: ceph_mount\n"); + ret = ceph_mount(mnt, NULL); + if (ret >= 0) { + goto cm_done; + } + + err_cm_release: + ceph_release(mnt); + mnt = NULL; + DBG_DEBUG("[CEPH] Error mounting fs: %s\n", strerror(-ret)); + cm_done: + /* + * Handle the error correctly. Ceph returns -errno. + */ + if (ret) { + errno = -ret; + } + return mnt; +} + +/* Check for NULL pointer parameters in vfs_ceph_* functions */ + +/* We don't want to have NULL function pointers lying around. Someone + is sure to try and execute them. These stubs are used to prevent + this possibility. */ + +static int vfs_ceph_connect(struct vfs_handle_struct *handle, + const char *service, const char *user) +{ + int ret = 0; + struct cephmount_cached *entry = NULL; + struct ceph_mount_info *cmount = NULL; + int snum = SNUM(handle->conn); + char *cookie = cephmount_get_cookie(handle, snum); + if (cookie == NULL) { + return -1; + } + + entry = cephmount_cache_update(cookie); + if (entry != NULL) { + goto connect_ok; + } + + cmount = cephmount_mount_fs(snum); + if (cmount == NULL) { + ret = -1; + goto connect_fail; + } + ret = cephmount_cache_add(cookie, cmount, &entry); + if (ret != 0) { + goto connect_fail; + } + +connect_ok: + handle->data = entry; + DBG_WARNING("Connection established with the server: %s\n", cookie); + + /* + * Unless we have an async implementation of getxattrat turn this off. + */ + lp_do_parameter(SNUM(handle->conn), "smbd async dosmode", "false"); +connect_fail: + talloc_free(cookie); + return ret; +} + +static struct ceph_mount_info *cmount_of(const struct vfs_handle_struct *handle) +{ + const struct cephmount_cached *entry = handle->data; + + return entry->mount; +} + +static void vfs_ceph_disconnect(struct vfs_handle_struct *handle) +{ + struct ceph_mount_info *cmount = cmount_of(handle); + int ret = 0; + + ret = cephmount_cache_remove(handle->data); + if (ret > 0) { + DBG_DEBUG("[CEPH] mount cache entry still in use\n"); + return; + } + + ret = ceph_unmount(cmount); + if (ret < 0) { + DBG_ERR("[CEPH] failed to unmount: %s\n", strerror(-ret)); + } + + ret = ceph_release(cmount); + if (ret < 0) { + DBG_ERR("[CEPH] failed to release: %s\n", strerror(-ret)); + } + handle->data = NULL; +} + +/* Ceph user-credentials */ +static struct UserPerm *vfs_ceph_userperm_new( + const struct vfs_handle_struct *handle) -- Samba Shared Repository