This is exactly the issue referenced in the LKML thread, and the one that Jeffrey Altman analyzed, so it seems the issue has not yet been fixed. Unfortunately, I don't think the patches dealing with credentials will fix this because dentry_open calls security_file_open regardless of credentials, and security_file_open expects process context.
I believe the way to fix this is that OpenAFS needs to open this file in a way that does not go through the LSM, because there is no process context here. From: Jonathan Billings <[email protected]> Sent: Tuesday, March 9, 2021 5:46 AM To: [email protected] Cc: Martin Kelly <[email protected]> Subject: [External] Re: [OpenAFS] OpenAFS 1.8.7 on Linux systems running Crowdstrike falcon-sensor Basically, this is what I'm running: # git describe --abbrev=4 openafs-stable-1_8_x openafs-stable-1_8_7-109-gb7bdd # rxdebug localhost 7001 -version Trying 127.0.0.1 [127.0.0.1] <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1&d=DwQFaQ&c=08 AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=wsraf2W9gLu3Kwvcq4lteczxlOggIsFy oEwCrqmJ1gE&m=YBSO0k1d9jL782APR0Fvy6pws5z_ZZ2yXwqqX2r4Tzk&s=za_1x9waWw1p_UYf sz-uz1ZXNQCsoJwjTXFBN_tLnSc&e=> (port 7001): AFS version: OpenAFS 1.8.7-109-gb7bdd 2021-03-08 mockbuild@ With this kmod and the latest RHEL7 kernel, this is the kernel backtrace: [ 170.503804] BUG: unable to handle kernel NULL pointer dereference at 0000000000000004 [ 170.506260] IP: [<ffffffff8238a6ec>] _raw_spin_lock+0xc/0x30 [ 170.507074] PGD 0 [ 170.507824] Oops: 0002 [#1] SMP [ 170.508596] Modules linked in: cts rpcsec_gss_krb5 nfsv4 dns_resolver nfs lockd grace falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_11308(E) nf_log_ipv4 nf_log_common xt_LOG ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock cachefiles fscache sb_edac ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper [ 170.512708] ablk_helper cryptd pcspkr joydev sg vmw_vmci i2c_piix4 parport_pc parport binfmt_misc openafs(POE) auth_rpcgss sunrpc ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptsas ata_piix scsi_transport_sas nfit drm crct10dif_pclmul mptscsih crct10dif_common libata serio_raw crc32c_intel libnvdimm mptbase vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse [ 170.516344] CPU: 6 PID: 2782 Comm: llvmpipe-8 Kdump: loaded Tainted: P OE ------------ 3.10.0-1160.15.2.el7.x86_64 #1 [ 170.517353] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018 [ 170.518320] task: ffff9c0127fe0000 ti: ffff9c0556be0000 task.ti: ffff9c0556be0000 [ 170.519315] RIP: 0010:[<ffffffff8238a6ec>] [<ffffffff8238a6ec>] _raw_spin_lock+0xc/0x30 [ 170.520311] RSP: 0018:ffff9c0556be3710 EFLAGS: 00010246 [ 170.521317] RAX: 0000000000000000 RBX: 0000000000000004 RCX: 0000000000000000 [ 170.522310] RDX: 0000000000000001 RSI: ffff9c013eb85d80 RDI: 0000000000000004 [ 170.523301] RBP: ffff9c0556be3730 R08: 000000000001f060 R09: ffff9c013efc67a0 [ 170.524292] R10: ffff9c0073e7aa80 R11: ffff9c013c7ae440 R12: ffff9c0556be3740 [ 170.525298] R13: 0000000000000000 R14: 0000000000000000 R15: ffff9c050c867810 [ 170.526303] FS: 00007f5b1dd69700(0000) GS:ffff9c0569300000(0000) knlGS:0000000000000000 [ 170.527315] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 170.528324] CR2: 0000000000000004 CR3: 00000004da610000 CR4: 00000000001607e0 [ 170.529415] Call Trace: [ 170.530459] [<ffffffffc0af7d9d>] ? 0xffffffffc0af7d9c [ 170.531482] [<ffffffffc0b56ab8>] _ZdlPv+0x48088/0x484f0 [falcon_lsm_serviceable] [ 170.532523] [<ffffffffc0b5725b>] cshook_security_file_permission+0x9b/0x8c0 [falcon_lsm_serviceable] [ 170.533579] [<ffffffffc0b5ea9e>] cshook_security_file_open+0xe/0x10 [falcon_lsm_serviceable] [ 170.534637] [<ffffffffc0ae4873>] pinnedhook_security_file_open+0x43/0x70 [falcon_lsm_pinned_11308] [ 170.535719] [<ffffffff81f08f12>] security_file_open+0x22/0x70 [ 170.536789] [<ffffffff81e4b339>] do_dentry_open+0xc9/0x2d0 [ 170.537850] [<ffffffff81e4b5da>] vfs_open+0x5a/0xb0 [ 170.538876] [<ffffffff81e4b679>] dentry_open+0x49/0xc0 [ 170.540008] [<ffffffffc06cdb0f>] afs_linux_raw_open+0x8f/0x140 [openafs] [ 170.541094] [<ffffffffc06cdc74>] osi_UFSOpen+0xb4/0x1a0 [openafs] [ 170.542140] [<ffffffffc06567f1>] DRead+0x341/0x520 [openafs] [ 170.543225] [<ffffffffc0669a7e>] FindItem+0x5e/0x1f0 [openafs] [ 170.544306] [<ffffffffc066a183>] afs_dir_Delete+0x33/0x1a0 [openafs] [ 170.545409] [<ffffffffc06a6d17>] ? RXAFS_RemoveFile+0x67/0x120 [openafs] [ 170.546510] [<ffffffffc068e1ad>] ? afs_LocalHero+0x11d/0x200 [openafs] [ 170.547616] [<ffffffffc069abc9>] afsremove+0x489/0x7d0 [openafs] [ 170.548705] [<ffffffffc069beca>] afs_remunlink+0x24a/0x2f0 [openafs] [ 170.549798] [<ffffffffc068489d>] afs_InactiveVCache+0x7d/0x80 [openafs] [ 170.550891] [<ffffffffc06d28e8>] afs_dentry_iput+0x58/0x140 [openafs] [ 170.551931] [<ffffffff81e670ca>] __dentry_kill+0x13a/0x1d0 [ 170.553003] [<ffffffff81e67785>] dput+0xb5/0x1a0 [ 170.554060] [<ffffffff81e500cd>] __fput+0x18d/0x230 [ 170.555118] [<ffffffff81e5025e>] ____fput+0xe/0x10 [ 170.556169] [<ffffffff81cc294b>] task_work_run+0xbb/0xe0 [ 170.557191] [<ffffffff81ca1894>] do_exit+0x2d4/0xa30 [ 170.558162] [<ffffffff81d1317f>] ? futex_wait+0x11f/0x280 [ 170.559126] [<ffffffff81c63d03>] ? x2apic_send_IPI_mask+0x13/0x20 [ 170.560048] [<ffffffff81ca206f>] do_group_exit+0x3f/0xa0 [ 170.560939] [<ffffffff81cb323e>] get_signal_to_deliver+0x1ce/0x5e0 [ 170.561802] [<ffffffff81c2c527>] do_signal+0x57/0x6f0 [ 170.562626] [<ffffffff81d14f46>] ? do_futex+0x106/0x5a0 [ 170.563420] [<ffffffff81c2cc32>] do_notify_resume+0x72/0xc0 [ 170.564184] [<ffffffff823952ef>] int_signal+0x12/0x17 [ 170.564878] Code: 5d c3 0f 1f 44 00 00 85 d2 74 e4 0f 1f 40 00 eb ed 66 0f 1f 44 00 00 b8 01 00 00 00 5d c3 90 0f 1f 44 00 00 31 c0 ba 01 00 00 00 <f0> 0f b1 17 85 c0 75 01 c3 55 89 c6 48 89 e5 e8 ea 18 ff ff 5d [ 170.566509] RIP [<ffffffff8238a6ec>] _raw_spin_lock+0xc/0x30 [ 170.567274] RSP <ffff9c0556be3710> [ 170.568018] CR2: 0000000000000004 On Mon, Mar 8, 2021 at 8:22 PM Jeffrey E Altman <[email protected] <mailto:[email protected]> > wrote: On 3/8/2021 7:20 PM, Benjamin Kaduk ([email protected] <mailto:[email protected]> ) wrote: > On Mon, Mar 08, 2021 at 07:35:19PM +0000, Martin Kelly wrote: >> Below is the LKML LSM thread regarding this. Please let me know if you have any other questions: >> >> https://www.spinics.net/lists/linux-security-module/msg39081.html [spinics.net] <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.spinics.net_lists_ linux-2Dsecurity-2Dmodule_msg39081.html&d=DwMFaQ&c=08AGY6txKsvMOP6lYkHQpPMRA 1U6kqhAwGa8-0QCg3M&r=wsraf2W9gLu3Kwvcq4lteczxlOggIsFyoEwCrqmJ1gE&m=YBSO0k1d9 jL782APR0Fvy6pws5z_ZZ2yXwqqX2r4Tzk&s=1KnxijRcsGp8k_dt6qTga6LJh5Nu53ja1ZuNA6x 9AQc&e=> >> https://www.spinics.net/lists/linux-security-module/msg39083.html [spinics.net] <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.spinics.net_lists_ linux-2Dsecurity-2Dmodule_msg39083.html&d=DwMFaQ&c=08AGY6txKsvMOP6lYkHQpPMRA 1U6kqhAwGa8-0QCg3M&r=wsraf2W9gLu3Kwvcq4lteczxlOggIsFyoEwCrqmJ1gE&m=YBSO0k1d9 jL782APR0Fvy6pws5z_ZZ2yXwqqX2r4Tzk&s=DWivoWdmUWYhc2fPQ040MVvE8ryH7ncDdVo9vQV V1A8&e=> > > Thanks for spotting this thread and the quick follow-up. This is the same thread that Yadav discussed with the openafs-release team on 11 Dec 2020. > I suspect that the changes at https://gerrit.openafs.org/#/c/13751/ [gerrit.openafs.org] <https://urldefense.proofpoint.com/v2/url?u=https-3A__gerrit.openafs.org_-23 _c_13751_&d=DwMFaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=wsraf2W9g Lu3Kwvcq4lteczxlOggIsFyoEwCrqmJ1gE&m=YBSO0k1d9jL782APR0Fvy6pws5z_ZZ2yXwqqX2r 4Tzk&s=IzzLrxc0PFa4NClPYSdo20jhIRaCdMK_JWp58-1aroQ&e=> are > going to be relevant in this space, but without seeing the stack trace of > the crash in question it's hard to be sure. Can you speak to whether this > is touching the "right" part of the code with respect to the crashes you > were investigating? The suggested change was cherry-picked to openafs-stable-1_8_x as https://gerrit.openafs.org/14082 [gerrit.openafs.org] <https://urldefense.proofpoint.com/v2/url?u=https-3A__gerrit.openafs.org_140 82&d=DwMFaQ&c=08AGY6txKsvMOP6lYkHQpPMRA1U6kqhAwGa8-0QCg3M&r=wsraf2W9gLu3Kwvc q4lteczxlOggIsFyoEwCrqmJ1gE&m=YBSO0k1d9jL782APR0Fvy6pws5z_ZZ2yXwqqX2r4Tzk&s= lHcFqdLdUXfADi3w5wJ7Ig4CBNsRsa9dqLCuSp2rJ10&e=> and merged as ee578e92d9f810d93659a9805d0c12084fe2bb95. As Jonathan wrote to IRC OpenAFS: > (4:53:15 PM) billings: I built openafs from the latest commit in > 1_8_x and crowdstrike still panics, so it doesnt look like any > merged commits there fix my issue. Martin's e-mail describes the call pattern: > - A process exits, calling task_exit(). I think Martin meant do_exit(). > - exit_fs() is called, setting current->fs = NULL. task_struct field struct fs_struct *fs; > - Next, exit_task_work() is called, exit_task_work() calls task_work_run() which flushes any pending works. > which calls fput(). which must have been called by a pending work. > - In response to the fput(), the filesystem opens a file disk cache > to update some metadata, calling dentry_open(). dentry_open() in turn will trigger a call to any configured LSM. If task_struct.fs is NULL, Kaboom!!! Jeffrey Altman -- Jonathan Billings <[email protected] <mailto:[email protected]> > (he/his) College of Engineering - CAEN - Linux Support
smime.p7s
Description: S/MIME cryptographic signature
