sftp activity logging / Fluent Bit
2023-08-31 (Thu) 16:29 UTC -0500 :
> I am setting an openbsd 7.3 stable system to serve files via ssh's sftp
> subsystem.
>
> Does openssh have a native way to audit what files were
> downloaded/uploaded with user/timestamp information?
>
> If not, are there any recommendations?
(I think you did not ask suggestions for the log parsing part but here is one
anyway)
This is what I have come up with Fluent Bit and OpenSSH portable on other OS:
---
[PARSER]
Name my_sftp_receiver
Format regex
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L%z
Regex ^\<(?[0-9]{1,5})\>(?.+) (?[^ ]*) (?[^ ]+):
close "(?.+)" bytes read (?.*) written (?.*)$
Types filename:string dlsize:int ulsize:int
---
My use case is to get triggered after succesful upload, so rest of the
Fluent Bit pipeline is set to do output action (HTTP POST) when
variable "ulsize" is greater than 0 and "filename" contains directory
path of special interest.
Hope this helps.
--
Mikko
Re: OpenBSD Wireguard implementation not copying ToS from inner to outer WG header
> On 29 Sep 2023, at 00:09, Sonic wrote: > > > Hopefully not as dumb of a question as I suspect it might be. > Does the generic... > = > match out on $ext_if inet proto tcp from ($ext_if) set prio (3, 7) > match in on $ext_if inet proto tcp to ($ext_if) set prio (3, 7) > = > ...take advantage of this patch when using wireguard or does the match need > to be applied to the wgx interface? Hi. I have a generic ‘match’ which does not specify interfaces or direction, and which sets prio, tos, and queue on all the important things. So yes, like your example but without direction or interface. And of course use specific interface dependant rules to pass or drop things as needed. Then as packets enter the inside Interface, the prio will be set. And when the packet gets to the wg interface, the prio is copied (with this patch), and finally the encrypted packet gets pulled out of the outside interface’s queue in prio order (there is more going on here, but you can assume prio order). You will not currently be able to get the encrypted packet into any queue other than the ‘default’ queue as the packet tos flags and queue state labels are lost during encryption. But now that the prio is copied with this patch, at least your ACKs etc will be pulled from the one default queue first. As this will be facing your internet connection (smaller than your LAN), you should still see a benefit even though TOS is not yet copied. The longer the pipe / greater the BDP, the more noticeable it will be. Hope this helps, Andy.
Re: Force Probing Intel GPU
On Thu, Sep 28, 2023 at 11:23:03AM +, Gökhan Özdemir wrote: > Hello, > > I have an Intel Arc A770 which requires force probe kernel parameter > (i915.force_probe=56a0) on linux 6.1. Since current drm stack on OpenBSD is > 6.1, I am looking for a way to find OpenBSD equivalent of force probe. > > I am running -current and manually installed inteldrm firmware via fw_update Matching isn't the whole problem. The dg2 cards take paths that are not fully implemented: https://marc.info/?l=openbsd-tech=2=1=dg2=b
Re: sftp activity logging?
Copy.. ;) Stuart Longland VK4MSL wrote: > On 28/9/23 18:29, Daniele B. wrote: > > Any more update? > > > > Beside FTPD having instead vsftpd like the ftp man one > > sftp ≠ ftp/ftps > > FTP is the original File Transfer Protocol, introduced in 1971 and > described in RFC-959. > > FTPS is FTP run over TLS. > > SFTP is a file transfer subsystem run over SSH.
Re: sftp activity logging?
On 28/9/23 18:29, Daniele B. wrote: Any more update? Beside FTPD having instead vsftpd like the ftp man one sftp ≠ ftp/ftps FTP is the original File Transfer Protocol, introduced in 1971 and described in RFC-959. FTPS is FTP run over TLS. SFTP is a file transfer subsystem run over SSH. -- Stuart Longland (aka Redhatter, VK4MSL) I haven't lost my mind... ...it's backed up on a tape somewhere.
mount invalid argument
Hello, wiz$ mount -t ffs /dev/sd1i /mnt/stick mount_ffs: /dev/sd1i on /mnt/stick: Invalid argument Maybe "Invalid argument" here could be expanded to "wrong -t argument", "wrong file system type argument"? We are continuing to guess in this way.. Hope it is helpful ;) -- Daniele Bonini
Re: OpenBSD 7.2 fw stack trace on Dell R740
On 26.9.2023. 9:24, Joerg Streckfuss wrote: > > Hi Stuart, > > Am 25.09.23 um 19:08 schrieb Stuart Henderson: >> That might possibly be the one fixed by 7.2 errata 008, so if you don't >> already have that you at least want to syspatch. > > That was my guess as well. However, the systems were patched up to 7.2 > errata-016. I applied the remaining patches. So far the systems are > running stable. Are there any changes between the 7.2 and 7.3 releases > that could indicate a bug? > > Many regards, > > Joerg Hi, If you can go to snapshot or wait for 7.4. Thanks to dlg@ my pfsync firewalls are rock stable after this commit https://marc.info/?l=openbsd-cvs=168861927203498=2
Force Probing Intel GPU
Hello,
I have an Intel Arc A770 which requires force probe kernel parameter
(i915.force_probe=56a0) on linux 6.1. Since current drm stack on OpenBSD is
6.1, I am looking for a way to find OpenBSD equivalent of force probe.
I am running -current and manually installed inteldrm firmware via fw_update
Methods i have tried so far:
Index: autoconf.h
===
RCS file: /cvs/src/sys/dev/pci/drm/include/generated/autoconf.h,v
retrieving revision 1.10
diff -u -p -r1.10 autoconf.h
--- autoconf.h 3 Oct 2022 10:07:01 - 1.10
+++ autoconf.h 28 Sep 2023 12:14:45 -
@@ -27,7 +27,7 @@
#define CONFIG_DRM_I915_STOP_TIMEOUT 100 /* ms */
#define CONFIG_DRM_I915_FENCE_TIMEOUT 1 /* ms */
#define CONFIG_DRM_I915_USERFAULT_AUTOSUSPEND 250 /* ms */
-#define CONFIG_DRM_I915_FORCE_PROBE ""
+#define CONFIG_DRM_I915_FORCE_PROBE "*,56a0"
#ifdef __HAVE_ACPI
#include "acpi.h"
also tried different combinations like "*" , "56a0", "56a0,56a1,56a5,56a6" and
built kernel but no luck.
modified functions where it checks if device is forced or blocked.
Index: i915_pci.c
===
RCS file: /cvs/src/sys/dev/pci/drm/i915/i915_pci.c,v
retrieving revision 1.19
diff -u -p -r1.19 i915_pci.c
--- i915_pci.c 15 Jun 2023 02:46:22 - 1.19
+++ i915_pci.c 28 Sep 2023 12:13:37 -
@@ -1293,12 +1293,12 @@ static bool device_id_in_list(u16 device
static bool id_forced(u16 device_id)
{
- return device_id_in_list(device_id, i915_modparams.force_probe, false);
+ return true;
}
static bool id_blocked(u16 device_id)
{
- return device_id_in_list(device_id, i915_modparams.force_probe, true);
+ return false;
}
bool i915_pci_resource_valid(struct pci_dev *pdev, int bar)
still, no luck.
As last resort, removed require_force_probe definition on dg2_info
Index: i915_pci.c
===
RCS file: /cvs/src/sys/dev/pci/drm/i915/i915_pci.c,v
retrieving revision 1.19
diff -u -p -r1.19 i915_pci.c
--- i915_pci.c 15 Jun 2023 02:46:22 - 1.19
+++ i915_pci.c 28 Sep 2023 12:22:00 -
@@ -1078,7 +1078,6 @@ static const struct intel_device_info dg
XE_LPD_FEATURES,
.__runtime.cpu_transcoder_mask = BIT(TRANSCODER_A) | BIT(TRANSCODER_B) |
BIT(TRANSCODER_C) | BIT(TRANSCODER_D),
- .require_force_probe = 1,
};
static const struct intel_device_info ats_m_info = {
with this change, gpu gets registered as inteldrm but falls back to efifb with
error
pci3 at ppb2 bus 3
inteldrm0 at pci3 dev 0 function 0 "Intel Arc A770" rev 0x08
drm0 at inteldrm0
inteldrm0: msi, DG2, gen 12
xehp_load_dss_mask: stub
xehp_load_dss_mask: stub
intel_slicemask_from_xehp_dssmask: stub
intel_slicemask_from_xehp_dssmask: stub
init_stolen_lmem: stub
drm:pid0:intel_memory_regions_hw_probe *ERROR* [drm] *ERROR* Failed to setup
region(-78) type=3
Device initialization failed (-78)
efifb0 at mainbus0: 1920x1080, 32bpp
dmesgs for both unmodified kernel and modified kernel are in attachment
Thanks for any help!
- Gökhan
real mem = 34122215424 (32541MB)
avail mem = 33068335104 (31536MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.4 @ 0x79d2a000 (118 entries)
bios0: vendor American Megatrends Inc. version "2611" date 08/11/2023
bios0: ASUS PRIME Z690-P D4
efi0 at bios0: UEFI 2.8
efi0: American Megatrends rev 0x5001b
acpi0 at bios0: ACPI 6.4
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP MCFG FIDT SSDT SSDT SSDT SSDT HPET APIC MCFG SSDT NHLT
LPIT SSDT SSDT DBGP DBG2 SSDT DMAR FPDT SSDT SSDT SSDT BGRT WPBT TPM2 PHAT WSMT
acpi0: wakeup devices PEG1(S4) PEGP(S4) PEGP(S4) PEG0(S4) PEGP(S4) RP09(S4)
PXSX(S4) RP10(S4) PXSX(S4) RP11(S4) PXSX(S4) RP12(S4) PXSX(S4) RP13(S4)
PXSX(S4) RP14(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimcfg0 at acpi0
acpimcfg0: addr 0xc000, bus 0-255
acpihpet0 at acpi0: 1920 Hz
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: 12th Gen Intel(R) Core(TM) i5-12600KF, 5002.58 MHz, 06-97-02, patch
002e
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PT,SHA,UMIP,PKU,WAITPKG,PKS,MD_CLEAR,IBT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,IBRS_ALL,SKIP_L1DFL,MDS_NO,IF_PSCHANGE,TAA_NO,MISC_PKG_CT,ENERGY_FILT,DOITM,SBDR_SSDP_N,FBSDP_NO,PSDP_NO,RRSBA,OVERCLOCK,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu0: 48KB 64b/line 12-way D-cache, 32KB 64b/line 8-way I-cache, 1MB 64b/line
10-way L2 cache, 20MB 64b/line 10-way L3 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var
Re: OpenBSD Wireguard implementation not copying ToS from inner to outer WG header
Hopefully not as dumb of a question as I suspect it might be. Does the generic... = match out on $ext_if inet proto tcp from ($ext_if) set prio (3, 7) match in on $ext_if inet proto tcp to ($ext_if) set prio (3, 7) = ...take advantage of this patch when using wireguard or does the match need to be applied to the wgx interface?
Re: I nuked my filesystem
Just want to close the loop. I ended up finding a backup of my mbox on a different machine with the disklabel in the insecurity update (thanks for the tip, Nick). I installed it on the disk and had access to everything except root! Out of curiosity, I generated a new disklabel for that same disk with disklabel -A and predictably, the offsets and sizes were all correct in that one too. Thank you all for your wisdom and reorienting my week around proper backups. Isaac > To be clear: having overwritten the starting 5GB of sd1c, > but not of the underlying sd0c, you still have the CRYPTO volume on > sd0c which you can attach with bioctl, right? If my ssd is sd0 and my bio(4) managed volume is sd1, then I dd'd over sd1 (all of sd1a, some of swap). On Wed, Sep 27, 2023 at 6:16 PM Stuart Henderson wrote: > > On 2023-09-27, Isaac Meerleo wrote: > > Thank you all for your replies. I needed to step away from my > > computer last night so I apologize for the terse email. > > sd0 is my physical hard disk with full disk encryption. I wrote a 5gb > > iso over the beginning of sd1c (my softraid volume). I rebooted. I > > installed openbsd on a separate drive, booted it, mounted the > > encrypted volume, and hopelessly ran scan_ffs. > > > > After reading Stuart's advice, I will attempt to reinstall the > > disklabel from scratch. I installed 7.2 on my second disk; the version > > I used to do the initial install. > > When I initially set up my disk, I used the auto partition defaults > > which left me with a 300gb /home directory. Later, I expanded the > > /home partition across the rest of the disk. When I recreate the > > disklabel, should I let it auto partition and then grow the file > > system like before? Or should I just set home to fill the rest of the > > disk? > > Set it to fill the rest of the disk - don't run growfs on it again. > > I would be wanting to mount filesystems read-only and try to copy data > off to another disk, then reinstall on the disk with damaged disklabel > and copy back, rather than relying on the existing filesystems long-term. > >
Re: sftp activity logging?
On 2023-09-28, Nick Holland wrote: > On 8/31/23 17:29, myml...@gmx.com wrote: >> Hi All, >> >> I am setting an openbsd 7.3 stable system to serve files via ssh's sftp >> subsystem. >> >> Does openssh have a native way to audit what files were >> downloaded/uploaded with user/timestamp information? >> >> If not, are there any recommendations? >> >> Thanks in advance. >> > > Try this, perhaps? > > man sftp-server, > options of interest may include -f, -l. > > You will probably have to have a /dev/log inside the chroot, which > also means the "nodev" option is not your friend. Files accessed: yes. Files opened are logged with flags, so you can distinguish between read/write. Files closed, renamed, removed are logged. User: yes, but you'll need to match lines together based on PID, look for the 'session opened/closed' lines which have username/IP. In a long running connection authlog may have rotated between the connection and transfer. You don't need a /dev/log socket to do this on OpenBSD, we have the mechanism described in sendsyslog(2). For sftp chroot with the internal-sftp implementation you can do "ForceCommand internal-sftp -l INFO".
Re: sftp activity logging?
On 8/31/23 17:29, myml...@gmx.com wrote: Hi All, I am setting an openbsd 7.3 stable system to serve files via ssh's sftp subsystem. Does openssh have a native way to audit what files were downloaded/uploaded with user/timestamp information? If not, are there any recommendations? Thanks in advance. Try this, perhaps? man sftp-server, options of interest may include -f, -l. You will probably have to have a /dev/log inside the chroot, which also means the "nodev" option is not your friend. Nick.
Re: sftp activity logging?
Any more update? Beside FTPD having instead vsftpd like the ftp man one: dual_log_enable If enabled, two log files are generated in parallel, going by default to /var/log/xferlog and /var/log/vsftpd.log. The former is a wu-ftpd style transfer log, parseable by standard tools. The latter is vsftpd's own style log. Default: NO log_ftp_protocol When enabled, all FTP requests and responses are logged, providing the option xferlog_std_format is not enabled. Useful for debugging. Default: NO syslog_enable If enabled, then any log output which would have gone to /var/log/vsftpd.log goes to the system log instead. Logging is done under the FTPD facility. Default: NO xferlog_enable If enabled, a log file will be maintained detailling uploads and downloads. By default, this file will be placed at /var/log/vsftpd.log, but this location may be overridden using the configuration setting vsftpd_log_file. Default: NO (but the sample config file enables it) xferlog_std_format If enabled, the transfer log file will be written in standard xferlog format, as used by wu-ftpd. This is useful because you can reuse existing transfer statistics generators. The default format is more readable, however. The default location for this style of log file is /var/log/xferlog, but you may change it with the setting xferlog_file. Default: NO -- Daniele Bonini "myml...@gmx.com" wrote: > Hi All, > > I am setting an openbsd 7.3 stable system to serve files via ssh's > sftp subsystem. > > Does openssh have a native way to audit what files were > downloaded/uploaded with user/timestamp information? > > If not, are there any recommendations? > > Thanks in advance. >
