Re: [PATCH 8/9] pstore/blk: use the normal block device I/O path
On Fri, Oct 16, 2020 at 11:33 PM Christoph Hellwig wrote: > > Stop poking into block layer internals and just open the block device > file an use kernel_read and kernel_write on it. Note that this means > the transformation from name_to_dev_t can't be used anymore, and proper > block device file names must be used. > > Signed-off-by: Christoph Hellwig > --- > fs/pstore/blk.c| 152 + > include/linux/pstore_blk.h | 2 + > init/main.c| 4 + > 3 files changed, 57 insertions(+), 101 deletions(-) > > diff --git a/fs/pstore/blk.c b/fs/pstore/blk.c > index 0430b190a1df2a..bd4eadfc9bd795 100644 > --- a/fs/pstore/blk.c > +++ b/fs/pstore/blk.c > @@ -8,15 +8,13 @@ > > #include > #include > -#include "../../block/blk.h" > #include > #include > -#include > -#include > -#include > #include > +#include > +#include > +#include > #include > -#include > > static long kmsg_size = CONFIG_PSTORE_BLK_KMSG_SIZE; > module_param(kmsg_size, long, 0400); > @@ -88,7 +86,6 @@ MODULE_PARM_DESC(blkdev, "block device for pstore storage"); > * during the register/unregister functions. > */ > static DEFINE_MUTEX(pstore_blk_lock); > -static struct block_device *psblk_bdev; > static struct pstore_zone_info *pstore_zone_info; > > #define check_size(name, alignsize) ({ \ > @@ -185,70 +182,20 @@ void unregister_pstore_device(const struct > pstore_zone_ops *ops) > } > EXPORT_SYMBOL_GPL(unregister_pstore_device); > > +static struct file *psblk_file; > + > static ssize_t psblk_generic_blk_read(char *buf, size_t bytes, loff_t pos) > { > - struct block_device *bdev = psblk_bdev; > - struct file file; > - struct kiocb kiocb; > - struct iov_iter iter; > - struct kvec iov = {.iov_base = buf, .iov_len = bytes}; > - > - if (!bdev) > - return -ENODEV; > - > - memset(, 0, sizeof(struct file)); > - file.f_mapping = bdev->bd_inode->i_mapping; > - file.f_flags = O_DSYNC | __O_SYNC | O_NOATIME; > - file.f_inode = bdev->bd_inode; > - file_ra_state_init(_ra, file.f_mapping); > - > - init_sync_kiocb(, ); > - kiocb.ki_pos = pos; > - iov_iter_kvec(, READ, , 1, bytes); > - > - return generic_file_read_iter(, ); > + return kernel_read(psblk_file, buf, bytes, ); > } > > static ssize_t psblk_generic_blk_write(const char *buf, size_t bytes, > loff_t pos) > { > - struct block_device *bdev = psblk_bdev; > - struct iov_iter iter; > - struct kiocb kiocb; > - struct file file; > - ssize_t ret; > - struct kvec iov = {.iov_base = (void *)buf, .iov_len = bytes}; > - > - if (!bdev) > - return -ENODEV; > - > /* Console/Ftrace backend may handle buffer until flush dirty zones */ > if (in_interrupt() || irqs_disabled()) > return -EBUSY; > - > - memset(, 0, sizeof(struct file)); > - file.f_mapping = bdev->bd_inode->i_mapping; > - file.f_flags = O_DSYNC | __O_SYNC | O_NOATIME; > - file.f_inode = bdev->bd_inode; > - > - init_sync_kiocb(, ); > - kiocb.ki_pos = pos; > - iov_iter_kvec(, WRITE, , 1, bytes); > - > - inode_lock(bdev->bd_inode); > - ret = generic_write_checks(, ); > - if (ret > 0) > - ret = generic_perform_write(, , pos); > - inode_unlock(bdev->bd_inode); > - > - if (likely(ret > 0)) { > - const struct file_operations f_op = {.fsync = blkdev_fsync}; > - > - file.f_op = _op; > - kiocb.ki_pos += ret; > - ret = generic_write_sync(, ret); > - } > - return ret; > + return kernel_write(psblk_file, buf, bytes, ); > } > > static const struct pstore_zone_ops pstore_blk_zone_ops = { > @@ -257,68 +204,71 @@ static const struct pstore_zone_ops pstore_blk_zone_ops > = { > .write = psblk_generic_blk_write, > }; > > -static int __init pstore_blk_init(void) > +static int __init __pstore_blk_init(const char *name) > { > - char bdev_name[BDEVNAME_SIZE]; > - struct block_device *bdev; > - int ret = -ENODEV; > - fmode_t mode = FMODE_READ | FMODE_WRITE | FMODE_EXCL; > - sector_t nr_sects; > - > + int ret = -EINVAL; > + > if (!best_effort || !blkdev[0]) > return 0; > > - /* hold bdev exclusively */ > - bdev = blkdev_get_by_path(blkdev, mode, blkdev); > - if (IS_ERR(bdev)) { > - dev_t devt; > - > - devt = name_to_dev_t(blkdev); > - if (devt == 0) { > - pr_err("failed to open '%s'!\n", blkdev); > - return -ENODEV; > - } > - bdev = blkdev_get_by_dev(devt, mode, blkdev); > - if (IS_ERR(bdev)) { > - pr_err("failed to open '%s'!\n", blkdev); > -
Re: simplify pstore-blk
hi Christoph, On Fri, Oct 16, 2020 at 9:27 PM Christoph Hellwig wrote: > > Hi all, > > this series cleans up and massively simplifies the pstore-blk code, > please take a look. Thanks for your code. I am going to redesign pstore/blk referred to your idea. I want to split pstore/blk into two parts, pstore/dev and pstore/blk. pstore/dev is a common layer that handles all configurations for users. All devices including block devices, ram devices and mtd devices register to pstore/dev with operations and some flags. Besides that, we can delete the codes for the block device to register a panic_wirte() interface since there is no block device needed. But it should be easy to implement registering if necessary.
Re: [PATCH 2/9] pstore/blk: update the command line example
On Fri, Oct 16, 2020 at 9:28 PM Christoph Hellwig wrote: > > Use the human readable device name instead of the device number, and > add the required best_effort parameter. > > Signed-off-by: Christoph Hellwig > --- > Documentation/admin-guide/pstore-blk.rst | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/Documentation/admin-guide/pstore-blk.rst > b/Documentation/admin-guide/pstore-blk.rst > index 296d5027787ac2..d9ec8b0572d3b2 100644 > --- a/Documentation/admin-guide/pstore-blk.rst > +++ b/Documentation/admin-guide/pstore-blk.rst > @@ -35,7 +35,7 @@ module parameters have priority over Kconfig. > > Here is an example for module parameters:: > > -pstore_blk.blkdev=179:7 pstore_blk.kmsg_size=64 > +pstore_blk.blkdev=/dev/mmcblk0p7 pstore_blk.kmsg_size=64 > best_effort=y > Reviewed-by: WeiXiong Liao > The detail of each configurations may be of interest to you. > > -- > 2.28.0 >
Re: [PATCH 1/9] pstore/zone: cap the maximum device size
On Fri, Oct 16, 2020 at 9:27 PM Christoph Hellwig wrote: > > Introduce an abritrary 128MiB cap to avoid malloc failures when using > a larger block device. > > Signed-off-by: Christoph Hellwig > --- > fs/pstore/zone.c | 4 > 1 file changed, 4 insertions(+) > > diff --git a/fs/pstore/zone.c b/fs/pstore/zone.c > index 3ce89216670c9b..5266ccbec007f3 100644 > --- a/fs/pstore/zone.c > +++ b/fs/pstore/zone.c > @@ -1299,6 +1299,10 @@ int register_pstore_zone(struct pstore_zone_info *info) > pr_warn("total_size must be >= 4096\n"); > return -EINVAL; > } > + if (info->total_size > SZ_128M) { > + pr_warn("capping size to 128MiB\n"); > + info->total_size = SZ_128M; > + } > Reviewed-by: WeiXiong Liao > if (!info->kmsg_size && !info->pmsg_size && !info->console_size && > !info->ftrace_size) { > -- > 2.28.0 >
Re: [RFC v5 3/4] pstore/blk: support pmsg for pstore block
On 2019-01-18 08:17, Kees Cook wrote: > On Mon, Jan 7, 2019 at 4:01 AM liaoweixiong > wrote: >> >> To enable pmsg, just set pmsg_size when block device register blkzone. > > At first pass, this looks like a reasonable extension of blkzone. Is > it possible to add console, ftrace, etc, too? > I plan to do that on other patch, because i am going to refine the public codes for all extensions before add console, ftrace, etc. It takes some time. > -Kees > >> >> Signed-off-by: liaoweixiong >> --- >> fs/pstore/blkzone.c| 254 >> + >> include/linux/pstore_blk.h | 1 + >> 2 files changed, 233 insertions(+), 22 deletions(-) >> >> diff --git a/fs/pstore/blkzone.c b/fs/pstore/blkzone.c >> index e1b7f26..157f1cfd 100644 >> --- a/fs/pstore/blkzone.c >> +++ b/fs/pstore/blkzone.c >> @@ -32,12 +32,14 @@ >> * >> * @sig: signature to indicate header (BLK_SIG xor BLKZONE-type value) >> * @datalen: length of data in @data >> + * @start: offset into @data where the beginning of the stored bytes begin >> * @data: zone data. >> */ >> struct blkz_buffer { >> #define BLK_SIG (0x43474244) /* DBGC */ >> uint32_t sig; >> atomic_t datalen; >> + atomic_t start; >> uint8_t data[0]; >> }; >> >> @@ -70,6 +72,9 @@ struct blkz_dmesg_header { >> * frontent name for this zone >> * @buffer: >> * pointer to data buffer managed by this zone >> + * @oldbuf: >> + * pointer to old data buffer. It is used for single zone such as pmsg, >> + * saving the old buffer. >> * @buffer_size: >> * bytes in @buffer->data >> * @should_recover: >> @@ -83,6 +88,7 @@ struct blkz_zone { >> enum pstore_type_id type; >> >> struct blkz_buffer *buffer; >> + struct blkz_buffer *oldbuf; >> size_t buffer_size; >> bool should_recover; >> atomic_t dirty; >> @@ -90,8 +96,10 @@ struct blkz_zone { >> >> struct blkoops_context { >> struct blkz_zone **dbzs;/* dmesg block zones */ >> + struct blkz_zone *pbz; /* Pmsg block zone */ >> unsigned int dmesg_max_cnt; >> unsigned int dmesg_read_cnt; >> + unsigned int pmsg_read_cnt; >> unsigned int dmesg_write_cnt; >> /** >> * the counter should be recovered when do recovery >> @@ -125,6 +133,11 @@ static inline int buffer_datalen(struct blkz_zone *zone) >> return atomic_read(>buffer->datalen); >> } >> >> +static inline int buffer_start(struct blkz_zone *zone) >> +{ >> + return atomic_read(>buffer->start); >> +} >> + >> static inline bool is_on_panic(void) >> { >> struct blkoops_context *cxt = _cxt; >> @@ -394,6 +407,72 @@ static int blkz_recover_dmesg(struct blkoops_context >> *cxt) >> return ret; >> } >> >> +static int blkz_recover_pmsg(struct blkoops_context *cxt) >> +{ >> + struct blkz_info *info = cxt->bzinfo; >> + struct blkz_buffer *oldbuf; >> + struct blkz_zone *zone = NULL; >> + ssize_t (*readop)(char *buf, size_t bytes, loff_t pos); >> + int ret = 0; >> + ssize_t rcnt, len; >> + >> + zone = cxt->pbz; >> + if (!zone || zone->oldbuf) >> + return 0; >> + >> + if (is_on_panic()) >> + goto out; >> + >> + readop = info->read; >> + if (unlikely(!readop)) >> + return -EINVAL; >> + >> + len = zone->buffer_size + sizeof(*oldbuf); >> + oldbuf = kzalloc(len, GFP_KERNEL); >> + if (!oldbuf) >> + return -ENOMEM; >> + >> + rcnt = readop((char *)oldbuf, len, zone->off); >> + if (rcnt != len) { >> + pr_debug("recovery pmsg failed\n"); >> + ret = (int)rcnt < 0 ? (int)rcnt : -EIO; >> + goto free_oldbuf; >> + } >> + >> + if (oldbuf->sig != zone->buffer->sig) { >> + pr_debug("no valid data in zone %s\n", zone->name); >> + goto free_oldbuf; >> + } >> + >> + if (zone->buffer_size < atomic_read(>datalen) || >> + zone->buffer_size < atomic_read(>start)) { >> + pr_info("found overtop zone: %s: off %lu, size %zu\n", >> + zone->name, zone->off, zone->buffer_size); >> + goto free_oldbuf; >> + } >> + >> + if (!atomic_read(>datalen)) { >> + pr_debug("found erased zone: %s: id 0, off %lu, size %zu, >> datalen %d\n", >> + zone->name, zone->off, zone->buffer_size, >> + atomic_read(>datalen)); >> + kfree(oldbuf); >> + goto out; >> + } >> + >> + pr_debug("found nice zone: %s: id 0, off %lu, size %zu, datalen >> %d\n", >> + zone->name, zone->off, zone->buffer_size, >> + atomic_read(>datalen)); >> + zone->oldbuf = oldbuf; >> +out: >> + if (atomic_read(>dirty)) >> +
Re: [RFC v5 2/4] pstore/blk: add sample for pstore_blk
On 2019-01-18 08:21, Kees Cook wrote: > On Thu, Jan 17, 2019 at 4:15 PM Kees Cook wrote: >> >> On Mon, Jan 7, 2019 at 4:01 AM liaoweixiong >> wrote: >>> >>> It is a sample for pstore_blk, using general ram rather than block device. >>> According to pstore_blk, the data will be saved to ram buffer if not >>> register device path and apis for panic. So, it can only used to dump >>> Oops and some things will not reboot. >> >> I'm not sure I see the purpose of this implementation? Doesn't this >> just cause all the pstore machinery to skip any actions? i.e. without >> bzinfo->part_path, won't blkz_sample_write() just return -EINVAL, etc? > > Say, instead of a no-op driver, can you build something like the how > ramoops processes module parameters, so that a person can define an > arbitrary device at boot time for blkoops? This also allows for easier > runtime testing too. Sure, i will do it in next version. But it can only use for oops, excluding panic. I have no idea how to pass panic operation parameters. > > This all looks good, with some minor tweaks as mentioned. And on > closer review, yeah, it doesn't look like it shares much with ramoops. > :) > > Thanks for sending this series; I look forward to the next version. :) > > -Kees >
Re: [RFC v5 0/4] pstore/block: new support logger for block devices
On 2019-01-08 05:47, Kees Cook wrote: > On Mon, Jan 7, 2019 at 4:01 AM liaoweixiong > wrote: >> >> Why should we need pstore_block? >> 1. Most embedded intelligent equipment have no persistent ram, which >> increases costs. We perfer to cheaper solutions, like block devices. >> In fast, there is already a sample for block device logger in driver >> MTD (drivers/mtd/mtdoops.c). >> 2. Do not any equipment have battery, which means that it lost all data >> on general ram if power failure. Pstore has little to do for these >> equipments. > > I'm catching up from being off over the holidays, but I just wanted to > say that I like the general idea here. Thanks for your work very much! :) > >> [PATCH v5] >> On patch 1: >> 1. rename pstore/rom to pstore/blk > > Thanks! The earlier name seemed wrong. :) > >> 2. Do not allocate any memory in the write path of panic. So, use local >> array instead in function romz_recover_dmesg_meta. >> 3. Add C header file "linux/fs.h" to fix implicit declaration of function >>'filp_open','kernel_read'... >> On patch 3: >> 1. If panic, do not recover pmsg but flush if it is dirty. >> 2. Fix erase pmsg failed. >> On patch 4: >> 1. Create a document for pstore/blk > > Overall, I wonder if more of the ram backend code could get reused for > the blk backend? I'd much rather be able to share that, since much of > the logic is the same. Before I wrote pstore/blk, I specially studied pstore/ram. As your words, much of the logic is the same, you will see some codes similar to pstore/ram. But there is still something different with pstore/ram. For example, block device maybe unready for read/write before kernel finish initializing. So, pstore/blk recovers old data when mount rather than initialize. In addition, pstore/blk could not be used for pstore-console, since block device much slower than ram. In my original design, pstore/blk works like a manager for storage space, don't care if it is ram or block device. So, it may works for persistent ram too. > > I'll get to a more detailed review in the coming days. Thanks for sending > this! > > -Kees > >> >> [PATCH v4] >> On patch 1: >> 1. Fix always true condition '(--i >= 0) => (0-u32max >= 0)' in function >>romz_init_zones by defining variable i to 'int' rahter than >>'unsigned int'. >> 2. To make codes more easily to read, we use macro READ_NEXT_ZONE for >>return value of romz_dmesg_read if it need to read next zone. >>Moveover, we assign READ_NEXT_ZONE -1024 rather than 0. >> 3. Add 'FLUSH_META' to 'enum romz_flush_mode' and rename 'NOT_FLUSH' to >>'FLUSH_NONE' >> 4. Function romz_zone_write work badly with FLUSH_PART mode as badly >>address and offset to write. >> On patch 3: >> NEW SUPPORT psmg for pstore_rom. >> >> [PATCH v3] >> On patch 1: >> Fix build as module error for undefined 'vfs_read' and 'vfs_write' >> Both of 'vfs_read' and 'vfs_write' haven't be exproted yet, so we use >> 'kernel_read' and 'kernel_write' instead. >> >> [PATCH v2] >> On patch 1: >> Fix build as module error for redefinition of 'romz_unregister' and >> 'romz_register' >> >> [PATCH v1] >> On patch 1: >> Core codes of pstore_rom, which works well on allwinner(sunxi) platform. >> On patch 2: >> A sample for pstore_rom, using general ram rather than block device. >> >> liaoweixiong (4): >> pstore/blk: new support logger for block devices >> pstore/blk: add sample for pstore_blk >> pstore/blk: support pmsg for pstore block >> Documentation: pstore/blk: create document for pstore_blk >> >> Documentation/admin-guide/pstore-block.rst | 226 ++ >> MAINTAINERS|1 + >> fs/pstore/Kconfig | 20 + >> fs/pstore/Makefile |5 + >> fs/pstore/blkbuf.c | 46 ++ >> fs/pstore/blkzone.c| 1168 >> >> include/linux/pstore_blk.h | 62 ++ >> 7 files changed, 1528 insertions(+) >> create mode 100644 Documentation/admin-guide/pstore-block.rst >> create mode 100644 fs/pstore/blkbuf.c >> create mode 100644 fs/pstore/blkzone.c >> create mode 100644 include/linux/pstore_blk.h >> >> -- >> 1.9.1 >> > >
Re: [RFC v4 0/3] pstore/rom: new support logger for block devices
hi Tony: On 2019-01-04 01:18, Luck, Tony wrote: > I'm curious why you call this "pstore/rom" rather than the more descriptive > "pstore/block". Because there is "pstore/ram", so i name it as "pstore/rom". It's nice to rename it "pstore/block", i will change it in next version of patch. > > It looks to be a really good idea. Should i add "Acked-by" in next version of patch? I have little experience in sending patch to upstream linux. > > I think you need to document how the "write" function for the block device > must be written. > Since pstore calls this at "panic" time, the write path: > > + Cannot allocate any memory > + Must be polled, not interrupt driven > + Cannot take any locks that may be held by regular code > + ... perhaps other restrictions that I can't think of right now > > The memory allocation restriction is likely easy to get around. Just allocate > anything > you need at pstore "init" time rather than waiting until the panic. >I will provide a documemt for it in next version of patch. > -Tony >
Re: [PATCH] pstore: fix crypto dependencies of 842/zstd compression
hi , On 2018/12/17 16:16, kbuild test robot wrote: > Hi liaoweixiong, > > Thank you for the patch! Yet something to improve: > > [auto build test ERROR on kees/for-next/pstore] > [also build test ERROR on v4.20-rc7 next-20181214] > [if your patch is applied to the wrong git tree, please drop us a note to > help improve the system] > > url: > https://github.com/0day-ci/linux/commits/liaoweixiong/pstore-fix-crypto-dependencies-of-842-zstd-compression/20181214-222645 > base: https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git > for-next/pstore > config: i386-allmodconfig (attached as .config) > compiler: gcc-7 (Debian 7.3.0-1) 7.3.0 > reproduce: > # save the attached .config to linux build tree > make ARCH=i386 > > All errors (new ones prefixed by >>): > >fs/pstore/platform.o: In function `zbufsize_zstd': >>> platform.c:(.text+0x30a): undefined reference to `ZSTD_compressBound' Yes, i reproduced it. I had made a new patch in version 2. On new patch, i try to trun sub-options of compression back to 'bool' for more rigorous. Commit info like this: On commit 58eb5b670747 ("pstore: fix crypto dependencies"), dependency bug was fixed by selecting the crypto core rather than turned compression sub-options to 'tristate'. In addition, these options are used to enable/disable compression. They are not modules, and mean nothing when set to 'M'. So, this patch is going to turn them back to 'bool'. > > --- > 0-DAY kernel test infrastructureOpen Source Technology Center > https://lists.01.org/pipermail/kbuild-all Intel Corporation >
Re: [PATCH] pstore: fix crypto dependencies of 842/zstd compression
在 2018年12月13日 02:43, Kees Cook 写道: On Wed, Dec 12, 2018 at 12:24 AM liaoweixiong wrote: Reference to commit 58eb5b670747 ("pstore: fix crypto dependencies"), which fixed crypto dependencies of deflate, lzo, lz4 and lz4hc compression, but omitted 842 and newer compression zstd from commit 1021bcf44d0e ("pstore: add zstd compression support") Signed-off-by: liaoweixiong Were you seeing build or config failures without this patch? testsdfsdf -Kees --- fs/pstore/Kconfig | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/pstore/Kconfig b/fs/pstore/Kconfig index 0d19d19..7068f45 100644 --- a/fs/pstore/Kconfig +++ b/fs/pstore/Kconfig @@ -44,14 +44,14 @@ config PSTORE_LZ4HC_COMPRESS This option enables LZ4HC (high compression) mode algorithm. config PSTORE_842_COMPRESS - bool "842 compression" + tristate "842 compression" depends on PSTORE select CRYPTO_842 help This option enables 842 compression algorithm support. config PSTORE_ZSTD_COMPRESS - bool "zstd compression" + tristate "zstd compression" depends on PSTORE select CRYPTO_ZSTD help -- 1.9.1
Re: [PATCH] pstore: fix crypto dependencies of 842/zstd compression
In fast, there is no any failure while building or configuring on the newest codes. The patch of commit 58eb5b670747 ("pstore: fix crypto dependencies") makes the pstore itself select the crypto core if PSTORE_COMPRESS is set. This fixes the dependence bug at all in my tests. But this patch also turns the sub-options from 'bool' into 'tristate'. It's ok, but makes sub-options different between deflate, lzo, lz4, lz4hc and 842, zstd. So, my patch just keeps them in line. How about to make all these sub-options as 'bool'? These compressions are just function options but not module. Otherwise, it's not about fixing crypto dependencies. 在 2018年12月13日 02:43, Kees Cook 写道: On Wed, Dec 12, 2018 at 12:24 AM liaoweixiong wrote: Reference to commit 58eb5b670747 ("pstore: fix crypto dependencies"), which fixed crypto dependencies of deflate, lzo, lz4 and lz4hc compression, but omitted 842 and newer compression zstd from commit 1021bcf44d0e ("pstore: add zstd compression support") Signed-off-by: liaoweixiong Were you seeing build or config failures without this patch? -Kees --- fs/pstore/Kconfig | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/pstore/Kconfig b/fs/pstore/Kconfig index 0d19d19..7068f45 100644 --- a/fs/pstore/Kconfig +++ b/fs/pstore/Kconfig @@ -44,14 +44,14 @@ config PSTORE_LZ4HC_COMPRESS This option enables LZ4HC (high compression) mode algorithm. config PSTORE_842_COMPRESS - bool "842 compression" + tristate "842 compression" depends on PSTORE select CRYPTO_842 help This option enables 842 compression algorithm support. config PSTORE_ZSTD_COMPRESS - bool "zstd compression" + tristate "zstd compression" depends on PSTORE select CRYPTO_ZSTD help -- 1.9.1