Re: [PATCH v2] lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number
On Fri, Jul 07, 2023 at 10:13:34AM +0200, Mattijs Korpershoek wrote: > Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") > fixed cache alignment for systems with a D-CACHE. > > However it introduced some performance regressions [1] on system > flashing huge images, such as Android. > > On AM62x SK EVM, we also observe such performance penalty: > Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.954s] > Writing 'super'OKAY [ 75.926s] > Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.641s] > Writing 'super'OKAY [ 62.849s] > Finished. Total time: 182.474s > > The reason for this is that we use an arbitrary small buffer > (info->blksz * 100) for transferring. > > Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE) > as suggested in the original's patch review [2]. > > With this patch, performance impact is mitigated: > Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.912s] > Writing 'super'OKAY [ 15.780s] > Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.581s] > Writing 'super'OKAY [ 17.192s] > Finished. Total time: 76.569s > > [1] > https://lore.kernel.org/r/20221118121323.4009193-1-gary.bis...@boundarydevices.com > [2] > https://lore.kernel.org/r/all/43e4c17c-4483-ec8e-f843-9b4c5569b...@seco.com/ > > Fixes: 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") > Signed-off-by: Mattijs Korpershoek Applied to u-boot/master, thanks! -- Tom signature.asc Description: PGP signature
Re: [PATCH v2] lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number
Hi Qianfan, Thank you for your review. On ven., juil. 07, 2023 at 18:54, qianfan wrote: > 在 2023/7/7 16:13, Mattijs Korpershoek 写道: >> Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") >> fixed cache alignment for systems with a D-CACHE. >> >> However it introduced some performance regressions [1] on system >> flashing huge images, such as Android. >> >> On AM62x SK EVM, we also observe such performance penalty: >> Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.954s] >> Writing 'super'OKAY [ 75.926s] >> Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.641s] >> Writing 'super'OKAY [ 62.849s] >> Finished. Total time: 182.474s >> >> The reason for this is that we use an arbitrary small buffer >> (info->blksz * 100) for transferring. >> >> Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE) >> as suggested in the original's patch review [2]. >> >> With this patch, performance impact is mitigated: >> Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.912s] >> Writing 'super'OKAY [ 15.780s] >> Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.581s] >> Writing 'super'OKAY [ 17.192s] >> Finished. Total time: 76.569s >> >> [1] >> https://lore.kernel.org/r/20221118121323.4009193-1-gary.bis...@boundarydevices.com >> [2] >> https://lore.kernel.org/r/all/43e4c17c-4483-ec8e-f843-9b4c5569b...@seco.com/ >> >> Fixes: 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") >> Signed-off-by: Mattijs Korpershoek >> --- >> Changes in v2: >> - Use FASTBOOT_MAX_BLK_WRITE instead of blkcnt (Qianfan) >> - Link to v1: >> https://lore.kernel.org/r/20230616-sparse-flash-fix-v1-1-6bafeacc5...@baylibre.com >> --- >> drivers/fastboot/fb_mmc.c | 2 -- >> include/image-sparse.h| 2 ++ >> lib/image-sparse.c| 3 ++- >> 3 files changed, 4 insertions(+), 3 deletions(-) >> >> diff --git a/drivers/fastboot/fb_mmc.c b/drivers/fastboot/fb_mmc.c >> index 9d25c402028a..060918e49109 100644 >> --- a/drivers/fastboot/fb_mmc.c >> +++ b/drivers/fastboot/fb_mmc.c >> @@ -19,8 +19,6 @@ >> #include >> #include >> >> -#define FASTBOOT_MAX_BLK_WRITE 16384 >> - >> #define BOOT_PARTITION_NAME "boot" >> >> struct fb_mmc_sparse { >> diff --git a/include/image-sparse.h b/include/image-sparse.h >> index 0572dbd0a283..282a0b256498 100644 >> --- a/include/image-sparse.h >> +++ b/include/image-sparse.h >> @@ -7,6 +7,8 @@ >> #include >> #include >> >> +#define FASTBOOT_MAX_BLK_WRITE 16384 > Hi: > > Just a personal suggestion, define sometings like FASTBOOT_MAX_BLK_WRITE in > image-sparse.c is very strange. > > Or maybe define a marco such as SPARSE_MAX_BLK_WRITE and set a default > value to 16384, and leave some comments for why we choice this value. I don't agree with having a duplicating between SPARSE_MAX_BLK_WRITE and FASTBOOT_MAX_BLK_WRITE. code comments can rot as well. And FASTBOOT_MAX_BLK_WRITE is used for both sparse and unsparsed image, which is why I chose to not rename it. > > Thanks. >> + >> #define ROUNDUP(x, y) (((x) + ((y) - 1)) & ~((y) - 1)) >> >> struct sparse_storage { >> diff --git a/lib/image-sparse.c b/lib/image-sparse.c >> index 5ec0f94ab3eb..8f8a67e15804 100644 >> --- a/lib/image-sparse.c >> +++ b/lib/image-sparse.c >> @@ -55,7 +55,8 @@ static lbaint_t write_sparse_chunk_raw(struct >> sparse_storage *info, >> void *data, >> char *response) >> { >> -lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100; >> +lbaint_t n = blkcnt, write_blks, blks = 0; >> +lbaint_t aligned_buf_blks = FASTBOOT_MAX_BLK_WRITE; >> uint32_t *aligned_buf = NULL; >> >> if (CONFIG_IS_ENABLED(SYS_DCACHE_OFF)) { >> >> --- >> base-commit: 923de765ee1a5b26310f02cb42dcbad9e2b011c5 >> change-id: 20230616-sparse-flash-fix-9c2852aa8d16 >> >> Best regards,
Re: [PATCH v2] lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number
在 2023/7/7 16:13, Mattijs Korpershoek 写道: Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") fixed cache alignment for systems with a D-CACHE. However it introduced some performance regressions [1] on system flashing huge images, such as Android. On AM62x SK EVM, we also observe such performance penalty: Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.954s] Writing 'super'OKAY [ 75.926s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.641s] Writing 'super'OKAY [ 62.849s] Finished. Total time: 182.474s The reason for this is that we use an arbitrary small buffer (info->blksz * 100) for transferring. Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE) as suggested in the original's patch review [2]. With this patch, performance impact is mitigated: Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.912s] Writing 'super'OKAY [ 15.780s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.581s] Writing 'super'OKAY [ 17.192s] Finished. Total time: 76.569s [1] https://lore.kernel.org/r/20221118121323.4009193-1-gary.bis...@boundarydevices.com [2] https://lore.kernel.org/r/all/43e4c17c-4483-ec8e-f843-9b4c5569b...@seco.com/ Fixes: 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") Signed-off-by: Mattijs Korpershoek --- Changes in v2: - Use FASTBOOT_MAX_BLK_WRITE instead of blkcnt (Qianfan) - Link to v1: https://lore.kernel.org/r/20230616-sparse-flash-fix-v1-1-6bafeacc5...@baylibre.com --- drivers/fastboot/fb_mmc.c | 2 -- include/image-sparse.h| 2 ++ lib/image-sparse.c| 3 ++- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/fastboot/fb_mmc.c b/drivers/fastboot/fb_mmc.c index 9d25c402028a..060918e49109 100644 --- a/drivers/fastboot/fb_mmc.c +++ b/drivers/fastboot/fb_mmc.c @@ -19,8 +19,6 @@ #include #include -#define FASTBOOT_MAX_BLK_WRITE 16384 - #define BOOT_PARTITION_NAME "boot" struct fb_mmc_sparse { diff --git a/include/image-sparse.h b/include/image-sparse.h index 0572dbd0a283..282a0b256498 100644 --- a/include/image-sparse.h +++ b/include/image-sparse.h @@ -7,6 +7,8 @@ #include #include +#define FASTBOOT_MAX_BLK_WRITE 16384 Hi: Just a personal suggestion, define sometings like FASTBOOT_MAX_BLK_WRITE in image-sparse.c is very strange. Or maybe define a marco such as SPARSE_MAX_BLK_WRITE and set a default value to 16384, and leave some comments for why we choice this value. Thanks. + #define ROUNDUP(x, y) (((x) + ((y) - 1)) & ~((y) - 1)) struct sparse_storage { diff --git a/lib/image-sparse.c b/lib/image-sparse.c index 5ec0f94ab3eb..8f8a67e15804 100644 --- a/lib/image-sparse.c +++ b/lib/image-sparse.c @@ -55,7 +55,8 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info, void *data, char *response) { - lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100; + lbaint_t n = blkcnt, write_blks, blks = 0; + lbaint_t aligned_buf_blks = FASTBOOT_MAX_BLK_WRITE; uint32_t *aligned_buf = NULL; if (CONFIG_IS_ENABLED(SYS_DCACHE_OFF)) { --- base-commit: 923de765ee1a5b26310f02cb42dcbad9e2b011c5 change-id: 20230616-sparse-flash-fix-9c2852aa8d16 Best regards,
[PATCH v2] lib: sparse: allocate FASTBOOT_MAX_BLK_WRITE instead of small number
Commit 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") fixed cache alignment for systems with a D-CACHE. However it introduced some performance regressions [1] on system flashing huge images, such as Android. On AM62x SK EVM, we also observe such performance penalty: Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.954s] Writing 'super'OKAY [ 75.926s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.641s] Writing 'super'OKAY [ 62.849s] Finished. Total time: 182.474s The reason for this is that we use an arbitrary small buffer (info->blksz * 100) for transferring. Fix it by using a bigger buffer (info->blksz * FASTBOOT_MAX_BLK_WRITE) as suggested in the original's patch review [2]. With this patch, performance impact is mitigated: Sending sparse 'super' 1/2 (768793 KB) OKAY [ 23.912s] Writing 'super'OKAY [ 15.780s] Sending sparse 'super' 2/2 (629819 KB) OKAY [ 19.581s] Writing 'super'OKAY [ 17.192s] Finished. Total time: 76.569s [1] https://lore.kernel.org/r/20221118121323.4009193-1-gary.bis...@boundarydevices.com [2] https://lore.kernel.org/r/all/43e4c17c-4483-ec8e-f843-9b4c5569b...@seco.com/ Fixes: 62649165cb02 ("lib: sparse: Make CHUNK_TYPE_RAW buffer aligned") Signed-off-by: Mattijs Korpershoek --- Changes in v2: - Use FASTBOOT_MAX_BLK_WRITE instead of blkcnt (Qianfan) - Link to v1: https://lore.kernel.org/r/20230616-sparse-flash-fix-v1-1-6bafeacc5...@baylibre.com --- drivers/fastboot/fb_mmc.c | 2 -- include/image-sparse.h| 2 ++ lib/image-sparse.c| 3 ++- 3 files changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/fastboot/fb_mmc.c b/drivers/fastboot/fb_mmc.c index 9d25c402028a..060918e49109 100644 --- a/drivers/fastboot/fb_mmc.c +++ b/drivers/fastboot/fb_mmc.c @@ -19,8 +19,6 @@ #include #include -#define FASTBOOT_MAX_BLK_WRITE 16384 - #define BOOT_PARTITION_NAME "boot" struct fb_mmc_sparse { diff --git a/include/image-sparse.h b/include/image-sparse.h index 0572dbd0a283..282a0b256498 100644 --- a/include/image-sparse.h +++ b/include/image-sparse.h @@ -7,6 +7,8 @@ #include #include +#define FASTBOOT_MAX_BLK_WRITE 16384 + #define ROUNDUP(x, y) (((x) + ((y) - 1)) & ~((y) - 1)) struct sparse_storage { diff --git a/lib/image-sparse.c b/lib/image-sparse.c index 5ec0f94ab3eb..8f8a67e15804 100644 --- a/lib/image-sparse.c +++ b/lib/image-sparse.c @@ -55,7 +55,8 @@ static lbaint_t write_sparse_chunk_raw(struct sparse_storage *info, void *data, char *response) { - lbaint_t n = blkcnt, write_blks, blks = 0, aligned_buf_blks = 100; + lbaint_t n = blkcnt, write_blks, blks = 0; + lbaint_t aligned_buf_blks = FASTBOOT_MAX_BLK_WRITE; uint32_t *aligned_buf = NULL; if (CONFIG_IS_ENABLED(SYS_DCACHE_OFF)) { --- base-commit: 923de765ee1a5b26310f02cb42dcbad9e2b011c5 change-id: 20230616-sparse-flash-fix-9c2852aa8d16 Best regards, -- Mattijs Korpershoek