Hello Community Members, I would like to propose a change to how we handle sparse image generation for ext* file-systems in OpenEmbedded, specifically moving away from the generic img2simg tool toward the specialised ext2simg_android in the e2fsprogs [contrib/android]
*1) Problem*
Currently, image_types_sparse.bbclass uses img2simg for all file-system types.
img2simg is a tool that reads the raw image to find zero-filled blocks.
For ext* file-systems, this is inefficient compared to ext2simg from e2fsprogs
[contrib/android]
which understands the file-system structure and uses the block bitmap to
determine sparse areas.
*2) History*
Previously, 'ext2simg' was bundled within android-tools (v5.1 and older).
The utility has since migrated to the e2fsprogs project (contrib/android).
However, upstream e2fsprogs does not provide a build system for this tool,
and it requires libsparse from android-tools to compile.
*3) Proposed Changes*
To implement this , below approach is proposed for feedback:
a. meta-oe (android-tools):
Export libsparse, libbase, and liblog headers and libraries to the sysroot.
Currently, these are often internal to the build.
Reorganise header installation to ${includedir}/sparse to match the expected
include paths for e2fsprogs contrib tools.
b. meta-oe (ext2simg_android )
Have created an fresh recipe which focus only on getting ext2simg.c compiled
with andorid sparse.h
(as per suggestion on first RFC
[email protected] | [RFC] Improving ext* sparse image
generation: migrating from img2simg to ext4sparse for ext* filesystem (
https://lists.openembedded.org/g/openembedded-core/topic/rfc_improving_ext_sparse/117715350
)
)
This would link against the exported libsparse from android-tools-native.
c. meta-oe (image_types_sparse.bbclass):
Update CONVERSION_CMD:sparse to detect ext* types.
Use ext2simg from the e2fsprogs [contrib/android] for these types while
falling back to img2simg for others (like f2fs).
*4) Points for Discussion / Feasibility*
I am seeking feedback on the following:
*a. Build Integration:*
In my PoC, I've used a manual ${CC} call in do_compile:append.
Would it be preferred to patch the e2fsprogs Makefile to handle this "contrib"
tool more natively?
*b. Library Handling:*
To ensure the native binary finds libsparse at image creation time, I am
currently using LD_LIBRARY_PATH in the class.
I suspect a better approach would be ensuring proper RPATH during the e2fsprogs
build.
If members can help for any better way of doing this
*c. I have a working proof of concept using RPI4 build*
Attached along are the patch
001-meta-oe-image_types_sparse-use-ext2simg_android-for-.patch
*5) Logical layout of current POC*
This commit bridges the gap between these two recipes:
1. android-tools:
- Updated to v29.0.6 and masked v5.1.1 to provide modern libraries.
- Modified do_install to export libsparse, libbase, and liblog headers and
libraries to the sysroot, enabling external linking.
2. image_types_sparse.bbclass:
- Updated CONVERSION_CMD to branch by filesystem type, calling ext2simg_android
specifically for ext* images.
- Exported LD_LIBRARY_PATH to the native sysroot to ensure the ext2simg_android
binary can locate its shared dependencies at runtime.
3. Layer Configuration:
- Updated layer.conf to support dynamic-layers for SELinux and synchronized
BBMASK entries to prioritize the updated tools.
*6) Benefit :*
This reduces the time and cost involved while flashing Android RFS / Fastboot
etc
Even in CI/CD , which all adds to ease of usage and have monetary benefit
involved with continous CI/CD
Looking forward to thoughts / feedback from community members
Please do let me know if any specific platform or group needs to be addressed
to have such discussion
If there is any specific meeting involved where i can join and explain, please
do let me know
I can connect to team members and works as per suggestion from community to
fine tune / rework if any specific changes community suggest
Thanks ,
Ashish Kumar Mishra
0001-meta-oe-image_types_sparse-use-ext2simg_android-for-.patch
Description: Binary data
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#124499): https://lists.openembedded.org/g/openembedded-devel/message/124499 Mute This Topic: https://lists.openembedded.org/mt/117889188/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
