Re: Checksummed environments
On 12/04/18 10:33, Daniel Kiper wrote: > On Tue, Apr 10, 2018 at 11:09:33PM +0200, Daniel Kiper wrote: >> On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote: >>> On 06/04/18 14:35, Daniel Kiper wrote: On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: > Hey, I work for Northern.tech, developing update software for embedded > Linux devices. > > I have a question about GRUB's environment block: This block is not I am not sure what exactly you mean by "GRUB's environment block". Could you send me some references to the code? >>> >>> Of course, sorry if the context wasn't clear. I'm talking about the >>> environment block mentioned here: >>> https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block, >>> which is used to store information from one boot to the next, using >>> save_env and load_env. >>> > checksummed, and hence I reckon it can become corrupt if power is lost > in the middle of a write. What about the other blocks? >>> >>> In theory, there is only one block, because it is written in-place, >>> directly on disk, without changing any other filesystem blocks. Is that >>> what you meant by "other blocks"? >>> > This is an important safety criterion for us, so we've been thinking of > developing environment block checksumming as an extension to the > existing save_env and load_env commands. The most likely approach will > be to grab X amount of bytes at the end of the block and use these for > the checksum. Could you tell us more about that? >>> >>> The idea comes from U-Boot [1], which uses a dedicated block on a >>> storage device to store data, and uses a CRC32 checksum to make sure it >>> is consistent. The motivation is to be able to detect that the block is >>> corrupt, rather than accepting a block of data that may have incorrect >>> data in if a write was interrupted midway by a powerloss. You can read >>> more about it in the link. >>> >>> [2] https://www.denx.de/wiki/DULG/UBootEnvVariables >> >> I am OK with the idea. However, I think that the feature should have >> a kind of switch to turn it off/on. At first sight it looks that new >> environment variable is sufficient for it. > > And/Or an argument for save_env/load_env... Yes, I'm fine with either. How about a variable that determines the default, and you can override it with flags? -- Kristian ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: Checksummed environments
On Tue, Apr 10, 2018 at 11:09:33PM +0200, Daniel Kiper wrote: > On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote: > > On 06/04/18 14:35, Daniel Kiper wrote: > > > On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: > > >> Hey, I work for Northern.tech, developing update software for embedded > > >> Linux devices. > > >> > > >> I have a question about GRUB's environment block: This block is not > > > > > > I am not sure what exactly you mean by "GRUB's environment block". > > > Could you send me some references to the code? > > > > Of course, sorry if the context wasn't clear. I'm talking about the > > environment block mentioned here: > > https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block, > > which is used to store information from one boot to the next, using > > save_env and load_env. > > > > >> checksummed, and hence I reckon it can become corrupt if power is lost > > >> in the middle of a write. > > > > > > What about the other blocks? > > > > In theory, there is only one block, because it is written in-place, > > directly on disk, without changing any other filesystem blocks. Is that > > what you meant by "other blocks"? > > > > >> This is an important safety criterion for us, so we've been thinking of > > >> developing environment block checksumming as an extension to the > > >> existing save_env and load_env commands. The most likely approach will > > >> be to grab X amount of bytes at the end of the block and use these for > > >> the checksum. > > > > > > Could you tell us more about that? > > > > The idea comes from U-Boot [1], which uses a dedicated block on a > > storage device to store data, and uses a CRC32 checksum to make sure it > > is consistent. The motivation is to be able to detect that the block is > > corrupt, rather than accepting a block of data that may have incorrect > > data in if a write was interrupted midway by a powerloss. You can read > > more about it in the link. > > > > [2] https://www.denx.de/wiki/DULG/UBootEnvVariables > > I am OK with the idea. However, I think that the feature should have > a kind of switch to turn it off/on. At first sight it looks that new > environment variable is sufficient for it. And/Or an argument for save_env/load_env... Daniel ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: Checksummed environments
On Fri, Apr 06, 2018 at 03:08:23PM +0200, Kristian Amlie wrote: > On 06/04/18 14:35, Daniel Kiper wrote: > > On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: > >> Hey, I work for Northern.tech, developing update software for embedded > >> Linux devices. > >> > >> I have a question about GRUB's environment block: This block is not > > > > I am not sure what exactly you mean by "GRUB's environment block". > > Could you send me some references to the code? > > Of course, sorry if the context wasn't clear. I'm talking about the > environment block mentioned here: > https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block, > which is used to store information from one boot to the next, using > save_env and load_env. > > >> checksummed, and hence I reckon it can become corrupt if power is lost > >> in the middle of a write. > > > > What about the other blocks? > > In theory, there is only one block, because it is written in-place, > directly on disk, without changing any other filesystem blocks. Is that > what you meant by "other blocks"? > > >> This is an important safety criterion for us, so we've been thinking of > >> developing environment block checksumming as an extension to the > >> existing save_env and load_env commands. The most likely approach will > >> be to grab X amount of bytes at the end of the block and use these for > >> the checksum. > > > > Could you tell us more about that? > > The idea comes from U-Boot [1], which uses a dedicated block on a > storage device to store data, and uses a CRC32 checksum to make sure it > is consistent. The motivation is to be able to detect that the block is > corrupt, rather than accepting a block of data that may have incorrect > data in if a write was interrupted midway by a powerloss. You can read > more about it in the link. > > [2] https://www.denx.de/wiki/DULG/UBootEnvVariables I am OK with the idea. However, I think that the feature should have a kind of switch to turn it off/on. At first sight it looks that new environment variable is sufficient for it. Daniel ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: Checksummed environments
On 06/04/18 14:35, Daniel Kiper wrote: > On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: >> Hey, I work for Northern.tech, developing update software for embedded >> Linux devices. >> >> I have a question about GRUB's environment block: This block is not > > I am not sure what exactly you mean by "GRUB's environment block". > Could you send me some references to the code? Of course, sorry if the context wasn't clear. I'm talking about the environment block mentioned here: https://www.gnu.org/software/grub/manual/grub/grub.html#Environment-block, which is used to store information from one boot to the next, using save_env and load_env. >> checksummed, and hence I reckon it can become corrupt if power is lost >> in the middle of a write. > > What about the other blocks? In theory, there is only one block, because it is written in-place, directly on disk, without changing any other filesystem blocks. Is that what you meant by "other blocks"? >> This is an important safety criterion for us, so we've been thinking of >> developing environment block checksumming as an extension to the >> existing save_env and load_env commands. The most likely approach will >> be to grab X amount of bytes at the end of the block and use these for >> the checksum. > > Could you tell us more about that? The idea comes from U-Boot [1], which uses a dedicated block on a storage device to store data, and uses a CRC32 checksum to make sure it is consistent. The motivation is to be able to detect that the block is corrupt, rather than accepting a block of data that may have incorrect data in if a write was interrupted midway by a powerloss. You can read more about it in the link. [2] https://www.denx.de/wiki/DULG/UBootEnvVariables -- Kristian > >> This would also allow us to fall back to an earlier environment file if >> the current one is corrupt, hence implementing redundancy. >> >> Is this something that the GRUB project would be interested in? We want >> to upstream this if possible, since we think many people may benefit >> from this. > > Great! However, first of all we need some more info about that. > > Daniel > ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
Re: Checksummed environments
On Fri, Apr 06, 2018 at 11:25:22AM +0200, Kristian Amlie wrote: > Hey, I work for Northern.tech, developing update software for embedded > Linux devices. > > I have a question about GRUB's environment block: This block is not I am not sure what exactly you mean by "GRUB's environment block". Could you send me some references to the code? > checksummed, and hence I reckon it can become corrupt if power is lost > in the middle of a write. What about the other blocks? > This is an important safety criterion for us, so we've been thinking of > developing environment block checksumming as an extension to the > existing save_env and load_env commands. The most likely approach will > be to grab X amount of bytes at the end of the block and use these for > the checksum. Could you tell us more about that? > This would also allow us to fall back to an earlier environment file if > the current one is corrupt, hence implementing redundancy. > > Is this something that the GRUB project would be interested in? We want > to upstream this if possible, since we think many people may benefit > from this. Great! However, first of all we need some more info about that. Daniel ___ Grub-devel mailing list Grub-devel@gnu.org https://lists.gnu.org/mailman/listinfo/grub-devel
