Re: Any way to "passive commit" to RAM without syncing ?

2016-03-08 Thread Hal Murray

swiftgri...@gmail.com said:
> Let's say I've got a machine with a boatload of RAM and I have an
> application which wants extremely fast writes, but doesn't really care  that
> much about data integrity, and this case, nor do I. Having the system  come
> back in the same state after rebooting is "nice to have" in this  thought
> experiment. Soo, here is the question:
...
> In other words, operate out of a RAM disk until my human operator gives  the
> go-ahead to flush out the writes to disk or the RAM disk gets full. I  think
> Sprite did something like this back in the Jurassic Period. I also  remember
> CacheFS for Solaris doing something similar for NFS.

I had a setup for a while to keep working log files in RAM.  I have forgotten 
the details.

The general idea was that the working log files were in RAM, old ones were on 
disk.  A script would rsync things from RAM to disk and delete stuff more than 
a few days old.  It was called by cron, the script that stopped the server, or 
by hand.  At boot time, another script would preload the RAM from disk so 
appending kept going at the right place..  If the system crashed, you lost the 
stuff in RAM that hadn't been backed up yet.

Overall, it was simple enough so that I (far from a wizard) could set it up 
without a lot of effort.

You might be able to do something similar for any specific application.  Please 
let us know if you come up with a good solution.


-- 
These are my opinions.  I hate spam.





Re: Any way to "passive commit" to RAM without syncing ?

2016-03-08 Thread Robert Elz
Sounds like you want a tmpfs, possibly union mounted over the
"stable" filesystem, with "rsync" being your :sync" command.

kre



Re: Any way to "passive commit" to RAM without syncing ?

2016-03-08 Thread Malcolm Herbert
Sounds like you're looking for a combo of overlay mounts on top of tmpfs
- the flush to disk you'd have to manage manually but I've done this
with some success in the past and I do seem to recall it being discussed
to provide pseudo read-write over a read-only medium such as a CDROM

http://netbsd.gw.com/cgi-bin/man-cgi?mount_overlay+8+NetBSD-current

the only caution is that I found it tricky to grok the sense of which
filesystem gets overlaid on what and where writes end up, but some
workshopping should show that up pretty quickly

Regards,
Malcolm

On Wed, Mar 9, 2016, at 10:49, Swift Griggs wrote:
> 
> I know NetBSD supports RAM disks. I also know it uses them for 
> installation, but I'm not sure if they are overlays on top of the static 
> disk images or if they get loaded/populated with the image after the 
> system boots. The effect is the same, either way. You end up with a 
> writable file system in RAM which is predicated on a disk image. I was 
> thinking about this in the context of another problem.
> 
> Let's say I've got a machine with a boatload of RAM and I have an 
> application which wants extremely fast writes, but doesn't really care 
> that much about data integrity, and this case, nor do I. Having the
> system 
> come back in the same state after rebooting is "nice to have" in this 
> thought experiment. Soo, here is the question:
> 
> Since a RAM disk is probably the fastest option, is there a way to 
> basically leave uncommitted data until I run "sync" or something calls 
> fsync() ? Even better, is there a way to fake a sync until I run
> something 
> else ?
> 
> In other words, operate out of a RAM disk until my human operator gives 
> the go-ahead to flush out the writes to disk or the RAM disk gets full. I 
> think Sprite did something like this back in the Jurassic Period. I also 
> remember CacheFS for Solaris doing something similar for NFS.
> 
> I can think of a few applications:
> 
> - Internet cafe style browsing where the machines are using older 
> hardware. They might benefit from using a small RAM disk overlay rather 
> than having to use something like a PATA disks for it's cache.
> 
> - Editing high bandwidth data like video or audio. Most of the time my 
> editing sessions don't last more than an hour and once I get a final cut
> I 
> just save the project to a server and don't care much about the recording 
> or editing workstation.
> 
> - Any kind of temporary storage that needs RAM disk speeds, but still 
> needs an option to sync/flush the file system back to long term storage.
> 
> I'm guessing this would be a little tricky with something like UFS since
> a 
> process could alter/write to a file and it'd need to reflect that when 
> another process wanted to do the same, fseek(), or dozens of other 
> operations. However, I just wonder if there is already a way to do this 
> and I'm just not aware of it.
> 
> I wonder if the LiveCD's for Linux that let you flush you changes to USB 
> do something similar. I doubt it. I'm guessing they just load a RAM disk 
> image and you operate on that until you reboot and it's lost.
> 
> -Swift


-- 
Malcolm Herbert
m...@mjch.net