> Yes, performance seems to be the main use case. Fuel files of hundreds of MB > take a long time to load and save, even if you only change a single string. > One instance where this is relevant is the Moose model. They currently use > .mse files, which use a textual representation (if I’m not mistaken). Usman > asked me if Fuel were an option, especially for updating only parts of the > model. > > > Ok, but how could patching the file be much faster? The file has to be > parsed into something; why not the objects themselves? To me loading, > editing and red acing is patching and done as fast as Fuel can do things. > > Depending on syntax you may be able to scan for a string and replace using > eg the sed stream editor. But that'll only work for simple data like > strings. > > If you really need this to be faster have you considered > > - decomposing the Moose model into smaller individual components? > > - keeping the moose model to be saved in an image (possibly live running in > a server) and patching the model in the image and saving it
This is a classic problem; as models start to get large they get harder to update. If there were some way to update the file with just the parts in memory that changed since the last save, it would perform a lot faster for the end user. Of course whatever makes models large is also what tends to make them more interesting to more users, but moves inversely to the sharability. Some kind of server is needed to serve up the fuel file and use the above to dish out whatever portions of the file needed by user for the part of the model they're accessing. When they save, only the parts of the model they changed, nothing more, are sent back to the server which performs a random-access update to the fuel file in a controlled, synchronous manner. Although, even with that I guess users could still overlay other users work very easily. Some kind of concurrency control mechanism should look at the changes being made and signal a warning if that happens.
