Hi Levente, 2010/12/9 Levente Uzonyi <[email protected]>
> On Thu, 9 Dec 2010, Stéphane Ducasse wrote: > > It does! >>>>>> It seems that you did not work in VW2.5 and 3.0 and when parcels >>>>>> arrived loading was realllllly a big difference >>>>>> I do not see why this would not the same with Fuel. >>>>>> >>>>> >>>>> No I didn't, but the version number of VW is around 7.x now, so I guess >>>>> the CPUs and VMs are now several times faster. Does it really matter if it >>>>> takes 200ms or 20ms to load a package? >>>>> >>>> >>>> When I load a package with MC so far I still notice it and I would not >>>> even think about it. >>>> It should load in a unnoticeable amount of time. >>>> >>> >>> Do you think MC is "slow" because the Compiler is "slow"? >>> >> >> not totally but I do not understand why this is obligatory to compile >> everything all the time. >> Then Opal will be slower than the old comcrapiler. >> >> > I did a quick test where it took 4294ms to load RoelTyper + OCompletion > from disk (no socket creation, no network latency, etc). These packages > contain 790 methods. 5.1% of the total time was spent for compilation, > that's 219ms. The rest is used by other stuff like: > - loading the files from disk > - writing the source code to disk (sources/changes files) > - evaluating class side #initialize methods > - processing system change notifications > > Another 51ms was spent in creating 63 classes and 107ms to install the > CompiledMethods to the classes. So that's 219+51+107 = 377ms for creating > all classes and their methods. The rest is administration which can not be > avoided by binary loading. > > Even if binary loading is 10x faster than compiling the code (which I > doubt), then you save 339ms. So it would take only 3917ms to load these > packages. That would save you 7.9% of the total time. > One of the important features of the VW parcels work is that one does not write the source code to the changes file. Instead teh system has a SourceFileManager that works like a dictionary mapping file indices to source files, so instead of the two element SourceFileArray one has an arbitrarily large collection of files. When a parcel file is loaded its source file is added to the SFM which returns an index and then all the file pointers in the methods in the parcel are swizzled to refer to their sources' position in their files' index. We changed the format of file pointers so something resembling four floating-point formats so we can have lots of small files (more space for file indices) and a few large files (more space for file offsets) before one overflows into large integer file pointers. Now Igor's trailer work makes this approach feasible but one wouldn't need the funky floating-point format stuff because one could easily allocate, say, 5 bytes to the file pointer, two for the file index for a maximum of 64k source files, and 3 for the file offset for a maximum of 16m of source per parcel source file (I think splitting the 40 bits as 14:26 might be better but that's details). One needs to predetermine the size of the file pointer so that the trailer can be modified in place since changing the length of a trailer means allocating a new comp[iled method and that will be slow. If this approach is taken how much does that change your analysis? Note that not writing source to the changes file has ancilliary benefits; change recovery is now not polluted with package loads and the changes file does not grow as packages are added, only as one's changes are made. Unloading a package doesn't leave garbage in the changes files. There are downsides. Deploying a development image means deploying all the associated parcel source files as well, and for this a platform-independent Filename abstraction really helps. best Eliot > > Levente
