ok, if saving is dumping all, then 3 is confirmed? After the first commit, I'd say so.
about 2, I don't know. I'm available to make tests and measure results have a nice trip, keep us tuned about any progress On Dec 11, 2013, at 2:09 PM, Goubier Thierry <[email protected]> wrote: > Yes, you're right in the general case. > > But a solution to that general problem will take time to be implemented (time > I lack at the moment, sadly) and if the main gain is a few % because it's > writing the version file and the metadata for methods which are the "slow" > factors, then we'll have worked hard for nothing. > > If you want to help, I'd really like to see either 2- or 3- confirmed. I can > produce a special gitfiletree to remove writing the metadata, that you can > try on a large project temporary copy; if the slow writing (and reading) is > confirmed, then this is 3- > > (But I'm leaving on a trip tomorrow early, so I have no idea of when I'll > have the time to do that :( ). > > Thierry > > Le 11/12/2013 16:44, Sebastian Sastre a écrit : >> Without entering in details, a cause for slow package write is dumping >> all every time. >> >> For that strategy, we already have the image save which is magically fast. >> >> So, if we make something to scan the code and write only when it's >> different from what's on disk, then we would be preventing tons of >> redundant writes >> >> sebastian <https://about.me/sebastianconcept> >> >> o/ >> >> >> >> >> >> On Dec 11, 2013, at 1:43 PM, Goubier Thierry <[email protected] >> <mailto:[email protected]>> wrote: >> >>> >>> >>> Le 11/12/2013 16:27, Esteban Lorenzano a écrit : >>>> ah, and IMHO the problem is not about reading... is about writing (if it >>>> has to write the metadata each time...). >>> >>> But, personnaly, I don't know if this is the reason for the lack of >>> performance... >>> >>> I have three hypothesis for Sebastian problem: >>> 1 - Slow read time for version metadata >>> - Confirmed because of the 16 seconds wait time for reading the >>> package metadata in the repository browser. >>> 2 - Slow metadata write >>> 3 - Slow package write >>> >>> I have an implemented solution for 1-, a very easy to implement for >>> 2-, and none yet for 3- >>> >>> So I'd really like to check if 3- is confirmed ;) >>> >>> Thierry >>> >>>> >>>> Esteban >>>> >>>> >>>> On Wed, Dec 11, 2013 at 4:24 PM, Esteban Lorenzano >>>> <[email protected] <mailto:[email protected]> >>>> <mailto:[email protected]>> wrote: >>>> >>>> Thierry, I know there is a working version... let me search... >>>> >>>> (5 mins later) >>>> >>>> >>>> here: >>>> >>>> https://github.com/rjsargent/CypressReferenceImplementation >>>> >>>> Dale says Richard made a metadata-less version. >>>> >>>> We should take a look at that. >>>> >>>> Esteban >>>> >>>> >>>> On Wed, Dec 11, 2013 at 4:28 PM, Goubier Thierry >>>> <[email protected] >>>> <mailto:[email protected]><mailto:[email protected]>> wrote: >>>> >>>> Esteban, Sebastian, >>>> >>>> In the filetree code, you will find a format without metadata, >>>> but it's not in use anymore. >>>> >>>> If you use gitfiletree, it will write the metadata for >>>> compatibility reasons with filetree, but it will never read it >>>> back. >>>> >>>> I'm pushing code to make filetree robust to absence of metadata, >>>> but I haven't worked on it for a while. >>>> >>>> gitfiletree has solved the problem of a slow metadata read. It >>>> does not solve any performance problem associated with >>>> writing, yet. >>>> >>>> Thierry >>>> >>>> Le 11/12/2013 16:12, Esteban Lorenzano a écrit : >>>> >>>> I know there is a version of filetree without metadata (more >>>> compelling >>>> for projects that will never use other formats). >>>> Dale told me that there was a preview somewhere, but I >>>> didn't tested yet >>>> (lack of time) and now I cannot find the mail... >>>> Dale, can you re-send the link? >>>> >>>> cheers, >>>> Esteban >>>> >>>> >>>> On Wed, Dec 11, 2013 at 4:08 PM, Sebastian Sastre >>>> <[email protected] >>>> <mailto:[email protected]> >>>> <mailto:[email protected]> >>>> <mailto:sebastian@__flowingconcept.com >>>> <mailto:[email protected]>>> wrote: >>>> >>>> I should breath before I type, but you probably already >>>> got that I >>>> meant /redundant writes/ (not reads)... >>>> >>>> >>>> Anyway.. I was talking with Esteban and he mentions >>>> some kind of >>>> compatibility metadata. >>>> >>>> If I'm going to give a leap of faith to filetree repos >>>> to save code >>>> why should I care about mcz compatibility? Paying a >>>> toll for no >>>> reason is evil. >>>> >>>> Maybe we could make that optional so those who don't >>>> extract value >>>> from that feature can opt-out? >>>> >>>> sebastian <https://about.me/__sebastianconcept >>>> <https://about.me/sebastianconcept>> >>>> >>>> >>>> o/ >>>> >>>> >>>> >>>> >>>> >>>> On Dec 11, 2013, at 12:44 PM, Sebastian Sastre >>>> <[email protected] >>>> <mailto:[email protected]> >>>> <mailto:[email protected]> >>>> <mailto:sebastian@__flowingconcept.com >>>> <mailto:[email protected]>>> >>>> wrote: >>>> >>>> Hi Thierry >>>> >>>> On Dec 11, 2013, at 12:43 PM, Goubier Thierry >>>> <[email protected] >>>> <mailto:[email protected]> >>>> <mailto:[email protected]> >>>> <mailto:[email protected] >>>> <mailto:[email protected]>__>> wrote: >>>> >>>> >>>> I have packages (in the order of hundreds >>>> of classes) and save >>>> delays >>>> and package click delays are starting to >>>> demand patience in a >>>> way that >>>> doesn't feel like the right path >>>> >>>> >>>> Which operations ? I didn't remember noticing >>>> much with 179 >>>> classes on a laptop without a SSD. >>>> >>>> >>>> choose one. Just for clicking the package that will >>>> should you >>>> UUID, version and author I need to wait ~16 >>>> seconds. Sounds like a >>>> lot of overhead for reading a small .json file. >>>> >>>> But the write is the most worrisome >>>> >>>> >>>> All that is with a SSD disk, otherwise save >>>> delays would be >>>> /way/ beyond >>>> unacceptable >>>> >>>> >>>> I'd like to know more, and understand the >>>> reason, for sure. As >>>> far as I know, filetree will rewrite the whole >>>> package to disk >>>> everytime... and maybe optimising that could be >>>> the solution. >>>> >>>> >>>> Well, that explains a lot. Writing all every time >>>> is the lazy >>>> thing that's okay for a prototype and temporary >>>> code in a proof of >>>> concept but that massive redundant reads certainly >>>> doesn't sounds >>>> like pro software. Specially for SSD's which has a >>>> limited >>>> quantity of writes >>>> >>>> >>>> Thierry >>>> >>>> sebastian >>>> <https://about.me/__sebastianconcept >>>> <https://about.me/sebastianconcept>> >>>> >>>> o/ >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Thierry Goubier >>>> CEA list >>>> Laboratoire des Fondations des Systèmes Temps >>>> Réel Embarqués >>>> 91191 Gif sur Yvette Cedex >>>> France >>>> Phone/Fax: +33 (0) 1 69 08 32 92 >>>> <tel:%2B33%20%280%29%201%2069%2008%2032%2092> >>>> <tel:%2B33%20%280%29%201%2069%__2008%2032%2092> >>>> / 83 95 >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Thierry Goubier >>>> CEA list >>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués >>>> 91191 Gif sur Yvette Cedex >>>> France >>>> Phone/Fax: +33 (0) 1 69 08 32 92 >>>> <tel:%2B33%20%280%29%201%2069%2008%2032%2092> / 83 95 >>>> >>>> >>>> >>> >>> -- >>> Thierry Goubier >>> CEA list >>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués >>> 91191 Gif sur Yvette Cedex >>> France >>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95 >> > > -- > Thierry Goubier > CEA list > Laboratoire des Fondations des Systèmes Temps Réel Embarqués > 91191 Gif sur Yvette Cedex > France > Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
