Pharo10 on SmalltalkHub is humongous. You can definitely do a stress test with it :)
Uko On 12 Dec 2013, at 15:43, GOUBIER Thierry <[email protected]> wrote: > I would need a large project, composed of one or more packages, with more > than 150~200 classes, which triggers the slow read and writing times > Sebastian experience. And, probably, to be complete, a long and complex > commit history in git (> 100 commits). > > I'll keep in mind the idea of creating one randomly ;) > > Thierry > > De : Pharo-dev [[email protected]] de la part de Yuriy > Tymchuk [[email protected]] > Date d'envoi : jeudi 12 décembre 2013 15:37 > À : Pharo Development List > Objet : Re: [Pharo-dev] Tell me about your workflow > > Are you interested in a package or a project? I can provide you information > based on size, etc… > > Uko > > On 12 Dec 2013, at 15:30, GOUBIER Thierry <[email protected]> wrote: > >> I gave up running gitfiletree on 1.4 :( >> >> It's possible to use gitfiletree from a 2.0 or a 3.0 image to browse your >> git repository, but testing the writing will be an issue. >> >> My best chance would be to find a large enough package I can use on 2.0 or >> 3.0 to test and profile. Does anybody has a large enough package which could >> fit? Anything that doesn't require a NDA to read it, of course. Is Roassal >> large enough? >> >> Thierry >> >> De : Pharo-dev [[email protected]] de la part de Sebastian >> Sastre [[email protected]] >> Date d'envoi : jeudi 12 décembre 2013 12:12 >> À : Pharo Development List >> Objet : Re: [Pharo-dev] Tell me about your workflow >> >> gee the big code package is airflowing which I have, quite conservatively, >> running on #14438 images >> >> I load filetree like this: >> >> Gofer new >> url: 'http://ss3.gemstone.com/ss/FileTree'; >> package: 'ConfigurationOfFileTree'; >> load. >> ((Smalltalk at: #ConfigurationOfFileTree) project version: #'stable') load. >> >> and it never complained >> >> let me know >> >> >> >> >> >> On Dec 12, 2013, at 3:53 AM, GOUBIER Thierry <[email protected]> wrote: >> >>> If you would be ready to profile a package save on your repository, it >>> would be great. In the mean time, I'll make available a special gitfiletree >>> package to test. Which version of Pharo you are using? 2.0 or 3.0? >>> >>> Regards, >>> >>> Thierry >>> >>> >>> De : Pharo-dev [[email protected]] de la part de Sebastian >>> Sastre [[email protected]] >>> Date d'envoi : mercredi 11 décembre 2013 17:09 >>> À : Pharo Development List >>> Objet : Re: [Pharo-dev] Tell me about your workflow >>> >>> ok, if saving is dumping all, then 3 is confirmed? After the first commit, >>> I'd say so. >>> >>> about 2, I don't know. I'm available to make tests and measure results >>> >>> have a nice trip, keep us tuned about any progress >>> >>> >>> >>> >>> >>> >>> >>> On Dec 11, 2013, at 2:09 PM, Goubier Thierry <[email protected]> wrote: >>> >>>> Yes, you're right in the general case. >>>> >>>> But a solution to that general problem will take time to be implemented >>>> (time I lack at the moment, sadly) and if the main gain is a few % because >>>> it's writing the version file and the metadata for methods which are the >>>> "slow" factors, then we'll have worked hard for nothing. >>>> >>>> If you want to help, I'd really like to see either 2- or 3- confirmed. I >>>> can produce a special gitfiletree to remove writing the metadata, that you >>>> can try on a large project temporary copy; if the slow writing (and >>>> reading) is confirmed, then this is 3- >>>> >>>> (But I'm leaving on a trip tomorrow early, so I have no idea of when I'll >>>> have the time to do that :( ). >>>> >>>> Thierry >>>> >>>> Le 11/12/2013 16:44, Sebastian Sastre a écrit : >>>>> Without entering in details, a cause for slow package write is dumping >>>>> all every time. >>>>> >>>>> For that strategy, we already have the image save which is magically fast. >>>>> >>>>> So, if we make something to scan the code and write only when it's >>>>> different from what's on disk, then we would be preventing tons of >>>>> redundant writes >>>>> >>>>> sebastian <https://about.me/sebastianconcept> >>>>> >>>>> o/ >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Dec 11, 2013, at 1:43 PM, Goubier Thierry <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>>> >>>>>> >>>>>> Le 11/12/2013 16:27, Esteban Lorenzano a écrit : >>>>>>> ah, and IMHO the problem is not about reading... is about writing (if it >>>>>>> has to write the metadata each time...). >>>>>> >>>>>> But, personnaly, I don't know if this is the reason for the lack of >>>>>> performance... >>>>>> >>>>>> I have three hypothesis for Sebastian problem: >>>>>> 1 - Slow read time for version metadata >>>>>> - Confirmed because of the 16 seconds wait time for reading the >>>>>> package metadata in the repository browser. >>>>>> 2 - Slow metadata write >>>>>> 3 - Slow package write >>>>>> >>>>>> I have an implemented solution for 1-, a very easy to implement for >>>>>> 2-, and none yet for 3- >>>>>> >>>>>> So I'd really like to check if 3- is confirmed ;) >>>>>> >>>>>> Thierry >>>>>> >>>>>>> >>>>>>> Esteban >>>>>>> >>>>>>> >>>>>>> On Wed, Dec 11, 2013 at 4:24 PM, Esteban Lorenzano >>>>>>> <[email protected] <mailto:[email protected]> >>>>>>> <mailto:[email protected]>> wrote: >>>>>>> >>>>>>> Thierry, I know there is a working version... let me search... >>>>>>> >>>>>>> (5 mins later) >>>>>>> >>>>>>> >>>>>>> here: >>>>>>> >>>>>>> https://github.com/rjsargent/CypressReferenceImplementation >>>>>>> >>>>>>> Dale says Richard made a metadata-less version. >>>>>>> >>>>>>> We should take a look at that. >>>>>>> >>>>>>> Esteban >>>>>>> >>>>>>> >>>>>>> On Wed, Dec 11, 2013 at 4:28 PM, Goubier Thierry >>>>>>> <[email protected] >>>>>>> <mailto:[email protected]><mailto:[email protected]>> wrote: >>>>>>> >>>>>>> Esteban, Sebastian, >>>>>>> >>>>>>> In the filetree code, you will find a format without metadata, >>>>>>> but it's not in use anymore. >>>>>>> >>>>>>> If you use gitfiletree, it will write the metadata for >>>>>>> compatibility reasons with filetree, but it will never read it >>>>>>> back. >>>>>>> >>>>>>> I'm pushing code to make filetree robust to absence of metadata, >>>>>>> but I haven't worked on it for a while. >>>>>>> >>>>>>> gitfiletree has solved the problem of a slow metadata read. It >>>>>>> does not solve any performance problem associated with >>>>>>> writing, yet. >>>>>>> >>>>>>> Thierry >>>>>>> >>>>>>> Le 11/12/2013 16:12, Esteban Lorenzano a écrit : >>>>>>> >>>>>>> I know there is a version of filetree without metadata (more >>>>>>> compelling >>>>>>> for projects that will never use other formats). >>>>>>> Dale told me that there was a preview somewhere, but I >>>>>>> didn't tested yet >>>>>>> (lack of time) and now I cannot find the mail... >>>>>>> Dale, can you re-send the link? >>>>>>> >>>>>>> cheers, >>>>>>> Esteban >>>>>>> >>>>>>> >>>>>>> On Wed, Dec 11, 2013 at 4:08 PM, Sebastian Sastre >>>>>>> <[email protected] >>>>>>> <mailto:[email protected]> >>>>>>> <mailto:[email protected]> >>>>>>> <mailto:sebastian@__flowingconcept.com >>>>>>> <mailto:[email protected]>>> wrote: >>>>>>> >>>>>>> I should breath before I type, but you probably already >>>>>>> got that I >>>>>>> meant /redundant writes/ (not reads)... >>>>>>> >>>>>>> >>>>>>> Anyway.. I was talking with Esteban and he mentions >>>>>>> some kind of >>>>>>> compatibility metadata. >>>>>>> >>>>>>> If I'm going to give a leap of faith to filetree repos >>>>>>> to save code >>>>>>> why should I care about mcz compatibility? Paying a >>>>>>> toll for no >>>>>>> reason is evil. >>>>>>> >>>>>>> Maybe we could make that optional so those who don't >>>>>>> extract value >>>>>>> from that feature can opt-out? >>>>>>> >>>>>>> sebastian <https://about.me/__sebastianconcept >>>>>>> <https://about.me/sebastianconcept>> >>>>>>> >>>>>>> >>>>>>> o/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Dec 11, 2013, at 12:44 PM, Sebastian Sastre >>>>>>> <[email protected] >>>>>>> <mailto:[email protected]> >>>>>>> <mailto:[email protected]> >>>>>>> <mailto:sebastian@__flowingconcept.com >>>>>>> <mailto:[email protected]>>> >>>>>>> wrote: >>>>>>> >>>>>>> Hi Thierry >>>>>>> >>>>>>> On Dec 11, 2013, at 12:43 PM, Goubier Thierry >>>>>>> <[email protected] >>>>>>> <mailto:[email protected]> >>>>>>> <mailto:[email protected]> >>>>>>> <mailto:[email protected] >>>>>>> <mailto:[email protected]>__>> wrote: >>>>>>> >>>>>>> >>>>>>> I have packages (in the order of hundreds >>>>>>> of classes) and save >>>>>>> delays >>>>>>> and package click delays are starting to >>>>>>> demand patience in a >>>>>>> way that >>>>>>> doesn't feel like the right path >>>>>>> >>>>>>> >>>>>>> Which operations ? I didn't remember noticing >>>>>>> much with 179 >>>>>>> classes on a laptop without a SSD. >>>>>>> >>>>>>> >>>>>>> choose one. Just for clicking the package that will >>>>>>> should you >>>>>>> UUID, version and author I need to wait ~16 >>>>>>> seconds. Sounds like a >>>>>>> lot of overhead for reading a small .json file. >>>>>>> >>>>>>> But the write is the most worrisome >>>>>>> >>>>>>> >>>>>>> All that is with a SSD disk, otherwise save >>>>>>> delays would be >>>>>>> /way/ beyond >>>>>>> unacceptable >>>>>>> >>>>>>> >>>>>>> I'd like to know more, and understand the >>>>>>> reason, for sure. As >>>>>>> far as I know, filetree will rewrite the whole >>>>>>> package to disk >>>>>>> everytime... and maybe optimising that could be >>>>>>> the solution. >>>>>>> >>>>>>> >>>>>>> Well, that explains a lot. Writing all every time >>>>>>> is the lazy >>>>>>> thing that's okay for a prototype and temporary >>>>>>> code in a proof of >>>>>>> concept but that massive redundant reads certainly >>>>>>> doesn't sounds >>>>>>> like pro software. Specially for SSD's which has a >>>>>>> limited >>>>>>> quantity of writes >>>>>>> >>>>>>> >>>>>>> Thierry >>>>>>> >>>>>>> sebastian >>>>>>> <https://about.me/__sebastianconcept >>>>>>> <https://about.me/sebastianconcept>> >>>>>>> >>>>>>> o/ >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thierry Goubier >>>>>>> CEA list >>>>>>> Laboratoire des Fondations des Systèmes Temps >>>>>>> Réel Embarqués >>>>>>> 91191 Gif sur Yvette Cedex >>>>>>> France >>>>>>> Phone/Fax: +33 (0) 1 69 08 32 92 >>>>>>> <tel:%2B33%20%280%29%201%2069%2008%2032%2092> >>>>>>> <tel:%2B33%20%280%29%201%2069%__2008%2032%2092> >>>>>>> / 83 95 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Thierry Goubier >>>>>>> CEA list >>>>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués >>>>>>> 91191 Gif sur Yvette Cedex >>>>>>> France >>>>>>> Phone/Fax: +33 (0) 1 69 08 32 92 >>>>>>> <tel:%2B33%20%280%29%201%2069%2008%2032%2092> / 83 95 >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> Thierry Goubier >>>>>> CEA list >>>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués >>>>>> 91191 Gif sur Yvette Cedex >>>>>> France >>>>>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95 >>>>> >>>> >>>> -- >>>> Thierry Goubier >>>> CEA list >>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués >>>> 91191 Gif sur Yvette Cedex >>>> France >>>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
