I can try a forced load on a 3.0 ignoring requisites for testing purposes but I'm using this: https://github.com/dalehenrich/filetree
I'm not sure what you're are using can you clarify? On Dec 13, 2013, at 11:43 AM, GOUBIER Thierry <[email protected]> wrote: > Ok. I'll try Roassal on a slow netbook to see. I don't see a factor of 10 > difference between yours and Roassal, so I'll have a look. > > Thierry > De : Pharo-dev [[email protected]] de la part de Sebastian > Sastre [[email protected]] > Date d'envoi : vendredi 13 décembre 2013 14:32 > À : Pharo Development List > Objet : Re: [Pharo-dev] Tell me about your workflow > > A bit. > > This is from today's current version (and is not all, it's only the two > biggest packages): > > (MCPackage named: 'flow') workingCopy packageInfo classes size. 363. > (MCPackage named: 'flow') workingCopy packageInfo coreMethods size. 4585. > > (MCPackage named: 'airflowing') workingCopy packageInfo classes size. 377. > (MCPackage named: 'airflowing') workingCopy packageInfo coreMethods size. > 5818. > > > > > > > > On Dec 13, 2013, at 11:25 AM, GOUBIER Thierry <[email protected]> wrote: > >> Roassal: 3493 >> >> Number of versions in the package history: 733. Size of the version file: >> 202796. >> >> Is that a lot lower than your count? >> >> Thierry >> >> De : Pharo-dev [[email protected]] de la part de Sebastian >> Sastre [[email protected]] >> Date d'envoi : vendredi 13 décembre 2013 13:34 >> À : Pharo Development List >> Objet : Re: [Pharo-dev] Tell me about your workflow >> >> how many coreMethods? >> >> >> >> >> On Dec 13, 2013, at 7:00 AM, GOUBIER Thierry <[email protected]> wrote: >> >>> Bad news. Roassal package directory has 355 entries (343 classes + a few >>> extensions) and I don't see much of a slow down (on 3.0). It's not >>> instantaneous, but with a bit of feedback, it doesn't seems long. >>> >>> I'll do some profiling. >>> >>> Thierry >>> De : Pharo-dev [[email protected]] de la part de GOUBIER >>> Thierry >>> Date d'envoi : jeudi 12 décembre 2013 17:07 >>> À : Pharo Development List >>> Objet : [PROVENANCE INTERNET] Re: [Pharo-dev] Tell me about your workflow >>> >>> Thanks for the pointers. >>> >>> I'll look at Seaside/Moose/Mondrian and Roassal, because I need code I can >>> load and save in an image without destroying the very image I use to test >>> (which would happen if I load Pharo10 stuff in a 3.0 image ;) ). >>> >>> Thierry >>> >>> De : Pharo-dev [[email protected]] de la part de Yuriy >>> Tymchuk [[email protected]] >>> Date d'envoi : jeudi 12 décembre 2013 16:24 >>> À : Pharo Development List >>> Objet : Re: [Pharo-dev] Tell me about your workflow >>> >>> So if you want something big and with a lot of commits you can use Pharo* >>> in general. Pharo10 has the most versions and Pharo30Inbox is the largest >>> one. If you want some other projects then you heve to take a look at >>> Seaside30, Mondrian, Moose, Glamour or Roassal. >>> >>> Uko >>> >>> On 12 Dec 2013, at 16:20, Yuriy Tymchuk <[email protected]> wrote: >>> >>>> Pharo10 on SmalltalkHub is humongous. You can definitely do a stress test >>>> with it :) >>>> >>>> Uko >>>> >>>> On 12 Dec 2013, at 15:43, GOUBIER Thierry <[email protected]> wrote: >>>> >>>>> I would need a large project, composed of one or more packages, with more >>>>> than 150~200 classes, which triggers the slow read and writing times >>>>> Sebastian experience. And, probably, to be complete, a long and complex >>>>> commit history in git (> 100 commits). >>>>> >>>>> I'll keep in mind the idea of creating one randomly ;) >>>>> >>>>> Thierry >>>>> >>>>> De : Pharo-dev [[email protected]] de la part de Yuriy >>>>> Tymchuk [[email protected]] >>>>> Date d'envoi : jeudi 12 décembre 2013 15:37 >>>>> À : Pharo Development List >>>>> Objet : Re: [Pharo-dev] Tell me about your workflow >>>>> >>>>> Are you interested in a package or a project? I can provide you >>>>> information based on size, etc… >>>>> >>>>> Uko >>>>> >>>>> On 12 Dec 2013, at 15:30, GOUBIER Thierry <[email protected]> wrote: >>>>> >>>>>> I gave up running gitfiletree on 1.4 :( >>>>>> >>>>>> It's possible to use gitfiletree from a 2.0 or a 3.0 image to browse >>>>>> your git repository, but testing the writing will be an issue. >>>>>> >>>>>> My best chance would be to find a large enough package I can use on 2.0 >>>>>> or 3.0 to test and profile. Does anybody has a large enough package >>>>>> which could fit? Anything that doesn't require a NDA to read it, of >>>>>> course. Is Roassal large enough? >>>>>> >>>>>> Thierry >>>>>> >>>>>> De : Pharo-dev [[email protected]] de la part de >>>>>> Sebastian Sastre [[email protected]] >>>>>> Date d'envoi : jeudi 12 décembre 2013 12:12 >>>>>> À : Pharo Development List >>>>>> Objet : Re: [Pharo-dev] Tell me about your workflow >>>>>> >>>>>> gee the big code package is airflowing which I have, quite >>>>>> conservatively, running on #14438 images >>>>>> >>>>>> I load filetree like this: >>>>>> >>>>>> Gofer new >>>>>> url: 'http://ss3.gemstone.com/ss/FileTree'; >>>>>> package: 'ConfigurationOfFileTree'; >>>>>> load. >>>>>> ((Smalltalk at: #ConfigurationOfFileTree) project version: #'stable') >>>>>> load. >>>>>> >>>>>> and it never complained >>>>>> >>>>>> let me know >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Dec 12, 2013, at 3:53 AM, GOUBIER Thierry <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> If you would be ready to profile a package save on your repository, it >>>>>>> would be great. In the mean time, I'll make available a special >>>>>>> gitfiletree package to test. Which version of Pharo you are using? 2.0 >>>>>>> or 3.0? >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Thierry >>>>>>> >>>>>>> >>>>>>> De : Pharo-dev [[email protected]] de la part de >>>>>>> Sebastian Sastre [[email protected]] >>>>>>> Date d'envoi : mercredi 11 décembre 2013 17:09 >>>>>>> À : Pharo Development List >>>>>>> Objet : Re: [Pharo-dev] Tell me about your workflow >>>>>>> >>>>>>> ok, if saving is dumping all, then 3 is confirmed? After the first >>>>>>> commit, I'd say so. >>>>>>> >>>>>>> about 2, I don't know. I'm available to make tests and measure results >>>>>>> >>>>>>> have a nice trip, keep us tuned about any progress >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Dec 11, 2013, at 2:09 PM, Goubier Thierry <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Yes, you're right in the general case. >>>>>>>> >>>>>>>> But a solution to that general problem will take time to be >>>>>>>> implemented (time I lack at the moment, sadly) and if the main gain is >>>>>>>> a few % because it's writing the version file and the metadata for >>>>>>>> methods which are the "slow" factors, then we'll have worked hard for >>>>>>>> nothing. >>>>>>>> >>>>>>>> If you want to help, I'd really like to see either 2- or 3- confirmed. >>>>>>>> I can produce a special gitfiletree to remove writing the metadata, >>>>>>>> that you can try on a large project temporary copy; if the slow >>>>>>>> writing (and reading) is confirmed, then this is 3- >>>>>>>> >>>>>>>> (But I'm leaving on a trip tomorrow early, so I have no idea of when >>>>>>>> I'll have the time to do that :( ). >>>>>>>> >>>>>>>> Thierry >>>>>>>> >>>>>>>> Le 11/12/2013 16:44, Sebastian Sastre a écrit : >>>>>>>>> Without entering in details, a cause for slow package write is dumping >>>>>>>>> all every time. >>>>>>>>> >>>>>>>>> For that strategy, we already have the image save which is magically >>>>>>>>> fast. >>>>>>>>> >>>>>>>>> So, if we make something to scan the code and write only when it's >>>>>>>>> different from what's on disk, then we would be preventing tons of >>>>>>>>> redundant writes >>>>>>>>> >>>>>>>>> sebastian <https://about.me/sebastianconcept> >>>>>>>>> >>>>>>>>> o/ >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Dec 11, 2013, at 1:43 PM, Goubier Thierry <[email protected] >>>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Le 11/12/2013 16:27, Esteban Lorenzano a écrit : >>>>>>>>>>> ah, and IMHO the problem is not about reading... is about writing >>>>>>>>>>> (if it >>>>>>>>>>> has to write the metadata each time...). >>>>>>>>>> >>>>>>>>>> But, personnaly, I don't know if this is the reason for the lack of >>>>>>>>>> performance... >>>>>>>>>> >>>>>>>>>> I have three hypothesis for Sebastian problem: >>>>>>>>>> 1 - Slow read time for version metadata >>>>>>>>>> - Confirmed because of the 16 seconds wait time for reading the >>>>>>>>>> package metadata in the repository browser. >>>>>>>>>> 2 - Slow metadata write >>>>>>>>>> 3 - Slow package write >>>>>>>>>> >>>>>>>>>> I have an implemented solution for 1-, a very easy to implement for >>>>>>>>>> 2-, and none yet for 3- >>>>>>>>>> >>>>>>>>>> So I'd really like to check if 3- is confirmed ;) >>>>>>>>>> >>>>>>>>>> Thierry >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Esteban >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Dec 11, 2013 at 4:24 PM, Esteban Lorenzano >>>>>>>>>>> <[email protected] <mailto:[email protected]> >>>>>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>>>>> >>>>>>>>>>> Thierry, I know there is a working version... let me search... >>>>>>>>>>> >>>>>>>>>>> (5 mins later) >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> here: >>>>>>>>>>> >>>>>>>>>>> https://github.com/rjsargent/CypressReferenceImplementation >>>>>>>>>>> >>>>>>>>>>> Dale says Richard made a metadata-less version. >>>>>>>>>>> >>>>>>>>>>> We should take a look at that. >>>>>>>>>>> >>>>>>>>>>> Esteban >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Dec 11, 2013 at 4:28 PM, Goubier Thierry >>>>>>>>>>> <[email protected] >>>>>>>>>>> <mailto:[email protected]><mailto:[email protected]>> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Esteban, Sebastian, >>>>>>>>>>> >>>>>>>>>>> In the filetree code, you will find a format without metadata, >>>>>>>>>>> but it's not in use anymore. >>>>>>>>>>> >>>>>>>>>>> If you use gitfiletree, it will write the metadata for >>>>>>>>>>> compatibility reasons with filetree, but it will never read it >>>>>>>>>>> back. >>>>>>>>>>> >>>>>>>>>>> I'm pushing code to make filetree robust to absence of >>>>>>>>>>> metadata, >>>>>>>>>>> but I haven't worked on it for a while. >>>>>>>>>>> >>>>>>>>>>> gitfiletree has solved the problem of a slow metadata read. It >>>>>>>>>>> does not solve any performance problem associated with >>>>>>>>>>> writing, yet. >>>>>>>>>>> >>>>>>>>>>> Thierry >>>>>>>>>>> >>>>>>>>>>> Le 11/12/2013 16:12, Esteban Lorenzano a écrit : >>>>>>>>>>> >>>>>>>>>>> I know there is a version of filetree without metadata >>>>>>>>>>> (more >>>>>>>>>>> compelling >>>>>>>>>>> for projects that will never use other formats). >>>>>>>>>>> Dale told me that there was a preview somewhere, but I >>>>>>>>>>> didn't tested yet >>>>>>>>>>> (lack of time) and now I cannot find the mail... >>>>>>>>>>> Dale, can you re-send the link? >>>>>>>>>>> >>>>>>>>>>> cheers, >>>>>>>>>>> Esteban >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Dec 11, 2013 at 4:08 PM, Sebastian Sastre >>>>>>>>>>> <[email protected] >>>>>>>>>>> <mailto:[email protected]> >>>>>>>>>>> <mailto:[email protected]> >>>>>>>>>>> <mailto:sebastian@__flowingconcept.com >>>>>>>>>>> <mailto:[email protected]>>> wrote: >>>>>>>>>>> >>>>>>>>>>> I should breath before I type, but you probably >>>>>>>>>>> already >>>>>>>>>>> got that I >>>>>>>>>>> meant /redundant writes/ (not reads)... >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Anyway.. I was talking with Esteban and he mentions >>>>>>>>>>> some kind of >>>>>>>>>>> compatibility metadata. >>>>>>>>>>> >>>>>>>>>>> If I'm going to give a leap of faith to filetree >>>>>>>>>>> repos >>>>>>>>>>> to save code >>>>>>>>>>> why should I care about mcz compatibility? Paying a >>>>>>>>>>> toll for no >>>>>>>>>>> reason is evil. >>>>>>>>>>> >>>>>>>>>>> Maybe we could make that optional so those who don't >>>>>>>>>>> extract value >>>>>>>>>>> from that feature can opt-out? >>>>>>>>>>> >>>>>>>>>>> sebastian <https://about.me/__sebastianconcept >>>>>>>>>>> <https://about.me/sebastianconcept>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> o/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Dec 11, 2013, at 12:44 PM, Sebastian Sastre >>>>>>>>>>> <[email protected] >>>>>>>>>>> <mailto:[email protected]> >>>>>>>>>>> <mailto:[email protected]> >>>>>>>>>>> <mailto:sebastian@__flowingconcept.com >>>>>>>>>>> <mailto:[email protected]>>> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Thierry >>>>>>>>>>> >>>>>>>>>>> On Dec 11, 2013, at 12:43 PM, Goubier Thierry >>>>>>>>>>> <[email protected] >>>>>>>>>>> <mailto:[email protected]> >>>>>>>>>>> <mailto:[email protected]> >>>>>>>>>>> <mailto:[email protected] >>>>>>>>>>> <mailto:[email protected]>__>> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I have packages (in the order of hundreds >>>>>>>>>>> of classes) and save >>>>>>>>>>> delays >>>>>>>>>>> and package click delays are starting to >>>>>>>>>>> demand patience in a >>>>>>>>>>> way that >>>>>>>>>>> doesn't feel like the right path >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Which operations ? I didn't remember noticing >>>>>>>>>>> much with 179 >>>>>>>>>>> classes on a laptop without a SSD. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> choose one. Just for clicking the package that >>>>>>>>>>> will >>>>>>>>>>> should you >>>>>>>>>>> UUID, version and author I need to wait ~16 >>>>>>>>>>> seconds. Sounds like a >>>>>>>>>>> lot of overhead for reading a small .json file. >>>>>>>>>>> >>>>>>>>>>> But the write is the most worrisome >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> All that is with a SSD disk, otherwise >>>>>>>>>>> save >>>>>>>>>>> delays would be >>>>>>>>>>> /way/ beyond >>>>>>>>>>> unacceptable >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I'd like to know more, and understand the >>>>>>>>>>> reason, for sure. As >>>>>>>>>>> far as I know, filetree will rewrite the >>>>>>>>>>> whole >>>>>>>>>>> package to disk >>>>>>>>>>> everytime... and maybe optimising that could >>>>>>>>>>> be >>>>>>>>>>> the solution. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Well, that explains a lot. Writing all every time >>>>>>>>>>> is the lazy >>>>>>>>>>> thing that's okay for a prototype and temporary >>>>>>>>>>> code in a proof of >>>>>>>>>>> concept but that massive redundant reads >>>>>>>>>>> certainly >>>>>>>>>>> doesn't sounds >>>>>>>>>>> like pro software. Specially for SSD's which has >>>>>>>>>>> a >>>>>>>>>>> limited >>>>>>>>>>> quantity of writes >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thierry >>>>>>>>>>> >>>>>>>>>>> sebastian >>>>>>>>>>> <https://about.me/__sebastianconcept >>>>>>>>>>> <https://about.me/sebastianconcept>> >>>>>>>>>>> >>>>>>>>>>> o/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Thierry Goubier >>>>>>>>>>> CEA list >>>>>>>>>>> Laboratoire des Fondations des Systèmes Temps >>>>>>>>>>> Réel Embarqués >>>>>>>>>>> 91191 Gif sur Yvette Cedex >>>>>>>>>>> France >>>>>>>>>>> Phone/Fax: +33 (0) 1 69 08 32 92 >>>>>>>>>>> <tel:%2B33%20%280%29%201%2069%2008%2032%2092> >>>>>>>>>>> >>>>>>>>>>> <tel:%2B33%20%280%29%201%2069%__2008%2032%2092> >>>>>>>>>>> / 83 95 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Thierry Goubier >>>>>>>>>>> CEA list >>>>>>>>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués >>>>>>>>>>> 91191 Gif sur Yvette Cedex >>>>>>>>>>> France >>>>>>>>>>> Phone/Fax: +33 (0) 1 69 08 32 92 >>>>>>>>>>> <tel:%2B33%20%280%29%201%2069%2008%2032%2092> / 83 95 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Thierry Goubier >>>>>>>>>> CEA list >>>>>>>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués >>>>>>>>>> 91191 Gif sur Yvette Cedex >>>>>>>>>> France >>>>>>>>>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95 >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Thierry Goubier >>>>>>>> CEA list >>>>>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués >>>>>>>> 91191 Gif sur Yvette Cedex >>>>>>>> France >>>>>>>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
