gee the big code package is airflowing which I have, quite conservatively,
running on #14438 images
I load filetree like this:
Gofer new
url: 'http://ss3.gemstone.com/ss/FileTree';
package: 'ConfigurationOfFileTree';
load.
((Smalltalk at: #ConfigurationOfFileTree) project version: #'stable') load.
and it never complained
let me know
On Dec 12, 2013, at 3:53 AM, GOUBIER Thierry <[email protected]> wrote:
> If you would be ready to profile a package save on your repository, it would
> be great. In the mean time, I'll make available a special gitfiletree package
> to test. Which version of Pharo you are using? 2.0 or 3.0?
>
> Regards,
>
> Thierry
>
>
> De : Pharo-dev [[email protected]] de la part de Sebastian
> Sastre [[email protected]]
> Date d'envoi : mercredi 11 décembre 2013 17:09
> À : Pharo Development List
> Objet : Re: [Pharo-dev] Tell me about your workflow
>
> ok, if saving is dumping all, then 3 is confirmed? After the first commit,
> I'd say so.
>
> about 2, I don't know. I'm available to make tests and measure results
>
> have a nice trip, keep us tuned about any progress
>
>
>
>
>
>
>
> On Dec 11, 2013, at 2:09 PM, Goubier Thierry <[email protected]> wrote:
>
>> Yes, you're right in the general case.
>>
>> But a solution to that general problem will take time to be implemented
>> (time I lack at the moment, sadly) and if the main gain is a few % because
>> it's writing the version file and the metadata for methods which are the
>> "slow" factors, then we'll have worked hard for nothing.
>>
>> If you want to help, I'd really like to see either 2- or 3- confirmed. I can
>> produce a special gitfiletree to remove writing the metadata, that you can
>> try on a large project temporary copy; if the slow writing (and reading) is
>> confirmed, then this is 3-
>>
>> (But I'm leaving on a trip tomorrow early, so I have no idea of when I'll
>> have the time to do that :( ).
>>
>> Thierry
>>
>> Le 11/12/2013 16:44, Sebastian Sastre a écrit :
>>> Without entering in details, a cause for slow package write is dumping
>>> all every time.
>>>
>>> For that strategy, we already have the image save which is magically fast.
>>>
>>> So, if we make something to scan the code and write only when it's
>>> different from what's on disk, then we would be preventing tons of
>>> redundant writes
>>>
>>> sebastian <https://about.me/sebastianconcept>
>>>
>>> o/
>>>
>>>
>>>
>>>
>>>
>>> On Dec 11, 2013, at 1:43 PM, Goubier Thierry <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>
>>>>
>>>>
>>>> Le 11/12/2013 16:27, Esteban Lorenzano a écrit :
>>>>> ah, and IMHO the problem is not about reading... is about writing (if it
>>>>> has to write the metadata each time...).
>>>>
>>>> But, personnaly, I don't know if this is the reason for the lack of
>>>> performance...
>>>>
>>>> I have three hypothesis for Sebastian problem:
>>>> 1 - Slow read time for version metadata
>>>> - Confirmed because of the 16 seconds wait time for reading the
>>>> package metadata in the repository browser.
>>>> 2 - Slow metadata write
>>>> 3 - Slow package write
>>>>
>>>> I have an implemented solution for 1-, a very easy to implement for
>>>> 2-, and none yet for 3-
>>>>
>>>> So I'd really like to check if 3- is confirmed ;)
>>>>
>>>> Thierry
>>>>
>>>>>
>>>>> Esteban
>>>>>
>>>>>
>>>>> On Wed, Dec 11, 2013 at 4:24 PM, Esteban Lorenzano
>>>>> <[email protected] <mailto:[email protected]>
>>>>> <mailto:[email protected]>> wrote:
>>>>>
>>>>> Thierry, I know there is a working version... let me search...
>>>>>
>>>>> (5 mins later)
>>>>>
>>>>>
>>>>> here:
>>>>>
>>>>> https://github.com/rjsargent/CypressReferenceImplementation
>>>>>
>>>>> Dale says Richard made a metadata-less version.
>>>>>
>>>>> We should take a look at that.
>>>>>
>>>>> Esteban
>>>>>
>>>>>
>>>>> On Wed, Dec 11, 2013 at 4:28 PM, Goubier Thierry
>>>>> <[email protected]
>>>>> <mailto:[email protected]><mailto:[email protected]>> wrote:
>>>>>
>>>>> Esteban, Sebastian,
>>>>>
>>>>> In the filetree code, you will find a format without metadata,
>>>>> but it's not in use anymore.
>>>>>
>>>>> If you use gitfiletree, it will write the metadata for
>>>>> compatibility reasons with filetree, but it will never read it
>>>>> back.
>>>>>
>>>>> I'm pushing code to make filetree robust to absence of metadata,
>>>>> but I haven't worked on it for a while.
>>>>>
>>>>> gitfiletree has solved the problem of a slow metadata read. It
>>>>> does not solve any performance problem associated with
>>>>> writing, yet.
>>>>>
>>>>> Thierry
>>>>>
>>>>> Le 11/12/2013 16:12, Esteban Lorenzano a écrit :
>>>>>
>>>>> I know there is a version of filetree without metadata (more
>>>>> compelling
>>>>> for projects that will never use other formats).
>>>>> Dale told me that there was a preview somewhere, but I
>>>>> didn't tested yet
>>>>> (lack of time) and now I cannot find the mail...
>>>>> Dale, can you re-send the link?
>>>>>
>>>>> cheers,
>>>>> Esteban
>>>>>
>>>>>
>>>>> On Wed, Dec 11, 2013 at 4:08 PM, Sebastian Sastre
>>>>> <[email protected]
>>>>> <mailto:[email protected]>
>>>>> <mailto:[email protected]>
>>>>> <mailto:sebastian@__flowingconcept.com
>>>>> <mailto:[email protected]>>> wrote:
>>>>>
>>>>> I should breath before I type, but you probably already
>>>>> got that I
>>>>> meant /redundant writes/ (not reads)...
>>>>>
>>>>>
>>>>> Anyway.. I was talking with Esteban and he mentions
>>>>> some kind of
>>>>> compatibility metadata.
>>>>>
>>>>> If I'm going to give a leap of faith to filetree repos
>>>>> to save code
>>>>> why should I care about mcz compatibility? Paying a
>>>>> toll for no
>>>>> reason is evil.
>>>>>
>>>>> Maybe we could make that optional so those who don't
>>>>> extract value
>>>>> from that feature can opt-out?
>>>>>
>>>>> sebastian <https://about.me/__sebastianconcept
>>>>> <https://about.me/sebastianconcept>>
>>>>>
>>>>>
>>>>> o/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Dec 11, 2013, at 12:44 PM, Sebastian Sastre
>>>>> <[email protected]
>>>>> <mailto:[email protected]>
>>>>> <mailto:[email protected]>
>>>>> <mailto:sebastian@__flowingconcept.com
>>>>> <mailto:[email protected]>>>
>>>>> wrote:
>>>>>
>>>>> Hi Thierry
>>>>>
>>>>> On Dec 11, 2013, at 12:43 PM, Goubier Thierry
>>>>> <[email protected]
>>>>> <mailto:[email protected]>
>>>>> <mailto:[email protected]>
>>>>> <mailto:[email protected]
>>>>> <mailto:[email protected]>__>> wrote:
>>>>>
>>>>>
>>>>> I have packages (in the order of hundreds
>>>>> of classes) and save
>>>>> delays
>>>>> and package click delays are starting to
>>>>> demand patience in a
>>>>> way that
>>>>> doesn't feel like the right path
>>>>>
>>>>>
>>>>> Which operations ? I didn't remember noticing
>>>>> much with 179
>>>>> classes on a laptop without a SSD.
>>>>>
>>>>>
>>>>> choose one. Just for clicking the package that will
>>>>> should you
>>>>> UUID, version and author I need to wait ~16
>>>>> seconds. Sounds like a
>>>>> lot of overhead for reading a small .json file.
>>>>>
>>>>> But the write is the most worrisome
>>>>>
>>>>>
>>>>> All that is with a SSD disk, otherwise save
>>>>> delays would be
>>>>> /way/ beyond
>>>>> unacceptable
>>>>>
>>>>>
>>>>> I'd like to know more, and understand the
>>>>> reason, for sure. As
>>>>> far as I know, filetree will rewrite the whole
>>>>> package to disk
>>>>> everytime... and maybe optimising that could be
>>>>> the solution.
>>>>>
>>>>>
>>>>> Well, that explains a lot. Writing all every time
>>>>> is the lazy
>>>>> thing that's okay for a prototype and temporary
>>>>> code in a proof of
>>>>> concept but that massive redundant reads certainly
>>>>> doesn't sounds
>>>>> like pro software. Specially for SSD's which has a
>>>>> limited
>>>>> quantity of writes
>>>>>
>>>>>
>>>>> Thierry
>>>>>
>>>>> sebastian
>>>>> <https://about.me/__sebastianconcept
>>>>> <https://about.me/sebastianconcept>>
>>>>>
>>>>> o/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Thierry Goubier
>>>>> CEA list
>>>>> Laboratoire des Fondations des Systèmes Temps
>>>>> Réel Embarqués
>>>>> 91191 Gif sur Yvette Cedex
>>>>> France
>>>>> Phone/Fax: +33 (0) 1 69 08 32 92
>>>>> <tel:%2B33%20%280%29%201%2069%2008%2032%2092>
>>>>> <tel:%2B33%20%280%29%201%2069%__2008%2032%2092>
>>>>> / 83 95
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Thierry Goubier
>>>>> CEA list
>>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>>> 91191 Gif sur Yvette Cedex
>>>>> France
>>>>> Phone/Fax: +33 (0) 1 69 08 32 92
>>>>> <tel:%2B33%20%280%29%201%2069%2008%2032%2092> / 83 95
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> Thierry Goubier
>>>> CEA list
>>>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>>>> 91191 Gif sur Yvette Cedex
>>>> France
>>>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
>>>
>>
>> --
>> Thierry Goubier
>> CEA list
>> Laboratoire des Fondations des Systèmes Temps Réel Embarqués
>> 91191 Gif sur Yvette Cedex
>> France
>> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95
