> On Sep 6, 2016, at 12:25 PM, Nicolai Hess <nicolaih...@gmail.com> wrote: > > > > 2016-09-06 11:47 GMT+02:00 Henrik Nergaard <henrik.nerga...@uia.no>: >> Rewriting the implementation of #initSelectorTable to: >> >> >> >> SelectorTable := WeakSet withAll: (CompiledMethod allInstances collect: [:m >> | m selector] as: IdentitySet). >> >> >> >> Makes it 20ms faster, so it looks like there is some overhead when using >> WeakSet compared to first use a normal set to remove duplicates. >> >> >> >> Another possibility could be to store the selectors in an IdentityDictionary >> as #selector->count which keep track of how many users of a selector there >> is itself, instead of using a WeakArray. >> >> >> >> Best regards, >> >> Henrik >> > > Maybe we can just move the rebuilding to the image shutdown instead of the > image startup ?
That's a much better idea. But only if the image snapshots. There is no point doing it on an exit without saving. And just to drive home the point, 70ms is an age in a web server. This kind of thing really damages Pharo performance in web servers. One wants startup times in the few tens of milliseconds or better. > > >> >> >> From: Pharo-dev [mailto:pharo-dev-boun...@lists.pharo.org] On Behalf Of >> Nicolai Hess >> Sent: Monday, September 5, 2016 7:52 PM >> To: Pharo Development List <pharo-dev@lists.pharo.org> >> Subject: Re: [Pharo-dev] Cycles lost on startup >> >> >> >> >> >> >> >> 2016-09-05 18:37 GMT+02:00 Eliot Miranda <eliot.mira...@gmail.com>: >> >> Hi All, >> >> >> >> we are simulating a full block Pharo image derived from Pharo 6. On >> startup we see a very sad loss of cycles due to a very slow rebuild of the >> selector table in Symbol class. Why is this done at each startup? Here is >> the stack: >> >> >> >> 16r107154 M ByteString class(ProtoObject)>identityHash 16r786AE8: a(n) >> ByteString class >> >> 16r10716C M ByteString class(Object)>hash 16r786AE8: a(n) ByteString class >> >> 16r10718C M ByteSymbol(String)>hash 16rE6A5C8: a(n) ByteSymbol >> >> 16r1071B0 M WeakSet>scanForEmptySlotFor: 16r26B968: a(n) WeakSet >> >> 16r1071DC M WeakSet>noCheckNoGrowFillFrom: 16r26B968: a(n) WeakSet >> >> 16r1071FC M WeakSet>growTo: 16r26B968: a(n) WeakSet >> >> 16r10721C M WeakSet>grow 16r26B968: a(n) WeakSet >> >> 16r107234 M WeakSet(HashedCollection)>fullCheck 16r26B968: a(n) WeakSet >> >> 16r10724C M WeakSet(HashedCollection)>atNewIndex:put: 16r26B968: a(n) >> WeakSet >> >> 16r107274 M WeakSet>add: 16r26B968: a(n) WeakSet >> >> 16r107294 M [] in Symbol class>(nil) 16r1D319A8: a(n) Symbol class >> >> 16r1072B8 M Array(SequenceableCollection)>do: 16r3C17370: a(n) Array >> >> 16r1072E8 I CompiledMethod class(Behavior)>allInstancesDo: 16r164E260: >> a(n) CompiledMethod class >> >> 16r10730C I Symbol class>initSelectorTable 16r1D319A8: a(n) Symbol class >> >> 16r10732C I Symbol class>selectorTable 16r1D319A8: a(n) Symbol class >> >> 16r107350 I Symbol class>internSelector: 16r1D319A8: a(n) Symbol class >> >> 16r10737C I CompiledMethod>selector: 16r26AB58: a(n) CompiledMethod >> >> 16r1073A4 I IRMethod>generate: 16r25A030: a(n) IRMethod >> >> 16r1073CC I FFICalloutMethodBuilder>generateMethodFromSpec: 16r255580: >> a(n) FFICalloutMethodBuilder >> >> 16r1073F0 I FFICalloutMethodBuilder>generate 16r255580: a(n) >> FFICalloutMethodBuilder >> >> 16r1030C8 I FFICalloutMethodBuilder>build: 16r255580: a(n) >> FFICalloutMethodBuilder >> >> 16r1030F4 I FFICalloutAPI>function:module: 16r255568: a(n) FFICalloutAPI >> >> 16r10311C I UnixEnvironment(Object)>ffiCall:module: 16r3816180: a(n) >> UnixEnvironment >> >> 16r103144 I UnixEnvironment(OSEnvironment)>getEnv: 16r3816180: a(n) >> UnixEnvironment >> >> 16r103168 I UnixEnvironment(OSEnvironment)>at:ifAbsent: 16r3816180: a(n) >> UnixEnvironment >> >> 16r10318C M [] in MacOSResolver>(nil) 16r241170: a(n) MacOSResolver >> >> 16r1031A4 M FullBlockClosure(BlockClosure)>on:do: 16r2553F0: a(n) >> FullBlockClosure >> >> 16r1031D0 I >> MacOSResolver(PlatformResolver)>directoryFromEnvVariableNamed:or: 16r241170: >> a(n) MacOSResolver >> >> 16r1031F8 I MacOSResolver(PlatformResolver)>directoryFromEnvVariableNamed: >> 16r241170: a(n) MacOSResolver >> >> 16r10321C I MacOSResolver>home 16r241170: a(n) MacOSResolver >> >> 16r10323C I MacOSResolver>userLibrary 16r241170: a(n) MacOSResolver >> >> 16r103254 M MacOSResolver>userApplicationSupport 16r241170: a(n) >> MacOSResolver >> >> 16r10326C M MacOSResolver(FileSystemResolver)>resolve: 16r241170: a(n) >> MacOSResolver >> >> 16r103288 M SystemResolver(FileSystemResolver)>unknownOrigin: 16r240D30: >> a(n) SystemResolver >> >> 16r1032A4 M SystemResolver(FileSystemResolver)>resolve: 16r240D30: a(n) >> SystemResolver >> >> 16r1032C4 M InteractiveResolver>unknownOrigin: 16r240CF0: a(n) >> InteractiveResolver >> >> 16r1032E4 M [] in InteractiveResolver>(nil) 16r240CF0: a(n) >> InteractiveResolver >> >> 16r103300 M IdentityDictionary(Dictionary)>at:ifAbsent: 16r240D00: a(n) >> IdentityDictionary >> >> 16r103320 M InteractiveResolver>resolve: 16r240CF0: a(n) >> InteractiveResolver >> >> 16r10333C M FileLocator>resolve 16r255258: a(n) FileLocator >> >> 16r103354 M FileLocator(AbstractFileReference)>exists 16r255258: a(n) >> FileLocator >> >> 16r10337C M [] in PharoFilesOpener>(nil) 16rE79FD8: a(n) PharoFilesOpener >> >> 16r103398 M [] in PharoFilesOpener>(nil) 16rE79FD8: a(n) PharoFilesOpener >> >> 16r1033B0 M FullBlockClosure(BlockClosure)>on:do: 16r255190: a(n) >> FullBlockClosure >> >> 16r1033D0 M FullBlockClosure(BlockClosure)>ifError: 16r255190: a(n) >> FullBlockClosure >> >> 16r1033EC M PharoFilesOpener>ignoreIfFail: 16rE79FD8: a(n) PharoFilesOpener >> >> 16r102188 M [] in PharoFilesOpener>(nil) 16rE79FD8: a(n) PharoFilesOpener >> >> 16r1021AC M Array(SequenceableCollection)>do: 16r2495E8: a(n) Array >> >> 16r1021DC I PharoFilesOpener>openSources:forImage: 16rE79FD8: a(n) >> PharoFilesOpener >> >> 16r102204 I PharoFilesOpener>openSources 16rE79FD8: a(n) PharoFilesOpener >> >> 16r102228 I PharoFilesOpener>sourcesFileOrNil 16rE79FD8: a(n) >> PharoFilesOpener >> >> 16r10224C I SourceFileArray>ensureOpen 16r8AA128: a(n) SourceFileArray >> >> 16r10226C I SmalltalkImage>openSourceFiles 16rA58A28: a(n) SmalltalkImage >> >> 16r10228C I SmalltalkImage class>startUp: 16rFDA6A8: a(n) SmalltalkImage >> class >> >> 16r1022A8 M ClassSessionHandler>startup: 16r8B8978: a(n) >> ClassSessionHandler >> >> 16r1022C8 M [] in WorkingSession>(nil) 16r1A45D0: a(n) WorkingSession >> >> 16r1022EC M [] in WorkingSession>(nil) 16r1A45D0: a(n) WorkingSession >> >> 16r102304 M FullBlockClosure(BlockClosure)>on:do: 16r245640: a(n) >> FullBlockClosure >> >> 16r102328 M [] in WorkingSession>(nil) 16r1A45D0: a(n) WorkingSession >> >> 16r10234C M Array(SequenceableCollection)>do: 16r1A47B8: a(n) Array >> >> 16r102370 I WorkingSession>runList:do: 16r1A45D0: a(n) WorkingSession >> >> 16r102398 I WorkingSession>runStartup: 16r1A45D0: a(n) WorkingSession >> >> 16r1023BC I WorkingSession>start: 16r1A45D0: a(n) WorkingSession >> >> 16r1023E8 I SessionManager>snapshot:andQuit: 16r17AA950: a(n) >> SessionManager >> >> >> >> You'll see that Symbol class>selectorTable calls Symbol class> >> initSelectorTable which then calls CompiledMethod >> class(Behavior)>allInstancesDo: to enumerate over all methods. This takes a >> significant amount of time, even on a fast machine. For example, on a >> 2.%GHz machine this takes about 70 milliseconds. That's a /lot/ of time to >> squander on startup. And it will grow the larger the image is. >> >> >> >> Hm, I didn't thought about that. >> >> >> >> >> >> The selector table seems to be discarded on shut-down, which seems >> pointless. If one wants to recompute the table then before or after methods >> are added or removed seems like a better choice, not on every image startup. >> >> >> >> I think I wanted to be on the save side and just discard on shutdown and >> rebuild on startup to be sure to have the correct set of selector symbols. >> >> >> >> >> >> In around 2000 we were able to startup VisualWorks images in about 40 >> milliseconds on the hardware available at the time. I hope someone >> reconsiders this code and a avoids recomputing the selector table on each >> startup. >> >> >> >> I will take a look and try to find a better way. >> >> >> nicolai >> >> >> >> _,,,^..^,,,_ >> >> best, Eliot >> >
