Hi Ben, as Nicolas said, yes, MOP is for Meta Object Protocol (really nice book btw https://www.amazon.com/Art-Metaobject-Protocol-Gregor-Kiczales/dp/0262610744 ).
Ben, the process is not carved in stone and we are fully aware that it can be enhanced :). Any enhancement, even the littlest one, is important. If it is important for your workflows to add more logging information, let's add them! Nicolas, Actually the special interpreter is not the simulator. We do not use the bytecode interpreter in the simulator, we use an AST interpreter that has "lazy class creation" semantics plus some other little tricks to initialize the circular dependencies of metaclasses and so on... Behind, we use the simulator only to store the objects and decouple ourselves from the object format (this removed for example a lot of work when migrating from V3 to Spur). On Wed, May 2, 2018 at 10:04 PM, Nicolas Cellier < [email protected]> wrote: > Nice reverse engineering! > > MOP is Meta Object Protocol (serach in reference [9]). > > Maybe you miss a reference to the definition of BaselineOfPharoBootstrap > https://github.com/pharo-project/pharo/blob/development/bootstrap/src/ > Pharo30Bootstrap.package/PBBootstrap.class/instance/ > allPackagesForHermes.st > https://github.com/pharo-project/pharo/blob/development/bootstrap/src/ > Pharo30Bootstrap.package/PBBootstrap.class/instance/ > ensureBaselineOfPharoBootstrap.st > > And the simulator too (the special interpreter). > > 2018-05-02 20:53 GMT+02:00 Ben Coman <[email protected]>: > >> I found it hard to find info on the Pharo Bootstrap, so I went looking to >> understand the it by identifying very first creation on disk of the >> smallest "seed".image file that will later become the final build >> artifact. I'm sharing here to have a record I can refer to later, and >> maybe others find interesting. >> >> Looking at the log from the latest build... >> https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20p >> ull%20request%20and%20branch%20Pipeline/job/development/822/consoleFull >> >> Searching on "BUILD_NUMBER=" indicates the first major CI step is... >> >> + BUILD_NUMBER=822 BOOTSTRAP_ARCH=32 bash ./bootstrap/scripts/2-download.sh >> [1] >> which downloads 24MB " bootstrapImage.zip" [2] >> which I presume is an almost full image to run the bootstrap process. >> >> [1] https://github.com/pharo-project/pharo/blob/development/ >> bootstrap/scripts/2-download.sh >> [2] https://github.com/guillep/PharoBootstrap/releases/download/ >> v1.4.1/bootstrapImage.zip >> >> >> The next major CI step is... >> >> + BUILD_NUMBER=822 BOOTSTRAP_ARCH=32 bash ./bootstrap/scripts/3-prepare.sh >> [3] >> which uses the downloaded "Pharo.image" to (I presume) produce the seed >> "bootstrap.image" file. >> (This is not indicated explicitly in the logs, but step 4 copies that file >> with a shell command, so my presumption. It would be nice for this to be >> explicitly logged" >> >> prepare.sh executes two commands... >> >> ./pharo Pharo.image ..../bootstrap/scripts/prepare_image.st --save >> --quit [4] >> which seems to modify the downloaded "Pharo.image", >> not create bootstrap.image >> >> ./pharo Pharo.image ..../bootstrap/scripts/bootstrap.st --ARCH=32 >> --BUILD_NUMBER=822 --quit [5] >> which does... "PBBootstrap fromCommandLine bootstrap" [6] [7] >> which does... >> PBBootstrap>>#bootstrap >> PBStepStartedSignal log: 'Starting Process.'. >> self prepareBootstrap. >> [self createImage ] on:AssertionFailure do: [ :e | e resume ]. >> self initializeImage. >> PBStepFinishedSignal log: 'Process ended.' >> and digging a bit... >> >> PBBootstrap>>#createImage >> | builder | >> builder := PBImageBuilderSpur50 forArchitecture: architecture. >> builder definitionFetcher: versionFetcher. >> builder buildNumber: buildNumber. >> builder imageFileReference: imageReference. >> builder systemDefinition: ringEnvironment. >> builder espellBackend instanceVariableMapping: >> (PBInstanceVariableMapping onEnvironment: ringEnvironment ). >> builder bootstrap >> >> >> PBImageBuilderSpur50-class>>#forArchitecture: architecture >> | candidates | >> candidates := self subclasses select: [ :sc | sc name endsWith: >> architecture, 'bit' ]. >> candidates >> ifEmpty: [ self error: 'No candidates for architecture: ', >> architecture ]. >> candidates size > 1 >> ifTrue: [ self error: 'More than one candidate for >> architecture: ', architecture ]. >> ^ candidates anyOne new >> >> >> PBImageBuilderSpur50 >>#initialize >> super initialize. >> statistics := PBBootstrapStatistics new. >> logger := PBBootstrapLogger new. >> imageFileReference := 'bootstrap.image' asFileReference. >> >> "the default espell backend for the bootstrap" >> self initializeBackend. >> self instantiateClassLoader. >> >> ahh, there is the reference to "bootstrap.image" >> >> So just to round out a high level overview, the last two messages sent >> are defined as... >> >> PBImageBuilderSpur5032bit>>#initializeBackend >> espellBackend := EPSimulatorBackend for32Bit forBootstrap >> >> which I won't dig further but the class seems available at [8] >> and Espell is described briefly at [9] (but what is a "MOP"? the term is >> not defined there) >> >> >> PBImageBuilderSpur50>>#instantiateClassLoader >> classLoader := PBSpurClassLoader new. >> >> which is defined at [10], and there is an interesting example at [11]. >> >> >> [3] https://github.com/pharo-project/pharo/blob/development/ >> bootstrap/scripts/3-prepare.sh >> [4] https://github.com/pharo-project/pharo/blob/development/ >> bootstrap/scripts/prepare_image.st >> [5] https://github.com/pharo-project/pharo/blob/development/ >> bootstrap/scripts/bootstrap.st >> [6] https://github.com/pharo-project/pharo/blob/development/ >> bootstrap/src/Pharo30Bootstrap.package/PBBootstrap.class/ >> class/fromCommandLine.st >> [7] https://github.com/pharo-project/pharo/blob/development/ >> bootstrap/src/Pharo30Bootstrap.package/PBBootstrap.class/ >> instance/bootstrap.st >> [8] https://github.com/guillep/espell/blob/master/src/ >> EspellSimulator.package/EPSimulatorBackend.class/class/for32Bit.st >> [9] https://rmod.inria.fr/archives/papers/Poli15a-Onward- >> Bootstrapping.pdf >> [10] https://github.com/pharo-project/pharo/tree/development >> /bootstrap/src/Pharo30Bootstrap.package/PBSpurClassLoader.class >> [11] https://github.com/pharo-project/pharo/blob/development >> /bootstrap/src/Pharo30Bootstrap.package/PBImageBuilderSpur50 >> .class/class/example5617.st >> >> >> So now I guess that bootstrap.image has been generated? >> Just curious, what is the size of this file right now, before the next >> step? >> Perhaps an `ls -l` could echo this to the build log. >> >> >> The penultimate major CI step seems to then start adding packages to >> bootstrap.image... >> >> + BUILD_NUMBER=822 BOOTSTRAP_ARCH=32 bash ./bootstrap/scripts/4-build.sh >> >> located at [12] and searching on "${VM}" is enlightening. >> Its help information says... >> Pharo Build Script >> ================== >> This script assumes the existence of a new uninitialized image. Then it >> proceeds to its initialization and \"growing\". >> * Step 1: >> - Initialize the image >> - output: core.image >> * Step 2: >> - Bootstrap Monticello local repositories >> - output: monticello_bootstrap.image and changes file >> * Step 3: >> - Load Monticello remote repositories >> - output: monticello.image and changes file >> * Step 4: >> - Load Metacello >> - output: metacello.image and changes file >> * Step 5: >> - Load the rest of the image using BaselineOfIDE >> - output: Pharo.image and changes file >> >> Unfortunately those steps are labelled in the script, >> but the first action shown in the log is... >> cp bootstrap.image Pharo7.0-compiler-32bit-9912d7c.image >> Initializing Bootstraped Image >> pharo --headless Pharo7.0-compiler-32bit-9912d7c.image >> >> [12] https://github.com/pharo-project/pharo/blob/development >> /bootstrap/scripts/4-build.sh >> >> >> >> The final major CI step was a fair way past what I was interested in >> right now. ... >> >> + BUILD_NUMBER=822 bash ../bootstrap/scripts/prepare_for_upload.sh >> >> >> >> cheers -ben >> >> > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13
