If Traits are supported then I must be doing something else wrong - maybe load order? (To be honest, I don’t fully understand how traits work - but I will keep digging).
It could be that my latest error (having just done Class compile: ‘…. Missing method code … ‘) is something deeper related to these traits methods not applying somehow… I’ll try a few things this morning. It’s amazing how easily the image size jumps up when you add things though. With the original minimal v6 64bit image you sorted out for me I had a footprint of 21.6mb (image was 12mb). Now with the v7 image - and adding fuel and some S3 handling - it jumps up to 32.4mb (image is 15mb). I know that the S3 support has a lot of XML crud - but it seems quite astounding that this takes almost 8mb of code/objects. AWS loading (and hence response time) seems quite sensitive to the size of the package it has to load - although equally I wonder if there is also more #startUp/initalisation cost that is going on as well as when you get a warm lambda instance (so its not just loading your package) I’m still not seeing the 400-500ms times but more 800-1000ms times. Its all quite interesting - and shows why the work behind a minimal image is very useful. Of course there is tiny-minimal (which I know you guys are after) and then there is small-server- minimal (which is realistically what I’m after, so I can just focus on creating my app and just carefully load code and dependencies - but still have core server services like Fuel, and Git loading). Thanks for doing this work - as it brings a lot of useful things together. Tim > On 4 Aug 2017, at 06:48, Pavel Krivanek <[email protected]> wrote: > > > > 2017-08-03 18:17 GMT+02:00 Tim Mackinnon <[email protected] > <mailto:[email protected]>>: > Hi guys - I think I’m getting close - I’m able to load remote packages, and > I’ve added fuel to my repo and its loading - however I am getting an error > when I run and test writing out a context - MessageNotUnderstood: Context > class>>fuelIgnoredInstanceVariableNames > > I can see that this is a trait - is trait support missing in the minimal > image? > > No, this image supports them > > (I also seemed to get this far just loading the Fuel package - but just to be > sure I added the other platform ones like you suggested and still get the > same error). Given how simple that method is, I could just compile it in - > but maybe there are many more? > > Tim > > > "errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences > process Lambda '{\"debug\":true,\"data\":5}'\n > "errorType": "Error", > "stackTrace": [ > "Writing fuel context to S3...", > "Using S3 bucket morethan-technology-projects", > "\u001b[31m", > "--- RUNNING ERROR HANDLER ---", > "MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", > "", > "\u001b[0m\u001b[31mMessageNotUnderstood: Context > class>>fuelIgnoredInstanceVariableNames", > "\u001b[0mContext class(Object)>>doesNotUnderstand: > #fuelIgnoredInstanceVariableNames", > "FLVariablesMapping>>instanceVariableNamesToSerialize", > "FLVariablesMapping>>initializeAnalyzing", > "FLVariablesMapping class>>newAnalyzing:references:", > "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", > "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", > "clusterKeyedByObjectClass: t1 class: t2", > " self error: 'No decompiler available' in > FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: > clusterKeyedByObjectClass: t1 class: t2...", > "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", > " self error: 'No decompiler available' in > FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in > Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", > "at: t1 ifAbsentPut: t2", > " self error: 'No decompiler available' in > IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: > t2...", > "IdentityDictionary(Dictionary)>>at:ifAbsent:", > "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", > "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory:", > "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", > "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", > "FLLightGeneralMapper>>visitMethodContext:", > "Context>>fuelAccept:", > "FLLightGeneralMapper>>mapAndTrace:", > "FLLightGlobalMapper>>mapAndTrace:", > "FLPluggableSubstitutionMapper>>mapAndTrace:", > "FLAnalysis>>mapAndTrace:", > "FLAnalysis>>run", > "setDefaultAnalysis", > " self error: 'No decompiler available' in > FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", > "FLAnalyzer>>analysisFor:", > "FLSerialization>>analysisStep", > "FLSerialization>>run", > "setDefaultSerialization", > " self error: 'No decompiler available' in > FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", > "serialize: t1 on: t2", > " self error: 'No decompiler available' in FLSerializer>>serialize:on: > in Block: serialize: t1 on: t2...", > "on: t1 globalEnvironment: t2 do: t3", > " self error: 'No decompiler available' in FLEncoder > class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: > t3...", > "BlockClosure>>ensure:", > "FLEncoder class>>on:globalEnvironment:do:", > "\u001b[0m" >> On 3 Aug 2017, at 13:16, Mariano Martinez Peck <[email protected] >> <mailto:[email protected]>> wrote: >> >> >> >> On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <[email protected] >> <mailto:[email protected]>> wrote: >> I’m so close now (bit of a diversion with gitlab yml not liking the eval >> string) - that did load and it got me really close - I’m just missing >> FlSerializer now (I have progressed my work to fuel out the context for >> debugging to S3)… so looking at the repo, there are quite a few fuel >> packages - but in a big image I can see that FLSerialiser is in Core - so do >> I just need core? But then there are a few fuel packages with Core? >> >> >> The easiest way to understand that is either checking ConfigurationOfFuel >> directly or some of the related Metacello tool. >> Anyway, for this case, I think you need the packages (in this order): >> FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions >> (which we use in Fuel)... so...it's not always easy to load a project >> without Metacello. >> >> Sometimes you can print the result of #record: >> >> Metacello new >> configuration: 'Fuel'; >> smalltalkhubUser: 'Pharo' project: 'Fuel'; >> version: #stable; >> record: #('Core'). >> >> -> >> >> "linear load : >> linear load : 2.1.10 [ConfigurationOfFuel] >> linear load : baseline [ConfigurationOfFuelPlatform] >> load : FuelPlatform-Core >> load : FuelPlatform-Pharo-Core >> load : FuelPlatform-Pharo-06" >> >> >> ------ >> >> Anyway.... I guess I would try to load these packages (in this order): >> >> FuelPlatform-Core >> FuelPlatform-Pharo-Core >> FuelPlatform-Pharo-06 >> Fuel >> >> And when you are down loading, then execute: >> >> (Smalltalk at: #FLPlatform) current addHacks >> >> >> I’m also not quite clear on how I would load packages without a Baseline >> (would I create my own baseline for the fuel packages I need?) >> >> Tim >> >>> On 3 Aug 2017, at 12:07, Pavel Krivanek <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> >>> >>> 2017-08-03 13:02 GMT+02:00 Tim Mackinnon <[email protected] >>> <mailto:[email protected]>>: >>> I really appreciate your patience and help on this - it looks very >>> promising and I am giving it a spin now… >>> >>> When you say the pharo repository - are you referring to this: >>> https://github.com/pharo-project/pharo/tree/master/src >>> <https://github.com/pharo-project/pharo/tree/master/src>. ? Just so I know >>> for the future. >>> >>> yes >>> >>> -- Pavel >>> >>> >>> Tim >>> >>>> On 3 Aug 2017, at 09:16, Pavel Krivanek <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> >>>> >>>> 2017-08-03 10:14 GMT+02:00 Pavel Krivanek <[email protected] >>>> <mailto:[email protected]>>: >>>> The easiest think you can do is to copy to your repository (or some other) >>>> related packages from the Pharo repository (to do not have to clone it >>>> all): >>>> >>>> Metacello-GitBasedRepository.package >>>> Metacello-GitHub.package >>>> SUnit-Core.package >>>> >>>> and create baseline to load them. I already tried it so you can test it: >>>> https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 >>>> <https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8> >>>> >>>> Then you will do something like: >>>> >>>> (accidental message send ;-) ) >>>> >>>> BASELINE=MetacelloGitBasedRepository >>>> pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ >>>> "Metacello new baseline: '$BASELINE'; repository: >>>> 'filetree://./PharoLambda/src <>'; load." >>>> >>>> BASELINE=Lambda >>>> pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ >>>> "Metacello new baseline: '$BASELINE'; repository: >>>> 'filetree://./PharoLambda/src <>'; load." >>>> >>>> As I tried that. The baseline of Lambda is then able to be loaded. >>>> >>>> -- Pavel >>>> >>>> >>>> >>>> >>>> >>>> 2017-08-02 15:18 GMT+02:00 Tim Mackinnon <[email protected] >>>> <mailto:[email protected]>>: >>>> Ah, I think I’m starting to get closer to what I need… >>>> >>>> So if I want to load in that last piece to enable more general remote >>>> loading - how do I figure out how to do that? I’m trying to work out where >>>> the build steps for building up the image (can I see them in Jenkins? It >>>> wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned >>>> - looking there I can see a few metacello and gofer packages - but then I >>>> guess I’m looking for an easy Mcz I can use with the example below? >>>> >>>> Or do I just load Metacello as a git submodule and then it will be on my >>>> local filesystem to then bootstrap up? >>>> >>>> I guess I’m trying to work out the best sustainable approach to getting to >>>> a good server based image that has minimal tools and the ability to easily >>>> load remote code for pre-req projects. >>>> >>>> Tim >>>> >>>>> On 2 Aug 2017, at 07:05, Guillermo Polito <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> Yes, you should be able to load an mcz in that image by doing: >>>>> >>>>> (MCDirectoryRepository new directory: 'where-your-mcz-is') >>>>> loadVersionFromFileNamed: >>>>> 'Metacello-GitBasedRepository-Author.1.mcz') load. >>>>> >>>>> Guille >>>>> >>>>> On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> 2017-08-01 23:13 GMT+02:00 Tim Mackinnon <[email protected] >>>>> <mailto:[email protected]>>: >>>>> Hi Pavel - I tried it again and the problem is do with Metacello >>>>> dependencies in your baseline. >>>>> >>>>> The SUnit baseline doesn’t specify any additional dependencies to load >>>>> (it just loads local packages), whereas my baseline that fails looks like >>>>> this: >>>>> >>>>> baseline: spec >>>>> <baseline> >>>>> >>>>> spec for: #common do: [ >>>>> spec configuration: 'ZTimestamp' with: [ >>>>> spec >>>>> versionString: #stable; >>>>> repository: 'http://mc.stfx.eu/Neo >>>>> <http://mc.stfx.eu/Neo>' ]. >>>>> >>>>> spec baseline: 'AWS' with: [ >>>>> spec repository: >>>>> 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository <>' ]. >>>>> >>>>> spec >>>>> package: 'Lambda' with: [ spec requires: {'ZTimestamp'. >>>>> 'AWS'}]. >>>>> >>>>> ]. >>>>> >>>>> >>>>> The “spec configuration: ….” Specifications cause the error: >>>>> >>>>> 25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: >>>>> 26 MCRepositoryGroup>>addRepository: >>>>> >>>>> If I remove those lines from my baseline above (as well the requires: >>>>> reference to them) I can then successfully load my packages. >>>>> >>>>> So is this something the minimal image should support (otherwise how do >>>>> you load external packages into your image without checking them in >>>>> locally?) OR is there something simple I can checkin and load locally to >>>>> return that behaviour? (Personally I think it would make sense to allow >>>>> remote loading - I’m guessing Pharo itself as a core has everything it >>>>> needs - but if you want to do any experimentation on the core and need >>>>> remote packages, you would hit this too - so it feels a bit limiting). >>>>> >>>>> The dependencies in th baseline are supported but the support for most of >>>>> external repositories (like package Metacello-GitBasedRepository) is >>>>> loaded at the end of bootstrapping process in BaselineOfIDE. We should >>>>> check/solve dependencies and move it into the minimal Pharo. For now you >>>>> can try to load it by yourself or work with already prepared local clones >>>>> of required repositories. >>>>> >>>>> -- Pavel >>>>> >>>>> >>>>> >>>>> >>>>> Tim >>>>> >>>>>> On 1 Aug 2017, at 10:06, Pavel Krivanek <[email protected] >>>>>> <mailto:[email protected]>> wrote: >>>>>> >>>>>> >>>>>> >>>>>> 2017-07-31 22:51 GMT+02:00 Tim Mackinnon <[email protected] >>>>>> <mailto:[email protected]>>: >>>>>> I wasn’t clear on which image to retry - the >>>>>> https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessfulBuild/artifact/Pharo-minimal-64.zip >>>>>> >>>>>> <https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessfulBuild/artifact/Pharo-minimal-64.zip> >>>>>> one still shows as being last updated 7 days ago. >>>>>> >>>>>> The >>>>>> https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-ConfigurationOfMinimalPharo/ >>>>>> >>>>>> <https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-ConfigurationOfMinimalPharo/> >>>>>> one gives me a mismatch error: This interpreter (vers. 68021) cannot >>>>>> read image file (vers. 6521). >>>>>> >>>>>> The >>>>>> https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuccessfulBuild/artifact/latest-minimal-64.zip >>>>>> >>>>>> <https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuccessfulBuild/artifact/latest-minimal-64.zip> >>>>>> one gives me a walkback when trying to run my install script : >>>>>> >>>>>> 25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: >>>>>> 26 MCRepositoryGroup>>addRepository: >>>>>> 27 createRepository >>>>>> | repo | >>>>>> repo := self project createRepository: self. >>>>>> ^ MCRepositoryGroup default repositories >>>>>> detect: [ :each | each = repo ] >>>>>> ifNone: [ >>>>>> MCRepositoryGroup default addRepository: repo. >>>>>> repo ] in MetacelloRepositorySpec>>createRepository >>>>>> >>>>>> I think this is because the image doesn’t support Metacello? As in: >>>>>> Metacello new >>>>>> repository: 'filetree://../src <>'; >>>>>> baseline: 'Lambda'; >>>>>> load. >>>>>> >>>>>> How do you guys load things into the minimal images (I thought you used >>>>>> Metacello - but maybe you do it some other way?) >>>>>> >>>>>> I can use a big 6.1 image fine (as it has Metacello loaded) but I’d >>>>>> really like a minimal solution - that can load in libraries like AWS S3, >>>>>> or XML parsing etc. and it seems like I should be a good customer for >>>>>> kicking the tires on all of this. >>>>>> >>>>>> Tim >>>>>> >>>>>> >>>>>> I checked the 64-bit Pharo 7 minimal image and loading of baseline (of >>>>>> SUnit from pharo-project/pharo) works: >>>>>> >>>>>> ./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new >>>>>> baseline: 'SUnit'; repository: 'filetree://./pharo-core/src <>'; load." >>>>>> >>>>>> ./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run" >>>>>> >>>>>> Can you test it too? >>>>>> >>>>>> -- Pavel >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Guille Polito >>>>> >>>>> Research Engineer >>>>> French National Center for Scientific Research - http://www.cnrs.fr >>>>> <http://www.cnrs.fr/> >>>>> >>>>> >>>>> Web: http://guillep.github.io <http://guillep.github.io/> >>>>> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013> >>>> >>>> >>> >>> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/> >
