[Setools] Mercurial
Gjallar Folks, I have been struggling to commit my changes into the mercurial repsitory. I discover that this is because i dont really know how mercurial works. I found this: http://hgbook.red-bean.com/hgbook.pdf and there is a video on google video about it. cheers Keith ___ Setools mailing list Setools@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/setools
[Setools] Gjallar and latest seaside code submitted
Dear All, I am pleased to announce that the building script can now run without human intervention needed. It auto-answers any questions that packages may ask on installation. With a little help from Installer, the script has been written without any variables, so as to be easily runnable in a workspace, bit by bit or as a whole. This is a really good resource to have in a workspace, for quickly updating packages if they change, simply select the lines in the script for that package and execute them. The script is (hopefully) readable enough for you to get a good idea of the context of each package, the order of install etc. To run the script from the command line, instructions are given in the Readme files in dist/scripts My working repository for the gjallar build scripts is being served at: http://squeak.warwick.st:8002/newgjallar On demo.gjallar.se, there is a built image in /home/gjallar/keith/gjallar , this working copy is a clone from hg.gjallar.se + a pull from the above repository. When we are satisfied with the scripts the push can be made back to hg.gjallar.se. Minor Bug: The script sets the seaside/config user/password to seaside/seaside At present the build from the scripts, includes the latest seaside, but not a Gjallar release that can use it. This is because the script loads the most recent gk edition, you will need (or need to merge) the latest kph version from the MC repository. I hope that is enough to get everyone started with Seaside2.7a1 latest best regards Keith -- From Q2v0.4-kph.263 (found the missing methods - the men in white coats had them) WADispatcher allInstances - Q2Session initialize callse super - Q2Canvas has key and allocateKey, I couldnt see any reason why these should be in WACanvaRenderer - WAAnchorTag text: deprecated in favour of with: (silly I think but this gets rid of the red deprecated sign in the tool bar) - WATagBrush-tabIndex is now included in Seaside, recategorised this method as an override see method comment for details. - Added Q2MacOSSwishEWrapper, www.darwinports.com installs at default path of /opt/local/bin/swish-e - Added 'Magma seaside' has potential for the future. - Added Q2MagmaLocalLocation being a database location that is defined in the ini file, thus being chooseable but not editable in seaside/config whereas MagmaLocalLocation represents a database configurable by setting the path, as can be set in seaside/config. Although Q2Session ignores the WAMagmaConfiguration setting but MagmaControl panel can use it to access the gjallar repository since it knows nothing about gjallar. ___ Setools mailing list Setools@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/setools
Re: [Setools] Introduction
Hello All, I was dissappointed when we decided to remove images from the main gjallar repository, so I have been seeking a solution. For managing gjallar images with mercurial I have attempted to use mecurial queues but this was not really workable. Instead I have been successfully using nested repositories. To set this up, You clone the main repository as usual. so the gjallar/.hg directory is the repository. Then in the dist directory where you are typically working you initialize a second repository. hg init then you hg add Gjallar0.4.image hg add Gjallar0.4.changes hg commit these are added and committed to the new inner gjallar/dist/.hg repository! So when you want to make changes to the outer repository just cd .. , into the outer gjallar directory and make your changes adds/updates/commits/push/pulls from there This scheme theoretically allows you to commit a snapshot of your image and also the database state if you add that as well. You can of course return to any previous snapshotted state, and I dont think it uses as much disk as having lots of images as per my usual practice. Its great! Keith ___ Setools mailing list Setools@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/setools
[Setools] Re: MagmaReadStrategies
RESEND slightly more recent version of previous mail Hear is an outline of my readStrategy creation/management/understanding scheme! enjoy Keith = buildReadStrategyAggressiveOn: strategy Very highly aggressive read everything unless it is gated. This analogy uses the idea of attempting to collect information from a guarded space, perhaps a normal KEEP describes the scene. (OT: What a cool name for a database!) Essentially this is how it works, whatever you can see you can have. BIG GATES (typically -20) Stop everything except what is explicitly visible through a KEYHOLE - MagmaCollections are natural gates, - MagmaCollection's query mechanism is the little 'porter window' where you ask for service. - e.g. In GJallar Q2Model has a BIG GATE to gate most things - When the GATE is defined for the whole class (on all attributes) it applies to subclasses also including extra attributes the subclass may define. KEYHOLE (typically 0) - allows any reasonably agressive (+ve) read strategy to pass through the BIG/SMALL GATE. - will loose one in 'range'. BIG KEYHOLE (typically 1) - allows any read strategy that gets this far to pass through to the next level without cost. OPEN ACCESS (typically 99) - Everything inside the area/object is visible. - Setting objects for OPEN ACCESS is not relevant when OPEN ACCESS is the default behaviour. These items may be left marked as such to indicate intent and to facilitate future variations. SMALL GATE - Non-Solid See Through (typically -10) - Most attributes are not visible unless made ACQUIRABLE meaning explicitly placed in the line of sight by their owner for easy 'target aquisition'. - In Gjallar domain objects are gated with a SMALL GATE (e.g. Q2Process Q2Case) ACQUIRABLE - (typically 15) - Contents of default index like attributes are set to be visible through small gates whatever they are. - i.e. id, name, description BLOCKADED (typically -2) - Some specific object attributes are never retrieved unless explicitly requested. Precedence is such that BLOCKADED objects, if accidentally, placed in an OPEN ACCESS space or in line of sight are still protected, where as lesser barriers are not. - i.e. In Gjallar the wikiPage content is only ever needed when explicitly requested ALARMED (typically -99) - Access to this object will raise an error in the Log. Use this if you want to be informed when this object is used. (This is the equivalent of a self halt!) RECONNAISANCE -- GLANCING - minimumDepth 1 - top level OPEN ACCESS, KEYHOLES, and their ACQUIRABLES only LOOKING - minimumDepth 5 - to view any OPEN ACCESS, even through KEYHOLES, and ACQUIRABLES SMALL KEY - minimumDepth 15 - to retrieve anything behind one SMALL GATE BIG KEY - minimumDepth 25 - to retrive anything behind one BIG GATE strategy minimumDepth: 5. LOOKING big first read not needed due to high min depth strategy onAny: Q2RepositoryRoot readToDepth: 5. the main BIG GATE strategy onAny: Q2Model readToDepth: -20. tiny KEY HOLES strategy forVariableNamed: 'caseCounter' onAny: Q2Model readToDepth: 0. strategy forVariableNamed: 'useIndexing' onAny: Q2Model readToDepth: 99. ProtoObject - Object - Q2Object - Q2CasePanel - filter tableCaseForm numberOfCases name description Q2CodeHolder - code selection result error Q2CustomObjectLoader - Q2CustomObjectFileLoader - fileName separator Q2CustomObjectODBCLoader - dsn user password query Q2Event - Q2CaseEvent - case Q2CreatedCaseEvent case - Q2ModifiedCaseAssignedToMeEvent case - assigned Q2ModifiedCaseEvent case - Q2ModifiedCaseResponsibleEvent case - newResponsible Q2ModifiedCaseResponsibleToMeEvent case newResponsible - Q2ModifiedCaseRolesEvent case - Q2ModifiedRelatedCaseEvent case - Q2NoteAddedEvent case - Q2StageTransitionEvent case - newStage Q2StageTransitionForRelatedCaseEvent case newStage - Q2UserEvent - user Q2UserAddedEvent user - Q2File - fileName content strategy onAny: Q2File readToDepth: -2. BLOCKADE strategy forVariableNamed: 'fileName' onAny: Q2File readToDepth: 15. ACQUIRABLE Q2FilterOperator - selector label Q2FilterOperatorReversed selector label - Q2FilterPredicate - operator value rank filter Q2FilterFieldPredicate operator value rank filter - field Q2FilterProcessPredicate operator value rank filter - Q2FilterResponsiblePredicate operator value rank filter - Q2FilterSelectorPredicate operator value rank filter - selector description label values operators multi Q2FilterStagePredicate operator value rank filter -
[Setools] MagmaServerRequests
Dear All, I have uploaded an add-on package to http://www.squeaksource.com/MagmaTester called 'Magma server requests'. This is as mentioned in a previous email to this list but it is now packaged in a generic form independent of Gjallar. to install with Installer use: Installer mc http: 'http://www.squeaksource.com'; project: 'MagmaTester'; install: 'Magma server requests'. After installing the package into a working Magma installation, you will need to do a MagmaSession cleanUp to get it adopted by the MagmaRepositoryController's I have made it fairly generic since it passes arbitrary messages, and returns any values, trapping server-side errors, it transports them back over the wire in order to signal them in the client. Actions are atomic since, and I quote from the Magma client server read-me text. The answer block is only ever evaluated for one client request at a time. So to use this all you have to do is implement the Client side and the server side in MyOwnServerRequestResponder e.g. Client side: MyClass-counterValue ^ (self magmaSession serverRequest counterValue) submit e.g Server side MagmaRequestResponder subclass: MyOwnRequestResponder -- MyOwnRequestResponder-#counterValue ^ Count := Count + 1 -- RequestResponder-class-#requestsSupported ^ #( #counterValue ) -- You can send a batch of requests like so... (self magmaSession serverRequest counterIncrement; counterIncrement; counterIncrement; counterValue) submit -- Then there is a handy server backdoor! Normally each request in a batch of requests is sent to the responder which can handle each request. However if we enable requests to be chained, such that the next request is sent to the result of the previous request we can gain full remote control of the server. This is enabled by MagmaServerBackdoorResponder which in itself only serves one request: #login:password: e.g. ((self magmaSession serverRequest login: 'admin' password: 'magma rocks') repositoryController isReadOnly) submit To do: The login could be tied in to magma's exisitng users management. Some responders need simple persistence for their use which I do not know how to do enjoy Keith ___ Setools mailing list Setools@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/setools
[Setools] Clock Faking Test Resource
Dear Ralph, you asked about a clock faking test resource. A while ago I developed a ClassClonerTestResource, this allows you to take a current class which is normally a singleton that is in use, e.g. DateAndTime, and to clone it especially for testing purposes so as to be able to test the clone separately. This would allow me to set my clock under test to some crucial rollover points and test its behaviour. I developed this especially to test bug fix 474 on mantis. [ Edgar/Ralph - here is a fix with tests for you!] However what this doesn't do is replace DateAndTime, so if you want to run code which calls DateAndTime now and have it do something clever like run last week, or run backwards it will not work. However you raise an interesting problem which I think it may be worth solving for everyone, and I think that as a result of my recent discovery of the amazing ProcessLocalVariables I may have an answer. At present ProcessLocalVariables only support value and #value: As it happens I added a #doesNotUnderstand: handler as an experiment in allowing gjallar Sessions to spoof as MagmaSessions for Magma. At the time I thought that this would have some interesting possibilities. For us it means that a ProcessLocalVariable can spoof a class for us within a defined scope (process or block). ProcessSpecificVariable-doesNotUnderstand: aMessage ^ aMessage sendTo: self value So I made a DateAndTimeWarp a subclass of DynamicVariable (a variant of the ProcessLocalVariable which has block scope as well as process scope) and set the default to return DateAndTime. We now have a psuedo DateAndTime class which is pluggable on a per process basis. If we swap DateAndTime in the system dictionary for DateAndTimeWarp, it works! (i.e. nothing breaks!!!) DateAndTimeWarp-installImposter (DateAndTime respondsTo: #installImposter) ifTrue: [ self error: 'DateAndTime is already spoofed' ]. DateAndTimeReal := Smalltalk at: #DateAndTime. Smalltalk at: #DateAndTime put: self. Now we can substitute our clone with some modifications. The fixed DateAndTime implementation available on mantis via Installer mantis fixBug: 474. (tested in 3.8 and Gjallar0,4) has already been refactored to refer to its ClassVar ClockProvider for all clock references. (default setting is class Time) So all we need to do is replace ClockProvider, we will replace it with an instance of DateAndTimeWarp, DateAndTimeWarp-class-warpedClockBeginAt: t speed: s | newClock newSystemClockProvider | newSystemClockProvider := self new setStart: t speed: s. newClock := ClassClonerTestResource cloneOf: DateAndTimeReal. newClock classPool at: #ClockProvider put: newSystemClockProvider. clock initializeOffsets. ^ newClock We instantiate our new clock provider with a start time, (nil for now), and a speed. The new clock provider saves some offsets, and re-implements #totalSeconds, #millisecondClockValue and #secondsWhenClockTicks (this method waits until the clock ticks and is used for syncing the seconds and milliseconds clocks together. The new clock is substituted for the default clock with the following code: DateAndTimeWarp-class-beginAt: start warpSpeed: s during: aBlock ^ self value: (self warpedClockBeginAt: start speed: s) during: aBlock To Test: DateAndTime beginAt: DateAndTime yesterday warpSpeed: 3 during: [ 10 timesRepeat: [ Transcript cr; show: DateAndTime now. (Delay forSeconds: 1) wait. ] ]. I thought that was pretty impressive stuff. I am well impressed with these process local variables! Also speed wise: au natural, DateAndTime nowBenchmark. '6.84 µs' (I did improve DateAndTime now by a factor of 50-80 in fix 474) The equivalent benchmark spoofed using process local variable DynamicVariable, benchmark gives: 29.09 µs, i.e. not slow. This code is available in Gjallar monticello repository 'http://mc.gjallar.se' in the ProcessSpecific package. enjoy Keith ___ Setools mailing list Setools@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/setools
Re: [Setools] MagmaServerRequests
Chris Muller wrote: I have made it fairly generic since it passes arbitrary messages, and returns any values, trapping server-side errors, it transports them back over the wire in order to signal them in the client. Ma client server does not support a dynamic protocol; only fixed which must be established just before session negotiation. How do you handle this? My MagmaServerRequest being a subclass of RepositoryRequest is registered as part of the Magma protocol as you say. It has an attribute 'requests'. This is passed as an Array of Message' instances, both of these appear to be understood by Magma for serialization. At the other end I check that the Message-#selector's are defined in the #requestsSupported and simply send them to the class that implements that selector. cheers Keith ___ Setools mailing list Setools@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/setools
[Setools] Gjallar settings contd.
I used a variety of settings and the autoLogin feature for debugging UI, so I thought that it would be useful to promote all the relevant settings into the ini file and Q2GjallarSettings interface. Keith = (*optional - indicates defaults supplied by Q2GjallarSettings) new iniFile settings [globals] (required) serverPort=8081 mirrorServerPort=8090 (optional) autoLoginUsername= #empty indicates autoLogin disabled autoLoginPassword= [seaside] (optional) sessionClass=Q2Session #alternative for debugging UI - Q2SessionWithoutMagma rootComponentClass=Q2Main #alternative for debugging UI - use the component you are working on as root. resourcesLibraryClass=Q2Resources #alternative in-image resource set (must be subclass of Q2Resources) magmaLocationClass=Q2MagmaLocalLocation These settings are used in Q2Main initialize, which is called before starting the server. ___ Setools mailing list Setools@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/setools
[Setools] [Fwd: Re: Hi Chris]
---BeginMessage--- On 8/7/07, Keith Hodges [EMAIL PROTECTED] wrote: We in the gjallar camp have been trying to get a remote server connection without success. I am using beta2. The Magma test cases run all tests across via the remote connection; hundreds of them with multiple connects, disconnects and even quitting the server image (twice!) in the middle of a huge commit and the clients reconnect succesfully after the server recovery.. So can you tell me what code you are using to start the server and then what code to connect the clients? It appears that it is sending 'connect' and it is being recieved as an MAMalformed etc due to the class id not being recognized. It is #207 when the largest id in the ClassIdManager is 206. This 'connect' request is sent only by the lower-level framework, Ma client server, on behalf of Magma, to establish the #protocol (merely a list of classes). You should check what class the client thinks is 207. 207 seems awfully high too, something is almost certainly not configured correctly. You should try two vanilla Magma images (without Gjaller), establish a server on one: MagmaServerConsole new open: '/path/to/the/repository' ; processOn: 1010 ; inspect and connect from another image: (MagmaSession host: 'serverName' port: 1010) connectAs: 'test' ; inspect Note you can connect to a Gjaller repository even without the Gjaller code in the client; the missing classes will be created on the fly in order to instantiate the domain instances (but with no methods). I also notice that the error occurs when it tries to send repositoryController to this MaMalformed error instance and this occurs outside of the MaError handler. Yes. The Malformed check is actually in MaTcpRequestServer#materializeRequest:. Is there some initialization that I need to do to get the class id's set up correctly? No, not for Magma. Secondly sig tried opening a telnet to the magma port and sending random characters in. he got a rather drastic Out of Memory error and a corrupted image. On the server or telnet client? I assume on the client right? That'll learn him. :) So he told Magma to listen on the port reserved for telnet? Actually the server should have signaled a MalformedRequest error which would have been serialized and shipped back to the telnet client. Is that what happened? I'm curious about this story, but this version of Magma is not designed to be strong against misuse or attack, so I don't know how concerned I'll be about it (again, for this version). It is late and I am not feeling very coherent. If you need more to go on I can give you an image tomorrow Send me more information about what is being done, what steps including the code snippets. Regards, Chris ---End Message--- ___ Setools mailing list Setools@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/setools
[Setools] Re: Counter in Gjallar
Göran Krampe wrote: Hi Chris! Hi Göran, no, I'm sorry to hear that since I spent a good 4 days on it and ended up quite pleased with the result. No need to be sorry! :) I haven't looked but IIRC from your post you implemented an RC counter a la GemStone, right? Which is quite useful for counting stuff of course. :) What's the problem with it? Well, we have multiple MagmaSessions creating Q2Txn objects, giving them a strictly increasing unique number and adding them to a MagmaCollection. In short we want the illusion of a very long OrderedCollection. :) In an OrderedCollection each object of course has its position and if you succeed adding an object to it (without conflicts) then its number == its position. With a MagmaCollection we need to give it a number ourselves (and use an index for it) - and the problem is how we generate that number so that we don't end up with multiple Q2Txns with the same number - and not having to do multiple commits. In other words - an integer series generator is not exactly the same problem as a reduced conflict counter. Note btw that I am describing this from MY understanding - Balazs and Levente are the ones who are coding on this and perhaps I have misunderstood some parts :) The simplistic solution is of course to let all MagmaSessions make a server call (to the Magma server) which has a counter wrapped up in critical: blocks for increasing it. Thus making sure we serialize all increments. This is what Keith implemented which apparently did not work I simply implemented a request-response which uses the existing protocol. It has a class which handles both ends of the communication. Of course this class (these classes) has to be present in the server for it to work. This effectively bypasses the database altogether, so there are no critical blocks involved. Since the server handles requests serially it does ensure that the counter does not conflict. The bit that I did not know how to do was how to persist the counter values. I just used a file on the server to serialize read/ unserialize write a dictionary with the counter values in it on every increment. I myself have not looked at MagmaCounter yet. in the Magma version I tried a week ago (the server seems to not deal with the MagmaServerRequest properly when running as a separate image - causing the client image to hang waiting for a response. I logged all Magma The server image has to have the responder classes loaded in order to respond. requests going and all seems fine until this MagmaServerRequest coming through the line, haven't debugged it further on the server side - failed to see where to put the darn halt. :)). best regards Keith ___ Setools mailing list Setools@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/setools
[Setools] [Ann] Logging updated
The logging front end to the 3 logging frameworks has been refactored and simplified. In the process it has become much more powerful. Highlights: Output Adapters, and AdapterSets: You can now easily set up as many output adapters as you like on any or all processes, each with different filters etc. Processes can share sets of adaptors. Lots of different output adapters are supported: Logging Frameworks: LogEngine, SimpleLog, Toothpick(not ready yet) Also direct support for: Transcript, nil, SLLogMorph, SLLogWatchDog (email), SLSyslogSender, SLLogFile New Feature: LogHistory. This adapter logs a given number of messages to memory. Then when you connect another adapter, it outputs the history to the new adapter. Thus giving you fast logging to memory, but still allowing you to catch up with a hard copy if you need it. It will only be a matter of time before we have an automatic, when a big error occurs dump the in memory logs to a file feature. If you are using Rio or Logging, please join the setools@lists.squeakfoundation.org list and put your feedback there or find me on irc #squeak. best regards Keith Installation: = Installer mantis ensureFix: '7219 Streams readability'. Installer squeaksource project: 'Logging'; install: 'Logging'. Installer squeaksource project: 'Logging'; install: 'SimpleLog'. Architecture: LogCurrent the class is also a ProcessSpecific variable that returns an different instance of LogCurrent for each process. Each of these maintains a buffer for that process's log output. Users use 'self log' to obtain a LogCurrent buffer instance for the current process from 'LogCurrent value'. Since LogCurrent buffers are per-process they are threadsafe, and can maintain state between calls (self log properties). Each LogCurrent instance maintains a stream (on its buffer) and provides the users front end interface for convenient logging. Each LogCurrent instance maintains an adapter, when a log line is completed (#endEntry), the line is sent to the adapter, the adapter being the backend interface. The adapter may be any class that implements #logTime: stamp line: line level: level category: category sender: sender pid: pid e.g. Transcript. The adapter may also be a LogAdapterSet, in which case, the line is sent to each adapter contained in the set in turn. LogAdapterSets may be used to allow a number of processes to share the same set of adapters. LogAdapterSets may be arbitrarily nested, and may define a filterBlock. e.g. self filterBlock: [ :level :category | #(#error #debug #info) includes: level ]. AdapterSet may be subclassed if more specific filtering behaviour is required. Any changes to LogAdapterSets anywhere are protected by a single Semaphore, but use of adapters for output is protected instead by using a safer implementation of #do: to iterate over the set should an adapter be removed by a different thread. All processes that do not have a LogCurrent instance defined will copy the default LogCurrent instance, and will therefore share the same adapter/adapterset as the default. When you assign a bespoke LogCurrent instance to a specific process, it is up to you to set its adapter. It can include the global adapter/adapterset, nested within the new set you give it. Additional logging utilities and tools may be defined by subclassing LogCurrent. LogAdapterSet may be subclassed to provide more interesting filtering behaviour. And individual Adapters may also be configured or subclassed as desired. Adapters: === 1) LogHistory e.g. LogHistory size: 20. If the LogHistory adapter is present in a LogAdapterSet, then any new adapters added to that set have the recorded history sent to them immediately. Only one LogHistory adapter per set is allowed. 2) Blocks, LogAdapterSet may contain a block which returns a real adaper or nil. This enables adapters to be written to if enabled, or ignored if not available (see SLLogMorph below). 3) Transcript - the standard Transcript is used, and the line is output in one hit within the Transcripts own protection semaphore. 4) LogEngine e.g. LogEngine for: 'seaside'. This adapter is for the LogEngine backend. 5) SLLog This adapter is for the SimpleLog backend. 6) Toothpick This adapter is for the Toothpick backend. (Not yet supported) Direct to 'SimpleLog' Components 7) SLSyslogSender e,g, SLSyslogSender new addHost: 'log.somewhere.else'. or SLSyslogSender localhost. 8) SLLogWatchdog e.g. SLLogWatchdog default. Emails log entries over the given threshold 9) SLLogMorph e.g. [ SLLogMorph current ]. The block will be evaluated for every log entry. Therefore this construct will only write to LogMorph if it is open. 10) SLLogFile named: 'current.log'. Subclass and override #output*, or add #output* methods for additional file output formats. Or use (SLLogFile named: 'custom.log') output: [ :entry | entry stamp, '-', entry line ] Configuration: To change the global, or local LogCurrent
