Re: Nice idea in blog: Should we toss x86 architecture - NOT.
32 and 16 bit. The boot loaders used and FreeDos required incorporation of 16 bit support. Monumental pain. We don't see 64 bit support being problematic though. Gary On 8/1/08 9:31 AM, "Adam Thornton" <[EMAIL PROTECTED]> wrote: > Does z/VOS do x86_64 or just 32-bit x86? Actually, can you list which > x86 extensions are included in it?
Re: Nice idea in blog: Should we toss x86 architecture - NOT.
On Jul 31, 2008, at 5:08 PM, Gary M. Dennis wrote: z/VOS translates guest OS code during initial execution. Code fragment storage, lookup, disposal and reuse for primary and sibling guests are addressed in a patent application. Suffice it to say that we don't interpret or emulate massive amounts of x86 code for use 2-n. Ah, the old ARDI Executor approach. Well, probably not *exactly* if there's enough novel in there to patent, but, yeah, JIT instruction-stream-translation with caching of commonly-reused fragments was the way ARDI was emulating 68K Macs on Pentiums back in, oh, 1996 or so. That would (of course) be a better way to cut your overhead, because if you have a long sequence of s390x instructions mapped (nearly 1-to-1) to a long sequence of x86(_64?) instructions, then you only take the "wrapping the registers and shifting the arguments around" hit once per chunk, and a lot of x86 executable code is a) boilerplate and b) common in most executables produced by the same compiler, so this could actually be a very big win. Does z/VOS do x86_64 or just 32-bit x86? Actually, can you list which x86 extensions are included in it? Or do I need to wait for release to find out. Adam
Re: Nice idea in blog: Should we toss x86 architecture - NOT.
Apologies for not responding to this thread in a more timely fashion. I had a flood of emails after the initial post. Speed OR Portability Adam is closer than he knows about the approach we have taken on z/VOS. First, he is right when he guessed "almost-certainly assembly". We have tried both QEMU and BOCH and you can forget your name before the target OS completes IPL in our current environment. This is not meant to be a criticism. Both of these systems were written for portability rather than platform specific speed. He is also on target about the instruction mapping. Obviously the closer you get to a one to one instruction relationship the better the performance. Pipeline/Instruction Overlap Making the instruction stream pipeline-friendly has been another consideration with z/VOS. David Bond gave a great presentation (Share Session 8192 in August 2006) on this so I leave it to you and Google if you are interested. Single Pass Translation For those of you that think emulation of x86 would be a bad idea, we agree. z/VOS translates guest OS code during initial execution. Code fragment storage, lookup, disposal and reuse for primary and sibling guests are addressed in a patent application. Suffice it to say that we don't interpret or emulate massive amounts of x86 code for use 2-n. z/VOS Development Environment Mantissa has a FLEX development environment that redefines and enhances the definition of "slow". We have two copies of VM operational at all times. One of the VM system supports z/OS and VSE development. The other system is dedicated to z/VOS development. Because Adam mentioned FreeDos, I will give this point of reference. Under z/VOS FreeDOS IPLs in 1 second with all debug logging interfaces enabled. After the IPL, it is split second responsive. --. .- .-. -.-- Gary Dennis Mantissa Corporation On 7/25/08 11:28 AM, "Adam Thornton" <[EMAIL PROTECTED]> wrote: > On Jul 25, 2008, at 8:48 AM, McKown, John wrote: >> >> Somewhat like BOCH? I remember somebody saying that they ran Windows >> on >> BOCH on an old P/390. > > A little more data: the straight-up portable-emulation x86 code-path > is still not a good idea. I got the current released bochs (20080720) > built (with all the cool stuff like x86_64, SSE, plenty of neat > features) on Linux s390x. The build was quite clean, actually. > > It is a lot less painful than on a P/390 or H70, but running Bochs on > a 2094 (z9 of some sort), where z/VM sees 2 CPUs but the Linux guest > has one, is still only giving me 3.4 to 4.0 million (x86) instructions > per second, which is...well, a LOT less than you'd get on a modern > Xeon. That's not to say that I'm necessarily CPU-bound. If I had > time to play with it, the VGA refresh rate is where I'd start, because > that probably isn't helping. > > FreeDOS installation was pokey but not really terrible. Performance > is, well pretty bad; it feels like working over a satellite link in > terms of latency. I think you'd have a really hard time making the > case to management that THIS was a good use for your zSeries. > > So here's hoping that the Mantissa product is focussed around an > efficient (and almost-certainly assembly) x86 emulation. Given the > richness of the s390x instruction set, and that a bunch of the > instructions fundamentally do the same thing in the x86 and the s390x > world (that is, "move something from a memory location to a named > register" is the same concept on either architecture), I would hope > that most of the user-mode instructions can be mapped close to 1-to-1, > and the mere fact of having to create an instruction translator is > going to mean that the actual performance will be several-host- > instructions-to-one-guest-instruction. Complicated instructions are > still going to be slowed significantly, of course. > > Adam > --. .- .-. -.-- Gary Dennis Mantissa Corporation 1121 Edenton Street Birmingham, Alabama 35242-9257 p: 205.968-3942 m: 205.218-3937 f: 205.968.3932 [EMAIL PROTECTED] http://www.mantissa.com http://www.idovos.com
Re: Nice idea in blog: Should we toss x86 architecture - NOT.
On Jul 25, 2008, at 8:48 AM, McKown, John wrote: Somewhat like BOCH? I remember somebody saying that they ran Windows on BOCH on an old P/390. A little more data: the straight-up portable-emulation x86 code-path is still not a good idea. I got the current released bochs (20080720) built (with all the cool stuff like x86_64, SSE, plenty of neat features) on Linux s390x. The build was quite clean, actually. It is a lot less painful than on a P/390 or H70, but running Bochs on a 2094 (z9 of some sort), where z/VM sees 2 CPUs but the Linux guest has one, is still only giving me 3.4 to 4.0 million (x86) instructions per second, which is...well, a LOT less than you'd get on a modern Xeon. That's not to say that I'm necessarily CPU-bound. If I had time to play with it, the VGA refresh rate is where I'd start, because that probably isn't helping. FreeDOS installation was pokey but not really terrible. Performance is, well pretty bad; it feels like working over a satellite link in terms of latency. I think you'd have a really hard time making the case to management that THIS was a good use for your zSeries. So here's hoping that the Mantissa product is focussed around an efficient (and almost-certainly assembly) x86 emulation. Given the richness of the s390x instruction set, and that a bunch of the instructions fundamentally do the same thing in the x86 and the s390x world (that is, "move something from a memory location to a named register" is the same concept on either architecture), I would hope that most of the user-mode instructions can be mapped close to 1-to-1, and the mere fact of having to create an instruction translator is going to mean that the actual performance will be several-host- instructions-to-one-guest-instruction. Complicated instructions are still going to be slowed significantly, of course. Adam
Re: Nice idea in blog: Should we toss x86 architecture - NOT.
On Jul 25, 2008, at 8:48 AM, McKown, John wrote: -Original Message- From: The IBM z/VM Operating System [mailto:[EMAIL PROTECTED] On Behalf Of Gary M. Dennis Sent: Thursday, July 24, 2008 9:34 PM To: IBMVM@LISTSERV.UARK.EDU Subject: Re: Nice idea in blog: Should we toss x86 architecture - NOT. z/VOS is written to support the x86 instruction set and the underlying hardware rather than a specific operating system. For example, FreeDos was used as the initial debug target operating system due to source code availability. --. .- .-. -.-- Gary Dennis Mantissa Corporation Somewhat like BOCH? I remember somebody saying that they ran Windows on BOCH on an old P/390. Yeah, that was me. I did indeed boot NT4 on Bochs on a P/390. Or maybe it was an H70: the fact that the Linux box was "h1.tx.sinenomine.net" makes me suspect that it was Dave Jones' (at the time) H70. http://www.fsf.net/~adam/NT-on-390-desktop.png I went and grabbed the latest Bochs last night. Apparently it now does x86_64 and can run Vista. It will be interesting, once Mantissa is released, to do a speed comparison of it versus Bochs, since both appear to be, essentially, x86 emulators. I haven't been able to build it yet; it, at the very least, requires X libraries, so I need to build a dev box with the appropriate libraries. And of course it's going to bespecialto run it on, ahem, a Flex box running in 64-bit mode on 32-bit (Intel) hardware, which is what we have in-house. Adam
Re: Nice idea in blog: Should we toss x86 architecture - NOT.
> -Original Message- > From: The IBM z/VM Operating System > [mailto:[EMAIL PROTECTED] On Behalf Of Gary M. Dennis > Sent: Thursday, July 24, 2008 9:34 PM > To: IBMVM@LISTSERV.UARK.EDU > Subject: Re: Nice idea in blog: Should we toss x86 architecture - NOT. > > z/VOS is written to support the x86 instruction set and the underlying > hardware rather than a specific operating system. For > example, FreeDos was > used as the initial debug target operating system due to source code > availability. > > --. .- .-. -.-- > > Gary Dennis > Mantissa Corporation Somewhat like BOCH? I remember somebody saying that they ran Windows on BOCH on an old P/390. -- John McKown Senior Systems Programmer HealthMarkets Keeping the Promise of Affordable Coverage Administrative Services Group Information Technology The information contained in this e-mail message may be privileged and/or confidential. It is for intended addressee(s) only. If you are not the intended recipient, you are hereby notified that any disclosure, reproduction, distribution or other use of this communication is strictly prohibited and could, in certain circumstances, be a criminal offense. If you have received this e-mail in error, please notify the sender by reply and delete this message without copying or disclosing it.
Re: Nice idea in blog: Should we toss x86 architecture - NOT.
z/VOS is written to support the x86 instruction set and the underlying hardware rather than a specific operating system. For example, FreeDos was used as the initial debug target operating system due to source code availability. --. .- .-. -.-- Gary Dennis Mantissa Corporation On 7/23/08 9:06 AM, "Mary Anne Matyaz" <[EMAIL PROTECTED]> wrote: > Gary, if it runs native windows, will it also then run x86 linux? That seems > to be one of the barriers for us, that z/linux may not support certain x86 > linux > applications. > Thanks, > Mary Anne > >> >> Gary M. Dennis wrote: >> >>> Z/VOS is a CMS application. The glass-side user will only see Windows via >>> RDC and know nothing of or about CMS or VM. >>> >>> Gary >>> >>> On 7/22/08 8:30 PM, "dave" <[EMAIL PROTECTED]> wrote: >>> >>> Good luck, Gary. I do hope your organization can pull this off. VM-ers need more employment possibilities:-) I gather from some of your previous posts to this list that your Windows support software, z/VOS, is in fact a sophisticated CMS-based application, that is a user would log onto a CMS user id to start his Windows systemis my understanding correct? Thanks and have a good one. DJ - Original Message - From: "Gary M. Dennis" <[EMAIL PROTECTED]> To: IBMVM@LISTSERV.UARK.EDU Subject: Re: Nice idea in blog: Should we toss x86 architecture Date: Tue, 22 Jul 2008 13:02:33 -0500 > This was our post to the zd net blog. > > > "Maybe we already have. > > In Q1 2009 Mantissa will deliver a system that permits > unaltered Windows operating systems to run under z/VM. > Using a desktop appliance running RDC, users will be able > to connect to their virtual Windows images running in the > VM environment. Goodbye desktop hardware, remote > maintenance, high power consumption, machine order lead > time. > > z/VOS began with the observation that most Windows > workstations do practically nothing 95% of the time and we > were so intrigued with the idea of being able to actually > run an intel-based operating system under IBM VM that we > never looked back. VM provided a natural platform for > development of this product. > > The product has been a bear for the development group but > the thought of being able to run 3000 copies of Windows on > one System z so fascinated the team that we needed very > little additional incentive. > > Let's hope IBM can ramp up System z production." > > > Why wait until 2016? > --. .- .-. -.-- > > Gary Dennis > Mantissa Corporation > > On 7/22/08 11:14 AM, "Bob Heerdink" > <[EMAIL PROTECTED]> wrote: > > >> http://blogs.zdnet.com/perlow/?p=9183 >> >> "Should we toss x86 architecture and wipe the slate with >> something greene r >> and more scalable?" >> >> "Windows Server 2016 128-bit edition running virtualized >> on z/VM in a gre en >> datacenter, accessed via my house from a thin client >> over high-speed fibe r >> optic connection. I can see it now." >> >> Hope this happens sooner than predicted, >> Bob >> >>> >>> > >
Re: Nice idea in blog: Should we toss x86 architecture - NOT.
If so, then unlikely that CMS would run on "cell blade engines", and emulation not required. With IBM now owning platform, who did seem to have this kind of technology, there are feasible options that would actually be marketable. Quay, Jonathan (IHG) wrote: It is my understanding that IBM intends to integrate Cell Blade engine (e.g. playstation 3) technology into the z/Series ecosystem. This would seem to me to be the place where massively parallel high intensity cpu workload would live in the not so far flung future. -Original Message- From: The IBM z/VM Operating System [mailto:[EMAIL PROTECTED] On Behalf Of Barton Robinson Sent: Wednesday, July 23, 2008 10:59 AM To: IBMVM@LISTSERV.UARK.EDU Subject: Re: Nice idea in blog: Should we toss x86 architecture - NOT. Ok, so reality check folks before y'all start drooling about jobs and can think you can run 47000 windows servers under VM. In Linux we learned that running compiled code "natively" on "z", megahertz is megahertz and a CPU intensive task would always run faster on Intel than on "z" (until we got z9 and z10). And that is "native" meaning the programs were compiled to run on z, and the operating system was compiled to run on z. So now, under CMS, this emulates intel. So megahertz is NOT megahertz. With emulating an architecture, one could easily imagine losing an order of magnitude. Thus a windows server that is running at 10% peak on a 4Ghz processor would consume a z10 IFL and want more. One does need to pay significant attention to the performance characteristics before thinking about something like this seriously. Sorry. Gary M. Dennis wrote: Z/VOS is a CMS application. The glass-side user will only see Windows via RDC and know nothing of or about CMS or VM. Gary On 7/22/08 8:30 PM, "dave" <[EMAIL PROTECTED]> wrote: Good luck, Gary. I do hope your organization can pull this off. VM-ers need more employment possibilities:-) I gather from some of your previous posts to this list that your Windows support software, z/VOS, is in fact a sophisticated CMS-based application, that is a user would log onto a CMS user id to start his Windows systemis my understanding correct? Thanks and have a good one. DJ - Original Message - From: "Gary M. Dennis" <[EMAIL PROTECTED]> To: IBMVM@LISTSERV.UARK.EDU Subject: Re: Nice idea in blog: Should we toss x86 architecture Date: Tue, 22 Jul 2008 13:02:33 -0500 This was our post to the zd net blog. "Maybe we already have. In Q1 2009 Mantissa will deliver a system that permits unaltered Windows operating systems to run under z/VM. Using a desktop appliance running RDC, users will be able to connect to their virtual Windows images running in the VM environment. Goodbye desktop hardware, remote maintenance, high power consumption, machine order lead time. z/VOS began with the observation that most Windows workstations do practically nothing 95% of the time and we were so intrigued with the idea of being able to actually run an intel-based operating system under IBM VM that we never looked back. VM provided a natural platform for development of this product. The product has been a bear for the development group but the thought of being able to run 3000 copies of Windows on one System z so fascinated the team that we needed very little additional incentive. Let's hope IBM can ramp up System z production." Why wait until 2016? --. .- .-. -.-- Gary Dennis Mantissa Corporation On 7/22/08 11:14 AM, "Bob Heerdink" <[EMAIL PROTECTED]> wrote: http://blogs.zdnet.com/perlow/?p=9183 "Should we toss x86 architecture and wipe the slate with something greene r and more scalable?" "Windows Server 2016 128-bit edition running virtualized on z/VM in a gre en datacenter, accessed via my house from a thin client over high-speed fibe r optic connection. I can see it now." Hope this happens sooner than predicted, Bob
Re: Nice idea in blog: Should we toss x86 architecture - NOT.
It is my understanding that IBM intends to integrate Cell Blade engine (e.g. playstation 3) technology into the z/Series ecosystem. This would seem to me to be the place where massively parallel high intensity cpu workload would live in the not so far flung future. -Original Message- From: The IBM z/VM Operating System [mailto:[EMAIL PROTECTED] On Behalf Of Barton Robinson Sent: Wednesday, July 23, 2008 10:59 AM To: IBMVM@LISTSERV.UARK.EDU Subject: Re: Nice idea in blog: Should we toss x86 architecture - NOT. Ok, so reality check folks before y'all start drooling about jobs and can think you can run 47000 windows servers under VM. In Linux we learned that running compiled code "natively" on "z", megahertz is megahertz and a CPU intensive task would always run faster on Intel than on "z" (until we got z9 and z10). And that is "native" meaning the programs were compiled to run on z, and the operating system was compiled to run on z. So now, under CMS, this emulates intel. So megahertz is NOT megahertz. With emulating an architecture, one could easily imagine losing an order of magnitude. Thus a windows server that is running at 10% peak on a 4Ghz processor would consume a z10 IFL and want more. One does need to pay significant attention to the performance characteristics before thinking about something like this seriously. Sorry. Gary M. Dennis wrote: > Z/VOS is a CMS application. The glass-side user will only see Windows via > RDC and know nothing of or about CMS or VM. > > Gary > > On 7/22/08 8:30 PM, "dave" <[EMAIL PROTECTED]> wrote: > > >>Good luck, Gary. I do hope your organization can pull this >>off. VM-ers need more employment possibilities:-) >> >>I gather from some of your previous posts to this list that >>your Windows support software, z/VOS, is in fact a >>sophisticated CMS-based application, that is a user would >>log onto a CMS user id to start his Windows systemis my >>understanding correct? >> >>Thanks and have a good one. >> >>DJ >>- Original Message - >>From: "Gary M. Dennis" <[EMAIL PROTECTED]> >>To: IBMVM@LISTSERV.UARK.EDU >>Subject: Re: Nice idea in blog: Should we toss x86 >>architecture >>Date: Tue, 22 Jul 2008 13:02:33 -0500 >> >> >>>This was our post to the zd net blog. >>> >>> >>>"Maybe we already have. >>> >>>In Q1 2009 Mantissa will deliver a system that permits >>>unaltered Windows operating systems to run under z/VM. >>>Using a desktop appliance running RDC, users will be able >>>to connect to their virtual Windows images running in the >>>VM environment. Goodbye desktop hardware, remote >>>maintenance, high power consumption, machine order lead >>>time. >>> >>>z/VOS began with the observation that most Windows >>>workstations do practically nothing 95% of the time and we >>>were so intrigued with the idea of being able to actually >>>run an intel-based operating system under IBM VM that we >>>never looked back. VM provided a natural platform for >>>development of this product. >>> >>>The product has been a bear for the development group but >>>the thought of being able to run 3000 copies of Windows on >>>one System z so fascinated the team that we needed very >>>little additional incentive. >>> >>>Let's hope IBM can ramp up System z production." >>> >>> >>>Why wait until 2016? >>>--. .- .-. -.-- >>> >>>Gary Dennis >>>Mantissa Corporation >>> >>>On 7/22/08 11:14 AM, "Bob Heerdink" >>><[EMAIL PROTECTED]> wrote: >>> >>> >>>>http://blogs.zdnet.com/perlow/?p=9183 >>>> >>>>"Should we toss x86 architecture and wipe the slate with >>>>something greene r >>>>and more scalable?" >>>> >>>>"Windows Server 2016 128-bit edition running virtualized >>>>on z/VM in a gre en >>>>datacenter, accessed via my house from a thin client >>>>over high-speed fibe r >>>>optic connection. I can see it now." >>>> >>>>Hope this happens sooner than predicted, >>>>Bob >>>> >> > >
Re: Nice idea in blog: Should we toss x86 architecture - NOT.
Gary, if it runs native windows, will it also then run x86 linux? That seems to be one of the barriers for us, that z/linux may not support certain x86 linux applications. Thanks, Mary Anne > Gary M. Dennis wrote: > > Z/VOS is a CMS application. The glass-side user will only see Windows via >> RDC and know nothing of or about CMS or VM. >> >> Gary >> >> On 7/22/08 8:30 PM, "dave" <[EMAIL PROTECTED]> wrote: >> >> >> Good luck, Gary. I do hope your organization can pull this >>> off. VM-ers need more employment possibilities:-) >>> >>> I gather from some of your previous posts to this list that >>> your Windows support software, z/VOS, is in fact a >>> sophisticated CMS-based application, that is a user would >>> log onto a CMS user id to start his Windows systemis my >>> understanding correct? >>> >>> Thanks and have a good one. >>> >>> DJ >>> - Original Message - >>> From: "Gary M. Dennis" <[EMAIL PROTECTED]> >>> To: IBMVM@LISTSERV.UARK.EDU >>> Subject: Re: Nice idea in blog: Should we toss x86 >>> architecture >>> Date: Tue, 22 Jul 2008 13:02:33 -0500 >>> >>> >>> This was our post to the zd net blog. "Maybe we already have. In Q1 2009 Mantissa will deliver a system that permits unaltered Windows operating systems to run under z/VM. Using a desktop appliance running RDC, users will be able to connect to their virtual Windows images running in the VM environment. Goodbye desktop hardware, remote maintenance, high power consumption, machine order lead time. z/VOS began with the observation that most Windows workstations do practically nothing 95% of the time and we were so intrigued with the idea of being able to actually run an intel-based operating system under IBM VM that we never looked back. VM provided a natural platform for development of this product. The product has been a bear for the development group but the thought of being able to run 3000 copies of Windows on one System z so fascinated the team that we needed very little additional incentive. Let's hope IBM can ramp up System z production." Why wait until 2016? --. .- .-. -.-- Gary Dennis Mantissa Corporation On 7/22/08 11:14 AM, "Bob Heerdink" <[EMAIL PROTECTED]> wrote: http://blogs.zdnet.com/perlow/?p=9183 > > "Should we toss x86 architecture and wipe the slate with > something greene r > and more scalable?" > > "Windows Server 2016 128-bit edition running virtualized > on z/VM in a gre en > datacenter, accessed via my house from a thin client > over high-speed fibe r > optic connection. I can see it now." > > Hope this happens sooner than predicted, > Bob > > >>> >> >>
Re: Nice idea in blog: Should we toss x86 architecture - NOT.
Ok, so reality check folks before y'all start drooling about jobs and can think you can run 47000 windows servers under VM. In Linux we learned that running compiled code "natively" on "z", megahertz is megahertz and a CPU intensive task would always run faster on Intel than on "z" (until we got z9 and z10). And that is "native" meaning the programs were compiled to run on z, and the operating system was compiled to run on z. So now, under CMS, this emulates intel. So megahertz is NOT megahertz. With emulating an architecture, one could easily imagine losing an order of magnitude. Thus a windows server that is running at 10% peak on a 4Ghz processor would consume a z10 IFL and want more. One does need to pay significant attention to the performance characteristics before thinking about something like this seriously. Sorry. Gary M. Dennis wrote: Z/VOS is a CMS application. The glass-side user will only see Windows via RDC and know nothing of or about CMS or VM. Gary On 7/22/08 8:30 PM, "dave" <[EMAIL PROTECTED]> wrote: Good luck, Gary. I do hope your organization can pull this off. VM-ers need more employment possibilities:-) I gather from some of your previous posts to this list that your Windows support software, z/VOS, is in fact a sophisticated CMS-based application, that is a user would log onto a CMS user id to start his Windows systemis my understanding correct? Thanks and have a good one. DJ - Original Message - From: "Gary M. Dennis" <[EMAIL PROTECTED]> To: IBMVM@LISTSERV.UARK.EDU Subject: Re: Nice idea in blog: Should we toss x86 architecture Date: Tue, 22 Jul 2008 13:02:33 -0500 This was our post to the zd net blog. "Maybe we already have. In Q1 2009 Mantissa will deliver a system that permits unaltered Windows operating systems to run under z/VM. Using a desktop appliance running RDC, users will be able to connect to their virtual Windows images running in the VM environment. Goodbye desktop hardware, remote maintenance, high power consumption, machine order lead time. z/VOS began with the observation that most Windows workstations do practically nothing 95% of the time and we were so intrigued with the idea of being able to actually run an intel-based operating system under IBM VM that we never looked back. VM provided a natural platform for development of this product. The product has been a bear for the development group but the thought of being able to run 3000 copies of Windows on one System z so fascinated the team that we needed very little additional incentive. Let's hope IBM can ramp up System z production." Why wait until 2016? --. .- .-. -.-- Gary Dennis Mantissa Corporation On 7/22/08 11:14 AM, "Bob Heerdink" <[EMAIL PROTECTED]> wrote: http://blogs.zdnet.com/perlow/?p=9183 "Should we toss x86 architecture and wipe the slate with something greene r and more scalable?" "Windows Server 2016 128-bit edition running virtualized on z/VM in a gre en datacenter, accessed via my house from a thin client over high-speed fibe r optic connection. I can see it now." Hope this happens sooner than predicted, Bob
