Re: [Freedos-user] Problem w/ CONFIG.SYS INSTALL
Thanks, Christian! When I first came across this problem, it seemed like maybe INSTALL= was using memory without an appropriate MCB, though from just perusing the code it looked like it was using INT 21.48 to allocate the memory. I had actually tried allocating an extra 64k segment at the top of conventional memory before, but that didn't fix it, so I assumed it was probably something else. I just did some testing, and it turns out that an extra 64k isn't enough to solve the problem, but 128k seems to be. The "real" number to use is probably somewhere in between. I'll leave it at 128k for now. Note to developers: If you're writing or modifying any new programs, and there's even a remote possibility that the user will try to use INSTALL= and will be using an un-patched version of FreeDOS, the the top part of conventional memory is off-limits! -- View this message in context: http://old.nabble.com/Problem-w--CONFIG.SYS-INSTALL-tp30076203p30107045.html Sent from the FreeDOS - User mailing list archive at Nabble.com. -- Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Problem w/ CONFIG.SYS INSTALL
Hi Bret & list, I found the problem. Both TESTPASS and TESTFAIL failed in my test environment, but that doesn't matter. The kernel does not correctly allocate MCBs for its configuration and initialization program code. The code that an application loaded by INSTALL= returns to via termination (pointed to by Int22) is in free memory. Your TSR relocates its installation code to the top of the low memory area as allocatable with Int21.58 and .48 calls. Thus, it overwrites some of the kernel's config/init code. For application developers (ie Bret): You can probably work around that by allocating 64 KiB (or something like that) too much for the relocated installation code. (I recommend to retry with the usual required size if the size+1000h 21.48 call fails.) If you don't modify this part (but free it as usual) that should do it for now. Please test. A check for affected FreeDOS builds could be done by checking whether Int22 points into unallocated memory. For kernel developers (I guess cross-posting this to -kernel is not necessary): The interrupt 22h handler is at address 8F8Eh:AB67h (at least with the supplied kernel binary, build 2038, from 2009-05-16, WATCOMC, FAT32, and in my test environment) for the application loaded with INSTALL=. This segmented address is at the linear address 9A447h (= segmented 9A44h:0007h). The only visible allocation of the kernel (owner = 60h) during this time is at segment 9DD8h (MCB at 9DD7h), and its size is 227h paragraphs. (The next MCB is the one at 9FFFh.) The affected functions are DoInstall, InstallExec, and init_DosExec, which call down in that order. init_DosExec is in intr.asm, the others are in config.c. init_DosExec calls Int21.4B, ie Int22 points to its code. All of these functions execute in unallocated memory, though most others in config.c probably do too. In my tests, Bret's application didn't overwrite InstallExec and init_DosExec but that depends entirely on how much memory the application uses. (DoInstall had been overwritten.) I tested this again with build 2039, from 2010-08-01, WATCOMC, FAT32, which is the latest kernel binary on the FreeDOS mirrors. In this case, Int22 is at 9016h:A098h (linear 9A1F8h) and the kernel allocation is at 9DC9h. The SVN repo does not appear to contain changes that might affect this problem. Please release a stable build as soon as you have fixed that bug! Regards, Christian -- Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Problem w/ CONFIG.SYS INSTALL
I'm attaching a file that has a "simple" TSR with the problem (TESTFAIL.COM), as well as one that doesn't (TESTPASS.COM). I'm also including the source code (TEST.A36, in A386 format), as well as the KERNEL.SYS & CONFIG.SYS I used to test with. The only difference between TESTFAIL & TESTPASS is the size of the executable file itself. In the source code, I change ProgStackSize (line 1364 in TEST.A36) from 1024 for TESTFAIL to (1024-16) for TESTPASS. IOW, I simply decrease the executable's size by 1 paragraph and it starts working. TESTPASS & TESTFAIL both work from the FreeDOS command-line, and they also work from MS-DOS 6.20 CONFIG.SYS and command-line. But only TESTPASS works from FreeDOS CONFIG.SYS. Changing the executable size up and down (by manipulating ProgStackSize in increments of 16 bytes) will cause different kinds of errors to occur. What the TSR does is simply intercept INT 8, and "toggles" the character at row 2, column 80 on the screen. There is still quite a bit of "extra" (uncalled) code in the program, but I stopped changing things when I got to a point where I could make it start and stop working with a single, simple change. Also, the program currently uses a TSR Exit (INT 21.31), even though it doesn't actually need to. I was just experimenting with INT 21.31 vs. INT 21.4C to see if it made any difference. So, as it is now, it leaves a "real" TSR segment in memory (one that intercepts INT 08 & INT 2D), as well as a small segment near the top of conventional memory that doesn't actually do anything. I figured if the TSR exit worked but a regular exit didn't (for whatever reason), I could simply trap INT 28 or INT 21 with the TSR and release the unnecessary segment at a later point in time. It's very possible that there's a problem with my code and there's nothing wrong with the FreeDOS kernel, so any help at all would be appreciated. http://old.nabble.com/file/p30101181/TEST.ZIP TEST.ZIP ___ Bret -- View this message in context: http://old.nabble.com/Problem-w--CONFIG.SYS-INSTALL-tp30076203p30101181.html Sent from the FreeDOS - User mailing list archive at Nabble.com. -- Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Problem w/ CONFIG.SYS INSTALL
I was using kernel 2036. Just tried 2038 -- no effective difference. The JEMM error is different now (Error 06 at a seemingly random CS:EIP, near the top of conventional memory but where there is no associated PSP). -- Bret -- Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Problem w/ CONFIG.SYS INSTALL
> The TSR I'm working on can't be published just yet, so that's not an > option right now. I thought so. > It's also _really_ complicated, so I'm not sure anyone would want to > mess with it anyway. You know me. > I'm using an older version of the kernel -- I'm not sure exactly which > one right now, but it's one from right after FreeDOS version 1 was > released. I'm working with some other programmers who are using the > latest versions of the kernel. I can find out exact details if it might > help. Yes, details would help. I would strongly recommend you to update your kernel, that might fix the problem. Regards, Christian -- Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Problem w/ CONFIG.SYS INSTALL
Hi Bret, > I checked the system's state with INSTALL=DEBUG.COM on a boot disk > (Rugxulo's bare DOS disk, with a 2008-03-08 kernel, build 2038) and > it appears fine to me. Memory that belongs to the > configuration/initialization program is allocated to a PSP at > segment 60h (!) which is properly set up except there's no MCB in > front of it. (This means programs that display a process's name > display garbage instead.) > Some simple tests, including resident installation of a TSR that > uses process relocation and the new Int21.4C method, passed. > > Tell us the programs that are used and their versions, that might > help. The TSR I'm working on can't be published just yet, so that's not an option right now. It's also _really_ complicated, so I'm not sure anyone would want to mess with it anyway. I'm using an older version of the kernel -- I'm not sure exactly which one right now, but it's one from right after FreeDOS version 1 was released. I'm working with some other programmers who are using the latest versions of the kernel. I can find out exact details if it might help. > Can you reproduce the crash with a minimal (dummy) TSR? Could you > make a TSR available that causes this crash? I don't know -- I'll see if I can create a "simple" TSR that still has the problem. -- Bret -- Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Problem w/ CONFIG.SYS INSTALL
> That could be related. How do you allocate the extra memory and > where? I manage memory using the standard INT 21.48, 49, & 4A. The segments sometimes are installed in a UMB ad other times not, depending on the exact circumstances. E.g., if UMB's are available and the CPU is in real mode, I assume that something like UMBPCI is installed and the UMB's are probably incompatible with DMA. Some of the segments need DMA and others don't, so some may intentionally get installed in UMB's and others not. I use INT 21.5801, 5802, 5803, & 5804 to manipulate exactly where the segment ends up before I issue the INT 21.48. I do relocate the installation program to the top of conventional memory and run it from there. I've tried both just copying the program but leaving the original in place (at the bottom of conventional memory), as well as relocating and completely terminating the original process, and neither one works. I've also tried with and without HIMEM(X) and (J)EMM, with the same results. > Maybe JEMMEX has problems with fragmented memory, did you try > JEMM386? The error itself does not tell much - GPF at a 64k segment > boundary... Could mean that code jumped into an empty segment and > fell of its end. I've tried JEMMEX, HIMEMX+JEMM386, and MS HIMEM+EMM386. None of them help. > While INSTALL differs from normal running of apps (as child tasks of > SHELL) it should do so in the same way for different versions of DOS. > > In FreeDOS, you get - apart from a not complete PSP which is > probably normal for all INSTALL situations anyway...: Do you know exactly what is "incomplete" about the PSP? > STATIC VOID InstallExec(struct instCmds *icmd) > ... > exb.exec.env_seg = 0; > exb.exec.cmd_line = (CommandTail FAR *) args; > exb.exec.fcb_1 = exb.exec.fcb_2 = (fcb FAR *) 0xul; > ... init_DosExec(icmd->mode, &exb, filename) ... > ... > > ...where AL = mode is either 0 or 0x80 for low / high install which is a > non-standard use of > the high bit of AL. A possible bug in FreeDOS could be: > > COUNT DosExec(COUNT mode, exec_blk FAR * ep, BYTE FAR * lp) > ... > if (mode == LOAD && rc == SUCCESS) > fmemcpy(ep, &TempExeBlock, sizeof(exec_blk)); > ... > > This should probably use "if mode and 0x7f is load" instead? > However, INSTALL does not use LOAD mode, it uses LOADNGO... > And value LOAD_HIGH which is 0x80 is defined but never used. > > Other functions: DosExeLoader, DosComLoader, load_transfer... > > It might be possible that DosComLoader tries to give the COM all available > memory and > unless you specify otherwise when going TSR none is left for others, but > thats just guessing. It's a .COM, so all memory is allocated. As discussed above, it does get resized (or else I couln't allocate any new segments in conventional memory). > Is your TSR a COM or is it EXE? COM > Does it keep handles open? Only while installing. If I exit as a TSR (INT 21.31), I close the handles manually. If a normal exit (INT 21.4C), I let DOS handle it. > Does it use environment variables or other PSP related data? It does use enviroment variables, but only while installing. It does copy and manipulate the original PSP data if it terminates the original process after relocating, otherwise not. > Does it make a difference whether to INSTALL or INSTALLHIGH? No. ___ Bret -- Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Problem w/ CONFIG.SYS INSTALL
> Maybe JEMMEX has problems with fragmented memory, did you try JEMM386? Maybe not..? The issue does not appear to be related to JEMM (as usual). > The error itself does not tell much - GPF at a 64k segment boundary... > Could mean that code jumped into an empty segment and fell of its end. It would appear so. > It might be possible that DosComLoader tries to give the COM > all available memory and unless you specify otherwise when > going TSR none is left for others, but thats just guessing. Bret allocates other memory blocks, so he does resize the process first as necessary. (As an aside, such a situation should not make the kernel crash like that.) > Is your TSR a COM or is it EXE? Does it keep handles open? > Does it use environment variables or other PSP related data? > Does it make a difference whether to INSTALL or INSTALLHIGH? Educated guess: COM, no (shouldn't matter though), no (except as allowed, before the TSR is resident), no (Bret said so). Regards, Christian -- Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Problem w/ CONFIG.SYS INSTALL
Hi Bret, I checked the system's state with INSTALL=DEBUG.COM on a boot disk (Rugxulo's bare DOS disk, with a 2008-03-08 kernel, build 2038) and it appears fine to me. Memory that belongs to the configuration/initialization program is allocated to a PSP at segment 60h (!) which is properly set up except there's no MCB in front of it. (This means programs that display a process's name display garbage instead.) Some simple tests, including resident installation of a TSR that uses process relocation and the new Int21.4C method, passed. Tell us the programs that are used and their versions, that might help. Can you reproduce the crash with a minimal (dummy) TSR? Could you make a TSR available that causes this crash? Regards, Christian -- Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
Re: [Freedos-user] Problem w/ CONFIG.SYS INSTALL
Hi Bret, > I am writing a TSR, and having trouble getting it to work with FreeDOS. It > installs just fine in MSDOS, also also installs just fine from a FreeDOS > command prompt. But, when I try to install it using an INSTALL= or > INSTALLHIGH= from CONFIG.SYS, the machine hangs. The hang occurs right > after the program exits (INT 21.4Ch or INT 21.31h). If it helps in > troubleshooting, the TSR does allocate and leave in place some "extra" > memory segments that the TSR needs. That could be related. How do you allocate the extra memory and where? > I can install some of my other TSR's just fine from CONFIG.SYS. If I load > JEMMEX, when the problem program exits, JEMM displays an Error 0D at CS:EIP > X:Y, where X is always the CS of the previous TSR that was installed, and Y > is always either or 0001. If I don't use JEMM, the machine > just hangs with no error messages at all. Maybe JEMMEX has problems with fragmented memory, did you try JEMM386? The error itself does not tell much - GPF at a 64k segment boundary... Could mean that code jumped into an empty segment and fell of its end. > I've been looking at some of the source code for FreeDOS, but nothing has > revealed itself yet. I'm just wondering if someone could explain, at least > in general terms, what is "not there" or is "different" (other than the > shell program) when INSTALL is being used as opposed to an installation from > the command-line? While INSTALL differs from normal running of apps (as child tasks of SHELL) it should do so in the same way for different versions of DOS. In FreeDOS, you get - apart from a not complete PSP which is probably normal for all INSTALL situations anyway...: STATIC VOID InstallExec(struct instCmds *icmd) ... exb.exec.env_seg = 0; exb.exec.cmd_line = (CommandTail FAR *) args; exb.exec.fcb_1 = exb.exec.fcb_2 = (fcb FAR *) 0xul; ... init_DosExec(icmd->mode, &exb, filename) ... ... ...where AL = mode is either 0 or 0x80 for low / high install which is a non-standard use of the high bit of AL. A possible bug in FreeDOS could be: COUNT DosExec(COUNT mode, exec_blk FAR * ep, BYTE FAR * lp) ... if (mode == LOAD && rc == SUCCESS) fmemcpy(ep, &TempExeBlock, sizeof(exec_blk)); ... This should probably use "if mode and 0x7f is load" instead? However, INSTALL does not use LOAD mode, it uses LOADNGO... And value LOAD_HIGH which is 0x80 is defined but never used. Other functions: DosExeLoader, DosComLoader, load_transfer... It might be possible that DosComLoader tries to give the COM all available memory and unless you specify otherwise when going TSR none is left for others, but thats just guessing. Is your TSR a COM or is it EXE? Does it keep handles open? Does it use environment variables or other PSP related data? Does it make a difference whether to INSTALL or INSTALLHIGH? Regards, Eric -- Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev ___ Freedos-user mailing list Freedos-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-user
