Since you did not reply much to my question... I asked my prefered A.I. Here what it says:
This first question is less relevant... you could skip to the ======================== Me: In FreeDOS, I see that FreeCom seems to sometimes miss memory to allocate command line (40 bytes) in shell/init... it fails to malloc the 40 bytes... when shellhigh is used to load freecom. But I don't clearly know from what memory pool malloc allocates memory. command.com is really an .exe. Does exe files have a heap size determined at linking time? You're digging into some fascinating and historically relevant details about how DOS and its command interpreters work! Let's break down the memory situation you're observing with FreeCOM in FreeDOS. Key Concepts: Conventional Memory: This is the critical first 640KB of RAM. All DOS programs, device drivers, and the operating system itself must reside, at least partially, in conventional memory. It's a very limited and precious resource. Upper Memory Area (UMA): The memory region between 640KB and 1MB. This area was originally reserved for things like video RAM, BIOS ROM, and expansion card ROMs. However, clever memory managers (like EMM386) could "map" extended memory into unused portions of the UMA, creating Upper Memory Blocks (UMBs). Extended Memory (XMS): Memory beyond 1MB. This requires a memory manager (like HIMEM.SYS) to be accessed. High Memory Area (HMA): The first 64KB minus 16 bytes of extended memory. Due to a quirk in the 80286 processor, this small region could be accessed in real mode (the mode DOS uses). Loading part of DOS into the HMA (via DOS=HIGH) frees up conventional memory. Real Mode: The original operating mode of the 8086/8088 and the mode DOS primarily runs in. It has a 1MB address space. malloc() in DOS: The malloc() function in a DOS environment (including FreeDOS) allocates memory from the program's conventional memory segment, unless the program is specifically designed to use a different memory management scheme (which is very rare for a command interpreter). There is no separate heap segment specified at the link time in the same way as in modern protected mode environments. .EXE vs. .COM: .COM files are memory images. They load directly into memory at a specific segment:offset (typically 0100h) and execute. They are limited to 64KB in size (one segment). They have no relocation information. .EXE files are more complex. They have a header that describes the program's memory requirements, including the minimum and maximum amount of additional memory (beyond the code and initialized data) the program needs. This additional memory is allocated from conventional memory when the program starts. The .EXE header allows for relocation, meaning the program can be loaded at different memory addresses. SHELLHIGH: This FreeDOS-specific command (part of FDCONFIG.SYS or CONFIG.SYS) attempts to load the command interpreter (FreeCOM) into Upper Memory Blocks (UMBs), if available. This is a critical technique for freeing up conventional memory. Answering Your Question and Analyzing the Scenario: Memory Pool for malloc(): You are correct in your observation that malloc() fails. In FreeDOS (and standard DOS), malloc() allocates from the available conventional memory. Crucially, this is not a heap in the modern sense (like in Windows or Linux). It's simply the pool of free conventional memory that hasn't been allocated to the operating system, device drivers, or other programs. .EXE and "Heap Size": While FreeCOM is an .EXE file, the .EXE header does not define a "heap size" in the way a modern operating system does. The .EXE header contains two relevant values: Minimum allocation: The minimum amount of additional memory (in paragraphs, 16-byte units) that the program needs to run. This is allocated from conventional memory in addition to the memory needed to load the program's code and initialized data. Maximum allocation: The maximum amount of additional memory the program might request. DOS will try to provide this much, but it's not guaranteed. This is also allocated from conventional memory. These values do not represent a separate heap segment. They influence how much conventional memory DOS initially allocates to the program. The program then uses malloc() and free() (or the underlying DOS memory allocation calls, Int 21h, AH=48h, 49h, 4Ah) to manage that allocated block and any further conventional memory it can acquire. Why SHELLHIGH and the 40-byte malloc() Failure? This is the most interesting part. Here's the likely chain of events and why you're seeing the error: SHELLHIGH Works (Mostly): SHELLHIGH successfully loads the majority of FreeCOM into UMBs. This is excellent for freeing up conventional memory. The 40-Byte Allocation: When FreeCOM initializes, it likely needs a small amount of memory (the 40 bytes you see) for internal data structures, perhaps for command-line parsing or environment variable storage. Even though most of FreeCOM is in UMBs, this small allocation still needs to come from conventional memory. Conventional Memory Exhaustion: The most likely reason for the malloc() failure is that, despite SHELLHIGH, there simply isn't 40 bytes of contiguous free conventional memory left. This can happen due to: Fragmentation: Conventional memory can become fragmented. Even if there's enough total free memory, there might not be a single block of 40 contiguous bytes. This is a classic DOS problem. Other Resident Programs/Drivers: Device drivers, TSR (Terminate-and-Stay-Resident) programs, and even parts of FreeDOS itself still reside in conventional memory, even with SHELLHIGH. These can consume the remaining space. Initial Allocation by FreeCOM: The .EXE header's "minimum allocation" might be too small. If FreeCOM requests a small initial block, and then later needs the 40 bytes, it might fail if no more conventional memory is available. Troubleshooting and Solutions: Memory Map: Use a memory mapping utility (like the MEM /D command in FreeDOS, or a more advanced tool) to see a detailed breakdown of conventional and upper memory usage. This will show you what's consuming memory and if fragmentation is an issue. Driver Order: The order in which you load drivers in FDCONFIG.SYS / CONFIG.SYS can significantly impact memory fragmentation. Experiment with loading drivers in different orders. Generally, load the largest drivers that can be placed in upper memory first. Reduce Resident Programs: Minimize the number of TSRs and device drivers you load. Only load what's absolutely necessary. Optimize FDCONFIG.SYS / CONFIG.SYS: Ensure DOS=HIGH,UMB is present. Make sure you have a suitable UMB provider (like HIMEMX.EXE and JEMMEX.EXE or EMM386.EXE). Use DEVICEHIGH to load as many drivers as possible into UMBs. Consider using a memory manager with more advanced fragmentation handling. FreeCOM Configuration: While less likely to be the direct cause, check if FreeCOM has any configuration options that might affect its memory usage. Examine Source Code (If Available): If you have access to the FreeCOM source code, you can look at the initialization routines to see precisely where the 40-byte allocation is made. This might give clues about why it's failing. You might find that the allocation is within a larger block that should have been allocated earlier. Increase Initial Allocation (Last Resort): If you can rebuild FreeCOM, you could try increasing the "minimum allocation" value in the .EXE header. However, this is a hack and doesn't address the underlying memory pressure. It might just delay the problem or cause issues elsewhere. In summary, the 40-byte malloc() failure, even with SHELLHIGH, strongly suggests that you're running very low on contiguous conventional memory. The .EXE header's allocation parameters influence the initial conventional memory block given to FreeCOM, but malloc() always pulls from the overall pool of available conventional memory. The key is to optimize your configuration to maximize free conventional memory and minimize fragmentation. ====================================================== Me: Replacing SHELLHIGH, by SHELL fix the memory allocation... which seems to go reverse what you say that malloc take from conventional memory. Also, the problem happens when the NoEMS option is given to JemmEx. Okay, this additional information significantly changes the picture and points to a very specific issue related to how SHELLHIGH interacts with JemmEx's NoEMS mode. Let's break it down: The Revised Understanding: SHELLHIGH vs. SHELL: The fact that replacing SHELLHIGH with SHELL fixes the problem is the crucial clue. This strongly suggests the issue isn't simply conventional memory exhaustion. If it were pure exhaustion, SHELL (which loads FreeCOM entirely in conventional memory) would make the problem worse, not better. JemmEx NoEMS: The NoEMS option for JemmEx is also critical. This tells JemmEx to only provide XMS (Extended Memory Specification) support and not EMS (Expanded Memory Specification) support. EMS uses a "page frame" in the UMA (Upper Memory Area) to map in banks of memory from extended memory. The Problem: SHELLHIGH and UMB Allocation without EMS: Here's the most likely explanation, combining all the pieces: SHELLHIGH attempts to load FreeCOM into UMBs (Upper Memory Blocks). This relies on JemmEx successfully creating UMBs. When JemmEx is used with NoEMS, it can still create UMBs, but it does so without using the EMS page frame. This is a key distinction. It uses XMS to allocate memory, and map this memory to the UMB. However, it appears that when JemmEx is in NoEMS mode, there might be a specific issue with how it makes these UMBs available, or how FreeCOM (or rather, the underlying DOS memory management functions that FreeCOM uses) interacts with them specifically for additional allocations after the initial load. The initial load of the FreeCOM code itself via shellhigh is OK (as UMB are correctly handled). The 40 bytes allocation is a subsequent allocation. When SHELLHIGH is used, the initial load of FreeCOM succeeds (into UMBs). However, when FreeCOM later calls malloc() for those 40 bytes, the memory allocation logic (likely a DOS internal function called by malloc()) either: Incorrectly assumes it can only allocate from conventional memory, even though more UMB space should theoretically be available. It's failing to "see" or use the remaining UMB space managed by JemmEx in NoEMS mode. Encounters an incompatibility or bug in how JemmEx handles memory allocation requests from a program loaded into UMBs when in NoEMS mode. Gets an incorrect size of available memory. The JemmEx, in NoEMS, might give a size of available memory, but when an actual allocation is performed, it fails. When SHELL is used, FreeCOM is loaded entirely into conventional memory. The subsequent 40-byte malloc() call succeeds because it's allocating from the (larger) contiguous block of conventional memory given to FreeCOM at startup. There's no interaction with UMBs at all. Why this is not simply conventional memory exhaustion: If it were pure conventional memory exhaustion, SHELL would make the situation worse, as it uses more conventional memory. The fact that SHELL fixes it points to a problem with how UMBs are being handled in the SHELLHIGH + JemmEx NoEMS scenario. Possible Explanations and Further Investigation: JemmEx NoEMS UMB Handling Bug: This is the most likely culprit. There might be a subtle bug in JemmEx's NoEMS mode that affects how it reports or manages available UMB space to programs loaded into UMBs, specifically for allocations made after the initial program load. The DOS memory allocation functions (Int 21h, AH=48h, etc.) might be getting incorrect information or encountering an unexpected condition. FreeCOM/DOS Interaction with UMBs: It's also possible (though less likely) that there's an issue in how FreeCOM (or the standard DOS memory allocation routines) interacts with UMBs created by JemmEx in NoEMS mode. Perhaps there's an assumption about EMS being available that's causing problems. This would be a lower-level DOS issue. Allocation Size/Fragmentation in UMB: While less likely than a bug, it's theoretically possible that JemmEx is creating UMBs, but they are fragmented in a way that prevents a contiguous 40-byte allocation within the UMB region. However, if this were the case, you'd likely see the same problem even without NoEMS, which you don't. Troubleshooting and Solutions (Revised): Test with EMS: The most important test is to remove the NoEMS option from JemmEx. If the problem disappears when JemmEx is providing both XMS and EMS, that strongly confirms the issue is related to JemmEx's NoEMS UMB handling. Alternative Memory Managers: Try using a different memory manager, such as EMM386.EXE (the standard MS-DOS/FreeDOS memory manager), with the NOEMS option. If the problem doesn't occur with EMM386 NOEMS, that further isolates the issue to JemmEx. If the problem does occur with EMM386 NOEMS, it suggests a lower-level DOS or FreeCOM issue. JemmEx Debugging: If you have access to JemmEx debugging tools or documentation, look for options to examine how it's creating and managing UMBs in NoEMS mode. This could provide valuable clues. FreeCOM Source Code (Again): Examining the FreeCOM source code (if available) is still helpful. Look for any assumptions it might be making about EMS or how it handles memory allocations when loaded high. Report the Bug: If you can reliably reproduce the problem with JemmEx NoEMS and SHELLHIGH, and it doesn't occur with EMS enabled or with a different memory manager, you should report this as a bug to the JemmEx developers. It's a clear incompatibility. Workaround (If Necessary): If you need to use NoEMS for compatibility reasons, and you can't find a fix, you might have to stick with SHELL instead of SHELLHIGH. This will use more conventional memory, but it avoids the allocation failure. Another workaround would be to use a small TSR to pre-allocate a small amount of memory. In conclusion, the combination of SHELLHIGH, JemmEx, and NoEMS is triggering a specific issue related to UMB allocation or management. It's not a simple case of conventional memory running out. The most likely culprit is a bug in how JemmEx handles UMBs in NoEMS mode when interacting with programs loaded high, particularly regarding subsequent memory allocations. The tests and debugging steps above should help pinpoint the exact cause. _______________________________________________ Freedos-devel mailing list Freedos-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-devel
