Re: [Freedos-kernel] Regarding commit 1702 (large sector sizes)
Note that the 1702 changes were effectively reversed by 1704. The 1705 changes are unrelated. Wrong From -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] Regarding commit 1702 (large sector sizes)
Dear PerditionC, UBYTE DiskTransferBuffer[MAX_SEC_SIZE]; wastes 3,5 KB low memory for *everybody*, not only when it's needed. regarding how much time we have spend until we had 64 byte free I think this is a bad idea please see my previous messages, I know exactly how much space is wasted and clearly stated this along with statement 512 will be default (and was already changed to such earlier) I also think that these experiments should NOT be in the stable branch. fork it, and make another 'unstable' branch if you want to experiment, but don't experiment with code that is supposed to be stable. Regards, Tom These are not experiments, these are - in the current state - even useless experiments. this is me working to improve compatibility with MSDOS and existing though rare disks. a) the only problematic disks have a 4K sectors. your fix doesn't work with them. b) these disks don't exist, as there is currently no USB stack that supports 4K sectors I don't mind if you work on an unstable branch, or on your own local copy of the freedos kernel. however generating AGAIN an unstable kernel 2041 is a real bad idea. and tagging current state (after the questionable 1705) '2041' without further discussion is shit. Tom -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] Regarding commit 1702 (large sector sizes)
Dear PerditionC, UBYTE DiskTransferBuffer[MAX_SEC_SIZE]; wastes 3,5 KB low memory for *everybody*, not only when it's needed. regarding how much time we have spend until we had 64 byte free I think this is a bad idea I also think that these experiments should NOT be in the stable branch. fork it, and make another 'unstable' branch if you want to experiment, but don't experiment with code that is supposed to be stable. Regards, Tom -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] Regarding commit 1702 (large sector sizes)
On Tue, Feb 7, 2012 at 9:51 AM, Tom Ehlert t...@drivesnapshot.de wrote: Dear PerditionC, UBYTE DiskTransferBuffer[MAX_SEC_SIZE]; wastes 3,5 KB low memory for *everybody*, not only when it's needed. regarding how much time we have spend until we had 64 byte free I think this is a bad idea please see my previous messages, I know exactly how much space is wasted and clearly stated this along with statement 512 will be default (and was already changed to such earlier) I also think that these experiments should NOT be in the stable branch. fork it, and make another 'unstable' branch if you want to experiment, but don't experiment with code that is supposed to be stable. Regards, Tom These are not experiments, this is me working to improve compatibility with MSDOS and existing though rare disks. I purposely left a default value larger than needed so I could track down a kernel issue with memory allocation during initialization. This issue still exists and I believe may be one of the reasons why there are occasional reports of invalid opcode/crash during boot. MSDOS sets the value based on known devices during startup, this is documented in books describing it. The FD kernel has been hardcoded for 512 bytes. My plan was first to ensure sizes 512 bytes work then make the changes to allow user to choose larger size than 512 so only those who need the different value are penalized with increased memory usage and can use FD kernel with their drive while everyone else only has minimally larger kernel but more like MSDOS. Part 1, the changes so sizes 512 can be used was committed and uncovered a memory allocation issue. I put part 2 on hold because this seemed important to find - instead I will leave the 512 hard coded which rarely triggers the bug. The experimental 4K build is based on my forked 0xdc (DOSC OEM id) kernel, and it is where I do most of my changes before merging/testing with the 0xfd kernel. Jeremy. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] Regarding commit 1702 (large sector sizes)
Hi Jeremy, Chris, others, I am aware they are different... That is risky - even for a demonstration implementation, consistency seems important to keep code maintainable... dynamically adjusted same as MSDOS. That currently is not done. that has low priority imho, as drives with large sectors are a temporary thing, will be gone with GPT partitions. For the same reason, I suggest to NOT put efforts into incompatible put 8 small sectors in each 4k buffer on mixed sector size computers BUFFERS code of any sorts. Note that larger caches already do put multiple sectors in one slot, but for other reasons and without mixing. BUFFERS=COUNT#,READAHEAD#,SECTORSIZE# or would it be better to add a whole new command MAXSECSIZE=SECTORSIZE# ? (the latter seems simpler so the one I will plan on for now). MAXSECSIZE will be fine, thanks :-) And really useful. Changeable drives are en vogue, so figuring out needed sector sizes at boot time is not always useful, and it is probably a lot of work as well... For my personal taste, a MAXSECSIZE, with a default of 512, would be a good small-amount-of-code way of having enough large sector support for those who need it. When you encounter a drive/partition with unsuitable sector sizes for the current settings, skip it with a message. agreed, though currently any value 3KB causes memory corruption evil again... you said you already have an idea to debug? I don't see us using max sector size 512, but using tdsk with smaller than 512 sector size does seem to work - though at this The last time when we discussed OTHER sector sizes on the lists, it was exactly for that reason - small RAMDISKS with small sectors and less fragmentation at the expense of slightly larger FATs. So I do think it would be useful to support BUT only with sectors of at least 64 bytes size (FAT32: 128, although without FSINFO). I suggest to NOT spend any code for BPB wraps sectors or such: sectors below 128 bytes are just way too exotic and support for dir entry sized micro sectors is more a theoretical exercise. Fun extra exercise: Make a bootable FAT12/16/32 sector with 128 bytes per sector, loading a 2nd stage loader (reserved sectors). Then construct a drive and BIOS for such drives, or ROM/MEMDISK. Regards, Eric -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
[Freedos-kernel] Regarding commit 1702 (large sector sizes)
Hello devs, especially Jeremy, I reviewed your recent commit 1702 to build 2041 regarding sector sizes larger than 512. Here are my thoughts and suggestions. - First, dsk.h defines the maximum sector size (MAX_SEC_SIZE) as 2048 (as the commit comment says) while the relevant LoL entry (maxsecsize) in kernel.asm is initialised to 4096. - Second, maxsecsize is erroneously used once in dsk.c while everything else uses the value from the dsk.h definition. - Third, if I'm not entirely mistaken, the patch makes all buffers grow to 2048 bytes, even in the absence of any block devices with large sectors. This will unnecessarily waste memory on most systems. MS-DOS initializes its 'maxsecsize' to 512 and will increase it (along with each buffer's size) during CONFIG processing whenever a block device with larger sectors is registered. (As far as I know/deduced, this works up to a sector size of 8192 bytes.) This way, if the need for large buffers wasn't apparent during CONFIG, they will stay smaller. This scheme could obviously be extended by additional tools/interfaces to increase the buffer size later on, and a CONFIG option to artificially increase it in anticipation of a post-CONFIG requirement. - Fourth, apart from the other points, I fail to see why you should not be able to simply increase MAX_SEC_SIZE to 8192. 4096 is definitely in need as evidenced by the list's requests for precisely that size. - Fifth, I think MS-DOS supports sector sizes below 512 too. If you were to look into that, I believe it wouldn't be too hard to implement. Due to the filesystem (size of directory entries), 32 bytes is the absolute minimum. I believe all sizes below 512 will require a slight change to the BPB loading (What to do about the sector end signature location?) while you might have to get creative for the very small sizes, because a full BPB would not fit. I'd suggest just expecting the BPB to stretch several sectors then; the filesystem already allows reserved sectors to be properly implemented using the BPB. Regards, C. Masloch -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
Re: [Freedos-kernel] Regarding commit 1702 (large sector sizes)
- First, dsk.h defines the maximum sector size (MAX_SEC_SIZE) as 2048 (as the commit comment says) while the relevant LoL entry (maxsecsize) in kernel.asm is initialised to 4096. Yes - this is simply for testing purposes; I plan to default both to 512. I would prefer both set from the same define to ensure they stay in sync - I am aware they are different, I am tracking down how to fix support for larger than 2KB sectors and MAX_SEC_SIZE will be set to 4096 to test or possibly eliminated in favor of using only maxsecsize. Ah, yes, Eric speculated that this might be a proof of concept for now, lacking an unstable branch. I happen to keep track of kernel commits recently. Sorry that I already reviewed this premature implementation! Values should definitely be tied later, either by consistently using MAX_SEC_SIZE, or (preferably) by initialising maxsecsize and the init buffer's size with MAX_SEC_SIZE and then always referring to maxsecsize afterwards. - Third, if I'm not entirely mistaken, the patch makes all buffers grow to 2048 bytes, even in the absence of any block devices with large sectors. This will unnecessarily waste memory on most systems. yes, buffers are allocated using size of max sector size so will waste lots of memory if 512 byte sectors are used. I will be committing updates to ensure max sector size defaults to 512 so as to avoid waste of space. Regarding that, I once considered that unused buffer space could be used to load multiple smaller sectors. Say, if you use 4096-byte buffers but a particular drive has 512-byte sectors, you can load eight sectors at once whenever reading. Would have to align sector reads to those boundaries though, plus a special check for the end of the file system. Additionally, the written bit would either have to be expanded to a bitmap, or you'd have to write eight sectors together all the time too. This would probably abandon any MS-DOS compatibility of the buffer layout, if the kernel currently does have any (not sure). This scheme could obviously be extended by additional tools/interfaces to increase the buffer size later on, and a CONFIG option to artificially increase it in anticipation of a post-CONFIG requirement. agreed, a config option to allow manually specifying a larger value than needed at boot time - should buffers be extended to BUFFERS=COUNT#,READAHEAD#,SECTORSIZE# or would it be better to add a whole new command MAXSECSIZE=SECTORSIZE# ? (the latter seems simpler so the one I will plan on for now). Yes, Eric suggested that one too. Plus, I think it's preferable because then a user can specify MAXSEXSIZE without having to specify the first two values for BUFFERS. I don't see us using max sector size 512, but using tdsk with smaller than 512 sector size does seem to work - though at this time I don't see me researching if it correctly works or merely happens to work on my test setup. I think you need to review at least dsk.c around line 388, where it says pbpbarray-bpb_nbyte % 512 now. I didn't look into what that check actually does, but it sure seems like it would have to be adjusted. While you're at it, you can make it the proper check for powers of two between 32 (or 64,128, regarding (E)(N)BPB size) and 8192 (or 2048 for now). currently any value 3KB causes memory corruption if too many buffers are specified (I have successfully gotten 4KB sectors to finish booting by changing buffers=2). I currently have tracked down part of my issue to allocation problems if buffers is too large, but still working on relearning the init time segment relocations and order to determine a proper fix for buffer allocations. It would help if printf worked correctly in all the init code... The virtues of high-level language kernel development ;) Regards, C. Masloch -- Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 ___ Freedos-kernel mailing list Freedos-kernel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/freedos-kernel
