Re: [oi-dev] [OpenIndiana-discuss] Hipster 2020.10 text installer ISO Wow!!!!
I tried that, but probably not at the appropriate time. I'd likely done the "dd if=/dev/zero..." already. I was annoyed and got out the big hammer. So by the time I tried it, it didn't matter. Also it was a FreeBSD "zroot" label. After I play with the AHCI stuff I may repeat the situation. Have Fun! Reg On Tuesday, March 2, 2021, 01:41:08 PM CST, Joshua M. Clulow wrote: On Tue, 2 Mar 2021 at 07:37, Reginald Beardsley via oi-dev wrote: > There clearly is a need for a "zfs unlabel " option to remove label > cruft. I ran into the problem with a 3 TB disk 7-8 years ago. On that > occasion I was a bit more thoughtful and searched for the pool label and did > a more surgical elimination. That was actually because the problem was the > duplicate label at the end of the disk. I believe you want "zpool labelclear", as per: https://illumos.org/man/1M/zpool -- Joshua M. Clulow http://blog.sysmgr.org ___ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] [OpenIndiana-discuss] Hipster 2020.10 text installer ISO Wow!!!!
On Tue, 2 Mar 2021 at 07:37, Reginald Beardsley via oi-dev wrote: > There clearly is a need for a "zfs unlabel " option to remove label > cruft. I ran into the problem with a 3 TB disk 7-8 years ago. On that > occasion I was a bit more thoughtful and searched for the pool label and did > a more surgical elimination. That was actually because the problem was the > duplicate label at the end of the disk. I believe you want "zpool labelclear", as per: https://illumos.org/man/1M/zpool -- Joshua M. Clulow http://blog.sysmgr.org ___ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] [OpenIndiana-discuss] Hipster 2020.10 text installer ISO Wow!!!!
Toomas, Thank you again. Very helpful. The Z400 will only support IDE mode. ACHI+RAID is the other option which I think only works with Windows. I know it will not work with Solaris and feel fairly certain it doesn't work with Linux or *BSD either. As pstack is not familiar to me I didn't interpret the output properly. It certainly would make good sense in a multicore world for system calls to be implemented in threads. That's a very different matter from what I assumed from the ptrace output. As the 4k sector size has been buried behind some layer of software in current drives I'm not sure how disks >2TB actually communicate the size of the disk. I'm curious and am going to try to obtain a fresh drive to determine what I clobbered. However, if the label information for the disk geometry is in the first 20K I think I can reconstruct the geometry from a dump I made of that. In any case it's an interesting technical challenge. I'm sure the format code is a mess. Old code usually is, and the last time I was asked for a disk speed was almost 30 years ago. In the intervening time I was the sole support for an aggregate of 2.5 million lines of scientific code. One especially lovely piece of work was about 400,000 lines in which the sole comment was the author's name. Fortunately, he had taken a software engineering course and chose good names for for functions and variables and I knew the science well. But he wrote about a dozen copy, paste, modify programs and I had to fix the exact same bug in many places with different names. I shall be reasonably conversant with the on disk structure of ZFS shortly as the books are due to arrive on Thursday. There clearly is a need for a "zfs unlabel " option to remove label cruft. I ran into the problem with a 3 TB disk 7-8 years ago. On that occasion I was a bit more thoughtful and searched for the pool label and did a more surgical elimination. That was actually because the problem was the duplicate label at the end of the disk. I came along just after SMD disks had disappeared from the landscape. Reading instructions for setting those up made me very glad that they were gone. I'm fairly sure the geometry stuff I ran into was part of that process and came from BSD. Have Fun! Reg On Tuesday, March 2, 2021, 02:35:33 AM CST, Toomas Soome wrote: For some unknown reason, your disks are recognized as PATA (IDE) disks and not SATA/AHCI. That alone is bad because IDE is slow and cmdk driver is not the fastest one either. You should start with checking out if AHCI (SATA) is available. That alone should not get to the crash but bad things happen. > Error: can't open disk '/dev/rdsk/c4d0p0'. Now this line is actual error - we do try to open whole disk device (*p0 is special device which does allow access to entire disk surface), but we fail. The bug there is obviously about this error - we failed to open the disk, we should not continue from this point, but we still do and the punishment is segmentation fault because some needed data structures are not properly set. It is not obvious why the disk open is failing there, the error code is not printed and you seem to have root rights. From this point, I’d run truss format and would seek out the failing open() to get the error code, perhaps it would explain. Yes, the code of format command is mess (I have been in it more than I would like to;), also we would need better tooling to handle disk partitioning, allowing easy scripting etc, something *like* freebsd gpart has been in my mind for a long time, but I haven't had time to think about it more… rgds, toomas ___ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] [OpenIndiana-discuss] Hipster 2020.10 text installer ISO Wow!!!!
> On 2. Mar 2021, at 13:08, cretin1997 wrote: > > ‐‐‐ Original Message ‐‐‐ > On Tuesday, March 2, 2021 3:35 PM, Toomas Soome via openindiana-discuss > wrote: >> >> Yes, the code of format command is mess (I have been in it more than I would >> like to;), also we would need better tooling to handle disk partitioning, >> allowing easy scripting etc, something like freebsd gpart has been in my >> mind for a long time, but I haven't had time to think about it more… >> >> rgds, >> toomas >> > > Too bad FreeBSD's gpart seemed to depend on FreeBSD's GEOM framework. Porting > it from FreeBSD seemed to not easy. Luckily, we have a snapshot of FreeBSD > before they switched to GEOM, DragonflyBSD. Do you think DragonflyBSD's gpt > tool is easier to port, sir? > > https://man.dragonflybsd.org/?command=gpt=8 no, it is not about porting it, but the command line logic and functionality. we do have format, fdisk, prtvtoc, fmthard, all doing some subset of tasks and even if you can feed some input data, the customization would need awk/sed and fancy scripting:D See /bin/usbgen for example;) rgds, toomas ___ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] [OpenIndiana-discuss] Hipster 2020.10 text installer ISO Wow!!!!
> On 2. Mar 2021, at 02:45, Reginald Beardsley wrote: > > > It is with deep regret that I say this. > > I shall not be working on Illumos/OI. Below is the terminal output using the > Hipster 2020.10 GUI install disk. I've always been amused by the enthusiasm > for threads. I bought John Lakos' book when it came out and unlike most > actually read much of it. I even posted a page on my office door. Many months > after I'd taken it down the entire project ground to a complete halt while > they refactored the sysem to address the linker issue I'd highlighted on my > office door. I've also read many books on operating systems and all the > issues that must be handled 100% correctly to make multiple processes running > in the same address space work reliably. Can you state the "Dining > Philospher's" problem? I can. This stuff is hard. That's why protected memory > is so important to a robust system. Threads, like shared memory are valuable > tools, but are best used with discretion. And only when there is no suitable > alternative. > > Rather well known, at least to my generation, is something called "Amdahl's > law". It's not particularly insightful. It just makes the observation that > pefectly optimizing something which does not take a lot of time has > neglegible benefit. > > That a seldom used admin utility would be rewritten as a threaded application > says that those responsible for this idiocy were solely interested in adding > "threaded programming" to their resumes. I neither know nor care if the > programmer was at fault or the manager who permitted it was at fault. In any > case it is so stupid as to begger belief. Sadly I have seen many people do > something similar. > > I have *never* written a program using pthreads or any other threading > library. I dealt with things like seismic imaging problems which ran on > 10,000 -20,000 cores or more for 8-10 days processing 10-30 TB of input and > generating similar output volumes. Fortunately these were trivially parallel > and could be broken up into separate jobs for each core. With that many > machines running, checkpointing and restarting was a critical issue. You > *were* going to lose a few machines during a run. So robust recovery was a > big deal. That much compute costs a lot of money and starting the whole thing > over simply not viable. If you did, you went out of business. The only upside > was because such users were running 50,000 cores 24x7 in a single location, > they got baseload electrical rates. Though it was not uncommon to have to > find additonal locations because the electrical power feed to that location > had been exceeded. In Houston, the AC costs as much or more than the > computers to run. > > I've seen an inappropriate tool used far too many times for the sole reason > that some one wanted to put something on their resume. For many years I'd > never written an IP based client server application. I'd taught other people > how to do it, but I had not done it myself for the simple reason that it was > not an appropriate tool for my assigned tasks. I did eventually get to write > a UDP usage logger that tracked usage statistics for the package we were > fielding. That was fun. I generated graphics that showed where each business > affiliate was in the exploration cycle from what programs they were running. > > Much too my surprise and some dismay, Mac OS X is the best supported > commercial Unix today. So I may well become a Mac user. > > A not very happy, > Reg > > jack@openindiana:/jack$ sudo /bin/su > root@openindiana:/jack# format -e > Searching for disks...done > > > AVAILABLE DISK SELECTIONS: > 0. c4d0 > /pci@0,0/pci-ide@1f,2/ide@0/cmdk@0,0 > Specify disk (enter its number): 0 > For some unknown reason, your disks are recognized as PATA (IDE) disks and not SATA/AHCI. That alone is bad because IDE is slow and cmdk driver is not the fastest one either. You should start with checking out if AHCI (SATA) is available. That alone should not get to the crash but bad things happen. > Error: can't open disk '/dev/rdsk/c4d0p0'. Now this line is actual error - we do try to open whole disk device (*p0 is special device which does allow access to entire disk surface), but we fail. The bug there is obviously about this error - we failed to open the disk, we should not continue from this point, but we still do and the punishment is segmentation fault because some needed data structures are not properly set. It is not obvious why the disk open is failing there, the error code is not printed and you seem to have root rights. From this point, I’d run truss format and would seek out the failing open() to get the error code, perhaps it would explain. Yes, the code of format command is mess (I have been in it more than I would like to;), also we would need better tooling to handle disk partitioning, allowing easy scripting etc, something *like* freebsd gpart has
Re: [oi-dev] [OpenIndiana-discuss] Hipster 2020.10 text installer ISO Wow!!!!
In light of the use cases for format(1m) I find it difficult to imagine why it would need to communicate with anything else other than to assert a mutex to prevent other processes from doing something stupid while it ran. I have the most recent monographs on FreeBSD, Solaris Internals and ZFS arriving later this week. *If* there is anything worth saying I shall comment once I've read them. But my admiration for Solaris has been severely affected. "Tis but a scratch, but it is enough. If you look for me on the morrow, you shall find me a very grave man." With apologies to Will for any failures of my memory. Reg On Monday, March 1, 2021, 07:11:15 PM CST, Joshua M. Clulow wrote: On Mon, 1 Mar 2021 at 16:45, Reginald Beardsley via oi-dev wrote: > That a seldom used admin utility would be rewritten as a threaded application > says that those responsible for this idiocy were solely interested in adding > "threaded programming" to their resumes. I neither know nor care if the > programmer was at fault or the manager who permitted it was at fault. In any > case it is so stupid as to begger belief. Sadly I have seen many people do > something similar. If you take a look, format doesn't create any threads directly -- not that it would be a problem if it did! > root@openindiana:/jack# pstack core > core 'core' of 3749: format -e > - thread# 1 / lwp# 1 - > 080658cb init_globals (8095bd8, 0, 8047d18, 806837c, 0, fefc3bb9) + 51f > 08068385 c_disk (fef70548, fef70548, 3, 2, 4, 806d608) + 3c8 > 08065bc1 main (8047d6c, fef685c8, 8047da8, 8057e47, 2, 8047dd4) + 2b4 > 08057e47 _start_crt (2, 8047dd4, fefd0c6f, 0, 0, 0) + 96 > 08057d1a _start (2, 8047eb4, 8047ebb, 0, 8047ebe, 8047ed2) + 1a > - thread# 2 / lwp# 2 - > feeed05e __door_return (0, 0, 0, 0, feb90240, fef5b000) + 2e > feed287c door_create_func (0) + 4a > feee7551 _thrp_setup (feb90240) + 81 > feee7800 _lwp_start (feb90240, 0, 0, 0, 0, 0) > - thread# 3 / lwp# 3 - > feee7859 __lwp_park (8097828, 8097838, 0, 0, 0, 0) + 19 > feee1607 cond_wait_queue (8097828, 8097838, 0) + 5f > feee1d82 __cond_wait (8097828, 8097838) + b1 > feee1e1c cond_wait (8097828, 8097838) + 2e > fe8a4986 subscriber_event_handler (8095ce0) + 9d > feee7551 _thrp_setup (feb90a40) + 81 > feee7800 _lwp_start (feb90a40, 0, 0, 0, 0, 0) > root@openindiana:/jack# Those other threads, beyond the main thread from format that is crashing, are almost certainly from libsysevent on behalf of the disk management library. They are an implementation detail of the door calls it uses to subscribe to events about devices coming and going in the system. Cheers. -- Joshua M. Clulow http://blog.sysmgr.org ___ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] [OpenIndiana-discuss] Hipster 2020.10 text installer ISO Wow!!!!
On Mon, 1 Mar 2021 at 16:45, Reginald Beardsley via oi-dev wrote: > That a seldom used admin utility would be rewritten as a threaded application > says that those responsible for this idiocy were solely interested in adding > "threaded programming" to their resumes. I neither know nor care if the > programmer was at fault or the manager who permitted it was at fault. In any > case it is so stupid as to begger belief. Sadly I have seen many people do > something similar. If you take a look, format doesn't create any threads directly -- not that it would be a problem if it did! > root@openindiana:/jack# pstack core > core 'core' of 3749: format -e > - thread# 1 / lwp# 1 - > 080658cb init_globals (8095bd8, 0, 8047d18, 806837c, 0, fefc3bb9) + 51f > 08068385 c_disk (fef70548, fef70548, 3, 2, 4, 806d608) + 3c8 > 08065bc1 main (8047d6c, fef685c8, 8047da8, 8057e47, 2, 8047dd4) + 2b4 > 08057e47 _start_crt (2, 8047dd4, fefd0c6f, 0, 0, 0) + 96 > 08057d1a _start (2, 8047eb4, 8047ebb, 0, 8047ebe, 8047ed2) + 1a > - thread# 2 / lwp# 2 - > feeed05e __door_return (0, 0, 0, 0, feb90240, fef5b000) + 2e > feed287c door_create_func (0) + 4a > feee7551 _thrp_setup (feb90240) + 81 > feee7800 _lwp_start (feb90240, 0, 0, 0, 0, 0) > - thread# 3 / lwp# 3 - > feee7859 __lwp_park (8097828, 8097838, 0, 0, 0, 0) + 19 > feee1607 cond_wait_queue (8097828, 8097838, 0) + 5f > feee1d82 __cond_wait (8097828, 8097838) + b1 > feee1e1c cond_wait (8097828, 8097838) + 2e > fe8a4986 subscriber_event_handler (8095ce0) + 9d > feee7551 _thrp_setup (feb90a40) + 81 > feee7800 _lwp_start (feb90a40, 0, 0, 0, 0, 0) > root@openindiana:/jack# Those other threads, beyond the main thread from format that is crashing, are almost certainly from libsysevent on behalf of the disk management library. They are an implementation detail of the door calls it uses to subscribe to events about devices coming and going in the system. Cheers. -- Joshua M. Clulow http://blog.sysmgr.org ___ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] [OpenIndiana-discuss] Hipster 2020.10 text installer ISO Wow!!!!
It is with deep regret that I say this. I shall not be working on Illumos/OI. Below is the terminal output using the Hipster 2020.10 GUI install disk. I've always been amused by the enthusiasm for threads. I bought John Lakos' book when it came out and unlike most actually read much of it. I even posted a page on my office door. Many months after I'd taken it down the entire project ground to a complete halt while they refactored the sysem to address the linker issue I'd highlighted on my office door. I've also read many books on operating systems and all the issues that must be handled 100% correctly to make multiple processes running in the same address space work reliably. Can you state the "Dining Philospher's" problem? I can. This stuff is hard. That's why protected memory is so important to a robust system. Threads, like shared memory are valuable tools, but are best used with discretion. And only when there is no suitable alternative. Rather well known, at least to my generation, is something called "Amdahl's law". It's not particularly insightful. It just makes the observation that pefectly optimizing something which does not take a lot of time has neglegible benefit. That a seldom used admin utility would be rewritten as a threaded application says that those responsible for this idiocy were solely interested in adding "threaded programming" to their resumes. I neither know nor care if the programmer was at fault or the manager who permitted it was at fault. In any case it is so stupid as to begger belief. Sadly I have seen many people do something similar. I have *never* written a program using pthreads or any other threading library. I dealt with things like seismic imaging problems which ran on 10,000 -20,000 cores or more for 8-10 days processing 10-30 TB of input and generating similar output volumes. Fortunately these were trivially parallel and could be broken up into separate jobs for each core. With that many machines running, checkpointing and restarting was a critical issue. You *were* going to lose a few machines during a run. So robust recovery was a big deal. That much compute costs a lot of money and starting the whole thing over simply not viable. If you did, you went out of business. The only upside was because such users were running 50,000 cores 24x7 in a single location, they got baseload electrical rates. Though it was not uncommon to have to find additonal locations because the electrical power feed to that location had been exceeded. In Houston, the AC costs as much or more than the computers to run. I've seen an inappropriate tool used far too many times for the sole reason that some one wanted to put something on their resume. For many years I'd never written an IP based client server application. I'd taught other people how to do it, but I had not done it myself for the simple reason that it was not an appropriate tool for my assigned tasks. I did eventually get to write a UDP usage logger that tracked usage statistics for the package we were fielding. That was fun. I generated graphics that showed where each business affiliate was in the exploration cycle from what programs they were running. Much too my surprise and some dismay, Mac OS X is the best supported commercial Unix today. So I may well become a Mac user. A not very happy, Reg jack@openindiana:/jack$ sudo /bin/su root@openindiana:/jack# format -e Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c4d0 /pci@0,0/pci-ide@1f,2/ide@0/cmdk@0,0 Specify disk (enter its number): 0 Error: can't open disk '/dev/rdsk/c4d0p0'. Segmentation Fault (core dumped) root@openindiana:/jack# pstack core core 'core' of 3749: format -e - thread# 1 / lwp# 1 - 080658cb init_globals (8095bd8, 0, 8047d18, 806837c, 0, fefc3bb9) + 51f 08068385 c_disk (fef70548, fef70548, 3, 2, 4, 806d608) + 3c8 08065bc1 main (8047d6c, fef685c8, 8047da8, 8057e47, 2, 8047dd4) + 2b4 08057e47 _start_crt (2, 8047dd4, fefd0c6f, 0, 0, 0) + 96 08057d1a _start (2, 8047eb4, 8047ebb, 0, 8047ebe, 8047ed2) + 1a - thread# 2 / lwp# 2 - feeed05e __door_return (0, 0, 0, 0, feb90240, fef5b000) + 2e feed287c door_create_func (0) + 4a feee7551 _thrp_setup (feb90240) + 81 feee7800 _lwp_start (feb90240, 0, 0, 0, 0, 0) - thread# 3 / lwp# 3 - feee7859 __lwp_park (8097828, 8097838, 0, 0, 0, 0) + 19 feee1607 cond_wait_queue (8097828, 8097838, 0) + 5f feee1d82 __cond_wait (8097828, 8097838) + b1 feee1e1c cond_wait (8097828, 8097838) + 2e fe8a4986 subscriber_event_handler (8095ce0) + 9d feee7551 _thrp_setup (feb90a40) + 81 feee7800 _lwp_start (feb90a40, 0, 0, 0, 0, 0) root@openindiana:/jack# ___ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] [OpenIndiana-discuss] Hipster 2020.10 text installer ISO Wow!!!!
On Mon, 1 Mar 2021 at 16:06, cretin1997 via openindiana-discuss wrote: > Could you try other Illumos to see if it's OI specific? I suggest you to try > OmniOS. If it's a generic problem for Illumos then the OI people can't do > anything, so don't blame them. It would be best to avoid the notion of "blame" altogether. If whatever problem you happen to be experiencing is generic to all distributions based on illumos, either folks from OpenIndiana or anywhere else are welcome to report bugs and to contribute fixes. At the end of the day it's all just software, the software won't fix itself, and we're all in this together. Cheers. -- Joshua M. Clulow http://blog.sysmgr.org ___ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev
