Re: [PLUG] How to restore external drive
A few comments... > However, I have long been impatient with the speed of the Mediasonic; > moving or renaming a file can take over a minute while the drives > whir and the lights dance. Renaming or moving files on the same drive should take fraction of a second regardless of the file size or the speed of connection to it. If this what you really experience, you should consider using mv on a command line. > During the restore from backup the GUI (Nautilus) gave me a popup > with continuous information on the progress, and it reported a steady > 115-116 Mbps transfer rate, so that is apparently the max that my > home ethernet can do. That 100-120MBs is about maximum you can observe over 1Gbs ethernet to a NAS for sequential, large file transfers. Recent Synology NAS`s are capable of that without problem even for RAID5 arrays. Please note - sequential, large file transfers - is the keyword. Once we talk about random or small file access - you are into 40MBs-ish territory even for fast SSDs - no what the speed of connection. See some benchmarks such as: SSDs: https://www.anandtech.com/show/12485/the-plextor-m8v-sata-ssd-rev iew/5 NVMEs: https://www.anandtech.com/show/13254/the-toshiba-xg6-1tb-ssd-rev iew-first-96l-3d-nand/5 > Thinkpad P72 (not yet available in the US) or a P71 that I can buy > right now. Both come with Intel Thunderbolt 3 USB-C ports that are > capable of 40Gbps, compared to the 5Gbps of my USB 3.0 connection or > 3GBps of eSATA. So, connecting to HDD or SSD from single user, single computer at 5Gbs, 10Gbs or even 40Gbs is rather irrelevant. Especially when you consider a laptop which is very limited in what it is capable of compared to a desktop - especially in high bandwidth usage applications. Anyway - large files are mainly media - Encoding a film with x264 proceeds at glacial 5Mbs-ish speed these days. So even if you could encode 5 streams at the same time, you would probably be fine with 1Gbs ethernet to a NAS. Backups are incremental, and do not slow you down, especially if your NAS writes to your DAS directly or between two NAS`s. I assume that you are not watching the backups to make them go faster. :-) --- If we would be talking (= not dreaming) about low cost 4-6 drive Synology NAS with 2.5/5/10Gbs ethernet + $150-ish 8 port 2.5/5/10Gbs switch to go with it - that would be different cup-a-tea. --- Storage is slow, and who likes politics or taxes. Hope it helps, Tomas On Mon, 2018-10-15 at 08:39 -0700, John Jason Jordan wrote: > On Mon, 15 Oct 2018 01:27:31 -0700 > Tomas K dijo: > > > It seems that it would be safer to have the DAS array attached > > permanently to your NAS and access it over the network. That is if > > your > > NAS has eSATA port. > > > > That way it would be permanently attached like internal drives and > > that > > should avoid these type of synchronization errors - Especially when > > you > > use it as RAID0. > > > > RAID arrays, especially in striping configuration, do not like to > > be > > detached or put to sleep without proper sync and umount. > > The Synology NAS does not have an eSATA port, but it does have a USB > port, so I could probably attach the Mediasonic to it. And I agree > that > doing so might make the connection more stable - the NAS is across > the > room, while the Mediasonic is on my desk where stuff gets moved > around > a bit. However, I have long been impatient with the speed of the > Mediasonic; moving or renaming a file can take over a minute while > the > drives whir and the lights dance. I lust for SSDs to replace the two > WD > Red Pro drives in it, but the cost of 16TB of SSDs always makes me > discard the notion. If I attach the Mediasonic to the Synology I > wonder > about access time compared to its current USB 3 connection. > > During the restore from backup the GUI (Nautilus) gave me a popup > with > continuous information on the progress, and it reported a steady 115- > 116 > Mbps transfer rate, so that is apparently the max that my home > ethernet > can do. > > I have been shopping for a new computer and I'm currently looking at > a > Thinkpad P72 (not yet available in the US) or a P71 that I can buy > right now. Both come with Intel Thunderbolt 3 USB-C ports that are > capable of 40Gbps, compared to the 5Gbps of my USB 3.0 connection or > 3GBps of eSATA. Then again, the WD Red Pro drives are SATA-3, which > are > rated at 6Gbps, so until I go to SSDs there wouldn't be much to gain. > And the only faster enclosures available are Mediasonic rated at > 10Gbps. What would be really cool is 16TB of SSDs mounted inside the > laptop itself. > > > If attaching it permanently to NAS is not an option, autofs with > > reasonable timeout avoiding the DAS power saving mode, instead of > > mount > > in fstab, would probably help too. > > The Synology is mounted by a line in fstab, but not the Mediasonic. I > have thought of doing so, but haven't bothered; it's trivial to mount > it with the GUI. But if doing so
Re: [PLUG] How to restore external drive
On Mon, 15 Oct 2018 01:27:31 -0700 Tomas K dijo: >It seems that it would be safer to have the DAS array attached >permanently to your NAS and access it over the network. That is if your >NAS has eSATA port. > >That way it would be permanently attached like internal drives and that >should avoid these type of synchronization errors - Especially when you >use it as RAID0. > >RAID arrays, especially in striping configuration, do not like to be >detached or put to sleep without proper sync and umount. The Synology NAS does not have an eSATA port, but it does have a USB port, so I could probably attach the Mediasonic to it. And I agree that doing so might make the connection more stable - the NAS is across the room, while the Mediasonic is on my desk where stuff gets moved around a bit. However, I have long been impatient with the speed of the Mediasonic; moving or renaming a file can take over a minute while the drives whir and the lights dance. I lust for SSDs to replace the two WD Red Pro drives in it, but the cost of 16TB of SSDs always makes me discard the notion. If I attach the Mediasonic to the Synology I wonder about access time compared to its current USB 3 connection. During the restore from backup the GUI (Nautilus) gave me a popup with continuous information on the progress, and it reported a steady 115-116 Mbps transfer rate, so that is apparently the max that my home ethernet can do. I have been shopping for a new computer and I'm currently looking at a Thinkpad P72 (not yet available in the US) or a P71 that I can buy right now. Both come with Intel Thunderbolt 3 USB-C ports that are capable of 40Gbps, compared to the 5Gbps of my USB 3.0 connection or 3GBps of eSATA. Then again, the WD Red Pro drives are SATA-3, which are rated at 6Gbps, so until I go to SSDs there wouldn't be much to gain. And the only faster enclosures available are Mediasonic rated at 10Gbps. What would be really cool is 16TB of SSDs mounted inside the laptop itself. >If attaching it permanently to NAS is not an option, autofs with >reasonable timeout avoiding the DAS power saving mode, instead of mount >in fstab, would probably help too. The Synology is mounted by a line in fstab, but not the Mediasonic. I have thought of doing so, but haven't bothered; it's trivial to mount it with the GUI. But if doing so can increase access speed I'm all for it. ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] How to restore external drive
It seems that it would be safer to have the DAS array attached permanently to your NAS and access it over the network. That is if your NAS has eSATA port. That way it would be permanently attached like internal drives and that should avoid these type of synchronization errors - Especially when you use it as RAID0. RAID arrays, especially in striping configuration, do not like to be detached or put to sleep without proper sync and umount. If attaching it permanently to NAS is not an option, autofs with reasonable timeout avoiding the DAS power saving mode, instead of mount in fstab, would probably help too. Hope it helps, Tomas On Sun, 2018-10-14 at 18:26 -0700, John Jason Jordan wrote: > On Sun, 14 Oct 2018 13:45:14 -0700 (PDT) > Rich Shepard dijo: > > > On Sun, 14 Oct 2018, Ben Koenig wrote: > > > > > Please copy/paste the following command into your terminal and > > > post > > > the output here so that we can offer sensible advice. > > > > > > ls -l /dev/sd* > > > > Ben, > > > > Might lsblk provide John with the same information? > > It does, and I already used it last night. It found /dev/sdc, but not > the partition /dev/sdc1. Gparted had a utility to search a disk for a > filesystem, which I left running overnight. It found nothing, so this > morning I bit the bullet, recreated and formatted the partition, and > then started copying everything from the backup Synology NAS. I > started > the copying at about 9am, and it is now working on letter 'O.' I hope > to be done before bedtime tonight. > ___ > PLUG mailing list > PLUG@pdxlinux.org > http://lists.pdxlinux.org/mailman/listinfo/plug ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] How to restore external drive
On Sun, 14 Oct 2018 13:45:14 -0700 (PDT) Rich Shepard dijo: >On Sun, 14 Oct 2018, Ben Koenig wrote: > >> Please copy/paste the following command into your terminal and post >> the output here so that we can offer sensible advice. >> >> ls -l /dev/sd* > >Ben, > > Might lsblk provide John with the same information? It does, and I already used it last night. It found /dev/sdc, but not the partition /dev/sdc1. Gparted had a utility to search a disk for a filesystem, which I left running overnight. It found nothing, so this morning I bit the bullet, recreated and formatted the partition, and then started copying everything from the backup Synology NAS. I started the copying at about 9am, and it is now working on letter 'O.' I hope to be done before bedtime tonight. ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] How to restore external drive
On Sun, 14 Oct 2018, Ben Koenig wrote: Please copy/paste the following command into your terminal and post the output here so that we can offer sensible advice. ls -l /dev/sd* Ben, Might lsblk provide John with the same information? Rich ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] How to restore external drive
John, Please copy/paste the following command into your terminal and post the output here so that we can offer sensible advice. ls -l /dev/sd* Making sure you know what is attached and mounted to your system should be your first step in a situation like this. --Ben On Sun, Oct 14, 2018 at 9:04 AM Rich Shepard wrote: > On Sun, 14 Oct 2018, John Jason Jordan wrote: > > > It is not referenced in fstab. Here is fstab: > > John, > >That's totally different from any I've seen so I have nothing useful to > add. > > > Altering your command because Ubuntu decided some time back to rename > > /var/log/messages to /var/log/syslog, I get nothing referring to the > > device. > >Did you follow /var/log/syslog for changes as you connected (or turned > on) > your problem device? If you look at the static log you might not see what > how the kernel reacts when a new device is seen. > >On my systems when I attached a USB external flash/hard drive I often > see > the log respond (not in these exact words), > >/dev/sdc seen >/dev/sdc1 attached as Logitech model xxx. > > This tells me that the kernel first sees the device, then the specific > partition. > >Others with much more knowledge than I have will probably help you more. > > Regards, > > Rich > ___ > PLUG mailing list > PLUG@pdxlinux.org > http://lists.pdxlinux.org/mailman/listinfo/plug > ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] How to restore external drive
On Sun, 14 Oct 2018, John Jason Jordan wrote: It is not referenced in fstab. Here is fstab: John, That's totally different from any I've seen so I have nothing useful to add. Altering your command because Ubuntu decided some time back to rename /var/log/messages to /var/log/syslog, I get nothing referring to the device. Did you follow /var/log/syslog for changes as you connected (or turned on) your problem device? If you look at the static log you might not see what how the kernel reacts when a new device is seen. On my systems when I attached a USB external flash/hard drive I often see the log respond (not in these exact words), /dev/sdc seen /dev/sdc1 attached as Logitech model xxx. This tells me that the kernel first sees the device, then the specific partition. Others with much more knowledge than I have will probably help you more. Regards, Rich ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] How to restore external drive
On Sun, 14 Oct 2018 05:47:15 -0700 (PDT) Rich Shepard dijo: >On Sat, 13 Oct 2018, John Jason Jordan wrote: > >> sudo mount /dev/sdc /media/jjj/Movies >> mount: wrong fs type, bad option, bad superblock on /dev/sdc, >> missing codepage or helper program, or other error > 1) What is the reference to that device in /etc/fstab? It is not referenced in fstab. Here is fstab: # /etc/fstab: static file system information. # Use 'blkid' to print the universally unique identifier for a # device; # # / was on /dev/sdb1 during installation UUID=27c11f6b-b443-417e-9853-12c99789d8d9 / ext4 errors=remount-ro 0 1 # /home was on /dev/sdb2 during installation UUID=9a201393-e364-4d11-b372-877cded3b9cc /home ext4 defaults 0 2 192.168.1.115:/volume1/Synology /media/jjj/Synology nfs auto,user 0 0 #so clients can see nfs share /media/jjj /export/users none bind 0 0 /dev/disk/by-label/256GB-1 /media/jjj/256GB-1 auto nosuid,nodev,nofail,noauto,x-gvfs-show,x-gvfs-name=256GB-1 0 0 The last line that starts with '/dev/disk/by-label/256GB-1' was added by something involving the two 256GB USB flash drives that I bought recently. When they arrived they were exFAT or something, so I used the GUI Disks utility that comes with Ubuntu to delete the original partition, then created a new one and format it ext4, and give them labels as '256GB-1' and '256GB-2.' I did not create that line in fstab. I asked about it here the other day and the only comment was that it might involve hotplug. I searched on 'hotplug linux' but didn't come up with any explanation. The last few lines of dmesg are: usb 4-5.2: new SuperSpeed USB device number 10 using xhci_hcd usb 4-5.2: New USB device found, idVendor=152d, idProduct=9561 usb 4-5.2: New USB device strings: Mfr=1, Product=2, SerialNumber=5 usb 4-5.2: Product: JMS56x Series usb 4-5.2: Manufacturer: JMicron usb 4-5.2: SerialNumber: scsi host11: uas scsi 11:0:0:0: Direct-Access JMicron Disk 0104 PQ: 0 ANSI: 6 sd 11:0:0:0: Attached scsi generic sg4 type 0 sd 11:0:0:0: [sdc] 23441833984 512-byte logical blocks: (12.0 TB/10.9 TiB) sd 11:0:0:0: [sdc] Write Protect is off sd 11:0:0:0: [sdc] Mode Sense: 67 00 10 08 sd 11:0:0:0: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA sd 11:0:0:0: [sdc] Attached SCSI disk [18690.215525] perf interrupt took too long (2503 > 2500), lowering kernel.perf_event_max_sample_rate to 5 > 2) When you run 'sudo tail -f /var/log/messages' and connect (or > turn on) that unit how does the kernel see that drive? I could > be /dev/sdc/ or /dev/sdc1/. The drive referenced to the mount point > needs to match what /var/log/messages shows. As you can see from the end of dmesg above, the kernel sees it as /dev/sdc, not as /dev/sdc1. My dim recollection is that it used to be sdc1 before I swapped the boards in the enclosure. Altering your command because Ubuntu decided some time back to rename /var/log/messages to /var/log/syslog, I get nothing referring to the device. Last night before going to bed it occurred to me to try the Gparted GUI from the Ubuntu repos. It sees the partition, but like the Disks GUI utility it doesn't know what to make of it. It displays the partition with dotted lines around it. But it offered something new - a utility to 'search for filesystems on /dev/sdc.' It said that it might take a long time, so I started it and left it running. This morning it is still running and apparently has found nothing. I hate to do it because it's such a pain (about 7.5TB of data), but I think the only thing I can do now is to recreate and reformat the partition, then restore the data from the Synology. ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
Re: [PLUG] How to restore external drive
On Sat, 13 Oct 2018, John Jason Jordan wrote: sudo mount /dev/sdc /media/jjj/Movies mount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error John, Two questions: 1) What is the reference to that device in /etc/fstab? 2) When you run 'sudo tail -f /var/log/messages' and connect (or turn on) that unit how does the kernel see that drive? I could be /dev/sdc/ or /dev/sdc1/. The drive referenced to the mount point needs to match what /var/log/messages shows. Regards, Rich ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
[PLUG] How to restore external drive
I have a Mediasonic two-bay external enclosure with two hard disks in Raid 0, for a total of 12TB. This is backed up regularly to a 16TB Synology NAS, most recently last night, so if I lose everything on the 12TB Mediasonic I will lose just a few things that I did today - easily replaced. The Mediasonic has both USB 3.0 and eSATA ports. Some time ago the USB port failed, so I bought an eSATA cable and switched to that port. Lately I have been losing the connection at random times again, so I contacted Mediasonic and for $20 bought a replacement card that has the two ports on it, which arrived a couple weeks ago. I intended to install it at the Clinic a week from tomorrow, but today I lost the connection permanently, so I got out my tools and installed the new card. When I powered it up it was in single-drive mode, not Raid 0, so I had to change the setting. This required powering it down, and when it started back up it appeared to be fine - both drive lights are on. But I can't mount it. Dmesg says it is /dev/sdc, but all I get are errors at the mount command: sudo mount /dev/sdc /media/jjj/Movies mount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error I tried adding '-t ext4', but no joy. Now, I dimly recall that this device used to be /dev/sdc1, but now the '1' is missing. Using the Disks GUI utility in Ubuntu it sees the device and calls it '12TB Unknown.' It offers to format it or create a partition image for me, but I haven't yet taken the plunge. Before I reformat and go through the travail of restoring from the Synology NAS, does anyone have any suggestions? ___ PLUG mailing list PLUG@pdxlinux.org http://lists.pdxlinux.org/mailman/listinfo/plug
