Re: [ccp4bb] RAID array

[Kay Diederichs]

Hi Peter,

indeed I was not trying to say that RAID1 is an insurance against disks going 
bad. It is only the first defense against sudden and unpredictable failure (and 
has saved us a couple of times). On the contrary, we regularly inspect 
/var/log/messages since (on RHELx) this has mdadm-related messages and also 
shows non-RAID disk and other errors. We also regularly run SMART tests, run 
mdadm RAID checks, look at webserver error logs, read security logs ... I 
didn't want to expand on this since it doesn't solve Vaheh's problem ...


Best,
Kay
Am 27. November 2019 17:53:37 MEZ schrieb Peter Keller 
:

Dear all,

On 27/11/2019 14:03, Kay Diederichs wrote:
> As an example, by default in my lab we have the operating system on mdadm 
> RAID1 which consists of two disks that mirror each other. If one of the disks 
> fails, typically we only notice this when inspecting the system log files.

This won't help Vaheh, but I highly recommend configuring notifications of 
changes in the state of the RAID in this kind of setup:

(1) configure /etc/mdadm.conf with (as a minumum) the MAILADDR keyword (see 
'man mdadm.conf' for a complete list of keywords). Alternatively, use the 
PROGRAM keyword and write a script that uses something like 'wall' to notify 
users.

(2) test that notifications get through to their intended destination with 
'mdadm --monitor --test'. If the local MTA (postfix, sendmail,...) isn't set up 
correctly, you may need to do that too.

(3) make sure that 'mdadm --monitor --scan' is running. Depending on 
distro, this will be done with the usual service enable and startup commands, 
something like

   systemctl enable --now mdmonitor

or

   chkconfig --add mdmonitor

   service mdmonitor start

And yes, you've guessed it, we got bitten once by a software RAID with 
multiple disk failures, and we only noticed it when an application complained 
that it couldn't write to a file ;-)

Regards,

Peter.

-- 
Peter Keller Tel.: +44 (0)1223 353033
Global Phasing Ltd., Fax.: +44 (0)1223 366889
Sheraton House,
Castle Park,
Cambridge CB3 0AX
United Kingdom





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1


smime.p7s
Description: S/MIME Cryptographic Signature

Re: [ccp4bb] RAID array

[George N. Reeke]

On Wed, 2019-11-27 at 14:03 +, Kay Diederichs wrote:
> Hi Vaheh,
> 
> RAID on Linux comes in different flavours and levels; the flavours are 
> software RAID (mdadm) and hardware RAID (dedicated RAID controller or 
> motherboard), and the levels are RAID0 RAID1 RAID5 RAID6 RAID10 and a few 
> others. These details influence what the user will notice when a disk goes 
> bad. Without knowing what you have, it is difficult to help.
> 
> As an example, by default in my lab we have the operating system on mdadm 
> RAID1 which consists of two disks that mirror each other. If one of the disks 
> fails, typically we only notice this when inspecting the system log files. 
> Replacing the disk, and re-silvering the RAID1 is not trivial and requires 
> some reading of material on the web. 
> 
> It sounds like you don't have this type of RAID1, or maybe there is some 
> mis-configuration.
> 
> good luck,
> Kay
> 
Just to add a bit of advice based on long experience:  Unless you really
do it every day, looking at system logs can be an unreliable way to see
when you have a bad disk.  Assuming you have root access on your Linux
system, there is a tool called logwatch that can be set to look at your
logs for you and send you an email every day with selected excerpts.
So I add a script (I call it runmdadm) to root's /etc/cron.daily to
run mdadm every night and get the RAID info into the log.  I use egrep
in the script to pick out the important output.  Part of my script looks
like this (details changed):

#!/bin/sh
# Run mdadm to check status of RAID arrays
echo "mdadm /dev/md0" > /var/tmp/mdadm.log
/sbin/mdadm --detail /dev/md0 | egrep 'active|removed' >> \
/var/tmp/mdadm.log

Then you can follow the instructions for logwatch to add this new
/var/tmp/mdadm.log to the list of logs it looks at.  (Or change the
above script to add the mdadm info to a log logwatch already looks at.)
If you have hardware RAID, the idea is the same, but you need to use
a command to the hardware controller instead of mdadm.

George Reeke (an old lurker)



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] RAID array

[Peter Keller]

Dear all,

On 27/11/2019 14:03, Kay Diederichs wrote:

As an example, by default in my lab we have the operating system on mdadm RAID1 
which consists of two disks that mirror each other. If one of the disks fails, 
typically we only notice this when inspecting the system log files.


This won't help Vaheh, but I highly recommend configuring notifications 
of changes in the state of the RAID in this kind of setup:


(1) configure /etc/mdadm.conf with (as a minumum) the MAILADDR keyword 
(see 'man mdadm.conf' for a complete list of keywords). Alternatively, 
use the PROGRAM keyword and write a script that uses something like 
'wall' to notify users.


(2) test that notifications get through to their intended destination 
with 'mdadm --monitor --test'. If the local MTA (postfix, sendmail,...) 
isn't set up correctly, you may need to do that too.


(3) make sure that 'mdadm --monitor --scan' is running. Depending on 
distro, this will be done with the usual service enable and startup 
commands, something like


   systemctl enable --now mdmonitor

or

   chkconfig --add mdmonitor

   service mdmonitor start

And yes, you've guessed it, we got bitten once by a software RAID with 
multiple disk failures, and we only noticed it when an application 
complained that it couldn't write to a file ;-)


Regards,

Peter.

--
Peter Keller Tel.: +44 (0)1223 353033
Global Phasing Ltd., Fax.: +44 (0)1223 366889
Sheraton House,
Castle Park,
Cambridge CB3 0AX
United Kingdom




To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] RAID array

[David J. Schuller]

I agree about the complexity of the RAID situation. But it can be narrowed down 
a bit.

Since the claim is made there were only two hard drives, the only possibilities 
are:
RAID 0 - "striping". In which case his data will probably not be recoverable, 
and he would be unable to boot.
RAID 1 - "mirroring". In which case the data may be recoverable. This is 
probably the case, since from the description the machine is booting but the 
login process is unsuccessful.

If the RAID is set up with hardware, there may be entry into the controller 
from the BIOS boot process.

If the RAID is set up in Linux software, then you need to get logged in somehow 
with root privileges. Details may vary a bit depending on which Linux 
distribution is involved (which was not mentioned).

You could try to boot into "rescue mode" from the Linux installation media (DVD 
or USB stick, usually). Or you could try to boot into "single user mode" by 
altering the boot command during the boot procedure (details will vary with 
distribution). Or you could try setting to boot up in nongraphical mode.

Like I say, this varies with Linux distribution so I won't try to give a line 
by line set of instructions. My experience is all with Fedor and Scientific 
Linux.

Once you are in, you need to assess the situation. If RAID is set up with 
software, there will be a configuration file, typically /etc/mdadm.conf, and a 
status file, typically /proc/mdstat. You can look for these files for clues to 
how the RAID is set up, and what its status is.
You would need to figure out how your existing drive is configured by checking 
out the partition arrangement with /usr/sbin/fdisk or /usr/sbin/parted.
Then you would need to set up partitions on the brand new drive in the same way.
Once the partitions exist, you would try to add the partitions into the RAID 
volume with /usr/sbin/mdadm. You are going to need to read the man pages for 
mdadm and probably search for help online. Once the RAID set is successfully 
reconfigured, it would take probably a few hours to copy the data from the good 
device to the new one.

BTW, if anyone is using HPE SSDs they need to be aware of this:
https://www.zdnet.com/article/hpe-tells-users-to-patch-ssds-to-prevent-failure-after-32768-hours-of-operation/
HPE tells users to patch SSDs to prevent failure after 32,768 hours of operation



On 2019-11-27 09:03, Kay Diederichs wrote:

Hi Vaheh,

RAID on Linux comes in different flavours and levels; the flavours are software 
RAID (mdadm) and hardware RAID (dedicated RAID controller or motherboard), and 
the levels are RAID0 RAID1 RAID5 RAID6 RAID10 and a few others. These details 
influence what the user will notice when a disk goes bad. Without knowing what 
you have, it is difficult to help.

As an example, by default in my lab we have the operating system on mdadm RAID1 
which consists of two disks that mirror each other. If one of the disks fails, 
typically we only notice this when inspecting the system log files. Replacing 
the disk, and re-silvering the RAID1 is not trivial and requires some reading 
of material on the web.

It sounds like you don't have this type of RAID1, or maybe there is some 
mis-configuration.

good luck,
Kay



On Tue, 26 Nov 2019 21:05:54 +, Oganesyan, Vaheh 
 wrote:



Hello ccp4-ers,

A bit off topic (actually a lot off topic) question regarding RAID array 
system. On linux box one of two hard drives failed. I've found identical one 
and replaced it. Can someone point me in the direction where I can get 
instructions on what to do next to be able to login? Currently computer starts 
with no error messages, comes to the login window, takes username and password 
from me and then flashes and comes back to login window.

Thank you for your help.
Regards,

Vaheh Oganesyan, Ph.D.
Scientist, Biologic Therapeutics

AstraZeneca
R | Antibody Discovery and Protein Engineering
One Medimmune Way, Gaithersburg, MD 20878
T:  301-398-5851
vaheh.oganes...@astrazeneca.com





To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1






To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1



--
===
All Things Serve the Beam
===
   David J. Schuller

Re: [ccp4bb] Rigaku Minstrel and Windows10

[Ahel,Juraj]

Dear Tommi,

we tried to update to Windows 7 a while back - it didn't work because of the 
drivers for the camera that were compatible with Windows 7 were not able to 
communicate with the software anymore. So we assumed it not to be possible 
without patching the Minstrel Imager software, which obviously isn't happening 
now.

If you do manage to make it work, do share your story.

All the best

Juraj

From: CCP4 bulletin board  on behalf of Kajander, Tommi 
A 
Sent: Wednesday, November 27, 2019 11:41 AM
To: CCP4BB@JISCMAIL.AC.UK 
Subject: [ccp4bb] Rigaku Minstrel and Windows10

Dear All,

Bit off-topic if allowed: I would like to ask has anyone had problems or 
successes updating the Rigaku Minstrel computers to Windows 10?

Would like to hear about experiences if anyone can provide insight.

Thank you very much,
Tommi



Tommi Kajander, PhD
Principal investigator
Structural Biology Program
Institute of Biotechnology
Biocenter 3, Viikinkaari 1 (PO Box 65)
University of Helsinki
tommi.kajan...@helsinki.fi
+358 294158904







To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] RAID array

[Kay Diederichs]

Hi Vaheh,

RAID on Linux comes in different flavours and levels; the flavours are software 
RAID (mdadm) and hardware RAID (dedicated RAID controller or motherboard), and 
the levels are RAID0 RAID1 RAID5 RAID6 RAID10 and a few others. These details 
influence what the user will notice when a disk goes bad. Without knowing what 
you have, it is difficult to help.

As an example, by default in my lab we have the operating system on mdadm RAID1 
which consists of two disks that mirror each other. If one of the disks fails, 
typically we only notice this when inspecting the system log files. Replacing 
the disk, and re-silvering the RAID1 is not trivial and requires some reading 
of material on the web. 

It sounds like you don't have this type of RAID1, or maybe there is some 
mis-configuration.

good luck,
Kay



On Tue, 26 Nov 2019 21:05:54 +, Oganesyan, Vaheh 
 wrote:

>Hello ccp4-ers,
>
>A bit off topic (actually a lot off topic) question regarding RAID array 
>system. On linux box one of two hard drives failed. I've found identical one 
>and replaced it. Can someone point me in the direction where I can get 
>instructions on what to do next to be able to login? Currently computer starts 
>with no error messages, comes to the login window, takes username and password 
>from me and then flashes and comes back to login window.
>
>Thank you for your help.
>Regards,
>
>Vaheh Oganesyan, Ph.D.
>Scientist, Biologic Therapeutics
>
>AstraZeneca
>R | Antibody Discovery and Protein Engineering
>One Medimmune Way, Gaithersburg, MD 20878
>T:  301-398-5851
>vaheh.oganes...@astrazeneca.com
>
>
>
>
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>
>
>
>To unsubscribe from the CCP4BB list, click the following link:
>https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] microscope camera

[Artem Evdokimov]

I bought a bunch of cheap cameras from Amscope. They were OK for the price.

Artem

On Wed, Nov 27, 2019, 12:46 Dean Derbyshire 
wrote:

> Forgive the off topic subject:
>
>
>
> Has anyone got any experience with moticam microscope cameras?
>
> We are looking into cheap cameras for record keeping etc and this supplier
> looks good but…
>
>
>
> Cheers in advance
>
>
>
> Sent from Mail  for
> Windows 10
>
>
>
> --
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
>



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

[ccp4bb] Rigaku Minstrel and Windows10

[Kajander, Tommi A]

Dear All,

Bit off-topic if allowed: I would like to ask has anyone had problems or 
successes updating the Rigaku Minstrel computers to Windows 10?

Would like to hear about experiences if anyone can provide insight.

Thank you very much,
Tommi



Tommi Kajander, PhD
Principal investigator
Structural Biology Program
Institute of Biotechnology
Biocenter 3, Viikinkaari 1 (PO Box 65)
University of Helsinki
tommi.kajan...@helsinki.fi
+358 294158904







To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1