Hi everyone, It looks like sometime during observations on Thursday the beam got stuck again. I killed the data recorder process, which you will soon receive an email about. I'll take a look at the new data we collect tonight to see if everything is back to normal. Clearly this is a recurring problem that we need to look into.
Laura On Tue, Apr 20, 2010 at 7:30 PM, Andrew Siemion <[email protected]> wrote: > Hi Dan, > > Thanks, a bottle of Ron del Barillitos would be great! > > - Andrew > > On 4/20/10 4:23 PM, "Dan Werthimer" <[email protected]> wrote: > >> >> >> andrew and laura, >> >> i'm at arecibo for another 13 hours, so let >> me know if you want me to do anthing. >> >> dan >> >> >> On 4/20/2010 4:12 PM, Andrew Siemion wrote: >>> Hi all, >>> >>> Laura noticed that sV.v had stopped switching beams. We have made some >>> changes to the status check script to: >>> >>> 1. check to make sure the ibob is responding to pings >>> 2. include the ibob in the power off - power on sequence >>> 3. power off - power on both the ibob and bee2 upon any error condition >>> >>> - Andrew >>> >>> >>> On 4/20/10 4:08 PM, "[email protected]"<[email protected]> wrote: >>> >>> >>>> SERENDIP V.5 CRITICAL ERROR REPORT >>>> >>>> The BEE2 is responding to pings: >>>> PING beecourageous (192.168.1.86) 56(84) bytes of data. >>>> 64 bytes from beecourageous (192.168.1.86): icmp_seq=1 ttl=64 time=0.369 ms >>>> >>>> --- beecourageous ping statistics --- >>>> 1 packets transmitted, 1 received, 0% packet loss, time 0ms >>>> rtt min/avg/max/mdev = 0.369/0.369/0.369/0.000 ms >>>> The iBOB is responding to pings: >>>> PING ddc (192.168.2.6) 56(84) bytes of data. >>>> 64 bytes from ddc (192.168.2.6): icmp_seq=1 ttl=64 time=5.56 ms >>>> >>>> --- ddc ping statistics --- >>>> 1 packets transmitted, 1 received, 0% packet loss, time 0ms >>>> rtt min/avg/max/mdev = 5.565/5.565/5.565/0.000 ms >>>> >>>> >>>> *****The data collection process is **NOT RUNNING** as of Tuesday 20th >>>> April >>>> 2010 07:00:01 PM >>>> Attempting to restart the BEE... >>>> >>>> Power off command output: >>>> >>>> Power on command output: >>>> Attempting to restart the IBOB... >>>> >>>> Power off command output: >>>> >>>> Power on command output: >>>> >>>> Current Status: 1 ON >>>> 2 OFF >>>> 3 ON >>>> 4 ON >>>> 5 ON >>>> 6 OFF >>>> 7 ON >>>> 8 OFF >>>> >>>> Sleeping 60 seconds to wait for fpga devices to come back up... >>>> >>>> !!Success!! The iBOB has awoken: >>>> PING ddc (192.168.2.6) 56(84) bytes of data. >>>> 64 bytes from ddc (192.168.2.6): icmp_seq=1 ttl=64 time=6.43 ms >>>> >>>> --- ddc ping statistics --- >>>> 1 packets transmitted, 1 received, 0% packet loss, time 0ms >>>> rtt min/avg/max/mdev = 6.434/6.434/6.434/0.000 ms >>>> !!Success!! The BEE2 lives! >>>> Sleeping 5 minutes to let the BEE2 boot... >>>> Attempting to restart the whole shebang... >>>> Check output below: >>>> root >>>> copying dr2_config_short into dr2_config >>>> copying old output files for safekeeping >>>> killing any previous setispec_dr runs >>>> killing old disk buf collector... >>>> killing previous ssh instances to run sendstatus on o...@beecourageous >>>> initalizing the iBOB >>>> >>>> 7 beam, 2 pol >>>> Number of beams: 7 >>>> Number of pols: 2 >>>> Number of cycles: 1 >>>> Dwelltime: 1 >>>> Max address: 14 >>>> >>>> >>>> 0x0000 / 00000 -> 0x000000ED / 0b00000000000000000000000011101101 / >>>> 0000000237 >>>> >>>> >>>> >>>> 0x0001 / 00001 -> 0x000000EB / 0b00000000000000000000000011101011 / >>>> 0000000235 >>>> >>>> >>>> >>>> 0x0002 / 00002 -> 0x000000E7 / 0b00000000000000000000000011100111 / >>>> 0000000231 >>>> >>>> >>>> >>>> 0x0003 / 00003 -> 0x000000DE / 0b00000000000000000000000011011110 / >>>> 0000000222 >>>> >>>> >>>> >>>> 0x0004 / 00004 -> 0x000000DD / 0b00000000000000000000000011011101 / >>>> 0000000221 >>>> >>>> >>>> >>>> 0x0005 / 00005 -> 0x000000DB / 0b00000000000000000000000011011011 / >>>> 0000000219 >>>> >>>> >>>> >>>> 0x0006 / 00006 -> 0x000000D7 / 0b00000000000000000000000011010111 / >>>> 0000000215 >>>> >>>> >>>> >>>> 0x0007 / 00007 -> 0x000000BE / 0b00000000000000000000000010111110 / >>>> 0000000190 >>>> >>>> >>>> >>>> 0x0008 / 00008 -> 0x000000BD / 0b00000000000000000000000010111101 / >>>> 0000000189 >>>> >>>> >>>> >>>> 0x0009 / 00009 -> 0x000000BB / 0b00000000000000000000000010111011 / >>>> 0000000187 >>>> >>>> >>>> >>>> 0x000A / 00010 -> 0x000000B7 / 0b00000000000000000000000010110111 / >>>> 0000000183 >>>> >>>> >>>> >>>> 0x000B / 00011 -> 0x0000007E / 0b00000000000000000000000001111110 / >>>> 0000000126 >>>> >>>> >>>> >>>> 0x000C / 00012 -> 0x0000007D / 0b00000000000000000000000001111101 / >>>> 0000000125 >>>> >>>> >>>> >>>> 0x000D / 00013 -> 0x0000007B / 0b00000000000000000000000001111011 / >>>> 0000000123 >>>> >>>> >>>> >>>> 0x000E / 00014 -> 0x00000000 / 0b00000000000000000000000000000000 / >>>> 0000000000 >>>> >>>> >>>> rebooting the bee2 >>>> tcgetattr: Inappropriate ioctl for device >>>> killing previous sendstatus >>>> deleting previous nohup file >>>> killing running bofs >>>> loading new bofs >>>> nohup: appending output to `nohup.out' >>>> nohup: appending output to `nohup.out' >>>> nohup: appending output to `nohup.out' >>>> nohup: appending output to `nohup.out' >>>> configuring chips >>>> Setting the maximum number of hits to: 25 >>>> Setting the scale threshold to 80 >>>> done >>>> Connection to beecourageous closed. >>>> starting sendstatus >>>> starting diskbuf cleaner >>>> starting new run >>>> imdonenow >>>> >>>> >>>> Disk Usage: >>>> Filesystem Size Used Avail Use% Mounted on >>>> /dev/md1 226G 71G 155G 32% / >>>> tmpfs 2.0G 0 2.0G 0% /lib/init/rw >>>> udev 10M 108K 9.9M 2% /dev >>>> tmpfs 2.0G 0 2.0G 0% /dev/shm >>>> /dev/md0 236M 35M 190M 16% /boot >>>> /dev/sdc1 1.4T 147G 1.3T 11% /mockdata >>>> >>>> >>>> >>>> >>> >>> >>> >> >> > > > >
