Re: [Elphel-support] Recording fullHD JP4 being limited to 7 seconds
Flavio, I am wondering about this. Can Elphel deal with RAID? If I format both my CF Cards to RAID 0, maybe I can improve (double) this performance, which could be something... I don't have such experience - probably not. You could try, but I have no idea why it should help - r/w speed will stay the same. I'd go with an external SATA HDD at this point. Best regards, Oleg Dzhimiev Electronics Engineer phone: +1 801 783 x124 Elphel, Inc. On 10 May 2012 14:01, The Apertus Team qazav...@gmail.com wrote: Hello, there, If I understood correctly, my card takes 6.75 seconds to write the 19MB and 19 MB / 6.75 = 2.8MB/s should be the maximum data rate my cards can handle, is that it? Yes, the cards are in PIO mode so seem to be disabled. And as I wrote - it is not only that they are ~16/2.8 ~= 5.5 times slower than those in DMA mode, they use CPU much more during recording, so no resources fro other CPU tasks. Can you try them with hdparm -I and send the reply? The CPU-limited recording rate in DMA mode should be ~16MB/s I am wondering about this. Can Elphel deal with RAID? If I format both my CF Cards to RAID 0, maybe I can improve (double) this performance, which could be something... thanks! flavio ___ Support-list mailing list Support-list@support.elphel.com http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com ___ Support-list mailing list Support-list@support.elphel.com http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
Re: [Elphel-support] Recording fullHD JP4 being limited to 7 seconds
Hello, there, If I understood correctly, my card takes 6.75 seconds to write the 19MB and 19 MB / 6.75 = 2.8MB/s should be the maximum data rate my cards can handle, is that it? Yes, the cards are in PIO mode so seem to be disabled. And as I wrote - it is not only that they are ~16/2.8 ~= 5.5 times slower than those in DMA mode, they use CPU much more during recording, so no resources fro other CPU tasks. Can you try them with hdparm -I and send the reply? The CPU-limited recording rate in DMA mode should be ~16MB/s I am wondering about this. Can Elphel deal with RAID? If I format both my CF Cards to RAID 0, maybe I can improve (double) this performance, which could be something... thanks! flavio ___ Support-list mailing list Support-list@support.elphel.com http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
Re: [Elphel-support] Recording fullHD JP4 being limited to 7 seconds
Thanks, Sebastian, that worked! Now I just need the indication for the CF card and we're go! ___ Support-list mailing list Support-list@support.elphel.com http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
Re: [Elphel-support] Recording fullHD JP4 being limited to 7 seconds
I remember settings FPS can be a bit tricky at first: There are 2 modes which require the FPS to be set differently: In parsedit/autocampars: Free running mode (default): FPSFLAGS=1 FP1000SLIM=desired_FPS x 1000 Triggered mode (requires some flags to be set in advance): FPSFLAGS=0 TRIG_PERIOD=9600 / desired_FPS ElphelVision deals with all the above automatically when you go to Settings-Custom FPS. Regards Sebastian On Sat, Apr 28, 2012 at 19:53, flavio soares qazav...@gmail.com wrote: Hello, everyone, I've rechecked pretty much everything, found an easy way to connect the camera directly to a PC (I'll update the info at the wiki page), installed GStreamer and everything is working fine. I haven't really studied GStreamer's documentation to feel comfortable about changing the parameters, but that will come with time. So now I can see and debayer the live streaming at a lower resolution, as expected. (an observation: the documentation on how to install and run GStreamer is quite good and the examples work. I have an issue on compiling for 32bits, where is the best place to get in touch with them? for 64 bits everything if fine). As to recording, I noticed that when I fix a video to be, say, 1024x768 in camvc, I refresh Elphel's page in Firefox/Iceweasel and that information is updated in the disk recorder section. I notice, however, that when I set camvc to have: Video Streamer: on; Limit the frame rate: on; Frame rate limit: 25 fps; Mantain the frame rate: on. Up there, right below the icons, it shows me this: 1024x768 @ 63.591 (in blue color, as expected) fps --- 47k. I mention this just to be sure I'm passing the right information. Now, this is what the web page shows me when I update it: Filename: - Image Resolution: 1024 x 768 JPEG Quality 80 % Framerate: 63.754 fps Record Time: - File Size: - Data Rate: - Data Rate: - The question is: if I fixed my framerate to be 25 fps, why doesn't it update here? Also, I see at the recorded videos that they are being recorded at this same fps, see below mplayer's (edited) output: livre@laika:~/Desktop$ mplayer http://192.168.1.9:var/hdd/1335633763_711797.mov Playing http://192.168.1.9:var/hdd/1335633763_711797.mov. [lavf] stream 0: video (mjpeg), -vid 0 VIDEO: [jpeg] 1024x768 24bpp 63.694 fps 0.0 kbps ( 0.0 kbyte/s) I guess that by fixing this issue, we'll be quite ready to do a first testing recording. =) Also, I will update my CF card then. It will give us a lot of mobility. So, I'll wait for your advice on which card exactly shoulf we buy. I believe a 8GB will be good enough, but it can be a 16GB. I also found the tests we made here quite important, so I'll also update this info at the wiki. Cheers everyone, and thanks for the help! flavio 2012/4/26, Andrey Filippov support-list@support.elphel.com: Flavio, That you for the nice words. The link a gave you is posted to the public mailing list, and it is hosted on the public server, so there is no secret about it and you may show it (all the content on the that server is released under the GNU FDL 1.3). We just did not want to make much buzz about it yet - these are our first images with the new series of the cameras and there is a lot to be done yet, and there are many processing parameters to tweak - i took them from the previous camera settings (i.e. some de-mosaic artifacts are visible). But we definitely consider it a proof that our concepts worked, the difference with the first Eyesis that had the same sensors and same lenses (not to count the top fisheye) is significant. We will publish more details later - so far we were very busy as we had to make our first customers to wait extra 4 months - we had to completely redesign and re-build the critical part of the cameras - the sensor front end mechanics. We are going to demonstrate the new camera together with the camera calibration machine and the process of calibration at this year SIGGRAPH in Los Angeles (https://www.google.com/search?q=siggraph+elphel) - the booth number is different as we had to increase the size - could not fit in 3x3 meters. Andrey On Thu, Apr 26, 2012 at 2:48 AM, flavio soares qazav...@gmail.com wrote: With Eyesis we use 97-98%, but that is only because we use aberration correction by de-convolution that amplifies the noise and artifacts. BTW - here are our last panoramas made with the new camera (viewing requires good videocard and WebGL-capable browser) http://community.elphel.com/files/eyesis/pano-db-3/webgl_panorama_editor.html?kml=mainstreet_ro.kmlproto=mainstreet_ro.kmlntxt=2as_camera=95start=result_1334548264_780764_1.jpegrange=95labels=falsekeepzoom=falseclosest2d=falseseethrough=0.4transition=5mask=azimuth=23.2elevation=-70.5zoom=0.386follow=falsemv3d=false o.O Man, this is really impressive. Is this link public, I mean, can I show it around?
Re: [Elphel-support] Recording fullHD JP4 being limited to 7 seconds
Hello Flavio, Most likely the problem is related to the CF - as I wrote in that old article, most CF cards in IDE mode lie about support of the DMA mode. They do support UDMA so most people never notice that small lie. But Axis CPU used in the camera has a bug of it's own - and this bug is related to the UDMA. If the card would hostly report that it does not support DMA camera driver would just go to PIO mode (ABSOLUTELY NOT SUITABLE for the video recording - both slow and using to much of the camera CPU resources), but it does not, so camera may hang up and never come out of the boot scripts. So what we did was to add some known bad cards to the driver black list: http://elphel.cvs.sourceforge.net/viewvc/elphel/elphel353-8.0/os/linux-2.6-tag--devboard-R2_10-4/drivers/ide/ide-dma.c?view=markup(lines 130..136). At some point we black-listed all no-name cards (we did not find any among them that did support DMA) You may use hdparm -I (from http://192.168.0.9/phpshell.php, telnet or serial console that you should have cable for) and the only way to re-enable (if you card matches the black-listed one) the card is to modify that file and recompile the camera firmware. The only type of CF cards we use ourselves is Sandisk Extreme III. When the card is blacklisted, camera bots correctly and is able to mount (and use) the card - it can be used to save images or very slow video (still useful for timelapse), but not the normal video. You may test the speed of the CF card (that test will destroy data on it!) with the following command (again - phpshell, telnet, serial console): time dd if=/dev/circbuf of=/dev/hda #(or hdb - depending how is the card inserted/configured) /dev/circbuf is the circular buffer ~19MB in size that is very fast to read for the camera, so the speed will be related to the CF itself You may try it on /dev/null first: [root@Goniometer /dev]16713# time dd if=/dev/circbuf of=/dev/null 38656+0 records in 38656+0 records out real0m 0.43s user0m 0.11s sys 0m 0.32s With phpshell you may just open the following link (provided you have the default IP) - just replace all spaces in the command line with + signs http://192.168.0.9/phpshell.php?command=time+dd+if=/dev/circbuf+of=/dev/null Then divide 19MB by the time measured. Andrey ___ Support-list mailing list Support-list@support.elphel.com http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
Re: [Elphel-support] Recording fullHD JP4 being limited to 7 seconds
Hi, Andrey, Sorry, I was rebooting, that's what kept me. Here is the result for both cards (even though I can't really understand what it means): [root@Elphel353 /root]3882# hdparm -I /dev/hdb /dev/hdb: ATA device, with non-removable media Model Number:CF CARD 4GB Serial Number: 3C2E Firmware Revision: Standards: Likely used: 5 Configuration: Logical max current cylinders 80608060 heads 16 16 sectors/track 63 63 -- CHS current addressable sectors:8124480 LBAuser addressable sectors:8124480 device size with M = 1024*1024:3967 MBytes device size with M = 1000*1000:4159 MBytes (4 GB) Capabilities: LBA, IORDY(may be)(cannot be disabled) bytes avail on r/w long: 4 Queue depth: 1 Standby timer values: spec'd by Vendor R/W multiple sector transfer: Max = 1 Current = 0 DMA: mdma0 mdma1 *mdma2 Cycle time: min=120ns recommended=120ns PIO: pio0 pio1 pio2 pio3 pio4 Cycle time: no flow control=120ns IORDY flow control=120ns HW reset results: CBLID- below Vih Device num = 1 [root@Elphel353 /root]3882# hdparm -I /dev/hda /dev/hda: ATA device, with non-removable media Model Number:CF CARD 4GB Serial Number: 3C48 Firmware Revision: Standards: Likely used: 5 Configuration: Logical max current cylinders 80608060 heads 16 16 sectors/track 63 63 -- CHS current addressable sectors:8124480 LBAuser addressable sectors:8124480 device size with M = 1024*1024:3967 MBytes device size with M = 1000*1000:4159 MBytes (4 GB) Capabilities: LBA, IORDY(may be)(cannot be disabled) bytes avail on r/w long: 4 Queue depth: 1 Standby timer values: spec'd by Vendor R/W multiple sector transfer: Max = 1 Current = 0 DMA: mdma0 mdma1 *mdma2 Cycle time: min=120ns recommended=120ns PIO: pio0 pio1 pio2 pio3 pio4 Cycle time: no flow control=120ns IORDY flow control=120ns HW reset results: CBLID- below Vih Device num = 0 Thanks for the link, I'll check on it soon. I have a really good graphic card here, but my Iceweasel is not up to the task =) I'll reboot and check it with chrome! ps.: Do you think it's too dangerous to test the Sata the way I mentioned? My computer is actually open at the moment with a Sata literally hanging on it (backuping). =) 2012/4/26, Andrey Filippov support-list@support.elphel.com: Hello, Andrey, thanks for the quick response! You are welcome , Flavio I'm also online (here it is 3:40am) =) Not that late he - less than 1AM :-) My Kingston cards are not listed in the blacklist. Let's get them to work and see how far they go. Here are the results of the tests, I'm ssh-ed into the cam: [root@Elphel353 /mnt/0]944# time dd if=/dev/circbuf of=/dev/null 38656+0 records in 38656+0 records out real0m 0.42s user0m 0.03s sys 0m 0.39s [root@Elphel353 /mnt/0]944# time dd if=/dev/circbuf of=/dev/hdb1 38656+0 records in 38656+0 records out real0m 6.75s user0m 0.19s sys 0m 2.46s [root@Elphel353 /mnt/0]944# If I understood correctly, my card takes 6.75 seconds to write the 19MB and 19 MB / 6.75 = 2.8MB/s should be the maximum data rate my cards can handle, is that it? Yes, the cards are in PIO mode so seem to be disabled. And as I wrote - it is not only that they are ~16/2.8 ~= 5.5 times slower than those in DMA mode, they use CPU much more during recording, so no resources fro other CPU tasks. Can you try them with hdparm -I and send the reply? The CPU-limited recording rate in DMA mode should be ~16MB/s Meanwhile, I've tested normal HD JP4, at 720p and observed that compression 100% is a killer, but at about 90%, the recording proceeded without cutting the video, even tough the gui refresh rate suffered. Yes that is so - 90% is really good quality. It may also make sense to fine-tune coring value () in addition to JPEG quality to find the best overall quality/image size ratio. With Eyesis we use 97-98%, but that is only because we use aberration correction by de-convolution that amplifies the noise and artifacts. BTW - here are our last panoramas made with the new camera (viewing requires good videocard and WebGL-capable browser) http://community.elphel.com/files/eyesis/pano-db-3/webgl_panorama_editor.html?kml=mainstreet_ro.kmlproto=mainstreet_ro.kmlntxt=2as_camera=95start=result_1334548264_780764_1.jpegrange=95labels=falsekeepzoom=falseclosest2d=falseseethrough=0.4transition=5mask=azimuth=23.2elevation=-70.5zoom=0.386follow=falsemv3d=false
Re: [Elphel-support] Recording fullHD JP4 being limited to 7 seconds
On Thu, Apr 26, 2012 at 09:32, flavio soares qazav...@gmail.com wrote: Hi, Andrey, Sorry, I was rebooting, that's what kept me. Here is the result for both cards (even though I can't really understand what it means): [root@Elphel353 /root]3882# hdparm -I /dev/hdb /dev/hdb: ATA device, with non-removable media Model Number: CF CARD 4GB Serial Number: 3C2E Firmware Revision: Standards: Likely used: 5 Configuration: Logical max current cylinders 8060 8060 heads 16 16 sectors/track 63 63 -- CHS current addressable sectors: 8124480 LBA user addressable sectors: 8124480 device size with M = 1024*1024: 3967 MBytes device size with M = 1000*1000: 4159 MBytes (4 GB) Capabilities: LBA, IORDY(may be)(cannot be disabled) bytes avail on r/w long: 4 Queue depth: 1 Standby timer values: spec'd by Vendor R/W multiple sector transfer: Max = 1 Current = 0 DMA: mdma0 mdma1 *mdma2 Cycle time: min=120ns recommended=120ns PIO: pio0 pio1 pio2 pio3 pio4 Cycle time: no flow control=120ns IORDY flow control=120ns HW reset results: CBLID- below Vih Device num = 1 [root@Elphel353 /root]3882# hdparm -I /dev/hda /dev/hda: ATA device, with non-removable media Model Number: CF CARD 4GB Serial Number: 3C48 Firmware Revision: Standards: Likely used: 5 Configuration: Logical max current cylinders 8060 8060 heads 16 16 sectors/track 63 63 -- CHS current addressable sectors: 8124480 LBA user addressable sectors: 8124480 device size with M = 1024*1024: 3967 MBytes device size with M = 1000*1000: 4159 MBytes (4 GB) Capabilities: LBA, IORDY(may be)(cannot be disabled) bytes avail on r/w long: 4 Queue depth: 1 Standby timer values: spec'd by Vendor R/W multiple sector transfer: Max = 1 Current = 0 DMA: mdma0 mdma1 *mdma2 Cycle time: min=120ns recommended=120ns PIO: pio0 pio1 pio2 pio3 pio4 Cycle time: no flow control=120ns IORDY flow control=120ns HW reset results: CBLID- below Vih Device num = 0 Thanks for the link, I'll check on it soon. I have a really good graphic card here, but my Iceweasel is not up to the task =) I'll reboot and check it with chrome! ps.: Do you think it's too dangerous to test the Sata the way I mentioned? My computer is actually open at the moment with a Sata literally hanging on it (backuping). =) It should work fine that way! With the right cards or an external HDD you should get up to 12-15 MByte/s writespeed There are 2 issues with datarate: -) When using streamer at the same time as camogm the streamer takes up CPU resources and reduces the maximum datarate camogm can record before it will overflow -) PHP calls can also take up CPU resources and reduce camogm datarate before it overflows, camogmgui itself polls the camera to get updates and therefore reduces datarate. Its a balance of staying informed and keeping resources free ;) I lowered the poll frequency in camogmgui to 2 per second (if I remember correctly) to help with PHP resources. If you want the best possible conditions you can try sshing to the camera and killing both str (streamer) and all php instances. Then you need to start recording from the command-line though - more infos on the well documented camogm wiki page: http://wiki.elphel.com/index.php?title=Camogm#start Regards Sebastian 2012/4/26, Andrey Filippov support-list@support.elphel.com: Hello, Andrey, thanks for the quick response! You are welcome , Flavio I'm also online (here it is 3:40am) =) Not that late he - less than 1AM :-) My Kingston cards are not listed in the blacklist. Let's get them to work and see how far they go. Here are the results of the tests, I'm ssh-ed into the cam: [root@Elphel353 /mnt/0]944# time dd if=/dev/circbuf of=/dev/null 38656+0 records in 38656+0 records out real 0m 0.42s user 0m 0.03s sys 0m 0.39s [root@Elphel353 /mnt/0]944# time dd if=/dev/circbuf of=/dev/hdb1 38656+0 records in 38656+0 records out real 0m 6.75s user 0m 0.19s sys 0m 2.46s [root@Elphel353 /mnt/0]944# If I understood correctly, my card takes 6.75 seconds to write the 19MB and 19 MB / 6.75 = 2.8MB/s should be the maximum data rate my cards can handle, is that it? Yes, the cards are in PIO mode so seem to be disabled. And as I wrote - it is not only that they are ~16/2.8 ~= 5.5 times slower than those in DMA mode, they use CPU much more during recording, so no resources fro other CPU tasks. Can you try them
Re: [Elphel-support] Recording fullHD JP4 being limited to 7 seconds
With Eyesis we use 97-98%, but that is only because we use aberration correction by de-convolution that amplifies the noise and artifacts. BTW - here are our last panoramas made with the new camera (viewing requires good videocard and WebGL-capable browser) http://community.elphel.com/files/eyesis/pano-db-3/webgl_panorama_editor.html?kml=mainstreet_ro.kmlproto=mainstreet_ro.kmlntxt=2as_camera=95start=result_1334548264_780764_1.jpegrange=95labels=falsekeepzoom=falseclosest2d=falseseethrough=0.4transition=5mask=azimuth=23.2elevation=-70.5zoom=0.386follow=falsemv3d=false o.O Man, this is really impressive. Is this link public, I mean, can I show it around? ___ Support-list mailing list Support-list@support.elphel.com http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
Re: [Elphel-support] Recording fullHD JP4 being limited to 7 seconds
Flavio, That you for the nice words. The link a gave you is posted to the public mailing list, and it is hosted on the public server, so there is no secret about it and you may show it (all the content on the that server is released under the GNU FDL 1.3). We just did not want to make much buzz about it yet - these are our first images with the new series of the cameras and there is a lot to be done yet, and there are many processing parameters to tweak - i took them from the previous camera settings (i.e. some de-mosaic artifacts are visible). But we definitely consider it a proof that our concepts worked, the difference with the first Eyesis that had the same sensors and same lenses (not to count the top fisheye) is significant. We will publish more details later - so far we were very busy as we had to make our first customers to wait extra 4 months - we had to completely redesign and re-build the critical part of the cameras - the sensor front end mechanics. We are going to demonstrate the new camera together with the camera calibration machine and the process of calibration at this year SIGGRAPH in Los Angeles (https://www.google.com/search?q=siggraph+elphel) - the booth number is different as we had to increase the size - could not fit in 3x3 meters. Andrey On Thu, Apr 26, 2012 at 2:48 AM, flavio soares qazav...@gmail.com wrote: With Eyesis we use 97-98%, but that is only because we use aberration correction by de-convolution that amplifies the noise and artifacts. BTW - here are our last panoramas made with the new camera (viewing requires good videocard and WebGL-capable browser) http://community.elphel.com/files/eyesis/pano-db-3/webgl_panorama_editor.html?kml=mainstreet_ro.kmlproto=mainstreet_ro.kmlntxt=2as_camera=95start=result_1334548264_780764_1.jpegrange=95labels=falsekeepzoom=falseclosest2d=falseseethrough=0.4transition=5mask=azimuth=23.2elevation=-70.5zoom=0.386follow=falsemv3d=false o.O Man, this is really impressive. Is this link public, I mean, can I show it around? ___ Support-list mailing list Support-list@support.elphel.com http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com ___ Support-list mailing list Support-list@support.elphel.com http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
