We're still trying without considerable progress; I switched on the
sdram sync debug bit in the /etc/init.d/fpga script, syslog reports a
number of errors, maybe they help to locate the issue:
Jan 1 00:00:40 Elphel353 kernel: Found MT9P001 2592x1944 sensor, chip ID=1801
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/framepars.c:420:initMultiPars
GLOBALPARS(G_MULTI_NUM)=f
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:922:multisensor_pgm_detectsensor
replaced: sensor->sensorDelay=0xed8
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1278:multisensor_pgm_sensorphase
cableDelay1=0x708 FPGADelay1= 0xf3e sensorproc_phys->sensor.sensorDelay=0x99c,
thisPhase1=0x9000f
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1283:multisensor_pgm_sensorphase
cableDelay2=0x708 FPGADelay2= 0xefc sensorproc_phys->sensor.sensorDelay=0x99c,
thisPhase3=0x9000c
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1288:multisensor_pgm_sensorphase
cableDelay3=0x708 FPGADelay3= 0x101a sensorproc_phys->sensor.sensorDelay=0x99c,
thisPhase3=0x90019
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x0 newPhase= 0x0 oldPhase= 0x0^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=0 newPhase90=0x0 newPhase= 0x0 oldPhase= 0x0^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1797:multisensor_memphase
centroid=0x93bc, OK=0
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=0 newPhase90=0x1 newPhase= 0x0 oldPhase= 0x0^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1797:multisensor_memphase
centroid=0x3d13, OK=0
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=0 newPhase90=0x2 newPhase= 0x0 oldPhase= 0x10000^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1797:multisensor_memphase
centroid=0xffff4835, OK=0
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=0 newPhase90=0x3 newPhase= 0x0 oldPhase= 0x20000^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1797:multisensor_memphase
centroid=0xffffd37b, OK=0
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0x0 oldPhase= 0x30000^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=0 newPhase90=0x3 newPhase= 0x7d oldPhase= 0x30000^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1797:multisensor_memphase
centroid=0xffffa2cb, OK=0
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=0 newPhase90=0x3 newPhase= 0xbb oldPhase= 0x3007d^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1646:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xda oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1646:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xea oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1646:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xf2 oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1646:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xf6 oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1646:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xf8 oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1646:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xf9 oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1646:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0x0 oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=0 newPhase90=0x3 newPhase= 0xff83 oldPhase= 0x30000^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1797:multisensor_memphase
centroid=0x285a, OK=0
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=0 newPhase90=0x3 newPhase= 0xff45 oldPhase= 0x3ff83^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1673:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xff26 oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1673:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xff16 oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1673:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xff0e oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1673:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xff0a oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1673:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xff08 oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1673:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x2 resetDCM=1 newPhase90=0x3 newPhase= 0xff07 oldPhase= 0xfffffffc^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1673:multisensor_adjustSDRAM
DCM error=4
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1685:multisensor_adjustSDRAM
low90=3, low=250, high90=3, high=-250
Jan 1 00:00:40 Elphel353 kernel: **** ERROR adjusting SDRAM clock phase in
arch/cris/arch-v32/drivers/elphel/multisensor.c:1697:multisensor_adjustSDRAM
Jan 1 00:00:40 Elphel353 kernel: low90=3, low_l=249, high90=3, high_l=-250
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1310:multisensor_pgm_sensorphase
**** ERROR adjusting SDRAM clock phase in
arch/cris/arch-v32/drivers/elphel/multisensor.c:1311:multisensor_pgm_sensorphase,
result=0xffffffff
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x3 resetDCM=1 newPhase90=0x9 newPhase= 0xf oldPhase= 0x0^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x4 resetDCM=1 newPhase90=0x9 newPhase= 0xc oldPhase= 0x0^M
Jan 1 00:00:40 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x5 resetDCM=1 newPhase90=0x9 newPhase= 0x19 oldPhase= 0x0^M
Jan 1 00:00:40 Elphel353 kernel: Resetting MT9X001 sensor^M
Jan 1 00:00:40 Elphel353 kernel: Reading sensor registers to the shadows:^M
Jan 1 00:00:40 Elphel353 kernel: Initializing MT9X001 registers with default
values:^M
Jan 1 00:00:40 Elphel353 kernel: Starting hardware sequencers
Jan 1 00:00:47 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x3 resetDCM=0 newPhase90=0x1 newPhase= 0xb0 oldPhase= 0x1000f^M
Jan 1 00:00:47 Elphel353 kernel:
arch/cris/arch-v32/drivers/elphel/multisensor.c:1506:multisensor_set_phase
reg_addr=0x4 resetDCM=0 newPhase90=0x1 newPhase= 0xb0 oldPhase= 0x1000c^M
HW needs replacement?
Simon
Am 22.06.20 um 20:35 schrieb Oleg:
Yes, 8.2.15 should be fine too... as long as you find the MULTI_*
parameters.
Regards,
Oleg
On Mon, Jun 22, 2020 at 12:25 PM Simon Vogl <off...@voxel.at
<mailto:off...@voxel.at>> wrote:
Hi Oleg,
thanks for the timely feedback, let me answer inline:
Am 22.06.20 um 19:44 schrieb Oleg:
Hi,
When stacking images vertically the mux board let's the 1st
sensor (top image) through w/o buffering in SDRAM. The second is
buffered. So, it could be SDRAM phase.
Well, this is definitely involved - when I set the right clock
parameters, I could get reasonable picture content again (although
only on one image).
1. What firmware version are you using?
~# cat /etc/issue
or
~# cat /etc/release
[root@Elphel353 /root]817# cat /etc/release
JFFSID="8.2.15"
close to .16 but not exactly - does an upgrade in the field make
sense?
2. If the firmware is 8.2.16 (hopefully), then the phase settings
are available through the camera parameters:
http://192.168.0.9/autocampars.php -> check 'unsafe' - in the
table you will find:
MULTI_PHASE_SDRAM
MULTI_PHASE1
MULTI_PHASE2
MULTI_PHASE3
Check the values from a camera that works ok. And try to fix them
for a bad camera from this autocampars page (not
10359_controls.html, which communicates directly with fpga -
cannot not store settings across reboots)
Once you set the values, go back to
http://192.168.0.9/autocampars.php and 'save' the config in the
second table. Then reboot.
ok great, will try it this way, guess that's what I was missing.
3. Has anything changed recently? Like you stored a new
configuration? Reboot often? Applied any changes?
The camera stores and loads its firmware from flash -
sometimes the fs or some file there would get corrupted when
changes to the filesystem are applied but not properly synced.
With a single lens camera most of the trouble was usually
caused by /etc/autocampars.xml going corrupt. Then backing it up,
removing the file and then rebooting (it autoregenerated on
reboot if not there) might help. Note - if you manipulate files
manually from a command line - run 'sync' when done. Then repeat
step 2 if firmware is 8.2.16.
The maintenaince guys did a few reboots last week (suspected
troubles with a poe switch) and I am hoping that this is the cause
of the trouble.... I bit-compared the bit files, they're identical
to a running camera. The xml files looked fine as well.
Will try around more and come back to you - thanks a lot!
Simon
Regards,
Oleg
On Mon, Jun 22, 2020 at 10:29 AM Simon Vogl <off...@voxel.at
<mailto:off...@voxel.at>> wrote:
Dear all,
we have a bunch of 353-based stereo cameras that are in
operation for seven years meanwhile and we've been very happy
with them. Over the last weeks, some of them have image
errors, at first sight it looks like one of the sensor is not
configured correctly, a combined image looks like:
I found that this seems to be a problem with the 10359
multiplexer board -- exchanging this resolves the issue for a
camera.
When booting, the relevant portion of dmesg reports this
error on the affected cameras:
arch/cris/arch-v32/drivers/elphel/framepars.c:390:initGlobalPars
GLOBALPARS(G_DEBUG)=0
initSequencers:resetting both sequencers
arch/cris/arch-v32/drivers/elphel/framepars.c:420:initMultiPars
GLOBALPARS(G_MULTI_NUM)=0
Initializing DMA registers for EXTDMA3 -
sensor clock set to 48000000
10359 bitstream version =0x0359a054
10353 sensor clock set to 96000000
10359A sensor clock set to 96000000
removing MRST from the sensor
Found MT9P001 2592x1944 sensor on 10359A port 1, chip ID=1801
Found MT9P001 2592x1944 sensor on 10359A port 2, chip ID=1801
Setting internal HACT generation
Found MT9P001 2592x1944 sensor, chip ID=1801
arch/cris/arch-v32/drivers/elphel/framepars.c:420:initMultiPars
GLOBALPARS(G_MULTI_NUM)=f
**** ERROR adjusting SDRAM clock phase in
arch/cris/arch-v32/drivers/elphel/multisensor.c:1697:multisensor_adjustSDRAM
low90=3, low_l=249, high90=3, high_l=-250
arch/cris/arch-v32/drivers/elphel/multisensor.c:1310:multisensor_pgm_sensorphase
**** ERROR adjusting SDRAM clock phase in
arch/cris/arch-v32/drivers/elphel/multisensor.c:1311:multisensor_pgm_sensorphase,
result=0xffffffff
Resetting MT9X001 sensor
Reading sensor registers to the shadows:
Initializing MT9X001 registers with default values:
Starting hardware sequencers
arch/cris/arch-v32/drivers/elphel/quantization_tables.c:211:init_qtable_head_cache
I have tried to set the clock parameters from a functioning
specimen via the /359/10359_controls.html which temporarily
resolves the issue, but after a reboot the problems reappear.
Any chance to resolve this in software or do we need to
exchange boards? If so - are they still available?
All the best from Austria, stay healthy,
Simon
--
VoXel Interaction Design |www.voxel.at <http://www.voxel.at>
DI Dr.techn. Simon Vogl |si...@voxel.at <mailto:si...@voxel.at>
Tomaschekweg 46 | +43 650 2323 555
A-4040 Linz - Austria |
Office address: Industriezeile 35, 4020 Linz (2nd floor)
_______________________________________________
Support-list mailing list
Support-list@support.elphel.com
<mailto:Support-list@support.elphel.com>
http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
--
Best regards,
Oleg Dzhimiev
Electronics Engineer
phone: +1 801 783 5555 x124
Elphel, Inc.
_______________________________________________
Support-list mailing list
Support-list@support.elphel.com <mailto:Support-list@support.elphel.com>
http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
--
VoXel Interaction Design |www.voxel.at <http://www.voxel.at>
DI Dr.techn. Simon Vogl |si...@voxel.at <mailto:si...@voxel.at>
Tomaschekweg 46 | +43 650 2323 555
A-4040 Linz - Austria |
Office address: Industriezeile 35, 4020 Linz (2nd floor)
_______________________________________________
Support-list mailing list
Support-list@support.elphel.com
<mailto:Support-list@support.elphel.com>
http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
--
Best regards,
Oleg Dzhimiev
Electronics Engineer
phone: +1 801 783 5555 x124
Elphel, Inc.
_______________________________________________
Support-list mailing list
Support-list@support.elphel.com
http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
--
VoXel Interaction Design | www.voxel.at
DI Dr.techn. Simon Vogl | si...@voxel.at
Tomaschekweg 46 | +43 650 2323 555
A-4040 Linz - Austria |
Office address: Industriezeile 35, 4020 Linz (2nd floor)
_______________________________________________
Support-list mailing list
Support-list@support.elphel.com
http://support.elphel.com/mailman/listinfo/support-list_support.elphel.com
