Hello Mike,

I have applied the patch and sent you the log out put.
would you please check and let me know .

I ran the following command to enable debug :
make DEBUG_TCP=1 DEBUG_SCSI=1
make DEBUG_TCP=1 DEBUG_SCSI=1 install


and here is the libiscsi.c file after applying the patch

void iscsi_conn_failure(struct iscsi_conn *conn, enum iscsi_err err)
{
        struct iscsi_session *session = conn->session;
        unsigned long flags;

        dump_stack();

        spin_lock_irqsave(&session->lock, flags);
        if (session->state == ISCSI_STATE_FAILED) {
                spin_unlock_irqrestore(&session->lock, flags);
                return;
        }






Mike Christie wrote:
iscsi wrote:
  
Hi Mike,

While back I provided the log file to iqstor tech support and they told 
me that
initiator logged out in the middle of the connection. and now that I do 
attention
I see that it is correct. initiator is logged out and I don't know if 
this is due to
blocked session error that we saw in open-iscsi debug file or it is just 
logged out due to
time out.
    

You get the blocked error message when the iniatiator logs out of the 
session (either because it detected a problem like a timeout, RFC issue, 
network layer returned a error, cable is unplugged, etc) or the target 
detected a problem and drops the session (like a timeout, RFC issue, 
etc), and we cannot log back in within 
node.session.timeo.replacement_timeout seconds.

Could you run the attached patch with the kernel debugging on and send 
me the output offlist? It will tell me where the conn error is being 
fired from.



  
would you please let me know what would be the reason that initiator is 
logged out ?
our program is writing continuously on  target,  and I don't think it is 
related to time out however
here is the time out on our open-iscsi, I am going to increase 
node.conn[0].timeo.noop_out_interval  to  10
and node.conn[0].timeo.noop_out_timeout to 15. but do you think 
increasing other time outs will help ?
    

Those would help if you saw some error message in the initiator log 
about a nop out timing out.

  
and also I am thinking to change node.conn[0].startup to automatic ( now 
it is  manual ) .

node.conn[0].startup = manual
    

That does not really do anything.

  
current time out setting :
node.conn[0].timeo.logout_timeout = 15
node.conn[0].timeo.login_timeout = 15
node.conn[0].timeo.auth_timeout = 45
node.conn[0].timeo.noop_out_interval = 5
node.conn[0].timeo.noop_out_timeout = 5


On initiator :
Nov 21 11:41:11 Yuan kernel: connection2:0: iscsi: detected conn error 
(1011)

on Target :
11:41:17 11/21/07; cxn_update_tgt(ddb_index 0x0000, state 0x0006, chipnum 0)
11:41:17 11/21/07; AME_daemon->cxn_update_tgt->update state to 0006, 
User=our_user_name WWN=0x827E0B2B090067B7 at (initID=0 chipnum=0)
11:41:17 11/21/07;      update SourceIP from 0x620A0A0A to 0x620A0A0A
11:41:17 11/21/07;      update NodeName from 
[iqn.2005-03.org.open-iscsi:8b9afe093c] to 
iqn.2005-03.org.open-iscsi:8b9afe093c
11:41:17 11/21/07; AME_daemon->cxn_pop_tgt: logout User=our_user_name 
WWN=0x827E0B2B090067B7 at (initID=0 chipnum=0) from 
host=iqn.2005-03.org.open-iscsi:8b9a
11:41:21 11/21/07; PRQ_daemon->cxn_push_tgt->login success: 
User=our_user_name WWN=0x827E0B2B090067B7 at (initID=0 chipnum=0)
11:41:21 11/21/07; handle_IOCB_Immediate_Notify: 1720270.88 
Authentication Success for 10.10.10.98 on chip 0, ddb_entry 0
11:41:21 11/21/07; handle_IOCB_Immediate_Notify: chip 0: index 0: Normal 
Session.
11:41:21 11/21/07; cxn_update_tgt(ddb_index 0x0000, state 0x0004, chipnum 0)
11:41:21 11/21/07; AME_daemon->cxn_update_tgt->update state to 0004, 
User=our_user_name WWN=0x827E0B2B090067B7 at (initID=0 chipnum=0)
11:41:21 11/21/07;      update SourceIP from 0x620A0A0A to 0x620A0A0A
11:41:21 11/21/07;      update NodeName from 
[iqn.2005-03.org.open-iscsi:8b9afe093c] to 
iqn.2005-03.org.open-iscsi:8b9afe093c
11:41:21 11/21/07; isp4010_Async_Event: Device 0,chipnum 0: is in active 
session!


iscsi wrote:
    
Mike Christie wrote:
  
      
iscsi wrote:
  
    
        
Hi Mike,

Please ignore that specific one specially in October 4 because at that 
week we upgraded
our target firmware and that firmware caused problem for us.
    
      
          
Ok.

  
    
        
after iscsi discovered and logged in to target, we started to create 
extd3 file system
on top of each LUN ( probably due to new firmware problem and then we 
downgraded the
target firmware ).

Debugging started at November 6 about 10 am , and we didn't receive any 
connection error
until we started to read from target heavily( Note that writing to 
discovered LUNs didn't cause any connection
errors, just reading caused those connection and scsi errors )
    
      
          
Did you still get "cdb alloc len 4 < needed len 152." errors on your target?
  
    
        
any time we login to target , we get cdb error, but some times at the 
same time we get connection error in initiator
side as well. then we will not see any error until we start reading 
heavy from target  ( heavy means reading a lot of
files per seconds, our file size are pretty small , but they are a lot 
in number )

after login to target, we will not see cdb error any more on target side.

here is the errors that shows up on target when we see continues 
connection error on initiator side, and now that I am
looking in to the log files , I see that there is a connection error 
even on target side, and I think this is happening because
either initiator side or target side can not keepup with reading files.
is there any setting in initiotor side that can improve reading ? or if 
it can act as a nice and put files in queue rather than
sending them to target !!!!

I was thinking to decrease buffer size on initiator side in case our 
target can not keep up with initiator speed.
is there any setting on open-iscsi that will help us to decrease the 
speed according to target responsiveness ?

Thanks for help

[EMAIL PROTECTED]:21:29 11/06/07:ERR:iSCSI_fs do_writev: 426710.13 CTIO fail at 
DataIn: initID=0 conID=0 CTIOstatus=0x28 hlun=0 taskTag=0x2E000000 
pid=3259 [EMAIL PROTECTED]:21:29 11/06/07:INFO:000 loginT=424949 conID=0 
initID=0 chipnum=0 state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:001 loginT=422836 conID=0 initID=0 chipnum=1 
state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:ihandleCTIOstatus: 426710.13 Connection 
unavailable, taskTag=0x13000000
[EMAIL PROTECTED]:21:29 11/06/07:ERR:iSCSI_fs do_writev: 426710.13 CTIO fail at 
DataIn: initID=0 conID=0 CTIOstatus=0x28 hlun=0 taskTag=0x13000000 
pid=3262 [EMAIL PROTECTED]:21:29 11/06/07:INFO:000 loginT=424949 conID=0 
initID=0 chipnum=0 state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:001 loginT=422836 conID=0 initID=0 chipnum=1 
state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:ihandleCTIOstatus: 426710.13 Connection 
unavailable, taskTag=0x3B000000
[EMAIL PROTECTED]:21:29 11/06/07:ERR:iSCSI_fs do_writev: 426710.13 CTIO fail at 
DataIn: initID=0 conID=0 CTIOstatus=0x28 hlun=0 taskTag=0x3B000000 
pid=3248 [EMAIL PROTECTED]:21:29 11/06/07:INFO:000 loginT=424949 conID=0 
initID=0 chipnum=0 state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:001 loginT=422836 conID=0 initID=0 chipnum=1 
state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:ihandleCTIOstatus: 426710.13 Connection 
unavailable, taskTag=0x21000000
[EMAIL PROTECTED]:21:29 11/06/07:ERR:iSCSI_fs do_writev: 426710.13 CTIO fail at 
DataIn: initID=0 conID=0 CTIOstatus=0x28 hlun=0 taskTag=0x21000000 
pid=3261 [EMAIL PROTECTED]:21:29 11/06/07:INFO:000 loginT=424949 conID=0 
initID=0 chipnum=0 state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:001 loginT=422836 conID=0 initID=0 chipnum=1 
state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:ihandleCTIOstatus: 426710.13 Connection 
unavailable, taskTag=0x13000000
[EMAIL PROTECTED]:21:29 11/06/07:ERR:CTIO fail at Status: CTIOstatus=0x28 
taskTag=0x13000000 pid=3262 delta_ms=10230 cmd=2800,2908530F,00,0020,00
[EMAIL PROTECTED]:21:29 11/06/07:ERR:do_iSCSI_status: 426710.13 CTIO fail at Status: 
initID=0 conID=0 CTIOstatus=0x28 hlun=0 taskTag=0x13000000 pid=3262 
delta_ms=10230
[EMAIL PROTECTED]:21:29 11/06/07:INFO:000 loginT=424949 conID=0 initID=0 chipnum=0 
state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:001 loginT=422836 conID=0 initID=0 chipnum=1 
state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:ihandleCTIOstatus: 426710.13 Connection 
unavailable, taskTag=0x2E000000
[EMAIL PROTECTED]:21:29 11/06/07:ERR:CTIO fail at Status: CTIOstatus=0x28 
taskTag=0x2E000000 pid=3259 delta_ms=10210 cmd=2800,046C003F,00,0008,00
[EMAIL PROTECTED]:21:29 11/06/07:ERR:do_iSCSI_status: 426710.13 CTIO fail at Status: 
initID=0 conID=0 CTIOstatus=0x28 hlun=0 taskTag=0x2E000000 pid=3259 
delta_ms=10210
[EMAIL PROTECTED]:21:29 11/06/07:INFO:000 loginT=424949 conID=0 initID=0 chipnum=0 
state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:001 loginT=422836 conID=0 initID=0 chipnum=1 
state=0004 uname=ourusername
[EMAIL PROTECTED]:21:29 11/06/07:INFO:ihandleCTIOstatus: 426710.13 Connection 
unavailable, taskTag=0x5D000000


  
      
  
    
        
we have configured RAID 5 on target, and each block is 32 KB, then we 
created ext3 file system  which its block size is 4K.
is this make our reading slower ? I mean should we create different file 
system which supports bigger block size ?

    
      
          
I do not know if it makes that much of a difference. I am not sure 
though. It should definately not cause connection errors like in your log.

  
    
        
I will ask iqstor to contact you off the list .
    
      
          
Thanks a lot.

  
    
        
Thanks for help.


mainly we are receiving
Mike Christie wrote:
    
      
          
iscsi wrote:
  
      
        
            
Hi Mike,

Did you find any thing on log files that would help us.
we did more testing and found that we receive those connection errors 
while we are
doing some heavy reading from target.
    
        
          
              
I did not see anything interesting in the logs. A ethereal trace might 
help, but it would be best to get the vendor involved so we can make 
sure we know what their error message means.

The thing is that we start to fall apart right away while the scsi layer 
is just trying to setup the device:

Oct  4 18:39:58 Ming kernel: scsi6 : iSCSI Initiator over TCP/IP
Oct  4 18:39:59 Ming kernel: sdb: unknown partition table
Oct  4 18:55:21 Ming kernel: sdb: sdb1
Oct  4 18:55:23 Ming kernel: sdb: sdb1
Oct  4 19:25:58 Ming kernel: connection1:0: iscsi: detected conn error 
(1011)

So we are not getting very far before we start to hit a problem.

Your target is a iqstor right? Not a istor? Does iqstor support linux 
and have you tried contacting them? If you could have them contact me 
offlist so we can work together on supporting their target it would be 
great.

Is there any iqstor people on the list?

  
      
        
            
Thanks for help.

 

Mike Christie wrote:
    
        
          
              
iscsi wrote:
  
      
          
            
                
Hi Mike,

What do you think about our log output.
can you please let me know what do you think.

    
        
            
              
                  
Give me another day. It is a big log. I am in the middle of trying to 
fix some iscsi tape issue.

  
      
          
            
                
    
        
          
              
  
      
        
            
    
      
          
    
  
    
        
  
      
    



  

diff -aurp open-iscsi-2.0-865.15/kernel/libiscsi.c open-iscsi-2.0-865.15.work/kernel/libiscsi.c --- open-iscsi-2.0-865.15/kernel/libiscsi.c 2007-08-15 18:51:53.000000000 -0500 +++ open-iscsi-2.0-865.15.work/kernel/libiscsi.c 2007-11-26 23:01:09.000000000 -0600 @@ -564,6 +564,8 @@ void iscsi_conn_failure(struct iscsi_con struct iscsi_session *session = conn->session; unsigned long flags; + dump_stack(); + spin_lock_irqsave(&session->lock, flags); if (session->state == ISCSI_STATE_FAILED) { spin_unlock_irqrestore(&session->lock, flags);


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To post to this group, send email to open-iscsi@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/open-iscsi
-~----------~----~----~----~------~----~------~--~---

Reply via email to