Hi Steve
Per our analysis, MaxMpxCount is enforced on all the commands including
Negotiate, Echo and SMBOplockBreak. Please find the answers as follows :
1. I would like clarification on the how the maximum number of cifs
simultaneous requests are enforced, and for which cifs operations they apply
to. In part this is to avoid triggering a serious problem in Windows 7 and
Windows Vista (when they are run as a server) when exceeding this limit on
simultaneous requests.
On both the client and the server, MaxMpxCount is used to limit
the number of outstanding commands Per SMB Connection. This behavior applies
to ALL commands.
2. What I have observed in my testing:
- Although Windows 7 and Vista allows more than 10 simultaneous SMB
write requests (and Windows 2008 allows more than 50 simultaneous write
requests), Windows 7 and Windows Vista appear to have problems when 20 or 30
are in flight at one time.
We would be requiring capture for this scenario for further analysis.
3. Windows does not enforce a limit on SMB Negotiate Protocol
Windows seems to ignore checks on maximum simultaneous requests on
certain handle based operations (SMB writeX in particular) until after the
file is closed (the writes don't get an error, but the next QueryPathInfo
after the file is closed gets an error).
Windows in fact sends STATUS_DATA_NOT_ACCEPTED. Please let us
know if you are not getting this error.
4. I need to understand on which requests (besides SMB Negotiate) I should
ignore MaxMpxCount. In particular, SMB Echo, seems like an obvious choice to
ALWAYS allow to be sent by the client, since SMB
Echo is sent to make sure the server is alive (when for example the limit
on simultaneous requests has been reached due to slow opens, or writes past end
of file) and to prevent timeout. If a client is restricted from sending SMB
Echo then there is no reasonably reliable mechanism available to determine
whether the server
is dead or just hung temporarily processing slow requests.
To really have a problem with SMB Echo or any other commands, total
pending requests(including SMB Echo) should be greater than what server could
really handle, which might be possible only when server is really busy. The
above mentioned scenario would not arise unless server is busy.
In this scenario, SMB should fail Echo if it cannot process more requests.
Client can keep retrying to check if server is available.
5. MS-CIFS is not clear on which commands MaxMpxCount is enforced.
MaxMpxCount will be enforced for all the commands in a particular
Connection.
6. I would like to ignore MaxMpxCount on (sending one of) SMBEcho (where you
could cause data integrity issues if you limited the ability to check server
state up/down) and SMBNegotiate (where it is obviously
not set yet) and SMBOplockBreak responses (since you can't guarantee the
order that these are received/sent from the server relative to other frames
which are being processed, and can cause the server
session to drop if you don't allow the client to send these) to the server
since there is no reasonable mechanism to limit these without risking problems
with prematurely taking down a slow session (perhaps
opens of offline files for example).
SMBEcho, SMB Negotiate & SMB Oplock response are NOT excluded from
MaxMpxCount limit. However the server does not enable Oplocks if MaxMpxCount =
1, as Oplock break notification/ack need an extra work item when current
create/lock/write request is pending on oplock break.
7. It looks like the safest way to handle this is for the client to limit
pending requests to the MaxMpxCount, with the exception of the three SMB types
listed above.
Client would always limit the pending requests to MaxMpxCount,
without the exception of any commands.
8. In previous versions of MS-CIFS there were mentions of different processes,
and uids on the negotiated tcp session and their relation to the MaxMpxCount -
and I would also like to verify whether the limits are considered relative to
some combination of the negotiated tcp session and uid and/or pid as earlier
discussions
implied (or file handle, as our testing hinted at).
MaxMpxCount on the client is based on the outstanding PID/MID pairs
in the Client.Connection.PIDMIDList per SMB Connection.
Please let us know if any further clarification is required.
Thanks
Tarun Chopra.
-----Original Message-----
From: Bryan Burgin
Sent: Friday, October 14, 2011 1:20 PM
To: 'Steve French'; Tarun Chopra
Cc: MSSolve Case Email; '[email protected]'; '[email protected]'
Subject: RE: [REG:111092973873305] [cifs-protocol] [Pfif] Maximum Simultaneous
Request for CIFS behavior clarification
[Adding Tarun]
Steve,
I am working with Windows Sustained Engineering on the Windows Vista and
Windows 7 issue you reported. It is not directly related to MaxMpx, which is
the topic of this thread.
Tarun is researching your questions re "I need to understand on which requests
(besides SMB Negotiate) I should ignore MaxMpxCount", "MS-CIFS is not clear on
which commands MaxMpxCount is enforced" and other protocol-related MaxMpx
questions.
Bryan
-----Original Message-----
From: Bryan Burgin
Sent: Thursday, September 29, 2011 2:19 PM
To: Steve French
Cc: MSSolve Case Email; [email protected]<mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Subject: [REG:111092973873305] [cifs-protocol] [Pfif] Maximum Simultaneous
Request for CIFS behavior clarification
Hi, Steve,
I'll look into this for you in the process of working the issue that is driving
this inquiry: the large file copy to Vista or Windows 7 clients resulting in
the DOS NOT_ENOUGH_MEMORY error on subsequent SMB session setups (among other
commands, like Queries, which return STATUS_INSUFF_SERVER_RESOURCES).
I have the repro of that issue and I'm presently debugging it. I will note
that on my Windows 7 system (acting as the server in this transaction), it
replies with a negotiation specifying 50 (0x32) for MaxMpxCount. Furthermore,
the "Client05" system we were using, also Windows 7, was forced to use 50 by
setting Services\LanmanSERVER\Parameters\MaxMpxCt = (DWORD) 50 (previously, it
returned 10 for MaxMpxCount). It continued to fail. (I also experimented with
Services\LanmanWORKSTATION\Parameters\MaxCmds = (DWORD) 50). Thus, the value
returned by the server side via Negotiate MaxMpxCount does not seem to be
linked to your initial issue. Setting \LanmanSERVER\Parameters\MaxWorkItems,
however, does seem to have a positive effect on your original issue. And, I
was able to find code that used MaxWorkItems in calculating if there was
sufficient buffers for an operation. So, there may be a direct collation yet
to be verified.
I am also aware that the failing scenario seems to be triggered by having a lot
of Writes-in-flight. In our testing together, we were seeing 15 to 20 to more
Writes-in-flight, which exceeded MaxMpxCount (then 10) by quite a bit.
I'm debugging the cause of your initial issue and in the process I'll research
your questions re MaxMpxCount. Once root cause is determined, we'll know more
if there was a connection between the Out-of-Memory errors and MaxMpxCount. In
the process of that work, I'll research the issues you listed below.
Bryan
-----Original Message-----
From: Christopher R. Hertel
[mailto:[email protected]]<mailto:[mailto:[email protected]]>
Sent: Thursday, September 29, 2011 12:16 PM
To: Steve French
Cc: Interoperability Documentation Help;
[email protected]<mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Subject: Re: [cifs-protocol] [Pfif] Maximum Simultaneous Request for CIFS
behavior clarification
Steve,
Only Microsoft can correctly answer these questions, at this point, but I am
adding my comments here to encourage them to spend a good deal of time studying
both the client and server code. The behavior of the MaxMpxCount value, in
particular, is buried in the source.
My team and I spent a great deal of time digging into this particular question
when writing the documentation. We did try to make things as clear as we
could. There is an internal table, as described in [MS-CIFS], that is keyed on
both PID and MID (not just MID). This table keeps track of outstanding
requests from the client. Note that an open OpLock is an out standing request,
as is a ChangeNotify.
What is difficult to determine when studying the code is how this internal
table actually performs under load.
Another thing to consider is how the Windows clients handle the MaxMpxCount
value. I do not believe that they put much stress on the server, in general.
Chris -)-----
Steve French wrote:
> Resending with correct title.
>
> On Thu, Sep 29, 2011 at 1:09 PM, Steve French
> <[email protected]<mailto:[email protected]>> wrote:
>> I would like clarification on the how the maximum number of cifs
>> simultaneous requests are enforced, and for which cifs operations
>> they apply to. In part this is to avoid triggering a serious problem
>> in Windows 7 and Windows Vista (when they are run as a server) when
>> exceeding this limit on simultaneous requests.
>>
>> What I have observed in my testing:
>> - Although Windows 7 and Vista allows more than 10 simultaneous SMB
>> write requests (and Windows 2008 allows more than 50 simultaneous
>> write requests), Windows 7 and Windows Vista appear to have problems
>> when 20 or 30 are in flight at one time.
>> - Windows does not enforce a limit on SMB Negotiate Protocol
>> - Windows seems to ignore checks on maximum simultaneous requests on
>> certain handle based operations (SMB writeX in particular) until
>> after the file is closed (the writes don't get an error, but the next
>> QueryPathInfo after the file is closed gets an error).
>>
>> I need to understand on which requests (besides SMB Negotiate) I
>> should ignore MaxMpxCount. In particular, SMB Echo, seems like an
>> obvious choice to ALWAYS allow to be sent by the client, since SMB
>> Echo is sent to make sure the server is alive (when for example the
>> limit on simultaneous requests has been reached due to slow opens, or
>> writes past end of file) and to prevent timeout. If a client is
>> restricted from sending SMB Echo then there is no reasonably reliable
>> mechanism available to determine whether the server is dead or just
>> hung temporarily processing slow requests.
>>
>> MS-CIFS is not clear on which commands MaxMpxCount is enforced.
>>
>> I would like to ignore MaxMpxCount on (sending one of) SMBEcho (where
>> you could cause data integrity issues if you limited the ability to
>> check server state up/down) and SMBNegotiate (where it is obviously
>> not set yet) and SMBOplockBreak responses (since you can't guarantee
>> the order that these are received/sent from the server relative to
>> other frames which are being processed, and can cause the server
>> session to drop if you don't allow the client to send these) to the
>> server since there is no reasonable mechanism to limit these without
>> risking problems with prematurely taking down a slow session (perhaps
>> opens of offline files for example).
>>
>> It looks like the safest way to handle this is for the client to
>> limit pending requests to the MaxMpxCount, with the exception of the
>> three SMB types listed above.
>>
>> In previous versions of MS-CIFS there were mentions of different
>> processes, and uids on the negotiated tcp session and their relation
>> to the MaxMpxCount - and I would also like to verify whether the
>> limits are considered relative to some combination of the negotiated
>> tcp session and uid and/or pid as earlier discussions implied (or
>> file handle, as our testing hinted at).
>>
>>
>>
>>
>>
>> --
>> Thanks,
>>
>> Steve
>>
>
>
>
--
"Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X
Samba Team -- http://www.samba.org/ -)----- Christopher R. Hertel
jCIFS Team -- http://jcifs.samba.org/ -)----- ubiqx development, uninq.
ubiqx Team -- http://www.ubiqx.org/ -)-----
[email protected]<mailto:[email protected]>
OnLineBook -- http://ubiqx.org/cifs/ -)-----
[email protected]<mailto:[email protected]>
_______________________________________________
cifs-protocol mailing list
[email protected]
https://lists.samba.org/mailman/listinfo/cifs-protocol
