RE: Patch to allow large messages to be logged via ColoredConsoleAppender
I had some issues getting robust behaviour from the WriteConsoleW call.
There appear to be internal buffers allocated by the WriteConsole call
which are dependant on the amount of data passed in the first (?) call.
The longest string that I was able to write out successfully on the
first call to WriteConsole was 30,704 chars long, rather an odd limit.
With this as the first call subsequent calls of up to the same length
also succeeded. However if the first call was with a string of 15,000
chars and the second was of 20,000 chars, then the second call would
fail with last error 8 (Not enough storage is available to process this
command).
To resolve this I have changed the implementation to use WriteFile
rather than WriteConsole. WriteFile supports partially writing from a
buffer (WriteConsoleW has a lpNumberOfCharsWritten param which could do
this, but it doesn't seem to ever partially write). The .NET Console
class provides a Stream instance that wraps up all the WriteFile calls.
This change is now committed to CVS.
In my testing this works at least up to 5,000,000 char output strings.
Nicko
-Original Message-
From: Wesley Smith [mailto:[EMAIL PROTECTED]
Sent: 28 January 2005 22:45
To: Log4NET Dev
Subject: RE: Patch to allow large messages to be logged via
ColoredConsoleAppender
From the documentation, I thought that 31999 characters would
work too, but I found that it didn't always work. At least
with my test case, if the first call to WriteConsoleW is
smaller than 26000ish, then you can call it subsequent times
with up to 31999 chars, but if the first call is with
anything 26000ish, then it prints nothing (for example,
passing 3 characters to the first call of WriteConsoleW
doesn't print anything - at least in my test it didn't).
I don't understand this behavior, and hope that you can come
up with something more reasonable and understandable - just
wanted to warn you to be on the lookout for this weird behavior.
-Original Message-
From: Nicko Cadell [mailto:[EMAIL PROTECTED]
Sent: Friday, January 28, 2005 2:35 PM
To: Log4NET Dev
Subject: RE: Patch to allow large messages to be logged via
ColoredConsoleAppender
It looks like the buffer passed to WriteConsoleW is limited
to 64KB,
i.e. 32000 WCHARs less 1 for the NULL if the P/Invoke marshalling
feels
like putting one on the end of the string.
I will update the appender along the lines of your patch.
Thanks,
Nicko
-Original Message-
From: Wesley Smith [mailto:[EMAIL PROTECTED]
Sent: 27 January 2005 20:03
To: log4net-dev@logging.apache.org
Subject: Patch to allow large messages to be logged via
ColoredConsoleAppender
I found that if you attempt to log a message of more than about
26000 characters, ColoredConsoleAppender will not log the message
(no error, just silently eats the message). One possible fix for
the problem is given in the patch below:
---
C:\TEMP\log4net-1.2.0-beta8\src\Appender\ColoredConsoleAppender.cs
Mon Jul 07 01:05:02 2003
+++ C:\Program
Files\log4net\log4net-1.2.0-beta8\src\Appender\ColoredConsoleA
ppender.cs
Wed Jan 26 12:57:22 2005
@@ -280,13 +280,32 @@
string strLoggingMessage =
RenderLoggingEvent(loggingEvent);
- // write the output.
UInt32 uiWritten = 0;
+
+ // WriteConsoleW can only handle a limited
number of characters at a time
+ // and emperically, 26000 is about right.
+ const int maxChunkSize = 26000;
+ while (strLoggingMessage.Length maxChunkSize)
+ {
+ string firstPart =
strLoggingMessage.Substring(0, maxChunkSize);
+ WriteConsoleW( iConsoleHandle,
+ firstPart,
+ (UInt32)firstPart.Length,
+ out (UInt32)uiWritten,
+ IntPtr.Zero);
+
+ // Remove the characters already output
from strLoggingMessage
+ strLoggingMessage =
strLoggingMessage.Substring(maxChunkSize);
+
+ }
+ // write the remaining output.
+
WriteConsoleW( iConsoleHandle,
strLoggingMessage,
(UInt32)strLoggingMessage.Length,
out
(UInt32)uiWritten,
IntPtr.Zero);
+
}
/// summary
RE: Patch to allow large messages to be logged via ColoredConsoleAppender
It looks like the buffer passed to WriteConsoleW is limited to 64KB,
i.e. 32000 WCHARs less 1 for the NULL if the P/Invoke marshalling feels
like putting one on the end of the string.
I will update the appender along the lines of your patch.
Thanks,
Nicko
-Original Message-
From: Wesley Smith [mailto:[EMAIL PROTECTED]
Sent: 27 January 2005 20:03
To: log4net-dev@logging.apache.org
Subject: Patch to allow large messages to be logged via
ColoredConsoleAppender
I found that if you attempt to log a message of more than
about 26000 characters, ColoredConsoleAppender will not log
the message (no error, just silently eats the message). One
possible fix for the problem is given in the patch below:
--- C:\TEMP\log4net-1.2.0-beta8\src\Appender\ColoredConsoleAppender.cs
Mon Jul 07 01:05:02 2003
+++ C:\Program
Files\log4net\log4net-1.2.0-beta8\src\Appender\ColoredConsoleA
ppender.cs
Wed Jan 26 12:57:22 2005
@@ -280,13 +280,32 @@
string strLoggingMessage =
RenderLoggingEvent(loggingEvent);
- // write the output.
UInt32 uiWritten = 0;
+
+ // WriteConsoleW can only handle a limited
number of characters at a time
+ // and emperically, 26000 is about right.
+ const int maxChunkSize = 26000;
+ while (strLoggingMessage.Length maxChunkSize)
+ {
+ string firstPart =
strLoggingMessage.Substring(0, maxChunkSize);
+ WriteConsoleW( iConsoleHandle,
+ firstPart,
+ (UInt32)firstPart.Length,
+ out (UInt32)uiWritten,
+ IntPtr.Zero);
+
+ // Remove the characters already output
from strLoggingMessage
+ strLoggingMessage =
strLoggingMessage.Substring(maxChunkSize);
+
+ }
+ // write the remaining output.
+
WriteConsoleW( iConsoleHandle,
strLoggingMessage,
(UInt32)strLoggingMessage.Length,
out
(UInt32)uiWritten,
IntPtr.Zero);
+
}
/// summary
RE: Patch to allow large messages to be logged via ColoredConsoleAppender
I will keep an eye out for weird behaviour in my testing.
What OS platform have you seen this behaviour on?
Cheers,
Nicko
-Original Message-
From: Wesley Smith [mailto:[EMAIL PROTECTED]
Sent: 28 January 2005 22:45
To: Log4NET Dev
Subject: RE: Patch to allow large messages to be logged via
ColoredConsoleAppender
From the documentation, I thought that 31999 characters would
work too, but I found that it didn't always work. At least
with my test case, if the first call to WriteConsoleW is
smaller than 26000ish, then you can call it subsequent times
with up to 31999 chars, but if the first call is with
anything 26000ish, then it prints nothing (for example,
passing 3 characters to the first call of WriteConsoleW
doesn't print anything - at least in my test it didn't).
I don't understand this behavior, and hope that you can come
up with something more reasonable and understandable - just
wanted to warn you to be on the lookout for this weird behavior.
-Original Message-
From: Nicko Cadell [mailto:[EMAIL PROTECTED]
Sent: Friday, January 28, 2005 2:35 PM
To: Log4NET Dev
Subject: RE: Patch to allow large messages to be logged via
ColoredConsoleAppender
It looks like the buffer passed to WriteConsoleW is limited
to 64KB,
i.e. 32000 WCHARs less 1 for the NULL if the P/Invoke marshalling
feels
like putting one on the end of the string.
I will update the appender along the lines of your patch.
Thanks,
Nicko
-Original Message-
From: Wesley Smith [mailto:[EMAIL PROTECTED]
Sent: 27 January 2005 20:03
To: log4net-dev@logging.apache.org
Subject: Patch to allow large messages to be logged via
ColoredConsoleAppender
I found that if you attempt to log a message of more than about
26000 characters, ColoredConsoleAppender will not log the message
(no error, just silently eats the message). One possible fix for
the problem is given in the patch below:
---
C:\TEMP\log4net-1.2.0-beta8\src\Appender\ColoredConsoleAppender.cs
Mon Jul 07 01:05:02 2003
+++ C:\Program
Files\log4net\log4net-1.2.0-beta8\src\Appender\ColoredConsoleA
ppender.cs
Wed Jan 26 12:57:22 2005
@@ -280,13 +280,32 @@
string strLoggingMessage =
RenderLoggingEvent(loggingEvent);
- // write the output.
UInt32 uiWritten = 0;
+
+ // WriteConsoleW can only handle a limited
number of characters at a time
+ // and emperically, 26000 is about right.
+ const int maxChunkSize = 26000;
+ while (strLoggingMessage.Length maxChunkSize)
+ {
+ string firstPart =
strLoggingMessage.Substring(0, maxChunkSize);
+ WriteConsoleW( iConsoleHandle,
+ firstPart,
+ (UInt32)firstPart.Length,
+ out (UInt32)uiWritten,
+ IntPtr.Zero);
+
+ // Remove the characters already output
from strLoggingMessage
+ strLoggingMessage =
strLoggingMessage.Substring(maxChunkSize);
+
+ }
+ // write the remaining output.
+
WriteConsoleW( iConsoleHandle,
strLoggingMessage,
(UInt32)strLoggingMessage.Length,
out
(UInt32)uiWritten,
IntPtr.Zero);
+
}
/// summary
RE: Patch to allow large messages to be logged via ColoredConsoleAppender
I saw this on Windows XP.
-Original Message-
From: Nicko Cadell [mailto:[EMAIL PROTECTED]
Sent: Friday, January 28, 2005 2:50 PM
To: Log4NET Dev
Subject: RE: Patch to allow large messages to be logged via
ColoredConsoleAppender
I will keep an eye out for weird behaviour in my testing.
What OS platform have you seen this behaviour on?
Cheers,
Nicko
-Original Message-
From: Wesley Smith [mailto:[EMAIL PROTECTED]
Sent: 28 January 2005 22:45
To: Log4NET Dev
Subject: RE: Patch to allow large messages to be logged via
ColoredConsoleAppender
From the documentation, I thought that 31999 characters would
work too, but I found that it didn't always work. At least
with my test case, if the first call to WriteConsoleW is
smaller than 26000ish, then you can call it subsequent times
with up to 31999 chars, but if the first call is with
anything 26000ish, then it prints nothing (for example,
passing 3 characters to the first call of WriteConsoleW
doesn't print anything - at least in my test it didn't).
I don't understand this behavior, and hope that you can come
up with something more reasonable and understandable - just
wanted to warn you to be on the lookout for this weird behavior.
-Original Message-
From: Nicko Cadell [mailto:[EMAIL PROTECTED]
Sent: Friday, January 28, 2005 2:35 PM
To: Log4NET Dev
Subject: RE: Patch to allow large messages to be logged via
ColoredConsoleAppender
It looks like the buffer passed to WriteConsoleW is limited
to 64KB,
i.e. 32000 WCHARs less 1 for the NULL if the P/Invoke marshalling
feels
like putting one on the end of the string.
I will update the appender along the lines of your patch.
Thanks,
Nicko
-Original Message-
From: Wesley Smith [mailto:[EMAIL PROTECTED]
Sent: 27 January 2005 20:03
To: log4net-dev@logging.apache.org
Subject: Patch to allow large messages to be logged via
ColoredConsoleAppender
I found that if you attempt to log a message of more than about
26000 characters, ColoredConsoleAppender will not log the
message
(no error, just silently eats the message). One possible fix
for
the problem is given in the patch below:
---
C:\TEMP\log4net-1.2.0-beta8\src\Appender\ColoredConsoleAppender.cs
Mon Jul 07 01:05:02 2003
+++ C:\Program
Files\log4net\log4net-1.2.0-beta8\src\Appender\ColoredConsoleA
ppender.cs
Wed Jan 26 12:57:22 2005
@@ -280,13 +280,32 @@
string strLoggingMessage =
RenderLoggingEvent(loggingEvent);
- // write the output.
UInt32 uiWritten = 0;
+
+ // WriteConsoleW can only handle a
limited
number of characters at a time
+ // and emperically, 26000 is about
right.
+ const int maxChunkSize = 26000;
+ while (strLoggingMessage.Length
maxChunkSize)
+ {
+ string firstPart =
strLoggingMessage.Substring(0, maxChunkSize);
+ WriteConsoleW( iConsoleHandle,
+ firstPart,
+
(UInt32)firstPart.Length,
+ out (UInt32)uiWritten,
+ IntPtr.Zero);
+
+ // Remove the characters already
output
from strLoggingMessage
+ strLoggingMessage =
strLoggingMessage.Substring(maxChunkSize);
+
+ }
+ // write the remaining output.
+
WriteConsoleW( iConsoleHandle,
strLoggingMessage,
(UInt32)strLoggingMessage.Length,
out
(UInt32)uiWritten,
IntPtr.Zero);
+
}
/// summary
