Why admit it unless your hand is forced? Since when did most
corporations own up to their faults simply to be good netizens? Then
there's always the anti-corporate train of thought that believes it is
possible that they don't mind such issues existing on old software
since it can cause upgrades to occur (see Ipswitch too). Of course
that may be borderline schizo thinking.
Matt
Andy Schmidt wrote:
Hi,
I finally was able to get a confirmation from Microsoft Support yesterday
afternoon (case: SRZ060911001854)
"We are aware the issue you are experiencing. A corresponding bugcheck
request is currently open, and the develop team is working on this issue.
However, the hotfix for this issue is not ready.
0xDF is the data pattern that NTFS returns when it has problem to decompress
the file (eg. the compression fragments are corrupted and can't be
decompressed). Based on my research, the actual raw data on the disk is not
changed, it shows as 0xDF because the system cannot decompress the file and
display the data correctly. So the corrupt is not permanent.
Further more, the issue only occurs on files which containing Hexadecimal
codes."
Apparently, Microsoft decided not to warn people about this problem - no
comment has been added to KF920958 warning people which system
configurations will cause data loss (who cares if it's not permanent if you
can't use your data for a few months).
Best Regards
Andy Schmidt
Phone: +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Heimir
Eidskrem
Sent: Thursday, August 24, 2006 03:21 PM
To: [email protected]
Subject: Re: [Declude.JunkMail] OT: Disk pattern 0xDF in files -> KB920958
may be bad!
Answers below.
Andy Schmidt wrote:
Hi Heimir:
I've been running a number of tests, am in contact with a third
Microsoft customer and some pattern seems to emerge. I also have a
"lead" to a questionable Hotfix, but I'm trying to qualify that first.
Can we first compare your systems to see what's the same (and may be
relevant) and what's different:
A) Disks are defined as "dynamic"
Dynamic
B) Disks are software mirrored using Win2k Disk Administration
no
C) The folders with the "problem" files have the "compression"
attribute set!
yes.
D) Did the problem occur at some point after KB920958 was installed?
yes, I think so.
E) Do the corrupted files have a content of all 0xDF (it looks a
little like an uppercase "B", the German special "s", or like the Beta
character)
Yes
F) Does it appear as if only NEW files are effected?
no, old files as well. BUT I think defrag ran this weekend and that would
have moved some files - if that matters.
G) Does it appear as if only files are effected that are close to a
multiple of 4K?
Yes.
I broke the mirrors on my effected two servers and ran ChkDsk /F. On
one server, ONE disk ChkDsk reported errors (including the files that
I knew were corrupted) - virtually all of them were image file types.
I reran the ChkDsk and it did NOT find errors. I then tried the second
disk of the mirror and it found no errors at all. I then restablished
the mirrors and my client continues to have problems with new files.
On the second server, I broke the mirror, again, the ChcDsk /F
repaired a long list of errors. I did NOT reestablish the mirror and
did not put that disk back in service.
Please contribute to the thread in the Microsoft newsgroup:
http://www.microsoft.com/technet/community/newsgroups/dgbrowser/en-us/
defaul
t.mspx?dg=microsoft.public.win2000.file_system&mid=d826afe9-2ab1-4b2f-
ae11-c
c27702f574a
Best Regards
Andy Schmidt
Phone: +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
Heimir Eidskrem
Sent: Thursday, August 24, 2006 12:29 PM
To: [email protected]
Subject: Re: [Declude.JunkMail] OT: Disk pattern 0xDF in files
Follow up:
During the day I did run chkdks with no switch to check the hard
drive, it reported errors and could not continue. Last night I did run
chkdsk /f on the partition and it did not find any errors this time.
i did process a few thumbnails and they worked fine at 12:30am today.
At 8:00am they still worked but now 11:27 they dont. This was old
photos that I did reprocess again. A couple of new photos that was
uploaded yesterday and processed yesterday is still working fine.
I can't make much sense out of this. Not sure what to next.
I dont think its hardware and I am certain its not our software.
So that leaves OS.
Heimir Eidskrem wrote:
we are having the exact problem on one of our servers.
We create small thumbnail pictures about 4k in size.
They work fine at first but later they are corrupted.
Windows 2000 server.
I have no clue what it could be at this time.
It started around this weekend I think.
Please keep me posted if you find something.
H.
Andy Schmidt wrote:
Hi,
I have two older servers (but not same models or same purchase
years) running Windows 2000 with mirrored disks (software Raid-1).
Two days ago a customer noticed that they uploaded files to their
FTP space, and initially they see the files on the browser - but a
while later the data is corrupted.
I investigated - and oddly enough the problem so far always seems to
appear with small thumbnail graphics files that occupy less than
4095 bytes.
When I
inspect the files I may see the "correct" data through a share, but
if I access the files through some other method, I always see the
byte pattern of 0xDF.
I ran a standalone checkdisk a day ago against the first server,
sure enough, it reported and fixed several problems "Windows
replaced bad clusters in file xxxx". But, the problem recurred the next
day.
Now, my first instinct was that ONE of the two mirrored disks was
truly on its way out and depending on which drive was being used to
read the data it would either get good or bad data.
However, a day later a second customer had the same complaint but on
an entirely different machine. In this case, the error occurs with a
set of relatively new SCSI drives (not even a year old).
So now that I'm looking at two totally different server models, from
entirely different years, one with fairly new disks - what are the
chances that the SAME problem and symptom would show at the same
time. Both on software mirrored disks, in both cases files that are
less than 4 MB large.
Now I'm wondering if this is some "software" issue.
Best Regards
Andy Schmidt
Phone: +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
David Barker
Sent: Wednesday, July 12, 2006 03:53 PM
To: [email protected]
Subject: RE: [Declude.JunkMail] Trying to install Declude 3.1.20
anew
When the decludeproc services start under your windows services and
the first email is processed. A file call diags.txt is created in
your \Declude directory.
This should contain the version and diagnostics. The valid options
on decludeproc from the cmd prompt are:
Decludeproc -v displays the version and build
Decludeproc -i installs the decludeproc service
Decludeproc -u uninstalls the decludeproc service
David B
www.declude.com
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
Andy Schmidt
Sent: Wednesday, July 12, 2006 3:43 PM
To: [email protected]
Subject: RE: [Declude.JunkMail] Trying to install Declude 3.1.20
anew
Dave -
That's what I call catch 22:
D:\IMail>decludeproc -diag
Invalid command line parameter:
-install Install Declude
-diag Print diagnostics
Hm - so let's see, after "-install", I used "-diag" to figure out
what's wrong. But, "-diag" is invalid. The ony valid parameters are...
"-install"
and "-diag"?
Best Regards
Andy Schmidt
Phone: +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206
________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
Andy Schmidt
Sent: Wednesday, July 12, 2006 03:09 PM
To: [email protected]
Subject: RE: [Declude.JunkMail] Trying to install Declude 3.1.20
anew
Hi Dave,
thanks.
Next question:
I noticed that your Virus.CFG is missing two options from Version 2:
AUTOFORGE ON
BANEZIPEXTS ON
If I recall correctly, the idea was that:
BANZIPEXTS OFF
# BANEXT EZIP
BANEZIPEXTS ON
would PERMIT banned extensions inside zipped files (where they could
be scanned), but DENY banned extensions if they were contained
inside encrypted zipped files.
Where those options forgotten in your config file - or are they no
longer available in Version 3?
Best Regards
Andy Schmidt
Phone: +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206
________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
David Barker
Sent: Wednesday, July 12, 2006 02:43 PM
To: [email protected]
Subject: RE: [Declude.JunkMail] Trying to install Declude 3.1.20
anew
The Program Files\Declude is a temp directory that can be deleted
after the install. The original purpose of this directory was to
make available the latest configs as we do not overwrite your
configs. This has since been removed in version 4.x where you will
find a \Declude\Resources directory which has the same purpose.
David B
www.declude.com
________________________________
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
Andy Schmidt
Sent: Wednesday, July 12, 2006 2:36 PM
To: [email protected]
Subject: [Declude.JunkMail] Trying to install Declude 3.1.20 anew
Hi,
I'm trying to set up a server from scratch and thus downloaded and ran:
Declude_IM_N310.exe
and chose the option to let it do its install (rather than the
option for "experienced" admins). PS - that screen has a typo!
The setup created a
C:\Program Files\Declude
folder that contains just the 5 config files it also created the
SAME files
in:
D:\Imail\Declude
together with binaries and the various other Declude files.
I'm at loss!
Which location is the "right" one for the config files (I'm assuming
the D:\Imail\Declude)?
What's the point of creating a "dummy" Folder in the C:\Program
Files\ that contains no programs and that contains files that are
not being used at all (assuming that being the case)?
Should I be deleting this Program Files folder to avoid confusion
when someone else maintains this server?
Come on, the cold war has been over since Reagan - are we still
trying to confuse the Russians?
Best Regards
Andy Schmidt
Phone: +1 201 934-3414 x20 (Business)
Fax: +1 201 934-9206
________________________________
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of Matt
Sent: Tuesday, May 23, 2006 03:25 PM
To: [email protected]
Subject: Re: [Declude.JunkMail] Experience with 4.x
Andrew,
Thanks for your notes and their history.
I'm using the following settings right now:
THREADS 30
WAITFORMAIL 500
WAITFORTHREADS 200
WAITBETWEENTHREADS 100
WINSOCKCLEANUP OFF
INVITEFIX ON
AUTOREVIEW ON
There are a few reasons for trying these values.
THREADS 30 - I'm pretty confident that dual 3.2 Ghz Xeons and RAID
can only handle 30 threads with average messages. In reality, one
single
message can spike the system to 100%, but these are uncommon. I
figure that
if I open this up too wide and I am dealing with a backup or something,
launching more threads when at 100% CPU utilization will actually
slow the
system down. This was the same with 2.x and before. There is added
overhead to managing threads and you don't want that to happen on top of
100% CPU utilization. I am going to back up my server later tonight
to see
if I can't find what the magic number is since I don't want to be
below that
magic number, and it would probably be best to be a little above it.
WAITFORMAIL 500 - On my server, this never kicks in, but if it did,
it wouldn't make sense to delay for too long because I could build up
messages. A half second seems good.
WAITFORTHREADS 200 - This apparently kicks in only when I reach my
thread limit; sort of like a throttle. I don't want it to be too long
because this should only happen when I am hammered, but it is wise
not to
keep hammering when you are at 100%. Sort of a mixed bag choice here.
WAITBETWEENTHREADS 100 - I see this setting as being the biggest
issue with sizing a server. Setting it at 100 ms means that I can only
handle 10 messages per second, and this establishes an upper limit
for what
the server can do. I currently average about 5 messages per second
coming
from my gateways at peak hours, so I figured that to be safe, I should
double that value.
INVITEFIX ON - I have it on because it comes on by default and I
don't know any better. I know nothing about the cause for needing this
outside of brief comments. It seems strange that my Declude setup could
ruin an invitation unless I was using footers. If this is only
triggered by
footer use, I would like to know so that I could turn it off. I would
imagine that this causes extra load to do the check.
AUTOREVIEW ON - I have this on for the same reason that Andrew
pointed out. When I restart Decludeproc, messages land in my review
folder,
and I don't wish to keep manually fishing things out. If there is an
issue
with looping, it would be wise for Declude to make this only trigger say
every 15 minutes instead of more regularly.
Feel free to add to this if you want.
Matt
Colbeck, Andrew wrote:
I'd second that... on both the observed behaviour and the request
for documentation.
I'm attaching my highly commented declude.cfg as a reasonable
sample.
Andrew 8)
________________________________
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of Matt
Sent: Tuesday, May 23, 2006 10:36 AM
To: [email protected]
Subject: Re: [Declude.JunkMail] Experience with 4.x
David,
That did the trick. I can't even see any messages in my
proc folder any more. I might suggest adding your explanation to the
comments in the file just in case others feel the need to turn this
on like
I did. I recalled the issues from the list and I turned it on because I
didn't want the possibility of DNS crapping out and the leakage that
this
would cause.
Here's a screen cap of what my processor graph looks like
now:
Thanks,
Matt
David Barker wrote:
The purpose of WINSOCKCLEANUP ON is to reset
the winsock, what
happens when using this setting is that when the
\proc directory hit 0
decludeproc will finish processing all the messages
in the \work before
checking the \proc again. As WINSOCKCLEANUP is to be
used only by those who
experience DNS issues I would suggest running your
tests again with
WINSOCKCLEANUP commented out and see how the
behavior differs. Also having
the WAITFORMAIL to low can cause the CPU to process
very high as it is
constantly checking the \proc I would suggest a
minimum of 500-1000
David B
www.declude.com
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On
Behalf Of Matt
Sent: Monday, May 22, 2006 8:12 PM
To: [email protected]
Subject: Re: [Declude.JunkMail] Experience with 4.x
Darrell,
I put up two Windows Explorer windows side-by-side
under normal volume and the pattern was consistent where
the proc folder
grows while the work folder shrinks until the work folder
hits zero
at which point the proc folder empties out and everything
lands in work
and then the pattern repeats with proc growing while work
shrinks.
My settings are as follows:
THREADS 50
WAITFORMAIL 100
WAITFORTHREADS 10
WAITBETWEENTHREADS 50
WINSOCKCLEANUP ON
AUTOREVIEW ON
INVITEFIX ON
Matt
Darrell ([EMAIL PROTECTED]) wrote:
It's a faulty design that leaves
more than half a server's CPU capacity unused due
to the mere fact
that they wait for all threads to complete before
moving in a new
batch.
I can't speak to what you see on your
server, but that is not how it is running on my
server. I just double
checked again to make sure I am not crazy, but as I
watch the thread
count on my server (decludeproc) the threads
fluctuate between
7 - 30 ( threads currently set to 50). It is not
uncommon to see the
threads move as follow: 11,8,10,7,15,.... While I
was watching it I
never seen a case where it went down low enough for
the WAITFORMAIL
setting to kick in. Watching the proc/work directory
you can see
files moving in and out, but never really emptying
out. Its possible
what I am seeing is an anomaly or maybe I am
interpreting it wrong.
Maybe David can comment on this.
Darrell
------------------------------------------------------------------------
invURIBL - Intelligent URI filtering plug-in
for Declude, mxGuard, and ORF. Stop spam at the
source the
spamvertised domain. More effective than traditional
RBL's. Try it today -
http://www.invariantsystems.com
---
This E-mail came from the Declude.JunkMail
mailing list. To
unsubscribe, just send an E-mail to
[EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The
archives can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing
list. To
unsubscribe, just send an E-mail to
[EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives
can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing
list. To
unsubscribe, just send an E-mail to
[EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives
can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To unsubscribe,
just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe
Declude.JunkMail". The archives can be found at
http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To unsubscribe,
just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe
Declude.JunkMail". The archives can be found at
http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To unsubscribe,
just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe
Declude.JunkMail". The archives can be found at
http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To unsubscribe,
just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe
Declude.JunkMail". The archives can be found at
http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe,
just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe
Declude.JunkMail". The archives can be found at
http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com. |
