Re[2]: [Declude.JunkMail] Determining a BCC Recipient
you shouldn't proceed under the assumption that government regulators are out there giving IT staff lists of words to be used in full-text search of E-mail archives. That is not the law, and it is not how subpoenas are issued. First: I clearly noted that legal (or compliance, if distinct) is given all documents, including criteria for an archive search, and that IT staff are not responsible for the search. IT is expected to create a system that compliance officers can use independent of IT (in turn respecting employees' privacy from sysadmins' snooping, restricting access to those that perform that role professionally). The full retention media must also be made available, but the regulators will request pruned material. You seem to think that you're really going to hit it off with regulators by coolly giving them hard drives with terabytes of raw mbox data and nothing more. You obviously don't know how it feels to be faced with hundreds of millions of dollars in fines and the knowledge that every day you delay is another day with your company name in the papers as an ongoing investigation. You do not mess around or play tough on producing records; you will only go down harder. The examples are legion. Second: last you wrote, you'd only been involved in an investigation that was not bound to SOX or SEC regulations. I see nothing in your new comments, though they're more verbose, that's any more authoritative. Your isolation of SOX seems deliberately naive, since it is commonplace for SOX's open-ended storage requirements to be allied with SEC 17a-4 requirements to ensure coordination between departments and guarantee prompt response to inquiries without the perception of considered obstruction through negligence. And no organization creates separate SOX-compliant systems and SEC-compliant systems if bound by both. Third: my notes are based on our work with three different clients' IT staffs, their inside and outside counsel (two different outside firms), and documents submitted by regulatory agencies that were specific to the cases; it is also based on the experience of building the original, incomplete archiving systems for these clients and later expansions and revisions of these systems to achieve independently verified SEC/NASD compliance. Fourth: there were no enemy lawyers involved, unless you consider those attempting to prevent criminal actions--in this case, stealing millions from individual investors to benefit secret corporate alliances--to be your enemies. Yet, if those are the enemies in question, I'm surprised you're opposed to _Ipswitch's_ recent activity. Aren't they just following in the footsteps of Enron by concealing their probable dead-end status while soliciting huge monies for nonexistent products? How can a private company's secrecy and price gouging be such an abomination, based on the insults you've used on the IMail list, while here you encourage a public company's destruction of records wherever you perceive a loophole? --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Determining a BCC Recipient
Uh oh. Time to backup up and take a breath. I have not been following this, but have meant to go back and read it because of the implications of the subject. Having gone back and read some of the posts, well, Matt, I like you a lot, but there are some issues. Matt said: Not to debate the applicability of the technology, but you shouldn't proceed under the assumption that government regulators are out there giving IT staff lists of words to be used in full-text search of E-mail archives. That is not the law, and it is not how subpoenas are issued In reality, that is exactly what they can indeed do. No, I have not reviewed the letter of the law, nor will I, nor do I have a desire to. However, I have been briefed on the matter by the in-house IT staff of clients I am involved with that are either subject to SOX or SEC regulations. Matt said: What is at question here is document retention, or more specifically in this case, E-mail retention. There is nothing specific in Sarbanes-Oxley that indicates anything other than destruction of records, thereby implying that records such as E-mail are required to maintained for a period of 5 years. There is absolutely no mention of required technologies, but it is clearly implied that you can't lose access to such documents due to a failure to properly apply a technological solution that survives that length of time (i.e. archival means need to be accessible going 5 years back at any time). While it is true that no mention of what technology is to be used, there are requirements, particularly in SEC regulations, that once a subpoena is presented, you have a time limit to comply and produce the requested information. This time period can be in as little as 4 hours. Obviously, you are going to need technology to provide copies of all e-mail to and from so and so for the last 3 years in 4 hours. Simply having an archive is not enough. You must have the means to search and retrieve quickly. Matt said: There are applications that archive and mine data from E-mail, but IMO, these are really just big-brother types of apps, and I've never been big on invading people's privacy. There are other services that some companies use under the general guise of policy enforcement which is just a fancy way of saying content screening. I think that Sniffer's engine could be set up to do at least part of this work (outside of attachments), but there are large companies out there that already offer such services and this is generally limited to only large customers. I consider this to be an ineffective solution since it can be so easily bypassed with a flash drive on a key chain, or missed by a set of keywords or phrases. Every one is intitled to their opinion. However, truth is the courts have found and upheld that e-mail using company assets are not private, and a company policy must be dictated to enforce such. This means that if a company policy states all e-mail is company property, and no personal e-mail is allowed, or words similar to that effect, the courts have upheld the companies explicit right to search, review, archive and take action on e-mails used within the company. Therefore, there is no question of privacy, as it is company property. Matt, I do not see any personal attack on you by Sandy. What I see is his response to specific things you have said which appear to be incorrect. The various regulations regarding e-mail are convoluted for us to understand at best, and while yes every one is entitled to an opinion, it should not be stated as fact. John Tolmachoff Engineer/Consultant/Owner eServices For You -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Sent: Friday, October 29, 2004 4:46 AM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Let's please try to keep the personal stuff off of this list for the good of everyone. Even though I might find it a tad bit amusing at times when it is directed at me, I don't think that others appreciate seeing it here, and I generally don't. I hesitated even to draft this reply except that I felt it would possibly help in the future seeing as how repeated this pattern has become. This is a support group where people come to share ideas and learn from others, and flame wars have no place in such a forum. One can express an opinion or attempt to establish fact without in effect attacking or belittling a fellow participant, and unlike the circumstance regarding IMail, there is no reason for anyone to become angry about things so insignificant. I don't claim to be perfect in this regard myself, but I think it needed to be said. Matt Sanford Whiteman wrote: you shouldn't proceed under the assumption that governmentregulators are out there giving IT staff lists of words to be usedin full-text search of E-mail archives. That is not the law, andit is not how
Re[2]: [Declude.JunkMail] Determining a BCC Recipient
Each company is different and therefore so are their needs. Okay, but _Rick's_ needs are SOX compliance. I don't have any interest in discussing general archiving methods; to each his/her own in that effort. Many that archive will never need to go through the data, primarily because many companies aren't so enormous that they have the legal liability nor the volume that would necessitate a preemptive indexing of content. I do not believe you are speaking from experience. I would consider it to be unrealistic to demand that a full text indexing be done of file attachments. . . That's nice, but it's not your choice. Regulators demand prompt, often overnight, responses to search requests (sometimes many concurrent requests). Do not think for a minute that unrealistic is the IT staff's trump card against compliance. This is not as simple as you think it is, but that's understable because you've evidently never been under the gun of a SOX or SEC investigation. Three of our clients have been through the latter (cleared, I might add, but now with double the liability insurance premium, probably forever). Two of these companies have less than 50 employees, and one has four full-timers. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Determining a BCC Recipient
Show me a search of a full text index that can positively give you 100% of the hits on a given topic and I'll let you have this one :) Manual review is necessary to verify, and chances are you would need to manually review every E-mail going to and from specific employees across a range of dates. A good law firm would do the review themselves before passing on the material to the regulators instead of relying on some tech to identify the subject matter by way of keyword. And no, I've never dealt with SOX compliance, but I was involved in a case where I had to produce over 700 E-mails between myself and employees of another company. That wasn't fun. It was easy to identify the messages, but very time consuming to do the review. Matt Sanford Whiteman wrote: Each company is different and therefore so are their needs. Okay, but _Rick's_ needs are SOX compliance. I don't have any interest in discussing general archiving methods; to each his/her own in that effort. Many that archive will never need to go through the data, primarily because many companies aren't so enormous that they have the legal liability nor the volume that would necessitate a preemptive indexing of content. I do not believe you are speaking from experience. I would consider it to be unrealistic to demand that a full text indexing be done of file attachments. . . That's nice, but it's not your choice. Regulators demand prompt, often overnight, responses to search requests (sometimes many concurrent requests). Do not think for a minute that "unrealistic" is the IT staff's trump card against compliance. This is not as simple as you think it is, but that's understable because you've evidently never been under the gun of a SOX or SEC investigation. Three of our clients have been through the latter (cleared, I might add, but now with double the liability insurance premium, probably forever). Two of these companies have less than 50 employees, and one has four full-timers. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
Re[2]: [Declude.JunkMail] Determining a BCC Recipient
Show me a search of a full text index that can positively give you 100% of the hits on a given topic and I'll let you have this one :) The regulators will typically give you a list of search terms to be used in a full-text search. Their specifications are what guide the accuracy of the search. Of course, deliberate and deep obfuscation of all nouns and verbs will elude the search. But you _must_ search all communications, including message bodies and attachments. This is the law. You can debate the constitutionality of the law or what-have-you, but the realities of an investigation are that all communications must be searched, and in any volume and with the deadlines one is always under, that mandates full-text indexing. Manual review is necessary to verify, and chances are you would need to manually review every E-mail going to and from specific employees across a range of dates. Wrong. The initial request is a list of search terms run through a compliant archiving system. The search results are vetted by counsel and submitted to the regulator. Pruned results may accompany the full results of the search, but the computer-generated results are the first line of compliance. At the regulator's discretion, manual review of all emails to detect anomalies, obfuscation, et al. might then be the next step. A good law firm would do the review themselves before passing on the material to the regulators instead of relying on some tech to identify the subject matter by way of keyword. The keyword search is part of the regulatory framework for electronic communications. Part of being compliant is ensuring that a search must be able to conducted by independent auditors _or the regulators themselves_ at any time. In a proper setup, a tech does not need to be involved in the actual search. I was involved in a case where I had to produce over 700 E-mails between myself and employees of another company. That wasn't fun. It was easy to identify the messages, but very time consuming to do the review. Yes, it is time-consuming. On that we agree. And shirking statutory obligations that in fact shorten the time to settlement/dismissal, and in turn bringing additional scrutiny, is not a wise tactic. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re[3]: [Declude.JunkMail] Determining a BCC Recipient
On Thursday, October 28, 2004, 4:08:30 PM, Sanford wrote: Show me a search of a full text index that can positively give you 100% of the hits on a given topic and I'll let you have this one :) SW The regulators will typically give you a list of search terms to be SW used in a full-text search. Their specifications are what guide the SW accuracy of the search. Of course, deliberate and deep obfuscation of SW all nouns and verbs will elude the search. But you _must_ search all SW communications, including message bodies and attachments. This is the SW law. You can debate the constitutionality of the law or what-have-you, SW but the realities of an investigation are that all communications must SW be searched, and in any volume and with the deadlines one is always SW under, that mandates full-text indexing. snip/ SW Yes, it is time-consuming. On that we agree. And shirking statutory SW obligations that in fact shorten the time to settlement/dismissal, and SW in turn bringing additional scrutiny, is not a wise tactic. All of this makes me wonder if our pattern matching engine and a simple archive of messages might be a useful product in this case. Picture if you will an MTA with Message Sniffer installed where an archive is generated automatically using a compressed format. Perhaps one file per day. If such a request were to come in then a rulebase would be generated to match the search phrases and then the sniffer engine would scan through the messages in the archives and deliver those that matched to a specified email address along with a list of the patterns that were found... that is, the matching message as an attachment to a report message that describes the original envelope and the list of matching patterns. Might this be a useful product do you think? The thing is - this is probably a fairly simple augmentation to the hold/release mechanism I'm working on for Message Sniffer - so I'm curious to know if the enhanced capability is worth the trip. My gut tells me it is. _M --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: Re[3]: [Declude.JunkMail] Determining a BCC Recipient
All of this makes me wonder if our pattern matching engine and a simple archive of messages might be a useful product in this case. Picture if you will an MTA with Message Sniffer installed where an archive is generated automatically using a compressed format. Perhaps one file per day. If such a request were to come in then a rulebase would be generated to match the search phrases and then the sniffer engine would scan through the messages in the archives and deliver those that matched to a specified email address along with a list of the patterns that were found... that is, the matching message as an attachment to a report message that describes the original envelope and the list of matching patterns. Might this be a useful product do you think? The thing is - this is probably a fairly simple augmentation to the hold/release mechanism I'm working on for Message Sniffer - so I'm curious to know if the enhanced capability is worth the trip. My gut tells me it is. _M Hi Pete, I think your gut is right. I'm pretty sure that I have 2 clients that would be quite interested in SOXsniffer. g ~Patrick --- [This E-mail scanned for viruses at HGBD by Declude/McAfee] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re[4]: [Declude.JunkMail] Determining a BCC Recipient
Picture if you will an MTA with Message Sniffer installed where an archive is generated automatically using a compressed format. The compression part is one thing I'm not too clear on, and. . . that is, the matching message as an attachment to a report message that describes the original envelope and the list of matching patterns. . . . I don't think e-mail is the right transport, since the results can run in the gigabytes, but. . . Might this be a useful product do you think? . . . yes! --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re[5]: [Declude.JunkMail] Determining a BCC Recipient
On Thursday, October 28, 2004, 7:55:09 PM, Sanford wrote: Picture if you will an MTA with Message Sniffer installed where an archive is generated automatically using a compressed format. SW The compression part is one thing I'm not too clear on, and. . . that is, the matching message as an attachment to a report message that describes the original envelope and the list of matching patterns. SW . . . I don't think e-mail is the right transport, since the results SW can run in the gigabytes, but. . . Might this be a useful product do you think? SW . . . yes! Interresting - let me clarify. I am working on a mechanism that would store held spam in a single file with a companion index file. One of these per day. Each message is packaged with it's envelope (in IMail D + Q file), compressed with gzip (or equiv) and appended to the file with an index to the ID. This will allow Message Sniffer to retrieve and deliver the message whenever it is called for. Since each message is compressed before storage and there is only one file per day the archival task for holding spam won't overwhelm most file systems. Step back for a moment and consider flipping a switch so that all messages are archived and marked. Under normal conditions held spam would be retrieved and sent to the requesting user. Under SOX conditions an alternate utility is used that searches for the appropriate keywords at sniffer speeds through all appropriate archives in the library. When a match is found, the message is attached to a report message and delivered to the requested mailbox. Email going to email boxes, one message per message so that they can be reviewed in the same way email might normally be reviewed (threading, chronology, From/To, etc -- all the stuff usually found in email clients). Of course, the messages could just as easily be decompressed and stored in an XML file suitable for import to a database of choice - but I digress. Point is, an existing mechanism can be extended to satisfy multiple needs. For example, SOX is not the only reason a company might want to dig into it's email archive --- there are lots of reasons both good and bad. I like the idea. I will extend the spec to accommodate hooks for the new features. _M --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: Re[5]: [Declude.JunkMail] Determining a BCC Recipient
Point is, an existing mechanism can be extended to satisfy multiple needs. For example, SOX is not the only reason a company might want to dig into it's email archive --- there are lots of reasons both good and bad. I like the idea. I will extend the spec to accommodate hooks for the new features. _M Excellent... ~Patrick --- [This E-mail scanned for viruses at HGBD by Declude/McAfee] --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Determining a BCC Recipient
Patrick Childers wrote: Hi Pete, I think your gut is right. I'm pretty sure that I have 2 clients that would be quite interested in SOXsniffer. g Not to debate the applicability of the technology, but you shouldn't proceed under the assumption that government regulators are out there giving IT staff lists of words to be used in full-text search of E-mail archives. That is not the law, and it is not how subpoenas are issued. What is at question here is document retention, or more specifically in this case, E-mail retention. There is nothing specific in Sarbanes-Oxley that indicates anything other than destruction of records, thereby implying that records such as E-mail are required to maintained for a period of 5 years. There is absolutely no mention of required technologies, but it is clearly implied that you can't lose access to such documents due to a failure to properly apply a technological solution that survives that length of time (i.e. archival means need to be accessible going 5 years back at any time). Given these requirements of such companies, it is now much more common to subpoena E-mail records for matters outside of that covered by the SEC, which is what Sarbanes-Oxley is all about. Subpoenas must be specific enough to comply with rather than having the implementation of an archive being capable of allowing enemy lawyers to mine your data for hits. If you use such technology however, you actually open up yourself for a higher likelihood and therefore liability of being required to do so based on preexisting capability. As far as I can tell, the only retrieval requirement is being able to identify messages by sender, recipients and date. If you wish to read the requirements as established under the law, it is Sarbanes-Oxley section 802. It may also be wise to establish a separate and shorter retention schedule for employees that are not involved in matters governed by the SEC. When I worked for a Fortune 500 company, they had a 1 year E-mail deletion policy purposefully to limit liability related to product liability lawsuits and the cost of compliance. This was pre-SOX, and there was no policy that retained deleted E-mail outside of backup tapes which were carefully controlled as well. There was nothing wrong with such policies so long as documents were not destroyed in response to an investigation or lawsuit (which is why Sarbanes-Oxley specifies retention of documents for audits which require 5 years of records). How one implements this is up to the company as long as they meet the basic requirements of the law. I certainly don't expect anyone to trust me on any of this, but I wouldn't go about designing a repository of any sort without consulting a lawyer with regulatory experience, and the same goes for developing products for use by such companies. There are applications that archive and mine data from E-mail, but IMO, these are really just big-brother types of apps, and I've never been big on invading people's privacy. There are other services that some companies use under the general guise of policy enforcement which is just a fancy way of saying content screening. I think that Sniffer's engine could be set up to do at least part of this work (outside of attachments), but there are large companies out there that already offer such services and this is generally limited to only large customers. I consider this to be an ineffective solution since it can be so easily bypassed with a flash drive on a key chain, or missed by a set of keywords or phrases. Matt -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re[2]: [Declude.JunkMail] Determining a BCC Recipient
On Thursday, October 28, 2004, 10:44:32 PM, Matt wrote: M Patrick Childers wrote: Hi Pete, I think your gut is right. I'm pretty sure that I have 2 clients that would be quite interested in SOXsniffer. g M Not to debate the applicability of the technology, but you shouldn't M proceed under the assumption that government regulators are out there M giving IT staff lists of words to be used in full-text search of M E-mail archives. That is not the law, and it is not how subpoenas are M issued. snip/ All really appreciated Matt. I think the point is that the basic requirements can easily be met, and the search capability, which can be very useful in mundane and even positive circumstances, can be provided without a significant additional effort. So, for a very low cost, those who might not otherwise be able to afford the high-end systems you allude to can have the core of a fairly robust capability. I'm sure that core capability can and will be extended as needed if I do the job right. No assumptions here about marketability or suitability - only a raw capability that has a high potential for a low cost... and, based on my own experiences, having this kind of thing in your back pocket can be very powerful. I can recall times when a mechanism like this would not only have saved me days - even weeks of work, but also would have provided a significant competitive advantage. Consider auditing an engineering (or any large) project near completion or after initial deployment. The ability to extract all correspondence on the project in an inexpensive and orderly fashion is mind-bendingly powerful. -- Dump the results into a searchable mail archive system and you have a searchable, threaded reference that you didn't know you would need until now. Or... when the boss comes down and says: I need you to tell me _exactly_ what happened here... in that uncomfortable way that only pointy-haired fellows can really achieve... Been there, done that, got the t-shirt and the bumper sticker. It just makes you shiver. (Where would we be without Dilbert?) Anyway - I recognize your point about setting an appropriate policy. I just make hammers... I'll let other folks drive the nails where they are needed ;-) This is now decidedly off topic for Declude. Sorry for the extra bandwidth. Best all, _M --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Determining a BCC Recipient
That's going to be one massive database :) I've become quite the VBScripter as of late (if that's something to brag about), so let me know if you need any help. Matt Rick Davidson wrote: Thanks Matt, COPYFILE is working perfectly, now its just a matter of writing the program to parse and insert it into the SQL database. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, October 26, 2004 5:15 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Rick, This information is in the Q* file. If you use the COPYFILE action, it will keep both the D* and the Q* file. The only issue is that the Declude headers are lost and each message is kept separately and not viewable without a special application like spamreview. IMO, this is appropriate for archiving due to legal requirement, but not for doing review. If you want to handle this in a different way by just sending to a mailbox, you can use a WARN action with the %ALLRECIPS% variable which will contain the BCC addresses as well. For instance, you could do the following: TESTNAMEWARN X-RECIPIENTS: %ALLRECIPS% This of course exposes the BCC info to all that might view the headers. Matt Rick Davidson wrote: I am looking at creating our own email archiving solution using sql, the main hurdle is how to handle and email sent to a user using BCC. Is there a way to use Declude to include that info in a recipient x-header? If I send myself using only the BCC field the header contains only this From: Rick Davidson [EMAIL PROTECTED] To: Undisclosed-Recipient:; Subject: test I assume the BCC info is lost once the message hits the senders SMTP server correct? Rick Davidson National Systems Manager North American Title Group - --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Determining a BCC Recipient
Rick, I am looking at creating our own email archiving solution using sql This, as Matt notes, could be monstrous. It certainly is not best-practice to store this many CLOBs (or BLOBs, if you're decoding MIME) in a generic DB. That's why the only RDBMS message stores worth their salt are Exchange, Notes (sort of), and the archiving vendors' back ends, as they are purpose-built on both client and server ends. If you do go the RDBMS route, you should definitely consider auto-splitting by date into separate tables and/or separate databases to enable scaling out. However, I'd suggest instead that you use a well-known format such as MBOX and an MBOX-aware, high-capacity indexing/search product like dtSearch. We've used dtSearch Web as a message archive-and-search mechanism and have been very happy with the speed (though, admittedly, the display needs a lot of tweaks). --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Determining a BCC Recipient
ok thanks Matt, we do have some programmers on staff here but I will sure conscript your help if we brick wall. Regardless of where it is stored its going to be a massive amount of data, my initial samplings show 1.5 to 2GB per day. Yikes! You wouldnt happen to know how to parse mime types and remove attachments would you? :-) Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 2:58 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's going to be one massive database :) I've become quite the VBScripter as of late (if that's something to brag about), so let me know if you need any help. Matt Rick Davidson wrote: Thanks Matt, COPYFILE is working perfectly, now its just a matter of writing the program to parse and insert it into the SQL database. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, October 26, 2004 5:15 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Rick, This information is in the Q* file. If you use the COPYFILE action, it will keep both the D* and the Q* file. The only issue is that the Declude headers are lost and each message is kept separately and not viewable without a special application like spamreview. IMO, this is appropriate for archiving due to legal requirement, but not for doing review. If you want to handle this in a different way by just sending to a mailbox, you can use a WARN action with the %ALLRECIPS% variable which will contain the BCC addresses as well. For instance, you could do the following: TESTNAMEWARN X-RECIPIENTS: %ALLRECIPS% This of course exposes the BCC info to all that might view the headers. Matt Rick Davidson wrote: I am looking at creating our own email archiving solution using sql, the main hurdle is how to handle and email sent to a user using BCC. Is there a way to use Declude to include that info in a recipient x-header? If I send myself using only the BCC field the header contains only this From: Rick Davidson [EMAIL PROTECTED] To: Undisclosed-Recipient:; Subject: test I assume the BCC info is lost once the message hits the senders SMTP server correct? Rick Davidson National Systems Manager North American Title Group - --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Determining a BCC Recipient
That's funny that you should ask. I just coded that one up in VBScript this last weekend. I even managed to decode base64 text attachments, remove quoted-printable encoding, and strip out all of the HTML code. If this is for archiving according to legal requirement, the attachments would probably be necessary however. Sandy had some good recommendations on how to archive. Maybe if you shared your requirements with the list, someone would have some recommendations as to how to approach this a better way. Matt Rick Davidson wrote: ok thanks Matt, we do have some programmers on staff here but I will sure conscript your help if we brick wall. Regardless of where it is stored its going to be a massive amount of data, my initial samplings show 1.5 to 2GB per day. Yikes! You wouldnt happen to know how to parse mime types and remove attachments would you? :-) Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 2:58 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's going to be one massive database :) I've become quite the VBScripter as of late (if that's something to brag about), so let me know if you need any help. Matt Rick Davidson wrote: Thanks Matt, COPYFILE is working perfectly, now its just a matter of writing the program to parse and insert it into the SQL database. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, October 26, 2004 5:15 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Rick, This information is in the Q* file. If you use the COPYFILE action, it will keep both the D* and the Q* file. The only issue is that the Declude headers are lost and each message is kept separately and not viewable without a special application like spamreview. IMO, this is appropriate for archiving due to legal requirement, but not for doing review. If you want to handle this in a different way by just sending to a mailbox, you can use a WARN action with the %ALLRECIPS% variable which will contain the BCC addresses as well. For instance, you could do the following: TESTNAMEWARN X-RECIPIENTS: %ALLRECIPS% This of course exposes the BCC info to all that might view the headers. Matt Rick Davidson wrote: I am looking at creating our own email archiving solution using sql, the main hurdle is how to handle and email sent to a user using BCC. Is there a way to use Declude to include that info in a recipient x-header? If I send myself using only the BCC field the header contains only this From: Rick Davidson [EMAIL PROTECTED] To: Undisclosed-Recipient:; Subject: test I assume the BCC info is lost once the message hits the senders SMTP server correct? Rick Davidson National Systems Manager North American Title Group - --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro
Re: [Declude.JunkMail] Determining a BCC Recipient
Thanks Sandy, I will look into those, the boss wants me to do this on the cheap, the sql idea was first so we could at least say we were archiving the email. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Sanford Whiteman [EMAIL PROTECTED] To: Rick Davidson [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 3:21 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Rick, I am looking at creating our own email archiving solution using sql This, as Matt notes, could be monstrous. It certainly is not best-practice to store this many CLOBs (or BLOBs, if you're decoding MIME) in a generic DB. That's why the only RDBMS message stores worth their salt are Exchange, Notes (sort of), and the archiving vendors' back ends, as they are purpose-built on both client and server ends. If you do go the RDBMS route, you should definitely consider auto-splitting by date into separate tables and/or separate databases to enable scaling out. However, I'd suggest instead that you use a well-known format such as MBOX and an MBOX-aware, high-capacity indexing/search product like dtSearch. We've used dtSearch Web as a message archive-and-search mechanism and have been very happy with the speed (though, admittedly, the display needs a lot of tweaks). --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Determining a BCC Recipient
Essentially the good folks at Enron and WorldComm brought us the Sarbanes-Oxley Act or SOX for short. Public companies have to keep a record of all communications, the details of this are vague but mostly apply to the money people and decision makers. Since we cant selectively catch that specific traffic we have to grab it all. Basicly all mail must be archived including the attachments and all mail must be retrievable in a reasonable amount of time, thats about it. We were considering stripping the attachments and storing them in a directory structure and storing the email text data in the sql database. Separate fields for the date, to, from, subject, the entire D file and the attachment names and their location. We figure we can get decent compression and searchabiltiy with the text info but the biggest hurdle is the attachments and being a Title company we have alot of large attachments to deal with. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 3:53 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's funny that you should ask. I just coded that one up in VBScript this last weekend. I even managed to decode base64 text attachments, remove quoted-printable encoding, and strip out all of the HTML code. If this is for archiving according to legal requirement, the attachments would probably be necessary however. Sandy had some good recommendations on how to archive. Maybe if you shared your requirements with the list, someone would have some recommendations as to how to approach this a better way. Matt Rick Davidson wrote: ok thanks Matt, we do have some programmers on staff here but I will sure conscript your help if we brick wall. Regardless of where it is stored its going to be a massive amount of data, my initial samplings show 1.5 to 2GB per day. Yikes! You wouldnt happen to know how to parse mime types and remove attachments would you? :-) Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 2:58 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's going to be one massive database :) I've become quite the VBScripter as of late (if that's something to brag about), so let me know if you need any help. Matt Rick Davidson wrote: Thanks Matt, COPYFILE is working perfectly, now its just a matter of writing the program to parse and insert it into the SQL database. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, October 26, 2004 5:15 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Rick, This information is in the Q* file. If you use the COPYFILE action, it will keep both the D* and the Q* file. The only issue is that the Declude headers are lost and each message is kept separately and not viewable without a special application like spamreview. IMO, this is appropriate for archiving due to legal requirement, but not for doing review. If you want to handle this in a different way by just sending to a mailbox, you can use a WARN action with the %ALLRECIPS% variable which will contain the BCC addresses as well. For instance, you could do the following: TESTNAMEWARN X-RECIPIENTS: %ALLRECIPS% This of course exposes the BCC info to all that might view the headers. Matt Rick Davidson wrote: I am looking at creating our own email archiving solution using sql, the main hurdle is how to handle and email sent to a user using BCC. Is there a way to use Declude to include that info in a recipient x-header? If I send myself using only the BCC field the header contains only this From: Rick Davidson [EMAIL PROTECTED] To: Undisclosed-Recipient:; Subject: test I assume the BCC info is lost once the message hits the senders SMTP server correct? Rick Davidson National Systems Manager North American Title Group - --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned
Re[2]: [Declude.JunkMail] Determining a BCC Recipient
I will look into those, the boss wants me to do this on the cheap, the sql idea was first so we could at least say we were archiving the email. If you just want archiving for independent audit and to show good faith, concatenate the Q and D into an envelope-preserving MBOX for each day. However, you have to plan for a real investigation, and retrievability and simple envelope and body searching requirements will not be met on the cheap--since maintaining terabyte databases with _any_ data isn't cheap. Full-text indexing of such dbs also not a small project no matter what the driver. FTR, dtSearch web costs, I believe, 1000 bucks ( + server + storage + labor ). --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: Re[2]: [Declude.JunkMail] Determining a BCC Recipient
I have an application that acts as a POP3 mail client and writes the message body (with basic header info) to disk as a .txt file. I drop them into a folder hierarchy based on the date, etc which Microsoft Index server indexes (free w/ Windows). Just look for the message via a query based web page... Not sure if that helps but that's what we did for archiving/searching. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Sanford Whiteman Sent: Wednesday, October 27, 2004 4:47 PM To: Rick Davidson Subject: Re[2]: [Declude.JunkMail] Determining a BCC Recipient I will look into those, the boss wants me to do this on the cheap, the sql idea was first so we could at least say we were archiving the email. If you just want archiving for independent audit and to show good faith, concatenate the Q and D into an envelope-preserving MBOX for each day. However, you have to plan for a real investigation, and retrievability and simple envelope and body searching requirements will not be met on the cheap--since maintaining terabyte databases with _any_ data isn't cheap. Full-text indexing of such dbs also not a small project no matter what the driver. FTR, dtSearch web costs, I believe, 1000 bucks ( + server + storage + labor ). --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/do wnload/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2a liases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2alias es/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Determining a BCC Recipient
Mabry Internet/X Controls has a very good Mime processing controls for easy reading of uue files. - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 3:53 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's funny that you should ask. I just coded that one up in VBScript this last weekend. I even managed to decode base64 text attachments, remove quoted-printable encoding, and strip out all of the HTML code. If this is for archiving according to legal requirement, the attachments would probably be necessary however. Sandy had some good recommendations on how to archive. Maybe if you shared your requirements with the list, someone would have some recommendations as to how to approach this a better way. Matt Rick Davidson wrote: ok thanks Matt, we do have some programmers on staff here but I will sure conscript your help if we brick wall. Regardless of where it is stored its going to be a massive amount of data, my initial samplings show 1.5 to 2GB per day. Yikes! You wouldnt happen to know how to parse mime types and remove attachments would you? :-) Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 2:58 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's going to be one massive database :) I've become quite the VBScripter as of late (if that's something to brag about), so let me know if you need any help. Matt Rick Davidson wrote: Thanks Matt, COPYFILE is working perfectly, now its just a matter of writing the program to parse and insert it into the SQL database. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, October 26, 2004 5:15 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Rick, This information is in the Q* file. If you use the COPYFILE action, it will keep both the D* and the Q* file. The only issue is that the Declude headers are lost and each message is kept separately and not viewable without a special application like spamreview. IMO, this is appropriate for archiving due to legal requirement, but not for doing review. If you want to handle this in a different way by just sending to a mailbox, you can use a WARN action with the %ALLRECIPS% variable which will contain the BCC addresses as well. For instance, you could do the following: TESTNAMEWARN X-RECIPIENTS: %ALLRECIPS% This of course exposes the BCC info to all that might view the headers. Matt Rick Davidson wrote: I am looking at creating our own email archiving solution using sql, the main hurdle is how to handle and email sent to a user using BCC. Is there a way to use Declude to include that info in a recipient x-header? If I send myself using only the BCC field the header contains only this From: Rick Davidson [EMAIL PROTECTED] To: Undisclosed-Recipient:; Subject: test I assume the BCC info is lost once the message hits the senders SMTP server correct? Rick Davidson National Systems Manager North American Title Group - --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from
Re: [Declude.JunkMail] Determining a BCC Recipient
If it were me I would just use the CATCHALLMAILS feature of Declude and COPY them to an archival e-mail address and then just burn the inbox of that address to disk once a month. - Original Message - From: Rick Davidson [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 4:29 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Essentially the good folks at Enron and WorldComm brought us the Sarbanes-Oxley Act or SOX for short. Public companies have to keep a record of all communications, the details of this are vague but mostly apply to the money people and decision makers. Since we cant selectively catch that specific traffic we have to grab it all. Basicly all mail must be archived including the attachments and all mail must be retrievable in a reasonable amount of time, thats about it. We were considering stripping the attachments and storing them in a directory structure and storing the email text data in the sql database. Separate fields for the date, to, from, subject, the entire D file and the attachment names and their location. We figure we can get decent compression and searchabiltiy with the text info but the biggest hurdle is the attachments and being a Title company we have alot of large attachments to deal with. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 3:53 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's funny that you should ask. I just coded that one up in VBScript this last weekend. I even managed to decode base64 text attachments, remove quoted-printable encoding, and strip out all of the HTML code. If this is for archiving according to legal requirement, the attachments would probably be necessary however. Sandy had some good recommendations on how to archive. Maybe if you shared your requirements with the list, someone would have some recommendations as to how to approach this a better way. Matt Rick Davidson wrote: ok thanks Matt, we do have some programmers on staff here but I will sure conscript your help if we brick wall. Regardless of where it is stored its going to be a massive amount of data, my initial samplings show 1.5 to 2GB per day. Yikes! You wouldnt happen to know how to parse mime types and remove attachments would you? :-) Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 2:58 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's going to be one massive database :) I've become quite the VBScripter as of late (if that's something to brag about), so let me know if you need any help. Matt Rick Davidson wrote: Thanks Matt, COPYFILE is working perfectly, now its just a matter of writing the program to parse and insert it into the SQL database. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, October 26, 2004 5:15 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Rick, This information is in the Q* file. If you use the COPYFILE action, it will keep both the D* and the Q* file. The only issue is that the Declude headers are lost and each message is kept separately and not viewable without a special application like spamreview. IMO, this is appropriate for archiving due to legal requirement, but not for doing review. If you want to handle this in a different way by just sending to a mailbox, you can use a WARN action with the %ALLRECIPS% variable which will contain the BCC addresses as well. For instance, you could do the following: TESTNAMEWARN X-RECIPIENTS: %ALLRECIPS% This of course exposes the BCC info to all that might view the headers. Matt Rick Davidson wrote: I am looking at creating our own email archiving solution using sql, the main hurdle is how to handle and email sent to a user using BCC. Is there a way to use Declude to include that info in a recipient x-header? If I send myself using only the BCC field the header contains only this From: Rick Davidson [EMAIL PROTECTED] To: Undisclosed-Recipient:; Subject: test I assume the BCC info is lost once the message hits the senders SMTP server correct? Rick Davidson National Systems Manager North American Title Group - --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found
Re[2]: [Declude.JunkMail] Determining a BCC Recipient
If it were me I would just use the CATCHALLMAILS feature of Declude and COPY them to an archival e-mail address and then just burn the inbox of that address to disk once a month. For low-volume and unregulated businesses, perhaps, but this will not accomplish compliance, since: - it does not preserve envelope routing information - at 1.5 GB per day, you could not actually read the monthly MBXs using a standard client, even if IMail and the filesystem allowed you to create them - it does not allow for keyword search and export over the volume of data in question - the monthly backup is too infrequent Remember, this is a question of regulations, not internal policies. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Determining a BCC Recipient
Why not just turn the Archive feature of IMAIL SMTP on and route that way? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dan Geiser Sent: Wednesday, October 27, 2004 5:20 PM To: [EMAIL PROTECTED] Subject: Re: [Declude.JunkMail] Determining a BCC Recipient If it were me I would just use the CATCHALLMAILS feature of Declude and COPY them to an archival e-mail address and then just burn the inbox of that address to disk once a month. - Original Message - From: Rick Davidson [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 4:29 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Essentially the good folks at Enron and WorldComm brought us the Sarbanes-Oxley Act or SOX for short. Public companies have to keep a record of all communications, the details of this are vague but mostly apply to the money people and decision makers. Since we cant selectively catch that specific traffic we have to grab it all. Basicly all mail must be archived including the attachments and all mail must be retrievable in a reasonable amount of time, thats about it. We were considering stripping the attachments and storing them in a directory structure and storing the email text data in the sql database. Separate fields for the date, to, from, subject, the entire D file and the attachment names and their location. We figure we can get decent compression and searchabiltiy with the text info but the biggest hurdle is the attachments and being a Title company we have alot of large attachments to deal with. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 3:53 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's funny that you should ask. I just coded that one up in VBScript this last weekend. I even managed to decode base64 text attachments, remove quoted-printable encoding, and strip out all of the HTML code. If this is for archiving according to legal requirement, the attachments would probably be necessary however. Sandy had some good recommendations on how to archive. Maybe if you shared your requirements with the list, someone would have some recommendations as to how to approach this a better way. Matt Rick Davidson wrote: ok thanks Matt, we do have some programmers on staff here but I will sure conscript your help if we brick wall. Regardless of where it is stored its going to be a massive amount of data, my initial samplings show 1.5 to 2GB per day. Yikes! You wouldnt happen to know how to parse mime types and remove attachments would you? :-) Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 2:58 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's going to be one massive database :) I've become quite the VBScripter as of late (if that's something to brag about), so let me know if you need any help. Matt Rick Davidson wrote: Thanks Matt, COPYFILE is working perfectly, now its just a matter of writing the program to parse and insert it into the SQL database. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, October 26, 2004 5:15 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Rick, This information is in the Q* file. If you use the COPYFILE action, it will keep both the D* and the Q* file. The only issue is that the Declude headers are lost and each message is kept separately and not viewable without a special application like spamreview. IMO, this is appropriate for archiving due to legal requirement, but not for doing review. If you want to handle this in a different way by just sending to a mailbox, you can use a WARN action with the %ALLRECIPS% variable which will contain the BCC addresses as well. For instance, you could do the following: TESTNAMEWARN X-RECIPIENTS: %ALLRECIPS% This of course exposes the BCC info to all that might view the headers. Matt Rick Davidson wrote: I am looking at creating our own email archiving solution using sql, the main hurdle is how to handle and email sent to a user using BCC. Is there a way to use Declude to include that info in a recipient x-header? If I send myself using only the BCC field the header contains only this From: Rick Davidson [EMAIL PROTECTED] To: Undisclosed-Recipient
Re: Re[2]: [Declude.JunkMail] Determining a BCC Recipient
OK, fine then. Don't do it every month. Pick the archival frequency of your choosing. And can't you use Declude to insert the routing information into the headers? And can't you download the e-mail from the inbox into the mail client of your choosing and archive it that way? Anyway, as usual someone's off on an unintended tangent here. All I'm saying is that if I worked for a company I would come up with a more elegant solution to mail archiving then being dependent on SQL Server or any other proprietary format. Plain old text files are just fine by me. - Original Message - From: Sanford Whiteman [EMAIL PROTECTED] To: Dan Geiser [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 5:33 PM Subject: Re[2]: [Declude.JunkMail] Determining a BCC Recipient If it were me I would just use the CATCHALLMAILS feature of Declude and COPY them to an archival e-mail address and then just burn the inbox of that address to disk once a month. For low-volume and unregulated businesses, perhaps, but this will not accomplish compliance, since: - it does not preserve envelope routing information - at 1.5 GB per day, you could not actually read the monthly MBXs using a standard client, even if IMail and the filesystem allowed you to create them - it does not allow for keyword search and export over the volume of data in question - the monthly backup is too infrequent Remember, this is a question of regulations, not internal policies. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- Sign up for virus-free and spam-free e-mail with Nexus Technology Group http://www.nexustechgroup.com/mailscan --- Sign up for virus-free and spam-free e-mail with Nexus Technology Group http://www.nexustechgroup.com/mailscan --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re[4]: [Declude.JunkMail] Determining a BCC Recipient
And can't you use Declude to insert the routing information into the headers? Not without compromising BCC recipients, which is unacceptable. And can't you download the e-mail from the inbox into the mail client of your choosing and archive it that way? Possibly, but that's an ingredient you're not mentioning. A client could be anything from IMail client (bad) to a full-text indexing web-based MUA with giant capacity (good). It hinges on this component, which is open to question. I'm saying is that if I worked for a company I would come up with a more elegant solution to mail archiving then being dependent on SQL Server or any other proprietary format. Well, as I suggested, daily MBOX files--always recoverable in raw form--with a proprietary dtSearch full-text index is robust and RDBMS-free. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Determining a BCC Recipient
I strongly recommend that you just simply keep these in their Q* and D* formats and zip up the directories every night and write them to a CD or something every so often. Retrieval of such E-mail should be rare if ever necessary, and you can easily write something that would unzip the files, search for addresses in the Q* files, and copy the needed files to a directory when needed. You can also of course script the zipping. Personally, I like the WinZip command line add-on as it seems to compress better than freeware components, and WinZip only costs $20. You can zip an entire directory with a short call, and in a script you can generate a time stamp for the file name and then set it up on the scheduler. Matt Rick Davidson wrote: Essentially the good folks at Enron and WorldComm brought us the Sarbanes-Oxley Act or SOX for short. Public companies have to keep a record of all communications, the details of this are vague but mostly apply to the money people and decision makers. Since we cant selectively catch that specific traffic we have to grab it all. Basicly all mail must be archived including the attachments and all mail must be retrievable in a reasonable amount of time, thats about it. We were considering stripping the attachments and storing them in a directory structure and storing the email text data in the sql database. Separate fields for the date, to, from, subject, the entire D file and the attachment names and their location. We figure we can get decent compression and searchabiltiy with the text info but the biggest hurdle is the attachments and being a Title company we have alot of large attachments to deal with. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 3:53 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's funny that you should ask. I just coded that one up in VBScript this last weekend. I even managed to decode base64 text attachments, remove quoted-printable encoding, and strip out all of the HTML code. If this is for archiving according to legal requirement, the attachments would probably be necessary however. Sandy had some good recommendations on how to archive. Maybe if you shared your requirements with the list, someone would have some recommendations as to how to approach this a better way. Matt Rick Davidson wrote: ok thanks Matt, we do have some programmers on staff here but I will sure conscript your help if we brick wall. Regardless of where it is stored its going to be a massive amount of data, my initial samplings show 1.5 to 2GB per day. Yikes! You wouldnt happen to know how to parse mime types and remove attachments would you? :-) Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 2:58 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient That's going to be one massive database :) I've become quite the VBScripter as of late (if that's something to brag about), so let me know if you need any help. Matt Rick Davidson wrote: Thanks Matt, COPYFILE is working perfectly, now its just a matter of writing the program to parse and insert it into the SQL database. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Matt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, October 26, 2004 5:15 PM Subject: Re: [Declude.JunkMail] Determining a BCC Recipient Rick, This information is in the Q* file. If you use the COPYFILE action, it will keep both the D* and the Q* file. The only issue is that the Declude headers are lost and each message is kept separately and not viewable without a special application like spamreview. IMO, this is appropriate for archiving due to legal requirement, but not for doing review. If you want to handle this in a different way by just sending to a mailbox, you can use a WARN action with the %ALLRECIPS% variable which will contain the BCC addresses as well. For instance, you could do the following: TESTNAMEWARN X-RECIPIENTS: %ALLRECIPS% This of course exposes the BCC info to all that might view the headers. Matt Rick Davidson wrote: I am looking at creating our own email archiving solution using sql, the main hurdle is how to handle and email sent to a user using BCC. Is there a way to use Declude to include that info in a recipient x-header? If I send myself using only the BCC field the header contains only this From: Rick Davidson [EMAIL PROTECTED] To: Undisclosed-Recipient:; Subject: test I assume the BCC info is lost once the message hits the senders SMTP server correct? Rick Davidson National Systems Manager North American Title Group - --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com
Re: Re[2]: [Declude.JunkMail] Determining a BCC Recipient
After all these suggestions I think concatenating the Q and D file and maintaining a text file is a much better way to go, dtsearch definately looks attractive. Thanks again for the suggestions. Rick Davidson National Systems Manager North American Title Group - - Original Message - From: Sanford Whiteman [EMAIL PROTECTED] To: Rick Davidson [EMAIL PROTECTED] Sent: Wednesday, October 27, 2004 4:46 PM Subject: Re[2]: [Declude.JunkMail] Determining a BCC Recipient I will look into those, the boss wants me to do this on the cheap, the sql idea was first so we could at least say we were archiving the email. If you just want archiving for independent audit and to show good faith, concatenate the Q and D into an envelope-preserving MBOX for each day. However, you have to plan for a real investigation, and retrievability and simple envelope and body searching requirements will not be met on the cheap--since maintaining terabyte databases with _any_ data isn't cheap. Full-text indexing of such dbs also not a small project no matter what the driver. FTR, dtSearch web costs, I believe, 1000 bucks ( + server + storage + labor ). --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re[2]: [Declude.JunkMail] Determining a BCC Recipient
I strongly recommend that you just simply keep these in their Q* and D* formats and zip up the directories every night and write them to a CD or something every so often. Like I keep trying to say, this isn't an every so often or best-effort regulation. It's strict and for-real. . . . you can easily write something that would unzip the files, search for addresses in the Q* files, and copy the needed files to a directory when needed. Searches are almost always by keyword, not by user. This is why full-text indexing of body and attachment is a must. And the restrictions on outside auditor access, et al. are too long a list to satisfy here. Just remember that this question relates to SOX, not random measures under the umbrella of archiving. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Determining a BCC Recipient
Please don't parse my words so carefully. Each company is different and therefore so are their needs. Many that archive will never need to go through the data, primarily because many companies aren't so enormous that they have the legal liability nor the volume that would necessitate a preemptive indexing of content. In such cases, constructing a solution to unzip and index the archives in response to a subpoena would be entirely appropriate, and probably quite simple to do. For more frequent access to such information, one could turn drive compression on in Windows and leave the files in raw format within massive directories and then use Index Server to get maybe a better effect. I would consider it to be unrealistic to demand that a full text indexing be done of file attachments, so this solution would probably be near perfect. A little bit of scripting in ASP could get you a search engine capable of identifying files by way of sender, recipient, and/or text, and display the contents of each message in a browser window (decoded even if you wished). The decoding part would be a fair deal of work. That's probably how I would approach it. Note that you probably would need to turn WHITELIST AUTH or any IP settings off for the COPYFILE filter to work on internal E-mail. You could replace this with a credit filter that won't result in disabling actions based on test name, probably one that combines both IP and MAILFROM as matches. Matt Sanford Whiteman wrote: I strongly recommend that you just simply keep these in their Q* and D* formats and zip up the directories every night and write them to a CD or something every so often. Like I keep trying to say, this isn't an "every so often" or best-effort regulation. It's strict and for-real. . . . you can easily write something that would unzip the files, search for addresses in the Q* files, and copy the needed files to a directory when needed. Searches are almost always by keyword, not by user. This is why full-text indexing of body and attachment is a must. And the restrictions on outside auditor access, et al. are too long a list to satisfy here. Just remember that this question relates to SOX, not random measures under the umbrella of archiving. --Sandy Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release/ Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/download/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/release/ --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ =
[Declude.JunkMail] Determining a BCC Recipient
I am looking at creating our own email archiving solution using sql, the main hurdle is how to handle and email sent to a user using BCC. Is there a way to use Declude to include that info in a recipient x-header? If I send myself using only the BCC field the header contains only this From: Rick Davidson [EMAIL PROTECTED] To: Undisclosed-Recipient:; Subject: test I assume the BCC info is lost once the message hits the senders SMTP server correct? Rick Davidson National Systems Manager North American Title Group - --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
Re: [Declude.JunkMail] Determining a BCC Recipient
Rick, This information is in the Q* file. If you use the COPYFILE action, it will keep both the D* and the Q* file. The only issue is that the Declude headers are lost and each message is kept separately and not viewable without a special application like spamreview. IMO, this is appropriate for archiving due to legal requirement, but not for doing review. If you want to handle this in a different way by just sending to a mailbox, you can use a WARN action with the %ALLRECIPS% variable which will contain the BCC addresses as well. For instance, you could do the following: TESTNAMEWARN X-RECIPIENTS: %ALLRECIPS% This of course exposes the BCC info to all that might view the headers. Matt Rick Davidson wrote: I am looking at creating our own email archiving solution using sql, the main hurdle is how to handle and email sent to a user using BCC. Is there a way to use Declude to include that info in a recipient x-header? If I send myself using only the BCC field the header contains only this From: Rick Davidson [EMAIL PROTECTED] To: Undisclosed-Recipient:; Subject: test I assume the BCC info is lost once the message hits the senders SMTP server correct? Rick Davidson National Systems Manager North American Title Group - --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com. -- = MailPure custom filters for Declude JunkMail Pro. http://www.mailpure.com/software/ = --- [This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
