Re: How to encrypt Text and still be able to use full text search? 3rd Attempt ++
mos schrieb: I posted this message twice in the past 3 days, and it never gets on the mailing list. Why? Here it is again: I have a Text field that contains paragraph text and for security reasons I need to have it encrypted. If I do this, how can I still implement full text search on it? Also, I have a lot of Float columns that need to be protected but the user has to use comparison operators like and on them. Any recommendations? Mike, What size ist the database? Could you create some kind of (temporary) table holding the data unencrypted? As fulltext search is only possible on myisam tables, you might want to put this on a ramdisk and create it during mysql startup (just make a symlink like /var/lib/mysql/UnencryptedDatabase - /ramdisk/ and use the |--init-file-Paramter for mysqld to create and fill the table). It would at least make it more difficult to get the data for somebody who has physical access to the machine as long as you have all your partitions encrypted as well have to enter your password during startup. ||I know there is still danger: somebody at the ISP could shut down the server and modify your initrd and try to get you password when you enter it during startup, but as long as you won't host the machine yourself, there probably is no better option. Get rackspace that has doors and can be locked... a little more security, but usually the ISP has a second key in their safe :( ||Or you might set it up so you have to enter 2 Passwords, the first one to decrypt and start a small program that checksums the kernel and initrd that is in memory, and then a second one to mount the partitions...| | If the value of the data is really a million, host it on your own and install security systems etc. and a 24/7 NOC keeping an eye on your server looking for hackers and so on. If your budget is only $100/month I would do the way I described above. ||| |Jan| || -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
How to encrypt Text and still be able to use full text search? 3rd Attempt ++
I posted this message twice in the past 3 days, and it never gets on the mailing list. Why? Here it is again: I have a Text field that contains paragraph text and for security reasons I need to have it encrypted. If I do this, how can I still implement full text search on it? Also, I have a lot of Float columns that need to be protected but the user has to use comparison operators like and on them. Any recommendations? TIA Mike -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: How to encrypt Text and still be able to use full text search? 3rd Attempt ++
On 26 Oct 2007 at 9:17, mos wrote: I posted this message twice in the past 3 days, and it never gets on the mailing list. Why? Here it is again: I have a Text field that contains paragraph text and for security reasons I need to have it encrypted. If I do this, how can I still implement full text search on it? Also, I have a lot of Float columns that need to be protected but the user has to use comparison operators like and on them. Any recommendations? Hi, This is quite a difficult one, and as usual in the field of security depends on how valuable the data is and how difficult you want it to be for an attacker to obtain it. If you let us know what type of data this is and how well it has to be protected, maybe we can help more. security reasons is a bit vague, but I can understand that you don't want to give too much away. I can say one thing though, in order for the data to be indexed by MySQL , it has to be in an unencrypted form somewhere in the database. There is no way I know to get around this, but I hope someone can correct me :) Regards Ian -- -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: How to encrypt Text and still be able to use full text search? 3rd Attempt ++
Ian, At 09:36 AM 10/26/2007, you wrote: On 26 Oct 2007 at 9:17, mos wrote: I posted this message twice in the past 3 days, and it never gets on the mailing list. Why? Here it is again: I have a Text field that contains paragraph text and for security reasons I need to have it encrypted. If I do this, how can I still implement full text search on it? Also, I have a lot of Float columns that need to be protected but the user has to use comparison operators like and on them. Any recommendations? Hi, This is quite a difficult one, and as usual in the field of security depends on how valuable the data is and how difficult you want it to be for an attacker to obtain it. If you let us know what type of data this is and how well it has to be protected, maybe we can help more. security reasons is a bit vague, but I can understand that you don't want to give too much away. The data is quite valuable because there is a lot of competition in this particular marketplace and my competitors would like to get their hands on it. I've spent 5 years writing the software and generating the data. Let's say for the sake of argument the data is worth $1 million. How do I stop my competitor from bribing some flunky at the ISP into turning over the backup of my data or just e-mailing the MySQL password file to him? Also I don't want anyone at the ISP viewing the data or changing it because I'd be liable for any data errors. I can say one thing though, in order for the data to be indexed by MySQL , it has to be in an unencrypted form somewhere in the database. There is no way I know to get around this, but I hope someone can correct me :) I hope so too. :) There are quite a few databases out there that have transparent encryption (Blowfish, AES etc.) and I'm wondering why MySQL haven't implemented it, especially now with the new laws that make the company liable for security breaches on the web. On the other databases I've used, I haven't noticed any speed decrease if the table is encrypted. Mike -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: How to encrypt Text and still be able to use full text search? 3rd Attempt ++
[EMAIL PROTECTED] wrote: mos wrote: The data is quite valuable because there is a lot of competition in this particular marketplace and my competitors would like to get their hands on it. I've spent 5 years writing the software and generating the data. Let's say for the sake of argument the data is worth $1 million. How do I stop my competitor from bribing some flunky at the ISP into turning over the backup of my data or just e-mailing the MySQL password file to him? Also I don't want anyone at the ISP viewing the data or changing it because I'd be liable for any data errors. Host the machines in-house. I think that could be done for less than a million bucks for a smallish setup. Of course, I've only ever been a bystander with that sort of project, so the figures may be a lot higher than I'm guessing. For instance, you'd want a beefy connection installed, of course. And then there's the salary for someone to administer to everything. I agree. If you're using shared hosting, forget about encryption. Physical access to the machines ALWAYS trumps every other kind of security, so you can't do what you're trying to do (secure data in an insecure environment). Rent a T1 line for $500/mo and charge customers what the data is worth. Baron -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: How to encrypt Text and still be able to use full text search? 3rd Attempt ++
mos wrote: The data is quite valuable because there is a lot of competition in this particular marketplace and my competitors would like to get their hands on it. I've spent 5 years writing the software and generating the data. Let's say for the sake of argument the data is worth $1 million. How do I stop my competitor from bribing some flunky at the ISP into turning over the backup of my data or just e-mailing the MySQL password file to him? Also I don't want anyone at the ISP viewing the data or changing it because I'd be liable for any data errors. Host the machines in-house. I think that could be done for less than a million bucks for a smallish setup. Of course, I've only ever been a bystander with that sort of project, so the figures may be a lot higher than I'm guessing. For instance, you'd want a beefy connection installed, of course. And then there's the salary for someone to administer to everything. brian -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: How to encrypt Text and still be able to use full text search? 3rd Attempt ++
On 10/26/07, Baron Schwartz [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: mos wrote: The data is quite valuable because there is a lot of competition in this particular marketplace and my competitors would like to get their hands on it. I've spent 5 years writing the software and generating the data. Let's say for the sake of argument the data is worth $1 million. How do I stop my competitor from bribing some flunky at the ISP into turning over the backup of my data or just e-mailing the MySQL password file to him? Also I don't want anyone at the ISP viewing the data or changing it because I'd be liable for any data errors. Host the machines in-house. I think that could be done for less than a million bucks for a smallish setup. Of course, I've only ever been a bystander with that sort of project, so the figures may be a lot higher than I'm guessing. For instance, you'd want a beefy connection installed, of course. And then there's the salary for someone to administer to everything. I agree. If you're using shared hosting, forget about encryption. Physical access to the machines ALWAYS trumps every other kind of security, so you can't do what you're trying to do (secure data in an insecure environment). Rent a T1 line for $500/mo and charge customers what the data is worth. Baron I also agree, however for the sake of argument could we assume that the order of the wording in the entry probably imparts a significant amount of it's value? If that is the case, I would think creating a second column of unencrypted text (with a full text index) which would be nothing more than copy of the the text with the words in a random order might provide a bit of the protection that the user is looking for.
Re: How to encrypt Text and still be able to use full text search? 3rd Attempt ++
At 12:31 PM 10/26/2007, you wrote: [EMAIL PROTECTED] wrote: mos wrote: The data is quite valuable because there is a lot of competition in this particular marketplace and my competitors would like to get their hands on it. I've spent 5 years writing the software and generating the data. Let's say for the sake of argument the data is worth $1 million. How do I stop my competitor from bribing some flunky at the ISP into turning over the backup of my data or just e-mailing the MySQL password file to him? Also I don't want anyone at the ISP viewing the data or changing it because I'd be liable for any data errors. Host the machines in-house. I think that could be done for less than a million bucks for a smallish setup. Of course, I've only ever been a bystander with that sort of project, so the figures may be a lot higher than I'm guessing. For instance, you'd want a beefy connection installed, of course. And then there's the salary for someone to administer to everything. I agree. If you're using shared hosting, forget about encryption. Physical access to the machines ALWAYS trumps every other kind of security, so you can't do what you're trying to do (secure data in an insecure environment). Out of curiosity, why can't you use an ISP if the table is encrypted on a dedicated server (using a database other than MySQL, say SQLite)? The pw would be entered via VPN and is not stored in memory or in any file. The data on disk is always encrypted and the selected rows are only decrypted in memory on the fly. The ISP administrator never sees the pw or the unencrypted data. Rent a T1 line for $500/mo and charge customers what the data is worth. Yes, that will solve the problem. I'd have to incur more up front costs but security would be under my control. Mike -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: How to encrypt Text and still be able to use full text search? 3rd Attempt ++
At 01:47 PM 10/26/2007, you wrote: On 10/26/07, Baron Schwartz [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: mos wrote: The data is quite valuable because there is a lot of competition in this particular marketplace and my competitors would like to get their hands on it. I've spent 5 years writing the software and generating the data. Let's say for the sake of argument the data is worth $1 million. How do I stop my competitor from bribing some flunky at the ISP into turning over the backup of my data or just e-mailing the MySQL password file to him? Also I don't want anyone at the ISP viewing the data or changing it because I'd be liable for any data errors. Host the machines in-house. I think that could be done for less than a million bucks for a smallish setup. Of course, I've only ever been a bystander with that sort of project, so the figures may be a lot higher than I'm guessing. For instance, you'd want a beefy connection installed, of course. And then there's the salary for someone to administer to everything. I agree. If you're using shared hosting, forget about encryption. Physical access to the machines ALWAYS trumps every other kind of security, so you can't do what you're trying to do (secure data in an insecure environment). Rent a T1 line for $500/mo and charge customers what the data is worth. Baron I also agree, however for the sake of argument could we assume that the order of the wording in the entry probably imparts a significant amount of it's value? If that is the case, I would think creating a second column of unencrypted text (with a full text index) which would be nothing more than copy of the the text with the words in a random order might provide a bit of the protection that the user is looking for. Good point. I hadn't thought of that. :) I also need to protect a couple dozen Float fields and thought I could obscure them a bit by adding an offset to them based on an encrypted id stored with each row. It is not going to be as good as encryption but will help to obfuscate the data. Mike -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
Re: How to encrypt Text and still be able to use full text search? 3rd Attempt ++
If you are going to rely on obfuscation to protect valuable data, you might want to consider not posting the particular method you will use on a public mailing list. I think any method you implement will lower the overall security of the system. But, if you must search for encrypted text, you could have another representation of the text salted and hashed word for word. Then salt and hash each search word and search for it in the hashed text. You're still leaking information about word popularity if you do this which may help a determined attacker. - Original Message From: Baron Schwartz [EMAIL PROTECTED] To: mos [EMAIL PROTECTED] Cc: mysql@lists.mysql.com Sent: Friday, October 26, 2007 3:54:11 PM Subject: Re: How to encrypt Text and still be able to use full text search? 3rd Attempt ++ I also need to protect a couple dozen Float fields and thought I could obscure them a bit by adding an offset to them based on an encrypted id stored with each row. It is not going to be as good as encryption but will help to obfuscate the data. How much will obfuscation save you? Are you saving nickels and dimes to protect millions of dollars? I've seen people get burned by rolling their own encryption (I could tell you a great war story about a consultant I worked with who invented encryption for SSNs in a database). An insurance policy is something else to consider. Heck, buy the insurance and do weak obfuscation, then get the insurance money and go to Mexico. ... I could put strychnine in the guacamole... Baron -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED] __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Re: How to encrypt Text and still be able to use full text search? 3rd Attempt ++
I also need to protect a couple dozen Float fields and thought I could obscure them a bit by adding an offset to them based on an encrypted id stored with each row. It is not going to be as good as encryption but will help to obfuscate the data. How much will obfuscation save you? Are you saving nickels and dimes to protect millions of dollars? I've seen people get burned by rolling their own encryption (I could tell you a great war story about a consultant I worked with who invented encryption for SSNs in a database). An insurance policy is something else to consider. Heck, buy the insurance and do weak obfuscation, then get the insurance money and go to Mexico. ... I could put strychnine in the guacamole... Baron -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/[EMAIL PROTECTED]
