Re: file corruption solution (soft-update or ZFS)
On May 23, 2013, at 11:09 AM, Michael Sierchio ku...@tenebras.com wrote: On Thu, May 23, 2013 at 5:33 AM, Warren Block wbl...@wonkity.com wrote: .. One thing mentioned earlier is that ZFS wants lots of memory. 4G-8G minimum, some might say as much as the server will hold. Not necessarily so - deduplication places great demands on memory, but that can be satisfied with dedicated cache devices (on SSD for performance and safety reasons). Without dedup, the requirements are more modest. The rule of thumb for DeDupe is 1GB physical RAM for every 1TB of capacity. The issue is that the DeDupe metadata table must live in the ARC for good performance. The discussion I have seen on the ZFS lists indicates that L2ARC is not really adequate for this, so adding cache devices (SSD's) don't really help. On the other hand, you can use ZFS without DeDupe with as little as 2GB of total system RAM (depending on what else the system is doing). In my experience, the amount of RAM depends on the amount of I/O not the amount of storage. I find between 1GB and 3GB space for the ARC is adequate. -- Paul Kraus Deputy Technical Director, LoneStarCon 3 Sound Coordinator, Schenectady Light Opera Company ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: file corruption solution (soft-update or ZFS)
On Thu, 23 May 2013, saeedeh motlagh wrote: hello every body i have a question about fixing file corruption in freebsd. now i have freebsd8.2 and some times file corruption happened on it. this issue has a heavy cost for me and i want to avoid it or fixit it completely. so my question is: is it better to upgrade my freebsd to 9.1 and use soft update or migrate from UFS to ZFS? That's a judgement call, which means it depends. i heard so much about soft update -that is added in freebsd9.1- which can fix file corruption in acceptable way with low cost but i don't know how much is reliable and efficient. Several things: Soft updates have been around for quite a while. Soft updates journaling is the new addition. Neither of these address file corruption. Their purpose is to make sure the filesystem does not get corrupted, but individual files could still contain bad data. in the other hand, i think migration from UFS to ZFS can be another solution. as i read ZFS is is created to solve all the problems related integrity file system. is it reliable enough in comparison soft-update? now, i want to know which solution is better and why? Again, it depends. Does the target system have enough RAM for ZFS? If the file corruption is due to a hardware problem or an application writing bad data, no filesystem can prevent that. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: file corruption solution (soft-update or ZFS)
thanks for your reply. you know i have a sensitive server and unfortunately it is located some where that power outage happens much. so i want guarantee my data and avoid data lost and file corruption in my server. i do not have any problem in RAM and hardware. i don't know which approach is more suitable for my server. using soft-update or ZFS. please help me to select the best one. thank you so much On Thu, May 23, 2013 at 4:28 PM, Warren Block wbl...@wonkity.com wrote: On Thu, 23 May 2013, saeedeh motlagh wrote: hello every body i have a question about fixing file corruption in freebsd. now i have freebsd8.2 and some times file corruption happened on it. this issue has a heavy cost for me and i want to avoid it or fixit it completely. so my question is: is it better to upgrade my freebsd to 9.1 and use soft update or migrate from UFS to ZFS? That's a judgement call, which means it depends. i heard so much about soft update -that is added in freebsd9.1- which can fix file corruption in acceptable way with low cost but i don't know how much is reliable and efficient. Several things: Soft updates have been around for quite a while. Soft updates journaling is the new addition. Neither of these address file corruption. Their purpose is to make sure the filesystem does not get corrupted, but individual files could still contain bad data. in the other hand, i think migration from UFS to ZFS can be another solution. as i read ZFS is is created to solve all the problems related integrity file system. is it reliable enough in comparison soft-update? now, i want to know which solution is better and why? Again, it depends. Does the target system have enough RAM for ZFS? If the file corruption is due to a hardware problem or an application writing bad data, no filesystem can prevent that. -- *Sa.M* ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: file corruption solution (soft-update or ZFS)
On Thu, 23 May 2013 16:44+0430, saeedeh motlagh wrote: thanks for your reply. you know i have a sensitive server and unfortunately it is located some where that power outage happens much. so i want guarantee my data and avoid data lost and file corruption in my server. Maybe you should also invest in a decent UPS. i do not have any problem in RAM and hardware. i don't know which approach is more suitable for my server. using soft-update or ZFS. please help me to select the best one. thank you so much On Thu, May 23, 2013 at 4:28 PM, Warren Block wbl...@wonkity.com wrote: On Thu, 23 May 2013, saeedeh motlagh wrote: hello every body i have a question about fixing file corruption in freebsd. now i have freebsd8.2 and some times file corruption happened on it. this issue has a heavy cost for me and i want to avoid it or fixit it completely. so my question is: is it better to upgrade my freebsd to 9.1 and use soft update or migrate from UFS to ZFS? That's a judgement call, which means it depends. i heard so much about soft update -that is added in freebsd9.1- which can fix file corruption in acceptable way with low cost but i don't know how much is reliable and efficient. Several things: Soft updates have been around for quite a while. Soft updates journaling is the new addition. Neither of these address file corruption. Their purpose is to make sure the filesystem does not get corrupted, but individual files could still contain bad data. in the other hand, i think migration from UFS to ZFS can be another solution. as i read ZFS is is created to solve all the problems related integrity file system. is it reliable enough in comparison soft-update? now, i want to know which solution is better and why? Again, it depends. Does the target system have enough RAM for ZFS? If the file corruption is due to a hardware problem or an application writing bad data, no filesystem can prevent that. -- +---++ | Vennlig hilsen, | Best regards, | | Trond Endrestøl, | Trond Endrestøl, | | IT-ansvarlig, | System administrator, | | Fagskolen Innlandet, | Gjøvik Technical College, Norway, | | tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, | | sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. | +---++___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: file corruption solution (soft-update or ZFS)
On Thu, 23 May 2013, saeedeh motlagh wrote: you know i have a sensitive server and unfortunately it is located some where that power outage happens much. so i want guarantee my data and avoid data lost and file corruption in my server. i do not have any problem in RAM and hardware. The lack of a UPS can be considered a hardware problem. i don't know which approach is more suitable for my server. using soft-update or ZFS. please help me to select the best one. Please don't top-post, as it makes responding to your message more difficult. One thing mentioned earlier is that ZFS wants lots of memory. 4G-8G minimum, some might say as much as the server will hold. But resilient filesystems still can't prevent data corruption. Fix the power problem with a UPS. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: file corruption solution (soft-update or ZFS)
On Thu, May 23, 2013 at 5:33 AM, Warren Block wbl...@wonkity.com wrote: .. One thing mentioned earlier is that ZFS wants lots of memory. 4G-8G minimum, some might say as much as the server will hold. Not necessarily so - deduplication places great demands on memory, but that can be satisfied with dedicated cache devices (on SSD for performance and safety reasons). Without dedup, the requirements are more modest. Softupdates guarantee metadata consistency, but do nothing to address data integrity. ZFS has copy-on-write semantics (which solve a problem that even hardware RAID can't), and end-to-end checksums to detect/prevent data corruption (large drives will have uncorrectable bit errors over their lifetime). - M ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: file corruption solution (soft-update or ZFS)
On 5/23/2013 7:14 AM, saeedeh motlagh wrote: thanks for your reply. you know i have a sensitive server and unfortunately it is located some where that power outage happens much. so i want guarantee my data and avoid data lost and file corruption in my server. Get a good reliable UPS. Test it regularly, the batteries do fail. Test to make sure that it will work, unplug it and let the computer drain the battery to time it. Consider that the battery will degrade over time. One thing google does is put a 12V battery inside the chassis to help with the power backup, you might look into it. i do not have any problem in RAM and hardware. i don't know which approach is more suitable for my server. using soft-update or ZFS. please help me to select the best one. If power failure is an issue, you have no guarantee of data loss protection unless you use networked storage to a safe place. UFS soft updates protects against file system corruption in case of power loss, no guarantees of individual file consistency. ZFS guarantees no silent failures, it doesn't guarantee protection, only that you'll know about it. There is no filesystem that can guarantee you won't lose data in a power failure. Hard drives are known to lie about what's been physically synced to disk out of cache in order to improve speed. If the power goes out at the wrong time, you can lose data. ZFS can find a corrupted file and tell you, everything else won't. If you have a back up of that file, you can restore it. thank you so much On Thu, May 23, 2013 at 4:28 PM, Warren Block wbl...@wonkity.com wrote: On Thu, 23 May 2013, saeedeh motlagh wrote: hello every body i have a question about fixing file corruption in freebsd. now i have freebsd8.2 and some times file corruption happened on it. this issue has a heavy cost for me and i want to avoid it or fixit it completely. so my question is: is it better to upgrade my freebsd to 9.1 and use soft update or migrate from UFS to ZFS? That's a judgement call, which means it depends. i heard so much about soft update -that is added in freebsd9.1- which can fix file corruption in acceptable way with low cost but i don't know how much is reliable and efficient. Several things: Soft updates have been around for quite a while. Soft updates journaling is the new addition. Neither of these address file corruption. Their purpose is to make sure the filesystem does not get corrupted, but individual files could still contain bad data. in the other hand, i think migration from UFS to ZFS can be another solution. as i read ZFS is is created to solve all the problems related integrity file system. is it reliable enough in comparison soft-update? now, i want to know which solution is better and why? Again, it depends. Does the target system have enough RAM for ZFS? If the file corruption is due to a hardware problem or an application writing bad data, no filesystem can prevent that. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
