Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-29 Thread Anshuman Aggarwal
On 25 November 2014 at 10:26, Greg Freemyer greg.freem...@gmail.com wrote: On November 24, 2014 12:28:08 PM EST, Anshuman Aggarwal anshuman.aggar...@gmail.com wrote: On 24 November 2014 at 18:49, Greg Freemyer greg.freem...@gmail.com wrote: On November 24, 2014 1:48:48 AM EST, Anshuman

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-29 Thread Greg Freemyer
On Thu, Nov 27, 2014 at 12:50 PM, Anshuman Aggarwal anshuman.aggar...@gmail.com wrote: On 25 November 2014 at 10:26, Greg Freemyer greg.freem...@gmail.com wrote: On November 24, 2014 12:28:08 PM EST, Anshuman Aggarwal anshuman.aggar...@gmail.com wrote: On 24 November 2014 at 18:49, Greg

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-24 Thread Anshuman Aggarwal
On 24 November 2014 at 18:49, Greg Freemyer greg.freem...@gmail.com wrote: On November 24, 2014 1:48:48 AM EST, Anshuman Aggarwal anshuman.aggar...@gmail.com wrote: Sandeep, This isn't exactly RAID4 (only thing in common is a single parity disk but the data is not striped at all). I did

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-24 Thread Valdis . Kletnieks
On Mon, 24 Nov 2014 22:58:08 +0530, Anshuman Aggarwal said: prevents it from directly recognized by file system code . I was wondering if Split RAID block devices can be made to be unaware to the RAID scheme on top and be fully mountable and usable without the raid drivers (of course

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-24 Thread Greg Freemyer
On November 24, 2014 12:28:08 PM EST, Anshuman Aggarwal anshuman.aggar...@gmail.com wrote: On 24 November 2014 at 18:49, Greg Freemyer greg.freem...@gmail.com wrote: On November 24, 2014 1:48:48 AM EST, Anshuman Aggarwal anshuman.aggar...@gmail.com wrote: Sandeep, This isn't exactly RAID4

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-23 Thread SandeepKsinha
On Sat, Nov 22, 2014 at 8:24 PM, Greg Freemyer greg.freem...@gmail.com wrote: On November 22, 2014 9:43:23 AM EST, Anshuman Aggarwal anshuman.aggar...@gmail.com wrote: On 22 November 2014 at 19:33, Greg Freemyer greg.freem...@gmail.com wrote: On Sat, Nov 22, 2014 at 8:22 AM, Anshuman

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-23 Thread Anshuman Aggarwal
Sandeep, This isn't exactly RAID4 (only thing in common is a single parity disk but the data is not striped at all). I did bring it up on the linux-raid mailing list and have had a short conversation with Neil. He wasn't too excited about device mapper but didn't indicate why or why not. I would

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-22 Thread Greg Freemyer
Top posting is strongly discouraged on all kernel related mailing lists including this one. I've moved your reply to the bottom and then replied after that. In future I will ignore replies that are top posted. On 21 November 2014 17:11, Greg Freemyer greg.freem...@gmail.com wrote: On

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-22 Thread Anshuman Aggarwal
On 22 November 2014 at 18:47, Greg Freemyer greg.freem...@gmail.com wrote: Top posting is strongly discouraged on all kernel related mailing lists including this one. I've moved your reply to the bottom and then replied after that. In future I will ignore replies that are top posted. On

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-22 Thread Greg Freemyer
On Sat, Nov 22, 2014 at 8:22 AM, Anshuman Aggarwal anshuman.aggar...@gmail.com wrote: By not using stripes, we restrict writes to happen to just 1 drive and the XOR output to the parity drive which then explains the delayed and batched checksum (resulting in fewer writes to the parity drive).

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-22 Thread Greg Freemyer
On November 22, 2014 9:43:23 AM EST, Anshuman Aggarwal anshuman.aggar...@gmail.com wrote: On 22 November 2014 at 19:33, Greg Freemyer greg.freem...@gmail.com wrote: On Sat, Nov 22, 2014 at 8:22 AM, Anshuman Aggarwal anshuman.aggar...@gmail.com wrote: By not using stripes, we restrict writes

Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-21 Thread Anshuman Aggarwal
I'd a appreciate any help/pointers in implementing the proposal below including the right path to get this into the kernel itself. -- I'm outlining below a proposal for a RAID device mapper virtual block device for the kernel which adds split raid functionality on

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-21 Thread Greg Freemyer
On November 21, 2014 5:15:43 AM EST, Anshuman Aggarwal anshuman.aggar...@gmail.com wrote: I'd a appreciate any help/pointers in implementing the proposal below including the right path to get this into the kernel itself. -- I'm outlining below a proposal for a

Re: Split RAID: Proposal for archival RAID using incremental batch checksum

2014-11-21 Thread Anshuman Aggarwal
N pass through but with their own filesystems. Concatenation is via some kind of union fs solution not at the block level. Data is not supposed to be striped (this is critical so as to prevent all drives to be required to be accessed for consecutive data) Idea is that each drive can work