Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-03-09 Thread Nikos Tsironis
On 3/9/22 10:51, Mikulas Patocka wrote: Hi Note that you must submit kcopyd callbacks from a single thread, otherwise there's a race condition in snapshot. Hi, Thanks for the feedback. Yes, I'm aware of that. The snapshot code doesn't take locks in the copy_callback and it expects that

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-03-09 Thread Mikulas Patocka
On Tue, 8 Mar 2022, Nikos Tsironis wrote: > My work focuses mainly on improving the IOPs and latency of the > dm-snapshot target, in order to bring the performance of short-lived > snapshots as close as possible to bare-metal performance. > > My initial performance evaluation of dm-snapshot

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-03-08 Thread Nikos Tsironis
On 3/1/22 23:32, Chaitanya Kulkarni wrote: Nikos, [8] https://kernel.dk/io_uring.pdf I would like to participate in the discussion too. The dm-clone target would also benefit from copy offload, as it heavily employs dm-kcopyd. I have been exploring redesigning kcopyd in order to achieve

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-03-03 Thread Nikos Tsironis
On 3/1/22 23:32, Chaitanya Kulkarni wrote: Nikos, [8] https://kernel.dk/io_uring.pdf I would like to participate in the discussion too. The dm-clone target would also benefit from copy offload, as it heavily employs dm-kcopyd. I have been exploring redesigning kcopyd in order to achieve

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-03-01 Thread Nikos Tsironis
On 1/27/22 09:14, Chaitanya Kulkarni wrote: Hi, * Background :- --- Copy offload is a feature that allows file-systems or storage devices to be instructed to copy files/logical blocks without requiring involvement of the local

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-02-08 Thread Nitesh Shetty
Chaitanya, I would like to join the conversation. Thanks, Nitesh On Sun, Feb 6, 2022 at 7:29 PM Javier González wrote: > > On 27.01.2022 07:14, Chaitanya Kulkarni wrote: > >Hi, > > > >* Background :- > >--- > > > >Copy offload

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-02-08 Thread David Disseldorp
On Thu, 27 Jan 2022 07:14:13 +, Chaitanya Kulkarni wrote: > Hi, > > * Background :- > --- > > Copy offload is a feature that allows file-systems or storage devices > to be instructed to copy files/logical blocks without

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-02-01 Thread Kanchan Joshi
On Thu, Jan 27, 2022 at 12:51 PM Chaitanya Kulkarni wrote: > > Hi, > > * Background :- > --- > > Copy offload is a feature that allows file-systems or storage devices > to be instructed to copy files/logical blocks without

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-01-31 Thread Luis Chamberlain
> * What we will discuss in the proposed session ? > --- > > I'd like to propose a session to go over this topic to understand :- > > 1. What are the blockers for Copy Offload implementation ? > 2. Discussion about having a file

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-01-31 Thread Bart Van Assche
On 1/26/22 23:14, Chaitanya Kulkarni wrote: [1]https://content.riscv.org/wp-content/uploads/2018/12/A-New-Golden-Age-for-Computer-Architecture-History-Challenges-and-Opportunities-David-Patterson-.pdf [2] https://www.snia.org/computational

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-01-31 Thread Johannes Thumshirn
On Fri, 2022-01-28 at 19:59 +, Adam Manzanares wrote: > On Thu, Jan 27, 2022 at 07:14:13AM +, Chaitanya Kulkarni wrote: > > > > * Current state of the work :- > > --- > > > > > > With [3] being hard to handle arbitrary

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2022-01-31 Thread Adam Manzanares
On Thu, Jan 27, 2022 at 07:14:13AM +, Chaitanya Kulkarni wrote: > Hi, > > * Background :- > --- > > Copy offload is a feature that allows file-systems or storage devices > to be instructed to copy files/logical blocks

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-11-19 Thread Bart Van Assche
On 11/19/21 02:47, Kanchan Joshi wrote: Given the multitude of things accumulated on this topic, Martin suggested to have a table/matrix. Some of those should go in the initial patchset, and the remaining are to be staged for subsequent work. Here is the attempt to split the stuff into two

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-11-17 Thread Bart Van Assche
On 11/17/21 04:53, Javier González wrote: Thanks for sharing this. We will make sure that DM / MD are supported and then we can cover examples. Hopefully, you guys can help with the bits for dm-crypt to make the decision to offload when it make sense. Will ask around to learn who should work

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-11-16 Thread Bart Van Assche
On 11/16/21 05:43, Javier González wrote:     - Here, we need copy emulation to support encryption without dealing with HW issues and garbage Hi Javier, Thanks very much for having taken notes and also for having shared these. Regarding the above comment, after the meeting I learned

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-11-04 Thread Javier González
On 29.10.2021 10:14, Javier González wrote: On 29.10.2021 00:21, Chaitanya Kulkarni wrote: On 10/7/21 11:49 PM, Javier González wrote: External email: Use caution opening links or attachments On 06.10.2021 10:33, Bart Van Assche wrote: On 10/6/21 3:05 AM, Javier González wrote: I agree

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-11-03 Thread Keith Busch
On Fri, Oct 29, 2021 at 09:15:43AM -0700, Bart Van Assche wrote: > On 10/28/21 10:51 PM, Hannes Reinecke wrote: > > Also Keith presented his work on a simple zone-based remapping block > > device, which included an in-kernel copy offload facility. > > Idea is to lift that as a standalone patch

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-10-29 Thread Bart Van Assche
On 10/28/21 10:51 PM, Hannes Reinecke wrote: Also Keith presented his work on a simple zone-based remapping block device, which included an in-kernel copy offload facility. Idea is to lift that as a standalone patch such that we can use it a fallback (ie software) implementation if no other

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-10-29 Thread Javier González
On 29.10.2021 07:51, Hannes Reinecke wrote: On 10/29/21 2:21 AM, Chaitanya Kulkarni wrote: On 10/7/21 11:49 PM, Javier González wrote: External email: Use caution opening links or attachments On 06.10.2021 10:33, Bart Van Assche wrote: On 10/6/21 3:05 AM, Javier González wrote: I agree

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-10-29 Thread Javier González
On 29.10.2021 00:21, Chaitanya Kulkarni wrote: On 10/7/21 11:49 PM, Javier González wrote: External email: Use caution opening links or attachments On 06.10.2021 10:33, Bart Van Assche wrote: On 10/6/21 3:05 AM, Javier González wrote: I agree that the topic is complex. However, we have not

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-10-28 Thread Hannes Reinecke
On 10/29/21 2:21 AM, Chaitanya Kulkarni wrote: On 10/7/21 11:49 PM, Javier González wrote: External email: Use caution opening links or attachments On 06.10.2021 10:33, Bart Van Assche wrote: On 10/6/21 3:05 AM, Javier González wrote: I agree that the topic is complex. However, we have not

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-10-14 Thread Javier González
Chaitanya, Did you have a chance to look at the answers below? I would like to start finding candidate dates throughout the next couple of weeks. Thanks, Javier On 06.10.2021 12:01, Javier González wrote: On 30.09.2021 09:43, Chaitanya Kulkarni wrote: Javier, Hi all, Since we are not

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-10-07 Thread Javier González
On 30.09.2021 09:20, Bart Van Assche wrote: On 9/28/21 12:13 PM, Javier González wrote: Since we are not going to be able to talk about this at LSF/MM, a few of us thought about holding a dedicated virtual discussion about Copy Offload. I believe we can use Chaitanya's thread as a start. Given

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-10-07 Thread Javier González
On 30.09.2021 09:43, Chaitanya Kulkarni wrote: Javier, Hi all, Since we are not going to be able to talk about this at LSF/MM, a few of us thought about holding a dedicated virtual discussion about Copy Offload. I believe we can use Chaitanya's thread as a start. Given the current state of

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-10-06 Thread Bart Van Assche
On 10/6/21 3:05 AM, Javier González wrote: I agree that the topic is complex. However, we have not been able to find a clear path forward in the mailing list. Hmm ... really? At least Martin Petersen and I consider device mapper support essential. How about starting from Mikulas' patch series

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-10-01 Thread Chaitanya Kulkarni
Javier, > > Hi all, > > Since we are not going to be able to talk about this at LSF/MM, a few of > us thought about holding a dedicated virtual discussion about Copy > Offload. I believe we can use Chaitanya's thread as a start. Given the > current state of the current patches, I would propose

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-10-01 Thread Javier González
On 30.09.2021 09:43, Chaitanya Kulkarni wrote: Javier, Hi all, Since we are not going to be able to talk about this at LSF/MM, a few of us thought about holding a dedicated virtual discussion about Copy Offload. I believe we can use Chaitanya's thread as a start. Given the current state of

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-09-30 Thread Bart Van Assche
On 9/28/21 12:13 PM, Javier González wrote: Since we are not going to be able to talk about this at LSF/MM, a few of us thought about holding a dedicated virtual discussion about Copy Offload. I believe we can use Chaitanya's thread as a start. Given the current state of the current patches, I

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-09-29 Thread Johannes Thumshirn
On 28/09/2021 21:13, Javier González wrote: > Since we are not going to be able to talk about this at LSF/MM, a few of > us thought about holding a dedicated virtual discussion about Copy > Offload. I believe we can use Chaitanya's thread as a start. Given the > current state of the current

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-09-29 Thread Javier González
On 12.05.2021 07:30, Johannes Thumshirn wrote: On 11/05/2021 02:15, Chaitanya Kulkarni wrote: Hi, * Background :- --- Copy offload is a feature that allows file-systems or storage devices to be instructed to copy

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-06-11 Thread Nikos Tsironis
On 5/11/21 3:15 AM, Chaitanya Kulkarni wrote: Hi, * Background :- --- Copy offload is a feature that allows file-systems or storage devices to be instructed to copy files/logical blocks without requiring involvement of the

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-06-11 Thread Chaitanya Kulkarni
On 5/10/21 17:15, Chaitanya Kulkarni wrote: > Hi, > > * Background :- > --- > > Copy offload is a feature that allows file-systems or storage devices > to be instructed to copy files/logical blocks without requiring > involvement

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-05-17 Thread Kanchan Joshi
> * What we will discuss in the proposed session ? > --- > > I'd like to propose a session to go over this topic to understand :- > > 1. What are the blockers for Copy Offload implementation ? > 2. Discussion about having a file

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-05-17 Thread Bart Van Assche
On 5/10/21 5:15 PM, Chaitanya Kulkarni wrote: > I'd like to propose a session to go over this topic to understand :- > > 1. What are the blockers for Copy Offload implementation ? > 2. Discussion about having a file system interface. > 3. Discussion about having right system call for user-space.

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-05-17 Thread Himanshu Madhani
> On May 10, 2021, at 7:15 PM, Chaitanya Kulkarni > wrote: > > * Background :- > --- > > Copy offload is a feature that allows file-systems or storage devices > to be instructed to copy files/logical blocks without

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-05-17 Thread Javier González
On 11.05.2021 00:15, Chaitanya Kulkarni wrote: Hi, * Background :- --- Copy offload is a feature that allows file-systems or storage devices to be instructed to copy files/logical blocks without requiring involvement of the

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-05-12 Thread Hannes Reinecke
On 5/11/21 2:15 AM, Chaitanya Kulkarni wrote: Hi, * Background :- --- Copy offload is a feature that allows file-systems or storage devices to be instructed to copy files/logical blocks without requiring involvement of the

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-05-12 Thread Knight, Frederick
I'd love to participate in this discussion. You mention the 2 different models (single command vs. multi-command). Just as a reminder, there are specific reasons for those 2 different models. Some applications know both the source and the destination, so can use the single command model (the

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-05-12 Thread Erwin van Londen
On Tue, 2021-05-11 at 00:15 +, Chaitanya Kulkarni wrote: > Hi, > > * Background :- > - > -- > > Copy offload is a feature that allows file-systems or storage devices > to be instructed to copy files/logical blocks without

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-05-12 Thread Johannes Thumshirn
On 11/05/2021 02:15, Chaitanya Kulkarni wrote: > Hi, > > * Background :- > --- > > Copy offload is a feature that allows file-systems or storage devices > to be instructed to copy files/logical blocks without requiring >

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-05-11 Thread Bart Van Assche
On 5/10/21 5:15 PM, Chaitanya Kulkarni wrote: > * What we will discuss in the proposed session ? > --- > > I'd like to propose a session to go over this topic to understand :- > > 1. What are the blockers for Copy Offload

[dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2021-05-10 Thread Chaitanya Kulkarni
Hi, * Background :- --- Copy offload is a feature that allows file-systems or storage devices to be instructed to copy files/logical blocks without requiring involvement of the local CPU. With reference to the RISC-V summit

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2020-02-20 Thread Knight, Frederick
FWIW - the design of NVMe Simply Copy specifically included versioning of the data structure that describes what to copy. The reason for that was random peoples desire to complexify the Simple Copy command. Specifically, there was room designed into the data structure to accommodate a source

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2020-02-20 Thread joshi.k
I am very keen on this topic. I've been doing some work for "NVMe simple copy", and would like to discuss and solicit opinion of community on the following: - Simple-copy, unlike XCOPY and P2P, is limited to copy within a single namespace. Some of the problems that original XCOPY work [2] faced

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2020-01-24 Thread Nikos Tsironis
On 1/7/20 8:14 PM, Chaitanya Kulkarni wrote: Hi all, * Background :- --- Copy offload is a feature that allows file-systems or storage devices to be instructed to copy files/logical blocks without requiring involvement of the

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2020-01-09 Thread Martin K. Petersen
Bart, > * Copying must be supported not only within a single storage device but > also between storage devices. Identifying which devices to permit copies between has been challenging. That has since been addressed in T10. > * VMware, which uses XCOPY (with a one-byte length ID, aka LID1).

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2020-01-09 Thread Logan Gunthorpe
On 2020-01-08 3:17 a.m., Javier González wrote: > I think this is good topic and I would like to participate in the > discussion too. I think that Logan Gunthorpe would also be interested > (Cc). Adding Kanchan too, who is also working on this and can contribute > to the discussion > > We

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2020-01-09 Thread Chaitanya Kulkarni
> Was this last discussed during the 2018 edition of LSF/MM (see also > https://www.spinics.net/lists/linux-block/msg24986.html)? Has anyone > taken notes during that session? I haven't found a report of that > session in the official proceedings (https://lwn.net/Articles/752509/). > > Thanks, > >

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2020-01-09 Thread Javier González
On 07.01.2020 18:14, Chaitanya Kulkarni wrote: Hi all, * Background :- --- Copy offload is a feature that allows file-systems or storage devices to be instructed to copy files/logical blocks without requiring involvement of

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2020-01-08 Thread Damien Le Moal
On 2020/01/09 12:19, Bart Van Assche wrote: > On 2020-01-07 10:14, Chaitanya Kulkarni wrote: >> * Current state of the work :- >> --- >> >> With [3] being hard to handle arbitrary DM/MD stacking without >> splitting the command in

Re: [dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2020-01-08 Thread Bart Van Assche
On 2020-01-07 10:14, Chaitanya Kulkarni wrote: > * Current state of the work :- > --- > > With [3] being hard to handle arbitrary DM/MD stacking without > splitting the command in two, one for copying IN and one for copying >

[dm-devel] [LSF/MM/BFP ATTEND] [LSF/MM/BFP TOPIC] Storage: Copy Offload

2020-01-08 Thread Chaitanya Kulkarni
Hi all, * Background :- --- Copy offload is a feature that allows file-systems or storage devices to be instructed to copy files/logical blocks without requiring involvement of the local CPU. With reference to the RISC-V summit