Re: [ceph-users] Read IO to object while new data still in journal

2015-12-30 Thread Dong Wu 
what i know is that librbd use applied callback, here is the code: in
send_write() calls librados::Rados::aio_create_completion,  parameter
rados_req_cb is cb_safe, and cb_complete is NULL, the cb_safe is just
applied callback.

  void AbstractWrite::send_write() {
ldout(m_ictx->cct, 20) << "send_write " << this << " " << m_oid << " "
   << m_object_off << "~" << m_object_len << dendl;

m_state = LIBRBD_AIO_WRITE_FLAT;
guard_write();
add_write_ops(_write);
assert(m_write.size() != 0);

librados::AioCompletion *rados_completion =
  librados::Rados::aio_create_completion(this, NULL, rados_req_cb);
int r = m_ictx->data_ctx.aio_operate(m_oid, rados_completion, _write,
 m_snap_seq, m_snaps);
assert(r == 0);
rados_completion->release();
  }


librados::AioCompletion *librados::Rados::aio_create_completion(void *cb_arg,
callback_t cb_complete,
callback_t cb_safe)
{
  AioCompletionImpl *c;
  int r = rados_aio_create_completion(cb_arg, cb_complete, cb_safe, (void**));
  assert(r == 0);
  return new AioCompletion(c);
}


anything wrong?

Regards,
Dong Wu

2015-12-31 10:33 GMT+08:00 min fang :
> yes, the question here is, librbd use the committed callback, as my
> understanding, when this callback returned, librbd write will be looked as
> completed. So I can issue a read IO even if the data is not readable. In
> this case, i would like to know what data will be returned for the read IO?
>
> 2015-12-31 10:29 GMT+08:00 Dong Wu :
>>
>> there are two callbacks: committed and applied, committed means write
>> to all replica's journal, applied means write to all replica's file
>> system. so when applied callback return to client, it means data can
>> be read.
>>
>> 2015-12-31 10:15 GMT+08:00 min fang :
>> > Hi, as my understanding, write IO will committed data to journal
>> > firstly,
>> > then give a safe callback to ceph client. So it is possible that data
>> > still
>> > in journal when I send a read IO to the same area. So what data will be
>> > returned if the new data still in journal?
>> >
>> > Thanks.
>> >
>> > ___
>> > ceph-users mailing list
>> > ceph-users@lists.ceph.com
>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> >
>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Read IO to object while new data still in journal

2015-12-30 Thread Dong Wu 
there are two callbacks: committed and applied, committed means write
to all replica's journal, applied means write to all replica's file
system. so when applied callback return to client, it means data can
be read.

2015-12-31 10:15 GMT+08:00 min fang :
> Hi, as my understanding, write IO will committed data to journal firstly,
> then give a safe callback to ceph client. So it is possible that data still
> in journal when I send a read IO to the same area. So what data will be
> returned if the new data still in journal?
>
> Thanks.
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Read IO to object while new data still in journal

2015-12-30 Thread min fang 
thanks, so ceph can guarantee after write commit call back, read IO can get
the new written data, right?

2015-12-31 10:55 GMT+08:00 Zhi Zhang :

> If the data has not been written to filestore, as you mentioned, it is
> still in journal, your following read op will be blocked until the
> data is written to filestore.
>
> This is because when writing this data, the related object context
> will hold ondisk_write_lock. This lock will be released in a callback
> after data is in filestore. When ondisk_write_lock is held, read op to
> this data will be blocked.
>
>
> Regards,
> Zhi Zhang (David)
> Contact: zhang.david2...@gmail.com
>   zhangz.da...@outlook.com
>
>
> On Thu, Dec 31, 2015 at 10:33 AM, min fang 
> wrote:
> > yes, the question here is, librbd use the committed callback, as my
> > understanding, when this callback returned, librbd write will be looked
> as
> > completed. So I can issue a read IO even if the data is not readable. In
> > this case, i would like to know what data will be returned for the read
> IO?
> >
> > 2015-12-31 10:29 GMT+08:00 Dong Wu :
> >>
> >> there are two callbacks: committed and applied, committed means write
> >> to all replica's journal, applied means write to all replica's file
> >> system. so when applied callback return to client, it means data can
> >> be read.
> >>
> >> 2015-12-31 10:15 GMT+08:00 min fang :
> >> > Hi, as my understanding, write IO will committed data to journal
> >> > firstly,
> >> > then give a safe callback to ceph client. So it is possible that data
> >> > still
> >> > in journal when I send a read IO to the same area. So what data will
> be
> >> > returned if the new data still in journal?
> >> >
> >> > Thanks.
> >> >
> >> > ___
> >> > ceph-users mailing list
> >> > ceph-users@lists.ceph.com
> >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >> >
> >
> >
> >
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Read IO to object while new data still in journal

2015-12-30 Thread Zhi Zhang 
If the data has not been written to filestore, as you mentioned, it is
still in journal, your following read op will be blocked until the
data is written to filestore.

This is because when writing this data, the related object context
will hold ondisk_write_lock. This lock will be released in a callback
after data is in filestore. When ondisk_write_lock is held, read op to
this data will be blocked.


Regards,
Zhi Zhang (David)
Contact: zhang.david2...@gmail.com
  zhangz.da...@outlook.com


On Thu, Dec 31, 2015 at 10:33 AM, min fang  wrote:
> yes, the question here is, librbd use the committed callback, as my
> understanding, when this callback returned, librbd write will be looked as
> completed. So I can issue a read IO even if the data is not readable. In
> this case, i would like to know what data will be returned for the read IO?
>
> 2015-12-31 10:29 GMT+08:00 Dong Wu :
>>
>> there are two callbacks: committed and applied, committed means write
>> to all replica's journal, applied means write to all replica's file
>> system. so when applied callback return to client, it means data can
>> be read.
>>
>> 2015-12-31 10:15 GMT+08:00 min fang :
>> > Hi, as my understanding, write IO will committed data to journal
>> > firstly,
>> > then give a safe callback to ceph client. So it is possible that data
>> > still
>> > in journal when I send a read IO to the same area. So what data will be
>> > returned if the new data still in journal?
>> >
>> > Thanks.
>> >
>> > ___
>> > ceph-users mailing list
>> > ceph-users@lists.ceph.com
>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> >
>
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Read IO to object while new data still in journal

2015-12-30 Thread min fang 
yes, the question here is, librbd use the committed callback, as my
understanding, when this callback returned, librbd write will be looked as
completed. So I can issue a read IO even if the data is not readable. In
this case, i would like to know what data will be returned for the read IO?

2015-12-31 10:29 GMT+08:00 Dong Wu :

> there are two callbacks: committed and applied, committed means write
> to all replica's journal, applied means write to all replica's file
> system. so when applied callback return to client, it means data can
> be read.
>
> 2015-12-31 10:15 GMT+08:00 min fang :
> > Hi, as my understanding, write IO will committed data to journal firstly,
> > then give a safe callback to ceph client. So it is possible that data
> still
> > in journal when I send a read IO to the same area. So what data will be
> > returned if the new data still in journal?
> >
> > Thanks.
> >
> > ___
> > ceph-users mailing list
> > ceph-users@lists.ceph.com
> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> >
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Read IO to object while new data still in journal

2015-12-30 Thread min fang 
Hi, as my understanding, write IO will committed data to journal firstly,
then give a safe callback to ceph client. So it is possible that data still
in journal when I send a read IO to the same area. So what data will be
returned if the new data still in journal?

Thanks.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Read IO to object while new data still in journal

2015-12-30 Thread Zhi Zhang 
Regards,
Zhi Zhang (David)
Contact: zhang.david2...@gmail.com
  zhangz.da...@outlook.com


On Thu, Dec 31, 2015 at 11:08 AM, min fang  wrote:
> thanks, so ceph can guarantee after write commit call back, read IO can get
> the new written data, right?

yep :-)

>
> 2015-12-31 10:55 GMT+08:00 Zhi Zhang :
>>
>> If the data has not been written to filestore, as you mentioned, it is
>> still in journal, your following read op will be blocked until the
>> data is written to filestore.
>>
>> This is because when writing this data, the related object context
>> will hold ondisk_write_lock. This lock will be released in a callback
>> after data is in filestore. When ondisk_write_lock is held, read op to
>> this data will be blocked.
>>
>>
>> Regards,
>> Zhi Zhang (David)
>> Contact: zhang.david2...@gmail.com
>>   zhangz.da...@outlook.com
>>
>>
>> On Thu, Dec 31, 2015 at 10:33 AM, min fang 
>> wrote:
>> > yes, the question here is, librbd use the committed callback, as my
>> > understanding, when this callback returned, librbd write will be looked
>> > as
>> > completed. So I can issue a read IO even if the data is not readable. In
>> > this case, i would like to know what data will be returned for the read
>> > IO?
>> >
>> > 2015-12-31 10:29 GMT+08:00 Dong Wu :
>> >>
>> >> there are two callbacks: committed and applied, committed means write
>> >> to all replica's journal, applied means write to all replica's file
>> >> system. so when applied callback return to client, it means data can
>> >> be read.
>> >>
>> >> 2015-12-31 10:15 GMT+08:00 min fang :
>> >> > Hi, as my understanding, write IO will committed data to journal
>> >> > firstly,
>> >> > then give a safe callback to ceph client. So it is possible that data
>> >> > still
>> >> > in journal when I send a read IO to the same area. So what data will
>> >> > be
>> >> > returned if the new data still in journal?
>> >> >
>> >> > Thanks.
>> >> >
>> >> > ___
>> >> > ceph-users mailing list
>> >> > ceph-users@lists.ceph.com
>> >> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> >> >
>> >
>> >
>> >
>> > ___
>> > ceph-users mailing list
>> > ceph-users@lists.ceph.com
>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> >
>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com