Re: /sys/block and /dev and partitions
On 16/08/2015 00:00, Ilya Dryomov wrote: On Sat, Aug 15, 2015 at 11:56 PM, Loic Dachary l...@dachary.org wrote: Hi Ilya, On 15/08/2015 19:42, Ilya Dryomov wrote: On Sat, Aug 15, 2015 at 6:35 PM, Loic Dachary l...@dachary.org wrote: Hi Sage, On 15/08/2015 16:28, Sage Weil wrote: On Sat, 15 Aug 2015, Loic Dachary wrote: Hi, Is there a portable and consistent way to figure out if a given /dev/XXX path (for instance /dev/dm-1) is a partition of a whole device ? Although checking /sys/block/dm-1/dm/name for a number at the end (like mpatha1 or mpatha2) would probably work, it feels like a fragile hack. Looking into /sys/block/dm-1/slaves will lead to /sys/block/dm-1/slaves/dm-0 and we can check that /sys/block/dm-*/subsystem is class/block. But that does not necessarily mean dm-1 is a partition of dm-0, just that it's a slave of dm-0. Take a look at is_partition in ceph-disk, whih is the best I came up with. Basically it checks if the device name appears as /sys/block/*/$foo... For regular devices, you can access() /sys/dev/block/maj:min/partition. If it's there, it's a partition - no need to iterate over /sys/block. I added http://tracker.ceph.com/issues/12706 for when someone has time to rework that part of the code. That is consistently updated for /dev/sdb or /dev/vdb but things are different when using multipath. I'll rely on /sys/block/dm-?/dm/name instead until a better solution is found. A better way might be to rely on the fact that a dm partition will necessarily have its uuid prefixed by part. In that case, it should be safe to assume that the thing in slaves is a whole disk - I think that's what various util-linux tools do. However, IIRC the dm uuid is optional, so that won't work on a dm device without a uuid. It looks like multipath on both CentOS 7 and Ubuntu 14.04 set the uuid in this way. Is it also safe to assume that if the uuid is: $ cat /sys/dev/block/253:?/dm/uuid mpath-3533007d0 part1-mpath-3533007d0 part2-mpath-3533007d0 it means these were created by multipath because of the mpath ? When asking dmsetup with: Yes, I think so. I'm pretty sure these partid- and mpath- prefixes were devised for exactly this purpose. Excellent ! Thanks, Ilya -- Loïc Dachary, Artisan Logiciel Libre signature.asc Description: OpenPGP digital signature
Re: /sys/block and /dev and partitions
Hi Ilya, On 15/08/2015 19:42, Ilya Dryomov wrote: On Sat, Aug 15, 2015 at 6:35 PM, Loic Dachary l...@dachary.org wrote: Hi Sage, On 15/08/2015 16:28, Sage Weil wrote: On Sat, 15 Aug 2015, Loic Dachary wrote: Hi, Is there a portable and consistent way to figure out if a given /dev/XXX path (for instance /dev/dm-1) is a partition of a whole device ? Although checking /sys/block/dm-1/dm/name for a number at the end (like mpatha1 or mpatha2) would probably work, it feels like a fragile hack. Looking into /sys/block/dm-1/slaves will lead to /sys/block/dm-1/slaves/dm-0 and we can check that /sys/block/dm-*/subsystem is class/block. But that does not necessarily mean dm-1 is a partition of dm-0, just that it's a slave of dm-0. Take a look at is_partition in ceph-disk, whih is the best I came up with. Basically it checks if the device name appears as /sys/block/*/$foo... For regular devices, you can access() /sys/dev/block/maj:min/partition. If it's there, it's a partition - no need to iterate over /sys/block. I added http://tracker.ceph.com/issues/12706 for when someone has time to rework that part of the code. That is consistently updated for /dev/sdb or /dev/vdb but things are different when using multipath. I'll rely on /sys/block/dm-?/dm/name instead until a better solution is found. A better way might be to rely on the fact that a dm partition will necessarily have its uuid prefixed by part. In that case, it should be safe to assume that the thing in slaves is a whole disk - I think that's what various util-linux tools do. However, IIRC the dm uuid is optional, so that won't work on a dm device without a uuid. It looks like multipath on both CentOS 7 and Ubuntu 14.04 set the uuid in this way. Is it also safe to assume that if the uuid is: $ cat /sys/dev/block/253:?/dm/uuid mpath-3533007d0 part1-mpath-3533007d0 part2-mpath-3533007d0 it means these were created by multipath because of the mpath ? When asking dmsetup with: $ sudo dmsetup info --columns --noheading --options subsystem mpath part2 part1 it looks like it assumes the first part of the uuid is always the subsystem. Which makes me wonder how reliable this is. What I'm really looking for is a way to assert that a give dm-* is owned by multipath. Thanks a lot ! Thanks, Ilya -- Loïc Dachary, Artisan Logiciel Libre signature.asc Description: OpenPGP digital signature
Re: /sys/block and /dev and partitions
On Sat, Aug 15, 2015 at 11:56 PM, Loic Dachary l...@dachary.org wrote: Hi Ilya, On 15/08/2015 19:42, Ilya Dryomov wrote: On Sat, Aug 15, 2015 at 6:35 PM, Loic Dachary l...@dachary.org wrote: Hi Sage, On 15/08/2015 16:28, Sage Weil wrote: On Sat, 15 Aug 2015, Loic Dachary wrote: Hi, Is there a portable and consistent way to figure out if a given /dev/XXX path (for instance /dev/dm-1) is a partition of a whole device ? Although checking /sys/block/dm-1/dm/name for a number at the end (like mpatha1 or mpatha2) would probably work, it feels like a fragile hack. Looking into /sys/block/dm-1/slaves will lead to /sys/block/dm-1/slaves/dm-0 and we can check that /sys/block/dm-*/subsystem is class/block. But that does not necessarily mean dm-1 is a partition of dm-0, just that it's a slave of dm-0. Take a look at is_partition in ceph-disk, whih is the best I came up with. Basically it checks if the device name appears as /sys/block/*/$foo... For regular devices, you can access() /sys/dev/block/maj:min/partition. If it's there, it's a partition - no need to iterate over /sys/block. I added http://tracker.ceph.com/issues/12706 for when someone has time to rework that part of the code. That is consistently updated for /dev/sdb or /dev/vdb but things are different when using multipath. I'll rely on /sys/block/dm-?/dm/name instead until a better solution is found. A better way might be to rely on the fact that a dm partition will necessarily have its uuid prefixed by part. In that case, it should be safe to assume that the thing in slaves is a whole disk - I think that's what various util-linux tools do. However, IIRC the dm uuid is optional, so that won't work on a dm device without a uuid. It looks like multipath on both CentOS 7 and Ubuntu 14.04 set the uuid in this way. Is it also safe to assume that if the uuid is: $ cat /sys/dev/block/253:?/dm/uuid mpath-3533007d0 part1-mpath-3533007d0 part2-mpath-3533007d0 it means these were created by multipath because of the mpath ? When asking dmsetup with: Yes, I think so. I'm pretty sure these partid- and mpath- prefixes were devised for exactly this purpose. Thanks, Ilya -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
/sys/block and /dev and partitions
Hi, Is there a portable and consistent way to figure out if a given /dev/XXX path (for instance /dev/dm-1) is a partition of a whole device ? Although checking /sys/block/dm-1/dm/name for a number at the end (like mpatha1 or mpatha2) would probably work, it feels like a fragile hack. Looking into /sys/block/dm-1/slaves will lead to /sys/block/dm-1/slaves/dm-0 and we can check that /sys/block/dm-*/subsystem is class/block. But that does not necessarily mean dm-1 is a partition of dm-0, just that it's a slave of dm-0. Any idea ? Cheers -- Loïc Dachary, Artisan Logiciel Libre signature.asc Description: OpenPGP digital signature
creating the issue for the v0.94.4 release
Hi Abhishek, I'll be driving the next hammer point release (v0.94.4) and theanalyst will participate as I did for v0.94.3 (i.e. helping and mentoring when there is a need). During the v0.94.4 release you will have the opportunity to participate in all the steps. When we move to v0.94.5 you'll hopefully know all you need to drive it, should you want to. As you discovered over the past few weeks, it's not a complex process, but there are a lot of details. Contrary to what I told you the other day http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_start_backporting_a_stable_release/diff?utf8=%E2%9C%93version=7version_from=6commit=View+differences is not the documentation you need to create a new ticket for v0.94.4 and I removed this chapter to avoid further confusion. There is a more detailed explanation at: http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_start_working_on_a_new_point_release A section was added as a reminder that someone has to volunteer to drive the release. http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_start_working_on_a_new_point_release/diff?utf8=%E2%9C%93version=9version_from=8commit=View+differences Let me know if you need anything else to create the issue vor v0.94.4. Cheers -- Loïc Dachary, Artisan Logiciel Libre signature.asc Description: OpenPGP digital signature
Re: /sys/block and /dev and partitions
On Sat, Aug 15, 2015 at 6:35 PM, Loic Dachary l...@dachary.org wrote: Hi Sage, On 15/08/2015 16:28, Sage Weil wrote: On Sat, 15 Aug 2015, Loic Dachary wrote: Hi, Is there a portable and consistent way to figure out if a given /dev/XXX path (for instance /dev/dm-1) is a partition of a whole device ? Although checking /sys/block/dm-1/dm/name for a number at the end (like mpatha1 or mpatha2) would probably work, it feels like a fragile hack. Looking into /sys/block/dm-1/slaves will lead to /sys/block/dm-1/slaves/dm-0 and we can check that /sys/block/dm-*/subsystem is class/block. But that does not necessarily mean dm-1 is a partition of dm-0, just that it's a slave of dm-0. Take a look at is_partition in ceph-disk, whih is the best I came up with. Basically it checks if the device name appears as /sys/block/*/$foo... For regular devices, you can access() /sys/dev/block/maj:min/partition. If it's there, it's a partition - no need to iterate over /sys/block. That is consistently updated for /dev/sdb or /dev/vdb but things are different when using multipath. I'll rely on /sys/block/dm-?/dm/name instead until a better solution is found. A better way might be to rely on the fact that a dm partition will necessarily have its uuid prefixed by part. In that case, it should be safe to assume that the thing in slaves is a whole disk - I think that's what various util-linux tools do. However, IIRC the dm uuid is optional, so that won't work on a dm device without a uuid. Thanks, Ilya -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: /sys/block and /dev and partitions
On Sat, 15 Aug 2015, Loic Dachary wrote: Hi, Is there a portable and consistent way to figure out if a given /dev/XXX path (for instance /dev/dm-1) is a partition of a whole device ? Although checking /sys/block/dm-1/dm/name for a number at the end (like mpatha1 or mpatha2) would probably work, it feels like a fragile hack. Looking into /sys/block/dm-1/slaves will lead to /sys/block/dm-1/slaves/dm-0 and we can check that /sys/block/dm-*/subsystem is class/block. But that does not necessarily mean dm-1 is a partition of dm-0, just that it's a slave of dm-0. Take a look at is_partition in ceph-disk, whih is the best I came up with. Basically it checks if the device name appears as /sys/block/*/$foo... sage -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: /sys/block and /dev and partitions
Hi Sage, On 15/08/2015 16:28, Sage Weil wrote: On Sat, 15 Aug 2015, Loic Dachary wrote: Hi, Is there a portable and consistent way to figure out if a given /dev/XXX path (for instance /dev/dm-1) is a partition of a whole device ? Although checking /sys/block/dm-1/dm/name for a number at the end (like mpatha1 or mpatha2) would probably work, it feels like a fragile hack. Looking into /sys/block/dm-1/slaves will lead to /sys/block/dm-1/slaves/dm-0 and we can check that /sys/block/dm-*/subsystem is class/block. But that does not necessarily mean dm-1 is a partition of dm-0, just that it's a slave of dm-0. Take a look at is_partition in ceph-disk, whih is the best I came up with. Basically it checks if the device name appears as /sys/block/*/$foo... That is consistently updated for /dev/sdb or /dev/vdb but things are different when using multipath. I'll rely on /sys/block/dm-?/dm/name instead until a better solution is found. Thanks ! sage -- To unsubscribe from this list: send the line unsubscribe ceph-devel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- Loïc Dachary, Artisan Logiciel Libre signature.asc Description: OpenPGP digital signature
Re: creating the issue for the v0.94.4 release
On 15/08/2015 17:49, Abhishek Varshney wrote: Hi Loic,
Thanks for the detailed explanation. I have created an issue to keep track of
hammer v0.94.4 release here : http://tracker.ceph.com/issues/12701. I guess
the HOWTO page (http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO)
would be updated with this tracker issue once v0.94.3 is released? Please
correct me if I am wrong.
You are correct.
Could you also update the header
http://tracker.ceph.com/issues/12701#teuthology-run-commit to mimic
http://tracker.ceph.com/issues/11990#teuthology-run-commit-1e841b08cc9534e654de50967f06113fb7383b0chammer-backports-July-20
? It should be
.h3 teuthology run commit e1dadd3da9e39daf669f94715c7833d2b280bbed
(hammer backports August-14)
Because $ git rev-parse ceph/hammer-backports
e1dadd3da9e39daf669f94715c7833d2b280bbed
which is the branch you populated and that was compiled by
http://ceph.com/gitbuilder.cgi.
The content of this h3. section should the the output of the command
git --no-pager log --format='%H %s' --graph ceph/hammer..ceph/hammer-backports
| perl -p -e 's// /g; if (/\w+\s+Merge (\d+)/) { s|\w+\s+Merge (\d+).*|Pull
request $1:https://github.com/ceph/ceph/pull/$1|; } else {
s|(\w+)\s+(.*)|$2:https://github.com/ceph/ceph/commit/$1|; } s/\*/+/; s/^/*
/;'
it starts with:
* + Pull request 5202:https://github.com/ceph/ceph/pull/5202
* |\
* | + rpm: add missing Java
conditionals:https://github.com/ceph/ceph/commit/6417e8eff942dbbf6cd231ffb73136b7bdcd837f
* + Pull request 5203:https://github.com/ceph/ceph/pull/5203
* |\
* | + Add rpm conditionals :
cephfs_java:https://github.com/ceph/ceph/commit/37284773a754ef4b91d2b5436d46a5fd2be4f67e
* + Pull request 5204:https://github.com/ceph/ceph/pull/5204
...
and should render nicely.
This should then be followed by h4. section, one per test suite we are going to
run.
h4. rbd
pre
./virtualenv/bin/teuthology-suite --priority 1000 --suite rbd --subset $(expr
$RANDOM % 5)/5 --suite-branch hammer --distro ubuntu --email l...@dachary.org
--ceph hammer-backports --machine-type plana,burnupi,mira
/pre
etc.
Once this is done, I will copy paste the command to actually schedule the test.
And then we'll have to wait a few days for it to complete.
Cheers
Regards
Abhishek Varshney
On Sat, Aug 15, 2015 at 4:00 PM, Loic Dachary l...@dachary.org
mailto:l...@dachary.org wrote:
Hi Abhishek,
I'll be driving the next hammer point release (v0.94.4) and theanalyst
will participate as I did for v0.94.3 (i.e. helping and mentoring when there
is a need). During the v0.94.4 release you will have the opportunity to
participate in all the steps. When we move to v0.94.5 you'll hopefully know
all you need to drive it, should you want to. As you discovered over the past
few weeks, it's not a complex process, but there are a lot of details.
Contrary to what I told you the other day
http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_start_backporting_a_stable_release/diff?utf8=%E2%9C%93version=7version_from=6commit=View+differences
is not the documentation you need to create a new ticket for v0.94.4 and
I removed this chapter to avoid further confusion. There is a more detailed
explanation at:
http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_start_working_on_a_new_point_release
A section was added as a reminder that someone has to volunteer to drive
the release.
http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_start_working_on_a_new_point_release/diff?utf8=%E2%9C%93version=9version_from=8commit=View+differences
Let me know if you need anything else to create the issue vor v0.94.4.
Cheers
--
Loïc Dachary, Artisan Logiciel Libre
--
This email and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed. If
you have received this email in error please notify the system manager. This
message contains confidential information and is intended only for the
individual named. If you are not the named addressee you should not
disseminate, distribute or copy this e-mail. Please notify the sender
immediately by e-mail if you have received this e-mail by mistake and delete
this e-mail from your system. If you are not the intended recipient you are
notified that disclosing, copying, distributing or taking any action in
reliance on the contents of this information is strictly prohibited. Although
Flipkart has taken reasonable precautions to ensure no viruses are present in
this email, the company cannot accept responsibility for any loss or damage
arising from the use of this email or attachments
--
Loïc Dachary, Artisan Logiciel Libre
signature.asc
Description: OpenPGP
Re: creating the issue for the v0.94.4 release
Hi,
On 15/08/2015 19:54, Abhishek Varshney wrote:
Done ! Let me know if it is fine.
Perfect ! I have run the commands for you and the corresponding suite can be
seen at http://pulpito.ceph.com/. We usually add a link to each h4. with
something like:
* *running*
http://pulpito.ceph.com/loic-2015-08-15_22:00:55-rados-hammer-backports---basic-multi
etc. I expect teuthology will do something similar eventually but right now we
have to do that manually.
As much as I like to be your command line proxy, things will be easier if you
can run those yourself in the future ;-) I opened an issue to that end at
http://tracker.ceph.com/issues/12705. The staff in charge of the sepia lab is
extremely busy and it may take a few weeks before you actually have access.
Cheers
Thanks
Abhishek
On Sat, Aug 15, 2015 at 9:47 PM, Loic Dachary l...@dachary.org
mailto:l...@dachary.org wrote:
On 15/08/2015 17:49, Abhishek Varshney wrote: Hi Loic,
Thanks for the detailed explanation. I have created an issue to keep
track of hammer v0.94.4 release here : http://tracker.ceph.com/issues/12701.
I guess the HOWTO page
(http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO) would be updated
with this tracker issue once v0.94.3 is released? Please correct me if I am
wrong.
You are correct.
Could you also update the header
http://tracker.ceph.com/issues/12701#teuthology-run-commit to mimic
http://tracker.ceph.com/issues/11990#teuthology-run-commit-1e841b08cc9534e654de50967f06113fb7383b0chammer-backports-July-20
? It should be
.h3 teuthology run commit e1dadd3da9e39daf669f94715c7833d2b280bbed
(hammer backports August-14)
Because $ git rev-parse ceph/hammer-backports
e1dadd3da9e39daf669f94715c7833d2b280bbed
which is the branch you populated and that was compiled by
http://ceph.com/gitbuilder.cgi.
The content of this h3. section should the the output of the command
git --no-pager log --format='%H %s' --graph
ceph/hammer..ceph/hammer-backports | perl -p -e 's// /g; if (/\w+\s+Merge
(\d+)/) { s|\w+\s+Merge (\d+).*|Pull request
$1:https://github.com/ceph/ceph/pull/$1|; } else {
s|(\w+)\s+(.*)|$2:https://github.com/ceph/ceph/commit/$1|; } s/\*/+/; s/^/*
/;'
it starts with:
* + Pull request 5202:https://github.com/ceph/ceph/pull/5202
* |\
* | + rpm: add missing Java
conditionals:https://github.com/ceph/ceph/commit/6417e8eff942dbbf6cd231ffb73136b7bdcd837f
* + Pull request 5203:https://github.com/ceph/ceph/pull/5203
* |\
* | + Add rpm conditionals :
cephfs_java:https://github.com/ceph/ceph/commit/37284773a754ef4b91d2b5436d46a5fd2be4f67e
* + Pull request 5204:https://github.com/ceph/ceph/pull/5204
...
and should render nicely.
This should then be followed by h4. section, one per test suite we are
going to run.
h4. rbd
pre
./virtualenv/bin/teuthology-suite --priority 1000 --suite rbd --subset
$(expr $RANDOM % 5)/5 --suite-branch hammer --distro ubuntu --email
l...@dachary.org mailto:l...@dachary.org --ceph hammer-backports
--machine-type plana,burnupi,mira
/pre
etc.
Once this is done, I will copy paste the command to actually schedule the
test. And then we'll have to wait a few days for it to complete.
Cheers
Regards
Abhishek Varshney
On Sat, Aug 15, 2015 at 4:00 PM, Loic Dachary l...@dachary.org
mailto:l...@dachary.org mailto:l...@dachary.org
mailto:l...@dachary.org wrote:
Hi Abhishek,
I'll be driving the next hammer point release (v0.94.4) and
theanalyst will participate as I did for v0.94.3 (i.e. helping and mentoring
when there is a need). During the v0.94.4 release you will have the
opportunity to participate in all the steps. When we move to v0.94.5 you'll
hopefully know all you need to drive it, should you want to. As you
discovered over the past few weeks, it's not a complex process, but there are
a lot of details.
Contrary to what I told you the other day
http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_start_backporting_a_stable_release/diff?utf8=%E2%9C%93version=7version_from=6commit=View+differences
is not the documentation you need to create a new ticket for
v0.94.4 and I removed this chapter to avoid further confusion. There is a
more detailed explanation at:
http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_start_working_on_a_new_point_release
A section was added as a reminder that someone has to volunteer to
drive the release.
http://tracker.ceph.com/projects/ceph-releases/wiki/HOWTO_start_working_on_a_new_point_release/diff?utf8=%E2%9C%93version=9version_from=8commit=View+differences
Let me know if you need anything else to create the issue vor
v0.94.4.
