Re: [ceph-users] mon service failed to start

2018-02-21 Thread knawnd 
Could the problem be related with some faulty hardware (RAID-controller, port, cable) but not disk? 
Does "faulty" disk works OK on other server?


Behnam Loghmani wrote on 21/02/18 16:09:

Hi there,

I changed the SSD on the problematic node with the new one and reconfigure OSDs 
and MON service on it.
but the problem occurred again with:

"rocksdb: submit_transaction error: Corruption: block checksum mismatch code = 
2"

I get fully confused now.



On Tue, Feb 20, 2018 at 5:16 PM, Behnam Loghmani > wrote:


Hi Caspar,

I checked the filesystem and there isn't any error on filesystem.
The disk is SSD and it doesn't any attribute related to Wear level in 
smartctl and filesystem is
mounted with default options and no discard.

my ceph structure on this node is like this:

it has osd,mon,rgw services
1 SSD for OS and WAL/DB
2 HDD

OSDs are created by ceph-volume lvm.

the whole SSD is on 1 vg.
OS is on root lv
OSD.1 DB is on db-a
OSD.1 WAL is on wal-a
OSD.2 DB is on db-b
OSD.2 WAL is on wal-b

output of lvs:

   data-a data-a -wi-a-
   data-b data-b -wi-a-
   db-a   vg0    -wi-a-
   db-b   vg0    -wi-a-
   root   vg0    -wi-ao
   wal-a  vg0    -wi-a-
   wal-b  vg0    -wi-a-

after making a heavy write on the radosgw, OSD.1 and OSD.2 has stopped with 
"block checksum
mismatch" error.
Now on this node MON and OSDs services has stopped working with this error

I think my issue is related to this bug: 
http://tracker.ceph.com/issues/22102


I ran
#ceph-bluestore-tool fsck --path /var/lib/ceph/osd/ceph-1 --deep 1
but it returns the same error:

*** Caught signal (Aborted) **
  in thread 7fbf6c923d00 thread_name:ceph-bluestore-
2018-02-20 16:44:30.128787 7fbf6c923d00 -1 abort: Corruption: block 
checksum mismatch
  ceph version 12.2.2 (cf0baba3b47f9427c6c97e2144b094b7e5ba) luminous 
(stable)
  1: (()+0x3eb0b1) [0x55f779e6e0b1]
  2: (()+0xf5e0) [0x7fbf61ae15e0]
  3: (gsignal()+0x37) [0x7fbf604d31f7]
  4: (abort()+0x148) [0x7fbf604d48e8]
  5: (RocksDBStore::get(std::string const&, char const*, unsigned long,
ceph::buffer::list*)+0x1ce) [0x55f779d2b5ce]
  6: (BlueStore::Collection::get_onode(ghobject_t const&, bool)+0x545) 
[0x55f779cd8f75]
  7: (BlueStore::_fsck(bool, bool)+0x1bb5) [0x55f779cf1a75]
  8: (main()+0xde0) [0x55f779baab90]
  9: (__libc_start_main()+0xf5) [0x7fbf604bfc05]
  10: (()+0x1bc59f) [0x55f779c3f59f]
2018-02-20 16:44:30.131334 7fbf6c923d00 -1 *** Caught signal (Aborted) **
  in thread 7fbf6c923d00 thread_name:ceph-bluestore-

  ceph version 12.2.2 (cf0baba3b47f9427c6c97e2144b094b7e5ba) luminous 
(stable)
  1: (()+0x3eb0b1) [0x55f779e6e0b1]
  2: (()+0xf5e0) [0x7fbf61ae15e0]
  3: (gsignal()+0x37) [0x7fbf604d31f7]
  4: (abort()+0x148) [0x7fbf604d48e8]
  5: (RocksDBStore::get(std::string const&, char const*, unsigned long,
ceph::buffer::list*)+0x1ce) [0x55f779d2b5ce]
  6: (BlueStore::Collection::get_onode(ghobject_t const&, bool)+0x545) 
[0x55f779cd8f75]
  7: (BlueStore::_fsck(bool, bool)+0x1bb5) [0x55f779cf1a75]
  8: (main()+0xde0) [0x55f779baab90]
  9: (__libc_start_main()+0xf5) [0x7fbf604bfc05]
  10: (()+0x1bc59f) [0x55f779c3f59f]
  NOTE: a copy of the executable, or `objdump -rdS ` is needed 
to interpret this.

     -1> 2018-02-20 16:44:30.128787 7fbf6c923d00 -1 abort: Corruption: 
block checksum mismatch
  0> 2018-02-20 16:44:30.131334 7fbf6c923d00 -1 *** Caught signal 
(Aborted) **
  in thread 7fbf6c923d00 thread_name:ceph-bluestore-

  ceph version 12.2.2 (cf0baba3b47f9427c6c97e2144b094b7e5ba) luminous 
(stable)
  1: (()+0x3eb0b1) [0x55f779e6e0b1]
  2: (()+0xf5e0) [0x7fbf61ae15e0]
  3: (gsignal()+0x37) [0x7fbf604d31f7]
  4: (abort()+0x148) [0x7fbf604d48e8]
  5: (RocksDBStore::get(std::string const&, char const*, unsigned long,
ceph::buffer::list*)+0x1ce) [0x55f779d2b5ce]
  6: (BlueStore::Collection::get_onode(ghobject_t const&, bool)+0x545) 
[0x55f779cd8f75]
  7: (BlueStore::_fsck(bool, bool)+0x1bb5) [0x55f779cf1a75]
  8: (main()+0xde0) [0x55f779baab90]
  9: (__libc_start_main()+0xf5) [0x7fbf604bfc05]
  10: (()+0x1bc59f) [0x55f779c3f59f]
  NOTE: a copy of the executable, or `objdump -rdS ` is needed 
to interpret this.



Could you please help me to recover this node or find a way to prove SSD 
disk problem.

Best regards,
Behnam Loghmani




On Mon, Feb 19, 2018 at 1:35 PM, Caspar Smit > wrote:

Hi Behnam,

I would firstly recommend running a filesystem check on the monitor 
disk first to see if
there are any inconsistencies.

Is the disk

Re: [ceph-users] "Cannot get stat of OSD" in ceph.mgr.log upon enabling influx plugin

2018-02-20 Thread knawnd 
Ben, first of all thanks a lot for such quick reply! I appreciate a provided explanation and info on 
things to check!
I am new to all that that including InfluxDB that's why I used wrong influx cli to check if there 
are actual data is coming. But 
https://docs.influxdata.com/influxdb/v1.4/query_language/schema_exploration/ page helped to figure 
it out. So data is coming from ceph mgr node to influx if ssl and verify_ssl options are set to 
false. But it seems with self-signed certificate, ssl=true and verify_ssl=false data is not filling 
up InfluxDB database. I will try to apply your suggestions on debug to find out a source of the 
problem. But that is a different story.


Thanks again for your prompt and informative reply!

Benjeman Meekhof wrote on 19/02/18 18:07:

The 'cannot stat' messages are normal at startup, we see them also in
our working setup with mgr influx module.  Maybe they could be fixed
by delaying the module startup,  or having it check for some other
'all good' status but I haven't looked into it.  You should only be
seeing them when the mgr initially loads.

As far as not getting data, if the self-test works and outputs metrics
then the module is reading metrics ok from the mgr.  A few things you
could try:

- Check that the user you setup has rights to the destination
database, or admin rights to create database if you did not create and
setup beforehand
- Increase mgr debug and see if anything is showing up:  ceph tell
mgr.* injectargs '--debug_mgr 20'(this will be a lot of logging,
be sure to reset to 1/5 default)
- Check that your influx server is getting the traffic:   ' tcpdump -i
eth1 port 8086 and src host.example '

thanks,
Ben

On Mon, Feb 19, 2018 at 9:36 AM,   wrote:

Forgot to mentioned that influx self-test produces a reasonable output too
(long json list with some metrics and timestamps) as well as there are the
following lines in mgr log:

2018-02-19 17:35:04.208858 7f33a50ec700  1 mgr.server reply handle_command
(0) Success
2018-02-19 17:35:04.245285 7f33a50ec700  0 log_channel(audit) log [DBG] :
from='client.344950 :0/3773014505' entity='client.admin'
cmd=[{"prefix": "influx self-test"}]: dispatch
2018-02-19 17:35:04.245314 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer status'
2018-02-19 17:35:04.245319 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer mode'
2018-02-19 17:35:04.245323 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer on'
2018-02-19 17:35:04.245327 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer off'
2018-02-19 17:35:04.245331 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer eval'
2018-02-19 17:35:04.245335 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer eval-verbose'
2018-02-19 17:35:04.245339 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer optimize'
2018-02-19 17:35:04.245343 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer show'
2018-02-19 17:35:04.245347 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer rm'
2018-02-19 17:35:04.245351 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer reset'
2018-02-19 17:35:04.245354 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer dump'
2018-02-19 17:35:04.245358 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'balancer execute'
2018-02-19 17:35:04.245363 7f33a50ec700  1 mgr.server handle_command
pyc_prefix: 'influx self-test'
2018-02-19 17:35:04.402782 7f33a58ed700  1 mgr.server reply handle_command
(0) Success Self-test OK

kna...@gmail.com wrote on 19/02/18 17:27:


Dear Ceph users,

I am trying to enable influx plugin for ceph following
http://docs.ceph.com/docs/master/mgr/influx/ but no data comes to influxdb
DB. As soon as 'ceph mgr module enable influx' command is executed on one of
ceph mgr node (running on CentOS 7.4.1708) there are the following messages
in /var/log/ceph/ceph-mgr..log:

2018-02-19 17:11:05.947122 7f33c9b43600  0 ceph version 12.2.2
(cf0baba3b47f9427c6c97e2144b094b7e5ba) luminous (stable), process
(unknown), pid 96425
2018-02-19 17:11:05.947737 7f33c9b43600  0 pidfile_write: ignore empty
--pid-file
2018-02-19 17:11:05.986676 7f33c9b43600  1 mgr send_beacon standby
2018-02-19 17:11:06.003029 7f33c0e2a700  1 mgr init Loading python module
'balancer'
2018-02-19 17:11:06.031293 7f33c0e2a700  1 mgr init Loading python module
'dashboard'
2018-02-19 17:11:06.119328 7f33c0e2a700  1 mgr init Loading python module
'influx'
2018-02-19 17:11:06.220394 7f33c0e2a700  1 mgr init Loading python module
'restful'
2018-02-19 17:11:06.398380 7f33c0e2a700  1 mgr init Loading python module
'status'
2018-02-19 17:11:06.919109 7f33c0e2a700  1 mgr handle_mgr_map Activating!
2018-02-19 17:11:06.919454 7f33c0e2a700  1 mgr handle_mgr_map I am now
activating
2018-02-19 17:11:06.952174 7f33a58ed700  1 mgr load Constructed class from
module: balancer
2018-02-19 17:11:06.953259 7f33a58ed700  1 mgr load Constructed class

Re: [ceph-users] mgr[influx] Cannot transmit statistics: influxdb python module not found.

2018-02-19 Thread knawnd 

Marc Roos wrote on 13/02/18 00:50:

why not use collectd? centos7 rpms  should do fine.
Marc, sorry I somehow missed your question. One of the reason could be that collectd is a additional 
daemon whereas influx plugin for ceph is just an additional part of the already running system (ceph).



On Feb 12, 2018 9:50 PM, Benjeman Meekhof  wrote:


In our case I think we grabbed the SRPM from Fedora and rebuilt it on
Scientific Linux (another RHEL derivative).  Presumably the binary
didn't work or I would have installed it directly.  I'm not quite sure
why it hasn't migrated to EPEL yet.

I haven't tried the SRPM for latest releases, we're actually quite far
behind the current python-influx version since I built it a while back
but if I were you I'd grab whatever SRPM gets you the latest
python-influxdb release and give it a try.

http://rpmfind.net/linux/rpm2html/search.php?query=python-influxdb

thanks,
Ben

On Mon, Feb 12, 2018 at 11:03 AM,   wrote:

Dear all,

I'd like to store ceph luminous metrics into influxdb. It seems like influx
plugin has been already backported for lumious:
rpm -ql ceph-mgr-12.2.2-0.el7.x86_64|grep -i influx
/usr/lib64/ceph/mgr/influx
/usr/lib64/ceph/mgr/influx/__init__.py
/usr/lib64/ceph/mgr/influx/__init__.pyc
/usr/lib64/ceph/mgr/influx/__init__.pyo
/usr/lib64/ceph/mgr/influx/module.py
/usr/lib64/ceph/mgr/influx/module.pyc
/usr/lib64/ceph/mgr/influx/module.pyo

So following http://docs.ceph.com/docs/master/mgr/influx/ doc I enabled
influx plugin by executing the following command on mgr node:
ceph mgr module enable influx

but in ceph log I see the following error:
2018-02-12 15:51:31.241854 7f95e7942600  0 ceph version 12.2.2
(cf0baba3b47f9427c6c97e2144b094b7e5ba) luminous (stable), process
(unknown), pid 96425
[]
2018-02-12 15:51:31.422414 7f95dec29700  1 mgr init Loading python module
'influx'
[]
2018-02-12 15:51:32.227206 7f95c36ec700  1 mgr load Constructed class from
module: influx
[]
2018-02-12 15:51:32.228163 7f95c0ee7700  0 mgr[influx] Cannot transmit
statistics: influxdb python module not found.  Did you install it?

Indeed there is no python-influxdb module install on my mgr node (CentOS 7
x64) but yum search can't find it with the following repos enabled:
repo id
repo name   status
Ceph/x86_64
Ceph packages for x86_64
Ceph-noarch
Ceph noarch packages
base/7/x86_64 CentOS-7 - Base
ceph-source
Ceph source packages
epel/x86_64
Extra Packages for Enterprise Linux 7 - x86_64
extras/7/x86_64 CentOS-7 - Extras
updates/7/x86_64 CentOS-7 - Updates

Python version is 2.7.5.

Is 'pip install' the only way to go or there is still some option to have
required python module via rpm? I wonder how other people deals with that
issue?
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] "Cannot get stat of OSD" in ceph.mgr.log upon enabling influx plugin

2018-02-19 Thread knawnd 
Forgot to mentioned that influx self-test produces a reasonable output too (long json list with some 
metrics and timestamps) as well as there are the following lines in mgr log:


2018-02-19 17:35:04.208858 7f33a50ec700  1 mgr.server reply handle_command (0) 
Success
2018-02-19 17:35:04.245285 7f33a50ec700  0 log_channel(audit) log [DBG] : from='client.344950 
:0/3773014505' entity='client.admin' cmd=[{"prefix": "influx self-test"}]: dispatch

2018-02-19 17:35:04.245314 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer status'
2018-02-19 17:35:04.245319 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer mode'
2018-02-19 17:35:04.245323 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer on'
2018-02-19 17:35:04.245327 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer off'
2018-02-19 17:35:04.245331 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer eval'
2018-02-19 17:35:04.245335 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer eval-verbose'
2018-02-19 17:35:04.245339 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer optimize'
2018-02-19 17:35:04.245343 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer show'
2018-02-19 17:35:04.245347 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer rm'
2018-02-19 17:35:04.245351 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer reset'
2018-02-19 17:35:04.245354 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer dump'
2018-02-19 17:35:04.245358 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'balancer execute'
2018-02-19 17:35:04.245363 7f33a50ec700  1 mgr.server handle_command 
pyc_prefix: 'influx self-test'
2018-02-19 17:35:04.402782 7f33a58ed700  1 mgr.server reply handle_command (0) 
Success Self-test OK

kna...@gmail.com wrote on 19/02/18 17:27:

Dear Ceph users,

I am trying to enable influx plugin for ceph following http://docs.ceph.com/docs/master/mgr/influx/ 
but no data comes to influxdb DB. As soon as 'ceph mgr module enable influx' command is executed on 
one of ceph mgr node (running on CentOS 7.4.1708) there are the following messages in 
/var/log/ceph/ceph-mgr..log:


2018-02-19 17:11:05.947122 7f33c9b43600  0 ceph version 12.2.2 
(cf0baba3b47f9427c6c97e2144b094b7e5ba) luminous (stable), process (unknown), pid 96425

2018-02-19 17:11:05.947737 7f33c9b43600  0 pidfile_write: ignore empty 
--pid-file
2018-02-19 17:11:05.986676 7f33c9b43600  1 mgr send_beacon standby
2018-02-19 17:11:06.003029 7f33c0e2a700  1 mgr init Loading python module 
'balancer'
2018-02-19 17:11:06.031293 7f33c0e2a700  1 mgr init Loading python module 
'dashboard'
2018-02-19 17:11:06.119328 7f33c0e2a700  1 mgr init Loading python module 
'influx'
2018-02-19 17:11:06.220394 7f33c0e2a700  1 mgr init Loading python module 
'restful'
2018-02-19 17:11:06.398380 7f33c0e2a700  1 mgr init Loading python module 
'status'
2018-02-19 17:11:06.919109 7f33c0e2a700  1 mgr handle_mgr_map Activating!
2018-02-19 17:11:06.919454 7f33c0e2a700  1 mgr handle_mgr_map I am now 
activating
2018-02-19 17:11:06.952174 7f33a58ed700  1 mgr load Constructed class from 
module: balancer
2018-02-19 17:11:06.953259 7f33a58ed700  1 mgr load Constructed class from 
module: dashboard
2018-02-19 17:11:06.953959 7f33a58ed700  1 mgr load Constructed class from 
module: influx
2018-02-19 17:11:06.954193 7f33a58ed700  1 mgr load Constructed class from 
module: restful
2018-02-19 17:11:06.955549 7f33a58ed700  1 mgr load Constructed class from 
module: status
2018-02-19 17:11:06.955613 7f33a58ed700  1 mgr send_beacon active
2018-02-19 17:11:06.960224 7f33a58ed700  1 mgr[restful] Unknown request ''
2018-02-19 17:11:06.961912 7f33a28e7700  1 mgr[restful] server not running: no 
certificate configured
2018-02-19 17:11:06.969027 7f33a30e8700  0 Cannot get stat of OSD 0
 on so on for all 64 OSD I have in a cluster.

'ceph osd tree' shows all OSD are up. 'ceph health' gives HEALTH_OK.

python-influxdb-5.0.0-2.el7.noarch is installed on ceph mgr node. That rpm was 
rebuilt from fc28 srpm.

'ceph config-key dump|grep influx' shows reasonable info:
     "mgr/influx/database": "ceph_stats",
     "mgr/influx/hostname": "",
     "mgr/influx/password": "",
     "mgr/influx/ssl": "false",
     "mgr/influx/username": "cephstat",
     "mgr/influx/verify_ssl": "false"


influxdb-1.4.2-1.x86_64 is installed on influxdb host with CentOS 7.4.1708.

I would appreciate any help on that issue.




___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] "Cannot get stat of OSD" in ceph.mgr.log upon enabling influx plugin

2018-02-19 Thread knawnd 

Dear Ceph users,

I am trying to enable influx plugin for ceph following http://docs.ceph.com/docs/master/mgr/influx/ 
but no data comes to influxdb DB. As soon as 'ceph mgr module enable influx' command is executed on 
one of ceph mgr node (running on CentOS 7.4.1708) there are the following messages in 
/var/log/ceph/ceph-mgr..log:


2018-02-19 17:11:05.947122 7f33c9b43600  0 ceph version 12.2.2 
(cf0baba3b47f9427c6c97e2144b094b7e5ba) luminous (stable), process (unknown), pid 96425

2018-02-19 17:11:05.947737 7f33c9b43600  0 pidfile_write: ignore empty 
--pid-file
2018-02-19 17:11:05.986676 7f33c9b43600  1 mgr send_beacon standby
2018-02-19 17:11:06.003029 7f33c0e2a700  1 mgr init Loading python module 
'balancer'
2018-02-19 17:11:06.031293 7f33c0e2a700  1 mgr init Loading python module 
'dashboard'
2018-02-19 17:11:06.119328 7f33c0e2a700  1 mgr init Loading python module 
'influx'
2018-02-19 17:11:06.220394 7f33c0e2a700  1 mgr init Loading python module 
'restful'
2018-02-19 17:11:06.398380 7f33c0e2a700  1 mgr init Loading python module 
'status'
2018-02-19 17:11:06.919109 7f33c0e2a700  1 mgr handle_mgr_map Activating!
2018-02-19 17:11:06.919454 7f33c0e2a700  1 mgr handle_mgr_map I am now 
activating
2018-02-19 17:11:06.952174 7f33a58ed700  1 mgr load Constructed class from 
module: balancer
2018-02-19 17:11:06.953259 7f33a58ed700  1 mgr load Constructed class from 
module: dashboard
2018-02-19 17:11:06.953959 7f33a58ed700  1 mgr load Constructed class from 
module: influx
2018-02-19 17:11:06.954193 7f33a58ed700  1 mgr load Constructed class from 
module: restful
2018-02-19 17:11:06.955549 7f33a58ed700  1 mgr load Constructed class from 
module: status
2018-02-19 17:11:06.955613 7f33a58ed700  1 mgr send_beacon active
2018-02-19 17:11:06.960224 7f33a58ed700  1 mgr[restful] Unknown request ''
2018-02-19 17:11:06.961912 7f33a28e7700  1 mgr[restful] server not running: no 
certificate configured
2018-02-19 17:11:06.969027 7f33a30e8700  0 Cannot get stat of OSD 0
 on so on for all 64 OSD I have in a cluster.

'ceph osd tree' shows all OSD are up. 'ceph health' gives HEALTH_OK.

python-influxdb-5.0.0-2.el7.noarch is installed on ceph mgr node. That rpm was 
rebuilt from fc28 srpm.

'ceph config-key dump|grep influx' shows reasonable info:
"mgr/influx/database": "ceph_stats",
"mgr/influx/hostname": "",
"mgr/influx/password": "",
"mgr/influx/ssl": "false",
"mgr/influx/username": "cephstat",
"mgr/influx/verify_ssl": "false"


influxdb-1.4.2-1.x86_64 is installed on influxdb host with CentOS 7.4.1708.

I would appreciate any help on that issue.



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] mgr[influx] Cannot transmit statistics: influxdb python module not found.

2018-02-14 Thread knawnd 

Benjeman Meekhof wrote on 12/02/18 23:50:

In our case I think we grabbed the SRPM from Fedora and rebuilt it on
Scientific Linux (another RHEL derivative).
I've just done the same: rebuild from fc28 srpm (some spec-file tunning was required to build it on 
centos 7).



 Presumably the binary
didn't work or I would have installed it directly.  I'm not quite sure
why it hasn't migrated to EPEL yet.

I guess one of the reason can be 
https://bugzilla.redhat.com/show_bug.cgi?id=1336097


I haven't tried the SRPM for latest releases, we're actually quite far
behind the current python-influx version since I built it a while back
but if I were you I'd grab whatever SRPM gets you the latest
python-influxdb release and give it a try.

the mentioned in my initial email error disappeared


http://rpmfind.net/linux/rpm2html/search.php?query=python-influxdb

Ben, thanks for sharing your experience and the rest of info!



thanks,
Ben

On Mon, Feb 12, 2018 at 11:03 AM,   wrote:

Dear all,

I'd like to store ceph luminous metrics into influxdb. It seems like influx
plugin has been already backported for lumious:
rpm -ql ceph-mgr-12.2.2-0.el7.x86_64|grep -i influx
/usr/lib64/ceph/mgr/influx
/usr/lib64/ceph/mgr/influx/__init__.py
/usr/lib64/ceph/mgr/influx/__init__.pyc
/usr/lib64/ceph/mgr/influx/__init__.pyo
/usr/lib64/ceph/mgr/influx/module.py
/usr/lib64/ceph/mgr/influx/module.pyc
/usr/lib64/ceph/mgr/influx/module.pyo

So following http://docs.ceph.com/docs/master/mgr/influx/ doc I enabled
influx plugin by executing the following command on mgr node:
ceph mgr module enable influx

but in ceph log I see the following error:
2018-02-12 15:51:31.241854 7f95e7942600  0 ceph version 12.2.2
(cf0baba3b47f9427c6c97e2144b094b7e5ba) luminous (stable), process
(unknown), pid 96425
[]
2018-02-12 15:51:31.422414 7f95dec29700  1 mgr init Loading python module
'influx'
[]
2018-02-12 15:51:32.227206 7f95c36ec700  1 mgr load Constructed class from
module: influx
[]
2018-02-12 15:51:32.228163 7f95c0ee7700  0 mgr[influx] Cannot transmit
statistics: influxdb python module not found.  Did you install it?

Indeed there is no python-influxdb module install on my mgr node (CentOS 7
x64) but yum search can't find it with the following repos enabled:
repo id
repo name   status
Ceph/x86_64
Ceph packages for x86_64
Ceph-noarch
Ceph noarch packages
base/7/x86_64 CentOS-7 - Base
ceph-source
Ceph source packages
epel/x86_64
Extra Packages for Enterprise Linux 7 - x86_64
extras/7/x86_64 CentOS-7 - Extras
updates/7/x86_64 CentOS-7 - Updates

Python version is 2.7.5.

Is 'pip install' the only way to go or there is still some option to have
required python module via rpm? I wonder how other people deals with that
issue?
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] mgr[influx] Cannot transmit statistics: influxdb python module not found.

2018-02-12 Thread knawnd 

Dear all,

I'd like to store ceph luminous metrics into influxdb. It seems like influx plugin has been already 
backported for lumious:

rpm -ql ceph-mgr-12.2.2-0.el7.x86_64|grep -i influx
/usr/lib64/ceph/mgr/influx
/usr/lib64/ceph/mgr/influx/__init__.py
/usr/lib64/ceph/mgr/influx/__init__.pyc
/usr/lib64/ceph/mgr/influx/__init__.pyo
/usr/lib64/ceph/mgr/influx/module.py
/usr/lib64/ceph/mgr/influx/module.pyc
/usr/lib64/ceph/mgr/influx/module.pyo

So following http://docs.ceph.com/docs/master/mgr/influx/ doc I enabled influx plugin by executing 
the following command on mgr node:

ceph mgr module enable influx

but in ceph log I see the following error:
2018-02-12 15:51:31.241854 7f95e7942600  0 ceph version 12.2.2 
(cf0baba3b47f9427c6c97e2144b094b7e5ba) luminous (stable), process (unknown), pid 96425

[]
2018-02-12 15:51:31.422414 7f95dec29700  1 mgr init Loading python module 
'influx'
[]
2018-02-12 15:51:32.227206 7f95c36ec700  1 mgr load Constructed class from 
module: influx
[]
2018-02-12 15:51:32.228163 7f95c0ee7700  0 mgr[influx] Cannot transmit statistics: influxdb python 
module not found.  Did you install it?


Indeed there is no python-influxdb module install on my mgr node (CentOS 7 x64) but yum search can't 
find it with the following repos enabled:
repo id  repo 
name 
  status
Ceph/x86_64  Ceph 
packages for x86_64
Ceph-noarch  Ceph 
noarch packages
base/7/x86_64 
CentOS-7 - Base
ceph-source  Ceph 
source packages
epel/x86_64  Extra 
Packages for Enterprise Linux 7 - x86_64
extras/7/x86_64 
CentOS-7 - Extras
updates/7/x86_64 
CentOS-7 - Updates


Python version is 2.7.5.

Is 'pip install' the only way to go or there is still some option to have required python module via 
rpm? I wonder how other people deals with that issue?

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] restrict user access to certain rbd image

2018-02-05 Thread knawnd 
Thanks a lot who shared thoughts and own experience on that topic! It seems that Frédéric's input is 
exactly I've been looking for. Thanks Frédéric!


Jason Dillaman wrote on 02/02/18 19:24:

Concur that it's technically feasible by restricting access to
"rbd_id.", "rbd_header..",
"rbd_object_map..", and "rbd_data.." objects using
the prefix restriction in the OSD caps. However, this really won't
scale beyond a small number of images per user since every IO will
need to traverse the list of caps to verify the user can touch the
object.

On Fri, Feb 2, 2018 at 11:05 AM, Gregory Farnum  wrote:

I don't think it's well-integrated with the tooling, but check out the cephx
docs for the "prefix" level of access. It lets you grant access only to
objects whose name matches a prefix, which for rbd would be the rbd volume
ID (or name? Something easy to identify).
-Greg


On Fri, Feb 2, 2018 at 7:42 AM  wrote:


Hello!

I wonder if it's possible in ceph Luminous to manage user access to rbd
images on per image (but not
the whole rbd pool) basis?
I need to provide rbd images for my users but would like to disable their
ability to list all images
in a pool as well as to somehow access/use ones if a ceph admin didn't
authorize that.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com






___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] restrict user access to certain rbd image

2018-02-02 Thread knawnd 

Hello!

I wonder if it's possible in ceph Luminous to manage user access to rbd images on per image (but not 
the whole rbd pool) basis?
I need to provide rbd images for my users but would like to disable their ability to list all images 
in a pool as well as to somehow access/use ones if a ceph admin didn't authorize that.

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] problem with automounting cephfs on KVM VM boot

2018-01-31 Thread knawnd 

Hello!

I need to mount automatically cephfs on KVM VM boot .

I tried to follow recommendations mentioned at http://docs.ceph.com/docs/master/cephfs/fstab/ but in 
both cases (kernel mode or fuse) as well as by specifying mounting command in /etc/rc.local  it 
always fails to get mounted cephfs share on VM boot. There are lines in /var/log/messages like below:

...
Jan 31 17:56:54 10-93-221-35 systemd: Reached target Remote File Systems (Pre).
Jan 31 17:56:54 10-93-221-35 systemd: Starting Remote File Systems (Pre).
Jan 31 17:56:54 10-93-221-35 systemd: MESSAGE=Failed to check directory /mnt/cephfs: No such file or 
directory

Jan 31 17:56:54 10-93-221-35 systemd: Mounting /mnt/cephfs...
Jan 31 17:56:54 10-93-221-35 systemd: Started OpenSSH server daemon.
Jan 31 17:56:54 10-93-221-35 kernel: FS-Cache: Loaded
Jan 31 17:56:54 10-93-221-35 htcondor: Not changing GLOBAL_MAX_FDS (/proc/sys/fs/file-max): new 
value (32768) <= old value (400704).

Jan 31 17:56:54 10-93-221-35 kernel: Key type ceph registered
Jan 31 17:56:54 10-93-221-35 kernel: libceph: loaded (mon/osd proto 15/24)
Jan 31 17:56:54 10-93-221-35 mount: mount error 2 = No such file or directory
Jan 31 17:56:54 10-93-221-35 systemd: mnt-cephfsshare.mount mount process 
exited, code=exited status=2
Jan 31 17:56:54 10-93-221-35 kernel: FS-Cache: Netfs 'ceph' registered for 
caching
Jan 31 17:56:54 10-93-221-35 kernel: ceph: loaded (mds proto 32)
Jan 31 17:56:54 10-93-221-35 systemd-vconsole-setup: /usr/bin/setfont failed 
with error code 71.
Jan 31 17:56:54 10-93-221-35 systemd: Failed to mount /mnt/cephfs.
Jan 31 17:56:54 10-93-221-35 systemd: Dependency failed for Remote File Systems.
Jan 31 17:56:54 10-93-221-35 systemd: Job remote-fs.target/start failed with 
result 'dependency'.
Jan 31 17:56:54 10-93-221-35 systemd: Unit mnt-cephfsshare.mount entered failed 
state.

. or in case of fuse 

Jan 31 17:42:18 10-93-221-35 rc.local: fuse: bad mount point `/mnt/cephfs/': No 
such file or directory
Jan 31 17:42:18 10-93-221-35 rc.local: 2018-01-31 17:42:18.056464 7ff0f09b3040 -1 fuse_parse_cmdline 
failed.

Jan 31 17:42:18 10-93-221-35 rc.local: ceph-fuse[1043]: fuse failed to 
initialize
Jan 31 17:42:18 10-93-221-35 systemd: Started /etc/rc.d/rc.local Compatibility.


But /mnt/cephfs dir exists on the host:
 $ ls -al /mnt/
total 4
drwxr-xr-x. 2 root root 4096 Nov 15 16:12 cephfs

I tried various entries in /etc/fstab:
#none/mnt/cephfsfuse.ceph 
ceph.id=cephfsshare,ceph.conf=/etc/ceph/ceph.conf,ceph.client_mountpoint=/service/cephfsshare,_netdev,defaults 
 0 0


#192.168.1.11:6789,192.168.1.12:6789,192.168.1.13:6789:/service/cephfsshare  /mnt/cephfs 
   cephname=cephfsshare,secretfile=/root/ceph.cephfsshare.keyring,noatime,_netdev  0 0


192.168.1.11:6789:/service/cephfsshare /mnt/cephfsceph 
name=cephfsshare,secretfile=/root/ceph.cephfsshare.keyring,noatime,_netdev  0 0


In all cases I was able successfully mount cephfs share manually as soon as VM 
booted.

On the cephfs client  with CentOS 7 x64 I have

$ rpm -qa|grep -i ceph
ceph-fuse-12.2.2-0.el7.x86_64
ceph-release-1-1.el7.noarch
libcephfs2-12.2.2-0.el7.x86_64
python-cephfs-12.2.2-0.el7.x86_64
ceph-common-12.2.2-0.el7.x86_64

$ uname -r
4.15.0-1.el7.elrepo.x86_64

Ceph release is Luminous (12.2.2).

I would appreciate any help on that issue.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com