Re: [ceph-users] Ceph Mimic on Debian 9 Stretch
On 05/06/18 05:58, kefu chai wrote: > On Tue, Jun 5, 2018 at 6:13 AM, Paul Emmerich wrote: >> Hi, >> >> 2018-06-04 20:39 GMT+02:00 Sage Weil : >>> We'd love to build for stretch, but until there is a newer gcc for that >>> distro it's not possible. We could build packages for 'testing', but I'm >>> not sure if those will be usable on stretch. >> >> you can install gcc (and only gcc) from testing on Stretch to build Ceph for >> Stretch: >> >> echo 'deb http://ftp.de.debian.org/debian/ testing main' >> >> /etc/apt/sources.list >> echo 'APT::Default-Release "stable";' > >> /etc/apt/apt.conf.d/stable-as-default >> apt update >> apt install g++ -t testing >> >> That's how we are building our Debian packages. > thanks for sharing this, Paul ! does the built binary require any > runtime dependency offered the testing repo? if the answer is no, i > think we should offer the pre-built package for debian stable then. > > We have some debian developers in our team - let me see if we can make something happen with backporting gcc. It would be a shame for Ceph to be more cumbersome to install on Debian. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] GDPR encryption at rest
On 02/05/18 16:12, David Turner wrote: > I've heard conflicting opinions if GDPR requires data to be encrypted > at rest Encryption both in transit and at rest is part of data protection by design: it is about making sure that you have control over the data that you hold/are processing and that if you lose physical control over the storage medium (at rest) or the communication channel (in transit) that you do not also have a loss of control (a data breach). Encrypted data, whether it includes a personal data or not, is 'protected' secure data. GDPR doesn't particularly describe encryption but the ICO guidance does and in particular "Where appropriate, you should look to use measures such as pseudonymisation and encryption." We're currently working on a Ceph based Document Management System with object encryption which needs to comply with GDPR for users - and we're opting for encrypting everything! ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Object Gateway - Server Side Encryption
On 15/03/18 10:45, Vik Tara wrote: > > On 14/03/18 12:31, Amardeep Singh wrote: > >> Though I have now another issue because I am using Multisite setup >> with one zone for data and second zone for metadata with elastic >> search tier. >> >> http://docs.ceph.com/docs/master/radosgw/elastic-sync-module/ >> >> When document is encrypted the metadata is not pushed to >> elasticsearch and if document is uploaded without encryption it works >> fine. >> >> /2018-03-14 15:48:02.397490 7f0b4cbce700 20 >> cr:s=0x560a726c4000:op=0x560a7276e800:20RGWPutRESTResourceCRI15es_obj_metadataiE: >> operate() >> 2018-03-14 15:48:02.397492 7f0b4cbce700 20 >> cr:s=0x560a726c4000:op=0x560a7276e800:20RGWPutRESTResourceCRI15es_obj_metadataiE: >> operate() >> *2018-03-14 15:48:02.397633 7f0b4cbce700 20 sending request to >> http://192.168.95.60:9200/newbucket/object/ee560b67-c330-4fd0-af50-aefff93735d2.4163.1:testdocument:null* >> 2018-03-14 15:48:02.397653 7f0b4cbce700 20 register_request >> mgr=0x560a720d5d58 req_data->id=1759, easy_handle=0x560a7348a000 >> 2018-03-14 15:48:02.397666 7f0b4cbce700 20 run: stack=0x560a726c4000 >> is io blocked >> 2018-03-14 15:48:02.397685 7f0b4b3cb700 20 link_request >> req_data=0x560a727fae00 req_data->id=1758, easy_handle=0x560a733e6000 >> 2018-03-14 15:48:02.397725 7f0b4b3cb700 20 link_request >> req_data=0x560a72f31e00 req_data->id=1759, easy_handle=0x560a7348a000 >> 2018-03-14 15:48:02.398609 7f0b4b3cb700 10 receive_http_header >> 2018-03-14 15:48:02.398631 7f0b4b3cb700 10 received header:HTTP/1.1 >> 100 Continue >> 2018-03-14 15:48:02.398636 7f0b4b3cb700 10 received header:HTTP/1.1 >> 2018-03-14 15:48:02.398638 7f0b4b3cb700 10 receive_http_header >> 2018-03-14 15:48:02.398639 7f0b4b3cb700 10 received header: >> 2018-03-14 15:48:02.398659 7f0b4b3cb700 10 receive_http_header >> 2018-03-14 15:48:02.398661 7f0b4b3cb700 10 received header:HTTP/1.1 >> 100 Continue >> 2018-03-14 15:48:02.398662 7f0b4b3cb700 10 received header:HTTP/1.1 >> 2018-03-14 15:48:02.398663 7f0b4b3cb700 10 receive_http_header >> 2018-03-14 15:48:02.398664 7f0b4b3cb700 10 received header: >> 2018-03-14 15:48:02.443530 7f0b4b3cb700 10 receive_http_header >> *2018-03-14 15:48:02.443556 7f0b4b3cb700 10 received header:HTTP/1.1 >> 400 Bad Request* >> 2018-03-14 15:48:02.443563 7f0b4b3cb700 10 receive_http_header >> 2018-03-14 15:48:02.443565 7f0b4b3cb700 10 received header:Warning: >> 299 Elasticsearch-5.6.2-57e20f3 "Content type detection for rest >> requests is deprecated. Specify the content type using the >> [Content-Type] header." "Wed, 14 Mar 2018 10:17:35 GMT" >> 2018-03-14 15:48:02.443574 7f0b4b3cb700 10 receive_http_header >> 2018-03-14 15:48:02.443575 7f0b4b3cb700 10 received >> header:content-type: application/json; charset=UTF-8 >> 2018-03-14 15:48:02.443588 7f0b4b3cb700 10 receive_http_header >> 2018-03-14 15:48:02.443591 7f0b4b3cb700 10 received >> header:content-length: 374 >> 2018-03-14 15:48:02.443594 7f0b4b3cb700 10 receive_http_header >> 2018-03-14 15:48:02.443595 7f0b4b3cb700 10 received header: >> 2018-03-14 15:48:02.443663 7f0b4cbce700 20 >> cr:s=0x560a732f4d20:op=0x560a72fa8000:20RGWPutRESTResourceCRI15es_obj_metadataiE: >> operate() >> 2018-03-14 15:48:02.443675 7f0b4cbce700 5 failed to wait for op, >> ret=-22: PUT >> http://192.168.95.60:9200/newbucket/object/ee560b67-c330-4fd0-af50-aefff93735d2.4163.1:testdocument:null >> 2018-03-14 15:48:02.443754 7f0b4cbce700 20 >> cr:s=0x560a732f4d20:op=0x560a72fa8000:20RGWPutRESTResourceCRI15es_obj_metadataiE: >> operate() returned r=-22 >> 2018-03-14 15:48:02.443773 7f0b4cbce700 20 >> cr:s=0x560a732f4d20:op=0x560a7276c800:29RGWElasticHandleRemoteObjCBCR: >> operate() >> 2018-03-14 15:48:02.443787 7f0b4cbce700 20 >> cr:s=0x560a732f4d20:op=0x560a7276c800:29RGWElasticHandleRemoteObjCBCR: >> operate() returned r=-22 >> 2018-03-14 15:48:02.443791 7f0b4cbce700 20 >> cr:s=0x560a732f4d20:op=0x560a72efb800:27RGWElasticHandleRemoteObjCR: >> operate()/ > > This change 7 days ago looks like it deals with the encoding that ES > requires. > > https://github.com/ceph/ceph/pull/20707/files/13978bb28b7be809033bf24550b21ed2713ddc9b > > I'll make a build that incorporates this commit so you can test it :) After testing, it looks like that commit does not solve the issue with indexing encrypted objects - will raise a bug for this / move the discussion to the dev list. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Object Gateway - Server Side Encryption
On 14/03/18 12:31, Amardeep Singh wrote: > Though I have now another issue because I am using Multisite setup > with one zone for data and second zone for metadata with elastic > search tier. > > http://docs.ceph.com/docs/master/radosgw/elastic-sync-module/ > > When document is encrypted the metadata is not pushed to elasticsearch > and if document is uploaded without encryption it works fine. > > /2018-03-14 15:48:02.397490 7f0b4cbce700 20 > cr:s=0x560a726c4000:op=0x560a7276e800:20RGWPutRESTResourceCRI15es_obj_metadataiE: > operate() > 2018-03-14 15:48:02.397492 7f0b4cbce700 20 > cr:s=0x560a726c4000:op=0x560a7276e800:20RGWPutRESTResourceCRI15es_obj_metadataiE: > operate() > *2018-03-14 15:48:02.397633 7f0b4cbce700 20 sending request to > http://192.168.95.60:9200/newbucket/object/ee560b67-c330-4fd0-af50-aefff93735d2.4163.1:testdocument:null* > 2018-03-14 15:48:02.397653 7f0b4cbce700 20 register_request > mgr=0x560a720d5d58 req_data->id=1759, easy_handle=0x560a7348a000 > 2018-03-14 15:48:02.397666 7f0b4cbce700 20 run: stack=0x560a726c4000 > is io blocked > 2018-03-14 15:48:02.397685 7f0b4b3cb700 20 link_request > req_data=0x560a727fae00 req_data->id=1758, easy_handle=0x560a733e6000 > 2018-03-14 15:48:02.397725 7f0b4b3cb700 20 link_request > req_data=0x560a72f31e00 req_data->id=1759, easy_handle=0x560a7348a000 > 2018-03-14 15:48:02.398609 7f0b4b3cb700 10 receive_http_header > 2018-03-14 15:48:02.398631 7f0b4b3cb700 10 received header:HTTP/1.1 > 100 Continue > 2018-03-14 15:48:02.398636 7f0b4b3cb700 10 received header:HTTP/1.1 > 2018-03-14 15:48:02.398638 7f0b4b3cb700 10 receive_http_header > 2018-03-14 15:48:02.398639 7f0b4b3cb700 10 received header: > 2018-03-14 15:48:02.398659 7f0b4b3cb700 10 receive_http_header > 2018-03-14 15:48:02.398661 7f0b4b3cb700 10 received header:HTTP/1.1 > 100 Continue > 2018-03-14 15:48:02.398662 7f0b4b3cb700 10 received header:HTTP/1.1 > 2018-03-14 15:48:02.398663 7f0b4b3cb700 10 receive_http_header > 2018-03-14 15:48:02.398664 7f0b4b3cb700 10 received header: > 2018-03-14 15:48:02.443530 7f0b4b3cb700 10 receive_http_header > *2018-03-14 15:48:02.443556 7f0b4b3cb700 10 received header:HTTP/1.1 > 400 Bad Request* > 2018-03-14 15:48:02.443563 7f0b4b3cb700 10 receive_http_header > 2018-03-14 15:48:02.443565 7f0b4b3cb700 10 received header:Warning: > 299 Elasticsearch-5.6.2-57e20f3 "Content type detection for rest > requests is deprecated. Specify the content type using the > [Content-Type] header." "Wed, 14 Mar 2018 10:17:35 GMT" > 2018-03-14 15:48:02.443574 7f0b4b3cb700 10 receive_http_header > 2018-03-14 15:48:02.443575 7f0b4b3cb700 10 received > header:content-type: application/json; charset=UTF-8 > 2018-03-14 15:48:02.443588 7f0b4b3cb700 10 receive_http_header > 2018-03-14 15:48:02.443591 7f0b4b3cb700 10 received > header:content-length: 374 > 2018-03-14 15:48:02.443594 7f0b4b3cb700 10 receive_http_header > 2018-03-14 15:48:02.443595 7f0b4b3cb700 10 received header: > 2018-03-14 15:48:02.443663 7f0b4cbce700 20 > cr:s=0x560a732f4d20:op=0x560a72fa8000:20RGWPutRESTResourceCRI15es_obj_metadataiE: > operate() > 2018-03-14 15:48:02.443675 7f0b4cbce700 5 failed to wait for op, > ret=-22: PUT > http://192.168.95.60:9200/newbucket/object/ee560b67-c330-4fd0-af50-aefff93735d2.4163.1:testdocument:null > 2018-03-14 15:48:02.443754 7f0b4cbce700 20 > cr:s=0x560a732f4d20:op=0x560a72fa8000:20RGWPutRESTResourceCRI15es_obj_metadataiE: > operate() returned r=-22 > 2018-03-14 15:48:02.443773 7f0b4cbce700 20 > cr:s=0x560a732f4d20:op=0x560a7276c800:29RGWElasticHandleRemoteObjCBCR: > operate() > 2018-03-14 15:48:02.443787 7f0b4cbce700 20 > cr:s=0x560a732f4d20:op=0x560a7276c800:29RGWElasticHandleRemoteObjCBCR: > operate() returned r=-22 > 2018-03-14 15:48:02.443791 7f0b4cbce700 20 > cr:s=0x560a732f4d20:op=0x560a72efb800:27RGWElasticHandleRemoteObjCR: > operate()/ This change 7 days ago looks like it deals with the encoding that ES requires. https://github.com/ceph/ceph/pull/20707/files/13978bb28b7be809033bf24550b21ed2713ddc9b I'll make a build that incorporates this commit so you can test it :) ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
