On Wed, Nov 29, 2017 at 12:43 AM, <[email protected]> wrote: > Hi Yehuda. > > Are there any examples (doc's, blog posts, ...): > - how to use that "framework" and especially for the "callbacks"
There's a minimal sync module implementation that does nothing other than write a debug log for each sync event: https://github.com/ceph/ceph/blob/master/src/rgw/rgw_sync_module_log.cc Docs-wise, the elasticsearch luminous blog has a concrete config example that can be applied to other sync modules: http://ceph.com/rgw/new-luminous-rgw-metadata-search/ > - for the latest "Metasearch" feature / usage with a S3 client/tools like > CyberDuck, s3cmd, AWSCLI or at least boto3? > - i.e. is an external ELK still needed or is this somehow included in RGW > now? > You still need an external elasticsearch server, it's not part of ceph. However, search can be done by sending requests through rgw's RESTful api. We have a test that uses boto to generate such requests, but might not be exactly what you're looking for: https://github.com/ceph/ceph/blob/master/src/test/rgw/rgw_multi/zone_es.py Yehuda > Thanks & regards > > > Gesendet: Dienstag, 28. November 2017 um 13:52 Uhr > Von: "Yehuda Sadeh-Weinraub" <[email protected]> > An: "Sean Purdy" <[email protected]> > Cc: "[email protected]" <[email protected]> > Betreff: Re: [ceph-users] S3 object notifications > rgw has a sync modules framework that allows you to write your own > sync plugins. The system identifies objects changes and triggers > callbacks that can then act on those changes. For example, the > metadata search feature that was added recently is using this to send > objects metadata into elasticsearch for indexing. > > Yehuda > > On Tue, Nov 28, 2017 at 2:22 PM, Sean Purdy <[email protected]> wrote: >> Hi, >> >> >> http://docs.ceph.com/docs/master/radosgw/s3/ says that S3 object >> notifications are not supported. I'd like something like object >> notifications so that we can backup new objects in realtime, instead of >> trawling the whole object list for what's changed. >> >> Is there anything similar I can use? I've found Spreadshirt's haproxy fork >> which traps requests and updates redis - >> https://github.com/spreadshirt/s3gw-haproxy[https://github.com/spreadshirt/s3gw-haproxy][https://github.com/spreadshirt/s3gw-haproxy[https://github.com/spreadshirt/s3gw-haproxy]] >> Anybody used that? >> >> >> Thanks, >> >> Sean Purdy >> _______________________________________________ >> ceph-users mailing list >> [email protected] >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com[http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com][http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com[http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com]] > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com[http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com][http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com[http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com]] > _______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
