Re: [openstack-dev] [Ceilometer] Question of necessary queries for Event implemented on HBase
Hello, For today I have prepared a spec for events feature in HBase https://review.openstack.org/#/c/96417/, and it was successfully merged. In spec was described the details of event features in HBase. Yours, Igor D. On Wed, May 21, 2014 at 3:03 PM, Dmitriy Ukhlov dukh...@mirantis.com wrote: Hello Igor, Sounds reasonable. On 05/21/2014 02:38 PM, Igor Degtiarov wrote: Hi, I have found that filter model for Events has mandatory parameters start_time and end_time of the events period. So, it seems that structure for rowkey as ''timestamp + event_id will be more suitable. -- Best regards, Dmitriy Ukhlov Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Question of necessary queries for Event implemented on HBase
Hi, I have found that filter model for Events has mandatory parameters start_time and end_time of the events period. So, it seems that structure for rowkey as ''timestamp + event_id will be more suitable. I have started to work on bp https://blueprints.launchpad.net/ceilometer/+spec/hbase-events-feature Partial implementation of events into HBase is available by link https://review.openstack.org/#/c/91408/ By now it was added record method that writes events into HBase and get method with filtering by period of generation time of events. Sincerely, Igor D. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Question of necessary queries for Event implemented on HBase
Hello Igor, Sounds reasonable. On 05/21/2014 02:38 PM, Igor Degtiarov wrote: Hi, I have found that filter model for Events has mandatory parameters start_time and end_time of the events period. So, it seems that structure for rowkey as ''timestamp + event_id will be more suitable. -- Best regards, Dmitriy Ukhlov Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Question of necessary queries for Event implemented on HBase
Hello Dmitriy! Of course event_id could identify row. Actually Row structure now is a question for discuss. The main feature of HBase that data is stored in sorted rows. To filter events by time interval we will need to scan all stored data, when rowkey is started with event_id. Instead of adding timestamp in rowkey, we could set timestamps as a additional column value or qualifier inside row. Another decision is to start rowkey with timestamp + event_id. In this case we will have events stored by timestamps in HBase. What type of rowkey will give more efficiency is a question. On Wed, Apr 30, 2014 at 4:36 PM, Dmitriy Ukhlov dukh...@mirantis.comwrote: Hello Igor! Could you clarify, please, Why do we need event_id + reversed_timestamp row key? Isn't event_id identify row? On Tue, Apr 29, 2014 at 11:08 AM, Igor Degtiarov idegtia...@mirantis.comwrote: Hi, everybody. I’ve started to work on implementation of Event in ceilometer on HBase backend in the edges of blueprint https://blueprints.launchpad.net/ceilometer/+spec/hbase-events-feature By now Events has been implemented only in SQL. You know, using SQL we can build any query we need. With HBase it is another story. The data structure is built basing on queries we need, so to construct the structure of Event on HBase, it is very important to answer the question what queries should be implemented to retrieve events from storage. I registered bp https://blueprints.launchpad.net/ceilometer/+spec/hbase-events-structurefor discussion Events structure in HBase. For today it is prepared preliminary structure of Events in HBase: table: Events - rowkey: event_id + reversed_timestamp - column: event_type = string with description of event - [list of columns: trait_id + trait_desc + trait_type= trait_data] Structure that is proposed will support next queries: - event’s generation time - event id - event type - trait: id, description, type Any thoughts about additional queries that are necessary for Events. I’ll publish the patch with current implementation soon. Sincerely, Igor Degtiarov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best regards, Dmitriy Ukhlov Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Ceilometer] Question of necessary queries for Event implemented on HBase
Hello Igor! Could you clarify, please, Why do we need event_id + reversed_timestamp row key? Isn't event_id identify row? On Tue, Apr 29, 2014 at 11:08 AM, Igor Degtiarov idegtia...@mirantis.comwrote: Hi, everybody. I’ve started to work on implementation of Event in ceilometer on HBase backend in the edges of blueprint https://blueprints.launchpad.net/ceilometer/+spec/hbase-events-feature By now Events has been implemented only in SQL. You know, using SQL we can build any query we need. With HBase it is another story. The data structure is built basing on queries we need, so to construct the structure of Event on HBase, it is very important to answer the question what queries should be implemented to retrieve events from storage. I registered bp https://blueprints.launchpad.net/ceilometer/+spec/hbase-events-structurefor discussion Events structure in HBase. For today it is prepared preliminary structure of Events in HBase: table: Events - rowkey: event_id + reversed_timestamp - column: event_type = string with description of event - [list of columns: trait_id + trait_desc + trait_type= trait_data] Structure that is proposed will support next queries: - event’s generation time - event id - event type - trait: id, description, type Any thoughts about additional queries that are necessary for Events. I’ll publish the patch with current implementation soon. Sincerely, Igor Degtiarov ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Best regards, Dmitriy Ukhlov Mirantis Inc. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
