Re: Apache Phoenix Tracing Dashboard-jira issue PHOENIX 1118
Ayola, Did you apply to GSoC with this? The submissions window is now closed! Lewis On Wednesday, April 22, 2015, Ayola Jayamaha raphaelan...@gmail.com wrote: Hi All, I'm a University Student applying for the GSOC Project [1] Dashboard for tracing information in Phoenix. In Apache Phoenix, HTrace is used to provide request level trace information. Below are the trace information found in Phoenix Table. - trace_id - parent_id - span_id - description - start_time - end_time - hostname - tags.count - annotations.count HTrace library is integrated with HBase tracing utilities. And HTrace latest release 3.2.0 will include many visualization improvements[2]. You can find the Mock Up UI [3] which gives advance search capabilities for trace attributes and slider functionality[4] to view the trace for the selected time period with zooming capabilities and parent trace info on display. A web UI is the solution proposed and a dashboard will show the required information. Can the Phoenix SYSTEM.TRACING_STATS table be made available in the htrace web gui? Which will further cater for the needs of visualizing the trace info as the Mock Up UIs suggest. Pagination will be tackled in the visualization of the tracing. [1] https://issues.apache.org/jira/browse/PHOENIX-1118 [2] http://mail-archives.apache.org/mod_mbox/htrace-dev/201504.mbox/%3cca+qbeuny9f9zloutoiefb9lekzvbkmkwzrjw52_5-u8jka+...@mail.gmail.com%3E [3] https://issues.apache.org/jira/secure/attachment/12726864/MockUp2-AdvanceSearch.png [4] https://issues.apache.org/jira/secure/attachment/12726863/MockUp1-TimeSlider.png -- Best Regards, Ayola Jayamaha http://ayolajayamaha.blogspot.com/ -- *Lewis*
Re: Apache Phoenix Tracing Dashboard-jira issue PHOENIX 1118
Hi Lewis, Yes I already submitted the proposal. On Wed, Apr 22, 2015 at 7:51 PM, Lewis John Mcgibbney lewis.mcgibb...@gmail.com wrote: Ayola, Did you apply to GSoC with this? The submissions window is now closed! Lewis On Wednesday, April 22, 2015, Ayola Jayamaha raphaelan...@gmail.com wrote: Hi All, I'm a University Student applying for the GSOC Project [1] Dashboard for tracing information in Phoenix. In Apache Phoenix, HTrace is used to provide request level trace information. Below are the trace information found in Phoenix Table. - trace_id - parent_id - span_id - description - start_time - end_time - hostname - tags.count - annotations.count HTrace library is integrated with HBase tracing utilities. And HTrace latest release 3.2.0 will include many visualization improvements[2]. You can find the Mock Up UI [3] which gives advance search capabilities for trace attributes and slider functionality[4] to view the trace for the selected time period with zooming capabilities and parent trace info on display. A web UI is the solution proposed and a dashboard will show the required information. Can the Phoenix SYSTEM.TRACING_STATS table be made available in the htrace web gui? Which will further cater for the needs of visualizing the trace info as the Mock Up UIs suggest. Pagination will be tackled in the visualization of the tracing. [1] https://issues.apache.org/jira/browse/PHOENIX-1118 [2] http://mail-archives.apache.org/mod_mbox/htrace-dev/201504.mbox/%3cca+qbeuny9f9zloutoiefb9lekzvbkmkwzrjw52_5-u8jka+...@mail.gmail.com%3E [3] https://issues.apache.org/jira/secure/attachment/12726864/MockUp2-AdvanceSearch.png [4] https://issues.apache.org/jira/secure/attachment/12726863/MockUp1-TimeSlider.png -- Best Regards, Ayola Jayamaha http://ayolajayamaha.blogspot.com/ -- *Lewis* -- Best Regards, Ayola Jayamaha http://ayolajayamaha.blogspot.com/
Re: Apache Phoenix Tracing Dashboard-jira issue PHOENIX 1118
bq. So a web UI in HTrace itself would help more people than a web UI in just Phoenix You trying to poach our GSoC student, Colin? :-) The end goal is to provide a UI to visualize performance characteristics of an HBase cluster running Phoenix queries. If the information can't be related back to a query, then it's likely not going to be successful. I suspect Ayola's question is geared more around whether or not there's code she can adapt/borrow to get a headstart. Thanks, James On Wed, Apr 22, 2015 at 12:01 PM, Colin P. McCabe cmcc...@apache.org wrote: Hi Nishani, It's great to see more interest in HTrace and Phoenix! One thing I think you might be interested in is that, as Lewis and others have mentioned, there is a project in the works to come up with a web UI for HTrace. Masatake, May, Abe, and myself had a meeting about the HTrace GUI a few weeks ago, and came up with some ideas. Keep in mind that Phoenix is just one project that uses HTrace. We also have HDFS, HBase, Accumulo, and others. So a web UI in HTrace itself would help more people than a web UI in just Phoenix. There is a lot of code to be written on the HTrace GUI... what we have now is just a beginning, I think. I think it would be a great project if you're interested. best, Colin On Wed, Apr 22, 2015 at 1:52 AM, Ayola Jayamaha raphaelan...@gmail.com wrote: Hi All, I'm a University Student applying for the GSOC Project [1] Dashboard for tracing information in Phoenix. In Apache Phoenix, HTrace is used to provide request level trace information. Below are the trace information found in Phoenix Table. - trace_id - parent_id - span_id - description - start_time - end_time - hostname - tags.count - annotations.count HTrace library is integrated with HBase tracing utilities. And HTrace latest release 3.2.0 will include many visualization improvements[2]. You can find the Mock Up UI [3] which gives advance search capabilities for trace attributes and slider functionality[4] to view the trace for the selected time period with zooming capabilities and parent trace info on display. A web UI is the solution proposed and a dashboard will show the required information. Can the Phoenix SYSTEM.TRACING_STATS table be made available in the htrace web gui? Which will further cater for the needs of visualizing the trace info as the Mock Up UIs suggest. Pagination will be tackled in the visualization of the tracing. [1] https://issues.apache.org/jira/browse/PHOENIX-1118 [2] http://mail-archives.apache.org/mod_mbox/htrace-dev/201504.mbox/%3cca+qbeuny9f9zloutoiefb9lekzvbkmkwzrjw52_5-u8jka+...@mail.gmail.com%3E [3] https://issues.apache.org/jira/secure/attachment/12726864/MockUp2-AdvanceSearch.png [4] https://issues.apache.org/jira/secure/attachment/12726863/MockUp1-TimeSlider.png -- Best Regards, Ayola Jayamaha http://ayolajayamaha.blogspot.com/
Re: Apache Phoenix Tracing Dashboard-jira issue PHOENIX 1118
On Wed, Apr 22, 2015 at 1:16 PM, James Taylor jamestay...@apache.org wrote: On Wed, Apr 22, 2015 at 12:52 PM, Stack st...@duboce.net wrote: On Wed, Apr 22, 2015 at 12:41 PM, James Taylor jamestay...@apache.org wrote: bq. So a web UI in HTrace itself would help more people than a web UI in just Phoenix You trying to poach our GSoC student, Colin? :-) The end goal is to provide a UI to visualize performance characteristics of an HBase cluster running Phoenix queries. Where are you thinking this UI should live James? (Where are you thinking the traces should reside? Would you want to query the traces table using Phoenix?) UI would live in Phoenix and be part of the distro. You'd likely want to be able to issue queries (maybe embed sqlline or some other SQL client). I think it'd be Phoenix/SQL-specific, but it's possible that shared UI components could live in HTrace. Yes, we have a custom sink that persists the trace information into a Phoenix table to allow adhoc querying. The idea would be that the UI would query that table. See http://phoenix.apache.org/tracing.html. You can play around with this today - it's available in our 4.3.1 release (including turning tracing on and off through sqlline). So, for this to work on any random cluster, HDFS, HBase, and Phoenix will all be configured to write traces to a Phoenix table. The UI will do (or does) SQL against this table or does it use HBase APIs? You would not be interested in deferring 'trace' to an external system? You are more interested in leveraging bits and pieces of htrace to host your own? Thanks, St.Ack
Re: Apache Phoenix Tracing Dashboard-jira issue PHOENIX 1118
On Wed, Apr 22, 2015 at 12:52 PM, Stack st...@duboce.net wrote: On Wed, Apr 22, 2015 at 12:41 PM, James Taylor jamestay...@apache.org wrote: bq. So a web UI in HTrace itself would help more people than a web UI in just Phoenix You trying to poach our GSoC student, Colin? :-) The end goal is to provide a UI to visualize performance characteristics of an HBase cluster running Phoenix queries. Where are you thinking this UI should live James? (Where are you thinking the traces should reside? Would you want to query the traces table using Phoenix?) UI would live in Phoenix and be part of the distro. You'd likely want to be able to issue queries (maybe embed sqlline or some other SQL client). I think it'd be Phoenix/SQL-specific, but it's possible that shared UI components could live in HTrace. Yes, we have a custom sink that persists the trace information into a Phoenix table to allow adhoc querying. The idea would be that the UI would query that table. See http://phoenix.apache.org/tracing.html. You can play around with this today - it's available in our 4.3.1 release (including turning tracing on and off through sqlline). If the information can't be related back to a query, then it's likely not going to be successful. In HTrace, there is a means of annotating traces; phoenix could write the query as annotation on the trace it had provoked. Would a UI that could 'front' these query annotations suffice? Yes, we already annotate the traces with the query (it's in the DESCRIPTION column of that Phoenix table). See above, though - I think it's likely that the UI would be Phoenix-specific, but there may be opportunities for code sharing at the UI widget level. Thanks, St.Ack I suspect Ayola's question is geared more around whether or not there's code she can adapt/borrow to get a headstart. Thanks, James On Wed, Apr 22, 2015 at 12:01 PM, Colin P. McCabe cmcc...@apache.org wrote: Hi Nishani, It's great to see more interest in HTrace and Phoenix! One thing I think you might be interested in is that, as Lewis and others have mentioned, there is a project in the works to come up with a web UI for HTrace. Masatake, May, Abe, and myself had a meeting about the HTrace GUI a few weeks ago, and came up with some ideas. Keep in mind that Phoenix is just one project that uses HTrace. We also have HDFS, HBase, Accumulo, and others. So a web UI in HTrace itself would help more people than a web UI in just Phoenix. There is a lot of code to be written on the HTrace GUI... what we have now is just a beginning, I think. I think it would be a great project if you're interested. best, Colin On Wed, Apr 22, 2015 at 1:52 AM, Ayola Jayamaha raphaelan...@gmail.com wrote: Hi All, I'm a University Student applying for the GSOC Project [1] Dashboard for tracing information in Phoenix. In Apache Phoenix, HTrace is used to provide request level trace information. Below are the trace information found in Phoenix Table. - trace_id - parent_id - span_id - description - start_time - end_time - hostname - tags.count - annotations.count HTrace library is integrated with HBase tracing utilities. And HTrace latest release 3.2.0 will include many visualization improvements[2]. You can find the Mock Up UI [3] which gives advance search capabilities for trace attributes and slider functionality[4] to view the trace for the selected time period with zooming capabilities and parent trace info on display. A web UI is the solution proposed and a dashboard will show the required information. Can the Phoenix SYSTEM.TRACING_STATS table be made available in the htrace web gui? Which will further cater for the needs of visualizing the trace info as the Mock Up UIs suggest. Pagination will be tackled in the visualization of the tracing. [1] https://issues.apache.org/jira/browse/PHOENIX-1118 [2] http://mail-archives.apache.org/mod_mbox/htrace-dev/201504.mbox/%3cca+qbeuny9f9zloutoiefb9lekzvbkmkwzrjw52_5-u8jka+...@mail.gmail.com%3E [3] https://issues.apache.org/jira/secure/attachment/12726864/MockUp2-AdvanceSearch.png [4] https://issues.apache.org/jira/secure/attachment/12726863/MockUp1-TimeSlider.png -- Best Regards, Ayola Jayamaha http://ayolajayamaha.blogspot.com/
Apache Phoenix Tracing Dashboard-jira issue PHOENIX 1118
Hi All, I'm a University Student applying for the GSOC Project [1] Dashboard for tracing information in Phoenix. In Apache Phoenix, HTrace is used to provide request level trace information. Below are the trace information found in Phoenix Table. - trace_id - parent_id - span_id - description - start_time - end_time - hostname - tags.count - annotations.count HTrace library is integrated with HBase tracing utilities. And HTrace latest release 3.2.0 will include many visualization improvements[2]. You can find the Mock Up UI [3] which gives advance search capabilities for trace attributes and slider functionality[4] to view the trace for the selected time period with zooming capabilities and parent trace info on display. A web UI is the solution proposed and a dashboard will show the required information. Can the Phoenix SYSTEM.TRACING_STATS table be made available in the htrace web gui? Which will further cater for the needs of visualizing the trace info as the Mock Up UIs suggest. Pagination will be tackled in the visualization of the tracing. [1] https://issues.apache.org/jira/browse/PHOENIX-1118 [2] http://mail-archives.apache.org/mod_mbox/htrace-dev/201504.mbox/%3cca+qbeuny9f9zloutoiefb9lekzvbkmkwzrjw52_5-u8jka+...@mail.gmail.com%3E [3] https://issues.apache.org/jira/secure/attachment/12726864/MockUp2-AdvanceSearch.png [4] https://issues.apache.org/jira/secure/attachment/12726863/MockUp1-TimeSlider.png -- Best Regards, Ayola Jayamaha http://ayolajayamaha.blogspot.com/
