Great, thanks, Josh! (no worries on timing). A longer term nagging question, that still seems relevant -- Are there any specific reasons that you've anchored on (what seems to only be) Elastic as a backend? Naively, it seems could view Flagon as a way to produce data, that then could go through a message queue and to one or many different backends depending on desired consumption patterns. Is this more a matter of convenience - Elastic suffices and therefore devote attention elsewhere -- or because Elastic does something particularly unique given the context of Flagon.
Cheers, Austin On Sat, Jan 25, 2020 at 5:15 PM Joshua Poore <[email protected]> wrote: > > Austin—good to hear from you. > > Apologies for the delay. Have been working with users last few days and > wanted to give your email the appropriate attention—some good questions. > > Replies inline > > > On Jan 22, 2020, at 4:31 PM, Austin Bennett <[email protected]> > > wrote: > > > > Hi Flagon, > > > > Looking for info: > > > > * I don't see much on analysing the data that gets collected. Very > > interested in understanding the types of insights that people are > > deriving. Pointers very welcome (esp. to specific use cases). > > Apologies if missing something obvious… > > Most of our users are interested purely in business analytics. Mostly we get > requests for guidance on Funnel’s and Heatmaps. We have a crude funnel mocked > in a Kibana dashboard which you can find in our Business Analytics Dashboard > here: > https://github.com/apache/incubator-flagon/tree/master/docker/kibana/6.8.2/Saved%20Objects > > <https://github.com/apache/incubator-flagon/tree/master/docker/kibana/6.8.2/Saved%20Objects> > > Of course you’ll need to set up an Elastic instance to use it. You can find a > sandbox ELK project in the parent directory: > https://github.com/apache/incubator-flagon/tree/master/docker > <https://github.com/apache/incubator-flagon/tree/master/docker> > > In the Kibana visualizations and dashboards you’ll find a number of other viz > elements that derive directly from user asks. You’ll need to customize a > little bit given you’ll have different values from your logs, but there is a > LOT of content there. > > We are working on a python package, too, for more advanced behavioral > analytics. We haven’t been able to devote much time to it as we’ve been > working on tightening up UserALE, but we’ve done some WIP experiments with an > analytic pipeline (seriously, very much a WIP): > https://github.com/apache/incubator-flagon-distill/tree/distill_toolkit_refactor/WIP-distill_examples > > <https://github.com/apache/incubator-flagon-distill/tree/distill_toolkit_refactor/WIP-distill_examples> > > > > * Additionally, is JIRA the best place to understand pinpoints and > > areas for improvement? Esp. upon recognizing value of using the data, > > would be very interested in using my backend/data-eng experience to > > help develop and increase the stability of system for high-scale > > production use (sounds like successfully used places already). > > For now, yes, JIRA is the best place. We really do keep track of things well > in JIRA. Most of the tickets in there are backlog ideas, wishes, task. We > pull from the backlog into releases and track that way. In the very near > term, we’ll be ditching JIRA and moving to Git Projects. JIRA seems to be a > wall for our users. For now, feel free to jump in and add tickets labeled by > component: > > For UserALE and log pipeline add tickets to: > https://issues.apache.org/jira/browse/FLAGON-483?jql=project%20%3D%20FLAGON%20AND%20component%20%3D%20UserALE.js > > <https://issues.apache.org/jira/browse/FLAGON-483?jql=project%20=%20FLAGON%20AND%20component%20=%20UserALE.js> > > For Analytical asks, add tickets to: > https://issues.apache.org/jira/browse/FLAGON-464?jql=project%20%3D%20FLAGON%20AND%20component%20%3D%20Distill > > <https://issues.apache.org/jira/browse/FLAGON-464?jql=project%20=%20FLAGON%20AND%20component%20=%20Distill> > > For Stack/back-end (ELK) improvements, add tickets to: > https://issues.apache.org/jira/browse/FLAGON-466?jql=project%20%3D%20FLAGON%20AND%20component%20%3D%20stack > > <https://issues.apache.org/jira/browse/FLAGON-466?jql=project%20=%20FLAGON%20AND%20component%20=%20stack> > > Every commit we make, traces to a ticket. However, JIRA feels unwieldy for a > lot of users. Feel free to add to JIRA, I will migrate to GIt Projects, when > we make our move. > > > > * Lastly, how would y'all characterize the docs vs state of the code > > (is it believed things are well documented and not much drift -- docs > > tend to lag behind). The repository is not changing rapidly, so seems > > like would be a slow drift if it has occurred. > > Lately, we’ve been burning down adoption issues for UserALE.js. We’re > actually in the final throes of testing and documenting for UserALE.js v > 2.1.0. This is actually a largish release: > > 1. Webpack/Module Bundler support > 2. SessionStorage Support > 3. custom auth headers > 4. comprehensive custom log support > 5. other things > > We’ve been pretty active there: > https://github.com/apache/incubator-flagon-useralejs/tree/FLAGON-469 > <https://github.com/apache/incubator-flagon-useralejs/tree/FLAGON-469> > > As per docs, our workflow generally follows a pattern such that all READMEs > are updated prior to pushing new release candidates. Once released, we update > docs on the website at the same time that we update our website to reflect > new release links, etc. > > Currently, docs on UserALE.js — > http://flagon.incubator.apache.org/docs/useralejs/ > <http://flagon.incubator.apache.org/docs/useralejs/> — and the stack — > http://flagon.incubator.apache.org/docs/stack/ > <http://flagon.incubator.apache.org/docs/stack/> — are about up to date, or > lag by a patch release. The website is generally a good source for guides and > considerations, but our readmes are pretty solid. We’re always willing to > field asks for more specific documentation. > > We would love some help on our back-end. We’re lagging a bit as we’ve not yet > moved our examples up from ELK 6.8—>7.0. That’s on the near term slate, as > well as more documentation on clustering and indexing options. But, we’d love > PRs. > > > > > Thanks, > > Austin > > Thank You, Austin! I hope this is helpful! >
