[
https://issues.apache.org/jira/browse/SOLR-13621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Bernstein updated SOLR-13621:
----------------------------------
Summary: Visual log parsing and data loading with Streaming Expressions
(was: Visual log parsing and data loading with for Streaming Expressions)
> Visual log parsing and data loading with Streaming Expressions
> --------------------------------------------------------------
>
> Key: SOLR-13621
> URL: https://issues.apache.org/jira/browse/SOLR-13621
> Project: Solr
> Issue Type: New Feature
> Components: streaming expressions
> Reporter: Joel Bernstein
> Assignee: Joel Bernstein
> Priority: Major
>
> Streaming Expressions and Math Expressions are now mature on the query side.
> This includes the ability to query, transform, analyze and visualize data.
> It's now time to build the data loading and log parsing capabilities to apply
> the full suite of mathematics and visualizations over log data and CSV files.
> The design is to have stream sources that read from a file system and stream
> decorators that parse different file and log formats. The data can be further
> transformed and joined with other data by stream decorators and sent to any
> Solr Cloud collection with the update Stream.
> This design also allows Streaming Expressions to perform regex filtering,
> aggregations, statistical analysis and visualization directly over CSV files
> and log files before the data is loaded to Solr. Because of Streaming
> Expressions built in paralyzation capabilities this allows Solr Cloud to
> behave like a massively parallel *grep* engine.
> It also allows users to visualize data using Apache Zeppelin as part of
> loading process, to make it easier to understand the data before it's loaded
> into an index.
> This ticket will track the sub-tickets for the different log formats that
> will be supported.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
