Hi Arina,
Thanks for launching this discussion. A few minor suggestions.
The developers have done a fantastic job stabilizing and improving Drill's core
functionality. Now the opportunity is to expand the use cases for Drill so that
it gets wider adoption within the community. Drill competes for mindshare with
Impala, Presto, Hive, Spark and others. A key differentiator for Drill can be
the ability to extend the core and integrate Drill into user applications. Of
these tools, only Spark has a fully ostensible model. Can Drill provide some of
the flexibility that has powered Spark to success?
1. You mentioned the metastore is under active investigation. Anything yet to
share? Didn't see any activity on the JIRA ticket. Metadata is a key gap in
Drill. Simply adding a Hive-like metastore would repeat the very errors that
Drill was meant to address. Maybe we can toss around ideas for a metadata API
that provides greater flexibility.
2. Users can extend the core with custom UDFs, storage engines, formats and so
on. At present, the code to do this is rather hard to write, debug and
maintain. Is there value in streamlining those interfaces so that a wider
audience can extend Drill for their specific needs?
3. Similarly, we've seen interest in integrating Drill with other systems,
which suggests an opportunity for improved APIs. Ability to associate options,
defaults and restrictions with users. Ability to use the REST API for larger
data sets and with stateful session options. And so on.
Such extensions are best guided by user demands: what can Drill provide for
production applications to enable simpler/faster/more complete integration?
Thanks,
- Paul
On Monday, August 13, 2018, 5:42:08 AM PDT, Arina Ielchiieva
<[email protected]> wrote:
Hi all,
as a new PMC Chair I would like to thank users for choosing and using
Apache Drill and contributors / committers for making improvements and
fixes. Recently Apache Drill 1.14 was released bundled up with many
improvements and new features. Please feel free to try it out and share
your experience. As always we would love to hear your success stories of
using Apache Drill.
Also I encourage users to share any problems found in Drill, as well as any
suggestions for future improvements. Feel free to start discussion on the
mailing list and then file a Jira with the summary. Contributions are
always welcome: minor, major, doc improvements or grammar fixes. Just file
a Jira and open the PR. Do not hesitate to ping developers on the mailing
list if PR is not being timely reviewed.
Latest project reports show:
Apache Drill project has healthy release schedule, each release includes
lots of features.
Mailing list (user / dev) are getting substantial support from the active
developers, including Stackoverflow and Twitter.
New committers are added on the steady basis.
Overall project is growing and moving forward. There have been discussions
about Drill 2.0 last year and currently Drill metastore feature is under
active investigation which might the breaking change for 2.0.
Please feel free to reply to this email with your comments / concerns /
ideas about current project state.
Kind regards,
Arina
