FYI, I hijacked Ted's DRILL-3516 [1] to create a number of subtasks to improve UDF experience. The issue really falls into two categories:
- Better documentation - Less sharp edges [1] https://issues.apache.org/jira/browse/DRILL-3516 On Mon, Jul 20, 2015 at 9:08 AM, Jim Scott <[email protected]> wrote: > To iterate on Ted's point... and to note, this is not disagreement from > Jacques point... > > There are a core set of capabilities within Drill that need to have > fantastic documentation... > > 1. User documentation for how to write queries - I personally feel this is > very good and continues to get better. Is it perfect? No, but I wouldn't > complain about this. > 2. Developer documentation for how to extend drill > a. UDF > b. Storage Plugins > c. Formats > > I believe that #2 is the one we should probably think the hardest about. > The easier we make it for the community to contribute the faster drill will > gain adoption and the better drill will be as a product. We should take > action to take down the walls preventing people from contributing code. > Enabling pull requests from github is an amazingly good step in the right > direction. Having better documentation for #2 must also be taken care of. > > > On Mon, Jul 20, 2015 at 10:59 AM, Jacques Nadeau <[email protected]> > wrote: > > > Ted, > > > > - I've heard from a number of people that Drill's documentation is one of > > the best of all Hadoop-related projects > > - There are--and will always be--places where the documentation could be > > improved. > > - The people working on documentation have always been very responsive to > > feedback and that shows in the docs > > > > It is completely natural that as people use the documentation, they will > > identify areas for improvement. The community lists exists precisely to > > help people like Stefan. Your characterization of this as problematic > > rather than expected confuses me. Of course, we'd love to have perfect > > documentation (and bug-free code) from the start. However, we all know > > that reality intervenes. As such, making sure the process is in place is > > central to success. Here is my understanding of what happened: > > > > 1. Someone tried to do something. > > 2. They reviewed the documentation > > 3. They hit issues or were confused. > > 4. They asked for help. > > 5. They received help. > > 6. They completed their task. > > 7. They provided feedback to incorporate into the documentation, > > identifying what was missing. > > 8. The feedback is incorporated into the documentation. (pending) > > > > This pattern has occurred previously and will continue to occur. As long > > as the community continues to be responsive to people's challenges, we > are > > in good shape. We know that sometimes people who are not as tenacious as > > Stefan will stop at #3 above. We can only do what we can do. Let's > > continue to improve things so that this becomes less likely over time. > > > > Jacques > > > > > > > > > > On Mon, Jul 20, 2015 at 8:32 AM, Ted Dunning <[email protected]> > > wrote: > > > > > We just had Stefan over on the user@drill list stick with us through > > thick > > > and thin to get a very simple UDF up. He isn't a dolt by any means, > but > > > here is his signature on email describing his state of mind: > > > > > > Regards, > > > > -Stefan (*not the happiest camper*) > > > > > > > > > The only reason that this guy is still with us is that Jim Bates > > exchanged > > > quite a number of emails. The clear reason that he had problems is lack > > of > > > documentation and thus understanding of how to write a UDF. > > > > > > The current state of affairs is that it is unnecessarily hard to > > contribute > > > to Drill. This is clearly having a deleterious effect on building the > > > community of contributors. We need to fix this. > > > > > > There is a place on the web site that describes how to write a UDF, but > > it > > > is clearly failing to explain the big picture of what is happening and > it > > > doesn't highlight the fact that you aren't really writing Java code > when > > > writing a Drill UDF, but instead are using Java to write in a more > > limited > > > language. > > > > > > Stefan's recent experience should be a perfect opportunity to improve > > this > > > documentation by describing the process better. > > > > > > > > > -- > *Jim Scott* > Director, Enterprise Strategy & Architecture > +1 (347) 746-9281 > > <http://www.mapr.com/> > [image: MapR Technologies] <http://www.mapr.com> > > Now Available - Free Hadoop On-Demand Training > < > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > > >
