Agree with points made by Jacques. Let's create the JIRA as/when they come up. And will have the Drill team to address them.
On Mon, Jul 20, 2015 at 9:18 AM, Jacques Nadeau <[email protected]> wrote: > FYI, I hijacked Ted's DRILL-3516 [1] to create a number of subtasks to > improve UDF experience. The issue really falls into two categories: > > - Better documentation > - Less sharp edges > > [1] https://issues.apache.org/jira/browse/DRILL-3516 > > On Mon, Jul 20, 2015 at 9:08 AM, Jim Scott <[email protected]> wrote: > >> To iterate on Ted's point... and to note, this is not disagreement from >> Jacques point... >> >> There are a core set of capabilities within Drill that need to have >> fantastic documentation... >> >> 1. User documentation for how to write queries - I personally feel this is >> very good and continues to get better. Is it perfect? No, but I wouldn't >> complain about this. >> 2. Developer documentation for how to extend drill >> a. UDF >> b. Storage Plugins >> c. Formats >> >> I believe that #2 is the one we should probably think the hardest about. >> The easier we make it for the community to contribute the faster drill >> will >> gain adoption and the better drill will be as a product. We should take >> action to take down the walls preventing people from contributing code. >> Enabling pull requests from github is an amazingly good step in the right >> direction. Having better documentation for #2 must also be taken care of. >> >> >> On Mon, Jul 20, 2015 at 10:59 AM, Jacques Nadeau <[email protected]> >> wrote: >> >> > Ted, >> > >> > - I've heard from a number of people that Drill's documentation is one >> of >> > the best of all Hadoop-related projects >> > - There are--and will always be--places where the documentation could be >> > improved. >> > - The people working on documentation have always been very responsive >> to >> > feedback and that shows in the docs >> > >> > It is completely natural that as people use the documentation, they will >> > identify areas for improvement. The community lists exists precisely to >> > help people like Stefan. Your characterization of this as problematic >> > rather than expected confuses me. Of course, we'd love to have perfect >> > documentation (and bug-free code) from the start. However, we all know >> > that reality intervenes. As such, making sure the process is in place >> is >> > central to success. Here is my understanding of what happened: >> > >> > 1. Someone tried to do something. >> > 2. They reviewed the documentation >> > 3. They hit issues or were confused. >> > 4. They asked for help. >> > 5. They received help. >> > 6. They completed their task. >> > 7. They provided feedback to incorporate into the documentation, >> > identifying what was missing. >> > 8. The feedback is incorporated into the documentation. (pending) >> > >> > This pattern has occurred previously and will continue to occur. As >> long >> > as the community continues to be responsive to people's challenges, we >> are >> > in good shape. We know that sometimes people who are not as tenacious as >> > Stefan will stop at #3 above. We can only do what we can do. Let's >> > continue to improve things so that this becomes less likely over time. >> > >> > Jacques >> > >> > >> > >> > >> > On Mon, Jul 20, 2015 at 8:32 AM, Ted Dunning <[email protected]> >> > wrote: >> > >> > > We just had Stefan over on the user@drill list stick with us through >> > thick >> > > and thin to get a very simple UDF up. He isn't a dolt by any means, >> but >> > > here is his signature on email describing his state of mind: >> > > >> > > Regards, >> > > > -Stefan (*not the happiest camper*) >> > > >> > > >> > > The only reason that this guy is still with us is that Jim Bates >> > exchanged >> > > quite a number of emails. The clear reason that he had problems is >> lack >> > of >> > > documentation and thus understanding of how to write a UDF. >> > > >> > > The current state of affairs is that it is unnecessarily hard to >> > contribute >> > > to Drill. This is clearly having a deleterious effect on building the >> > > community of contributors. We need to fix this. >> > > >> > > There is a place on the web site that describes how to write a UDF, >> but >> > it >> > > is clearly failing to explain the big picture of what is happening >> and it >> > > doesn't highlight the fact that you aren't really writing Java code >> when >> > > writing a Drill UDF, but instead are using Java to write in a more >> > limited >> > > language. >> > > >> > > Stefan's recent experience should be a perfect opportunity to improve >> > this >> > > documentation by describing the process better. >> > > >> > >> >> >> >> -- >> *Jim Scott* >> Director, Enterprise Strategy & Architecture >> +1 (347) 746-9281 >> >> <http://www.mapr.com/> >> [image: MapR Technologies] <http://www.mapr.com> >> >> Now Available - Free Hadoop On-Demand Training >> < >> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available >> > >> > > -- Pinaki Mukerji Senior VP, Engineering 408.856.6955 *www.mapr.com <http://mapr.com>*
