Re: requirements for Pig 1.0?

2009-06-24 Thread Russell Jurney
For 1.0 - complete Owl? http://wiki.apache.org/pig/Metadata Russell Jurney rjur...@cloudstenography.com On Jun 23, 2009, at 4:40 PM, Alan Gates wrote: I don't believe there's a solid list of want to haves for 1.0. The big issue I see is that there are too many interfaces that are still

Re: requirements for Pig 1.0?

2009-06-24 Thread Alan Gates
Integration with Owl is something we want for 1.0. I am hopeful that by Pig's 1.0 Owl will have flown the coop and become either a subproject or found a home in Hadoop's common, since it will hopefully be used by multiple other subprojects. Alan. On Jun 23, 2009, at 11:42 PM, Russell

Re: requirements for Pig 1.0?

2009-06-24 Thread Dmitriy Ryaboy
Alan, any thoughts on performance baselines and benchmarks? I am a little surprised that you think SQL is a requirement for 1.0, since it's essentially an overlay, not core functionality. What about the storage layer rewrite (or is that what you referred to with your first bullet-point)? Also,

Re: requirements for Pig 1.0?

2009-06-24 Thread Alan Gates
To be clear, going to 1.0 is not about having a certain set of features. It is about stability and usability. When a project declares itself 1.0 it is making some guarantees regarding the stability of its interfaces (in Pig's case this is Pig Latin, UDFs, and command line usage). It is

RE: requirements for Pig 1.0?

2009-06-23 Thread Santhosh Srinivasan
:40 PM To: pig-dev@hadoop.apache.org Subject: Re: requirements for Pig 1.0? I don't believe there's a solid list of want to haves for 1.0. The big issue I see is that there are too many interfaces that are still shifting, such as: 1) Data input/output formats. The way we do slicing