Hi Russell and Daniel, I have set my proposal on google-melange public and available at * http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/grassonsnad/4001 *.
I read through the original idea page as well as all the email discussions concerning with the PIG-2586 project again, and found some problems that I'm not quite sure about the visualization: 1. input to the visualizer Is the input a (1).pig script file, or (2) a command entered from a *grunt* console? If the input is script file which contains a series of operations for a Pig task, we could visualize the whole plan/data at one time, and meanwhile provide the ability of step-by-step visualization of individual operations. If the input is from the *grunt* command line, we need to update visualization on the fly. 2. content of the visualization >From the original description of this project and earlier email discussion, I feel that the visualizer needs to provide information for the 3 layers of (1)schema, (2)plan and (3)data, collected from DESCRIBE, EXPLAIN and ILLUSTRATE respectively, all on one canvas, but with the ability of switching on and off the display of one or two layers. But later discussions made me feel that we need to separate the visualization of EXPLAIN and ILLUSTRATE, if so, I am wondering what the updated, detailed requirements of the visualizer are for the project. (one small question: does the visualizer need to provide all the three kinds of plan diagrams – logical, physical and mapreduce plan diagrams, or only the physical plan diagram as shown in the sigmod09 paper?) Looking forward to your reply. Thank you very much. Regards, On Fri, Apr 6, 2012 at 1:37 PM, Russell Jurney <russell.jur...@gmail.com>wrote: > I can't see the proposal, as I'm not an approved mentor. I'd suggest > working with rcmper...@gmail.com if possible. You might focus on > visualization of ILLUSTRATE while he focuses on the editor? > On Fri, Mar 23, 2012 at 11:21 PM, Shasha Liu <grassons...@gmail.com>wrote: > >> Hi Daniel, >> >> Thanks a lot for the reply. >> I installed the latest Pig and read through the book of "programming in >> pig". >> I manged to use "-dot -out filename" to produce three graphs in dot file >> format. >> >> Based on the existing dot file, my next question is what is the >> requirement regarding a better visualizer? >> Are we going to generate a picture (e.g., .png) for different plans >> (logical plan, physical plan, map reduce plan), or provide a web interface >> to visualize these graphs of plans? >> >> Best regards, >> -- >> Shasha(Amy) Liu >> >> >> On Sun, Mar 18, 2012 at 3:30 AM, Daniel Dai <da...@hortonworks.com>wrote: >> >>> See comments inline. >>> >>> On Sat, Mar 17, 2012 at 6:52 AM, grassonsand <grassons...@gmail.com> >>> wrote: >>> > Dear all, >>> > >>> > I am a Ph.D. student in Computer Science and have 4-year Java >>> programming >>> > experience focusing on Java Web development. >>> > In the candidate projects in PIG, I am interested in PIG-2586 (A better >>> > plan/data flow visualizer) and PIG-2599 (Mavenize Pig). >>> > >>> > In my on-going research project, I am in charge of (1). web user >>> interface >>> > development and (2). build system. Now I am working on adding hadoop >>> > capability to the project. The main reason I am interested in the PIG >>> > project is that I can make a contribution to the PIG community based >>> on my >>> > previous experience, and learn from the participant in GSoC this year >>> and >>> > benefit my on-going research project at the same time. >>> > >>> > (1). User interface development >>> > I have used several graphic libraries to visualize semantic data and >>> our own >>> > data set, e.g., Jung, graphviz, BIRT, and several plot plugins in >>> jquery. >>> > Therefore, I am interested in working on a new tool for PIG visualizer. >>> > After looking through the bug issue, I have several questions: >>> > (i) As both swing and javascript are mentioned, is this project a >>> web or >>> > standalone application? >>> > (ii) As ruby-graphviz is included, Is ruby required for this >>> project? >>> >>> I envision two visualize components in Pig. One is a lightweight >>> visualizer invoked by Grunt, which should be fast and concise, and >>> integrated into explain command. The other is a standalone composer >>> similar to PigPen, which should be much powerful. PIG-2586 is intended >>> to track the first, but Russel's comment is talking about the second. >>> Both are acceptable as a GSoC project. I leave it to Russel. >>> >>> > >>> > (2). Build system >>> > The code base of my research project is 40K loc and the build script >>> was >>> > written in Ant. Part of my duty is to convert the ant build script to >>> maven >>> > and maintain the build script. Therefore, Mavenize Pig is of interest >>> to me >>> > too. The build.xml in PIG project is more complicated than the one I >>> worked >>> > before. It includes ant, maven and ivy. Do we need to use maven to do >>> all >>> > the tasks and get rid of all the dependency on ant, maven and ivy? >>> >>> Yes >>> >>> > >>> > Best regards >>> > Shasha(Amy) Liu >>> >> >> >> > > > -- > Russell Jurney twitter.com/rjurney russell.jur...@gmail.com datasyndrome. > com > -- Shasha(Amy) Liu
