Having multiple block to display all magic blocks sounds good too. But having this: myVar: org.apache.spark.rdd.RDD[Int] = ... Printed in the middle would look bad
On Wed, Jul 22, 2015 at 4:08 PM, DuyHai Doan <[email protected]> wrote: > Your alternative sounds interesting. > > I had something in mind lately, e.g. enclosing block of magic > > val myVar = ... > println("%table .... %endTable") > val anotherVar = ... > println("%table .... % endTable") > > The output would be: > > myVar: org.apache.spark.rdd.RDD[Int] = ... > %table > ... > %endTable > anotherVar: org.apache.spark.RDD[...] = ... > % table > ... > % endTable > > There is some work to do on the Angular side to extract block of magic and > display each of them accordingly. The entry point in the code is here: > > > https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js#L623-L669 > > Instead of saving table data in result.columnNames and result.rows we can > have a list of table data. Optionally we can index table by their name with > %table[name1] and put their data into a map on the front-end > > It will open the possibility of having multiple tables display in the same > paragraph > > > > On Wed, Jul 22, 2015 at 8:22 AM, Corneau Damien <[email protected]> > wrote: > > > It wouldn't be enough. > > Even if you could detect get the %table, you wouldn't now what to show > > (scala text or table) > > Here the problem is that the paragraph expect one type of result only. > > So when you start typing scala code, you get scala type result which is > > some text (even if in the middle you print a different type) > > It is even more annoying that it is forcing you to enclose any code > inside > > a function. > > > > One option could also be to separate output from visualization by > creating > > a special function in zeppelin, that would work like print and replace > > default return type by new one. > > For example: > > > > *Case1* > > > > %spark > > val bankText = sc.parallelize(1 to 10) > > println("%table col1\tcol2\naaa\t123\n") > > > > /* Would print by default as result the scala type result (text) */ > > > > Case 2 > > > > %spark > > val bankText = sc.parallelize(1 to 10) > > z.showResult("%table col1\tcol2\naaa\t123\n") > > > > or even > > > > z.showResult("TABLE", "col1\tcol2\naaa\t123\n") > > > > /* Would replace original result type by new one, and add the scala type > > result in an output variable > > > > On Wed, Jul 22, 2015 at 3:00 PM, IT CTO <[email protected]> wrote: > > > > > How about changing getType and getData code to use contains and not > > > startWith and trim the head before the magic word? My assumption that > if > > > the user used the magic word that is what he wants to see. > > > Later we can add your suggestion for complex type. > > > With the current state, I find it hard to use the tables UI when I > > > manipulate data in spark and python > > > > > > Eran > > > > > > On Tue, Jul 21, 2015 at 10:53 PM DuyHai Doan <[email protected]> > > wrote: > > > > > > > Yes you may create a JIRA "Improve magic display system" > > > > > > > > On Tue, Jul 21, 2015 at 9:46 PM, IT CTO <[email protected]> wrote: > > > > > > > > > Thanks for the detailed answer. As a work-around, can we suppress > > the > > > sc > > > > > output? > > > > > BTW, should I open a jira bug for this? > > > > > Eran > > > > > > > > > > בתאריך יום ג׳, 21 ביולי 2015, 21:53 מאת DuyHai Doan < > > > > [email protected] > > > > > >: > > > > > > > > > > > Ha, this is a very CLASSICAL caveat with Zeppelin display system. > > > > > > > > > > > > How is that ? Because to determine the type of the returned > message > > > > > > (InterpreterResult class), Zeppelin is using this piece of code: > > > > > > > > > > > > private Type getType(String msg) { > > > > > > if (msg == null) { > > > > > > return Type.TEXT; > > > > > > } > > > > > > Type[] types = Type.values(); > > > > > > for (Type t : types) { > > > > > > String magic = "%" + t.name().toLowerCase(); > > > > > > if (msg.startsWith(magic + " ") || msg.startsWith(magic + > > "\n")) > > > { > > > > > > return t; > > > > > > } > > > > > > } > > > > > > return Type.TEXT; > > > > > > } > > > > > > > > > > > > > > > > > > Obviously, the statement "val bankText = sc.parallelize(1 to 10)" > > is > > > > > > interpreter by the Spark REPL and will generate the output > > > > > > > > > > > > "bankText: org.apache.spark.rdd.RDD[Int] = > ParallelCollectionRDD[4] > > > at" > > > > > > > > > > > > So the returned message does NOT start with %table but with the > > above > > > > > text > > > > > > so for Zeppelin, the Type is Type.TEXT. > > > > > > > > > > > > The caveat here is that Zeppelin code is using > msg.startsWith(xxx) > > to > > > > > check > > > > > > for the %magic. > > > > > > > > > > > > We should definitely refactor the result message display system > > with: > > > > > > > > > > > > 1) parse the whole output String and every time the is a %magic > > mixed > > > > > with > > > > > > plain text, return a Type.COMPOSITE (new type to be added) > > > > > > > > > > > > 2) On the AngularJS side, we need to extract any %magic bloc, > > display > > > > > them > > > > > > properly and display the remaining text as plain text > > > > > > > > > > > > Again, I'll have a look into it when I have some bandwidth > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Jul 21, 2015 at 7:14 PM, IT CTO <[email protected]> > wrote: > > > > > > > > > > > > > Code is very simple > > > > > > > /// Not working paragraph : > > > > > > > %spark > > > > > > > val bankText = sc.parallelize(1 to 10) > > > > > > > println("%table col1\tcol2\naaa\t123\n") > > > > > > > > > > > > > > OUTPUT IS: > > > > > > > > > > > > > > bankText: org.apache.spark.rdd.RDD[Int] = > > ParallelCollectionRDD[4] > > > at > > > > > > > parallelize at <console>:23 %table col1 col2 aaa 123 > > > > > > > > > > > > > > > > > > > > > > > > > > > > /// working paragraph: > > > > > > > println("%table col1\tcol2\naaa\t123\n") > > > > > > > > > > > > > > OUTPUT is the table display as expected with all buttons and > > > options > > > > > > > col1col2aaa123 > > > > > > > Eran > > > > > > > > > > > > > > On Tue, Jul 21, 2015 at 8:09 PM madhuka udantha < > > > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > Hi Eran, > > > > > > > > > > > > > > > > We can't see the images that you are sharing. I think it is > not > > > > > > possible > > > > > > > to > > > > > > > > attach them directly on ML. IMO you could try dropbox etc. > > > > > > > > > > > > > > > > Thanks. > > > > > > > > > > > > > > > > On Tue, Jul 21, 2015 at 7:52 PM, IT CTO <[email protected]> > > > wrote: > > > > > > > > > > > > > > > > > Just install the latest build and tried the following: > > > > > > > > > [image: Selection_001.png] > > > > > > > > > > > > > > > > > > this is not what I was expecting from the print. > > > > > > > > > actually, if I remove the sc line it works well... > > > > > > > > > [image: Selection_002.png] > > > > > > > > > > > > > > > > > > Any idea why this is happening? > > > > > > > > > Eran > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Cheers, > > > > > > > > Madhuka Udantha > > > > > > > > http://madhukaudantha.blogspot.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
