Your alternative sounds interesting.
I had something in mind lately, e.g. enclosing block of magic
val myVar = ...
println("%table .... %endTable")
val anotherVar = ...
println("%table .... % endTable")
The output would be:
myVar: org.apache.spark.rdd.RDD[Int] = ...
%table
...
%endTable
anotherVar: org.apache.spark.RDD[...] = ...
% table
...
% endTable
There is some work to do on the Angular side to extract block of magic and
display each of them accordingly. The entry point in the code is here:
https://github.com/apache/incubator-zeppelin/blob/master/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js#L623-L669
Instead of saving table data in result.columnNames and result.rows we can
have a list of table data. Optionally we can index table by their name with
%table[name1] and put their data into a map on the front-end
It will open the possibility of having multiple tables display in the same
paragraph
On Wed, Jul 22, 2015 at 8:22 AM, Corneau Damien <[email protected]>
wrote:
> It wouldn't be enough.
> Even if you could detect get the %table, you wouldn't now what to show
> (scala text or table)
> Here the problem is that the paragraph expect one type of result only.
> So when you start typing scala code, you get scala type result which is
> some text (even if in the middle you print a different type)
> It is even more annoying that it is forcing you to enclose any code inside
> a function.
>
> One option could also be to separate output from visualization by creating
> a special function in zeppelin, that would work like print and replace
> default return type by new one.
> For example:
>
> *Case1*
>
> %spark
> val bankText = sc.parallelize(1 to 10)
> println("%table col1\tcol2\naaa\t123\n")
>
> /* Would print by default as result the scala type result (text) */
>
> Case 2
>
> %spark
> val bankText = sc.parallelize(1 to 10)
> z.showResult("%table col1\tcol2\naaa\t123\n")
>
> or even
>
> z.showResult("TABLE", "col1\tcol2\naaa\t123\n")
>
> /* Would replace original result type by new one, and add the scala type
> result in an output variable
>
> On Wed, Jul 22, 2015 at 3:00 PM, IT CTO <[email protected]> wrote:
>
> > How about changing getType and getData code to use contains and not
> > startWith and trim the head before the magic word? My assumption that if
> > the user used the magic word that is what he wants to see.
> > Later we can add your suggestion for complex type.
> > With the current state, I find it hard to use the tables UI when I
> > manipulate data in spark and python
> >
> > Eran
> >
> > On Tue, Jul 21, 2015 at 10:53 PM DuyHai Doan <[email protected]>
> wrote:
> >
> > > Yes you may create a JIRA "Improve magic display system"
> > >
> > > On Tue, Jul 21, 2015 at 9:46 PM, IT CTO <[email protected]> wrote:
> > >
> > > > Thanks for the detailed answer. As a work-around, can we suppress
> the
> > sc
> > > > output?
> > > > BTW, should I open a jira bug for this?
> > > > Eran
> > > >
> > > > בתאריך יום ג׳, 21 ביולי 2015, 21:53 מאת DuyHai Doan <
> > > [email protected]
> > > > >:
> > > >
> > > > > Ha, this is a very CLASSICAL caveat with Zeppelin display system.
> > > > >
> > > > > How is that ? Because to determine the type of the returned message
> > > > > (InterpreterResult class), Zeppelin is using this piece of code:
> > > > >
> > > > > private Type getType(String msg) {
> > > > > if (msg == null) {
> > > > > return Type.TEXT;
> > > > > }
> > > > > Type[] types = Type.values();
> > > > > for (Type t : types) {
> > > > > String magic = "%" + t.name().toLowerCase();
> > > > > if (msg.startsWith(magic + " ") || msg.startsWith(magic +
> "\n"))
> > {
> > > > > return t;
> > > > > }
> > > > > }
> > > > > return Type.TEXT;
> > > > > }
> > > > >
> > > > >
> > > > > Obviously, the statement "val bankText = sc.parallelize(1 to 10)"
> is
> > > > > interpreter by the Spark REPL and will generate the output
> > > > >
> > > > > "bankText: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[4]
> > at"
> > > > >
> > > > > So the returned message does NOT start with %table but with the
> above
> > > > text
> > > > > so for Zeppelin, the Type is Type.TEXT.
> > > > >
> > > > > The caveat here is that Zeppelin code is using msg.startsWith(xxx)
> to
> > > > check
> > > > > for the %magic.
> > > > >
> > > > > We should definitely refactor the result message display system
> with:
> > > > >
> > > > > 1) parse the whole output String and every time the is a %magic
> mixed
> > > > with
> > > > > plain text, return a Type.COMPOSITE (new type to be added)
> > > > >
> > > > > 2) On the AngularJS side, we need to extract any %magic bloc,
> display
> > > > them
> > > > > properly and display the remaining text as plain text
> > > > >
> > > > > Again, I'll have a look into it when I have some bandwidth
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Jul 21, 2015 at 7:14 PM, IT CTO <[email protected]> wrote:
> > > > >
> > > > > > Code is very simple
> > > > > > /// Not working paragraph :
> > > > > > %spark
> > > > > > val bankText = sc.parallelize(1 to 10)
> > > > > > println("%table col1\tcol2\naaa\t123\n")
> > > > > >
> > > > > > OUTPUT IS:
> > > > > >
> > > > > > bankText: org.apache.spark.rdd.RDD[Int] =
> ParallelCollectionRDD[4]
> > at
> > > > > > parallelize at <console>:23 %table col1 col2 aaa 123
> > > > > >
> > > > > >
> > > > > >
> > > > > > /// working paragraph:
> > > > > > println("%table col1\tcol2\naaa\t123\n")
> > > > > >
> > > > > > OUTPUT is the table display as expected with all buttons and
> > options
> > > > > > col1col2aaa123
> > > > > > Eran
> > > > > >
> > > > > > On Tue, Jul 21, 2015 at 8:09 PM madhuka udantha <
> > > > > [email protected]>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Eran,
> > > > > > >
> > > > > > > We can't see the images that you are sharing. I think it is not
> > > > > possible
> > > > > > to
> > > > > > > attach them directly on ML. IMO you could try dropbox etc.
> > > > > > >
> > > > > > > Thanks.
> > > > > > >
> > > > > > > On Tue, Jul 21, 2015 at 7:52 PM, IT CTO <[email protected]>
> > wrote:
> > > > > > >
> > > > > > > > Just install the latest build and tried the following:
> > > > > > > > [image: Selection_001.png]
> > > > > > > >
> > > > > > > > this is not what I was expecting from the print.
> > > > > > > > actually, if I remove the sc line it works well...
> > > > > > > > [image: Selection_002.png]
> > > > > > > >
> > > > > > > > Any idea why this is happening?
> > > > > > > > Eran
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Cheers,
> > > > > > > Madhuka Udantha
> > > > > > > http://madhukaudantha.blogspot.com
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
