UI way requires interpreter process restart. So the behavior will be different from %spark z.load() you use right now. And...new UI way keeps list of dependencies. It does not reset after interpreter/zeppelin restart. So without api for removing dependency, dependencies will never decrease.
Hence, to me, the best option seems - bringing %spark z.load() back with correct implementation - deprecate and remove %dep z.load() while it was alternative for buggy %spark z.load() - keep new UI way, while not only Spark but all other interpreters can leverage it. Thanks, moon On Fri, Feb 19, 2016 at 2:30 PM Aish Fenton <afen...@netflix.com> wrote: > > I mean add in the dependency in the same way that the UI does. > > > On Fri, Feb 19, 2016 at 2:30 PM Aish Fenton <afen...@netflix.com> wrote: > >> >> Hmm, >> Maybe you could change the semantics of z.load to be automatically add in >> the dependency? The only advantage I was really after with z.load was that >> self-documenting feature. >> >> >> >> >> On Fri, Feb 19, 2016 at 2:11 PM moon soo Lee <m...@apache.org> wrote: >> >>> Right, new UI provides more generalized way for dependency loading for >>> all interpreters. not only spark but also jdbc, etc. >>> >>> For spark, >>> >>> We have had %spark z.load() at first, but it was too buggy, so we have >>> created %dep z.load(), which is far less buggy but still some library >>> couldn't loaded in this way. Finally, the new UI way, handles all the case >>> correctly. >>> >>> I agree if we could have z.load for spark, it's self-documenting. And it >>> may be convenient for multiple people working. >>> >>> If you're interested, i can work on bringing %spark z.load() back. But >>> then we'll have 3 different way of dependency loading for Spark interpreter. >>> >>> - %spark z.load() (removed in master branch) >>> - %dep z.load() (marked as deprecated in master branch) >>> - new UI way. >>> >>> I think new UI way is for all interpreters. so we definitely want to >>> keep it. >>> So i want to ask people which one do we want to keep? %spark z.load(), >>> %dep z.load(). Neither, both, one of it? >>> >>> Thanks, >>> moon >>> >>> >>> On Fri, Feb 19, 2016 at 10:52 AM Aish Fenton <afen...@netflix.com> >>> wrote: >>> >>>> >>>> z.load was buggy, but it was self-documenting. I'm wondering with the >>>> new UI way, how will this work for people sharing/importing notebook? We >>>> quite commonly share notebooks. >>>> >>>> >>>> >>>> On Fri, Feb 19, 2016 at 10:09 AM moon soo Lee <m...@apache.org> wrote: >>>> >>>>> Hi Ankur, >>>>> >>>>> It's bit depends on you Zeppelin version, but basically >>>>> z.load() inside of Spark interpreter was buggy and we have now removed >>>>> from master branch. >>>>> >>>>> You can check "Dependency Management" section, from >>>>> http://zeppelin.incubator.apache.org/docs/latest/interpreter/spark.html >>>>> >>>>> Basically, two different way >>>>> one using %dep interpreter, the other using spark properties. >>>>> >>>>> If you can use master branch, you can load dependencies from >>>>> Interpreter menu via GUI. >>>>> >>>>> Hope this helps. >>>>> >>>>> Best, >>>>> moon >>>>> >>>>> On Fri, Feb 19, 2016 at 5:50 AM Ankur Jain <ankur.j...@yash.com> >>>>> wrote: >>>>> >>>>>> Hi Team, >>>>>> >>>>>> >>>>>> >>>>>> I loaded avro dependencies, and then tried to read avro files, but it >>>>>> is throwing below exception… >>>>>> >>>>>> >>>>>> >>>>>> [image: image002.jpg] >>>>>> >>>>> >>>>>> >>>>>> Can you please guide on this? >>>>>> >>>>>> >>>>>> >>>>>> My Environment details are as mentioned below… >>>>>> >>>>>> >>>>>> https://blogs.aws.amazon.com/bigdata/post/Tx2HJD3Z74J2U8U/Running-an-External-Zeppelin-Instance-using-S3-Backed-Notebooks-with-Spark-on-Am >>>>>> >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Ankur >>>>>> >>>>> Information transmitted by this e-mail is proprietary to YASH >>>>>> Technologies and/ or its Customers and is intended for use only by the >>>>>> individual or entity to which it is addressed, and may contain >>>>>> information >>>>>> that is privileged, confidential or exempt from disclosure under >>>>>> applicable >>>>>> law. If you are not the intended recipient or it appears that this mail >>>>>> has >>>>>> been forwarded to you without proper authority, you are notified that any >>>>>> use or dissemination of this information in any manner is strictly >>>>>> prohibited. In such cases, please notify us immediately at >>>>>> i...@yash.com and delete this mail from your records. >>>>>> >>>>>
