[jira] [Comment Edited] (TOREE-407) Improve Branding on Site

2017-05-02 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993861#comment-15993861 ] Jakob Odersky edited comment on TOREE-407 at 5/2/17 9:5

[jira] [Commented] (TOREE-407) Improve Branding on Site

2017-05-02 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993861#comment-15993861 ] Jakob Odersky commented on TOREE-407: - That's a valid point, I also think

[jira] [Resolved] (TOREE-403) Temporary files/directories should be deleted after a kernel shutdown

2017-04-10 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky resolved TOREE-403. - Resolution: Fixed Fix Version/s: 0.2.0 > Temporary files/directories should be dele

[jira] [Commented] (TOREE-402) Installer should support parameterized kernel names

2017-04-05 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15957571#comment-15957571 ] Jakob Odersky commented on TOREE-402: - Good point; that's an aspect I did

[jira] [Comment Edited] (TOREE-402) Installer should support parameterized kernel names

2017-04-05 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15957443#comment-15957443 ] Jakob Odersky edited comment on TOREE-402 at 4/5/17 6:5

[jira] [Commented] (TOREE-402) Installer should support parameterized kernel names

2017-04-05 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15957443#comment-15957443 ] Jakob Odersky commented on TOREE-402: - Thanks for the detailed description. This c

[jira] [Commented] (TOREE-399) Make Spark Kernel work on Windows

2017-03-30 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15949642#comment-15949642 ] Jakob Odersky commented on TOREE-399: - Hi Aldo, the run.sh script is a launcher sc

[jira] [Updated] (TOREE-399) Make Spark Kernel work on Windows

2017-03-30 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated TOREE-399: Priority: Major (was: Blocker) > Make Spark Kernel work on Wind

Re: want to join toree mailing list

2017-03-28 Thread Jakob Odersky
Hi, the list system is automated. Send an email to dev-subscr...@toree.incubator.apache.org to subscribe. Check this page for more information http://toree.apache.org/community/get-involved --Jakob On Sun, Mar 26, 2017 at 2:33 PM, Rajkumar Natarajan wrote: > Hi Team, > > Please add me to the ma

[jira] [Commented] (TOREE-375) Incorrect fully qualified name for spark context

2017-03-10 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905637#comment-15905637 ] Jakob Odersky commented on TOREE-375: - Closing this as it is not related to T

[jira] [Closed] (TOREE-375) Incorrect fully qualified name for spark context

2017-03-10 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky closed TOREE-375. --- Resolution: Done > Incorrect fully qualified name for spark cont

[jira] [Resolved] (TOREE-386) spark kernel `--name test` or `--conf spark.app.name=test` parameter to spark_opts is not applied

2017-03-02 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky resolved TOREE-386. - Resolution: Fixed > spark kernel `--name test` or `--conf spark.app.name=test` parameter

[jira] [Resolved] (TOREE-383) Fix flaky tests

2017-02-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky resolved TOREE-383. - Resolution: Fixed > Fix flaky tests > --- > > Ke

[jira] [Commented] (TOREE-377) When magic fails, the error is swallowed

2017-02-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886512#comment-15886512 ] Jakob Odersky commented on TOREE-377: - Fixed in PR, thanks! > When magic fai

[jira] [Resolved] (TOREE-377) When magic fails, the error is swallowed

2017-02-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky resolved TOREE-377. - Resolution: Fixed > When magic fails, the error is swallo

[jira] [Resolved] (TOREE-376) Scala interpreter should make Spark SQL implicits available

2017-02-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky resolved TOREE-376. - Resolution: Fixed Merged, thanks! > Scala interpreter should make Spark SQL implicits availa

[jira] [Resolved] (TOREE-378) Usability improvements in jupyter console

2017-02-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky resolved TOREE-378. - Resolution: Fixed Fixed in PR, thanks! > Usability improvements in jupyter cons

[jira] [Closed] (TOREE-379) Tab completion doesn't replace partial words

2017-02-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky closed TOREE-379. --- Resolution: Fixed Fixed in PR, thanks! > Tab completion doesn't replace parti

[jira] [Resolved] (TOREE-387) Kernel should not store SparkSession

2017-02-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky resolved TOREE-387. - Resolution: Fixed fixed in pr > Kernel should not store SparkSess

[jira] [Assigned] (TOREE-386) spark kernel `--name test` or `--conf spark.app.name=test` parameter to spark_opts is not applied

2017-02-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky reassigned TOREE-386: --- Assignee: Jakob Odersky > spark kernel `--name test` or `--conf spark.app.name=t

[jira] [Commented] (TOREE-375) Incorrect fully qualified name for spark context

2017-02-21 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876909#comment-15876909 ] Jakob Odersky commented on TOREE-375: - Contrary to my initial comment, I can con

[jira] [Comment Edited] (TOREE-375) Incorrect fully qualified name for spark context

2017-02-21 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876892#comment-15876892 ] Jakob Odersky edited comment on TOREE-375 at 2/21/17 10:3

[jira] [Commented] (TOREE-375) Incorrect fully qualified name for spark context

2017-02-21 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876892#comment-15876892 ] Jakob Odersky commented on TOREE-375: - [~fschueler] so I'm actually not enti

[jira] [Comment Edited] (TOREE-375) Incorrect fully qualified name for spark context

2017-02-21 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876892#comment-15876892 ] Jakob Odersky edited comment on TOREE-375 at 2/21/17 10:3

Supported Scala versions

2017-02-16 Thread Jakob Odersky
Hi everyone, during a recent discussion with Marius https://github.com/apache/incubator-toree/pull/93, I found out that Toree does not support being built with Scala 2.10 anymore. I am all in favor of dropping the EOL scala version, however considering that there are still various references to t

[jira] [Resolved] (TOREE-382) Revamp the sbt build and consolidate dependencies

2017-02-16 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky resolved TOREE-382. - Resolution: Fixed > Revamp the sbt build and consolidate dependenc

[jira] [Commented] (TOREE-386) toree spark kernel --name parameter to spark-submit is not applied

2017-02-15 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868606#comment-15868606 ] Jakob Odersky commented on TOREE-386: - Hmm, I'm not sure where the name is co

[jira] [Updated] (TOREE-375) Incorrect fully qualified name for spark context

2017-02-15 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated TOREE-375: Priority: Critical (was: Major) > Incorrect fully qualified name for spark cont

[jira] [Assigned] (TOREE-382) Revamp the sbt build and consolidate dependencies

2017-02-14 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky reassigned TOREE-382: --- Assignee: Jakob Odersky > Revamp the sbt build and consolidate dependenc

[jira] [Assigned] (TOREE-383) Fix flaky tests

2017-02-14 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky reassigned TOREE-383: --- Assignee: Jakob Odersky > Fix flaky tests > --- > >

[jira] [Created] (TOREE-385) Refactor travis build to be runnable as a container (as opposed to a vm)

2017-02-14 Thread Jakob Odersky (JIRA)
Jakob Odersky created TOREE-385: --- Summary: Refactor travis build to be runnable as a container (as opposed to a vm) Key: TOREE-385 URL: https://issues.apache.org/jira/browse/TOREE-385 Project: TOREE

[jira] [Created] (TOREE-383) Fix flaky tests

2017-02-14 Thread Jakob Odersky (JIRA)
Jakob Odersky created TOREE-383: --- Summary: Fix flaky tests Key: TOREE-383 URL: https://issues.apache.org/jira/browse/TOREE-383 Project: TOREE Issue Type: Sub-task Reporter: Jakob

[jira] [Updated] (TOREE-382) Revamp the sbt build and consolidate dependencies

2017-02-14 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated TOREE-382: Priority: Minor (was: Major) > Revamp the sbt build and consolidate dependenc

[jira] [Created] (TOREE-382) Revamp the sbt build and consolidate dependencies

2017-02-14 Thread Jakob Odersky (JIRA)
Jakob Odersky created TOREE-382: --- Summary: Revamp the sbt build and consolidate dependencies Key: TOREE-382 URL: https://issues.apache.org/jira/browse/TOREE-382 Project: TOREE Issue Type: Sub

[jira] [Created] (TOREE-381) Revamp the build

2017-02-14 Thread Jakob Odersky (JIRA)
Jakob Odersky created TOREE-381: --- Summary: Revamp the build Key: TOREE-381 URL: https://issues.apache.org/jira/browse/TOREE-381 Project: TOREE Issue Type: Improvement Reporter

[jira] [Closed] (TOREE-372) stream corruption cased by big-endian and little-endian

2017-02-11 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky closed TOREE-372. --- Resolution: Won't Fix The issue isn't related to Toree. However a mixed endian environm

Classloader exceptions when testing

2017-02-11 Thread Jakob Odersky
Hi everyone, I may have found a fix for the classloader issues I'm having when running unit tests (https://groups.google.com/forum/#!topic/akka-user/U0mLX3mCmAk). Basically the issue appears to happen when a test suite that uses the Akka TestKit base class does not define an explicit classloader.

Linking JIRA with GitHub

2017-02-11 Thread Jakob Odersky
Hi, does anyone how we can link jira with github, so that pull requests with a title of the form [TOREE-X] will close the issue X when merged. Basically I'm looking for something similar to the way Spark handles issues and pull requests. cheers, --Jakob

[jira] [Closed] (TOREE-361) Spark examples that use Spark 2 fail because docker image contains 1.6

2017-02-11 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky closed TOREE-361. --- Resolution: Fixed fixed in pr > Spark examples that use Spark 2 fail because docker image conta

[jira] [Commented] (TOREE-354) Scala Error with Apache Spark when run in Jupyter

2017-02-09 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860179#comment-15860179 ] Jakob Odersky commented on TOREE-354: - Can you try a `pip install --pre `? Pip wil

[jira] [Closed] (TOREE-354) Scala Error with Apache Spark when run in Jupyter

2017-02-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky closed TOREE-354. --- Resolution: Not A Problem There is a mismatch between the scala versions used by Toree and Spark. The

[jira] [Commented] (TOREE-363) Syntax Highlighting Breaks

2017-02-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15857140#comment-15857140 ] Jakob Odersky commented on TOREE-363: - AFAIK syntax highlighting is not handled by

[jira] [Closed] (TOREE-331) jupyter: 'toree' is not a Jupyter command

2017-02-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky closed TOREE-331. --- Resolution: Cannot Reproduce Can't reproduce this either > jupyter: 'toree' is not

[jira] [Closed] (TOREE-281) Fix some typos in comments/docs/testnames.

2017-02-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky closed TOREE-281. --- Resolution: Fixed > Fix some typos in comments/docs/testna

[jira] [Closed] (TOREE-282) Remove `Created by` comments from source code

2017-02-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky closed TOREE-282. --- Resolution: Fixed > Remove `Created by` comments from source c

[jira] [Commented] (TOREE-340) Output of the form "a = b" returns "b"

2017-02-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15856973#comment-15856973 ] Jakob Odersky commented on TOREE-340: - [PR #94|https://github.com/apache/incub

[jira] [Closed] (TOREE-373) scala 2.11 library incompatible with pyspark 2.1.0

2017-02-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky closed TOREE-373. --- Resolution: Not A Bug +1 to Marius' answer. I would be surprised it worked with 2.12 t

[jira] [Commented] (TOREE-374) Variables declared on the Notebook are not garbage collected

2017-02-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15856880#comment-15856880 ] Jakob Odersky commented on TOREE-374: - Thanks for the reproduction steps, [~dt

[jira] [Comment Edited] (TOREE-374) Variables declared on the Notebook are not garbage collected

2017-02-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15856880#comment-15856880 ] Jakob Odersky edited comment on TOREE-374 at 2/7/17 9:5

[jira] [Commented] (TOREE-375) Incorrect fully qualified name for spark context

2017-02-07 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15856668#comment-15856668 ] Jakob Odersky commented on TOREE-375: - I checked the implementation of `valueOf

[jira] [Closed] (TOREE-365) Certain interpreter evaluations do not return result strings

2017-02-06 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky closed TOREE-365. --- Resolution: Won't Fix Closing as discussed in the pull request. See TOREE-368 for continu

[jira] [Commented] (TOREE-374) Variables declared on the Notebook are not garbage collected

2017-02-06 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855081#comment-15855081 ] Jakob Odersky commented on TOREE-374: - [~dtaieb] Could you provide some step

[jira] [Comment Edited] (TOREE-374) Variables declared on the Notebook are not garbage collected

2017-02-06 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855055#comment-15855055 ] Jakob Odersky edited comment on TOREE-374 at 2/7/17 12:4

[jira] [Commented] (TOREE-374) Variables declared on the Notebook are not garbage collected

2017-02-06 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855055#comment-15855055 ] Jakob Odersky commented on TOREE-374: - Hmm, I wonder if this is related to the Y

[jira] [Comment Edited] (TOREE-375) Incorrect fully qualified name for spark context

2017-02-06 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855019#comment-15855019 ] Jakob Odersky edited comment on TOREE-375 at 2/7/17 12:1

[jira] [Commented] (TOREE-375) Incorrect fully qualified name for spark context

2017-02-06 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855019#comment-15855019 ] Jakob Odersky commented on TOREE-375: - -Yrepl-class-based strikes again. I manage

[jira] [Commented] (TOREE-371) $SPARK_HOME environment variable not recognised

2017-02-06 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854859#comment-15854859 ] Jakob Odersky commented on TOREE-371: - I'm not sure how changing the defa

[jira] [Commented] (TOREE-372) stream corruption cased by big-endian and little-endian

2017-02-06 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854877#comment-15854877 ] Jakob Odersky commented on TOREE-372: - Spark requires its driver app (SparkCon

Development model

2017-01-30 Thread Jakob Odersky
Hi everyone, I was wondering how everyone usually develops toree, specifically how changes to toree are tested with a jupyter notebook? I couldn't find any documentation on the website so I thought I'd ask here. I tried running the various makefile targets, including `make dev` and `make jupyter`,

[jira] [Updated] (TOREE-365) Certain interpreter evaluations do not return result strings

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated TOREE-365: Description: The scala interpreter currently only returns results for expressions. Import

[jira] [Updated] (TOREE-365) Certain interpreter evaluations do not return result strings

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated TOREE-365: Description: The scala interpreter currently only returns results for expressions. Import

[jira] [Updated] (TOREE-365) Certain interpreter evaluations do not return result strings

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated TOREE-365: Description: The scala interpreter currently only returns results for expressions. Import

[jira] [Created] (TOREE-365) Certain interpreter evaluations do not return result strings

2017-01-27 Thread Jakob Odersky (JIRA)
Jakob Odersky created TOREE-365: --- Summary: Certain interpreter evaluations do not return result strings Key: TOREE-365 URL: https://issues.apache.org/jira/browse/TOREE-365 Project: TOREE

[jira] [Updated] (TOREE-333) make sbt-publishM2

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated TOREE-333: Priority: Minor (was: Major) > make sbt-publishM2 > -- > >

[jira] [Updated] (TOREE-340) Output of the form "a = b" returns "b"

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated TOREE-340: Priority: Major (was: Minor) > Output of the form "a = b&quo

[jira] [Commented] (TOREE-340) Output of the form "a = b" returns "b"

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15843676#comment-15843676 ] Jakob Odersky commented on TOREE-340: - Unfortunately a similar manifestation of

[jira] [Comment Edited] (TOREE-262) Resolve LGPL Dependency in project

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15843653#comment-15843653 ] Jakob Odersky edited comment on TOREE-262 at 1/27/17 11:3

[jira] [Commented] (TOREE-262) Resolve LGPL Dependency in project

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15843653#comment-15843653 ] Jakob Odersky commented on TOREE-262: - The JeroMQ project is now released under

[jira] [Resolved] (TOREE-322) Building Error with github snapshot

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky resolved TOREE-322. - Resolution: Fixed > Building Error with github snaps

[jira] [Commented] (TOREE-322) Building Error with github snapshot

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15843649#comment-15843649 ] Jakob Odersky commented on TOREE-322: - Latest master works, I assume it must have

[jira] [Updated] (TOREE-362) How do I define ports specifically?

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated TOREE-362: Priority: Major (was: Blocker) > How do I define ports specifica

[jira] [Updated] (TOREE-333) make sbt-publishM2

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated TOREE-333: Priority: Major (was: Critical) > make sbt-publishM2 > -- > >

[jira] [Commented] (TOREE-333) make sbt-publishM2

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15843642#comment-15843642 ] Jakob Odersky commented on TOREE-333: - The good news is that despite the error mes

[jira] [Commented] (TOREE-362) How do I define ports specifically?

2017-01-27 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/TOREE-362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15843631#comment-15843631 ] Jakob Odersky commented on TOREE-362: - I'm not sure I understand your ques

[jira] [Commented] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2017-01-10 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816398#comment-15816398 ] Jakob Odersky commented on SPARK-14280: --- Twitter chill for scala 2.12 is fin

Re: Third party library

2016-12-13 Thread Jakob Odersky
Hi Vineet, great to see you solved the problem! Since this just appeared in my inbox, I wanted to take the opportunity for a shameless plug: https://github.com/jodersky/sbt-jni. In case you're using sbt and also developing the native library, this plugin may help with the pains of building and pack

Re: Optimization for Processing a million of HTML files

2016-12-12 Thread Jakob Odersky
Assuming the bottleneck is IO, you could try saving your files to HDFS. This will distribute your data and allow for better concurrent reads. On Mon, Dec 12, 2016 at 3:06 PM, Reth RM wrote: > Hi, > > I have millions of html files in a directory, using "wholeTextFiles" api to > load them and proce

Re: wholeTextFiles()

2016-12-12 Thread Jakob Odersky
Also, in case the issue was not due to the string length (however it is still valid and may get you later), the issue may be due to some other indexing issues which are currently being worked on here https://issues.apache.org/jira/browse/SPARK-6235 On Mon, Dec 12, 2016 at 8:18 PM, Jakob Odersky

Re: wholeTextFiles()

2016-12-12 Thread Jakob Odersky
Hi Pradeep, I'm afraid you're running into a hard Java issue. Strings are indexed with signed integers and can therefore not be longer than approximately 2 billion characters. Could you use `textFile` as a workaround? It will give you an RDD of the files' lines instead. In general, this guide htt

[jira] [Updated] (SPARK-14519) Cross-publish Kafka for Scala 2.12

2016-12-12 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Odersky updated SPARK-14519: -- Summary: Cross-publish Kafka for Scala 2.12 (was: Cross-publish Kafka for Scala 2.12.0-M4

[jira] [Commented] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2016-12-09 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15736107#comment-15736107 ] Jakob Odersky commented on SPARK-17647: --- I rebased the PR and resolved

Re: Can I add a new method to RDD class?

2016-12-06 Thread Jakob Odersky
thods in. > > How can I specify a custom version? modify version numbers in all the > pom.xml file? > > > > On Dec 5, 2016, at 9:12 PM, Jakob Odersky wrote: > > m rdds in an "org.apache.spark" package as well > > -

Re: Can I add a new method to RDD class?

2016-12-05 Thread Jakob Odersky
It looks like you're having issues with including your custom spark version (with the extensions) in your test project. To use your local spark version: 1) make sure it has a custom version (let's call it 2.1.0-CUSTOM) 2) publish it to your local machine with `sbt publishLocal` 3) include the modif

Re: custom generate spark application id

2016-12-05 Thread Jakob Odersky
The app ID is assigned internally by spark's task scheduler https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala#L35. You could probably change the naming, however I'm pretty sure that the ID will always have to be unique for a context on a

[jira] [Commented] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-12-05 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15723814#comment-15723814 ] Jakob Odersky commented on SPARK-14280: --- You're welcome pull the changes

[jira] [Commented] (SPARK-14280) Update change-version.sh and pom.xml to add Scala 2.12 profiles

2016-12-05 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15723807#comment-15723807 ] Jakob Odersky commented on SPARK-14280: --- Hi [~joshrosen], I rebased your ini

Re: SparkILoop doesn't run

2016-11-21 Thread Jakob Odersky
libraries of multiple scala versions on the same classpath. You mention that it worked before, can you recall what libraries you upgraded before it broke? --Jakob On Mon, Nov 21, 2016 at 2:34 PM, Jakob Odersky wrote: > Trying it out locally gave me an NPE. I'll look into it in more > deta

Re: SparkILoop doesn't run

2016-11-21 Thread Jakob Odersky
Trying it out locally gave me an NPE. I'll look into it in more detail, however the SparkILoop.run() method is dead code. It's used nowhere in spark and can be removed without any issues. On Thu, Nov 17, 2016 at 11:16 AM, Mohit Jaggi wrote: > Thanks Holden. I did post to the user list but since t

[jira] [Comment Edited] (SPARK-14222) Cross-publish jackson-module-scala for Scala 2.12

2016-11-03 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634143#comment-15634143 ] Jakob Odersky edited comment on SPARK-14222 at 11/3/16 8:3

[jira] [Comment Edited] (SPARK-14222) Cross-publish jackson-module-scala for Scala 2.12

2016-11-03 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634143#comment-15634143 ] Jakob Odersky edited comment on SPARK-14222 at 11/3/16 8:3

[jira] [Commented] (SPARK-14222) Cross-publish jackson-module-scala for Scala 2.12

2016-11-03 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634143#comment-15634143 ] Jakob Odersky commented on SPARK-14222: --- Thanks Sean, however I realized that

[jira] [Commented] (SPARK-14222) Cross-publish jackson-module-scala for Scala 2.12

2016-11-03 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634117#comment-15634117 ] Jakob Odersky commented on SPARK-14222: --- A newer version of module (vertsion 2

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

2016-11-03 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634027#comment-15634027 ] Jakob Odersky commented on SPARK-14220: --- at least most dependencies will prob

[jira] [Comment Edited] (SPARK-14220) Build and test Spark against Scala 2.12

2016-11-03 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15634027#comment-15634027 ] Jakob Odersky edited comment on SPARK-14220 at 11/3/16 7:5

[jira] [Commented] (SPARK-14220) Build and test Spark against Scala 2.12

2016-11-03 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15633762#comment-15633762 ] Jakob Odersky commented on SPARK-14220: --- Scala 2.12 was just officially annou

Re: why spark driver program is creating so many threads? How can I limit this number?

2016-10-31 Thread Jakob Odersky
> how do I tell my spark driver program to not create so many? This may depend on your driver program. Do you spawn any threads in it? Could you share some more information on the driver program, spark version and your environment? It would greatly help others to help you On Mon, Oct 31, 2016 at

Re: [Spark 2] BigDecimal and 0

2016-10-24 Thread Jakob Odersky
further then. Thanks for the quick help. > > On Mon, Oct 24, 2016 at 7:34 PM Jakob Odersky wrote: >> >> What you're seeing is merely a strange representation, 0E-18 is zero. >> The E-18 represents the precision that Spark uses to store the decimal >> >> On

Re: [Spark 2] BigDecimal and 0

2016-10-24 Thread Jakob Odersky
What you're seeing is merely a strange representation, 0E-18 is zero. The E-18 represents the precision that Spark uses to store the decimal On Mon, Oct 24, 2016 at 7:32 PM, Jakob Odersky wrote: > An even smaller example that demonstrates the same behaviour: > > Seq(Dat

Re: [Spark 2] BigDecimal and 0

2016-10-24 Thread Jakob Odersky
An even smaller example that demonstrates the same behaviour: Seq(Data(BigDecimal(0))).toDS.head On Mon, Oct 24, 2016 at 7:03 PM, Efe Selcuk wrote: > I’m trying to track down what seems to be a very slight imprecision in our > Spark application; two of our columns, which should be netting ou

[jira] [Commented] (SPARK-18018) Specify alternate escape character in 'LIKE' expression

2016-10-19 Thread Jakob Odersky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15590136#comment-15590136 ] Jakob Odersky commented on SPARK-18018: --- I've started a very early

  1   2   3   4   >