[jira] [Created] (SPARK-6496) Multinomial Logistic Regression failed when initialWeights is not null

2015-03-24 Thread Yanbo Liang (JIRA)
Yanbo Liang created SPARK-6496: -- Summary: Multinomial Logistic Regression failed when initialWeights is not null Key: SPARK-6496 URL: https://issues.apache.org/jira/browse/SPARK-6496 Project: Spark

[jira] [Updated] (SPARK-6500) Scala code example in README.md does not compile

2015-03-24 Thread Nick (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick updated SPARK-6500: Description: I just downloaded and installed Spark 1.3. Inside README.md there is this example {code} And run

[jira] [Commented] (SPARK-5479) PySpark on yarn mode need to support non-local python files

2015-03-24 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377881#comment-14377881 ] Thomas Graves commented on SPARK-5479: -- Was this fixed by

[jira] [Created] (SPARK-6502) HiveThriftServer2 fails to inspect underlying Hive version when compiled against Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6502: - Summary: HiveThriftServer2 fails to inspect underlying Hive version when compiled against Hive 0.12.0 Key: SPARK-6502 URL: https://issues.apache.org/jira/browse/SPARK-6502

[jira] [Commented] (SPARK-2429) Hierarchical Implementation of KMeans

2015-03-24 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377840#comment-14377840 ] Yu Ishikawa commented on SPARK-2429: [~freeman-lab], [~mengxr], [~josephkb],

[jira] [Updated] (SPARK-6387) HTTP mode of HiveThriftServer2 doesn't work when built with Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6387: -- Issue Type: Sub-task (was: Bug) Parent: SPARK-6109 HTTP mode of HiveThriftServer2 doesn't

[jira] [Commented] (SPARK-6109) Unit tests fail when compiled against Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377897#comment-14377897 ] Cheng Lian commented on SPARK-6109: --- I had created [PR

[jira] [Commented] (SPARK-6505) Remove the reflection call in HiveFunctionWrapper

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377915#comment-14377915 ] Cheng Lian commented on SPARK-6505: --- Here is [a WiP simpler fix for

[jira] [Created] (SPARK-6503) Create Jenkins builder for testing Spark SQL with Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6503: - Summary: Create Jenkins builder for testing Spark SQL with Hive 0.12.0 Key: SPARK-6503 URL: https://issues.apache.org/jira/browse/SPARK-6503 Project: Spark Issue

[jira] [Created] (SPARK-6505) Remove the reflection call in HiveFunctionWrapper

2015-03-24 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6505: - Summary: Remove the reflection call in HiveFunctionWrapper Key: SPARK-6505 URL: https://issues.apache.org/jira/browse/SPARK-6505 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-6500) Scala code example in README.md does not compile

2015-03-24 Thread Nick (JIRA)
Nick created SPARK-6500: --- Summary: Scala code example in README.md does not compile Key: SPARK-6500 URL: https://issues.apache.org/jira/browse/SPARK-6500 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-5173) support python application running on yarn cluster mode

2015-03-24 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377829#comment-14377829 ] Thomas Graves commented on SPARK-5173: -- [~andrewor14] This pull request has went in,

[jira] [Created] (SPARK-6501) Blacklist Hive 0.13.1 specific tests when compiled against Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
Cheng Lian created SPARK-6501: - Summary: Blacklist Hive 0.13.1 specific tests when compiled against Hive 0.12.0 Key: SPARK-6501 URL: https://issues.apache.org/jira/browse/SPARK-6501 Project: Spark

[jira] [Resolved] (SPARK-6473) Launcher lib shouldn't try to figure out Scala version when not in dev mode

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6473. -- Resolution: Fixed Issue resolved by pull request 5143 [https://github.com/apache/spark/pull/5143]

[jira] [Updated] (SPARK-6473) Launcher lib shouldn't try to figure out Scala version when not in dev mode

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6473: - Component/s: (was: Spark Core) Spark Submit Priority: Minor (was: Major)

[jira] [Created] (SPARK-6504) Cannot read Parquet files generated from different versions at once

2015-03-24 Thread Marius Soutier (JIRA)
Marius Soutier created SPARK-6504: - Summary: Cannot read Parquet files generated from different versions at once Key: SPARK-6504 URL: https://issues.apache.org/jira/browse/SPARK-6504 Project: Spark

[jira] [Closed] (SPARK-6500) Scala code example in README.md does not compile

2015-03-24 Thread Nick (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick closed SPARK-6500. --- Scala code example in README.md does not compile

[jira] [Commented] (SPARK-5162) Python yarn-cluster mode

2015-03-24 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377887#comment-14377887 ] Thomas Graves commented on SPARK-5162: -- So is there anything left on this jira to do?

[jira] [Resolved] (SPARK-6297) EventLog permissions are always set to 770 which causes problems

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6297. -- Resolution: Not a Problem Provisionally closing this as not a problem unless there is more indication

[jira] [Updated] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-4814: - Labels: (was: backport-needed) Enable assertions in SBT, Maven tests / AssertionError from Hive's

[jira] [Resolved] (SPARK-4814) Enable assertions in SBT, Maven tests / AssertionError from Hive's LazyBinaryInteger

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-4814. -- Resolution: Fixed Target Version/s: (was: 1.0.3) Provisionally deciding that it's not worth

[jira] [Updated] (SPARK-6493) Support numeric(a,b) in the sqlContext

2015-03-24 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-6493: -- Summary: Support numeric(a,b) in the sqlContext (was: Support numeric(a,b) in the parser) Support

[jira] [Commented] (SPARK-6483) Spark SQL udf(ScalaUdf) is very slow

2015-03-24 Thread zzc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377746#comment-14377746 ] zzc commented on SPARK-6483: I test it later. Spark SQL udf(ScalaUdf) is very slow

[jira] [Updated] (SPARK-6499) pyspark: printSchema command on a dataframe hangs

2015-03-24 Thread cynepia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cynepia updated SPARK-6499: --- Attachment: airports.json pyspark.txt pyspark: printSchema command on a dataframe hangs

[jira] [Resolved] (SPARK-6494) rdd polymorphic method zipPartitions refactor

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6494. -- Resolution: Won't Fix Please see PR comments. The changes that are intended as in the PR are

[jira] [Updated] (SPARK-6383) Few examples on Dataframe operation give compiler errors

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6383: - Assignee: Tijo Thomas Few examples on Dataframe operation give compiler errors

[jira] [Commented] (SPARK-6383) Few examples on Dataframe operation give compiler errors

2015-03-24 Thread Tijo Thomas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377571#comment-14377571 ] Tijo Thomas commented on SPARK-6383: The Assignee: for this issues appeared as

[jira] [Closed] (SPARK-6456) Spark Sql throwing exception on large partitioned data

2015-03-24 Thread pankaj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] pankaj closed SPARK-6456. - Resolution: Fixed It was the issue of large number of partition. actually the number was too high. i removed old

[jira] [Commented] (SPARK-6469) The YARN driver in yarn-client mode will not use the local directories configured for YARN

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377599#comment-14377599 ] Apache Spark commented on SPARK-6469: - User 'preaudc' has created a pull request for

[jira] [Updated] (SPARK-6493) Support numeric(a,b) in the sqlContext

2015-03-24 Thread DoingDone9 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DoingDone9 updated SPARK-6493: -- Description: support sql like that : select cast(20.12 as numeric(4,2)) from src limit 1; was:

[jira] [Updated] (SPARK-6499) pyspark: printSchema command on a dataframe hangs

2015-03-24 Thread cynepia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cynepia updated SPARK-6499: --- Summary: pyspark: printSchema command on a dataframe hangs (was: pyspark dataframe filter does not work as

[jira] [Created] (SPARK-6499) pyspark dataframe filter does not work as expected

2015-03-24 Thread cynepia (JIRA)
cynepia created SPARK-6499: -- Summary: pyspark dataframe filter does not work as expected Key: SPARK-6499 URL: https://issues.apache.org/jira/browse/SPARK-6499 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-6449) Driver OOM results in reported application result SUCCESS

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6449. -- Resolution: Duplicate Fix Version/s: (was: 1.3.0) Driver OOM results in reported

[jira] [Reopened] (SPARK-6449) Driver OOM results in reported application result SUCCESS

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-6449: -- (Just doing this to link to SPARK-6018 as a Duplicate which will link it in the other issue too) Driver

[jira] [Commented] (SPARK-6484) Ganglia metrics xml reporter doesn't escape correctly

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377692#comment-14377692 ] Sean Owen commented on SPARK-6484: -- [~joshrosen] see

[jira] [Resolved] (SPARK-5368) Spark should support NAT (via akka improvements)

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-5368. -- Resolution: Duplicate Fix Version/s: (was: 1.2.2) Target Version/s: (was: 1.2.2)

[jira] [Commented] (SPARK-4922) Support dynamic allocation for coarse-grained Mesos

2015-03-24 Thread Hans van den Bogert (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377760#comment-14377760 ] Hans van den Bogert commented on SPARK-4922: What were/are the reasons for not

[jira] [Updated] (SPARK-6482) Remove synchronization of Hive Native commands

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6482: - Component/s: SQL [~dyross] let's assign Components Remove synchronization of Hive Native commands

[jira] [Updated] (SPARK-6491) Spark will put the current working dir to the CLASSPATH

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6491: - Component/s: Spark Submit [~marsishandsome] please assign a Component to JIRAs Spark will put the

[jira] [Commented] (SPARK-6480) histogram() bucket function is wrong in some simple edge cases

2015-03-24 Thread Frank Rosner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377611#comment-14377611 ] Frank Rosner commented on SPARK-6480: - Thanks for picking it up [~srowen]!

[jira] [Resolved] (SPARK-6477) Run MIMA tests before the Spark test suite

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6477. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5145

[jira] [Updated] (SPARK-6477) Run MIMA tests before the Spark test suite

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-6477: - Assignee: Brennon York Run MIMA tests before the Spark test suite

[jira] [Commented] (SPARK-6496) Multinomial Logistic Regression failed when initialWeights is not null

2015-03-24 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377713#comment-14377713 ] Sean Owen commented on SPARK-6496: -- The problem is numFeatures = -1, not initialWeights.

[jira] [Created] (SPARK-6497) Class is not registered: scala.reflect.ManifestFactory$$anon$9

2015-03-24 Thread Daniel Darabos (JIRA)
Daniel Darabos created SPARK-6497: - Summary: Class is not registered: scala.reflect.ManifestFactory$$anon$9 Key: SPARK-6497 URL: https://issues.apache.org/jira/browse/SPARK-6497 Project: Spark

[jira] [Commented] (SPARK-6493) Support numeric(a,b) in the sqlContext

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377736#comment-14377736 ] Apache Spark commented on SPARK-6493: - User 'DoingDone9' has created a pull request

[jira] [Commented] (SPARK-6496) Multinomial Logistic Regression failed when initialWeights is not null

2015-03-24 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377759#comment-14377759 ] Yanbo Liang commented on SPARK-6496: [~srowen] I have address this issue at github.

[jira] [Updated] (SPARK-6496) Multinomial Logistic Regression failed when initialWeights is not null

2015-03-24 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-6496: --- Description: This bug is easy to reproduce, when use Multinomial Logistic Regression to train

[jira] [Commented] (SPARK-6496) Multinomial Logistic Regression failed when initialWeights is not null

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377740#comment-14377740 ] Apache Spark commented on SPARK-6496: - User 'yanboliang' has created a pull request

[jira] [Commented] (SPARK-6481) Set In Progress when a PR is opened for an issue

2015-03-24 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378838#comment-14378838 ] Nicholas Chammas commented on SPARK-6481: - Since there is no guaranteed way to map

[jira] [Created] (SPARK-6515) OpenHashSet returns invalid position when the data size is 1

2015-03-24 Thread Xiangrui Meng (JIRA)
Xiangrui Meng created SPARK-6515: Summary: OpenHashSet returns invalid position when the data size is 1 Key: SPARK-6515 URL: https://issues.apache.org/jira/browse/SPARK-6515 Project: Spark

[jira] [Resolved] (SPARK-6476) Spark fileserver not started on same IP as using spark.driver.host

2015-03-24 Thread Rares Vernica (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rares Vernica resolved SPARK-6476. -- Resolution: Not a Problem After investigating more and trying the suggested change, I think

[jira] [Commented] (SPARK-6152) Spark does not support Java 8 compiled Scala classes

2015-03-24 Thread Martin Grotzke (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378918#comment-14378918 ] Martin Grotzke commented on SPARK-6152: --- Btw, we just released kryo 3.0.1:

[jira] [Updated] (SPARK-6380) Resolution of equi-join key in post-join projection

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6380: Target Version/s: 1.4.0 Resolution of equi-join key in post-join projection

[jira] [Commented] (SPARK-6385) ISO 8601 timestamp parsing does not support arbitrary precision second fractions

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378690#comment-14378690 ] Michael Armbrust commented on SPARK-6385: - A PR would be great. Let me know if

[jira] [Updated] (SPARK-6513) Regression - Adding zipWithUniqueId (and other missing RDD APIs) to RDDApi.scala

2015-03-24 Thread Eran Medan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eran Medan updated SPARK-6513: -- Priority: Blocker (was: Major) Regression - Adding zipWithUniqueId (and other missing RDD APIs) to

[jira] [Commented] (SPARK-6209) ExecutorClassLoader can leak connections after failing to load classes from the REPL class server

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378788#comment-14378788 ] Apache Spark commented on SPARK-6209: - User 'JoshRosen' has created a pull request for

[jira] [Updated] (SPARK-6513) Add zipWithUniqueId (and other RDD APIs) to RDDApi.scala

2015-03-24 Thread Eran Medan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eran Medan updated SPARK-6513: -- Summary: Add zipWithUniqueId (and other RDD APIs) to RDDApi.scala (was: Regression - missing

[jira] [Updated] (SPARK-6513) Regression - missing zipWithUniqueId (and other RDD APIs) in RDDApi.scala

2015-03-24 Thread Eran Medan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eran Medan updated SPARK-6513: -- Priority: Minor (was: Blocker) Regression - missing zipWithUniqueId (and other RDD APIs) in

[jira] [Created] (SPARK-6514) For Kinesis Streaming, use the same region for DynamoDB (KCL checkpoints) as the Kinesis stream itself

2015-03-24 Thread Chris Fregly (JIRA)
Chris Fregly created SPARK-6514: --- Summary: For Kinesis Streaming, use the same region for DynamoDB (KCL checkpoints) as the Kinesis stream itself Key: SPARK-6514 URL:

[jira] [Updated] (SPARK-6513) Add zipWithUniqueId (and other RDD APIs) to RDDApi

2015-03-24 Thread Eran Medan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eran Medan updated SPARK-6513: -- Description: It will be nice if we could treat a Dataframe just like an RDD (wherever it makes sense)

[jira] [Closed] (SPARK-3570) Shuffle write time does not include time to open shuffle files

2015-03-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-3570. Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Target Version/s: 1.3.1,

[jira] [Updated] (SPARK-6387) HTTP mode of HiveThriftServer2 doesn't work when built with Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6387: -- Shepherd: Cheng Lian HTTP mode of HiveThriftServer2 doesn't work when built with Hive 0.12.0

[jira] [Updated] (SPARK-6501) Blacklist Hive 0.13.1 specific tests when compiled against Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6501: -- Shepherd: Cheng Lian Blacklist Hive 0.13.1 specific tests when compiled against Hive 0.12.0

[jira] [Updated] (SPARK-6109) Unit tests fail when compiled against Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6109: -- Shepherd: Cheng Lian Assignee: (was: Cheng Lian) Unit tests fail when compiled against Hive

[jira] [Updated] (SPARK-6513) Regression - missing zipWithUniqueId (and other RDD APIs) in RDDApi.scala

2015-03-24 Thread Eran Medan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eran Medan updated SPARK-6513: -- Issue Type: Improvement (was: Bug) Regression - missing zipWithUniqueId (and other RDD APIs) in

[jira] [Updated] (SPARK-6513) Regression - missing zipWithUniqueId (and other RDD APIs) in RDDApi.scala

2015-03-24 Thread Eran Medan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eran Medan updated SPARK-6513: -- Description: I'm sure this has an Issue somewhere but I can't find it. I see this is not a regression

[jira] [Commented] (SPARK-6510) Add Graph#minus method to act as Set#difference

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378913#comment-14378913 ] Apache Spark commented on SPARK-6510: - User 'brennonyork' has created a pull request

[jira] [Commented] (SPARK-6385) ISO 8601 timestamp parsing does not support arbitrary precision second fractions

2015-03-24 Thread Nick Bruun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378836#comment-14378836 ] Nick Bruun commented on SPARK-6385: --- Strictly speaking, the ISO 8601 standard does not

[jira] [Created] (SPARK-6516) Coupling between default Hadoop versions in Spark build vs. ec2 scripts

2015-03-24 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-6516: Summary: Coupling between default Hadoop versions in Spark build vs. ec2 scripts Key: SPARK-6516 URL: https://issues.apache.org/jira/browse/SPARK-6516

[jira] [Commented] (SPARK-6385) ISO 8601 timestamp parsing does not support arbitrary precision second fractions

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378689#comment-14378689 ] Michael Armbrust commented on SPARK-6385: - [~bruun] I think all you need to do is

[jira] [Updated] (SPARK-6079) Use index to speed up StatusTracker.getJobIdsForGroup()

2015-03-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6079: - Affects Version/s: 1.3.0 Use index to speed up StatusTracker.getJobIdsForGroup()

[jira] [Updated] (SPARK-6513) Regression - Adding zipWithUniqueId (and other missing RDD APIs) to RDDApi.scala

2015-03-24 Thread Eran Medan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eran Medan updated SPARK-6513: -- Summary: Regression - Adding zipWithUniqueId (and other missing RDD APIs) to RDDApi.scala (was:

[jira] [Created] (SPARK-6513) Regression Adding zipWithUniqueId (and other missing RDD APIs) to RDDApi.scala

2015-03-24 Thread Eran Medan (JIRA)
Eran Medan created SPARK-6513: - Summary: Regression Adding zipWithUniqueId (and other missing RDD APIs) to RDDApi.scala Key: SPARK-6513 URL: https://issues.apache.org/jira/browse/SPARK-6513 Project:

[jira] [Assigned] (SPARK-6481) Set In Progress when a PR is opened for an issue

2015-03-24 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6481: --- Assignee: Nicholas Chammas (was: Apache Spark) Set In Progress when a PR is opened for an

[jira] [Assigned] (SPARK-6430) Cannot resolve column correctlly when using left semi join

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-6430: --- Assignee: Michael Armbrust Cannot resolve column correctlly when using left semi

[jira] [Closed] (SPARK-6088) UI is malformed when tasks fetch remote results

2015-03-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or closed SPARK-6088. Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Target Version/s: 1.3.1,

[jira] [Updated] (SPARK-6413) For data source tables, we should provide better output for described extended/formatted.

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6413: Assignee: Yin Huai For data source tables, we should provide better output for described

[jira] [Updated] (SPARK-6465) GenericRowWithSchema: KryoException: Class cannot be created (missing no-arg constructor):

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6465: Target Version/s: 1.3.1, 1.4.0 (was: 1.4.0) GenericRowWithSchema: KryoException: Class

[jira] [Updated] (SPARK-6209) ExecutorClassLoader can leak connections after failing to load classes from the REPL class server

2015-03-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6209: - Fix Version/s: 1.4.0 1.3.1 ExecutorClassLoader can leak connections after failing to

[jira] [Updated] (SPARK-6513) Add zipWithUniqueId (and other RDD APIs) to RDDApi

2015-03-24 Thread Eran Medan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eran Medan updated SPARK-6513: -- Summary: Add zipWithUniqueId (and other RDD APIs) to RDDApi (was: Add zipWithUniqueId (and other RDD

[jira] [Commented] (SPARK-6385) ISO 8601 timestamp parsing does not support arbitrary precision second fractions

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378816#comment-14378816 ] Michael Armbrust commented on SPARK-6385: - Oh, I see. Looks like this is actually

[jira] [Updated] (SPARK-6513) Add zipWithUniqueId (and other RDD APIs) to RDDApi.scala

2015-03-24 Thread Eran Medan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eran Medan updated SPARK-6513: -- Description: It will be nice if we could treat a Dataframe just like an RDD (wherever it makes sense)

[jira] [Commented] (SPARK-5508) Arrays and Maps stored with Hive Parquet Serde may not be able to read by the Parquet support in the Data Souce API

2015-03-24 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378848#comment-14378848 ] Ryan Blue commented on SPARK-5508: -- [~yhuai], we've been working to standardize nested

[jira] [Commented] (SPARK-2429) Hierarchical Implementation of KMeans

2015-03-24 Thread Yu Ishikawa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378877#comment-14378877 ] Yu Ishikawa commented on SPARK-2429: I got it. Thanks! Hierarchical Implementation

[jira] [Assigned] (SPARK-6450) Native Parquet reader does not assign table name as qualifier

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-6450: --- Assignee: Michael Armbrust (was: Cheng Lian) Native Parquet reader does not assign

[jira] [Updated] (SPARK-6088) UI is malformed when tasks fetch remote results

2015-03-24 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6088: - Affects Version/s: 1.3.0 UI is malformed when tasks fetch remote results

[jira] [Updated] (SPARK-6503) Create Jenkins builder for testing Spark SQL with Hive 0.12.0

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6503: -- Shepherd: Cheng Lian Create Jenkins builder for testing Spark SQL with Hive 0.12.0

[jira] [Updated] (SPARK-6507) Create separate Hive Driver instance for each SQL query in HiveThriftServer2

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6507: -- Shepherd: Cheng Lian Create separate Hive Driver instance for each SQL query in HiveThriftServer2

[jira] [Updated] (SPARK-6505) Remove the reflection call in HiveFunctionWrapper

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6505: -- Shepherd: Cheng Lian Remove the reflection call in HiveFunctionWrapper

[jira] [Updated] (SPARK-6465) GenericRowWithSchema: KryoException: Class cannot be created (missing no-arg constructor):

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6465: Priority: Critical (was: Major) GenericRowWithSchema: KryoException: Class cannot be

[jira] [Updated] (SPARK-6465) GenericRowWithSchema: KryoException: Class cannot be created (missing no-arg constructor):

2015-03-24 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-6465: Description: I can not find a issue for this. register for GenericRowWithSchema is lost

[jira] [Updated] (SPARK-6513) Regression - Adding zipWithUniqueId (and other missing RDD APIs) to RDDApi.scala

2015-03-24 Thread Eran Medan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eran Medan updated SPARK-6513: -- Description: I'm sure this has an Issue somewhere but I can't find it. I see this as a regression

[jira] [Updated] (SPARK-6513) Regression - missing zipWithUniqueId (and other RDD APIs) in RDDApi.scala

2015-03-24 Thread Eran Medan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eran Medan updated SPARK-6513: -- Summary: Regression - missing zipWithUniqueId (and other RDD APIs) in RDDApi.scala (was: Regression -

[jira] [Commented] (SPARK-6385) ISO 8601 timestamp parsing does not support arbitrary precision second fractions

2015-03-24 Thread Nick Bruun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378801#comment-14378801 ] Nick Bruun commented on SPARK-6385: --- An extra {{S}} does not seem to do the trick, as

[jira] [Created] (SPARK-6506) python support yarn cluster mode requires SPARK_HOME to be set

2015-03-24 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-6506: Summary: python support yarn cluster mode requires SPARK_HOME to be set Key: SPARK-6506 URL: https://issues.apache.org/jira/browse/SPARK-6506 Project: Spark

[jira] [Commented] (SPARK-6495) DataFrame#insertInto method should support insert rows with sub-columns

2015-03-24 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378051#comment-14378051 ] Cheng Lian commented on SPARK-6495: --- Inserting a subset of columns in the original

[jira] [Comment Edited] (SPARK-2394) Make it easier to read LZO-compressed files from EC2 clusters

2015-03-24 Thread Theodore Vasiloudis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14375766#comment-14375766 ] Theodore Vasiloudis edited comment on SPARK-2394 at 3/24/15 3:08 PM:

[jira] [Comment Edited] (SPARK-2394) Make it easier to read LZO-compressed files from EC2 clusters

2015-03-24 Thread Theodore Vasiloudis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14375766#comment-14375766 ] Theodore Vasiloudis edited comment on SPARK-2394 at 3/24/15 3:09 PM:

[jira] [Commented] (SPARK-6479) Create off-heap block storage API (internal)

2015-03-24 Thread Sandy Ryza (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378011#comment-14378011 ] Sandy Ryza commented on SPARK-6479: --- I believe he means wrapping Spark's call-outs to

[jira] [Commented] (SPARK-3306) Addition of external resource dependency in executors

2015-03-24 Thread Yan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14378073#comment-14378073 ] Yan commented on SPARK-3306: If by global singleton object, you meant it to be in the Executor

  1   2   3   >