[jira] [Created] (SPARK-11850) Spark StdDev/Variance defaults are incompatible with Hive

2015-11-19 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-11850: - Summary: Spark StdDev/Variance defaults are incompatible with Hive Key: SPARK-11850 URL: https://issues.apache.org/jira/browse/SPARK-11850 Project: Spark

[jira] [Created] (SPARK-11851) Unable to start spark thrift server against secured hive metastore

2015-11-19 Thread bit1129 (JIRA)
bit1129 created SPARK-11851: --- Summary: Unable to start spark thrift server against secured hive metastore Key: SPARK-11851 URL: https://issues.apache.org/jira/browse/SPARK-11851 Project: Spark

[jira] [Assigned] (SPARK-11829) Model export/import for spark.ml: estimators under ml.feature (II)

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11829: Assignee: Apache Spark > Model export/import for spark.ml: estimators under ml.feature

[jira] [Assigned] (SPARK-11829) Model export/import for spark.ml: estimators under ml.feature (II)

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11829: Assignee: (was: Apache Spark) > Model export/import for spark.ml: estimators under

[jira] [Updated] (SPARK-11851) Unable to start spark thrift server against secured hive metastore(GSS initiate failed)

2015-11-19 Thread bit1129 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bit1129 updated SPARK-11851: Description: I am using Spark 1.5.X to work with Hive 0.14.0. a. spark-defaults.conf: {code}

[jira] [Commented] (SPARK-11829) Model export/import for spark.ml: estimators under ml.feature (II)

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013300#comment-15013300 ] Apache Spark commented on SPARK-11829: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Updated] (SPARK-11829) Model export/import for spark.ml: estimators under ml.feature (II)

2015-11-19 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-11829: Description: Add read/write support to the following estimators and models under spark.ml: *

[jira] [Updated] (SPARK-3947) Support Scala/Java UDAF

2015-11-19 Thread Milad Bourhani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Milad Bourhani updated SPARK-3947: -- Attachment: spark-udaf.zip Hi :) I've tried using your {{GeometricMean}} example with both a

[jira] [Closed] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-19 Thread Stanislav Hadjiiski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Hadjiiski closed SPARK-11777. --- See [Yin Huai's

[jira] [Resolved] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11777. --- Resolution: Not A Problem OK, then this isn't a Spark problem, right? this is where we started the

[jira] [Commented] (SPARK-11777) HiveContext.saveAsTable does not update the metastore on overwrite

2015-11-19 Thread Stanislav Hadjiiski (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013130#comment-15013130 ] Stanislav Hadjiiski commented on SPARK-11777: - long story short, yes, that is the case >

[jira] [Commented] (SPARK-11846) Model export/import for spark.ml: AFTSurvivalRegression and IsotonicRegression

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013105#comment-15013105 ] Apache Spark commented on SPARK-11846: -- User 'yinxusen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11846) Model export/import for spark.ml: AFTSurvivalRegression and IsotonicRegression

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11846: Assignee: Xusen Yin (was: Apache Spark) > Model export/import for spark.ml:

[jira] [Resolved] (SPARK-11021) SparkSQL cli throws exception when using with Hive 0.12 metastore in spark-1.5.0 version

2015-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-11021. --- Resolution: Not A Problem OK, so this is just a Hive config issue? > SparkSQL cli throws exception

[jira] [Commented] (SPARK-11700) Memory leak at SparkContext jobProgressListener stageIdToData map

2015-11-19 Thread Kostas papageorgopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013158#comment-15013158 ] Kostas papageorgopoulos commented on SPARK-11700: - Hi Shixiong Zhu. To my understanding

[jira] [Commented] (SPARK-10367) Support Parquet logical type INTERVAL

2015-11-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013199#comment-15013199 ] Cheng Lian commented on SPARK-10367: Dropped. > Support Parquet logical type INTERVAL >

[jira] [Updated] (SPARK-10367) Support Parquet logical type INTERVAL

2015-11-19 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-10367: --- Target Version/s: (was: 1.6.0) > Support Parquet logical type INTERVAL >

[jira] [Comment Edited] (SPARK-11638) Apache Spark in Docker with Bridge networking / run Spark on Mesos, in Docker with Bridge networking

2015-11-19 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013261#comment-15013261 ] Stavros Kontopoulos edited comment on SPARK-11638 at 11/19/15 9:55 AM:

[jira] [Commented] (SPARK-11323) Add History Service Provider to service application histories from YARN timeline server

2015-11-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013260#comment-15013260 ] Steve Loughran commented on SPARK-11323: I had hoped, but the gate has closed. Let's target 1.7

[jira] [Commented] (SPARK-11638) Apache Spark in Docker with Bridge networking / run Spark on Mesos, in Docker with Bridge networking

2015-11-19 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013261#comment-15013261 ] Stavros Kontopoulos commented on SPARK-11638: - My new trial... I have them all fixed...1 to

[jira] [Commented] (SPARK-11821) Spark-Hive doesn't work with Kerberos in local mode

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013191#comment-15013191 ] Apache Spark commented on SPARK-11821: -- User 'woj-i' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11821) Spark-Hive doesn't work with Kerberos in local mode

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11821: Assignee: (was: Apache Spark) > Spark-Hive doesn't work with Kerberos in local mode >

[jira] [Assigned] (SPARK-11821) Spark-Hive doesn't work with Kerberos in local mode

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11821: Assignee: Apache Spark > Spark-Hive doesn't work with Kerberos in local mode >

[jira] [Commented] (SPARK-7457) Perf test for ALS.recommendAll

2015-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013221#comment-15013221 ] Sean Owen commented on SPARK-7457: -- This should be MatrixFactorizationModel.recommendProductsForUsers

[jira] [Commented] (SPARK-7457) Perf test for ALS.recommendAll

2015-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15013222#comment-15013222 ] Sean Owen commented on SPARK-7457: -- This should be MatrixFactorizationModel.recommendProductsForUsers

[jira] [Updated] (SPARK-11314) Add service API and test service for Yarn Cluster schedulers

2015-11-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-11314: --- Target Version/s: (was: 1.6.0) > Add service API and test service for Yarn Cluster

[jira] [Updated] (SPARK-11323) Add History Service Provider to service application histories from YARN timeline server

2015-11-19 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-11323: --- Target Version/s: (was: 1.6.0) > Add History Service Provider to service application

[jira] [Resolved] (SPARK-11830) Make NettyRpcEnv bind to the specified host

2015-11-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-11830. - Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 1.6.0 > Make NettyRpcEnv

[jira] [Updated] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

2015-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-11016: -- Assignee: Charles Allen > Spark fails when running with a task that requires a more recent version of

[jira] [Updated] (SPARK-11783) When deployed against remote Hive metastore, HiveContext.executionHive points to wrong metastore

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11783: --- Priority: Critical (was: Blocker) > When deployed against remote Hive metastore,

[jira] [Updated] (SPARK-11831) AkkaRpcEnvSuite is prone to port-contention-related flakiness

2015-11-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-11831: -- Target Version/s: 1.5.3, 1.6.0 > AkkaRpcEnvSuite is prone to port-contention-related flakiness >

[jira] [Resolved] (SPARK-10567) Reducer locality follow-up for Spark 1.6

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10567. Resolution: Fixed Assignee: Matei Zaharia Fix Version/s: 1.6.0 > Reducer locality

[jira] [Updated] (SPARK-11831) AkkaRpcEnvSuite is prone to port-contention-related flakiness

2015-11-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-11831: -- Labels: flaky-test (was: ) > AkkaRpcEnvSuite is prone to port-contention-related flakiness >

[jira] [Commented] (SPARK-11855) Catalyst breaks backwards compatibility in branch-1.6

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014135#comment-15014135 ] Davies Liu commented on SPARK-11855: cc [~marmbrus] > Catalyst breaks backwards compatibility in

[jira] [Commented] (SPARK-11853) java.lang.ClassNotFoundException for no reason

2015-11-19 Thread sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014149#comment-15014149 ] sam commented on SPARK-11853: - [~srowen] Take another look, I edited the description shortly after creating

[jira] [Commented] (SPARK-11850) Spark StdDev/Variance defaults are incompatible with Hive

2015-11-19 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014197#comment-15014197 ] Herman van Hovell commented on SPARK-11850: --- No problem. I'll disable those tests. This perhaps

[jira] [Updated] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11016: --- Assignee: (was: Davies Liu) > Spark fails when running with a task that requires a more recent

[jira] [Commented] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014055#comment-15014055 ] Davies Liu commented on SPARK-11016: [~sowen] I tried to assigned this to [~drcrallen] (who did the

[jira] [Comment Edited] (SPARK-11319) PySpark silently Accepts null values in non-nullable DataFrame fields.

2015-11-19 Thread Harry Brundage (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014098#comment-15014098 ] Harry Brundage edited comment on SPARK-11319 at 11/19/15 6:30 PM: --

[jira] [Commented] (SPARK-11319) PySpark silently Accepts null values in non-nullable DataFrame fields.

2015-11-19 Thread Harry Brundage (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014098#comment-15014098 ] Harry Brundage commented on SPARK-11319: Forgive my frankness but that is ridiculous. This means

[jira] [Updated] (SPARK-9506) DataFrames Postgresql JDBC unable to support most of the Postgresql's Data Type

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9506: -- Priority: Major (was: Blocker) > DataFrames Postgresql JDBC unable to support most of the Postgresql's

[jira] [Assigned] (SPARK-11831) AkkaRpcEnvSuite is prone to port-contention-related flakiness

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11831: Assignee: (was: Apache Spark) > AkkaRpcEnvSuite is prone to port-contention-related

[jira] [Commented] (SPARK-11831) AkkaRpcEnvSuite is prone to port-contention-related flakiness

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014122#comment-15014122 ] Apache Spark commented on SPARK-11831: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Updated] (SPARK-11831) AkkaRpcEnvSuite is prone to port-contention-related flakiness

2015-11-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-11831: -- Assignee: Shixiong Zhu > AkkaRpcEnvSuite is prone to port-contention-related flakiness >

[jira] [Updated] (SPARK-9278) DataFrameWriter.insertInto inserts incorrect data

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9278: -- Assignee: Cheng Lian > DataFrameWriter.insertInto inserts incorrect data >

[jira] [Commented] (SPARK-11850) Spark StdDev/Variance defaults are incompatible with Hive

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014142#comment-15014142 ] Davies Liu commented on SPARK-11850: [~hvanhovell] This is on purpose, we had a long discussion about

[jira] [Commented] (SPARK-11850) Spark StdDev/Variance defaults are incompatible with Hive

2015-11-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014203#comment-15014203 ] Reynold Xin commented on SPARK-11850: - Absolutely - that's why SPARK-11490 has a "releasenotes" label

[jira] [Commented] (SPARK-10582) using dynamic-executor-allocation, if AM failed. the new AM will be started. But the new AM does not allocate executors to dirver

2015-11-19 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014230#comment-15014230 ] Imran Rashid commented on SPARK-10582: -- I'm pretty sure this is distinct from SPARK-10515, so I've

[jira] [Reopened] (SPARK-10582) using dynamic-executor-allocation, if AM failed. the new AM will be started. But the new AM does not allocate executors to dirver

2015-11-19 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid reopened SPARK-10582: -- > using dynamic-executor-allocation, if AM failed. the new AM will be started. > But the new AM

[jira] [Commented] (SPARK-11730) Feature Importance for GBT

2015-11-19 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014087#comment-15014087 ] Seth Hendrickson commented on SPARK-11730: -- I can work on this. I was taking a look at the

[jira] [Updated] (SPARK-9686) Spark Thrift server doesn't return correct JDBC metadata

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9686: -- Priority: Critical (was: Blocker) > Spark Thrift server doesn't return correct JDBC metadata >

[jira] [Commented] (SPARK-10567) Reducer locality follow-up for Spark 1.6

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014125#comment-15014125 ] Davies Liu commented on SPARK-10567: Since https://issues.apache.org/jira/browse/SPARK-9852 is

[jira] [Updated] (SPARK-11831) AkkaRpcEnvSuite is prone to port-contention-related flakiness

2015-11-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-11831: -- Component/s: Tests > AkkaRpcEnvSuite is prone to port-contention-related flakiness >

[jira] [Resolved] (SPARK-9604) Unsafe ArrayData and MapData is very very slow

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9604. --- Resolution: Fixed > Unsafe ArrayData and MapData is very very slow >

[jira] [Closed] (SPARK-11850) Spark StdDev/Variance defaults are incompatible with Hive

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-11850. -- Resolution: Not A Problem > Spark StdDev/Variance defaults are incompatible with Hive >

[jira] [Assigned] (SPARK-11806) Spark 2.0 deprecations and removals

2015-11-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-11806: --- Assignee: Reynold Xin > Spark 2.0 deprecations and removals >

[jira] [Commented] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

2015-11-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014056#comment-15014056 ] Sean Owen commented on SPARK-11016: --- No it's just that people have to be in the Contributors role to be

[jira] [Commented] (SPARK-11016) Spark fails when running with a task that requires a more recent version of RoaringBitmaps

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014078#comment-15014078 ] Davies Liu commented on SPARK-11016: Thanks! > Spark fails when running with a task that requires a

[jira] [Commented] (SPARK-9506) DataFrames Postgresql JDBC unable to support most of the Postgresql's Data Type

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014100#comment-15014100 ] Davies Liu commented on SPARK-9506: --- [~cloud_fan] Could this be workaround by using a customized dialect

[jira] [Commented] (SPARK-9271) Concurrency bug triggered by partition predicate push-down

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014128#comment-15014128 ] Davies Liu commented on SPARK-9271: --- [~lian cheng] Is this still a problem? > Concurrency bug triggered

[jira] [Commented] (SPARK-11850) Spark StdDev/Variance defaults are incompatible with Hive

2015-11-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014145#comment-15014145 ] Reynold Xin commented on SPARK-11850: - Yup we decided that Hive was weird and not the example to

[jira] [Commented] (SPARK-6725) Model export/import for Pipeline API

2015-11-19 Thread Alexandru Rosianu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014151#comment-15014151 ] Alexandru Rosianu commented on SPARK-6725: -- Is there an alternative we can use in 1.5.2 to

[jira] [Updated] (SPARK-11806) Spark 2.0 deprecations and removals

2015-11-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-11806: Description: This is an umbrella ticket to track things we are deprecating and removing in Spark

[jira] [Commented] (SPARK-11855) Catalyst breaks backwards compatibility in branch-1.6

2015-11-19 Thread Santiago M. Mola (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014193#comment-15014193 ] Santiago M. Mola commented on SPARK-11855: -- Thanks Michael. Sounds reasonable. I'll prepare a PR

[jira] [Created] (SPARK-11858) Move sql.columnar into sql.execution

2015-11-19 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-11858: --- Summary: Move sql.columnar into sql.execution Key: SPARK-11858 URL: https://issues.apache.org/jira/browse/SPARK-11858 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-11700) Memory leak at SparkContext jobProgressListener stageIdToData map

2015-11-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014022#comment-15014022 ] Shixiong Zhu commented on SPARK-11700: -- Even if maps in jobProgressListener are cleared, the

[jira] [Updated] (SPARK-11785) When deployed against remote Hive metastore with lower versions, JDBC metadata calls throws exception

2015-11-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-11785: --- Priority: Critical (was: Blocker) > When deployed against remote Hive metastore with lower

[jira] [Commented] (SPARK-11854) The exit code of spark-submit is still 0 when an yarn application fails

2015-11-19 Thread sam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014144#comment-15014144 ] sam commented on SPARK-11854: - [~srowen] It's on emr-4.1.0 with latest Spark EMR uses (so 1.5.1 I guess). My

[jira] [Commented] (SPARK-11649) "SparkListenerSuite.onTaskGettingResult() called when result fetched remotely" test is very slow

2015-11-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014163#comment-15014163 ] Marcelo Vanzin commented on SPARK-11649: But master and 1.6 don't use akka by default. So the

[jira] [Commented] (SPARK-11855) Catalyst breaks backwards compatibility in branch-1.6

2015-11-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014180#comment-15014180 ] Michael Armbrust commented on SPARK-11855: -- Hey, very sorry for the confusion here. Catalyst is

[jira] [Assigned] (SPARK-2331) SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

2015-11-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-2331: -- Assignee: Reynold Xin > SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T] >

[jira] [Assigned] (SPARK-10580) Remove Bagel

2015-11-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-10580: --- Assignee: Reynold Xin > Remove Bagel > > > Key: SPARK-10580 >

[jira] [Created] (SPARK-11857) Remove Mesos fine-grained mode subject to discussions

2015-11-19 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-11857: --- Summary: Remove Mesos fine-grained mode subject to discussions Key: SPARK-11857 URL: https://issues.apache.org/jira/browse/SPARK-11857 Project: Spark Issue

[jira] [Assigned] (SPARK-4819) Remove Guava's "Optional" from public API

2015-11-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-4819: -- Assignee: Reynold Xin > Remove Guava's "Optional" from public API >

[jira] [Comment Edited] (SPARK-11700) Memory leak at SparkContext jobProgressListener stageIdToData map

2015-11-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014195#comment-15014195 ] Shixiong Zhu edited comment on SPARK-11700 at 11/19/15 7:23 PM: Yeah,

[jira] [Commented] (SPARK-11700) Memory leak at SparkContext jobProgressListener stageIdToData map

2015-11-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014195#comment-15014195 ] Shixiong Zhu commented on SPARK-11700: -- Yeah, just found the issue is in this line:

[jira] [Commented] (SPARK-11700) Memory leak at SparkContext jobProgressListener stageIdToData map

2015-11-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014196#comment-15014196 ] Shixiong Zhu commented on SPARK-11700: -- [~davies] > Memory leak at SparkContext jobProgressListener

[jira] [Created] (SPARK-11859) Regex for master URL for Mesos accepts incorrect zk:// prefix

2015-11-19 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-11859: --- Summary: Regex for master URL for Mesos accepts incorrect zk:// prefix Key: SPARK-11859 URL: https://issues.apache.org/jira/browse/SPARK-11859 Project: Spark

[jira] [Updated] (SPARK-11778) HiveContext.read.table does not support user-specified database names

2015-11-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-11778: - Labels: releasenotes (was: ) > HiveContext.read.table does not support user-specified

[jira] [Commented] (SPARK-11319) PySpark silently Accepts null values in non-nullable DataFrame fields.

2015-11-19 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014360#comment-15014360 ] Yin Huai commented on SPARK-11319: -- For those data sources, the nullable field is always true unless you

[jira] [Updated] (SPARK-11778) HiveContext.read.table does not support user-specified database names

2015-11-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-11778: - Assignee: Huaxin Gao > HiveContext.read.table does not support user-specified database

[jira] [Resolved] (SPARK-11778) HiveContext.read.table does not support user-specified database names

2015-11-19 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-11778. -- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 9773

[jira] [Commented] (SPARK-11319) PySpark silently Accepts null values in non-nullable DataFrame fields.

2015-11-19 Thread Harry Brundage (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014405#comment-15014405 ] Harry Brundage commented on SPARK-11319: Why force all users everywhere to do a pass over the

[jira] [Resolved] (SPARK-11845) Add unit tests to verify correct checkpointing of TrackStateRDD

2015-11-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-11845. --- Resolution: Fixed Fix Version/s: 1.6.0 > Add unit tests to verify correct checkpointing of

[jira] [Resolved] (SPARK-11864) Improve performance of max/min

2015-11-19 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-11864. - Resolution: Fixed Assignee: Davies Liu Fix Version/s: 1.6.0 > Improve

[jira] [Created] (SPARK-11874) DistributedCache for PySpark

2015-11-19 Thread Ranjana Rajendran (JIRA)
Ranjana Rajendran created SPARK-11874: - Summary: DistributedCache for PySpark Key: SPARK-11874 URL: https://issues.apache.org/jira/browse/SPARK-11874 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-11874) DistributedCache for PySpark

2015-11-19 Thread Ranjana Rajendran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ranjana Rajendran updated SPARK-11874: -- Description: I have access only to the workbench of a cluster. All the nodes have only

[jira] [Commented] (SPARK-11843) Isolate staging directory across applications on same YARN cluster

2015-11-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014994#comment-15014994 ] Andrew Or commented on SPARK-11843: --- Oops, that seems to be the case. I'm closing this. > Isolate

[jira] [Resolved] (SPARK-11843) Isolate staging directory across applications on same YARN cluster

2015-11-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or resolved SPARK-11843. --- Resolution: Won't Fix > Isolate staging directory across applications on same YARN cluster >

[jira] [Reopened] (SPARK-11843) Isolate staging directory across applications on same YARN cluster

2015-11-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reopened SPARK-11843: > Isolate staging directory across applications on same YARN cluster >

[jira] [Commented] (SPARK-11831) AkkaRpcEnvSuite is prone to port-contention-related flakiness

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014787#comment-15014787 ] Apache Spark commented on SPARK-11831: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11869) ML test TempDirectory trait does not clean up properly

2015-11-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-11869: - Assignee: Joseph K. Bradley > ML test TempDirectory trait does not clean up

[jira] [Updated] (SPARK-11869) ML test TempDirectory trait does not clean up properly

2015-11-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-11869: -- Target Version/s: 1.6.0 > ML test TempDirectory trait does not clean up properly >

[jira] [Commented] (SPARK-11869) ML test TempDirectory trait does not clean up properly

2015-11-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014807#comment-15014807 ] Joseph K. Bradley commented on SPARK-11869: --- It's because {{TempDirectory.afterAll}} needs to

[jira] [Assigned] (SPARK-11869) ML test TempDirectory trait does not clean up properly

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11869: Assignee: Joseph K. Bradley (was: Apache Spark) > ML test TempDirectory trait does not

[jira] [Commented] (SPARK-11869) ML test TempDirectory trait does not clean up properly

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15014817#comment-15014817 ] Apache Spark commented on SPARK-11869: -- User 'jkbradley' has created a pull request for this issue:

[jira] [Assigned] (SPARK-11869) ML test TempDirectory trait does not clean up properly

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11869: Assignee: Apache Spark (was: Joseph K. Bradley) > ML test TempDirectory trait does not

[jira] [Assigned] (SPARK-11872) Prevent the call to SparkContext#stop() in the listener bus's thread

2015-11-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-11872: Assignee: Apache Spark > Prevent the call to SparkContext#stop() in the listener bus's

[jira] [Updated] (SPARK-11873) Regression for TPC-DS query 63 when used with decimal datatype and windows function

2015-11-19 Thread Dileep Kumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dileep Kumar updated SPARK-11873: - Attachment: 63.double_schema 63.decimal_schema_windows_function

[jira] [Updated] (SPARK-11831) AkkaRpcEnvSuite is prone to port-contention-related flakiness

2015-11-19 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-11831: -- Fix Version/s: 1.5.3 > AkkaRpcEnvSuite is prone to port-contention-related flakiness >

  1   2   3   >