[jira] [Created] (SPARK-6932) A Prototype of Parameter Server

2015-04-15 Thread Qiping Li (JIRA)
Qiping Li created SPARK-6932: Summary: A Prototype of Parameter Server Key: SPARK-6932 URL: https://issues.apache.org/jira/browse/SPARK-6932 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-6933) Thrift Server couldn't strip .inprogress suffix after being stopped

2015-04-15 Thread Tao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Wang updated SPARK-6933: Affects Version/s: 1.3.0 Thrift Server couldn't strip .inprogress suffix after being stopped

[jira] [Assigned] (SPARK-6846) Stage kill URL easy to accidentally trigger and possibility for security issue.

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6846: --- Assignee: Apache Spark Stage kill URL easy to accidentally trigger and possibility for

[jira] [Assigned] (SPARK-6846) Stage kill URL easy to accidentally trigger and possibility for security issue.

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6846: --- Assignee: (was: Apache Spark) Stage kill URL easy to accidentally trigger and

[jira] [Commented] (SPARK-6846) Stage kill URL easy to accidentally trigger and possibility for security issue.

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496222#comment-14496222 ] Apache Spark commented on SPARK-6846: - User 'srowen' has created a pull request for

[jira] [Updated] (SPARK-6932) A Prototype of Parameter Server

2015-04-15 Thread Qiping Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiping Li updated SPARK-6932: - Description: h2. Introduction As specified in

[jira] [Assigned] (SPARK-5352) Add getPartitionStrategy in Graph

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5352: --- Assignee: Apache Spark Add getPartitionStrategy in Graph -

[jira] [Assigned] (SPARK-5352) Add getPartitionStrategy in Graph

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5352: --- Assignee: (was: Apache Spark) Add getPartitionStrategy in Graph

[jira] [Assigned] (SPARK-6857) Python SQL schema inference should support numpy types

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6857: --- Assignee: (was: Apache Spark) Python SQL schema inference should support numpy types

[jira] [Updated] (SPARK-6932) A Prototype of Parameter Server

2015-04-15 Thread Qiping Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiping Li updated SPARK-6932: - Description: h2. Introduction As specified in

[jira] [Assigned] (SPARK-5697) Allow Spark driver to wait longer before giving up connecting to the master

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5697: --- Assignee: Apache Spark Allow Spark driver to wait longer before giving up connecting to the

[jira] [Assigned] (SPARK-5697) Allow Spark driver to wait longer before giving up connecting to the master

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5697: --- Assignee: (was: Apache Spark) Allow Spark driver to wait longer before giving up

[jira] [Assigned] (SPARK-5484) Pregel should checkpoint periodically to avoid StackOverflowError

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5484: --- Assignee: Apache Spark (was: Ankur Dave) Pregel should checkpoint periodically to avoid

[jira] [Assigned] (SPARK-3580) Add Consistent Method To Get Number of RDD Partitions Across Different Languages

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3580: --- Assignee: (was: Apache Spark) Add Consistent Method To Get Number of RDD Partitions

[jira] [Assigned] (SPARK-3580) Add Consistent Method To Get Number of RDD Partitions Across Different Languages

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-3580: --- Assignee: Apache Spark Add Consistent Method To Get Number of RDD Partitions Across

[jira] [Assigned] (SPARK-6934) Fix the bug that using a wrong configuration for “ask” timeout in RpcEnv

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6934: --- Assignee: Apache Spark Fix the bug that using a wrong configuration for “ask” timeout in

[jira] [Commented] (SPARK-6933) Thrift Server couldn't strip .inprogress suffix after being stopped

2015-04-15 Thread Tao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496200#comment-14496200 ] Tao Wang commented on SPARK-6933: - BTW the event log file on hdfs is still with

[jira] [Assigned] (SPARK-6857) Python SQL schema inference should support numpy types

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6857: --- Assignee: Apache Spark Python SQL schema inference should support numpy types

[jira] [Commented] (SPARK-6857) Python SQL schema inference should support numpy types

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496201#comment-14496201 ] Apache Spark commented on SPARK-6857: - User 'viirya' has created a pull request for

[jira] [Resolved] (SPARK-6861) Scalastyle config prevents building Maven child modules alone

2015-04-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-6861. -- Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5471

[jira] [Assigned] (SPARK-5112) Expose SizeEstimator as a developer API

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5112: --- Assignee: Sandy Ryza (was: Apache Spark) Expose SizeEstimator as a developer API

[jira] [Assigned] (SPARK-5112) Expose SizeEstimator as a developer API

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5112: --- Assignee: Apache Spark (was: Sandy Ryza) Expose SizeEstimator as a developer API

[jira] [Updated] (SPARK-6932) A Prototype of Parameter Server

2015-04-15 Thread Qiping Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiping Li updated SPARK-6932: - Description: h2. Introduction As specified in

[jira] [Commented] (SPARK-6932) A Prototype of Parameter Server

2015-04-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496098#comment-14496098 ] Sean Owen commented on SPARK-6932: -- Why is this a separate JIRA rather than, say, a

[jira] [Assigned] (SPARK-5484) Pregel should checkpoint periodically to avoid StackOverflowError

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5484: --- Assignee: Ankur Dave (was: Apache Spark) Pregel should checkpoint periodically to avoid

[jira] [Created] (SPARK-6933) Thrift Server couldn't strip .inprogress suffix after being stopped

2015-04-15 Thread Tao Wang (JIRA)
Tao Wang created SPARK-6933: --- Summary: Thrift Server couldn't strip .inprogress suffix after being stopped Key: SPARK-6933 URL: https://issues.apache.org/jira/browse/SPARK-6933 Project: Spark

[jira] [Created] (SPARK-6934) Fix the bug that using a wrong configuration for “ask” timeout in RpcEnv

2015-04-15 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-6934: --- Summary: Fix the bug that using a wrong configuration for “ask” timeout in RpcEnv Key: SPARK-6934 URL: https://issues.apache.org/jira/browse/SPARK-6934 Project: Spark

[jira] [Assigned] (SPARK-5623) Replace an obsolete mapReduceTriplets with a new aggregateMessages in GraphSuite

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5623: --- Assignee: Apache Spark Replace an obsolete mapReduceTriplets with a new aggregateMessages

[jira] [Assigned] (SPARK-5623) Replace an obsolete mapReduceTriplets with a new aggregateMessages in GraphSuite

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5623: --- Assignee: (was: Apache Spark) Replace an obsolete mapReduceTriplets with a new

[jira] [Commented] (SPARK-6934) Fix the bug that using a wrong configuration for “ask” timeout in RpcEnv

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496223#comment-14496223 ] Apache Spark commented on SPARK-6934: - User 'zsxwing' has created a pull request for

[jira] [Assigned] (SPARK-6934) Fix the bug that using a wrong configuration for “ask” timeout in RpcEnv

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6934: --- Assignee: (was: Apache Spark) Fix the bug that using a wrong configuration for “ask”

[jira] [Commented] (SPARK-6889) Streamline contribution process with update to Contribution wiki, JIRA rules

2015-04-15 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496251#comment-14496251 ] Imran Rashid commented on SPARK-6889: - +1 on the proposed changes. I think that

[jira] [Updated] (SPARK-6777) Implement backwards-compatibility rules in Parquet schema converters

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6777: -- Priority: Critical (was: Major) Implement backwards-compatibility rules in Parquet schema converters

[jira] [Updated] (SPARK-6759) Do not borrow/release a kryo instance for every value in a complex type value when doing serialization/deserialization in in-memory columnar store

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6759: -- Priority: Critical (was: Major) Do not borrow/release a kryo instance for every value in a complex

[jira] [Commented] (SPARK-3702) Standardize MLlib classes for learners, models

2015-04-15 Thread Daniel Erenrich (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496274#comment-14496274 ] Daniel Erenrich commented on SPARK-3702: One thing I'm interested in along these

[jira] [Assigned] (SPARK-6581) Metadata is missing when saving parquet file using hadoop 1.0.4

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-6581: - Assignee: Cheng Lian Metadata is missing when saving parquet file using hadoop 1.0.4

[jira] [Assigned] (SPARK-6570) Spark SQL arrays: explode() fails and cannot save array type to Parquet

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-6570: - Assignee: Cheng Lian Spark SQL arrays: explode() fails and cannot save array type to Parquet

[jira] [Resolved] (SPARK-5697) Allow Spark driver to wait longer before giving up connecting to the master

2015-04-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah resolved SPARK-5697. --- Resolution: Won't Fix Allow Spark driver to wait longer before giving up connecting to the master

[jira] [Created] (SPARK-6935) spark/spark-ec2.py add parameters to give different instance types for master and slaves

2015-04-15 Thread Oleksii Mandrychenko (JIRA)
Oleksii Mandrychenko created SPARK-6935: --- Summary: spark/spark-ec2.py add parameters to give different instance types for master and slaves Key: SPARK-6935 URL:

[jira] [Commented] (SPARK-3702) Standardize MLlib classes for learners, models

2015-04-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496324#comment-14496324 ] Joseph K. Bradley commented on SPARK-3702: -- That's supported via the

[jira] [Updated] (SPARK-6581) Metadata is missing when saving parquet file using hadoop 1.0.4

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6581: -- Priority: Critical (was: Major) Metadata is missing when saving parquet file using hadoop 1.0.4

[jira] [Commented] (SPARK-6889) Streamline contribution process with update to Contribution wiki, JIRA rules

2015-04-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496373#comment-14496373 ] Nicholas Chammas commented on SPARK-6889: - {quote} I think that really the most

[jira] [Updated] (SPARK-6482) Remove synchronization of Hive Native commands

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6482: -- Priority: Critical (was: Major) Remove synchronization of Hive Native commands

[jira] [Commented] (SPARK-6935) spark/spark-ec2.py add parameters to give different instance types for master and slaves

2015-04-15 Thread Oleksii Mandrychenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496453#comment-14496453 ] Oleksii Mandrychenko commented on SPARK-6935: - Added support for default flag

[jira] [Updated] (SPARK-6935) spark/spark-ec2.py add parameters to give different instance types for master and slaves

2015-04-15 Thread Oleksii Mandrychenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksii Mandrychenko updated SPARK-6935: Description: I want to start a cluster where I give beefy AWS instances to slaves,

[jira] [Created] (SPARK-6936) SQLContext.sql() caused deadlock in multi-thread env

2015-04-15 Thread Paul Wu (JIRA)
Paul Wu created SPARK-6936: -- Summary: SQLContext.sql() caused deadlock in multi-thread env Key: SPARK-6936 URL: https://issues.apache.org/jira/browse/SPARK-6936 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-4629) Spark SQL uses Hadoop Configuration in a thread-unsafe manner when writing Parquet files

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-4629: - Assignee: Cheng Lian Spark SQL uses Hadoop Configuration in a thread-unsafe manner when writing

[jira] [Updated] (SPARK-4629) Spark SQL uses Hadoop Configuration in a thread-unsafe manner when writing Parquet files

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4629: -- Priority: Critical (was: Major) Spark SQL uses Hadoop Configuration in a thread-unsafe manner when

[jira] [Commented] (SPARK-6012) Deadlock when asking for partitions from CoalescedRDD on top of a TakeOrdered operator

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496469#comment-14496469 ] Yin Huai commented on SPARK-6012: - [~maxseiden] Can you try 1.3 and see if this issue has

[jira] [Resolved] (SPARK-6217) insertInto doesn't work in PySpark

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-6217. - Resolution: Not A Problem We have not implemented the support for inserting into a table created from

[jira] [Updated] (SPARK-4521) Parquet fails to read columns with spaces in the name

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4521: -- Description: I think this is actually a bug in parquet, but it would be good to track it here as well.

[jira] [Updated] (SPARK-4944) Table Not Found exception in Create Table Like registered RDD table

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4944: -- Description: {code} rdd_table.saveAsParquetFile(/user/spark/my_data.parquet)

[jira] [Commented] (SPARK-6916) StringIndexer should preserve non-ML metadata

2015-04-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496456#comment-14496456 ] Joseph K. Bradley commented on SPARK-6916: -- Yeah, I guess I don't know what users

[jira] [Commented] (SPARK-6113) Stabilize DecisionTree and ensembles APIs

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496475#comment-14496475 ] Apache Spark commented on SPARK-6113: - User 'jkbradley' has created a pull request for

[jira] [Commented] (SPARK-6889) Streamline contribution process with update to Contribution wiki, JIRA rules

2015-04-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496481#comment-14496481 ] Sean Owen commented on SPARK-6889: -- We can take away old docs that encourage people to

[jira] [Assigned] (SPARK-6937) [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6937: --- Assignee: (was: Apache Spark) [MLLIB] Tiny bug in PowerIterationClusteringExample in

[jira] [Assigned] (SPARK-6937) [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6937: --- Assignee: Apache Spark [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius

[jira] [Commented] (SPARK-6937) [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496523#comment-14496523 ] Apache Spark commented on SPARK-6937: - User 'javadba' has created a pull request for

[jira] [Commented] (SPARK-6933) Thrift Server couldn't strip .inprogress suffix after being stopped

2015-04-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496474#comment-14496474 ] Marcelo Vanzin commented on SPARK-6933: --- I think this is actually caused by the same

[jira] [Updated] (SPARK-6570) Spark SQL arrays: explode() fails and cannot save array type to Parquet

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6570: -- Priority: Critical (was: Major) Spark SQL arrays: explode() fails and cannot save array type to

[jira] [Commented] (SPARK-6935) spark/spark-ec2.py add parameters to give different instance types for master and slaves

2015-04-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496443#comment-14496443 ] Sean Owen commented on SPARK-6935: -- Sounds pretty reasonable, though you may need to

[jira] [Commented] (SPARK-6933) Thrift Server couldn't strip .inprogress suffix after being stopped

2015-04-15 Thread Tao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496437#comment-14496437 ] Tao Wang commented on SPARK-6933: - P.P.S: Tested with SparkPi, it worked fine. Now this

[jira] [Updated] (SPARK-6432) Cannot load parquet data with partitions if not all partition columns match data columns

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6432: -- Priority: Critical (was: Major) Cannot load parquet data with partitions if not all partition columns

[jira] [Commented] (SPARK-6548) Adding stddev to DataFrame functions

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496477#comment-14496477 ] Cheng Lian commented on SPARK-6548: --- Hey [~dreamquster], are you still working on this?

[jira] [Updated] (SPARK-5948) Support writing to partitioned table for the Parquet data source

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5948: -- Priority: Blocker (was: Major) Support writing to partitioned table for the Parquet data source

[jira] [Updated] (SPARK-5947) First class partitioning support in data sources API

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5947: -- Priority: Blocker (was: Major) First class partitioning support in data sources API

[jira] [Assigned] (SPARK-6123) Parquet reader should use the schema of every file to create converter

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-6123: - Assignee: Cheng Lian Parquet reader should use the schema of every file to create converter

[jira] [Commented] (SPARK-6217) insertInto doesn't work in PySpark

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496522#comment-14496522 ] Yin Huai commented on SPARK-6217: - [~cpcloud] Right now, we do not support inserting into

[jira] [Updated] (SPARK-6123) Parquet reader should use the schema of every file to create converter

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6123: -- Priority: Critical (was: Major) Parquet reader should use the schema of every file to create

[jira] [Created] (SPARK-6937) [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line

2015-04-15 Thread Stephen Boesch (JIRA)
Stephen Boesch created SPARK-6937: - Summary: [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line Key: SPARK-6937 URL:

[jira] [Assigned] (SPARK-5251) Using `tableIdentifier` in hive metastore

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5251: --- Assignee: (was: Apache Spark) Using `tableIdentifier` in hive metastore

[jira] [Assigned] (SPARK-5251) Using `tableIdentifier` in hive metastore

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5251: --- Assignee: Apache Spark Using `tableIdentifier` in hive metastore

[jira] [Commented] (SPARK-4521) Parquet fails to read columns with spaces in the name

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496542#comment-14496542 ] Cheng Lian commented on SPARK-4521: --- This is because Parquet {{MessageTypeParser}}

[jira] [Commented] (SPARK-3937) Unsafe memory access inside of Snappy library

2015-04-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496550#comment-14496550 ] Josh Rosen commented on SPARK-3937: --- [~witgo], is there any way to reproduce this

[jira] [Commented] (SPARK-4854) Custom UDTF with Lateral View throws ClassNotFound exception in Spark SQL CLI

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496552#comment-14496552 ] Cheng Lian commented on SPARK-4854: --- [~wanshenghua] Would you mind to verify whether [PR

[jira] [Assigned] (SPARK-4176) Support decimals with precision 18 in Parquet

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-4176: - Assignee: Cheng Lian Support decimals with precision 18 in Parquet

[jira] [Commented] (SPARK-6831) Document how to use external data sources

2015-04-15 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496548#comment-14496548 ] Shivaram Venkataraman commented on SPARK-6831: -- I think we should give an

[jira] [Updated] (SPARK-6774) Implement Parquet complex types backwards-compatiblity rules

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6774: -- Priority: Critical (was: Major) Implement Parquet complex types backwards-compatiblity rules

[jira] [Updated] (SPARK-6694) SparkSQL CLI must be able to specify an option --database on the command line.

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6694: -- Priority: Critical (was: Major) SparkSQL CLI must be able to specify an option --database on the

[jira] [Resolved] (SPARK-4804) StringContext method to allow using Strings for column names in catalyst DSL

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-4804. - Resolution: Duplicate I has been resolved by SPARK-5040. StringContext method to allow using Strings

[jira] [Created] (SPARK-6938) Add informative error messages to require statements.

2015-04-15 Thread Juliet Hougland (JIRA)
Juliet Hougland created SPARK-6938: -- Summary: Add informative error messages to require statements. Key: SPARK-6938 URL: https://issues.apache.org/jira/browse/SPARK-6938 Project: Spark

[jira] [Resolved] (SPARK-2984) FileNotFoundException on _temporary directory

2015-04-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2984. --- Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Josh Rosen For speculative tasks,

[jira] [Commented] (SPARK-4967) File name with comma will cause exception for SQLContext.parquetFile

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496635#comment-14496635 ] Yin Huai commented on SPARK-4967: - For new parquet, because we only accept a single path

[jira] [Updated] (SPARK-6869) Pass PYTHONPATH to executor, so that executor can read pyspark file from local file system on executor node

2015-04-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6869: - Affects Version/s: (was: 1.1.0) 1.0.0 Pass PYTHONPATH to executor, so that

[jira] [Updated] (SPARK-4176) Support decimals with precision 18 in Parquet

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4176: -- Priority: Major (was: Critical) Support decimals with precision 18 in Parquet

[jira] [Created] (SPARK-6939) Refactoring existing batch statistics into the new UI

2015-04-15 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-6939: --- Summary: Refactoring existing batch statistics into the new UI Key: SPARK-6939 URL: https://issues.apache.org/jira/browse/SPARK-6939 Project: Spark Issue

[jira] [Updated] (SPARK-6937) Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line

2015-04-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6937: - Summary: Tiny bug in PowerIterationClusteringExample in which radius not accepted from

[jira] [Updated] (SPARK-6915) VectorIndexer improvements

2015-04-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6915: - Description: This covers several improvements to VectorIndexer. They could be handled

[jira] [Closed] (SPARK-4967) File name with comma will cause exception for SQLContext.parquetFile

2015-04-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao closed SPARK-4967. Resolution: Won't Fix File name with comma will cause exception for SQLContext.parquetFile

[jira] [Commented] (SPARK-4521) Parquet fails to read columns with spaces in the name

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496646#comment-14496646 ] Yin Huai commented on SPARK-4521: - https://github.com/apache/spark/pull/5263 is for

[jira] [Created] (SPARK-6940) PySpark ML.Tuning Wrappers are missing

2015-04-15 Thread Omede Firouz (JIRA)
Omede Firouz created SPARK-6940: --- Summary: PySpark ML.Tuning Wrappers are missing Key: SPARK-6940 URL: https://issues.apache.org/jira/browse/SPARK-6940 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-6940) PySpark ML.Tuning Wrappers are missing

2015-04-15 Thread Omede Firouz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496672#comment-14496672 ] Omede Firouz commented on SPARK-6940: - I'm beginning work on this ticket, please let

[jira] [Assigned] (SPARK-6938) Add informative error messages to require statements.

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6938: --- Assignee: Apache Spark Add informative error messages to require statements.

[jira] [Commented] (SPARK-6938) Add informative error messages to require statements.

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496575#comment-14496575 ] Apache Spark commented on SPARK-6938: - User 'jhlch' has created a pull request for

[jira] [Assigned] (SPARK-6938) Add informative error messages to require statements.

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6938: --- Assignee: (was: Apache Spark) Add informative error messages to require statements.

[jira] [Closed] (SPARK-6916) StringIndexer should preserve non-ML metadata

2015-04-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-6916. Resolution: Not A Problem Target Version/s: (was: 1.4.0) StringIndexer should

[jira] [Commented] (SPARK-5133) Feature Importance for Decision Tree (Ensembles)

2015-04-15 Thread Parv Oberoi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496625#comment-14496625 ] Parv Oberoi commented on SPARK-5133: this would be a really useful feature to have in

[jira] [Commented] (SPARK-4944) Table Not Found exception in Create Table Like registered RDD table

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496641#comment-14496641 ] Yin Huai commented on SPARK-4944: - Seems we have to handle Create Table Like in Spark SQL

[jira] [Updated] (SPARK-6869) Pass PYTHONPATH to executor, so that executor can read pyspark file from local file system on executor node

2015-04-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6869: - Affects Version/s: 1.1.0 Pass PYTHONPATH to executor, so that executor can read pyspark file from

[jira] [Resolved] (SPARK-6657) Fix Python doc build warnings

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-6657. - Resolution: Fixed Fix Version/s: 1.3.1 The pr has been merged. I am resolving it. Fix Python doc

  1   2   3   >