[jira] [Commented] (SPARK-14460) DataFrameWriter JDBC doesn't Quote/Escape column names

2016-04-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242231#comment-15242231 ] Andrew Ash commented on SPARK-14460: I also had a problem with a column named "order" so we should

[jira] [Updated] (SPARK-12154) Upgrade to Jersey 2

2016-02-12 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-12154: --- Issue Type: Sub-task (was: Improvement) Parent: SPARK-11806 > Upgrade to Jersey 2 >

[jira] [Commented] (SPARK-12154) Upgrade to Jersey 2

2016-02-12 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15145746#comment-15145746 ] Andrew Ash commented on SPARK-12154: [~khajavi] would you please give it a go? [~mcheah] must have

[jira] [Commented] (SPARK-11081) Make spark-core pull in Jersey and javax.ws.rs dependencies separately for easier overriding

2015-12-04 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042048#comment-15042048 ] Andrew Ash commented on SPARK-11081: Agreed, Spark 2.0 is the main opportunity to bump as many

[jira] [Updated] (SPARK-1021) sortByKey() launches a cluster job when it shouldn't

2015-11-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-1021: -- Labels: (was: starter) > sortByKey() launches a cluster job when it shouldn't >

[jira] [Updated] (SPARK-10934) hashCode of unsafe array may crash

2015-10-20 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-10934: --- Summary: hashCode of unsafe array may crash (was: hashCode of unsafe array may crush) > hashCode

[jira] [Commented] (SPARK-7263) Add new shuffle manager which stores shuffle blocks in Parquet

2015-04-30 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521729#comment-14521729 ] Andrew Ash commented on SPARK-7263: --- [~massie] this is really exciting work! Thinking

[jira] [Commented] (SPARK-761) Print a nicer error message when incompatible Spark binaries try to talk

2015-04-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14491094#comment-14491094 ] Andrew Ash commented on SPARK-761: -- An implementation I can imagine would be to do an

[jira] [Updated] (SPARK-5594) SparkException: Failed to get broadcast (TorrentBroadcast)

2015-04-08 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-5594: -- Description: I am uncertain whether this is a bug, however I am getting the error below when running

[jira] [Created] (SPARK-5997) Increase partition count without performing a shuffle

2015-02-24 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-5997: - Summary: Increase partition count without performing a shuffle Key: SPARK-5997 URL: https://issues.apache.org/jira/browse/SPARK-5997 Project: Spark Issue Type:

[jira] [Updated] (SPARK-5997) Increase partition count without performing a shuffle

2015-02-24 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-5997: -- Description: When decreasing partition count with rdd.repartition() or rdd.coalesce(), the user has

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-02-18 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14325969#comment-14325969 ] Andrew Ash commented on SPARK-4879: --- That sounds exactly like what I'd expect to happen

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-02-17 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14324418#comment-14324418 ] Andrew Ash commented on SPARK-4879: --- [~romi-totango] what filesystem are you writing to?

[jira] [Commented] (SPARK-4879) Missing output partitions after job completes with speculative execution

2015-02-10 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315087#comment-14315087 ] Andrew Ash commented on SPARK-4879: --- This is really great work [~joshrosen]! I really

[jira] [Commented] (SPARK-761) Print a nicer error message when incompatible Spark binaries try to talk

2015-02-08 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14311485#comment-14311485 ] Andrew Ash commented on SPARK-761: -- Another thing could be a basic check for version

[jira] [Commented] (SPARK-1882) Support dynamic memory sharing in Mesos

2015-01-08 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14270559#comment-14270559 ] Andrew Ash commented on SPARK-1882: --- Yes -- in the simplest case where you have one

[jira] [Updated] (SPARK-4915) Wrong classname of external shuffle service in the doc for dynamic allocation

2014-12-22 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-4915: -- Fix Version/s: (was: 1.2.0) 1.2.1 Wrong classname of external shuffle service

[jira] [Commented] (SPARK-4915) Wrong classname of external shuffle service in the doc for dynamic allocation

2014-12-22 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14256084#comment-14256084 ] Andrew Ash commented on SPARK-4915: --- Changing fix version to 1.2.1 from 1.2.0 because

[jira] [Updated] (SPARK-3619) Upgrade to Mesos 0.21 to work around MESOS-1688

2014-12-19 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3619: -- Assignee: Timothy Chen Upgrade to Mesos 0.21 to work around MESOS-1688

[jira] [Commented] (SPARK-3619) Upgrade to Mesos 0.21 to work around MESOS-1688

2014-12-19 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14253150#comment-14253150 ] Andrew Ash commented on SPARK-3619: --- [~activars] Spark 1.2.0 is being released with a

[jira] [Updated] (SPARK-3619) Upgrade to Mesos 0.21 to work around MESOS-1688

2014-12-19 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3619: -- Description: The Mesos 0.21 release has a fix for https://issues.apache.org/jira/browse/MESOS-1688,

[jira] [Commented] (SPARK-4837) NettyBlockTransferService does not abide by spark.blockManager.port config option

2014-12-15 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14247030#comment-14247030 ] Andrew Ash commented on SPARK-4837: --- Ok that's fair -- a release note and targeting

[jira] [Updated] (SPARK-4837) NettyBlockTransferService does not abide by spark.blockManager.port config option

2014-12-13 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-4837: -- Priority: Blocker (was: Critical) NettyBlockTransferService does not abide by spark.blockManager.port

[jira] [Commented] (SPARK-4837) NettyBlockTransferService does not abide by spark.blockManager.port config option

2014-12-13 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14245497#comment-14245497 ] Andrew Ash commented on SPARK-4837: --- This is a regression from features that landed in

[jira] [Commented] (SPARK-3595) Spark should respect configured OutputCommitter when using saveAsHadoopFile

2014-12-12 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14244609#comment-14244609 ] Andrew Ash commented on SPARK-3595: --- Hi [~pwendell], with the past couple months of

[jira] [Commented] (SPARK-907) Add JSON endpoints to SparkUI

2014-11-25 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224811#comment-14224811 ] Andrew Ash commented on SPARK-907: -- [~dmccauley] this report looks like a duplicate of

[jira] [Commented] (SPARK-1358) Continuous integrated test should be involved in Spark ecosystem

2014-11-21 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221605#comment-14221605 ] Andrew Ash commented on SPARK-1358: --- Those machines sound more than powerful enough to

[jira] [Created] (SPARK-4437) Docs for difference between WholeTextFileRecordReader and WholeCombineFileRecordReader

2014-11-16 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-4437: - Summary: Docs for difference between WholeTextFileRecordReader and WholeCombineFileRecordReader Key: SPARK-4437 URL: https://issues.apache.org/jira/browse/SPARK-4437

[jira] [Commented] (SPARK-4399) Support multiple cloud providers

2014-11-16 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14214167#comment-14214167 ] Andrew Ash commented on SPARK-4399: --- Agreed that it does seem out of scope for what an

[jira] [Comment Edited] (SPARK-689) Task will crash when setting SPARK_WORKER_CORES 128

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207721#comment-14207721 ] Andrew Ash edited comment on SPARK-689 at 11/14/14 8:46 AM: I

[jira] [Commented] (SPARK-755) Kryo serialization failing - MLbase

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212033#comment-14212033 ] Andrew Ash commented on SPARK-755: -- Quick check [~sparks], this hasn't been updated in

[jira] [Commented] (SPARK-664) Accumulator updates should get locally merged before sent to the driver

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212046#comment-14212046 ] Andrew Ash commented on SPARK-664: -- [~irashid] it sounds like your proposal is to batch

[jira] [Commented] (SPARK-748) Add documentation page describing interoperability with other software (e.g. HBase, JDBC, Kafka, etc.)

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212049#comment-14212049 ] Andrew Ash commented on SPARK-748: -- I agree this would be valuable -- almost like a Spark

[jira] [Commented] (SPARK-625) Client hangs when connecting to standalone cluster using wrong address

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212062#comment-14212062 ] Andrew Ash commented on SPARK-625: -- Spark is very sensitive to hostnames in Spark URLs,

[jira] [Commented] (SPARK-809) Give newly registered apps a set of executors right away

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212068#comment-14212068 ] Andrew Ash commented on SPARK-809: -- I believe this situation hasn't changed? Looking a

[jira] [Updated] (SPARK-835) RDD$parallelize() should use object serializer (not closure serializer) for collection objects

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-835: - Fix Version/s: 0.8.0 RDD$parallelize() should use object serializer (not closure serializer) for

[jira] [Resolved] (SPARK-835) RDD$parallelize() should use object serializer (not closure serializer) for collection objects

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash resolved SPARK-835. -- Resolution: Fixed Closing per [~dlyubimov] with the Fix Version from SPARK-826 RDD$parallelize()

[jira] [Commented] (SPARK-904) Not able to Start/Stop Spark Worker from Remote Machine

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212078#comment-14212078 ] Andrew Ash commented on SPARK-904: -- [~ayushmishra2005] I suspect you don't have Spark

[jira] [Closed] (SPARK-904) Not able to Start/Stop Spark Worker from Remote Machine

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash closed SPARK-904. Resolution: Not a Problem Not able to Start/Stop Spark Worker from Remote Machine

[jira] [Commented] (SPARK-957) The problem that repeated computation among iterations

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212084#comment-14212084 ] Andrew Ash commented on SPARK-957: -- Hi [~caizhua], are you still having issues with your

[jira] [Commented] (SPARK-794) Remove sleep() in ClusterScheduler.stop

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212088#comment-14212088 ] Andrew Ash commented on SPARK-794: -- I don't see a {{ClusterScheduler}} class on master --

[jira] [Commented] (SPARK-665) Create RPM packages for Spark

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212094#comment-14212094 ] Andrew Ash commented on SPARK-665: -- This role of creating RPM packages seems to have been

[jira] [Closed] (SPARK-1206) Add python support for average and other summary satistics

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash closed SPARK-1206. - Resolution: Implemented Closing per [~holdenk_amp] as already done. Add python support for average and

[jira] [Commented] (SPARK-665) Create RPM packages for Spark

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212104#comment-14212104 ] Andrew Ash commented on SPARK-665: -- Sean are you suggesting dropping the .deb packages

[jira] [Closed] (SPARK-818) Design Spark Job Server

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash closed SPARK-818. Resolution: Won't Fix The community consensus was that the Spark job server should live in a separate

[jira] [Closed] (SPARK-1231) DEAD worker should recover automaticly

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash closed SPARK-1231. - Resolution: Duplicate DEAD worker should recover automaticly --

[jira] [Commented] (SPARK-1231) DEAD worker should recover automaticly

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212110#comment-14212110 ] Andrew Ash commented on SPARK-1231: --- Sorry [~tianyi], when I did my search for prior

[jira] [Commented] (SPARK-1169) Add countApproxDistinct and countApproxDistinctByKey to PySpark

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212112#comment-14212112 ] Andrew Ash commented on SPARK-1169: --- On current master (1.2) I see that rdd.py now has a

[jira] [Commented] (SPARK-754) Multiple Spark Contexts active in a single Spark Context

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212113#comment-14212113 ] Andrew Ash commented on SPARK-754: -- This is actually currently unsupported, and a ticket

[jira] [Closed] (SPARK-754) Multiple Spark Contexts active in a single Spark Context

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash closed SPARK-754. Resolution: Duplicate Multiple Spark Contexts active in a single Spark Context

[jira] [Updated] (SPARK-2243) Support multiple SparkContexts in the same JVM

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-2243: -- Affects Version/s: 0.7.0 Support multiple SparkContexts in the same JVM

[jira] [Created] (SPARK-4399) Support multiple cloud providers

2014-11-14 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-4399: - Summary: Support multiple cloud providers Key: SPARK-4399 URL: https://issues.apache.org/jira/browse/SPARK-4399 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-4400) Add scripts for launching Spark on Google Compute Engine (GCE)

2014-11-14 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-4400: - Summary: Add scripts for launching Spark on Google Compute Engine (GCE) Key: SPARK-4400 URL: https://issues.apache.org/jira/browse/SPARK-4400 Project: Spark

[jira] [Commented] (SPARK-1358) Continuous integrated test should be involved in Spark ecosystem

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212125#comment-14212125 ] Andrew Ash commented on SPARK-1358: --- I've heard these sorts of extended tests called end

[jira] [Closed] (SPARK-4400) Add scripts for launching Spark on Google Compute Engine (GCE)

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash closed SPARK-4400. - Resolution: Duplicate Add scripts for launching Spark on Google Compute Engine (GCE)

[jira] [Commented] (SPARK-928) Add support for Unsafe-based serializer in Kryo 2.22

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212132#comment-14212132 ] Andrew Ash commented on SPARK-928: -- Latest Chill (0.5.0) is still using Kryo 2.21 so this

[jira] [Commented] (SPARK-1568) Spark 0.9.0 hangs reading s3

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212145#comment-14212145 ] Andrew Ash commented on SPARK-1568: --- [~sams] did you see an improvement when you

[jira] [Commented] (SPARK-4338) Remove yarn-alpha support

2014-11-14 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14212485#comment-14212485 ] Andrew Ash commented on SPARK-4338: --- From discussion on the dev list today, Sandy aims

[jira] [Updated] (SPARK-3647) Shaded Guava patch causes access issues with package private classes

2014-11-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3647: -- Fix Version/s: 1.2.0 Shaded Guava patch causes access issues with package private classes

[jira] [Commented] (SPARK-3647) Shaded Guava patch causes access issues with package private classes

2014-11-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14206112#comment-14206112 ] Andrew Ash commented on SPARK-3647: --- Based on poking at the git repo below I'm marking

[jira] [Commented] (SPARK-572) Forbid update of static mutable variables

2014-11-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14206143#comment-14206143 ] Andrew Ash commented on SPARK-572: -- Static mutable variables are now a standard way of

[jira] [Commented] (SPARK-632) Akka system names need to be normalized (since they are case-sensitive)

2014-11-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14206158#comment-14206158 ] Andrew Ash commented on SPARK-632: -- // link moved to

[jira] [Updated] (SPARK-689) Task will crash when setting SPARK_WORKER_CORES 128

2014-11-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-689: - Description: when I set SPARK_WORKER_CORES 128(for example 200), and run a job in standalone mode that

[jira] [Commented] (SPARK-650) Add a setup hook API for running initialization code on each executor

2014-11-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14206182#comment-14206182 ] Andrew Ash commented on SPARK-650: -- As mentioned in SPARK-572 static classes'

[jira] [Commented] (SPARK-689) Task will crash when setting SPARK_WORKER_CORES 128

2014-11-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207721#comment-14207721 ] Andrew Ash commented on SPARK-689: -- I attempted a repro on a one-node cluster (my laptop)

[jira] [Comment Edited] (SPARK-689) Task will crash when setting SPARK_WORKER_CORES 128

2014-11-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207721#comment-14207721 ] Andrew Ash edited comment on SPARK-689 at 11/12/14 5:52 AM: I

[jira] [Closed] (SPARK-689) Task will crash when setting SPARK_WORKER_CORES 128

2014-11-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash closed SPARK-689. Resolution: Cannot Reproduce Task will crash when setting SPARK_WORKER_CORES 128

[jira] [Commented] (SPARK-720) Statically guarantee serialization will succeed

2014-11-11 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14207732#comment-14207732 ] Andrew Ash commented on SPARK-720: -- A big design goal of Spark is that you don't have to

[jira] [Commented] (SPARK-1882) Support dynamic memory sharing in Mesos

2014-11-10 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14205068#comment-14205068 ] Andrew Ash commented on SPARK-1882: --- Another major issue is to dynamically scale

[jira] [Updated] (SPARK-3996) Shade Jetty in Spark deliverables

2014-10-20 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3996: -- Target Version/s: 1.2.0 Affects Version/s: 1.0.2 1.1.0

[jira] [Commented] (SPARK-3996) Shade Jetty in Spark deliverables

2014-10-20 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14177736#comment-14177736 ] Andrew Ash commented on SPARK-3996: --- [~mkim] I think the practice the group's been

[jira] [Commented] (SPARK-3736) Workers should reconnect to Master if disconnected

2014-10-17 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14174753#comment-14174753 ] Andrew Ash commented on SPARK-3736: --- The configuration for Hadoop's retry policy was

[jira] [Updated] (SPARK-3940) Print the error code three times

2014-10-17 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3940: -- Description: if an error of SQL,the console print Error three times。 eg: {noformat} spark-sql show

[jira] [Updated] (SPARK-3940) Print the error code three times

2014-10-17 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3940: -- Fix Version/s: (was: 1.3.0) (was: 1.0.3) Print the error code three times

[jira] [Updated] (SPARK-3940) Print the error code three times

2014-10-17 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3940: -- Target Version/s: 1.1.1, 1.2.0 (was: 1.1.0) Print the error code three times

[jira] [Updated] (SPARK-534) Make SparkContext thread-safe

2014-10-17 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-534: - Description: SparkEnv (used by SparkContext) is not thread-safe and it causes issues with scala's Futures

[jira] [Updated] (SPARK-3466) Limit size of results that a driver collects for each action

2014-10-16 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3466: -- Assignee: Matthew Cheah Limit size of results that a driver collects for each action

[jira] [Commented] (SPARK-3819) Jenkins should compile Spark against multiple versions of Hadoop

2014-10-08 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164468#comment-14164468 ] Andrew Ash commented on SPARK-3819: --- Patrick do you think it's feasible to add to the

[jira] [Commented] (SPARK-3805) Enable Standalone worker cleanup by default

2014-10-08 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164584#comment-14164584 ] Andrew Ash commented on SPARK-3805: --- That sounds very reasonable -- I'll close. Maybe in

[jira] [Closed] (SPARK-3805) Enable Standalone worker cleanup by default

2014-10-08 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash closed SPARK-3805. - Resolution: Won't Fix Enable Standalone worker cleanup by default

[jira] [Commented] (SPARK-3135) Avoid memory copy in TorrentBroadcast serialization

2014-10-08 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164645#comment-14164645 ] Andrew Ash commented on SPARK-3135: --- [~pwendell] how do you feel about backporting this

[jira] [Commented] (SPARK-2585) Remove special handling of Hadoop JobConf

2014-10-06 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14160642#comment-14160642 ] Andrew Ash commented on SPARK-2585: --- I also vote for correct by default and there are

[jira] [Commented] (SPARK-2546) Configuration object thread safety issue

2014-10-06 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14161013#comment-14161013 ] Andrew Ash commented on SPARK-2546: --- Excellent research Josh! I agree that we should

[jira] [Commented] (SPARK-1860) Standalone Worker cleanup should not clean up running executors

2014-10-05 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159707#comment-14159707 ] Andrew Ash commented on SPARK-1860: --- Filed as SPARK-3805 Standalone Worker cleanup

[jira] [Created] (SPARK-3805) Enable Standalone worker cleanup by default

2014-10-05 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-3805: - Summary: Enable Standalone worker cleanup by default Key: SPARK-3805 URL: https://issues.apache.org/jira/browse/SPARK-3805 Project: Spark Issue Type: Task

[jira] [Commented] (SPARK-3464) Graceful decommission of executors

2014-10-05 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159712#comment-14159712 ] Andrew Ash commented on SPARK-3464: --- [~pwendell] what JIRA replaced this one? Graceful

[jira] [Updated] (SPARK-3166) Custom serialisers can't be shipped in application jars

2014-10-05 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3166: -- Target Version/s: 1.2.0 Custom serialisers can't be shipped in application jars

[jira] [Commented] (SPARK-3755) Do not bind port 1 - 1024 to server in spark

2014-10-05 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14159737#comment-14159737 ] Andrew Ash commented on SPARK-3755: --- I think you can only edit the title and description

[jira] [Updated] (SPARK-3369) Java mapPartitions Iterator-Iterable is inconsistent with Scala's Iterator-Iterator

2014-10-05 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3369: -- Labels: breaking_change (was: ) Java mapPartitions Iterator-Iterable is inconsistent with Scala's

[jira] [Commented] (SPARK-1860) Standalone Worker cleanup should not clean up running executors

2014-10-03 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14158646#comment-14158646 ] Andrew Ash commented on SPARK-1860: --- [~ilikerps] this ticket mentioned turning the

[jira] [Created] (SPARK-3750) Log ulimit settings at warning if they are too low

2014-09-30 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-3750: - Summary: Log ulimit settings at warning if they are too low Key: SPARK-3750 URL: https://issues.apache.org/jira/browse/SPARK-3750 Project: Spark Issue Type:

[jira] [Commented] (SPARK-1860) Standalone Worker cleanup should not clean up running executors

2014-09-30 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14153752#comment-14153752 ] Andrew Ash commented on SPARK-1860: --- That matches my expectations for this ticket Matt

[jira] [Commented] (SPARK-1860) Standalone Worker cleanup should not clean up running executors

2014-09-29 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152552#comment-14152552 ] Andrew Ash commented on SPARK-1860: --- Cleanup on executor shutdown is part of the

[jira] [Created] (SPARK-3736) Workers should reconnect to Master if disconnected

2014-09-29 Thread Andrew Ash (JIRA)
Andrew Ash created SPARK-3736: - Summary: Workers should reconnect to Master if disconnected Key: SPARK-3736 URL: https://issues.apache.org/jira/browse/SPARK-3736 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-3736) Workers should reconnect to Master if disconnected

2014-09-29 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Ash updated SPARK-3736: -- Affects Version/s: 1.1.0 Workers should reconnect to Master if disconnected

[jira] [Commented] (SPARK-3736) Workers should reconnect to Master if disconnected

2014-09-29 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14152594#comment-14152594 ] Andrew Ash commented on SPARK-3736: --- I can't tell for sure but this is possibly related

[jira] [Commented] (SPARK-2546) Configuration object thread safety issue

2014-09-25 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148244#comment-14148244 ] Andrew Ash commented on SPARK-2546: --- Another proposed fix: extend JobConf as a shim and

[jira] [Comment Edited] (SPARK-3032) Potential bug when running sort-based shuffle with sorting using TimSort

2014-09-25 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148411#comment-14148411 ] Andrew Ash edited comment on SPARK-3032 at 9/25/14 10:35 PM: -

[jira] [Commented] (SPARK-3032) Potential bug when running sort-based shuffle with sorting using TimSort

2014-09-25 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148411#comment-14148411 ] Andrew Ash commented on SPARK-3032: --- This bug prevents people from doing testing of

[jira] [Commented] (SPARK-2532) Fix issues with consolidated shuffle

2014-09-25 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14148666#comment-14148666 ] Andrew Ash commented on SPARK-2532: --- [~pwendell] should we close this ticket and track

<    1   2   3   4   >