[jira] [Updated] (SPARK-21249) Is it possible to use File Sink with mapGroupsWithState in Structured Streaming?

2017-06-29 Thread Amit Baghel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Baghel updated SPARK-21249: Description: I am working with 2.2.0-SNAPSHOT and Structured Streaming. Is it possible to use

[jira] [Commented] (SPARK-21249) Is it possible to use File Sink with mapGroupsWithState in Structured Streaming?

2017-06-29 Thread Amit Baghel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069528#comment-16069528 ] Amit Baghel commented on SPARK-21249: - [~hyukjin.kwon] Sorry for this jira post as I couldn't decide

[jira] [Updated] (SPARK-21249) Is it possible to use File Sink with mapGroupsWithState in Structured Streaming?

2017-06-29 Thread Amit Baghel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Baghel updated SPARK-21249: Description: I am working with 2.2.0-SNAPSHOT and Structured Streaming. Is it possible to use

[jira] [Updated] (SPARK-21249) Is it possible to use File Sink with mapGroupsWithState in Structured Streaming?

2017-06-29 Thread Amit Baghel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Amit Baghel updated SPARK-21249: Description: I am working with 2.2.0-SNAPSHOT and Structured Streaming. Is it possible to use

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-06-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069490#comment-16069490 ] Reynold Xin commented on SPARK-21190: - That makes a lot of sense. So to design APIs similar to a lot

[jira] [Resolved] (SPARK-21258) Window result incorrect using complex object with spilling

2017-06-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21258. - Resolution: Fixed Fix Version/s: 2.1.2 2.2.0 Issue resolved by pull

[jira] [Assigned] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

2017-06-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21176: --- Assignee: Ingo Schuster > Master UI hangs with spark.ui.reverseProxy=true if the master

[jira] [Commented] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069424#comment-16069424 ] Apache Spark commented on SPARK-21253: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21261) SparkSQL regexpExpressions example

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21261: Assignee: Apache Spark > SparkSQL regexpExpressions example >

[jira] [Assigned] (SPARK-21261) SparkSQL regexpExpressions example

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21261: Assignee: (was: Apache Spark) > SparkSQL regexpExpressions example >

[jira] [Commented] (SPARK-21261) SparkSQL regexpExpressions example

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069416#comment-16069416 ] Apache Spark commented on SPARK-21261: -- User 'visaxin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-21176) Master UI hangs with spark.ui.reverseProxy=true if the master node has many CPUs

2017-06-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21176. - Resolution: Fixed Fix Version/s: 2.1.2 2.2.0 Issue resolved by pull

[jira] [Assigned] (SPARK-19659) Fetch big blocks to disk when shuffle-read

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19659: Assignee: jin xing (was: Apache Spark) > Fetch big blocks to disk when shuffle-read >

[jira] [Assigned] (SPARK-19659) Fetch big blocks to disk when shuffle-read

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19659: Assignee: Apache Spark (was: jin xing) > Fetch big blocks to disk when shuffle-read >

[jira] [Updated] (SPARK-19659) Fetch big blocks to disk when shuffle-read

2017-06-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19659: - Fix Version/s: (was: 2.2.0) > Fetch big blocks to disk when shuffle-read >

[jira] [Reopened] (SPARK-19659) Fetch big blocks to disk when shuffle-read

2017-06-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reopened SPARK-19659: -- Reopened this as it's disabled. > Fetch big blocks to disk when shuffle-read >

[jira] [Commented] (SPARK-19659) Fetch big blocks to disk when shuffle-read

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069396#comment-16069396 ] Apache Spark commented on SPARK-19659: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Created] (SPARK-21261) SparkSQL regexpExpressions example

2017-06-29 Thread zhangxin (JIRA)
zhangxin created SPARK-21261: Summary: SparkSQL regexpExpressions example Key: SPARK-21261 URL: https://issues.apache.org/jira/browse/SPARK-21261 Project: Spark Issue Type: Documentation

[jira] [Resolved] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21253. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 18472

[jira] [Commented] (SPARK-20858) Document ListenerBus event queue size property

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069387#comment-16069387 ] Apache Spark commented on SPARK-20858: -- User 'sadikovi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20858) Document ListenerBus event queue size property

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20858: Assignee: (was: Apache Spark) > Document ListenerBus event queue size property >

[jira] [Assigned] (SPARK-20858) Document ListenerBus event queue size property

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20858: Assignee: Apache Spark > Document ListenerBus event queue size property >

[jira] [Assigned] (SPARK-21235) UTest should clear temp results when run case

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21235: Assignee: Apache Spark > UTest should clear temp results when run case >

[jira] [Assigned] (SPARK-21235) UTest should clear temp results when run case

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21235: Assignee: (was: Apache Spark) > UTest should clear temp results when run case >

[jira] [Commented] (SPARK-21224) Support a DDL-formatted string as schema in reading for R

2017-06-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069330#comment-16069330 ] Hyukjin Kwon commented on SPARK-21224: -- Oh! I almost missed this comment. Sure, I will soon. Yes, I

[jira] [Commented] (SPARK-21235) UTest should clear temp results when run case

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069340#comment-16069340 ] Apache Spark commented on SPARK-21235: -- User 'wangjiaochun' has created a pull request for this

[jira] [Assigned] (SPARK-21260) Remove the unused OutputFakerExec

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21260: Assignee: (was: Apache Spark) > Remove the unused OutputFakerExec >

[jira] [Assigned] (SPARK-21260) Remove the unused OutputFakerExec

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21260: Assignee: Apache Spark > Remove the unused OutputFakerExec >

[jira] [Created] (SPARK-21260) Remove the unused OutputFakerExec

2017-06-29 Thread Jiang Xingbo (JIRA)
Jiang Xingbo created SPARK-21260: Summary: Remove the unused OutputFakerExec Key: SPARK-21260 URL: https://issues.apache.org/jira/browse/SPARK-21260 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-21260) Remove the unused OutputFakerExec

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069332#comment-16069332 ] Apache Spark commented on SPARK-21260: -- User 'jiangxb1987' has created a pull request for this

[jira] [Comment Edited] (SPARK-21246) Unexpected Data Type conversion from LONG to BIGINT

2017-06-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069281#comment-16069281 ] Hyukjin Kwon edited comment on SPARK-21246 at 6/30/17 1:27 AM: --- Of course,

[jira] [Closed] (SPARK-18199) Support appending to Parquet files

2017-06-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-18199. --- Resolution: Invalid I'm closing this as invalid. It is not a good idea to append to an existing

[jira] [Resolved] (SPARK-21249) Is it possible to use File Sink with mapGroupsWithState in Structured Streaming?

2017-06-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21249. -- Resolution: Invalid I am resolving this per {{Type: Question}}. Questions should go to the

[jira] [Resolved] (SPARK-21230) Spark Encoder with mysql Enum and data truncated Error

2017-06-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21230. -- Resolution: Invalid It is hard for me to read and reproduce as well. Let's close unless you

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

2017-06-29 Thread Helena Edelson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069300#comment-16069300 ] Helena Edelson commented on SPARK-18057: IMHO kafka-0-11 to be explicit and wait until kafka

[jira] [Resolved] (SPARK-21246) Unexpected Data Type conversion from LONG to BIGINT

2017-06-29 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21246. -- Resolution: Invalid Of course, it follows the schema an user specified. {code} scala>

[jira] [Commented] (SPARK-21190) SPIP: Vectorized UDFs in Python

2017-06-29 Thread Leif Walsh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069269#comment-16069269 ] Leif Walsh commented on SPARK-21190: I agree with [~icexelloss] that we should aim to provide an API

[jira] [Closed] (SPARK-21148) Set SparkUncaughtExceptionHandler to the Master

2017-06-29 Thread Devaraj K (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K closed SPARK-21148. - Resolution: Duplicate > Set SparkUncaughtExceptionHandler to the Master >

[jira] [Commented] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069227#comment-16069227 ] Apache Spark commented on SPARK-21253: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21259) More rules for scalastyle

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21259: Assignee: (was: Apache Spark) > More rules for scalastyle > -

[jira] [Commented] (SPARK-21259) More rules for scalastyle

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069208#comment-16069208 ] Apache Spark commented on SPARK-21259: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-21259) More rules for scalastyle

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21259: Assignee: Apache Spark > More rules for scalastyle > - > >

[jira] [Updated] (SPARK-20690) Subqueries in FROM should have alias names

2017-06-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-20690: Description: We add missing attributes into Filter in Analyzer. But we shouldn't do it

[jira] [Created] (SPARK-21259) More rules for scalastyle

2017-06-29 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-21259: -- Summary: More rules for scalastyle Key: SPARK-21259 URL: https://issues.apache.org/jira/browse/SPARK-21259 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-20690) Subqueries in FROM should have alias names

2017-06-29 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-20690: Summary: Subqueries in FROM should have alias names (was: Analyzer shouldn't add missing

[jira] [Resolved] (SPARK-21188) releaseAllLocksForTask should synchronize the whole method

2017-06-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-21188. -- Resolution: Fixed Assignee: Feng Liu Fix Version/s: 2.3.0 >

[jira] [Commented] (SPARK-21255) NPE when creating encoder for enum

2017-06-29 Thread Mike (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069146#comment-16069146 ] Mike commented on SPARK-21255: -- Yes, but there may be other problems (at least I cannot guarantee there

[jira] [Commented] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069109#comment-16069109 ] Yuming Wang commented on SPARK-21253: - I checked it, all jars are latest 2.2.0-rcX. > Cannot fetch

[jira] [Commented] (SPARK-21258) Window result incorrect using complex object with spilling

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16069084#comment-16069084 ] Apache Spark commented on SPARK-21258: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Assigned] (SPARK-21258) Window result incorrect using complex object with spilling

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21258: Assignee: Herman van Hovell (was: Apache Spark) > Window result incorrect using complex

[jira] [Assigned] (SPARK-21258) Window result incorrect using complex object with spilling

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21258: Assignee: Apache Spark (was: Herman van Hovell) > Window result incorrect using complex

[jira] [Created] (SPARK-21258) Window result incorrect using complex object with spilling

2017-06-29 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-21258: - Summary: Window result incorrect using complex object with spilling Key: SPARK-21258 URL: https://issues.apache.org/jira/browse/SPARK-21258 Project: Spark

[jira] [Commented] (SPARK-20797) mllib lda's LocalLDAModel's save: out of memory.

2017-06-29 Thread Asher Krim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068988#comment-16068988 ] Asher Krim commented on SPARK-20797: This looks like a duplicate of

[jira] [Comment Edited] (SPARK-20797) mllib lda's LocalLDAModel's save: out of memory.

2017-06-29 Thread Asher Krim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068988#comment-16068988 ] Asher Krim edited comment on SPARK-20797 at 6/29/17 9:19 PM: - This looks like

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

2017-06-29 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068980#comment-16068980 ] Cody Koeninger commented on SPARK-18057: Kafka 0.11 is now released. Are we upgrading spark

[jira] [Commented] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068876#comment-16068876 ] Shixiong Zhu commented on SPARK-21253: -- [~q79969786] did you run Spark 2.2.0-rcX on Yarn which has a

[jira] [Updated] (SPARK-21257) LDA : create an Evaluator to enable cross validation

2017-06-29 Thread Mathieu DESPRIEE (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mathieu DESPRIEE updated SPARK-21257: - Issue Type: Improvement (was: New Feature) > LDA : create an Evaluator to enable cross

[jira] [Updated] (SPARK-21257) LDA : create an Evaluator to enable cross validation

2017-06-29 Thread Mathieu DESPRIEE (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mathieu DESPRIEE updated SPARK-21257: - Issue Type: New Feature (was: Improvement) > LDA : create an Evaluator to enable cross

[jira] [Created] (SPARK-21257) LDA : create an Evaluator to enable cross validation

2017-06-29 Thread Mathieu DESPRIEE (JIRA)
Mathieu DESPRIEE created SPARK-21257: Summary: LDA : create an Evaluator to enable cross validation Key: SPARK-21257 URL: https://issues.apache.org/jira/browse/SPARK-21257 Project: Spark

[jira] [Assigned] (SPARK-21256) Add WithSQLConf to Catalyst Test

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21256: Assignee: Xiao Li (was: Apache Spark) > Add WithSQLConf to Catalyst Test >

[jira] [Assigned] (SPARK-21256) Add WithSQLConf to Catalyst Test

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21256: Assignee: Apache Spark (was: Xiao Li) > Add WithSQLConf to Catalyst Test >

[jira] [Commented] (SPARK-21256) Add WithSQLConf to Catalyst Test

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068832#comment-16068832 ] Apache Spark commented on SPARK-21256: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Updated] (SPARK-21256) Add WithSQLConf to Catalyst Test

2017-06-29 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-21256: Summary: Add WithSQLConf to Catalyst Test (was: Add WithSQLConf to Catalyst) > Add WithSQLConf to

[jira] [Created] (SPARK-21256) Add WithSQLConf to Catalyst

2017-06-29 Thread Xiao Li (JIRA)
Xiao Li created SPARK-21256: --- Summary: Add WithSQLConf to Catalyst Key: SPARK-21256 URL: https://issues.apache.org/jira/browse/SPARK-21256 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-20873) Improve the error message for unsupported Column Type

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068800#comment-16068800 ] Apache Spark commented on SPARK-20873: -- User 'kiszk' has created a pull request for this issue:

[jira] [Commented] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068793#comment-16068793 ] Apache Spark commented on SPARK-21253: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Updated] (SPARK-20783) Enhance ColumnVector to support compressed representation

2017-06-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-20783: - Description: Current {{ColumnVector}} handles uncompressed data for parquet. For

[jira] [Updated] (SPARK-20783) Enhance ColumnVector to support compressed representation

2017-06-29 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-20783: - Summary: Enhance ColumnVector to support compressed representation (was: Enhance

[jira] [Updated] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-21253: - Target Version/s: 2.2.0 > Cannot fetch big blocks to disk >

[jira] [Assigned] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-21253: Assignee: Shixiong Zhu > Cannot fetch big blocks to disk >

[jira] [Commented] (SPARK-4131) Support "Writing data into the filesystem from queries"

2017-06-29 Thread ARUNA KIRAN NULU (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068700#comment-16068700 ] ARUNA KIRAN NULU commented on SPARK-4131: - Is this feature available ? > Support "Writing data

[jira] [Commented] (SPARK-21246) Unexpected Data Type conversion from LONG to BIGINT

2017-06-29 Thread Monica Raj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068553#comment-16068553 ] Monica Raj commented on SPARK-21246: Thanks for your response. I also tried with Seq(3) as Seq(3L),

[jira] [Resolved] (SPARK-21254) History UI: Taking over 1 minute for initial page display

2017-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21254. --- Resolution: Duplicate Duplicate of lots of JIRAs related to making the initial read faster >

[jira] [Commented] (SPARK-21255) NPE when creating encoder for enum

2017-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068520#comment-16068520 ] Sean Owen commented on SPARK-21255: --- Is the change to omit declaringClass too? > NPE when creating

[jira] [Updated] (SPARK-21255) NPE when creating encoder for enum

2017-06-29 Thread Mike (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike updated SPARK-21255: - Description: When you try to create an encoder for Enum type (or bean with enum property) via

[jira] [Created] (SPARK-21255) NPE when creating encoder for enum

2017-06-29 Thread Mike (JIRA)
Mike created SPARK-21255: Summary: NPE when creating encoder for enum Key: SPARK-21255 URL: https://issues.apache.org/jira/browse/SPARK-21255 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-21254) History UI: Taking over 1 minute for initial page display

2017-06-29 Thread Dmitry Parfenchik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Parfenchik updated SPARK-21254: -- Priority: Minor (was: Major) > History UI: Taking over 1 minute for initial page

[jira] [Updated] (SPARK-21254) History UI: Taking over 1 minute for initial page display

2017-06-29 Thread Dmitry Parfenchik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Parfenchik updated SPARK-21254: -- Description: Currently on the first page load (if there is no limit set) the whole

[jira] [Updated] (SPARK-21254) History UI: Taking over 1 minute for initial page display

2017-06-29 Thread Dmitry Parfenchik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Parfenchik updated SPARK-21254: -- Description: Currently on the first page load (if there is no limit set) the whole

[jira] [Updated] (SPARK-21254) History UI: Taking over 1 minute for initial page display

2017-06-29 Thread Dmitry Parfenchik (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Parfenchik updated SPARK-21254: -- Attachment: screenshot-1.png > History UI: Taking over 1 minute for initial page

[jira] [Created] (SPARK-21254) History UI: Taking over 1 minute for initial page display

2017-06-29 Thread Dmitry Parfenchik (JIRA)
Dmitry Parfenchik created SPARK-21254: - Summary: History UI: Taking over 1 minute for initial page display Key: SPARK-21254 URL: https://issues.apache.org/jira/browse/SPARK-21254 Project: Spark

[jira] [Resolved] (SPARK-21052) Add hash map metrics to join

2017-06-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21052. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18301

[jira] [Assigned] (SPARK-21052) Add hash map metrics to join

2017-06-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21052: --- Assignee: Liang-Chi Hsieh > Add hash map metrics to join > > >

[jira] [Commented] (SPARK-12868) ADD JAR via sparkSQL JDBC will fail when using a HDFS URL

2017-06-29 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068301#comment-16068301 ] Steve Loughran commented on SPARK-12868: It's actually not the cause of that, merely the

[jira] [Updated] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-21253: Description: Spark *cluster* can reproduce, *local* can't: 1. Start a spark context with

[jira] [Assigned] (SPARK-21225) decrease the Mem using for variable 'tasks' in function resourceOffers

2017-06-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-21225: --- Assignee: yangZhiguo > decrease the Mem using for variable 'tasks' in function

[jira] [Resolved] (SPARK-21225) decrease the Mem using for variable 'tasks' in function resourceOffers

2017-06-29 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-21225. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18435

[jira] [Updated] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-21253: Description: Spark *cluster* can reproduce, *local* can't: 1. Start a spark context with

[jira] [Commented] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068232#comment-16068232 ] Yuming Wang commented on SPARK-21253: - It may be hang for a {{spark-sql}} application also:

[jira] [Updated] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-21253: Attachment: ui-thread-dump-jqhadoop221-154.gif > Cannot fetch big blocks to disk >

[jira] [Updated] (SPARK-21252) The duration times showed by spark web UI are inaccurate

2017-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-21252: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) I don't know what the

[jira] [Assigned] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21253: Assignee: Apache Spark > Cannot fetch big blocks to disk >

[jira] [Assigned] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21253: Assignee: (was: Apache Spark) > Cannot fetch big blocks to disk >

[jira] [Commented] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068169#comment-16068169 ] Apache Spark commented on SPARK-21253: -- User 'wangyum' has created a pull request for this issue:

[jira] [Created] (SPARK-21253) Cannot fetch big blocks to disk

2017-06-29 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-21253: --- Summary: Cannot fetch big blocks to disk Key: SPARK-21253 URL: https://issues.apache.org/jira/browse/SPARK-21253 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-21252) The duration times showed by spark web UI are inaccurate

2017-06-29 Thread igor mazor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068147#comment-16068147 ] igor mazor edited comment on SPARK-21252 at 6/29/17 10:42 AM: -- Yes, its

[jira] [Commented] (SPARK-21252) The duration times showed by spark web UI are inaccurate

2017-06-29 Thread igor mazor (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068147#comment-16068147 ] igor mazor commented on SPARK-21252: Yes, its indeed possible to drill to individual stages, however

[jira] [Resolved] (SPARK-17689) _temporary files breaks the Spark SQL streaming job.

2017-06-29 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma resolved SPARK-17689. - Resolution: Cannot Reproduce I am unable to reproduce this anymore, looks like this

[jira] [Commented] (SPARK-19908) Direct buffer memory OOM should not cause stage retries.

2017-06-29 Thread Kaushal Prajapati (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068106#comment-16068106 ] Kaushal Prajapati commented on SPARK-19908: --- [~zhanzhang] can you plz share some example code

[jira] [Commented] (SPARK-21252) The duration times showed by spark web UI are inaccurate

2017-06-29 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16068085#comment-16068085 ] Sean Owen commented on SPARK-21252: --- You can drill into individual stage times, right? > The duration

  1   2   >