[jira] [Created] (SPARK-23933) High-order function: map(array, array) → map

2018-04-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-23933: --- Summary: High-order function: map(array, array) → map Key: SPARK-23933 URL: https://issues.apache.org/jira/browse/SPARK-23933 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-23934) High-order function: map_from_entries(array>) → map

2018-04-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-23934: --- Summary: High-order function: map_from_entries(array>) → map Key: SPARK-23934 URL: https://issues.apache.org/jira/browse/SPARK-23934 Project: Spark Issue Type: Sub-ta

[jira] [Updated] (SPARK-23934) High-order function: map_from_entries(array>) → map

2018-04-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23934: Description: Ref: https://prestodb.io/docs/current/functions/map.html Returns a map created from the given

[jira] [Created] (SPARK-23935) High-order function: map_entries(map) → array>

2018-04-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-23935: --- Summary: High-order function: map_entries(map) → array> Key: SPARK-23935 URL: https://issues.apache.org/jira/browse/SPARK-23935 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-23937) High-order function: map_filter(map, function) → MAP

2018-04-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-23937: --- Summary: High-order function: map_filter(map, function) → MAP Key: SPARK-23937 URL: https://issues.apache.org/jira/browse/SPARK-23937 Project: Spark Issue Type: Sub-ta

[jira] [Created] (SPARK-23936) High-order function: map_concat(map1, map2, ..., mapN) → map

2018-04-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-23936: --- Summary: High-order function: map_concat(map1, map2, ..., mapN) → map Key: SPARK-23936 URL: https://issues.apache.org/jira/browse/SPARK-23936 Project: Spark Issue Typ

[jira] [Created] (SPARK-23938) High-order function: map_zip_with(map, map, function) → map

2018-04-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-23938: --- Summary: High-order function: map_zip_with(map, map, function) → map Key: SPARK-23938 URL: https://issues.apache.org/jira/browse/SPARK-23938 Project: Spark Issue Type

[jira] [Created] (SPARK-23939) High-order function: transform_keys(map, function) → map

2018-04-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-23939: --- Summary: High-order function: transform_keys(map, function) → map Key: SPARK-23939 URL: https://issues.apache.org/jira/browse/SPARK-23939 Project: Spark Issue Type: S

[jira] [Updated] (SPARK-23939) High-order function: transform_keys(map, function) → map

2018-04-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23939: Description: Ref: https://prestodb.io/docs/current/functions/map.html Returns a map that applies function

[jira] [Created] (SPARK-23940) High-ofer function: transform_values(map, function) → map

2018-04-09 Thread Xiao Li (JIRA)
Xiao Li created SPARK-23940: --- Summary: High-ofer function: transform_values(map, function) → map Key: SPARK-23940 URL: https://issues.apache.org/jira/browse/SPARK-23940 Project: Spark Issue Type:

[jira] [Updated] (SPARK-23938) High-order function: map_zip_with(map, map, function) → map

2018-04-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-23938: Description: Ref: https://prestodb.io/docs/current/functions/map.html Merges the two given maps into a si

[jira] [Resolved] (SPARK-23883) Error with conversion to arrow while using pandas_udf

2018-04-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23883. -- Resolution: Not A Problem Thanks. Let me leave this one resolved then. > Error with conversion

[jira] [Commented] (SPARK-23929) pandas_udf schema mapped by position and not by name

2018-04-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430169#comment-16430169 ] Hyukjin Kwon commented on SPARK-23929: -- I think documentation would be good enough f

[jira] [Assigned] (SPARK-23909) High-order function: filter(array, function) → array

2018-04-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-23909: - Assignee: Herman van Hovell > High-order function: filter(array, function) → arr

[jira] [Assigned] (SPARK-23908) High-order function: transform(array, function) → array

2018-04-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-23908: - Assignee: Herman van Hovell > High-order function: transform(array, function) →

[jira] [Assigned] (SPARK-23911) High-order function: reduce(array, initialState S, inputFunction, outputFunction) → R

2018-04-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell reassigned SPARK-23911: - Assignee: Herman van Hovell > High-order function: reduce(array, initialState S,

[jira] [Commented] (SPARK-23902) Provide an option in months_between UDF to disable rounding-off

2018-04-09 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430214#comment-16430214 ] Marco Gaido commented on SPARK-23902: - I will work on this, thanks. > Provide an opt

[jira] [Commented] (SPARK-23916) High-order function: array_join(x, delimiter, null_replacement) → varchar

2018-04-09 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430230#comment-16430230 ] Marco Gaido commented on SPARK-23916: - I will work on this, thanks. > High-order fun

[jira] [Commented] (SPARK-23907) Support regr_* functions

2018-04-09 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430218#comment-16430218 ] Marco Gaido commented on SPARK-23907: - I will work on this, thanks. > Support regr_*

[jira] [Commented] (SPARK-23918) High-order function: array_min(x) → x

2018-04-09 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430232#comment-16430232 ] Marco Gaido commented on SPARK-23918: - I will work on this, thanks. > High-order fun

[jira] [Commented] (SPARK-23917) High-order function: array_max(x) → x

2018-04-09 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430233#comment-16430233 ] Marco Gaido commented on SPARK-23917: - I will work on this, thanks. > High-order fun

[jira] [Updated] (SPARK-22256) Introduce spark.mesos.driver.memoryOverhead

2018-04-09 Thread paul mackles (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul mackles updated SPARK-22256: - Affects Version/s: 2.3.0 > Introduce spark.mesos.driver.memoryOverhead > ---

[jira] [Commented] (SPARK-22256) Introduce spark.mesos.driver.memoryOverhead

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430260#comment-16430260 ] Apache Spark commented on SPARK-22256: -- User 'pmackles' has created a pull request f

[jira] [Updated] (SPARK-22256) Introduce spark.mesos.driver.memoryOverhead

2018-04-09 Thread paul mackles (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul mackles updated SPARK-22256: - Fix Version/s: 2.4.0 2.3.1 > Introduce spark.mesos.driver.memoryOverhead > --

[jira] [Commented] (SPARK-22256) Introduce spark.mesos.driver.memoryOverhead

2018-04-09 Thread paul mackles (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430262#comment-16430262 ] paul mackles commented on SPARK-22256: -- created a PR on behalf of [~clehene] since h

[jira] [Assigned] (SPARK-22256) Introduce spark.mesos.driver.memoryOverhead

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22256: Assignee: Apache Spark > Introduce spark.mesos.driver.memoryOverhead > --

[jira] [Assigned] (SPARK-22256) Introduce spark.mesos.driver.memoryOverhead

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22256: Assignee: (was: Apache Spark) > Introduce spark.mesos.driver.memoryOverhead > ---

[jira] [Commented] (SPARK-23886) update query.status

2018-04-09 Thread Efim Poberezkin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430267#comment-16430267 ] Efim Poberezkin commented on SPARK-23886: - I'd like to work on this issue if nobo

[jira] [Created] (SPARK-23941) Mesos task failed on specific spark app name

2018-04-09 Thread bounkong khamphousone (JIRA)
bounkong khamphousone created SPARK-23941: - Summary: Mesos task failed on specific spark app name Key: SPARK-23941 URL: https://issues.apache.org/jira/browse/SPARK-23941 Project: Spark

[jira] [Commented] (SPARK-23897) Guava version

2018-04-09 Thread Jakub Dubovsky (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430297#comment-16430297 ] Jakub Dubovsky commented on SPARK-23897: [~hvanhovell] not necessarily related to

[jira] [Commented] (SPARK-23897) Guava version

2018-04-09 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430301#comment-16430301 ] Herman van Hovell commented on SPARK-23897: --- It is currently being discussed on

[jira] [Commented] (SPARK-23929) pandas_udf schema mapped by position and not by name

2018-04-09 Thread Omri (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430321#comment-16430321 ] Omri commented on SPARK-23929: -- [~hyukjin.kwon] I can do a PR, however I think this should o

[jira] [Created] (SPARK-23942) PySpark's collect doesn't trigger QueryExecutionListener

2018-04-09 Thread Hyukjin Kwon (JIRA)
Hyukjin Kwon created SPARK-23942: Summary: PySpark's collect doesn't trigger QueryExecutionListener Key: SPARK-23942 URL: https://issues.apache.org/jira/browse/SPARK-23942 Project: Spark Issu

[jira] [Assigned] (SPARK-23706) spark.conf.get(value, default=None) should produce None in PySpark

2018-04-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23706: Assignee: Hyukjin Kwon > spark.conf.get(value, default=None) should produce None in PySpar

[jira] [Updated] (SPARK-23942) PySpark's collect doesn't trigger QueryExecutionListener

2018-04-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23942: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-23483 > PySpark's collect doesn't

[jira] [Assigned] (SPARK-23942) PySpark's collect doesn't trigger QueryExecutionListener

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23942: Assignee: Apache Spark > PySpark's collect doesn't trigger QueryExecutionListener > --

[jira] [Commented] (SPARK-23942) PySpark's collect doesn't trigger QueryExecutionListener

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430335#comment-16430335 ] Apache Spark commented on SPARK-23942: -- User 'HyukjinKwon' has created a pull reques

[jira] [Assigned] (SPARK-23942) PySpark's collect doesn't trigger QueryExecutionListener

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23942: Assignee: (was: Apache Spark) > PySpark's collect doesn't trigger QueryExecutionListen

[jira] [Assigned] (SPARK-23902) Provide an option in months_between UDF to disable rounding-off

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23902: Assignee: Apache Spark > Provide an option in months_between UDF to disable rounding-off >

[jira] [Commented] (SPARK-23902) Provide an option in months_between UDF to disable rounding-off

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430404#comment-16430404 ] Apache Spark commented on SPARK-23902: -- User 'mgaido91' has created a pull request f

[jira] [Assigned] (SPARK-23902) Provide an option in months_between UDF to disable rounding-off

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23902: Assignee: (was: Apache Spark) > Provide an option in months_between UDF to disable rou

[jira] [Commented] (SPARK-22342) refactor schedulerDriver registration

2018-04-09 Thread Stavros Kontopoulos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430520#comment-16430520 ] Stavros Kontopoulos commented on SPARK-22342: - great news [~susanxhuynh] than

[jira] [Assigned] (SPARK-23905) Add UDF weekday

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23905: Assignee: Apache Spark > Add UDF weekday > --- > > Key: SPARK-

[jira] [Assigned] (SPARK-23905) Add UDF weekday

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23905: Assignee: (was: Apache Spark) > Add UDF weekday > --- > >

[jira] [Commented] (SPARK-23905) Add UDF weekday

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430524#comment-16430524 ] Apache Spark commented on SPARK-23905: -- User 'yucai' has created a pull request for

[jira] [Updated] (SPARK-23775) Flaky test: DataFrameRangeSuite

2018-04-09 Thread Gabor Somogyi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-23775: -- Attachment: filtered_more_logs.log > Flaky test: DataFrameRangeSuite >

[jira] [Commented] (SPARK-23926) High-order function: reverse(x) → array

2018-04-09 Thread Marek Novotny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430539#comment-16430539 ] Marek Novotny commented on SPARK-23926: --- I take this one. > High-order function: r

[jira] [Created] (SPARK-23943) add more specific health check to MesosRestServer

2018-04-09 Thread paul mackles (JIRA)
paul mackles created SPARK-23943: Summary: add more specific health check to MesosRestServer Key: SPARK-23943 URL: https://issues.apache.org/jira/browse/SPARK-23943 Project: Spark Issue Type:

[jira] [Updated] (SPARK-23943) Add more specific health check to MesosRestServer

2018-04-09 Thread paul mackles (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul mackles updated SPARK-23943: - Summary: Add more specific health check to MesosRestServer (was: add more specific health check

[jira] [Updated] (SPARK-23943) Add more specific health check to MesosRestServer

2018-04-09 Thread paul mackles (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul mackles updated SPARK-23943: - Environment:     was: Added a more robust health-check to MesosRestServer so that anyone who

[jira] [Updated] (SPARK-23943) Add more specific health check to MesosRestServer

2018-04-09 Thread paul mackles (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul mackles updated SPARK-23943: - Description: Add a more robust health-check to MesosRestServer so that anyone who runs MesosClus

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-09 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430616#comment-16430616 ] Imran Rashid commented on SPARK-23206: -- I have a question about this part of the des

[jira] [Assigned] (SPARK-23900) format_number udf should take user specifed format as argument

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23900: Assignee: (was: Apache Spark) > format_number udf should take user specifed format as

[jira] [Commented] (SPARK-23900) format_number udf should take user specifed format as argument

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430620#comment-16430620 ] Apache Spark commented on SPARK-23900: -- User 'wangyum' has created a pull request fo

[jira] [Assigned] (SPARK-23900) format_number udf should take user specifed format as argument

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23900: Assignee: Apache Spark > format_number udf should take user specifed format as argument >

[jira] [Updated] (SPARK-23943) Improve observability of MesosRestServer

2018-04-09 Thread paul mackles (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul mackles updated SPARK-23943: - Summary: Improve observability of MesosRestServer (was: Add more specific health check to MesosR

[jira] [Updated] (SPARK-23943) Improve observability of MesosRestServer/MesosClusterDispatcher

2018-04-09 Thread paul mackles (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul mackles updated SPARK-23943: - Summary: Improve observability of MesosRestServer/MesosClusterDispatcher (was: Improve observabi

[jira] [Commented] (SPARK-23906) Add UDF trunc(numeric)

2018-04-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430632#comment-16430632 ] Yuming Wang commented on SPARK-23906: - Duplicate with SPARK-20754's {{TRUNC}}? > Add

[jira] [Updated] (SPARK-23943) Improve observability of MesosRestServer/MesosClusterDispatcher

2018-04-09 Thread paul mackles (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] paul mackles updated SPARK-23943: - Description: Two changes: First, a more robust [health-check|[http://mesosphere.github.io/marat

[jira] [Commented] (SPARK-23906) Add UDF trunc(numeric)

2018-04-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430761#comment-16430761 ] Xiao Li commented on SPARK-23906: - [~q79969786] Could you link your PR to this JIRA? > A

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430763#comment-16430763 ] Xiao Li commented on SPARK-23206: - cc [~jiangxb1987] [~Gengliang.Wang]  > Additional Mem

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-09 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430765#comment-16430765 ] Edwina Lu commented on SPARK-23206: --- [~irashid], this would be quantile values for peak

[jira] [Commented] (SPARK-23925) High-order function: repeat(element, count) → array

2018-04-09 Thread Florent Pepin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430771#comment-16430771 ] Florent Pepin commented on SPARK-23925: --- Hi, I am new to Spark and I am interested

[jira] [Issue Comment Deleted] (SPARK-23906) Add UDF trunc(numeric)

2018-04-09 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-23906: Comment: was deleted (was: Duplicate with SPARK-20754's {{TRUNC}}?) > Add UDF trunc(numeric) > ---

[jira] [Assigned] (SPARK-23916) High-order function: array_join(x, delimiter, null_replacement) → varchar

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23916: Assignee: (was: Apache Spark) > High-order function: array_join(x, delimiter, null_rep

[jira] [Assigned] (SPARK-23916) High-order function: array_join(x, delimiter, null_replacement) → varchar

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23916: Assignee: Apache Spark > High-order function: array_join(x, delimiter, null_replacement) →

[jira] [Commented] (SPARK-23916) High-order function: array_join(x, delimiter, null_replacement) → varchar

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430808#comment-16430808 ] Apache Spark commented on SPARK-23916: -- User 'mgaido91' has created a pull request f

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-09 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430836#comment-16430836 ] Imran Rashid commented on SPARK-23206: -- ah of course, that makes sense -- quantiles

[jira] [Assigned] (SPARK-23906) Add UDF trunc(numeric)

2018-04-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-23906: --- Assignee: Yuming Wang > Add UDF trunc(numeric) > -- > > Key: SPA

[jira] [Commented] (SPARK-23206) Additional Memory Tuning Metrics

2018-04-09 Thread Edwina Lu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430922#comment-16430922 ] Edwina Lu commented on SPARK-23206: --- The doc wasn't very clear what the quantile values

[jira] [Resolved] (SPARK-23881) Flaky test: JobCancellationSuite."interruptible iterator of shuffle reader"

2018-04-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23881. - Resolution: Fixed Assignee: Jiang Xingbo Fix Version/s: 2.4.0 2.3.1 >

[jira] [Commented] (SPARK-23924) High-order function: element_at

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431002#comment-16431002 ] Kazuaki Ishizaki commented on SPARK-23924: -- I will work for this. > High-order

[jira] [Commented] (SPARK-23919) High-order function: array_position(x, element) → bigint

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431004#comment-16431004 ] Kazuaki Ishizaki commented on SPARK-23919: -- I am working for this. > High-order

[jira] [Commented] (SPARK-23923) High-order function: cardinality(x) → bigint

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431009#comment-16431009 ] Kazuaki Ishizaki commented on SPARK-23923: -- I am working for this. > High-order

[jira] [Comment Edited] (SPARK-23919) High-order function: array_position(x, element) → bigint

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431004#comment-16431004 ] Kazuaki Ishizaki edited comment on SPARK-23919 at 4/9/18 6:19 PM: -

[jira] [Commented] (SPARK-23929) pandas_udf schema mapped by position and not by name

2018-04-09 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431014#comment-16431014 ] Bryan Cutler commented on SPARK-23929: -- cc [~icexelloss] > pandas_udf schema mapped

[jira] [Commented] (SPARK-23921) High-order function: array_sort(x) → array

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431037#comment-16431037 ] Kazuaki Ishizaki commented on SPARK-23921: -- I am working for this > High-order

[jira] [Assigned] (SPARK-23816) FetchFailedException when killing speculative task

2018-04-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23816: -- Assignee: Imran Rashid > FetchFailedException when killing speculative task >

[jira] [Resolved] (SPARK-23816) FetchFailedException when killing speculative task

2018-04-09 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23816. Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 2.

[jira] [Comment Edited] (SPARK-23923) High-order function: cardinality(x) → bigint

2018-04-09 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431009#comment-16431009 ] Kazuaki Ishizaki edited comment on SPARK-23923 at 4/9/18 6:36 PM: -

[jira] [Resolved] (SPARK-22856) Add wrapper for codegen output and nullability

2018-04-09 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22856. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.4.0 > Add wrapper for codege

[jira] [Commented] (SPARK-21005) VectorIndexerModel does not prepare output column field correctly

2018-04-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431079#comment-16431079 ] Joseph K. Bradley commented on SPARK-21005: --- I don't actually see why this is a

[jira] [Updated] (SPARK-23890) Hive ALTER TABLE CHANGE COLUMN for struct type no longer works

2018-04-09 Thread Andrew Otto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Otto updated SPARK-23890: External issue URL: https://github.com/apache/spark/pull/21012 > Hive ALTER TABLE CHANGE COLUMN for

[jira] [Assigned] (SPARK-23890) Hive ALTER TABLE CHANGE COLUMN for struct type no longer works

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23890: Assignee: (was: Apache Spark) > Hive ALTER TABLE CHANGE COLUMN for struct type no long

[jira] [Assigned] (SPARK-23890) Hive ALTER TABLE CHANGE COLUMN for struct type no longer works

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23890: Assignee: Apache Spark > Hive ALTER TABLE CHANGE COLUMN for struct type no longer works >

[jira] [Commented] (SPARK-23890) Hive ALTER TABLE CHANGE COLUMN for struct type no longer works

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431082#comment-16431082 ] Apache Spark commented on SPARK-23890: -- User 'ottomata' has created a pull request f

[jira] [Assigned] (SPARK-14681) Provide label/impurity stats for spark.ml decision tree nodes

2018-04-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-14681: - Assignee: Weichen Xu > Provide label/impurity stats for spark.ml decision tree n

[jira] [Resolved] (SPARK-14681) Provide label/impurity stats for spark.ml decision tree nodes

2018-04-09 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-14681. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20786 [h

[jira] [Comment Edited] (SPARK-23890) Hive ALTER TABLE CHANGE COLUMN for struct type no longer works

2018-04-09 Thread Andrew Otto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431099#comment-16431099 ] Andrew Otto edited comment on SPARK-23890 at 4/9/18 7:31 PM: -

[jira] [Comment Edited] (SPARK-23890) Hive ALTER TABLE CHANGE COLUMN for struct type no longer works

2018-04-09 Thread Andrew Otto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431099#comment-16431099 ] Andrew Otto edited comment on SPARK-23890 at 4/9/18 7:31 PM: -

[jira] [Commented] (SPARK-16996) Hive ACID delta files not seen

2018-04-09 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-16996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431100#comment-16431100 ] Maciej Bryński commented on SPARK-16996: [~ste...@apache.org] Are you prepared to

[jira] [Comment Edited] (SPARK-23890) Hive ALTER TABLE CHANGE COLUMN for struct type no longer works

2018-04-09 Thread Andrew Otto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431099#comment-16431099 ] Andrew Otto edited comment on SPARK-23890 at 4/9/18 7:31 PM: -

[jira] [Commented] (SPARK-23890) Hive ALTER TABLE CHANGE COLUMN for struct type no longer works

2018-04-09 Thread Andrew Otto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431099#comment-16431099 ] Andrew Otto commented on SPARK-23890: - Hah! As a temporary workaround, we are [instan

[jira] [Comment Edited] (SPARK-23890) Hive ALTER TABLE CHANGE COLUMN for struct type no longer works

2018-04-09 Thread Andrew Otto (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431099#comment-16431099 ] Andrew Otto edited comment on SPARK-23890 at 4/9/18 7:32 PM: -

[jira] [Commented] (SPARK-23929) pandas_udf schema mapped by position and not by name

2018-04-09 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431188#comment-16431188 ] Li Jin commented on SPARK-23929: I think there are pros and cons for both matching by pos

[jira] [Created] (SPARK-23944) Add Param set functions to LSHModel types

2018-04-09 Thread Lu Wang (JIRA)
Lu Wang created SPARK-23944: --- Summary: Add Param set functions to LSHModel types Key: SPARK-23944 URL: https://issues.apache.org/jira/browse/SPARK-23944 Project: Spark Issue Type: Improvement

[jira] [Comment Edited] (SPARK-23929) pandas_udf schema mapped by position and not by name

2018-04-09 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431188#comment-16431188 ] Li Jin edited comment on SPARK-23929 at 4/9/18 8:39 PM: I think t

[jira] [Comment Edited] (SPARK-23929) pandas_udf schema mapped by position and not by name

2018-04-09 Thread Li Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431188#comment-16431188 ] Li Jin edited comment on SPARK-23929 at 4/9/18 8:39 PM: I think t

[jira] [Commented] (SPARK-22919) Bump Apache httpclient versions

2018-04-09 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431198#comment-16431198 ] Steve Loughran commented on SPARK-22919: going to highlight this appears to break

[jira] [Commented] (SPARK-23874) Upgrade apache/arrow to 0.9.0

2018-04-09 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431287#comment-16431287 ] Apache Spark commented on SPARK-23874: -- User 'BryanCutler' has created a pull reques

  1   2   >