[jira] [Resolved] (SPARK-13259) SPARK_HOME should not be used as the CWD in docker executors

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13259. -- Resolution: Incomplete > SPARK_HOME should not be used as the CWD in docker executors > --

[jira] [Resolved] (SPARK-8587) Return cost and cluster index KMeansModel.predict

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8587. - Resolution: Incomplete > Return cost and cluster index KMeansModel.predict >

[jira] [Resolved] (SPARK-10776) Pass location of SparkR source files from R process to JVM

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10776. -- Resolution: Incomplete > Pass location of SparkR source files from R process to JVM >

[jira] [Resolved] (SPARK-14035) Flaky spark.mllib.classification.NaiveBayesSuite test

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14035. -- Resolution: Incomplete > Flaky spark.mllib.classification.NaiveBayesSuite test > -

[jira] [Resolved] (SPARK-17155) usage of a Dataset inside a Future throws MissingRequirementError

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17155. -- Resolution: Incomplete > usage of a Dataset inside a Future throws MissingRequirementError > -

[jira] [Resolved] (SPARK-14723) A new way to support dynamic allocation in Spark Streaming

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14723. -- Resolution: Incomplete > A new way to support dynamic allocation in Spark Streaming >

[jira] [Resolved] (SPARK-8789) improve SQLQuerySuite resilience by dropping tables in setup

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8789. - Resolution: Incomplete > improve SQLQuerySuite resilience by dropping tables in setup > -

[jira] [Resolved] (SPARK-10444) Remove duplication in Mesos schedulers

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10444. -- Resolution: Incomplete > Remove duplication in Mesos schedulers >

[jira] [Resolved] (SPARK-14947) Showtable Command - Can't List Tables Using JDBC Connector

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14947. -- Resolution: Incomplete > Showtable Command - Can't List Tables Using JDBC Connector >

[jira] [Resolved] (SPARK-16767) existsRecursively method in UserDefinedType is not correct

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16767. -- Resolution: Incomplete > existsRecursively method in UserDefinedType is not correct >

[jira] [Resolved] (SPARK-9695) Add random seed Param to ML Pipeline

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9695. - Resolution: Incomplete > Add random seed Param to ML Pipeline > -

[jira] [Resolved] (SPARK-16244) Failed job/stage couldn't stop JobGenerator immediately.

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16244. -- Resolution: Incomplete > Failed job/stage couldn't stop JobGenerator immediately. > --

[jira] [Resolved] (SPARK-14980) Spark is not picking up classes in the MutableURLClassLoader causing errors with drools

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14980. -- Resolution: Incomplete > Spark is not picking up classes in the MutableURLClassLoader causing

[jira] [Resolved] (SPARK-8983) ML Tuning Cross-Validation Improvements

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8983. - Resolution: Incomplete > ML Tuning Cross-Validation Improvements > --

[jira] [Resolved] (SPARK-17600) Cannot set public address for Worker and Master Web UI

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17600. -- Resolution: Incomplete > Cannot set public address for Worker and Master Web UI >

[jira] [Resolved] (SPARK-12178) Expose reporting of StreamInputInfo for custom made streams

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12178. -- Resolution: Incomplete > Expose reporting of StreamInputInfo for custom made streams > ---

[jira] [Resolved] (SPARK-12622) spark-submit fails on executors when jar has a space in it

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12622. -- Resolution: Incomplete > spark-submit fails on executors when jar has a space in it >

[jira] [Resolved] (SPARK-9599) Dynamic partitioning based on key-distribution

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9599. - Resolution: Incomplete > Dynamic partitioning based on key-distribution > ---

[jira] [Resolved] (SPARK-13683) Finalize the public interface for OutputWriter[Factory]

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13683. -- Resolution: Incomplete > Finalize the public interface for OutputWriter[Factory] > ---

[jira] [Resolved] (SPARK-9442) java.lang.ArithmeticException: / by zero when reading Parquet

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9442. - Resolution: Incomplete > java.lang.ArithmeticException: / by zero when reading Parquet >

[jira] [Resolved] (SPARK-17156) Add multiclass logistic regression Scala Example

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17156. -- Resolution: Incomplete > Add multiclass logistic regression Scala Example > --

[jira] [Resolved] (SPARK-14926) OneVsRest labelMetadata uses incorrect name

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14926. -- Resolution: Incomplete > OneVsRest labelMetadata uses incorrect name > ---

[jira] [Resolved] (SPARK-7597) Make default doc build avoid search engine indexing

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7597. - Resolution: Incomplete > Make default doc build avoid search engine indexing > --

[jira] [Resolved] (SPARK-9237) Added Top N Column Values for DataFrames

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9237. - Resolution: Incomplete > Added Top N Column Values for DataFrames > -

[jira] [Resolved] (SPARK-12099) Standalone and Mesos Should use OnOutOfMemoryError handlers

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12099. -- Resolution: Incomplete > Standalone and Mesos Should use OnOutOfMemoryError handlers > ---

[jira] [Resolved] (SPARK-10659) DataFrames and SparkSQL saveAsParquetFile does not preserve REQUIRED (not nullable) flag in schema

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10659. -- Resolution: Incomplete > DataFrames and SparkSQL saveAsParquetFile does not preserve REQUIRED

[jira] [Resolved] (SPARK-12319) ExchangeCoordinatorSuite fails on big-endian platforms

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12319. -- Resolution: Incomplete > ExchangeCoordinatorSuite fails on big-endian platforms >

[jira] [Resolved] (SPARK-15389) DataFrame filter by isNotNull fails in complex, large case

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15389. -- Resolution: Incomplete > DataFrame filter by isNotNull fails in complex, large case >

[jira] [Resolved] (SPARK-7653) ML Pipeline and meta-algs should take random seed param

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7653. - Resolution: Incomplete > ML Pipeline and meta-algs should take random seed param > --

[jira] [Resolved] (SPARK-14054) Support parameters for UDTs

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14054. -- Resolution: Incomplete > Support parameters for UDTs > --- > >

[jira] [Resolved] (SPARK-14256) Remove parameter sqlContext from as.DataFrame

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14256. -- Resolution: Incomplete > Remove parameter sqlContext from as.DataFrame > -

[jira] [Resolved] (SPARK-17357) Simplified predicates can't be pushed down through operators because of the rule order in Optimizer

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17357. -- Resolution: Incomplete > Simplified predicates can't be pushed down through operators because

[jira] [Resolved] (SPARK-11823) HiveThriftBinaryServerSuite tests timing out, leaves hanging processes

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11823. -- Resolution: Incomplete > HiveThriftBinaryServerSuite tests timing out, leaves hanging processe

[jira] [Resolved] (SPARK-10551) Successful task-end event after task failed due to executor loss

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10551. -- Resolution: Incomplete > Successful task-end event after task failed due to executor loss > --

[jira] [Resolved] (SPARK-14890) DAGScheduler should not accept the result of a previous task attempt, since its stage attempt has been completed.

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14890. -- Resolution: Incomplete > DAGScheduler should not accept the result of a previous task attempt,

[jira] [Resolved] (SPARK-7379) pickle.loads expects a string instead of bytes in Python 3.

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7379. - Resolution: Incomplete > pickle.loads expects a string instead of bytes in Python 3. > --

[jira] [Resolved] (SPARK-12208) Abstract the examples into a common place

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12208. -- Resolution: Incomplete > Abstract the examples into a common place > -

[jira] [Resolved] (SPARK-12506) Push down WHERE clause arithmetic operator to JDBC layer

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12506. -- Resolution: Incomplete > Push down WHERE clause arithmetic operator to JDBC layer > --

[jira] [Resolved] (SPARK-10250) Scala PairRDDFunctions.groupByKey() should be fault-tolerant of single large groups

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10250. -- Resolution: Incomplete > Scala PairRDDFunctions.groupByKey() should be fault-tolerant of singl

[jira] [Resolved] (SPARK-12013) Add a Hive context method to retrieve the database Names

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-12013. -- Resolution: Incomplete > Add a Hive context method to retrieve the database Names > --

[jira] [Resolved] (SPARK-16108) Why is KMeansModel (scala) private?

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16108. -- Resolution: Incomplete > Why is KMeansModel (scala) private? > ---

[jira] [Resolved] (SPARK-15903) Support AllColumn expression in UDF functions

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15903. -- Resolution: Incomplete > Support AllColumn expression in UDF functions > -

[jira] [Resolved] (SPARK-11517) Calc partitions in parallel for multiple partitions table

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11517. -- Resolution: Incomplete > Calc partitions in parallel for multiple partitions table > -

[jira] [Resolved] (SPARK-14026) Subquery not brodcasted

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14026. -- Resolution: Incomplete > Subquery not brodcasted > --- > >

[jira] [Resolved] (SPARK-13653) Split DiskBlockObjectWriter to separate object- and byte-based write interfaces

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13653. -- Resolution: Incomplete > Split DiskBlockObjectWriter to separate object- and byte-based write

[jira] [Resolved] (SPARK-15143) CSV data source is not being tested as HadoopFsRelation

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15143. -- Resolution: Incomplete > CSV data source is not being tested as HadoopFsRelation > ---

[jira] [Resolved] (SPARK-16969) GBTClassifier needs a raw prediction column

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16969. -- Resolution: Incomplete > GBTClassifier needs a raw prediction column > ---

[jira] [Resolved] (SPARK-10150) --force=true option is not working in beeline

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10150. -- Resolution: Incomplete > --force=true option is not working in beeline >

[jira] [Resolved] (SPARK-16504) UDAF should be typed

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16504. -- Resolution: Incomplete > UDAF should be typed > > > Key:

[jira] [Resolved] (SPARK-14193) Skip unnecessary sorts if input data have been already ordered in InMemoryRelation

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14193. -- Resolution: Incomplete > Skip unnecessary sorts if input data have been already ordered in >

[jira] [Resolved] (SPARK-17397) Show example of what to do when awaitTermination() throws an Exception

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17397. -- Resolution: Incomplete > Show example of what to do when awaitTermination() throws an Exceptio

[jira] [Resolved] (SPARK-15418) SparkSQL does not support using a UDAF in a CREATE VIEW clause

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15418. -- Resolution: Incomplete > SparkSQL does not support using a UDAF in a CREATE VIEW clause >

[jira] [Resolved] (SPARK-10972) UDFs in SQL joins

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10972. -- Resolution: Incomplete > UDFs in SQL joins > - > > Key: SPARK-

[jira] [Resolved] (SPARK-16737) ListingFileCatalog comments about RPC calls in object store isn't correct

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16737. -- Resolution: Incomplete > ListingFileCatalog comments about RPC calls in object store isn't cor

[jira] [Resolved] (SPARK-16532) Provide a REST API for submitting and tracking status of jobs

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16532. -- Resolution: Incomplete > Provide a REST API for submitting and tracking status of jobs > -

[jira] [Resolved] (SPARK-18421) Dynamic disk allocation

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18421. -- Resolution: Incomplete > Dynamic disk allocation > --- > >

[jira] [Resolved] (SPARK-8514) LU factorization on BlockMatrix

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8514. - Resolution: Incomplete > LU factorization on BlockMatrix > --- > >

[jira] [Resolved] (SPARK-14534) Should SparkContext.parallelize(List) take an Iterable instead?

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14534. -- Resolution: Incomplete > Should SparkContext.parallelize(List) take an Iterable instead? > ---

[jira] [Resolved] (SPARK-17482) Analyzer should be able run on top of optimized rule

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17482. -- Resolution: Incomplete > Analyzer should be able run on top of optimized rule > --

[jira] [Resolved] (SPARK-9588) spark sql cache: partition level cache eviction

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9588. - Resolution: Incomplete > spark sql cache: partition level cache eviction > --

[jira] [Resolved] (SPARK-13222) checkpoint latest stateful RDD on graceful shutdown

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13222. -- Resolution: Incomplete > checkpoint latest stateful RDD on graceful shutdown > ---

[jira] [Resolved] (SPARK-13707) Streaming UI tab misleading for window operations

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13707. -- Resolution: Incomplete > Streaming UI tab misleading for window operations > -

[jira] [Resolved] (SPARK-10420) Implementing Reactive Streams based Spark Streaming Receiver

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10420. -- Resolution: Incomplete > Implementing Reactive Streams based Spark Streaming Receiver > --

[jira] [Resolved] (SPARK-7338) Survival Modelling - Cox proportional hazards

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7338. - Resolution: Incomplete > Survival Modelling - Cox proportional hazards >

[jira] [Resolved] (SPARK-14186) Size awareness during spill and shuffle

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14186. -- Resolution: Incomplete > Size awareness during spill and shuffle > ---

[jira] [Resolved] (SPARK-9739) Execution visualizer

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-9739. - Resolution: Incomplete > Execution visualizer > > > Key: SPA

[jira] [Resolved] (SPARK-13762) support only column names in schema string at createDataFrame

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13762. -- Resolution: Incomplete > support only column names in schema string at createDataFrame > -

[jira] [Resolved] (SPARK-16743) converter and access code out of sync: createDataFrame on RDD[Option[C]] fails with MatchError

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16743. -- Resolution: Incomplete > converter and access code out of sync: createDataFrame on RDD[Option[

[jira] [Resolved] (SPARK-17775) pyspark: take(num) failed, but collect() worked for big dataset

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17775. -- Resolution: Incomplete > pyspark: take(num) failed, but collect() worked for big dataset > ---

[jira] [Resolved] (SPARK-17754) DataFrame reader and writer don't show Input/Output metrics in Spark UI

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17754. -- Resolution: Incomplete > DataFrame reader and writer don't show Input/Output metrics in Spark

[jira] [Resolved] (SPARK-14330) Spark SQL does not infer correct type for joda DateTime

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14330. -- Resolution: Incomplete > Spark SQL does not infer correct type for joda DateTime > ---

[jira] [Resolved] (SPARK-13795) ClassCast Exception while attempting to show() a DataFrame

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13795. -- Resolution: Incomplete > ClassCast Exception while attempting to show() a DataFrame >

[jira] [Resolved] (SPARK-18100) Improve the performance of get_json_object using Gson

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18100. -- Resolution: Incomplete > Improve the performance of get_json_object using Gson > -

[jira] [Resolved] (SPARK-11288) Specify the return type for UDF in Scala

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11288. -- Resolution: Incomplete > Specify the return type for UDF in Scala > --

[jira] [Resolved] (SPARK-10702) Dynamic Allocation in Standalone Breaking Parallelism

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10702. -- Resolution: Incomplete > Dynamic Allocation in Standalone Breaking Parallelism > -

[jira] [Resolved] (SPARK-11632) Filter out empty partition for KafkaRDD

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11632. -- Resolution: Incomplete > Filter out empty partition for KafkaRDD > ---

[jira] [Resolved] (SPARK-10105) Adding most k frequent words parameter to Word2Vec implementation

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-10105. -- Resolution: Incomplete > Adding most k frequent words parameter to Word2Vec implementation > -

[jira] [Resolved] (SPARK-15015) Log statements lack file name/number

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15015. -- Resolution: Incomplete > Log statements lack file name/number > --

[jira] [Resolved] (SPARK-16999) Convert Keyword `path` of Data Source Tables to Lower Case

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16999. -- Resolution: Incomplete > Convert Keyword `path` of Data Source Tables to Lower Case >

[jira] [Resolved] (SPARK-11524) Support SparkR with Mesos cluster

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11524. -- Resolution: Incomplete > Support SparkR with Mesos cluster > -

[jira] [Resolved] (SPARK-15258) Nested/Chained case statements generate codegen over 64k exception

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15258. -- Resolution: Incomplete > Nested/Chained case statements generate codegen over 64k exception >

[jira] [Resolved] (SPARK-7852) Set the initial weights based on the previous when GLMs are run with multiple regParams

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-7852. - Resolution: Incomplete > Set the initial weights based on the previous when GLMs are run with mul

[jira] [Resolved] (SPARK-8523) Spark streaming static files not found when running web ui under virtual host

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8523. - Resolution: Incomplete > Spark streaming static files not found when running web ui under virtual

[jira] [Resolved] (SPARK-16537) Extracting partition columns from the hive table.

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16537. -- Resolution: Incomplete > Extracting partition columns from the hive table. > -

[jira] [Resolved] (SPARK-15130) PySpark shared params should include default values to match Scala

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15130. -- Resolution: Incomplete > PySpark shared params should include default values to match Scala >

[jira] [Resolved] (SPARK-11038) Consolidate the format of UnsafeArrayData and UnsafeMapData

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-11038. -- Resolution: Incomplete > Consolidate the format of UnsafeArrayData and UnsafeMapData > ---

[jira] [Resolved] (SPARK-8946) Intermittent SnappyCompressionCodec. IllegalArgumentException

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8946. - Resolution: Incomplete > Intermittent SnappyCompressionCodec. IllegalArgumentException >

[jira] [Resolved] (SPARK-14583) SparkSQL doesn't apply TBLPROPERTIES('serialization.null.format'='') when Hive Table has partitions

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14583. -- Resolution: Incomplete > SparkSQL doesn't apply TBLPROPERTIES('serialization.null.format'='')

[jira] [Resolved] (SPARK-13155) add runtime null check when convert catalyst array to external array

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13155. -- Resolution: Incomplete > add runtime null check when convert catalyst array to external array

[jira] [Resolved] (SPARK-15993) PySpark RuntimeConfig should be immutable

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15993. -- Resolution: Incomplete > PySpark RuntimeConfig should be immutable > -

[jira] [Resolved] (SPARK-14425) SQL/dataframe join error: mixes up the columns

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14425. -- Resolution: Incomplete > SQL/dataframe join error: mixes up the columns > ---

[jira] [Resolved] (SPARK-14708) Repl Serialization Issue

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14708. -- Resolution: Incomplete > Repl Serialization Issue > > >

[jira] [Resolved] (SPARK-13216) Spark streaming application not honoring --num-executors in restarting of an application from a checkpoint

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-13216. -- Resolution: Incomplete > Spark streaming application not honoring --num-executors in restartin

[jira] [Resolved] (SPARK-8082) Functionality to Reset DF Schemas/Cast Multiple Columns

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-8082. - Resolution: Incomplete > Functionality to Reset DF Schemas/Cast Multiple Columns > --

[jira] [Resolved] (SPARK-17603) Utilize Hive-generated Statistics For Partitioned Tables

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-17603. -- Resolution: Incomplete > Utilize Hive-generated Statistics For Partitioned Tables > --

[jira] [Resolved] (SPARK-14405) broadcast variables can cause lots of warning messages on shutdown

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14405. -- Resolution: Incomplete > broadcast variables can cause lots of warning messages on shutdown >

[jira] [Resolved] (SPARK-15482) ClassCast exception when join two tables.

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-15482. -- Resolution: Incomplete > ClassCast exception when join two tables. > -

[jira] [Resolved] (SPARK-16728) migrate internal API for MLlib trees from spark.mllib to spark.ml

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-16728. -- Resolution: Incomplete > migrate internal API for MLlib trees from spark.mllib to spark.ml > -

[jira] [Resolved] (SPARK-18340) Inconsistent error messages in launching scripts and hanging in sparkr script for wrong options

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-18340. -- Resolution: Incomplete > Inconsistent error messages in launching scripts and hanging in spark

[jira] [Resolved] (SPARK-14003) Multi-session can not work when one session is moving files for "INSERT ... SELECT" clause

2019-05-20 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-14003. -- Resolution: Incomplete > Multi-session can not work when one session is moving files for "INSE

<    3   4   5   6   7   8   9   10   11   12   >