[jira] [Assigned] (SPARK-18522) Create explicit contract for column stats serialization

[ https://issues.apache.org/jira/browse/SPARK-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18522: Assignee: Apache Spark (was: Reynold Xin) > Create explicit contract for column stats

[jira] [Assigned] (SPARK-18522) Create explicit contract for column stats serialization

[ https://issues.apache.org/jira/browse/SPARK-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18522: Assignee: Reynold Xin (was: Apache Spark) > Create explicit contract for column stats

[jira] [Commented] (SPARK-18522) Create explicit contract for column stats serialization

[ https://issues.apache.org/jira/browse/SPARK-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682815#comment-15682815 ] Apache Spark commented on SPARK-18522: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-17932) Failed to run SQL "show table extended like table_name" in Spark2.0.0

[ https://issues.apache.org/jira/browse/SPARK-17932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17932: Assignee: Apache Spark > Failed to run SQL "show table extended like table_name" in

[jira] [Assigned] (SPARK-17932) Failed to run SQL "show table extended like table_name" in Spark2.0.0

[ https://issues.apache.org/jira/browse/SPARK-17932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17932: Assignee: (was: Apache Spark) > Failed to run SQL "show table extended like

[jira] [Commented] (SPARK-17932) Failed to run SQL "show table extended like table_name" in Spark2.0.0

[ https://issues.apache.org/jira/browse/SPARK-17932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682810#comment-15682810 ] Apache Spark commented on SPARK-17932: -- User 'jiangxb1987' has created a pull request for this

[jira] [Created] (SPARK-18522) Create explicit contract for column stats serialization

Reynold Xin created SPARK-18522: --- Summary: Create explicit contract for column stats serialization Key: SPARK-18522 URL: https://issues.apache.org/jira/browse/SPARK-18522 Project: Spark Issue

[jira] [Commented] (SPARK-17755) Master may ask a worker to launch an executor before the worker actually got the response of registration

[ https://issues.apache.org/jira/browse/SPARK-17755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682778#comment-15682778 ] Shixiong Zhu commented on SPARK-17755: -- It's not easy to fix. The root cause is messages are sent

[jira] [Created] (SPARK-18521) Add `NoRedundantStringInterpolator` Scala rule

Weiqing Yang created SPARK-18521: Summary: Add `NoRedundantStringInterpolator` Scala rule Key: SPARK-18521 URL: https://issues.apache.org/jira/browse/SPARK-18521 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-18515) AlterTableDropPartitions fails for non-string columns

[ https://issues.apache.org/jira/browse/SPARK-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682732#comment-15682732 ] Dongjoon Hyun edited comment on SPARK-18515 at 11/21/16 7:10 AM: - ~While

[jira] [Comment Edited] (SPARK-18515) AlterTableDropPartitions fails for non-string columns

[ https://issues.apache.org/jira/browse/SPARK-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682732#comment-15682732 ] Dongjoon Hyun edited comment on SPARK-18515 at 11/21/16 7:10 AM: - ~~While

[jira] [Comment Edited] (SPARK-18515) AlterTableDropPartitions fails for non-string columns

[ https://issues.apache.org/jira/browse/SPARK-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682732#comment-15682732 ] Dongjoon Hyun edited comment on SPARK-18515 at 11/21/16 7:10 AM: - -While

[jira] [Updated] (SPARK-18450) Add AND-amplification to Locality Sensitive Hashing

[ https://issues.apache.org/jira/browse/SPARK-18450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-18450: --- Component/s: ML > Add AND-amplification to Locality Sensitive Hashing >

[jira] [Updated] (SPARK-18454) Changes to fix Nearest Neighbor Search for LSH

[ https://issues.apache.org/jira/browse/SPARK-18454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-18454: --- Component/s: ML > Changes to fix Nearest Neighbor Search for LSH >

[jira] [Updated] (SPARK-18408) API Improvements for LSH

[ https://issues.apache.org/jira/browse/SPARK-18408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath updated SPARK-18408: --- Component/s: ML > API Improvements for LSH > > >

[jira] [Commented] (SPARK-18515) AlterTableDropPartitions fails for non-string columns

[ https://issues.apache.org/jira/browse/SPARK-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682732#comment-15682732 ] Dongjoon Hyun commented on SPARK-18515: --- While digging this, option 1 seems to increase complexity.

[jira] [Commented] (SPARK-18134) SQL: MapType in Group BY and Joins not working

[ https://issues.apache.org/jira/browse/SPARK-18134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682730#comment-15682730 ] Takeshi Yamamuro commented on SPARK-18134: -- In v1.5, the comparison and ordering of map-types

[jira] [Updated] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-12978: Affects Version/s: (was: 1.6.0) Target Version/s: 2.2.0 > Skip unnecessary final group-by

[jira] [Assigned] (SPARK-18520) Add missing setXXXCol methods for BisectingKMeansModel and GaussianMixtureModel

[ https://issues.apache.org/jira/browse/SPARK-18520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18520: Assignee: (was: Apache Spark) > Add missing setXXXCol methods for

[jira] [Assigned] (SPARK-18520) Add missing setXXXCol methods for BisectingKMeansModel and GaussianMixtureModel

[ https://issues.apache.org/jira/browse/SPARK-18520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18520: Assignee: Apache Spark > Add missing setXXXCol methods for BisectingKMeansModel and >

[jira] [Commented] (SPARK-18520) Add missing setXXXCol methods for BisectingKMeansModel and GaussianMixtureModel

[ https://issues.apache.org/jira/browse/SPARK-18520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682692#comment-15682692 ] Apache Spark commented on SPARK-18520: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-18520) Add missing setXXXCol methods for BisectingKMeansModel and GaussianMixtureModel

zhengruifeng created SPARK-18520: Summary: Add missing setXXXCol methods for BisectingKMeansModel and GaussianMixtureModel Key: SPARK-18520 URL: https://issues.apache.org/jira/browse/SPARK-18520

[jira] [Assigned] (SPARK-18519) map type can not be used in binary comparison

[ https://issues.apache.org/jira/browse/SPARK-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18519: Assignee: Wenchen Fan (was: Apache Spark) > map type can not be used in binary

[jira] [Assigned] (SPARK-18519) map type can not be used in binary comparison

[ https://issues.apache.org/jira/browse/SPARK-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18519: Assignee: Apache Spark (was: Wenchen Fan) > map type can not be used in binary

[jira] [Commented] (SPARK-18519) map type can not be used in binary comparison

[ https://issues.apache.org/jira/browse/SPARK-18519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682677#comment-15682677 ] Apache Spark commented on SPARK-18519: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Created] (SPARK-18519) map type can not be used in binary comparison

Wenchen Fan created SPARK-18519: --- Summary: map type can not be used in binary comparison Key: SPARK-18519 URL: https://issues.apache.org/jira/browse/SPARK-18519 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-18456) Use matrix abstraction for LogisitRegression coefficients during training

[ https://issues.apache.org/jira/browse/SPARK-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-18456. Resolution: Fixed Assignee: Seth Hendrickson Fix Version/s: 2.1.0 > Use

[jira] [Created] (SPARK-18518) HasSolver should support allowed values

zhengruifeng created SPARK-18518: Summary: HasSolver should support allowed values Key: SPARK-18518 URL: https://issues.apache.org/jira/browse/SPARK-18518 Project: Spark Issue Type:

[jira] [Commented] (SPARK-18434) Add missing ParamValidations for ML algos

[ https://issues.apache.org/jira/browse/SPARK-18434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682601#comment-15682601 ] Apache Spark commented on SPARK-18434: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-17591) Fix/investigate the failure of tests in Scala On Windows

[ https://issues.apache.org/jira/browse/SPARK-17591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682578#comment-15682578 ] Hyukjin Kwon commented on SPARK-17591: -- I will close this when I am able to proceed further the

[jira] [Assigned] (SPARK-18516) Separate instantaneous state from progress performance statistics

[ https://issues.apache.org/jira/browse/SPARK-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18516: Assignee: Michael Armbrust (was: Apache Spark) > Separate instantaneous state from

[jira] [Assigned] (SPARK-18516) Separate instantaneous state from progress performance statistics

[ https://issues.apache.org/jira/browse/SPARK-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18516: Assignee: Apache Spark (was: Michael Armbrust) > Separate instantaneous state from

[jira] [Commented] (SPARK-18516) Separate instantaneous state from progress performance statistics

[ https://issues.apache.org/jira/browse/SPARK-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682564#comment-15682564 ] Apache Spark commented on SPARK-18516: -- User 'marmbrus' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18517) DROP TABLE IF EXISTS should not warn for non-existing tables

[ https://issues.apache.org/jira/browse/SPARK-18517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18517: Assignee: Apache Spark > DROP TABLE IF EXISTS should not warn for non-existing tables >

[jira] [Commented] (SPARK-18517) DROP TABLE IF EXISTS should not warn for non-existing tables

[ https://issues.apache.org/jira/browse/SPARK-18517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682523#comment-15682523 ] Apache Spark commented on SPARK-18517: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-18517) DROP TABLE IF EXISTS should not warn for non-existing tables

[ https://issues.apache.org/jira/browse/SPARK-18517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18517: Assignee: (was: Apache Spark) > DROP TABLE IF EXISTS should not warn for non-existing

[jira] [Created] (SPARK-18517) DROP TABLE IF EXISTS should not warn for non-existing tables

Dongjoon Hyun created SPARK-18517: - Summary: DROP TABLE IF EXISTS should not warn for non-existing tables Key: SPARK-18517 URL: https://issues.apache.org/jira/browse/SPARK-18517 Project: Spark

[jira] [Commented] (SPARK-18023) Adam optimizer

[ https://issues.apache.org/jira/browse/SPARK-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682491#comment-15682491 ] Vincent commented on SPARK-18023: - thanks [~mlnick] that's really what we need. when I wrote the code for

[jira] [Commented] (SPARK-18023) Adam optimizer

[ https://issues.apache.org/jira/browse/SPARK-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682471#comment-15682471 ] Nick Pentreath commented on SPARK-18023: Linking SPARK-17136 which is really a blocker for adding

[jira] [Commented] (SPARK-16377) Spark MLlib: MultilayerPerceptronClassifier - error while training

[ https://issues.apache.org/jira/browse/SPARK-16377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682458#comment-15682458 ] Nick Pentreath commented on SPARK-16377: Is this still a bug? As per your above comment seems we

[jira] [Commented] (SPARK-6346) Use faster converging optimization method in MLlib

[ https://issues.apache.org/jira/browse/SPARK-6346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682455#comment-15682455 ] Nick Pentreath commented on SPARK-6346: --- I think we can close this ticket? It's pretty old, and

[jira] [Reopened] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-12978: - > Skip unnecessary final group-by when input data already clustered with > group-by keys >

[jira] [Resolved] (SPARK-18467) Refactor StaticInvoke, Invoke and NewInstance.

[ https://issues.apache.org/jira/browse/SPARK-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18467. - Resolution: Fixed Fix Version/s: 2.1.0 > Refactor StaticInvoke, Invoke and NewInstance. >

[jira] [Updated] (SPARK-18467) Refactor StaticInvoke, Invoke and NewInstance.

[ https://issues.apache.org/jira/browse/SPARK-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18467: Assignee: Takuya Ueshin > Refactor StaticInvoke, Invoke and NewInstance. >

[jira] [Commented] (SPARK-2941) Add config option to support NIO vs OIO in Netty network module

[ https://issues.apache.org/jira/browse/SPARK-2941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682355#comment-15682355 ] Qiao Yifan commented on SPARK-2941: --- where is the config file locates? I can't find it > Add config

[jira] [Updated] (SPARK-18514) Fix `Note:`/`NOTE:`/`Note that` across R API documentation

[ https://issues.apache.org/jira/browse/SPARK-18514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-18514: - Priority: Trivial (was: Minor) > Fix `Note:`/`NOTE:`/`Note that` across R API documentation >

[jira] [Assigned] (SPARK-18514) Fix `Note:`/`NOTE:`/`Note that` across R API documentation

[ https://issues.apache.org/jira/browse/SPARK-18514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18514: Assignee: Apache Spark > Fix `Note:`/`NOTE:`/`Note that` across R API documentation >

[jira] [Assigned] (SPARK-18514) Fix `Note:`/`NOTE:`/`Note that` across R API documentation

[ https://issues.apache.org/jira/browse/SPARK-18514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18514: Assignee: (was: Apache Spark) > Fix `Note:`/`NOTE:`/`Note that` across R API

[jira] [Commented] (SPARK-18514) Fix `Note:`/`NOTE:`/`Note that` across R API documentation

[ https://issues.apache.org/jira/browse/SPARK-18514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682286#comment-15682286 ] Apache Spark commented on SPARK-18514: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-18516) Separate instantaneous state from progress performance statistics

[ https://issues.apache.org/jira/browse/SPARK-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18516: - Description: There are two types of information that you want to be able to extract from

[jira] [Updated] (SPARK-18516) Separate instantaneous state from progress performance statistics

[ https://issues.apache.org/jira/browse/SPARK-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18516: - Description: There are two types of information that you want to be able to extract from

[jira] [Commented] (SPARK-18506) kafka 0.10 with Spark 2.02 auto.offset.reset=earliest will only read from a single partition on a multi partition topic

[ https://issues.apache.org/jira/browse/SPARK-18506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682216#comment-15682216 ] Cody Koeninger commented on SPARK-18506: I tried your example code on an AWS 2-node spark

[jira] [Created] (SPARK-18516) Separate instantaneous state from progress performance statistics

Michael Armbrust created SPARK-18516: Summary: Separate instantaneous state from progress performance statistics Key: SPARK-18516 URL: https://issues.apache.org/jira/browse/SPARK-18516 Project:

[jira] [Commented] (SPARK-18475) Be able to provide higher parallelization for StructuredStreaming Kafka Source

[ https://issues.apache.org/jira/browse/SPARK-18475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15682171#comment-15682171 ] Cody Koeninger commented on SPARK-18475: An iterator certainly does have an ordering guarantee,

[jira] [Assigned] (SPARK-18510) Partition schema inference corrupts data

[ https://issues.apache.org/jira/browse/SPARK-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18510: Assignee: (was: Apache Spark) > Partition schema inference corrupts data >

[jira] [Assigned] (SPARK-18510) Partition schema inference corrupts data

[ https://issues.apache.org/jira/browse/SPARK-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18510: Assignee: Apache Spark > Partition schema inference corrupts data >

[jira] [Commented] (SPARK-18510) Partition schema inference corrupts data

[ https://issues.apache.org/jira/browse/SPARK-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681953#comment-15681953 ] Apache Spark commented on SPARK-18510: -- User 'brkyvz' has created a pull request for this issue:

[jira] [Commented] (SPARK-17062) Add --conf to mesos dispatcher process

[ https://issues.apache.org/jira/browse/SPARK-17062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681890#comment-15681890 ] Apache Spark commented on SPARK-17062: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Updated] (SPARK-17732) ALTER TABLE DROP PARTITION should support comparators

[ https://issues.apache.org/jira/browse/SPARK-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell updated SPARK-17732: -- Fix Version/s: (was: 2.1.0) 2.2.0 > ALTER TABLE DROP PARTITION

[jira] [Assigned] (SPARK-18339) Don't push down current_timestamp for filters in StructuredStreaming

[ https://issues.apache.org/jira/browse/SPARK-18339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18339: Assignee: Tyson Condie (was: Apache Spark) > Don't push down current_timestamp for

[jira] [Commented] (SPARK-18339) Don't push down current_timestamp for filters in StructuredStreaming

[ https://issues.apache.org/jira/browse/SPARK-18339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681821#comment-15681821 ] Apache Spark commented on SPARK-18339: -- User 'tcondie' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18339) Don't push down current_timestamp for filters in StructuredStreaming

[ https://issues.apache.org/jira/browse/SPARK-18339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18339: Assignee: Apache Spark (was: Tyson Condie) > Don't push down current_timestamp for

[jira] [Commented] (SPARK-18510) Partition schema inference corrupts data

[ https://issues.apache.org/jira/browse/SPARK-18510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681677#comment-15681677 ] Burak Yavuz commented on SPARK-18510: - No. Working on a separate fix > Partition schema inference

[jira] [Commented] (SPARK-18515) AlterTableDropPartitions fails for non-string columns

[ https://issues.apache.org/jira/browse/SPARK-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681626#comment-15681626 ] Dongjoon Hyun commented on SPARK-18515: --- Yes, indeed. Thank you for advice. Also I think we need

[jira] [Commented] (SPARK-18515) AlterTableDropPartitions fails for non-string columns

[ https://issues.apache.org/jira/browse/SPARK-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681605#comment-15681605 ] Herman van Hovell commented on SPARK-18515: --- The Analyzer is injecting casts because the

[jira] [Commented] (SPARK-17732) ALTER TABLE DROP PARTITION should support comparators

[ https://issues.apache.org/jira/browse/SPARK-17732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681570#comment-15681570 ] Apache Spark commented on SPARK-17732: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Commented] (SPARK-18515) AlterTableDropPartitions fails for non-string columns

[ https://issues.apache.org/jira/browse/SPARK-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681480#comment-15681480 ] Dongjoon Hyun commented on SPARK-18515: --- Ah, not only `Cast` occurs, but also the literal also is

[jira] [Comment Edited] (SPARK-18515) AlterTableDropPartitions fails for non-string columns

[ https://issues.apache.org/jira/browse/SPARK-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681450#comment-15681450 ] Dongjoon Hyun edited comment on SPARK-18515 at 11/20/16 4:50 PM: - I see,

[jira] [Commented] (SPARK-18515) AlterTableDropPartitions fails for non-string columns

[ https://issues.apache.org/jira/browse/SPARK-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681450#comment-15681450 ] Dongjoon Hyun commented on SPARK-18515: --- I see, [~hvanhovell]. Anyway, I'll fix this today. Thank

[jira] [Commented] (SPARK-18515) AlterTableDropPartitions fails for non-string columns

[ https://issues.apache.org/jira/browse/SPARK-18515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681435#comment-15681435 ] Herman van Hovell commented on SPARK-18515: --- [~dongjoon] I am reverting this from branch-2.1:

[jira] [Created] (SPARK-18515) AlterTableDropPartitions fails for non-string columns

Herman van Hovell created SPARK-18515: - Summary: AlterTableDropPartitions fails for non-string columns Key: SPARK-18515 URL: https://issues.apache.org/jira/browse/SPARK-18515 Project: Spark

[jira] [Created] (SPARK-18514) Fix `Note:`/`NOTE:`/`Note that` across R API documentation

Hyukjin Kwon created SPARK-18514: Summary: Fix `Note:`/`NOTE:`/`Note that` across R API documentation Key: SPARK-18514 URL: https://issues.apache.org/jira/browse/SPARK-18514 Project: Spark

[jira] [Commented] (SPARK-18447) Fix `Note:`/`NOTE:`/`Note that` across Python API documentation

[ https://issues.apache.org/jira/browse/SPARK-18447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681396#comment-15681396 ] Apache Spark commented on SPARK-18447: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-18513) Record and recover watermark

[ https://issues.apache.org/jira/browse/SPARK-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18513: Assignee: (was: Apache Spark) > Record and recover watermark >

[jira] [Commented] (SPARK-18513) Record and recover watermark

[ https://issues.apache.org/jira/browse/SPARK-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681385#comment-15681385 ] Apache Spark commented on SPARK-18513: -- User 'lw-lin' has created a pull request for this issue:

[jira] [Updated] (SPARK-18513) Record and recover watermark

[ https://issues.apache.org/jira/browse/SPARK-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-18513: -- Summary: Record and recover watermark (was: Record and recover waterwark) > Record and recover

[jira] [Assigned] (SPARK-18513) Record and recover watermark

[ https://issues.apache.org/jira/browse/SPARK-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18513: Assignee: Apache Spark > Record and recover watermark > > >

[jira] [Created] (SPARK-18513) Record and recover waterwark

Liwei Lin created SPARK-18513: - Summary: Record and recover waterwark Key: SPARK-18513 URL: https://issues.apache.org/jira/browse/SPARK-18513 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-16998) select($"column1", explode($"column2")) is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-16998. --- Resolution: Fixed Assignee: Herman van Hovell Fix Version/s: 2.2.0 >

[jira] [Commented] (SPARK-16998) select($"column1", explode($"column2")) is extremely slow

[ https://issues.apache.org/jira/browse/SPARK-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681374#comment-15681374 ] Takeshi Yamamuro commented on SPARK-16998: -- [~hvanhovell] Since SPARK-15214 improves this query

[jira] [Commented] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15681349#comment-15681349 ] Apache Spark commented on SPARK-12978: -- User 'maropu' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-17436) dataframe.write sometimes does not keep sorting

[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680994#comment-15680994 ] Ran Haim edited comment on SPARK-17436 at 11/20/16 12:06 PM: - Sure , I

[jira] [Updated] (SPARK-17436) dataframe.write sometimes does not keep sorting

[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ran Haim updated SPARK-17436: - Description: update *** It seems that in spark 2.1 code, the sorting issue is resolved. The

[jira] [Updated] (SPARK-17436) dataframe.write sometimes does not keep sorting

[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ran Haim updated SPARK-17436: - Description: *** update It seems that in spark 2.0 code, the sorting issue is resolved. The sorter

[jira] [Updated] (SPARK-17436) dataframe.write sometimes does not keep sorting

[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ran Haim updated SPARK-17436: - Description: update *** It seems that in spark 2.0 code, the sorting issue is resolved. The

[jira] [Updated] (SPARK-17436) dataframe.write sometimes does not keep sorting

[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ran Haim updated SPARK-17436: - Priority: Minor (was: Major) > dataframe.write sometimes does not keep sorting >

[jira] [Updated] (SPARK-17436) dataframe.write sometimes does not keep sorting

[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ran Haim updated SPARK-17436: - Description: When using partition by, datawriter can sometimes mess up an ordered dataframe. The

[jira] [Comment Edited] (SPARK-17436) dataframe.write sometimes does not keep sorting

[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680994#comment-15680994 ] Ran Haim edited comment on SPARK-17436 at 11/20/16 11:30 AM: - Sure , I

[jira] [Comment Edited] (SPARK-17436) dataframe.write sometimes does not keep sorting

[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680994#comment-15680994 ] Ran Haim edited comment on SPARK-17436 at 11/20/16 11:29 AM: - Sure -

[jira] [Created] (SPARK-18512) FileNotFoundException on _temporary directory with Spark Streaming 2.0.1 and S3A

Giuseppe Bonaccorso created SPARK-18512: --- Summary: FileNotFoundException on _temporary directory with Spark Streaming 2.0.1 and S3A Key: SPARK-18512 URL: https://issues.apache.org/jira/browse/SPARK-18512

[jira] [Commented] (SPARK-17436) dataframe.write sometimes does not keep sorting

[ https://issues.apache.org/jira/browse/SPARK-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680994#comment-15680994 ] Ran Haim commented on SPARK-17436: -- Sure. Basically I propose to stop using UnsafeKVExternalSorter, and

[jira] [Commented] (SPARK-3359) `sbt/sbt unidoc` doesn't work with Java 8

[ https://issues.apache.org/jira/browse/SPARK-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680887#comment-15680887 ] Sean Owen commented on SPARK-3359: -- Current state: lots of PRs over time have removed most

[jira] [Commented] (SPARK-18511) Add an api for join operation with just the column name and join type.

[ https://issues.apache.org/jira/browse/SPARK-18511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680822#comment-15680822 ] Apache Spark commented on SPARK-18511: -- User 'shiv4nsh' has created a pull request for this issue:

[jira] [Commented] (SPARK-18475) Be able to provide higher parallelization for StructuredStreaming Kafka Source

[ https://issues.apache.org/jira/browse/SPARK-18475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15680780#comment-15680780 ] Ofir Manor commented on SPARK-18475: I understand, but I disagree with you. I think you are mixing