[jira] [Resolved] (SPARK-18144) StreamingQueryListener.QueryStartedEvent is not written to event log

[ https://issues.apache.org/jira/browse/SPARK-18144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-18144. -- Resolution: Fixed Assignee: Nan Zhu Fix Version/s: 2.1.0 2.0.

[jira] [Updated] (SPARK-18144) StreamingQueryListener.QueryStartedEvent is not written to event log

[ https://issues.apache.org/jira/browse/SPARK-18144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18144: - Affects Version/s: 2.0.2 > StreamingQueryListener.QueryStartedEvent is not written to event log >

[jira] [Resolved] (SPARK-18192) Support all file formats in structured streaming

[ https://issues.apache.org/jira/browse/SPARK-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18192. - Resolution: Fixed Fix Version/s: 2.1.0 > Support all file formats in structured streaming

[jira] [Resolved] (SPARK-15472) Add support for writing partitioned `csv`, `json`, `text` formats in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-15472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15472. - Resolution: Fixed Fix Version/s: 2.1.0 > Add support for writing partitioned `csv`, `json`

[jira] [Updated] (SPARK-16575) partition calculation mismatch with sc.binaryFiles

[ https://issues.apache.org/jira/browse/SPARK-16575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16575: Target Version/s: 2.1.0 > partition calculation mismatch with sc.binaryFiles >

[jira] [Updated] (SPARK-13184) Support minPartitions parameter for JSON and CSV datasources as options

[ https://issues.apache.org/jira/browse/SPARK-13184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-13184: Target Version/s: 2.2.0 (was: 2.1.0) > Support minPartitions parameter for JSON and CSV datasource

[jira] [Commented] (SPARK-10840) SparkSQL doesn't work well with JSON

[ https://issues.apache.org/jira/browse/SPARK-10840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627929#comment-15627929 ] Hyukjin Kwon commented on SPARK-10840: -- I guess this is a duplicate of SPARK-17969 (

[jira] [Updated] (SPARK-14878) Support Trim characters in the string trim function

[ https://issues.apache.org/jira/browse/SPARK-14878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14878: Target Version/s: 2.2.0 (was: 2.1.0) > Support Trim characters in the string trim function > -

[jira] [Updated] (SPARK-18183) INSERT OVERWRITE TABLE ... PARTITION will overwrite the entire Datasource table instead of just the specified partition

[ https://issues.apache.org/jira/browse/SPARK-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-18183: Assignee: Eric Liang > INSERT OVERWRITE TABLE ... PARTITION will overwrite the entire Datasource >

[jira] [Updated] (SPARK-14878) Support Trim characters in the string trim function

[ https://issues.apache.org/jira/browse/SPARK-14878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-14878: Description: The current Spark SQL does not support the trim characters in the string trim functio

[jira] [Closed] (SPARK-16972) Move DriverEndpoint out of CoarseGrainedSchedulerBackend

[ https://issues.apache.org/jira/browse/SPARK-16972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-16972. --- Resolution: Won't Fix Closing this as won't fix for now given the comments on the pull request. The

[jira] [Updated] (SPARK-16972) Move DriverEndpoint out of CoarseGrainedSchedulerBackend

[ https://issues.apache.org/jira/browse/SPARK-16972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-16972: Target Version/s: (was: 2.1.0) > Move DriverEndpoint out of CoarseGrainedSchedulerBackend > -

[jira] [Resolved] (SPARK-18184) INSERT [INTO|OVERWRITE] TABLE ... PARTITION for Datasource tables cannot handle partitions with custom locations

[ https://issues.apache.org/jira/browse/SPARK-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18184. - Resolution: Fixed Assignee: Eric Liang Fix Version/s: 2.1.0 Target Ve

[jira] [Resolved] (SPARK-18183) INSERT OVERWRITE TABLE ... PARTITION will overwrite the entire Datasource table instead of just the specified partition

[ https://issues.apache.org/jira/browse/SPARK-18183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18183. - Resolution: Fixed Fix Version/s: 2.1.0 Target Version/s: 2.1.0 > INSERT OVERWRIT

[jira] [Commented] (SPARK-14960) Don't perform treeAggregation in local mode

[ https://issues.apache.org/jira/browse/SPARK-14960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627902#comment-15627902 ] Reynold Xin commented on SPARK-14960: - [~joshrosen] any reason why you closed the pul

[jira] [Updated] (SPARK-17058) Add maven snapshots-and-staging profile to build/test against staging artifacts

[ https://issues.apache.org/jira/browse/SPARK-17058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17058: Target Version/s: (was: 2.1.0) > Add maven snapshots-and-staging profile to build/test against st

[jira] [Updated] (SPARK-17267) Long running structured streaming requirements

[ https://issues.apache.org/jira/browse/SPARK-17267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17267: Assignee: Peter Lee > Long running structured streaming requirements >

[jira] [Resolved] (SPARK-17475) HDFSMetadataLog should not leak CRC files

[ https://issues.apache.org/jira/browse/SPARK-17475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17475. - Resolution: Fixed Fix Version/s: 2.1.0 > HDFSMetadataLog should not leak CRC files > -

[jira] [Resolved] (SPARK-17267) Long running structured streaming requirements

[ https://issues.apache.org/jira/browse/SPARK-17267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17267. - Resolution: Fixed Fix Version/s: 2.1.0 > Long running structured streaming requirements >

[jira] [Comment Edited] (SPARK-15420) Repartition and sort before Parquet writes

[ https://issues.apache.org/jira/browse/SPARK-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627859#comment-15627859 ] Reynold Xin edited comment on SPARK-15420 at 11/2/16 5:57 AM: -

[jira] [Commented] (SPARK-15420) Repartition and sort before Parquet writes

[ https://issues.apache.org/jira/browse/SPARK-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627859#comment-15627859 ] Reynold Xin commented on SPARK-15420: - Ryan I looked at this just now (sorry not look

[jira] [Commented] (SPARK-18133) Python ML Pipeline Example has syntax errors

[ https://issues.apache.org/jira/browse/SPARK-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627856#comment-15627856 ] Apache Spark commented on SPARK-18133: -- User 'jagadeesanas2' has created a pull requ

[jira] [Updated] (SPARK-15420) Repartition and sort before Parquet writes

[ https://issues.apache.org/jira/browse/SPARK-15420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-15420: Target Version/s: 2.2.0 (was: 2.1.0) > Repartition and sort before Parquet writes > --

[jira] [Commented] (SPARK-17822) JVMObjectTracker.objMap may leak JVM objects

[ https://issues.apache.org/jira/browse/SPARK-17822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627846#comment-15627846 ] Felix Cheung commented on SPARK-17822: -- I don't have a good handle on what actually

[jira] [Commented] (SPARK-18212) Flaky test: org.apache.spark.sql.kafka010.KafkaSourceSuite.assign from specific offsets

[ https://issues.apache.org/jira/browse/SPARK-18212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627838#comment-15627838 ] Cody Koeninger commented on SPARK-18212: So here's a heavily excerpted version of

[jira] [Updated] (SPARK-17868) Do not use bitmasks during parsing and analysis of CUBE/ROLLUP/GROUPING SETS

[ https://issues.apache.org/jira/browse/SPARK-17868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17868: Target Version/s: 2.2.0 (was: 2.1.0) > Do not use bitmasks during parsing and analysis of CUBE/ROL

[jira] [Closed] (SPARK-17402) separate the management of temp views and metastore tables/views in SessionCatalog

[ https://issues.apache.org/jira/browse/SPARK-17402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-17402. --- Resolution: Fixed > separate the management of temp views and metastore tables/views in > SessionCat

[jira] [Commented] (SPARK-18193) queueStream not updated if rddQueue.add after create queueStream in Java

[ https://issues.apache.org/jira/browse/SPARK-18193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627774#comment-15627774 ] Hubert Kang commented on SPARK-18193: - Thanks Sean. While it's inconsistent with tha

[jira] [Resolved] (SPARK-17992) HiveClient.getPartitionsByFilter throws an exception for some unsupported filters when hive.metastore.try.direct.sql=false

[ https://issues.apache.org/jira/browse/SPARK-17992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17992. - Resolution: Fixed Assignee: Michael Allman Fix Version/s: 2.1.0 > HiveClient.getP

[jira] [Commented] (SPARK-17838) Strict type checking for arguments with a better messages across APIs.

[ https://issues.apache.org/jira/browse/SPARK-17838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627770#comment-15627770 ] Felix Cheung commented on SPARK-17838: -- merged to master. this should be very safe t

[jira] [Resolved] (SPARK-17838) Strict type checking for arguments with a better messages across APIs.

[ https://issues.apache.org/jira/browse/SPARK-17838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-17838. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.2.0 Targe

[jira] [Commented] (SPARK-7755) MetadataCache.refresh does not take into account _SUCCESS

[ https://issues.apache.org/jira/browse/SPARK-7755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627754#comment-15627754 ] Hyukjin Kwon commented on SPARK-7755: - Hi [~liancheng], didn't we remove {{DirectParqu

[jira] [Commented] (SPARK-18073) Migrate wiki to spark.apache.org web site

[ https://issues.apache.org/jira/browse/SPARK-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627731#comment-15627731 ] Siddharth Ahuja commented on SPARK-18073: - Hi [~srowen], I would be happy to work

[jira] [Closed] (SPARK-6825) Data sources implementation to support `sequenceFile`

[ https://issues.apache.org/jira/browse/SPARK-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-6825. -- Resolution: Won't Fix I'm marking this as won't fix for now. It's unclear how the interface would look

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627715#comment-15627715 ] Xiao Li commented on SPARK-18209: - Ok, will do it. Thanks! > More robust view canonicali

[jira] [Commented] (SPARK-6825) Data sources implementation to support `sequenceFile`

[ https://issues.apache.org/jira/browse/SPARK-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627714#comment-15627714 ] Hyukjin Kwon commented on SPARK-6825: - Hi [~shivaram], do we still need this? If so, I

[jira] [Created] (SPARK-18217) Disallow creating permanent views based on temporary views

Reynold Xin created SPARK-18217: --- Summary: Disallow creating permanent views based on temporary views Key: SPARK-18217 URL: https://issues.apache.org/jira/browse/SPARK-18217 Project: Spark Issu

[jira] [Assigned] (SPARK-18217) Disallow creating permanent views based on temporary views

[ https://issues.apache.org/jira/browse/SPARK-18217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-18217: --- Assignee: Xiao Li > Disallow creating permanent views based on temporary views > ---

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627710#comment-15627710 ] Reynold Xin commented on SPARK-18209: - Actually I'd consider it a "bug" and fix this

[jira] [Commented] (SPARK-16808) History Server main page does not honor APPLICATION_WEB_PROXY_BASE

[ https://issues.apache.org/jira/browse/SPARK-16808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627706#comment-15627706 ] Vinayak Joshi commented on SPARK-16808: --- The same issue also affects the case where

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627702#comment-15627702 ] Xiao Li commented on SPARK-18209: - True. {code} Seq((1, (1, 1))).toDF().createTemp

[jira] [Comment Edited] (SPARK-4549) Support BigInt -> Decimal in convertToCatalyst in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627695#comment-15627695 ] Hyukjin Kwon edited comment on SPARK-4549 at 11/2/16 4:41 AM: --

[jira] [Commented] (SPARK-4549) Support BigInt -> Decimal in convertToCatalyst in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627695#comment-15627695 ] Hyukjin Kwon commented on SPARK-4549: - Could we maybe close this for now if no one can

[jira] [Commented] (SPARK-17967) Support for list or other types as an option for datasources

[ https://issues.apache.org/jira/browse/SPARK-17967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627698#comment-15627698 ] Reynold Xin commented on SPARK-17967: - +1 on json arrays. > Support for list or othe

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627679#comment-15627679 ] Reynold Xin commented on SPARK-18209: - Yes, both global and local temp views. The iss

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627673#comment-15627673 ] Xiao Li commented on SPARK-18209: - Without SQL expansion, also need to block the global t

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627664#comment-15627664 ] Reynold Xin commented on SPARK-18209: - Yup - we will have to disallow it, for good re

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627662#comment-15627662 ] Xiao Li commented on SPARK-18209: - Yeah, I just posted the example to show it. I mentione

[jira] [Commented] (SPARK-18133) Python ML Pipeline Example has syntax errors

[ https://issues.apache.org/jira/browse/SPARK-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627648#comment-15627648 ] Nirmal Fernando commented on SPARK-18133: - Thanks All. > Python ML Pipeline Exam

[jira] [Resolved] (SPARK-18216) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-18216. - Resolution: Fixed Fix Version/s: 2.1.0 > Make Column.expr public > ---

[jira] [Commented] (SPARK-18133) Python ML Pipeline Example has syntax errors

[ https://issues.apache.org/jira/browse/SPARK-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627633#comment-15627633 ] Apache Spark commented on SPARK-18133: -- User 'jagadeesanas2' has created a pull requ

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627628#comment-15627628 ] Reynold Xin commented on SPARK-18209: - That's what [~vssrinath] pointed out isn't tha

[jira] [Commented] (SPARK-17816) Json serialzation of accumulators are failing with ConcurrentModificationException

[ https://issues.apache.org/jira/browse/SPARK-17816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627622#comment-15627622 ] Jonathan Alvarado commented on SPARK-17816: --- Can I assume that I can disregard

[jira] [Comment Edited] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627595#comment-15627595 ] Xiao Li edited comment on SPARK-18209 at 11/2/16 4:00 AM: -- If we

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627599#comment-15627599 ] Xiao Li commented on SPARK-18209: - Please hold on until we finalize the design. > More r

[jira] [Assigned] (SPARK-17895) Improve documentation of "rowsBetween" and "rangeBetween"

[ https://issues.apache.org/jira/browse/SPARK-17895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17895: Assignee: Apache Spark > Improve documentation of "rowsBetween" and "rangeBetween" > -

[jira] [Assigned] (SPARK-17895) Improve documentation of "rowsBetween" and "rangeBetween"

[ https://issues.apache.org/jira/browse/SPARK-17895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17895: Assignee: (was: Apache Spark) > Improve documentation of "rowsBetween" and "rangeBetwe

[jira] [Commented] (SPARK-17895) Improve documentation of "rowsBetween" and "rangeBetween"

[ https://issues.apache.org/jira/browse/SPARK-17895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627597#comment-15627597 ] Apache Spark commented on SPARK-17895: -- User 'david-weiluo-ren' has created a pull r

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627595#comment-15627595 ] Xiao Li commented on SPARK-18209: - If we do not qualify the table/persistent view name wh

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627588#comment-15627588 ] Xiao Li commented on SPARK-18209: - {code} sql("CREATE VIEW jtv1 AS SELECT * FROM jt

[jira] [Updated] (SPARK-18198) Highlight code snippets for Streaming integretion docs

[ https://issues.apache.org/jira/browse/SPARK-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-18198: -- Component/s: (was: SQL) Structured Streaming > Highlight code snippets for Streami

[jira] [Commented] (SPARK-16545) Structured Streaming : foreachSink creates the Physical Plan multiple times per TriggerInterval

[ https://issues.apache.org/jira/browse/SPARK-16545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627554#comment-15627554 ] Liwei Lin commented on SPARK-16545: --- hi [~mariobriggs], per discussion on the PR, would

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627518#comment-15627518 ] Dongjoon Hyun commented on SPARK-18209: --- Thank you. Now, I understand. I have been

[jira] [Comment Edited] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627518#comment-15627518 ] Dongjoon Hyun edited comment on SPARK-18209 at 11/2/16 3:21 AM: ---

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627502#comment-15627502 ] Reynold Xin commented on SPARK-18209: - The pr is still useful -- the only thing that

[jira] [Commented] (SPARK-18206) Log instrumentation in MPC, NB, LDA, AFT, GLR, Isotonic, LinReg

[ https://issues.apache.org/jira/browse/SPARK-18206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627497#comment-15627497 ] Apache Spark commented on SPARK-18206: -- User 'zhengruifeng' has created a pull reque

[jira] [Assigned] (SPARK-18206) Log instrumentation in MPC, NB, LDA, AFT, GLR, Isotonic, LinReg

[ https://issues.apache.org/jira/browse/SPARK-18206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18206: Assignee: zhengruifeng (was: Apache Spark) > Log instrumentation in MPC, NB, LDA, AFT, GL

[jira] [Assigned] (SPARK-18206) Log instrumentation in MPC, NB, LDA, AFT, GLR, Isotonic, LinReg

[ https://issues.apache.org/jira/browse/SPARK-18206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18206: Assignee: Apache Spark (was: zhengruifeng) > Log instrumentation in MPC, NB, LDA, AFT, GL

[jira] [Commented] (SPARK-18107) Insert overwrite statement runs much slower in spark-sql than it does in hive-client

[ https://issues.apache.org/jira/browse/SPARK-18107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627426#comment-15627426 ] Apache Spark commented on SPARK-18107: -- User 'viirya' has created a pull request for

[jira] [Commented] (SPARK-17879) Don't compact metadata logs constantly into a single compacted file

[ https://issues.apache.org/jira/browse/SPARK-17879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627421#comment-15627421 ] Burak Yavuz commented on SPARK-17879: - We should be doing the second. What you said m

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627407#comment-15627407 ] Dongjoon Hyun commented on SPARK-18209: --- BTW, [~rxin]. I didn't notice that the fo

[jira] [Commented] (SPARK-17982) Spark 2.0.0 CREATE VIEW statement fails :: java.lang.RuntimeException: Failed to analyze the canonicalized SQL. It is possible there is a bug in Spark.

[ https://issues.apache.org/jira/browse/SPARK-17982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627406#comment-15627406 ] Franck Tago commented on SPARK-17982: - Wanted to mention that I was able to successfu

[jira] [Commented] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627367#comment-15627367 ] Jiang Xingbo commented on SPARK-18209: -- I'm working on this, thanks! > More robust

[jira] [Commented] (SPARK-18167) Flaky test when hive partition pruning is enabled

[ https://issues.apache.org/jira/browse/SPARK-18167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627343#comment-15627343 ] Apache Spark commented on SPARK-18167: -- User 'ericl' has created a pull request for

[jira] [Updated] (SPARK-17937) Clarify Kafka offset semantics for Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17937: - Component/s: Structured Streaming > Clarify Kafka offset semantics for Structured Streami

[jira] [Updated] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.1.0

[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18057: - Component/s: Structured Streaming > Update structured streaming kafka from 10.0.1 to 10.1

[jira] [Updated] (SPARK-17343) Prerequisites for Kafka 0.8 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17343: - Component/s: (was: DStreams) Structured Streaming > Prerequisites fo

[jira] [Updated] (SPARK-17837) Disaster recovery of offsets from WAL

[ https://issues.apache.org/jira/browse/SPARK-17837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17837: - Component/s: Structured Streaming > Disaster recovery of offsets from WAL > -

[jira] [Updated] (SPARK-17834) Fetch the earliest offsets manually in KafkaSource instead of counting on KafkaConsumer

[ https://issues.apache.org/jira/browse/SPARK-17834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17834: - Component/s: (was: SQL) Structured Streaming > Fetch the earliest of

[jira] [Updated] (SPARK-17346) Kafka 0.10 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17346: - Component/s: (was: DStreams) Structured Streaming > Kafka 0.10 suppo

[jira] [Updated] (SPARK-17345) Prerequisites for Kafka 0.10 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17345: - Component/s: (was: DStreams) Structured Streaming > Prerequisites fo

[jira] [Closed] (SPARK-18201) add toDense and toSparse into Matrix trait, like Vector design

[ https://issues.apache.org/jira/browse/SPARK-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu closed SPARK-18201. -- Resolution: Duplicate It will fix in this PR https://github.com/apache/spark/pull/15628 > add toDense

[jira] [Updated] (SPARK-17815) Report committed offsets

[ https://issues.apache.org/jira/browse/SPARK-17815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17815: - Component/s: (was: SQL) Structured Streaming > Report committed offs

[jira] [Updated] (SPARK-17812) More granular control of starting offsets (assign)

[ https://issues.apache.org/jira/browse/SPARK-17812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17812: - Component/s: (was: SQL) Structured Streaming > More granular control

[jira] [Updated] (SPARK-17813) Maximum data per trigger

[ https://issues.apache.org/jira/browse/SPARK-17813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17813: - Component/s: (was: SQL) Structured Streaming > Maximum data per trig

[jira] [Updated] (SPARK-17344) Kafka 0.8 support for Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-17344: - Component/s: (was: DStreams) Structured Streaming > Kafka 0.8 suppor

[jira] [Closed] (SPARK-17345) Prerequisites for Kafka 0.10 support in Structured Streaming

[ https://issues.apache.org/jira/browse/SPARK-17345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust closed SPARK-17345. Resolution: Fixed > Prerequisites for Kafka 0.10 support in Structured Streaming >

[jira] [Updated] (SPARK-15406) Structured streaming support for consuming from Kafka

[ https://issues.apache.org/jira/browse/SPARK-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-15406: - Component/s: Structured Streaming > Structured streaming support for consuming from Kafka

[jira] [Updated] (SPARK-17183) put hive serde table schema to table properties like data source table

[ https://issues.apache.org/jira/browse/SPARK-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17183: - Priority: Blocker (was: Major) > put hive serde table schema to table properties like data source table

[jira] [Resolved] (SPARK-18025) Port streaming to use the commit protocol API

[ https://issues.apache.org/jira/browse/SPARK-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-18025. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 15710 [htt

[jira] [Updated] (SPARK-18192) Support all file formats in structured streaming

[ https://issues.apache.org/jira/browse/SPARK-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18192: Component/s: Structured Streaming > Support all file formats in structured streaming >

[jira] [Updated] (SPARK-18025) Port streaming to use the commit protocol API

[ https://issues.apache.org/jira/browse/SPARK-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18025: - Component/s: Structured Streaming > Port streaming to use the commit protocol API > -

[jira] [Closed] (SPARK-18215) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-18215. --- Resolution: Duplicate Target Version/s: (was: 2.1.0) > Make Column.expr public >

[jira] [Assigned] (SPARK-18216) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-18216: --- Assignee: Reynold Xin > Make Column.expr public > --- > >

[jira] [Assigned] (SPARK-18215) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-18215: --- Assignee: Reynold Xin > Make Column.expr public > --- > >

[jira] [Assigned] (SPARK-18216) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18216: Assignee: (was: Apache Spark) > Make Column.expr public > --- > >

[jira] [Assigned] (SPARK-18216) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18216: Assignee: Apache Spark > Make Column.expr public > --- > >

[jira] [Commented] (SPARK-18216) Make Column.expr public

[ https://issues.apache.org/jira/browse/SPARK-18216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15627220#comment-15627220 ] Apache Spark commented on SPARK-18216: -- User 'rxin' has created a pull request for t

[jira] [Created] (SPARK-18215) Make Column.expr public

Reynold Xin created SPARK-18215: --- Summary: Make Column.expr public Key: SPARK-18215 URL: https://issues.apache.org/jira/browse/SPARK-18215 Project: Spark Issue Type: Improvement Compo

[jira] [Created] (SPARK-18216) Make Column.expr public

Reynold Xin created SPARK-18216: --- Summary: Make Column.expr public Key: SPARK-18216 URL: https://issues.apache.org/jira/browse/SPARK-18216 Project: Spark Issue Type: Improvement Compo

  1   2   3   4   >