[jira] [Created] (SPARK-22936) providing HttpStreamSource and HttpStreamSink

2018-01-01 Thread bluejoe (JIRA)
bluejoe created SPARK-22936: --- Summary: providing HttpStreamSource and HttpStreamSink Key: SPARK-22936 URL: https://issues.apache.org/jira/browse/SPARK-22936 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-22936) providing HttpStreamSource and HttpStreamSink

2018-01-01 Thread bluejoe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bluejoe updated SPARK-22936: Description: Hi, in my project I completed a spark-http-stream, which is now available on

[jira] [Updated] (SPARK-22936) providing HttpStreamSource and HttpStreamSink

2018-01-01 Thread bluejoe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bluejoe updated SPARK-22936: Description: Hi, in my project I completed a spark-http-stream, which is now available on

[jira] [Updated] (SPARK-22936) providing HttpStreamSource and HttpStreamSink

2018-01-01 Thread bluejoe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bluejoe updated SPARK-22936: Description: Hi, in my project I completed a spark-http-stream, which is now available on

[jira] [Commented] (SPARK-17762) invokeJava fails when serialized argument list is larger than INT_MAX (2,147,483,647) bytes

2018-01-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307724#comment-16307724 ] Felix Cheung commented on SPARK-17762: -- is this still needed after SPARK-17790 is fixed? >

[jira] [Commented] (SPARK-16693) Remove R deprecated methods

2018-01-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307725#comment-16307725 ] Felix Cheung commented on SPARK-16693: -- [~shivaram]we didn't do this, not sure if we should to get

[jira] [Commented] (SPARK-16366) Time comparison failures in SparkR unit tests

2018-01-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307728#comment-16307728 ] Felix Cheung commented on SPARK-16366: -- not 100% sure, but a a similar timestamp test failure was

[jira] [Updated] (SPARK-14037) count(df) is very slow for dataframe constructed using SparkR::createDataFrame

2018-01-01 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-14037: - Summary: count(df) is very slow for dataframe constructed using SparkR::createDataFrame (was:

[jira] [Comment Edited] (SPARK-22918) sbt test (spark - local) fail after upgrading to 2.2.1 with: java.security.AccessControlException: access denied org.apache.derby.security.SystemPermission( "engi

2018-01-01 Thread Damian Momot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16306316#comment-16306316 ] Damian Momot edited comment on SPARK-22918 at 1/2/18 7:37 AM: -- As a quick

[jira] [Comment Edited] (SPARK-22918) sbt test (spark - local) fail after upgrading to 2.2.1 with: java.security.AccessControlException: access denied org.apache.derby.security.SystemPermission( "engi

2018-01-01 Thread Damian Momot (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16306316#comment-16306316 ] Damian Momot edited comment on SPARK-22918 at 1/2/18 7:37 AM: -- As a quick

[jira] [Commented] (SPARK-21773) Should Install mkdocs if missing in the path in SQL documentation build

2018-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307650#comment-16307650 ] Apache Spark commented on SPARK-21773: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-21724) Missing since information in the documentation of date functions

2018-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307648#comment-16307648 ] Apache Spark commented on SPARK-21724: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-19165) UserDefinedFunction should verify call arguments and provide readable exception in case of mismatch

2018-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307649#comment-16307649 ] Apache Spark commented on SPARK-19165: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-21536) Remove the workaroud to allow dots in field names in R's createDataFame

2018-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307658#comment-16307658 ] Apache Spark commented on SPARK-21536: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-21536) Remove the workaroud to allow dots in field names in R's createDataFame

2018-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21536: Assignee: Apache Spark > Remove the workaroud to allow dots in field names in R's

[jira] [Assigned] (SPARK-21536) Remove the workaroud to allow dots in field names in R's createDataFame

2018-01-01 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21536: Assignee: (was: Apache Spark) > Remove the workaroud to allow dots in field names in

[jira] [Commented] (SPARK-22903) AlreadyBeingCreatedException in stage retry caused by wrong attemptNumber

2018-01-01 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307663#comment-16307663 ] liupengcheng commented on SPARK-22903: -- [~imranr] I think keep the fetchfailed stage tasks running

[jira] [Commented] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Suchith J N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307462#comment-16307462 ] Suchith J N commented on SPARK-22553: -

[jira] [Commented] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Suchith J N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307464#comment-16307464 ] Suchith J N commented on SPARK-22553: - {panel:title=Removed_FROM_from_nonReserved_list.patch} ---

[jira] [Issue Comment Deleted] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Suchith J N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suchith J N updated SPARK-22553: Comment: was deleted (was:

[jira] [Commented] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Suchith J N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307458#comment-16307458 ] Suchith J N commented on SPARK-22553: - Shall I work on this issue? > Drop FROM in nonReserved >

[jira] [Comment Edited] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Suchith J N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307467#comment-16307467 ] Suchith J N edited comment on SPARK-22553 at 1/1/18 3:01 PM: - Removed FROM

[jira] [Commented] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Suchith J N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307499#comment-16307499 ] Suchith J N commented on SPARK-22553: - Shall I submit a PR? > Drop FROM in nonReserved >

[jira] [Updated] (SPARK-22935) Dataset with Java Beans for java.sql.Date produces incorrect result

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22935: - Description: The following code prints {{c=0}}. The value of {{c}} must be 2. {{ds}}

[jira] [Created] (SPARK-22935) Dataset with Java Beans for java.sql.Date produces incorrect result

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-22935: Summary: Dataset with Java Beans for java.sql.Date produces incorrect result Key: SPARK-22935 URL: https://issues.apache.org/jira/browse/SPARK-22935 Project:

[jira] [Commented] (SPARK-22935) Dataset with Java Beans for java.sql.Date produces incorrect result

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307520#comment-16307520 ] Kazuaki Ishizaki commented on SPARK-22935: -- I am working for this. > Dataset with Java Beans

[jira] [Resolved] (SPARK-22530) Add ArrayType Support for working with Pandas and Arrow

2018-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-22530. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/20114 > Add ArrayType Support

[jira] [Commented] (SPARK-21413) Multiple projections with CASE WHEN fails to run generated codes

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307531#comment-16307531 ] Kazuaki Ishizaki commented on SPARK-21413: -- SPARK-22520 solved this issue. > Multiple

[jira] [Commented] (SPARK-22902) do not count failures if the speculative task is killed as the same task finished in other executor

2018-01-01 Thread Keith Sun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307596#comment-16307596 ] Keith Sun commented on SPARK-22902: --- I could propose a simple solution for this : once the task in one

[jira] [Updated] (SPARK-22530) Add ArrayType Support for working with Pandas and Arrow

2018-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-22530: - Fix Version/s: 2.3.0 > Add ArrayType Support for working with Pandas and Arrow >

[jira] [Assigned] (SPARK-22530) Add ArrayType Support for working with Pandas and Arrow

2018-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-22530: Assignee: Bryan Cutler > Add ArrayType Support for working with Pandas and Arrow >

[jira] [Commented] (SPARK-22898) collect_set aggregation on bucketed table causes an exchange stage

2018-01-01 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307591#comment-16307591 ] Liang-Chi Hsieh commented on SPARK-22898: - I think this should already be fixed by SPARK-3.

[jira] [Resolved] (SPARK-21413) Multiple projections with CASE WHEN fails to run generated codes

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki resolved SPARK-21413. -- Resolution: Duplicate > Multiple projections with CASE WHEN fails to run generated

[jira] [Resolved] (SPARK-22932) Refactor AnalysisContext

2018-01-01 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-22932. - Resolution: Fixed Fix Version/s: 2.3.0 > Refactor AnalysisContext > > >

[jira] [Comment Edited] (SPARK-22898) collect_set aggregation on bucketed table causes an exchange stage

2018-01-01 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307591#comment-16307591 ] Liang-Chi Hsieh edited comment on SPARK-22898 at 1/2/18 1:46 AM: - I think

[jira] [Resolved] (SPARK-21616) SparkR 2.3.0 migration guide, release note

2018-01-01 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21616. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 20106

[jira] [Commented] (SPARK-22903) AlreadyBeingCreatedException in stage retry caused by wrong attemptNumber

2018-01-01 Thread liupengcheng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307633#comment-16307633 ] liupengcheng commented on SPARK-22903: -- [~imranr] I agree what you said about the cleanup did not

[jira] [Updated] (SPARK-22935) Dataset with Java Beans for java.sql.Date throws CompileException

2018-01-01 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-22935: - Description: The following code can throw an exception with or without whole-stage

[jira] [Commented] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307469#comment-16307469 ] Takeshi Yamamuro commented on SPARK-22553: -- Look fine to me > Drop FROM in nonReserved >

[jira] [Commented] (SPARK-22297) Flaky test: BlockManagerSuite "Shuffle registration timeout and maxAttempts conf"

2018-01-01 Thread Jiang Xingbo (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307501#comment-16307501 ] Jiang Xingbo commented on SPARK-22297: -- How often do we run into this? Personally I can't repro this

[jira] [Resolved] (SPARK-22298) SparkUI executor URL encode appID

2018-01-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22298. --- Resolution: Won't Fix > SparkUI executor URL encode appID > - > >

[jira] [Commented] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Suchith J N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307466#comment-16307466 ] Suchith J N commented on SPARK-22553: - {code:java} ---

[jira] [Comment Edited] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Suchith J N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16307466#comment-16307466 ] Suchith J N edited comment on SPARK-22553 at 1/1/18 2:58 PM: - {code} ---

[jira] [Issue Comment Deleted] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Suchith J N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suchith J N updated SPARK-22553: Comment: was deleted (was: {panel:title=Removed_FROM_from_nonReserved_list.patch} ---

[jira] [Updated] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Suchith J N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suchith J N updated SPARK-22553: Attachment: Removed_FROM_from_nonReserved_list.patch Removed FROM from nonReserved > Drop FROM in

[jira] [Issue Comment Deleted] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Suchith J N (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suchith J N updated SPARK-22553: Comment: was deleted (was: {code} ---