[jira] [Commented] (SPARK-20199) GradientBoostedTreesModel doesn't have Column Sampling Rate Paramenter

2017-04-18 Thread pralabhkumar (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974126#comment-15974126 ] pralabhkumar commented on SPARK-20199: -- any update ? > GradientBoostedTreesModel doesn't have

[jira] [Updated] (SPARK-20382) fileSystem.closed when we run load data on beeline

2017-04-18 Thread QQShu1 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QQShu1 updated SPARK-20382: --- Summary: fileSystem.closed when we run load data on beeline (was: we meet fileSystem.closed when we run

[jira] [Updated] (SPARK-20382) fileSystem.closed when run load data on beeline

2017-04-18 Thread QQShu1 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QQShu1 updated SPARK-20382: --- Summary: fileSystem.closed when run load data on beeline (was: fileSystem.closed when we run load data

[jira] [Comment Edited] (SPARK-20382) we meet fileSystem.closed when we run load data on beeline

2017-04-18 Thread QQShu1 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974022#comment-15974022 ] QQShu1 edited comment on SPARK-20382 at 4/19/17 3:58 AM: - I find in Hive.java

[jira] [Commented] (SPARK-20382) we meet fileSystem.closed when we run load data on beeline

2017-04-18 Thread QQShu1 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974022#comment-15974022 ] QQShu1 commented on SPARK-20382: I find in Hive.java needToCopy HadoopShims.HdfsEncryptionShim

[jira] [Closed] (SPARK-20363) sessionstate.get is get the same object in hive project, when I use spark-beeline

2017-04-18 Thread QQShu1 (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] QQShu1 closed SPARK-20363. -- > sessionstate.get is get the same object in hive project, when I use > spark-beeline >

[jira] [Created] (SPARK-20382) we meet fileSystem.closed when we run load data on beeline

2017-04-18 Thread QQShu1 (JIRA)
QQShu1 created SPARK-20382: -- Summary: we meet fileSystem.closed when we run load data on beeline Key: SPARK-20382 URL: https://issues.apache.org/jira/browse/SPARK-20382 Project: Spark Issue Type:

[jira] [Commented] (SPARK-20263) create empty dataframes in sparkR

2017-04-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974001#comment-15974001 ] Felix Cheung commented on SPARK-20263: -- I see - so you would need a way to create an empty dataframe

[jira] [Assigned] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20356: Assignee: Apache Spark > Spark sql group by returns incorrect results after join +

[jira] [Assigned] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20356: Assignee: (was: Apache Spark) > Spark sql group by returns incorrect results after

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973996#comment-15973996 ] Apache Spark commented on SPARK-20356: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-20381) ObjectHashAggregateExec is missing numOutputRows

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973980#comment-15973980 ] Apache Spark commented on SPARK-20381: -- User 'yucai' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20381) ObjectHashAggregateExec is missing numOutputRows

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20381: Assignee: (was: Apache Spark) > ObjectHashAggregateExec is missing numOutputRows >

[jira] [Assigned] (SPARK-20381) ObjectHashAggregateExec is missing numOutputRows

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20381: Assignee: Apache Spark > ObjectHashAggregateExec is missing numOutputRows >

[jira] [Commented] (SPARK-20208) Document R fpGrowth support in vignettes, programming guide and code example

2017-04-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973976#comment-15973976 ] Felix Cheung commented on SPARK-20208: -- I [~zero323] I checked there didn't seem to have ML

[jira] [Assigned] (SPARK-20208) Document R fpGrowth support in vignettes, programming guide and code example

2017-04-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-20208: Assignee: Maciej Szymkiewicz > Document R fpGrowth support in vignettes, programming

[jira] [Updated] (SPARK-20254) SPARK-19716 generates unnecessary data conversion for Dataset with primitive array

2017-04-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-20254: Fix Version/s: 2.3.0 > SPARK-19716 generates unnecessary data conversion for Dataset with

[jira] [Assigned] (SPARK-20254) SPARK-19716 generates unnecessary data conversion for Dataset with primitive array

2017-04-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-20254: --- Assignee: Kazuaki Ishizaki > SPARK-19716 generates unnecessary data conversion for Dataset

[jira] [Resolved] (SPARK-20254) SPARK-19716 generates unnecessary data conversion for Dataset with primitive array

2017-04-18 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20254. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17568

[jira] [Created] (SPARK-20381) ObjectHashAggregateExec is missing numOutputRows

2017-04-18 Thread yucai (JIRA)
yucai created SPARK-20381: - Summary: ObjectHashAggregateExec is missing numOutputRows Key: SPARK-20381 URL: https://issues.apache.org/jira/browse/SPARK-20381 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973936#comment-15973936 ] Liang-Chi Hsieh commented on SPARK-20356: - [~dkbiswal] Yeah, right. Thanks. We need to force the

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973929#comment-15973929 ] Dilip Biswal commented on SPARK-20356: -- [~viirya] Did you try from spark-shell or from one of our

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973928#comment-15973928 ] Liang-Chi Hsieh commented on SPARK-20356: - I think I found the reason of the issue. I am working

[jira] [Updated] (SPARK-18570) Consider supporting other R formula operators

2017-04-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-18570: - Target Version/s: 2.3.0 (was: 2.2.0) > Consider supporting other R formula operators >

[jira] [Updated] (SPARK-18569) Support R formula arithmetic

2017-04-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-18569: - Target Version/s: 2.3.0 (was: 2.2.0) > Support R formula arithmetic >

[jira] [Commented] (SPARK-19282) RandomForestRegressionModel summary should expose getMaxDepth

2017-04-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973926#comment-15973926 ] Felix Cheung commented on SPARK-19282: -- hi - the R side changes are done, correct? >

[jira] [Commented] (SPARK-20015) Document R Structured Streaming (experimental) in R vignettes and R & SS programming guide

2017-04-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973924#comment-15973924 ] Felix Cheung commented on SPARK-20015: -- yes, sorry, it's my bad - they should be documentation >

[jira] [Updated] (SPARK-20015) Document R Structured Streaming (experimental) in R vignettes and R & SS programming guide

2017-04-18 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20015: - Issue Type: Documentation (was: Bug) > Document R Structured Streaming (experimental) in R

[jira] [Comment Edited] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973902#comment-15973902 ] Liang-Chi Hsieh edited comment on SPARK-20356 at 4/19/17 2:05 AM: --

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973902#comment-15973902 ] Liang-Chi Hsieh commented on SPARK-20356: - [~hvanhovell] I can't reproduce it with your example

[jira] [Assigned] (SPARK-20380) describe table not showing updated table comment after alter operation

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20380: Assignee: (was: Apache Spark) > describe table not showing updated table comment

[jira] [Commented] (SPARK-20380) describe table not showing updated table comment after alter operation

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973856#comment-15973856 ] Apache Spark commented on SPARK-20380: -- User 'sujith71955' has created a pull request for this

[jira] [Assigned] (SPARK-20380) describe table not showing updated table comment after alter operation

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20380: Assignee: Apache Spark > describe table not showing updated table comment after alter

[jira] [Updated] (SPARK-20380) describe table not showing updated table comment after alter operation

2017-04-18 Thread Sujith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith updated SPARK-20380: --- Description: When user alter the table properties and adds/updates table comment. table comment which is

[jira] [Created] (SPARK-20380) describe table not showing updated table comment after alter operation

2017-04-18 Thread Sujith (JIRA)
Sujith created SPARK-20380: -- Summary: describe table not showing updated table comment after alter operation Key: SPARK-20380 URL: https://issues.apache.org/jira/browse/SPARK-20380 Project: Spark

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973793#comment-15973793 ] Liang-Chi Hsieh commented on SPARK-20356: - [~dkbiswal] Thanks for pinging me. I will look into

[jira] [Comment Edited] (SPARK-20364) Parquet predicate pushdown on columns with dots return empty results

2017-04-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973777#comment-15973777 ] Hyukjin Kwon edited comment on SPARK-20364 at 4/19/17 12:15 AM: Ah,

[jira] [Commented] (SPARK-20364) Parquet predicate pushdown on columns with dots return empty results

2017-04-18 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973777#comment-15973777 ] Hyukjin Kwon commented on SPARK-20364: -- Ah, actually, I ran some of tests already with the latter

[jira] [Commented] (SPARK-20364) Parquet predicate pushdown on columns with dots return empty results

2017-04-18 Thread Andrew Ash (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973763#comment-15973763 ] Andrew Ash commented on SPARK-20364: Thanks for the investigation [~hyukjin.kwon]! The proposal you

[jira] [Resolved] (SPARK-20377) Fix JavaStructuredSessionization example

2017-04-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-20377. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17676

[jira] [Assigned] (SPARK-20379) Allow setting SSL-related passwords through env variables

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20379: Assignee: Apache Spark > Allow setting SSL-related passwords through env variables >

[jira] [Commented] (SPARK-20379) Allow setting SSL-related passwords through env variables

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973633#comment-15973633 ] Apache Spark commented on SPARK-20379: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20379) Allow setting SSL-related passwords through env variables

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20379: Assignee: (was: Apache Spark) > Allow setting SSL-related passwords through env

[jira] [Created] (SPARK-20379) Allow setting SSL-related passwords through env variables

2017-04-18 Thread Marcelo Vanzin (JIRA)
Marcelo Vanzin created SPARK-20379: -- Summary: Allow setting SSL-related passwords through env variables Key: SPARK-20379 URL: https://issues.apache.org/jira/browse/SPARK-20379 Project: Spark

[jira] [Assigned] (SPARK-20377) Fix JavaStructuredSessionization example

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20377: Assignee: Apache Spark (was: Tathagata Das) > Fix JavaStructuredSessionization example >

[jira] [Commented] (SPARK-20377) Fix JavaStructuredSessionization example

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973594#comment-15973594 ] Apache Spark commented on SPARK-20377: -- User 'tdas' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20377) Fix JavaStructuredSessionization example

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20377: Assignee: Tathagata Das (was: Apache Spark) > Fix JavaStructuredSessionization example >

[jira] [Created] (SPARK-20378) StreamSinkProvider should provide schema in createSink.

2017-04-18 Thread Yogesh Mahajan (JIRA)
Yogesh Mahajan created SPARK-20378: -- Summary: StreamSinkProvider should provide schema in createSink. Key: SPARK-20378 URL: https://issues.apache.org/jira/browse/SPARK-20378 Project: Spark

[jira] [Updated] (SPARK-20378) StreamSinkProvider should provide schema in createSink.

2017-04-18 Thread Yogesh Mahajan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yogesh Mahajan updated SPARK-20378: --- Description: We have our own Sink implementation based on our in memory store and this sink

[jira] [Updated] (SPARK-20377) Fix JavaStructuredSessionization example

2017-04-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-20377: -- Description: Extra accessors in java bean class causes incorrect encoder generation. This led

[jira] [Created] (SPARK-20377) Fix JavaStructuredSessionization example

2017-04-18 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-20377: - Summary: Fix JavaStructuredSessionization example Key: SPARK-20377 URL: https://issues.apache.org/jira/browse/SPARK-20377 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-20376) Make StateStoreProvider plugable

2017-04-18 Thread Yogesh Mahajan (JIRA)
Yogesh Mahajan created SPARK-20376: -- Summary: Make StateStoreProvider plugable Key: SPARK-20376 URL: https://issues.apache.org/jira/browse/SPARK-20376 Project: Spark Issue Type: Improvement

[jira] [Closed] (SPARK-20374) Encoder generated using Java beans causes corruption in MapGroupsWithState

2017-04-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das closed SPARK-20374. - Resolution: Duplicate > Encoder generated using Java beans causes corruption in

[jira] [Updated] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-04-18 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-12297: - Description: Spark copied Hive's behavior for parquet, but this was inconsistent with other

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973554#comment-15973554 ] Cody Koeninger commented on SPARK-20036: [~danielnuriyev] what actually happened when you removed

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Daniel Nuriyev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973544#comment-15973544 ] Daniel Nuriyev commented on SPARK-20036: I placed the explicit dependencies in my pom because the

[jira] [Commented] (SPARK-20153) Support Multiple aws credentials in order to access multiple Hive on S3 table in spark application

2017-04-18 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973541#comment-15973541 ] Steve Loughran commented on SPARK-20153: [~tafra...@gmail.com] : thanks for discovering that. I

[jira] [Assigned] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20036: Assignee: Apache Spark > impossible to read a whole kafka topic using kafka 0.10 and

[jira] [Assigned] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20036: Assignee: (was: Apache Spark) > impossible to read a whole kafka topic using kafka

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973538#comment-15973538 ] Apache Spark commented on SPARK-20036: -- User 'koeninger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20375) R wrappers for array and map

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20375: Assignee: Apache Spark > R wrappers for array and map > > >

[jira] [Assigned] (SPARK-20375) R wrappers for array and map

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20375: Assignee: (was: Apache Spark) > R wrappers for array and map >

[jira] [Commented] (SPARK-20375) R wrappers for array and map

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973524#comment-15973524 ] Apache Spark commented on SPARK-20375: -- User 'zero323' has created a pull request for this issue:

[jira] [Created] (SPARK-20375) R wrappers for array and map

2017-04-18 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20375: -- Summary: R wrappers for array and map Key: SPARK-20375 URL: https://issues.apache.org/jira/browse/SPARK-20375 Project: Spark Issue Type:

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973507#comment-15973507 ] Cody Koeninger commented on SPARK-20036: I'll submit a PR to add a note to the docs about this.

[jira] [Commented] (SPARK-20036) impossible to read a whole kafka topic using kafka 0.10 and spark 2.0.0

2017-04-18 Thread Ryan Stack (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973497#comment-15973497 ] Ryan Stack commented on SPARK-20036: I had the same issue. Removing the explicit dependencies in my

[jira] [Updated] (SPARK-20374) Encoder generated using Java beans causes corruption in MapGroupsWithState

2017-04-18 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-20374: -- Description: Running the example

[jira] [Commented] (SPARK-20015) Document R Structured Streaming (experimental) in R vignettes and R & SS programming guide

2017-04-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973461#comment-15973461 ] Joseph K. Bradley commented on SPARK-20015: --- [~felixcheung] I see several R issues listed as

[jira] [Updated] (SPARK-20192) SparkR 2.2.0 release note

2017-04-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-20192: -- Issue Type: Documentation (was: Bug) > SparkR 2.2.0 release note >

[jira] [Updated] (SPARK-20208) Document R fpGrowth support in vignettes, programming guide and code example

2017-04-18 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-20208: -- Issue Type: Documentation (was: Bug) > Document R fpGrowth support in vignettes,

[jira] [Comment Edited] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973455#comment-15973455 ] Dilip Biswal edited comment on SPARK-20356 at 4/18/17 8:47 PM: --- [~viirya]

[jira] [Comment Edited] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973455#comment-15973455 ] Dilip Biswal edited comment on SPARK-20356 at 4/18/17 8:46 PM: --- [~viirya]

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Dilip Biswal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973455#comment-15973455 ] Dilip Biswal commented on SPARK-20356: -- [~viirya] [~hvanhovell] [~cloud_fan] [~smilegator] I took a

[jira] [Created] (SPARK-20374) Encoder generated using Java beans causes corruption in MapGroupsWithState

2017-04-18 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-20374: - Summary: Encoder generated using Java beans causes corruption in MapGroupsWithState Key: SPARK-20374 URL: https://issues.apache.org/jira/browse/SPARK-20374

[jira] [Created] (SPARK-20373) Batch queries with 'Dataset/DataFrame.withWatermark()` does not execute

2017-04-18 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-20373: - Summary: Batch queries with 'Dataset/DataFrame.withWatermark()` does not execute Key: SPARK-20373 URL: https://issues.apache.org/jira/browse/SPARK-20373 Project:

[jira] [Assigned] (SPARK-20360) Create repr functions for interpreters to use

2017-04-18 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-20360: --- Assignee: Kyle Kelley > Create repr functions for interpreters to use >

[jira] [Resolved] (SPARK-20360) Create repr functions for interpreters to use

2017-04-18 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk resolved SPARK-20360. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17662

[jira] [Commented] (SPARK-20341) Support BigIngeger values > 19 precision

2017-04-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973343#comment-15973343 ] Kazuaki Ishizaki commented on SPARK-20341: -- This exception is thrown since {{BigInt}} and

[jira] [Commented] (SPARK-20341) Support BigIngeger values > 19 precision

2017-04-18 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973227#comment-15973227 ] Kazuaki Ishizaki commented on SPARK-20341: -- Master branch also throws an exception. > Support

[jira] [Updated] (SPARK-19968) Use a cached instance of KafkaProducer for writing to kafka via KafkaSink.

2017-04-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19968: - Target Version/s: 2.3.0 (was: 2.2.0) > Use a cached instance of KafkaProducer for writing to

[jira] [Assigned] (SPARK-20372) Word2Vec Continuous Bag Of Words model

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20372: Assignee: (was: Apache Spark) > Word2Vec Continuous Bag Of Words model >

[jira] [Commented] (SPARK-20372) Word2Vec Continuous Bag Of Words model

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973187#comment-15973187 ] Apache Spark commented on SPARK-20372: -- User 'shubhamchopra' has created a pull request for this

[jira] [Assigned] (SPARK-20372) Word2Vec Continuous Bag Of Words model

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20372: Assignee: Apache Spark > Word2Vec Continuous Bag Of Words model >

[jira] [Updated] (SPARK-20371) R wrappers for collect_list and collect_set

2017-04-18 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-20371: --- Affects Version/s: 2.0.0 > R wrappers for collect_list and collect_set >

[jira] [Created] (SPARK-20372) Word2Vec Continuous Bag Of Words model

2017-04-18 Thread Shubham Chopra (JIRA)
Shubham Chopra created SPARK-20372: -- Summary: Word2Vec Continuous Bag Of Words model Key: SPARK-20372 URL: https://issues.apache.org/jira/browse/SPARK-20372 Project: Spark Issue Type:

[jira] [Commented] (SPARK-20371) R wrappers for collect_list and collect_set

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973165#comment-15973165 ] Apache Spark commented on SPARK-20371: -- User 'zero323' has created a pull request for this issue:

[jira] [Updated] (SPARK-20370) create external table on read only location fails

2017-04-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20370: - Component/s: (was: Spark Core) SQL > create external table on read only

[jira] [Assigned] (SPARK-20371) R wrappers for collect_list and collect_set

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20371: Assignee: Apache Spark > R wrappers for collect_list and collect_set >

[jira] [Updated] (SPARK-20367) Spark silently escapes partition column names

2017-04-18 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20367: - Component/s: (was: Spark Core) SQL > Spark silently escapes partition

[jira] [Assigned] (SPARK-20371) R wrappers for collect_list and collect_set

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20371: Assignee: (was: Apache Spark) > R wrappers for collect_list and collect_set >

[jira] [Comment Edited] (SPARK-20263) create empty dataframes in sparkR

2017-04-18 Thread Ott Toomet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973157#comment-15973157 ] Ott Toomet edited comment on SPARK-20263 at 4/18/17 5:51 PM: - for instance,

[jira] [Comment Edited] (SPARK-20263) create empty dataframes in sparkR

2017-04-18 Thread Ott Toomet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973157#comment-15973157 ] Ott Toomet edited comment on SPARK-20263 at 4/18/17 5:52 PM: - for instance,

[jira] [Commented] (SPARK-20263) create empty dataframes in sparkR

2017-04-18 Thread Ott Toomet (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973157#comment-15973157 ] Ott Toomet commented on SPARK-20263: for instance, by rbinding in a loop. I have several pieces of

[jira] [Commented] (SPARK-20356) Spark sql group by returns incorrect results after join + distinct transformations

2017-04-18 Thread Job Evers-Meltzer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15973100#comment-15973100 ] Job Evers-Meltzer commented on SPARK-20356: --- Commit 5ed397baa758c29c54a853d3f8fee0ad44e97c14

[jira] [Resolved] (SPARK-20354) When I request access to the 'http: //ip:port/api/v1/applications' link, return 'sparkUser' is empty in REST API.

2017-04-18 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-20354. Resolution: Fixed Assignee: guoxiaolongzte Fix Version/s: 2.2.0 > When I

[jira] [Created] (SPARK-20371) R wrappers for collect_list and collect_set

2017-04-18 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-20371: -- Summary: R wrappers for collect_list and collect_set Key: SPARK-20371 URL: https://issues.apache.org/jira/browse/SPARK-20371 Project: Spark

[jira] [Commented] (SPARK-15816) SQL server based on Postgres protocol

2017-04-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15972971#comment-15972971 ] Takeshi Yamamuro commented on SPARK-15816: -- I put the prototype a little forward in my repo:

[jira] [Created] (SPARK-20370) create external table on read only location fails

2017-04-18 Thread Gaurav Shah (JIRA)
Gaurav Shah created SPARK-20370: --- Summary: create external table on read only location fails Key: SPARK-20370 URL: https://issues.apache.org/jira/browse/SPARK-20370 Project: Spark Issue Type:

[jira] [Commented] (SPARK-20368) Support Sentry on PySpark workers

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15972914#comment-15972914 ] Apache Spark commented on SPARK-20368: -- User 'kxepal' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20368) Support Sentry on PySpark workers

2017-04-18 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20368: Assignee: (was: Apache Spark) > Support Sentry on PySpark workers >

  1   2   >