[jira] [Commented] (SPARK-24549) 32BitDecimalType and 64BitDecimalType support push down to the data sources

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511049#comment-16511049 ] Apache Spark commented on SPARK-24549: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-24549) 32BitDecimalType and 64BitDecimalType support push down to the data sources

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24549: Assignee: Apache Spark > 32BitDecimalType and 64BitDecimalType support push down to the d

[jira] [Created] (SPARK-24549) 32BitDecimalType and 64BitDecimalType support push down to the data sources

2018-06-13 Thread Yuming Wang (JIRA)
Yuming Wang created SPARK-24549: --- Summary: 32BitDecimalType and 64BitDecimalType support push down to the data sources Key: SPARK-24549 URL: https://issues.apache.org/jira/browse/SPARK-24549 Project: Sp

[jira] [Commented] (SPARK-24549) 32BitDecimalType and 64BitDecimalType support push down to the data sources

2018-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511034#comment-16511034 ] Yuming Wang commented on SPARK-24549: - I'm working on this > 32BitDecimalType and 6

[jira] [Updated] (SPARK-24538) ByteArrayDecimalType support push down to the data sources

2018-06-13 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24538: Summary: ByteArrayDecimalType support push down to the data sources (was: Decimal type support pu

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

2018-06-13 Thread Sina Madani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511033#comment-16511033 ] Sina Madani commented on SPARK-650: --- I too have this problem. It seems that Apache Flink

[jira] [Reopened] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco reopened SPARK-24545: - > Function hour not working as expected for hour 2 in PySpark >

[jira] [Commented] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511003#comment-16511003 ] Eric Blanco commented on SPARK-24545: - !image-2018-06-13-13-52-06-165.png! In Scala

[jira] [Updated] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco updated SPARK-24545: Attachment: image-2018-06-13-13-53-21-185.png > Function hour not working as expected for hour 2 i

[jira] [Updated] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco updated SPARK-24545: Attachment: image-2018-06-13-13-52-06-165.png > Function hour not working as expected for hour 2 i

[jira] [Updated] (SPARK-24548) JavaPairRDD to Dataset in SPARK generates ambiguous results

2018-06-13 Thread Jackson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackson updated SPARK-24548: Component/s: Java API Summary: JavaPairRDD to Dataset in SPARK generates ambiguous results (was:

[jira] [Resolved] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco resolved SPARK-24545. - Resolution: Not A Bug > Function hour not working as expected for hour 2 in PySpark > --

[jira] [Commented] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510999#comment-16510999 ] Eric Blanco commented on SPARK-24545: - Ok apparenlty this is due to a change of hour

[jira] [Updated] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco updated SPARK-24545: Description: Hello, I tried to get the hour out of a date and it works except if the hour is 2. I

[jira] [Updated] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco updated SPARK-24545: Description: Hello, I tried to get the hour out of a date and it works except if the hour is 2. I

[jira] [Updated] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco updated SPARK-24545: Description: Hello, I tried to get the hour out of a date and it works except if the hour is 2. I

[jira] [Created] (SPARK-24548) JavaPairRDD to Dataset in SPARK returns error

2018-06-13 Thread Jackson (JIRA)
Jackson created SPARK-24548: --- Summary: JavaPairRDD to Dataset in SPARK returns error Key: SPARK-24548 URL: https://issues.apache.org/jira/browse/SPARK-24548 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-24547) Spark on K8s docker-image-tool.sh improvements

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24547: Assignee: (was: Apache Spark) > Spark on K8s docker-image-tool.sh improvements >

[jira] [Commented] (SPARK-24547) Spark on K8s docker-image-tool.sh improvements

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510971#comment-16510971 ] Apache Spark commented on SPARK-24547: -- User 'rayburgemeestre' has created a pull r

[jira] [Assigned] (SPARK-24547) Spark on K8s docker-image-tool.sh improvements

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24547: Assignee: Apache Spark > Spark on K8s docker-image-tool.sh improvements > ---

[jira] [Updated] (SPARK-24547) Spark on K8s docker-image-tool.sh improvements

2018-06-13 Thread Ray Burgemeestre (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Burgemeestre updated SPARK-24547: - Description: *Context* PySpark support for Spark on k8s was merged with [https://githu

[jira] [Updated] (SPARK-24547) Spark on K8s docker-image-tool.sh improvements

2018-06-13 Thread Ray Burgemeestre (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Burgemeestre updated SPARK-24547: - Summary: Spark on K8s docker-image-tool.sh improvements (was: Spark on K8s ) > Spark o

[jira] [Created] (SPARK-24547) Spark on K8s

2018-06-13 Thread Ray Burgemeestre (JIRA)
Ray Burgemeestre created SPARK-24547: Summary: Spark on K8s Key: SPARK-24547 URL: https://issues.apache.org/jira/browse/SPARK-24547 Project: Spark Issue Type: Improvement Compo

[jira] [Assigned] (SPARK-24546) InsertIntoDataSourceCommand make dataframe with wrong schema

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24546: Assignee: Apache Spark > InsertIntoDataSourceCommand make dataframe with wrong schema > -

[jira] [Commented] (SPARK-24546) InsertIntoDataSourceCommand make dataframe with wrong schema

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510957#comment-16510957 ] Apache Spark commented on SPARK-24546: -- User 'zheh12' has created a pull request fo

[jira] [Assigned] (SPARK-24546) InsertIntoDataSourceCommand make dataframe with wrong schema

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24546: Assignee: (was: Apache Spark) > InsertIntoDataSourceCommand make dataframe with wrong

[jira] [Commented] (SPARK-24215) Implement __repr__ and _repr_html_ for dataframes in PySpark

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510946#comment-16510946 ] Apache Spark commented on SPARK-24215: -- User 'xuanyuanking' has created a pull requ

[jira] [Updated] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco updated SPARK-24545: Description: Hello, I tried to get the hour out of a date and it works except if the hour is 2. I

[jira] [Updated] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco updated SPARK-24545: Description: Hello, I tried to get the hour out of a date and it works except if the hour is 2. I

[jira] [Updated] (SPARK-24546) InsertIntoDataSourceCommand make dataframe with wrong schema

2018-06-13 Thread yangz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangz updated SPARK-24546: -- Description: I have a hdfs table with schema  {code:java} hdfs_table(a int, b int, c int){code}   and a kudu

[jira] [Updated] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco updated SPARK-24545: Priority: Minor (was: Major) > Function hour not working as expected for hour 2 in PySpark >

[jira] [Updated] (SPARK-24545) Function hour not working as expected for hour 2 in PySpark

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco updated SPARK-24545: Description: Hello, I tried to get the hour out of a date and it works except if the hour is 2. I

[jira] [Created] (SPARK-24546) InsertIntoDataSourceCommand make dataframe with wrong schema

2018-06-13 Thread yangz (JIRA)
yangz created SPARK-24546: - Summary: InsertIntoDataSourceCommand make dataframe with wrong schema Key: SPARK-24546 URL: https://issues.apache.org/jira/browse/SPARK-24546 Project: Spark Issue Type: B

[jira] [Created] (SPARK-24545) Function hour not working as expecte for hour 2

2018-06-13 Thread Eric Blanco (JIRA)
Eric Blanco created SPARK-24545: --- Summary: Function hour not working as expecte for hour 2 Key: SPARK-24545 URL: https://issues.apache.org/jira/browse/SPARK-24545 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-24545) Function hour not working as expected for hour 2

2018-06-13 Thread Eric Blanco (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Blanco updated SPARK-24545: Summary: Function hour not working as expected for hour 2 (was: Function hour not working as expe

[jira] [Assigned] (SPARK-24544) Print actual failure cause when look up function failed

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24544: Assignee: (was: Apache Spark) > Print actual failure cause when look up function fail

[jira] [Commented] (SPARK-24544) Print actual failure cause when look up function failed

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510933#comment-16510933 ] Apache Spark commented on SPARK-24544: -- User 'caneGuy' has created a pull request f

[jira] [Assigned] (SPARK-24544) Print actual failure cause when look up function failed

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24544: Assignee: Apache Spark > Print actual failure cause when look up function failed > --

[jira] [Created] (SPARK-24544) Print actual failure cause when look up function failed

2018-06-13 Thread zhoukang (JIRA)
zhoukang created SPARK-24544: Summary: Print actual failure cause when look up function failed Key: SPARK-24544 URL: https://issues.apache.org/jira/browse/SPARK-24544 Project: Spark Issue Type: I

[jira] [Commented] (SPARK-23486) LookupFunctions should not check the same function name more than once

2018-06-13 Thread LANDAIS Christophe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510917#comment-16510917 ] LANDAIS Christophe commented on SPARK-23486: Hi,   What is the process to

[jira] [Commented] (SPARK-24514) Exception while converting RDD to DataFrame

2018-06-13 Thread SHAILENDRA SHAHANE (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510875#comment-16510875 ] SHAILENDRA SHAHANE commented on SPARK-24514: Attached Source code and Except

[jira] [Updated] (SPARK-24514) Exception while converting RDD to DataFrame

2018-06-13 Thread SHAILENDRA SHAHANE (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHAILENDRA SHAHANE updated SPARK-24514: --- Attachment: SparkMongoException.txt > Exception while converting RDD to DataFrame >

[jira] [Updated] (SPARK-24514) Exception while converting RDD to DataFrame

2018-06-13 Thread SHAILENDRA SHAHANE (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SHAILENDRA SHAHANE updated SPARK-24514: --- Attachment: MongoSparkTest.java > Exception while converting RDD to DataFrame >

[jira] [Commented] (SPARK-24543) Support any DataType as DDL string for from_json's schema

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510769#comment-16510769 ] Apache Spark commented on SPARK-24543: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-24543) Support any DataType as DDL string for from_json's schema

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24543: Assignee: (was: Apache Spark) > Support any DataType as DDL string for from_json's sc

[jira] [Assigned] (SPARK-24543) Support any DataType as DDL string for from_json's schema

2018-06-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24543: Assignee: Apache Spark > Support any DataType as DDL string for from_json's schema >

[jira] [Created] (SPARK-24543) Support any DataType as DDL string for from_json's schema

2018-06-13 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-24543: -- Summary: Support any DataType as DDL string for from_json's schema Key: SPARK-24543 URL: https://issues.apache.org/jira/browse/SPARK-24543 Project: Spark Issue T

[jira] [Commented] (SPARK-24543) Support any DataType as DDL string for from_json's schema

2018-06-13 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510753#comment-16510753 ] Maxim Gekk commented on SPARK-24543: I am working on the feature at the moment. > S

<    1   2