[jira] [Commented] (SPARK-32779) Spark/Hive3 interaction potentially causes deadlock

[ https://issues.apache.org/jira/browse/SPARK-32779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17190014#comment-17190014 ] JinxinTang commented on SPARK-32779: Hi [~bersprockets], Seems spark cannot work wi

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176966#comment-17176966 ] JinxinTang commented on SPARK-32500: Kindly ping [~hyukjin.kwon] It's my pleasure : 

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173633#comment-17173633 ] JinxinTang commented on SPARK-32500: I have found the root cause is `org.apache.spar

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170095#comment-17170095 ] JinxinTang commented on SPARK-32500: Thank you for your issue, I have understand and

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:09 PM:

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:08 PM:

[jira] [Comment Edited] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169653#comment-17169653 ] JinxinTang edited comment on SPARK-32515 at 8/2/20, 11:08 PM:

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169653#comment-17169653 ] JinxinTang commented on SPARK-32515: Hi [~tigaiii123],  Could you please try the da

[jira] [Updated] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32515: --- Attachment: image-2020-08-03-07-03-55-716.png > Distinct Function Weird Bug > --

[jira] [Commented] (SPARK-32515) Distinct Function Weird Bug

[ https://issues.apache.org/jira/browse/SPARK-32515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169444#comment-17169444 ] JinxinTang commented on SPARK-32515: Hi [~tigaiii123] , Could you please provide re

[jira] [Comment Edited] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169219#comment-17169219 ] JinxinTang edited comment on SPARK-32500 at 8/1/20, 2:24 AM: -

[jira] [Commented] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169219#comment-17169219 ] JinxinTang commented on SPARK-32500: Hi, [~abhishekd0907], I have test the code in

[jira] [Updated] (SPARK-32500) Query and Batch Id not set for Structured Streaming Jobs in case of ForeachBatch in PySpark

[ https://issues.apache.org/jira/browse/SPARK-32500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32500: --- Attachment: image-2020-08-01-10-21-51-246.png > Query and Batch Id not set for Structured Streaming

[jira] [Commented] (SPARK-32483) spark-shell: error: value topByKey is not a member of org.apache.spark.rdd.RDD[(String, (String, Double))]

[ https://issues.apache.org/jira/browse/SPARK-32483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17167158#comment-17167158 ] JinxinTang commented on SPARK-32483: [~ukiml] Please `import org.apache.spark.mllib

[jira] [Commented] (SPARK-32425) Spark sequence() fails if start and end of range are identical timestamps

[ https://issues.apache.org/jira/browse/SPARK-32425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17165634#comment-17165634 ] JinxinTang commented on SPARK-32425: cc [~viirya]  Because I have fixed it by  https

[jira] [Commented] (SPARK-32347) BROADCAST hint makes a weird message that "column can't be resolved" (it was OK in Spark 2.4)

[ https://issues.apache.org/jira/browse/SPARK-32347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160627#comment-17160627 ] JinxinTang commented on SPARK-32347: cc [~ibobak] Thanks for your issue, I have just

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160370#comment-17160370 ] JinxinTang edited comment on SPARK-32317 at 7/18/20, 8:55 AM:

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160370#comment-17160370 ] JinxinTang edited comment on SPARK-32317 at 7/18/20, 8:54 AM:

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160370#comment-17160370 ] JinxinTang edited comment on SPARK-32317 at 7/18/20, 8:53 AM:

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160370#comment-17160370 ] JinxinTang edited comment on SPARK-32317 at 7/18/20, 8:51 AM:

[jira] [Commented] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160370#comment-17160370 ] JinxinTang commented on SPARK-32317: Please try use `spark.read.*option("mergeSchema

[jira] [Comment Edited] (SPARK-32347) BROADCAST hint makes a weird message that "column can't be resolved" (it was OK in Spark 2.4)

[ https://issues.apache.org/jira/browse/SPARK-32347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160284#comment-17160284 ] JinxinTang edited comment on SPARK-32347 at 7/18/20, 12:22 AM: ---

[jira] [Commented] (SPARK-32347) BROADCAST hint makes a weird message that "column can't be resolved" (it was OK in Spark 2.4)

[ https://issues.apache.org/jira/browse/SPARK-32347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17160284#comment-17160284 ] JinxinTang commented on SPARK-32347: Same the CTE issue as https://issues.apache.org

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159864#comment-17159864 ] JinxinTang edited comment on SPARK-32317 at 7/17/20, 11:12 AM: ---

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159864#comment-17159864 ] JinxinTang edited comment on SPARK-32317 at 7/17/20, 11:11 AM: ---

[jira] [Comment Edited] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159864#comment-17159864 ] JinxinTang edited comment on SPARK-32317 at 7/17/20, 11:07 AM: ---

[jira] [Commented] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159864#comment-17159864 ] JinxinTang commented on SPARK-32317: Yes, maybe we should check the decimal schema a

[jira] [Issue Comment Deleted] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32317: --- Comment: was deleted (was: Seems a little hard to solve, because current spark use `org.apache.parq

[jira] [Commented] (SPARK-32317) Parquet file loading with different schema(Decimal(N, P)) in files is not working as expected

[ https://issues.apache.org/jira/browse/SPARK-32317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159796#comment-17159796 ] JinxinTang commented on SPARK-32317: Seems a little hard to solve, because current s

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153131#comment-17153131 ] JinxinTang edited comment on SPARK-32205 at 7/8/20, 5:10 AM: -

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153131#comment-17153131 ] JinxinTang edited comment on SPARK-32205 at 7/8/20, 12:41 AM:

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153131#comment-17153131 ] JinxinTang edited comment on SPARK-32205 at 7/7/20, 11:31 PM:

[jira] [Commented] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153131#comment-17153131 ] JinxinTang commented on SPARK-32205: [~nileshr.patil] Seems a issue, we can insert t

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152679#comment-17152679 ] JinxinTang edited comment on SPARK-32205 at 7/7/20, 11:44 AM:

[jira] [Commented] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152679#comment-17152679 ] JinxinTang commented on SPARK-32205: Sure, this problem seems in mysql side, because

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/7/20, 9:21 AM: -

[jira] [Comment Edited] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152574#comment-17152574 ] JinxinTang edited comment on SPARK-32205 at 7/7/20, 9:16 AM: -

[jira] [Commented] (SPARK-32206) Enable multi-line true could break the read csv in Azure Data Lake Storage gen2

[ https://issues.apache.org/jira/browse/SPARK-32206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152588#comment-17152588 ] JinxinTang commented on SPARK-32206: [~vicentezqh] Could you please reproduce it in

[jira] [Commented] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152574#comment-17152574 ] JinxinTang commented on SPARK-32205: We can see from the doc in mysql Date type has

[jira] [Issue Comment Deleted] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32205: --- Comment: was deleted (was: Could you please try insert into mysql directly, it seems mysql not supp

[jira] [Commented] (SPARK-32205) Writing timestamp in mysql gets fails

[ https://issues.apache.org/jira/browse/SPARK-32205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152568#comment-17152568 ] JinxinTang commented on SPARK-32205: Could you please try insert into mysql directly

[jira] [Commented] (SPARK-32206) Enable multi-line true could break the read csv in Azure Data Lake Storage gen2

[ https://issues.apache.org/jira/browse/SPARK-32206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17152567#comment-17152567 ] JinxinTang commented on SPARK-32206: Please use `spark.read.format("xxx").load("fil

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:47 AM:

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:43 AM:

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:43 AM:

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:42 AM:

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:41 AM:

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:40 AM:

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:40 AM:

[jira] [Comment Edited] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151682#comment-17151682 ] JinxinTang edited comment on SPARK-32176 at 7/6/20, 12:40 AM:

[jira] [Commented] (SPARK-32176) Automatic type promotion to ArrayType in defined schema in from_json is broken

[ https://issues.apache.org/jira/browse/SPARK-32176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151682#comment-17151682 ] JinxinTang commented on SPARK-32176: cc [~abhi92544] Seems can not reproduce： [https

[jira] [Created] (SPARK-32156) SPARK-31061 has two very similar tests could merge and somewhere could be improved

JinxinTang created SPARK-32156: -- Summary: SPARK-31061 has two very similar tests could merge and somewhere could be improved Key: SPARK-32156 URL: https://issues.apache.org/jira/browse/SPARK-32156 Projec

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148501#comment-17148501 ] JinxinTang commented on SPARK-32130: [~gourav.sengupta] Nice notebook, is seems the

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 8:07 AM:

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 8:05 AM:

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 8:05 AM:

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 7:59 AM:

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 7:58 AM:

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang edited comment on SPARK-32130 at 6/30/20, 7:39 AM:

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17148393#comment-17148393 ] JinxinTang commented on SPARK-32130: [~lotus2you] Thank you for your feedback. Lots

[jira] [Updated] (SPARK-32133) Forbid time field steps for date start/end in Sequence

[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32133: --- Description: *Sequence time field steps for date start/end looks strange in spark as follows:* sca

[jira] [Updated] (SPARK-32133) Forbid time field steps for date start/end in Sequence

[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32133: --- Summary: Forbid time field steps for date start/end in Sequence (was: Forbid time field steps for d

[jira] [Created] (SPARK-32133) Forbid time field steps for date start/end

JinxinTang created SPARK-32133: -- Summary: Forbid time field steps for date start/end Key: SPARK-32133 URL: https://issues.apache.org/jira/browse/SPARK-32133 Project: Spark Issue Type: Improvemen

[jira] [Commented] (SPARK-32068) Spark 3 UI task launch time show in error time zone

[ https://issues.apache.org/jira/browse/SPARK-32068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17143619#comment-17143619 ] JinxinTang commented on SPARK-32068: Thank you for the issue, hope my PR can help.

[jira] [Commented] (SPARK-31751) spark serde property path overwrites table property location

[ https://issues.apache.org/jira/browse/SPARK-31751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17141320#comment-17141320 ] JinxinTang commented on SPARK-31751: Thank you for your issue, In order to compatibl

[jira] [Comment Edited] (SPARK-31980) Spark sequence() fails if start and end of range are identical dates

[ https://issues.apache.org/jira/browse/SPARK-31980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136248#comment-17136248 ] JinxinTang edited comment on SPARK-31980 at 6/16/20, 3:01 AM:

[jira] [Commented] (SPARK-31980) Spark sequence() fails if start and end of range are identical dates

[ https://issues.apache.org/jira/browse/SPARK-31980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17136248#comment-17136248 ] JinxinTang commented on SPARK-31980: [~DaveDeCaprio] Maybe one of this PRs could reb

[jira] [Commented] (SPARK-31968) write.partitionBy() creates duplicate subdirectories when user provides duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-31968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17135342#comment-17135342 ] JinxinTang commented on SPARK-31968: [~qxzzxq] Thank you for this issue, it is my pl

[jira] [Commented] (SPARK-31968) write.partitionBy() creates duplicate subdirectories when user provides duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-31968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17135340#comment-17135340 ] JinxinTang commented on SPARK-31968: Hi, [~dongjoon], [~hyukjin.kwon], I am  TJX2014

[jira] [Updated] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31797: --- Description: 1.Add and register three new functions: {{TIMESTAMP_SECONDS}},  {{TIMESTAMP_MILLIS}} an

[jira] [Updated] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31797: --- Description: (was: ### What changes were proposed in this pull request? Add and register three ne

[jira] [Created] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

JinxinTang created SPARK-31797: -- Summary: Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions Key: SPARK-31797 URL: https://issues.apache.org/jira/browse/SPARK-31797 Project: Spark

[jira] [Comment Edited] (SPARK-31598) LegacySimpleTimestampFormatter incorrectly interprets pre-Gregorian timestamps

[ https://issues.apache.org/jira/browse/SPARK-31598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17097202#comment-17097202 ] JinxinTang edited comment on SPARK-31598 at 5/1/20, 7:00 AM: -

[jira] [Commented] (SPARK-31598) LegacySimpleTimestampFormatter incorrectly interprets pre-Gregorian timestamps

[ https://issues.apache.org/jira/browse/SPARK-31598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17097202#comment-17097202 ] JinxinTang commented on SPARK-31598: already fix: [#anchor]https://issues.apache.org

[jira] [Issue Comment Deleted] (SPARK-31550) nondeterministic configurations with general meanings in sql configuration doc

[ https://issues.apache.org/jira/browse/SPARK-31550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31550: --- Comment: was deleted (was: try to specify conf in spark-defaults.conf spark.sql.warehouse.dir /tmp

[jira] [Issue Comment Deleted] (SPARK-31532) SparkSessionBuilder shoud not propagate static sql configurations to the existing active/default SparkSession

[ https://issues.apache.org/jira/browse/SPARK-31532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31532: --- Comment: was deleted (was: Thanks for your issue, these followings may not be allowed to modify aft

[jira] [Commented] (SPARK-31550) nondeterministic configurations with general meanings in sql configuration doc

[ https://issues.apache.org/jira/browse/SPARK-31550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091218#comment-17091218 ] JinxinTang commented on SPARK-31550: try to specify conf in spark-defaults.conf spa

[jira] [Comment Edited] (SPARK-31532) SparkSessionBuilder shoud not propagate static sql configurations to the existing active/default SparkSession

[ https://issues.apache.org/jira/browse/SPARK-31532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091067#comment-17091067 ] JinxinTang edited comment on SPARK-31532 at 4/24/20, 1:08 AM:

[jira] [Commented] (SPARK-31532) SparkSessionBuilder shoud not propagate static sql configurations to the existing active/default SparkSession

[ https://issues.apache.org/jira/browse/SPARK-31532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091067#comment-17091067 ] JinxinTang commented on SPARK-31532: Thanks for your issue, these followings not be

[jira] [Comment Edited] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089596#comment-17089596 ] JinxinTang edited comment on SPARK-31508 at 4/22/20, 12:00 PM: ---

[jira] [Issue Comment Deleted] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31508: --- Comment: was deleted (was: such '2...' is far larger than long, I think we should not covert to

[jira] [Comment Edited] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089598#comment-17089598 ] JinxinTang edited comment on SPARK-31508 at 4/22/20, 11:49 AM: ---

[jira] [Commented] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089598#comment-17089598 ] JinxinTang commented on SPARK-31508: such '2...' is far larger than big, I think

[jira] [Commented] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089596#comment-17089596 ] JinxinTang commented on SPARK-31508: for example： select * from default.test1 where

[jira] [Commented] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089583#comment-17089583 ] JinxinTang commented on SPARK-31508: You are right,We can see from code gen in branc

[jira] [Issue Comment Deleted] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31508: --- Comment: was deleted (was: You are right,We can see from code gen in branch 2.4 which will convet u

[jira] [Comment Edited] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089449#comment-17089449 ] JinxinTang edited comment on SPARK-31508 at 4/22/20, 8:33 AM:

[jira] [Commented] (SPARK-31508) string type compare with numberic cause data inaccurate

[ https://issues.apache.org/jira/browse/SPARK-31508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089449#comment-17089449 ] JinxinTang commented on SPARK-31508: select id from test1 where id>'0';  may be see

[jira] [Commented] (SPARK-31515) Canonicalize Cast should consider the value of needTimeZone

[ https://issues.apache.org/jira/browse/SPARK-31515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089372#comment-17089372 ] JinxinTang commented on SPARK-31515: Could you please provide code piece or picture

[jira] [Commented] (SPARK-31400) The catalogString doesn't distinguish Vectors in ml and mllib

[ https://issues.apache.org/jira/browse/SPARK-31400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089317#comment-17089317 ] JinxinTang commented on SPARK-31400: Thanks for report this issue, this is in progre

[jira] [Issue Comment Deleted] (SPARK-31256) Dropna doesn't work for struct columns

[ https://issues.apache.org/jira/browse/SPARK-31256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31256: --- Comment: was deleted (was: This feature may not be supported since spark 2.4.5，due to former bug is

[jira] [Commented] (SPARK-31256) Dropna doesn't work for struct columns

[ https://issues.apache.org/jira/browse/SPARK-31256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17087306#comment-17087306 ] JinxinTang commented on SPARK-31256: This feature may not be supported since spark 2

[jira] [Commented] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17085455#comment-17085455 ] JinxinTang commented on SPARK-31386: hi [~hyukjin.kwon] , I test pyspark from my lo

[jira] [Updated] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31386: --- Attachment: 选区_001.png > Reading broadcast in UDF raises MemoryError when > spark.executor.pyspark.

[jira] [Issue Comment Deleted] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31386: --- Comment: was deleted (was: hi [~hyukjin.kwon] , I test pyspark from my local spark cluster with the

[jira] [Comment Edited] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17085451#comment-17085451 ] JinxinTang edited comment on SPARK-31386 at 4/17/20, 5:45 AM:

[jira] [Commented] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17085451#comment-17085451 ] JinxinTang commented on SPARK-31386: hi [~hyukjin.kwon] , I test pyspark from my lo

[jira] [Updated] (SPARK-31386) Reading broadcast in UDF raises MemoryError when spark.executor.pyspark.memory is set

[ https://issues.apache.org/jira/browse/SPARK-31386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31386: --- Attachment: 选区_001.png > Reading broadcast in UDF raises MemoryError when > spark.executor.pyspark.

[jira] [Issue Comment Deleted] (SPARK-31256) Dropna doesn't work for struct columns

[ https://issues.apache.org/jira/browse/SPARK-31256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-31256: --- Comment: was deleted (was: I will try to solve it) > Dropna doesn't work for struct columns > -

[jira] [Commented] (SPARK-31256) Dropna doesn't work for struct columns

[ https://issues.apache.org/jira/browse/SPARK-31256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17081084#comment-17081084 ] JinxinTang commented on SPARK-31256: I will try to solve it > Dropna doesn't work f

  1   2   >