[
https://issues.apache.org/jira/browse/SPARK-25801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-25801.
--
Resolution: Fixed
Fix Version/s: 2.4.0
> pandas_udf grouped_map fails with input
[
https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-22809:
-
Fix Version/s: 2.3.2
> pyspark is sensitive to imports with dots
>
[
https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16661502#comment-16661502
]
Bryan Cutler commented on SPARK-22809:
--
Sure, I probably shouldn't have tested out of the branches.
[
https://issues.apache.org/jira/browse/SPARK-25079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675866#comment-16675866
]
Bryan Cutler edited comment on SPARK-25079 at 11/5/18 11:09 PM:
Sounds
[
https://issues.apache.org/jira/browse/SPARK-25079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675866#comment-16675866
]
Bryan Cutler commented on SPARK-25079:
--
Sounds like a good plan [~shaneknapp]! The instances of
[
https://issues.apache.org/jira/browse/SPARK-25079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675866#comment-16675866
]
Bryan Cutler edited comment on SPARK-25079 at 11/5/18 11:08 PM:
Sounds
[
https://issues.apache.org/jira/browse/SPARK-25344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685613#comment-16685613
]
Bryan Cutler commented on SPARK-25344:
--
[~hyukjin.kwon] no problem, I can take on ML and MLlib
>
[
https://issues.apache.org/jira/browse/SPARK-25344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643762#comment-16643762
]
Bryan Cutler commented on SPARK-25344:
--
No I don't have strong feelings, my only preference was to
[
https://issues.apache.org/jira/browse/SPARK-25461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16635762#comment-16635762
]
Bryan Cutler commented on SPARK-25461:
--
Thanks for looking into this [~viirya]! You are right that
[
https://issues.apache.org/jira/browse/SPARK-25461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637626#comment-16637626
]
Bryan Cutler commented on SPARK-25461:
--
I file ARROW-3428, which deals with the incorrect cast from
[
https://issues.apache.org/jira/browse/SPARK-25461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16637626#comment-16637626
]
Bryan Cutler edited comment on SPARK-25461 at 10/3/18 11:53 PM:
I filed
[
https://issues.apache.org/jira/browse/SPARK-25461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642379#comment-16642379
]
Bryan Cutler commented on SPARK-25461:
--
Just wanted to add that the resolution here added a note
[
https://issues.apache.org/jira/browse/SPARK-25471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-25471:
Assignee: (was: Bryan Cutler)
> Fix tests for Python 3.6 with Pandas 0.23+
>
Bryan Cutler created SPARK-25471:
Summary: Fix tests for Python 3.6 with Pandas 0.23+
Key: SPARK-25471
URL: https://issues.apache.org/jira/browse/SPARK-25471
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-25432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626148#comment-16626148
]
Bryan Cutler commented on SPARK-25432:
--
moved description :)
> Consider if using standard
[
https://issues.apache.org/jira/browse/SPARK-25432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-25432:
-
Description: As we saw in
[https://github.com/apache/spark/pull/22295/files] the logic can get
[
https://issues.apache.org/jira/browse/SPARK-25432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-25432:
-
Environment: (was: As we saw in
[https://github.com/apache/spark/pull/22295/files] the
[
https://issues.apache.org/jira/browse/SPARK-25351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629572#comment-16629572
]
Bryan Cutler commented on SPARK-25351:
--
Hi [~pgadige], yes please go ahead with this issue! When
[
https://issues.apache.org/jira/browse/SPARK-26591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744288#comment-16744288
]
Bryan Cutler commented on SPARK-26591:
--
[~elch10] please go ahead and make a Jira for Arrow
[
https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-26566:
-
Description:
_This is just a placeholder for now to collect what needs to be fixed when we
[
https://issues.apache.org/jira/browse/SPARK-26591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742517#comment-16742517
]
Bryan Cutler commented on SPARK-26591:
--
I created the same virtual environment and could not
[
https://issues.apache.org/jira/browse/SPARK-26591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743460#comment-16743460
]
Bryan Cutler commented on SPARK-26591:
--
[~elch10] this seems like it is more an Arrow issue with
[
https://issues.apache.org/jira/browse/SPARK-26591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16743463#comment-16743463
]
Bryan Cutler commented on SPARK-26591:
--
And yes, you could build pyarrow yourself, but it shouldn't
[
https://issues.apache.org/jira/browse/SPARK-26676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-26676.
--
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 23604
[
https://issues.apache.org/jira/browse/SPARK-26676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-26676:
Assignee: Hyukjin Kwon
> Make HiveContextSQLTests.test_unbounded_frames test compatible
[
https://issues.apache.org/jira/browse/SPARK-26315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16717873#comment-16717873
]
Bryan Cutler commented on SPARK-26315:
--
I believe {{def approxSimilarityJoin(...)}} in LSHModelf in
[
https://issues.apache.org/jira/browse/SPARK-26200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704092#comment-16704092
]
Bryan Cutler edited comment on SPARK-26200 at 11/30/18 12:56 AM:
-
I
[
https://issues.apache.org/jira/browse/SPARK-26200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704092#comment-16704092
]
Bryan Cutler commented on SPARK-26200:
--
I think this is a duplicate of
[
https://issues.apache.org/jira/browse/SPARK-26200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-26200.
--
Resolution: Duplicate
> Column values are incorrectly transposed when a field in a PySpark
[
https://issues.apache.org/jira/browse/SPARK-24333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-24333:
Assignee: Huaxin Gao
> Add fit with validation set to spark.ml GBT: Python API
>
[
https://issues.apache.org/jira/browse/SPARK-24333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-24333.
--
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 21465
[
https://issues.apache.org/jira/browse/SPARK-25274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-25274:
Assignee: Bryan Cutler
> Improve toPandas with Arrow by sending out-of-order record
[
https://issues.apache.org/jira/browse/SPARK-25274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-25274.
--
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 22275
Bryan Cutler created SPARK-26573:
Summary: Python worker not reused with mapPartitions if not
consuming iterator
Key: SPARK-26573
URL: https://issues.apache.org/jira/browse/SPARK-26573
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-26349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-26349:
Assignee: Imran Rashid
> Pyspark should not accept insecure p4yj gateways
>
[
https://issues.apache.org/jira/browse/SPARK-26349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-26349.
--
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 23441
[
https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-26566:
-
Target Version/s: (was: 2.4.0)
> Upgrade apache/arrow to 0.12.0
>
Bryan Cutler created SPARK-26566:
Summary: Upgrade apache/arrow to 0.12.0
Key: SPARK-26566
URL: https://issues.apache.org/jira/browse/SPARK-26566
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16736540#comment-16736540
]
Bryan Cutler commented on SPARK-26566:
--
Version 0.12.0 is slated to be released in mid January
>
[
https://issues.apache.org/jira/browse/SPARK-25272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-25272.
--
Resolution: Won't Fix
> Show some kind of test output to indicate pyarrow tests were run
>
[
https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-26566:
-
Fix Version/s: (was: 2.4.0)
> Upgrade apache/arrow to 0.12.0
>
[
https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-26566:
-
Affects Version/s: (was: 2.3.0)
2.4.0
> Upgrade apache/arrow to
[
https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-26566:
Assignee: (was: Bryan Cutler)
> Upgrade apache/arrow to 0.12.0
>
[
https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-26566:
-
Description:
_This is just a placeholder for now to collect what needs to be fixed when we
[
https://issues.apache.org/jira/browse/SPARK-26591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740571#comment-16740571
]
Bryan Cutler commented on SPARK-26591:
--
Could you share some details of your pyarrow installation -
[
https://issues.apache.org/jira/browse/SPARK-25344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16614176#comment-16614176
]
Bryan Cutler commented on SPARK-25344:
--
>From the mailing list I think we should agree on a few
[
https://issues.apache.org/jira/browse/SPARK-26200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16705224#comment-16705224
]
Bryan Cutler commented on SPARK-26200:
--
Thanks [~davidlyness], I'll mark this as a duplicate since
[
https://issues.apache.org/jira/browse/SPARK-26412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751785#comment-16751785
]
Bryan Cutler commented on SPARK-26412:
--
[~mengxr] I think Arrow record batches would be a much more
[
https://issues.apache.org/jira/browse/SPARK-26410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751771#comment-16751771
]
Bryan Cutler commented on SPARK-26410:
--
This could be useful to have, but it does seem a little
[
https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16751802#comment-16751802
]
Bryan Cutler commented on SPARK-24579:
--
It would be great to start up this discussion again, I saw
[
https://issues.apache.org/jira/browse/SPARK-27276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801250#comment-16801250
]
Bryan Cutler commented on SPARK-27276:
--
[~shaneknapp] this will need an upgrade on Jenkins, so let
Bryan Cutler created SPARK-27276:
Summary: Increase the minimum pyarrow version to 0.12.0
Key: SPARK-27276
URL: https://issues.apache.org/jira/browse/SPARK-27276
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-27276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-27276:
-
Description:
The current minimum version is 0.8.0, which is pretty ancient since Arrow has
[
https://issues.apache.org/jira/browse/SPARK-27276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-27276:
-
Summary: Increase the minimum pyarrow version to 0.12.1 (was: Increase the
minimum pyarrow
[
https://issues.apache.org/jira/browse/SPARK-27276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810024#comment-16810024
]
Bryan Cutler commented on SPARK-27276:
--
I think we should use 0.12.1, there was a bug fix
[
https://issues.apache.org/jira/browse/SPARK-27353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810009#comment-16810009
]
Bryan Cutler commented on SPARK-27353:
--
Works for me out of master, can you provide a script to
Bryan Cutler created SPARK-27387:
Summary: Replace sqlutils assertPandasEqual with Pandas
assert_frame_equal in tests
Key: SPARK-27387
URL: https://issues.apache.org/jira/browse/SPARK-27387
Project:
[
https://issues.apache.org/jira/browse/SPARK-27387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810197#comment-16810197
]
Bryan Cutler commented on SPARK-27387:
--
This can be done after the upgrade of pyarrow version to
[
https://issues.apache.org/jira/browse/SPARK-27387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16810200#comment-16810200
]
Bryan Cutler commented on SPARK-27387:
--
I can work on this
> Replace sqlutils assertPandasEqual
[
https://issues.apache.org/jira/browse/SPARK-27389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16811355#comment-16811355
]
Bryan Cutler commented on SPARK-27389:
--
>From the stacktrace, it looks like it's getting this from
[
https://issues.apache.org/jira/browse/SPARK-27293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-27293:
-
Summary: Setting random seed produces different results in
RandomForestRegressor (was: I am
[
https://issues.apache.org/jira/browse/SPARK-27293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-27293:
-
Description:
I am interested in finding out if there is a bug in the implementation of
[
https://issues.apache.org/jira/browse/SPARK-27293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-27293:
-
Component/s: ML
> Setting random seed produces different results in RandomForestRegressor
>
[
https://issues.apache.org/jira/browse/SPARK-27293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16804124#comment-16804124
]
Bryan Cutler commented on SPARK-27293:
--
Setting the seed like in your example for randomSplit and
[
https://issues.apache.org/jira/browse/SPARK-27240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-27240.
--
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24177
[
https://issues.apache.org/jira/browse/SPARK-27240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-27240:
Assignee: Takuya Ueshin
> Use pandas DataFrame for struct type argument in Scalar Pandas
[
https://issues.apache.org/jira/browse/SPARK-23836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16777416#comment-16777416
]
Bryan Cutler commented on SPARK-23836:
--
I can work on this
> Support returning StructType to the
[
https://issues.apache.org/jira/browse/SPARK-25147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-25147.
--
Resolution: Cannot Reproduce
Going to resolve this for now, please reopen if the above
[
https://issues.apache.org/jira/browse/SPARK-26943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16780751#comment-16780751
]
Bryan Cutler commented on SPARK-26943:
--
If you can try to reproduce locally, that would be ideal.
[
https://issues.apache.org/jira/browse/SPARK-26858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773284#comment-16773284
]
Bryan Cutler commented on SPARK-26858:
--
{quote}
(One other possibility I was thinking about batches
[
https://issues.apache.org/jira/browse/SPARK-23961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16784843#comment-16784843
]
Bryan Cutler commented on SPARK-23961:
--
I could also reproduce with a nearly identical error using
[
https://issues.apache.org/jira/browse/SPARK-27039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16783690#comment-16783690
]
Bryan Cutler commented on SPARK-27039:
--
I was able to reproduce in v2.4.0, but it looks like
[
https://issues.apache.org/jira/browse/SPARK-26943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774566#comment-16774566
]
Bryan Cutler commented on SPARK-26943:
--
Could you please provide a complete script to reproduce?
Bryan Cutler created SPARK-27163:
Summary: Cleanup and consolidate Pandas UDF functionality
Key: SPARK-27163
URL: https://issues.apache.org/jira/browse/SPARK-27163
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-27163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-27163:
-
Priority: Minor (was: Major)
> Cleanup and consolidate Pandas UDF functionality
>
[
https://issues.apache.org/jira/browse/SPARK-23836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-23836.
--
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 23900
[
https://issues.apache.org/jira/browse/SPARK-23836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-23836:
Assignee: Bryan Cutler
> Support returning StructType to the level support in GroupedMap
[
https://issues.apache.org/jira/browse/SPARK-26858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772266#comment-16772266
]
Bryan Cutler commented on SPARK-26858:
--
[~hyukjin.kwon] actually {{pyarrow.Table.from_batches}}
[
https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-26566:
-
Description:
Version 0.12.0 includes the following selected fixes/improvements relevant to
[
https://issues.apache.org/jira/browse/SPARK-26566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-26566:
-
Description:
Version 0.12.0 includes the following selected fixes/improvements relevant to
[
https://issues.apache.org/jira/browse/SPARK-27389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813896#comment-16813896
]
Bryan Cutler commented on SPARK-27389:
--
Thanks [~shaneknapp] for the fix. I couldn't come up with
[
https://issues.apache.org/jira/browse/SPARK-27389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812877#comment-16812877
]
Bryan Cutler commented on SPARK-27389:
--
[~shaneknapp], I had a couple of successful tests with
[
https://issues.apache.org/jira/browse/SPARK-27387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-27387:
Assignee: Bryan Cutler
> Replace sqlutils assertPandasEqual with Pandas
[
https://issues.apache.org/jira/browse/SPARK-27463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16842335#comment-16842335
]
Bryan Cutler commented on SPARK-27463:
--
[~d80tb7] I think you could remove the SPIP label from this
[
https://issues.apache.org/jira/browse/SPARK-27712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-27712.
--
Resolution: Duplicate
> createDataFrame() reorders row
> --
>
>
[
https://issues.apache.org/jira/browse/SPARK-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-27805:
-
Affects Version/s: (was: 3.1.0)
2.4.3
> toPandas does not propagate
[
https://issues.apache.org/jira/browse/SPARK-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-27805.
--
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24677
[
https://issues.apache.org/jira/browse/SPARK-27805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-27805:
Assignee: David Vogelbacher
> toPandas does not propagate SparkExceptions with arrow
[
https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855969#comment-16855969
]
Bryan Cutler edited comment on SPARK-27939 at 6/4/19 6:13 PM:
--
Linked to a
[
https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-27939.
--
Resolution: Not A Problem
> Defining a schema with VectorUDT
>
[
https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855969#comment-16855969
]
Bryan Cutler commented on SPARK-27939:
--
Another problem with Python {{Row}} class
> Defining a
[
https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855966#comment-16855966
]
Bryan Cutler edited comment on SPARK-27939 at 6/4/19 6:11 PM:
--
The problem
[
https://issues.apache.org/jira/browse/SPARK-27939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16855966#comment-16855966
]
Bryan Cutler commented on SPARK-27939:
--
The problem is the {{Row}} class sorts the field names
[
https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-27992:
-
Description:
Both SPARK-27805 and SPARK-27548 identified an issue that errors in a Spark job
[
https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-27992:
-
Environment: (was: Both SPARK-27805 and SPARK-27548 identified an issue
that errors in a
Bryan Cutler created SPARK-27992:
Summary: PySpark socket server should sync with JVM connection
thread future
Key: SPARK-27992
URL: https://issues.apache.org/jira/browse/SPARK-27992
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-27992:
-
Affects Version/s: (was: 2.4.3)
3.0.0
> PySpark socket server should
[
https://issues.apache.org/jira/browse/SPARK-27992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler updated SPARK-27992:
-
Description:
Both SPARK-27805 and SPARK-27548 identified an issue that errors in a Spark job
[
https://issues.apache.org/jira/browse/SPARK-28003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler resolved SPARK-28003.
--
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24844
[
https://issues.apache.org/jira/browse/SPARK-28003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Cutler reassigned SPARK-28003:
Assignee: Li Jin
> spark.createDataFrame with Arrow doesn't work with pandas.NaT
>
501 - 600 of 779 matches
Mail list logo
