Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/2920#discussion_r19719423
--- Diff: python/pyspark/tests.py ---
@@ -1216,51 +1216,6 @@ def test_reserialization(self):
result5 = sorted(self.sc.sequenceFile(basepath +
"/reserialize/newdataset").collect())
self.assertEqual(result5, data)
- def test_unbatched_save_and_read(self):
--- End diff --
With or without batching, it looks like the old code ended up flattening
out all of the objects when writing them to a SequenceFile, so the end result /
data was the same. It looks like we already have other tests for reading
Hadoop files, so I agree that this is probably safe to remove.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
