Github user skparkes commented on a diff in the pull request:
https://github.com/apache/spark/pull/12251#discussion_r60162696
--- Diff: python/pyspark/sql/types.py ---
@@ -507,20 +503,52 @@ def add(self, field, data_type=None, nullable=True,
metadata=None):
else:
data_type_f = data_type
self.fields.append(StructField(field, data_type_f, nullable,
metadata))
- self.names.append(field)
- self._needSerializeAnyField = any(f.needConversion() for f in
self.fields)
return self
+ @property
+ def names(self):
+ """Return a list of the field names"""
+ return [field.name for field in self]
+
+ @property
+ def _needSerializeAnyField(self):
+ """Determine if any field needs conversion"""
+ return any(field.needConversion() for field in self)
+
+ def __iter__(self):
+ """Iterate the fields"""
+ return iter(self.fields)
+
+ def __len__(self):
+ """Return the number of fields."""
+ return len(self.fields)
+
+ def __getitem__(self, key):
+ """Access fields by name or slice."""
+ if isinstance(key, str):
+ _dict_fields = {field.name: field for field in self}
--- End diff --
There we go, I added another commit to correctly return `StructType`
objects when accessing fields via a `slice`. This was a quirk of Python I was
only recently made aware of. So in summary, this pull request proposes to:
- Using a `str` key will return the named `StructField`:
`my_structtype['my_structfield_name`]`
- Using a single `int` key will return the `StructField` by index:
`my_structtype[2]`
- Using a `slice` object (e.g. `[2:3]`) will return a new `StructType`
object with the chosen fields: `my_structtype[2:3]`
I'm happy to expand the tests and/or the documentation if you think any
section could be clearer.
Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
