I've filed SPARK-7507 <https://issues.apache.org/jira/browse/SPARK-7507> for this.
On Fri, May 8, 2015 at 5:57 PM Reynold Xin <r...@databricks.com> wrote: > Sure. > > > On Fri, May 8, 2015 at 2:43 PM, Nicholas Chammas < > nicholas.cham...@gmail.com> wrote: > >> StructType looks an awful lot like a Python dictionary. >> >> However, it doesn’t implement __iter__() >> > <https://docs.python.org/3/library/stdtypes.html#iterator-types>, so doing > > >> a quick conversion like this doesn’t work: >> >> >>> df = sqlContext.jsonRDD(sc.parallelize(['{"name": "El >> Magnifico"}']))>>> df.schema >> StructType(List(StructField(name,StringType,true)))>>> dict(df.schema) >> Traceback (most recent call last): >> File "<stdin>", line 1, in <module> >> TypeError: 'StructType' object is not iterable >> >> This would be super helpful for doing any custom schema manipulations >> without having to go through the whole .json() -> json.loads() -> >> manipulate() -> json.dumps() -> .fromJson() charade. >> >> Same goes for Row, which offers an asDict() >> > < >> https://spark.apache.org/docs/1.3.1/api/python/pyspark.sql.html#pyspark.sql.Row.asDict >> > > > >> method but doesn’t support the more Pythonic dict(Row). >> >> Does this make sense? >> >> Nick >> >> >
