[
https://issues.apache.org/jira/browse/SPARK-6574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384172#comment-14384172
]
Apache Spark commented on SPARK-6574:
-------------------------------------
User 'davies' has created a pull request for this issue:
https://github.com/apache/spark/pull/5230
> Python Example sql.py not working in version 1.3
> ------------------------------------------------
>
> Key: SPARK-6574
> URL: https://issues.apache.org/jira/browse/SPARK-6574
> Project: Spark
> Issue Type: Bug
> Components: PySpark, SQL
> Affects Versions: 1.3.0
> Reporter: Davies Liu
> Assignee: Davies Liu
> Priority: Blocker
>
> I downloaded spark version spark-1.3.0-bin-hadoop2.4.
> When the python version of sql.py is run the following error occurs:
> [root@nde-dev8-template python]#
> /root/spark-1.3.0-bin-hadoop2.4/bin/spark-submit sql.py
> Spark assembly has been built with Hive, including Datanucleus jars on
> classpath
> Traceback (most recent call last):
> File "/root/spark-1.3.0-bin-hadoop2.4/examples/src/main/python/sql.py",
> line 22, in <module>
> from pyspark.sql import Row, StructField, StructType, StringType,
> IntegerType
> ImportError: cannot import name StructField
> ----------------------------------------------------------------------
> The sql.py version, spark-1.2.1-bin-hadoop2.4, does not throw the error:
> [root@nde-dev8-template python]#
> /root/spark-1.2.1-bin-hadoop2.4/bin/spark-submit sql.py
> Spark assembly has been built with Hive, including Datanucleus jars on
> classpath
> 15/03/27 14:18:44 WARN NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 15/03/27 14:19:41 WARN ThreadLocalRandom: Failed to generate a seed from
> SecureRandom within 3 seconds. Not enough entrophy?
> root
> |-- age: integer (nullable = true)
> |-- name: string (nullable = true)
> root
> |-- person_name: string (nullable = false)
> |-- person_age: integer (nullable = false)
> root
> |-- age: integer (nullable = true)
> |-- name: string (nullable = true)
> Justin
> -------------------------------------------------
> The OS/JAVA environments are:
> OS: Linux nde-dev8-template 2.6.32-431.17.1.el6.x86_64 #1 SMP Fri Apr 11
> 17:27:00 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux
> JAVA: [root@nde-dev8-template bin]# java -version
> java version "1.7.0_51"
> Java(TM) SE Runtime Environment (build 1.7.0_51-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)
> The same error occurs when using bin/pyspark shell.
> >>> from pyspark.sql import StructField
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> ImportError: cannot import name StructField
> ---------------------------------------------------
> Any advice for resolving? Thanks in advance.
> Peter
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
