Hi，
Here is my javacode;
SparkConf sparkConf = Constance.getSparkConf();
JavaSparkContext sc = new JavaSparkContext(sparkConf);
SQLContext sql = new SQLContext(sc);
HiveContext sqlContext = new HiveContext(sc.sc());
List<StructField> fields = new ArrayList<StructField>();
fields.add(DataTypes.createStructField("id", DataTypes.IntegerType,
true));
List<Integer> data = Arrays.asList(1, 2, 3, 4, 5);
JavaRDD<Integer> distData = sc.parallelize(data);
StructType schema = DataTypes.createStructType(fields);
JavaRDD<Row> rdd = distData.map(new Function<Integer, Row>() {
@Override
public Row call(Integer v1) throws Exception {
return RowFactory.create(v1);
}
});
DataFrame peopleDataFrame = sqlContext.createDataFrame(rdd, schema);
peopleDataFrame.write().mode(SaveMode.Overwrite).format("json")
.saveAsTable("shop.id");
System.out.println("**************"+sqlContext.sql("select * from
shop.id").toJavaRDD()
.collect());
I run java on my pc to create a table named "shop.id" in Hive,the
table have 5 records;
And then I run "select * from shop.id" through beeline that connect
to server;
It works well;
But when I run the shell second or above;
The javacode could read the data well,but beeline throw an exception
Error: org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist:
hdfs://namenode:9000/user/hive/warehouse/shop.db/id/part-r-00003-4af3afb8-e029-4540-9788-9e56a9ef78c7
Input path does not exist:
hdfs://namenode:9000/user/hive/warehouse/shop.db/id/part-r-00001-4af3afb8-e029-4540-9788-9e56a9ef78c7
Input path does not exist:
hdfs://namenode:9000/user/hive/warehouse/shop.db/id/part-r-00000-4af3afb8-e029-4540-9788-9e56a9ef78c7
Input path does not exist:
hdfs://namenode:9000/user/hive/warehouse/shop.db/id/part-r-00002-4af3afb8-e029-4540-9788-9e56a9ef78c7
Why??? java code read well but beeline fail!
I'm sure that beeline and shell read the same hive-site.xml ,the
metastore is saved with mysql;
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Can-t-read-data-correctly-through-beeline-when-data-is-save-by-HiveContext-tp25774.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
