Re: spark 1.2 three times slower than spark 1.1, why?

2015-01-21 Thread JaeBoo Jung
Title: Samsung Enterprise Portal mySingle I was recently faced with a similar issue, but unfortunatelyIcould notfind out why it happened. Here'sjira ticket https://issues.apache.org/jira/browse/SPARK-5081of my previous post. Please checkyour shuffle I/O differences between the two in spark

use spark-shell in the source

2014-06-11 Thread JaeBoo Jung
shell style and get outputs. run(line); } Thanks _ JaeBoo, Jung AssistantEngineer/BDA Lab/ Samsung SDS

Re: Re: spark table to hive table

2014-05-27 Thread JaeBoo Jung
Title: Samsung Enterprise Portal mySingle I already tried HiveContext as well as SqlContext. Butitseems that Spark'sHiveContext is not completely same as Apache Hive. For example, SQL like 'SELECT RANK() OVER(ORDER BY VAL1 ASC) FROM TEST LIMIT 10' works fine in Apache Hive, butSpark's Hive