Hi, Guys,
Where can I find the console log file of CoarseGrainedExecutorBackend
process?
Thanks!
Tao
Hi, Guys,
Is it possible that RDD created by driver A be used driver B?
Thanks!
You may consider storing it in one big HDFS file, and to keep appending new
messages to it.
For instance,
one message -> zip it -> append it to the HDFS as one line
On Wed, Sep 2, 2015 at 12:43 PM, wrote:
> Hi,
> I already store them in MongoDB in parralel for operational access and
> don't wa
Your requirements conflict with each other.
1. You want to dump all your messages somewhere
2. You want to be able to update/delete individual message
3. You don't want to introduce anther NOSQL database(like HBASE) since you
already have all messages stored in MongoDB
My suggestion is:
1. Don't i
You situation is special. It seems to me Spark may not fit well in your
case.
You want to process the individual files (500M~2G) as a whole, you want
good performance.
You may want to write our own Scala/Java programs and distribute it along
with those files across your cluster, and run them in p
Yes. Ayan, you approach will work.
Or alternatively, use Spark, and write a Scala/Java function which
implements similar logic in your Pig UDF.
Both approaches look similar.
Personally, I would go with Spark solution, it will be slightly faster, and
easier if you already have Spark cluster setup
> more money for maintenance and development.
>
> Le jeu. 3 sept. 2015 à 17:16, Tao Lu a écrit :
>
>> Yes. Ayan, you approach will work.
>>
>> Or alternatively, use Spark, and write a Scala/Java function which
>> implements similar logic in your Pig UDF.
>>
>>
Basically they need NOSQL like random update access.
On Fri, Sep 4, 2015 at 9:56 AM, Ted Yu wrote:
> What about concurrent access (read / update) to the small file with same
> key ?
>
> That can get a bit tricky.
>
> On Thu, Sep 3, 2015 at 2:47 PM, Jörn Franke wrote:
>
>> Well it is the sam
unsubscribe
Unsubscribe
unsubscribe
Hi, Guys,
I have some code which runs will using Spark-Submit command.
$SPARK_HOME/bin/spark-submit --class com.myorg.service.SparkService ./Search.jar
How can I deploy it to Tomcat? If I simply deploy the jar file, I
will get ClassNotFound error.
Thanks!
-
13 matches
Mail list logo