Is spoutLog just a non-spark file writer? If you run that in the map call
on a cluster its going to be writing in the filesystem of the executor its
being run on. I'm not sure if that's what you intended.
On Mon, Jun 22, 2015 at 1:35 PM, anshu shukla <anshushuk...@gmail.com>
wrote:
> Running perfectly in local system but not writing to file in cluster mode
> .ANY suggestions please ..
>
>
> //msgid is long counter
>
> JavaDStream<String> newinputStream=inputStream.map(new Function<String,
> String>() {
> @Override
> public String call(String v1) throws Exception {
> String s1=msgId+"@"+v1;
> System.out.println(s1);
> msgId++;
> try {
> *//filewriter logic
> spoutlog.batchLogwriter(System.currentTimeMillis(), "spout-MSGID," +
> msgeditor.getMessageId(s1));*
> } catch (Exception e) {
>
> System.out.println("exeception is here");
> e.printStackTrace();
> throw e;
> }
> System.out.println("msgid,"+msgId);
> return msgeditor.addMessageId(v1,msgId);
> }
> });
>
>
> --
> Thanks & Regards,
> Anshu Shukla
>
> On Mon, Jun 22, 2015 at 10:50 PM, anshu shukla <anshushuk...@gmail.com>
> wrote:
>
>> Can not we write some data to a txt file in parallel with multiple
>> executors running in parallel ??
>>
>>
>> --
>> Thanks & Regards,
>> Anshu Shukla
>>
>
>
>
> --
> Thanks & Regards,
> Anshu Shukla
>
