Re: Variable not initialized in the open() method of RichMapFunction

2016-07-22 Thread Stephan Ewen 
Initializing in "open(Configuration)" means that the ObjectMapper is
created only in the cluster once the MapFunction is started.

Otherwise it is created before (on the client) and Serialization-copied
into the cluster, together with the MapFunction.

If the second approach works well (i.e., the ObjectMapper is serialization
friendly), then there is no downside to it.

On Fri, Jul 22, 2016 at 6:01 PM, Dong iL, Kim  wrote:

> declare objectMapper out of map class.
>
> final ObjectMapper objectMapper = new ObjectMapper();
>
> source.map(str -> objectMapper.readValue(value, Request.class));
>
> On Sat, Jul 23, 2016 at 12:28 AM, Yassin Marzouki 
> wrote:
>
>> Thank you Stephan and Kim, that solved the problem.
>> Just to make sure, is using a MapFunction as in the following code any
>> different? i.e. does it initialize the objectMapper for every element in
>> the stream?
>>
>> .map(new MapFunction() {
>>
>> private ObjectMapper objectMapper = new ObjectMapper();
>>
>> @Override
>>  public Request map(String value) throws Exception {
>>  return objectMapper.readValue(value, Request.class);
>> }
>> })
>>
>> On Fri, Jul 22, 2016 at 5:20 PM, Dong iL, Kim  wrote:
>>
>>> oops. stephan already answered.
>>> sorry. T^T
>>>
>>> On Sat, Jul 23, 2016 at 12:16 AM, Dong iL, Kim 
>>> wrote:
>>>
 is open method signature right? or typo?

 void open(Configuration parameters) throws Exception;

 On Sat, Jul 23, 2016 at 12:09 AM, Stephan Ewen 
 wrote:

> I think you overrode the open method with the wrong signature. The
> right signature would be "open(Configuration cfg) {...}". You probably
> overlooked this because you missed the "@Override" annotation.
>
> On Fri, Jul 22, 2016 at 4:49 PM, Yassin Marzouki  > wrote:
>
>> Hi everyone,
>>
>> I want to convert a stream of json strings to POJOs using Jackson, so
>> I did the following:
>>
>> .map(new RichMapFunction() {
>>
>> private ObjectMapper objectMapper;
>>
>> public void open() {
>> objectMapper = new ObjectMapper();
>> }
>>
>> @Override
>>  public Request map(String value) throws Exception {
>>  return objectMapper.readValue(value, Request.class);
>> }
>> })
>>
>> But this code gave me a NullPointerException because the objectMapper
>> was not initialized successfully.
>>
>> 1. Isn't the open() method supposed to be called before map() and
>> initialize objectMapper?
>> 2. I figured out that initializing objectMapper before the open()
>> method resolves the problem, and that it works also with a simple
>> MapFunction. In that case, is there an advantage for using a
>> RichMapFunction?
>>
>> Best,
>> Yassine
>>
>
>


 --
 http://www.kiva.org; TARGET="_top">
 http://www.kiva.org/images/bannerlong.png; WIDTH="460"
 HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
 ALIGN="BOTTOM">

>>>
>>>
>>>
>>> --
>>> http://www.kiva.org; TARGET="_top">
>>> http://www.kiva.org/images/bannerlong.png; WIDTH="460"
>>> HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
>>> ALIGN="BOTTOM">
>>>
>>
>>
>
>
> --
> http://www.kiva.org; TARGET="_top">
> http://www.kiva.org/images/bannerlong.png; WIDTH="460"
> HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
> ALIGN="BOTTOM">
>

Re: Variable not initialized in the open() method of RichMapFunction

2016-07-22 Thread Dong iL, Kim 
declare objectMapper out of map class.

final ObjectMapper objectMapper = new ObjectMapper();

source.map(str -> objectMapper.readValue(value, Request.class));

On Sat, Jul 23, 2016 at 12:28 AM, Yassin Marzouki 
wrote:

> Thank you Stephan and Kim, that solved the problem.
> Just to make sure, is using a MapFunction as in the following code any
> different? i.e. does it initialize the objectMapper for every element in
> the stream?
>
> .map(new MapFunction() {
>
> private ObjectMapper objectMapper = new ObjectMapper();
>
> @Override
>  public Request map(String value) throws Exception {
>  return objectMapper.readValue(value, Request.class);
> }
> })
>
> On Fri, Jul 22, 2016 at 5:20 PM, Dong iL, Kim  wrote:
>
>> oops. stephan already answered.
>> sorry. T^T
>>
>> On Sat, Jul 23, 2016 at 12:16 AM, Dong iL, Kim 
>> wrote:
>>
>>> is open method signature right? or typo?
>>>
>>> void open(Configuration parameters) throws Exception;
>>>
>>> On Sat, Jul 23, 2016 at 12:09 AM, Stephan Ewen  wrote:
>>>
 I think you overrode the open method with the wrong signature. The
 right signature would be "open(Configuration cfg) {...}". You probably
 overlooked this because you missed the "@Override" annotation.

 On Fri, Jul 22, 2016 at 4:49 PM, Yassin Marzouki 
 wrote:

> Hi everyone,
>
> I want to convert a stream of json strings to POJOs using Jackson, so
> I did the following:
>
> .map(new RichMapFunction() {
>
> private ObjectMapper objectMapper;
>
> public void open() {
> objectMapper = new ObjectMapper();
> }
>
> @Override
>  public Request map(String value) throws Exception {
>  return objectMapper.readValue(value, Request.class);
> }
> })
>
> But this code gave me a NullPointerException because the objectMapper
> was not initialized successfully.
>
> 1. Isn't the open() method supposed to be called before map() and
> initialize objectMapper?
> 2. I figured out that initializing objectMapper before the open()
> method resolves the problem, and that it works also with a simple
> MapFunction. In that case, is there an advantage for using a
> RichMapFunction?
>
> Best,
> Yassine
>


>>>
>>>
>>> --
>>> http://www.kiva.org; TARGET="_top">
>>> http://www.kiva.org/images/bannerlong.png; WIDTH="460"
>>> HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
>>> ALIGN="BOTTOM">
>>>
>>
>>
>>
>> --
>> http://www.kiva.org; TARGET="_top">
>> http://www.kiva.org/images/bannerlong.png; WIDTH="460"
>> HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
>> ALIGN="BOTTOM">
>>
>
>


-- 
http://www.kiva.org; TARGET="_top">
http://www.kiva.org/images/bannerlong.png; WIDTH="460"
HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
ALIGN="BOTTOM">

Re: Variable not initialized in the open() method of RichMapFunction

2016-07-22 Thread Yassin Marzouki 
Thank you Stephan and Kim, that solved the problem.
Just to make sure, is using a MapFunction as in the following code any
different? i.e. does it initialize the objectMapper for every element in
the stream?

.map(new MapFunction() {

private ObjectMapper objectMapper = new ObjectMapper();

@Override
 public Request map(String value) throws Exception {
 return objectMapper.readValue(value, Request.class);
}
})

On Fri, Jul 22, 2016 at 5:20 PM, Dong iL, Kim  wrote:

> oops. stephan already answered.
> sorry. T^T
>
> On Sat, Jul 23, 2016 at 12:16 AM, Dong iL, Kim  wrote:
>
>> is open method signature right? or typo?
>>
>> void open(Configuration parameters) throws Exception;
>>
>> On Sat, Jul 23, 2016 at 12:09 AM, Stephan Ewen  wrote:
>>
>>> I think you overrode the open method with the wrong signature. The right
>>> signature would be "open(Configuration cfg) {...}". You probably overlooked
>>> this because you missed the "@Override" annotation.
>>>
>>> On Fri, Jul 22, 2016 at 4:49 PM, Yassin Marzouki 
>>> wrote:
>>>
 Hi everyone,

 I want to convert a stream of json strings to POJOs using Jackson, so I
 did the following:

 .map(new RichMapFunction() {

 private ObjectMapper objectMapper;

 public void open() {
 objectMapper = new ObjectMapper();
 }

 @Override
  public Request map(String value) throws Exception {
  return objectMapper.readValue(value, Request.class);
 }
 })

 But this code gave me a NullPointerException because the objectMapper
 was not initialized successfully.

 1. Isn't the open() method supposed to be called before map() and
 initialize objectMapper?
 2. I figured out that initializing objectMapper before the open()
 method resolves the problem, and that it works also with a simple
 MapFunction. In that case, is there an advantage for using a
 RichMapFunction?

 Best,
 Yassine

>>>
>>>
>>
>>
>> --
>> http://www.kiva.org; TARGET="_top">
>> http://www.kiva.org/images/bannerlong.png; WIDTH="460"
>> HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
>> ALIGN="BOTTOM">
>>
>
>
>
> --
> http://www.kiva.org; TARGET="_top">
> http://www.kiva.org/images/bannerlong.png; WIDTH="460"
> HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
> ALIGN="BOTTOM">
>

Re: Variable not initialized in the open() method of RichMapFunction

2016-07-22 Thread Dong iL, Kim 
oops. stephan already answered.
sorry. T^T

On Sat, Jul 23, 2016 at 12:16 AM, Dong iL, Kim  wrote:

> is open method signature right? or typo?
>
> void open(Configuration parameters) throws Exception;
>
> On Sat, Jul 23, 2016 at 12:09 AM, Stephan Ewen  wrote:
>
>> I think you overrode the open method with the wrong signature. The right
>> signature would be "open(Configuration cfg) {...}". You probably overlooked
>> this because you missed the "@Override" annotation.
>>
>> On Fri, Jul 22, 2016 at 4:49 PM, Yassin Marzouki 
>> wrote:
>>
>>> Hi everyone,
>>>
>>> I want to convert a stream of json strings to POJOs using Jackson, so I
>>> did the following:
>>>
>>> .map(new RichMapFunction() {
>>>
>>> private ObjectMapper objectMapper;
>>>
>>> public void open() {
>>> objectMapper = new ObjectMapper();
>>> }
>>>
>>> @Override
>>>  public Request map(String value) throws Exception {
>>>  return objectMapper.readValue(value, Request.class);
>>> }
>>> })
>>>
>>> But this code gave me a NullPointerException because the objectMapper
>>> was not initialized successfully.
>>>
>>> 1. Isn't the open() method supposed to be called before map() and
>>> initialize objectMapper?
>>> 2. I figured out that initializing objectMapper before the open() method
>>> resolves the problem, and that it works also with a simple MapFunction. In
>>> that case, is there an advantage for using a RichMapFunction?
>>>
>>> Best,
>>> Yassine
>>>
>>
>>
>
>
> --
> http://www.kiva.org; TARGET="_top">
> http://www.kiva.org/images/bannerlong.png; WIDTH="460"
> HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
> ALIGN="BOTTOM">
>



-- 
http://www.kiva.org; TARGET="_top">
http://www.kiva.org/images/bannerlong.png; WIDTH="460"
HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
ALIGN="BOTTOM">

Re: Variable not initialized in the open() method of RichMapFunction

2016-07-22 Thread Dong iL, Kim 
is open method signature right? or typo?

void open(Configuration parameters) throws Exception;

On Sat, Jul 23, 2016 at 12:09 AM, Stephan Ewen  wrote:

> I think you overrode the open method with the wrong signature. The right
> signature would be "open(Configuration cfg) {...}". You probably overlooked
> this because you missed the "@Override" annotation.
>
> On Fri, Jul 22, 2016 at 4:49 PM, Yassin Marzouki 
> wrote:
>
>> Hi everyone,
>>
>> I want to convert a stream of json strings to POJOs using Jackson, so I
>> did the following:
>>
>> .map(new RichMapFunction() {
>>
>> private ObjectMapper objectMapper;
>>
>> public void open() {
>> objectMapper = new ObjectMapper();
>> }
>>
>> @Override
>>  public Request map(String value) throws Exception {
>>  return objectMapper.readValue(value, Request.class);
>> }
>> })
>>
>> But this code gave me a NullPointerException because the objectMapper was
>> not initialized successfully.
>>
>> 1. Isn't the open() method supposed to be called before map() and
>> initialize objectMapper?
>> 2. I figured out that initializing objectMapper before the open() method
>> resolves the problem, and that it works also with a simple MapFunction. In
>> that case, is there an advantage for using a RichMapFunction?
>>
>> Best,
>> Yassine
>>
>
>


-- 
http://www.kiva.org; TARGET="_top">
http://www.kiva.org/images/bannerlong.png; WIDTH="460"
HEIGHT="60" ALT="Kiva - loans that change lives" BORDER="0"
ALIGN="BOTTOM">

Re: Variable not initialized in the open() method of RichMapFunction

2016-07-22 Thread Stephan Ewen 
I think you overrode the open method with the wrong signature. The right
signature would be "open(Configuration cfg) {...}". You probably overlooked
this because you missed the "@Override" annotation.

On Fri, Jul 22, 2016 at 4:49 PM, Yassin Marzouki 
wrote:

> Hi everyone,
>
> I want to convert a stream of json strings to POJOs using Jackson, so I
> did the following:
>
> .map(new RichMapFunction() {
>
> private ObjectMapper objectMapper;
>
> public void open() {
> objectMapper = new ObjectMapper();
> }
>
> @Override
>  public Request map(String value) throws Exception {
>  return objectMapper.readValue(value, Request.class);
> }
> })
>
> But this code gave me a NullPointerException because the objectMapper was
> not initialized successfully.
>
> 1. Isn't the open() method supposed to be called before map() and
> initialize objectMapper?
> 2. I figured out that initializing objectMapper before the open() method
> resolves the problem, and that it works also with a simple MapFunction. In
> that case, is there an advantage for using a RichMapFunction?
>
> Best,
> Yassine
>

Variable not initialized in the open() method of RichMapFunction

2016-07-22 Thread Yassin Marzouki 
Hi everyone,

I want to convert a stream of json strings to POJOs using Jackson, so I did
the following:

.map(new RichMapFunction() {

private ObjectMapper objectMapper;

public void open() {
objectMapper = new ObjectMapper();
}

@Override
 public Request map(String value) throws Exception {
 return objectMapper.readValue(value, Request.class);
}
})

But this code gave me a NullPointerException because the objectMapper was
not initialized successfully.

1. Isn't the open() method supposed to be called before map() and
initialize objectMapper?
2. I figured out that initializing objectMapper before the open() method
resolves the problem, and that it works also with a simple MapFunction. In
that case, is there an advantage for using a RichMapFunction?

Best,
Yassine