Gian,
I have tested storm-integration code locally. Its working fine.
On Tue, Sep 29, 2015 at 4:12 PM, Gianfranco Murador <mura...@apache.org>
wrote:
> Yes, in mater branch of my fork.
>
> 2015-09-29 8:45 GMT+02:00 chandresh pancholi <
> chandreshpancholi...@gmail.com
> >:
>
> > Gian,
> >
> > is code for ignite-429 in master branch?
> >
> > On Mon, Sep 28, 2015 at 5:27 PM, chandresh pancholi <
> > chandreshpancholi...@gmail.com> wrote:
> >
> > > Gian,
> > >
> > > Lets use this and modify our tests. I will push the code by tomorrow
> EOD.
> > >
> > > Thanks
> > > Chandresh Pancholi
> > >
> > > On Mon, Sep 28, 2015 at 3:58 PM, Gianfranco Murador <
> > > murador.gianfra...@gmail.com> wrote:
> > >
> > >> Chandresh,
> > >> I think that the suggestion of Roman is preferable, we can use the
> > >> following code :
> > >> Let me know if you have problems
> > >>
> > >> /** Start a simulated topology */
> > >> public void startTopology(StormStreamer stormStreamer) {
> > >> MkClusterParam mkClusterParam = new MkClusterParam();
> > >> mkClusterParam.setSupervisors(4);
> > >> Config daemonConf = new Config();
> > >> daemonConf.put(Config.STORM_LOCAL_MODE_ZMQ, false);
> > >> mkClusterParam.setDaemonConf(daemonConf);
> > >>
> > >> Testing.withSimulatedTimeLocalCluster(mkClusterParam, new
> > >> TestJob()
> > >> {
> > >> @Override
> > >> public void run(ILocalCluster cluster) throws
> > >> IOException {
> > >> TopologyBuilder builder = new
> TopologyBuilder();
> > >>
> > >> builder.setSpout("testSpout", new
> StormSpout());
> > >>
> > >> builder.setBolt("bolt", stormStreamer)
> > >> .shuffleGrouping("spout");
> > >>
> > >> StormTopology topology =
> > builder.createTopology();
> > >>
> > >> MockedSources mockedSources = new
> > MockedSources();
> > >>
> > >> //Our spout will be processing this values.
> > >> mockedSources.addMockData("testSpout",new
> > >> Values(stormStream.getKeyValMap()));
> > >>
> > >>
> > >> // prepare the config
> > >> Config conf = new Config();
> > >> conf.setNumWorkers(2);
> > >>
> > >> CompleteTopologyParam completeTopologyParam =
> > new
> > >> CompleteTopologyParam();
> > >>
> > >> completeTopologyParam.setMockedSources(mockedSources);
> > >> completeTopologyParam.setStormConf(conf);
> > >>
> > >> Map result = Testing.completeTopology(cluster,
> > >> topology, completeTopologyParam);
> > >> }
> > >> }
> > >> );
> > >> }
> > >>
> > >> 2015-09-28 6:11 GMT+02:00 chandresh pancholi <
> > >> chandreshpancholi...@gmail.com
> > >> >:
> > >>
> > >> > Agree,
> > >> > Will check for some other solution and update!!
> > >> >
> > >> > On Mon, Sep 28, 2015 at 9:12 AM, Vishal Garg <gargv...@gmail.com>
> > >> wrote:
> > >> >
> > >> > > Chandresh,
> > >> > >
> > >> > > Generally sleep in tests is not a good idea. Will try to look into
> > >> your
> > >> > > test this week, see if we can avoid it.
> > >> > > Meanwhile you can look at how storm does its own unit testing if
> you
> > >> > could
> > >> > > find one?
> > >> > > Vishal
> > >> > >
> > >> > > Sent from my iPhone
> > >> > >
> > >> > > > On Sep 27, 2015, at 8:23 PM, Roman <rsht...@yahoo.com.INVALID>
> > >> wrote:
> > >> > > >
> > >> > > > I also have concerns about this sleep().Storm has testing
> > facilities
> > >> > you
> > >> > > can use for your test. How about
> > >> Testing.withSimulatedTimeLocalCluster()?
> > >> > > > Roman
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Monday, September 28, 2015 5:39 AM, Dmitriy Setrakyan <
> > >> > > dsetrak...@apache.org> wrote:
> > >> > > >
> > >> > > >
> > >> > > > On Sun, Sep 27, 2015 at 1:36 PM, chandresh pancholi <
> > >> > > > chandreshpancholi...@gmail.com> wrote:
> > >> > > >
> > >> > > >> Gian,
> > >> > > >> I have checked the code. It looks good.
> > >> > > >> Just one little change make it 20 sec.
> > >> > > >>
> > >> > > >> /* Topology will run for 10sec */
> > >> > > >> Utils.sleep(20000);
> > >> > > >
> > >> > > > Hm... Can someone explain what is this sleep() for?
> > >> > > >
> > >> > > >
> > >> > > >>
> > >> > > >>
> > >> > > >>
> > >> > > >> On Sun, Sep 27, 2015 at 4:30 PM, Gianfranco Murador <
> > >> > > >> murador.gianfra...@gmail.com> wrote:
> > >> > > >>
> > >> > > >>> Chandresh, we're ready to do a pull request. Look here:
> > >> > > >>>
> https://github.com/murador/ignite/tree/IGNITE-429/modules/storm
> > >> > > >>>
> > >> > > >>> 2015-09-27 8:49 GMT+02:00 chandresh pancholi <
> > >> > > >>> chandreshpancholi...@gmail.com
> > >> > > >>>> :
> > >> > > >>>
> > >> > > >>>> Here is the log file.
> > >> > > >>>>
> > >> > > >>>> Thanks
> > >> > > >>>>
> > >> > > >>>> On Sat, Sep 26, 2015 at 10:40 PM, Gianfranco Murador <
> > >> > > >>>> murador.gianfra...@gmail.com> wrote:
> > >> > > >>>>
> > >> > > >>>>> Chan,
> > >> > > >>>>> I can run the test succefully, but sometimes I get this
> > error:
> > >> > > >>>>> Caused by: class org.apache.ignite.spi.IgniteSpiException:
> > >> Failed
> > >> > to
> > >> > > >> add
> > >> > > >>>>> node to topology because remote node is configured to use
> > >> loopback
> > >> > > >>>>> address,
> > >> > > >>>>> but local node is not (consider changing 'localAddress'
> > >> > configuration
> > >> > > >>>>> parameter).
> > >> > > >>>>>
> > >> > > >>>>> can you send the log file?
> > >> > > >>>>> Regards, Gianfranco
> > >> > > >>>>>
> > >> > > >>>>>
> > >> > > >>>>>
> > >> > > >>>>> 2015-09-26 17:13 GMT+02:00 chandresh pancholi <
> > >> > > >>>>> chandreshpancholi...@gmail.com>:
> > >> > > >>>>>
> > >> > > >>>>>> Gian,
> > >> > > >>>>>> Its still giving "Ouch! Argument cannot be null: streamer"
> > >> > > >>>>>>
> > >> > > >>>>>> On Sat, Sep 26, 2015 at 7:24 PM, Gianfranco Murador <
> > >> > > >>>>>> murador.gianfra...@gmail.com> wrote:
> > >> > > >>>>>>
> > >> > > >>>>>>> Chan,
> > >> > > >>>>>>> I've fixed some issue, please take a look here:
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
> https://github.com/murador/ignite/blob/IGNITE-429/modules/storm/src/test/java/org/apache/ignite/stream/storm/StormIgniteStreamerSelfTest.java
> > >> > > >>>>>>>
> > >> > > >>>>>>> 2015-09-26 8:29 GMT+02:00 chandresh pancholi <
> > >> > > >>>>>>> chandreshpancholi...@gmail.com
> > >> > > >>>>>>>> :
> > >> > > >>>>>>>
> > >> > > >>>>>>>> Gian,
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> I have followed instructions given by you. I instantiate
> > >> > > >>>>>>> IgniteDateStreamer
> > >> > > >>>>>>>> and ignite in prepare() method of bolt. but getting
> "*Cache
> > >> > > >>> doesn't
> > >> > > >>>>>>>> exist*".
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> Code snippet from stormStreamer class in prepare() method
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> StormStreamer<String, String, String> stormStreamer =
> null;
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> Ignite ignite = Ignition.start();
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> System.out.println(" ignite ====== "+ignite.toString());
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> try(IgniteDataStreamer<String, String> stmr =
> > >> > > >>>>>>>> ignite.dataStreamer("cache-0")) {
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> stormStreamer = new StormStreamer<>();
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> IgniteCache<String, String> cache =
> > >> ignite.cache("cache-0");
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> stmr.allowOverwrite(true);
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> /* Set ignite instance */
> > >> > > >>>>>>>> stormStreamer.setIgnite(ignite);
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> /* Set streamer instance */
> > >> > > >>>>>>>> stormStreamer.setStreamer(stmr);
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> /* set thread count */
> > >> > > >>>>>>>> setThreads(5);
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> }
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> On Thu, Sep 24, 2015 at 4:19 PM, Anton Vinogradov <
> > >> > > >>>>>>>> avinogra...@gridgain.com>
> > >> > > >>>>>>>> wrote:
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>> Chandresh,
> > >> > > >>>>>>>>> As far as understand IBolt implementation should setup
> all
> > >> > > >>>>> external
> > >> > > >>>>>>>>> connections at .prepare() method.
> > >> > > >>>>>>>>> So, better way is to get existing Ignite instance or
> > create
> > >> > > >> new
> > >> > > >>> at
> > >> > > >>>>>>>>> .prepare()
> > >> > > >>>>>>>>> method.
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>> On Thu, Sep 24, 2015 at 10:55 AM, Gianfranco Murador <
> > >> > > >>>>>>>>> murador.gianfra...@gmail.com> wrote:
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>>> Chandresh,
> > >> > > >>>>>>>>>> I will launch the unit test later this week , if the
> > >> problem
> > >> > > >>>>>>> persists.
> > >> > > >>>>>>>> I
> > >> > > >>>>>>>>>> think that you should instantiate Ignite through the
> > >> start()
> > >> > > >>>>> method
> > >> > > >>>>>>> of
> > >> > > >>>>>>>>>> Ignition class
> > >> > > >>>>>>>>>> and retrieve the Ignite cache specifying a name for the
> > >> > > >>> default
> > >> > > >>>>>>> cache.
> > >> > > >>>>>>>>> You
> > >> > > >>>>>>>>>> should provide a name also for data streamer, not
> passing
> > >> > > >>> null.
> > >> > > >>>>>>>>>> Could you do this and run again the unit test ?
> > >> > > >>>>>>>>>> Thank you,
> > >> > > >>>>>>>>>> Regards,
> > >> > > >>>>>>>>>> Gianfranco
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>> 2015-09-24 5:40 GMT+02:00 chandresh pancholi <
> > >> > > >>>>>>>>>> chandreshpancholi...@gmail.com
> > >> > > >>>>>>>>>>> :
> > >> > > >>>>>>>>>>
> > >> > > >>>>>>>>>>> Anton,
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> Clone this repo
> > >> > > >>> https://github.com/chandresh-pancholi/ignite
> > >> > > >>>>> and
> > >> > > >>>>>>> run
> > >> > > >>>>>>>>>> this
> > >> > > >>>>>>>>>>> class
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
> https://github.com/chandresh-pancholi/ignite/blob/master/modules/storm/src/test/java/org/apache/ignite/stream/storm/IgniteStormStreamerSelfTestSuite.java
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> You will get the Error saying "Oouch,Argument is Null"
> > for
> > >> > > >>>>>>>>> getStreamer().
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> On Wed, Sep 23, 2015 at 9:46 PM, Anton Vinogradov <
> > >> > > >>>>>>>>>>> avinogra...@gridgain.com>
> > >> > > >>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>>> Chandresh,
> > >> > > >>>>>>>>>>>> I'm trying to analize problem.
> > >> > > >>>>>>>>>>>> Could you please provide more details and explain
> > >> > > >>>>> step-by-step
> > >> > > >>>>>>> how
> > >> > > >>>>>>>>> can
> > >> > > >>>>>>>>>> I
> > >> > > >>>>>>>>>>>> reproduce problem?
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>> On Tue, Sep 22, 2015 at 11:06 PM, chandresh pancholi
> <
> > >> > > >>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
> > >> > > >>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> Not yet.
> > >> > > >>>>>>>>>>>>> Still doing brainstorming why StormStreamer object
> is
> > >> > > >>>>> getting
> > >> > > >>>>>>>> NULLL
> > >> > > >>>>>>>>>>>>> in-spite setting up ignite and streamer instances.
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> On Tue, Sep 22, 2015 at 7:22 PM, Vishal Garg <
> > >> > > >>>>>>> gargv...@gmail.com
> > >> > > >>>>>>>>>
> > >> > > >>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Did you figure it out? Something related to your
> > >> > > >> storm
> > >> > > >>>>>>>>> integration?
> > >> > > >>>>>>>>>>>>>> Vishal
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>> Sent from my iPhone
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> On Sep 22, 2015, at 4:50 AM, chandresh pancholi <
> > >> > > >>>>>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> Problem here is with Storm. Its making streamer
> > >> > > >>> object
> > >> > > >>>>>> null
> > >> > > >>>>>>>>> once
> > >> > > >>>>>>>>>> it
> > >> > > >>>>>>>>>>>>> goes
> > >> > > >>>>>>>>>>>>>>> into submit topology.
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> On Mon, Sep 21, 2015 at 7:13 PM, Lalit Kumar Jha <
> > >> > > >>>>>>>>>>>> lalitj....@gmail.com
> > >> > > >>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> wrote:
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> Hi Chandresh,
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> See test suite class here, its based on
> > >> > > >> annotations
> > >> > > >>
> > >> > >
> > >> >
> > >>
> >
> https://github.com/sylentprayer/ignite/blob/ignite-530/modules/twitter/src/test/java/org/apache/ignite/stream/twitter/IgniteTwitterStreamerTestSuite.java
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> No code required in class body.
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>> On Mon, Sep 21, 2015 at 6:24 PM, chandresh
> > >> > > >>> pancholi <
> > >> > > >>>>>>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
> > >> > > >>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> Hi Gian/Vishal/Lalit,
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> You guys have worked on Streamer integration
> > >> > > >> with
> > >> > > >>>>>>> Ignite. I
> > >> > > >>>>>>>>> am
> > >> > > >>>>>>>>>>>>> working
> > >> > > >>>>>>>>>>>>>> on
> > >> > > >>>>>>>>>>>>>>>>> Storm-ignite integration.
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> I am setting up setStreamer() and setIgnite() in
> > >> > > >>>>> Test
> > >> > > >>>>>>> class
> > >> > > >>>>>>>>> and
> > >> > > >>>>>>>>>>>>> sending
> > >> > > >>>>>>>>>>>>>>>>> this object to setBolt() method.
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> When i try to read via gettreamer() and
> > >> > > >>> getIgnite()
> > >> > > >>>>> i
> > >> > > >>>>>> am
> > >> > > >>>>>>>>>> getting
> > >> > > >>>>>>>>>>>>> NULL.
> > >> > > >>>>>>>>>>>>>> I
> > >> > > >>>>>>>>>>>>>>>> am
> > >> > > >>>>>>>>>>>>>>>>> trying to find out why is it happening but No
> > >> > > >>> Luck.
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> If any one of you find some time and look into
> > >> > > >> the
> > >> > > >>>>> code
> > >> > > >>>>>>> and
> > >> > > >>>>>>>>>> help
> > >> > > >>>>>>>>>>> me
> > >> > > >>>>>>>>>>>>> to
> > >> > > >>>>>>>>>>>>>>>>> resolve this. It would be great.
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> Git :
> > >> > > >>> https://github.com/chandresh-pancholi/ignite
> > >> > > >>>>>>>>>>>>>>>>> Module : storm
> > >> > > >>>>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>>>> Chandresh Pancholi
> > >> > > >>>>>>>>>>>>>>>>> Senior Software Engineer
> > >> > > >>>>>>>>>>>>>>>>> Flipkart.com
> > >> > > >>>>>>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> > >> > > >>>>>>>>>>>>>>>>> Contact:08951803660
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>>>> Chandresh Pancholi
> > >> > > >>>>>>>>>>>>>>> Senior Software Engineer
> > >> > > >>>>>>>>>>>>>>> Flipkart.com
> > >> > > >>>>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> > >> > > >>>>>>>>>>>>>>> Contact:08951803660
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>>
> > >> > > >>>>>>>>>>>>> --
> > >> > > >>>>>>>>>>>>> Chandresh Pancholi
> > >> > > >>>>>>>>>>>>> Senior Software Engineer
> > >> > > >>>>>>>>>>>>> Flipkart.com
> > >> > > >>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> > >> > > >>>>>>>>>>>>> Contact:08951803660
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>>
> > >> > > >>>>>>>>>>> --
> > >> > > >>>>>>>>>>> Chandresh Pancholi
> > >> > > >>>>>>>>>>> Senior Software Engineer
> > >> > > >>>>>>>>>>> Flipkart.com
> > >> > > >>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> > >> > > >>>>>>>>>>> Contact:08951803660
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>>
> > >> > > >>>>>>>> --
> > >> > > >>>>>>>> Chandresh Pancholi
> > >> > > >>>>>>>> Senior Software Engineer
> > >> > > >>>>>>>> Flipkart.com
> > >> > > >>>>>>>> Email-id:chandresh.panch...@flipkart.com
> > >> > > >>>>>>>> Contact:08951803660
> > >> > > >>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>>
> > >> > > >>>>>> --
> > >> > > >>>>>> Chandresh Pancholi
> > >> > > >>>>>> Senior Software Engineer
> > >> > > >>>>>> Flipkart.com
> > >> > > >>>>>> Email-id:chandresh.panch...@flipkart.com
> > >> > > >>>>>> Contact:08951803660
> > >> > > >>>>
> > >> > > >>>>
> > >> > > >>>>
> > >> > > >>>> --
> > >> > > >>>> Chandresh Pancholi
> > >> > > >>>> Senior Software Engineer
> > >> > > >>>> Flipkart.com
> > >> > > >>>> Email-id:chandresh.panch...@flipkart.com
> > >> > > >>>> Contact:08951803660
> > >> > > >>
> > >> > > >>
> > >> > > >>
> > >> > > >> --
> > >> > > >> Chandresh Pancholi
> > >> > > >> Senior Software Engineer
> > >> > > >> Flipkart.com
> > >> > > >> Email-id:chandresh.panch...@flipkart.com
> > >> > > >> Contact:08951803660
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Chandresh Pancholi
> > >> > Senior Software Engineer
> > >> > Flipkart.com
> > >> > Email-id:chandresh.panch...@flipkart.com
> > >> > Contact:08951803660
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Chandresh Pancholi
> > > Senior Software Engineer
> > > Flipkart.com
> > > Email-id:chandresh.panch...@flipkart.com
> > > Contact:08951803660
> > >
> >
> >
> >
> > --
> > Chandresh Pancholi
> > Senior Software Engineer
> > Flipkart.com
> > Email-id:chandresh.panch...@flipkart.com
> > Contact:08951803660
> >
>
>
>
> --
> Gianfranco Murador
> Igniter and Software Engineer.
>
--
Chandresh Pancholi
Senior Software Engineer
Flipkart.com
Email-id:chandresh.panch...@flipkart.com
Contact:08951803660
