Gian,
Lets use this and modify our tests. I will push the code by tomorrow EOD.
Thanks
Chandresh Pancholi
On Mon, Sep 28, 2015 at 3:58 PM, Gianfranco Murador <
murador.gianfra...@gmail.com> wrote:
> Chandresh,
> I think that the suggestion of Roman is preferable, we can use the
> following code :
> Let me know if you have problems
>
> /** Start a simulated topology */
> public void startTopology(StormStreamer stormStreamer) {
> MkClusterParam mkClusterParam = new MkClusterParam();
> mkClusterParam.setSupervisors(4);
> Config daemonConf = new Config();
> daemonConf.put(Config.STORM_LOCAL_MODE_ZMQ, false);
> mkClusterParam.setDaemonConf(daemonConf);
>
> Testing.withSimulatedTimeLocalCluster(mkClusterParam, new TestJob()
> {
> @Override
> public void run(ILocalCluster cluster) throws
> IOException {
> TopologyBuilder builder = new TopologyBuilder();
>
> builder.setSpout("testSpout", new StormSpout());
>
> builder.setBolt("bolt", stormStreamer)
> .shuffleGrouping("spout");
>
> StormTopology topology = builder.createTopology();
>
> MockedSources mockedSources = new MockedSources();
>
> //Our spout will be processing this values.
> mockedSources.addMockData("testSpout",new
> Values(stormStream.getKeyValMap()));
>
>
> // prepare the config
> Config conf = new Config();
> conf.setNumWorkers(2);
>
> CompleteTopologyParam completeTopologyParam = new
> CompleteTopologyParam();
>
> completeTopologyParam.setMockedSources(mockedSources);
> completeTopologyParam.setStormConf(conf);
>
> Map result = Testing.completeTopology(cluster,
> topology, completeTopologyParam);
> }
> }
> );
> }
>
> 2015-09-28 6:11 GMT+02:00 chandresh pancholi <
> chandreshpancholi...@gmail.com
> >:
>
> > Agree,
> > Will check for some other solution and update!!
> >
> > On Mon, Sep 28, 2015 at 9:12 AM, Vishal Garg <gargv...@gmail.com> wrote:
> >
> > > Chandresh,
> > >
> > > Generally sleep in tests is not a good idea. Will try to look into your
> > > test this week, see if we can avoid it.
> > > Meanwhile you can look at how storm does its own unit testing if you
> > could
> > > find one?
> > > Vishal
> > >
> > > Sent from my iPhone
> > >
> > > > On Sep 27, 2015, at 8:23 PM, Roman <rsht...@yahoo.com.INVALID>
> wrote:
> > > >
> > > > I also have concerns about this sleep().Storm has testing facilities
> > you
> > > can use for your test. How about
> Testing.withSimulatedTimeLocalCluster()?
> > > > Roman
> > > >
> > > >
> > > >
> > > > On Monday, September 28, 2015 5:39 AM, Dmitriy Setrakyan <
> > > dsetrak...@apache.org> wrote:
> > > >
> > > >
> > > > On Sun, Sep 27, 2015 at 1:36 PM, chandresh pancholi <
> > > > chandreshpancholi...@gmail.com> wrote:
> > > >
> > > >> Gian,
> > > >> I have checked the code. It looks good.
> > > >> Just one little change make it 20 sec.
> > > >>
> > > >> /* Topology will run for 10sec */
> > > >> Utils.sleep(20000);
> > > >
> > > > Hm... Can someone explain what is this sleep() for?
> > > >
> > > >
> > > >>
> > > >>
> > > >>
> > > >> On Sun, Sep 27, 2015 at 4:30 PM, Gianfranco Murador <
> > > >> murador.gianfra...@gmail.com> wrote:
> > > >>
> > > >>> Chandresh, we're ready to do a pull request. Look here:
> > > >>> https://github.com/murador/ignite/tree/IGNITE-429/modules/storm
> > > >>>
> > > >>> 2015-09-27 8:49 GMT+02:00 chandresh pancholi <
> > > >>> chandreshpancholi...@gmail.com
> > > >>>> :
> > > >>>
> > > >>>> Here is the log file.
> > > >>>>
> > > >>>> Thanks
> > > >>>>
> > > >>>> On Sat, Sep 26, 2015 at 10:40 PM, Gianfranco Murador <
> > > >>>> murador.gianfra...@gmail.com> wrote:
> > > >>>>
> > > >>>>> Chan,
> > > >>>>> I can run the test succefully, but sometimes I get this error:
> > > >>>>> Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed
> > to
> > > >> add
> > > >>>>> node to topology because remote node is configured to use
> loopback
> > > >>>>> address,
> > > >>>>> but local node is not (consider changing 'localAddress'
> > configuration
> > > >>>>> parameter).
> > > >>>>>
> > > >>>>> can you send the log file?
> > > >>>>> Regards, Gianfranco
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> 2015-09-26 17:13 GMT+02:00 chandresh pancholi <
> > > >>>>> chandreshpancholi...@gmail.com>:
> > > >>>>>
> > > >>>>>> Gian,
> > > >>>>>> Its still giving "Ouch! Argument cannot be null: streamer"
> > > >>>>>>
> > > >>>>>> On Sat, Sep 26, 2015 at 7:24 PM, Gianfranco Murador <
> > > >>>>>> murador.gianfra...@gmail.com> wrote:
> > > >>>>>>
> > > >>>>>>> Chan,
> > > >>>>>>> I've fixed some issue, please take a look here:
> > > >>
> > >
> >
> https://github.com/murador/ignite/blob/IGNITE-429/modules/storm/src/test/java/org/apache/ignite/stream/storm/StormIgniteStreamerSelfTest.java
> > > >>>>>>>
> > > >>>>>>> 2015-09-26 8:29 GMT+02:00 chandresh pancholi <
> > > >>>>>>> chandreshpancholi...@gmail.com
> > > >>>>>>>> :
> > > >>>>>>>
> > > >>>>>>>> Gian,
> > > >>>>>>>>
> > > >>>>>>>> I have followed instructions given by you. I instantiate
> > > >>>>>>> IgniteDateStreamer
> > > >>>>>>>> and ignite in prepare() method of bolt. but getting "*Cache
> > > >>> doesn't
> > > >>>>>>>> exist*".
> > > >>>>>>>>
> > > >>>>>>>> Code snippet from stormStreamer class in prepare() method
> > > >>>>>>>>
> > > >>>>>>>> StormStreamer<String, String, String> stormStreamer = null;
> > > >>>>>>>>
> > > >>>>>>>> Ignite ignite = Ignition.start();
> > > >>>>>>>>
> > > >>>>>>>> System.out.println(" ignite ====== "+ignite.toString());
> > > >>>>>>>>
> > > >>>>>>>> try(IgniteDataStreamer<String, String> stmr =
> > > >>>>>>>> ignite.dataStreamer("cache-0")) {
> > > >>>>>>>>
> > > >>>>>>>> stormStreamer = new StormStreamer<>();
> > > >>>>>>>>
> > > >>>>>>>> IgniteCache<String, String> cache =
> ignite.cache("cache-0");
> > > >>>>>>>>
> > > >>>>>>>> stmr.allowOverwrite(true);
> > > >>>>>>>>
> > > >>>>>>>> /* Set ignite instance */
> > > >>>>>>>> stormStreamer.setIgnite(ignite);
> > > >>>>>>>>
> > > >>>>>>>> /* Set streamer instance */
> > > >>>>>>>> stormStreamer.setStreamer(stmr);
> > > >>>>>>>>
> > > >>>>>>>> /* set thread count */
> > > >>>>>>>> setThreads(5);
> > > >>>>>>>>
> > > >>>>>>>> }
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Thu, Sep 24, 2015 at 4:19 PM, Anton Vinogradov <
> > > >>>>>>>> avinogra...@gridgain.com>
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Chandresh,
> > > >>>>>>>>> As far as understand IBolt implementation should setup all
> > > >>>>> external
> > > >>>>>>>>> connections at .prepare() method.
> > > >>>>>>>>> So, better way is to get existing Ignite instance or create
> > > >> new
> > > >>> at
> > > >>>>>>>>> .prepare()
> > > >>>>>>>>> method.
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> On Thu, Sep 24, 2015 at 10:55 AM, Gianfranco Murador <
> > > >>>>>>>>> murador.gianfra...@gmail.com> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> Chandresh,
> > > >>>>>>>>>> I will launch the unit test later this week , if the problem
> > > >>>>>>> persists.
> > > >>>>>>>> I
> > > >>>>>>>>>> think that you should instantiate Ignite through the start()
> > > >>>>> method
> > > >>>>>>> of
> > > >>>>>>>>>> Ignition class
> > > >>>>>>>>>> and retrieve the Ignite cache specifying a name for the
> > > >>> default
> > > >>>>>>> cache.
> > > >>>>>>>>> You
> > > >>>>>>>>>> should provide a name also for data streamer, not passing
> > > >>> null.
> > > >>>>>>>>>> Could you do this and run again the unit test ?
> > > >>>>>>>>>> Thank you,
> > > >>>>>>>>>> Regards,
> > > >>>>>>>>>> Gianfranco
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>> 2015-09-24 5:40 GMT+02:00 chandresh pancholi <
> > > >>>>>>>>>> chandreshpancholi...@gmail.com
> > > >>>>>>>>>>> :
> > > >>>>>>>>>>
> > > >>>>>>>>>>> Anton,
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Clone this repo
> > > >>> https://github.com/chandresh-pancholi/ignite
> > > >>>>> and
> > > >>>>>>> run
> > > >>>>>>>>>> this
> > > >>>>>>>>>>> class
> > > >>
> > >
> >
> https://github.com/chandresh-pancholi/ignite/blob/master/modules/storm/src/test/java/org/apache/ignite/stream/storm/IgniteStormStreamerSelfTestSuite.java
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> You will get the Error saying "Oouch,Argument is Null" for
> > > >>>>>>>>> getStreamer().
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Wed, Sep 23, 2015 at 9:46 PM, Anton Vinogradov <
> > > >>>>>>>>>>> avinogra...@gridgain.com>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> Chandresh,
> > > >>>>>>>>>>>> I'm trying to analize problem.
> > > >>>>>>>>>>>> Could you please provide more details and explain
> > > >>>>> step-by-step
> > > >>>>>>> how
> > > >>>>>>>>> can
> > > >>>>>>>>>> I
> > > >>>>>>>>>>>> reproduce problem?
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> On Tue, Sep 22, 2015 at 11:06 PM, chandresh pancholi <
> > > >>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>> Not yet.
> > > >>>>>>>>>>>>> Still doing brainstorming why StormStreamer object is
> > > >>>>> getting
> > > >>>>>>>> NULLL
> > > >>>>>>>>>>>>> in-spite setting up ignite and streamer instances.
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> On Tue, Sep 22, 2015 at 7:22 PM, Vishal Garg <
> > > >>>>>>> gargv...@gmail.com
> > > >>>>>>>>>
> > > >>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Did you figure it out? Something related to your
> > > >> storm
> > > >>>>>>>>> integration?
> > > >>>>>>>>>>>>>> Vishal
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>> Sent from my iPhone
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> On Sep 22, 2015, at 4:50 AM, chandresh pancholi <
> > > >>>>>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> Problem here is with Storm. Its making streamer
> > > >>> object
> > > >>>>>> null
> > > >>>>>>>>> once
> > > >>>>>>>>>> it
> > > >>>>>>>>>>>>> goes
> > > >>>>>>>>>>>>>>> into submit topology.
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> On Mon, Sep 21, 2015 at 7:13 PM, Lalit Kumar Jha <
> > > >>>>>>>>>>>> lalitj....@gmail.com
> > > >>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> wrote:
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> Hi Chandresh,
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> See test suite class here, its based on
> > > >> annotations
> > > >>
> > >
> >
> https://github.com/sylentprayer/ignite/blob/ignite-530/modules/twitter/src/test/java/org/apache/ignite/stream/twitter/IgniteTwitterStreamerTestSuite.java
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> No code required in class body.
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>> On Mon, Sep 21, 2015 at 6:24 PM, chandresh
> > > >>> pancholi <
> > > >>>>>>>>>>>>>>>> chandreshpancholi...@gmail.com> wrote:
> > > >>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Hi Gian/Vishal/Lalit,
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> You guys have worked on Streamer integration
> > > >> with
> > > >>>>>>> Ignite. I
> > > >>>>>>>>> am
> > > >>>>>>>>>>>>> working
> > > >>>>>>>>>>>>>> on
> > > >>>>>>>>>>>>>>>>> Storm-ignite integration.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> I am setting up setStreamer() and setIgnite() in
> > > >>>>> Test
> > > >>>>>>> class
> > > >>>>>>>>> and
> > > >>>>>>>>>>>>> sending
> > > >>>>>>>>>>>>>>>>> this object to setBolt() method.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> When i try to read via gettreamer() and
> > > >>> getIgnite()
> > > >>>>> i
> > > >>>>>> am
> > > >>>>>>>>>> getting
> > > >>>>>>>>>>>>> NULL.
> > > >>>>>>>>>>>>>> I
> > > >>>>>>>>>>>>>>>> am
> > > >>>>>>>>>>>>>>>>> trying to find out why is it happening but No
> > > >>> Luck.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> If any one of you find some time and look into
> > > >> the
> > > >>>>> code
> > > >>>>>>> and
> > > >>>>>>>>>> help
> > > >>>>>>>>>>> me
> > > >>>>>>>>>>>>> to
> > > >>>>>>>>>>>>>>>>> resolve this. It would be great.
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> Git :
> > > >>> https://github.com/chandresh-pancholi/ignite
> > > >>>>>>>>>>>>>>>>> Module : storm
> > > >>>>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>>>> Chandresh Pancholi
> > > >>>>>>>>>>>>>>>>> Senior Software Engineer
> > > >>>>>>>>>>>>>>>>> Flipkart.com
> > > >>>>>>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> > > >>>>>>>>>>>>>>>>> Contact:08951803660
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>>
> > > >>>>>>>>>>>>>>> --
> > > >>>>>>>>>>>>>>> Chandresh Pancholi
> > > >>>>>>>>>>>>>>> Senior Software Engineer
> > > >>>>>>>>>>>>>>> Flipkart.com
> > > >>>>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> > > >>>>>>>>>>>>>>> Contact:08951803660
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>>
> > > >>>>>>>>>>>>> --
> > > >>>>>>>>>>>>> Chandresh Pancholi
> > > >>>>>>>>>>>>> Senior Software Engineer
> > > >>>>>>>>>>>>> Flipkart.com
> > > >>>>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> > > >>>>>>>>>>>>> Contact:08951803660
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> --
> > > >>>>>>>>>>> Chandresh Pancholi
> > > >>>>>>>>>>> Senior Software Engineer
> > > >>>>>>>>>>> Flipkart.com
> > > >>>>>>>>>>> Email-id:chandresh.panch...@flipkart.com
> > > >>>>>>>>>>> Contact:08951803660
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> --
> > > >>>>>>>> Chandresh Pancholi
> > > >>>>>>>> Senior Software Engineer
> > > >>>>>>>> Flipkart.com
> > > >>>>>>>> Email-id:chandresh.panch...@flipkart.com
> > > >>>>>>>> Contact:08951803660
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> --
> > > >>>>>> Chandresh Pancholi
> > > >>>>>> Senior Software Engineer
> > > >>>>>> Flipkart.com
> > > >>>>>> Email-id:chandresh.panch...@flipkart.com
> > > >>>>>> Contact:08951803660
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> --
> > > >>>> Chandresh Pancholi
> > > >>>> Senior Software Engineer
> > > >>>> Flipkart.com
> > > >>>> Email-id:chandresh.panch...@flipkart.com
> > > >>>> Contact:08951803660
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Chandresh Pancholi
> > > >> Senior Software Engineer
> > > >> Flipkart.com
> > > >> Email-id:chandresh.panch...@flipkart.com
> > > >> Contact:08951803660
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Chandresh Pancholi
> > Senior Software Engineer
> > Flipkart.com
> > Email-id:chandresh.panch...@flipkart.com
> > Contact:08951803660
> >
>
--
Chandresh Pancholi
Senior Software Engineer
Flipkart.com
Email-id:chandresh.panch...@flipkart.com
Contact:08951803660
