Ah ok that's a nice idea, but in my situation it's like this: I want to initialize a database connection and also hold some database rows and other values in a class. Once these are initialized in a class, I don't want any other class to initialize it for the sake of saving memory and for referential integrity. It is this initialized object that I want all my Spouts to be able to access. That's why I was hoping that at least if Config could hold the class, then all Spouts and Bolts in the topology would be able to access the initialized values, because Config is part of the topology and can be accessed in open() and prepare().
On Thu, Apr 21, 2016 at 11:08 AM, Abhishek Agarwal <[email protected]> wrote: > In the prepare/open method, you can get the class name and initialize the > instance from class name > > Excuse typos > On Apr 21, 2016 9:17 AM, "Navin Ipe" <[email protected]> > wrote: > >> Thanks Abhishek, but I don't quite understand what you meant. Do you mean >> I pass a class name of a static class and then use it? >> >> Also took a look at the source code and this is what it says: >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> */** * Topology configs are specified as a plain old map. This class >> provides a * convenient way to create a topology config map by providing >> setter methods for * all the configs that can be set. It also makes it >> easier to do things like add * serializations. * * This class also provides >> constants for all the configurations possible on * a Storm cluster and >> Storm topology. Each constant is paired with an annotation * that defines >> the validity criterion of the corresponding field. Default * values for >> these configs can be found in defaults.yaml. * * Note that you may put >> other configurations in any of the configs. Storm * will ignore anything it >> doesn't recognize, but your topologies are free to make * use of them by >> reading them in the prepare method of Bolts or the open method of * >> Spouts. */public class Config extends HashMap<String, Object> {* >> >> So if Storm is supposed to ignore anything it does not recognize, then >> it's strange it's throwing an error for my serializable class. >> >> On Thu, Apr 21, 2016 at 12:01 AM, Abhishek Agarwal <[email protected]> >> wrote: >> >>> Pass the class name in config instead of concrete object. >>> >>> Excuse typos >>> On Apr 20, 2016 9:43 PM, "Navin Ipe" <[email protected]> >>> wrote: >>> >>>> Does anyone know how to do this? If not adding a custom class, then at >>>> least about how to maintain information retrieved from a database in a >>>> single class which all spouts/bolts in all the workers will be able to >>>> access? >>>> >>>> On Tue, Apr 19, 2016 at 2:52 PM, Navin Ipe < >>>> [email protected]> wrote: >>>> >>>>> I have this >>>>> >>>>> Config config = new Config(); >>>>> MongoDatabaseManager mongoManager = new MongoDatabaseManager(); >>>>> config.put("MongoManager", mongoManager); >>>>> >>>>> and MongoDatabaseManager is an empty class: >>>>> public class MongoDatabaseManager implements Serializable {} >>>>> >>>>> But after submitting the topology, I get this error: >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> *14:45:18.320 [timer] DEBUG o.a.s.scheduler.SupervisorDetails - >>>>> Creating a new supervisor (null-5b950364-129e-4e04-b10c-d1e496c12043) with >>>>> resources: {"supervisor.cpu.capacity" 400.0, >>>>> "supervisor.memory.capacity.mb" 3072.0}14:45:18.321 [main] ERROR >>>>> o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Thread Thread[main,5,main] >>>>> diedjava.lang.IllegalArgumentException: Topology conf is not >>>>> json-serializable at >>>>> org.apache.storm.testing$submit_local_topology.invoke(testing.clj:299) >>>>> ~[storm-core-1.0.0.jar:1.0.0] at >>>>> org.apache.storm.LocalCluster$_submitTopology.invoke(LocalCluster.clj:49) >>>>> ~[storm-core-1.0.0.jar:1.0.0] at >>>>> org.apache.storm.LocalCluster.submitTopology(Unknown Source) >>>>> ~[storm-core-1.0.0.jar:1.0.0] at >>>>> com.slh.Basicstorm.BasicStorm.main(BasicStorm.java:44) ~[main/:na]* >>>>> >>>>> Why can't I add a custom class to config? The idea here is to store >>>>> some info in the class so that all workers can access the information >>>>> (because the class is part of the topology) even though they are part of >>>>> different JVM's (the class can be accessed in the open() or prepare() >>>>> functions of spouts or bolts, but first, I don't understand why this crash >>>>> is happening). >>>>> >>>>> -- >>>>> Regards, >>>>> Navin >>>>> >>>> >>>> >>>> >>>> -- >>>> Regards, >>>> Navin >>>> >>> >> >> >> -- >> Regards, >> Navin >> > -- Regards, Navin
