So as an end-user where exactly do I add my new storage plugin configurations if I choose to add them in the future?
--- Mufeed Usman My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400> | My Social Cause <http://www.vision2016.org.in/> | My Blogs : LiveJournal <http://mufeed.livejournal.com> On Sat, Jun 28, 2014 at 3:17 AM, Aditya <[email protected]> wrote: > To be more precise, the configuration file "storage-plugins.json" has been > replaced with "bootstrap-storage-plugins.json" which resides inside the > Drill jars. > > When a cluster is started for the first time and no existing storage plugin > definition is found in the Zookeeper, this file will be read and copied > into the Zookeeper. From that point onward, the all Drillbits in the > cluster will use only the information stored in the ZK and would ignore " > bootstrap-storage-plugins.json" as well. > > > On Fri, Jun 27, 2014 at 2:36 PM, Aditya <[email protected]> wrote: > > > The configuration file "storage-plugins.json" is no longer used by Drill. > > > > Any change in the plugin configuration is stored only in the persistent > > store (Zookeeper by default). > > > > > > On Fri, Jun 27, 2014 at 12:09 AM, mufy <[email protected]> wrote: > > > >> Hello Anil, > >> > >> Hmm... so the ZK will have the info. Agreed. But what about the > statement > >> where it says that any new storage plugin added/updated will be > broadcast > >> to all the nodes running drillbit service. I took that statement to > mean - > >> the config file will be synchronized across all the nodes running the > >> drillbit service. Did I get it wrong? > >> > >> > >> --- > >> Mufeed Usman > >> My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400> | My > >> Social Cause <http://www.vision2016.org.in/> | My Blogs : LiveJournal > >> <http://mufeed.livejournal.com> > >> > >> > >> > >> > >> > >> On Fri, Jun 27, 2014 at 11:57 AM, AnilKumar B <[email protected]> > >> wrote: > >> > >> > Hi Murfy, > >> > > >> > Based on my understanding, registered storage plugin info will be > >> stored on > >> > ZK. As below. > >> > > >> > [zk: localhost:2181(CONNECTED) 10] ls /drill/sys.storage_plugins > >> > [hive, cp, dfs, hbase] > >> > [zk: localhost:2181(CONNECTED) 11] get > /drill/sys.storage_plugins/hbase > >> > { > >> > "type" : "hbase", > >> > "config" : { > >> > "hbase.zookeeper.quorum" : "localhost", > >> > "hbase.zookeeper.property.clientPort" : "2181" > >> > }, > >> > "enabled" : true > >> > } > >> > cZxid = 0x428 > >> > ctime = Tue Jun 24 13:30:18 IST 2014 > >> > mZxid = 0x450 > >> > mtime = Tue Jun 24 13:31:41 IST 2014 > >> > pZxid = 0x428 > >> > cversion = 0 > >> > dataVersion = 1 > >> > aclVersion = 0 > >> > ephemeralOwner = 0x0 > >> > dataLength = 157 > >> > numChildren = 0 > >> > [zk: localhost:2181(CONNECTED) 12] get /drill/sys.storage_plugins/cp > >> > { > >> > "type" : "file", > >> > "enabled" : true, > >> > "connection" : "classpath:///", > >> > "workspaces" : null, > >> > "formats" : null > >> > } > >> > cZxid = 0x426 > >> > ctime = Tue Jun 24 13:30:18 IST 2014 > >> > mZxid = 0x426 > >> > mtime = Tue Jun 24 13:30:18 IST 2014 > >> > pZxid = 0x426 > >> > cversion = 0 > >> > dataVersion = 0 > >> > aclVersion = 0 > >> > ephemeralOwner = 0x0 > >> > dataLength = 118 > >> > numChildren = 0 > >> > [zk: localhost:2181(CONNECTED) 13] > >> > > >> > Thanks, > >> > B Anil Kumar. > >> > > >> > Thanks & Regards, > >> > B Anil Kumar. > >> > > >> > > >> > On Fri, Jun 27, 2014 at 11:12 AM, mufy <[email protected]> > wrote: > >> > > >> > > Stated here > >> > > > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/DRILL/Connecting+Apache+Drill+to+Data+Sources > >> > > that, > >> > > > >> > > "When you add or update storage plugin instances on one node, Drill > >> > > broadcasts the information to all of the other Drill nodes so they > >> have > >> > > identical storage plugin instances registered. You do not need to > >> restart > >> > > any of the Drillbits when you add or update a storage plugin > >> instance." > >> > > > >> > > Does that mean if I were to have the following on one node: > >> > > [root@mufeed-drill1 ~]# cat > >> > > > >> > > > >> > > >> > /opt/mapr/drill/drill-1.0.0/apache-drill-1.0.0-m2-incubating-SNAPSHOT/conf/storage-plugins.json > >> > > { > >> > > "storage": { > >> > > > >> > > dfs: { > >> > > type: "file", > >> > > connection: "maprfs:///" > >> > > }, > >> > > > >> > > cp: { > >> > > type: "file", > >> > > connection: "classpath:///" > >> > > }, > >> > > > >> > > cp1: { > >> > > type: "file", > >> > > connection: "classpath:///" > >> > > }, > >> > > } > >> > > } > >> > > > >> > > The same content would get replicated to other nodes, on a file > >> content > >> > > level? If yes, I do not see that happening. If not, please clarify. > >> > > > >> > > --- > >> > > Mufeed Usman > >> > > My LinkedIn <http://www.linkedin.com/pub/mufeed-usman/28/254/400> | > >> My > >> > > Social Cause <http://www.vision2016.org.in/> | My Blogs : > LiveJournal > >> > > <http://mufeed.livejournal.com> > >> > > > >> > > >> > > > > >
