Ah, Satish you got the point. I meant copied version of files in supervisor, but itself can be isolated. I didn't think about removing blobs, and it seems not easy to do.
Jungtaek Lim (HeartSaVIoR) 2016년 8월 2일 (화) 오후 7:35, Satish Duggana <[email protected]>님이 작성: > Hi Jungtaek, > With the current proposal, are we removing blob store files referred by a > topology when it is killed? > > Thanks, > Satish. > > On Tue, Aug 2, 2016 at 3:50 PM, Jungtaek Lim <[email protected]> wrote: > > > Hi Satish, > > > > Thanks for reviewing and share your idea. > > > > Yes this is shared dependencies vs isolated dependencies. > > If we name file of dependency to contain group name, artifact name, and > > version, that can be shared. > > One downside of this approach is storage space since we don't know when > > it's safe to delete without additional care, but I'm curious that disk > > fills up due to dependency blob jar files in normal situation. > > So I think we're OK to do this but I would like to see others opinions. > > > > Btw, I'm designing details based on proposal. Will update to this thread > if > > there're not covered things with initial design. > > > > Thanks, > > Jungtaek Lim (HeartSaVioR) > > > > 2016년 8월 2일 (화) 오후 6:58, Satish Duggana <[email protected]>님이 작성: > > > > > Hi Jungtaek, > > > Proposal looks good to me. Good that we are not going with other > > > alternative using mutable classloader etc. > > > > > > Good to have the mentioned config in proposal to add those jars before > or > > > after storm core/libs. There is a property Config. > > > TOPOLOGY_CLASSPATH_BEGINNING which is to have that value as initial > > > classpath and that should continue to be working as expected even with > > the > > > new configuration. > > > > > > One enhancement which we may want to add to the existing proposal. > > > When --packages are used, storm submitter can upload those dependencies > > in > > > blob store with a defined naming convention so that same set of > packages > > > are not uploaded again and they can be used again for other topologies > if > > > they use same package. > > > > > > Thanks, > > > Satish. > > > > > > > > > On Tue, Aug 2, 2016 at 7:25 AM, Jungtaek Lim <[email protected]> > wrote: > > > > > > > Hi dev, > > > > > > > > This is proposal review thread for submitting topology with adding > jars > > > and > > > > maven artifacts. This is also following up discussion thread for > > > > [DISCUSSION] > > > > Policy of resolving dependencies for non storm-core modules.[1] > > > > > > > > I've written design doc which also describes motivation on this. > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/STORM/A.+Design+doc%3A+adding+jars+and+maven+artifacts+at+submission > > > > > > > > Please review this and comment to "this thread" instead of wiki page > so > > > > that all devs can be notified for the update. > > > > > > > > Thanks, > > > > Jungtaek Lim (HeartSaVioR) > > > > > > > > [1] > > > > > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/storm-dev/201607.mbox/%3CCAF5108jByyJLTKrV_P4fS=dj8rsr_o5oubzqbviscggsc1c...@mail.gmail.com%3E > > > > > > > > > >
