Hi Lei, yes, metaspace would run out eventually if you run too much in parallel. All finished jobs will close the classloaders and free the metaspace memory.
For newer setups, we recommend creating an ad-hoc cluster for each Flink application for this and several other reasons. If you are already using K8s, I'd definitively switch to per-application clusters. You could also have a look at the Ververica community editions for easy managing of such applications. [1] [1] https://www.ververica.com/getting-started On Fri, Mar 12, 2021 at 4:15 AM Lei Wang <leiwang...@gmail.com> wrote: > Thanks Arvid, > > If too many jobs run in the same task manager JVM, will it cause too > much metaspace memory occupation? > > > Thanks, > Lei > > On Thu, Mar 11, 2021 at 11:54 PM Arvid Heise <ar...@apache.org> wrote: > >> Hi Lei, >> >> each application has its own classloader as such each static constant >> exists multiple times (1 per job). So there should be no interference. You >> could verify it by logging the value of the constant and see it yourself. >> >> Best, >> >> Arvid >> >> On Thu, Mar 11, 2021 at 7:11 AM Lei Wang <leiwang...@gmail.com> wrote: >> >>> Consider the following situation. >>> >>> Two jobs builed in the same jar, they will share some common code, for >>> example, some static constants variables. >>> Currently they are running on the same task manager process. >>> >>> I killed job1, changed the static variable to another and then submit >>> it again. >>> Does another job will get the new value of the static variable or still >>> use the old value? >>> >>> Thanks, >>> Lei >>> >>> >>>
