> On Feb 15, 2018, at 1:30 PM, kedar mhaswade <kedar.mhasw...@gmail.com> wrote: > > This appears useful. > > Will Runtime.getRuntime().availableProcessors() return the processors > available to the container after this integration, or will a new API be > provided? I have seen thread pools being misconfigured (e.g. it returns the > number of processors available on the host which is far more than that of > processors allotted by cgroups to the container) because of the confusion (or > because of just using the non-container-aware code).
I fixed your above problem in JDK 10 (https://bugs.openjdk.java.net/browse/JDK-8146115 <https://bugs.openjdk.java.net/browse/JDK-8146115>). This work exports much more detail about the container to monitoring tools. Bob. > > Regards, > Kedar > > On Thu, Feb 15, 2018 at 10:04 AM, Bob Vandette <bob.vande...@oracle.com > <mailto:bob.vande...@oracle.com>> wrote: > > > On Feb 15, 2018, at 12:52 PM, Volker Simonis <volker.simo...@gmail.com > > <mailto:volker.simo...@gmail.com>> wrote: > > > > Sounds cool! > > > > Is this JEP only about providing the corresponding API or also about > > using it internally (within HotSpot and class library) to better adopt > > to environment the JVM is running in? > > Thanks Volker. > > I integrated JDK-8146115 into JDK 10 which allows Hotspot to adapt to the > container it’s running in. The configuration that is examined includes > memory limits, > cpu count including cpu shares and quotas. > > This JEP’s main focus is to provide an internal API that can be used by JDK > tools > such as JFR and JMX to export container statistics and configuration data. > This JEP > does not include the JFR and JMX implementations. They will hopefully follow > shortly after. > > Bob. > > > > > Either way, looking forward to see (and test) the first implementation bits! > > > > Regards, > > Volker > > > > > > On Thu, Feb 15, 2018 at 6:07 PM, Bob Vandette <bob.vande...@oracle.com > > <mailto:bob.vande...@oracle.com>> wrote: > >> I’d like to re-propose the following JEP that will enhance the Java > >> runtime to be more container aware. > >> This will add an Internal Java API that will provide container specific > >> statistics. Some of the initial goals > >> of the previous JEP proposal has been integrated into JDK 10 under an RFE > >> (JDK-8146115). > >> This JEP is now focused on providing a Java API that exports Container > >> runtime configuration and metrics. > >> > >> Since the scope of this JEP have changed, I’m re-submitting it for comment > >> and endorsement. > >> > >> > >> JEP Issue: > >> > >> https://bugs.openjdk.java.net/browse/JDK-8182070 > >> <https://bugs.openjdk.java.net/browse/JDK-8182070> > >> > >> Here’s a Text dump of the JEP contents for your convenience: > >> > >> Summary > >> ------- > >> > >> Container aware Java runtime > >> > >> Goals > >> ----- > >> > >> Provide an internal API that can be used to extract container specific > >> configuration and runtime statistics. This JEP will only support Docker > >> on Linux-x64 although the design should be flexible enough to allow > >> support for other platforms and container technologies. The initial focus > >> will be on Linux cgroups technology so that we will be able to easily > >> support other container technologies running on Linux in addition to > >> Docker. > >> > >> Non-Goals > >> --------- > >> > >> It is not a goal of this JEP to support any platform other than Docker > >> container technology running on Linux x64. > >> > >> Success Metrics > >> --------------- > >> > >> Success will be measured by the improvement in information that will be > >> available to tools which visualize resource usage of containers that are > >> running Java processes. > >> > >> Motivation > >> ---------- > >> > >> Container technology is becoming more and more prevalent in Cloud based > >> applications. The Cloud Serverless application programming model > >> motivates developers to split large monolithic applications into 100s of > >> smaller pieces each running in thier own container. This move increases > >> the importance of the observability of each running container process. > >> Adding the proposed set of APIs will allow more details related to each > >> container process to be made available to external tools thereby improving > >> the observability. > >> > >> Description > >> ----------- > >> > >> This enhancement will be made up of the following work items: > >> > >> A. Detecting if Java is running in a container. > >> > >> The Java runtime, as well as any tests that we might write for this > >> feature, will need to be able to detect that the current Java process is > >> running in a container. A new API will be made available for this purpose. > >> > >> B. Exposing container resource limits, configuration and runtime > >> statistics. > >> > >> There are several configuration options and limits that can be imposed > >> upon a running container. Not all of these > >> are important to a running Java process. We clearly want to be able to > >> detect how many CPUs have been allocated to our process along with the > >> maximum amount of memory that the process has been allocated but there are > >> other options that we might want to base runtime decisions on. > >> > >> In addition, since Container typically impose limits on system resources, > >> they also provide the ability to easily access the amount of consumption > >> of these resources. The goal is to provide this information in addition > >> to the configuration data. > >> > >> I propose adding a new jdk.internal.Platform class that will allow access > >> to this information. > >> > >> Here are some of the types of configuration and consumption statistics > >> that would be made available: > >> > >> isContainerized > >> Memory Limit > >> Total Memory Limit > >> Soft Memory Limit > >> Max Memory Usage > >> Current Memory Usage > >> Maximum Kernel Memory > >> CPU Shares > >> CPU Period > >> CPU Quota > >> Number of CPUs > >> CPU Sets > >> CPU Set Memory Nodes > >> CPU Usage > >> CPU Usage Per CPU > >> Block I/O Weight > >> Block I/O Device Weight > >> Device I/O Read Rate > >> Device I/O Write Rate > >> OOM Kill Enabled > >> OOM Score Adjustment > >> Memory Swappiness > >> Shared Memory Size > >> > >> Alternatives > >> ------------ > >> > >> There are a few existing tools available to extract some of the same > >> container statistics. These tools could be used instead. The benefit of > >> providing a core Java internal API is that this information can be expose > >> by current Java serviceability tools such as JMX and JFR along side other > >> JVM specific information. > >> > >> Testing > >> ------- > >> > >> Docker/container specific tests should be added in order to validate the > >> functionality being provided with this JEP. > >> > >> Risks and Assumptions > >> --------------------- > >> > >> Docker is currently based on cgroups v1. Cgroups v2 is also available but > >> is incomplete and not yet supported by Docker. It's possible that v2 could > >> replace v1 in an incompatible way rendering this work unusable until it is > >> upgraded. > >> > >> Other alternative container technologies based on hypervisors are being > >> developed that could replace the use of cgroups for container isloation. > >> > >> Dependencies > >> ----------- > >> > >> None at this time. > >> > >
