Do you have more details on the resource you were running out of on the
Tomcast containers? Were you CPU bound? Perhaps K8s is limiting the number
of CPU's available in a different fashion than what docker was.
Could you include the Tomcat iostat, vmstat, etc. output?
-EJ
On Tue, Sep 19, 2017 at 10:16 PM, Vinoth Narasimhan <talk2.v...@gmail.com>
wrote:
> Tim u mean the backend for k8s node is same as the result of the backend
> on native tomcat test as well as on the docker.
>
> k8s node backend is different that the tomcat and docker test.
>
>
> Tomcat/docker test did on the GCP machine with ubuntu flavour with
> 8cpu/30Gig machine
>
> k8s test did on 3 node k8s backend with default cos (container optimised
> OS).
>
>
>
> On Tuesday, September 19, 2017 at 9:59:52 PM UTC+5:30, Tim Hockin wrote:
>>
>> NodePort vs VIP should have no difference - they traverse the same paths.
>>
>> This is a much steeper difference than what I measured and more than I
>> would expect.
>>
>> Is this 8k new connections per second? Could you be exhausting
>> conntrack records and getting some failures? It would be interesting
>> to distinguish connections per second vs request throughput. We
>> should also clarify whether this is to a single backend on the same
>> node, or if this is across multiple backends and nodes. Testing to a
>> backend on the same node should, of course, be faster than testing a
>> backend on a different node.
>>
>> On Tue, Sep 19, 2017 at 8:55 AM, Warren Strange
>> <warren....@gmail.com> wrote:
>> >
>> > Debugging performance issues on Docker/Kube can be interesting....
>> >
>> > You could try exposing the service through a Nodeport, and run your
>> > benchmark directly against the node IP. That would at least tell you if
>> the
>> > GKE LB is a factor or not.
>> >
>> > Also - are your pods possibly CPU or memory limited (i.e, have you
>> > explicitly set resource limits - making Kube throttle your pods?)
>> >
>> >
>> > Please share your findings!
>> >
>> >
>> > On Tuesday, September 19, 2017 at 12:25:05 AM UTC-6, Vinoth Narasimhan
>> > wrote:
>> >>
>> >> Environment:
>> >>
>> >> Kubernetes version (use kubectl version):
>> >> kubectl version
>> >> Client Version: version.Info{Major:"1", Minor:"7",
>> GitVersion:"v1.7.3",
>> >> GitCommit:"2c2fe6e8278a5db2d15a013987b53968c743f2a1",
>> GitTreeState:"clean",
>> >> BuildDate:"2017-08-03T07:00:21Z", GoVersion:"go1.8.3", Compiler:"gc",
>> >> Platform:"linux/amd64"}
>> >> Server Version: version.Info{Major:"1", Minor:"6",
>> GitVersion:"v1.6.9",
>> >> GitCommit:"a3d1dfa6f433575ce50522888a0409397b294c7b",
>> GitTreeState:"clean",
>> >> BuildDate:"2017-08-23T16:58:45Z", GoVersion:"go1.7.6", Compiler:"gc",
>> >> Platform:"linux/amd64"}
>> >>
>> >> Cloud provider or hardware configuration**:
>> >>
>> >> Google Container Engine.
>> >>
>> >> What happened:
>> >>
>> >> We are in testing phase of springboot based microservice deployment on
>> >> GKE. During testing QA filed a performance issue , stats that the
>> throughput
>> >> of the service in k8s is low when compared to run the java app in
>> >>
>> >> java -jar method
>> >> docker run
>> >> For testing i skip those springboot stuff and take native tomcat home
>> page
>> >> as the test bed for the "ab" testing.
>> >>
>> >> The test setup looks like:
>> >>
>> >> Create an 8cpu/30Gig RAM ubuntu server in GCP and install native
>> >> tomact-8.5.20(80) and test the home page.
>> >>
>> >> Stop the native tomcat. Create the docker tomcat instances on the same
>> >> host and test the same home page.
>> >> The docker version is: Version: 17.06.2-ce
>> >>
>> >> Create the 3 node K8s cluster 1.6.9. Run the tomcat deployment the
>> same
>> >> 8.5.20 and expose the service through LB and test the same home page.
>> >>
>> >> I install the ab tool in other GCP instances and hit the above 3
>> different
>> >> endpoints.
>> >>
>> >> What's the Result:
>> >>
>> >> The first 2 test with native tomcat and docker run the throughput i
>> got is
>> >> nearly 8k Req/sec on avg on different request/concurrent level.
>> >>
>> >> But the same on K8s LB the throughput i got on the average of 2k
>> req/sec
>> >> on avg on different request/concurrency level.
>> >>
>> >> Is this something am i missing on the test. Or this is how the GKE LB
>> >> store and forward the request at this rate.
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "Kubernetes user discussion and Q&A" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an
>> > email to kubernetes-use...@googlegroups.com.
>> > To post to this group, send email to kubernet...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/kubernetes-users.
>> > For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Kubernetes user discussion and Q&A" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kubernetes-users+unsubscr...@googlegroups.com.
> To post to this group, send email to kubernetes-users@googlegroups.com.
> Visit this group at https://groups.google.com/group/kubernetes-users.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Kubernetes user discussion and Q&A" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to kubernetes-users+unsubscr...@googlegroups.com.
To post to this group, send email to kubernetes-users@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-users.
For more options, visit https://groups.google.com/d/optout.
