Looks like the requests to create volume has increased significantly. Heketi can handle 8 parallel volume create requests, and looks like there are 429 volume create requests pending.
I am not an expert in this. Added few more people in CC to help when they get to see this. -Amar On Thu, Jan 24, 2019 at 11:28 AM Shaik Salam <[email protected]> wrote: > Hi Surya, > > Could you please help us to resolve below issue (at lease workaround for > creating volume) > Attached db dump and log. Please let me know any other things need to > check. > Please guide us. > > BR > Salam > > > > From: Shaik Salam/HYD/TCS > To: "Amar Tumballi Suryanarayan" <[email protected]>, > [email protected], "[email protected] List" < > [email protected]> > Cc: "Murali Kottakota" <[email protected]>, "Sanju Rakonde" > <[email protected]> > Date: 01/23/2019 06:19 PM > Subject: Failed to provision volume with StorageClass > "glusterfs-storage": glusterfs: create volume err: error creating volume > ------------------------------ > > > > > > Hi, > > We are facing also following issue on openshift origin while we are > creating pvc for pods. > > Failed to provision volume with StorageClass "glusterfs-storage": > glusterfs: create volume err: error creating volume > Failed to provision volume with StorageClass "glusterfs-storage": > glusterfs: create volume err: error creating volume Server busy. Retry > operation later.. > > Please find heketidb dump and log > > [negroni] Completed 429 Too Many Requests in 250.763µs > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:07:49 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 169.08µs > [negroni] Started DELETE /volumes/520bc5f4e1bfd029855a72f9ca7ebf6c > [negroni] Completed 404 Not Found in 148.125µs > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:08:04 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 496.624µs > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:08:04 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 101.673µs > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:08:19 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 209.681µs > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:08:19 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 103.595µs > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:08:34 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 297.594µs > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:08:34 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 96.75µs > [negroni] Started POST /volumes > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:08:49 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 477.007µs > [heketi] WARNING 2019/01/23 12:08:49 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 165.38µs > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:09:04 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 488.253µs > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:09:04 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 171.836µs > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:09:19 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 208.59µs > [negroni] Started POST /volumes > [heketi] WARNING 2019/01/23 12:09:19 operations in-flight (8) exceeds > limit (8) > [negroni] Completed 429 Too Many Requests in 125.141µs > [negroni] Started DELETE /volumes/99e87ecd0a816ac34ae5a04eabc1d606 > [negroni] Completed 404 Not Found in 138.687µs > [negroni] Started POST /volumes > > > BR > Salam > > =====-----=====-----===== > Notice: The information contained in this e-mail > message and/or attachments to it may contain > confidential or privileged information. If you are > not the intended recipient, any dissemination, use, > review, distribution, printing or copying of the > information contained in this e-mail message > and/or attachments to it are strictly prohibited. If > you have received this communication in error, > please notify us by reply e-mail or telephone and > immediately and permanently delete the message > and any attachments. Thank you > > -- Amar Tumballi (amarts)
_______________________________________________ Gluster-users mailing list [email protected] https://lists.gluster.org/mailman/listinfo/gluster-users
