You could queue the tasks in a broker (rabbitmq for example) and use a fixed number of celery workers to process them and save them to the database, assuming:
- you don't have 1000 requests per second all the time - you don't need the data to be stored immediately On 1/4/17, Avraham Serour <[email protected]> wrote: > can you elaborate on this scenario? how and why can your database shutdown? > > I second the suggestion about returning error and telling the client to try > later, but it should be >=500 not 400. 4xx Are for client errors, 5xx are > for server errors. see http://www.restapitutorial.com/httpstatuscodes.html > > On Wed, Jan 4, 2017 at 10:47 PM, Vinicius Assef <[email protected]> > wrote: > >> Or you could return a http status 400 if the transaction couldn't be >> committed. So, the client should re-run the request later. >> >> If this is not an option, you should lay upon a queue as you suggested. >> >> Bad things always can happen. So, try your best to avoid i/o problems >> if your clients cannot re-make requests in case of problems in your >> side. >> >> -- >> Vinicius Assef >> >> >> >> On 4 January 2017 at 14:39, Bruno Barbosa <[email protected]> wrote: >> > Hello everybody! >> > >> > I have an application that receives about 1000 POST requests per >> > second. >> On >> > each request I writte a line on my database table. On this scenario my >> > database can shutdown during execution. >> > >> > Is it a good pratice to store these data on array in cache (redis) and >> > on >> > each minute through celery get then and use a bulk_create to insert all >> data >> > on database? >> > >> > Bruno >> > -- >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Django users" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to [email protected]. >> > To post to this group, send email to [email protected]. >> > Visit this group at https://groups.google.com/group/django-users. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msgid/django-users/CAHxcCH5t7FWRoXbCFvUez- >> hAzLk0b08KideZ_D_A6oj6a_8L1A%40mail.gmail.com. >> > For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/django-users. >> To view this discussion on the web visit https://groups.google.com/d/ >> msgid/django-users/CAFmXjSDsYp44V8mVwkdwX-FVww%3D-NhYm4w-KnDZ4rw5SQo5owg% >> 40mail.gmail.com. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/django-users. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/CAFWa6tJQCkmJ%2BnM4nbw2snR0TguG7DPYvKLqN2__OtdFK314Dg%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CALn3ei0b6zf2WpjDe0dHk5j8e91SDvX33CXFW86MrH0Yo%3D-spA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
