Hi Alexei, Thanks but I copied the celery.py file from the Arches template folder so it already had those lines in it. My file now looks like yours from the earlier reply.
The issue is that the actual management command that starts celery never uses that app variable. Instead it uses the elastic search prefix — see lines 41 and 43. Because my elastic search prefix is different from my app name, I get an error. Can you please try retesting by setting your elastic search prefix setting to something that doesn’t equal your project name to see if you hit the same error as me? Thanks, Angela On Thu, Feb 11, 2021 at 8:04 PM Alexei Peters <[email protected]> wrote: > Hi Angela, > After you copy that file over, you'll need to edit a couple of lines in > that file: > > THESE LINES: > os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings") > app = Celery("arches") > > SHOULD CHANGE TO THIS: > os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{my project > name}.settings") > app = Celery("{my project name}") > > Hope that helps! > -Alexei > > Director of Web Development - Farallon Geographics, Inc. - 971.227.3173 > > > On Wed, Feb 10, 2021 at 12:22 PM Angela Labrador <[email protected]> wrote: > >> Hi Alexei, >> >> I'm back! I have copied the new celery.py file over into my project >> directory as suggested above, however, I still get the same error. >> Error: >> Unable to load celery application. >> The module XXX was not found. >> >> where XXX = {settings.ELASTICSEARCH_PREFIX} >> >> The only celery management command >> <https://github.com/archesproject/arches/blob/stable/5.1.x/arches/management/commands/celery.py> >> that I have is in the arches/arches/management/commands/ directory, and I >> see that is using the elasticsearch_prefix variable on lines 41 and 43 and >> not the project name. When I edit those lines manually and change them to >> use my project's name, I can run the command fine. >> >> Am I supposed to have a separate celery management command within my >> project folder or is this a bug with the arches management command? >> >> Thanks, >> Angela >> >> >> On Friday, January 29, 2021 at 5:44:25 PM UTC-5 Alexei Peters wrote: >> >>> Hi Angela, >>> It looks like you identified the issue. Older projects weren't created >>> with that celery file and that explains the problem you're seeing. >>> Copying that file over to your project should solve the problem. >>> This was most likely an oversight in our upgrade announcement (although >>> I can't be sure). I'm actually not sure how/if this should be documented. >>> Suggestions are welcome there! :) >>> Glad we solved this! >>> Cheers, >>> Alexei >>> >>> Director of Web Development - Farallon Geographics, Inc. - 971.227.3173 >>> <(971)%20227-3173> >>> >>> On Fri, Jan 29, 2021 at 12:17 PM Angela Labrador <[email protected]> >>> wrote: >>> >>>> Hi Alexei, >>>> >>>> Yep, I'm running the command local to where the manage.py file is >>>> located (running the command just like any other python manage.py command >>>> with no dir in front of the command). >>>> >>>> But -- I think you just identified the problem! I do not have a >>>> celery.py file within my-project folder besides the celery.py files in >>>> media/packages/arches. This is a project that has been migrated from 4.0 >>>> through the major versions over some time. So, maybe it never created the >>>> celery.py file during one of those migrations? >>>> >>>> I jumped on another virtual machine and installed a brand new arches >>>> 5.1 project, and it did create a celery.py file with my project name as you >>>> pasted. >>>> >>>> Just for reference, on my migrated project, running >>>> find -iname celery.py >>>> >>>> results in: >>>> >>>> ./[my-project]/[my-project]/media/packages/arches/arches/management/commands/celery.py >>>> ./[my-project]/[my-project]/media/packages/arches/arches/celery.py >>>> >>>> ./[my-project]/static/packages/arches/tests/fixtures/testing_prj/testing_prj/celery.py >>>> >>>> ./[my-project]/static/packages/arches/arches/management/commands/celery.py >>>> ./[my-project]/static/packages/arches/arches/celery.py >>>> ./env/lib/python3.7/site-packages/celery/bin/celery.py >>>> ./arches/tests/fixtures/testing_prj/testing_prj/celery.py >>>> ./arches/arches/management/commands/celery.py >>>> ./arches/arches/celery.py >>>> >>>> So, should I just copy the contents of the celery.py file from the >>>> fresh project to my migrated project? And do you think this is an anomaly >>>> or a "feature" of migrating from much earlier versions? If the latter, >>>> should we add something to the docs to cover this? >>>> >>>> Thank you for sticking with this, >>>> Angela >>>> >>>> >>>> >>>> On Friday, January 29, 2021 at 2:39:06 PM UTC-5 Alexei Peters wrote: >>>> >>>>> Hi Angela, >>>>> Just to be clear the command should be run from the directory where >>>>> the "manage.py" file is located. >>>>> EG: you CAN'T do this --> python somedir/manage.py celery start >>>>> >>>>> Also, does your celery file look something like this? >>>>> >>>>> from __future__ import absolute_import, unicode_literals >>>>> import os >>>>> from celery import Celery >>>>> >>>>> os.environ.setdefault('DJANGO_SETTINGS_MODULE', >>>>> '{project_name}.settings') >>>>> app = Celery('{project_name}') >>>>> app.config_from_object('django.conf:settings', namespace='CELERY') >>>>> app.autodiscover_tasks() >>>>> >>>>> >>>>> where {project_name} is the name of the directory where your project >>>>> exists >>>>> >>>>> Cheers, >>>>> Alexei >>>>> >>>>> Director of Web Development - Farallon Geographics, Inc. - >>>>> 971.227.3173 <(971)%20227-3173> >>>>> On Wed, Jan 27, 2021 at 7:31 AM Angela Labrador <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Alexei, >>>>>> >>>>>> Correct. I am running python manage.py celery start from within >>>>>> [my_project] folder, as per docs >>>>>> <https://arches.readthedocs.io/en/stable/task-management/?highlight=celery#running-celery>, >>>>>> which results in the >>>>>> Error: >>>>>> Unable to load celery application. >>>>>> The module [ my settings.ELASTICSEARCH_PREFIX ] was not found. >>>>>> >>>>>> My project folder structure is standard -- no renaming of anything. >>>>>> >>>>>> Thanks, >>>>>> Angela >>>>>> On Tuesday, January 26, 2021 at 3:44:15 PM UTC-5 Alexei Peters wrote: >>>>>> >>>>>>> Hi Angela, >>>>>>> I assume that you're running the "python manage.py celery start" >>>>>>> command from your project and not from the root of the Arches code base >>>>>>> correct? >>>>>>> The command also assumes that you haven't rename your folders in >>>>>>> your project directory. >>>>>>> -Alexei >>>>>>> >>>>>>> Director of Web Development - Farallon Geographics, Inc. - >>>>>>> 971.227.3173 <(971)%20227-3173> >>>>>>> >>>>>>> >>>>>>> On Mon, Jan 25, 2021 at 11:35 AM Angela Labrador <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Lindsay, >>>>>>>> >>>>>>>> Thanks for checking in -- no, I haven't really solved it. I've got >>>>>>>> a workaround, but would appreciate any ideas or explanations so I can >>>>>>>> understand how celery is supposed to be configured (or so that we can >>>>>>>> update the docs if they need to be edited). >>>>>>>> >>>>>>>> Angela >>>>>>>> >>>>>>>> >>>>>>>> On Tuesday, January 19, 2021 at 7:48:47 PM UTC-5 Lindsey Gant wrote: >>>>>>>> >>>>>>>>> Hi Angela, >>>>>>>>> >>>>>>>>> Were you able to find a solution to your celery error? If not, let >>>>>>>>> me see if someone in the group can follow up. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Lindsey >>>>>>>>> GCI Arches Project Team >>>>>>>>> >>>>>>>>> On Thursday, January 14, 2021 at 2:30:56 PM UTC-8 [email protected] >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hello all, >>>>>>>>>> >>>>>>>>>> I'm testing an upgrade to Arches 5.1 and have hit a stumbling >>>>>>>>>> block with trying to get celery running. I've followed the >>>>>>>>>> directions on the doc >>>>>>>>>> <https://arches.readthedocs.io/en/stable/task-management/?highlight=celery#running-celery>, >>>>>>>>>> and have RabbitMQ installed and my settings.py file updated. I did >>>>>>>>>> not >>>>>>>>>> create a tasks.py file within my project folder because the docs >>>>>>>>>> imply this >>>>>>>>>> is only needed if I need to add additional tasks for my project (I >>>>>>>>>> just >>>>>>>>>> need the default Arches tasks). >>>>>>>>>> >>>>>>>>>> When I run python manage.py celery start from within my >>>>>>>>>> virtualenv, I get >>>>>>>>>> >>>>>>>>>> Error: >>>>>>>>>> Unable to load celery application. >>>>>>>>>> The module XXX was not found. >>>>>>>>>> >>>>>>>>>> where XXX = my settings.ELASTICSEARCH_PREFIX >>>>>>>>>> >>>>>>>>>> However, when I edit the celery.py management command to use >>>>>>>>>> cmd = f"celery -A *arches* worker -B -l info" >>>>>>>>>> instead of settings.ELASTICSEARCH_PREFIX, celery starts up and >>>>>>>>>> connects to rabbitmq. >>>>>>>>>> >>>>>>>>>> I'm wondering why that management command is using the ES prefix >>>>>>>>>> and what I could be missing in my own project config that is causing >>>>>>>>>> this. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Angela >>>>>>>>>> >>>>>>>>> -- >>>>>>>> -- To post, send email to [email protected]. To >>>>>>>> unsubscribe, send email to [email protected]. For >>>>>>>> more information, visit >>>>>>>> https://groups.google.com/d/forum/archesproject?hl=en >>>>>>>> --- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Arches Project" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/archesproject/17891260-c650-469a-bdff-9e68f9b5b462n%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/archesproject/17891260-c650-469a-bdff-9e68f9b5b462n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>> -- To post, send email to [email protected]. To >>>>>> unsubscribe, send email to [email protected]. For >>>>>> more information, visit >>>>>> https://groups.google.com/d/forum/archesproject?hl=en >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Arches Project" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> >>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/archesproject/4a95537c-db9b-4935-8144-37e930706511n%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/archesproject/4a95537c-db9b-4935-8144-37e930706511n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>> -- To post, send email to [email protected]. To unsubscribe, >>>> send email to [email protected]. For more information, >>>> visit https://groups.google.com/d/forum/archesproject?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Arches Project" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/archesproject/78043179-6107-4ac3-b597-7184cb61a571n%40googlegroups.com >>>> <https://groups.google.com/d/msgid/archesproject/78043179-6107-4ac3-b597-7184cb61a571n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> -- To post, send email to [email protected]. To >> unsubscribe, send email to [email protected]. >> For more information, visit >> https://groups.google.com/d/forum/archesproject?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Arches Project" group. >> > To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. > > >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/archesproject/a116bc89-5359-4c30-a832-e58701240284n%40googlegroups.com >> <https://groups.google.com/d/msgid/archesproject/a116bc89-5359-4c30-a832-e58701240284n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- -- To post, send email to [email protected]. To unsubscribe, send email to [email protected]. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/archesproject/CAJ1rqujZ7Bm5gSQOHYXerVCaGfYob_r6QBxs7ozD9wc5anVvOg%40mail.gmail.com.
