Hi Alexei, Ok that workaround makes sense. Thanks for confirming! I will file an issue on GitHub for this too (unless you beat me to it).
Enjoy your weekend, Angela On Fri, Feb 12, 2021 at 8:14 PM Alexei Peters <[email protected]> wrote: > Hi Angela, > Sorry, I see now what you're talking about. I wasn't looking at the > management command. So, having that set to use the ELASTICSEARCH_PREFIX is > probably a mistake. > Instead of running the command you can run what the command runs. > > "celery -A {your project name} worker -l info" > > That should work. > Cheers, > -Alexei > > Director of Web Development - Farallon Geographics, Inc. - 971.227.3173 > > > On Thu, Feb 11, 2021 at 5:34 PM Angela Labrador <[email protected]> wrote: > >> Hi Alexei, >> >> Thanks but I copied the celery.py file from the Arches template folder so >> it already had those lines in it. My file now looks like yours from the >> earlier reply. >> >> The issue is that the actual management command that starts celery never >> uses that app variable. Instead it uses the elastic search prefix — see >> lines 41 and 43. Because my elastic search prefix is different from my app >> name, I get an error. Can you please try retesting by setting your elastic >> search prefix setting to something that doesn’t equal your project name to >> see if you hit the same error as me? >> >> Thanks, >> Angela >> >> On Thu, Feb 11, 2021 at 8:04 PM Alexei Peters <[email protected]> wrote: >> >>> Hi Angela, >>> After you copy that file over, you'll need to edit a couple of lines in >>> that file: >>> >>> THESE LINES: >>> os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings") >>> app = Celery("arches") >>> >>> SHOULD CHANGE TO THIS: >>> os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{my project >>> name}.settings") >>> app = Celery("{my project name}") >>> >>> Hope that helps! >>> -Alexei >>> >>> Director of Web Development - Farallon Geographics, Inc. - 971.227.3173 >>> >>> >>> On Wed, Feb 10, 2021 at 12:22 PM Angela Labrador <[email protected]> >>> wrote: >>> >>>> Hi Alexei, >>>> >>>> I'm back! I have copied the new celery.py file over into my project >>>> directory as suggested above, however, I still get the same error. >>>> Error: >>>> Unable to load celery application. >>>> The module XXX was not found. >>>> >>>> where XXX = {settings.ELASTICSEARCH_PREFIX} >>>> >>>> The only celery management command >>>> <https://github.com/archesproject/arches/blob/stable/5.1.x/arches/management/commands/celery.py> >>>> that I have is in the arches/arches/management/commands/ directory, and I >>>> see that is using the elasticsearch_prefix variable on lines 41 and 43 and >>>> not the project name. When I edit those lines manually and change them to >>>> use my project's name, I can run the command fine. >>>> >>>> Am I supposed to have a separate celery management command within my >>>> project folder or is this a bug with the arches management command? >>>> >>>> Thanks, >>>> Angela >>>> >>>> >>>> On Friday, January 29, 2021 at 5:44:25 PM UTC-5 Alexei Peters wrote: >>>> >>>>> Hi Angela, >>>>> It looks like you identified the issue. Older projects weren't >>>>> created with that celery file and that explains the problem you're seeing. >>>>> Copying that file over to your project should solve the problem. >>>>> This was most likely an oversight in our upgrade announcement >>>>> (although I can't be sure). I'm actually not sure how/if this should be >>>>> documented. Suggestions are welcome there! :) >>>>> Glad we solved this! >>>>> Cheers, >>>>> Alexei >>>>> >>>>> Director of Web Development - Farallon Geographics, Inc. - >>>>> 971.227.3173 <(971)%20227-3173> >>>>> >>>>> On Fri, Jan 29, 2021 at 12:17 PM Angela Labrador <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Alexei, >>>>>> >>>>>> Yep, I'm running the command local to where the manage.py file is >>>>>> located (running the command just like any other python manage.py command >>>>>> with no dir in front of the command). >>>>>> >>>>>> But -- I think you just identified the problem! I do not have a >>>>>> celery.py file within my-project folder besides the celery.py files in >>>>>> media/packages/arches. This is a project that has been migrated from 4.0 >>>>>> through the major versions over some time. So, maybe it never created the >>>>>> celery.py file during one of those migrations? >>>>>> >>>>>> I jumped on another virtual machine and installed a brand new arches >>>>>> 5.1 project, and it did create a celery.py file with my project name as >>>>>> you >>>>>> pasted. >>>>>> >>>>>> Just for reference, on my migrated project, running >>>>>> find -iname celery.py >>>>>> >>>>>> results in: >>>>>> >>>>>> ./[my-project]/[my-project]/media/packages/arches/arches/management/commands/celery.py >>>>>> ./[my-project]/[my-project]/media/packages/arches/arches/celery.py >>>>>> >>>>>> ./[my-project]/static/packages/arches/tests/fixtures/testing_prj/testing_prj/celery.py >>>>>> >>>>>> ./[my-project]/static/packages/arches/arches/management/commands/celery.py >>>>>> ./[my-project]/static/packages/arches/arches/celery.py >>>>>> ./env/lib/python3.7/site-packages/celery/bin/celery.py >>>>>> ./arches/tests/fixtures/testing_prj/testing_prj/celery.py >>>>>> ./arches/arches/management/commands/celery.py >>>>>> ./arches/arches/celery.py >>>>>> >>>>>> So, should I just copy the contents of the celery.py file from the >>>>>> fresh project to my migrated project? And do you think this is an anomaly >>>>>> or a "feature" of migrating from much earlier versions? If the latter, >>>>>> should we add something to the docs to cover this? >>>>>> >>>>>> Thank you for sticking with this, >>>>>> Angela >>>>>> >>>>>> >>>>>> >>>>>> On Friday, January 29, 2021 at 2:39:06 PM UTC-5 Alexei Peters wrote: >>>>>> >>>>>>> Hi Angela, >>>>>>> Just to be clear the command should be run from the directory where >>>>>>> the "manage.py" file is located. >>>>>>> EG: you CAN'T do this --> python somedir/manage.py celery start >>>>>>> >>>>>>> Also, does your celery file look something like this? >>>>>>> >>>>>>> from __future__ import absolute_import, unicode_literals >>>>>>> import os >>>>>>> from celery import Celery >>>>>>> >>>>>>> os.environ.setdefault('DJANGO_SETTINGS_MODULE', >>>>>>> '{project_name}.settings') >>>>>>> app = Celery('{project_name}') >>>>>>> app.config_from_object('django.conf:settings', namespace='CELERY') >>>>>>> app.autodiscover_tasks() >>>>>>> >>>>>>> >>>>>>> where {project_name} is the name of the directory where your project >>>>>>> exists >>>>>>> >>>>>>> Cheers, >>>>>>> Alexei >>>>>>> >>>>>>> Director of Web Development - Farallon Geographics, Inc. - >>>>>>> 971.227.3173 <(971)%20227-3173> >>>>>>> On Wed, Jan 27, 2021 at 7:31 AM Angela Labrador <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Alexei, >>>>>>>> >>>>>>>> Correct. I am running python manage.py celery start from within >>>>>>>> [my_project] folder, as per docs >>>>>>>> <https://arches.readthedocs.io/en/stable/task-management/?highlight=celery#running-celery>, >>>>>>>> which results in the >>>>>>>> Error: >>>>>>>> Unable to load celery application. >>>>>>>> The module [ my settings.ELASTICSEARCH_PREFIX ] was not found. >>>>>>>> >>>>>>>> My project folder structure is standard -- no renaming of anything. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Angela >>>>>>>> On Tuesday, January 26, 2021 at 3:44:15 PM UTC-5 Alexei Peters >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Angela, >>>>>>>>> I assume that you're running the "python manage.py celery start" >>>>>>>>> command from your project and not from the root of the Arches code >>>>>>>>> base >>>>>>>>> correct? >>>>>>>>> The command also assumes that you haven't rename your folders in >>>>>>>>> your project directory. >>>>>>>>> -Alexei >>>>>>>>> >>>>>>>>> Director of Web Development - Farallon Geographics, Inc. - >>>>>>>>> 971.227.3173 <(971)%20227-3173> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Jan 25, 2021 at 11:35 AM Angela Labrador <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Lindsay, >>>>>>>>>> >>>>>>>>>> Thanks for checking in -- no, I haven't really solved it. I've >>>>>>>>>> got a workaround, but would appreciate any ideas or explanations so >>>>>>>>>> I can >>>>>>>>>> understand how celery is supposed to be configured (or so that we can >>>>>>>>>> update the docs if they need to be edited). >>>>>>>>>> >>>>>>>>>> Angela >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tuesday, January 19, 2021 at 7:48:47 PM UTC-5 Lindsey Gant >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Angela, >>>>>>>>>>> >>>>>>>>>>> Were you able to find a solution to your celery error? If not, >>>>>>>>>>> let me see if someone in the group can follow up. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Lindsey >>>>>>>>>>> GCI Arches Project Team >>>>>>>>>>> >>>>>>>>>>> On Thursday, January 14, 2021 at 2:30:56 PM UTC-8 >>>>>>>>>>> [email protected] wrote: >>>>>>>>>>> >>>>>>>>>>>> Hello all, >>>>>>>>>>>> >>>>>>>>>>>> I'm testing an upgrade to Arches 5.1 and have hit a stumbling >>>>>>>>>>>> block with trying to get celery running. I've followed the >>>>>>>>>>>> directions on the doc >>>>>>>>>>>> <https://arches.readthedocs.io/en/stable/task-management/?highlight=celery#running-celery>, >>>>>>>>>>>> and have RabbitMQ installed and my settings.py file updated. I did >>>>>>>>>>>> not >>>>>>>>>>>> create a tasks.py file within my project folder because the docs >>>>>>>>>>>> imply this >>>>>>>>>>>> is only needed if I need to add additional tasks for my project (I >>>>>>>>>>>> just >>>>>>>>>>>> need the default Arches tasks). >>>>>>>>>>>> >>>>>>>>>>>> When I run python manage.py celery start from within my >>>>>>>>>>>> virtualenv, I get >>>>>>>>>>>> >>>>>>>>>>>> Error: >>>>>>>>>>>> Unable to load celery application. >>>>>>>>>>>> The module XXX was not found. >>>>>>>>>>>> >>>>>>>>>>>> where XXX = my settings.ELASTICSEARCH_PREFIX >>>>>>>>>>>> >>>>>>>>>>>> However, when I edit the celery.py management command to use >>>>>>>>>>>> cmd = f"celery -A *arches* worker -B -l info" >>>>>>>>>>>> instead of settings.ELASTICSEARCH_PREFIX, celery starts up and >>>>>>>>>>>> connects to rabbitmq. >>>>>>>>>>>> >>>>>>>>>>>> I'm wondering why that management command is using the ES >>>>>>>>>>>> prefix and what I could be missing in my own project config that >>>>>>>>>>>> is causing >>>>>>>>>>>> this. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Angela >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>> -- To post, send email to [email protected]. To >>>>>>>>>> unsubscribe, send email to [email protected]. For >>>>>>>>>> more information, visit >>>>>>>>>> https://groups.google.com/d/forum/archesproject?hl=en >>>>>>>>>> --- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "Arches Project" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to [email protected]. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/archesproject/17891260-c650-469a-bdff-9e68f9b5b462n%40googlegroups.com >>>>>>>>>> <https://groups.google.com/d/msgid/archesproject/17891260-c650-469a-bdff-9e68f9b5b462n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> -- >>>>>>>> -- To post, send email to [email protected]. To >>>>>>>> unsubscribe, send email to [email protected]. For >>>>>>>> more information, visit >>>>>>>> https://groups.google.com/d/forum/archesproject?hl=en >>>>>>>> --- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Arches Project" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> >>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/archesproject/4a95537c-db9b-4935-8144-37e930706511n%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/archesproject/4a95537c-db9b-4935-8144-37e930706511n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> >>>>>>> -- >>>>>> -- To post, send email to [email protected]. To >>>>>> unsubscribe, send email to [email protected]. For >>>>>> more information, visit >>>>>> https://groups.google.com/d/forum/archesproject?hl=en >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Arches Project" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> >>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/archesproject/78043179-6107-4ac3-b597-7184cb61a571n%40googlegroups.com >>>>>> <https://groups.google.com/d/msgid/archesproject/78043179-6107-4ac3-b597-7184cb61a571n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>> -- To post, send email to [email protected]. To >>>> unsubscribe, send email to [email protected]. >>>> For more information, visit >>>> https://groups.google.com/d/forum/archesproject?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Arches Project" group. >>>> >>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>> >>> >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/archesproject/a116bc89-5359-4c30-a832-e58701240284n%40googlegroups.com >>>> <https://groups.google.com/d/msgid/archesproject/a116bc89-5359-4c30-a832-e58701240284n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- -- To post, send email to [email protected]. To unsubscribe, send email to [email protected]. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en --- You received this message because you are subscribed to the Google Groups "Arches Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/archesproject/CAJ1rqugb3guRB46ZgG9VYkqZZPimcByPofUZRGUQi-iyja%2B9MA%40mail.gmail.com.
