Hi Angela,
Sorry, I see now what you're talking about. I wasn't looking at the
management command. So, having that set to use the ELASTICSEARCH_PREFIX is
probably a mistake.
Instead of running the command you can run what the command runs.
"celery -A {your project name} worker -l info"
That should work.
Cheers,
-Alexei
Director of Web Development - Farallon Geographics, Inc. - 971.227.3173
On Thu, Feb 11, 2021 at 5:34 PM Angela Labrador <[email protected]> wrote:
> Hi Alexei,
>
> Thanks but I copied the celery.py file from the Arches template folder so
> it already had those lines in it. My file now looks like yours from the
> earlier reply.
>
> The issue is that the actual management command that starts celery never
> uses that app variable. Instead it uses the elastic search prefix — see
> lines 41 and 43. Because my elastic search prefix is different from my app
> name, I get an error. Can you please try retesting by setting your elastic
> search prefix setting to something that doesn’t equal your project name to
> see if you hit the same error as me?
>
> Thanks,
> Angela
>
> On Thu, Feb 11, 2021 at 8:04 PM Alexei Peters <[email protected]> wrote:
>
>> Hi Angela,
>> After you copy that file over, you'll need to edit a couple of lines in
>> that file:
>>
>> THESE LINES:
>> os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings")
>> app = Celery("arches")
>>
>> SHOULD CHANGE TO THIS:
>> os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{my project
>> name}.settings")
>> app = Celery("{my project name}")
>>
>> Hope that helps!
>> -Alexei
>>
>> Director of Web Development - Farallon Geographics, Inc. - 971.227.3173
>>
>>
>> On Wed, Feb 10, 2021 at 12:22 PM Angela Labrador <[email protected]>
>> wrote:
>>
>>> Hi Alexei,
>>>
>>> I'm back! I have copied the new celery.py file over into my project
>>> directory as suggested above, however, I still get the same error.
>>> Error:
>>> Unable to load celery application.
>>> The module XXX was not found.
>>>
>>> where XXX = {settings.ELASTICSEARCH_PREFIX}
>>>
>>> The only celery management command
>>> <https://github.com/archesproject/arches/blob/stable/5.1.x/arches/management/commands/celery.py>
>>> that I have is in the arches/arches/management/commands/ directory, and I
>>> see that is using the elasticsearch_prefix variable on lines 41 and 43 and
>>> not the project name. When I edit those lines manually and change them to
>>> use my project's name, I can run the command fine.
>>>
>>> Am I supposed to have a separate celery management command within my
>>> project folder or is this a bug with the arches management command?
>>>
>>> Thanks,
>>> Angela
>>>
>>>
>>> On Friday, January 29, 2021 at 5:44:25 PM UTC-5 Alexei Peters wrote:
>>>
>>>> Hi Angela,
>>>> It looks like you identified the issue. Older projects weren't created
>>>> with that celery file and that explains the problem you're seeing.
>>>> Copying that file over to your project should solve the problem.
>>>> This was most likely an oversight in our upgrade announcement (although
>>>> I can't be sure). I'm actually not sure how/if this should be documented.
>>>> Suggestions are welcome there! :)
>>>> Glad we solved this!
>>>> Cheers,
>>>> Alexei
>>>>
>>>> Director of Web Development - Farallon Geographics, Inc. - 971.227.3173
>>>> <(971)%20227-3173>
>>>>
>>>> On Fri, Jan 29, 2021 at 12:17 PM Angela Labrador <[email protected]>
>>>> wrote:
>>>>
>>>>> Hi Alexei,
>>>>>
>>>>> Yep, I'm running the command local to where the manage.py file is
>>>>> located (running the command just like any other python manage.py command
>>>>> with no dir in front of the command).
>>>>>
>>>>> But -- I think you just identified the problem! I do not have a
>>>>> celery.py file within my-project folder besides the celery.py files in
>>>>> media/packages/arches. This is a project that has been migrated from 4.0
>>>>> through the major versions over some time. So, maybe it never created the
>>>>> celery.py file during one of those migrations?
>>>>>
>>>>> I jumped on another virtual machine and installed a brand new arches
>>>>> 5.1 project, and it did create a celery.py file with my project name as
>>>>> you
>>>>> pasted.
>>>>>
>>>>> Just for reference, on my migrated project, running
>>>>> find -iname celery.py
>>>>>
>>>>> results in:
>>>>>
>>>>> ./[my-project]/[my-project]/media/packages/arches/arches/management/commands/celery.py
>>>>> ./[my-project]/[my-project]/media/packages/arches/arches/celery.py
>>>>>
>>>>> ./[my-project]/static/packages/arches/tests/fixtures/testing_prj/testing_prj/celery.py
>>>>>
>>>>> ./[my-project]/static/packages/arches/arches/management/commands/celery.py
>>>>> ./[my-project]/static/packages/arches/arches/celery.py
>>>>> ./env/lib/python3.7/site-packages/celery/bin/celery.py
>>>>> ./arches/tests/fixtures/testing_prj/testing_prj/celery.py
>>>>> ./arches/arches/management/commands/celery.py
>>>>> ./arches/arches/celery.py
>>>>>
>>>>> So, should I just copy the contents of the celery.py file from the
>>>>> fresh project to my migrated project? And do you think this is an anomaly
>>>>> or a "feature" of migrating from much earlier versions? If the latter,
>>>>> should we add something to the docs to cover this?
>>>>>
>>>>> Thank you for sticking with this,
>>>>> Angela
>>>>>
>>>>>
>>>>>
>>>>> On Friday, January 29, 2021 at 2:39:06 PM UTC-5 Alexei Peters wrote:
>>>>>
>>>>>> Hi Angela,
>>>>>> Just to be clear the command should be run from the directory where
>>>>>> the "manage.py" file is located.
>>>>>> EG: you CAN'T do this --> python somedir/manage.py celery start
>>>>>>
>>>>>> Also, does your celery file look something like this?
>>>>>>
>>>>>> from __future__ import absolute_import, unicode_literals
>>>>>> import os
>>>>>> from celery import Celery
>>>>>>
>>>>>> os.environ.setdefault('DJANGO_SETTINGS_MODULE',
>>>>>> '{project_name}.settings')
>>>>>> app = Celery('{project_name}')
>>>>>> app.config_from_object('django.conf:settings', namespace='CELERY')
>>>>>> app.autodiscover_tasks()
>>>>>>
>>>>>>
>>>>>> where {project_name} is the name of the directory where your project
>>>>>> exists
>>>>>>
>>>>>> Cheers,
>>>>>> Alexei
>>>>>>
>>>>>> Director of Web Development - Farallon Geographics, Inc. -
>>>>>> 971.227.3173 <(971)%20227-3173>
>>>>>> On Wed, Jan 27, 2021 at 7:31 AM Angela Labrador <[email protected]>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Alexei,
>>>>>>>
>>>>>>> Correct. I am running python manage.py celery start from within
>>>>>>> [my_project] folder, as per docs
>>>>>>> <https://arches.readthedocs.io/en/stable/task-management/?highlight=celery#running-celery>,
>>>>>>> which results in the
>>>>>>> Error:
>>>>>>> Unable to load celery application.
>>>>>>> The module [ my settings.ELASTICSEARCH_PREFIX ] was not found.
>>>>>>>
>>>>>>> My project folder structure is standard -- no renaming of anything.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Angela
>>>>>>> On Tuesday, January 26, 2021 at 3:44:15 PM UTC-5 Alexei Peters wrote:
>>>>>>>
>>>>>>>> Hi Angela,
>>>>>>>> I assume that you're running the "python manage.py celery start"
>>>>>>>> command from your project and not from the root of the Arches code base
>>>>>>>> correct?
>>>>>>>> The command also assumes that you haven't rename your folders in
>>>>>>>> your project directory.
>>>>>>>> -Alexei
>>>>>>>>
>>>>>>>> Director of Web Development - Farallon Geographics, Inc. -
>>>>>>>> 971.227.3173 <(971)%20227-3173>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Jan 25, 2021 at 11:35 AM Angela Labrador <[email protected]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Lindsay,
>>>>>>>>>
>>>>>>>>> Thanks for checking in -- no, I haven't really solved it. I've got
>>>>>>>>> a workaround, but would appreciate any ideas or explanations so I can
>>>>>>>>> understand how celery is supposed to be configured (or so that we can
>>>>>>>>> update the docs if they need to be edited).
>>>>>>>>>
>>>>>>>>> Angela
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tuesday, January 19, 2021 at 7:48:47 PM UTC-5 Lindsey Gant
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Angela,
>>>>>>>>>>
>>>>>>>>>> Were you able to find a solution to your celery error? If not,
>>>>>>>>>> let me see if someone in the group can follow up.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Lindsey
>>>>>>>>>> GCI Arches Project Team
>>>>>>>>>>
>>>>>>>>>> On Thursday, January 14, 2021 at 2:30:56 PM UTC-8
>>>>>>>>>> [email protected] wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello all,
>>>>>>>>>>>
>>>>>>>>>>> I'm testing an upgrade to Arches 5.1 and have hit a stumbling
>>>>>>>>>>> block with trying to get celery running. I've followed the
>>>>>>>>>>> directions on the doc
>>>>>>>>>>> <https://arches.readthedocs.io/en/stable/task-management/?highlight=celery#running-celery>,
>>>>>>>>>>> and have RabbitMQ installed and my settings.py file updated. I did
>>>>>>>>>>> not
>>>>>>>>>>> create a tasks.py file within my project folder because the docs
>>>>>>>>>>> imply this
>>>>>>>>>>> is only needed if I need to add additional tasks for my project (I
>>>>>>>>>>> just
>>>>>>>>>>> need the default Arches tasks).
>>>>>>>>>>>
>>>>>>>>>>> When I run python manage.py celery start from within my
>>>>>>>>>>> virtualenv, I get
>>>>>>>>>>>
>>>>>>>>>>> Error:
>>>>>>>>>>> Unable to load celery application.
>>>>>>>>>>> The module XXX was not found.
>>>>>>>>>>>
>>>>>>>>>>> where XXX = my settings.ELASTICSEARCH_PREFIX
>>>>>>>>>>>
>>>>>>>>>>> However, when I edit the celery.py management command to use
>>>>>>>>>>> cmd = f"celery -A *arches* worker -B -l info"
>>>>>>>>>>> instead of settings.ELASTICSEARCH_PREFIX, celery starts up and
>>>>>>>>>>> connects to rabbitmq.
>>>>>>>>>>>
>>>>>>>>>>> I'm wondering why that management command is using the ES prefix
>>>>>>>>>>> and what I could be missing in my own project config that is
>>>>>>>>>>> causing this.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Angela
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>> -- To post, send email to [email protected]. To
>>>>>>>>> unsubscribe, send email to [email protected]. For
>>>>>>>>> more information, visit
>>>>>>>>> https://groups.google.com/d/forum/archesproject?hl=en
>>>>>>>>> ---
>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>> Groups "Arches Project" group.
>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>> send an email to [email protected].
>>>>>>>>> To view this discussion on the web visit
>>>>>>>>> https://groups.google.com/d/msgid/archesproject/17891260-c650-469a-bdff-9e68f9b5b462n%40googlegroups.com
>>>>>>>>> <https://groups.google.com/d/msgid/archesproject/17891260-c650-469a-bdff-9e68f9b5b462n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>> --
>>>>>>> -- To post, send email to [email protected]. To
>>>>>>> unsubscribe, send email to [email protected]. For
>>>>>>> more information, visit
>>>>>>> https://groups.google.com/d/forum/archesproject?hl=en
>>>>>>> ---
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "Arches Project" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to [email protected].
>>>>>>>
>>>>>> To view this discussion on the web visit
>>>>>>> https://groups.google.com/d/msgid/archesproject/4a95537c-db9b-4935-8144-37e930706511n%40googlegroups.com
>>>>>>> <https://groups.google.com/d/msgid/archesproject/4a95537c-db9b-4935-8144-37e930706511n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>> .
>>>>>>>
>>>>>> --
>>>>> -- To post, send email to [email protected]. To unsubscribe,
>>>>> send email to [email protected]. For more information,
>>>>> visit https://groups.google.com/d/forum/archesproject?hl=en
>>>>> ---
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Arches Project" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>>
>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/archesproject/78043179-6107-4ac3-b597-7184cb61a571n%40googlegroups.com
>>>>> <https://groups.google.com/d/msgid/archesproject/78043179-6107-4ac3-b597-7184cb61a571n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>> --
>>> -- To post, send email to [email protected]. To
>>> unsubscribe, send email to [email protected].
>>> For more information, visit
>>> https://groups.google.com/d/forum/archesproject?hl=en
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "Arches Project" group.
>>>
>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to [email protected].
>>
>>
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/archesproject/a116bc89-5359-4c30-a832-e58701240284n%40googlegroups.com
>>> <https://groups.google.com/d/msgid/archesproject/a116bc89-5359-4c30-a832-e58701240284n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>
--
-- To post, send email to [email protected]. To unsubscribe, send
email to [email protected]. For more information,
visit https://groups.google.com/d/forum/archesproject?hl=en
---
You received this message because you are subscribed to the Google Groups
"Arches Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/archesproject/CA%2BZLqy8D7HuYe9QN0rByXqDH8yWd4t2mOY217MpBCR_Ysek6SQ%40mail.gmail.com.
