For deployments, I agree with you. Our pipeline works similar to what you describe.
We open-sourced this tool and I'm cleaning some stuff up for a release. I am trying to safeguard against the most likely user-errors based on past experiences. The hashing is working well, and was inspired by poetry lockfiles. I just compute a md5 of the filepath and filecontents for the config_uri and stick it in the settings. On Friday, January 10, 2025 at 4:34:18 PM UTC-5 Ian Wilson wrote: > I didn't understand the issue at first. The thing you are trying to avoid > is that a script is run with the wrong configuration file because multiple > configuration files exist on every server instance, ie. a production.ini is > on the staging server and a staging.ini is also on the production server? > I haven't used the cookie cutter in a while but can't you just only include > the correct configuration file with each deployment, and don't send > the others, then you don't have to worry about using the wrong file? > > I think I'm using exclusively environment variables now but I am using a > service which provides a heroku-esque management of those. But a given > server instance just has a preloaded single database url in an environment > variable (amongst other things) and does not know about anything else. > > > > On Fri, Jan 10, 2025 at 12:32 PM Jonathan Vanasco <[email protected]> > wrote: > >> That just overly complicates things by requiring Celery and having to set >> it up. Some commandline scripts require Pyramid to be running, others >> don't. I'd like to keep this simple. >> >> On Friday, January 10, 2025 at 3:48:30 AM UTC-5 Luke Crooks wrote: >> >>> "I invoke periodic operations that are often long running" >>> >>> If you invoke celery to manage these, and have celery execute from >>> inside the pyramid, it will be running if the pyramid application is... >>> >>> Then execute any shell scripts with subprocess? >>> >>> Regards, >>> -- >>> Luke Crooks >>> Solent Wholesale Carpets >>> www.solentwholesale.com >>> >>> On Fri, Jan 10, 2025 at 2:10 AM Jonathan Vanasco <[email protected]> >>> wrote: >>> >>>> In this particular use case: >>>> >>>> * The Pyramid Application may only be occasionally publicly available >>>> (firewall rules) >>>> * The Scripts and Server are on the same machine. >>>> >>>> I've done a test implementation pretty quickly using config-file hashes >>>> and that is working well. A Pyramid route publishes 2 hashes: the config >>>> file's filepath, and the config file's contents. Based on that info, a >>>> "client" script can reasonably ascertain it is being invoked by the same >>>> configuration and is probably communicating with "itself". I've also >>>> tested >>>> adding this to my exception logging, as it looks promising for >>>> troubleshooting and identifying regressions. >>>> >>>> The reason for hashes is that a Pyramid application can function very >>>> differently based on the config file - so I can't risk a script invoked >>>> with `staging.ini` thinking that is talking to a server running >>>> `production.ini`. They may have completely different databases, or >>>> functional behaviors. >>>> >>>> The actual use case: >>>> >>>> The Pyramid Application is a TLS Certificate Manager and centralized >>>> ACME Client; when running, it responds to public ACME challenges, and >>>> offers an API to OpenResty/Nginx servers for dynamic certificate loading. >>>> >>>> I invoke periodic operations that are often long running, such as >>>> renewing enrolled certificates, pulling ARI information, etc. These all >>>> need commandline scripts to run on demand anyways, so I didn't bother with >>>> Celery. I need checks in place to ensure both the server and the script >>>> are invoked with the same configuration file. I guess I could probably >>>> add >>>> the MAC address into there to identify the machine... >>>> On Thursday, January 9, 2025 at 5:10:32 PM UTC-5 Luke Crooks wrote: >>>> >>>>> Unless I have missed something, why not run celery from inside pyramid? >>>>> >>>>> >>>>> Regards, >>>>> -- >>>>> Luke Crooks >>>>> Solent Wholesale Carpets >>>>> www.solentwholesale.com >>>>> >>>>> >>>>> On Thu, 9 Jan 2025 at 15:00, Jonathan Vanasco <[email protected]> >>>>> wrote: >>>>> >>>>>> I have a Pyramid project that has some commandline routines via >>>>>> registered script entrypoints (e.g. the /scripts/ directory on the >>>>>> current >>>>>> cookiecutters). >>>>>> >>>>>> A few of these routines need to be run while the pyramid web >>>>>> application is running. >>>>>> >>>>>> I'd like to dummyproof this as much as possible and am trying to >>>>>> think of ways to ensure the server is running. >>>>>> >>>>>> Has anyone done something like this? >>>>>> >>>>>> First I thought of PIDs/Semaphores, but there could be different >>>>>> configuration files used. >>>>>> >>>>>> My current thought is calculating a hash of the ini file by both the >>>>>> commandline and server, and having an endpoint on the server publish it >>>>>> for >>>>>> comparison. >>>>>> >>>>>> Does anyone have a better idea? >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "pylons-discuss" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To view this discussion visit >>>>>> https://groups.google.com/d/msgid/pylons-discuss/8f7b3127-ccbe-4533-a988-1c1034f316dbn%40googlegroups.com >>>>>> >>>>>> <https://groups.google.com/d/msgid/pylons-discuss/8f7b3127-ccbe-4533-a988-1c1034f316dbn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "pylons-discuss" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>> To view this discussion visit >>>> https://groups.google.com/d/msgid/pylons-discuss/b2e4bb56-ee07-4c94-98d4-ff78d8182519n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/pylons-discuss/b2e4bb56-ee07-4c94-98d4-ff78d8182519n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "pylons-discuss" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion visit >> https://groups.google.com/d/msgid/pylons-discuss/7fd0dcf0-13ce-4ce7-a4c6-7be4e57c8607n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/pylons-discuss/7fd0dcf0-13ce-4ce7-a4c6-7be4e57c8607n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/pylons-discuss/408161c2-0290-4924-85b1-2adce3bf5737n%40googlegroups.com.
