[pylons-discuss] Re: What should I use to inject data into multiple view responses before JSON rendering?

[Jonathan Vanasco]

I don't know what is recommended, but have done similar things before.

One way I've done something similar is to use a custom renderer in the 
`@view_config`.

Another way I've accomplished this is to set a request attribute early on - 
either in a tween or in the view class instantiation or even a request 
attribute, and having my return value reflect this.

To illustrate the second example, in this project each view is configured 
to handle 2 different API endpoints - one that is human HTML, and another 
that is programmatic API for JSON.  If the route is accessed for JSON, the 
view returns a slightly different payload.

1- This code adds a "Request.wants_json" attribute:

https://github.com/aptise/peter_sslers/blob/main/src/peter_sslers/web/__init__.py#L136-L140

2- This view returns a different payload, depending upon the wants_json 
attribute.

https://github.com/aptise/peter_sslers/blob/main/src/peter_sslers/web/views_admin/acme_challenge.py#L114-L126

This was a quick, lazy and functional way to proceed.  If I had to do this 
again, I would probably use a single payload with a custom JSON renderer, 
and have a function in the custom JSON renderer to only render specific 
values from the payload (which would either be attached to the request, or 
listed in a namespace in the payload).

For your description though, I would probably just write a custom JSON 
renderer.

For example, here is a custom jsonp and gif renderer:

def jsonp_renderer_factory(info):
def _render(value, system):
value = py2json(value)
request = system.get("request")
if request is None:
return value
# JSONP is GET only, unless you do some weird shit that doesn't 
work reliably
callback = request.GET.get("callback", None)
if callback is None:
ct_json = "application/json"
else:
ct_json = "application/javascript"
value = "%s(%s)" % (callback, value)
response = request.response
if response.content_type == response.default_content_type:
response.content_type = ct_json
return value

return _render

def gif_renderer_factory(info):
def _render(value, system):
request = system.get("request")
if request is not None:
response = request.response
ct = response.content_type
if ct == response.default_content_type:
response.content_type = "image/gif"
return value

return _render

config.add_renderer("jsonp", jsonp_renderer_factory)
config.add_renderer("gif", gif_renderer_factory)



On Wednesday, April 10, 2024 at 4:44:27 PM UTC-4 Pablo Díaz Ogni wrote:

> Hello guys! I'm building an API using pyramid + pyramid_openapi3 package 
> to use openapi3 features such as request validation, the web interface, and 
> have well documented the endpoints. I'm having trouble understanding the 
> different ways I can use views, view_derivers, tweens, decorators, etc.
>
> Let's say I have multiple views that return its own data as a 
> dictionary/list and I want to wrap the response with more info before 
> rendering to JSON:
> {request: , response:  view>}
>
> I first tried with tweens, but that's post renderization so I have to 
> deserialize and serialize the json. I also tried view_derivers and I was 
> facing the same issue. I also saw that you can match a view with a Context 
> such as an exception, but not sure if there is a way to use that Context in 
> other way such as "all the View objects that have this specific attribute"
>
> I also tried a decorator with wrapper(context, request), but i switched 
> from functions with @view_config to use View classes with get() method 
> instead, so I should change the approach of the decorator
>
> I'm new to pyramid and it has its own way of plugging views so I would 
> like to know what is the recommended way of doing this?
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/e22a2d93-bf29-4393-bb71-b0499846ea30n%40googlegroups.com.

[pylons-discuss] Re: Question about serving video and audio (e.g .mp4 and .ogg)

[Jonathan Vanasco]

Unless you need to lock those files down with pyramid auth, I would isolate 
them away for Apache to serve.

I am very familiar with Dreamhost, and running fcgi stuff on it.  I keep 
some subversion/trac stuff on it - I really need to upgrade those to git!

I would avoid serving anything static on Dreamhost through Pyramid.  They 
have some throttling and process monitoring on their virtualization stack - 
and do not have a nice cleanup of old fcgi processes.  All your static 
assets are going to be competing for limited resources with the urls that 
can only be handled by the Pyramid app.

If you can map the apache subdir to the pyramid static folder, that would 
be ideal.  Another option is serving the static files as a separate 
subdomain, mapping the static dir to that domain on dreamhost, and using a 
toggle in your app to handle the url prefix for the assets (same domain on 
testing environment, subdomain on production).




On Saturday, March 23, 2024 at 4:08:55 PM UTC-4 Aaron Krister Johnson wrote:

> Hi all,
>
> I have a pyramid app on Dreamhost, and they are reverting from using 
> Ruby's passenger back to fcgi fro shared hosting accounts. The server runs 
> Apache, and I have control of .htaccess configuration.
>
> At the moment, I am able to serve static files by using 
> `config.add_static_view`, but it does not appear to be working for things 
> like videos hosted on site (.mp4) nor audio (.ogg).
>
> I thought to bypass `add_static_view` and have Apache serve static assets 
> itself, but it's brought no joy, since apparently, my setup doesn't allow 
> `Alias` directives, and the `RewriteCond` for files doesn't play well with 
> Pyramid's layout, with or without `add_static_view`. The basic site seems 
> to work better when I use Pyramid to serve, but like I said, video and 
> audio files are blocked somehow. How do I serve .mp4 and .ogg? Do I have to 
> get into declaring things iwth the `mimetypes` lib?
>
> Here's my .htaccess, for reference:
>
> $ cat .htaccess
> #Disables GZIP
> SetEnv no-gzip 1
> 
> #Turns off the expires headers for Apache
> 
>   ExpiresActive Off
> 
>
> # DISABLE CACHING
> 
>   Header set Cache-Control "no-cache, no-store, must-revalidate"
>   Header set Pragma "no-cache"
>   Header set Expires 0
>   Header always set Content-Security-Policy "upgrade-insecure-requests;"
> 
>
> Options +ExecCGI
> AddHandler fcgid-script .fcgi
> SetHandler fcgid-script
> RewriteEngine On
> RewriteBase /
> RewriteCond %{HTTPS} !=on
> RewriteRule ^(.*) https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301,NE]
> RewriteCond %{REQUEST_FILENAME} !-f
> RewriteRule ^(.*)$ dispatch.fcgi/$1 [QSA,L]
>
> And here is my `dispatch.fcgi`:
>
> #!/home/untwelve_admin/untwelve.dreamhosters.com/bin/python3.12
>
> import sys
>
> from paste.deploy import loadapp
> from flup.server.fcgi_fork import WSGIServer
>
> sys.path.append("UnTwelveDotOrg")
>
> app = loadapp(
> "config:/home/untwelve_admin/
> untwelve.dreamhosters.com/UnTwelveDotOrg/development.ini"
> )
>
> if __name__ == "__main__":
> WSGIServer(app).run()
>
>
> Thanks for any insights you can provide!
>
> Best,
> Aaron
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/8fdecb11-6fec-446e-8eda-1f68b7969ff5n%40googlegroups.com.

[pylons-discuss] Re: Question about creating routes

[Jonathan Vanasco]

I'm not sure about lambdas, but this reminded me the tutorial with 
"discriminators", which goes into some troubleshooting:

https://docs.pylonsproject.org/projects/pyramid_cookbook/en/latest/configuration/whirlwind_tour.html

I feel like I've encountered something around this concept before , but I 
can't recall what project it was in to look at code. :/

On Wednesday, March 13, 2024 at 1:50:07 PM UTC-4 Laurent Daverio wrote:

> Hello list,
>
> it seems that Google is allowing me to post on here again. A couple of 
> weeks ago, I was banned, both from emailing the list, AND from posting on 
> the web group 
>
> I am trying to create a series of routes based on lambda functions, 
> something like:
>
> ```
> config.add_route('doc1.delete_selection', '/doc1/-/delete_selection')
> config.add_route('doc2.delete_selection', '/doc2/-/delete_selection')
> config.add_view(
> lambda request: _delete_multi(request, Doc1Resource),
> route_name='doc1.delete_selection', request_method='POST', 
> renderer='json')
> config.add_view(
> lambda request: _delete_multi(request, Doc2Resource),
> route_name='doc2.delete_selection', request_method='POST', 
> renderer='json')
> ```
>
> (I also have `archive_selection` and `unarchive_selection` routes)
>
> I'm under the impression that `config.add_view` can't take lambda 
> functions as arguments, as it tries to resolve their names ("maybe_dotted", 
> "maybe_resolve", etc.). All I can say it that the views seem to be calling 
> the wrong functions. This prevents me from using for loops and lambda for 
> generating the routes in a more concise way :/
>
> Could anyone confirm if I'm correct?
>
> Many thanks,
>
> Laurent.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/61212748-c660-443d-bdb0-89a84d08cd89n%40googlegroups.com.

Re: [pylons-discuss] request for help upgrading the cookiecutter to support sqlalchemy 2.0

[Jonathan Vanasco]

I don't have time to do this, but would be happy to be looped in for code 
review if there is another PR.

It looks like you've gotten most things done already.

I think the alembic integration may be out of date, as those files haven't 
been touched for a while.  I can ask Federico on the SqlAlchemy team to 
take a look at the current setup, he's been the main force behind 
maintaining and expanding Alembic.  The core devs meet on Wednesdays on 
gitter. 


On Monday, January 29, 2024 at 5:08:54 AM UTC-5 Laurent Daverio wrote:

> Hello list,
>
> I think I could do that, unless someone has volunteered already (I have A 
> LOT to do these days, wouldn't want to duplicate someone else's work).
>
> Laurent.
>
>
> Le lun. 29 janv. 2024 à 07:51, Michael Merickel  a 
> écrit :
>
>> Hey folks,
>>
>> I'd really appreciate it if someone was willing to take the time to 
>> upgrade the pyramid-cookiecutter-starter to use SQLAlchemy 2.0 best 
>> practices. Things like rewriting the queries away from Query to simple 
>> select(), and using the more modern declarative mapper that is typing 
>> friendly.
>>
>> https://github.com/Pylons/pyramid-cookiecutter-starter/tree/main
>>
>> Once we get that upgraded, we'd need to fix up the wiki2 tutorial to 
>> reference the updated files accurately - but first step is just getting the 
>> cookiecutter updated.
>>
>> https://github.com/Pylons/pyramid/tree/main/docs/tutorials/wiki2
>>
>> Thanks!
>>
>> - Michael
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discus...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/54913600-1D9F-47E9-B991-136D0B5A73F3%40gmail.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/ea574da1-7b7e-4a88-ac4b-83950161a5dcn%40googlegroups.com.

[pylons-discuss] Re: Enabling Github Discussions

[Jonathan Vanasco]

After using github discussions as a secondary support channel, SqlAlchemy 
froze the Google Group a few months ago and uses it as the only official 
channel now.  It has worked fairly well.

On Wednesday, January 31, 2024 at 8:57:29 PM UTC-5 Michael Merickel wrote:

> Hey folks,
>
> I've enabled the discussions feature on the pyramid repository as an 
> alternative form of communication if it simplifies communication for 
> anyone. We shall see!
>
> https://github.com/Pylons/pyramid/discussions
>
> - Michael
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/5ff2c175-97a7-4f61-8806-370fc4eed8c6n%40googlegroups.com.

Re: [pylons-discuss] feedback on Supabase auth sample app for Pyramid

[Jonathan Vanasco]

Fair enough!  I wrote `pyramid_session_multi` to let me map multiple 
ISession Session libraries onto a request.  On our largest project, we run 
a secondary server side session and mix of encrypted cookies to handle some 
"secret" data.  This library would let me drop one of the server side 
sessions and replace the encrypted cookie with something more automated.  I 
am so excited and thankful you shared it!

On Friday, December 1, 2023 at 2:43:20 PM UTC-5 Delta Regeer wrote:

> Nope. I know it was created for a client of Agendaless, and I know it 
> exists. I have not used it because I don’t store secret data in cookies, so 
> using the signed factory one is easier and didn’t require additional crypto 
> libraries to be added to my stack.
>
> On Nov 30, 2023, at 15:26, Jonathan Vanasco  wrote:
>
> Wow. This looks great. I wish I knew about it sooner.
>
> Digging into the code, there was a PR to split things out and support JSON 
> serialization – however there are no unit tests covering this or docs for 
> it.  @Delta do you know of any public examples of this usage?  If so I'd be 
> happy to play around with it and generate a PR for unit tests.
>
> I often manually generate and read encrypted cookies, which can be a 
> chore.  This would be incredibly useful to me in a few projects.
>
> On Thursday, November 30, 2023 at 2:48:03 PM UTC-5 Delta Regeer wrote:
>
>> Use 
>> https://docs.pylonsproject.org/projects/pyramid-nacl-session/en/latest/usage.html
>>
>> It encrypts the session the is stored in the cookie with NACL. No longer 
>> is the content if the cookie something that an attacker can read/do 
>> anything with.
>>
>>
>> On Nov 28, 2023, at 12:12, Scott Lawton  wrote:
>>
>> Some followup:
>> - 
>> https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/sessions.html 
>> has a big section in red: 'By default the SignedCookieSessionFactory() 
>> <https://docs.pylonsproject.org/projects/pyramid/en/latest/api/session.html#pyramid.session.SignedCookieSessionFactory>
>>  implementation 
>> contains the following security concerns:
>>
>> ... which seems to argue against session, but maybe doesn't apply to 
>> access/refresh tokens? And/or maybe setting the cookie like we do isn't any 
>> better?
>>
>> We also tried to follow 
>> https://docs.pylonsproject.org/projects/pyramid/en/latest/whatsnew-2.0.html#upgrading-auth-20
>>  
>> ... but not sure we did so correctly. That's what we're looking for 
>> feedback!
>>
>> Scott
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discus...@googlegroups.com.
>>
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/2dedd1e5-cffc-45c4-84b6-ebb142a68368n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/pylons-discuss/2dedd1e5-cffc-45c4-84b6-ebb142a68368n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>>
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
>
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/b8039844-e54d-4b80-be42-ec56dca2e066n%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/pylons-discuss/b8039844-e54d-4b80-be42-ec56dca2e066n%40googlegroups.com?utm_medium=email_source=footer>
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/7b4d41f8-526b-4330-8991-594e0f09d432n%40googlegroups.com.

Re: [pylons-discuss] feedback on Supabase auth sample app for Pyramid

[Jonathan Vanasco]

Wow. This looks great. I wish I knew about it sooner.

Digging into the code, there was a PR to split things out and support JSON 
serialization – however there are no unit tests covering this or docs for 
it.  @Delta do you know of any public examples of this usage?  If so I'd be 
happy to play around with it and generate a PR for unit tests.

I often manually generate and read encrypted cookies, which can be a 
chore.  This would be incredibly useful to me in a few projects.

On Thursday, November 30, 2023 at 2:48:03 PM UTC-5 Delta Regeer wrote:

> Use 
> https://docs.pylonsproject.org/projects/pyramid-nacl-session/en/latest/usage.html
>
> It encrypts the session the is stored in the cookie with NACL. No longer 
> is the content if the cookie something that an attacker can read/do 
> anything with.
>
>
> On Nov 28, 2023, at 12:12, Scott Lawton  wrote:
>
> Some followup:
> - 
> https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/sessions.html 
> has a big section in red: 'By default the SignedCookieSessionFactory() 
> 
>  implementation 
> contains the following security concerns:
>
> ... which seems to argue against session, but maybe doesn't apply to 
> access/refresh tokens? And/or maybe setting the cookie like we do isn't any 
> better?
>
> We also tried to follow 
> https://docs.pylonsproject.org/projects/pyramid/en/latest/whatsnew-2.0.html#upgrading-auth-20
>  
> ... but not sure we did so correctly. That's what we're looking for 
> feedback!
>
> Scott
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
>
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/2dedd1e5-cffc-45c4-84b6-ebb142a68368n%40googlegroups.com
>  
> 
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/b8039844-e54d-4b80-be42-ec56dca2e066n%40googlegroups.com.

Re: [pylons-discuss] feedback on Supabase auth sample app for Pyramid

[Jonathan Vanasco]

> Any suggestions for 1-3 simple examples? Ideally with only Python 
dependencies -- I'd rather not add Redis, MongoDB etc. since we already 
have Supabase.

Aside from beaker, no.  I maintain `pyramid_session_redis` and - as long as 
you disable redis administration - it is relatively insignificant on a 
server that has available memory.  Unless you have very high traffic and 
require more memory, running Redis with a 100MB limit is pretty 
insignificant.  

If you do use pyramid_beaker, i strongly suggest having it save to a 
dedicated disk partition so that it does not overrun your server, which can 
lead to downtime.

While there are memory backed options, there are two drawbacks for this:
* sessions do not persist across server or application restarts
* you can run into complications with forking servers

> Looks like there are enough docs available for us to proceed. Still, if 
anyone knows of a sample app that includes that + oauth, would be great to 
see. (I learn more from examples than from docs.)

I also main pyramid_oauthlib_lowlevel

 https://github.com/jvanasco/pyramid_oauthlib_lowlevel

The test suites have fully functional pyramid applications for oauth 1 and 
oauth 2 flows, each of which mimic how a consumer server and provider 
server will work with one another.




On Thursday, November 30, 2023 at 1:04:12 PM UTC-5 Scott Lawton wrote:

> Thanks for the link and the 'second' that it's still a practical solution; 
> I was a bit hesitant given that it's 10 years old: 0.8 (2013-06-28).
>
> Looks like there are enough docs available for us to proceed. Still, if 
> anyone knows of a sample app that includes that + oauth, would be great to 
> see. (I learn more from examples than from docs.)
>
> Scott
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/6bdc092c-9cd4-465e-bc41-3ba36df3ffe4n%40googlegroups.com.

Re: [pylons-discuss] feedback on Supabase auth sample app for Pyramid

[Jonathan Vanasco]

First off, the refresh/access tokens are sensitive credentials that must be 
protected from exposure to third parties.

I would personally consider this implementation as a low security risk when 
it comes to the two tokens, as they'd be left as plaintext in the browser's 
settings/profile whether the connection was on http or HTTPS.

This would be completely insecure and a high security risk if the 
connection/server was http, as the data would be transferred in plaintext 
and is fully susceptible to MITM attacks or network packet sniffing.

To be clear, IMHO: storing this stuff in a cookie is not an issue.  Storing 
this stuff in an **unencrypted** cookie without TLS is an issue.

For this type of data, at a minimum, calls to request.response.set_cookie 
should contain `httponly=True` and `secure=True`; the app should also be 
under https only. If I understand how your system uses these credentials 
correctly, `samesite=strict` should also be added.

There are two concepts when it comes to Cookie Data.  
* signing
* encryption

Signing the cookie just creates a digest of the payload with a site secret. 
 It is used to prove the data originated from the site.  

Encrypting the cookie makes it unreadable without the key.

Pyramid's sessions, by default, are the following: 
(https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/sessions.html)
* unencrypted, but signed/authenticated
* fully contained in the cookie (Client-Side Sessions)

IMHO, putting this data in a standard Pyramid client-side session wouldn't 
fix the security concerns either - even though the information is signed, 
which authenticates it originated from your app, it could be extracted from 
the user's browser and would be susceptible to mitm attacks if the 
connection were not https.  There are also the concerns with the cookie not 
being locked down with the kwargs mentioned above.

If I were to be stashing this information, I would consider the following 
two options:

1- Store in an Encrypted cookie.  I am not sure if there are any open 
source projects on PyPi that automate this within Pyramid, but there are 
many that will handle the generic encryption/decryption.  Alternatively, 
you can use middleware to encrypt/decrypt the cookie data.
2- Store in a Server-Side session.  There are many projects in the pyramid 
ecosystem for this.  If this method is used, you need to ensure the cookies 
used for authentication/authorization/login-status are either encrypted or 
are HTTPS only.  If they are not, you should require re-authentication for 
the user to access this information.


On Tuesday, November 28, 2023 at 2:12:33 PM UTC-5 Scott Lawton wrote:

> Some followup:
> - 
> https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/sessions.html 
> has a big section in red: 'By default the SignedCookieSessionFactory() 
> 
>  implementation 
> contains the following security concerns:
>
> ... which seems to argue against session, but maybe doesn't apply to 
> access/refresh tokens? And/or maybe setting the cookie like we do isn't any 
> better?
>
> We also tried to follow 
> https://docs.pylonsproject.org/projects/pyramid/en/latest/whatsnew-2.0.html#upgrading-auth-20
>  
> ... but not sure we did so correctly. That's what we're looking for 
> feedback!
>
> Scott
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/93a2b140-d1ab-430d-9e48-a8975c4f2b5an%40googlegroups.com.

Re: [pylons-discuss] Problems when using Gunicorn and Postgresql together

[Jonathan Vanasco]

> Namely, if you deploy with Gunicorn a Pyramid + PostgreSQL app based on 
the standard cookiecutter, you will run into problems, because the 
connection to the DB can't be shared between the processes, so each process 
needs to have its own connection to the DB.

I forgot to mention...

This should not happen. Do you know which cookiecutter you used, and when?  
This should not happen in the most recent cookiecutter.

The cookiecutter works by adding a transaction/thread safe session to each 
request: 
https://github.com/Pylons/pyramid-cookiecutter-starter/blob/latest/%7B%7Bcookiecutter.repo_name%7D%7D/%7B%7Bcookiecutter.repo_name%7D%7D/sqlalchemy_models/__init__.py#L87-L127

It also requires pyramid_tm, which will cleanup the connections at the end 
of the request lifecycle.  If you decide to not use pyramid_tm, then you'll 
have to use a cleanup routine like this one when you grab the request.  
This was one the pattern in the pyramid cookiecutter, but it was taken out 
because pyramid_tm effectively does that itself.

  
  
https://github.com/jvanasco/pyramid_oauthlib_lowlevel/blob/main/tests/oauth2_app/model/__init__.py#L71-L77

The only way the connection pool sharing should be able to happen is if you 
grabbed a database connection before the fork - like during application 
setup, which would then require a call to Engine.dispose.

If you are not consciously grabbing a database connection before the fork, 
you should do a code audit to figure out where the first connection is 
being made and how it is recycled, as the last few versions of the 
cookiecutter do defend against this behavior.




On Tuesday, November 21, 2023 at 1:06:43 AM UTC-5 Mike Orr wrote:

> On Mon, Nov 20, 2023 at 4:14 PM Jonathan Vanasco  
> wrote:
> >
> > SQLAlchemy supports this via `Engine.dispose()`, which is the documented 
> way of handling a post-fork connection:
> >
> > https://docs.sqlalchemy.org/en/13/core/connections.html#engine-disposal
>
> Yes, that sounds familiar.
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/984f5ea0-5d10-416b-83f0-c6d19d30ad9dn%40googlegroups.com.

Re: [pylons-discuss] Re: Using SSL client certificate in a Pyramid application

[Jonathan Vanasco]

100% agree with Michael's comments about signed headers.  When dealing with 
credentials, a lot of people distrust proxies and testing them - so on top 
of removing all headers, anything added will be signed (or the aggregate 
signed) with a low-cost method, which can easily be tested.  
On Friday, November 17, 2023 at 2:30:12 PM UTC-5 Michael Merickel wrote:

> You don’t need a signed header. You just need to make sure your proxy is 
> configured to remove that header from an incoming request and only put 
> validated data into it before sending it to your app. This is standard for 
> other headers set by your proxy as well. Definitely never trust a header 
> without checking those properties of your deployment though. 
>
> - Michael
>
> On Nov 17, 2023, at 10:02, Jonathan Vanasco  wrote:
>
> 
>
> I do a lot of work with SSL Certs.  If you are using publicly trusted 
> client certificates (i.e. LetsEncrypt, Digisign, etc), then you basically 
> just need to do what Michael suggested - ensure your gateway or server 
> populates the headers or environment variables with the information for the 
> client.  Most implementations for that stuff will also insert a 
> secret-signed header field to ensure the client did not try and stuff 
> headers themselves.  You'd then need to validate the certificate again 
> within python (see workaround #3 below)
>
> I think this is going to be a bit of a headache if you are using untrusted 
> CAs for the client certificates (ie. you control the CA and allocate the 
> certs to your clients/partners) and trying to implement the sort of policy 
> that PostgreSQL and other systems offer with client verification.
>
> The complexity and level of work required come down to the following 
> factors:
>
> * Who will issue the client certificates (a public trusted CA or your own 
> private CA)
> * How much validation can you split between the gateway/server and Python 
> (based on application design and business logic constraints), and where can 
> the errors pop-up?
>
> If you are talking about custom CA certs, the two approaches I know of for 
> this sort of stuff:
>
> 1- (Easy for developer, harder for client) The client application stuffs 
> the certificate into a header.  Everything happens as normal Pyramid app.  
> You just write pyramid code to require and validate the certificate 
> header.  This isn't actually mTLS, but uses the client certificate similar 
> to ssl keychains.
>
> 2. (Hard for developer, easy for client) Actually use TLS Mutual Auth, 
> which is what Theron first brought up.  With mTLS, the client certificate 
> is presented and validated during the TLS handshake - which means you would 
> need to configure the gateway or mod_wsgi to handle the validation.
>
> The large complexity of #2 is that many web servers / gateways can only be 
> configured with a single client CA or a static client CA store - so 
> building this as scalable to multiple clients may not necessarily be 
> possible.
>
> The three workarounds I can think of are:
> 1- Don't terminate TLS or validate the client certificate on the gateway, 
> instead pass it all back to a python server like Waitress and see if you 
> can handle everything via middleware to process the SSL connection.
>
> 2- Terminate TLS on OpenResty(nginx) and use a Lua script to handle the 
> client validation.  Then you can proxy that back to mod_wsgi.  I 
> opensourced our dynamic SSL certificate tool for OpenResty here - 
> https://github.com/aptise/lua-resty-peter_sslers - it does not do client 
> certificates, but IMHO is a good reference for what you want to 
> accomplish.it looks like OpenResty has some builtin stuff for this, but 
> if not - those are the correct hooks.  
>
> 3- Terminate TLS on the gateway but make it very lax for the client 
> certificate to pass - basically just checking to see if the certificate is 
> valid and from the CA.  Then validate the client a second time in 
> middleware/tween to ensure the certificate is active and allowed.  
>
>
> On Wednesday, November 15, 2023 at 9:13:39 AM UTC-5 Thierry Florac wrote:
>
>> Hi,
>> My problem is probably quite simple: I would like to be able, in a 
>> Pyramid application, to create a custom security policy which could use an 
>> SSL client certificate as a request credential to handle authentication 
>> (authorized certificates being referenced in a database or stored in a 
>> specific server directory).
>> This application is then supposed to be published via mod_wsgi in an 
>> Apache server located behind an HAProxy.
>> I tried to search here and there but didn't find any information about 
>> this...
>> Any hint?
>>
>> Best regards,
>> 

Re: [pylons-discuss] Problems when using Gunicorn and Postgresql together

[Jonathan Vanasco]

SQLAlchemy supports this via `Engine.dispose()`, which is the documented 
way of handling a post-fork connection:

   https://docs.sqlalchemy.org/en/13/core/connections.html#engine-disposal

Invoking `Pool.recreate()` should be fine, but the documented pattern is to 
call `Engine.dispose`

> How should I modify my app to make it compatible with both Waitress and 
Gunicorn?

I've been trying to standardize this with pyramid_forkfsafe when I was 
deploying projects on uwsgi and gunicorn simultaneously:

https://pypi.org/project/pyramid-forksafe/
https://github.com/jvanasco/pyramid_forksafe

I don't know if the gunicorn is currently working, as I only work with 
uwsgi now and never got around to writing tests for gunicorn.

Ideally, my package works like this:

* it defines a new `ApplicationPostFork` event
* you write your post-fork code in an event subscriber
* the package invokes the event at the right time

This allows you to just define the post-fork routine, and if everything 
goes correctly pyramid_forksafe will detect the correct post-fork event 
and invoke your routine.
On Thursday, November 16, 2023 at 12:18:59 PM UTC-5 Theron Luhn wrote:

> If you aren’t using `—preload` then gunicorn should load the application 
> fresh for each worker and you shouldn’t have any issues.
>
> If you are using preload, you have to recreate any existing connections on 
> fork.  For SQLAlchemy I use:
>
> def after_fork(registry):
> registry['db_engine'].pool.recreate()
>
> def includeme(config):
> os.register_at_fork(
> after_in_child=functools.partial(after_fork, config.registry),
> )
>
>
> — Theron
>
>
>
> On Nov 16, 2023, at 7:41 AM, Laurent Daverio  wrote:
>
> Hello list,
>
> this page seems to describe perfectly a problem I've stumbled on: 
>
>
> https://stackoverflow.com/questions/64995178/decryption-failed-or-bad-record-mac-in-multiprocessing
>
> Namely, if you deploy with Gunicorn a Pyramid + PostgreSQL app based on 
> the standard cookiecutter, you will run into problems, because the 
> connection to the DB can't be shared between the processes, so each process 
> needs to have its own connection to the DB.
>
> Before I start trying to develop a workaround, has anybody encountered the 
> problem? How should I modify my app to make it compatible with both 
> Waitress and Gunicorn?
>
> Thanks in advance,
>
> Laurent.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAB7cU6z1DqHpEazrrJ1sPHmSPQvYtfkmeKfsJP_jLmsDyPA96w%40mail.gmail.com
>  
> 
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/459deec2-f880-494d-afc4-bfc300f15093n%40googlegroups.com.

[pylons-discuss] Re: Using SSL client certificate in a Pyramid application

[Jonathan Vanasco]

I do a lot of work with SSL Certs.  If you are using publicly trusted 
client certificates (i.e. LetsEncrypt, Digisign, etc), then you basically 
just need to do what Michael suggested - ensure your gateway or server 
populates the headers or environment variables with the information for the 
client.  Most implementations for that stuff will also insert a 
secret-signed header field to ensure the client did not try and stuff 
headers themselves.  You'd then need to validate the certificate again 
within python (see workaround #3 below)

I think this is going to be a bit of a headache if you are using untrusted 
CAs for the client certificates (ie. you control the CA and allocate the 
certs to your clients/partners) and trying to implement the sort of policy 
that PostgreSQL and other systems offer with client verification.

The complexity and level of work required come down to the following 
factors:

* Who will issue the client certificates (a public trusted CA or your own 
private CA)
* How much validation can you split between the gateway/server and Python 
(based on application design and business logic constraints), and where can 
the errors pop-up?

If you are talking about custom CA certs, the two approaches I know of for 
this sort of stuff:

1- (Easy for developer, harder for client) The client application stuffs 
the certificate into a header.  Everything happens as normal Pyramid app.  
You just write pyramid code to require and validate the certificate 
header.  This isn't actually mTLS, but uses the client certificate similar 
to ssl keychains.

2. (Hard for developer, easy for client) Actually use TLS Mutual Auth, 
which is what Theron first brought up.  With mTLS, the client certificate 
is presented and validated during the TLS handshake - which means you would 
need to configure the gateway or mod_wsgi to handle the validation.

The large complexity of #2 is that many web servers / gateways can only be 
configured with a single client CA or a static client CA store - so 
building this as scalable to multiple clients may not necessarily be 
possible.

The three workarounds I can think of are:
1- Don't terminate TLS or validate the client certificate on the gateway, 
instead pass it all back to a python server like Waitress and see if you 
can handle everything via middleware to process the SSL connection.

2- Terminate TLS on OpenResty(nginx) and use a Lua script to handle the 
client validation.  Then you can proxy that back to mod_wsgi.  I 
opensourced our dynamic SSL certificate tool for OpenResty here - 
https://github.com/aptise/lua-resty-peter_sslers - it does not do client 
certificates, but IMHO is a good reference for what you want to 
accomplish.it looks like OpenResty has some builtin stuff for this, but 
if not - those are the correct hooks.  

3- Terminate TLS on the gateway but make it very lax for the client 
certificate to pass - basically just checking to see if the certificate is 
valid and from the CA.  Then validate the client a second time in 
middleware/tween to ensure the certificate is active and allowed.  


On Wednesday, November 15, 2023 at 9:13:39 AM UTC-5 Thierry Florac wrote:

> Hi,
> My problem is probably quite simple: I would like to be able, in a Pyramid 
> application, to create a custom security policy which could use an SSL 
> client certificate as a request credential to handle authentication 
> (authorized certificates being referenced in a database or stored in a 
> specific server directory).
> This application is then supposed to be published via mod_wsgi in an 
> Apache server located behind an HAProxy.
> I tried to search here and there but didn't find any information about 
> this...
> Any hint?
>
> Best regards,
> Thierry
> -- 
>   https://www.ulthar.net -- http://pyams.readthedocs.io
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/39f47b19-0e08-4936-aaff-3da5822dc90fn%40googlegroups.com.

[pylons-discuss] Re: `add_request_method` doesn't seem to work within an `includeme` anymore

[Jonathan Vanasco]

I just remembered the debugtoolbar works as a separate application, which 
explains why I'm seeing different values for `id(config)`  in the toolbar 
and main app during setup.  unless I am crazy, this should never have 
worked.



On Wednesday, September 20, 2023 at 5:46:46 PM UTC-4 Jonathan Vanasco wrote:

> This is driving me a bit crazy.  I hope someone can see the issue or 
> changelog I am missing in the current pyramid and debugtoolbar.
>
> I am adding some typing and tests to a debug toolbar that has the 
> following in the includeme:
>
> ```
> def includeme(config: "Configurator"):
> config.add_debugtoolbar_panel(DogpileDebugPanel)
> config.add_request_method(
> "pyramid_debugtoolbar_dogpile.setup_dogpile_logging",
> "dogpile_logging",
> reify=True,
> )
> ```
>
> Through debugging, i see the includeme is run and the debugtoolbar panel 
> is enabled when using the following:
>
> with Configurator() as config:
> config.add_settings({"debugtoolbar.includes": 
> ["pyramid_debugtoolbar_dogpile"]})
> config.include("pyramid_debugtoolbar")
> ...
> app = config.make_wsgi_app()
> serve(app, host='0.0.0.0', port=6543)
>
> or
>
> def setUp(self):
> self.config = config = testing.setUp()
> config.add_settings({"debugtoolbar.includes": 
> ["pyramid_debugtoolbar_dogpile"]})
> config.include("pyramid_debugtoolbar")
>
> The request method, however, is not added.  No request methods I add 
> within the includeme persist.
>
> Adding the request method in the main application (copy paste that same 
> snippet) will work.
>
> Can anyone help set me on the right path ?
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/dec013a0-7188-4e6c-97fa-3e38494aefc4n%40googlegroups.com.

[pylons-discuss] `add_request_method` doesn't seem to work within an `includeme` anymore

[Jonathan Vanasco]

This is driving me a bit crazy.  I hope someone can see the issue or 
changelog I am missing in the current pyramid and debugtoolbar.

I am adding some typing and tests to a debug toolbar that has the following 
in the includeme:

```
def includeme(config: "Configurator"):
config.add_debugtoolbar_panel(DogpileDebugPanel)
config.add_request_method(
"pyramid_debugtoolbar_dogpile.setup_dogpile_logging",
"dogpile_logging",
reify=True,
)
```

Through debugging, i see the includeme is run and the debugtoolbar panel is 
enabled when using the following:

with Configurator() as config:
config.add_settings({"debugtoolbar.includes": 
["pyramid_debugtoolbar_dogpile"]})
config.include("pyramid_debugtoolbar")
...
app = config.make_wsgi_app()
serve(app, host='0.0.0.0', port=6543)

or

def setUp(self):
self.config = config = testing.setUp()
config.add_settings({"debugtoolbar.includes": 
["pyramid_debugtoolbar_dogpile"]})
config.include("pyramid_debugtoolbar")

The request method, however, is not added.  No request methods I add within 
the includeme persist.

Adding the request method in the main application (copy paste that same 
snippet) will work.

Can anyone help set me on the right path ?





-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/03343090-4714-4374-93b9-87394f1fa966n%40googlegroups.com.

[pylons-discuss] Re: Functional tetsing of an app using pyramid_cas (Apereo CAS)

[Jonathan Vanasco]

There are a few ways I have done this.

1-In the functional tests, just rely on the headers being set/unset.   Then 
test the full SSO to set headers on integrated tests.
2- If the SSO functions use the Requests library, you can use Responses to 
mock the response. This way you can simulate a SSO on the functional tests.

> tried setting the headers of a TestRequest.blank, but nothing works

You should debug why that's failing.




On Friday, July 28, 2023 at 3:40:56 AM UTC-4 Rafael Lopez wrote:

> I've started porting my apps away from pyramid_ldap to pyramid_cas  to 
> rely on the university's CAS SSO and match other apps developed in-house by 
> my department.
>
> I started on a smaller app, and while the transition went fine, I am stuck 
> with tests that cannot possibly pass or go on because as far as I can do, I 
> can't get my test user recognized as logged in.
>
> In normal usage, the way it works is : go to a restricted page => 
> forbidden_view => redirect to SSO, login there => redirected back with a 
> ticket => decode it => validate auth => view page. When testing, I can't 
> just pass a random user/password to the external SSO and expect it to work, 
> and using my own login/password is also out.
>
> I tried the suggestions from 
> https://docs.pylonsproject.org/projects/webtest/en/latest/testapp.html?highlight=authorization%20#modifying-the-environment-simulating-authentication,
>  
> tried setting the headers of a TestRequest.blank, but nothing works, I 
> can't get the app to recognize my test user as authenticated, so it will 
> try to redirect and fail not-gracefully. I could take a week or two and see 
> if I can manage to run an ersatz of my university's CAS server using the 
> docker image from Apereo, but I'd rather spend that time resting or working 
> on proper projects ;p
>
> I'm using Pyramid 1.10.8, yes I know I need to move on, but that's for 
> later. What would the best way to test the parts of my app that require 
> authentication without having to actually do a full CAS SSO authentication ?
>
> RL
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/1e4b78ac-fd7f-435d-9559-7bccbd612bban%40googlegroups.com.

Re: [pylons-discuss] pyramid_session_redis - v1.7.0rc1

[Jonathan Vanasco]

It's just mypy and dropping python2.

There was one small change, in which session ids must now be a string.  

Integrating mypy has been great.  It's been surfacing so many edge cases 
for me.

On Thursday, June 15, 2023 at 12:41:29 PM UTC-4 Mike Orr wrote:

> I can try it in my current project update. It will have a code freeze
> at the beginning of July. Do you expect a release by then? Although I
> don't need to upgrade it if the changes are mainy for mypy, which I
> don't use.
>
> On Tue, Jun 13, 2023 at 10:04 AM Jonathan Vanasco  
> wrote:
> >
> > I've released a new branch of pyramid_sesion_redis with an rc1 status: 
> 1.7.0rc1
> >
> > https://pypi.org/project/pyramid-session-redis/
> > https://github.com/jvanasco/pyramid_session_redis
> >
> > If anyone is able to test this on their staging systems, I would be very 
> appreciative. This has been working well for our needs, but the package is 
> implemented in many different ways.
> >
> > This is the fist version to drop py2 and pyramid1 support. The minimum 
> requirements are Py3.6 and Pyramid2.0
> >
> > Most of the changes are internal and have to deal with supporting mypy 
> and handling some edge-cases that mypy surfaced.
> >
> > Default usage should be fully backwards compatible. The only known 
> breaking change so far involves specifying a custom id_generator. The 
> library now requires session_ids to be strings, and will convert an integer 
> id into a string.
> >
> > --
> > You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to pylons-discus...@googlegroups.com.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/632b73fe-818f-42b3-b035-b61fb518df85n%40googlegroups.com
> .
>
>
>
> -- 
> Mike Orr 
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/2ede9be4-e0d8-459d-abed-bdf799a52099n%40googlegroups.com.

[pylons-discuss] pyramid_session_redis - v1.7.0rc1

[Jonathan Vanasco]

I've released a new branch of pyramid_sesion_redis with an rc1 status:  
1.7.0rc1

 https://pypi.org/project/pyramid-session-redis/
 https://github.com/jvanasco/pyramid_session_redis

If anyone is able to test this on their staging systems, I would be very 
appreciative.  This has been working well for our needs, but the package is 
implemented in many different ways.  

This is the fist version to drop py2 and pyramid1 support.  The minimum 
requirements are Py3.6 and Pyramid2.0

Most of the changes are internal and have to deal with supporting mypy and 
handling some edge-cases that mypy surfaced.  

Default usage should be fully backwards compatible.  The only known 
breaking change so far involves specifying a custom id_generator.  The 
library now requires session_ids to be strings, and will convert an integer 
id into a string.  

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/632b73fe-818f-42b3-b035-b61fb518df85n%40googlegroups.com.

Re: [pylons-discuss] webob SignedSerializer questions

[Jonathan Vanasco]

Thanks for the quick response.

After digging through this for a bit, I think there just may be a lot of 
artifacts in here from years of code updates and specs changing.

In webob's response.Response, set_cookie accepts a string 
(https://github.com/Pylons/webob/blob/main/src/webob/response.py#L1023-L1028),
then encodes it to utf-8 bytes (
https://github.com/Pylons/webob/blob/main/src/webob/response.py#L1086 
/ https://github.com/Pylons/webob/blob/main/src/webob/util.py#L47-L51 )
and invokes webob.cookies's make_cookie  
(https://github.com/Pylons/webob/blob/main/src/webob/cookies.py#L492 )
which does a lot of work with bytes.. but then serializes it to text 
(https://github.com/Pylons/webob/blob/main/src/webob/cookies.py#L286)

_parse_cookie is dealing with latin-1 
(https://github.com/Pylons/webob/blob/main/src/webob/cookies.py#L201-L202)
mutate_header is dealing with ascii, utf-8, latin-1 
(https://github.com/Pylons/webob/blob/main/src/webob/cookies.py#L56) for 
different bits of the spec

It's just really interesting and seems to be all over.  I'm familiar with 
most of the specs, so understand the ascii requirements for names, but it's 
just hard to decipher from scratch.

The thing that triggered me on this path, is this behavior:
SignedSerializer expects strings 
(https://github.com/Pylons/webob/blob/main/src/webob/cookies.py#L663-L688) 
and returns bytes.
But the default serializer converts strings to bytes 
(https://github.com/Pylons/webob/blob/main/src/webob/cookies.py#L620) 
The output of that is commonly used as cookie.  
While set_cookie deals expects strings 
(https://github.com/Pylons/webob/blob/main/src/webob/response.py#L1001-L1029).  
It immediately turns it into bytes 
(https://github.com/Pylons/webob/blob/main/src/webob/response.py#L1086), 
and sends it to make_cookie
make_cookie does a lot of work on bytes 
(https://github.com/Pylons/webob/blob/main/src/webob/cookies.py#L492)
and then invokes a serialization  
(https://github.com/Pylons/webob/blob/main/src/webob/cookies.py#L613)
said serialization works on bytes, and returns strings 
( https://github.com/Pylons/webob/blob/main/src/webob/cookies.py#L319)
At this point I lose track of the next few steps.

It just seems very odd to me that there are so many conversions going on, 
without detailed documentation disclosing why.





On Monday, June 12, 2023 at 4:38:05 PM UTC-4 Bert JW Regeer wrote:

>
> On Jun 12, 2023, at 12:42, Jonathan Vanasco  wrote:
>
> I've been updating my session library to utilize typing/mypy and this 
> implementation detail surfaced between mypy integration and some new tests.
>
> I hope someone can enlighten me on this and some best testing practices...
>
> Webob's SignedSerializer expects to dump to, and load from, `bytes`.  The 
> default behavior is handled by taking json.dumps/loads - which operates on 
> strings - and transcoding it between bytes and string. (See 
> https://github.com/Pylons/webob/blob/main/src/webob/cookies.py)
>
> Webob's set_cookie is designed to accept strings, and transcodes it to 
> bytes with utf8 encoding.
> (https://github.com/Pylons/webob/blob/main/src/webob/response.py)
>
> Webob's request does a lot of transcoding between bytes and str for the 
> cookies, and both utf8 and latin1 charsets are used.
>
> Can anyone share why these design decisions were made?  There just seems 
> to be a lot of needless transcoding going around, and the design decisions 
> in webob/pyramid tend to be very deliberate and thought out - so I must be 
> missing something.  
>
> Can anyone also share exactly why there are some latin-1 charsets in the 
> webob cookie logic?
>
>
> I can’t directly comment on what part of the code you are looking at, but 
> it is likely because Cookies are set using HTTP headers, and HTTP headers 
> are limited in subset to latin-1 only (minus a bunch of other 
> characters/control characters/whatnot). The reality is that they tend to 
> get mangled and eaten alive, and or modified every which way. The only safe 
> set is ASCII characters.
>
> The same is true for other parts of the HTTP request and response.
>
>
> Thank you in advance.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/c2c36996-39c3-434a-91a9-a77bb2343027n%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/pylons-discuss/c2c36996-39c3-434a-91a9-a77bb2343027n%40googlegroups.com?utm_medium=email_source=footer>
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe f

[pylons-discuss] webob SignedSerializer questions

[Jonathan Vanasco]

I've been updating my session library to utilize typing/mypy and this 
implementation detail surfaced between mypy integration and some new tests.

I hope someone can enlighten me on this and some best testing practices...

Webob's SignedSerializer expects to dump to, and load from, `bytes`.  The 
default behavior is handled by taking json.dumps/loads - which operates on 
strings - and transcoding it between bytes and string. 
(See https://github.com/Pylons/webob/blob/main/src/webob/cookies.py)

Webob's set_cookie is designed to accept strings, and transcodes it to 
bytes with utf8 encoding.
(https://github.com/Pylons/webob/blob/main/src/webob/response.py)

Webob's request does a lot of transcoding between bytes and str for the 
cookies, and both utf8 and latin1 charsets are used.

Can anyone share why these design decisions were made?  There just seems to 
be a lot of needless transcoding going around, and the design decisions in 
webob/pyramid tend to be very deliberate and thought out - so I must be 
missing something.  

Can anyone also share exactly why there are some latin-1 charsets in the 
webob cookie logic?

Thank you in advance.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/c2c36996-39c3-434a-91a9-a77bb2343027n%40googlegroups.com.

[pylons-discuss] Re: Question about Pyramid + ZODB + RelStorage performances

[Jonathan Vanasco]

> So, does anyone have any idea about the origin of this increase in 
PostgreSQL activity and performance decrease?

Since pg_stat_activity is not showing anything, my typical troubleshooting 
is this:


1. Have your app log the exact queries, along with the time it takes to 
run.  You can do this yourself, or with the debugtoolbar.   I wrote a few 
debugtoolbar extensions that can help; they not only track these things, 
but expose the data as .csv files via an API. They may not work with the 
current version of pyramid.  My production environment is a bit behind.
https://github.com/jvanasco/pyramid_debugtoolbar_api_sqlalchemy
https://github.com/jvanasco/pyramid_debugtoolbar_api_performance

2. Wait for things to get jammed again.

3. When things get jammed, run the slow queries as an EXPLAIN in a pg 
console. ** make sure you do this while the app is jammed **.  e.g.  
 "EXPLAIN {QUERY}" 

4. Compare this EXPLAIN strategy to running the same query on a freshly 
restarted postgresql, and again on a freshly started app.

5. Also look at the history of where things slowed down, and see if it is 
happening after a "very different" query

In my experience, this stuff often happens because of memory and index 
loading, and can be fixed by either creating partial indexes on the 
database and hinting the queries to use those indexes OR increasing the 
memory allocation.

The most typical causes I've seen is the app runs really fast while a 
certain query/index is being used (query A), however running a "different" 
query (query B) will unload all of the indexes/data for Query A from memory 
as postgres needs to load the data for Query B. This can often be fixed by 
trying to minimize the footprint of indexes used for each query, and making 
sure both queries use indexes.

I've also had situations where many "test indexes" were still on a table, 
and writing 1 row of data was really writing 30+ rows instead of 6 rows.  
(The first row is the data, the additional ones are the indexes).  There 
have also been situations where the rows have triggers and check 
constraints that involve other rows, however those triggers and check 
constraints did not utilize indexes, so they start scanning the entire 
table for that data. Doing this can also unload indexes used by the main 
query.  This sort of stuff gets buried from normal view, and people forget 
to check for it.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/5771cbaf-f437-4bb7-b1ce-20d0b763ef10n%40googlegroups.com.

[pylons-discuss] Re: Hypatia - drop Python 2, GitHub Actions?

[Jonathan Vanasco]

I can't speak for the maintainers, but the Pylons Projects have been 
standardizing onto Github Actions so there is appeal.  IIRC, support for 
projects has also been targeting 3.7+.

But as a personal preference, while I don't use this library myself, 
whenever I move a project onto GitHub actions I initially maintain support 
for 2.7 and 3.6 then drop those versions in a second commit. The reason is 
that many people still run 2.7/3.6 in production, so if a security fix is 
released for those versions the fix can be applied to a commit with a 
functional test environment.  I find it much easier to just spend an extra 
5-10 minutes setting up that environment in advance "just in case" while 
I'm already working on GitHub Actions, than spending 1-2 hours to suddenly 
re-familiarize myself with the CI environment/setup and trying to jam 
legacy Python support into it.

On Saturday, December 17, 2022 at 9:33:40 PM UTC-5 p...@thirdfloor.com.au 
wrote:

> Hi,
>
> I’ve been a user of Hypatia for a fair while now and have an additional 
> index type I’m just cleaning up at the moment to propose in a pull request 
> - it’s a spatial index based on a port of RBush (
> https://github.com/mourner/rbush) from JavaScript. As part of this I’ve 
> been testing with all supported Python 3 versions and 3.12 alpha versions 
> as I’d like to be able to take advantage of some of the Python 3 additions 
> and the code it is used in is only Python 3. I’ve also made some changes to 
> allow for running the tox tests via GitHub actions rather than using Travis 
> CI - it’s a little simpler with one less moving part and I’ve had recent 
> experience with Actions (
> https://github.com/pfw/hypatia/actions/runs/3722816192) which might be 
> useful. I’d like to get the docs building and published as well.
>
> How would people feel about dropping support for anything below 3.7? Given 
> there is a now a 0.4 version that is on PyPI I don’t think there is much 
> loss if future additions only work on Python3.
>
> Is there any appeal in using GitHub Actions for the tests?
>
> Regards,
> Peter Wilkinson
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/3464d329-6d19-4f9a-b323-78005e574ab3n%40googlegroups.com.

[pylons-discuss] Pyramid Authentication/Authorization Questions

[Jonathan Vanasco]

I have 2 large Pyramid applications that use an in-house 
authentication/authorization and request pre-processing system.  I'm in the 
middle of porting a legacy mod_perl app to Pyramid, and weighing the 
options of: (i) converting everything to more native Pyramid code, or (ii) 
abstracting the current system into something more extensible.  

One of two existing apps were originally built on Pylons, the other dates 
back to either before Pyramid's current auth existed, or at least was as 
robust as it is now.

Both apps use the Pylons style "handler" approach to writing view 
callables.  A base handler class exists with some core pre-processing logic:
* initialize some request vars
* handle any cookie processing (autologin cookie, preview access, etc)
* handle API vs Web request differences; setup context
* require HTTPS or specific TLS versions

The base handlers are then subclassed into Authorization policies, topline 
options like : LoggedIn/ LoggedOut/ LoggedAny (e.g. 
pyramid.authorization.Everyone).  The "LoggedIn" are further subclassed 
into discrete permissions, and eventually we have a  `/views` directory 
with subclasses, and their methods are hooked into Pyramid via 
`@view_config` decorators.

This has worked well for 10+ years and the applications have scaled very 
well performance-wise, but the maintenance on them leaves a bit to be 
desired as we need to upgrade/patch each application separately for most 
auth related issues and improvements.  

I'm wondering if there are any components in Pyramid2 that may be used to 
develop a simpler-to-manage replacement for existing apps and give a good 
blueprint for the future.

I think a bunch of the pre-processing could be handled on Tweens (much of 
this is invoking or manipulating request properties configured via 
`add_request_method`).  

The big disconnect I keep having with Pyramid's Auth(s) system(s) are:

* requiring an unauthenticated user.  
* applying different styles of required authorizations

Our authz tend to be more complex than the standard Pyramid examples and 
any open source Pyramid app I've seen.  I think the easiest way to describe 
it would be like Facebook's group policies:

* user type - normal, group administrator, group owner, app employee
* group administrator permissions (computed or database stored based on 
user+group combination)
* user or group qualities (some things require a specific TOS version 
optin, others require a photo to have been uploaded)

Beyond that, we are also able to use the subclasses to set the failure 
policies - e.g. where the redirects are routed to - as these are logically 
grouped by the handlers. 

The app I am currently porting does not have all these constraints, and is 
much simpler, but this seems like the perfect time to rethink and 
potentially redeploy some legacy code.

Has anyone worked on something similar and can share tips, or seen any 
potentially similar open source projects we can look to for inspiration?














-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/a04332da-c249-4426-8d3a-1b9126dd64cen%40googlegroups.com.

Re: [pylons-discuss] Re: SQLAlchemy 2.0 support

[Jonathan Vanasco]

That should be fine.  If you see any warnings, that's unfortunately more 
than most developers see!


On Monday, November 28, 2022 at 4:14:16 PM UTC-5 zsol...@gmail.com wrote:

> I just use "PYTHONWARNINGS=default", afaik that's all I need to do right? 
> It shows me all the Pyramid and waitress warnings.
>
>
>
> On 28. Nov 2022 at 19:53:27, Jonathan Vanasco  wrote:
>
>> On Sunday, November 27, 2022 at 1:23:21 PM UTC-5 zsol...@gmail.com wrote:
>>
>>> Great to know! About the warnings, I'm on 2.0 and it works, so either 
>>> some of those RemovedIn20Warning are not removed or none of them are left.
>>>
>>
>> The warnings are still there, you most likely have fully compatible code. 
>> Congrats!
>>
>> One small thing to look out for: SqlAlchemy does some extra stuff now to 
>> make sure warnings are visible.  I don't think zope.sqlalchemy or many 
>> other projects do this.  TLDR: Python started to hide "warnings" around 
>> 2010, and you need to opt-in to seeing them. This caused many issues with 
>> SqlAlchemy, because the "postgres" driver name had been deprecated for 10+ 
>> years but people were still using it because they missed the "warnings".  
>>
>> I mean I rewrote my queries to 2.0 style, but I've read that 1.x style 
>>> queries will continue to work, they are just removed from the documentation 
>>> now.
>>>
>>
>> Yes.  AFAIK, there is no planned deprecation for it.  Mike re-wrote the 
>> tutorial for 2.0 and we had a few people help out in older docs to remove 
>> all the 1.x query instructions.  IIRC, they should still be around in the 
>> FAQ and some various docs for legacy troubleshooting, but there is a 
>> decision for all new development to happen in the 2.0 API style and to help 
>> people migrate to the new syntax.
>>
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "pylons-discuss" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/pylons-discuss/sDMJlpQQedM/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> pylons-discus...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/0a5f033b-4bc5-45c7-88c1-f6c8c56a600cn%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/pylons-discuss/0a5f033b-4bc5-45c7-88c1-f6c8c56a600cn%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/297d2da8-f46d-4ced-b304-35d5c26beee0n%40googlegroups.com.

Re: [pylons-discuss] Re: SQLAlchemy 2.0 support

[Jonathan Vanasco]

On Sunday, November 27, 2022 at 1:23:21 PM UTC-5 zsol...@gmail.com wrote:

> Great to know! About the warnings, I'm on 2.0 and it works, so either some 
> of those RemovedIn20Warning are not removed or none of them are left.
>

The warnings are still there, you most likely have fully compatible code. 
Congrats!

One small thing to look out for: SqlAlchemy does some extra stuff now to 
make sure warnings are visible.  I don't think zope.sqlalchemy or many 
other projects do this.  TLDR: Python started to hide "warnings" around 
2010, and you need to opt-in to seeing them. This caused many issues with 
SqlAlchemy, because the "postgres" driver name had been deprecated for 10+ 
years but people were still using it because they missed the "warnings".  

I mean I rewrote my queries to 2.0 style, but I've read that 1.x style 
> queries will continue to work, they are just removed from the documentation 
> now.
>

Yes.  AFAIK, there is no planned deprecation for it.  Mike re-wrote the 
tutorial for 2.0 and we had a few people help out in older docs to remove 
all the 1.x query instructions.  IIRC, they should still be around in the 
FAQ and some various docs for legacy troubleshooting, but there is a 
decision for all new development to happen in the 2.0 API style and to help 
people migrate to the new syntax.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/0a5f033b-4bc5-45c7-88c1-f6c8c56a600cn%40googlegroups.com.

[pylons-discuss] Re: SQLAlchemy 2.0 support

[Jonathan Vanasco]

There was one potential incompatibility with transaction, but the 
zope.sqlalchemy team addressed it already ( 
see https://github.com/zopefoundation/zope.sqlalchemy/issues/60 ).  There 
are deprecation warnings that are still unhandled ( 
see https://github.com/zopefoundation/zope.sqlalchemy/pull/64 ) but 
everything else should be fine.

Most of the 2.0 sqlalchemy changes are in the engine and query systems, and 
pretty isolated from the support the cookiecutter and zope.sqlalchemy 
provide.
On Thursday, November 24, 2022 at 1:38:01 PM UTC-5 zsol...@gmail.com wrote:

> Hi,
>
> I'm starting a new project from cookiecutter and I wanted to use 
> SQLAlchemy 2.0 (pre-release currently). 
>
> So far I changed meta.py to 
>
> class Base(DeclarativeBase):
> metadata = my_metadata
>
> as well as changing the model to the new Mapped / mapped_column syntax.
>
> My question is that is there anything in pyramid_tm, transaction or 
> zope.sqlalchemy packages which needs to be changed, or this was all I 
> needed?
>
> SQLAlchemy changed a *lot* in 2.0, that's why I think the whole 
> transaction related part might be affected.
>
> If it's the case then I'll just go back to 1.4 and wait a year or so till 
> it matures. What do you think?
>
> Zsolt
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/bb45eb53-c263-4d35-af74-60691ca010d5n%40googlegroups.com.

[pylons-discuss] feedback? cookiecutter for maintaining larger projects

[Jonathan Vanasco]

I recently had to upgrade some legacy systems and build out a few new 
projects.

While doing this, I decided to standardize everything to a 
pattern/framework we've settled on over the past few years.  This involved 
creating a cookiecutter based on the standard Pyramid cookiecutter, and I 
decided to open-source it as I've found it very useful.

The cookiecutter does NOT generate a pyramid application, but instead 
generates the scaffolding to maintain complex Pyramid projects in version 
control. You still need to generate apps with the default cookiecutters (or 
import them into this scaffold)

It's available via MIT license here:

https://github.com/jvanasco/pyramid-cookiecutter-complex

How does it work?

The scaffold is designed to segment a Pyramid application into reusable 
components.  While 95% of Pyramid users will be ecstatic with the standard 
project layout, some of us have seen projects grow incredibly large or had 
numerous side-projects and micro-sites bolted onto them.  To handle this, 
the framework creates 3 python packages:  project_core for shared 
functions, project_model for sqlalchemy/etc and project_pyramid for the 
pyramid app.  

The intent is for the output of a standard pyramid cookiecutter (your app) 
to exist in "project_pyramid".  Various functions that are likely to be 
used across packages should be migrated to "project_core", and the 
sqlalchemy model migrated to "project_model".  Additional packages can be 
developed alongside these.

Several directories are used to organize server configuration files (e.g. 
nginx/apache), ssl certificates, sql schemas/triggers/views/etc, cronjobs, 
etc.

Fabric is used to manage various operations like setup and deployment. The 
cookiecutter only ships with some installation and setup routines, as well 
as some code-quality routines that simply run black/flake8 on all the 
python packages. 

We also use Fabric for deployment and integrated testing, but those 
routines are all too specialized across our projects and I couldn't really 
abstract them.

I'll make a few updates over the next week (or so) that better document 
some functions and try to open source more.   The framework uses 
environment variables to change deployment context (development vs 
production), and file based semaphores to handle downtime (e.g. a Fabric 
command can touch a file to tell nginx we're in maintenance mode and serve 
a static site instead).  

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/fbe3048d-a140-4a11-a450-0b05b3191f45n%40googlegroups.com.

Re: [pylons-discuss] [ANN] web-top

['Jonathan Vanasco' via pylons-discuss]

This looks interesting. I'll try to test it on uwsgi and contribute a PR if 
it works.  If anyone can beat me to it though...
On Wednesday, August 24, 2022 at 4:44:09 AM UTC-4 mi...@redinnovation.com 
wrote:

> Hi Thierry,
>
>
>> That seems to be a nice project!
>> I would like to provide an integration package for Pyramid (probably 
>> based on tweens)!!
>> Would it be better to create a dedicated module in web-top, or to create 
>> a new "pyramid-top" package?
>>
>
> I am in fact using webtop at the front of Pyramid (through Gunicorn):
>
> Please 
>
> - drop a folder `top/pyramid" where you can add Pyramid code and 
>
> - then necessary unit test
>
> - add optional dependencies with `top-framework[pyramid]` label
>
> - Add a documentation page on how to integrate with Pyramid
>
> - Open a PR
>
> Here is the example code for Gunicorn:
>
>
> https://github.com/tradingstrategy-ai/top-framework/blob/master/top/gunicorn/hooks.py
>
> Because it is WSGI it should be pretty much copy-paste for Pyramid.
>
> Br,
> Mikko
>  
>
>>
>> Best regards,
>> Thierry
>> -- 
>>   https://www.ulthar.net -- http://pyams.readthedocs.io
>>
>>
>> Le mer. 24 août 2022 à 00:14, Mikko Ohtamaa  a 
>> écrit :
>>
>>> Hi all,
>>>
>>> I am happy to announce a new project: web-top.
>>>
>>> Web-top is a UNIX top-like tool to live HTTP requests and responses of 
>>> any web server. It is built using Top Framework for Python 
>>> .
>>>
>>> [image: image.png]
>>>
>>>
>>> Sometimes you just need to log in to your web server and see what's 
>>> going on. web-top allows you to do. Instead of focusing on Application 
>>> Performance Management (APM), metrics or log storage web-top displays 
>>> what's currently happening in your web server. web-top is ideal for 
>>> observing and catching issues when they happen.
>>>
>>>
>>>- Installation 
>>>
>>> 
>>>- Usage 
>>>
>>>
>>> Features 
>>>
>>>- 
>>>
>>>Easily integrate with any web server
>>>- 
>>>
>>>Display active and recently completed HTTP requests/responses
>>>- 
>>>
>>>Live top-like monitoring
>>>- 
>>>
>>>Easily locate stalled requests, as requests not making progress are 
>>>sorted first and marked red
>>>- 
>>>
>>>IP address and country geolocation to understand who is using your 
>>>web service
>>>
>>> 
>>> Documentation 
>>>
>>>- Browse documentation 
>>>
>>>
>>> Community 
>>>
>>>- Join Discord for any questions 
>>>
>>>
>>>
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "pylons-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to pylons-discus...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/pylons-discuss/CAK8RCUug5zFRKp6xzv--bdyTbVsB9Vj2pVOZ7Ynsm0Aq%2BfdnVQ%40mail.gmail.com
>>>  
>>> 
>>> .
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discus...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWDTBx5jjxDgOB1YwpVKJgQ940m6SU1-Pg%2BaFnVqpcP8Qw%40mail.gmail.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/8d3e436a-86ab-4616-bcff-1dd6b96a57fen%40googlegroups.com.

[pylons-discuss] Re: Top like monitoring for Pyramid / Gunicorn

['Jonathan Vanasco' via pylons-discuss]

Have you tried using statsd ?  (https://github.com/statsd/statsd)

See 
* https://www.etsy.com/codeascraft/measure-anything-measure-everything/
* 
https://thenewstack.io/collecting-metrics-using-statsd-a-standard-for-real-time-monitoring/

Typically you log each request start and the stop reason (success, type of 
error, etc) and major actions -- e.g. login attempt + login result, inbound 
request, etc.  Everything looks nice in graphite, and you can see when you 
start having issues.  It's incredibly useful for monitoring the effects of 
pushing a new release to production, but it is often used for debugging 
logic issues as well.



On Sunday, August 21, 2022 at 2:23:33 PM UTC-4 mi...@redinnovation.com 
wrote:

> Hi all,
>
> I am running Pyramid + SQLAlchemy website on the top of Gunicorn. 
> Sometimes requests get stuck of the Pyramid application becomes 
> non-responsive for certain requests (not all). Despite good attempts using 
> Gunicorn statsd monitoring, I have had no good success to pin down these 
> problematic requests.
>
> Do you know any tools that could offer UNIX top like monitoring for the 
> currently active requests? I know this is a far shot - Gunicorn did not 
> seem to offer any interface in its documentation to offer anything like 
> this. And if this is not the case, which is unfortunately likely, what 
> would be nice way to pull start request and stop request events from 
> Pyramid to a separate CLI tool, so one could see when bad things start to 
> happen?
>
> Br,
> Mikko
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/3e762838-2bbe-4298-b969-1198a9ea60a6n%40googlegroups.com.

Re: [pylons-discuss] Re: using pyramid without an ORM ?

['Jonathan Vanasco' via pylons-discuss]

On Tuesday, August 16, 2022 at 11:45:24 AM UTC-4 Mike Orr wrote:
> It is rolling back in some of my testing when there's no
> insert/delete/update, but I want to make sure it always does, just in
> case something somehow modifies the database when we didn't intend to.
> It's not that big a deal but it's what I'd like. I'm not sure if
> SQLAlchemy is issuing rollback if there were no changes, or if it will
> always do so.

That's from SQLAlchemy. It will rollback if there were no database writes.  
SQLAlchemy is unaware of raw sql being a write operation, so you need to 
use the `mark_changed` function from zope.sqlalchemy.   This is a weird 
idiosyncrasy of SQLAlchemy and transaction - the transaction could be 
completely successful, but SQLAlchemy will rollback because there was no 
activity within it's scope. 

It sounds like you're trying to do the opposite of what the `transaction` 
package is designed to do.

The way I normally deal with situations like that is to control if 
SQLAlchemy joins the transaction or not.  In most projects, I only use the 
transaction on specific views that require this type of integration - such 
as anything that sends an email (pyramid_mailer integrates with pyramid_tm).

It also sounds like your concern is mostly in testing.  The approach I've 
started to standardize on is to have a database snapshot for tests and just 
recreate the database from that on every run.  If you just want to 
completely disable database commits though, you could have your test 
harness set up a SQLAlchemy event listener for "commit", and then issue a 
"rollback" within the event.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/771e180a-ca5b-4625-baf7-972d237ea45an%40googlegroups.com.

Re: [pylons-discuss] Re: using pyramid without an ORM ?

[Jonathan Vanasco]

I second what Michael said.  The sqlalchemy starter template is the right 
way to go.

The major thing this template does, is provide you with the glue between a 
SQLAlchemy "Session" and the pyramid request.  See : 
https://github.com/Pylons/pyramid-cookiecutter-starter/blob/latest/%7B%7Bcookiecutter.repo_name%7D%7D/%7B%7Bcookiecutter.repo_name%7D%7D/sqlalchemy_models/__init__.py#L87-L127

If you run pyramid_tm (https://pypi.org/project/pyramid-tm/) you can then 
use zope.sqlalchemy (https://pypi.org/project/zope.sqlalchemy/) to bind the 
session to the transaction.

You don't need to use SQLAlchemy's ORM.  You can just use SQLAlchemy Core 
(https://docs.sqlalchemy.org/en/14/core/) to do everything.  You can also 
access the underlying psycopg2 connections through SQLAlchemy when you want.

There is not a lot of code to rewrite if you want to eliminate SqlAlchemy 
and start from scratch - probably only 100-400 lines worth, and as Arndt 
Droullier said it is pretty straightforward -- but you're going to have to 
maintain that, and the entrypoints are somewhat rough.  Even in that 
situation, starting with the sqlalchemy cookiecutter is your best option as 
it will give you a better idea as to where your code needs to go.


On Thursday, August 11, 2022 at 12:38:36 PM UTC-4 mmer...@gmail.com wrote:

> As Laurent said, if I were starting from scratch a new project I would 
> personally use the pyramid-cookiecutter-starter sqlalchemy and then rip out 
> what I didn’t need but only one part of that cookie cutter is the ORM. It 
> has several other features that you need to solve one way or another so 
> you’d at least want to know what you’re getting rid of. It is modular and 
> you can pick the layers you want.
>
>  

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/fd53e026-af8b-45ed-b4fe-df073e50f0e7n%40googlegroups.com.

Re: [pylons-discuss] Pyramid, partial and resumable downloads, HTTP Accept Ranges

['Jonathan Vanasco' via pylons-discuss]

^ ditto

On Tuesday, June 28, 2022 at 6:28:23 PM UTC-4 the...@luhn.com wrote:

> Easiest way is to not do it :)  I try to offload downloads/uploads to a 
> service like S3.  Let somebody else do the heavy lifting.  If 
> authentication is needed, I’ll generate a presigned URL in Pyramid and 
> redirect.
>
> AFAIA, Accept-Ranges is the only real HTTP solution for 
> resumable/multipart downloads.
>
> If you’re using nginx in front of gunicorn (which you should be), you can 
> have nginx cache the response and handle Range for you.  
> https://www.nginx.com/blog/smart-efficient-byte-range-caching-nginx/  You 
> could also skip Pyramid altogether and serve file directly with nginx, 
> using the auth_request module if you need auth.  
> https://nginx.org/en/docs/http/ngx_http_auth_request_module.html 
>
> If none of that is feasible, rolling your own FileResponse that handles 
> Range shouldn’t be too difficult, although personally that’d be my very 
> last option.
>
> — Theron
>
>
>
> On Jun 28, 2022, at 1:01 PM, Mikko Ohtamaa  
> wrote:
>
> Hi,
>
> What would be an easy way to support partial downloads / resumes with 
> Pyramid? Namely HTTP Accept Ranges come to my mind, but there could be 
> others.
>
> - Serving files directly from FS using FileResponse
>
> - Using gunicorn as a web server
>
> https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.5
>
> Br,
> Mikko
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAK8RCUurRhwaqBem%3DVi3OEYWw-oMZh0fhRqCqZcjPiGibyO8Rw%40mail.gmail.com
>  
> 
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/ddd5db00-5343-400d-881e-87e1a06e1548n%40googlegroups.com.

Re: [pylons-discuss] pserve run in a child process under Fabric/Invoke + VirtualEnv ?

['Jonathan Vanasco' via pylons-discuss]

Well, it was worth a shot and is giving me some ideas to try.  (I just 
tested this myself too, just to be sure it wouldn't work.)

On Wednesday, June 8, 2022 at 8:24:44 PM UTC-4 mmer...@gmail.com wrote:

> Well I might have answered too hastily. Seems the first process is 
> controlled by fabric and not virtualenv. 
>
> - Michael
>
> On Jun 8, 2022, at 19:23, Michael Merickel  wrote:
>
> I doubt you’ll have the same problem with “python -m 
> pyramid.scripts.pserve dev_admin.ini” but would be good to find out. The 
> process list appears to be due to the virtualenv wrapper around a console 
> script which this should skip. 
>
>
>
> - Michael
>
> On Jun 8, 2022, at 15:31, 'Jonathan Vanasco' via pylons-discuss <
> pylons-...@googlegroups.com> wrote:
>
> I doubt anyone here may have experienced this, but I've run out of 
> resources to explore on this...
>
> We use Fabric (fabfile.org) to automate a lot of things. It is great.
>
> I built a new routine in it this week, and I can't get it to clean up 
> properly.  The routine simply spins up an admin version of a pyramid app, 
> then hits some API endpoints to POST some filesystem data to it.
>
> This is executed in a virtualenv. The problematic part of the routine is 
> roughly...
>
> @task
> def import_data(c):
> with c.cd("localpath/to/pyramid_app"):
> proc_server = c.run("pserve dev_admin.ini", replace_env=False, 
> asynchronous=True)
>
> The issue is that I see two different processes on the operating system:
> * cd localpath/to/pyramid_app && pserve dev_admin.ini
> * /Library/Frameworks...Python /virtualenv/bin/pserve dev_admin.ini
>
> asynchronous is used because running pyramid would otherwise block 
> forever.  i just analyze the process stderr for the "Serving" line, and 
> continue once it is emitted. 
>
> In fabric, I can access the FIRST process via `proc_server.runner` and I 
> can stop/kill/terminate that -- but that leaves the second process 
> running.  That second process is one PID higher, and is the actual process 
> that is running Pyramid and bound to the ports.  
>
> I have a temporary workaround where I increase the PID by 1 and 
> `c.run("kill %s" % pid2)` that process, but this is janky.
>
> Has anyone encountered this before or have an idea on how to better handle 
> this?
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/45ca3e17-e8a7-459a-ac69-492b3fadaeccn%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/pylons-discuss/45ca3e17-e8a7-459a-ac69-492b3fadaeccn%40googlegroups.com?utm_medium=email_source=footer>
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/30f90196-e89c-45fb-8ae0-3918a63f4768n%40googlegroups.com.

[pylons-discuss] pserve run in a child process under Fabric/Invoke + VirtualEnv ?

['Jonathan Vanasco' via pylons-discuss]

I doubt anyone here may have experienced this, but I've run out of 
resources to explore on this...

We use Fabric (fabfile.org) to automate a lot of things. It is great.

I built a new routine in it this week, and I can't get it to clean up 
properly.  The routine simply spins up an admin version of a pyramid app, 
then hits some API endpoints to POST some filesystem data to it.

This is executed in a virtualenv. The problematic part of the routine is 
roughly...

@task
def import_data(c):
with c.cd("localpath/to/pyramid_app"):
proc_server = c.run("pserve dev_admin.ini", replace_env=False, 
asynchronous=True)

The issue is that I see two different processes on the operating system:
* cd localpath/to/pyramid_app && pserve dev_admin.ini
* /Library/Frameworks...Python /virtualenv/bin/pserve dev_admin.ini

asynchronous is used because running pyramid would otherwise block 
forever.  i just analyze the process stderr for the "Serving" line, and 
continue once it is emitted. 

In fabric, I can access the FIRST process via `proc_server.runner` and I 
can stop/kill/terminate that -- but that leaves the second process 
running.  That second process is one PID higher, and is the actual process 
that is running Pyramid and bound to the ports.  

I have a temporary workaround where I increase the PID by 1 and 
`c.run("kill %s" % pid2)` that process, but this is janky.

Has anyone encountered this before or have an idea on how to better handle 
this?

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/45ca3e17-e8a7-459a-ac69-492b3fadaeccn%40googlegroups.com.

[pylons-discuss] Re: Waitress and multiprocess pooling

[Jonathan Vanasco]

- What's the recommended modern multiprocess enabled web server to do more 
scaleable Pyramid hosting?

I like uWSGI, but others like gunicorn and I think there is another popular 
option. 

Regardless of the server you choose, please be aware they may (though I am 
pretty sure they all will) all cause issues with your current codebase due 
to the characteristics of copy-on-write memory optimization.  Usually the 
issues I've seen are with database connections and random seed generators.  
Most of the libraries you are probably using will support correcting this 
with specific calls to make after the process forks.  If you're using 
SqlAlchemy, look at Engine.dispose() and most crypto libraries have an 
`atfork` function. 

I just wanted to put that on your radar now, so you can audit your code for 
this stuff if you decide to switch your deployment.

On Thursday, May 12, 2022 at 4:46:55 PM UTC-4 mi...@redinnovation.com wrote:

> Hi,
>
> I am running a Pyramid based website and now traffic is picking up. I feel 
> I might bump the problems of Waitress scalability soon. As far as I 
> understand, Waitress does not offer multiple process pooling modes, only 
> threading. This, combined with Python GIL, might cause problems if requests 
> start to be more and more CPU bound. 
>
> - Am I correct and Waitress is limited by Python threading?
>
> - What's the recommended modern multiprocess enabled web server to do more 
> scaleable Pyramid hosting?
>
> Thank you,
> Mikko
>
> Ps. Site is here, also we are hiring if anyone is looking for gigs or jobs 
> in Pyramid + SQLAlchemy:
>
> https://tradingstrategy.ai/
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/20f1a129-1853-412b-a707-ebeb11db10e2n%40googlegroups.com.

Re: [pylons-discuss] Downsides of committing by default?

[Jonathan Vanasco]

IMHO it's not a micro-optimization. RDBMs systems will often take a 
performance hit on the COMMIT vs rollback when there are multiple 
simultaneous transactions, and it can cause issues on clustered/replicant 
systems.

I often forget about this too. The techniques that have worked for me:

* using sqlalchemy events to issue a mark_changed on the `before_execute` 
event. sometimes i'll have the event parse the statement sent to the 
backend for an insert/update/etc. 
* never using `session.execute` directly. instead i use a helper function 
that takes the session and execute params, it then does the work and 
applies mark changed.

I've done a few other things too, but those are the ones I recall the most

On Tuesday, May 3, 2022 at 12:25:46 AM UTC-4 mmer...@gmail.com wrote:

> My understanding is that it’s a micro optimization that a rollback is 
> slightly more efficient than a commit. I’m not aware of any good arguments 
> against defaulting to changed. Would need to ask the zope.sqlalchemy folks 
> for some insights. 
>
> - Michael
>
> On May 2, 2022, at 17:55, Theron Luhn  wrote:
>
> I’m using a cookiecutter-based Pyramid+zope.sqlalchemy+SQLAlchemy stack. 
>  More and more lately I’ve been skipping the ORM and using Core for write 
> operations, and I frequently run into issues where I forget to mark_changed 
> and zope.sqlalchemy ROLLBACKs by default:
>
>
> By default, zope.sqlalchemy puts sessions in an ‘active’ state when they 
> are first used. ORM write operations automatically move the session into a 
> ‘changed’ state. This avoids unnecessary database commits. Sometimes it is 
> necessary to interact with the database directly through SQL. It is not 
> possible to guess whether such an operation is a read or a write. Therefore 
> we must manually mark the session as changed when manual SQL statements 
> write to the DB.
>
>
> The docs go on to describe how I can change the behavior to COMMIT by 
> default:
>
> If this is a problem you may register the events and tell them to place 
> the session in the ‘changed’ state initially.
>
>
> My question:  Is there any downside to COMMITing by default?  I assume 
> there’s a good reason why the default is to ROLLBACK.  I’m using 
> PostgreSQL, but I’d be interested in hearing about how this affects RDBMSs 
> too.
>
> — Theron
>
>
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/598CAFEC-A22F-47E1-8AFE-7B06F29EC5CF%40luhn.com
>  
> 
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/82d0f7e8-9aea-408a-8fcd-141251cae390n%40googlegroups.com.

Re: [pylons-discuss] Re: pyramid_tm 2.5 has been released

['Jonathan Vanasco' via pylons-discuss]

No worries. I'm just glad it's basically a typo and you didn't lose a chunk 
of your build process!

On Wednesday, March 16, 2022 at 4:14:27 PM UTC-4 Steve Piercy wrote:

> Apologies. The correct link is here:
>
> https://docs.pylonsproject.org/projects/pyramid_tm/en/latest/changes.html
>
> --steve
>
>
> On 3/16/22 10:10 AM, 'Jonathan Vanasco' via pylons-discuss wrote:
> > I assume this message is automated by a release script, and that is out 
> of date.
> > 
> > The changes and latest aren't rendered onto those docs, and the anchors 
> don't exist.
> > 
> > The changes do appear here:
> > 
> > https://github.com/Pylons/pyramid_tm/blob/master/CHANGES.rst
> > 
> > 
> > On Sunday, March 13, 2022 at 4:01:18 AM UTC-4 Steve Piercy wrote:
> > 
> > pyramid_tm 2.5 has been released.
> > 
> > The full changelog is here:
> > https://docs.pylonsproject.org/projects/pyramid_tm/en/latest/#changes <
> https://docs.pylonsproject.org/projects/pyramid_tm/en/latest/#changes>
> > 
> > What's New In pyramid_tm 2.5:
> > https://docs.pylonsproject.org/projects/pyramid_tm/en/latest/#id3 <
> https://docs.pylonsproject.org/projects/pyramid_tm/en/latest/#id3>
> > 
> > Documentation:
> > https://docs.pylonsproject.org/projects/pyramid_tm/en/latest/ <
> https://docs.pylonsproject.org/projects/pyramid_tm/en/latest/>
> > 
> > You can install it via PyPI:
> > 
> > pip install pyramid-tm==2.5
> > 
> > Enjoy, and please report any issues you find to the issue tracker at
> > https://github.com/Pylons/pyramid_tm/issues <
> https://github.com/Pylons/pyramid_tm/issues>
> > 
> > Thanks!
> > 
> > - pyramid_tm core developers
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to pylons-discus...@googlegroups.com  pylons-discus...@googlegroups.com>.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/4f8a1bf1-62cc-4105-8b5f-68ae4147be14n%40googlegroups.com
>  
> <
> https://groups.google.com/d/msgid/pylons-discuss/4f8a1bf1-62cc-4105-8b5f-68ae4147be14n%40googlegroups.com?utm_medium=email_source=footer
> >.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/c0a07c55-c407-4a4f-a2f9-d8650fd5e821n%40googlegroups.com.

[pylons-discuss] Re: pyramid_tm 2.5 has been released

['Jonathan Vanasco' via pylons-discuss]

I assume this message is automated by a release script, and that is out of 
date.

The changes and latest aren't rendered onto those docs, and the anchors 
don't exist.

The changes do appear here:

https://github.com/Pylons/pyramid_tm/blob/master/CHANGES.rst


On Sunday, March 13, 2022 at 4:01:18 AM UTC-4 Steve Piercy wrote:

> pyramid_tm 2.5 has been released.
>
> The full changelog is here:
> https://docs.pylonsproject.org/projects/pyramid_tm/en/latest/#changes
>
> What's New In pyramid_tm 2.5:
> https://docs.pylonsproject.org/projects/pyramid_tm/en/latest/#id3
>
> Documentation:
> https://docs.pylonsproject.org/projects/pyramid_tm/en/latest/
>
> You can install it via PyPI:
>
> pip install pyramid-tm==2.5
>
> Enjoy, and please report any issues you find to the issue tracker at
> https://github.com/Pylons/pyramid_tm/issues
>
> Thanks!
>
> - pyramid_tm core developers
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/4f8a1bf1-62cc-4105-8b5f-68ae4147be14n%40googlegroups.com.

[pylons-discuss] Re: Looking for best option for Pyramid deployment

['Jonathan Vanasco' via pylons-discuss]

We run Pyramid via uwsgi, behind an OpenResty server (Nginx fork with 
embedded lua interpreter).  

IMHO the "best" stack is somewhat dependent on your application.  We have a 
large application and the forking model of uwsgi - combined with many of 
its management hooks - lets us aggressively leverage a lot of efficiencies 
via the copy-on-write memory behavior.  gunicorn was not nearly as 
performant for us with our core app, however it has been more performant on 
other apps.  We've run some smaller apps via waitress as a test, and forgot 
about it for months - it worked perfectly with no complaints.

Caddy is certainly a decent server - and has some of the best https 
termination and ssl certificate management in the field.  It has a built in 
ACME client, and can store provisioned certificates in the cloud, which 
simplifies a lot of devops in clustered environments.

I stay away from Apache and personally don't recommend it.  Nginx (and 
caddy) is far better with concurrent requests and throughput; they also 
have much lighter memory footprints. We can run a lot more uwsgi processes 
behind Nginx than any Apache deployment option - which means we can scale 
to more processes on a node before having to scale onto more nodes in a 
cluster.  In my experience, from a devops management and billing 
perspective, Apache tends to be much more expensive.

On Sunday, February 20, 2022 at 8:53:02 AM UTC-5 tfl...@gmail.com wrote:

> Hi,
> I've built a custom content management framework based on Pyramid; it's a 
> classic web application, based on ZODB and RelStorage with a PostgreSQL 
> backend, a Redis cache and an Elasticsearch index, and I'm actually looking 
> for the best production deployment option.
> Until now, I always used Apache with mod_wsgi, and I'm actually quite 
> happy with it, but I'd like to know if there is a better option with better 
> performance!
> In development mode, I already tried using "native" Waitress server, a 
> GUnicorn server, and I just tried using Pypy with Waitress or GUnicorn but 
> performances were quite "disappointing"!
>
> Any advice?
>
> Best regards,
> Thierry
> -- 
>   https://www.ulthar.net -- http://pyams.readthedocs.io
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/0ef924de-ef7d-4152-bb70-b266f0a368c8n%40googlegroups.com.

Re: [pylons-discuss] Subscribing to connection lost / failed transaction event?

['Jonathan Vanasco' via pylons-discuss]

but hit Ctrl + C on curl a 
> second or two after you start curl:
>
> You should see something like the following:
>
> python client_disconnected.py
> 2022-03-08 20:12:01,081 INFO waitress Serving on http://0.0.0.0:8080
> 2022-03-08 20:12:04,206 DEBUG __main__ Starting computation for 0
> 2022-03-08 20:12:06,211 DEBUG __main__ Completed computation for 0
> 2022-03-08 20:12:06,211 DEBUG __main__ Starting computation for 1
> 2022-03-08 20:12:08,215 DEBUG __main__ Completed computation for 1
> 2022-03-08 20:12:08,216 DEBUG __main__ Remote client went away, processed 
> 2 items
> 2022-03-08 20:12:08,217 INFO waitress Client disconnected while serving /
>
> An app developer who knows that the clients are always going to be 
> directly connected, can add code similar to the above in their response 
> code and do these checks manually during their computation, and if they 
> raise an error, pyramid_tm will appropriate abort the transaction, and 
> pyramid should run the exception view machinery (although that response 
> will never make it back to the client, it should be possible to use it at 
> that point to do any extra cleanup or whatnot though)
>
> Hopefully Andrew Free this helps somewhat, in that it is possible, it’s 
> just extra code you have to write and be aware of, it is not something that 
> comes for free, and requires that you use waitress, and it requires that 
> you set the `channel_request_lookhead` flag, and it requires that you know 
> your clients are directly connected.
>
> Caveat emptor.
>
> Thanks,
> Bert JW Regeer
>
> > On Mar 7, 2022, at 13:05, Jonathan Vanasco  wrote:
> > 
> > Just to clarify the above comment, this concept isn't really possible 
> with any internet technology in general, not just Pyramid.
> > On Thursday, February 17, 2022 at 8:35:25 PM UTC-5 Bert JW Regeer wrote:
> > No, this is not possible *.
> > 
> > * Except under some very narrow circumstances, but none that are easy to 
> use or directly supported in Pyramid
> > 
> >> On Feb 17, 2022, at 13:12, Andrew Free  wrote:
> >> 
> >> Is there a way to subscribe to any events of a dropped/lost connection?
> >> 
> >> For example, if the user closed the browser window in the middle of a 
> request. I am using pyramid_tm and having a hard time finding a method for 
> this. I just want to run some code based on the request object state in the 
> event that the response doesn't make it back to the client and the 
> transaction does not complete. I've looked into the exception_view_config 
> and this doesn't appear to help. Would a tween be the best way to handle 
> this?
> >> Thanks.
> >> 
> >> 
> >> --
> >> You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
> >> To unsubscribe from this group and stop receiving emails from it, send 
> an email to pylons-discus...@googlegroups.com.
> >> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/cab03e28-c370-4dcb-917a-7b5d36e7a86fn%40googlegroups.com
> .
> > 
> > 
> > --
> > You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to pylons-discus...@googlegroups.com.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/113341b5-529b-4bb8-b1e8-5a3d28ce028dn%40googlegroups.com
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/d2f50f9a-1510-4e08-a49d-b0fd405cba1dn%40googlegroups.com.

Re: [pylons-discuss] Subscribing to connection lost / failed transaction event?

[Jonathan Vanasco]

Just to clarify the above comment, this concept isn't really possible with 
any internet technology in general, not just Pyramid. 
On Thursday, February 17, 2022 at 8:35:25 PM UTC-5 Bert JW Regeer wrote:

> No, this is not possible *.
>
> * Except under some very narrow circumstances, but none that are easy to 
> use or directly supported in Pyramid
>
> On Feb 17, 2022, at 13:12, Andrew Free  wrote:
>
> Is there a way to subscribe to any events of a dropped/lost connection? 
>
> For example, if the user closed the browser window in the middle of a 
> request. I am using pyramid_tm and having a hard time finding a method 
> for this. I just want to run some code based on the request object state in 
> the event that the response doesn't make it back to the client and the 
> transaction does not complete. I've looked into the exception_view_config 
> and this doesn't appear to help. Would a tween be the best way to handle 
> this?
> Thanks. 
>
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/cab03e28-c370-4dcb-917a-7b5d36e7a86fn%40googlegroups.com
>  
> 
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/113341b5-529b-4bb8-b1e8-5a3d28ce028dn%40googlegroups.com.

Re: [pylons-discuss] Best practices for running Pyramid (docker) and nginx (host) deployment

['Jonathan Vanasco' via pylons-discuss]

On Monday, December 13, 2021 at 9:14:30 PM UTC-5 the...@luhn.com wrote:

> 1)  pserve isn’t really comparable with gunicorn, its just a way to launch 
> a server, such as gunicorn or waitress.  You’re probably using waitress, 
> that’s what the Pyramid docs use.
>
> I personally use gunicorn, but many on this mailing list are using 
> waitress with success, so I think it’s a fine choice.
>

I wish I saw this thread last month!

I personally use uWSGI.  waitress, gunicorn and uWSGI are **all** great 
application servers (as are some others) - however they each have their own 
sets of advantages/strengths and drawbacks/weaknesses across: concurrency, 
latency, cpu, ram, etc.

Depending on your exact application and traffic, you may get a significant 
performance boost by using one platform over the others. you may also see 
no discernible difference between the platform options. 

IMHO: as waitress is the default and production ready, you generally don't 
really need to consider other platforms until you need to scale into more 
than two nodes (redundancy is good), start to run into issues with capacity 
(concurrency, latency, cpu, ram, etc), or are using some sort of automatic 
scaling system to deploy more nodes. In those cases, waitress still might 
be the best option for you - but doing an audit and comparative benchmark 
of your applications' use of resources is warranted.





 

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/f330c75c-07f2-4f2e-949a-4174806cd99fn%40googlegroups.com.

[pylons-discuss] pyramid_session_redis 1.6.3 released

[Jonathan Vanasco]

Pyramid Session Redis 1.6.3 is now available on Github and PyPi

* https://github.com/jvanasco/pyramid_session_redis
* https://pypi.org/project/pyramid-session-redis/

The new version was released to support developers who do not actively 
monitor deprecation warnings, and have not already adjusted their code to 
support Redis 4.0.0 (released yesterday) removing the previously deprecated 
"charset" and "errors" kwargs in favor of "encoding" and "encoding_errors", 
respectively. The Python core developers decided to silence deprecation 
warnings in the 2.7 and 3.2 releases to hide them from end-users – which 
has caused endless troubles for library/package maintainers who deprecate 
things, as these warnings must now be actively monitored.

Some logic was added to detect what version of Redis (python library) is 
installed, and act accordingly.  The test matrix was extended to test 
against versions of Redis that use each set of kwargs.

Pyramid Session Redis 1.6.2, released in August, added support for the new 
kwargs.  Users who have already migrated to that and use the new kwargs 
will not need to migrate.  All users of pyramid_session_redis are 
encouraged to update their install AND to audit their integrations to 
ensure they are not using outdated kwargs in their configuration.


-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/8d9fb1a4-18f4-413a-9ada-48688698f822n%40googlegroups.com.

[pylons-discuss] Re: keeping the tests out of the wheel

[Jonathan Vanasco]

About a year ago, I moved all my projects to the pattern that Pyramid uses 
itself...

* /src/PROJECT
* /tests
* /MANIFEST.in
* /pyproject.toml
* /setup.cfg
* /setup.py
* /tox.ini

That has eliminated 99% of the packaging issues I've encountered in the 
past.

IIRC, that became the standardized repo design for Python projects a few 
years ago, due to constant issues in package/file discovery.

On Tuesday, October 26, 2021 at 7:56:21 AM UTC-4 est...@augusta.de wrote:

> Hi,
>
> I'm using pyramid for some web application. I .whl files to install the 
> application inside a container.
>
> It is working perfectly. But setup.py bdist_wheel always add the 
> unittests to the wheel. Unittests reside in the tests folder in the same 
> folder as the application folder. So the tests get added in the root 
> folder of the wheel and when extracted. They are places in the 
> site_packages folder.
>
> How can I prevent the tests to be added.
>
> I tried
>
> packages=find_packages(exclude=('tests',)),
>
> They still are added
>
> i tried
>
> packages=['']
>
> The are not included. But some other folders like locale are missing now.
>
> i tried why Manifest.in. No success there either.
>
> Can someone give me some pointers, what I'm missing.
>
> regards
> Estartu
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/9d2e711c-8801-4236-8e8a-182c4f77d909n%40googlegroups.com.

Re: [pylons-discuss] SQLAlchemy create_engine future argument

['Jonathan Vanasco' via pylons-discuss]

> It looks like the `future` flag on Engine and the `future` flag on 
Session are two separate things.  So you need to add `future=True` to 
either sessionmaker or the Session constructor.

Confirming this.  There is also a `future` on `Connection` objects, but 
that is inherited from `Engine`.  Session and Engine/Connection are 
entirely unrelated.  Please see:

* 
https://docs.sqlalchemy.org/en/14/changelog/migration_20.html#migration-to-2-0-step-four-use-the-future-flag-on-engine
* 
https://docs.sqlalchemy.org/en/14/changelog/migration_20.html#migration-to-2-0-step-four-use-the-future-flag-on-session

> However, I did find self.request.dbsession.bind._is_future.
>

that is `Session.bind`, which will be an `Engine` or `Connection` object - 
not the Session itself.
 

> If I use the code below in __init__.py:main() before settings is passed to 
> Configurator():
>
> settings['sqlalchemy.future'] = True
>

Assuming you are using the cookiecutter setup, this only affects the 
Engine. Please see:

https://github.com/Pylons/pyramid-cookiecutter-starter/blob/latest/%7B%7Bcookiecutter.repo_name%7D%7D/%7B%7Bcookiecutter.repo_name%7D%7D/sqlalchemy_models/__init__.py#L15-L16

So, if this is the correct flag SQLAlchemy sets/uses, it appears to be 
> making the trip all the way into the view, which is what I wanted to 
> validate.
>

Your Engine is in future mode, but your Session is not.  As Theron noted, 
you need to change your sessionmaker construction.  If you are using the 
cookiecutter setup, this is what must be adjusted:

https://github.com/Pylons/pyramid-cookiecutter-starter/blob/latest/%7B%7Bcookiecutter.repo_name%7D%7D/%7B%7Bcookiecutter.repo_name%7D%7D/sqlalchemy_models/__init__.py#L19-L22

 
 

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/74d9b5e4-1006-4d70-b61b-af20d35c00bfn%40googlegroups.com.

[pylons-discuss] Re: debugtoolbar - don't intercept a raised exception ?

['Jonathan Vanasco' via pylons-discuss]

I *temporarily* solved this by modifying an existing tween.  The relevant 
code looks like this:

def my__tween(request):
_redirect = None
try:
response = handler(request)
except HTTPException as exc:
_redirect = exc
if _redirect:
return _redirect
return response

I'll likely migrate this to it's own tween for dev testing, unless I can 
figure out if the toolbar is loaded and run the code conditionally.  In any 
event, this approach lets me run my full test suite with the toolbar 
enabled.


On Friday, May 21, 2021 at 3:55:34 PM UTC-4 Jonathan Vanasco wrote:

> One of my CI tests deals with an edge case, in which a 
> HTTPMovedPermanently is raised in deeply nested code. This almost never 
> gets raised in Production - rules on the loadbalancer/gateway typically 
> catch it.
>
> With the debugtoolbar off, Pyramid serves the redirect.  With the 
> debugtoolbar enabled, it's caught by toolbar.py as an Exception and never 
> makes it to the block of code dealing with `intercept_redirects`.
>
> Has anyone encountered this before and developed a good workaround?  I 
> feel like this test used to pass, but I can't find anything on our change 
> history or Pyramid/Debugtoolbar's that would cause this change.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/9e810b38-4c1e-45a5-9ad4-b5d9f8983d77n%40googlegroups.com.

[pylons-discuss] debugtoolbar - don't intercept a raised exception ?

['Jonathan Vanasco' via pylons-discuss]

One of my CI tests deals with an edge case, in which a HTTPMovedPermanently 
is raised in deeply nested code. This almost never gets raised in 
Production - rules on the loadbalancer/gateway typically catch it.

With the debugtoolbar off, Pyramid serves the redirect.  With the 
debugtoolbar enabled, it's caught by toolbar.py as an Exception and never 
makes it to the block of code dealing with `intercept_redirects`.

Has anyone encountered this before and developed a good workaround?  I feel 
like this test used to pass, but I can't find anything on our change 
history or Pyramid/Debugtoolbar's that would cause this change.


-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/ea13f955-007b-403f-a009-1a4e7ac5d288n%40googlegroups.com.

Re: [pylons-discuss] Replacing Freenode?

['Jonathan Vanasco' via pylons-discuss]

just to be clear, my main concern is the majority of staff have quit -- so 
reliability is likely to be an issue. the only one I have not seen an 
announcement from is their infrastructure lead. 
On Wednesday, May 19, 2021 at 4:11:44 PM UTC-4 Steve Piercy wrote:

> We have not discussed it. It's a matter of inertia and the effort required 
> to move official Pylons Project support channels to something else. No one 
> wants to do that work as a volunteer.
>
> It is fine if someone takes the initiative to establish an unofficial 
> community-driven Pylons Project channel on whatever they like, Discord, 
> Slack, Gitter, carrier pigeon...
>
> --steve
>
>
> On 5/19/21 9:07 AM, Joshua Drake wrote:
> > Without getting into the politics of it...
> > 
> > OFTC could be a good choice. It is what Software in the Public Interest 
> uses which is behind such stalwarts as Debian, Libreoffice (in the U.S.) 
> and PostgreSQL.Org
> > 
> > On Wed, May 19, 2021 at 7:24 AM 'Jonathan Vanasco' via pylons-discuss <
> pylons-...@googlegroups.com <mailto:pylons-...@googlegroups.com>> wrote:
> > 
> > Are there any plans to replace Freenode, in light of the recent 
> developments?
> > 
> > See:
> > 
> https://boingboing.net/2021/05/19/freenode-irc-staff-quit-after-new-owner-seizes-control.html
>  
> <
> https://boingboing.net/2021/05/19/freenode-irc-staff-quit-after-new-owner-seizes-control.html
> >
> > https://libera.chat/ <https://libera.chat/>
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to pylons-discus...@googlegroups.com  pylons-discus...@googlegroups.com>.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/0d58265b-cc5d-467e-ae95-a2d5f5c4d1d3n%40googlegroups.com
>  
> <
> https://groups.google.com/d/msgid/pylons-discuss/0d58265b-cc5d-467e-ae95-a2d5f5c4d1d3n%40googlegroups.com?utm_medium=email_source=footer
> >.
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to pylons-discus...@googlegroups.com  pylons-discus...@googlegroups.com>.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAJvJg-SrpSoqYW6qMDAxqcvFW9F5rJzCHQO1ocwBOszUQeXFDQ%40mail.gmail.com
>  
> <
> https://groups.google.com/d/msgid/pylons-discuss/CAJvJg-SrpSoqYW6qMDAxqcvFW9F5rJzCHQO1ocwBOszUQeXFDQ%40mail.gmail.com?utm_medium=email_source=footer
> >.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/9dcb1fd9-35a8-49fc-b54c-5e1d18a888acn%40googlegroups.com.

[pylons-discuss] Replacing Freenode?

['Jonathan Vanasco' via pylons-discuss]

Are there any plans to replace Freenode, in light of the recent 
developments?

See:
  
  
https://boingboing.net/2021/05/19/freenode-irc-staff-quit-after-new-owner-seizes-control.html
https://libera.chat/

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/0d58265b-cc5d-467e-ae95-a2d5f5c4d1d3n%40googlegroups.com.

Re: [pylons-discuss] Problem between Pyramid's CSRF protection and Deform

['Jonathan Vanasco' via pylons-discuss]

They're not the same at all.

The difference is on purpose.

Janzert is correct, though his description may not necessarily be clear.

The following might make more sense:

The two functions do the following:

  pyramid.csrf.get_csrf_token(request)
discern active ICSRFStoragePolicy
invoke {ICSRFStoragePolicy.get_csrf_token()}

  request.session.get_csrf_token()
invoke {self.get_csrf_token()}

Because of that difference, the following happens.

  1. when using `LegacySessionCSRFStoragePolicy`, `
SessionCSRFStoragePolicy`, or `None` (which is one of those IIRC):
  a) pyramid.csrf.get_csrf_token always uses the Session
  b) request.session.get_csrf_token is just a shortcut to the above

  2. when using `CookieCSRFStoragePolicy`:
  a) pyramid.csrf.get_csrf_token always uses the dedicated Session 
cookie, as you instructed it to.
  b) request.session.get_csrf_token is referencing a csrf_token within 
the session itself. Pyramid is not configured to ever look in the session 
for a csrf, because you told it to use a dedicated session cookie.

The `session.get_csrf_token` method is really a helper function for Pyramid 
and add-ons to have a standard interface for stashing a CSRF token in the 
session.  it might be worth changing those interface names to have leading 
underscores, so the public doesn't rely on them.  Or maybe add a warning 
docstring that `pyramid.csrf.get_csrf_token` should be used by developers.

In any event, you are getting different results because you are telling 
pyramid to use a cookie for the csrf token, then using one method that 
queries that cookie (correct value!) and a second method that queries the 
active session for a token -- which is not tied to the pyramid csrf system 
in any way.
On Sunday, May 2, 2021 at 6:14:47 PM UTC-4 Steve Piercy wrote:

> They are effectively the same.
>
>
> https://docs.pylonsproject.org/projects/pyramid/en/latest/_modules/pyramid/csrf.html#LegacySessionCSRFStoragePolicy.get_csrf_token
>
> In your code, you have configured two session factories. I assume you get 
> the CSRF unique to each factory. ¯\_(ツ)_/¯
>
> --steve
>
>
> On 5/2/21 10:25 AM, Laurent Daverio wrote:
> > So, if I follow this line of reasoning, the way to get the same value
> > as in the template is to use :
> > 
> > from pyramid.csrf import get_csrf_token
> > print get_csrf_token(request)
> > 
> > and *not* :
> > 
> > print request.session.get_csrf_token()
> > 
> > Le dim. 2 mai 2021 à 19:11, Laurent Daverio  a écrit 
> :
> >>
> >> OK, I've been able to nail it down on a simple example : depending on
> >> the CSRF storage policy I use, "request.session.get_csrf_token()"
> >> (called from python or a template) and "get_csrf_token()" (called from
> >> a template) return the same value *or not*.
> >>
> >> - no storage policy => ok
> >> - LegacySessionCSRFStoragePolicy => ok
> >> - CookieCSRFStoragePolicy => ko
> >>
> >> I'm attaching my example, I called it "onefile.py", although I needed
> >> two files actually (one python file + one mako template). Sorry ;)
> >>
> >> Le mer. 28 avr. 2021 à 22:32, Laurent Daverio  a 
> écrit :
> >>>
> >>> Thank you Steve. I'll have to think about it, not that the code is
> >>> secret, just a matter of knowing what to post to be relevant.
> >>>
> >>> Le mer. 28 avr. 2021 à 22:10, Steve Piercy
> >>>  a écrit :
> 
>  It's difficult to say without your example. I've been using CSRF as 
> shown in the Deform demo without any issues.
> 
>  --steve
> 
> 
>  On 4/28/21 10:32 AM, Laurent Daverio wrote:
> > Hello List,
> >
> > I'd like to report a problem I've just encountered, occurring betwen
> > Pyramid's CSRF protection and Deform.
> >
> > Basically, I have a Pyramid 2.0 web app configured along the lines of
> > the "URL dispatch wiki tutorial"
> > (
> https://docs.pylonsproject.org/projects/pyramid/en/2.0-branch/tutorials/wiki2/authentication.html
> ),
> > with some Deform forms in it.
> >
> > The Deform Demo
> > (https://deformdemo.pylonsproject.org/pyramid_csrf_demo/) shows how 
> to
> > use a deferred value to create hidden field "csrf_token" in the
> > generated forms.
> >
> > But there's a problem: the token generated that way doesn't have the
> > same value as when I directly call get_csrf_token() in a template.
> >
> > As I don't have the time/energy to fully investigate the problem 
> right
> > now, I think I will just use a workaround: as I'm using Diazo as a
> > theming engine (awesome tech, btw), I think I will add a rule to
> > inject the token into every form. Should work.
> >
> > Still, I wanted to take the time to report the problem, in case it
> > could be useful.
> >
> > Laurent.
> >
> 
>  --
>  You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
>  To unsubscribe from this group and stop receiving emails 

Re: [pylons-discuss] how does pyramid know what "create" , "view", "edit" etc is ?

['Jonathan Vanasco' via pylons-discuss]

Extending and hoping to clarify Steve's example above, Pyramid users 
typically leverage the "predicates" to handle this mapping. by using the 
`request_method` predicate to map "GET" to "view", "POST" to "create", etc.
On Monday, May 10, 2021 at 4:27:09 PM UTC-4 grc...@gmail.com wrote:

>
> excellent !! 
>
> I am very happy with pyramid and some things don't make sense *at the 
> beginning* but the end result is a much cleaner / simpler / modular code.
>
> thank you so much everyone !! 
> On Monday, May 10, 2021 at 11:03:50 PM UTC+3 mmer...@gmail.com wrote:
>
>> The permission strings are arbitrary. For examples that are using ACLs 
>> (like the wiki tutorials) the only requirement is that the strings should 
>> match up to ACL entries that you are generating on your context objects. 
>> Pyramid does not care about the values of the strings and you could use 
>> "update" or "edit" or "foo". If a principal in the request and the 
>> permission on the view do not match an ACL entry then an HTTPForbidden 
>> exception is raised.
>>
>> On May 10, 2021, at 14:32, pzzcc  wrote:
>>
>> thank you for the input everyone.
>>
>> please correct me if I am wrong , does pyramid know what a ( *view* ) 
>> action  is ? 
>>
>> does it know that an Edit action ( is a form that is being POSTed or 
>> Restful call to update ?
>>
>> same goes for create , does it have a way to figure out that create is ( 
>> PUT )?
>>
>>
>> *in other words , if I go to a view and change the view config to have a 
>> permssion of ( update ) instead of ( edit ) ,*
>>
>> *and then go to principals  and update them accordingly , Pyramid it self 
>> wont care,  would it ?  *
>>
>> *can a view have more than one permission like ( update , create , view ) 
>> ? *
>>
>> *I am trying to figure out how it works so I can write a better code 
>> because I have gone through the wiki tutorial , it is great but it leaves 
>> you with a lot of question to be able to understand how things are put 
>> together .*
>>
>>
>> On Monday, May 10, 2021 at 12:30:03 PM UTC+3 Eldav wrote:
>>
>>> Hello, 
>>>
>>> you could have a look at the "Authorization" page of the SQLAlchemy + 
>>> URL dispatch wiki tutorial: 
>>>
>>>
>>> https://pyramid.readthedocs.io/en/latest/tutorials/wiki2/authorization.html
>>>  
>>>
>>> Basically : you define your permission as string via an ACL mechanism. 
>>> Your permissions may be global (e.g. all members of the "managers" 
>>> group get the "manage" permission), or defined via a route factory. 
>>> Route factories allow for policies such as: every authenticated user 
>>> can "view" a page, its author can "edit" it. They also allow you to 
>>> simplify the code of your views. 
>>>
>>> Hope this helps, 
>>>
>>> Laurent. 
>>>
>>> Le dim. 9 mai 2021 à 20:17, Thierry Florac  a écrit : 
>>> > 
>>> > Hi, 
>>> > Are you asking about the way to protect a view with a permission, or 
>>> about the way to grant this permission to a request? 
>>> > Best regards, 
>>> > Thierry 
>>> > -- 
>>> >  https://www.ulthar.net -- http://pyams.readthedocs.io 
>>> > 
>>> > 
>>> > Le dim. 9 mai 2021 à 19:00, pzzcc  a écrit : 
>>> >> 
>>> >> Hi, 
>>> >> 
>>> >> I am trying to wrap my head around some pyramid concepts and I am 
>>> trying to figure out how does a view config know what a permission like ( 
>>> view , edit , create ) is ? 
>>> >> 
>>> >> does it rely on the pyramid_tm r or the routes or what ? 
>>> >> 
>>> >> I know how to use them but I need to wrap my head againts some 
>>> concepts. 
>>> >> 
>>> >> thanks. 
>>> >> 
>>> >> -- 
>>> >> You received this message because you are subscribed to the Google 
>>> Groups "pylons-discuss" group. 
>>> >> To unsubscribe from this group and stop receiving emails from it, 
>>> send an email to pylons-discus...@googlegroups.com. 
>>> >> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/pylons-discuss/2b676239-b805-40d6-9ae2-1e4c60a9a7dcn%40googlegroups.com
>>> . 
>>> > 
>>> > -- 
>>> > You received this message because you are subscribed to the Google 
>>> Groups "pylons-discuss" group. 
>>> > To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to pylons-discus...@googlegroups.com. 
>>> > To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWCYnWP_Rrbgk1ZBP1JBUN8KNztgj5%3DJ_Q_8%2B_uvAXAv_A%40mail.gmail.com
>>> . 
>>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discus...@googlegroups.com.
>>
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/97b621fe-4b8b-4a44-884a-079813495ff4n%40googlegroups.com
>>  
>> 
>> .
>>
>>
>>

-- 
You received this message because you 

[pylons-discuss] pyramid_session_redis 1.6.0 released

['Jonathan Vanasco' via pylons-discuss]

pyramid_session_redis 1.6.0 is now released on Github and PyPi.

Notable changes:

* Pyramid2 is now supported.  
* Pyramid1.x and Python2 are still fully supported.
* The Session object has two new methods:
* Session.adjust_cookie_expires
* Session.adjust_cookie_max_age
* These methods ONLY provide a mechanism to adjust the cookie expiry 
values. They do not affect any Session logic.

Much thanks to Theron Luhn for offering a PR which preserved backwards 
compatibility in some ways better than my approach AND vastly improving the 
test matrix.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/2402f2de-f100-471b-a415-09261f2bbeecn%40googlegroups.com.

[pylons-discuss] debugging JSON encoder issues

['Jonathan Vanasco' via pylons-discuss]

I've run into this same problem a few times in unittest. I'm hoping someone 
else has a better solution.

1. The test harness uses webtest.TestApp to mount the Pyramid application 
2. A route has a payload with an object that can not be rendered into json
3. The traceback looks like the item below.

The problem is call stack doesn't have any information about my app-code 
that generated the render/exception.  I can match it to a route, via the 
line in the test harness, but that's it.  Because I have no idea where the 
return/response happened in the route logic, I often spend a lot of time 
with pdb breakpoints in the route trying to figure out what line caused it 
- as the return value could have been generated in multiple spots.

Has anyone figured out a standard way of dealing with this type of 
situation?



Traceback (most recent call last):

  File 
"/Users/jvanasco/webserver/os-projects-mine/project/tests/test_pyramid_app.py", 
line 6402, in test_post_required_json

res = self.testapp.get(



lots of webob/pyramid/etc items in the stack

... 

  File 
"/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyramid/renderers.py",
 
line 298, in default

raise TypeError('%r is not JSON serializable' % (obj,))

TypeError:  is not JSON 
serializable


-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/f11c3d71-a7f5-44c1-90eb-7600cb86cf73n%40googlegroups.com.

Re: [pylons-discuss] generating docs for views

['Jonathan Vanasco' via pylons-discuss]

Yes, it is much closer to your pyramid_openapi3 demos.  I'm not sure that 
project is a good candidate for my requirements or bandwidth, but I will 
dig into it - thanks!




On Friday, March 12, 2021 at 4:52:45 PM UTC-5 Steve Piercy wrote:

> My first thought was to use docstrings and Sphinx, but I don't think 
> that's what you want.
>
> It sounds like you want to document an API, something like 
> pyramid_openapi3.
>
> https://github.com/Pylons/pyramid_openapi3/
>
> RealWorld.io app API Demo (it might take a while to start up the app, as 
> it is on-demand only).
>
> https://pyramid-realworld.herokuapp.com/api
>
> --steve
>
>
> On 3/12/21 1:36 PM, 'Jonathan Vanasco' via pylons-discuss wrote:
> > I had a small Pyramid project which has grown into a much larger one.
> > 
> > Many Views service multiple Routes (multiple calls to `@view_config()`), 
> typically offering a default HTML version on a "bare" url, and a JSON 
> version with a ".json" suffix to the url.
> > 
> > Documenting these routes has become increasingly burdensome, and I 
> really need to simplify it.  For clarity, the docs are served by the 
> application itself and in a structured format.
> > 
> > Right now I have two types of documentation, and I'd like to consolidate 
> it into a single one.
> > 
> > * The original documentation was built off a Python file, 
> `/lib/docs.py`, which just notes the routes which have JSON endpoints, 
> gives an "about" docstring, and offers a dict about how the route functions 
> (GET vs POST, is it self-documenting, etc).
> > 
> > * As the project progressed, the routes started self-documenting.  Forms 
> were moved to POST only, with GET offering structured details 
> (instructions, required vs optional form elements, etc). Most of this is 
> defined by just returning a dict on GET operations.
> > 
> > I've been trying to figure out the best ways to consolidate this. 
> Perhaps using a custom decorator to declare and track this information into 
> a namespace?
> > 
> > Has anyone else worked on a scenario like this?  How did you handle it?
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to pylons-discus...@googlegroups.com  pylons-discus...@googlegroups.com>.
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/2cc3c09f-037a-4cf0-9a18-b464540a2305n%40googlegroups.com
>  
> <
> https://groups.google.com/d/msgid/pylons-discuss/2cc3c09f-037a-4cf0-9a18-b464540a2305n%40googlegroups.com?utm_medium=email_source=footer
> >.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/b2946103-6911-48d7-b352-b75911b96d9dn%40googlegroups.com.

[pylons-discuss] generating docs for views

['Jonathan Vanasco' via pylons-discuss]

I had a small Pyramid project which has grown into a much larger one.

Many Views service multiple Routes (multiple calls to `@view_config()`), 
typically offering a default HTML version on a "bare" url, and a JSON 
version with a ".json" suffix to the url.

Documenting these routes has become increasingly burdensome, and I really 
need to simplify it.  For clarity, the docs are served by the application 
itself and in a structured format.

Right now I have two types of documentation, and I'd like to consolidate it 
into a single one.

* The original documentation was built off a Python file, `/lib/docs.py`, 
which just notes the routes which have JSON endpoints, gives an "about" 
docstring, and offers a dict about how the route functions (GET vs POST, is 
it self-documenting, etc).

* As the project progressed, the routes started self-documenting.  Forms 
were moved to POST only, with GET offering structured details 
(instructions, required vs optional form elements, etc). Most of this is 
defined by just returning a dict on GET operations.

I've been trying to figure out the best ways to consolidate this. Perhaps 
using a custom decorator to declare and track this information into a 
namespace?

Has anyone else worked on a scenario like this?  How did you handle it?

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/2cc3c09f-037a-4cf0-9a18-b464540a2305n%40googlegroups.com.

Re: [pylons-discuss] Pyramid 2.0 released

['Jonathan Vanasco' via pylons-discuss]

For those that need 2.0 support with pyramid_session_redis, I have pushed a 
Pyramid2 compatible branch of pyramid_session_redis to 
https://github.com/jvanasco/pyramid_session_redis/tree/1.6-branch-concept_a

This branch is NOT likely to mature to an actual release, and is basically 
offered as a hotfix to bootstrap projects.  It supports Pyramid2 by 
dropping support for Python2 and Pyramid 1.x, both of which are still 
supported by my project.


-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/068396d5-a3f4-4c15-ab54-c7459968c8d9n%40googlegroups.com.

Re: [pylons-discuss] Pyramid 2.0b1 released

['Jonathan Vanasco' via pylons-discuss]

> I'd just say though that if you need the request and don't have it,
> it's often an indication that the program's structure is wrong.

I authored this change, and you are absolutely correct.  This technique is 
not ideal, but it is designed to stop several anti-patterns that happen 
with great frequency in many SQLAlchemy web integrations in a "less evil" 
but "supported" pattern.

A lot of web implementations will use global variables to access the active 
web request when dealing with "Model" concepts  (examples include 
Flask-SQLAlchemy and legacy Pylons) which is an anti-pattern; a lot of 
Pyramid applications used `get_current_request`, which is explicitly warned 
against.  

In this technique, the "info" dict - which is the official SQLAlchemy place 
for end-users to place data into a session - is used to stash the request.  
It's a bit odd -- the request has a session which has the request - but 
it's the safest way to make the request available to any SQLAlchemy 
object.  (note: this breaks once the session is closed or the object leaves 
the session)  This also makes testing much, much, much easier.

Models shouldn't need to be aware of the context, but sometimes it's the 
best option.  I specifically wrote it so people can  get to the 
`request.registry` to access application settings, like Michael mentioned.  

FWIW, my preferred technique is to structure a SQLAlchemy model independent 
of Pyramid (either as a separate package or part of a larger package at the 
same "level" as the Pyramid app), and have Pyramid/Celery/Commandline-tools 
etc import that model.  Each framework then notes in the "info" dict (i) 
which system SQLAlchemy is being run under and (ii) the current request or 
context. For the sake of simplicity, I left that stuff out of the PR. 

On Sunday, February 21, 2021 at 2:10:13 PM UTC-5 Mike Orr wrote:

> Oh, nifty. And good docstring.
>
> I'd just say though that if you need the request and don't have it,
> it's often an indication that the program's structure is wrong.
> Although there are certainly times when you need something like this,
> such as when you call a third-party function that doesn't take a
> request, then it calls you and you need the request but the caller
> hasn't passed it through.
>
> On Sat, Feb 20, 2021 at 3:42 PM Michael Merickel  
> wrote:
> >
> > It's in the cookiecutter's "get_tm_session" on the master branch. Which 
> is not the default branch.
> >
> > The purpose of the pattern is if your model objects needed access to 
> some settings or some other request properties. It's up to you to decide if 
> that's good or bad. Of course you don't need to use it.
> >
> > - Michael
> >
> > > On Feb 20, 2021, at 17:39, Mike Orr  wrote:
> > >
> > > On Sat, Feb 20, 2021 at 12:57 PM Michael Merickel  
> wrote:
> > >> Check out the new pattern of storing the request in the SQLAlchemy 
> session object for easier access in your model layer without threadlocals!
> > >
> > > Where is that? I don't see it in 'pyramid-cookiecutter-starter' or the 
> 2.0 docs.
> > >
> > > I do essentially the opposite. I have a request subclass with reified
> > > methods for external resources like 'request.sa_session()'.
> > > Cookiecutter is still using this concept although using
> > > 'config.add_request_method()' instead of a subclass:
> > >
> > > 
> https://github.com/Pylons/pyramid-cookiecutter-starter/blob/latest/%7B%7Bcookiecutter.repo_name%7D%7D/%7B%7Bcookiecutter.repo_name%7D%7D/sqlalchemy_models/__init__.py
> > > (Lines 71-75.)
> > >
> > > My model classes used to have querying methods but I moved away from
> > > that because they ended up putting too much into the models and being
> > > too limiting (my querying needs inevitably expanded beyond the method
> > > arguments). I now have just the columns and relations in the models,
> > > and separate lib modules for high-level queries. These take a session
> > > argument. I don't usually need a request argument except occasionally
> > > for URL generation. For that I have a postprocessing function that
> > > adds the URL attributes to the query results. For instance:
> > >
> > > # View
> > > results = myapp.lib.mysearch.do_search(sa_session, ...criteria..)
> > > # Return list of ORM objects,
> > > postprocess(results, request)
> > > # Iterate through objects, adding URL attributes for HTML links,
> > > # formatting dates, calculating display values, etc.
> > >
> > > --
> > > You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
> > > To unsubscribe from this group and stop receiving emails from it, send 
> an email to pylons-discus...@googlegroups.com.
> > > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/CAH9f%3Duo821nzLFSttTcvn5xBqbAL2hisuQWUE_%2BCS8q15zUgRA%40mail.gmail.com
> .
> >
> > --
> > You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
> > To unsubscribe from this group and 

Re: [pylons-discuss] Pyramid packaging

['Jonathan Vanasco' via pylons-discuss]

FWIW, Based on the stacktrace, I do not think this is the cause -- but it 
could be. There have been a few related bugs in setuptools and pip 
regarding the version parser and tag_build directives; the errors I've seen 
were not like this - but I saw 3 completely different errors from these 
over the past 2 weeks:

https://github.com/pypa/pip/issues/9446
https://github.com/pypa/setuptools/issues/2529



On Tuesday, January 26, 2021 at 8:28:44 AM UTC-5 Jason Madden wrote:

>
> > Am Di., 26. Jan. 2021 um 11:43 Uhr schrieb Thierry Florac <
> tfl...@gmail.com>:
> >> 
> >> Well...
> >> It seems that the last release of zc.buildout is not compatible with 
> the last setuptools release!
> >> I downgraded setuptools to 50.3.2 and everything seems OK...
> >> 
>
> zc.buildout 3.0a2 should be compatible with modern setuptools. This is 
> because 3.0a1 began using `pip install` instead of calling 
> `setuptools.easy_install`.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/f0df1513-1697-4ae6-807d-41d42915659en%40googlegroups.com.

[pylons-discuss] Re: Pyramid maintenance mode

['Jonathan Vanasco' via pylons-discuss]

i should add, the pattern for doing this in nginx is often called a flag or 
semaphore

i found this article that goes through a 
how-to  
https://blog.devcloud.hosting/configuring-nginx-for-quickly-switching-to-maintenance-mode-e4136cf497f3

basically, you just touch/remove a specific file to change the routing on 
nginx. while the existence of the file is checked on every request, due to 
the way nginx and operating system cache the information this is negligible 
and essentially handled in memory.

On Thursday, January 7, 2021 at 12:43:46 PM UTC-5 Jonathan Vanasco wrote:

> I typically handle this on nginx which sites in front of Pyramid.  if you 
> wanted to do everything in python, you could probably use WSGI middleware 
> to route to a separate maintenance application or html file.
>
> On Thursday, January 7, 2021 at 10:09:34 AM UTC-5 C J wrote:
>
>> Hi everybody,
>>
>> I am looking for an easy way to temporarily redirect all the users of my 
>> pyramid website to a maintenance vue without having to comment/delete, etc 
>> my routes.
>> I would like to make it easy to re-activate the others routes when the 
>> maintenance is done.
>> I found this :
>> https://pypi.org/project/pyramid_maintenance/
>> but  I always get :
>>
>> in renderer
>> raise ValueError('No such renderer factory %s' % str(self.type))
>> ValueError: No such renderer factory .jinja2"
>>
>> with my browser displaying :
>> "Internal Server Error The server encountered an unexpected internal 
>> server error (generated by waitress)"
>>
>> I am new to Pyramid so please give me the necessary details step by step.
>>
>> Best regards.
>> Cedric J.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/17953aa8-8bef-4ae8-a2dc-96a2bd7ae45en%40googlegroups.com.

[pylons-discuss] Re: Pyramid maintenance mode

['Jonathan Vanasco' via pylons-discuss]

I typically handle this on nginx which sites in front of Pyramid.  if you 
wanted to do everything in python, you could probably use WSGI middleware 
to route to a separate maintenance application or html file.

On Thursday, January 7, 2021 at 10:09:34 AM UTC-5 C J wrote:

> Hi everybody,
>
> I am looking for an easy way to temporarily redirect all the users of my 
> pyramid website to a maintenance vue without having to comment/delete, etc 
> my routes.
> I would like to make it easy to re-activate the others routes when the 
> maintenance is done.
> I found this :
> https://pypi.org/project/pyramid_maintenance/
> but  I always get :
>
> in renderer
> raise ValueError('No such renderer factory %s' % str(self.type))
> ValueError: No such renderer factory .jinja2"
>
> with my browser displaying :
> "Internal Server Error The server encountered an unexpected internal 
> server error (generated by waitress)"
>
> I am new to Pyramid so please give me the necessary details step by step.
>
> Best regards.
> Cedric J.
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/98a779a5-5bcc-4971-a271-a202cc49f732n%40googlegroups.com.

Re: [pylons-discuss] Creating sub-processes from Pyramid application

['Jonathan Vanasco' via pylons-discuss]

pyramid_tasks looks very promising! thanks for sharing this.  i may use it 
in the future.  i'll have to see if it is compatible with our setup.

our celery app is mostly tasks that have been migrated from a twisted 
daemon.  we already ran web spiders and social-media-account importers 
through twisted, so all the resizing stuff was originally pyramid creating 
database entries (messages) for twisted to process.  although we still use 
twisted for those tasks (and more!) migrating to celery for these simpler 
functions has been really productive and timesaving.  the code is cleaner 
and easier to maintain, and troubleshooting/bugfixing is so much simpler.
On Wednesday, January 6, 2021 at 12:51:37 PM UTC-5 the...@luhn.com wrote:

> Seems like a good time to plug my new project:  
> https://pypi.org/project/pyramid-tasks/
>
> Pyramid and Celery are so dramatically different it’s a pain to integrate 
> them, and any code shared between them can’t access anything 
> Pyramid-specific.  Got tired of trying to make a square peg fit in a round 
> hole, so I brought my entire Pyramid application over to my Celery worker. 
>  Having a request object available in a Celery task is incredibly 
> convenient, and my code is much tidier not having to tiptoe around the 
> differences in environments.
>
> On Jan 6, 2021, at 2:14 AM, Adam Groszer  wrote:
>
> Hi,
>
> Yeah celery is the way to go.
> There's https://github.com/sontek/pyramid_celery to check out.
> Some things to keep in mind is, you'll need a shared DB, good-old ZODB 
> with filestorage is not enough, because more processes will need to have 
> access.
> Do not store valuable data in the celery queue, consider it ephemeral, go 
> through a real DB.
>
> On Tuesday, January 5, 2021 at 10:31:15 PM UTC+1 tfl...@gmail.com wrote:
>
>> Hi Jonathan,
>>
>> Thank you for this description of Celery!
>> I'll try to have a look at it if I can get a little time, it seems to be 
>> a good replacement (probably more reliable!) of my own developments...
>>
>> Best regards,
>> Thierry
>> -- 
>>   https://www.ulthar.net -- http://pyams.readthedocs.io
>>
>> Le mar. 5 janv. 2021 à 22:18, 'Jonathan Vanasco' via pylons-discuss <
>> pylons-...@googlegroups.com> a écrit :
>>
>>> Thierry,
>>>
>>> That is what I mostly use Celery for (it's also used for generating 
>>> Reports, ACME SSL Certificates and a few other things). A Pyramid Request 
>>> will defer a job(s) to Celery via the "Transport Backend" and receive a 
>>> task/messaging ID.  Subsequent requests will poll the Celery "Result 
>>> Backend"  for the status of that job, based on the ID.  This way we show 
>>> the "Still processing!" message to users.
>>>
>>> Celery is run as a worker with multiple processes. The Celery task 
>>> manager grabs a task off the queue, then does the resizing, uploads to S3, 
>>> and notifies the Result Backend when complete.  I use Redis for Result and 
>>> Transport.  I think there was once a ZeroMQ integration; there is 
>>> definitely RabbitMQ integration, it's one of the more popular options.
>>>
>>> On Monday, January 4, 2021 at 12:25:44 PM UTC-5 tfl...@gmail.com wrote:
>>>
>>>> Hi Jonathan,
>>>>
>>>> I didn't have a look at Celery yet, maybe it could work...
>>>> My goal is to be able to start long-running tasks (medias files 
>>>> conversions for example) whose definition is stored in database and 
>>>> managed 
>>>> by end users from my main Pyramid application; for this, I create a 
>>>> dedicated sub-process which is "managed" using ZeroMQ messages (I use this 
>>>> because it's also used for other non-Python/Pyramid applications).
>>>> Tasks are then started using dedicated threads.
>>>>
>>>> Everything is OK until now... My only requirement now is to be able to 
>>>> get access to main Pyramid's registry from my sub-process and it's 
>>>> threads, 
>>>> and I don't really understand why I get a pointer to
>>>> ZCA global registry...  :(
>>>>
>>>> Best regards,
>>>> Thierry
>>>> -- 
>>>>   https://www.ulthar.net -- http://pyams.readthedocs.io
>>>>
>>>> Le lun. 4 janv. 2021 à 17:51, 'Jonathan Vanasco' via pylons-discuss <
>>>> pylons-...@googlegroups.com> a écrit :
>>>>
>>>>> Have you considered using Celery for this?  I started offloading 
>>>>> everything related to subprocesses and mess

Re: [pylons-discuss] Re: Creating sub-processes from Pyramid application

['Jonathan Vanasco' via pylons-discuss]

Thierry,

That is what I mostly use Celery for (it's also used for generating 
Reports, ACME SSL Certificates and a few other things). A Pyramid Request 
will defer a job(s) to Celery via the "Transport Backend" and receive a 
task/messaging ID.  Subsequent requests will poll the Celery "Result 
Backend"  for the status of that job, based on the ID.  This way we show 
the "Still processing!" message to users.

Celery is run as a worker with multiple processes. The Celery task manager 
grabs a task off the queue, then does the resizing, uploads to S3, and 
notifies the Result Backend when complete.  I use Redis for Result and 
Transport.  I think there was once a ZeroMQ integration; there is 
definitely RabbitMQ integration, it's one of the more popular options.

On Monday, January 4, 2021 at 12:25:44 PM UTC-5 tfl...@gmail.com wrote:

> Hi Jonathan,
>
> I didn't have a look at Celery yet, maybe it could work...
> My goal is to be able to start long-running tasks (medias files 
> conversions for example) whose definition is stored in database and managed 
> by end users from my main Pyramid application; for this, I create a 
> dedicated sub-process which is "managed" using ZeroMQ messages (I use this 
> because it's also used for other non-Python/Pyramid applications).
> Tasks are then started using dedicated threads.
>
> Everything is OK until now... My only requirement now is to be able to get 
> access to main Pyramid's registry from my sub-process and it's threads, and 
> I don't really understand why I get a pointer to
> ZCA global registry...  :(
>
> Best regards,
> Thierry
> -- 
>   https://www.ulthar.net -- http://pyams.readthedocs.io
>
> Le lun. 4 janv. 2021 à 17:51, 'Jonathan Vanasco' via pylons-discuss <
> pylons-...@googlegroups.com> a écrit :
>
>> Have you considered using Celery for this?  I started offloading 
>> everything related to subprocesses and message queues to it a while back, 
>> and have been much happier.
>>
>> On Monday, January 4, 2021 at 11:20:46 AM UTC-5 tfl...@gmail.com wrote:
>>
>>> Hi,
>>>
>>> I need to create custom sub-processes from my main Pyramid application; 
>>> these processes are used to handle "commands" received from ZeroMQ messages.
>>> If I use the components registry to register adapters and utilities 
>>> which are also required in these processes, is there a way to start these 
>>> processes so that they can "inherit" from the main application registry ?
>>>
>>> I'm actually using Pyramid 1.10.5, and I generally do a 
>>> "config.hook_zca()" on application startup before doing 
>>> the "config.setup_registry()" and "config.scan()".
>>> I tried to do the same operations in my sub-process "run" method, but I 
>>> always get the "global" registry instead of my "webapp" registry which was 
>>> configured in my main application...
>>>
>>> Best regards,
>>> Thierry
>>> -- 
>>>   https://www.ulthar.net -- http://pyams.readthedocs.io
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discus...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/9a3e8a04-1a29-4f18-885e-dc3ac1c759b3n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/pylons-discuss/9a3e8a04-1a29-4f18-885e-dc3ac1c759b3n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/5ac83ff1-747f-4df9-994c-ffa48f339013n%40googlegroups.com.

[pylons-discuss] Re: Creating sub-processes from Pyramid application

['Jonathan Vanasco' via pylons-discuss]

Have you considered using Celery for this?  I started offloading everything 
related to subprocesses and message queues to it a while back, and have 
been much happier.

On Monday, January 4, 2021 at 11:20:46 AM UTC-5 tfl...@gmail.com wrote:

> Hi,
>
> I need to create custom sub-processes from my main Pyramid application; 
> these processes are used to handle "commands" received from ZeroMQ messages.
> If I use the components registry to register adapters and utilities which 
> are also required in these processes, is there a way to start these 
> processes so that they can "inherit" from the main application registry ?
>
> I'm actually using Pyramid 1.10.5, and I generally do a 
> "config.hook_zca()" on application startup before doing 
> the "config.setup_registry()" and "config.scan()".
> I tried to do the same operations in my sub-process "run" method, but I 
> always get the "global" registry instead of my "webapp" registry which was 
> configured in my main application...
>
> Best regards,
> Thierry
> -- 
>   https://www.ulthar.net -- http://pyams.readthedocs.io
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/9a3e8a04-1a29-4f18-885e-dc3ac1c759b3n%40googlegroups.com.

[pylons-discuss] Re: Request attributes vs request environment

['Jonathan Vanasco' via pylons-discuss]

I don't know about "correct", but I use `add_request_method` to attach a 
custom object(s) with `reify=True,` and then store all the information in 
those objects.  IIRC, the narrative documentation and tutorials use it for 
similar purposes.

https://docs.pylonsproject.org/projects/pyramid/en/latest/api/config.html#pyramid.config.Configurator.add_request_method


On Thursday, November 12, 2020 at 8:57:58 AM UTC-5 tfl...@gmail.com wrote:

> Hi,
> When we have to store custom information about current request, we can use 
> it's properties, attributes or environment (or even annotations).
> What is the correct usage of each of them, and is there any benefit of 
> using one of them against the others?
>
> Best regards,
> Thierry
> -- 
>   https://www.ulthar.net -- http://pyams.readthedocs.io
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/344da45e-9eef-436a-9eb9-66e40dda0e56n%40googlegroups.com.

[pylons-discuss] Re: Appengine Logging with Pyramid, need to get current request

['Jonathan Vanasco' via pylons-discuss]

I don't use Google's platform, but from what I can tell:

1. Their platform adds in the header "X_CLOUD_TRACE_CONTEXT"
2. There are middleware tools for aggregating logs based on that
3. I don't know if the software you linked is using the middleware, but the 
popular libraries seem to be:

* https://github.com/salrashid123/flask-gcp-log-groups
* https://github.com/christophski/django-gcp-log-groups

I could be wrong.

IMHO, for a variety of reasons `get_current_request` is the wrong approach 
here.  If you need to create the header - and it's not part of google's 
platform - then you'd be best off writing middleware or a tween to add 
header using a UUID.

BUT it looks like your best option would be to fork the flask/django 
middleware project and adapt it to Pyramid.  



On Sunday, November 8, 2020 at 10:55:12 PM UTC-5 jdavi...@gmail.com wrote:

> Using get_current_request, it does seem to end up working, sort of.
>
> I still get a duplicate, I imagined to somehow get it down to just 1:
>
> [image: chrome_P3YiTNWlNu.png]
>
> I'm not getting the root log entry to show the log level though, which 
> makes filtering by error or warning certainly a nightmare. Part of that, I 
> think, is that it's emitting a log entry on each call to `logging`, rather 
> than pooling them and emitting at the end of the request. Clicking that 
> parent log entry, there are no children entries, but after about a second, 
> that one^ pops in, as if it's querying for any matching trace IDs. This 
> isn't how appengine Standard appears to work. Child log entries seem to 
> exist on the parent as a whole/no querying needed.
>
> Given that I am able to "group" these, maybe this is less a Pyramid issue 
> and more a google one. Looking further into get_current_request, it seems 
> to do basically the same thing as django and flask so I suppose for the 
> purposes of logging, it's good enough?
> On Sunday, November 8, 2020 at 9:36:10 PM UTC-6 jdavi...@gmail.com wrote:
>
>> Trying to get appengine logging working under Pyramid. 
>>
>> Log entries are appearing, but they aren't grouped in the same request. 
>> Not only are they not grouped in the same request, but I'm getting a 
>> duplicate as well.
>>
>> [image: chrome_hZKb2p0Zdr.png]
>>
>> For those that aren't familiar with googles Logging system, the top line 
>> should have the blue icon indicating the request contains an INFO log, and 
>> clicking on it should expand to show all of the logging calls that were 
>> made during that request. For example:
>>
>> [image: chrome_LiowoZYVa5.png]
>>
>> As you can see there, that particular request contains 2 logging entries, 
>> so we see them there when that line is expanded.
>>
>> In order for log entries to be grouped together, the api needs to know 
>> what request its a part of. Both django and flask appear to have methods 
>> available to determine what the current request is, and look for a 
>> particular header within that request. 
>>
>> Here is the place where I think needs an addition for Pyramid to be 
>> supported here.   
>>
>>
>> https://github.com/googleapis/python-logging/blob/master/google/cloud/logging/handlers/_helpers.py#L49-L65
>>
>> I see that pyramid has "pyramid.threadlocal.get_current_request()" but 
>> the documentation seems to indicate that that maybe shouldn't be used.
>>
>> Can anyone point me in the right direction?
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/076d8fc4-168f-4f20-a8e9-e1f87172b05dn%40googlegroups.com.

Re: [pylons-discuss] Issues migrating dynamic CSV to Python3

[Jonathan Vanasco]

Thanks, everyone!

Now I've got both these debugtoolbar plugins updated PyPi and running on 
Github Actions with expanded test-suites:

* https://github.com/jvanasco/pyramid_debugtoolbar_api_performance
* https://github.com/jvanasco/pyramid_debugtoolbar_api_sqlalchemy


On Monday, October 19, 2020 at 11:59:05 AM UTC-4 Jonathan Vanasco wrote:

> Actually,  just using `body=csvfile.read()` works on Python2 & Python3 !
>
>
> On Monday, October 19, 2020 at 11:32:48 AM UTC-4 the...@luhn.com wrote:
>
>> Since you’re just dumping the entire file into BytesIO anyways, wouldn’t 
>> it be easier to do `body=csvfile.read().encode(“utf8”)` and skip BytesIO?
>>
>> On Oct 18, 2020, at 3:33 PM, Jonathan Vanasco  wrote:
>>
>> Thanks so much, Bert. That should have been obvious!
>>
>> As an interim solution, I'm just wrapping the inputs:
>>
>> if six.PY3:
>> csvfile = BytesIO(csvfile.read().encode("utf-8"))
>> csvfile.seek(0)
>>
>>
>>
>> On Sunday, October 18, 2020 at 4:22:16 PM UTC-4 Bert JW Regeer wrote:
>>
>>> Your body_file is not bytes, but str. You need to make sure that what 
>>> you pass to body_file returns bytes.
>>>
>>> On Oct 16, 2020, at 15:03, Jonathan Vanasco  wrote:
>>>
>>> I discovered an issue with our code when generating dynamic CSVs under 
>>> Python3.  I'm hoping someone can point me in the right direction.  I 
>>> couldn't find the appropriate migration/changelog information.  This works 
>>> fine under Python2.
>>>
>>> The generic way we create/serve the CSV and check it in tests are below.
>>>
>>> The two problems:
>>>
>>> 1. Under Python3, an exception is thrown under waitress, because of 
>>> attempted string+byte concatenation. (
>>> https://github.com/Pylons/waitress/blob/master/src/waitress/task.py#L316
>>> )
>>>
>>> >towrite += data + b"\r\n"
>>>
>>> > TypeError: can only concatenate str (not "bytes") to str
>>>
>>> 2. I picked this up in a unit test; i can't seem to access any sort of 
>>> body/text/content off the response.
>>>
>>> from pyramid.request import Request
>>> from pyramid import testing
>>>
>>> self.config = config = testing.setUp()
>>> app = self.config.make_wsgi_app()
>>>
>>> req = Request.blank(csv_link)
>>> req.remote_addr = "127.0.0.1"
>>> resp = req.get_response(app)
>>> self.assertEqual(resp.status_code, 200)
>>> self.assertTrue(resp.text.startswith("User CPU time,"))
>>>
>>>
>>>
>>> ---
>>>
>>>  def view_csv(request):
>>> csvfile = StringIO()
>>> csvwriter = csv.writer(csvfile)
>>> for row in ((1, "a"), (2, "b")):
>>> csvwriter.writerow(row)
>>> csvfile.seek(0)
>>> as_csv = Response(content_type="text/csv", body_file=csvfile, 
>>> status=200)
>>> as_csv.headers["Content-Disposition"] = str("attachment; 
>>> filename=example.csv")
>>> return as_csv
>>>
>>> ---
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "pylons-discuss" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to pylons-discus...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/pylons-discuss/7d3c278f-8d20-4456-b05c-33c8d2cb6a67n%40googlegroups.com
>>>  
>>> <https://groups.google.com/d/msgid/pylons-discuss/7d3c278f-8d20-4456-b05c-33c8d2cb6a67n%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>>>
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discus...@googlegroups.com.
>>
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/e73aae83-0f7e-414c-a9c3-c20957793a0dn%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/pylons-discuss/e73aae83-0f7e-414c-a9c3-c20957793a0dn%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/19714c15-c0db-44bf-b9d3-653d814725d1n%40googlegroups.com.

[pylons-discuss] Re: Serving an S3 file

[Jonathan Vanasco]

> What is 'wsgi.file_wrapper' anyway?

It's a WSGI hook defined in PEP 333 and  
(https://www.python.org/dev/peps/pep-/#optional-platform-specific-file-handling)

It basically lets your application defer to an iterator defined in 
middleware file responses.  In the Pyramid code, you'll see that it checks 
to see if there is a middleware filewrapper, and falls back to Pyramid's 
option.

If you have loaded the bytes, you can use a BytesIO object, which is 
in-memory. 

If the file might be large, you could write the data to tempfile.
SpooledTemporaryFile -- which will be in-memory until it hits a max-size, 
and then writes to disk.  


On Saturday, October 17, 2020 at 3:07:13 AM UTC-4 Mike Orr wrote:

> I'm migrating user-uploaded files from a local filesystem to MinIO (an
> S3-compatible file server). I can't use FileResponse because I get
> back a filehandle or a bytes, and I'd have to write it to a temporary
> file to use FileResponse.I'm also exploring presigned URLs to have the
> browser fetch the file, but right now I want to serve it server-side.
> What Pyramid response args should I use, and how should I stream the
> content? I looked at the source of FileResponse, and it creates an
> 'app_iter' object using 'environ["wsgi.file_wrapper"]'. Do I have to
> do all that? Or can I just do 'body_file=result["body"]', which is the
> filehandle and has a 'read' method. What is 'wsgi.file_wrapper'
> anyway?
>
> -- 
> Mike Orr 
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/39ecc9ad-23b3-4541-a4c3-a4d4b52973c7n%40googlegroups.com.

Re: [pylons-discuss] Issues migrating dynamic CSV to Python3

[Jonathan Vanasco]

Actually,  just using `body=csvfile.read()` works on Python2 & Python3 !


On Monday, October 19, 2020 at 11:32:48 AM UTC-4 the...@luhn.com wrote:

> Since you’re just dumping the entire file into BytesIO anyways, wouldn’t 
> it be easier to do `body=csvfile.read().encode(“utf8”)` and skip BytesIO?
>
> On Oct 18, 2020, at 3:33 PM, Jonathan Vanasco  wrote:
>
> Thanks so much, Bert. That should have been obvious!
>
> As an interim solution, I'm just wrapping the inputs:
>
> if six.PY3:
> csvfile = BytesIO(csvfile.read().encode("utf-8"))
> csvfile.seek(0)
>
>
>
> On Sunday, October 18, 2020 at 4:22:16 PM UTC-4 Bert JW Regeer wrote:
>
>> Your body_file is not bytes, but str. You need to make sure that what you 
>> pass to body_file returns bytes.
>>
>> On Oct 16, 2020, at 15:03, Jonathan Vanasco  wrote:
>>
>> I discovered an issue with our code when generating dynamic CSVs under 
>> Python3.  I'm hoping someone can point me in the right direction.  I 
>> couldn't find the appropriate migration/changelog information.  This works 
>> fine under Python2.
>>
>> The generic way we create/serve the CSV and check it in tests are below.
>>
>> The two problems:
>>
>> 1. Under Python3, an exception is thrown under waitress, because of 
>> attempted string+byte concatenation. (
>> https://github.com/Pylons/waitress/blob/master/src/waitress/task.py#L316)
>>
>> >towrite += data + b"\r\n"
>>
>> > TypeError: can only concatenate str (not "bytes") to str
>>
>> 2. I picked this up in a unit test; i can't seem to access any sort of 
>> body/text/content off the response.
>>
>> from pyramid.request import Request
>> from pyramid import testing
>>
>> self.config = config = testing.setUp()
>> app = self.config.make_wsgi_app()
>>
>> req = Request.blank(csv_link)
>> req.remote_addr = "127.0.0.1"
>> resp = req.get_response(app)
>> self.assertEqual(resp.status_code, 200)
>> self.assertTrue(resp.text.startswith("User CPU time,"))
>>
>>
>>
>> ---
>>
>>  def view_csv(request):
>> csvfile = StringIO()
>> csvwriter = csv.writer(csvfile)
>> for row in ((1, "a"), (2, "b")):
>> csvwriter.writerow(row)
>> csvfile.seek(0)
>> as_csv = Response(content_type="text/csv", body_file=csvfile, 
>> status=200)
>> as_csv.headers["Content-Disposition"] = str("attachment; 
>> filename=example.csv")
>> return as_csv
>>
>> ---
>>
>>
>>
>>
>>
>>
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discus...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/7d3c278f-8d20-4456-b05c-33c8d2cb6a67n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/pylons-discuss/7d3c278f-8d20-4456-b05c-33c8d2cb6a67n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>>
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
>
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/e73aae83-0f7e-414c-a9c3-c20957793a0dn%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/pylons-discuss/e73aae83-0f7e-414c-a9c3-c20957793a0dn%40googlegroups.com?utm_medium=email_source=footer>
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/ce9e5409-eaac-4014-a98f-df5d1b808734n%40googlegroups.com.

Re: [pylons-discuss] Issues migrating dynamic CSV to Python3

[Jonathan Vanasco]

Thanks so much, Bert. That should have been obvious!

As an interim solution, I'm just wrapping the inputs:

if six.PY3:
csvfile = BytesIO(csvfile.read().encode("utf-8"))
csvfile.seek(0)



On Sunday, October 18, 2020 at 4:22:16 PM UTC-4 Bert JW Regeer wrote:

> Your body_file is not bytes, but str. You need to make sure that what you 
> pass to body_file returns bytes.
>
> On Oct 16, 2020, at 15:03, Jonathan Vanasco  wrote:
>
> I discovered an issue with our code when generating dynamic CSVs under 
> Python3.  I'm hoping someone can point me in the right direction.  I 
> couldn't find the appropriate migration/changelog information.  This works 
> fine under Python2.
>
> The generic way we create/serve the CSV and check it in tests are below.
>
> The two problems:
>
> 1. Under Python3, an exception is thrown under waitress, because of 
> attempted string+byte concatenation. (
> https://github.com/Pylons/waitress/blob/master/src/waitress/task.py#L316)
>
> >towrite += data + b"\r\n"
>
> > TypeError: can only concatenate str (not "bytes") to str
>
> 2. I picked this up in a unit test; i can't seem to access any sort of 
> body/text/content off the response.
>
> from pyramid.request import Request
> from pyramid import testing
>
> self.config = config = testing.setUp()
> app = self.config.make_wsgi_app()
>
> req = Request.blank(csv_link)
> req.remote_addr = "127.0.0.1"
> resp = req.get_response(app)
> self.assertEqual(resp.status_code, 200)
> self.assertTrue(resp.text.startswith("User CPU time,"))
>
>
>
> ---
>
>  def view_csv(request):
> csvfile = StringIO()
> csvwriter = csv.writer(csvfile)
> for row in ((1, "a"), (2, "b")):
> csvwriter.writerow(row)
> csvfile.seek(0)
> as_csv = Response(content_type="text/csv", body_file=csvfile, 
> status=200)
> as_csv.headers["Content-Disposition"] = str("attachment; 
> filename=example.csv")
> return as_csv
>
> ---
>
>
>
>
>
>
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/7d3c278f-8d20-4456-b05c-33c8d2cb6a67n%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/pylons-discuss/7d3c278f-8d20-4456-b05c-33c8d2cb6a67n%40googlegroups.com?utm_medium=email_source=footer>
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/e73aae83-0f7e-414c-a9c3-c20957793a0dn%40googlegroups.com.

[pylons-discuss] Issues migrating dynamic CSV to Python3

[Jonathan Vanasco]

I discovered an issue with our code when generating dynamic CSVs under 
Python3.  I'm hoping someone can point me in the right direction.  I 
couldn't find the appropriate migration/changelog information.  This works 
fine under Python2.

The generic way we create/serve the CSV and check it in tests are below.

The two problems:

1. Under Python3, an exception is thrown under waitress, because of 
attempted string+byte concatenation. 
(https://github.com/Pylons/waitress/blob/master/src/waitress/task.py#L316)

>towrite += data + b"\r\n"

> TypeError: can only concatenate str (not "bytes") to str

2. I picked this up in a unit test; i can't seem to access any sort of 
body/text/content off the response.

from pyramid.request import Request
from pyramid import testing

self.config = config = testing.setUp()
app = self.config.make_wsgi_app()

req = Request.blank(csv_link)
req.remote_addr = "127.0.0.1"
resp = req.get_response(app)
self.assertEqual(resp.status_code, 200)
self.assertTrue(resp.text.startswith("User CPU time,"))



---

 def view_csv(request):
csvfile = StringIO()
csvwriter = csv.writer(csvfile)
for row in ((1, "a"), (2, "b")):
csvwriter.writerow(row)
csvfile.seek(0)
as_csv = Response(content_type="text/csv", body_file=csvfile, 
status=200)
as_csv.headers["Content-Disposition"] = str("attachment; 
filename=example.csv")
return as_csv

---






-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/7d3c278f-8d20-4456-b05c-33c8d2cb6a67n%40googlegroups.com.

Re: [pylons-discuss] Will `render_to_response` ever return something that is not `pyramid.response.Response`?

['Jonathan Vanasco' via pylons-discuss]

Thank you, Michael!

This helps me a lot.

On Monday, October 12, 2020 at 4:16:40 PM UTC-4 mmer...@gmail.com wrote:

> There is a null_renderer in Pyramid, but it's not available via 
> render_to_response. I do not see a code path that would return anything 
> other than an object implementing pyramid.interfaces.IResponse (the 
> expected return value from the app's configured IResponseFactory). This is, 
> of course, slightly more general than pyramid.response.Response.
>
> - Michael
>
> On Oct 12, 2020, at 13:48, 'Jonathan Vanasco' via pylons-discuss <
> pylons-...@googlegroups.com> wrote:
>
> I've looked at the code and am pretty sure the answer is no, but a bunch 
> of subclasses are used here, so I think it's best to ask --  Will 
> render_to_response ever return something that is not 
> `pyramid.response.Response`?
>
> I have a small library that will act on the output of render_to_response; 
> I am trying to catch edge cases where it is improperly invoked and a 
> developer passes in a normal dict (which would be rendered via view_config 
> template or json renderer) instead of the expected output of 
> render_to_response.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/32fddda9-dba9-4c84-a26a-c0a28a6953dbn%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/pylons-discuss/32fddda9-dba9-4c84-a26a-c0a28a6953dbn%40googlegroups.com?utm_medium=email_source=footer>
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/13360489-4e50-440e-8553-6ee26d838da0n%40googlegroups.com.

[pylons-discuss] Will `render_to_response` ever return something that is not `pyramid.response.Response`?

['Jonathan Vanasco' via pylons-discuss]

 

I've looked at the code and am pretty sure the answer is no, but a bunch of 
subclasses are used here, so I think it's best to ask --  Will 
render_to_response ever return something that is not 
`pyramid.response.Response`?

I have a small library that will act on the output of render_to_response; I 
am trying to catch edge cases where it is improperly invoked and a 
developer passes in a normal dict (which would be rendered via view_config 
template or json renderer) instead of the expected output of 
render_to_response.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/32fddda9-dba9-4c84-a26a-c0a28a6953dbn%40googlegroups.com.

Re: [pylons-discuss] migrating away from formencode?

['Jonathan Vanasco' via pylons-discuss]

It's not a big deal and only shows the wrong message if your file is 
required.

We have many forms with optional file uploads.  Not including a file would 
break the form validation.
On Monday, October 5, 2020 at 11:44:45 PM UTC-4 Mike Orr wrote:

> I'm using Formencode 1.3.1. My application switched to Python 3 in
> 2017 and I haven't had any problems with Formencode. It has two file
> uploads, one with a form, and if I submit it without a file I get the
> error message "Please provide only one value" mentioned in the issue.
> This is not a big deal in my use case. Now that Formencode 2 is out
> I'll upgrade to it.
>
> On Mon, Oct 5, 2020 at 7:34 AM 'Jonathan Vanasco' via pylons-discuss
>  wrote:
> >
> > Talk about timing. Several hours after I posted this, Formencode merged 
> the Python3 fix(es) and did a 2.0 release. I guess I'm safe for now, and 
> can migrate away at a more leisurely pace.
> >
> > The big bug was on file uploads (
> https://github.com/formencode/formencode/issues/101), which broke under 
> Python3.
> >
> > I think the other tickers were all housekeeping.
> >
> > On Friday, October 2, 2020 at 4:28:29 PM UTC-4 i...@laspilitas.com 
> wrote:
> >>
> >> I guess I'm still using formencode too with python3 but maybe I just 
> worked around the bugs. Do you know what is holding up the release? Is it 
> for sure a dead-end?
> >>
> >> - Ian
> >>
> >> On Fri, Oct 2, 2020 at 1:20 PM Steve Piercy  
> wrote:
> >>>
> >>> Deform uses Colander and Peppercorn. Use Colander for schemas and 
> validation. There are no tutorials to migrate, but the demo should provide 
> sufficient Colander schema examples for each Deform widget that you want. 
> If you don't see something, ask!
> >>>
> >>> --steve
> >>>
> >>>
> >>> On 10/2/20 1:00 PM, 'Jonathan Vanasco' via pylons-discuss wrote:
> >>> > Thanks, Stev!
> >>> >
> >>> > Deform is high on the list; I should have been more specific - I'm 
> looking for any guidelines/tutorials/tools to migrate the code from 
> Formencode to other libraries. I have a lot of forms schemas/definitions to 
> migrate. I wrote my own validation layer, so I'm not too worried about 
> migrating the code that interacts with forms.
> >>> >
> >>> > FormEncode should my *last* Python2 library to support! There is a 
> longstanding bug on it file uploads breaking on Python3 . I've been using 
> someone's stale PR as a patch for 2 years now, but that is just making 
> everything too fragile so I try to keep all those apps running under 
> Python2..
> >>> >
> >>> >
> >>> > On Friday, October 2, 2020 at 3:40:22 PM UTC-4 Steve Piercy wrote:
> >>> >
> >>> > On 10/2/20 10:06 AM, 'Jonathan Vanasco' via pylons-discuss wrote:
> >>> > > Does anyone have tips/advice for migrating away from Formencode?
> >>> >
> >>> > For server side rendering of forms, Deform is under active 
> development and works under both Python 2 and 3.
> >>> >
> >>> > Demo:
> >>> > https://deformdemo.pylonsproject.org/
> >>> >
> >>> > GitHub:
> >>> > https://github.com/pylons/deform
> >>> >
> >>> > Deform 3.0.0 will use Bootstrap 4 for its templates and drop Python 
> 2 support, although it might still work under Python 2.
> >>> > https://github.com/Pylons/deform/milestone/3
> >>> >
> >>> > --steve
> >>> >
> >>> > --
> >>> > You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
> >>> > To unsubscribe from this group and stop receiving emails from it, 
> send an email to pylons-discus...@googlegroups.com  pylons-discus...@googlegroups.com>.
> >>> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/d0a0d404-f081-41f8-865f-b073ed17bc78n%40googlegroups.com
>  
> <
> https://groups.google.com/d/msgid/pylons-discuss/d0a0d404-f081-41f8-865f-b073ed17bc78n%40googlegroups.com?utm_medium=email_source=footer
> >.
> >>>
> >>> --
> >>> You received this message because you are subscribed to the Google 
> Groups "pylons-discuss" group.
> >>>
> >>> To unsubscribe from this group and stop receiving emails from it, send 
> an email to pylons-discus...@googlegroups.com.
> >>> To view this discussion on the web visit 

Re: [pylons-discuss] migrating away from formencode?

['Jonathan Vanasco' via pylons-discuss]

Talk about timing. Several hours after I posted this, Formencode merged the 
Python3 fix(es) and did a 2.0 release.  I guess I'm safe for now, and can 
migrate away at a more leisurely pace.

The big bug was on file uploads 
(https://github.com/formencode/formencode/issues/101), which broke under 
Python3.

I think the other tickers were all housekeeping.

On Friday, October 2, 2020 at 4:28:29 PM UTC-4 i...@laspilitas.com wrote:

> I guess I'm still using formencode too with python3 but maybe I just 
> worked around the bugs.  Do you know what is holding up the release?  Is it 
> for sure a dead-end?
>
> - Ian
>
> On Fri, Oct 2, 2020 at 1:20 PM Steve Piercy  wrote:
>
>> Deform uses Colander and Peppercorn.  Use Colander for schemas and 
>> validation.  There are no tutorials to migrate, but the demo should provide 
>> sufficient Colander schema examples for each Deform widget that you want.  
>> If you don't see something, ask!
>>
>> --steve
>>
>>
>> On 10/2/20 1:00 PM, 'Jonathan Vanasco' via pylons-discuss wrote:
>> > Thanks, Stev!
>> > 
>> > Deform is high on the list; I should have been more specific - I'm 
>> looking for any guidelines/tutorials/tools to migrate the code from 
>> Formencode to other libraries.  I have a lot of forms schemas/definitions 
>> to migrate.  I wrote my own validation layer, so I'm not too worried about 
>> migrating the code that interacts with forms.
>> > 
>> > FormEncode should my *last* Python2 library to support!  There is a 
>> longstanding bug on it file uploads breaking on Python3 . I've been using 
>> someone's stale PR as a patch for 2 years now, but that is just making 
>> everything too fragile so I try to keep all those apps running under 
>> Python2..
>> > 
>> > 
>> > On Friday, October 2, 2020 at 3:40:22 PM UTC-4 Steve Piercy wrote:
>> > 
>> > On 10/2/20 10:06 AM, 'Jonathan Vanasco' via pylons-discuss wrote:
>> > > Does anyone have tips/advice for migrating away from Formencode?
>> > 
>> > For server side rendering of forms, Deform is under active 
>> development and works under both Python 2 and 3.
>> > 
>> > Demo:
>> > https://deformdemo.pylonsproject.org/
>> > 
>> > GitHub:
>> > https://github.com/pylons/deform
>> > 
>> > Deform 3.0.0 will use Bootstrap 4 for its templates and drop Python 
>> 2 support, although it might still work under Python 2.
>> > https://github.com/Pylons/deform/milestone/3
>> > 
>> > --steve
>> > 
>> > -- 
>> > You received this message because you are subscribed to the Google 
>> Groups "pylons-discuss" group.
>> > To unsubscribe from this group and stop receiving emails from it, send 
>> an email to pylons-discus...@googlegroups.com > pylons-discus...@googlegroups.com>.
>> > To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/d0a0d404-f081-41f8-865f-b073ed17bc78n%40googlegroups.com
>>  
>> <
>> https://groups.google.com/d/msgid/pylons-discuss/d0a0d404-f081-41f8-865f-b073ed17bc78n%40googlegroups.com?utm_medium=email_source=footer
>> >.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "pylons-discuss" group.
>>
> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to pylons-discus...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/pylons-discuss/6f0f2332-daa6-70a1-f148-2b32abdbb459%40gmail.com
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/1d1850a5-2e49-47f3-835d-c44a6595c60an%40googlegroups.com.

Re: [pylons-discuss] migrating away from formencode?

['Jonathan Vanasco' via pylons-discuss]

Thanks, Stev!

Deform is high on the list; I should have been more specific - I'm looking 
for any guidelines/tutorials/tools to migrate the code from Formencode to 
other libraries.  I have a lot of forms schemas/definitions to migrate.  I 
wrote my own validation layer, so I'm not too worried about migrating the 
code that interacts with forms.

FormEncode should my *last* Python2 library to support!  There is a 
longstanding bug on it file uploads breaking on Python3 . I've been using 
someone's stale PR as a patch for 2 years now, but that is just making 
everything too fragile so I try to keep all those apps running under 
Python2..


On Friday, October 2, 2020 at 3:40:22 PM UTC-4 Steve Piercy wrote:

> On 10/2/20 10:06 AM, 'Jonathan Vanasco' via pylons-discuss wrote:
> > Does anyone have tips/advice for migrating away from Formencode?
>
> For server side rendering of forms, Deform is under active development and 
> works under both Python 2 and 3.
>
> Demo:
> https://deformdemo.pylonsproject.org/
>
> GitHub:
> https://github.com/pylons/deform
>
> Deform 3.0.0 will use Bootstrap 4 for its templates and drop Python 2 
> support, although it might still work under Python 2.
> https://github.com/Pylons/deform/milestone/3
>
> --steve
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/d0a0d404-f081-41f8-865f-b073ed17bc78n%40googlegroups.com.

[pylons-discuss] migrating away from formencode?

['Jonathan Vanasco' via pylons-discuss]

Does anyone have tips/advice for migrating away from Formencode?

I have a lot of apps using Formencode and will likely need to move off it.  
It's no longer maintained, and needs to a patch/fork to run under Python3. 
That's been fine for internal apps, but it's a pain for open sourced 
efforts.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/22190c16-6299-41c3-a1e4-25fed04606ccn%40googlegroups.com.

[pylons-discuss] Re: Pyramid docs copyright

[Jonathan Vanasco]

I am not a package maintainer, but if I recall correctly, this is a benefit 
and not a desire.

I believe one of (several) reasons for this licensing decision was to stop 
third-party groups from monetizing the documentation at the expense of the 
project. IIRC, several commercial sites had mirrored outdated versions of 
the documentation and used advanced Search Engine Optimization techniques 
to overtake the rankings of the official and current documentation.  This 
is a common plague to Open Source Projects.

IMHO this is a benefit because the software and documentation change often, 
and should be installed or upgraded using the Python Package manager.  The 
official installation instructions (
https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/install.html) 
target package manager installations, not operating system installers.

On Monday, September 21, 2020 at 5:40:47 AM UTC-4 frispete wrote:

> Hi,
>
> is it desired, that Linux distributions are not allowed to package the 
> Pyramid 
> docs due to the CC-BY-NC-SA-3.0 license?
>
> Pete
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/c7533691-ca7e-44a6-bbe4-7df68c9f0824n%40googlegroups.com.

Re: [pylons-discuss] changing session cookie max_age?

[Jonathan Vanasco]

On Saturday, September 19, 2020 at 12:00:28 PM UTC-4 mmer...@gmail.com 
wrote:

> It could support changing the max_age when you invoke 
> `adjust_timeout_for_session` but it apparently is not.
>

I'm the package author. `adjust_timeout_for_session` doesn't affect 
`max_age` because that defect was in the original project 
"pyramid_redis_sessions" that my project was forked from. The "expires" 
cookie attribute is also not supported in any way, for the same reason.  
I've never needed either of these, so the issue never came up before.  I 
have no issue with supporting these features, but it's likely not going to 
happen until I need to use these features myself or someone else issues a 
PR. Writing support for this is pretty simple, but there are a few things 
it may affect in the overall API (there are both `timeout` and `expires` 
concepts this could alter) and writing enough test coverage to make it safe 
to merge it will be a few hours of work.
 

> On Sep 19, 2020, at 07:34, zsol...@gmail.com  wrote:
>
> Ideally, I'd like to achieve never logging out logged-in users, as it's 
> bad for user experience, but at the same time limit bots and non-logged-in 
> users to 1800 seconds.
>
>
IMHO the only way to achieve the UX of "never logging out logged-in users" 
is to use an ancillary client-side secure/encrypted "autologin" cookie, 
which will establish a new session for a given user if their session cookie 
is expired or missing.  There are far too many scenarios that can cause a 
loss of ServerSide/Redis session data, including: bot traffic, ddos, usage 
spikes, server crashes, etc (all of which I've experienced).  Implementing 
this in Pyramid is fairly simple, thanks to the tweens.


 

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/8753876a-bf3f-46e7-ac3b-6f03730462e1n%40googlegroups.com.

Re: [pylons-discuss] Migration from Flask - options?

['Jonathan Vanasco' via pylons-discuss]

I've done a handful of side-by-side migrations or deployments.

I think you already identified the generally best approach IMHO:

>  Conceptually the simplest would be to have a auth cookie that is 
valid in both, it could be set to only be created in one and honoured in 
the other.

However I recommend it with one caveat -- I would approach this 
functionality as if it were it's own authentication micro-service.  It can 
live in one of the two processes, it doesn't need a third, but I would 
structure the UX, tests and development as if it were totally independent 
-- so the cookie value would just contain login state the core account info 
needed for auth and permissions.  If either process needs to store other 
data in a cookie, use another cookie.

Reading Pyramid cookies in Flask is pretty simple, Pyramid sessions just 
automate loading webob cookies with defaults

* https://github.com/Pylons/pyramid/blob/master/src/pyramid/session.py
* https://github.com/Pylons/webob/blob/master/src/webob/cookies.py


In terms of Pyramid versions, if you need Python2 support - 1.10 is your 
only option.  Otherwise, just pay attention to the deprecations on Pyramid2 
and you should be able to transition from 1.10 to 2 very easily if you 
don't want to run the pre-release.  Michael Merickel is the expert on this, 
but I think he and his team have done a great job in planning the 2.0 
changes by offering a lot of "forward compatibility' options in 1.10 to 
prepare us for switching to 2.0.  I could be wrong, but I think almost 
every one of the 2.0 changes can be elected in 1.10.  The only difference I 
know of is the drastic change to authentication/authorization (which may be 
worth using the pre-release). 


-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/b0d9e559-d070-4522-82ef-217af2c63de2o%40googlegroups.com.

Re: [pylons-discuss] externally hosted static files

['Jonathan Vanasco' via pylons-discuss]

This is a very simplified version of my approach:

I typically handle this by injecting a `cdn` variable into the default 
rendering environment, and then just having the templates reference 

href="${cdn}/path/to/file"

I specify the `cdn` var in the `.ini` file for the application. on startup, 
i load it into the registry and a few other places.

On some projects, I have routine that preprocesses the templates for a 
handful of "global" variables.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/6ef9cf2d-5c65-4b5c-a534-555785bcc0f0o%40googlegroups.com.

[pylons-discuss] docs down?

[Jonathan Vanasco]

I'm getting 403 errors from cloudflare on all the 
https://docs.pylonsproject.org urls.  i tried accessing from several 
networks in the usa to ensure it wasn't a local routing or blacklist issue

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/e8021f24-9f2e-4775-928e-54ef8c26c581%40googlegroups.com.

[pylons-discuss] Re: strategies for marking test coverage of routes?

[Jonathan Vanasco]

I ended up doing a quick proof of concept that works pretty well:

_ROUTES_TESTED = {}



def tests_routes(*args):
"""
`@tests_routes` is a decorator
when writing/editing a test, declare what routes the test covers, like 
such:


@tests_routes(("foo", "bar"))
def test_foo_bar(self):
...

this will populate a global variable `_ROUTES_TESTED` with the name of 
the 
tested routes.

invoking the Audit test:

python -m unittest tests.FunctionalTests_AuditRoutes


will ensure all routes in Pyramid have test coverage
"""
_routes = args[0]
if isinstance(_routes, (list, tuple)):
for _r in _routes:
_ROUTES_TESTED[_r] = True
else:
_ROUTES_TESTED[_routes] = True


def _decorator(_function):
def _wrapper(*args, **kwargs):
_function(*args, **kwargs)
return _wrapper
return _decorator




class FunctionalTests_AuditRoutes(AppTest):
"""
python -m unittest tests.FunctionalTests_AuditRoutes
"""


def test_audit(self):
pyramid_route_names = [i.name for i in self.testapp.app.
routes_mapper.routelist]
names_missing = [r for r in pyramid_route_names if r not in 
_ROUTES_TESTED.keys()]
if names_missing:
raise ValueError("no coverage for %s routes: %s" % (len(
names_missing), names_missing))




This still needs work, but implementing this into the test suite of an app 
with 150+ routes, I found 20 without test coverage

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/ad28fe5b-6c78-4a5d-9a09-ccffb18d2805%40googlegroups.com.

[pylons-discuss] strategies for marking test coverage of routes?

[Jonathan Vanasco]

I'd like to ensure 100% test coverage of every route.

Has anyone worked on a solution to ensuring/reporting this?

I assume it might involve a decorator on my tests stating the routes 
covered, and then a command to match that against what is registered into 
the Pyramid app via @view_config or other mechansism.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/a06fe39a-02e7-4928-bef7-0e7b985c74df%40googlegroups.com.

[pylons-discuss] proper way to stash an object into the application registry?

[Jonathan Vanasco]

I am decoupling some code from Pyramid and am creating a dict-like object 
to live alongside the traditional application "settings" object available 
as `config.registry.settings` and `request.registry.settings`.  I am 
basically moving the non-Pyramid-centric application settings out of the 
ISettings container and into another object.

Is it appropriate to simply stash this object into the "config.registry" 
during the `__init__.py` `main()` def, or must it be registered?

An alternative option could be to place it within the 
`request.registry.settings` object too. I would like to avoid that if 
possible, just because I think it could be too confusing for maintenance.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/42cf539b-2eb0-4687-9288-fc06bc4a8fd3%40googlegroups.com.

Re: [pylons-discuss] Python 3.8 and __init__.py in views directory

[Jonathan Vanasco]

Stuff like this often happens because Python has a compiled/cached version. 
Under Python2,  `.pyc` files were in the same directory as the `.py` file ; 
under Python3 they are in a `__pycache__` directory and versioned to the 
interpreter.

you probably had a `views/__pycache__/__init__.cpython-35.pyc` file, which 
python 3.5 picked up but 3.7 and 3.8 will not.


On Wednesday, March 25, 2020 at 2:06:26 PM UTC-4, Sydo Luciani wrote:
>
> Yes, I was missing __init__.py in views folder, but didn't have any 
> problem while working in Python 3.5 environment,
> error started when switched to Python 3.7 and Python 3.8, and creating 
> __init__ fixed the issue.
>
> The different behavior brought up curiosity.
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/25856fcc-e569-4fe7-a138-6f3bb9917849%40googlegroups.com.

[pylons-discuss] Re: pyramid_beaker, pyramid_redis_sessions, pyramid_session_redis

[Jonathan Vanasco]

Mike- 

I maintain `pyramid_session_redis`.

The package allows for session data to be serialized with `pickle`.  I 
don't necessarily recommend it and prefer for `msgpack` myself, but it 
supports `pickle` and `pickle` is the default.

A recent-ish release dropped another usage of `pickle` in the library which 
was a potential security issue. Pyramid had finally removed it from one of 
the more recent branches.  The original behavior is still present 
in `pyramid_redis_sessions`:

https://github.com/ericrasmussen/pyramid_redis_sessions/blob/master/pyramid_redis_sessions/__init__.py#L284

cookieval = signed_serialize(session.session_id, secret)

Pyramid's `signed_serialize` was deprecated in 1.10 and has been removed 
from Pyramid 2.0

if you look at the legacy code from a PR ( 
https://github.com/Pylons/pyramid/commit/ 

https://github.com/Pylons/pyramid/commit/133db09d179c3f5afe7e02dc13ab6687517db5a1#diff-66ef7a67eff67171f5600c8c2841a754L70
)

You'll see this happens in the function:

pickled = pickle.dumps(data, pickle.HIGHEST_PROTOCOL) 
The pyramid_session_redis behavior was to take a session_id, serialize it 
with pickle, then calculate a signature of that and use the pickled value + 
signature as a cookie value.

To retrieve a session, the signature of the cookie's value is checked, and 
it is deserialized if it passes. then that deserialized value is used to 
lookup the record in redis.

If your site's secret became compromised or decrypted, an attacker could 
compromise your servers by sending in carefully crafted payloads - as the 
value will be piped into pickle.loads().

The new behavior in my package is to leverage webob's `SignedSerializer` 
directly, 
and sign the session_id - as it is guaranteed by this library to be a 
string that can be fed into the serializer directly.

Since your concern is bots, something you may also consider is to use two 
different sessions or leverage other signed cookies into your session 
strategy. Usually I dedicate a browser cookie to 'session_status' of just 
being logged in or the user id -- either as a signed browser cookie or a 
primary session.  The more typical "session" stuff is then used via a 
server-side session that is only accessed when logged in. 

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/ed1eb5e1-c650-48c3-99f3-f55334373709%40googlegroups.com.

[pylons-discuss] Re: pyramid_tm - is there an acceptable way to unjoin the transaction and use explicit commits?

[Jonathan Vanasco]

>
> `request.tm.commit()`, `request.tm.abort()` and `request.tm.begin()` 
> instead of `request.dbsession.commit()`. 


Thanks, Michael. This obvious answer is what I'll use.

I am very familiar with Celery and task runners, and have a few app using 
it with Pyramid.

In this particular application, the work/results are all instant - there 
are no potentially long-running subprocesses.  I just need to commit 
multiple times within a view to ensure the database records an action was 
performed.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/eb885f42-86b0-413b-9f2e-e1e91d9c95a3%40googlegroups.com.

[pylons-discuss] pyramid_tm - is there an acceptable way to unjoin the transaction and use explicit commits?

[Jonathan Vanasco]

one of my apps uses pyramid_tm, but has reached a point where one (possibly 
two) views (out of dozens) need to make explicit commits with the 
SqlAlchemy session.  

does anyone know if it's currently possible to handle this?  i have a few 
apps that only integrate pyramid_tm on a small subset of views, but i 
haven't tried to bypass it on a single view in many years.  I'm trying to 
avoid a secondary database connection, because this app can support a 
sqlite database and I've had problems with too many connections and sqlite 
in the past.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/c3d3f561-8bcc-4c20-b1e8-f2aa147483f0%40googlegroups.com.

Re: [pylons-discuss] Re: tracking down a test issue - ResourceWarning: unclosed file <_io.BufferedRandom

[Jonathan Vanasco]

On Wednesday, February 5, 2020 at 1:15:23 PM UTC-5, Michael Merickel wrote:
>
> If you're opening files you should use the context manager - for example:
>
> with request.POST['file'] as field:
> data = field.file.read()
>

Thanks, Michael. My application code did that... but somewhere in the form 
management/validation stack, file objects were being opened and not 
closed.  I haven't tracked down exactly where that happens yet, so I went 
with the finished callback to close file-like objects.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/696afa5d-13ad-4a3e-9e07-c7c74e75710a%40googlegroups.com.

[pylons-discuss] Re: tracking down a test issue - ResourceWarning: unclosed file <_io.BufferedRandom

[Jonathan Vanasco]

After even more testing (half my day!)...

My app needed an `add_finished_callback` to close any 
`webob.compat.cgi_FieldStorage` objects in forms.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/b87e949c-62f7-43b9-a609-d78bd8ea6015%40googlegroups.com.

[pylons-discuss] tracking down a test issue - ResourceWarning: unclosed file <_io.BufferedRandom

[Jonathan Vanasco]

I keep encountering a particular `ResourceWarning: unclosed file 
<_io.BufferedRandom` in my test suite that I can not track down.  I'm 
hoping someone else may have an idea:

* In a suite of 10+ test classes, it only happens when 2 particular classes 
are run consecutively and in a particular order. It does not happen in the 
inverse.  
* this happens in a testapp setup, on the consecutive setup
* the error happens in this stack:

pyramid.paster.get_appsettings("test.ini")
...
plaster.loaders.find_loader()

It seems to trigger right after the final `return [EntryPointLoaderInfo...] 

The error is also coming from `packaging/version.py`

site-packages/pkg_resources/_vendor/packaging/version.py:141: 
ResourceWarning: unclosed file <_io.BufferedRandom name=5>

and is in the _cmpkey function:

release = tuple(
reversed(list(
itertools.dropwhile(
lambda x: x == 0,
reversed(release),
)
))
)

I can't seem to recreate this, and I can't pinpoint things further to fix.  
It happens somewhere in the pyramid/plaster/paste configuration.  

Does this situation look familiar to anyone?  This makes no sense.  I have 
a small hope that someone has experienced this before and knows where I 
should be focusing my efforts.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/b109be13-a837-48b0-8ca7-6c061396d6d8%40googlegroups.com.

Re: [pylons-discuss] is it possible to 'nest' a pyramid application within a larger library?

[Jonathan Vanasco]

thank you! 

On Monday, January 27, 2020 at 5:41:42 PM UTC-5, Theron Luhn wrote:
>
> Your setup.py is correct.
>
> In the ini file, `use` points not to a module but an entry point.  
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/7df57ab7-b9c6-4bf0-b1e2-b37b495eefe4%40googlegroups.com.

Re: [pylons-discuss] is it possible to 'nest' a pyramid application within a larger library?

[Jonathan Vanasco]

wow. it would be great if I can get this to work then.  this package 
started out as a pyramid app, but now the pyramid/web usage is 1/3 of 
typical usage patterns (it also has a terminal console and a python 
developer library).  i'd prefer to restructure the distribution so the 
pyramid-only stuff is reorganized under "web".

I can't match up the right combo of these 2 things into something that will 
install/run.  i assume these are what I need to configure:

environment.ini
  [app:main]
  use = egg:myapp.web

setup.py
  [paste.app_factory]
  main =  myapp.web:main



On Monday, January 27, 2020 at 3:44:11 PM UTC-5, Theron Luhn wrote:
>
> This is totally possible.  I actually have a live application that’s set 
> up in much the same way.  (Pyramid app nested inside another Pyramid app.)
>
> It was pretty straightforward when I did it, I don’t think there’s any 
> assumptions made about where the application lives.  What particular 
> troubles you’re having?
>
> — Theron
>
>
>
> On Jan 27, 2020, at 12:37 PM, Jonathan Vanasco  > wrote:
>
> does anyone know if it possible to nest a pyramid application within a 
> larger library?
>
> for example, I would like a Pyramid application's root to be `myapp.web`, 
> instead of `myapp`.  
>
> I don't think this is possible, which is fine - I can just split the 
> pyramid app into a separate package.
>
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to pylons-...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/pylons-discuss/66bd44da-f33b-44db-b479-b5d060091067%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/pylons-discuss/66bd44da-f33b-44db-b479-b5d060091067%40googlegroups.com?utm_medium=email_source=footer>
> .
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/ba721314-be8b-481e-a686-8dfe05f2ef25%40googlegroups.com.

[pylons-discuss] is it possible to 'nest' a pyramid application within a larger library?

[Jonathan Vanasco]

does anyone know if it possible to nest a pyramid application within a 
larger library?

for example, I would like a Pyramid application's root to be `myapp.web`, 
instead of `myapp`.  

I don't think this is possible, which is fine - I can just split the 
pyramid app into a separate package.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/66bd44da-f33b-44db-b479-b5d060091067%40googlegroups.com.

[pylons-discuss] Re: 'samesite' cookie attribute and Chrome changes

[Jonathan Vanasco]

1. Bert, thank you!

2. Mike, this stuff is generally a mess:

There are now 4 valid options for a cookie:

Python Value | Cookie Value (all strings)
None | 
"None"   | None< this is the new "experimental" one that google has 
forced
"Strict" | Strict
"Lax"| Lax




You are supposed to still have the behavior you desire as long as the 
samesite is "lax" and the cookie has a "domain" attribute.  That's the 
public hope, at least.  There are some releases of specific browsers that 
are broken.  It is a mess.

However, you need to understand these changes mean that the experience is 
definitely going to break for some amount of users, as some browsers can't 
handle the samesite values.  Browser sniffing can help get around that.

My approach would be deciding which users to prioritize and if you need to 
work on browser sniffing.  I would look at your traffic logs to decide 
that.  

If you have a lot of chrome traffic, you need to do browser testing WITH 
SPECIFIC CHROME FLAGS SET.  Not all installations of chrome have these set.

IIRC the flags you want to enable are:

chrome://flags/#same-site-by-default-cookies
chrome://flags/#cookies-without-same-site-must-be-secure


-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/63b211be-0190-4458-a376-8efc2817644a%40googlegroups.com.

Re: [pylons-discuss] upgraded to 1.4.2 and requests got stuck (load 1.0)

[Jonathan Vanasco]

Not Heroku, but in general...


On Friday, January 10, 2020 at 9:21:36 PM UTC-5, Peter Lada wrote:
>
>
> If anyone has a good insight on how to enable further request logging 
> (beyond the path that heroku already gives me) that would be great.
>

You could write a simple tween, modeled after pyramid_exclog (
https://github.com/Pylons/pyramid_exclog) that times the request (and 
possibly checks the CPU load); then logs pertinent information about the 
request to a database if thresholds are met.  I've used this approach to 
grab GET/POST vars and Paths to identify and recreate edge cases. 

You could also do this with Sentry; their API allow custom events, beyond 
standard exception logging.

I'd also look at your HTTP and Google Analytics logs to for any patterns 
when these issues spike.

I don't know if this is possible on Heroku, but I still use Statsd timing 
to log certain types of sever activity (like db connection management 
actions) - and look for any changes in behavior after a release.

But...

Based on what you've disclosed:

1) nothing makes this look particularly waitress related, and not something 
like traffic spikes, or spikes or problematic routes. I know nothing about 
your apps, but most sites I've worked on have the lowest traffic of the 
year between Dec 23-Jan 7.

2) if this were waitress related, i'd be suspect of the entire 1.4 release 
family and not just the ones you identified.   there were a handful of 
changes in in 1.4.0 (
https://docs.pylonsproject.org/projects/waitress/en/stable/#id4) 

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/23bde29f-7661-4b23-ab7a-66e8a68565c3%40googlegroups.com.

Re: [pylons-discuss] upgraded to 1.4.2 and requests got stuck (load 1.0)

[Jonathan Vanasco]

On Thursday, January 9, 2020 at 5:16:12 PM UTC-5, Jonathan Vanasco wrote:
>
> Is there a reason why you're both using `1.4.2` and not `1.4.9` ?
>

Nevermind, just realized you're talking about waitress which just pushed a 
1.4.2 and not Pyramid.

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/aa5bd9e4-8872-451d-b54d-031337ad5a3b%40googlegroups.com.

Re: [pylons-discuss] upgraded to 1.4.2 and requests got stuck (load 1.0)

[Jonathan Vanasco]

Is there a reason why you're both using `1.4.2` and not `1.4.9` ?

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/d2e453a2-4944-4b38-9c8e-dadb6d1491e5%40googlegroups.com.

Re: [pylons-discuss] Confused accessing db sessions

[Jonathan Vanasco]

On Monday, January 6, 2020 at 4:03:19 PM UTC-5, Kate Boelhauf wrote:
>
> This is insanely helpful - thank you Michael.  I'm going to pass the 
> dbsession into the class method for now but look into a service layer.  
>

A few years ago I adopted the pattern of passing Pyramid's `request` object 
into all class methods - and pretty much everything else - as the first 
argument representing the "context".  From that, I can grab my active core 
database session via `request.dbsession`.  If I need to override with an 
explicit dbSession, I pass in a dbsession kwarg or will inspect the object 
using `sqlalchemy.orm.object_session(user)` as Michael suggested.

At one point I passed in the dbSession, but I found myself with a chunks of 
code that expected a dbSession and chunks that expected a request.  
Standardizing everything to a request saved a lot of headaches down the 
line for me.

>

-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/89768643-b6bc-4458-b0ee-b8b8f3d57b54%40googlegroups.com.

Re: [pylons-discuss] question regarding url dispatch with very similar routes

[Jonathan Vanasco]

On Thursday, December 19, 2019 at 12:26:34 PM UTC-5, Michael Merickel wrote:
>
> You didn't make it completely clear what you're doing with your 
> view_config but I assume it's something to do with the match_param 
> predicate after your factory updates the matchdict. That's fine, and you 
> could use it to annotate the request with another attribute and then write 
> a custom predicate for your views to use instead of the match_param. Think 
> something like request.user_focus = 'a_username', 
> and @view_config(route_name='user_focus', user_type='a_username') which 
> will work well if they are mutually exclusive.
>

Thanks, Michael.  I'm not doing that, but it looks like i should!

Your reply gave me enough insight/hints to get something working, and I'm 
pretty happy with it from a maintainability standpoint.

I'm now doing the following:



def factory__user_type_discriminator(request):
"""inspects the matchdict's 'integrated_user_identifier' placeholder;
   sets 'user_type=A' or 'user_type=B' in the matchdict
"""
pass

config.add_route("integrated_user_focus",
 "/users/{integrated_user_identifier}",
 factory=factory__user_type_discriminator
 )


class ViewClass(handler):

@view_config(route_name="integrated_user_focus", match_param=
'user_type=A')
 def view_type_a(self, request):
pass
  
@view_config(route_name="integrated_user_focus", match_param=
'user_type=B')
 def view_type_b(self, request):
pass







-- 
You received this message because you are subscribed to the Google Groups 
"pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to pylons-discuss+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/pylons-discuss/351ad616-1ba0-4f16-b20d-4b8a59590230%40googlegroups.com.