[google-appengine] Re: Use of Pandas in GAE

[Wesley C (Google)]

This topic came up recently again in Reddit . I 
answered there as well as this similar Q on SO 
. The bottom line is that this 
shouldn't be an issue after 2018 when the 2nd-gen App Engine service 
launched. You don't have to use GAE flexible, and if you think you do, try 
Cloud Run 1st.

Cheers,
--Wesley

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
"A computer never does what you want... only what you tell it."
wesley chun :: @wescpy  :: Software 
Architect & Engineer
Developer Advocate at Google 
 by day; at 
night: Core Python 


On Wednesday, February 15, 2017 at 3:10:19 PM UTC-8 pay...@google.com wrote:

> Hey Juan,
>
> In the App Engine Standard Environment 
> , 
> the docs explain that code and libraries must be pure python 
> 
> :
>
> All code for the Python runtime environment must be pure Python, and not 
> include any C extensions or other code that must be compiled. 
>
>
> However, for the Flexible Environment 
> , you can use C 
> modules, certainly, along with any code that would run on a linux machine. 
> The code runs inside a Docker container, but it's not quite the same as 
> deploying on a Compute Engine instance and is more like programming for App 
> Engine. 
>
> You should check out the quickstart for a Flexible Environment python app 
>  and 
> then see about reading my last comment which showed a Dockerfile which 
> would allow you to install libraries on the python instance container image.
>
>
> Cheers,
>
> Nick
> Cloud Platform Community Support
>
> On Wednesday, February 15, 2017 at 8:10:48 AM UTC-5, Juan Antonio 
> Fernández Sánchez wrote:
>>
>> Hi Nick, 
>>
>> Thank you for your answer. The problem is that I would like to run these 
>> libraries in a service inside a normal GAE app, not in a container in
>> Compute Engine. 
>>
>> Thanks!
>>
>> El lunes, 13 de febrero de 2017, 20:12:12 (UTC+1), Nick (Cloud Platform 
>> Support) escribió:
>>>
>>> Hey Juan,
>>>
>>> It could be because in installing the C -based python modules locally, 
>>> pip is compiling C code for your system's architecture, which wouldn't 
>>> translate to the deployed machine. If this is the case, see if the 
>>> following works as a solution:
>>>
>>> You can write a Dockerfile for your Custom Runtime app that will install 
>>> the requirements when the container image is built, that way it'll use the 
>>> architecture of the container that will actually run the app. Here's an 
>>> example Dockerfile from our python custom runtime github repository 
>>> :
>>>
>>> FROM gcr.io/google-appengine/python
>>>
>>> # Create a virtualenv for dependencies. This isolates these packages from
>>> # system-level packages.
>>> RUN virtualenv /env
>>>
>>> # Setting these environment variables are the same as running
>>> # source /env/bin/activate.
>>> ENV VIRTUAL_ENV /env
>>> ENV PATH /env/bin:$PATH
>>>
>>> # Copy the application's requirements.txt and run pip to install all
>>> # dependencies into the virtualenv.*ADD requirements.txt 
>>> /app/requirements.txt
>>> RUN pip install -r /app/requirements.txt*
>>>
>>> # Add the application source code.
>>> ADD . /app
>>>
>>> # Run a WSGI server to serve the application. gunicorn must be declared as
>>> # a dependency in requirements.txt.
>>> CMD gunicorn -b :$PORT main:app
>>>
>>> You can see the lines which install dependencies in bold.
>>>
>>> Let me know how that works out for you! We could see about getting this 
>>> caveat for custom runtime deployment and dependencies with c-based python 
>>> modules documented if that turns out to be the case.
>>>
>>> In order for me to be able to test and reproduce what you've seen in 
>>> future, could you also attach the requirements.txt to your reply?
>>>
>>> Cheers,
>>>
>>> Nick
>>> Cloud Platform Community Support
>>>
>>> On Sunday, February 12, 2017 at 3:13:01 PM UTC-5, Juan Antonio Fernández 
>>> Sánchez wrote:

 Thank you so much for the answer Nick, I'm trying to do this without 
 success. 

 I've the dependencie in my requirements.txt file and in the module 
 where I import the module like this: import pandas as pd-

 If I try to execute the code I've this exception:

 ImportError: Missing required dependencies ['numpy']

 And if I add numpy in requirements.txt (executing it) and add the 
 module in the same file where I've pandas I've this exception:

 ImportError: 
 Importing the multiarray numpy extension module failed.  Most
 

Re: [google-appengine] Updates from the Google App Engine team (Fall 2021)

['David (Cloud Platform Support)' via Google App Engine]

Hello,

Other than applying changes within your own application so it will go easy 
at startup, you can try using warmup requests 

 
along with tweaking the min_idle_instances element 

 
in your app.yaml, in order to reduce request and response latency during 
the time when your app's code is being loaded to a newly created instance.
There’s not an official way of forcing the scaling to behave like it used 
to be on python 2.7. However, this type of feedback can be passed to the 
App Engine engineering team in the form of a feature request. You can 
create such a feature request here 
. 
The App Engine engineering team would then evaluate it and decide whether 
it could be implemented or not.
On Thursday, November 4, 2021 at 11:38:52 AM UTC-4 Joshua Smith wrote:

> Thanks for sending out this update. I did, indeed, miss most of this news.
>
> This item, in particular, is *awesome*:
>
>
>
>- 
>
>Extending support for App Engine bundled services 
>
> 
>  
>(Sep 2021) 
>
>
> This sounds like it will make it so much easier to migrate to Python 3.7.
>
> One thing I've noticed is that my older apps on 2.7 seem to handle peak 
> scaling a lot better than my newer apps on 3.7. For example, if I have a 
> web page that hits a 2.7 app with 100 REST calls at startup (bad design, 
> but it happens), the old app serves them all eventually. But if I do the 
> same thing in a 3.7 app, it's likely to choke and fail a bunch of those 
> requests with these:
>
>
> The specific pattern is that after the first couple requests, it spins up 
> a new instance. That takes 5 seconds to serve its first request (simple 
> app, so I guess that's just GAE overhead). Then it spins up a couple more. 
> Then I start getting those errors on some of the requests, because 15s have 
> passed. In this capture the blue ones are new instances spinning up, and 
> the orange ones are timeout errors (note the 15s time on those).
>
>
> I've been designing around the issue by making sure my web apps go easy on 
> the server on startup. But it is yet another concern about migrating.
>
> Is there a way to tune the auto-scaling for 3.7 to behave like the 2.7? 
> Here's a similar set of requests to my 2.7 app, which only spins up one 
> extra instance, and never throws timeout errors, ever:
>
>
> -Joshua
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/be0758b0-a47c-4d4e-bdee-6b27ec308e3fn%40googlegroups.com.

[google-appengine] Updates from the Google App Engine team (Fall 2021)

[wesley chun]

App Engine and Google Cloud serverless users:

The App Engine  team at Google Cloud
 had a flurry of announcements and product updates
over the past few months. In case you missed them, we've summarized them
here with relevant links.


   -

   New features to better secure your Google App Engine apps
   

   (Aug 2021) —  Introducing a pair of networking and security features:
   -

  Egress Controls for Serverless VPC Access
  

  — consistent app (static) outbound IP address
  -

  User-managed service accounts
  

  — custom service accounts to restrict apps to only the permissions
  necessary, following the best practice of "least privileges"
  -

   Exploring serverless with a nebulous app
   

   (Sep 2021) — Deploy/shift the same app to App Engine, Cloud Functions,
   or Cloud Run without any code changes, and show how to access Cloud APIs
   from serverless
   -

   Extending support for App Engine bundled services
   

   (Sep 2021) — Legacy App Engine APIs/bundled services, e.g., Memcache
   ,
   Datastore
   , etc.,
   were left out of the next-generation service (to fight against "vendor
   lock-in" not because they were deprecated). Many matured into standalone
   products, e.g., Cloud Memorystore
, Cloud
   Datastore , etc., respectively
   anyway. To help with updating language versions (e.g., Python 2 to 3, Java
   8 to 11, etc.), we've added many of these bundled services "back" to the
   latest App Engine runtimes.
   -

   Modernizing your serverless applications
   

   (Sep 2021) — Introducing codelabs (hands-on tutorials) and new video
   series  to help customers migrate to standalone
   services to modernize & make their apps more portable (second-generation
   App Engine, Cloud Functions, Cloud Run; Kubernetes Engine (GKE), Compute
   Engine VMs; other cloud, multi-cloud, hybrid cloud, on-prem, etc.):
   -

  App Engine webapp2 to Flask
  

  -

  App Engine taskqueue to Cloud Tasks
  

  (push queues)
  -

  App Engine ndb (Datastore) to Cloud NDB
  


  -

  Cloud NDB to Cloud Datastore
  

  -

  App Engine to Cloud Run (with Docker
  

  or without Docker using Cloud Buildpacks
  

  )
  -

  More are on the way…


While we are working hard to improve your App Engine experience, recognize
App Engine is no longer the only serverless option available from Google
Cloud:



   -

   If you don't have an entire app, have single-function utilities or
   microservices, Cloud Functions  is a
   great alternative; it also supports event-driven workloads
   -

   For those leveraging containerization as part of their app modernization
   or software development workflows or wish to avoid some of the constraints
   from App Engine or Cloud Functions, consider Cloud Run 
   

[google-appengine] How to start to use GCP Retail console

[水口和也]

We want to use GCP Retail ( include  Recommendations AI and Retail Search).

The Retail Dashboard on GCP is only accessible for allowlisted projects.
So we submitted the form according to  the below instructions on GCP Retail 
console.

*Please submit a "Contact sales" form at https://cloud.google.com/contact  
 to get this project 
allowlisted to use this Retail Admin Console for Product Discovery (Retail 
Search and Recommendations AI).*

However, Sales Development contact personnel replied to the form "We cannot 
support this case, please use Google Cloud support".

Can anyone tell me how to start to use GCP Retail console if anyone else is 
using?

Thanks,
Kazuya Minakuchi

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/00b1bbe7-e693-402b-a5e0-8b6f71cae37en%40googlegroups.com.