Hi Katayoon, Here is my use case: 1) source : CSV file 2) one master table of size 500 GB and 500 million records. 3) daily updates of size 10GB and 10 million records.
I need to create 2) one time. Every day I need to update 3). Right now I import the CSV files and create tables in BigQuery. Then I can do additional work and export as CSV to Cloud Storage. With BigQuery I can prepare the tables in advance. Let say I have 1 table with 100 columns and 10 indexes on only 10 fields which are fields in the CSV exported from BigQuery in CloudStorage. Let say the master table is 500 GB/500 mil records and the indexes should be 1/10 or 50 GB. Every day I change/delete about 10 GB/10 million records. BigQuery is not for customer facing apps so I'm looking for the best way to achieve that most effective and cheapest way. For customer facing app I don't need full SQL - some limited SQL or if not possible KV store is OK. Best, --Constantine On Thursday, May 31, 2018 at 11:26:26 AM UTC-7, Katayoon (Cloud Platform Support) wrote: > > If you mean using Search API in App Engine to index your structured data > (your JSON files), you may take a look at this documentation > <https://cloud.google.com/appengine/docs/standard/python/search/> and the > pricing > <https://cloud.google.com/appengine/docs/standard/python/search/#Python_Search_API_pricing> > > is the same as brought in the App Engine tab of Pricing Calculator > <https://cloud.google.com/products/calculator/> (Indexing Documents > section). If not, could you provide more clarification about your use case? > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/1c4160f1-6f5f-49de-bcaa-24f787e25dc7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
