You can simply pass --publish_to_big_query=false - this flag is used to publish test results to BigQuery, for us to keep a history of the performance of the tests. If you want that, then provide a BQ dataset for your performance history, but if you only need to run the test once and see the results, you don't need to pass any GCP-related info. Best -P.
On Mon, Dec 21, 2020 at 10:57 AM Ahmet Altay <[email protected]> wrote: > In case this helps there is an example of running tests on Dataflow here > [1]. Runner related information needs to be changed for running locally. > Does this help? > > Faisal, what is the purpose for running the load test locally (assuming > you mean direct runner)? Is this related to BEAM-10029? > > [1] > https://github.com/apache/beam/blob/164a7b68a477978d869c6a2bf3faf80135373390/.test-infra/jenkins/job_PerformanceTests_BigQueryIO_Python.groovy#L26 > > On Wed, Dec 16, 2020 at 11:13 AM Chamikara Jayalath <[email protected]> > wrote: > >> I haven't tried this, but in general. >> >> metrics_dataset = A BigQuery dataset (to publish metrics) >> metrics_table = A BigQuery table name >> staging_location = A GCS bucket (of the form gs://...) (also I think this >> should be optional since 'staging_location' defaults to 'temp_location') >> temp_location = A GCS bucket (of the form gs://...) >> >> Also, note that this command assumes a DataflowRunner, hence you'll have >> to perform Dataflow specific setup mentioned here: >> https://beam.apache.org/documentation/runners/dataflow/#setup >> >> Thanks, >> Cham >> >> >> On Wed, Dec 16, 2020 at 10:51 AM Ahmet Altay <[email protected]> wrote: >> >>> +Pablo Estrada <[email protected]> +Heejong Lee <[email protected]> >>> +Chamikara >>> Jayalath <[email protected]> >>> >>> On Wed, Dec 16, 2020 at 9:15 AM Faisal Maqsood < >>> [email protected]> wrote: >>> >>>> Hey everyone, >>>> I need some help related to running LoadTest on the local machine. >>>> Reference to the file (*apache_beam.io.gcp.bigquery_read_perf_test* ) >>>> I find the command to run LoadTest but I am not sure about the parameters >>>> to be passed in this command can someone please help me out on this and >>>> give any example command. >>>> >>>> what value should be passed in these parameters >>>> >>>> *metrics_dataset = ...metrics_table* *= ...* >>>> >>>> *staging_location = ...* >>>> >>>> *temp_location = ...* >>>> >>>> Example test run on DataflowRunner: >>>> >>>> python -m apache_beam.io.gcp.bigquery_read_perf_test \ >>>> --test-pipeline-options=" >>>> --runner=TestDataflowRunner >>>> --project=... >>>> --region=... >>>> --staging_location=gs://... >>>> --temp_location=gs://... >>>> --sdk_location=.../dist/apache-beam-x.x.x.dev0.tar.gz >>>> --publish_to_big_query=true >>>> --metrics_dataset=gs://... >>>> --metrics_table=... >>>> --input_dataset=... >>>> --input_table=... >>>> --input_options='{ >>>> \"num_records\": 1024, >>>> \"key_size\": 1, >>>> \"value_size\": 1024, >>>> } >>>> >>>> Thanks in advance :) >>>> >>>> *Faisal Ali* >>>> Senior Software Engineer >>>> Mobile: +92 3433016854 <+92%20343%203016854> >>>> Skype: fessimax >>>> >>>> <http://venturedive.com/> >>>> >>>
