Hi Micah,
Le 27/07/2019 à 05:43, Micah Kornfield a écrit : > Hi Arrow Dev, > As a follow-up to an old thread [1] on working with BigQuery and Arrow. I > just wanted to share some work that Brian Hulette and I helped out with. > > I'm happy to announce there is now preliminary support for reading Arrow > data in the BigQuery Storage API [1]. Python library support is available > in the latest release of google-cloud-bigquery-storage [2][3]. This is nice. Reading the original ML thread [1], does this mean that high-speed Avro-to-Arrow parsing has become less important now? Regards Antoine. > > Caveats: > - Small cached tables are not supported (same with Avro) > - Row filters aren't supported yet. > > Cheers, > Micah > > [1] > https://lists.apache.org/thread.html/6d374dc6c948d3e84b1f0feda1d48eddf905a99c0ef569d46af7f7af@%3Cdev.arrow.apache.org%3E > [2] https://cloud.google.com/bigquery/docs/reference/storage/ > [3] https://pypi.org/project/google-cloud-bigquery-storage/ > [4] > https://googleapis.github.io/google-cloud-python/latest/bigquery_storage/gapic/v1beta1/reader.html#google.cloud.bigquery_storage_v1beta1.reader.ReadRowsIterable.to_arrow >
