Mgmaplus opened a new issue #10408: URL: https://github.com/apache/arrow/issues/10408
Hi, I'm using the bigquery_storage_v1 and I have an error when it calls read_schema. It seems this function is in pyarrow.lib I have pyarrow 4.0 installed which makes me think I need to install an earlier release. Any help is appreciated. <!--StartFragment--> [ERROR] AttributeError: module 'pyarrow' has no attribute 'ipc'Traceback (most recent call last): File "/var/task/lambda_handler.py", line 89, in lambda_handler raise err File "/var/task/lambda_handler.py", line 80, in lambda_handler report_json = fetchPlatform(job['platform'], job['credentials'], job['ranges'], date_ranges, job['labels_to_add']) File "/var/task/generators/gen_reports_rev.py", line 81, in fetchPlatform listOfFigs = BQ_report_builder(cred_dict, listOfDateRanges[i][0], listOfDateRanges[i][1], labelsToAdd) File "/var/task/generators/gen_reports_rev.py", line 583, in BQ_report_builder df = BIGQ_downloadLastDay(json_cert, projectID, bucketID, params, date = end) File "/var/task/generators/data_download.py", line 604, in BIGQ_downloadLastDay dataframe = (bqclient.query(query_string).result().to_datafram e()) File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1852, in to_dataframe record_batch = self.to_arrow( File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1667, in to_arrow for record_batch in self._to_arrow_iterable( File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1567, in _to_page_iterable yield from result_pages File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 717, in _download_table_bqstorage future.result() File "/var/lang/lib/python3.8/concurrent/futures/_base.py", line 437, in result return self.__get_result() File "/var/lang/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result raise self._exception File "/var/lang/lib/python3.8/concurrent/fut ures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 598, in _download_table_bqstorage_stream item = page_to_item(page) File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 581, in _bqstorage_page_to_arrow return page.to_arrow() File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 453, in to_arrow return self._stream_parser.to_arrow(self._message) File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 626, in to_arrow return self._parse_arrow_message(message) File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 650, in _parse_arrow_message self._parse_arrow_schema()& nbsp; File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 661, in _parse_arrow_schema self._schema = pyarrow.ipc.read_schema( | [ERROR] AttributeError: module 'pyarrow' has no attribute 'ipc' Traceback (most recent call last): File "/var/task/lambda_handler.py", line 89, in lambda_handler raise err File "/var/task/lambda_handler.py", line 80, in lambda_handler report_json = fetchPlatform(job['platform'], job['credentials'], job['ranges'], date_ranges, job['labels_to_add']) File "/var/task/generators/gen_reports_rev.py", line 81, in fetchPlatform listOfFigs = BQ_report_builder(cred_dict, listOfDateRanges[i][0], listOfDateRanges[i][1], labelsToAdd) File "/var/task/generators/gen_reports_rev.py", line 583, in BQ_report_builder df = BIGQ_downloadLastDay(json_cert, projectID, bucketID, params, date = end) File "/var/task/generators/data_download.py", line 604, in BIGQ_downloadLastDay dataframe = (bqclient.query(query_string).result().to_dataframe()) File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1852, in to_dataframe record_batch = self.to_arrow( File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1667, in to_arrow for record_batch in self._to_arrow_iterable( File "/mnt/efs/__python_packages__/google/cloud/bigquery/table.py", line 1567, in _to_page_iterable yield from result_pages File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 717, in _download_table_bqstorage future.result() File "/var/lang/lib/python3.8/concurrent/futures/_base.py", line 437, in result return self.__ge t_result() File "/var/lang/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result raise self._exception File "/var/lang/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 598, in _download_table_bqstorage_stream item = page_to_item(page) File "/mnt/efs/__python_packages__/google/cloud/bigquery/_pandas_helpers.py", line 581, in _bqstorage_page_to_arrow return page.to_arrow() File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 453, in to_arrow return self._stream_parser.to_arrow(self._message) File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 626, in to_arrow &nb sp;return self._parse_arrow_message(message) File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 650, in _parse_arrow_message self._parse_arrow_schema() File "/mnt/efs/__python_packages__/google/cloud/bigquery_storage_v1/reader.py", line 661, in _parse_arrow_schema self._schema = pyarrow.ipc.read_schema( -- | -- <!--EndFragment--> STOP! Are you reporting a bug, a possible bug, or requesting a feature? If so, please report under the ARROW project on the ASF JIRA server https://issues.apache.org/jira/browse/ARROW. This JIRA server is free to use and open to the public, but you must create an account if it is your first time. See our contribution guidelines for more information: http://arrow.apache.org/docs/developers/contributing.html We have GitHub issues available as a way for new contributors and passers-by who are unfamiliar with Apache Software Foundation projects to ask questions and interact with the project. Do not be surprised if the first response is to open a JIRA issue or to write an e-mail to one of the public mailing lists: * Development discussions: [email protected] (first subscribe by sending an e-mail to [email protected]). * User discussions: [email protected] (first subscribe by sending an e-mail to [email protected]). Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
