Hi all, I can convert my arrow table to buffer and can read back/convert that buffer to table thorough below parquet routines. But getting corrupt footer error when trying to convert plasma stored output buffer to table using parquet routines.
I have an arrow table as "my_arrow_table", //------------------------------------------ auto sink = std::make_shared<parquet::InMemoryOutputStream>(); const std::shared_ptr<parquet::arrow::ArrowWriterProperties>& arrow_properties = parquet::arrow::default_arrow_writer_properties(); WriteTable(*my_arrow_table, ::arrow::default_memory_pool(), sink, arrow_alignments_table->num_columns(), parquet::default_writer_properties(), arrow_properties); std::shared_ptr<arrow::Buffer> table_data = sink->GetBuffer(); //------------------------------------------ std::string metadata = "Table"; client.Create(object_id, table_data->size(), (uint8_t*) metadata.data(), metadata.size(), &table_data); ObjectBuffer object_buffer; ARROW_CHECK_OK(client1.Get(&object_id1, 1, -1, &object_buffer)); // Retrieve object data. auto outbuffer = object_buffer.data; std::shared_ptr<arrow::Buffer> plasmaData = outbuffer; //------------------------------------------ auto reader = parquet::ParquetFileReader::Open(std::make_shared<::arrow::io::BufferReader>(plasmaData)); parquet::arrow::FileReader filereader(::arrow::default_memory_pool(), std::move(reader)); std::shared_ptr<::arrow::Table> table; filereader.ReadTable(&table); //------------------------------------------ I'm getting following error: terminate called after throwing an instance of 'parquet::ParquetException' what(): Invalid parquet file. Corrupt footer. Aborted (core dumped) Even the outbuffer size is being shown same as without using Plasma object store/get. Any help in this regard will be highly appreciated. Thanks. Regards, Tanveer Ahmad
