I tried making some simple changes to the reader-writer.cc example to test
it working with hdfs. I am pretty sure I have set up my hdfs properly
because I can do things like create directories. I copied over the parquet
file generate by the test to hdfs and tried to load it in as follows:
/**********************************************************************************
PARQUET READER EXAMPLE
**********************************************************************************/
try {
// Create a ParquetReader instance
std::shared_ptr<arrow::io::HdfsClient> hdfs;
arrow::Status connectionStat = arrow::io::HdfsClient::Connect(&hdfsConfig,
&hdfs);
hdfs->MakeDirectory("/tester");
std::cout<<"made directory"<<std::endl;
std::shared_ptr<arrow::io::HdfsReadableFile> in_file;
arrow::Status stat =
hdfs->OpenReadable("/testing2/test2.parquet",1024*1024,&in_file);
std::cout<<"opened readable"<<std::endl;
std::shared_ptr<parquet::FileMetaData> file_metadata;
std::unique_ptr<parquet::ParquetFileReader> parquet_reader =
parquet::ParquetFileReader::Open(in_file,parquet::default_reader_properties(),file_metadata);
Is this the preferred way of doing this? I should be clear its segfaulting
but due to a linker error that I am not able to debug the parquet-cpp
files at the moment (will fix anc update with more info as soon as I can).
I did want to make sure that this is the right way of doing this. I had
trouble finding documentation on this.
Felipe
ᐧ
