hi Felipe -- you need to check the Status results returned by these Arrow function calls. Possibly one of the calls is failing, but the failure is going unchecked.
If you compile with clang, it will warn you when you forget to check a Status. If you use C++ exceptions for error handling, it may be useful to have a macro that converts Arrow errors into C++ exceptions -- see what we did in MapD, for example: https://github.com/mapd/mapd-core/blob/9301b845a1662165d2b20381afdfbc3e5cf174db/QueryEngine/ArrowUtil.h#L28 - Wes On Mon, Oct 30, 2017 at 9:16 AM, Uwe L. Korn <uw...@xhochy.com> wrote: > Hello Felipe, > > from a brief look, this code seems to be fine. Note that in the recent > version of Arrow, HdfsClient was merged into HadoopFileSystem and thus > the code will slightly differ there (should be only naming changes for > you). You should update to avoid running into any difficulties because > of that. With a backtrace or the position of the actual segfault, we > should be able to help you better. > > Uwe > > On Mon, Oct 30, 2017, at 07:49 AM, Felipe Aramburu wrote: >> I tried making some simple changes to the reader-writer.cc example to >> test >> it working with hdfs. I am pretty sure I have set up my hdfs properly >> because I can do things like create directories. I copied over the >> parquet >> file generate by the test to hdfs and tried to load it in as follows: >> >> >> >> /********************************************************************************** >> PARQUET READER EXAMPLE >> >> **********************************************************************************/ >> >> try { >> // Create a ParquetReader instance >> >> std::shared_ptr<arrow::io::HdfsClient> hdfs; >> arrow::Status connectionStat = >> arrow::io::HdfsClient::Connect(&hdfsConfig, >> &hdfs); >> >> hdfs->MakeDirectory("/tester"); >> std::cout<<"made directory"<<std::endl; >> std::shared_ptr<arrow::io::HdfsReadableFile> in_file; >> arrow::Status stat = >> hdfs->OpenReadable("/testing2/test2.parquet",1024*1024,&in_file); >> std::cout<<"opened readable"<<std::endl; >> std::shared_ptr<parquet::FileMetaData> file_metadata; >> std::unique_ptr<parquet::ParquetFileReader> parquet_reader = >> >> >> parquet::ParquetFileReader::Open(in_file,parquet::default_reader_properties(),file_metadata); >> >> >> Is this the preferred way of doing this? I should be clear its >> segfaulting >> but due to a linker error that I am not able to debug the parquet-cpp >> files at the moment (will fix anc update with more info as soon as I >> can). >> I did want to make sure that this is the right way of doing this. I had >> trouble finding documentation on this. >> >> Felipe >> ᐧ
