Thanks Wes! Very useful Felipe
On Mon, Oct 30, 2017 at 9:55 AM, Wes McKinney <[email protected]> wrote: > hi Felipe -- you need to check the Status results returned by these > Arrow function calls. Possibly one of the calls is failing, but the > failure is going unchecked. > > If you compile with clang, it will warn you when you forget to check a > Status. > > If you use C++ exceptions for error handling, it may be useful to have > a macro that converts Arrow errors into C++ exceptions -- see what we > did in MapD, for example: > > https://github.com/mapd/mapd-core/blob/9301b845a1662165d2b20381afdfbc > 3e5cf174db/QueryEngine/ArrowUtil.h#L28 > > - Wes > > On Mon, Oct 30, 2017 at 9:16 AM, Uwe L. Korn <[email protected]> wrote: > > Hello Felipe, > > > > from a brief look, this code seems to be fine. Note that in the recent > > version of Arrow, HdfsClient was merged into HadoopFileSystem and thus > > the code will slightly differ there (should be only naming changes for > > you). You should update to avoid running into any difficulties because > > of that. With a backtrace or the position of the actual segfault, we > > should be able to help you better. > > > > Uwe > > > > On Mon, Oct 30, 2017, at 07:49 AM, Felipe Aramburu wrote: > >> I tried making some simple changes to the reader-writer.cc example to > >> test > >> it working with hdfs. I am pretty sure I have set up my hdfs properly > >> because I can do things like create directories. I copied over the > >> parquet > >> file generate by the test to hdfs and tried to load it in as follows: > >> > >> > >> > >> /*********************************************************** > *********************** > >> PARQUET READER EXAMPLE > >> > >> ************************************************************ > **********************/ > >> > >> try { > >> // Create a ParquetReader instance > >> > >> std::shared_ptr<arrow::io::HdfsClient> hdfs; > >> arrow::Status connectionStat = > >> arrow::io::HdfsClient::Connect(&hdfsConfig, > >> &hdfs); > >> > >> hdfs->MakeDirectory("/tester"); > >> std::cout<<"made directory"<<std::endl; > >> std::shared_ptr<arrow::io::HdfsReadableFile> in_file; > >> arrow::Status stat = > >> hdfs->OpenReadable("/testing2/test2.parquet",1024*1024,&in_file); > >> std::cout<<"opened readable"<<std::endl; > >> std::shared_ptr<parquet::FileMetaData> file_metadata; > >> std::unique_ptr<parquet::ParquetFileReader> parquet_reader = > >> > >> parquet::ParquetFileReader::Open(in_file,parquet::default_ > reader_properties(),file_metadata); > >> > >> > >> Is this the preferred way of doing this? I should be clear its > >> segfaulting > >> but due to a linker error that I am not able to debug the parquet-cpp > >> files at the moment (will fix anc update with more info as soon as I > >> can). > >> I did want to make sure that this is the right way of doing this. I had > >> trouble finding documentation on this. > >> > >> Felipe > >> ᐧ >
