No, it will not prevent this. We already build against parquet-cpp master in the Arrow build (https://github.com/apache/arrow/blob/master/ci/travis_build_parquet_cpp.sh), but API breaks going the other way cause brief instability that's unavoidable. We do our best to make this "outages" brief but are beholden to the time it takes for Travis CI / Appveyor to run, which can sometimes be 6-12 hours
Thanks Wes On Sun, Nov 26, 2017 at 8:12 AM, Sandeep Joshi <[email protected]> wrote: >> I am not sure what additional benefit using a git submodule would be. > > It might prevent build errors arising from master-to-master instability. > > -Sandeep > > > > On Thu, Nov 23, 2017 at 9:40 PM, Wes McKinney <[email protected]> wrote: > >> We build Arrow automatically in the toolchain if the developer has not >> provided another install location via $ARROW_HOME or >> $PARQUET_BUILD_TOOLCHAIN: >> >> https://github.com/apache/parquet-cpp/blob/master/cmake_ >> modules/ThirdpartyToolchain.cmake#L340 >> >> I am not sure what additional benefit using a git submodule would be. >> >> - Wes >> >> On Thu, Nov 23, 2017 at 10:36 AM, Sandeep Joshi <[email protected]> >> wrote: >> > Wes, >> > >> > Build worked after latest git pull but I do wish it arrow were a >> submodule >> > of parquet-cpp ! >> > >> > thanks >> > -Sandeep >> > >> > On Thu, Nov 23, 2017 at 8:23 PM, Wes McKinney <[email protected]> >> wrote: >> > >> >> hi Sandeep, >> >> >> >> If you are not using the latest tagged releases of the projects, you >> >> should be using the master branches of each project for development. >> >> We did make some API changes in the last days that impacted >> >> parquet-cpp (which will be released as Arrow 0.8.0 and parquet-cpp >> >> 1.4.0 respectively), but master-to-master is stable now. >> >> >> >> - Wes >> >> >> >> On Thu, Nov 23, 2017 at 2:08 AM, Sandeep Joshi <[email protected]> >> >> wrote: >> >> > I am getting build errors seen below in parquet-cpp because of >> >> > incompatibilities with arrow version (latest) >> >> > To get a stable build, should I use master branch of both or some >> other >> >> > branch ? >> >> > Arrow does not appear as a git submodule of parquet-cpp so there is no >> >> way >> >> > to track dependencies or flag incompatible changes. >> >> > >> >> > The arrow build is as of git commit hash fc4e2c36d2c56 >> >> > Author: Wes McKinney <[email protected]> >> >> > Date: Tue Nov 21 19:01:48 2017 -0500 >> >> > >> >> > The parquet-cpp build has git commit hash 6a2ed4f29731 >> >> > Author: Phillip Cloud <[email protected]> >> >> > Date: Sun Nov 19 23:19:03 2017 -0500 >> >> > >> >> > In file included from >> >> > /usr/include/x86_64-linux-gnu/c++/7/bits/c++allocator.h:33:0, >> >> > from /usr/include/c++/7/bits/allocator.h:46, >> >> > from /usr/include/c++/7/memory:63, >> >> > from >> >> > /home/sandeep/utils/FILE_FORMAT/parquet-cpp/src/ >> >> parquet/arrow/reader.h:21, >> >> > from >> >> > /home/sandeep/utils/FILE_FORMAT/parquet-cpp/src/ >> >> parquet/arrow/reader.cc:18: >> >> > /usr/include/c++/7/ext/new_allocator.h: In instantiation of ‘void >> >> > __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with >> _Up = >> >> > arrow::Table; _Args = {std::shared_ptr<arrow::Schema>&, >> >> > std::vector<std::shared_ptr<arrow::Column>, >> >> > std::allocator<std::shared_ptr<arrow::Column> > >&}; _Tp = >> >> arrow::Table]’: >> >> > /usr/include/c++/7/bits/alloc_traits.h:475:4: required from ‘static >> >> void >> >> > std::allocator_traits<std::allocator<_Tp1> >> >> >>::construct(std::allocator_traits<std::allocator<_Tp1> >> >> >::allocator_type&, >> >> > _Up*, _Args&& ...) [with _Up = arrow::Table; _Args = >> >> > {std::shared_ptr<arrow::Schema>&, >> >> > std::vector<std::shared_ptr<arrow::Column>, >> >> > std::allocator<std::shared_ptr<arrow::Column> > >&}; _Tp = >> arrow::Table; >> >> > std::allocator_traits<std::allocator<_Tp1> >::allocator_type = >> >> > std::allocator<arrow::Table>]’ >> >> > /usr/include/c++/7/bits/shared_ptr_base.h:526:39: required from >> >> > ‘std::_Sp_counted_ptr_inplace<_Tp, _Alloc, >> >> > _Lp>::_Sp_counted_ptr_inplace(_Alloc, _Args&& ...) [with _Args = >> >> > {std::shared_ptr<arrow::Schema>&, >> >> > std::vector<std::shared_ptr<arrow::Column>, >> >> > std::allocator<std::shared_ptr<arrow::Column> > >&}; _Tp = >> arrow::Table; >> >> > _Alloc = std::allocator<arrow::Table>; __gnu_cxx::_Lock_policy _Lp = >> >> > (__gnu_cxx::_Lock_policy)2]’ >> >> > /usr/include/c++/7/bits/shared_ptr_base.h:637:4: required from >> >> > ‘std::__shared_count<_Lp>::__shared_count(std::_Sp_make_shared_tag, >> >> _Tp*, >> >> > const _Alloc&, _Args&& ...) [with _Tp = arrow::Table; _Alloc = >> >> > std::allocator<arrow::Table>; _Args = {std::shared_ptr<arrow:: >> Schema>&, >> >> > std::vector<std::shared_ptr<arrow::Column>, >> >> > std::allocator<std::shared_ptr<arrow::Column> > >&}; >> >> > __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2]’ >> >> > /usr/include/c++/7/bits/shared_ptr_base.h:1295:35: required from >> >> > ‘std::__shared_ptr<_Tp, _Lp>::__shared_ptr(std::_Sp_make_shared_tag, >> >> const >> >> > _Alloc&, _Args&& ...) [with _Alloc = std::allocator<arrow::Table>; >> _Args >> >> = >> >> > {std::shared_ptr<arrow::Schema>&, >> >> > std::vector<std::shared_ptr<arrow::Column>, >> >> > std::allocator<std::shared_ptr<arrow::Column> > >&}; _Tp = >> arrow::Table; >> >> > __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2]’ >> >> > /usr/include/c++/7/bits/shared_ptr.h:344:64: required from >> >> > ‘std::shared_ptr<_Tp>::shared_ptr(std::_Sp_make_shared_tag, const >> >> _Alloc&, >> >> > _Args&& ...) [with _Alloc = std::allocator<arrow::Table>; _Args = >> >> > {std::shared_ptr<arrow::Schema>&, >> >> > std::vector<std::shared_ptr<arrow::Column>, >> >> > std::allocator<std::shared_ptr<arrow::Column> > >&}; _Tp = >> >> arrow::Table]’ >> >> > /usr/include/c++/7/bits/shared_ptr.h:690:14: required from >> >> > ‘std::shared_ptr<_Tp> std::allocate_shared(const _Alloc&, _Args&& ...) >> >> > [with _Tp = arrow::Table; _Alloc = std::allocator<arrow::Table>; >> _Args = >> >> > {std::shared_ptr<arrow::Schema>&, >> >> > std::vector<std::shared_ptr<arrow::Column>, >> >> > std::allocator<std::shared_ptr<arrow::Column> > >&}]’ >> >> > /usr/include/c++/7/bits/shared_ptr.h:706:39: required from >> >> > ‘std::shared_ptr<_Tp> std::make_shared(_Args&& ...) [with _Tp = >> >> > arrow::Table; _Args = {std::shared_ptr<arrow::Schema>&, >> >> > std::vector<std::shared_ptr<arrow::Column>, >> >> > std::allocator<std::shared_ptr<arrow::Column> > >&}]’ >> >> > /home/sandeep/utils/FILE_FORMAT/parquet-cpp/src/ >> >> parquet/arrow/reader.cc:434:49: >> >> > required from here >> >> > /usr/include/c++/7/ext/new_allocator.h:136:4: error: invalid >> >> new-expression >> >> > of abstract class type ‘arrow::Table’ >> >> > { ::new((void *)__p) _Up(std::forward<_Args>(__args)...); } >> >> > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ >> >> > In file included from /usr/local/include/arrow/api.h:31:0, >> >> > from >> >> > /home/sandeep/utils/FILE_FORMAT/parquet-cpp/src/ >> >> parquet/arrow/reader.cc:30: >> >> > /usr/local/include/arrow/table.h:117:46: note: because the >> following >> >> > virtual functions are pure within ‘arrow::Table’: >> >> > class ARROW_EXPORT Table { >> >> > ^ >> >> > /usr/local/include/arrow/table.h:149:35: note: virtual >> >> > std::shared_ptr<arrow::Column> arrow::Table::column(int) const >> >> > virtual std::shared_ptr<Column> column(int i) const = 0; >> >> > ^~~~~~ >> >> > /usr/local/include/arrow/table.h:152:18: note: virtual arrow::Status >> >> > arrow::Table::RemoveColumn(int, std::shared_ptr<arrow::Table>*) const >> >> > virtual Status RemoveColumn(int i, std::shared_ptr<Table>* out) >> const >> >> = >> >> > 0; >> >> > ^~~~~~~~~~~~ >> >> > /usr/local/include/arrow/table.h:155:18: note: virtual arrow::Status >> >> > arrow::Table::AddColumn(int, const std::shared_ptr<arrow::Column>&, >> >> > std::shared_ptr<arrow::Table>*) const >> >> > virtual Status AddColumn(int i, const std::shared_ptr<Column>& >> column, >> >> > ^~~~~~~~~ >> >> > /usr/local/include/arrow/table.h:163:34: note: virtual >> >> > std::shared_ptr<arrow::Table> arrow::Table:: >> ReplaceSchemaMetadata(const >> >> > std::shared_ptr<const arrow::KeyValueMetadata>&) const >> >> > virtual std::shared_ptr<Table> ReplaceSchemaMetadata( >> >> > ^~~~~~~~~~~~~~~~~~~~~ >> >> > /usr/local/include/arrow/table.h:167:18: note: virtual arrow::Status >> >> > arrow::Table::Validate() const >> >> > virtual Status Validate() const = 0; >> >> > ^~~~~~~~ >> >> > CMakeFiles/parquet_objlib.dir/build.make:124: recipe for target >> >> > 'CMakeFiles/parquet_objlib.dir/src/parquet/arrow/reader.cc.o' failed >> >> > make[2]: *** [CMakeFiles/parquet_ob/jlib. >> dir/src/parquet/arrow/reader. >> >> cc.o] >> >> > Error 1 >> >> > CMakeFiles/Makefile2:462: recipe for target >> >> > 'CMakeFiles/parquet_objlib.dir/all' failed >> >> >>
