[ https://issues.apache.org/jira/browse/IMPALA-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-6363. ----------------------------------- Resolution: Fixed Fix Version/s: Impala 2.12.0 IMPALA-6363: avoid cscope build races Use the -ignore_readdir_race flag for find so that find doesn't fail if a directory disappears under it. From what I could tell the flag has been in GNU find for a long time and is also available in other OS flavours like BSD and OS X. Make the step depend on gen-deps so that it can index thrift, protobuf, etc, output. Change-Id: I22bdb7c64036cb88a8a10907af35c5e3a55a9195 Reviewed-on: [http://gerrit.cloudera.org:8080/9007] Reviewed-by: Tim Armstrong <[tarmstr...@cloudera.com|mailto:tarmstr...@cloudera.com]> Tested-by: Impala Public Jenkins > cscope build step seems racy, breaks compilation > ------------------------------------------------ > > Key: IMPALA-6363 > URL: https://issues.apache.org/jira/browse/IMPALA-6363 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure > Affects Versions: Impala 2.12.0 > Reporter: Michael Brown > Assignee: Tim Armstrong > Priority: Critical > Labels: flaky > Fix For: Impala 2.12.0 > > > In some downstream compilation environments of Impala, the compilation is > failing early with little to go on except: > {noformat} > Generating IR description files > [ 0%] Built target gen_ir_descriptions > find: ‘./be/src/codegen/CMakeFiles/test-loop.bc.dir/depend.make.tmp’: No such > file or directory > Scanning dependencies of target krb5_realm_override > Scanning dependencies of target function-registry > Scanning dependencies of target fb-deps > Scanning dependencies of target gutil > Scanning dependencies of target test-loop.bc > Scanning dependencies of target thrift-cpp > Scanning dependencies of target impala-parent > [ 0%] Generating files for builtins. > [ 0%] Running FlatBuffers compiler on CatalogObjects.fbs > [ 0%] Running thrift compiler on ErrorCodes.thrift > ======================================================================== > [ 1%] Building CXX object > be/src/kudu/security/CMakeFiles/krb5_realm_override.dir/krb5_realm_override.cc.o > Running mvn install -DskipTests > [ 1%] Building CXX object > be/src/gutil/CMakeFiles/gutil.dir/atomicops-internals-x86.cc.o > Directory: > /data/jenkins/workspace/impala-asf-master-exhaustive-rhel7/repos/Impala/impala-parent > ======================================================================== > [ 1%] Built target test-loop.bc > [ 1%] Building CXX object be/src/gutil/CMakeFiles/gutil.dir/bits.cc.o > [ 1%] Built target function-registry > [ 1%] Built target fb-deps > [ 1%] Running thrift compiler on beeswax.thrift > [ 1%] Running thrift compiler on BackendGflags.thrift > [ 2%] Building CXX object > be/src/gutil/CMakeFiles/gutil.dir/callback_internal.cc.o > [ 2%] Running thrift compiler on CatalogInternalService.thrift > make[2]: *** [CMakeFiles/cscope] Error 1 > make[1]: *** [CMakeFiles/cscope.dir/all] Error 2 > make[1]: *** Waiting for unfinished jobs.... > {noformat} > I've seen this twice, and the keys in both are: > # No such file or directory on an apparent tempfile, reported by {{find}} > # Failure related to {{cscope}} > It's possible the cscope target is racy if it is being run with other steps > in parallel. From {{bin/gen-cscope.sh}}: > {noformat} > # Generate list of files for Cscope to index > cd $IMPALA_HOME > find . -regex '.*\.\(cc\|c\|hh\|h\|java\|thrift\|flex\|y\)$' > cscope.files > {noformat} > If other make steps are being run in parallel, there could be races where > items that {{find}} is supposed to inspect become unlinked underneath it, and > it fails. > In the environment where I see this, it's happened twice, with successful > compilations in between. -- This message was sent by Atlassian JIRA (v7.6.3#76005)