I think the second option is what we'll go for. I'm going to add a flag to pull that will allow the user to specify extra flags to be given to rsync. Then we can call crave pull like this: ./crave pull --extra-rsync-flags ' --ignore-missing-args' '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
*** Note the additional space before the hypen in ' --ignore-missing-args' . This should handle the missing source files/directories. What do you think? Thanks, -Uv On Dec 1 2023, at 12:56 am, Mikhail Khludnev <m...@apache.org> wrote: > Hello Yuvraaj, > Thanks for taking care of this. Honestly it's not my wheelhouse. > I seems like there's a consideration that a test getting out of heap will > create heapdumps folder and put a file into. I don't know wether test/gradle > can dump heap there ever. At least we don't have tests dumps heap there now. > So, whether this folder exists or is absent is not certain. > We have a few options: > - drop heapdumps/** from crave pull until someone needs to investigate a test > falling out of memory. > - hack crave pull to ignore path wildcards for absent dir > - execute $mkdir heapdumps or $mkdir -p heapdumps (depending on script's > error handling more) before $crave pull > > > On Thu, Nov 30, 2023 at 11:24 PM Yuvraaj Kelkar <u...@crave.io > (mailto:u...@crave.io)> wrote: > > I just started a build with crave: > > crave run ./gradlew --console=plain check integrationTests > > > > And at the end of it, looked for the patterns in the crave pull command: > > > > admin@171074329f9e:/tmp/src/solr$ find . -name '*.events' > > admin@171074329f9e:/tmp/src/solr$ find . -name 'hs_err_pid*' > > admin@171074329f9e:/tmp/src/solr$ > > admin@171074329f9e:/tmp/src/solr$ ls -l heapdumps > > ls: cannot access 'heapdumps': No such file or directory > > > > The only thing I could get a lot of output on was > > > > admin@171074329f9e:/tmp/src/solr$ find . | grep 'build.*test.TEST' | head > > ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.JsonRequestApiTest.xml > > ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.UsingSolrJRefGuideExamplesTest.xml > > > > (https://link.getmailspring.com/link/a1463805-bfd9-43a5-bd19-08e9ae1a4...@getmailspring.com/0?redirect=TEST-org.apache.solr.client.ref_guide_examples.UsingSolrJRefGuideExamplesTest.xml&recipient=ZGV2QGx1Y2VuZS5hcGFjaGUub3Jn) > > ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.IndexingNestedDocuments.xml > > ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.ZkConfigFilesTest.xml > > ./solr/solr-ref-guide/build/test-results/test/TEST-org.apache.solr.client.ref_guide_examples.JsonRequestApiHeatmapFacetingTest.xml > > ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.exporter.SolrExporterIntegrationTest.xml > > ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.scraper.SolrStandaloneScraperBasicAuthTest.xml > > ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.exporter.MetricsQueryTemplateTest.xml > > ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.scraper.SolrStandaloneScraperTest.xml > > ./solr/prometheus-exporter/build/test-results/test/TEST-org.apache.solr.prometheus.scraper.SolrCloudScraperTest.xml > > > > Is there some other build command required to generate the other file > > patterns? > > Thanks, > > -Uv > > > > On Nov 30 2023, at 11:33 am, Yuvraaj Kelkar <u...@crave.io > > (mailto:u...@crave.io)> wrote: > > > Investigating. > > > > > > On Nov 26 2023, at 12:32 am, Mikhail Khludnev <m...@apache.org > > > (mailto:m...@apache.org)> wrote: > > > > Pardon > > > > > > > > On Sun, Nov 26, 2023 at 11:28 AM Gautam Worah <worah.gau...@gmail.com > > > > (mailto:worah.gau...@gmail.com)> wrote: > > > > > I think you meant to send it to d...@solr.apache.org > > > > > (mailto:d...@solr.apache.org)? > > > > > > > > > > On Sun, Nov 26, 2023 at 12:24 AM Mikhail Khludnev <m...@apache.org > > > > > (mailto:m...@apache.org)> wrote: > > > > > > Hello > > > > > > It's rather like a logical error in crave pull. How to work around > > > > > > it? > > > > > > > > > > > > + status=0 > > > > > > + ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' > > > > > > 'heapdumps/**' '**/hs_err_pid*' > > > > > > Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: > > > > > > No such file or directory (2) > > > > > > rsync error: some files/attrs were not transferred (see previous > > > > > > errors) (code 23) at main.c(1682) [Receiver=3.1.3] > > > > > > rsync: [Receiver] write error: Broken pipe (32) > > > > > > > > > > > > + exit 0 > > > > > > ---------- Forwarded message --------- > > > > > > From: Apache Jenkins Server <jenk...@builds.apache.org > > > > > > (mailto:jenk...@builds.apache.org)> > > > > > > Date: Sun, Nov 26, 2023 at 11:17 AM > > > > > > Subject: [JENKINS] Solr » Solr-Check-9.x - Build # 5949 - Still > > > > > > Failing! > > > > > > To: <bui...@solr.apache.org (mailto:bui...@solr.apache.org)> > > > > > > > > > > > > > > > > > > > > > > > > Build: > > > > > > https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/5949/ > > > > > > No tests ran. > > > > > > Build Log: > > > > > > [...truncated 1490 lines...] > > > > > > ERROR: Step ‘Publish JUnit test result report’ failed: No test > > > > > > report files were found. Configuration error? > > > > > > Email was triggered for: Failure - Any > > > > > > Sending email for trigger: Failure - Any > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > > To unsubscribe, e-mail: builds-unsubscr...@solr.apache.org > > > > > > (mailto:builds-unsubscr...@solr.apache.org) > > > > > > For additional commands, e-mail: builds-h...@solr.apache.org > > > > > > (mailto:builds-h...@solr.apache.org) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Sincerely yours > > > > > > Mikhail Khludnev > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Sincerely yours > > > > Mikhail Khludnev > > > > > > > > > > > > > > > > > > -- > Sincerely yours > Mikhail Khludnev > > >
