I thought the purpose of using Docker was to create a simpler, more reproducible environment. I don't use Docker regularly (in fact, my employer bans it from my work machine) and so, from my perspective, any use of Docker makes the process more complicated.
I also hate having to put all of my most important passwords into a single text file to make this process work. Julian On Tue, Oct 5, 2021 at 1:39 AM Francis Chuang <[email protected]> wrote: > > Hey Julian, > > Thanks for reporting that. I think the instructions in the howto needs > to be clearer regarding the steps to start asflike-release-environment. > I think we should put the actual commands used to start the mock servers > directly in there. > > Hmm, the docker build should not write to your home directory at all. It > creates a separate volume to mount into the container and uses that as > the home directory in the container, so it should never touch your real > home directory. > > I have been using it for the past few releases and it's always worked > for me. My only suggestion if you're still willing to try is run > `docker-compose down` from the root directory of the git repo to clear > all containers and to remove the `gradle-cache` from docker, then > starting over from there. > > As for pushing to gitbox, I think you can use > -Pasf.git.pushRepositoryProvider=GITBOX, that's what I have for docker: > https://github.com/apache/calcite-avatica/blob/89e0deb510311b85b8c8bacde6d2ff70c309930e/docker.sh#L251 > > Francis > > On 5/10/2021 7:24 pm, Julian Hyde wrote: > > Francis, > > > > That helped somewhat. I was able to get the Docker-powered server > > running using your instructions. (From reading the HOWTO, I had no > > idea that I was creating a server in a separate terminal.) I was > > running Tomcat on port 80, and so I had to shut that down to prevent > > port clashes. > > > > But then: > > > > $ ./gradlew prepareVote -Prc=0 > > Starting a Gradle Daemon (subsequent builds will be faster) > > > > FAILURE: Build failed with an exception. > > > > * What went wrong: > > Gradle could not start your build. > >> Cannot create service of type DefaultConfigurationCache using > >> DefaultConfigurationCache constructor as there is a problem with parameter > >> #10 of type ConfigurationCacheFingerprintController. > > > Cannot create service of type > > ConfigurationCacheFingerprintController using > > ConfigurationCacheFingerprintController constructor as there is a > > problem with parameter #5 of type FileCollectionFingerprinterRegistry. > > > Cannot create service of type > > FileCollectionFingerprinterRegistry using method > > VirtualFileSystemServices$BuildSessionServices.createFileCollectionFingerprinterRegistry() > > as there is a problem with parameter #1 of type > > List<FileCollectionFingerprinter>. > > > Could not create service of type CrossBuildFileHashCache > > using BuildSessionServices.createCrossBuildFileHashCache(). > > > Failed to create parent directory > > '/home/jhyde/dev/calcite-avatica/.gradle/6.8.1' when creating > > directory '/home/jhyde/dev/calcite-avatica/.gradle/6.8.1/fileHashes' > > > > > > I think the Docker build had created a .gradle directory as root, and > > when I ran as myself, I couldn't write to it. And then I gave up > > trying to do 'dry runs' and added the -Pasf flag to do it for real: > > > > $ ./gradlew prepareVote -Prc=0 -Pasf > > > >> Task :pushRcTag FAILED > > > > Build calcite-avatica FAILURE reason: > > Execution failed for task ':pushRcTag': > > Caused by: org.eclipse.jgit.api.errors.TransportException: > > https://github.com/apache/calcite-avatica.git: not authorized > > at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:180) > > at > > com.github.vlsi.gradle.release.jgit.dsl.GitExtensionsKt.push(GitExtensions.kt:132) > > at > > com.github.vlsi.gradle.release.GitPushTask$pushTag$1.invoke(GitPushTask.kt:54) > > > > After an hour checking my github username and password, I found this: > > > > $ git push origin test-tag-1 > > Username for 'https://github.com': julianhyde > > Password for 'https://[email protected]': > > remote: Support for password authentication was removed on August 13, > > 2021. Please use a personal access token instead. > > remote: Please see > > https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/ > > for more information. > > fatal: Authentication failed for > > 'https://github.com/apache/calcite-avatica/' > > > > If the remote was > > https://gitbox.apache.org/repos/asf/calcite-avatica.git or > > [email protected]:apache/calcite-avatica.git I would probably hve been > > able to authenticate, but it seems to be hard-wired to > > https://github.com/apache/calcite-avatica. > > > > There's a whole day gone up in smoke. Sometimes the 'productivity > > gains' of Gradle + Docker are pretty hard to discern. > > > > Julian > > > > > > > > > > > > On Mon, Oct 4, 2021 at 2:25 PM Francis Chuang <[email protected]> > > wrote: > >> > >> Hey Julian, > >> > >> I just tested everything in docker and it works if I do the following: > >> > >> Terminal 1: > >> git clone https://github.com/vlsi/asflike-release-environment > >> cd asflike-release-environment > >> ./recreate.sh calcite-avatica > >> > >> ... wait for the mock release environment to finish starting ... > >> > >> Terminal 2: > >> git clone https://github.com/apache/calcite-avatica > >> cd calcite-avatica > >> docker-compose run -v ~/.gnupg:/.gnupg dry-run > >> > >> This is the result: > >> --- snipped --- > >> > Task :createRcTag > >> Created tag avatica-1.19.0-rc0 -> > >> Ref[refs/tags/avatica-1.19.0-rc0=2df9697c97350c9e92f9336c2b8c1270076e45a9(-1)] > >> > >> > Task :pushRcTag > >> Pushing tag to Git remote release-origin-test: > >> git://127.0.0.1/calcite-avatica.git > >> Message from release-origin-test: > >> refs/tags/avatica-1.19.0-rc0: OK, > >> 2df9697c97350c9e92f9336c2b8c1270076e45a9 (fastForward) > >> > >> > Task :generateVoteText > >> Listing SVN folders [] at > >> http://127.0.0.1/svn/dist/dev/calcite/apache-calcite-avatica-1.19.0-rc0 > >> Fetching > >> http://127.0.0.1/svn/dist/dev/calcite/apache-calcite-avatica-1.19.0-rc0/apache-calcite-avatica-1.19.0-src.tar.gz.sha512@2 > >> Please find draft vote text in /src/build/prepareVote/mail.txt > >> > >> > Task :prepareVote > >> --- vote email snipped --- > >> > >> Deprecated Gradle features were used in this build, making it > >> incompatible with Gradle 7.0. > >> Use '--warning-mode all' to show the individual deprecation warnings. > >> See > >> https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings > >> > >> BUILD SUCCESSFUL in 5m 8s > >> 105 actionable tasks: 105 executed > >> > >> Francis > >> > >> On 5/10/2021 7:40 am, Stamatis Zampetakis wrote: > >>> I haven't done an avatica release so I don't know if using docker is must > >>> but maybe running directly the prepareVote task could work: > >>> > >>> ./gradlew prepareVote -Prc=0 -Pasf > >>> > >>> It may complain about missing properties. I have the following on my > >>> ~/.gradle/gradle.properties file: > >>> > >>> useGpgCmd=true > >>> signing.gnupg.executable=gpg > >>> signing.gnupg.useLegacyGpg=false > >>> signing.gnupg.keyName=D77C3383F1927570 > >>> signing.gnupg.passphrase=XXXX > >>> asfSvnUsername=zabetak > >>> asfSvnPassword=YYY > >>> asfGitSourceUsername=zabetak > >>> asfGitSourcePassword=ZZZZ > >>> asfNexusUsername=zabetak > >>> asfNexusPassword=YYY > >>> > >>> Best, > >>> Stamatis > >>> > >>> On Mon, Oct 4, 2021 at 10:03 PM Julian Hyde <[email protected]> wrote: > >>> > >>>> I'm still running into this 'Connection refused' error. It's a > >>>> show-stopper. I cannot make a release candidate unless this problem is > >>>> solved. > >>>> > >>>> On Mon, Oct 4, 2021 at 10:58 AM Julian Hyde <[email protected]> wrote: > >>>>> > >>>>> This is the command I used: > >>>>> > >>>>> docker-compose run -v ~/.gnupg:/.gnupg dry-run > >>>>> > >>>>> I'm running Docker 18.06.1-ce on Ubuntu 20.04.3 LTS. > >>>>> > >>>>> By the way, I was thinking of creating a "stage" branch in the Avatica > >>>>> git repo. We no longer use a branch per release (which is fine) but > >>>>> this means that the commit is not pushed to the git repo until after > >>>>> the vote passes. I propose to push the RC commit to the "stage" branch > >>>>> (and destructively force push if we need another RC) to give CI a > >>>>> chance to run on the RC. After the vote passes I will push a tag. > >>>>> > >>>>> Julian > >>>>> > >>>>> > >>>>> On Mon, Oct 4, 2021 at 10:48 AM Vladimir Sitnikov > >>>>> <[email protected]> wrote: > >>>>>> > >>>>>> I'm not sure which steps are you following, however, the same sequence > >>>>>> seems to work fine: > >>>>>> > >>>>>> Commands: > >>>>>> > >>>> https://github.com/vlsi/vlsi-release-plugins/blob/83c85c5faa4c7cd1fe0173b75c1cba5e60c3f209/.github/workflows/release-test.yml#L33-L60 > >>>>>> Logs: > >>>>>> > >>>> https://github.com/vlsi/vlsi-release-plugins/runs/3794304055?check_suite_focus=true > >>>>>> > >>>>>> Vladimir > >>>>>> > >>>>>> > >>>>>> пн, 4 окт. 2021 г. в 20:15, Julian Hyde <[email protected]>: > >>>>>> > >>>>>>> As release manager for the upcoming Avatica 1.19, I just tried to use > >>>>>>> the docker-based dry-run. I got the following failure: > >>>>>>> > >>>>>>> Build calcite-avatica FAILURE reason: > >>>>>>> Execution failed for task ':initializeNexusStagingRepository': > >>>>>>> java.io.UncheckedIOException: java.net.ConnectException: > >>>>>>> Failed to connect to /127.0.0.1:8080 > >>>>>>> at > >>>>>>> > >>>> de.marcphilipp.gradle.nexus.internal.NexusClient.findStagingProfileId(NexusClient.kt:84) > >>>>>>> at > >>>>>>> > >>>> de.marcphilipp.gradle.nexus.InitializeNexusStagingRepository.determineStagingProfileId(InitializeNexusStagingRepository.kt:113) > >>>>>>> at > >>>>>>> > >>>> de.marcphilipp.gradle.nexus.InitializeNexusStagingRepository.access$determineStagingProfileId(InitializeNexusStagingRepository.kt:38) > >>>>>>> > >>>>>>> I don't want to spend a morning debugging Docker port-mappings so I'm > >>>>>>> moving on. > >>>>>>> > >>>>>>> Julian > >>>>>>> > >>>> > >>>
