Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package velero for openSUSE:Factory checked in at 2021-01-19 16:02:15 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/velero (Old) and /work/SRC/openSUSE:Factory/.velero.new.28504 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "velero" Tue Jan 19 16:02:15 2021 rev:5 rq:864113 version:1.5.3 Changes: -------- --- /work/SRC/openSUSE:Factory/velero/velero.changes 2020-11-13 19:00:46.510229425 +0100 +++ /work/SRC/openSUSE:Factory/.velero.new.28504/velero.changes 2021-01-19 16:02:43.263427731 +0100 @@ -1,0 +2,15 @@ +Sat Jan 16 13:59:15 UTC 2021 - [email protected] + +- Update to version 1.5.3: + * Add changelog for v1.5.3 + * Increased limit for Velero pod to 512M. Fixes #3234 + * ???? BSLs with validation disabled should be validated at least once (#3084) + * Don't fail backup deletion if downloading tarball fails (#2993) + * ???? Do not run ItemAction plugins for unresolvable types for all types (#3059) + * ???? Use namespace and name to match PVB to Pod restore (#3051) + * Adding fix for restic init container index on restores. (#3011) + * v1.5.2 changelogs and cherry-picks (#3023) + * Add changelog and docs for v1.5 release (#2941) + * Spruce up release instructions and release scripts (#2931) + +------------------------------------------------------------------- Old: ---- velero-1.5.2.tar.gz New: ---- _servicedata velero-1.5.3.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ velero.spec ++++++ --- /var/tmp/diff_new_pack.ivelXZ/_old 2021-01-19 16:02:44.131429043 +0100 +++ /var/tmp/diff_new_pack.ivelXZ/_new 2021-01-19 16:02:44.131429043 +0100 @@ -1,7 +1,7 @@ # # spec file for package velero # -# Copyright (c) 2020 SUSE LLC +# Copyright (c) 2021 SUSE LLC # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -17,11 +17,11 @@ %define goipath github.com/vmware-tanzu/velero -%define commit e115e5a191b1fdb5d379b62a35916115e77124a4 +%define commit 123109a3bcac11dbb6783d2758207bac0d0817cb %define gitstate clean Name: velero -Version: 1.5.2 +Version: 1.5.3 Release: 0 Summary: Backup program with deduplication and encryption License: Apache-2.0 ++++++ _service ++++++ --- /var/tmp/diff_new_pack.ivelXZ/_old 2021-01-19 16:02:44.171429104 +0100 +++ /var/tmp/diff_new_pack.ivelXZ/_new 2021-01-19 16:02:44.171429104 +0100 @@ -1,4 +1,17 @@ <services> + <service name="tar_scm" mode="disabled"> + <param name="url">https://github.com/vmware-tanzu/velero</param> + <param name="scm">git</param> + <param name="exclude">.git</param> + <param name="versionformat">@PARENT_TAG@</param> + <param name="versionrewrite-pattern">v(.*)</param> + <param name="revision">v1.5.3</param> + <param name="changesgenerate">enable</param> + </service> + <service name="recompress" mode="disabled"> + <param name="file">velero-*.tar</param> + <param name="compression">gz</param> + </service> <service name="set_version" mode="disabled"> <param name="basename">velero</param> </service> ++++++ _servicedata ++++++ <servicedata> <service name="tar_scm"> <param name="url">https://github.com/vmware-tanzu/velero</param> <param name="changesrevision">123109a3bcac11dbb6783d2758207bac0d0817cb</param></service></servicedata>++++++ velero-1.5.2.tar.gz -> velero-1.5.3.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/changelogs/CHANGELOG-1.5.md new/velero-1.5.3/changelogs/CHANGELOG-1.5.md --- old/velero-1.5.2/changelogs/CHANGELOG-1.5.md 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/changelogs/CHANGELOG-1.5.md 2021-01-14 16:13:14.000000000 +0100 @@ -1,3 +1,25 @@ +## v1.5.3 +### 2021-01-14 +### Download +https://github.com/vmware-tanzu/velero/releases/tag/v1.5.3 + +### Container Image +`velero/velero:v1.5.3` + +### Documentation +https://velero.io/docs/v1.5/ + +### Upgrading +https://velero.io/docs/v1.5/upgrade-to-1.5/ + +### All Changes + * Increased default Velero pod memory limit to 512Mi (#3234, @dsmithuchida) + * ???? BSLs with validation disabled should be validated at least once (#3084, @ashish-amarnath) + * Fixed an issue where the deletion of a backup would fail if the backup tarball couldn't be downloaded from object storage. Now the tarball is only downloaded if there are associated DeleteItemAction plugins and if downloading the tarball fails, the plugins are skipped. (#2993, @zubron) + * ???? ItemAction plugins for unresolvable types should not be run for all types (#3059, @ashish-amarnath) + * ???? Use namespace and name to match PVB to Pod restore (#3051, @ashish-amarnath) + * Allows the restic-wait container to exist in any order in the pod being restored. Prints a warning message in the case where the restic-wait container isn't the first container in the list of initialization containers. (#3011, @doughepi) + ## v1.5.2 ### 2020-10-20 ### Download diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/internal/storage/storagelocation.go new/velero-1.5.3/internal/storage/storagelocation.go --- old/velero-1.5.2/internal/storage/storagelocation.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/internal/storage/storagelocation.go 2021-01-14 16:13:14.000000000 +0100 @@ -51,24 +51,28 @@ validationFrequency = bslValidationFrequency.Duration } - if validationFrequency == 0 { - log.Debug("Validation period for this backup location is set to 0, skipping validation") - return false - } - if validationFrequency < 0 { log.Debugf("Validation period must be non-negative, changing from %d to %d", validationFrequency, defaultLocationInfo.StoreValidationFrequency) validationFrequency = defaultLocationInfo.StoreValidationFrequency } lastValidation := lastValidationTime - if lastValidation != nil { // always ready to validate the first time around, so only even do this check if this has happened before - nextValidation := lastValidation.Add(validationFrequency) // next validation time: last validation time + validation frequency - if time.Now().UTC().Before(nextValidation) { // ready only when NOW is equal to or after the next validation time - return false - } + if lastValidation == nil { + // Regardless of validation frequency, we want to validate all BSLs at least once. + return true + } + + if validationFrequency == 0 { + // Validation was disabled so return false. + log.Debug("Validation period for this backup location is set to 0, skipping validation") + return false } + // We want to validate BSL only if the set validation frequency/ interval has elapsed. + nextValidation := lastValidation.Add(validationFrequency) // next validation time: last validation time + validation frequency + if time.Now().UTC().Before(nextValidation) { // ready only when NOW is equal to or after the next validation time + return false + } return true } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/internal/storage/storagelocation_test.go new/velero-1.5.3/internal/storage/storagelocation_test.go --- old/velero-1.5.2/internal/storage/storagelocation_test.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/internal/storage/storagelocation_test.go 2021-01-14 16:13:14.000000000 +0100 @@ -36,14 +36,20 @@ bslValidationFrequency *metav1.Duration lastValidationTime *metav1.Time defaultLocationInfo DefaultBackupLocationInfo - - // serverDefaultValidationFrequency time.Duration - // backupLocation *velerov1api.BackupStorageLocation - ready bool + ready bool }{ { - name: "don't validate, since frequency is set to zero", + name: "validate when true when validation frequency is zero and lastValidationTime is nil", + bslValidationFrequency: &metav1.Duration{Duration: 0}, + defaultLocationInfo: DefaultBackupLocationInfo{ + StoreValidationFrequency: 0, + }, + ready: true, + }, + { + name: "don't validate when false when validation is disabled and lastValidationTime is not nil", bslValidationFrequency: &metav1.Duration{Duration: 0}, + lastValidationTime: &metav1.Time{Time: time.Now()}, defaultLocationInfo: DefaultBackupLocationInfo{ StoreValidationFrequency: 0, }, @@ -63,7 +69,8 @@ defaultLocationInfo: DefaultBackupLocationInfo{ StoreValidationFrequency: 1, }, - ready: false, + lastValidationTime: &metav1.Time{Time: time.Now()}, + ready: false, }, { name: "validate as per default setting when location setting is not set", @@ -77,7 +84,8 @@ defaultLocationInfo: DefaultBackupLocationInfo{ StoreValidationFrequency: 0, }, - ready: false, + lastValidationTime: &metav1.Time{Time: time.Now()}, + ready: false, }, { name: "don't validate when now is before the NEXT validation time (validation frequency + last validation time)", @@ -112,8 +120,8 @@ t.Run(tt.name, func(t *testing.T) { g := NewWithT(t) log := velerotest.NewLogger() - - g.Expect(IsReadyToValidate(tt.bslValidationFrequency, tt.lastValidationTime, tt.defaultLocationInfo, log)).To(BeIdenticalTo(tt.ready)) + actual := IsReadyToValidate(tt.bslValidationFrequency, tt.lastValidationTime, tt.defaultLocationInfo, log) + g.Expect(actual).To(BeIdenticalTo(tt.ready)) }) } } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/backup/backup.go new/velero-1.5.3/pkg/backup/backup.go --- old/velero-1.5.2/pkg/backup/backup.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/backup/backup.go 2021-01-14 16:13:14.000000000 +0100 @@ -127,7 +127,7 @@ return nil, err } - resources := getResourceIncludesExcludes(helper, resourceSelector.IncludedResources, resourceSelector.ExcludedResources) + resources := collections.GetResourceIncludesExcludes(helper, resourceSelector.IncludedResources, resourceSelector.ExcludedResources) namespaces := collections.NewIncludesExcludes().Includes(resourceSelector.IncludedNamespaces...).Excludes(resourceSelector.ExcludedNamespaces...) selector := labels.Everything() @@ -150,30 +150,6 @@ return resolved, nil } -// getResourceIncludesExcludes takes the lists of resources to include and exclude, uses the -// discovery helper to resolve them to fully-qualified group-resource names, and returns an -// IncludesExcludes list. -func getResourceIncludesExcludes(helper discovery.Helper, includes, excludes []string) *collections.IncludesExcludes { - resources := collections.GenerateIncludesExcludes( - includes, - excludes, - func(item string) string { - gvr, _, err := helper.ResourceFor(schema.ParseGroupResource(item).WithVersion("")) - if err != nil { - // If we can't resolve it, return it as-is. This prevents the generated - // includes-excludes list from including *everything*, if none of the includes - // can be resolved. ref. https://github.com/vmware-tanzu/velero/issues/2461 - return item - } - - gr := gvr.GroupResource() - return gr.String() - }, - ) - - return resources -} - // getNamespaceIncludesExcludes returns an IncludesExcludes list containing which namespaces to // include and exclude from the backup. func getNamespaceIncludesExcludes(backup *velerov1api.Backup) *collections.IncludesExcludes { @@ -200,7 +176,7 @@ Name: hookSpec.Name, Selector: hook.ResourceHookSelector{ Namespaces: collections.NewIncludesExcludes().Includes(hookSpec.IncludedNamespaces...).Excludes(hookSpec.ExcludedNamespaces...), - Resources: getResourceIncludesExcludes(discoveryHelper, hookSpec.IncludedResources, hookSpec.ExcludedResources), + Resources: collections.GetResourceIncludesExcludes(discoveryHelper, hookSpec.IncludedResources, hookSpec.ExcludedResources), }, Pre: hookSpec.PreHooks, Post: hookSpec.PostHooks, @@ -242,7 +218,7 @@ log.Infof("Including namespaces: %s", backupRequest.NamespaceIncludesExcludes.IncludesString()) log.Infof("Excluding namespaces: %s", backupRequest.NamespaceIncludesExcludes.ExcludesString()) - backupRequest.ResourceIncludesExcludes = getResourceIncludesExcludes(kb.discoveryHelper, backupRequest.Spec.IncludedResources, backupRequest.Spec.ExcludedResources) + backupRequest.ResourceIncludesExcludes = collections.GetResourceIncludesExcludes(kb.discoveryHelper, backupRequest.Spec.IncludedResources, backupRequest.Spec.ExcludedResources) log.Infof("Including resources: %s", backupRequest.ResourceIncludesExcludes.IncludesString()) log.Infof("Excluding resources: %s", backupRequest.ResourceIncludesExcludes.ExcludesString()) log.Infof("Backing up all pod volumes using restic: %t", *backupRequest.Backup.Spec.DefaultVolumesToRestic) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/builder/pod_volume_backup_builder.go new/velero-1.5.3/pkg/builder/pod_volume_backup_builder.go --- old/velero-1.5.2/pkg/builder/pod_volume_backup_builder.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/builder/pod_volume_backup_builder.go 2021-01-14 16:13:14.000000000 +0100 @@ -75,6 +75,12 @@ return b } +// PodNamespace sets the name of the pod associated with this PodVolumeBackup. +func (b *PodVolumeBackupBuilder) PodNamespace(ns string) *PodVolumeBackupBuilder { + b.object.Spec.Pod.Namespace = ns + return b +} + // Volume sets the name of the volume associated with this PodVolumeBackup. func (b *PodVolumeBackupBuilder) Volume(volume string) *PodVolumeBackupBuilder { b.object.Spec.Volume = volume diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/controller/backup_controller.go new/velero-1.5.3/pkg/controller/backup_controller.go --- old/velero-1.5.2/pkg/controller/backup_controller.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/controller/backup_controller.go 2021-01-14 16:13:14.000000000 +0100 @@ -734,6 +734,10 @@ } func closeAndRemoveFile(file *os.File, log logrus.FieldLogger) { + if file == nil { + log.Debug("Skipping removal of file due to nil file pointer") + return + } if err := file.Close(); err != nil { log.WithError(err).WithField("file", file.Name()).Error("error closing file") } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/controller/backup_deletion_controller.go new/velero-1.5.3/pkg/controller/backup_deletion_controller.go --- old/velero-1.5.2/pkg/controller/backup_deletion_controller.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/controller/backup_deletion_controller.go 2021-01-14 16:13:14.000000000 +0100 @@ -295,13 +295,6 @@ errs = append(errs, err.Error()) } - // Download the tarball - backupFile, err := downloadToTempFile(backup.Name, backupStore, log) - if err != nil { - return errors.Wrap(err, "error downloading backup") - } - defer closeAndRemoveFile(backupFile, c.logger) - actions, err := pluginManager.GetDeleteItemActions() log.Debugf("%d actions before invoking actions", len(actions)) if err != nil { @@ -309,20 +302,30 @@ } // don't defer CleanupClients here, since it was already called above. - ctx := &delete.Context{ - Backup: backup, - BackupReader: backupFile, - Actions: actions, - Log: c.logger, - DiscoveryHelper: c.helper, - Filesystem: filesystem.NewFileSystem(), - } + if len(actions) > 0 { + // Download the tarball + backupFile, err := downloadToTempFile(backup.Name, backupStore, log) + defer closeAndRemoveFile(backupFile, c.logger) - // Optimization: wrap in a gofunc? Would be useful for large backups with lots of objects. - // but what do we do with the error returned? We can't just swallow it as that may lead to dangling resources. - err = delete.InvokeDeleteActions(ctx) - if err != nil { - return errors.Wrap(err, "error invoking delete item actions") + if err != nil { + log.WithError(err).Errorf("Unable to download tarball for backup %s, skipping associated DeleteItemAction plugins", backup.Name) + } else { + ctx := &delete.Context{ + Backup: backup, + BackupReader: backupFile, + Actions: actions, + Log: c.logger, + DiscoveryHelper: c.helper, + Filesystem: filesystem.NewFileSystem(), + } + + // Optimization: wrap in a gofunc? Would be useful for large backups with lots of objects. + // but what do we do with the error returned? We can't just swallow it as that may lead to dangling resources. + err = delete.InvokeDeleteActions(ctx) + if err != nil { + return errors.Wrap(err, "error invoking delete item actions") + } + } } if backupStore != nil { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/controller/backup_deletion_controller_test.go new/velero-1.5.3/pkg/controller/backup_deletion_controller_test.go --- old/velero-1.5.2/pkg/controller/backup_deletion_controller_test.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/controller/backup_deletion_controller_test.go 2021-01-14 16:13:14.000000000 +0100 @@ -49,6 +49,8 @@ persistencemocks "github.com/vmware-tanzu/velero/pkg/persistence/mocks" "github.com/vmware-tanzu/velero/pkg/plugin/clientmgmt" pluginmocks "github.com/vmware-tanzu/velero/pkg/plugin/mocks" + "github.com/vmware-tanzu/velero/pkg/plugin/velero" + "github.com/vmware-tanzu/velero/pkg/plugin/velero/mocks" velerotest "github.com/vmware-tanzu/velero/pkg/test" "github.com/vmware-tanzu/velero/pkg/volume" ) @@ -717,6 +719,265 @@ ), core.NewDeleteAction( velerov1api.SchemeGroupVersion.WithResource("backups"), + td.req.Namespace, + td.req.Spec.BackupName, + ), + core.NewPatchAction( + velerov1api.SchemeGroupVersion.WithResource("deletebackuprequests"), + td.req.Namespace, + td.req.Name, + types.MergePatchType, + []byte(`{"status":{"phase":"Processed"}}`), + ), + core.NewDeleteCollectionAction( + velerov1api.SchemeGroupVersion.WithResource("deletebackuprequests"), + td.req.Namespace, + pkgbackup.NewDeleteBackupRequestListOptions(td.req.Spec.BackupName, "uid"), + ), + } + + velerotest.CompareActions(t, expectedActions, td.client.Actions()) + + // Make sure snapshot was deleted + assert.Equal(t, 0, td.volumeSnapshotter.SnapshotsTaken.Len()) + }) + + t.Run("backup is not downloaded when there are no DeleteItemAction plugins", func(t *testing.T) { + backup := builder.ForBackup(velerov1api.DefaultNamespace, "foo").Result() + backup.UID = "uid" + backup.Spec.StorageLocation = "primary" + + td := setupBackupDeletionControllerTest(t, backup) + + location := &velerov1api.BackupStorageLocation{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: backup.Namespace, + Name: backup.Spec.StorageLocation, + }, + Spec: velerov1api.BackupStorageLocationSpec{ + Provider: "objStoreProvider", + StorageType: velerov1api.StorageType{ + ObjectStorage: &velerov1api.ObjectStorageLocation{ + Bucket: "bucket", + }, + }, + }, + } + + require.NoError(t, td.fakeClient.Create(context.Background(), location)) + + snapshotLocation := &velerov1api.VolumeSnapshotLocation{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: backup.Namespace, + Name: "vsl-1", + }, + Spec: velerov1api.VolumeSnapshotLocationSpec{ + Provider: "provider-1", + }, + } + require.NoError(t, td.sharedInformers.Velero().V1().VolumeSnapshotLocations().Informer().GetStore().Add(snapshotLocation)) + + // Clear out req labels to make sure the controller adds them and does not + // panic when encountering a nil Labels map + // (https://github.com/vmware-tanzu/velero/issues/1546) + td.req.Labels = nil + + td.client.PrependReactor("get", "backups", func(action core.Action) (bool, runtime.Object, error) { + return true, backup, nil + }) + td.volumeSnapshotter.SnapshotsTaken.Insert("snap-1") + + td.client.PrependReactor("patch", "deletebackuprequests", func(action core.Action) (bool, runtime.Object, error) { + return true, td.req, nil + }) + + td.client.PrependReactor("patch", "backups", func(action core.Action) (bool, runtime.Object, error) { + return true, backup, nil + }) + + snapshots := []*volume.Snapshot{ + { + Spec: volume.SnapshotSpec{ + Location: "vsl-1", + }, + Status: volume.SnapshotStatus{ + ProviderSnapshotID: "snap-1", + }, + }, + } + + pluginManager := &pluginmocks.Manager{} + pluginManager.On("GetVolumeSnapshotter", "provider-1").Return(td.volumeSnapshotter, nil) + pluginManager.On("GetDeleteItemActions").Return([]velero.DeleteItemAction{}, nil) + pluginManager.On("CleanupClients") + td.controller.newPluginManager = func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager } + + td.backupStore.On("GetBackupVolumeSnapshots", td.req.Spec.BackupName).Return(snapshots, nil) + td.backupStore.On("DeleteBackup", td.req.Spec.BackupName).Return(nil) + + err := td.controller.processRequest(td.req) + require.NoError(t, err) + + td.backupStore.AssertNotCalled(t, "GetBackupContents", td.req.Spec.BackupName) + + expectedActions := []core.Action{ + core.NewPatchAction( + velerov1api.SchemeGroupVersion.WithResource("deletebackuprequests"), + td.req.Namespace, + td.req.Name, + types.MergePatchType, + []byte(`{"metadata":{"labels":{"velero.io/backup-name":"foo"}},"status":{"phase":"InProgress"}}`), + ), + core.NewGetAction( + velerov1api.SchemeGroupVersion.WithResource("backups"), + td.req.Namespace, + td.req.Spec.BackupName, + ), + core.NewPatchAction( + velerov1api.SchemeGroupVersion.WithResource("deletebackuprequests"), + td.req.Namespace, + td.req.Name, + types.MergePatchType, + []byte(`{"metadata":{"labels":{"velero.io/backup-uid":"uid"}}}`), + ), + core.NewPatchAction( + velerov1api.SchemeGroupVersion.WithResource("backups"), + td.req.Namespace, + td.req.Spec.BackupName, + types.MergePatchType, + []byte(`{"status":{"phase":"Deleting"}}`), + ), + core.NewDeleteAction( + velerov1api.SchemeGroupVersion.WithResource("backups"), + td.req.Namespace, + td.req.Spec.BackupName, + ), + core.NewPatchAction( + velerov1api.SchemeGroupVersion.WithResource("deletebackuprequests"), + td.req.Namespace, + td.req.Name, + types.MergePatchType, + []byte(`{"status":{"phase":"Processed"}}`), + ), + core.NewDeleteCollectionAction( + velerov1api.SchemeGroupVersion.WithResource("deletebackuprequests"), + td.req.Namespace, + pkgbackup.NewDeleteBackupRequestListOptions(td.req.Spec.BackupName, "uid"), + ), + } + + velerotest.CompareActions(t, expectedActions, td.client.Actions()) + + // Make sure snapshot was deleted + assert.Equal(t, 0, td.volumeSnapshotter.SnapshotsTaken.Len()) + }) + + t.Run("backup is still deleted if downloading tarball fails for DeleteItemAction plugins", func(t *testing.T) { + backup := builder.ForBackup(velerov1api.DefaultNamespace, "foo").Result() + backup.UID = "uid" + backup.Spec.StorageLocation = "primary" + + td := setupBackupDeletionControllerTest(t, backup) + + location := &velerov1api.BackupStorageLocation{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: backup.Namespace, + Name: backup.Spec.StorageLocation, + }, + Spec: velerov1api.BackupStorageLocationSpec{ + Provider: "objStoreProvider", + StorageType: velerov1api.StorageType{ + ObjectStorage: &velerov1api.ObjectStorageLocation{ + Bucket: "bucket", + }, + }, + }, + } + + require.NoError(t, td.fakeClient.Create(context.Background(), location)) + + snapshotLocation := &velerov1api.VolumeSnapshotLocation{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: backup.Namespace, + Name: "vsl-1", + }, + Spec: velerov1api.VolumeSnapshotLocationSpec{ + Provider: "provider-1", + }, + } + require.NoError(t, td.sharedInformers.Velero().V1().VolumeSnapshotLocations().Informer().GetStore().Add(snapshotLocation)) + + // Clear out req labels to make sure the controller adds them and does not + // panic when encountering a nil Labels map + // (https://github.com/vmware-tanzu/velero/issues/1546) + td.req.Labels = nil + + td.client.PrependReactor("get", "backups", func(action core.Action) (bool, runtime.Object, error) { + return true, backup, nil + }) + td.volumeSnapshotter.SnapshotsTaken.Insert("snap-1") + + td.client.PrependReactor("patch", "deletebackuprequests", func(action core.Action) (bool, runtime.Object, error) { + return true, td.req, nil + }) + + td.client.PrependReactor("patch", "backups", func(action core.Action) (bool, runtime.Object, error) { + return true, backup, nil + }) + + snapshots := []*volume.Snapshot{ + { + Spec: volume.SnapshotSpec{ + Location: "vsl-1", + }, + Status: volume.SnapshotStatus{ + ProviderSnapshotID: "snap-1", + }, + }, + } + + pluginManager := &pluginmocks.Manager{} + pluginManager.On("GetVolumeSnapshotter", "provider-1").Return(td.volumeSnapshotter, nil) + pluginManager.On("GetDeleteItemActions").Return([]velero.DeleteItemAction{new(mocks.DeleteItemAction)}, nil) + pluginManager.On("CleanupClients") + td.controller.newPluginManager = func(logrus.FieldLogger) clientmgmt.Manager { return pluginManager } + + td.backupStore.On("GetBackupVolumeSnapshots", td.req.Spec.BackupName).Return(snapshots, nil) + td.backupStore.On("GetBackupContents", td.req.Spec.BackupName).Return(nil, fmt.Errorf("error downloading tarball")) + td.backupStore.On("DeleteBackup", td.req.Spec.BackupName).Return(nil) + + err := td.controller.processRequest(td.req) + require.NoError(t, err) + + expectedActions := []core.Action{ + core.NewPatchAction( + velerov1api.SchemeGroupVersion.WithResource("deletebackuprequests"), + td.req.Namespace, + td.req.Name, + types.MergePatchType, + []byte(`{"metadata":{"labels":{"velero.io/backup-name":"foo"}},"status":{"phase":"InProgress"}}`), + ), + core.NewGetAction( + velerov1api.SchemeGroupVersion.WithResource("backups"), + td.req.Namespace, + td.req.Spec.BackupName, + ), + core.NewPatchAction( + velerov1api.SchemeGroupVersion.WithResource("deletebackuprequests"), + td.req.Namespace, + td.req.Name, + types.MergePatchType, + []byte(`{"metadata":{"labels":{"velero.io/backup-uid":"uid"}}}`), + ), + core.NewPatchAction( + velerov1api.SchemeGroupVersion.WithResource("backups"), + td.req.Namespace, + td.req.Spec.BackupName, + types.MergePatchType, + []byte(`{"status":{"phase":"Deleting"}}`), + ), + core.NewDeleteAction( + velerov1api.SchemeGroupVersion.WithResource("backups"), td.req.Namespace, td.req.Spec.BackupName, ), diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/controller/backupstoragelocation_controller_test.go new/velero-1.5.3/pkg/controller/backupstoragelocation_controller_test.go --- old/velero-1.5.2/pkg/controller/backupstoragelocation_controller_test.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/controller/backupstoragelocation_controller_test.go 2021-01-14 16:13:14.000000000 +0100 @@ -120,13 +120,13 @@ wantErr bool }{ { - backupLocation: builder.ForBackupStorageLocation("ns-1", "location-1").ValidationFrequency(0).Result(), + backupLocation: builder.ForBackupStorageLocation("ns-1", "location-1").ValidationFrequency(0).LastValidationTime(time.Now()).Result(), isValidError: nil, expectedPhase: "", wantErr: false, }, { - backupLocation: builder.ForBackupStorageLocation("ns-1", "location-2").ValidationFrequency(0).Result(), + backupLocation: builder.ForBackupStorageLocation("ns-1", "location-2").ValidationFrequency(0).LastValidationTime(time.Now()).Result(), isValidError: nil, expectedPhase: "", wantErr: false, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/controller/pod_volume_restore_controller.go new/velero-1.5.3/pkg/controller/pod_volume_restore_controller.go --- old/velero-1.5.2/pkg/controller/pod_volume_restore_controller.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/controller/pod_volume_restore_controller.go 2021-01-14 16:13:14.000000000 +0100 @@ -1,5 +1,5 @@ /* -Copyright 2018 the Velero contributors. +Copyright 2020 the Velero contributors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -155,6 +155,12 @@ return } + resticInitContainerIndex := getResticInitContainerIndex(pod) + if resticInitContainerIndex > 0 { + log.Warnf(`Init containers before the %s container may cause issues + if they interfere with volumes being restored: %s index %d`, restic.InitContainer, restic.InitContainer, resticInitContainerIndex) + } + log.Debug("Enqueueing") c.enqueue(obj) } @@ -174,6 +180,12 @@ return } + resticInitContainerIndex := getResticInitContainerIndex(pod) + if resticInitContainerIndex > 0 { + log.Warnf(`Init containers before the %s container may cause issues + if they interfere with volumes being restored: %s index %d`, restic.InitContainer, restic.InitContainer, resticInitContainerIndex) + } + selector := labels.Set(map[string]string{ velerov1api.PodUIDLabel: string(pod.UID), }).AsSelector() @@ -208,18 +220,21 @@ } func isResticInitContainerRunning(pod *corev1api.Pod) bool { - // no init containers, or the first one is not the velero restic one: return false - if len(pod.Spec.InitContainers) == 0 || pod.Spec.InitContainers[0].Name != restic.InitContainer { - return false - } - // status hasn't been created yet, or the first one is not yet running: return false - if len(pod.Status.InitContainerStatuses) == 0 || pod.Status.InitContainerStatuses[0].State.Running == nil { - return false + // Restic wait container can be anywhere in the list of init containers, but must be running. + i := getResticInitContainerIndex(pod) + return i >= 0 && pod.Status.InitContainerStatuses[i].State.Running != nil +} + +func getResticInitContainerIndex(pod *corev1api.Pod) int { + // Restic wait container can be anywhere in the list of init containers so locate it. + for i, initContainer := range pod.Spec.InitContainers { + if initContainer.Name == restic.InitContainer { + return i + } } - // else, it's running - return true + return -1 } func (c *podVolumeRestoreController) processQueueItem(key string) error { diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/controller/pod_volume_restore_controller_test.go new/velero-1.5.3/pkg/controller/pod_volume_restore_controller_test.go --- old/velero-1.5.2/pkg/controller/pod_volume_restore_controller_test.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/controller/pod_volume_restore_controller_test.go 2021-01-14 16:13:14.000000000 +0100 @@ -1,5 +1,5 @@ /* -Copyright 2018 the Velero contributors. +Copyright 2020 the Velero contributors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. @@ -491,7 +491,7 @@ expected: false, }, { - name: "pod with running restic init container that's not first should return false", + name: "pod with running restic init container that's not first should still work", pod: &corev1api.Pod{ ObjectMeta: metav1.ObjectMeta{ Namespace: "ns-1", @@ -522,7 +522,7 @@ }, }, }, - expected: false, + expected: true, }, { name: "pod with restic init container as first initContainer that's not running should return false", @@ -598,3 +598,105 @@ }) } } + +func TestGetResticInitContainerIndex(t *testing.T) { + tests := []struct { + name string + pod *corev1api.Pod + expected int + }{ + { + name: "init container is not present return -1", + pod: &corev1api.Pod{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-1", + Name: "pod-1", + }, + }, + expected: -1, + }, + { + name: "pod with no restic init container return -1", + pod: &corev1api.Pod{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-1", + Name: "pod-1", + }, + Spec: corev1api.PodSpec{ + InitContainers: []corev1api.Container{ + { + Name: "non-restic-init", + }, + }, + }, + }, + expected: -1, + }, + { + name: "pod with restic container as second initContainern should return 1", + pod: &corev1api.Pod{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-1", + Name: "pod-1", + }, + Spec: corev1api.PodSpec{ + InitContainers: []corev1api.Container{ + { + Name: "non-restic-init", + }, + { + Name: restic.InitContainer, + }, + }, + }, + }, + expected: 1, + }, + { + name: "pod with restic init container as first initContainer should return 0", + pod: &corev1api.Pod{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-1", + Name: "pod-1", + }, + Spec: corev1api.PodSpec{ + InitContainers: []corev1api.Container{ + { + Name: restic.InitContainer, + }, + { + Name: "non-restic-init", + }, + }, + }, + }, + expected: 0, + }, + { + name: "pod with restic init container as first initContainer should return 0", + pod: &corev1api.Pod{ + ObjectMeta: metav1.ObjectMeta{ + Namespace: "ns-1", + Name: "pod-1", + }, + Spec: corev1api.PodSpec{ + InitContainers: []corev1api.Container{ + { + Name: restic.InitContainer, + }, + { + Name: "non-restic-init", + }, + }, + }, + }, + expected: 0, + }, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + assert.Equal(t, test.expected, getResticInitContainerIndex(test.pod)) + }) + } +} diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/install/resources.go new/velero-1.5.3/pkg/install/resources.go --- old/velero-1.5.2/pkg/install/resources.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/install/resources.go 2021-01-14 16:13:14.000000000 +0100 @@ -46,7 +46,7 @@ DefaultVeleroPodCPURequest = "500m" DefaultVeleroPodMemRequest = "128Mi" DefaultVeleroPodCPULimit = "1000m" - DefaultVeleroPodMemLimit = "256Mi" + DefaultVeleroPodMemLimit = "512Mi" DefaultResticPodCPURequest = "500m" DefaultResticPodMemRequest = "512Mi" DefaultResticPodCPULimit = "1000m" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/restic/common.go new/velero-1.5.3/pkg/restic/common.go --- old/velero-1.5.2/pkg/restic/common.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/restic/common.go 2021-01-14 16:13:14.000000000 +0100 @@ -95,13 +95,17 @@ return res } +func isPVBMatchPod(pvb *velerov1api.PodVolumeBackup, pod metav1.Object) bool { + return pod.GetName() == pvb.Spec.Pod.Name && pod.GetNamespace() == pvb.Spec.Pod.Namespace +} + // GetVolumeBackupsForPod returns a map, of volume name -> snapshot id, // of the PodVolumeBackups that exist for the provided pod. func GetVolumeBackupsForPod(podVolumeBackups []*velerov1api.PodVolumeBackup, pod metav1.Object) map[string]string { volumes := make(map[string]string) for _, pvb := range podVolumeBackups { - if pod.GetName() != pvb.Spec.Pod.Name { + if !isPVBMatchPod(pvb, pod) { continue } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/restic/common_test.go new/velero-1.5.3/pkg/restic/common_test.go --- old/velero-1.5.2/pkg/restic/common_test.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/restic/common_test.go 2021-01-14 16:13:14.000000000 +0100 @@ -564,6 +564,88 @@ } } +func TestIsPVBMatchPod(t *testing.T) { + testCases := []struct { + name string + pod metav1.Object + pvb velerov1api.PodVolumeBackup + expected bool + }{ + { + name: "should match PVB and pod", + pod: &metav1.ObjectMeta{ + Name: "matching-pod", + Namespace: "matching-namespace", + }, + pvb: velerov1api.PodVolumeBackup{ + Spec: velerov1api.PodVolumeBackupSpec{ + Pod: corev1api.ObjectReference{ + Name: "matching-pod", + Namespace: "matching-namespace", + }, + }, + }, + expected: true, + }, + { + name: "should not match PVB and pod, pod name mismatch", + pod: &metav1.ObjectMeta{ + Name: "not-matching-pod", + Namespace: "matching-namespace", + }, + pvb: velerov1api.PodVolumeBackup{ + Spec: velerov1api.PodVolumeBackupSpec{ + Pod: corev1api.ObjectReference{ + Name: "matching-pod", + Namespace: "matching-namespace", + }, + }, + }, + expected: false, + }, + { + name: "should not match PVB and pod, pod namespace mismatch", + pod: &metav1.ObjectMeta{ + Name: "matching-pod", + Namespace: "not-matching-namespace", + }, + pvb: velerov1api.PodVolumeBackup{ + Spec: velerov1api.PodVolumeBackupSpec{ + Pod: corev1api.ObjectReference{ + Name: "matching-pod", + Namespace: "matching-namespace", + }, + }, + }, + expected: false, + }, + { + name: "should not match PVB and pod, pod name and namespace mismatch", + pod: &metav1.ObjectMeta{ + Name: "not-matching-pod", + Namespace: "not-matching-namespace", + }, + pvb: velerov1api.PodVolumeBackup{ + Spec: velerov1api.PodVolumeBackupSpec{ + Pod: corev1api.ObjectReference{ + Name: "matching-pod", + Namespace: "matching-namespace", + }, + }, + }, + expected: false, + }, + } + + for _, tc := range testCases { + t.Run(tc.name, func(t *testing.T) { + actual := isPVBMatchPod(&tc.pvb, tc.pod) + assert.Equal(t, tc.expected, actual) + }) + + } +} + func newFakeClient(t *testing.T, initObjs ...runtime.Object) client.Client { err := velerov1api.AddToScheme(scheme.Scheme) require.NoError(t, err) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/restore/restic_restore_action_test.go new/velero-1.5.3/pkg/restore/restic_restore_action_test.go --- old/velero-1.5.2/pkg/restore/restic_restore_action_test.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/restore/restic_restore_action_test.go 2021-01-14 16:13:14.000000000 +0100 @@ -173,12 +173,14 @@ podVolumeBackups: []*velerov1api.PodVolumeBackup{ builder.ForPodVolumeBackup(veleroNs, "pvb-1"). PodName("my-pod"). + PodNamespace("ns-1"). Volume("vol-1"). ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, backupName)). SnapshotID("foo"). Result(), builder.ForPodVolumeBackup(veleroNs, "pvb-2"). PodName("my-pod"). + PodNamespace("ns-1"). Volume("vol-2"). ObjectMeta(builder.WithLabels(velerov1api.BackupNameLabel, backupName)). SnapshotID("foo"). diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/restore/restore_test.go new/velero-1.5.3/pkg/restore/restore_test.go --- old/velero-1.5.2/pkg/restore/restore_test.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/restore/restore_test.go 2021-01-14 16:13:14.000000000 +0100 @@ -2435,14 +2435,14 @@ want map[*test.APIResource][]string }{ { - name: "a pod that exists in given backup and contains associated PVBs should have should have RestorePodVolumes called", + name: "a pod that exists in given backup and contains associated PVBs should have RestorePodVolumes called", restore: defaultRestore().Result(), backup: defaultBackup().Result(), apiResources: []*test.APIResource{test.Pods()}, podVolumeBackups: []*velerov1api.PodVolumeBackup{ builder.ForPodVolumeBackup("velero", "pvb-1").PodName("pod-1").SnapshotID("foo").Result(), - builder.ForPodVolumeBackup("velero", "pvb-2").PodName("pod-2").SnapshotID("foo").Result(), - builder.ForPodVolumeBackup("velero", "pvb-3").PodName("pod-4").SnapshotID("foo").Result(), + builder.ForPodVolumeBackup("velero", "pvb-2").PodName("pod-2").PodNamespace("ns-1").SnapshotID("foo").Result(), + builder.ForPodVolumeBackup("velero", "pvb-3").PodName("pod-4").PodNamespace("ns-2").SnapshotID("foo").Result(), }, podWithPVBs: []*corev1api.Pod{ builder.ForPod("ns-1", "pod-2"). diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/pkg/util/collections/includes_excludes.go new/velero-1.5.3/pkg/util/collections/includes_excludes.go --- old/velero-1.5.2/pkg/util/collections/includes_excludes.go 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/pkg/util/collections/includes_excludes.go 2021-01-14 16:13:14.000000000 +0100 @@ -201,7 +201,10 @@ func(item string) string { gvr, _, err := helper.ResourceFor(schema.ParseGroupResource(item).WithVersion("")) if err != nil { - return "" + // If we can't resolve it, return it as-is. This prevents the generated + // includes-excludes list from including *everything*, if none of the includes + // can be resolved. ref. https://github.com/vmware-tanzu/velero/issues/2461 + return item } gr := gvr.GroupResource() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/site/content/docs/main/customize-installation.md new/velero-1.5.3/site/content/docs/main/customize-installation.md --- old/velero-1.5.2/site/content/docs/main/customize-installation.md 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/site/content/docs/main/customize-installation.md 2021-01-14 16:13:14.000000000 +0100 @@ -80,7 +80,7 @@ |CPU request|500m|500m| |Memory requests|128Mi|512Mi| |CPU limit|1000m (1 CPU)|1000m (1 CPU)| -|Memory limit|256Mi|1024Mi| +|Memory limit|512Mi|1024Mi| {{< /table >}} ### Install with custom resource requests and limits @@ -111,7 +111,7 @@ ```bash kubectl patch deployment velero -n velero --patch \ -'{"spec":{"template":{"spec":{"containers":[{"name": "velero", "resources": {"limits":{"cpu": "1", "memory": "256Mi"}, "requests": {"cpu": "1", "memory": "128Mi"}}}]}}}}' +'{"spec":{"template":{"spec":{"containers":[{"name": "velero", "resources": {"limits":{"cpu": "1", "memory": "512Mi"}, "requests": {"cpu": "1", "memory": "128Mi"}}}]}}}}' ``` **restic pod** diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/velero-1.5.2/site/content/docs/v1.5/customize-installation.md new/velero-1.5.3/site/content/docs/v1.5/customize-installation.md --- old/velero-1.5.2/site/content/docs/v1.5/customize-installation.md 2020-10-20 19:37:45.000000000 +0200 +++ new/velero-1.5.3/site/content/docs/v1.5/customize-installation.md 2021-01-14 16:13:14.000000000 +0100 @@ -80,7 +80,7 @@ |CPU request|500m|500m| |Memory requests|128Mi|512Mi| |CPU limit|1000m (1 CPU)|1000m (1 CPU)| -|Memory limit|256Mi|1024Mi| +|Memory limit|512Mi|1024Mi| {{< /table >}} ### Install with custom resource requests and limits @@ -111,7 +111,7 @@ ```bash kubectl patch deployment velero -n velero --patch \ -'{"spec":{"template":{"spec":{"containers":[{"name": "velero", "resources": {"limits":{"cpu": "1", "memory": "256Mi"}, "requests": {"cpu": "1", "memory": "128Mi"}}}]}}}}' +'{"spec":{"template":{"spec":{"containers":[{"name": "velero", "resources": {"limits":{"cpu": "1", "memory": "512Mi"}, "requests": {"cpu": "1", "memory": "128Mi"}}}]}}}}' ``` **restic pod** ++++++ vendor.tar.gz ++++++ /work/SRC/openSUSE:Factory/velero/vendor.tar.gz /work/SRC/openSUSE:Factory/.velero.new.28504/vendor.tar.gz differ: char 5, line 1
