Re: [PATCH 03/16] refs: add methods for the ref iterators
On Sat, Jan 2, 2016 at 4:06 PM, David Aguilar <dav...@gmail.com> wrote:
> Apologies for the late review, and this review should probably
> go on patch 01 or 02 but I don't have it in my mbox atm...
>
> On Wed, Dec 02, 2015 at 07:35:08PM -0500, David Turner wrote:
>> From: Ronnie Sahlberg <sahlb...@google.com>
>>
>> Signed-off-by: Ronnie Sahlberg <sahlb...@google.com>
>> Signed-off-by: David Turner <dtur...@twopensource.com>
>> ---
>> refs.c | 54
>>
>> refs/files-backend.c | 41 +++
>> refs/refs-internal.h | 29
>> 3 files changed, 112 insertions(+), 12 deletions(-)
>>
>> diff --git a/refs.c b/refs.c
>> index 9562325..b9b0244 100644
>> --- a/refs.c
>> +++ b/refs.c
>> @@ -1150,3 +1150,57 @@ int resolve_gitlink_ref(const char *path, const char
>> *refname,
>> {
>> return the_refs_backend->resolve_gitlink_ref(path, refname, sha1);
>> }
>> +
>> +int head_ref(each_ref_fn fn, void *cb_data)
>> +{
>> + return the_refs_backend->head_ref(fn, cb_data);
>> +}
>
> My only comment is that it seems like having a single static global
> the_refs_backend seems like it should be avoided.
>
> It seems like the intention was to keep the existing interface
> as-is, which explains why this is using globals, but it seems
> like the refs backend should be part of some "application
> context" struct on the stack or allocated during main(). It can
> then be passed around in the API so that we do not need to have
> a global.
Not commenting on whether this is the right direction or not. A global
variable holding a methods table might not be most aesthetic design,
but there are practicalities.
However, that kind of change would change the function signatures for
all public refs functions and probably most private refs functions as
well and will likely have massive conflicts with almost any other
patch that touches the refs code.
If doing this API change is desired it is probably best to do that as
a separate patch later that ONLY does this signature change and
nothing else to make review easier and to possibly make merge conflict
changes easier to manage.
>
> That way the code will not be tied to a specific single
> the_refs_backend and could in theory have multiple backends
> working at the same time. If submodule were ever rewritten in C
> then this could potentially be a useful feature.
>
> That said, the refs backend is not the only global static data
> in git that would need to be factored out, but it wouldn't hurt
>ice, to make this part a little more tidy.
>
> Thoughts?
> --
> David
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/8] Making reflog modifications part of the transactions API
On Fri, Dec 12, 2014 at 11:17 AM, Michael Haggerty mhag...@alum.mit.edu wrote: On 12/06/2014 03:46 AM, Stefan Beller wrote: This goes on top of Michaels series. The idea of this series is make the reflogs being part of the transaction API, so it will be part of the contract of transaction_commit to either commit all the changes or none at all. Currently when using the transaction API to change refs, also reflogs are changed. But the changes to the reflogs just happen as a side effect and not as part of the atomic part of changes we want to commit altogether. Would you please explain why this patch series is still needed if my reflog_expire() series is accepted? reflog_expire() already has its own little transaction. Reflog expiration never needs to be combined with other reference changes, so there is no need for reflog expiration to be done via a ref_transaction. ref_transaction_commit() already updates the reflog if necessary when a reference is updated. That update is part of the containing ref_transaction, because it is locked by the lock on the reference itself at the beginning of the transaction and unlocked at the end of the transaction [1]. It can't fail in normal circumstances because the preconditions for the transaction have already been checked. As far as I can tell, the only thing left to do is provide a substitute for log_ref_setup() a.k.a. create_reflog() that can be triggered within a transaction. In my opinion the easiest way to do that is to give ref_transaction_update()'s flag parameter an additional option, REF_CREATE_REFLOG, which forces the reference's reflog to be created if it does not already exist. What am I missing? My original idea was to clean up a bit of the reflog handling API and have one single transaction API for all ref and reflog operations. Think future use cases where you have a database backend for both refs and reflogs. It would be very nice to have a single atomic transaction that would either commit or fail atomically any update to refs and/or reflogs. Otherwise you would have all consumers of the API have to invent their own transaction and rewind support : 'oh the ref transaction failed and I have already done the reflog commit, have to manually uncommit And this quickly becomes quite burdensome for consumers. I think a transaction API should remove this burden from consumers and make it as easy as possible to use the API. Conditional of if it is desireable to have transactions for reflogs at all. About the cleanup part. The current API, and I think also the current direction of both my old patches (which I think did not go far enough in the transactifications) or this current patchseries is that they all have a very confusing and inconsistent API for reflog updates. With this I mean, sometimes reflog updates happen within a transaction as a side effect of a ref_transaction_update(). Other times reflog updates happens ooutside of transactions by calling a special reflog API. I.e. reflogs sometimes update as part of a transaction and sometimes not. A follow up question then on this API is what should happen if you have a transaction open, but not committed, and while the transaction is open you call the non-transactional reflog API for a reflog for the same ref that is already beeing/or going to be/ updated as the ref-update-side-effect ? I think an api where sometimes you operate on foo from within a transaction and sometimes you do not, and if you do the latter when a transaction is open, it is unclear what should happen is not great. IMHO, refs and reflog updates are related and I think: * a transaction should be the ONLY way to mutate either a ref or a reflog or both. * if you update both a ref and a reflog then that should happen as part of one single transaction. * (later) it would probably make the API better if the code was refactored so write_ref_sha1() will NOT call log_ref_write() anymore and instead make the reflog update that happens explicit perhaps by calling something like a ref_transaction_update_reflog() as part of the ref_transaction_update() call. Michael [1] The reflog updates are not atomic in the face of disk errors and power outages and stuff, but neither are reference updates. In other words, I think reflog updates in ref_transaction_commit() are done as safely as reference updates, which is surely good enough for this reference backend. Other reference backends can do a better job with both while staying within the existing API. -- Michael Haggerty mhag...@alum.mit.edu -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 07/23] expire_reflog(): use a lock_file for rewriting the reflog file
On Thu, Dec 4, 2014 at 3:08 PM, Michael Haggerty mhag...@alum.mit.edu wrote:
We don't actually need the locking functionality, because we already
hold the lock on the reference itself,
No. You do need the lock.
The ref is locked only during transaction_commit()
If you don't want to lock the reflog file and instead rely on the lock
on the ref itself you will need to
rework your patches so that the lock on the ref is taken already
during, for example, transaction_update_ref() instead.
But without doing those changes and moving the ref locking from
_commit() to _update_ref() you will risk reflog corruption/surprises
if two operations collide and both rewrite the reflog without any lock held.
which is how the reflog file is
locked. But the lock_file code still does some of the bookkeeping for
us and is more careful than the old code here was.
For example:
* Correctly handle the case that the reflog lock file already exists
for some reason or cannot be opened.
* Correctly clean up the lockfile if the program dies.
Signed-off-by: Michael Haggerty mhag...@alum.mit.edu
---
builtin/reflog.c | 37 ++---
1 file changed, 22 insertions(+), 15 deletions(-)
diff --git a/builtin/reflog.c b/builtin/reflog.c
index a282e60..d344d45 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -349,12 +349,14 @@ static int push_tip_to_list(const char *refname, const
unsigned char *sha1, int
return 0;
}
+static struct lock_file reflog_lock;
+
static int expire_reflog(const char *refname, const unsigned char *sha1,
void *cb_data)
{
struct cmd_reflog_expire_cb *cmd = cb_data;
struct expire_reflog_cb cb;
struct ref_lock *lock;
- char *log_file, *newlog_path = NULL;
+ char *log_file;
struct commit *tip_commit;
struct commit_list *tips;
int status = 0;
@@ -372,10 +374,14 @@ static int expire_reflog(const char *refname, const
unsigned char *sha1, void *c
unlock_ref(lock);
return 0;
}
+
log_file = git_pathdup(logs/%s, refname);
if (!cmd-dry_run) {
- newlog_path = git_pathdup(logs/%s.lock, refname);
- cb.newlog = fopen(newlog_path, w);
+ if (hold_lock_file_for_update(reflog_lock, log_file, 0) 0)
+ goto failure;
+ cb.newlog = fdopen_lock_file(reflog_lock, w);
+ if (!cb.newlog)
+ goto failure;
}
cb.cmd = cmd;
@@ -423,10 +429,9 @@ static int expire_reflog(const char *refname, const
unsigned char *sha1, void *c
}
if (cb.newlog) {
- if (fclose(cb.newlog)) {
- status |= error(%s: %s, strerror(errno),
- newlog_path);
- unlink(newlog_path);
+ if (close_lock_file(reflog_lock)) {
+ status |= error(Couldn't write %s: %s, log_file,
+ strerror(errno));
} else if (cmd-updateref
(write_in_full(lock-lock_fd,
sha1_to_hex(cb.last_kept_sha1), 40) != 40 ||
@@ -434,21 +439,23 @@ static int expire_reflog(const char *refname, const
unsigned char *sha1, void *c
close_ref(lock) 0)) {
status |= error(Couldn't write %s,
lock-lk-filename.buf);
- unlink(newlog_path);
- } else if (rename(newlog_path, log_file)) {
- status |= error(cannot rename %s to %s,
- newlog_path, log_file);
- unlink(newlog_path);
+ rollback_lock_file(reflog_lock);
+ } else if (commit_lock_file(reflog_lock)) {
+ status |= error(cannot rename %s.lock to %s,
+ log_file, log_file);
} else if (cmd-updateref commit_ref(lock)) {
status |= error(Couldn't set %s, lock-ref_name);
- } else {
- adjust_shared_perm(log_file);
}
}
- free(newlog_path);
free(log_file);
unlock_ref(lock);
return status;
+
+ failure:
+ rollback_lock_file(reflog_lock);
+ free(log_file);
+ unlock_ref(lock);
+ return -1;
}
static int collect_reflog(const char *ref, const unsigned char *sha1, int
unused, void *cb_data)
--
2.1.3
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] refs.c: repack_without_refs may be called without error string buffer
On Thu, Nov 20, 2014 at 10:10 AM, Stefan Beller sbel...@google.com wrote:
If we don't pass in the error string buffer, we skip over all
parts dealing with preparing error messages.
Signed-off-by: Stefan Beller sbel...@google.com
---
This goes ontop of [PATCH v5] refs.c: use a stringlist for repack_without_refs
if that makes sense.
refs.c | 8
refs.h | 1 -
2 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/refs.c b/refs.c
index ebcd90f..3c85ea6 100644
--- a/refs.c
+++ b/refs.c
@@ -2646,8 +2646,6 @@ int repack_without_refs(struct string_list *without,
struct strbuf *err)
struct string_list_item *ref_to_delete;
int ret, needs_repacking = 0, removed = 0;
- assert(err);
-
/* Look for a packed ref */
for_each_string_list_item(ref_to_delete, without) {
if (get_packed_ref(ref_to_delete-string)) {
@@ -2661,7 +2659,9 @@ int repack_without_refs(struct string_list *without,
struct strbuf *err)
return 0;
if (lock_packed_refs(0)) {
- unable_to_lock_message(git_path(packed-refs), errno, err);
+ if (err)
+ unable_to_lock_message(git_path(packed-refs),
+ errno, err);
return -1;
}
packed = get_packed_refs(ref_cache);
@@ -2688,7 +2688,7 @@ int repack_without_refs(struct string_list *without,
struct strbuf *err)
/* Write what remains */
ret = commit_packed_refs();
- if (ret)
+ if (ret err)
strbuf_addf(err, unable to overwrite old ref-pack file: %s,
strerror(errno));
return ret;
diff --git a/refs.h b/refs.h
index c7323ff..b71fb79 100644
--- a/refs.h
+++ b/refs.h
@@ -170,7 +170,6 @@ int pack_refs(unsigned int flags);
* strbuf.
*
* The refs in 'without' may have any order.
- * The err buffer must not be omitted.
*/
extern int repack_without_refs(struct string_list *without, struct strbuf
*err);
--
2.2.0.rc2.23.gca0107e
LGTM
Reviewed-by: Ronnie Sahlberg sahlb...@google.com
Nit:
While it does not hurt to allow passing NULL, at some stage later
this function will become
private to refs.c and ONLY be called from within transaction_commit()
which will always
pass a non-NULL err argument.
At that stage we will not strictly need to allow err==NULL since all
callers are guaranteed to
always pass err!=NULL.
That said, having err being optional is probably a better API. Maybe
err should be made optional for all other functions that take
an err strbuf too so that the calling conventions become more consistent?
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] refs.c: repack_without_refs may be called without error string buffer
On Thu, Nov 20, 2014 at 10:35 AM, Jonathan Nieder jrnie...@gmail.com wrote:
Stefan Beller wrote:
If we don't pass in the error string buffer, we skip over all
parts dealing with preparing error messages.
Please no.
We tried this with the ref transaction code. When someone wants
to silence the message, it is cheap enough to do
struct strbuf ignore = STRBUF_INIT;
if (thing_that_can_fail_in_an_ignorable_way(..., ignore)) {
... handle the failure ...
}
The extra lines of code make it obvious that the error message is
being dropped, which is a very good thing. The extra work to format a
message in the error case is not so bad and can be mitigated if the
error is a common normal case by passing a flag to not consider it an
error.
Silently losing good diagnostic messages when err == NULL would have
the opposite effect: when there isn't a spare strbuf to put errors in
around, it would be tempting for people coding in a hurry to just pass
NULL, and to readers it would look at first glance like oh, an
optional paramter was not passed and we are getting the good default
behavior.
This is not a theoretical concern --- it actually happened.
Fair enough.
Un-LGTM my message above.
My two cents,
Jonathan
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3 00/14] ref-transactions-reflog
On Tue, Nov 18, 2014 at 3:26 AM, Michael Haggerty mhag...@alum.mit.edu wrote: On 11/18/2014 02:35 AM, Stefan Beller wrote: The following patch series updates the reflog handling to use transactions. This patch series has previously been sent to the list[1]. [...] I was reviewing this patch series (I left some comments in Gerrit about the first few patches) when I realized that I'm having trouble understanding the big picture of where you want to go with this. I have the feeling that the operations that you are implementing are at too low a level of abstraction. What are the elementary write operations that are needed for a reflog? Off the top of my head, 1. Add a reflog entry when a reference is updated in a transaction. 2. Rename a reflog file when the corresponding reference is renamed. 3. Delete the reflog when the corresponding reference is deleted [1]. 4. Configure a reference to be reflogged. 5. Configure a reference to not be reflogged anymore and delete any existing reflog. 6. Selectively expire old reflog entries, e.g., based on their age. Have I forgotten any? The first three should be side-effects of the corresponding reference updates. Aside from the fact that renames are not yet done within a transaction, I think this is already the case. Number 4, I think, currently only happens in conjunction with adding a line to the reflog. So it could be implemented, say, as a FORCE_CREATE_REFLOG flag on a ref_update within a transaction. Number 5 is not very interesting, I think. For example, it could be a separate API function, disconnected from any transactions. Number 6 is more interesting, and from my quick reading, it looks like a lot of the work of this patch series is to allow number 6 to be implemented in builtin/reflog.c:expire_reflog(). But it seems to me that you are building API calls at the wrong level of abstraction. Expiring a reflog should be a single API call to the refs API, and ultimately it should be left up to the refs backend to decide how to implement it. For a filesystem-based backend, it would do what it does now. But (for example) a SQL-based backend might implement this as a single SELECT statement. I agree in principle. But things are more difficult since expire_reflog() has very complex semantics. To keep things simple for the reviews at this stage the logic is the same as the original code: loop over all entries: use very complex conditionals to decide which entries to keep/remove optionally modify the sha1 values for the records we keep write records we keep back to the file one record at a time So that as far as possible, we keep the same rules and behavior but we use a different API for the actual write entry to new reflog. We could wrap this inside a new specific transaction_expire_reflog() function so that other types of backends, for example an SQL backend, could optimize, but I think that should be in a separate later patch because expire_reflog is almost impossibly complex. It will not be a simple SELECT unfortunately. The current expire logic is something like : 1, expire all entries older than timestamp 2, optionally, also expire all entries that refer to unreachable objects using a different timestamp This involves actually reading the objects that the sha1 points to and parsing them! 3, optionally, if the sha1 objects can not be referenced, they are not commit objects or if they don't exist, then expire them too. This also involves reading the objects behind the sha1. 4, optionally, delete reflog entry #foo 5, optionally, if any log entries were discarded due to 2,3,4 then we might also re-write and modify some of the reflog entries we keep. or any combination thereof (6, if --dry-run is specified, just print what we would have expired) 2 and 3 requires that we need to read the objects for the entry 4 allows us to delete a specific entry 5 means that even for entries we keep we will need to mutate them. I also don't have the feeling that reflog expiration has to be done within a ref_transaction. For example, is there ever a reason to combine expiration with other reference updates in a single atomic transaction? --updateref In expire_reflog() we not only prune the reflog. When --updateref is used we update the actual ref itself. I think we want to have both the ref update and also the reflog update both be part of a single atomic transaction. I think not. So it seems to me that it would be more practical to have a separate API function that is called to expire selected entries from a reflog [2], unconnected with any transaction. I think it makes the API cleaner if we have a 'you can only update a ref/reflog/other things added in the future/ from within a transaction.' Since we need to do reflog changes within a transaction for the expire reflog case as well as the rename ref case I think it makes sense to enforce that reflog changes must be done
Re: [PATCH 02/15] refs.c: return error instead of dying when locking fails during transaction
On Tue, Nov 11, 2014 at 2:34 AM, Jeff King p...@peff.net wrote:
On Tue, Oct 21, 2014 at 01:36:47PM -0700, Ronnie Sahlberg wrote:
commit e193c10fc4f9274d1e751cfcdcc4507818e8d498 upstream.
Change lock_ref_sha1_basic to return an error instead of dying when
we fail to lock a file during a transaction.
This function is only called from transaction_commit() and it knows how
to handle these failures.
[...]
- else
- unable_to_lock_die(ref_file, errno);
+ else {
+ struct strbuf err = STRBUF_INIT;
+ unable_to_lock_message(ref_file, errno, err);
+ error(%s, err.buf);
+ strbuf_reset(err);
+ goto error_return;
+ }
I coincidentally just wrote almost the identical patch, because this
isn't just a cleanup; it fixes a real bug. During pack_refs, we call
prune_ref to lock and delete the loose ref. If the lock fails, that's
OK; that just means somebody else is updating it at the same time, and
we can skip our pruning step. But due to the unable_to_lock_die call
here in lock_ref_sha1_basic, the pack-refs process may die prematurely.
I wonder if it is worth pulling this one out from the rest of the
series, as it has value (and can be applied) on its own. I did some
digging on the history of this, too. Here's the rationale I wrote:
lock_ref_sha1_basic: do not die on locking errors
lock_ref_sha1_basic is inconsistent about when it calls
die() and when it returns NULL to signal an error. This is
annoying to any callers that want to recover from a locking
error.
This seems to be mostly historical accident. It was added in
4bd18c4 (Improve abstraction of ref lock/write.,
2006-05-17), which returned an error in all cases except
calling safe_create_leading_directories, in which case it
died. Later, 40aaae8 (Better error message when we are
unable to lock the index file, 2006-08-12) asked
hold_lock_file_for_update to die for us, leaving the
resolve_ref code-path the only one which returned NULL.
We tried to correct that in 5cc3cef (lock_ref_sha1(): do not
sometimes error() and sometimes die()., 2006-09-30),
by converting all of the die() calls into returns. But we
missed the die flag passed to the lock code, leaving us
inconsistent. This state persisted until e5c223e
(lock_ref_sha1_basic(): if locking fails with ENOENT, retry,
2014-01-18). Because of its retry scheme, it does not ask
the lock code to die, but instead manually dies with
unable_to_lock_die().
We can make this consistent with the other return paths by
converting this to use unable_to_lock_message(), and
returning NULL. This is safe to do because all callers
already needed to check the return value of the function,
since it could fail (and return NULL) for other reasons.
I also have some other cleanups to lock_ref_sha1_basic's error handling.
I'd be happy to take over this patch and send it along with those
cleanups as a separate series.
Sounds Good To Me.
Thanks,
Ronnie Sahlberg
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 00/16] ref-transaction-rename
List, Thsi series builds on the previous series : ref-transaction-reflog as applied to next. This series has been sent to the list before but is now rebased to current git next. This series can also be found at : https://github.com/rsahlberg/git/tree/ref-transactions-rename This series converts ref rename to use a transaction. This addesses several issues in the old implementation, such as colliding renames might overwrite someone elses reflog, and it makes the rename atomic. As part of the series we also move changes that cover multiple refs to happen as an atomic transaction/rename to the pacekd refs file. This makes it possible to have both the rename case (one deleted ref + one created ref) as well as any operation that updates multiple refs to become one atomic rename() applied to the packed refs file. Thus all such changes are now also atomic to all external observers. Version 2: - Changed to not use potentially iterators to copy the reflog entries one by one. Instead adding two new functions. One to read an existing reflog as one big blob, and a second function to, in a transaction, write a new complete reflog from said blob. The idea is that each future reflog backend will provide optimized versions for these read whole reflog write whole reflog functions. Version 3: - Rename and redo the API for updating a whole reflog in one single operation to transaction_rename_reflog() Ronnie Sahlberg (16): refs.c: allow passing raw git_committer_info as email to _update_reflog refs.c: return error instead of dying when locking fails during transaction refs.c: use packed refs when deleting refs during a transaction refs.c: use a stringlist for repack_without_refs refs.c: add transaction support for renaming a reflog refs.c: update rename_ref to use a transaction refs.c: rollback the lockfile before we die() in repack_without_refs refs.c: move reflog updates into its own function refs.c: write updates to packed refs when a transaction has more than one ref remote.c: use a transaction for deleting refs refs.c: make repack_without_refs static refs.c: make the *_packed_refs functions static refs.c: replace the onerr argument in update_ref with a strbuf err refs.c: make add_packed_ref return an error instead of calling die refs.c: make lock_packed_refs take an err argument refs.c: add an err argument to pack_refs builtin/checkout.c| 7 +- builtin/clone.c | 36 ++- builtin/merge.c | 20 +- builtin/notes.c | 24 +- builtin/pack-refs.c | 8 +- builtin/reflog.c | 19 +- builtin/remote.c | 69 +++--- builtin/reset.c | 12 +- builtin/update-ref.c | 7 +- notes-cache.c | 2 +- notes-utils.c | 5 +- refs.c| 616 ++ refs.h| 79 +++ t/t3200-branch.sh | 7 - t/t5516-fetch-push.sh | 2 +- transport-helper.c| 7 +- transport.c | 9 +- 17 files changed, 552 insertions(+), 377 deletions(-) -- 2.1.0.rc2.206.gedb03e5 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 04/16] refs.c: use a stringlist for repack_without_refs
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/remote.c | 23 ---
refs.c | 42 +-
refs.h | 2 +-
3 files changed, 30 insertions(+), 37 deletions(-)
diff --git a/builtin/remote.c b/builtin/remote.c
index c25420f..6806251 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -751,7 +751,6 @@ static int mv(int argc, const char **argv)
static int remove_branches(struct string_list *branches)
{
struct strbuf err = STRBUF_INIT;
- const char **branch_names;
int i, result = 0;
if (lock_packed_refs(0)) {
@@ -763,13 +762,9 @@ static int remove_branches(struct string_list *branches)
return -1;
}
- branch_names = xmalloc(branches-nr * sizeof(*branch_names));
- for (i = 0; i branches-nr; i++)
- branch_names[i] = branches-items[i].string;
- if (repack_without_refs(branch_names, branches-nr, err))
+ if (repack_without_refs(branches, err))
result |= error(%s, err.buf);
strbuf_release(err);
- free(branch_names);
for (i = 0; i branches-nr; i++) {
struct string_list_item *item = branches-items + i;
@@ -1327,7 +1322,6 @@ static int prune_remote(const char *remote, int dry_run)
int result = 0, i;
struct ref_states states;
struct string_list delete_refs_list = STRING_LIST_INIT_NODUP;
- const char **delete_refs;
const char *dangling_msg = dry_run
? _( %s will become dangling!)
: _( %s has become dangling!);
@@ -1335,6 +1329,11 @@ static int prune_remote(const char *remote, int dry_run)
memset(states, 0, sizeof(states));
get_remote_ref_states(remote, states, GET_REF_STATES);
+ for (i = 0; i states.stale.nr; i++) {
+ string_list_insert(delete_refs_list,
+ states.stale.items[i].util);
+ }
+
if (states.stale.nr) {
printf_ln(_(Pruning %s), remote);
printf_ln(_(URL: %s),
@@ -1342,9 +1341,6 @@ static int prune_remote(const char *remote, int dry_run)
? states.remote-url[0]
: _((no URL)));
- delete_refs = xmalloc(states.stale.nr * sizeof(*delete_refs));
- for (i = 0; i states.stale.nr; i++)
- delete_refs[i] = states.stale.items[i].util;
if (!dry_run) {
struct strbuf err = STRBUF_INIT;
@@ -1353,19 +1349,16 @@ static int prune_remote(const char *remote, int dry_run)
errno, err);
result |= error(%s, err.buf);
} else
- if (repack_without_refs(delete_refs,
- states.stale.nr, err))
+ if (repack_without_refs(delete_refs_list,
+ err))
result |= error(%s, err.buf);
strbuf_release(err);
}
- free(delete_refs);
}
for (i = 0; i states.stale.nr; i++) {
const char *refname = states.stale.items[i].util;
- string_list_insert(delete_refs_list, refname);
-
if (!dry_run)
result |= delete_ref(refname, NULL, 0);
diff --git a/refs.c b/refs.c
index 809bd3f..e9e321e 100644
--- a/refs.c
+++ b/refs.c
@@ -2663,31 +2663,32 @@ static int curate_packed_ref_fn(struct ref_entry
*entry, void *cb_data)
/*
* Must be called with packed refs already locked (and sorted)
*/
-int repack_without_refs(const char **refnames, int n, struct strbuf *err)
+int repack_without_refs(struct string_list *without, struct strbuf *err)
{
struct ref_dir *packed;
struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
struct string_list_item *ref_to_delete;
- int i, ret, removed = 0;
+ int count, ret, removed = 0;
assert(err);
/* Look for a packed ref */
- for (i = 0; i n; i++)
- if (get_packed_ref(refnames[i]))
- break;
+ count = 0;
+ for_each_string_list_item(ref_to_delete, without)
+ if (get_packed_ref(ref_to_delete-string))
+ count++;
- /* Avoid processing if we have nothing to do */
- if (i == n) {
+ /* No refname exists in packed refs */
+ if (!count) {
rollback_packed_refs();
- return 0; /* no refname exists in packed refs */
+ return 0;
}
packed = get_packed_refs(ref_cache);
/* Remove refnames from the cache */
- for (i = 0; i n; i++)
- if (remove_entry(packed, refnames[i]) != -1
[PATCH v3 01/16] refs.c: allow passing raw git_committer_info as email to _update_reflog
In many places in the code we do not have access to the individual fields
in the committer data. Instead we might only have access to prebaked data
such as what is returned by git_committer_info() containing a string
that consists of email, timestamp, zone etc.
This makes it inconvenient to use transaction_update_reflog since it means
you would have to first parse git_committer_info before you can call
update_reflog.
Add a new flag REFLOG_COMMITTER_INFO_IS_VALID to _update_reflog to tell it
that we pass in a fully prebaked committer info string that can be used as is.
At the same time, also go over and change all references from email
to id where the code actually refers to a committer id and not just an email
address. I.e. where the string is : NAME EMAIL
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
builtin/reflog.c | 19 +--
refs.c | 21 -
refs.h | 25 +++--
3 files changed, 48 insertions(+), 17 deletions(-)
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 6bb7454..be88a53 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -292,7 +292,7 @@ static int unreachable(struct expire_reflog_cb *cb, struct
commit *commit, unsig
}
static int expire_reflog_ent(unsigned char *osha1, unsigned char *nsha1,
- const char *email, unsigned long timestamp, int tz,
+ const char *id, unsigned long timestamp, int tz,
const char *message, void *cb_data)
{
struct expire_reflog_cb *cb = cb_data;
@@ -320,9 +320,14 @@ static int expire_reflog_ent(unsigned char *osha1,
unsigned char *nsha1,
goto prune;
if (cb-t) {
+ struct reflog_committer_info ci;
+
+ memset(ci, 0, sizeof(ci));
+ ci.id = id;
+ ci.timestamp = timestamp;
+ ci.tz = tz;
if (transaction_update_reflog(cb-t, cb-refname, nsha1, osha1,
- email, timestamp, tz, message, 0,
- err))
+ ci, message, 0, err))
return -1;
hashcpy(cb-last_kept_sha1, nsha1);
}
@@ -356,6 +361,7 @@ static int expire_reflog(const char *ref, const unsigned
char *sha1, int unused,
struct expire_reflog_cb cb;
struct commit *tip_commit;
struct commit_list *tips;
+ struct reflog_committer_info ci;
int status = 0;
memset(cb, 0, sizeof(cb));
@@ -368,9 +374,10 @@ static int expire_reflog(const char *ref, const unsigned
char *sha1, int unused,
status |= error(%s, err.buf);
goto cleanup;
}
+
+ memset(ci, 0, sizeof(ci));
if (transaction_update_reflog(cb.t, cb.refname, null_sha1, null_sha1,
- NULL, 0, 0, NULL, REFLOG_TRUNCATE,
- err)) {
+ ci, NULL, REFLOG_TRUNCATE, err)) {
status |= error(%s, err.buf);
goto cleanup;
}
@@ -672,7 +679,7 @@ static int cmd_reflog_expire(int argc, const char **argv,
const char *prefix)
}
static int count_reflog_ent(unsigned char *osha1, unsigned char *nsha1,
- const char *email, unsigned long timestamp, int tz,
+ const char *id, unsigned long timestamp, int tz,
const char *message, void *cb_data)
{
struct cmd_reflog_expire_cb *cb = cb_data;
diff --git a/refs.c b/refs.c
index f1ca9e4..1791166 100644
--- a/refs.c
+++ b/refs.c
@@ -3226,7 +3226,7 @@ struct read_ref_at_cb {
};
static int read_ref_at_ent(unsigned char *osha1, unsigned char *nsha1,
- const char *email, unsigned long timestamp, int tz,
+ const char *id, unsigned long timestamp, int tz,
const char *message, void *cb_data)
{
struct read_ref_at_cb *cb = cb_data;
@@ -3273,7 +3273,7 @@ static int read_ref_at_ent(unsigned char *osha1, unsigned
char *nsha1,
}
static int read_ref_at_ent_oldest(unsigned char *osha1, unsigned char *nsha1,
- const char *email, unsigned long timestamp,
+ const char *id, unsigned long timestamp,
int tz, const char *message, void *cb_data)
{
struct read_ref_at_cb *cb = cb_data;
@@ -3625,8 +3625,7 @@ int transaction_update_reflog(struct transaction
*transaction,
const char *refname,
const unsigned char *new_sha1,
const unsigned char *old_sha1,
- const char *email,
- unsigned long timestamp, int tz,
+ struct reflog_committer_info *ci,
const
[PATCH v3 07/16] refs.c: rollback the lockfile before we die() in repack_without_refs
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/refs.c b/refs.c
index 9a3c7fe..5a8f3da 100644
--- a/refs.c
+++ b/refs.c
@@ -2707,8 +2707,10 @@ int repack_without_refs(struct string_list *without,
struct strbuf *err)
/* Remove any other accumulated cruft */
do_for_each_entry_in_dir(packed, 0, curate_packed_ref_fn,
refs_to_delete);
for_each_string_list_item(ref_to_delete, refs_to_delete) {
- if (remove_entry(packed, ref_to_delete-string) == -1)
+ if (remove_entry(packed, ref_to_delete-string) == -1) {
+ rollback_packed_refs();
die(internal error);
+ }
}
/* Write what remains */
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 16/16] refs.c: add an err argument to pack_refs
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/pack-refs.c | 8 +++-
refs.c | 7 +++
refs.h | 3 ++-
3 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/builtin/pack-refs.c b/builtin/pack-refs.c
index b20b1ec..299768e 100644
--- a/builtin/pack-refs.c
+++ b/builtin/pack-refs.c
@@ -10,6 +10,7 @@ static char const * const pack_refs_usage[] = {
int cmd_pack_refs(int argc, const char **argv, const char *prefix)
{
unsigned int flags = PACK_REFS_PRUNE;
+ struct strbuf err = STRBUF_INIT;
struct option opts[] = {
OPT_BIT(0, all, flags, N_(pack everything),
PACK_REFS_ALL),
OPT_BIT(0, prune, flags, N_(prune loose refs (default)),
PACK_REFS_PRUNE),
@@ -17,5 +18,10 @@ int cmd_pack_refs(int argc, const char **argv, const char
*prefix)
};
if (parse_options(argc, argv, prefix, opts, pack_refs_usage, 0))
usage_with_options(pack_refs_usage, opts);
- return pack_refs(flags);
+ if (pack_refs(flags, err)) {
+ error(%s, err.buf);
+ strbuf_release(err);
+ return -1;
+ }
+ return 0;
}
diff --git a/refs.c b/refs.c
index 1314a9a..cafb4aa 100644
--- a/refs.c
+++ b/refs.c
@@ -2593,16 +2593,15 @@ static void prune_refs(struct ref_to_prune *r)
}
}
-int pack_refs(unsigned int flags)
+int pack_refs(unsigned int flags, struct strbuf *err)
{
struct pack_refs_cb_data cbdata;
- struct strbuf err = STRBUF_INIT;
memset(cbdata, 0, sizeof(cbdata));
cbdata.flags = flags;
- if (lock_packed_refs(err))
- die(%s, err.buf);
+ if (lock_packed_refs(err))
+ return -1;
cbdata.packed_refs = get_packed_refs(ref_cache);
diff --git a/refs.h b/refs.h
index b5ba685..489aa9d 100644
--- a/refs.h
+++ b/refs.h
@@ -130,8 +130,9 @@ extern void warn_dangling_symrefs(FILE *fp, const char
*msg_fmt, const struct st
/*
* Write a packed-refs file for the current repository.
* flags: Combination of the above PACK_REFS_* flags.
+ * Returns 0 on success and fills in err on failure.
*/
-int pack_refs(unsigned int flags);
+int pack_refs(unsigned int flags, struct strbuf *err);
extern int ref_exists(const char *);
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 11/16] refs.c: make repack_without_refs static
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 2 +-
refs.h | 3 ---
2 files changed, 1 insertion(+), 4 deletions(-)
diff --git a/refs.c b/refs.c
index c1db86f..2c6b0f6 100644
--- a/refs.c
+++ b/refs.c
@@ -2668,7 +2668,7 @@ static int curate_packed_ref_fn(struct ref_entry *entry,
void *cb_data)
/*
* Must be called with packed refs already locked (and sorted)
*/
-int repack_without_refs(struct string_list *without, struct strbuf *err)
+static int repack_without_refs(struct string_list *without, struct strbuf *err)
{
struct ref_dir *packed;
struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
diff --git a/refs.h b/refs.h
index 0ba078e..ce290b1 100644
--- a/refs.h
+++ b/refs.h
@@ -163,9 +163,6 @@ extern void rollback_packed_refs(void);
*/
int pack_refs(unsigned int flags);
-extern int repack_without_refs(struct string_list *without,
- struct strbuf *err);
-
extern int ref_exists(const char *);
extern int is_branch(const char *refname);
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 10/16] remote.c: use a transaction for deleting refs
Transactions now use packed refs when deleting multiple refs so there is no
need to do it manually from remote.c any more.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/remote.c | 80
1 file changed, 40 insertions(+), 40 deletions(-)
diff --git a/builtin/remote.c b/builtin/remote.c
index 6806251..42702d7 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -750,30 +750,27 @@ static int mv(int argc, const char **argv)
static int remove_branches(struct string_list *branches)
{
- struct strbuf err = STRBUF_INIT;
int i, result = 0;
+ struct transaction *transaction;
+ struct strbuf err = STRBUF_INIT;
- if (lock_packed_refs(0)) {
- struct strbuf err = STRBUF_INIT;
-
- unable_to_lock_message(git_path(packed-refs), errno, err);
- error(%s, err.buf);
- strbuf_release(err);
- return -1;
- }
+ transaction = transaction_begin(err);
+ if (!transaction)
+ die(%s, err.buf);
- if (repack_without_refs(branches, err))
+ for (i = 0; i branches-nr; i++)
+ if (transaction_delete_ref(transaction,
+ branches-items[i].string, NULL,
+ 0, 0, remote-branches, err)) {
+ result |= error(%s, err.buf);
+ goto cleanup;
+ }
+ if (transaction_commit(transaction, err))
result |= error(%s, err.buf);
- strbuf_release(err);
-
- for (i = 0; i branches-nr; i++) {
- struct string_list_item *item = branches-items + i;
- const char *refname = item-string;
-
- if (delete_ref(refname, NULL, 0))
- result |= error(_(Could not remove branch %s),
refname);
- }
+ cleanup:
+ strbuf_release(err);
+ transaction_free(transaction);
return result;
}
@@ -1325,42 +1322,38 @@ static int prune_remote(const char *remote, int dry_run)
const char *dangling_msg = dry_run
? _( %s will become dangling!)
: _( %s has become dangling!);
+ struct transaction *transaction = NULL;
+ struct strbuf err = STRBUF_INIT;
memset(states, 0, sizeof(states));
get_remote_ref_states(remote, states, GET_REF_STATES);
- for (i = 0; i states.stale.nr; i++) {
- string_list_insert(delete_refs_list,
- states.stale.items[i].util);
- }
-
if (states.stale.nr) {
printf_ln(_(Pruning %s), remote);
printf_ln(_(URL: %s),
states.remote-url_nr
? states.remote-url[0]
: _((no URL)));
-
- if (!dry_run) {
- struct strbuf err = STRBUF_INIT;
-
- if (lock_packed_refs(0)) {
- unable_to_lock_message(git_path(packed-refs),
- errno, err);
- result |= error(%s, err.buf);
- } else
- if (repack_without_refs(delete_refs_list,
- err))
- result |= error(%s, err.buf);
- strbuf_release(err);
- }
}
+ if (!dry_run) {
+ transaction = transaction_begin(err);
+ if (!transaction)
+ die(%s, err.buf);
+ }
for (i = 0; i states.stale.nr; i++) {
const char *refname = states.stale.items[i].util;
- if (!dry_run)
- result |= delete_ref(refname, NULL, 0);
+ string_list_insert(delete_refs_list, refname);
+
+ if (!dry_run) {
+ if (transaction_delete_ref(transaction,
+ refname, NULL,
+ 0, 0, remote-branches, err)) {
+ result |= error(%s, err.buf);
+ goto cleanup;
+ }
+ }
if (dry_run)
printf_ln(_( * [would prune] %s),
@@ -1370,6 +1363,13 @@ static int prune_remote(const char *remote, int dry_run)
abbrev_ref(refname, refs/remotes/));
}
+ if (!dry_run)
+ if (transaction_commit(transaction, err))
+ result |= error(%s, err.buf);
+
+ cleanup:
+ strbuf_release(err);
+ transaction_free(transaction);
warn_dangling_symrefs(stdout, dangling_msg, delete_refs_list);
string_list_clear(delete_refs_list, 0);
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe
[PATCH v3 02/16] refs.c: return error instead of dying when locking fails during transaction
Change lock_ref_sha1_basic to return an error instead of dying when
we fail to lock a file during a transaction.
This function is only called from transaction_commit() and it knows how
to handle these failures.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 10 --
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index 1791166..d4ab65c 100644
--- a/refs.c
+++ b/refs.c
@@ -2340,6 +2340,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char
*refname,
lock-lock_fd = hold_lock_file_for_update(lock-lk, ref_file, lflags);
if (lock-lock_fd 0) {
+ last_errno = errno;
if (errno == ENOENT --attempts_remaining 0)
/*
* Maybe somebody just deleted one of the
@@ -2347,8 +2348,13 @@ static struct ref_lock *lock_ref_sha1_basic(const char
*refname,
* again:
*/
goto retry;
- else
- unable_to_lock_die(ref_file, errno);
+ else {
+ struct strbuf err = STRBUF_INIT;
+ unable_to_lock_message(ref_file, errno, err);
+ error(%s, err.buf);
+ strbuf_reset(err);
+ goto error_return;
+ }
}
return old_sha1 ? verify_lock(lock, old_sha1, mustexist) : lock;
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 03/16] refs.c: use packed refs when deleting refs during a transaction
Make the deletion of refs during a transaction more atomic.
Start by first copying all loose refs we will be deleting to the packed
refs file and then commit the packed refs file. Then re-lock the packed refs
file to stop anyone else from modifying these refs and keep it locked until
we are finished.
Since all refs we are about to delete are now safely held in the packed refs
file we can proceed to immediately unlink any corresponding loose refs
and still be fully rollback-able.
The exception is for refs that can not be resolved. Those refs are never
added to the packed refs and will just be un-rollback-ably deleted during
commit.
By deleting all the loose refs at the start of the transaction we make make
it possible to both delete one ref and then re-create a different ref in
the same transaction even if the two refs would normally conflict.
Example: rename m-m/m
In that example we want to delete the file 'm' so that we make room so
that we can create a directory with the same name in order to lock and
write to the ref m/m and its lock-file m/m.lock.
If there is a failure during the commit phase we can rollback without losing
any refs since we have so far only deleted loose refs that that are
guaranteed to also have a corresponding entry in the packed refs file.
Once we have finished all updates for refs and their reflogs we can repack
the packed refs file and remove the to-be-deleted refs from the packed refs,
at which point all the deleted refs will disappear in one atomic rename
operation.
This also means that for an outside observer, deletion of multiple refs
in a single transaction will look atomic instead of one ref being deleted
at a time.
In order to do all this we need to change the semantics for the
repack_without_refs function so that we can lock the packed refs file,
do other stuff, and later be able to call repack_without_refs with the
lock already taken.
This means we need some additional changes in remote.c to reflect the
changes to the repack_without_refs semantics.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
builtin/remote.c | 20 ++-
refs.c | 155 ++-
2 files changed, 138 insertions(+), 37 deletions(-)
diff --git a/builtin/remote.c b/builtin/remote.c
index 7f28f92..c25420f 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -1,4 +1,5 @@
#include builtin.h
+#include lockfile.h
#include parse-options.h
#include transport.h
#include remote.h
@@ -753,6 +754,15 @@ static int remove_branches(struct string_list *branches)
const char **branch_names;
int i, result = 0;
+ if (lock_packed_refs(0)) {
+ struct strbuf err = STRBUF_INIT;
+
+ unable_to_lock_message(git_path(packed-refs), errno, err);
+ error(%s, err.buf);
+ strbuf_release(err);
+ return -1;
+ }
+
branch_names = xmalloc(branches-nr * sizeof(*branch_names));
for (i = 0; i branches-nr; i++)
branch_names[i] = branches-items[i].string;
@@ -1337,9 +1347,15 @@ static int prune_remote(const char *remote, int dry_run)
delete_refs[i] = states.stale.items[i].util;
if (!dry_run) {
struct strbuf err = STRBUF_INIT;
- if (repack_without_refs(delete_refs, states.stale.nr,
- err))
+
+ if (lock_packed_refs(0)) {
+ unable_to_lock_message(git_path(packed-refs),
+ errno, err);
result |= error(%s, err.buf);
+ } else
+ if (repack_without_refs(delete_refs,
+ states.stale.nr, err))
+ result |= error(%s, err.buf);
strbuf_release(err);
}
free(delete_refs);
diff --git a/refs.c b/refs.c
index d4ab65c..809bd3f 100644
--- a/refs.c
+++ b/refs.c
@@ -2660,6 +2660,9 @@ static int curate_packed_ref_fn(struct ref_entry *entry,
void *cb_data)
return 0;
}
+/*
+ * Must be called with packed refs already locked (and sorted)
+ */
int repack_without_refs(const char **refnames, int n, struct strbuf *err)
{
struct ref_dir *packed;
@@ -2674,14 +2677,12 @@ int repack_without_refs(const char **refnames, int n,
struct strbuf *err)
if (get_packed_ref(refnames[i]))
break;
- /* Avoid locking if we have nothing to do */
- if (i == n)
+ /* Avoid processing if we have nothing to do */
+ if (i == n) {
+ rollback_packed_refs();
return 0; /* no refname exists in packed refs */
-
- if (lock_packed_refs(0
[PATCH v3 09/16] refs.c: write updates to packed refs when a transaction has more than one ref
When we are updating more than one single ref, i.e. not a commit, then
write the updated refs directly to the packed refs file instead of writing
them as loose refs.
Change clone to use a transaction instead of using the packed refs API.
This changes the behavior of clone slightly. Previously clone would always
clone all refs into a packed refs file. With this change clone will only
clone into packed refs iff there are two or more refs being cloned.
If the repository we are cloning from only contains exactly one single ref
then clone will now store this as a loose ref. The benefit here is that
we no longer need to export a bunch of API functions to clone to access
packed refs directly. Clone can now just use a normal transaction and all
the packed refs goodness will happen automatically.
Update the t5516 test to cope with the fact that clone now only uses
packed refs if there are more than one ref being cloned.
We still use loose refs for single ref transactions, such as are used
by 'git commit' and friends. The reason for this is that if you have very
many refs then having to re-write the whole packed refs file for every
common operation like commit would have a performance impact.
That said, with these changes it should now be fairly straightforward to
add support to optionally start using packed refs for ALL updates
which could solve existing issues with name clashes in case insensitive
filesystems.
This change also means that multi-ref updates will now appear as a single
atomic change to any external observers instead of a sequence of discreete
changes.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/clone.c | 16 ++---
refs.c| 89 ++-
t/t5516-fetch-push.sh | 2 +-
3 files changed, 72 insertions(+), 35 deletions(-)
diff --git a/builtin/clone.c b/builtin/clone.c
index 316c75d..bb2c058 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -498,17 +498,25 @@ static struct ref *wanted_peer_refs(const struct ref
*refs,
static void write_remote_refs(const struct ref *local_refs)
{
const struct ref *r;
+ struct transaction *transaction;
+ struct strbuf err = STRBUF_INIT;
- lock_packed_refs(LOCK_DIE_ON_ERROR);
+ transaction = transaction_begin(err);
+ if (!transaction)
+ die(%s, err.buf);
for (r = local_refs; r; r = r-next) {
if (!r-peer_ref)
continue;
- add_packed_ref(r-peer_ref-name, r-old_sha1);
+ if (transaction_update_ref(transaction, r-peer_ref-name,
+ r-old_sha1, NULL, 0, 0, NULL,
+ err))
+ die(%s, err.buf);
}
- if (commit_packed_refs())
- die_errno(unable to overwrite old ref-pack file);
+ if (transaction_commit(transaction, err))
+ die(%s, err.buf);
+ transaction_free(transaction);
}
static void write_followtags(const struct ref *refs, const char *msg)
diff --git a/refs.c b/refs.c
index 7f4b4cb..c1db86f 100644
--- a/refs.c
+++ b/refs.c
@@ -2673,36 +2673,15 @@ int repack_without_refs(struct string_list *without,
struct strbuf *err)
struct ref_dir *packed;
struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
struct string_list_item *ref_to_delete;
- int count, ret, removed = 0;
+ int ret;
assert(err);
- /* Look for a packed ref */
- count = 0;
- for_each_string_list_item(ref_to_delete, without)
- if (get_packed_ref(ref_to_delete-string))
- count++;
-
- /* No refname exists in packed refs */
- if (!count) {
- rollback_packed_refs();
- return 0;
- }
-
packed = get_packed_refs(ref_cache);
/* Remove refnames from the cache */
for_each_string_list_item(ref_to_delete, without)
- if (remove_entry(packed, ref_to_delete-string) != -1)
- removed = 1;
- if (!removed) {
- /*
-* All packed entries disappeared while we were
-* acquiring the lock.
-*/
- rollback_packed_refs();
- return 0;
- }
+ remove_entry(packed, ref_to_delete-string);
/* Remove any other accumulated cruft */
do_for_each_entry_in_dir(packed, 0, curate_packed_ref_fn,
refs_to_delete);
@@ -3791,6 +3770,7 @@ int transaction_commit(struct transaction *transaction,
struct strbuf *err)
{
int ret = 0, i, need_repack = 0;
+ int num_updates = 0;
int n = transaction-nr;
struct packed_ref_cache *packed_ref_cache;
struct ref_update **updates = transaction-updates;
@@ -3824,14 +3804,30 @@ int transaction_commit(struct transaction *transaction,
goto
[PATCH v3 06/16] refs.c: update rename_ref to use a transaction
Change refs.c to use a single transaction to perform the rename.
Change the function to return 1 on failure instead of either -1 or 1.
These changes make the rename_ref operation atomic.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c| 168 ++
t/t3200-branch.sh | 7 ---
2 files changed, 43 insertions(+), 132 deletions(-)
diff --git a/refs.c b/refs.c
index 8ca6add..9a3c7fe 100644
--- a/refs.c
+++ b/refs.c
@@ -2757,60 +2757,6 @@ int delete_ref(const char *refname, const unsigned char
*sha1, int delopt)
return 0;
}
-/*
- * People using contrib's git-new-workdir have .git/logs/refs -
- * /some/other/path/.git/logs/refs, and that may live on another device.
- *
- * IOW, to avoid cross device rename errors, the temporary renamed log must
- * live into logs/refs.
- */
-#define TMP_RENAMED_LOG logs/refs/.tmp-renamed-log
-
-static int rename_tmp_log(const char *newrefname)
-{
- int attempts_remaining = 4;
-
- retry:
- switch (safe_create_leading_directories(git_path(logs/%s,
newrefname))) {
- case SCLD_OK:
- break; /* success */
- case SCLD_VANISHED:
- if (--attempts_remaining 0)
- goto retry;
- /* fall through */
- default:
- error(unable to create directory for %s, newrefname);
- return -1;
- }
-
- if (rename(git_path(TMP_RENAMED_LOG), git_path(logs/%s, newrefname)))
{
- if ((errno==EISDIR || errno==ENOTDIR) --attempts_remaining
0) {
- /*
-* rename(a, b) when b is an existing
-* directory ought to result in ISDIR, but
-* Solaris 5.8 gives ENOTDIR. Sheesh.
-*/
- if (remove_empty_directories(git_path(logs/%s,
newrefname))) {
- error(Directory not empty: logs/%s,
newrefname);
- return -1;
- }
- goto retry;
- } else if (errno == ENOENT --attempts_remaining 0) {
- /*
-* Maybe another process just deleted one of
-* the directories in the path to newrefname.
-* Try again from the beginning.
-*/
- goto retry;
- } else {
- error(unable to move logfile TMP_RENAMED_LOG to
logs/%s: %s,
- newrefname, strerror(errno));
- return -1;
- }
- }
- return 0;
-}
-
static int rename_ref_available(const char *oldname, const char *newname)
{
struct string_list skip = STRING_LIST_INIT_NODUP;
@@ -2859,91 +2805,63 @@ static int copy_reflog_into_strbuf(const char *refname,
struct strbuf *buf)
int rename_ref(const char *oldrefname, const char *newrefname, const char
*logmsg)
{
- unsigned char sha1[20], orig_sha1[20];
- int flag = 0, logmoved = 0;
- struct ref_lock *lock;
- struct stat loginfo;
- int log = !lstat(git_path(logs/%s, oldrefname), loginfo);
+ unsigned char sha1[20];
+ int flag = 0;
+ int log;
+ struct transaction *transaction = NULL;
+ struct strbuf err = STRBUF_INIT;
const char *symref = NULL;
+ struct reflog_committer_info ci;
- if (log S_ISLNK(loginfo.st_mode))
- return error(reflog for %s is a symlink, oldrefname);
+ memset(ci, 0, sizeof(ci));
+ ci.committer_info = git_committer_info(0);
symref = resolve_ref_unsafe(oldrefname, RESOLVE_REF_READING,
- orig_sha1, flag);
- if (flag REF_ISSYMREF)
- return error(refname %s is a symbolic ref, renaming it is not
supported,
- oldrefname);
- if (!symref)
- return error(refname %s not found, oldrefname);
-
- if (!rename_ref_available(oldrefname, newrefname))
+ sha1, flag);
+ if (flag REF_ISSYMREF) {
+ error(refname %s is a symbolic ref, renaming it is not
+ supported, oldrefname);
return 1;
-
- if (log rename(git_path(logs/%s, oldrefname),
git_path(TMP_RENAMED_LOG)))
- return error(unable to move logfile logs/%s to
TMP_RENAMED_LOG: %s,
- oldrefname, strerror(errno));
-
- if (delete_ref(oldrefname, orig_sha1, REF_NODEREF)) {
- error(unable to delete old %s, oldrefname);
- goto rollback;
}
-
- if (!read_ref_full(newrefname, RESOLVE_REF_READING, sha1, NULL)
- delete_ref(newrefname, sha1, REF_NODEREF)) {
- if (errno==EISDIR) {
- if (remove_empty_directories(git_path(%s
[PATCH v3 2/7] send-pack.c: add an --atomic-push command line argument
This adds support to send-pack to negotiate and use atomic pushes
iff the server supports it. Atomic pushes are activated by a new command
line flag --atomic-push.
In order to do this we also need to change the semantics for send_pack()
slightly. The existing send_pack() function actually don't sent all the
refs back to the server when multiple refs are involved, for example
when using --all. Several of the failure modes for pushes can already be
detected locally in the send_pack client based on the information from the
initial server side list of all the refs as generated by receive-pack.
Any such refs that we thus know would fail to push are thus pruned from
the list of refs we send to the server to update.
For atomic pushes, we have to deal thus with both failures that are detected
locally as well as failures that are reported back from the server. In order
to do so we treat all local failures as push failures too.
We introduce a new status code REF_STATUS_ATOMIC_PUSH_FAILED so we can
flag all refs that we would normally have tried to push to the server
but we did not due to local failures. This is to improve the error message
back to the end user to flag that these refs failed to update since the
atomic push operation failed.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
Documentation/git-send-pack.txt | 7 ++-
builtin/send-pack.c | 6 +-
remote.h| 3 ++-
send-pack.c | 39 ++-
send-pack.h | 1 +
transport.c | 4
6 files changed, 52 insertions(+), 8 deletions(-)
diff --git a/Documentation/git-send-pack.txt b/Documentation/git-send-pack.txt
index 2a0de42..9296587 100644
--- a/Documentation/git-send-pack.txt
+++ b/Documentation/git-send-pack.txt
@@ -9,7 +9,7 @@ git-send-pack - Push objects over Git protocol to another
repository
SYNOPSIS
[verse]
-'git send-pack' [--all] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin] [host:]directory
[ref...]
+'git send-pack' [--all] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin] [--atomic-push]
[host:]directory [ref...]
DESCRIPTION
---
@@ -62,6 +62,11 @@ be in a separate packet, and the list must end with a flush
packet.
Send a thin pack, which records objects in deltified form based
on objects not included in the pack to reduce network traffic.
+--atomic-push::
+ Use an atomic transaction for updating the refs. If any of the refs
+ fails to update then the entire push will fail without changing any
+ refs.
+
host::
A remote host to house the repository. When this
part is specified, 'git-receive-pack' is invoked via
diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index b564a77..93cb17c 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -13,7 +13,7 @@
#include sha1-array.h
static const char send_pack_usage[] =
-git send-pack [--all | --mirror] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin] [host:]directory
[ref...]\n
+git send-pack [--all | --mirror] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin] [--atomic-push]
[host:]directory [ref...]\n
--all and explicit ref specification are mutually exclusive.;
static struct send_pack_args args;
@@ -170,6 +170,10 @@ int cmd_send_pack(int argc, const char **argv, const char
*prefix)
args.use_thin_pack = 1;
continue;
}
+ if (!strcmp(arg, --atomic-push)) {
+ args.use_atomic_push = 1;
+ continue;
+ }
if (!strcmp(arg, --stateless-rpc)) {
args.stateless_rpc = 1;
continue;
diff --git a/remote.h b/remote.h
index 8b62efd..f346524 100644
--- a/remote.h
+++ b/remote.h
@@ -115,7 +115,8 @@ struct ref {
REF_STATUS_REJECT_SHALLOW,
REF_STATUS_UPTODATE,
REF_STATUS_REMOTE_REJECT,
- REF_STATUS_EXPECTING_REPORT
+ REF_STATUS_EXPECTING_REPORT,
+ REF_STATUS_ATOMIC_PUSH_FAILED
} status;
char *remote_status;
struct ref *peer_ref; /* when renaming */
diff --git a/send-pack.c b/send-pack.c
index 1ccc84c..08602a8 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -190,7 +190,7 @@ static void advertise_shallow_grafts_buf(struct strbuf *sb)
for_each_commit_graft(advertise_shallow_grafts_cb, sb);
}
-static int ref_update_to_be_sent(const struct ref *ref, const struct
send_pack_args *args)
+static int ref_update_to_be_sent(const struct ref *ref, const struct
send_pack_args *args, int *atomic_push_failed)
{
if (!ref-peer_ref !args-send_mirror)
return
[PATCH v3 7/7] refs.c: add an err argument to create_symref
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/branch.c | 7 +--
builtin/checkout.c | 13 ++---
builtin/clone.c| 15 +++
builtin/init-db.c | 8 ++--
builtin/notes.c| 7 ---
builtin/remote.c | 26 ++
builtin/symbolic-ref.c | 6 +-
cache.h| 1 -
refs.c | 30 ++
refs.h | 1 +
10 files changed, 78 insertions(+), 36 deletions(-)
diff --git a/builtin/branch.c b/builtin/branch.c
index 04f57d4..ab6d9f4 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -698,6 +698,7 @@ static void rename_branch(const char *oldname, const char
*newname, int force)
{
struct strbuf oldref = STRBUF_INIT, newref = STRBUF_INIT, logmsg =
STRBUF_INIT;
struct strbuf oldsection = STRBUF_INIT, newsection = STRBUF_INIT;
+ struct strbuf err = STRBUF_INIT;
int recovery = 0;
int clobber_head_ok;
@@ -734,8 +735,10 @@ static void rename_branch(const char *oldname, const char
*newname, int force)
warning(_(Renamed a misnamed branch '%s' away), oldref.buf +
11);
/* no need to pass logmsg here as HEAD didn't really move */
- if (!strcmp(oldname, head) create_symref(HEAD, newref.buf, NULL))
- die(_(Branch renamed to %s, but HEAD is not updated!),
newname);
+ if (!strcmp(oldname, head)
+ create_symref(HEAD, newref.buf, NULL, err))
+ die(_(Branch renamed to %s, but HEAD is not updated!. %s),
+ newname, err.buf);
strbuf_addf(oldsection, branch.%s, oldref.buf + 11);
strbuf_release(oldref);
diff --git a/builtin/checkout.c b/builtin/checkout.c
index d9cb9c3..1efe353 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -634,7 +634,10 @@ static void update_refs_for_switch(const struct
checkout_opts *opts,
describe_detached_head(_(HEAD is now at),
new-commit);
}
} else if (new-path) { /* Switch branches. */
- create_symref(HEAD, new-path, msg.buf);
+ if (create_symref(HEAD, new-path, msg.buf, err)) {
+ error(%s, err.buf);
+ strbuf_release(err);
+ }
if (!opts-quiet) {
if (old-path !strcmp(new-path, old-path)) {
if (opts-new_branch_force)
@@ -1020,12 +1023,16 @@ static int parse_branchname_arg(int argc, const char
**argv,
static int switch_unborn_to_new_branch(const struct checkout_opts *opts)
{
int status;
- struct strbuf branch_ref = STRBUF_INIT;
+ struct strbuf branch_ref = STRBUF_INIT, err = STRBUF_INIT;
if (!opts-new_branch)
die(_(You are on a branch yet to be born));
strbuf_addf(branch_ref, refs/heads/%s, opts-new_branch);
- status = create_symref(HEAD, branch_ref.buf, checkout -b);
+ status = create_symref(HEAD, branch_ref.buf, checkout -b, err);
+ if (status) {
+ error(%s, err.buf);
+ strbuf_release(err);
+ }
strbuf_release(branch_ref);
if (!opts-quiet)
fprintf(stderr, _(Switched to a new branch '%s'\n),
diff --git a/builtin/clone.c b/builtin/clone.c
index 5577b5b..17b6ae8 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -565,6 +565,7 @@ static void update_remote_refs(const struct ref *refs,
int check_connectivity)
{
const struct ref *rm = mapped_refs;
+ struct strbuf err = STRBUF_INIT;
if (check_connectivity) {
if (transport-progress)
@@ -586,9 +587,12 @@ static void update_remote_refs(const struct ref *refs,
struct strbuf head_ref = STRBUF_INIT;
strbuf_addstr(head_ref, branch_top);
strbuf_addstr(head_ref, HEAD);
- create_symref(head_ref.buf,
- remote_head_points_at-peer_ref-name,
- msg);
+ if (create_symref(head_ref.buf,
+ remote_head_points_at-peer_ref-name,
+ msg, err)) {
+ error(%s, err.buf);
+ strbuf_release(err);
+ }
}
}
@@ -599,7 +603,10 @@ static void update_head(const struct ref *our, const
struct ref *remote,
const char *head;
if (our skip_prefix(our-name, refs/heads/, head)) {
/* Local default branch link */
- create_symref(HEAD, our-name, NULL);
+ if (create_symref(HEAD, our-name, NULL, err)) {
+ error(%s, err.buf);
+ strbuf_release(err);
+ }
if (!option_bare) {
update_ref(msg, HEAD, our-old_sha1, NULL, 0, err
[PATCH v3 0/7] ref-transaction-send-pack
List, This series has been posted before but is now rebased on the previous ref-transaction-rename series that are against next. This series can also be found at : https://github.com/rsahlberg/git/tree/ref-transactions-send-pack This series finishes the transaction work to provide atomic pushes. With this series we can now perform atomic pushes to a repository. Version 2: - Reordered the capabilities we send so that agent= remains the last capability listed. - Reworded the paragraph for atomic push in git-send-pack.txt - Dropped the patch for receive.preferatomicpush Version 3: - Fix a typo in a commit message. Ronnie Sahlberg (7): receive-pack.c: add protocol support to negotiate atomic-push send-pack.c: add an --atomic-push command line argument receive-pack.c: use a single transaction when atomic-push is negotiated push.c: add an --atomic-push argument t5543-atomic-push.sh: add basic tests for atomic pushes refs.c: add an err argument to create_reflog refs.c: add an err argument to create_symref Documentation/git-push.txt| 7 +- Documentation/git-send-pack.txt | 7 +- Documentation/technical/protocol-capabilities.txt | 12 ++- builtin/branch.c | 7 +- builtin/checkout.c| 21 +++-- builtin/clone.c | 15 +++- builtin/init-db.c | 8 +- builtin/notes.c | 7 +- builtin/push.c| 2 + builtin/receive-pack.c| 79 + builtin/remote.c | 26 -- builtin/send-pack.c | 6 +- builtin/symbolic-ref.c| 6 +- cache.h | 1 - refs.c| 93 ++-- refs.h| 5 +- remote.h | 3 +- send-pack.c | 45 -- send-pack.h | 1 + t/t5543-atomic-push.sh| 101 ++ transport.c | 5 ++ transport.h | 1 + 22 files changed, 358 insertions(+), 100 deletions(-) create mode 100755 t/t5543-atomic-push.sh -- 2.1.0.rc2.206.gedb03e5 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 6/7] refs.c: add an err argument to create_reflog
Add err argument to create_reflog that can explain the reason for a
failure. This then eliminates the need to manage errno through this
function since we can just add strerror(errno) to the err string when
meaningful. No callers relied on errno from this function for anything
else than the error message.
log_ref_write is a private function that calls create_reflog. Update
this function to also take an err argument and pass it back to the caller.
This again eliminates the need to manage errno in this function.
Update the private function write_sha1_update_reflog to also take an
err argument.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/checkout.c | 8 +++---
refs.c | 71 +++---
refs.h | 4 +--
3 files changed, 42 insertions(+), 41 deletions(-)
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 60a68f7..d9cb9c3 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -590,10 +590,12 @@ static void update_refs_for_switch(const struct
checkout_opts *opts,
if (opts-new_orphan_branch) {
if (opts-new_branch_log !log_all_ref_updates) {
char *ref_name = mkpath(refs/heads/%s,
opts-new_orphan_branch);
+ struct strbuf err = STRBUF_INIT;
- if (create_reflog(ref_name)) {
- fprintf(stderr, _(Can not do reflog
for '%s'\n),
- opts-new_orphan_branch);
+ if (create_reflog(ref_name, err)) {
+ fprintf(stderr, _(Can not do reflog
for '%s'. %s\n),
+ opts-new_orphan_branch, err.buf);
+ strbuf_release(err);
return;
}
}
diff --git a/refs.c b/refs.c
index cafb4aa..fc9ace2 100644
--- a/refs.c
+++ b/refs.c
@@ -2895,8 +2895,7 @@ static int copy_msg(char *buf, const char *msg)
return cp - buf;
}
-/* This function must set a meaningful errno on failure */
-int create_reflog(const char *refname)
+int create_reflog(const char *refname, struct strbuf *err)
{
int logfd, oflags = O_APPEND | O_WRONLY;
char logfile[PATH_MAX];
@@ -2907,9 +2906,8 @@ int create_reflog(const char *refname)
starts_with(refname, refs/notes/) ||
!strcmp(refname, HEAD)) {
if (safe_create_leading_directories(logfile) 0) {
- int save_errno = errno;
- error(unable to create directory for %s, logfile);
- errno = save_errno;
+ strbuf_addf(err, unable to create directory for %s.
+ %s, logfile, strerror(errno));
return -1;
}
oflags |= O_CREAT;
@@ -2922,20 +2920,16 @@ int create_reflog(const char *refname)
if ((oflags O_CREAT) errno == EISDIR) {
if (remove_empty_directories(logfile)) {
- int save_errno = errno;
- error(There are still logs under '%s',
- logfile);
- errno = save_errno;
+ strbuf_addf(err, There are still logs under
+ '%s', logfile);
return -1;
}
logfd = open(logfile, oflags, 0666);
}
if (logfd 0) {
- int save_errno = errno;
- error(Unable to append to %s: %s, logfile,
- strerror(errno));
- errno = save_errno;
+ strbuf_addf(err, Unable to append to %s: %s,
+ logfile, strerror(errno));
return -1;
}
}
@@ -2972,7 +2966,8 @@ static int log_ref_write_fd(int fd, const unsigned char
*old_sha1,
}
static int log_ref_write(const char *refname, const unsigned char *old_sha1,
-const unsigned char *new_sha1, const char *msg)
+const unsigned char *new_sha1, const char *msg,
+struct strbuf *err)
{
int logfd, result = 0, oflags = O_APPEND | O_WRONLY;
char log_file[PATH_MAX];
@@ -2981,7 +2976,7 @@ static int log_ref_write(const char *refname, const
unsigned char *old_sha1,
log_all_ref_updates = !is_bare_repository();
if (log_all_ref_updates !reflog_exists(refname))
- result = create_reflog(refname);
+ result = create_reflog(refname, err);
if (result)
return
[PATCH v3 5/7] t5543-atomic-push.sh: add basic tests for atomic pushes
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
t/t5543-atomic-push.sh | 101 +
1 file changed, 101 insertions(+)
create mode 100755 t/t5543-atomic-push.sh
diff --git a/t/t5543-atomic-push.sh b/t/t5543-atomic-push.sh
new file mode 100755
index 000..4903227
--- /dev/null
+++ b/t/t5543-atomic-push.sh
@@ -0,0 +1,101 @@
+#!/bin/sh
+
+test_description='pushing to a mirror repository'
+
+. ./test-lib.sh
+
+D=`pwd`
+
+invert () {
+ if $@; then
+ return 1
+ else
+ return 0
+ fi
+}
+
+mk_repo_pair () {
+ rm -rf master mirror
+ mkdir mirror
+ (
+ cd mirror
+ git init
+ git config receive.denyCurrentBranch warn
+ )
+ mkdir master
+ (
+ cd master
+ git init
+ git remote add $1 up ../mirror
+ )
+}
+
+
+test_expect_success 'atomic push works for a single branch' '
+
+ mk_repo_pair
+ (
+ cd master
+ echo one foo git add foo git commit -m one
+ git push --mirror up
+ echo two foo git add foo git commit -m two
+ git push --atomic-push --mirror up
+ )
+ master_master=$(cd master git show-ref -s --verify
refs/heads/master)
+ mirror_master=$(cd mirror git show-ref -s --verify
refs/heads/master)
+ test $master_master = $mirror_master
+
+'
+
+test_expect_success 'atomic push works for two branches' '
+
+ mk_repo_pair
+ (
+ cd master
+ echo one foo git add foo git commit -m one
+ git branch second
+ git push --mirror up
+ echo two foo git add foo git commit -m two
+ git checkout second
+ echo three foo git add foo git commit -m three
+ git checkout master
+ git push --atomic-push --mirror up
+ )
+ master_master=$(cd master git show-ref -s --verify
refs/heads/master)
+ mirror_master=$(cd mirror git show-ref -s --verify
refs/heads/master)
+ test $master_master = $mirror_master
+
+ master_second=$(cd master git show-ref -s --verify
refs/heads/second)
+ mirror_second=$(cd mirror git show-ref -s --verify
refs/heads/second)
+ test $master_second = $mirror_second
+'
+
+# set up two branches where master can be pushed but second can not
+# (non-fast-forward). Since second can not be pushed the whole operation
+# will fail and leave master untouched.
+test_expect_success 'atomic push fails if one branch fails' '
+ mk_repo_pair
+ (
+ cd master
+ echo one foo git add foo git commit -m one
+ git branch second
+ git checkout second
+ echo two foo git add foo git commit -m two
+ echo three foo git add foo git commit -m three
+ echo four foo git add foo git commit -m four
+ git push --mirror up
+ git reset --hard HEAD~2
+ git checkout master
+ echo five foo git add foo git commit -m five
+ ! git push --atomic-push --all up
+ )
+ master_master=$(cd master git show-ref -s --verify
refs/heads/master)
+ mirror_master=$(cd mirror git show-ref -s --verify
refs/heads/master)
+ test $master_master != $mirror_master
+
+ master_second=$(cd master git show-ref -s --verify
refs/heads/second)
+ mirror_second=$(cd mirror git show-ref -s --verify
refs/heads/second)
+ test $master_second != $mirror_second
+'
+
+test_done
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 3/7] receive-pack.c: use a single transaction when atomic-push is negotiated
Update receive-pack to use an atomic transaction iff the client negotiated
that it wanted atomic-push.
This leaves the default behavior to be the old non-atomic one ref at a
time update. This is to cause as little disruption as possible to existing
clients. It is unknown if there are client scripts that depend on the old
non-atomic behavior so we make it opt-in for now.
Later patch in this series also adds a configuration variable where you can
override the atomic push behavior on the receiving repo and force it
to use atomic updates always.
If it turns out over time that there are no client scripts that depend on the
old behavior we can change git to default to use atomic pushes and instead
offer an opt-out argument for people that do not want atomic pushes.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/receive-pack.c | 73 +++---
1 file changed, 58 insertions(+), 15 deletions(-)
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 65d9a7e..27b49dd 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -67,6 +67,8 @@ static const char *NONCE_SLOP = SLOP;
static const char *nonce_status;
static long nonce_stamp_slop;
static unsigned long nonce_stamp_slop_limit;
+struct strbuf err = STRBUF_INIT;
+struct transaction *transaction;
static enum deny_action parse_deny_action(const char *var, const char *value)
{
@@ -832,33 +834,55 @@ static const char *update(struct command *cmd, struct
shallow_info *si)
cmd-did_not_exist = 1;
}
}
- if (delete_ref(namespaced_name, old_sha1, 0)) {
- rp_error(failed to delete %s, name);
- return failed to delete;
+ if (!use_atomic_push) {
+ if (delete_ref(namespaced_name, old_sha1, 0)) {
+ rp_error(failed to delete %s, name);
+ return failed to delete;
+ }
+ } else {
+ if (transaction_delete_ref(transaction,
+ namespaced_name,
+ old_sha1,
+ 0, old_sha1 != NULL,
+ push, err)) {
+ rp_error(%s, err.buf);
+ strbuf_release(err);
+ return failed to delete;
+ }
}
return NULL; /* good */
}
else {
- struct strbuf err = STRBUF_INIT;
- struct transaction *transaction;
-
if (shallow_update si-shallow_ref[cmd-index]
update_shallow_ref(cmd, si))
return shallow error;
- transaction = transaction_begin(err);
- if (!transaction ||
- transaction_update_ref(transaction, namespaced_name,
- new_sha1, old_sha1, 0, 1, push,
- err) ||
- transaction_commit(transaction, err)) {
- transaction_free(transaction);
+ if (!use_atomic_push) {
+ transaction = transaction_begin(err);
+ if (!transaction) {
+ rp_error(%s, err.buf);
+ strbuf_release(err);
+ return failed to start transaction;
+ }
+ }
+ if (transaction_update_ref(transaction,
+ namespaced_name,
+ new_sha1, old_sha1,
+ 0, 1, push,
+ err)) {
rp_error(%s, err.buf);
strbuf_release(err);
return failed to update ref;
}
-
- transaction_free(transaction);
+ if (!use_atomic_push) {
+ if (transaction_commit(transaction, err)) {
+ transaction_free(transaction);
+ rp_error(%s, err.buf);
+ strbuf_release(err);
+ return failed to update ref;
+ }
+ transaction_free(transaction);
+ }
strbuf_release(err);
return NULL; /* good */
}
@@ -1058,6 +1082,16 @@ static void execute_commands(struct command *commands,
return;
}
+ if (use_atomic_push) {
+ transaction = transaction_begin(err);
+ if (!transaction
[PATCH v3 05/16] refs.c: add transaction support for renaming a reflog
Add a new transaction function transaction_rename_reflog.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 72 +-
refs.h | 8
2 files changed, 79 insertions(+), 1 deletion(-)
diff --git a/refs.c b/refs.c
index e9e321e..8ca6add 100644
--- a/refs.c
+++ b/refs.c
@@ -35,6 +35,11 @@ static unsigned char refname_disposition[256] = {
* just use the lock taken by the first update.
*/
#define UPDATE_REFLOG_NOLOCK 0x0200
+/*
+ * This update is used to replace a new or existing reflog with new content
+ * held in update-new_reflog.
+ */
+#define REFLOG_REPLACE 0x0400
/*
* Try to read one refname component from the front of refname.
@@ -2821,6 +2826,37 @@ static int rename_ref_available(const char *oldname,
const char *newname)
static int write_ref_sha1(struct ref_lock *lock, const unsigned char *sha1,
const char *logmsg);
+/*
+ * This is an optimized function to read the whole reflog as a blob
+ * into a strbuf. It is used during ref_rename so that we can use an
+ * efficient method to read the whole log and later write it back to a
+ * different file.
+ */
+static int copy_reflog_into_strbuf(const char *refname, struct strbuf *buf)
+{
+ struct stat st;
+ int fd;
+
+ if (lstat(git_path(logs/%s, refname), st) == -1)
+ return 1;
+ if ((fd = open(git_path(logs/%s, refname), O_RDONLY)) == -1) {
+ error(failed to open reflog %s, %s,
+ refname, strerror(errno));
+ return 1;
+ }
+ strbuf_init(buf, st.st_size);
+ strbuf_setlen(buf, st.st_size);
+ if (read_in_full(fd, buf-buf, st.st_size) != st.st_size) {
+ close(fd);
+ error(failed to read reflog %s, %s,
+ refname, strerror(errno));
+ return 1;
+ }
+ close(fd);
+
+ return 0;
+}
+
int rename_ref(const char *oldrefname, const char *newrefname, const char
*logmsg)
{
unsigned char sha1[20], orig_sha1[20];
@@ -3561,6 +3597,7 @@ struct ref_update {
struct lock_file *reflog_lock;
char *committer;
struct ref_update *orig_update; /* For UPDATE_REFLOG_NOLOCK */
+ struct strbuf new_reflog;
const char refname[FLEX_ARRAY];
};
@@ -3607,6 +3644,7 @@ void transaction_free(struct transaction *transaction)
return;
for (i = 0; i transaction-nr; i++) {
+ strbuf_release(transaction-updates[i]-new_reflog);
free(transaction-updates[i]-msg);
free(transaction-updates[i]-committer);
free(transaction-updates[i]);
@@ -3622,6 +3660,7 @@ static struct ref_update *add_update(struct transaction
*transaction,
size_t len = strlen(refname);
struct ref_update *update = xcalloc(1, sizeof(*update) + len + 1);
+ strbuf_init(update-new_reflog, 0);
strcpy((char *)update-refname, refname);
update-update_type = update_type;
ALLOC_GROW(transaction-updates, transaction-nr + 1,
transaction-alloc);
@@ -3681,6 +3720,27 @@ int transaction_update_reflog(struct transaction
*transaction,
return 0;
}
+int transaction_rename_reflog(struct transaction *transaction,
+ const char *oldrefname,
+ const char *newrefname,
+ struct strbuf *err)
+{
+ struct ref_update *update;
+
+ if (transaction-state != TRANSACTION_OPEN)
+ die(BUG: transaction_replace_reflog called for transaction
+ that is not open);
+
+ update = add_update(transaction, newrefname, UPDATE_LOG);
+ update-flags = REFLOG_REPLACE;
+ update-reflog_lock = xcalloc(1, sizeof(struct lock_file));
+ if (copy_reflog_into_strbuf(oldrefname, update-new_reflog))
+ die(BUG: failed to read old reflog in
+ transaction_rename_reflog);
+
+ return 0;
+}
+
int transaction_update_ref(struct transaction *transaction,
const char *refname,
const unsigned char *new_sha1,
@@ -4012,7 +4072,7 @@ int transaction_commit(struct transaction *transaction,
continue;
if (update-reflog_fd == -1)
continue;
- if (update-flags REFLOG_TRUNCATE)
+ if (update-flags (REFLOG_TRUNCATE|REFLOG_REPLACE))
if (lseek(update-reflog_fd, 0, SEEK_SET) 0 ||
ftruncate(update-reflog_fd, 0)) {
error(Could not truncate reflog: %s. %s,
@@ -4030,6 +4090,16 @@ int transaction_commit(struct transaction *transaction,
rollback_lock_file(update-reflog_lock);
update-reflog_fd = -1;
}
+ if (update-flags REFLOG_REPLACE
[PATCH v3 14/16] refs.c: make add_packed_ref return an error instead of calling die
Change add_packed_ref to return an error instead of calling die().
Update all callers to check the return value of add_packed_ref.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 21 +
1 file changed, 17 insertions(+), 4 deletions(-)
diff --git a/refs.c b/refs.c
index b59e2b8..0829c55 100644
--- a/refs.c
+++ b/refs.c
@@ -1229,15 +1229,16 @@ static struct ref_dir *get_packed_refs(struct ref_cache
*refs)
return get_packed_ref_dir(get_packed_ref_cache(refs));
}
-static void add_packed_ref(const char *refname, const unsigned char *sha1)
+static int add_packed_ref(const char *refname, const unsigned char *sha1)
{
struct packed_ref_cache *packed_ref_cache =
get_packed_ref_cache(ref_cache);
if (!packed_ref_cache-lock)
- die(internal error: packed refs not locked);
+ return -1;
add_ref(get_packed_ref_dir(packed_ref_cache),
create_ref_entry(refname, sha1, REF_ISPACKED, 1));
+ return 0;
}
/*
@@ -3827,7 +3828,13 @@ int transaction_commit(struct transaction *transaction,
sha1, NULL))
continue;
- add_packed_ref(update-refname, sha1);
+ if (add_packed_ref(update-refname, sha1)) {
+ if (err)
+ strbuf_addf(err, Failed to add %s to packed
+ refs, update-refname);
+ ret = -1;
+ goto cleanup;
+ }
need_repack = 1;
}
if (need_repack) {
@@ -3941,7 +3948,13 @@ int transaction_commit(struct transaction *transaction,
packed = get_packed_refs(ref_cache);
remove_entry(packed, update-refname);
- add_packed_ref(update-refname, update-new_sha1);
+ if (add_packed_ref(update-refname, update-new_sha1)) {
+ if (err)
+ strbuf_addf(err, Failed to add %s to packed
+ refs, update-refname);
+ ret = -1;
+ goto cleanup;
+ }
need_repack = 1;
try_remove_empty_parents((char *)update-refname);
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 15/16] refs.c: make lock_packed_refs take an err argument
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 25 +
1 file changed, 13 insertions(+), 12 deletions(-)
diff --git a/refs.c b/refs.c
index 0829c55..1314a9a 100644
--- a/refs.c
+++ b/refs.c
@@ -2398,13 +2398,17 @@ static int write_packed_entry_fn(struct ref_entry
*entry, void *cb_data)
return 0;
}
-/* This should return a meaningful errno on failure */
-static int lock_packed_refs(int flags)
+static int lock_packed_refs(struct strbuf *err)
{
struct packed_ref_cache *packed_ref_cache;
- if (hold_lock_file_for_update(packlock, git_path(packed-refs),
flags) 0)
+ if (hold_lock_file_for_update(packlock, git_path(packed-refs),
+ 0) 0) {
+ if (err)
+ unable_to_lock_message(git_path(packed-refs),
+ errno, err);
return -1;
+ }
/*
* Get the current packed-refs while holding the lock. If the
* packed-refs file has been modified since we last read it,
@@ -2592,11 +2596,14 @@ static void prune_refs(struct ref_to_prune *r)
int pack_refs(unsigned int flags)
{
struct pack_refs_cb_data cbdata;
+ struct strbuf err = STRBUF_INIT;
memset(cbdata, 0, sizeof(cbdata));
cbdata.flags = flags;
- lock_packed_refs(LOCK_DIE_ON_ERROR);
+ if (lock_packed_refs(err))
+ die(%s, err.buf);
+
cbdata.packed_refs = get_packed_refs(ref_cache);
do_for_each_entry_in_dir(get_loose_refs(ref_cache), 0,
@@ -3789,10 +3796,7 @@ int transaction_commit(struct transaction *transaction,
}
/* Lock packed refs during commit */
- if (lock_packed_refs(0)) {
- if (err)
- unable_to_lock_message(git_path(packed-refs),
- errno, err);
+ if (lock_packed_refs(err)) {
ret = -1;
goto cleanup;
}
@@ -3847,10 +3851,7 @@ int transaction_commit(struct transaction *transaction,
goto cleanup;
}
/* lock the packed refs again so no one can change it */
- if (lock_packed_refs(0)) {
- if (err)
- unable_to_lock_message(git_path(packed-refs),
- errno, err);
+ if (lock_packed_refs(err)) {
ret = -1;
goto cleanup;
}
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 12/16] refs.c: make the *_packed_refs functions static
We no longer need to expose the lock/add/commit/rollback functions
for packed refs anymore so make them static and remove them from the
public api.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 8
refs.h | 30 --
2 files changed, 4 insertions(+), 34 deletions(-)
diff --git a/refs.c b/refs.c
index 2c6b0f6..eee9a14 100644
--- a/refs.c
+++ b/refs.c
@@ -1229,7 +1229,7 @@ static struct ref_dir *get_packed_refs(struct ref_cache
*refs)
return get_packed_ref_dir(get_packed_ref_cache(refs));
}
-void add_packed_ref(const char *refname, const unsigned char *sha1)
+static void add_packed_ref(const char *refname, const unsigned char *sha1)
{
struct packed_ref_cache *packed_ref_cache =
get_packed_ref_cache(ref_cache);
@@ -2398,7 +2398,7 @@ static int write_packed_entry_fn(struct ref_entry *entry,
void *cb_data)
}
/* This should return a meaningful errno on failure */
-int lock_packed_refs(int flags)
+static int lock_packed_refs(int flags)
{
struct packed_ref_cache *packed_ref_cache;
@@ -2421,7 +2421,7 @@ int lock_packed_refs(int flags)
* Commit the packed refs changes.
* On error we must make sure that errno contains a meaningful value.
*/
-int commit_packed_refs(void)
+static int commit_packed_refs(void)
{
struct packed_ref_cache *packed_ref_cache =
get_packed_ref_cache(ref_cache);
@@ -2450,7 +2450,7 @@ int commit_packed_refs(void)
return error;
}
-void rollback_packed_refs(void)
+static void rollback_packed_refs(void)
{
struct packed_ref_cache *packed_ref_cache =
get_packed_ref_cache(ref_cache);
diff --git a/refs.h b/refs.h
index ce290b1..70a2819 100644
--- a/refs.h
+++ b/refs.h
@@ -120,36 +120,6 @@ extern void warn_dangling_symref(FILE *fp, const char
*msg_fmt, const char *refn
extern void warn_dangling_symrefs(FILE *fp, const char *msg_fmt, const struct
string_list *refnames);
/*
- * Lock the packed-refs file for writing. Flags is passed to
- * hold_lock_file_for_update(). Return 0 on success.
- * Errno is set to something meaningful on error.
- */
-extern int lock_packed_refs(int flags);
-
-/*
- * Add a reference to the in-memory packed reference cache. This may
- * only be called while the packed-refs file is locked (see
- * lock_packed_refs()). To actually write the packed-refs file, call
- * commit_packed_refs().
- */
-extern void add_packed_ref(const char *refname, const unsigned char *sha1);
-
-/*
- * Write the current version of the packed refs cache from memory to
- * disk. The packed-refs file must already be locked for writing (see
- * lock_packed_refs()). Return zero on success.
- * Sets errno to something meaningful on error.
- */
-extern int commit_packed_refs(void);
-
-/*
- * Rollback the lockfile for the packed-refs file, and discard the
- * in-memory packed reference cache. (The packed-refs file will be
- * read anew if it is needed again after this function is called.)
- */
-extern void rollback_packed_refs(void);
-
-/*
* Flags for controlling behaviour of pack_refs()
* PACK_REFS_PRUNE: Prune loose refs after packing
* PACK_REFS_ALL: Pack _all_ refs, not just tags and already packed refs
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 13/16] refs.c: replace the onerr argument in update_ref with a strbuf err
Get rid of the action_on_err enum and replace the action argument to
update_ref with a strbuf *err for error reporting.
Update all callers to the new api including two callers in transport*.c
which used the literal 0 instead of an enum.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/checkout.c | 7 +--
builtin/clone.c | 20
builtin/merge.c | 20 +---
builtin/notes.c | 24 ++--
builtin/reset.c | 12
builtin/update-ref.c | 7 +--
notes-cache.c| 2 +-
notes-utils.c| 5 +++--
refs.c | 14 +++---
refs.h | 10 ++
transport-helper.c | 7 ++-
transport.c | 9 ++---
12 files changed, 78 insertions(+), 59 deletions(-)
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 8550b6d..60a68f7 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -584,6 +584,8 @@ static void update_refs_for_switch(const struct
checkout_opts *opts,
{
struct strbuf msg = STRBUF_INIT;
const char *old_desc, *reflog_msg;
+ struct strbuf err = STRBUF_INIT;
+
if (opts-new_branch) {
if (opts-new_orphan_branch) {
if (opts-new_branch_log !log_all_ref_updates) {
@@ -621,8 +623,9 @@ static void update_refs_for_switch(const struct
checkout_opts *opts,
if (!strcmp(new-name, HEAD) !new-path !opts-force_detach) {
/* Nothing to do. */
} else if (opts-force_detach || !new-path) { /* No longer on any
branch. */
- update_ref(msg.buf, HEAD, new-commit-object.sha1, NULL,
- REF_NODEREF, UPDATE_REFS_DIE_ON_ERR);
+ if (update_ref(msg.buf, HEAD, new-commit-object.sha1, NULL,
+ REF_NODEREF, err))
+ die(%s, err.buf);
if (!opts-quiet) {
if (old-path advice_detached_head)
detach_advice(new-name);
diff --git a/builtin/clone.c b/builtin/clone.c
index bb2c058..5577b5b 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -522,6 +522,7 @@ static void write_remote_refs(const struct ref *local_refs)
static void write_followtags(const struct ref *refs, const char *msg)
{
const struct ref *ref;
+ struct strbuf err = STRBUF_INIT;
for (ref = refs; ref; ref = ref-next) {
if (!starts_with(ref-name, refs/tags/))
continue;
@@ -529,8 +530,9 @@ static void write_followtags(const struct ref *refs, const
char *msg)
continue;
if (!has_sha1_file(ref-old_sha1))
continue;
- update_ref(msg, ref-name, ref-old_sha1,
- NULL, 0, UPDATE_REFS_DIE_ON_ERR);
+ if (update_ref(msg, ref-name, ref-old_sha1,
+ NULL, 0, err))
+ die(%s, err.buf);
}
}
@@ -593,28 +595,30 @@ static void update_remote_refs(const struct ref *refs,
static void update_head(const struct ref *our, const struct ref *remote,
const char *msg)
{
+ struct strbuf err = STRBUF_INIT;
const char *head;
if (our skip_prefix(our-name, refs/heads/, head)) {
/* Local default branch link */
create_symref(HEAD, our-name, NULL);
if (!option_bare) {
- update_ref(msg, HEAD, our-old_sha1, NULL, 0,
- UPDATE_REFS_DIE_ON_ERR);
+ update_ref(msg, HEAD, our-old_sha1, NULL, 0, err);
install_branch_config(0, head, option_origin,
our-name);
}
} else if (our) {
struct commit *c = lookup_commit_reference(our-old_sha1);
/* --branch specifies a non-branch (i.e. tags), detach HEAD */
- update_ref(msg, HEAD, c-object.sha1,
- NULL, REF_NODEREF, UPDATE_REFS_DIE_ON_ERR);
+ if (update_ref(msg, HEAD, c-object.sha1,
+ NULL, REF_NODEREF, err))
+ die(%s, err.buf);
} else if (remote) {
/*
* We know remote HEAD points to a non-branch, or
* HEAD points to a branch but we don't know which one.
* Detach HEAD in all these cases.
*/
- update_ref(msg, HEAD, remote-old_sha1,
- NULL, REF_NODEREF, UPDATE_REFS_DIE_ON_ERR);
+ if (update_ref(msg, HEAD, remote-old_sha1,
+NULL, REF_NODEREF, err))
+ die(%s, err.buf);
}
}
diff --git a/builtin/merge.c b/builtin/merge.c
index bebbe5b..a787b6a 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -396,9 +396,11 @@ static void finish(struct commit *head_commit
[PATCH v3 4/7] push.c: add an --atomic-push argument
Add a command line argument to the git push command to request atomic
pushes.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
Documentation/git-push.txt | 7 ++-
builtin/push.c | 2 ++
transport.c| 1 +
transport.h| 1 +
4 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-push.txt b/Documentation/git-push.txt
index 21b3f29..04de8d8 100644
--- a/Documentation/git-push.txt
+++ b/Documentation/git-push.txt
@@ -9,7 +9,7 @@ git-push - Update remote refs along with associated objects
SYNOPSIS
[verse]
-'git push' [--all | --mirror | --tags] [--follow-tags] [-n | --dry-run]
[--receive-pack=git-receive-pack]
+'git push' [--all | --mirror | --tags] [--follow-tags] [--atomic-push] [-n |
--dry-run] [--receive-pack=git-receive-pack]
[--repo=repository] [-f | --force] [--prune] [-v | --verbose]
[-u | --set-upstream] [--signed]
[--force-with-lease[=refname[:expect]]]
@@ -136,6 +136,11 @@ already exists on the remote side.
logged. See linkgit:git-receive-pack[1] for the details
on the receiving end.
+--atomic-push::
+ Try using atomic push. If atomic push is negotiated with the server
+ then any push covering multiple refs will be atomic. Either all
+ refs are updated, or on error, no refs are updated.
+
--receive-pack=git-receive-pack::
--exec=git-receive-pack::
Path to the 'git-receive-pack' program on the remote
diff --git a/builtin/push.c b/builtin/push.c
index ae56f73..0b9f21a 100644
--- a/builtin/push.c
+++ b/builtin/push.c
@@ -507,6 +507,8 @@ int cmd_push(int argc, const char **argv, const char
*prefix)
OPT_BIT(0, follow-tags, flags, N_(push missing but relevant
tags),
TRANSPORT_PUSH_FOLLOW_TAGS),
OPT_BIT(0, signed, flags, N_(GPG sign the push),
TRANSPORT_PUSH_CERT),
+ OPT_BIT(0, atomic-push, flags, N_(use atomic push, if
available),
+ TRANSPORT_ATOMIC_PUSH),
OPT_END()
};
diff --git a/transport.c b/transport.c
index 2111986..cd2b63a 100644
--- a/transport.c
+++ b/transport.c
@@ -833,6 +833,7 @@ static int git_transport_push(struct transport *transport,
struct ref *remote_re
args.dry_run = !!(flags TRANSPORT_PUSH_DRY_RUN);
args.porcelain = !!(flags TRANSPORT_PUSH_PORCELAIN);
args.push_cert = !!(flags TRANSPORT_PUSH_CERT);
+ args.use_atomic_push = !!(flags TRANSPORT_ATOMIC_PUSH);
args.url = transport-url;
ret = send_pack(args, data-fd, data-conn, remote_refs,
diff --git a/transport.h b/transport.h
index 3e0091e..25fa1da 100644
--- a/transport.h
+++ b/transport.h
@@ -125,6 +125,7 @@ struct transport {
#define TRANSPORT_PUSH_NO_HOOK 512
#define TRANSPORT_PUSH_FOLLOW_TAGS 1024
#define TRANSPORT_PUSH_CERT 2048
+#define TRANSPORT_ATOMIC_PUSH 4096
#define TRANSPORT_SUMMARY_WIDTH (2 * DEFAULT_ABBREV + 3)
#define TRANSPORT_SUMMARY(x) (int)(TRANSPORT_SUMMARY_WIDTH + strlen(x) -
gettext_width(x)), (x)
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 2/7] send-pack.c: add an --atomic-push command line argument
Fixed. Thanks
On Tue, Nov 4, 2014 at 2:17 PM, Stefan Beller sbel...@google.com wrote:
On Mon, Nov 3, 2014 at 11:12 AM, Ronnie Sahlberg sahlb...@google.com wrote:
This adds support to send-pack to to negotiate and use atomic pushes
/s/to to/to/
iff the server supports it. Atomic pushes are activated by a new command
line flag --atomic-push.
In order to do this we also need to change the semantics for send_pack()
slightly. The existing send_pack() function actually don't sent all the
refs back to the server when multiple refs are involved, for example
when using --all. Several of the failure modes for pushes can already be
detected locally in the send_pack client based on the information from the
initial server side list of all the refs as generated by receive-pack.
Any such refs that we thus know would fail to push are thus pruned from
the list of refs we send to the server to update.
For atomic pushes, we have to deal thus with both failures that are detected
locally as well as failures that are reported back from the server. In order
to do so we treat all local failures as push failures too.
We introduce a new status code REF_STATUS_ATOMIC_PUSH_FAILED so we can
flag all refs that we would normally have tried to push to the server
but we did not due to local failures. This is to improve the error message
back to the end user to flag that these refs failed to update since the
atomic push operation failed.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
Documentation/git-send-pack.txt | 7 ++-
builtin/send-pack.c | 6 +-
remote.h| 3 ++-
send-pack.c | 39 ++-
send-pack.h | 1 +
transport.c | 4
6 files changed, 52 insertions(+), 8 deletions(-)
diff --git a/Documentation/git-send-pack.txt
b/Documentation/git-send-pack.txt
index 2a0de42..9296587 100644
--- a/Documentation/git-send-pack.txt
+++ b/Documentation/git-send-pack.txt
@@ -9,7 +9,7 @@ git-send-pack - Push objects over Git protocol to another
repository
SYNOPSIS
[verse]
-'git send-pack' [--all] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin]
[host:]directory [ref...]
+'git send-pack' [--all] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin] [--atomic-push]
[host:]directory [ref...]
DESCRIPTION
---
@@ -62,6 +62,11 @@ be in a separate packet, and the list must end with a
flush packet.
Send a thin pack, which records objects in deltified form based
on objects not included in the pack to reduce network traffic.
+--atomic-push::
+ Use an atomic transaction for updating the refs. If any of the refs
+ fails to update then the entire push will fail without changing any
+ refs.
+
host::
A remote host to house the repository. When this
part is specified, 'git-receive-pack' is invoked via
diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index b564a77..93cb17c 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -13,7 +13,7 @@
#include sha1-array.h
static const char send_pack_usage[] =
-git send-pack [--all | --mirror] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin]
[host:]directory [ref...]\n
+git send-pack [--all | --mirror] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin] [--atomic-push]
[host:]directory [ref...]\n
--all and explicit ref specification are mutually exclusive.;
static struct send_pack_args args;
@@ -170,6 +170,10 @@ int cmd_send_pack(int argc, const char **argv, const
char *prefix)
args.use_thin_pack = 1;
continue;
}
+ if (!strcmp(arg, --atomic-push)) {
+ args.use_atomic_push = 1;
+ continue;
+ }
if (!strcmp(arg, --stateless-rpc)) {
args.stateless_rpc = 1;
continue;
diff --git a/remote.h b/remote.h
index 8b62efd..f346524 100644
--- a/remote.h
+++ b/remote.h
@@ -115,7 +115,8 @@ struct ref {
REF_STATUS_REJECT_SHALLOW,
REF_STATUS_UPTODATE,
REF_STATUS_REMOTE_REJECT,
- REF_STATUS_EXPECTING_REPORT
+ REF_STATUS_EXPECTING_REPORT,
+ REF_STATUS_ATOMIC_PUSH_FAILED
} status;
char *remote_status;
struct ref *peer_ref; /* when renaming */
diff --git a/send-pack.c b/send-pack.c
index 1ccc84c..08602a8 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -190,7 +190,7 @@ static void advertise_shallow_grafts_buf(struct strbuf
*sb)
for_each_commit_graft(advertise_shallow_grafts_cb, sb
[PATCH v2 11/15] reflog.c: use a reflog transaction when writing during expire
Use a transaction for all updates during expire_reflog.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
builtin/reflog.c | 85
refs.c | 4 +--
refs.h | 2 +-
3 files changed, 40 insertions(+), 51 deletions(-)
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 2d85d26..6bb7454 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -32,8 +32,11 @@ struct cmd_reflog_expire_cb {
int recno;
};
+static struct strbuf err = STRBUF_INIT;
+
struct expire_reflog_cb {
- FILE *newlog;
+ struct transaction *t;
+ const char *refname;
enum {
UE_NORMAL,
UE_ALWAYS,
@@ -316,20 +319,18 @@ static int expire_reflog_ent(unsigned char *osha1,
unsigned char *nsha1,
if (cb-cmd-recno --(cb-cmd-recno) == 0)
goto prune;
- if (cb-newlog) {
- char sign = (tz 0) ? '-' : '+';
- int zone = (tz 0) ? (-tz) : tz;
- fprintf(cb-newlog, %s %s %s %lu %c%04d\t%s,
- sha1_to_hex(osha1), sha1_to_hex(nsha1),
- email, timestamp, sign, zone,
- message);
+ if (cb-t) {
+ if (transaction_update_reflog(cb-t, cb-refname, nsha1, osha1,
+ email, timestamp, tz, message, 0,
+ err))
+ return -1;
hashcpy(cb-last_kept_sha1, nsha1);
}
if (cb-cmd-verbose)
printf(keep %s, message);
return 0;
prune:
- if (!cb-newlog)
+ if (!cb-t)
printf(would prune %s, message);
else if (cb-cmd-verbose)
printf(prune %s, message);
@@ -353,29 +354,26 @@ static int expire_reflog(const char *ref, const unsigned
char *sha1, int unused,
{
struct cmd_reflog_expire_cb *cmd = cb_data;
struct expire_reflog_cb cb;
- struct ref_lock *lock;
- char *log_file, *newlog_path = NULL;
struct commit *tip_commit;
struct commit_list *tips;
int status = 0;
memset(cb, 0, sizeof(cb));
+ cb.refname = ref;
- /*
-* we take the lock for the ref itself to prevent it from
-* getting updated.
-*/
- lock = lock_any_ref_for_update(ref, sha1, 0, NULL);
- if (!lock)
- return error(cannot lock ref '%s', ref);
- log_file = git_pathdup(logs/%s, ref);
if (!reflog_exists(ref))
goto finish;
- if (!cmd-dry_run) {
- newlog_path = git_pathdup(logs/%s.lock, ref);
- cb.newlog = fopen(newlog_path, w);
+ cb.t = transaction_begin(err);
+ if (!cb.t) {
+ status |= error(%s, err.buf);
+ goto cleanup;
+ }
+ if (transaction_update_reflog(cb.t, cb.refname, null_sha1, null_sha1,
+ NULL, 0, 0, NULL, REFLOG_TRUNCATE,
+ err)) {
+ status |= error(%s, err.buf);
+ goto cleanup;
}
-
cb.cmd = cmd;
if (!cmd-expire_unreachable || !strcmp(ref, HEAD)) {
@@ -407,7 +405,10 @@ static int expire_reflog(const char *ref, const unsigned
char *sha1, int unused,
mark_reachable(cb);
}
- for_each_reflog_ent(ref, expire_reflog_ent, cb);
+ if (for_each_reflog_ent(ref, expire_reflog_ent, cb)) {
+ status |= error(%s, err.buf);
+ goto cleanup;
+ }
if (cb.unreachable_expire_kind != UE_ALWAYS) {
if (cb.unreachable_expire_kind == UE_HEAD) {
@@ -420,32 +421,20 @@ static int expire_reflog(const char *ref, const unsigned
char *sha1, int unused,
}
}
finish:
- if (cb.newlog) {
- if (fclose(cb.newlog)) {
- status |= error(%s: %s, strerror(errno),
- newlog_path);
- unlink(newlog_path);
- } else if (cmd-updateref
- (write_in_full(lock-lock_fd,
- sha1_to_hex(cb.last_kept_sha1), 40) != 40 ||
-write_str_in_full(lock-lock_fd, \n) != 1 ||
-close_ref(lock) 0)) {
- status |= error(Couldn't write %s,
- lock-lk-filename.buf);
- unlink(newlog_path);
- } else if (rename(newlog_path, log_file)) {
- status |= error(cannot rename %s to %s,
- newlog_path, log_file);
- unlink(newlog_path);
- } else if (cmd-updateref commit_ref(lock)) {
- status |= error(Couldn't set %s, lock-ref_name
[PATCH v2 03/15] refs.c: rename the transaction functions
Rename the transaction functions. Remove the leading ref_ from the
names and append _ref to the names for functions that create/delete/
update sha1 refs.
This also makes the names more appropriate for future changes that have been
discussed when the transactions could also operate on non ref objects. Such as
..git/config and .gitmodule.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
branch.c | 14 -
builtin/commit.c | 10 +++
builtin/fetch.c| 12
builtin/receive-pack.c | 13 -
builtin/replace.c | 10 +++
builtin/tag.c | 10 +++
builtin/update-ref.c | 26 -
fast-import.c | 22 +++---
refs.c | 78 +-
refs.h | 36 +++
sequencer.c| 12
walker.c | 10 +++
12 files changed, 126 insertions(+), 127 deletions(-)
diff --git a/branch.c b/branch.c
index 4bab55a..d9b1174 100644
--- a/branch.c
+++ b/branch.c
@@ -279,17 +279,17 @@ void create_branch(const char *head,
log_all_ref_updates = 1;
if (!dont_change_ref) {
- struct ref_transaction *transaction;
+ struct transaction *transaction;
struct strbuf err = STRBUF_INIT;
- transaction = ref_transaction_begin(err);
+ transaction = transaction_begin(err);
if (!transaction ||
- ref_transaction_update(transaction, ref.buf, sha1,
- null_sha1, 0, !forcing, msg, err) ||
- ref_transaction_commit(transaction, err))
+ transaction_update_ref(transaction, ref.buf, sha1,
+ null_sha1, 0, !forcing, msg,
+ err) ||
+ transaction_commit(transaction, err))
die(%s, err.buf);
- ref_transaction_free(transaction);
- strbuf_release(err);
+ transaction_free(transaction);
}
if (real_ref track)
diff --git a/builtin/commit.c b/builtin/commit.c
index e108c53..f50b7df 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1673,7 +1673,7 @@ int cmd_commit(int argc, const char **argv, const char
*prefix)
struct stat statbuf;
struct commit *current_head = NULL;
struct commit_extra_header *extra = NULL;
- struct ref_transaction *transaction;
+ struct transaction *transaction;
struct strbuf err = STRBUF_INIT;
if (argc == 2 !strcmp(argv[1], -h))
@@ -1804,17 +1804,17 @@ int cmd_commit(int argc, const char **argv, const char
*prefix)
strbuf_insert(sb, 0, reflog_msg, strlen(reflog_msg));
strbuf_insert(sb, strlen(reflog_msg), : , 2);
- transaction = ref_transaction_begin(err);
+ transaction = transaction_begin(err);
if (!transaction ||
- ref_transaction_update(transaction, HEAD, sha1,
+ transaction_update_ref(transaction, HEAD, sha1,
current_head
? current_head-object.sha1 : NULL,
0, !!current_head, sb.buf, err) ||
- ref_transaction_commit(transaction, err)) {
+ transaction_commit(transaction, err)) {
rollback_index_files();
die(%s, err.buf);
}
- ref_transaction_free(transaction);
+ transaction_free(transaction);
unlink(git_path(CHERRY_PICK_HEAD));
unlink(git_path(REVERT_HEAD));
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 6ffd023..323ffd9 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -404,7 +404,7 @@ static int s_update_ref(const char *action,
{
char msg[1024];
char *rla = getenv(GIT_REFLOG_ACTION);
- struct ref_transaction *transaction;
+ struct transaction *transaction;
struct strbuf err = STRBUF_INIT;
int ret, df_conflict = 0;
@@ -414,23 +414,23 @@ static int s_update_ref(const char *action,
rla = default_rla.buf;
snprintf(msg, sizeof(msg), %s: %s, rla, action);
- transaction = ref_transaction_begin(err);
+ transaction = transaction_begin(err);
if (!transaction ||
- ref_transaction_update(transaction, ref-name, ref-new_sha1,
+ transaction_update_ref(transaction, ref-name, ref-new_sha1,
ref-old_sha1, 0, check_old, msg, err))
goto fail;
- ret = ref_transaction_commit(transaction, err);
+ ret = transaction_commit(transaction, err);
if (ret) {
df_conflict = (ret == TRANSACTION_NAME_CONFLICT);
goto fail;
}
- ref_transaction_free(transaction
[PATCH v2 15/15] refs.c: allow deleting refs with a broken sha1
Add back support to make it possible to delete refs that have a broken
sha1.
Add new internal flags REF_ALLOW_BROKEN and RESOLVE_REF_ALLOW_BAD_SHA1
to pass intent from branch.c that we are willing to allow
resolve_ref_unsafe and lock_ref_sha1_basic to allow broken refs.
Since these refs can not actually be resolved to a sha1, they instead resolve
to null_sha1 when these flags are used.
For example, the ref:
echo Broken ref .git/refs/heads/foo-broken-1
can now be deleted using git branch -d foo-broken-1
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
builtin/branch.c| 5 +++--
cache.h | 7 +++
refs.c | 6 ++
refs.h | 6 --
t/t1402-check-ref-format.sh | 8
5 files changed, 28 insertions(+), 4 deletions(-)
diff --git a/builtin/branch.c b/builtin/branch.c
index 3b79c50..04f57d4 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -238,7 +238,8 @@ static int delete_branches(int argc, const char **argv, int
force, int kinds,
target = resolve_ref_unsafe(name,
RESOLVE_REF_READING
| RESOLVE_REF_NO_RECURSE
- | RESOLVE_REF_ALLOW_BAD_NAME,
+ | RESOLVE_REF_ALLOW_BAD_NAME
+ | RESOLVE_REF_ALLOW_BAD_SHA1,
sha1, flags);
if (!target) {
error(remote_branch
@@ -255,7 +256,7 @@ static int delete_branches(int argc, const char **argv, int
force, int kinds,
continue;
}
- if (delete_ref(name, sha1, REF_NODEREF)) {
+ if (delete_ref(name, sha1, REF_NODEREF|REF_ALLOW_BROKEN)) {
error(remote_branch
? _(Error deleting remote branch '%s')
: _(Error deleting branch '%s'),
diff --git a/cache.h b/cache.h
index 99ed096..61e61af 100644
--- a/cache.h
+++ b/cache.h
@@ -1000,10 +1000,17 @@ extern int read_ref(const char *refname, unsigned char
*sha1);
* resolved. The function returns NULL for such ref names.
* Caps and underscores refers to the special refs, such as HEAD,
* FETCH_HEAD and friends, that all live outside of the refs/ directory.
+ *
+ * RESOLVE_REF_ALLOW_BAD_SHA1 when this flag is set and the ref contains
+ * an invalid sha1, resolve_ref_unsafe will clear the sha1 argument,
+ * set the REF_ISBROKEN flag and return the refname.
+ * This allows using resolve_ref_unsafe to check for existence of such
+ * broken refs.
*/
#define RESOLVE_REF_READING 0x01
#define RESOLVE_REF_NO_RECURSE 0x02
#define RESOLVE_REF_ALLOW_BAD_NAME 0x04
+#define RESOLVE_REF_ALLOW_BAD_SHA1 0x08
extern const char *resolve_ref_unsafe(const char *ref, int resolve_flags,
unsigned char *sha1, int *flags);
extern char *resolve_refdup(const char *ref, int resolve_flags, unsigned char
*sha1, int *flags);
diff --git a/refs.c b/refs.c
index 6d50a32..f1ca9e4 100644
--- a/refs.c
+++ b/refs.c
@@ -1584,6 +1584,10 @@ const char *resolve_ref_unsafe(const char *refname, int
resolve_flags, unsigned
(buffer[40] != '\0' !isspace(buffer[40]))) {
if (flags)
*flags |= REF_ISBROKEN;
+ if (resolve_flags RESOLVE_REF_ALLOW_BAD_SHA1)
{
+ hashclr(sha1);
+ return refname;
+ }
errno = EINVAL;
return NULL;
}
@@ -2265,6 +2269,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char
*refname,
if (flags REF_NODEREF)
resolve_flags |= RESOLVE_REF_NO_RECURSE;
}
+ if (flags REF_ALLOW_BROKEN)
+ resolve_flags |= RESOLVE_REF_ALLOW_BAD_SHA1;
refname = resolve_ref_unsafe(refname, resolve_flags,
lock-old_sha1, type);
diff --git a/refs.h b/refs.h
index 721e21f..2e97f4f 100644
--- a/refs.h
+++ b/refs.h
@@ -185,11 +185,13 @@ extern int peel_ref(const char *refname, unsigned char
*sha1);
* REF_NODEREF: act on the ref directly, instead of dereferencing
* symbolic references.
* REF_DELETING: tolerate broken refs
+ * REF_ALLOW_BROKEN: allow locking refs that are broken.
*
* Flags = 0x100 are reserved for internal use.
*/
-#define REF_NODEREF0x01
-#define REF_DELETING 0x02
+#define REF_NODEREF0x01
+#define REF_DELETING 0x02
+#define REF_ALLOW_BROKEN 0x04
/** Reads log for the value of ref during at_time. **/
extern int read_ref_at(const char *refname, unsigned int
[PATCH v2 00/15] ref-transactions-reflog
List, Please find a patch that updates the reflog handling to use transactions. This patch series has previously been sent to the list but is now rebased on the current content of next which contains ref changes we depend on in this series. This series converts the reflog handling and builtin/reflog.c to use a transaction for both the ref as well as the reflog updates. As a side effect of this it simplifies the reflog marshalling code so that we only have one place where we marshall the entry. It also means that we can remove several functions from the public api towards the end of the series since we no longer need those functions. This series can also be found at https://github.com/rsahlberg/git/tree/ref-transactions-reflog Version 2: - Clarify the commit message to refs.c: only write reflog update if msg is non-NULL to highlight that msg==NULL and REFLOG_TRUNCATE control othogonal features. - Improve the comments for the transaction flags field in refs.c: allow multiple reflog updates during a single transaction. Ronnie Sahlberg (15): refs.c make ref_transaction_create a wrapper to ref_transaction_update refs.c: make ref_transaction_delete a wrapper for ref_transaction_update refs.c: rename the transaction functions refs.c: add a new update_type field to ref_update refs.c: add a function to append a reflog entry to a fd copy.c: make copy_fd preserve meaningful errno refs.c: add a transaction function to append a reflog entry refs.c: add a flag to allow reflog updates to truncate the log refs.c: only write reflog update if msg is non-NULL refs.c: allow multiple reflog updates during a single transaction reflog.c: use a reflog transaction when writing during expire refs.c: rename log_ref_setup to create_reflog refs.c: make unlock_ref/close_ref/commit_ref static refs.c: remove lock_any_ref_for_update refs.c: allow deleting refs with a broken sha1 branch.c| 14 +- builtin/branch.c| 5 +- builtin/checkout.c | 8 +- builtin/commit.c| 10 +- builtin/fetch.c | 12 +- builtin/receive-pack.c | 13 +- builtin/reflog.c| 85 -- builtin/replace.c | 10 +- builtin/tag.c | 10 +- builtin/update-ref.c| 26 +-- cache.h | 7 + copy.c | 15 +- fast-import.c | 22 +-- refs.c | 403 +--- refs.h | 87 +- sequencer.c | 12 +- t/t1402-check-ref-format.sh | 8 + walker.c| 10 +- 18 files changed, 450 insertions(+), 307 deletions(-) -- 2.1.0.rc2.206.gedb03e5 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 13/15] refs.c: make unlock_ref/close_ref/commit_ref static
unlock|close|commit_ref can be made static since there are no more external
callers.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 24
refs.h | 9 -
2 files changed, 12 insertions(+), 21 deletions(-)
diff --git a/refs.c b/refs.c
index 0b46bd9..e4ad4f4 100644
--- a/refs.c
+++ b/refs.c
@@ -2096,6 +2096,16 @@ int refname_match(const char *abbrev_name, const char
*full_name)
return 0;
}
+static void unlock_ref(struct ref_lock *lock)
+{
+ /* Do not free lock-lk -- atexit() still looks at them */
+ if (lock-lk)
+ rollback_lock_file(lock-lk);
+ free(lock-ref_name);
+ free(lock-orig_ref_name);
+ free(lock);
+}
+
/* This function should make sure errno is meaningful on error */
static struct ref_lock *verify_lock(struct ref_lock *lock,
const unsigned char *old_sha1, int mustexist)
@@ -2894,7 +2904,7 @@ int rename_ref(const char *oldrefname, const char
*newrefname, const char *logms
return 1;
}
-int close_ref(struct ref_lock *lock)
+static int close_ref(struct ref_lock *lock)
{
if (close_lock_file(lock-lk))
return -1;
@@ -2902,7 +2912,7 @@ int close_ref(struct ref_lock *lock)
return 0;
}
-int commit_ref(struct ref_lock *lock)
+static int commit_ref(struct ref_lock *lock)
{
if (commit_lock_file(lock-lk))
return -1;
@@ -2910,16 +2920,6 @@ int commit_ref(struct ref_lock *lock)
return 0;
}
-void unlock_ref(struct ref_lock *lock)
-{
- /* Do not free lock-lk -- atexit() still looks at them */
- if (lock-lk)
- rollback_lock_file(lock-lk);
- free(lock-ref_name);
- free(lock-orig_ref_name);
- free(lock);
-}
-
/*
* copy the reflog message msg to buf, which has been allocated sufficiently
* large, while cleaning up the whitespaces. Especially, convert LF to space,
diff --git a/refs.h b/refs.h
index 17e3a3c..025e2cb 100644
--- a/refs.h
+++ b/refs.h
@@ -198,15 +198,6 @@ extern struct ref_lock *lock_any_ref_for_update(const char
*refname,
const unsigned char *old_sha1,
int flags, int *type_p);
-/** Close the file descriptor owned by a lock and return the status */
-extern int close_ref(struct ref_lock *lock);
-
-/** Close and commit the ref locked by the lock */
-extern int commit_ref(struct ref_lock *lock);
-
-/** Release any lock taken but not written. **/
-extern void unlock_ref(struct ref_lock *lock);
-
/** Reads log for the value of ref during at_time. **/
extern int read_ref_at(const char *refname, unsigned int flags,
unsigned long at_time, int cnt,
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 12/15] refs.c: rename log_ref_setup to create_reflog
log_ref_setup is used to do several semi-related things :
* sometimes it will create a new reflog including missing parent directories
and cleaning up any conflicting stale directories in the path.
* fill in a filename buffer for the full path to the reflog.
* unconditionally re-adjust the permissions for the file.
This function is only called from two places: checkout.c where it is always
used to create a reflog and refs.c:log_ref_write where it sometimes are
used to create a reflog and sometimes just used to fill in the filename.
Rename log_ref_setup to create_reflog and change it to only take the
refname as argument to make its signature similar to delete_reflog and
reflog_exists. Change create_reflog to ignore log_all_ref_updates and
unconditionally create the reflog when called. Since checkout.c always
wants to create a reflog we can call create_reflog directly and avoid the
temp-and-log_all_ref_update dance.
In log_ref_write, only call create_reflog iff we want to create a reflog
and if the reflog does not yet exist. This means that for the common case
where the log already exists we now only need to perform a single lstat()
instead of a open(O_CREAT)+lstat()+close().
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
builtin/checkout.c | 8 +---
refs.c | 22 +-
refs.h | 8 +++-
3 files changed, 17 insertions(+), 21 deletions(-)
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 5410dac..8550b6d 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -587,19 +587,13 @@ static void update_refs_for_switch(const struct
checkout_opts *opts,
if (opts-new_branch) {
if (opts-new_orphan_branch) {
if (opts-new_branch_log !log_all_ref_updates) {
- int temp;
- char log_file[PATH_MAX];
char *ref_name = mkpath(refs/heads/%s,
opts-new_orphan_branch);
- temp = log_all_ref_updates;
- log_all_ref_updates = 1;
- if (log_ref_setup(ref_name, log_file,
sizeof(log_file))) {
+ if (create_reflog(ref_name)) {
fprintf(stderr, _(Can not do reflog
for '%s'\n),
opts-new_orphan_branch);
- log_all_ref_updates = temp;
return;
}
- log_all_ref_updates = temp;
}
}
else
diff --git a/refs.c b/refs.c
index a4bdfaf..0b46bd9 100644
--- a/refs.c
+++ b/refs.c
@@ -2947,16 +2947,16 @@ static int copy_msg(char *buf, const char *msg)
}
/* This function must set a meaningful errno on failure */
-int log_ref_setup(const char *refname, char *logfile, int bufsize)
+int create_reflog(const char *refname)
{
int logfd, oflags = O_APPEND | O_WRONLY;
+ char logfile[PATH_MAX];
- git_snpath(logfile, bufsize, logs/%s, refname);
- if (log_all_ref_updates
- (starts_with(refname, refs/heads/) ||
-starts_with(refname, refs/remotes/) ||
-starts_with(refname, refs/notes/) ||
-!strcmp(refname, HEAD))) {
+ git_snpath(logfile, sizeof(logfile), logs/%s, refname);
+ if (starts_with(refname, refs/heads/) ||
+ starts_with(refname, refs/remotes/) ||
+ starts_with(refname, refs/notes/) ||
+ !strcmp(refname, HEAD)) {
if (safe_create_leading_directories(logfile) 0) {
int save_errno = errno;
error(unable to create directory for %s, logfile);
@@ -3025,16 +3025,20 @@ static int log_ref_write_fd(int fd, const unsigned char
*old_sha1,
static int log_ref_write(const char *refname, const unsigned char *old_sha1,
const unsigned char *new_sha1, const char *msg)
{
- int logfd, result, oflags = O_APPEND | O_WRONLY;
+ int logfd, result = 0, oflags = O_APPEND | O_WRONLY;
char log_file[PATH_MAX];
if (log_all_ref_updates 0)
log_all_ref_updates = !is_bare_repository();
- result = log_ref_setup(refname, log_file, sizeof(log_file));
+ if (log_all_ref_updates !reflog_exists(refname))
+ result = create_reflog(refname);
+
if (result)
return result;
+ git_snpath(log_file, sizeof(log_file), logs/%s, refname);
+
logfd = open(log_file, oflags);
if (logfd 0)
return 0;
diff --git a/refs.h b/refs.h
index 9f70b89..17e3a3c 100644
--- a/refs.h
+++ b/refs.h
@@ -207,11 +207,6 @@ extern int commit_ref(struct ref_lock *lock);
/** Release any lock taken but not written. **/
extern void
[PATCH v2 04/15] refs.c: add a new update_type field to ref_update
Add a field that describes what type of update this refers to. For now
the only type is UPDATE_SHA1 but we will soon add more types.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 27 +++
1 file changed, 23 insertions(+), 4 deletions(-)
diff --git a/refs.c b/refs.c
index a1bfaa2..8803b95 100644
--- a/refs.c
+++ b/refs.c
@@ -3504,6 +3504,10 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
return retval;
}
+enum transaction_update_type {
+ UPDATE_SHA1 = 0
+};
+
/**
* Information needed for a single ref update. Set new_sha1 to the
* new value or to zero to delete the ref. To check the old value
@@ -3511,6 +3515,7 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
* value or to zero to ensure the ref does not exist before update.
*/
struct ref_update {
+ enum transaction_update_type update_type;
unsigned char new_sha1[20];
unsigned char old_sha1[20];
int flags; /* REF_NODEREF? */
@@ -3571,12 +3576,14 @@ void transaction_free(struct transaction *transaction)
}
static struct ref_update *add_update(struct transaction *transaction,
-const char *refname)
+const char *refname,
+enum transaction_update_type update_type)
{
size_t len = strlen(refname);
struct ref_update *update = xcalloc(1, sizeof(*update) + len + 1);
strcpy((char *)update-refname, refname);
+ update-update_type = update_type;
ALLOC_GROW(transaction-updates, transaction-nr + 1,
transaction-alloc);
transaction-updates[transaction-nr++] = update;
return update;
@@ -3606,7 +3613,7 @@ int transaction_update_ref(struct transaction
*transaction,
return -1;
}
- update = add_update(transaction, refname);
+ update = add_update(transaction, refname, UPDATE_SHA1);
hashcpy(update-new_sha1, new_sha1);
update-flags = flags;
update-have_old = have_old;
@@ -3684,13 +3691,17 @@ static int ref_update_reject_duplicates(struct
ref_update **updates, int n,
assert(err);
- for (i = 1; i n; i++)
+ for (i = 1; i n; i++) {
+ if (updates[i - 1]-update_type != UPDATE_SHA1 ||
+ updates[i]-update_type != UPDATE_SHA1)
+ continue;
if (!strcmp(updates[i - 1]-refname, updates[i]-refname)) {
strbuf_addf(err,
Multiple updates for ref '%s' not
allowed.,
updates[i]-refname);
return 1;
}
+ }
return 0;
}
@@ -3722,13 +3733,17 @@ int transaction_commit(struct transaction *transaction,
goto cleanup;
}
- /* Acquire all locks while verifying old values */
+ /* Acquire all ref locks while verifying old values */
for (i = 0; i n; i++) {
struct ref_update *update = updates[i];
int flags = update-flags;
+ if (update-update_type != UPDATE_SHA1)
+ continue;
+
if (is_null_sha1(update-new_sha1))
flags |= REF_DELETING;
+
update-lock = lock_ref_sha1_basic(update-refname,
(update-have_old ?
update-old_sha1 :
@@ -3750,6 +3765,8 @@ int transaction_commit(struct transaction *transaction,
for (i = 0; i n; i++) {
struct ref_update *update = updates[i];
+ if (update-update_type != UPDATE_SHA1)
+ continue;
if (!is_null_sha1(update-new_sha1)) {
if (write_ref_sha1(update-lock, update-new_sha1,
update-msg)) {
@@ -3767,6 +3784,8 @@ int transaction_commit(struct transaction *transaction,
for (i = 0; i n; i++) {
struct ref_update *update = updates[i];
+ if (update-update_type != UPDATE_SHA1)
+ continue;
if (update-lock) {
if (delete_ref_loose(update-lock, update-type, err)) {
ret = TRANSACTION_GENERIC_ERROR;
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 05/15] refs.c: add a function to append a reflog entry to a fd
Break out the code to create the string and writing it to the file
descriptor from log_ref_write and into a dedicated function log_ref_write_fd.
For now this is only used from log_ref_write but later on we will call
this function from reflog transactions too which means that we will end
up with only a single place where we write a reflog entry to a file instead
of the current two places (log_ref_write and builtin/reflog.c).
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 48 ++--
1 file changed, 30 insertions(+), 18 deletions(-)
diff --git a/refs.c b/refs.c
index 8803b95..0e11b1c 100644
--- a/refs.c
+++ b/refs.c
@@ -2990,15 +2990,37 @@ int log_ref_setup(const char *refname, char *logfile,
int bufsize)
return 0;
}
+static int log_ref_write_fd(int fd, const unsigned char *old_sha1,
+ const unsigned char *new_sha1,
+ const char *committer, const char *msg)
+{
+ int msglen, written;
+ unsigned maxlen, len;
+ char *logrec;
+
+ msglen = msg ? strlen(msg) : 0;
+ maxlen = strlen(committer) + msglen + 100;
+ logrec = xmalloc(maxlen);
+ len = sprintf(logrec, %s %s %s\n,
+ sha1_to_hex(old_sha1),
+ sha1_to_hex(new_sha1),
+ committer);
+ if (msglen)
+ len += copy_msg(logrec + len - 1, msg) - 1;
+
+ written = len = maxlen ? write_in_full(fd, logrec, len) : -1;
+ free(logrec);
+ if (written != len)
+ return -1;
+
+ return 0;
+}
+
static int log_ref_write(const char *refname, const unsigned char *old_sha1,
const unsigned char *new_sha1, const char *msg)
{
- int logfd, result, written, oflags = O_APPEND | O_WRONLY;
- unsigned maxlen, len;
- int msglen;
+ int logfd, result, oflags = O_APPEND | O_WRONLY;
char log_file[PATH_MAX];
- char *logrec;
- const char *committer;
if (log_all_ref_updates 0)
log_all_ref_updates = !is_bare_repository();
@@ -3010,19 +3032,9 @@ static int log_ref_write(const char *refname, const
unsigned char *old_sha1,
logfd = open(log_file, oflags);
if (logfd 0)
return 0;
- msglen = msg ? strlen(msg) : 0;
- committer = git_committer_info(0);
- maxlen = strlen(committer) + msglen + 100;
- logrec = xmalloc(maxlen);
- len = sprintf(logrec, %s %s %s\n,
- sha1_to_hex(old_sha1),
- sha1_to_hex(new_sha1),
- committer);
- if (msglen)
- len += copy_msg(logrec + len - 1, msg) - 1;
- written = len = maxlen ? write_in_full(logfd, logrec, len) : -1;
- free(logrec);
- if (written != len) {
+ result = log_ref_write_fd(logfd, old_sha1, new_sha1,
+ git_committer_info(0), msg);
+ if (result) {
int save_errno = errno;
close(logfd);
error(Unable to append to %s, log_file);
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 09/15] refs.c: only write reflog update if msg is non-NULL
When performing a reflog transaction update, only write to the reflog iff
msg is non-NULL. This can then be combined with REFLOG_TRUNCATE to perform
an update that only truncates but does not write.
This change only affects whether or not a reflog entry should be generated
and written. If msg==NULL then no such entry will be written.
Orthogonal to this we have a boolean flag REFLOG_TRUNCATE which is used to
tell the transaction system to truncate the reflog and thus discard all
previous users.
At the current time the only place where we use msg==NULL is also the place
where we use REFLOG_TRUNCATE. Eveni though these two settings are currently
only ever used together it still makes sense to have them through two
separate knobs.
This allows future consumers of this API that may want to do things
differently. For example someone can do:
msg=Reflog truncated by Bob because ... + REFLOG_TRUNCATE
and have it truncate the log and have it start fresh with an initial message
that explains the log was truncated. This API allows that.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 5 +++--
refs.h | 1 +
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index d54c3b9..f14b76e 100644
--- a/refs.c
+++ b/refs.c
@@ -3895,8 +3895,9 @@ int transaction_commit(struct transaction *transaction,
update-reflog_fd = -1;
continue;
}
- if (log_ref_write_fd(update-reflog_fd, update-old_sha1,
-update-new_sha1,
+ if (update-msg
+ log_ref_write_fd(update-reflog_fd,
+update-old_sha1, update-new_sha1,
update-committer, update-msg)) {
error(Could write to reflog: %s. %s,
update-refname, strerror(errno));
diff --git a/refs.h b/refs.h
index 5075073..bf96b36 100644
--- a/refs.h
+++ b/refs.h
@@ -337,6 +337,7 @@ int transaction_delete_ref(struct transaction *transaction,
/*
* Append a reflog entry for refname. If the REFLOG_TRUNCATE flag is set
* this update will first truncate the reflog before writing the entry.
+ * If msg is NULL no update will be written to the log.
*/
int transaction_update_reflog(struct transaction *transaction,
const char *refname,
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 10/15] refs.c: allow multiple reflog updates during a single transaction
Allow to make multiple reflog updates to the same ref during a transaction.
This means we only need to lock the reflog once, during the first update
that touches the reflog, and that all further updates can just write the
reflog entry since the reflog is already locked.
This allows us to write code such as:
t = transaction_begin()
transaction_reflog_update(t, foo, REFLOG_TRUNCATE, NULL);
loop-over-something...
transaction_reflog_update(t, foo, 0, message);
transaction_commit(t)
where we first truncate the reflog and then build the new content one line
at a time.
While this technically looks like O(n2) behavior it not that bad.
We only do this loop for transactions that cover a single ref during
reflog expire. This means that the linear search inside
transaction_update_reflog() will find the match on the very first entry
thus making it O(1) and not O(n) or our usecases. Thus the whole expire
becomes O(n) instead of O(n2). If in the future we start doing this for many
refs in one single transaction we might want to optimize this.
But there is no need to complexify the code and optimize for future usecases
that might never materialize at this stage.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 48 +++-
1 file changed, 39 insertions(+), 9 deletions(-)
diff --git a/refs.c b/refs.c
index f14b76e..1c3e674 100644
--- a/refs.c
+++ b/refs.c
@@ -31,6 +31,12 @@ static unsigned char refname_disposition[256] = {
*/
#define REF_ISPRUNING 0x0100
/*
+ * Only the first reflog update needs to lock the reflog file. Further updates
+ * just use the lock taken by the first update.
+ */
+#define UPDATE_REFLOG_NOLOCK 0x0200
+
+/*
* Try to read one refname component from the front of refname.
* Return the length of the component found, or -1 if the component is
* not legal. It is legal if it is something reasonable to have under
@@ -3521,7 +3527,7 @@ enum transaction_update_type {
UPDATE_LOG = 1
};
-/**
+/*
* Information needed for a single ref update. Set new_sha1 to the
* new value or to zero to delete the ref. To check the old value
* while locking the ref, set have_old to 1 and set old_sha1 to the
@@ -3531,7 +3537,9 @@ struct ref_update {
enum transaction_update_type update_type;
unsigned char new_sha1[20];
unsigned char old_sha1[20];
- int flags; /* REF_NODEREF? */
+ int flags; /* The flags to transaction_update_ref[log] are defined
+* in refs.h
+*/
int have_old; /* 1 if old_sha1 is valid, 0 otherwise */
struct ref_lock *lock;
int type;
@@ -3539,8 +3547,9 @@ struct ref_update {
/* used by reflog updates */
int reflog_fd;
- struct lock_file reflog_lock;
+ struct lock_file *reflog_lock;
char *committer;
+ struct ref_update *orig_update; /* For UPDATE_REFLOG_NOLOCK */
const char refname[FLEX_ARRAY];
};
@@ -3619,11 +3628,26 @@ int transaction_update_reflog(struct transaction
*transaction,
struct strbuf *err)
{
struct ref_update *update;
+ int i;
if (transaction-state != TRANSACTION_OPEN)
die(BUG: update_reflog called for transaction that is not
open);
update = add_update(transaction, refname, UPDATE_LOG);
+ update-flags = flags;
+ for (i = 0; i transaction-nr - 1; i++) {
+ if (transaction-updates[i]-update_type != UPDATE_LOG)
+ continue;
+ if (!strcmp(transaction-updates[i]-refname,
+ update-refname)) {
+ update-flags |= UPDATE_REFLOG_NOLOCK;
+ update-orig_update = transaction-updates[i];
+ break;
+ }
+ }
+ if (!(update-flags UPDATE_REFLOG_NOLOCK))
+ update-reflog_lock = xcalloc(1, sizeof(struct lock_file));
+
hashcpy(update-new_sha1, new_sha1);
hashcpy(update-old_sha1, old_sha1);
update-reflog_fd = -1;
@@ -3639,7 +3663,6 @@ int transaction_update_reflog(struct transaction
*transaction,
}
if (msg)
update-msg = xstrdup(msg);
- update-flags = flags;
return 0;
}
@@ -3822,10 +3845,15 @@ int transaction_commit(struct transaction *transaction,
if (update-update_type != UPDATE_LOG)
continue;
+ if (update-flags UPDATE_REFLOG_NOLOCK) {
+ update-reflog_fd = update-orig_update-reflog_fd;
+ update-reflog_lock = update-orig_update-reflog_lock;
+ continue;
+ }
update-reflog_fd = hold_lock_file_for_append(
- update-reflog_lock,
+ update-reflog_lock
[PATCH v2 07/15] refs.c: add a transaction function to append a reflog entry
Define a new transaction update type, UPDATE_LOG, and a new function
transaction_update_reflog. This function will lock the reflog and append
an entry to it during transaction commit.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 102 +++--
refs.h | 12
2 files changed, 112 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index 0e11b1c..100b3a3 100644
--- a/refs.c
+++ b/refs.c
@@ -3517,7 +3517,8 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
}
enum transaction_update_type {
- UPDATE_SHA1 = 0
+ UPDATE_SHA1 = 0,
+ UPDATE_LOG = 1
};
/**
@@ -3535,6 +3536,12 @@ struct ref_update {
struct ref_lock *lock;
int type;
char *msg;
+
+ /* used by reflog updates */
+ int reflog_fd;
+ struct lock_file reflog_lock;
+ char *committer;
+
const char refname[FLEX_ARRAY];
};
@@ -3581,6 +3588,7 @@ void transaction_free(struct transaction *transaction)
for (i = 0; i transaction-nr; i++) {
free(transaction-updates[i]-msg);
+ free(transaction-updates[i]-committer);
free(transaction-updates[i]);
}
free(transaction-updates);
@@ -3601,6 +3609,41 @@ static struct ref_update *add_update(struct transaction
*transaction,
return update;
}
+int transaction_update_reflog(struct transaction *transaction,
+ const char *refname,
+ const unsigned char *new_sha1,
+ const unsigned char *old_sha1,
+ const unsigned char *email,
+ unsigned long timestamp, int tz,
+ const char *msg, int flags,
+ struct strbuf *err)
+{
+ struct ref_update *update;
+
+ if (transaction-state != TRANSACTION_OPEN)
+ die(BUG: update_reflog called for transaction that is not
open);
+
+ update = add_update(transaction, refname, UPDATE_LOG);
+ hashcpy(update-new_sha1, new_sha1);
+ hashcpy(update-old_sha1, old_sha1);
+ update-reflog_fd = -1;
+ if (email) {
+ struct strbuf buf = STRBUF_INIT;
+ char sign = (tz 0) ? '-' : '+';
+ int zone = (tz 0) ? (-tz) : tz;
+
+ strbuf_addf(buf, %s %lu %c%04d, email, timestamp, sign,
+ zone);
+ update-committer = xstrdup(buf.buf);
+ strbuf_release(buf);
+ }
+ if (msg)
+ update-msg = xstrdup(msg);
+ update-flags = flags;
+
+ return 0;
+}
+
int transaction_update_ref(struct transaction *transaction,
const char *refname,
const unsigned char *new_sha1,
@@ -3773,7 +3816,28 @@ int transaction_commit(struct transaction *transaction,
}
}
- /* Perform updates first so live commits remain referenced */
+ /* Lock all reflog files */
+ for (i = 0; i n; i++) {
+ struct ref_update *update = updates[i];
+
+ if (update-update_type != UPDATE_LOG)
+ continue;
+ update-reflog_fd = hold_lock_file_for_append(
+ update-reflog_lock,
+ git_path(logs/%s, update-refname),
+ 0);
+ if (update-reflog_fd 0) {
+ const char *str = Cannot lock reflog for '%s'. %s;
+
+ ret = -1;
+ if (err)
+ strbuf_addf(err, str, update-refname,
+ strerror(errno));
+ goto cleanup;
+ }
+ }
+
+ /* Perform ref updates first so live commits remain referenced */
for (i = 0; i n; i++) {
struct ref_update *update = updates[i];
@@ -3809,6 +3873,40 @@ int transaction_commit(struct transaction *transaction,
}
}
+ /* Update all reflog files */
+ for (i = 0; i n; i++) {
+ struct ref_update *update = updates[i];
+
+ if (update-update_type != UPDATE_LOG)
+ continue;
+ if (update-reflog_fd == -1)
+ continue;
+
+ if (log_ref_write_fd(update-reflog_fd, update-old_sha1,
+update-new_sha1,
+update-committer, update-msg)) {
+ error(Could write to reflog: %s. %s,
+ update-refname, strerror(errno));
+ rollback_lock_file(update-reflog_lock);
+ update-reflog_fd = -1
[PATCH v2 06/15] copy.c: make copy_fd preserve meaningful errno
Update copy_fd to return a meaningful errno on failure.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
copy.c | 15 ++-
1 file changed, 10 insertions(+), 5 deletions(-)
diff --git a/copy.c b/copy.c
index f2970ec..a8d366e 100644
--- a/copy.c
+++ b/copy.c
@@ -8,12 +8,17 @@ int copy_fd(int ifd, int ofd)
if (!len)
break;
if (len 0) {
- return error(copy-fd: read returned %s,
-strerror(errno));
+ int save_errno = errno;
+ error(copy-fd: read returned %s, strerror(errno));
+ errno = save_errno;
+ return -1;
+ }
+ if (write_in_full(ofd, buffer, len) 0) {
+ int save_errno = errno;
+ error(copy-fd: write returned %s, strerror(errno));
+ errno = save_errno;
+ return -1;
}
- if (write_in_full(ofd, buffer, len) 0)
- return error(copy-fd: write returned %s,
-strerror(errno));
}
return 0;
}
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 08/15] refs.c: add a flag to allow reflog updates to truncate the log
Add a flag that allows us to truncate the reflog before we write the
update.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 17 +++--
refs.h | 10 +-
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/refs.c b/refs.c
index 100b3a3..d54c3b9 100644
--- a/refs.c
+++ b/refs.c
@@ -3873,7 +3873,12 @@ int transaction_commit(struct transaction *transaction,
}
}
- /* Update all reflog files */
+ /*
+* Update all reflog files
+* We have already done all ref updates and deletes.
+* There is not much we can do here if there are any reflog
+* update errors other than complain.
+*/
for (i = 0; i n; i++) {
struct ref_update *update = updates[i];
@@ -3881,7 +3886,15 @@ int transaction_commit(struct transaction *transaction,
continue;
if (update-reflog_fd == -1)
continue;
-
+ if (update-flags REFLOG_TRUNCATE)
+ if (lseek(update-reflog_fd, 0, SEEK_SET) 0 ||
+ ftruncate(update-reflog_fd, 0)) {
+ error(Could not truncate reflog: %s. %s,
+ update-refname, strerror(errno));
+ rollback_lock_file(update-reflog_lock);
+ update-reflog_fd = -1;
+ continue;
+ }
if (log_ref_write_fd(update-reflog_fd, update-old_sha1,
update-new_sha1,
update-committer, update-msg)) {
diff --git a/refs.h b/refs.h
index 8220d18..5075073 100644
--- a/refs.h
+++ b/refs.h
@@ -328,7 +328,15 @@ int transaction_delete_ref(struct transaction *transaction,
struct strbuf *err);
/*
- * Append a reflog entry for refname.
+ * Flags controlling transaction_update_reflog().
+ * REFLOG_TRUNCATE: Truncate the reflog.
+ *
+ * Flags = 0x100 are reserved for internal use.
+ */
+#define REFLOG_TRUNCATE 0x01
+/*
+ * Append a reflog entry for refname. If the REFLOG_TRUNCATE flag is set
+ * this update will first truncate the reflog before writing the entry.
*/
int transaction_update_reflog(struct transaction *transaction,
const char *refname,
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 01/15] refs.c make ref_transaction_create a wrapper to ref_transaction_update
The ref_transaction_update function can already be used to create refs by
passing null_sha1 as the old_sha1 parameter. Simplify by replacing
transaction_create with a thin wrapper.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 27 ++-
1 file changed, 2 insertions(+), 25 deletions(-)
diff --git a/refs.c b/refs.c
index 0368ed4..ed0485e 100644
--- a/refs.c
+++ b/refs.c
@@ -3623,31 +3623,8 @@ int ref_transaction_create(struct ref_transaction
*transaction,
int flags, const char *msg,
struct strbuf *err)
{
- struct ref_update *update;
-
- assert(err);
-
- if (transaction-state != REF_TRANSACTION_OPEN)
- die(BUG: create called for transaction that is not open);
-
- if (!new_sha1 || is_null_sha1(new_sha1))
- die(BUG: create ref with null new_sha1);
-
- if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
- strbuf_addf(err, refusing to create ref with bad name %s,
- refname);
- return -1;
- }
-
- update = add_update(transaction, refname);
-
- hashcpy(update-new_sha1, new_sha1);
- hashclr(update-old_sha1);
- update-flags = flags;
- update-have_old = 1;
- if (msg)
- update-msg = xstrdup(msg);
- return 0;
+ return ref_transaction_update(transaction, refname, new_sha1,
+ null_sha1, flags, 1, msg, err);
}
int ref_transaction_delete(struct ref_transaction *transaction,
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 02/15] refs.c: make ref_transaction_delete a wrapper for ref_transaction_update
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 22 ++
refs.h | 2 +-
2 files changed, 3 insertions(+), 21 deletions(-)
diff --git a/refs.c b/refs.c
index ed0485e..c607ab7 100644
--- a/refs.c
+++ b/refs.c
@@ -3633,26 +3633,8 @@ int ref_transaction_delete(struct ref_transaction
*transaction,
int flags, int have_old, const char *msg,
struct strbuf *err)
{
- struct ref_update *update;
-
- assert(err);
-
- if (transaction-state != REF_TRANSACTION_OPEN)
- die(BUG: delete called for transaction that is not open);
-
- if (have_old !old_sha1)
- die(BUG: have_old is true but old_sha1 is NULL);
-
- update = add_update(transaction, refname);
- update-flags = flags;
- update-have_old = have_old;
- if (have_old) {
- assert(!is_null_sha1(old_sha1));
- hashcpy(update-old_sha1, old_sha1);
- }
- if (msg)
- update-msg = xstrdup(msg);
- return 0;
+ return ref_transaction_update(transaction, refname, null_sha1,
+ old_sha1, flags, have_old, msg, err);
}
int update_ref(const char *action, const char *refname,
diff --git a/refs.h b/refs.h
index 2bc3556..7d675b7 100644
--- a/refs.h
+++ b/refs.h
@@ -283,7 +283,7 @@ struct ref_transaction *ref_transaction_begin(struct strbuf
*err);
/*
* Add a reference update to transaction. new_sha1 is the value that
- * the reference should have after the update, or zeros if it should
+ * the reference should have after the update, or null_sha1 if it should
* be deleted. If have_old is true, then old_sha1 holds the value
* that the reference should have had before the update, or zeros if
* it must not have existed beforehand.
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 14/15] refs.c: remove lock_any_ref_for_update
No one is using this function so we can delete it.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 7 ---
refs.h | 9 +
2 files changed, 1 insertion(+), 15 deletions(-)
diff --git a/refs.c b/refs.c
index e4ad4f4..6d50a32 100644
--- a/refs.c
+++ b/refs.c
@@ -2352,13 +2352,6 @@ static struct ref_lock *lock_ref_sha1_basic(const char
*refname,
return NULL;
}
-struct ref_lock *lock_any_ref_for_update(const char *refname,
-const unsigned char *old_sha1,
-int flags, int *type_p)
-{
- return lock_ref_sha1_basic(refname, old_sha1, NULL, flags, type_p);
-}
-
/*
* Write an entry to the packed-refs file for the specified refname.
* If peeled is non-NULL, write it as the entry's peeled value.
diff --git a/refs.h b/refs.h
index 025e2cb..721e21f 100644
--- a/refs.h
+++ b/refs.h
@@ -181,8 +181,7 @@ extern int is_branch(const char *refname);
extern int peel_ref(const char *refname, unsigned char *sha1);
/*
- * Flags controlling lock_any_ref_for_update(), transaction_update_ref(),
- * transaction_create_ref(), etc.
+ * Flags controlling transaction_update_ref(), transaction_create_ref(), etc.
* REF_NODEREF: act on the ref directly, instead of dereferencing
* symbolic references.
* REF_DELETING: tolerate broken refs
@@ -191,12 +190,6 @@ extern int peel_ref(const char *refname, unsigned char
*sha1);
*/
#define REF_NODEREF0x01
#define REF_DELETING 0x02
-/*
- * This function sets errno to something meaningful on failure.
- */
-extern struct ref_lock *lock_any_ref_for_update(const char *refname,
- const unsigned char *old_sha1,
- int flags, int *type_p);
/** Reads log for the value of ref during at_time. **/
extern int read_ref_at(const char *refname, unsigned int flags,
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 01/17] refs.c: allow passing raw git_committer_info as email to _update_reflog
In many places in the code we do not have access to the individual fields
in the committer data. Instead we might only have access to prebaked data
such as what is returned by git_committer_info() containing a string
that consists of email, timestamp, zone etc.
This makes it inconvenient to use transaction_update_reflog since it means
you would have to first parse git_committer_info before you can call
update_reflog.
Add a new flag REFLOG_COMMITTER_INFO_IS_VALID to _update_reflog to tell it
that we pass in a fully prebaked committer info string that can be used as is.
At the same time, also go over and change all references from email
to id where the code actually refers to a committer id and not just an email
address. I.e. where the string is : NAME EMAIL
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
builtin/reflog.c | 19 +--
refs.c | 21 -
refs.h | 25 +++--
3 files changed, 48 insertions(+), 17 deletions(-)
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 6bb7454..be88a53 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -292,7 +292,7 @@ static int unreachable(struct expire_reflog_cb *cb, struct
commit *commit, unsig
}
static int expire_reflog_ent(unsigned char *osha1, unsigned char *nsha1,
- const char *email, unsigned long timestamp, int tz,
+ const char *id, unsigned long timestamp, int tz,
const char *message, void *cb_data)
{
struct expire_reflog_cb *cb = cb_data;
@@ -320,9 +320,14 @@ static int expire_reflog_ent(unsigned char *osha1,
unsigned char *nsha1,
goto prune;
if (cb-t) {
+ struct reflog_committer_info ci;
+
+ memset(ci, 0, sizeof(ci));
+ ci.id = id;
+ ci.timestamp = timestamp;
+ ci.tz = tz;
if (transaction_update_reflog(cb-t, cb-refname, nsha1, osha1,
- email, timestamp, tz, message, 0,
- err))
+ ci, message, 0, err))
return -1;
hashcpy(cb-last_kept_sha1, nsha1);
}
@@ -356,6 +361,7 @@ static int expire_reflog(const char *ref, const unsigned
char *sha1, int unused,
struct expire_reflog_cb cb;
struct commit *tip_commit;
struct commit_list *tips;
+ struct reflog_committer_info ci;
int status = 0;
memset(cb, 0, sizeof(cb));
@@ -368,9 +374,10 @@ static int expire_reflog(const char *ref, const unsigned
char *sha1, int unused,
status |= error(%s, err.buf);
goto cleanup;
}
+
+ memset(ci, 0, sizeof(ci));
if (transaction_update_reflog(cb.t, cb.refname, null_sha1, null_sha1,
- NULL, 0, 0, NULL, REFLOG_TRUNCATE,
- err)) {
+ ci, NULL, REFLOG_TRUNCATE, err)) {
status |= error(%s, err.buf);
goto cleanup;
}
@@ -672,7 +679,7 @@ static int cmd_reflog_expire(int argc, const char **argv,
const char *prefix)
}
static int count_reflog_ent(unsigned char *osha1, unsigned char *nsha1,
- const char *email, unsigned long timestamp, int tz,
+ const char *id, unsigned long timestamp, int tz,
const char *message, void *cb_data)
{
struct cmd_reflog_expire_cb *cb = cb_data;
diff --git a/refs.c b/refs.c
index f1ca9e4..1791166 100644
--- a/refs.c
+++ b/refs.c
@@ -3226,7 +3226,7 @@ struct read_ref_at_cb {
};
static int read_ref_at_ent(unsigned char *osha1, unsigned char *nsha1,
- const char *email, unsigned long timestamp, int tz,
+ const char *id, unsigned long timestamp, int tz,
const char *message, void *cb_data)
{
struct read_ref_at_cb *cb = cb_data;
@@ -3273,7 +3273,7 @@ static int read_ref_at_ent(unsigned char *osha1, unsigned
char *nsha1,
}
static int read_ref_at_ent_oldest(unsigned char *osha1, unsigned char *nsha1,
- const char *email, unsigned long timestamp,
+ const char *id, unsigned long timestamp,
int tz, const char *message, void *cb_data)
{
struct read_ref_at_cb *cb = cb_data;
@@ -3625,8 +3625,7 @@ int transaction_update_reflog(struct transaction
*transaction,
const char *refname,
const unsigned char *new_sha1,
const unsigned char *old_sha1,
- const char *email,
- unsigned long timestamp, int tz,
+ struct reflog_committer_info *ci,
const
[PATCH v2 00/17] ref-transaction-rename
List, Thsi series builds on the previous series : ref-transaction-reflog as applied to next. This series has been sent to the list before but is now rebased to current git next. This series can also be found at : https://github.com/rsahlberg/git/tree/ref-transactions-rename This series converts ref rename to use a transaction. This addesses several issues in the old implementation, such as colliding renames might overwrite someone elses reflog, and it makes the rename atomic. As part of the series we also move changes that cover multiple refs to happen as an atomic transaction/rename to the pacekd refs file. This makes it possible to have both the rename case (one deleted ref + one created ref) as well as any operation that updates multiple refs to become one atomic rename() applied to the packed refs file. Thus all such changes are now also atomic to all external observers. Version 2: - Changed to not use potentially iterators to copy the reflog entries one by one. Instead adding two new functions. One to read an existing reflog as one big blob, and a second function to, in a transaction, write a new complete reflog from said blob. The idea is that each future reflog backend will provide optimized versions for these read whole reflog write whole reflog functions. Ronnie Sahlberg (17): refs.c: allow passing raw git_committer_info as email to _update_reflog refs.c: return error instead of dying when locking fails during transaction refs.c: use packed refs when deleting refs during a transaction refs.c: use a stringlist for repack_without_refs refs.c: add transaction support for replacing a reflog refs.c: add new function copy_reflog_into_strbuf refs.c: update rename_ref to use a transaction refs.c: rollback the lockfile before we die() in repack_without_refs refs.c: move reflog updates into its own function refs.c: write updates to packed refs when a transaction has more than one ref remote.c: use a transaction for deleting refs refs.c: make repack_without_refs static refs.c: make the *_packed_refs functions static refs.c: replace the onerr argument in update_ref with a strbuf err refs.c: make add_packed_ref return an error instead of calling die refs.c: make lock_packed_refs take an err argument refs.c: add an err argument to pack_refs builtin/checkout.c| 7 +- builtin/clone.c | 36 ++- builtin/merge.c | 20 +- builtin/notes.c | 24 +- builtin/pack-refs.c | 8 +- builtin/reflog.c | 19 +- builtin/remote.c | 69 +++--- builtin/reset.c | 12 +- builtin/update-ref.c | 7 +- notes-cache.c | 2 +- notes-utils.c | 5 +- refs.c| 618 ++ refs.h| 87 --- t/t3200-branch.sh | 7 - t/t5516-fetch-push.sh | 2 +- transport-helper.c| 7 +- transport.c | 9 +- 17 files changed, 562 insertions(+), 377 deletions(-) -- 2.1.0.rc2.206.gedb03e5 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 02/17] refs.c: return error instead of dying when locking fails during transaction
Change lock_ref_sha1_basic to return an error instead of dying when
we fail to lock a file during a transaction.
This function is only called from transaction_commit() and it knows how
to handle these failures.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 10 --
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index 1791166..d4ab65c 100644
--- a/refs.c
+++ b/refs.c
@@ -2340,6 +2340,7 @@ static struct ref_lock *lock_ref_sha1_basic(const char
*refname,
lock-lock_fd = hold_lock_file_for_update(lock-lk, ref_file, lflags);
if (lock-lock_fd 0) {
+ last_errno = errno;
if (errno == ENOENT --attempts_remaining 0)
/*
* Maybe somebody just deleted one of the
@@ -2347,8 +2348,13 @@ static struct ref_lock *lock_ref_sha1_basic(const char
*refname,
* again:
*/
goto retry;
- else
- unable_to_lock_die(ref_file, errno);
+ else {
+ struct strbuf err = STRBUF_INIT;
+ unable_to_lock_message(ref_file, errno, err);
+ error(%s, err.buf);
+ strbuf_reset(err);
+ goto error_return;
+ }
}
return old_sha1 ? verify_lock(lock, old_sha1, mustexist) : lock;
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 05/17] refs.c: add transaction support for replacing a reflog
Add a new transaction function transaction_replace_reflog.
This function takes a blob and replaces the new or existing reflog with the
content of this blob. This will be used by rename_ref where we basically want
to copy the existing blob as is.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 38 +-
refs.h | 16
2 files changed, 53 insertions(+), 1 deletion(-)
diff --git a/refs.c b/refs.c
index e9e321e..3d13624 100644
--- a/refs.c
+++ b/refs.c
@@ -35,6 +35,11 @@ static unsigned char refname_disposition[256] = {
* just use the lock taken by the first update.
*/
#define UPDATE_REFLOG_NOLOCK 0x0200
+/*
+ * This update is used to replace a new or existing reflog with new content
+ * held in update-new_reflog.
+ */
+#define REFLOG_REPLACE 0x0400
/*
* Try to read one refname component from the front of refname.
@@ -3561,6 +3566,7 @@ struct ref_update {
struct lock_file *reflog_lock;
char *committer;
struct ref_update *orig_update; /* For UPDATE_REFLOG_NOLOCK */
+ struct strbuf new_reflog;
const char refname[FLEX_ARRAY];
};
@@ -3607,6 +3613,7 @@ void transaction_free(struct transaction *transaction)
return;
for (i = 0; i transaction-nr; i++) {
+ strbuf_release(transaction-updates[i]-new_reflog);
free(transaction-updates[i]-msg);
free(transaction-updates[i]-committer);
free(transaction-updates[i]);
@@ -3622,6 +3629,7 @@ static struct ref_update *add_update(struct transaction
*transaction,
size_t len = strlen(refname);
struct ref_update *update = xcalloc(1, sizeof(*update) + len + 1);
+ strbuf_init(update-new_reflog, 0);
strcpy((char *)update-refname, refname);
update-update_type = update_type;
ALLOC_GROW(transaction-updates, transaction-nr + 1,
transaction-alloc);
@@ -3681,6 +3689,24 @@ int transaction_update_reflog(struct transaction
*transaction,
return 0;
}
+int transaction_replace_reflog(struct transaction *transaction,
+ const char *refname,
+ struct strbuf *buf,
+ struct strbuf *err)
+{
+ struct ref_update *update;
+
+ if (transaction-state != TRANSACTION_OPEN)
+ die(BUG: replace_reflog called for transaction that is
+ not open);
+
+ update = add_update(transaction, refname, UPDATE_LOG);
+ update-flags = REFLOG_REPLACE;
+ strbuf_swap(update-new_reflog, buf);
+ update-reflog_lock = xcalloc(1, sizeof(struct lock_file));
+ return 0;
+}
+
int transaction_update_ref(struct transaction *transaction,
const char *refname,
const unsigned char *new_sha1,
@@ -4012,7 +4038,7 @@ int transaction_commit(struct transaction *transaction,
continue;
if (update-reflog_fd == -1)
continue;
- if (update-flags REFLOG_TRUNCATE)
+ if (update-flags (REFLOG_TRUNCATE|REFLOG_REPLACE))
if (lseek(update-reflog_fd, 0, SEEK_SET) 0 ||
ftruncate(update-reflog_fd, 0)) {
error(Could not truncate reflog: %s. %s,
@@ -4030,6 +4056,16 @@ int transaction_commit(struct transaction *transaction,
rollback_lock_file(update-reflog_lock);
update-reflog_fd = -1;
}
+ if (update-flags REFLOG_REPLACE)
+ if (write_in_full(update-reflog_fd,
+ update-new_reflog.buf,
+ update-new_reflog.len) !=
+ update-new_reflog.len) {
+ error(Could write to reflog: %s. %s,
+ update-refname, strerror(errno));
+ rollback_lock_file(update-reflog_lock);
+ update-reflog_fd = -1;
+ }
}
/* Commit all reflog files */
diff --git a/refs.h b/refs.h
index d174380..ec4965f 100644
--- a/refs.h
+++ b/refs.h
@@ -354,6 +354,22 @@ int transaction_update_reflog(struct transaction
*transaction,
struct strbuf *err);
/*
+ * Replace the reflog with the content of buf.
+ * This function can be used when replacing the whole content of the reflog
+ * during for example a rename operation. For these cases we do not want
+ * to have to spend time processing and marshaling each entry individually
+ * if instead we can just pass one single blob to the transaction system and
+ * say write this blob, it is the new reflog.
+ *
+ * When successful, this function will take over ownership of buf.
+ * Thus meaning that buf will become cleared from
[PATCH v2 11/17] remote.c: use a transaction for deleting refs
Transactions now use packed refs when deleting multiple refs so there is no
need to do it manually from remote.c any more.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/remote.c | 80
1 file changed, 40 insertions(+), 40 deletions(-)
diff --git a/builtin/remote.c b/builtin/remote.c
index 6806251..42702d7 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -750,30 +750,27 @@ static int mv(int argc, const char **argv)
static int remove_branches(struct string_list *branches)
{
- struct strbuf err = STRBUF_INIT;
int i, result = 0;
+ struct transaction *transaction;
+ struct strbuf err = STRBUF_INIT;
- if (lock_packed_refs(0)) {
- struct strbuf err = STRBUF_INIT;
-
- unable_to_lock_message(git_path(packed-refs), errno, err);
- error(%s, err.buf);
- strbuf_release(err);
- return -1;
- }
+ transaction = transaction_begin(err);
+ if (!transaction)
+ die(%s, err.buf);
- if (repack_without_refs(branches, err))
+ for (i = 0; i branches-nr; i++)
+ if (transaction_delete_ref(transaction,
+ branches-items[i].string, NULL,
+ 0, 0, remote-branches, err)) {
+ result |= error(%s, err.buf);
+ goto cleanup;
+ }
+ if (transaction_commit(transaction, err))
result |= error(%s, err.buf);
- strbuf_release(err);
-
- for (i = 0; i branches-nr; i++) {
- struct string_list_item *item = branches-items + i;
- const char *refname = item-string;
-
- if (delete_ref(refname, NULL, 0))
- result |= error(_(Could not remove branch %s),
refname);
- }
+ cleanup:
+ strbuf_release(err);
+ transaction_free(transaction);
return result;
}
@@ -1325,42 +1322,38 @@ static int prune_remote(const char *remote, int dry_run)
const char *dangling_msg = dry_run
? _( %s will become dangling!)
: _( %s has become dangling!);
+ struct transaction *transaction = NULL;
+ struct strbuf err = STRBUF_INIT;
memset(states, 0, sizeof(states));
get_remote_ref_states(remote, states, GET_REF_STATES);
- for (i = 0; i states.stale.nr; i++) {
- string_list_insert(delete_refs_list,
- states.stale.items[i].util);
- }
-
if (states.stale.nr) {
printf_ln(_(Pruning %s), remote);
printf_ln(_(URL: %s),
states.remote-url_nr
? states.remote-url[0]
: _((no URL)));
-
- if (!dry_run) {
- struct strbuf err = STRBUF_INIT;
-
- if (lock_packed_refs(0)) {
- unable_to_lock_message(git_path(packed-refs),
- errno, err);
- result |= error(%s, err.buf);
- } else
- if (repack_without_refs(delete_refs_list,
- err))
- result |= error(%s, err.buf);
- strbuf_release(err);
- }
}
+ if (!dry_run) {
+ transaction = transaction_begin(err);
+ if (!transaction)
+ die(%s, err.buf);
+ }
for (i = 0; i states.stale.nr; i++) {
const char *refname = states.stale.items[i].util;
- if (!dry_run)
- result |= delete_ref(refname, NULL, 0);
+ string_list_insert(delete_refs_list, refname);
+
+ if (!dry_run) {
+ if (transaction_delete_ref(transaction,
+ refname, NULL,
+ 0, 0, remote-branches, err)) {
+ result |= error(%s, err.buf);
+ goto cleanup;
+ }
+ }
if (dry_run)
printf_ln(_( * [would prune] %s),
@@ -1370,6 +1363,13 @@ static int prune_remote(const char *remote, int dry_run)
abbrev_ref(refname, refs/remotes/));
}
+ if (!dry_run)
+ if (transaction_commit(transaction, err))
+ result |= error(%s, err.buf);
+
+ cleanup:
+ strbuf_release(err);
+ transaction_free(transaction);
warn_dangling_symrefs(stdout, dangling_msg, delete_refs_list);
string_list_clear(delete_refs_list, 0);
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe
[PATCH v2 13/17] refs.c: make the *_packed_refs functions static
We no longer need to expose the lock/add/commit/rollback functions
for packed refs anymore so make them static and remove them from the
public api.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 8
refs.h | 30 --
2 files changed, 4 insertions(+), 34 deletions(-)
diff --git a/refs.c b/refs.c
index 56b0360..c9c03f2 100644
--- a/refs.c
+++ b/refs.c
@@ -1229,7 +1229,7 @@ static struct ref_dir *get_packed_refs(struct ref_cache
*refs)
return get_packed_ref_dir(get_packed_ref_cache(refs));
}
-void add_packed_ref(const char *refname, const unsigned char *sha1)
+static void add_packed_ref(const char *refname, const unsigned char *sha1)
{
struct packed_ref_cache *packed_ref_cache =
get_packed_ref_cache(ref_cache);
@@ -2398,7 +2398,7 @@ static int write_packed_entry_fn(struct ref_entry *entry,
void *cb_data)
}
/* This should return a meaningful errno on failure */
-int lock_packed_refs(int flags)
+static int lock_packed_refs(int flags)
{
struct packed_ref_cache *packed_ref_cache;
@@ -2421,7 +2421,7 @@ int lock_packed_refs(int flags)
* Commit the packed refs changes.
* On error we must make sure that errno contains a meaningful value.
*/
-int commit_packed_refs(void)
+static int commit_packed_refs(void)
{
struct packed_ref_cache *packed_ref_cache =
get_packed_ref_cache(ref_cache);
@@ -2450,7 +2450,7 @@ int commit_packed_refs(void)
return error;
}
-void rollback_packed_refs(void)
+static void rollback_packed_refs(void)
{
struct packed_ref_cache *packed_ref_cache =
get_packed_ref_cache(ref_cache);
diff --git a/refs.h b/refs.h
index f5ba534..1747d5f 100644
--- a/refs.h
+++ b/refs.h
@@ -120,36 +120,6 @@ extern void warn_dangling_symref(FILE *fp, const char
*msg_fmt, const char *refn
extern void warn_dangling_symrefs(FILE *fp, const char *msg_fmt, const struct
string_list *refnames);
/*
- * Lock the packed-refs file for writing. Flags is passed to
- * hold_lock_file_for_update(). Return 0 on success.
- * Errno is set to something meaningful on error.
- */
-extern int lock_packed_refs(int flags);
-
-/*
- * Add a reference to the in-memory packed reference cache. This may
- * only be called while the packed-refs file is locked (see
- * lock_packed_refs()). To actually write the packed-refs file, call
- * commit_packed_refs().
- */
-extern void add_packed_ref(const char *refname, const unsigned char *sha1);
-
-/*
- * Write the current version of the packed refs cache from memory to
- * disk. The packed-refs file must already be locked for writing (see
- * lock_packed_refs()). Return zero on success.
- * Sets errno to something meaningful on error.
- */
-extern int commit_packed_refs(void);
-
-/*
- * Rollback the lockfile for the packed-refs file, and discard the
- * in-memory packed reference cache. (The packed-refs file will be
- * read anew if it is needed again after this function is called.)
- */
-extern void rollback_packed_refs(void);
-
-/*
* Flags for controlling behaviour of pack_refs()
* PACK_REFS_PRUNE: Prune loose refs after packing
* PACK_REFS_ALL: Pack _all_ refs, not just tags and already packed refs
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 03/17] refs.c: use packed refs when deleting refs during a transaction
Make the deletion of refs during a transaction more atomic.
Start by first copying all loose refs we will be deleting to the packed
refs file and then commit the packed refs file. Then re-lock the packed refs
file to stop anyone else from modifying these refs and keep it locked until
we are finished.
Since all refs we are about to delete are now safely held in the packed refs
file we can proceed to immediately unlink any corresponding loose refs
and still be fully rollback-able.
The exception is for refs that can not be resolved. Those refs are never
added to the packed refs and will just be un-rollback-ably deleted during
commit.
By deleting all the loose refs at the start of the transaction we make make
it possible to both delete one ref and then re-create a different ref in
the same transaction even if the two refs would normally conflict.
Example: rename m-m/m
In that example we want to delete the file 'm' so that we make room so
that we can create a directory with the same name in order to lock and
write to the ref m/m and its lock-file m/m.lock.
If there is a failure during the commit phase we can rollback without losing
any refs since we have so far only deleted loose refs that that are
guaranteed to also have a corresponding entry in the packed refs file.
Once we have finished all updates for refs and their reflogs we can repack
the packed refs file and remove the to-be-deleted refs from the packed refs,
at which point all the deleted refs will disappear in one atomic rename
operation.
This also means that for an outside observer, deletion of multiple refs
in a single transaction will look atomic instead of one ref being deleted
at a time.
In order to do all this we need to change the semantics for the
repack_without_refs function so that we can lock the packed refs file,
do other stuff, and later be able to call repack_without_refs with the
lock already taken.
This means we need some additional changes in remote.c to reflect the
changes to the repack_without_refs semantics.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
builtin/remote.c | 20 ++-
refs.c | 155 ++-
2 files changed, 138 insertions(+), 37 deletions(-)
diff --git a/builtin/remote.c b/builtin/remote.c
index 7f28f92..c25420f 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -1,4 +1,5 @@
#include builtin.h
+#include lockfile.h
#include parse-options.h
#include transport.h
#include remote.h
@@ -753,6 +754,15 @@ static int remove_branches(struct string_list *branches)
const char **branch_names;
int i, result = 0;
+ if (lock_packed_refs(0)) {
+ struct strbuf err = STRBUF_INIT;
+
+ unable_to_lock_message(git_path(packed-refs), errno, err);
+ error(%s, err.buf);
+ strbuf_release(err);
+ return -1;
+ }
+
branch_names = xmalloc(branches-nr * sizeof(*branch_names));
for (i = 0; i branches-nr; i++)
branch_names[i] = branches-items[i].string;
@@ -1337,9 +1347,15 @@ static int prune_remote(const char *remote, int dry_run)
delete_refs[i] = states.stale.items[i].util;
if (!dry_run) {
struct strbuf err = STRBUF_INIT;
- if (repack_without_refs(delete_refs, states.stale.nr,
- err))
+
+ if (lock_packed_refs(0)) {
+ unable_to_lock_message(git_path(packed-refs),
+ errno, err);
result |= error(%s, err.buf);
+ } else
+ if (repack_without_refs(delete_refs,
+ states.stale.nr, err))
+ result |= error(%s, err.buf);
strbuf_release(err);
}
free(delete_refs);
diff --git a/refs.c b/refs.c
index d4ab65c..809bd3f 100644
--- a/refs.c
+++ b/refs.c
@@ -2660,6 +2660,9 @@ static int curate_packed_ref_fn(struct ref_entry *entry,
void *cb_data)
return 0;
}
+/*
+ * Must be called with packed refs already locked (and sorted)
+ */
int repack_without_refs(const char **refnames, int n, struct strbuf *err)
{
struct ref_dir *packed;
@@ -2674,14 +2677,12 @@ int repack_without_refs(const char **refnames, int n,
struct strbuf *err)
if (get_packed_ref(refnames[i]))
break;
- /* Avoid locking if we have nothing to do */
- if (i == n)
+ /* Avoid processing if we have nothing to do */
+ if (i == n) {
+ rollback_packed_refs();
return 0; /* no refname exists in packed refs */
-
- if (lock_packed_refs(0
[PATCH v2 07/17] refs.c: update rename_ref to use a transaction
Change refs.c to use a single transaction to perform the rename.
Change the function to return 1 on failure instead of either -1 or 1.
These changes make the rename_ref operation atomic.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c| 173 +++---
t/t3200-branch.sh | 7 ---
2 files changed, 48 insertions(+), 132 deletions(-)
diff --git a/refs.c b/refs.c
index 1c968f5..44d38ab 100644
--- a/refs.c
+++ b/refs.c
@@ -2757,60 +2757,6 @@ int delete_ref(const char *refname, const unsigned char
*sha1, int delopt)
return 0;
}
-/*
- * People using contrib's git-new-workdir have .git/logs/refs -
- * /some/other/path/.git/logs/refs, and that may live on another device.
- *
- * IOW, to avoid cross device rename errors, the temporary renamed log must
- * live into logs/refs.
- */
-#define TMP_RENAMED_LOG logs/refs/.tmp-renamed-log
-
-static int rename_tmp_log(const char *newrefname)
-{
- int attempts_remaining = 4;
-
- retry:
- switch (safe_create_leading_directories(git_path(logs/%s,
newrefname))) {
- case SCLD_OK:
- break; /* success */
- case SCLD_VANISHED:
- if (--attempts_remaining 0)
- goto retry;
- /* fall through */
- default:
- error(unable to create directory for %s, newrefname);
- return -1;
- }
-
- if (rename(git_path(TMP_RENAMED_LOG), git_path(logs/%s, newrefname)))
{
- if ((errno==EISDIR || errno==ENOTDIR) --attempts_remaining
0) {
- /*
-* rename(a, b) when b is an existing
-* directory ought to result in ISDIR, but
-* Solaris 5.8 gives ENOTDIR. Sheesh.
-*/
- if (remove_empty_directories(git_path(logs/%s,
newrefname))) {
- error(Directory not empty: logs/%s,
newrefname);
- return -1;
- }
- goto retry;
- } else if (errno == ENOENT --attempts_remaining 0) {
- /*
-* Maybe another process just deleted one of
-* the directories in the path to newrefname.
-* Try again from the beginning.
-*/
- goto retry;
- } else {
- error(unable to move logfile TMP_RENAMED_LOG to
logs/%s: %s,
- newrefname, strerror(errno));
- return -1;
- }
- }
- return 0;
-}
-
static int rename_ref_available(const char *oldname, const char *newname)
{
struct string_list skip = STRING_LIST_INIT_NODUP;
@@ -2859,91 +2805,68 @@ static int copy_reflog_into_strbuf(const char *refname,
struct strbuf *buf)
int rename_ref(const char *oldrefname, const char *newrefname, const char
*logmsg)
{
- unsigned char sha1[20], orig_sha1[20];
- int flag = 0, logmoved = 0;
- struct ref_lock *lock;
- struct stat loginfo;
- int log = !lstat(git_path(logs/%s, oldrefname), loginfo);
+ unsigned char sha1[20];
+ int flag = 0;
+ int log;
+ struct transaction *transaction = NULL;
+ struct strbuf err = STRBUF_INIT;
+ struct strbuf buf = STRBUF_INIT;
const char *symref = NULL;
+ struct reflog_committer_info ci;
- if (log S_ISLNK(loginfo.st_mode))
- return error(reflog for %s is a symlink, oldrefname);
+ memset(ci, 0, sizeof(ci));
+ ci.committer_info = git_committer_info(0);
symref = resolve_ref_unsafe(oldrefname, RESOLVE_REF_READING,
- orig_sha1, flag);
- if (flag REF_ISSYMREF)
- return error(refname %s is a symbolic ref, renaming it is not
supported,
- oldrefname);
- if (!symref)
- return error(refname %s not found, oldrefname);
-
- if (!rename_ref_available(oldrefname, newrefname))
+ sha1, flag);
+ if (flag REF_ISSYMREF) {
+ error(refname %s is a symbolic ref, renaming it is not
+ supported, oldrefname);
return 1;
-
- if (log rename(git_path(logs/%s, oldrefname),
git_path(TMP_RENAMED_LOG)))
- return error(unable to move logfile logs/%s to
TMP_RENAMED_LOG: %s,
- oldrefname, strerror(errno));
-
- if (delete_ref(oldrefname, orig_sha1, REF_NODEREF)) {
- error(unable to delete old %s, oldrefname);
- goto rollback;
}
-
- if (!read_ref_full(newrefname, RESOLVE_REF_READING, sha1, NULL)
- delete_ref(newrefname, sha1, REF_NODEREF)) {
- if (errno==EISDIR
[PATCH v2 16/17] refs.c: make lock_packed_refs take an err argument
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 25 +
1 file changed, 13 insertions(+), 12 deletions(-)
diff --git a/refs.c b/refs.c
index 0b9cb8d..92e7714 100644
--- a/refs.c
+++ b/refs.c
@@ -2398,13 +2398,17 @@ static int write_packed_entry_fn(struct ref_entry
*entry, void *cb_data)
return 0;
}
-/* This should return a meaningful errno on failure */
-static int lock_packed_refs(int flags)
+static int lock_packed_refs(struct strbuf *err)
{
struct packed_ref_cache *packed_ref_cache;
- if (hold_lock_file_for_update(packlock, git_path(packed-refs),
flags) 0)
+ if (hold_lock_file_for_update(packlock, git_path(packed-refs),
+ 0) 0) {
+ if (err)
+ unable_to_lock_message(git_path(packed-refs),
+ errno, err);
return -1;
+ }
/*
* Get the current packed-refs while holding the lock. If the
* packed-refs file has been modified since we last read it,
@@ -2592,11 +2596,14 @@ static void prune_refs(struct ref_to_prune *r)
int pack_refs(unsigned int flags)
{
struct pack_refs_cb_data cbdata;
+ struct strbuf err = STRBUF_INIT;
memset(cbdata, 0, sizeof(cbdata));
cbdata.flags = flags;
- lock_packed_refs(LOCK_DIE_ON_ERROR);
+ if (lock_packed_refs(err))
+ die(%s, err.buf);
+
cbdata.packed_refs = get_packed_refs(ref_cache);
do_for_each_entry_in_dir(get_loose_refs(ref_cache), 0,
@@ -3791,10 +3798,7 @@ int transaction_commit(struct transaction *transaction,
}
/* Lock packed refs during commit */
- if (lock_packed_refs(0)) {
- if (err)
- unable_to_lock_message(git_path(packed-refs),
- errno, err);
+ if (lock_packed_refs(err)) {
ret = -1;
goto cleanup;
}
@@ -3849,10 +3853,7 @@ int transaction_commit(struct transaction *transaction,
goto cleanup;
}
/* lock the packed refs again so no one can change it */
- if (lock_packed_refs(0)) {
- if (err)
- unable_to_lock_message(git_path(packed-refs),
- errno, err);
+ if (lock_packed_refs(err)) {
ret = -1;
goto cleanup;
}
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 04/17] refs.c: use a stringlist for repack_without_refs
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/remote.c | 23 ---
refs.c | 42 +-
refs.h | 2 +-
3 files changed, 30 insertions(+), 37 deletions(-)
diff --git a/builtin/remote.c b/builtin/remote.c
index c25420f..6806251 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -751,7 +751,6 @@ static int mv(int argc, const char **argv)
static int remove_branches(struct string_list *branches)
{
struct strbuf err = STRBUF_INIT;
- const char **branch_names;
int i, result = 0;
if (lock_packed_refs(0)) {
@@ -763,13 +762,9 @@ static int remove_branches(struct string_list *branches)
return -1;
}
- branch_names = xmalloc(branches-nr * sizeof(*branch_names));
- for (i = 0; i branches-nr; i++)
- branch_names[i] = branches-items[i].string;
- if (repack_without_refs(branch_names, branches-nr, err))
+ if (repack_without_refs(branches, err))
result |= error(%s, err.buf);
strbuf_release(err);
- free(branch_names);
for (i = 0; i branches-nr; i++) {
struct string_list_item *item = branches-items + i;
@@ -1327,7 +1322,6 @@ static int prune_remote(const char *remote, int dry_run)
int result = 0, i;
struct ref_states states;
struct string_list delete_refs_list = STRING_LIST_INIT_NODUP;
- const char **delete_refs;
const char *dangling_msg = dry_run
? _( %s will become dangling!)
: _( %s has become dangling!);
@@ -1335,6 +1329,11 @@ static int prune_remote(const char *remote, int dry_run)
memset(states, 0, sizeof(states));
get_remote_ref_states(remote, states, GET_REF_STATES);
+ for (i = 0; i states.stale.nr; i++) {
+ string_list_insert(delete_refs_list,
+ states.stale.items[i].util);
+ }
+
if (states.stale.nr) {
printf_ln(_(Pruning %s), remote);
printf_ln(_(URL: %s),
@@ -1342,9 +1341,6 @@ static int prune_remote(const char *remote, int dry_run)
? states.remote-url[0]
: _((no URL)));
- delete_refs = xmalloc(states.stale.nr * sizeof(*delete_refs));
- for (i = 0; i states.stale.nr; i++)
- delete_refs[i] = states.stale.items[i].util;
if (!dry_run) {
struct strbuf err = STRBUF_INIT;
@@ -1353,19 +1349,16 @@ static int prune_remote(const char *remote, int dry_run)
errno, err);
result |= error(%s, err.buf);
} else
- if (repack_without_refs(delete_refs,
- states.stale.nr, err))
+ if (repack_without_refs(delete_refs_list,
+ err))
result |= error(%s, err.buf);
strbuf_release(err);
}
- free(delete_refs);
}
for (i = 0; i states.stale.nr; i++) {
const char *refname = states.stale.items[i].util;
- string_list_insert(delete_refs_list, refname);
-
if (!dry_run)
result |= delete_ref(refname, NULL, 0);
diff --git a/refs.c b/refs.c
index 809bd3f..e9e321e 100644
--- a/refs.c
+++ b/refs.c
@@ -2663,31 +2663,32 @@ static int curate_packed_ref_fn(struct ref_entry
*entry, void *cb_data)
/*
* Must be called with packed refs already locked (and sorted)
*/
-int repack_without_refs(const char **refnames, int n, struct strbuf *err)
+int repack_without_refs(struct string_list *without, struct strbuf *err)
{
struct ref_dir *packed;
struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
struct string_list_item *ref_to_delete;
- int i, ret, removed = 0;
+ int count, ret, removed = 0;
assert(err);
/* Look for a packed ref */
- for (i = 0; i n; i++)
- if (get_packed_ref(refnames[i]))
- break;
+ count = 0;
+ for_each_string_list_item(ref_to_delete, without)
+ if (get_packed_ref(ref_to_delete-string))
+ count++;
- /* Avoid processing if we have nothing to do */
- if (i == n) {
+ /* No refname exists in packed refs */
+ if (!count) {
rollback_packed_refs();
- return 0; /* no refname exists in packed refs */
+ return 0;
}
packed = get_packed_refs(ref_cache);
/* Remove refnames from the cache */
- for (i = 0; i n; i++)
- if (remove_entry(packed, refnames[i]) != -1
[PATCH v2 06/17] refs.c: add new function copy_reflog_into_strbuf
Add a new function copy_reflog_into_strbuf() that is a backend specific
optimized way to read the whole reflog as is into a strbuf so we can
copy it elsewhere instead of having to iterate over all the entries using
the iterators.
For the current type of refs backend that use files we simple read_in_full
the whole file into a strbuf. When we add other types of backends for refs
we will need to implement similarly optimized versions of this functions for
them.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 31 +++
1 file changed, 31 insertions(+)
diff --git a/refs.c b/refs.c
index 3d13624..1c968f5 100644
--- a/refs.c
+++ b/refs.c
@@ -2826,6 +2826,37 @@ static int rename_ref_available(const char *oldname,
const char *newname)
static int write_ref_sha1(struct ref_lock *lock, const unsigned char *sha1,
const char *logmsg);
+/*
+ * This is an optimized function to read the whole reflog as a blob
+ * into a strbuf. It is used during ref_rename so that we can use an
+ * efficient method to read the whole log and later write it back to a
+ * different file.
+ */
+static int copy_reflog_into_strbuf(const char *refname, struct strbuf *buf)
+{
+ struct stat st;
+ int fd;
+
+ if (lstat(git_path(logs/%s, refname), st) == -1)
+ return 1;
+ if ((fd = open(git_path(logs/%s, refname), O_RDONLY)) == -1) {
+ error(failed to open reflog %s, %s,
+ refname, strerror(errno));
+ return 1;
+ }
+ strbuf_init(buf, st.st_size);
+ strbuf_setlen(buf, st.st_size);
+ if (read_in_full(fd, buf-buf, st.st_size) != st.st_size) {
+ close(fd);
+ error(failed to read reflog %s, %s,
+ refname, strerror(errno));
+ return 1;
+ }
+ close(fd);
+
+ return 0;
+}
+
int rename_ref(const char *oldrefname, const char *newrefname, const char
*logmsg)
{
unsigned char sha1[20], orig_sha1[20];
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 12/17] refs.c: make repack_without_refs static
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 2 +-
refs.h | 3 ---
2 files changed, 1 insertion(+), 4 deletions(-)
diff --git a/refs.c b/refs.c
index 57e5d2f..56b0360 100644
--- a/refs.c
+++ b/refs.c
@@ -2668,7 +2668,7 @@ static int curate_packed_ref_fn(struct ref_entry *entry,
void *cb_data)
/*
* Must be called with packed refs already locked (and sorted)
*/
-int repack_without_refs(struct string_list *without, struct strbuf *err)
+static int repack_without_refs(struct string_list *without, struct strbuf *err)
{
struct ref_dir *packed;
struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
diff --git a/refs.h b/refs.h
index ec4965f..f5ba534 100644
--- a/refs.h
+++ b/refs.h
@@ -163,9 +163,6 @@ extern void rollback_packed_refs(void);
*/
int pack_refs(unsigned int flags);
-extern int repack_without_refs(struct string_list *without,
- struct strbuf *err);
-
extern int ref_exists(const char *);
extern int is_branch(const char *refname);
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 09/17] refs.c: move reflog updates into its own function
write_ref_sha1 tries to update the reflog while updating the ref.
Move these reflog changes out into its own function so that we can do the
same thing if we write a sha1 ref differently, for example by writing a ref
to the packed refs file instead.
No functional changes intended. We only move some code out into a separate
function.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 60 +++-
1 file changed, 35 insertions(+), 25 deletions(-)
diff --git a/refs.c b/refs.c
index becf188..a3815d1 100644
--- a/refs.c
+++ b/refs.c
@@ -3033,6 +3033,40 @@ int is_branch(const char *refname)
return !strcmp(refname, HEAD) || starts_with(refname, refs/heads/);
}
+static int write_sha1_update_reflog(struct ref_lock *lock,
+ const unsigned char *sha1, const char *logmsg)
+{
+ if (log_ref_write(lock-ref_name, lock-old_sha1, sha1, logmsg) 0 ||
+ (strcmp(lock-ref_name, lock-orig_ref_name)
+log_ref_write(lock-orig_ref_name, lock-old_sha1, sha1, logmsg)
0)) {
+ unlock_ref(lock);
+ return -1;
+ }
+ if (strcmp(lock-orig_ref_name, HEAD) != 0) {
+ /*
+* Special hack: If a branch is updated directly and HEAD
+* points to it (may happen on the remote side of a push
+* for example) then logically the HEAD reflog should be
+* updated too.
+* A generic solution implies reverse symref information,
+* but finding all symrefs pointing to the given branch
+* would be rather costly for this rare event (the direct
+* update of a branch) to be worth it. So let's cheat and
+* check with HEAD only which should cover 99% of all usage
+* scenarios (even 100% of the default ones).
+*/
+ unsigned char head_sha1[20];
+ int head_flag;
+ const char *head_ref;
+ head_ref = resolve_ref_unsafe(HEAD, RESOLVE_REF_READING,
+ head_sha1, head_flag);
+ if (head_ref (head_flag REF_ISSYMREF)
+ !strcmp(head_ref, lock-ref_name))
+ log_ref_write(HEAD, lock-old_sha1, sha1, logmsg);
+ }
+ return 0;
+}
+
/*
* Write sha1 into the ref specified by the lock. Make sure that errno
* is sane on error.
@@ -3076,34 +3110,10 @@ static int write_ref_sha1(struct ref_lock *lock,
return -1;
}
clear_loose_ref_cache(ref_cache);
- if (log_ref_write(lock-ref_name, lock-old_sha1, sha1, logmsg) 0 ||
- (strcmp(lock-ref_name, lock-orig_ref_name)
-log_ref_write(lock-orig_ref_name, lock-old_sha1, sha1, logmsg)
0)) {
+ if (write_sha1_update_reflog(lock, sha1, logmsg)) {
unlock_ref(lock);
return -1;
}
- if (strcmp(lock-orig_ref_name, HEAD) != 0) {
- /*
-* Special hack: If a branch is updated directly and HEAD
-* points to it (may happen on the remote side of a push
-* for example) then logically the HEAD reflog should be
-* updated too.
-* A generic solution implies reverse symref information,
-* but finding all symrefs pointing to the given branch
-* would be rather costly for this rare event (the direct
-* update of a branch) to be worth it. So let's cheat and
-* check with HEAD only which should cover 99% of all usage
-* scenarios (even 100% of the default ones).
-*/
- unsigned char head_sha1[20];
- int head_flag;
- const char *head_ref;
- head_ref = resolve_ref_unsafe(HEAD, RESOLVE_REF_READING,
- head_sha1, head_flag);
- if (head_ref (head_flag REF_ISSYMREF)
- !strcmp(head_ref, lock-ref_name))
- log_ref_write(HEAD, lock-old_sha1, sha1, logmsg);
- }
if (commit_ref(lock)) {
error(Couldn't set %s, lock-ref_name);
unlock_ref(lock);
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 17/17] refs.c: add an err argument to pack_refs
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/pack-refs.c | 8 +++-
refs.c | 7 +++
refs.h | 3 ++-
3 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/builtin/pack-refs.c b/builtin/pack-refs.c
index b20b1ec..299768e 100644
--- a/builtin/pack-refs.c
+++ b/builtin/pack-refs.c
@@ -10,6 +10,7 @@ static char const * const pack_refs_usage[] = {
int cmd_pack_refs(int argc, const char **argv, const char *prefix)
{
unsigned int flags = PACK_REFS_PRUNE;
+ struct strbuf err = STRBUF_INIT;
struct option opts[] = {
OPT_BIT(0, all, flags, N_(pack everything),
PACK_REFS_ALL),
OPT_BIT(0, prune, flags, N_(prune loose refs (default)),
PACK_REFS_PRUNE),
@@ -17,5 +18,10 @@ int cmd_pack_refs(int argc, const char **argv, const char
*prefix)
};
if (parse_options(argc, argv, prefix, opts, pack_refs_usage, 0))
usage_with_options(pack_refs_usage, opts);
- return pack_refs(flags);
+ if (pack_refs(flags, err)) {
+ error(%s, err.buf);
+ strbuf_release(err);
+ return -1;
+ }
+ return 0;
}
diff --git a/refs.c b/refs.c
index 92e7714..c98d594 100644
--- a/refs.c
+++ b/refs.c
@@ -2593,16 +2593,15 @@ static void prune_refs(struct ref_to_prune *r)
}
}
-int pack_refs(unsigned int flags)
+int pack_refs(unsigned int flags, struct strbuf *err)
{
struct pack_refs_cb_data cbdata;
- struct strbuf err = STRBUF_INIT;
memset(cbdata, 0, sizeof(cbdata));
cbdata.flags = flags;
- if (lock_packed_refs(err))
- die(%s, err.buf);
+ if (lock_packed_refs(err))
+ return -1;
cbdata.packed_refs = get_packed_refs(ref_cache);
diff --git a/refs.h b/refs.h
index af20bd7..7c2f6c0 100644
--- a/refs.h
+++ b/refs.h
@@ -130,8 +130,9 @@ extern void warn_dangling_symrefs(FILE *fp, const char
*msg_fmt, const struct st
/*
* Write a packed-refs file for the current repository.
* flags: Combination of the above PACK_REFS_* flags.
+ * Returns 0 on success and fills in err on failure.
*/
-int pack_refs(unsigned int flags);
+int pack_refs(unsigned int flags, struct strbuf *err);
extern int ref_exists(const char *);
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 15/17] refs.c: make add_packed_ref return an error instead of calling die
Change add_packed_ref to return an error instead of calling die().
Update all callers to check the return value of add_packed_ref.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
refs.c | 21 +
1 file changed, 17 insertions(+), 4 deletions(-)
diff --git a/refs.c b/refs.c
index aefbce4..0b9cb8d 100644
--- a/refs.c
+++ b/refs.c
@@ -1229,15 +1229,16 @@ static struct ref_dir *get_packed_refs(struct ref_cache
*refs)
return get_packed_ref_dir(get_packed_ref_cache(refs));
}
-static void add_packed_ref(const char *refname, const unsigned char *sha1)
+static int add_packed_ref(const char *refname, const unsigned char *sha1)
{
struct packed_ref_cache *packed_ref_cache =
get_packed_ref_cache(ref_cache);
if (!packed_ref_cache-lock)
- die(internal error: packed refs not locked);
+ return -1;
add_ref(get_packed_ref_dir(packed_ref_cache),
create_ref_entry(refname, sha1, REF_ISPACKED, 1));
+ return 0;
}
/*
@@ -3829,7 +3830,13 @@ int transaction_commit(struct transaction *transaction,
sha1, NULL))
continue;
- add_packed_ref(update-refname, sha1);
+ if (add_packed_ref(update-refname, sha1)) {
+ if (err)
+ strbuf_addf(err, Failed to add %s to packed
+ refs, update-refname);
+ ret = -1;
+ goto cleanup;
+ }
need_repack = 1;
}
if (need_repack) {
@@ -3943,7 +3950,13 @@ int transaction_commit(struct transaction *transaction,
packed = get_packed_refs(ref_cache);
remove_entry(packed, update-refname);
- add_packed_ref(update-refname, update-new_sha1);
+ if (add_packed_ref(update-refname, update-new_sha1)) {
+ if (err)
+ strbuf_addf(err, Failed to add %s to packed
+ refs, update-refname);
+ ret = -1;
+ goto cleanup;
+ }
need_repack = 1;
try_remove_empty_parents((char *)update-refname);
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 08/17] refs.c: rollback the lockfile before we die() in repack_without_refs
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/refs.c b/refs.c
index 44d38ab..becf188 100644
--- a/refs.c
+++ b/refs.c
@@ -2707,8 +2707,10 @@ int repack_without_refs(struct string_list *without,
struct strbuf *err)
/* Remove any other accumulated cruft */
do_for_each_entry_in_dir(packed, 0, curate_packed_ref_fn,
refs_to_delete);
for_each_string_list_item(ref_to_delete, refs_to_delete) {
- if (remove_entry(packed, ref_to_delete-string) == -1)
+ if (remove_entry(packed, ref_to_delete-string) == -1) {
+ rollback_packed_refs();
die(internal error);
+ }
}
/* Write what remains */
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 10/17] refs.c: write updates to packed refs when a transaction has more than one ref
When we are updating more than one single ref, i.e. not a commit, then
write the updated refs directly to the packed refs file instead of writing
them as loose refs.
Change clone to use a transaction instead of using the packed refs API.
This changes the behavior of clone slightly. Previously clone would always
clone all refs into a packed refs file. With this change clone will only
clone into packed refs iff there are two or more refs being cloned.
If the repository we are cloning from only contains exactly one single ref
then clone will now store this as a loose ref. The benefit here is that
we no longer need to export a bunch of API functions to clone to access
packed refs directly. Clone can now just use a normal transaction and all
the packed refs goodness will happen automatically.
Update the t5516 test to cope with the fact that clone now only uses
packed refs if there are more than one ref being cloned.
We still use loose refs for single ref transactions, such as are used
by 'git commit' and friends. The reason for this is that if you have very
many refs then having to re-write the whole packed refs file for every
common operation like commit would have a performance impact.
That said, with these changes it should now be fairly straightforward to
add support to optionally start using packed refs for ALL updates
which could solve existing issues with name clashes in case insensitive
filesystems.
This change also means that multi-ref updates will now appear as a single
atomic change to any external observers instead of a sequence of discreete
changes.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/clone.c | 16 ++---
refs.c| 89 ++-
t/t5516-fetch-push.sh | 2 +-
3 files changed, 72 insertions(+), 35 deletions(-)
diff --git a/builtin/clone.c b/builtin/clone.c
index 316c75d..bb2c058 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -498,17 +498,25 @@ static struct ref *wanted_peer_refs(const struct ref
*refs,
static void write_remote_refs(const struct ref *local_refs)
{
const struct ref *r;
+ struct transaction *transaction;
+ struct strbuf err = STRBUF_INIT;
- lock_packed_refs(LOCK_DIE_ON_ERROR);
+ transaction = transaction_begin(err);
+ if (!transaction)
+ die(%s, err.buf);
for (r = local_refs; r; r = r-next) {
if (!r-peer_ref)
continue;
- add_packed_ref(r-peer_ref-name, r-old_sha1);
+ if (transaction_update_ref(transaction, r-peer_ref-name,
+ r-old_sha1, NULL, 0, 0, NULL,
+ err))
+ die(%s, err.buf);
}
- if (commit_packed_refs())
- die_errno(unable to overwrite old ref-pack file);
+ if (transaction_commit(transaction, err))
+ die(%s, err.buf);
+ transaction_free(transaction);
}
static void write_followtags(const struct ref *refs, const char *msg)
diff --git a/refs.c b/refs.c
index a3815d1..57e5d2f 100644
--- a/refs.c
+++ b/refs.c
@@ -2673,36 +2673,15 @@ int repack_without_refs(struct string_list *without,
struct strbuf *err)
struct ref_dir *packed;
struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
struct string_list_item *ref_to_delete;
- int count, ret, removed = 0;
+ int ret;
assert(err);
- /* Look for a packed ref */
- count = 0;
- for_each_string_list_item(ref_to_delete, without)
- if (get_packed_ref(ref_to_delete-string))
- count++;
-
- /* No refname exists in packed refs */
- if (!count) {
- rollback_packed_refs();
- return 0;
- }
-
packed = get_packed_refs(ref_cache);
/* Remove refnames from the cache */
for_each_string_list_item(ref_to_delete, without)
- if (remove_entry(packed, ref_to_delete-string) != -1)
- removed = 1;
- if (!removed) {
- /*
-* All packed entries disappeared while we were
-* acquiring the lock.
-*/
- rollback_packed_refs();
- return 0;
- }
+ remove_entry(packed, ref_to_delete-string);
/* Remove any other accumulated cruft */
do_for_each_entry_in_dir(packed, 0, curate_packed_ref_fn,
refs_to_delete);
@@ -3793,6 +3772,7 @@ int transaction_commit(struct transaction *transaction,
struct strbuf *err)
{
int ret = 0, i, need_repack = 0;
+ int num_updates = 0;
int n = transaction-nr;
struct packed_ref_cache *packed_ref_cache;
struct ref_update **updates = transaction-updates;
@@ -3826,14 +3806,30 @@ int transaction_commit(struct transaction *transaction,
goto
[PATCH v2 14/17] refs.c: replace the onerr argument in update_ref with a strbuf err
Get rid of the action_on_err enum and replace the action argument to
update_ref with a strbuf *err for error reporting.
Update all callers to the new api including two callers in transport*.c
which used the literal 0 instead of an enum.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/checkout.c | 7 +--
builtin/clone.c | 20
builtin/merge.c | 20 +---
builtin/notes.c | 24 ++--
builtin/reset.c | 12
builtin/update-ref.c | 7 +--
notes-cache.c| 2 +-
notes-utils.c| 5 +++--
refs.c | 14 +++---
refs.h | 10 ++
transport-helper.c | 7 ++-
transport.c | 9 ++---
12 files changed, 78 insertions(+), 59 deletions(-)
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 8550b6d..60a68f7 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -584,6 +584,8 @@ static void update_refs_for_switch(const struct
checkout_opts *opts,
{
struct strbuf msg = STRBUF_INIT;
const char *old_desc, *reflog_msg;
+ struct strbuf err = STRBUF_INIT;
+
if (opts-new_branch) {
if (opts-new_orphan_branch) {
if (opts-new_branch_log !log_all_ref_updates) {
@@ -621,8 +623,9 @@ static void update_refs_for_switch(const struct
checkout_opts *opts,
if (!strcmp(new-name, HEAD) !new-path !opts-force_detach) {
/* Nothing to do. */
} else if (opts-force_detach || !new-path) { /* No longer on any
branch. */
- update_ref(msg.buf, HEAD, new-commit-object.sha1, NULL,
- REF_NODEREF, UPDATE_REFS_DIE_ON_ERR);
+ if (update_ref(msg.buf, HEAD, new-commit-object.sha1, NULL,
+ REF_NODEREF, err))
+ die(%s, err.buf);
if (!opts-quiet) {
if (old-path advice_detached_head)
detach_advice(new-name);
diff --git a/builtin/clone.c b/builtin/clone.c
index bb2c058..5577b5b 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -522,6 +522,7 @@ static void write_remote_refs(const struct ref *local_refs)
static void write_followtags(const struct ref *refs, const char *msg)
{
const struct ref *ref;
+ struct strbuf err = STRBUF_INIT;
for (ref = refs; ref; ref = ref-next) {
if (!starts_with(ref-name, refs/tags/))
continue;
@@ -529,8 +530,9 @@ static void write_followtags(const struct ref *refs, const
char *msg)
continue;
if (!has_sha1_file(ref-old_sha1))
continue;
- update_ref(msg, ref-name, ref-old_sha1,
- NULL, 0, UPDATE_REFS_DIE_ON_ERR);
+ if (update_ref(msg, ref-name, ref-old_sha1,
+ NULL, 0, err))
+ die(%s, err.buf);
}
}
@@ -593,28 +595,30 @@ static void update_remote_refs(const struct ref *refs,
static void update_head(const struct ref *our, const struct ref *remote,
const char *msg)
{
+ struct strbuf err = STRBUF_INIT;
const char *head;
if (our skip_prefix(our-name, refs/heads/, head)) {
/* Local default branch link */
create_symref(HEAD, our-name, NULL);
if (!option_bare) {
- update_ref(msg, HEAD, our-old_sha1, NULL, 0,
- UPDATE_REFS_DIE_ON_ERR);
+ update_ref(msg, HEAD, our-old_sha1, NULL, 0, err);
install_branch_config(0, head, option_origin,
our-name);
}
} else if (our) {
struct commit *c = lookup_commit_reference(our-old_sha1);
/* --branch specifies a non-branch (i.e. tags), detach HEAD */
- update_ref(msg, HEAD, c-object.sha1,
- NULL, REF_NODEREF, UPDATE_REFS_DIE_ON_ERR);
+ if (update_ref(msg, HEAD, c-object.sha1,
+ NULL, REF_NODEREF, err))
+ die(%s, err.buf);
} else if (remote) {
/*
* We know remote HEAD points to a non-branch, or
* HEAD points to a branch but we don't know which one.
* Detach HEAD in all these cases.
*/
- update_ref(msg, HEAD, remote-old_sha1,
- NULL, REF_NODEREF, UPDATE_REFS_DIE_ON_ERR);
+ if (update_ref(msg, HEAD, remote-old_sha1,
+NULL, REF_NODEREF, err))
+ die(%s, err.buf);
}
}
diff --git a/builtin/merge.c b/builtin/merge.c
index bebbe5b..a787b6a 100644
--- a/builtin/merge.c
+++ b/builtin/merge.c
@@ -396,9 +396,11 @@ static void finish(struct commit *head_commit
[PATCH v2 5/7] t5543-atomic-push.sh: add basic tests for atomic pushes
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
t/t5543-atomic-push.sh | 101 +
1 file changed, 101 insertions(+)
create mode 100755 t/t5543-atomic-push.sh
diff --git a/t/t5543-atomic-push.sh b/t/t5543-atomic-push.sh
new file mode 100755
index 000..4903227
--- /dev/null
+++ b/t/t5543-atomic-push.sh
@@ -0,0 +1,101 @@
+#!/bin/sh
+
+test_description='pushing to a mirror repository'
+
+. ./test-lib.sh
+
+D=`pwd`
+
+invert () {
+ if $@; then
+ return 1
+ else
+ return 0
+ fi
+}
+
+mk_repo_pair () {
+ rm -rf master mirror
+ mkdir mirror
+ (
+ cd mirror
+ git init
+ git config receive.denyCurrentBranch warn
+ )
+ mkdir master
+ (
+ cd master
+ git init
+ git remote add $1 up ../mirror
+ )
+}
+
+
+test_expect_success 'atomic push works for a single branch' '
+
+ mk_repo_pair
+ (
+ cd master
+ echo one foo git add foo git commit -m one
+ git push --mirror up
+ echo two foo git add foo git commit -m two
+ git push --atomic-push --mirror up
+ )
+ master_master=$(cd master git show-ref -s --verify
refs/heads/master)
+ mirror_master=$(cd mirror git show-ref -s --verify
refs/heads/master)
+ test $master_master = $mirror_master
+
+'
+
+test_expect_success 'atomic push works for two branches' '
+
+ mk_repo_pair
+ (
+ cd master
+ echo one foo git add foo git commit -m one
+ git branch second
+ git push --mirror up
+ echo two foo git add foo git commit -m two
+ git checkout second
+ echo three foo git add foo git commit -m three
+ git checkout master
+ git push --atomic-push --mirror up
+ )
+ master_master=$(cd master git show-ref -s --verify
refs/heads/master)
+ mirror_master=$(cd mirror git show-ref -s --verify
refs/heads/master)
+ test $master_master = $mirror_master
+
+ master_second=$(cd master git show-ref -s --verify
refs/heads/second)
+ mirror_second=$(cd mirror git show-ref -s --verify
refs/heads/second)
+ test $master_second = $mirror_second
+'
+
+# set up two branches where master can be pushed but second can not
+# (non-fast-forward). Since second can not be pushed the whole operation
+# will fail and leave master untouched.
+test_expect_success 'atomic push fails if one branch fails' '
+ mk_repo_pair
+ (
+ cd master
+ echo one foo git add foo git commit -m one
+ git branch second
+ git checkout second
+ echo two foo git add foo git commit -m two
+ echo three foo git add foo git commit -m three
+ echo four foo git add foo git commit -m four
+ git push --mirror up
+ git reset --hard HEAD~2
+ git checkout master
+ echo five foo git add foo git commit -m five
+ ! git push --atomic-push --all up
+ )
+ master_master=$(cd master git show-ref -s --verify
refs/heads/master)
+ mirror_master=$(cd mirror git show-ref -s --verify
refs/heads/master)
+ test $master_master != $mirror_master
+
+ master_second=$(cd master git show-ref -s --verify
refs/heads/second)
+ mirror_second=$(cd mirror git show-ref -s --verify
refs/heads/second)
+ test $master_second != $mirror_second
+'
+
+test_done
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 4/7] push.c: add an --atomic-push argument
Add a command line argument to the git push command to request atomic
pushes.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
Documentation/git-push.txt | 7 ++-
builtin/push.c | 2 ++
transport.c| 1 +
transport.h| 1 +
4 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/Documentation/git-push.txt b/Documentation/git-push.txt
index 21b3f29..04de8d8 100644
--- a/Documentation/git-push.txt
+++ b/Documentation/git-push.txt
@@ -9,7 +9,7 @@ git-push - Update remote refs along with associated objects
SYNOPSIS
[verse]
-'git push' [--all | --mirror | --tags] [--follow-tags] [-n | --dry-run]
[--receive-pack=git-receive-pack]
+'git push' [--all | --mirror | --tags] [--follow-tags] [--atomic-push] [-n |
--dry-run] [--receive-pack=git-receive-pack]
[--repo=repository] [-f | --force] [--prune] [-v | --verbose]
[-u | --set-upstream] [--signed]
[--force-with-lease[=refname[:expect]]]
@@ -136,6 +136,11 @@ already exists on the remote side.
logged. See linkgit:git-receive-pack[1] for the details
on the receiving end.
+--atomic-push::
+ Try using atomic push. If atomic push is negotiated with the server
+ then any push covering multiple refs will be atomic. Either all
+ refs are updated, or on error, no refs are updated.
+
--receive-pack=git-receive-pack::
--exec=git-receive-pack::
Path to the 'git-receive-pack' program on the remote
diff --git a/builtin/push.c b/builtin/push.c
index ae56f73..0b9f21a 100644
--- a/builtin/push.c
+++ b/builtin/push.c
@@ -507,6 +507,8 @@ int cmd_push(int argc, const char **argv, const char
*prefix)
OPT_BIT(0, follow-tags, flags, N_(push missing but relevant
tags),
TRANSPORT_PUSH_FOLLOW_TAGS),
OPT_BIT(0, signed, flags, N_(GPG sign the push),
TRANSPORT_PUSH_CERT),
+ OPT_BIT(0, atomic-push, flags, N_(use atomic push, if
available),
+ TRANSPORT_ATOMIC_PUSH),
OPT_END()
};
diff --git a/transport.c b/transport.c
index 2111986..cd2b63a 100644
--- a/transport.c
+++ b/transport.c
@@ -833,6 +833,7 @@ static int git_transport_push(struct transport *transport,
struct ref *remote_re
args.dry_run = !!(flags TRANSPORT_PUSH_DRY_RUN);
args.porcelain = !!(flags TRANSPORT_PUSH_PORCELAIN);
args.push_cert = !!(flags TRANSPORT_PUSH_CERT);
+ args.use_atomic_push = !!(flags TRANSPORT_ATOMIC_PUSH);
args.url = transport-url;
ret = send_pack(args, data-fd, data-conn, remote_refs,
diff --git a/transport.h b/transport.h
index 3e0091e..25fa1da 100644
--- a/transport.h
+++ b/transport.h
@@ -125,6 +125,7 @@ struct transport {
#define TRANSPORT_PUSH_NO_HOOK 512
#define TRANSPORT_PUSH_FOLLOW_TAGS 1024
#define TRANSPORT_PUSH_CERT 2048
+#define TRANSPORT_ATOMIC_PUSH 4096
#define TRANSPORT_SUMMARY_WIDTH (2 * DEFAULT_ABBREV + 3)
#define TRANSPORT_SUMMARY(x) (int)(TRANSPORT_SUMMARY_WIDTH + strlen(x) -
gettext_width(x)), (x)
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 0/7] ref-transaction-send-pack
List, This series has been posted before but is now rebased on the previous ref-transaction-rename series that are against next. This series can also be found at : https://github.com/rsahlberg/git/tree/ref-transactions-send-pack This series finishes the transaction work to provide atomic pushes. With this series we can now perform atomic pushes to a repository. Version 2: - Reordered the capabilities we send so that agent= remains the last capability listed. - Reworded the paragraph for atomic push in git-send-pack.txt - Dropped the patch for receive.preferatomicpush Ronnie Sahlberg (7): receive-pack.c: add protocol support to negotiate atomic-push send-pack.c: add an --atomic-push command line argument receive-pack.c: use a single transaction when atomic-push is negotiated push.c: add an --atomic-push argument t5543-atomic-push.sh: add basic tests for atomic pushes refs.c: add an err argument to create_reflog refs.c: add an err argument to create_symref Documentation/git-push.txt| 7 +- Documentation/git-send-pack.txt | 7 +- Documentation/technical/protocol-capabilities.txt | 12 ++- builtin/branch.c | 7 +- builtin/checkout.c| 21 +++-- builtin/clone.c | 15 +++- builtin/init-db.c | 8 +- builtin/notes.c | 7 +- builtin/push.c| 2 + builtin/receive-pack.c| 79 + builtin/remote.c | 26 -- builtin/send-pack.c | 6 +- builtin/symbolic-ref.c| 6 +- cache.h | 1 - refs.c| 93 ++-- refs.h| 5 +- remote.h | 3 +- send-pack.c | 45 -- send-pack.h | 1 + t/t5543-atomic-push.sh| 101 ++ transport.c | 5 ++ transport.h | 1 + 22 files changed, 358 insertions(+), 100 deletions(-) create mode 100755 t/t5543-atomic-push.sh -- 2.1.0.rc2.206.gedb03e5 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 2/7] send-pack.c: add an --atomic-push command line argument
This adds support to send-pack to to negotiate and use atomic pushes
iff the server supports it. Atomic pushes are activated by a new command
line flag --atomic-push.
In order to do this we also need to change the semantics for send_pack()
slightly. The existing send_pack() function actually don't sent all the
refs back to the server when multiple refs are involved, for example
when using --all. Several of the failure modes for pushes can already be
detected locally in the send_pack client based on the information from the
initial server side list of all the refs as generated by receive-pack.
Any such refs that we thus know would fail to push are thus pruned from
the list of refs we send to the server to update.
For atomic pushes, we have to deal thus with both failures that are detected
locally as well as failures that are reported back from the server. In order
to do so we treat all local failures as push failures too.
We introduce a new status code REF_STATUS_ATOMIC_PUSH_FAILED so we can
flag all refs that we would normally have tried to push to the server
but we did not due to local failures. This is to improve the error message
back to the end user to flag that these refs failed to update since the
atomic push operation failed.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
Documentation/git-send-pack.txt | 7 ++-
builtin/send-pack.c | 6 +-
remote.h| 3 ++-
send-pack.c | 39 ++-
send-pack.h | 1 +
transport.c | 4
6 files changed, 52 insertions(+), 8 deletions(-)
diff --git a/Documentation/git-send-pack.txt b/Documentation/git-send-pack.txt
index 2a0de42..9296587 100644
--- a/Documentation/git-send-pack.txt
+++ b/Documentation/git-send-pack.txt
@@ -9,7 +9,7 @@ git-send-pack - Push objects over Git protocol to another
repository
SYNOPSIS
[verse]
-'git send-pack' [--all] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin] [host:]directory
[ref...]
+'git send-pack' [--all] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin] [--atomic-push]
[host:]directory [ref...]
DESCRIPTION
---
@@ -62,6 +62,11 @@ be in a separate packet, and the list must end with a flush
packet.
Send a thin pack, which records objects in deltified form based
on objects not included in the pack to reduce network traffic.
+--atomic-push::
+ Use an atomic transaction for updating the refs. If any of the refs
+ fails to update then the entire push will fail without changing any
+ refs.
+
host::
A remote host to house the repository. When this
part is specified, 'git-receive-pack' is invoked via
diff --git a/builtin/send-pack.c b/builtin/send-pack.c
index b564a77..93cb17c 100644
--- a/builtin/send-pack.c
+++ b/builtin/send-pack.c
@@ -13,7 +13,7 @@
#include sha1-array.h
static const char send_pack_usage[] =
-git send-pack [--all | --mirror] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin] [host:]directory
[ref...]\n
+git send-pack [--all | --mirror] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin] [--atomic-push]
[host:]directory [ref...]\n
--all and explicit ref specification are mutually exclusive.;
static struct send_pack_args args;
@@ -170,6 +170,10 @@ int cmd_send_pack(int argc, const char **argv, const char
*prefix)
args.use_thin_pack = 1;
continue;
}
+ if (!strcmp(arg, --atomic-push)) {
+ args.use_atomic_push = 1;
+ continue;
+ }
if (!strcmp(arg, --stateless-rpc)) {
args.stateless_rpc = 1;
continue;
diff --git a/remote.h b/remote.h
index 8b62efd..f346524 100644
--- a/remote.h
+++ b/remote.h
@@ -115,7 +115,8 @@ struct ref {
REF_STATUS_REJECT_SHALLOW,
REF_STATUS_UPTODATE,
REF_STATUS_REMOTE_REJECT,
- REF_STATUS_EXPECTING_REPORT
+ REF_STATUS_EXPECTING_REPORT,
+ REF_STATUS_ATOMIC_PUSH_FAILED
} status;
char *remote_status;
struct ref *peer_ref; /* when renaming */
diff --git a/send-pack.c b/send-pack.c
index 1ccc84c..08602a8 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -190,7 +190,7 @@ static void advertise_shallow_grafts_buf(struct strbuf *sb)
for_each_commit_graft(advertise_shallow_grafts_cb, sb);
}
-static int ref_update_to_be_sent(const struct ref *ref, const struct
send_pack_args *args)
+static int ref_update_to_be_sent(const struct ref *ref, const struct
send_pack_args *args, int *atomic_push_failed)
{
if (!ref-peer_ref !args-send_mirror
[PATCH v2 7/7] refs.c: add an err argument to create_symref
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/branch.c | 7 +--
builtin/checkout.c | 13 ++---
builtin/clone.c| 15 +++
builtin/init-db.c | 8 ++--
builtin/notes.c| 7 ---
builtin/remote.c | 26 ++
builtin/symbolic-ref.c | 6 +-
cache.h| 1 -
refs.c | 30 ++
refs.h | 1 +
10 files changed, 78 insertions(+), 36 deletions(-)
diff --git a/builtin/branch.c b/builtin/branch.c
index 04f57d4..ab6d9f4 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -698,6 +698,7 @@ static void rename_branch(const char *oldname, const char
*newname, int force)
{
struct strbuf oldref = STRBUF_INIT, newref = STRBUF_INIT, logmsg =
STRBUF_INIT;
struct strbuf oldsection = STRBUF_INIT, newsection = STRBUF_INIT;
+ struct strbuf err = STRBUF_INIT;
int recovery = 0;
int clobber_head_ok;
@@ -734,8 +735,10 @@ static void rename_branch(const char *oldname, const char
*newname, int force)
warning(_(Renamed a misnamed branch '%s' away), oldref.buf +
11);
/* no need to pass logmsg here as HEAD didn't really move */
- if (!strcmp(oldname, head) create_symref(HEAD, newref.buf, NULL))
- die(_(Branch renamed to %s, but HEAD is not updated!),
newname);
+ if (!strcmp(oldname, head)
+ create_symref(HEAD, newref.buf, NULL, err))
+ die(_(Branch renamed to %s, but HEAD is not updated!. %s),
+ newname, err.buf);
strbuf_addf(oldsection, branch.%s, oldref.buf + 11);
strbuf_release(oldref);
diff --git a/builtin/checkout.c b/builtin/checkout.c
index d9cb9c3..1efe353 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -634,7 +634,10 @@ static void update_refs_for_switch(const struct
checkout_opts *opts,
describe_detached_head(_(HEAD is now at),
new-commit);
}
} else if (new-path) { /* Switch branches. */
- create_symref(HEAD, new-path, msg.buf);
+ if (create_symref(HEAD, new-path, msg.buf, err)) {
+ error(%s, err.buf);
+ strbuf_release(err);
+ }
if (!opts-quiet) {
if (old-path !strcmp(new-path, old-path)) {
if (opts-new_branch_force)
@@ -1020,12 +1023,16 @@ static int parse_branchname_arg(int argc, const char
**argv,
static int switch_unborn_to_new_branch(const struct checkout_opts *opts)
{
int status;
- struct strbuf branch_ref = STRBUF_INIT;
+ struct strbuf branch_ref = STRBUF_INIT, err = STRBUF_INIT;
if (!opts-new_branch)
die(_(You are on a branch yet to be born));
strbuf_addf(branch_ref, refs/heads/%s, opts-new_branch);
- status = create_symref(HEAD, branch_ref.buf, checkout -b);
+ status = create_symref(HEAD, branch_ref.buf, checkout -b, err);
+ if (status) {
+ error(%s, err.buf);
+ strbuf_release(err);
+ }
strbuf_release(branch_ref);
if (!opts-quiet)
fprintf(stderr, _(Switched to a new branch '%s'\n),
diff --git a/builtin/clone.c b/builtin/clone.c
index 5577b5b..17b6ae8 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -565,6 +565,7 @@ static void update_remote_refs(const struct ref *refs,
int check_connectivity)
{
const struct ref *rm = mapped_refs;
+ struct strbuf err = STRBUF_INIT;
if (check_connectivity) {
if (transport-progress)
@@ -586,9 +587,12 @@ static void update_remote_refs(const struct ref *refs,
struct strbuf head_ref = STRBUF_INIT;
strbuf_addstr(head_ref, branch_top);
strbuf_addstr(head_ref, HEAD);
- create_symref(head_ref.buf,
- remote_head_points_at-peer_ref-name,
- msg);
+ if (create_symref(head_ref.buf,
+ remote_head_points_at-peer_ref-name,
+ msg, err)) {
+ error(%s, err.buf);
+ strbuf_release(err);
+ }
}
}
@@ -599,7 +603,10 @@ static void update_head(const struct ref *our, const
struct ref *remote,
const char *head;
if (our skip_prefix(our-name, refs/heads/, head)) {
/* Local default branch link */
- create_symref(HEAD, our-name, NULL);
+ if (create_symref(HEAD, our-name, NULL, err)) {
+ error(%s, err.buf);
+ strbuf_release(err);
+ }
if (!option_bare) {
update_ref(msg, HEAD, our-old_sha1, NULL, 0, err
[PATCH v2 3/7] receive-pack.c: use a single transaction when atomic-push is negotiated
Update receive-pack to use an atomic transaction iff the client negotiated
that it wanted atomic-push.
This leaves the default behavior to be the old non-atomic one ref at a
time update. This is to cause as little disruption as possible to existing
clients. It is unknown if there are client scripts that depend on the old
non-atomic behavior so we make it opt-in for now.
Later patch in this series also adds a configuration variable where you can
override the atomic push behavior on the receiving repo and force it
to use atomic updates always.
If it turns out over time that there are no client scripts that depend on the
old behavior we can change git to default to use atomic pushes and instead
offer an opt-out argument for people that do not want atomic pushes.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/receive-pack.c | 73 +++---
1 file changed, 58 insertions(+), 15 deletions(-)
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 65d9a7e..27b49dd 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -67,6 +67,8 @@ static const char *NONCE_SLOP = SLOP;
static const char *nonce_status;
static long nonce_stamp_slop;
static unsigned long nonce_stamp_slop_limit;
+struct strbuf err = STRBUF_INIT;
+struct transaction *transaction;
static enum deny_action parse_deny_action(const char *var, const char *value)
{
@@ -832,33 +834,55 @@ static const char *update(struct command *cmd, struct
shallow_info *si)
cmd-did_not_exist = 1;
}
}
- if (delete_ref(namespaced_name, old_sha1, 0)) {
- rp_error(failed to delete %s, name);
- return failed to delete;
+ if (!use_atomic_push) {
+ if (delete_ref(namespaced_name, old_sha1, 0)) {
+ rp_error(failed to delete %s, name);
+ return failed to delete;
+ }
+ } else {
+ if (transaction_delete_ref(transaction,
+ namespaced_name,
+ old_sha1,
+ 0, old_sha1 != NULL,
+ push, err)) {
+ rp_error(%s, err.buf);
+ strbuf_release(err);
+ return failed to delete;
+ }
}
return NULL; /* good */
}
else {
- struct strbuf err = STRBUF_INIT;
- struct transaction *transaction;
-
if (shallow_update si-shallow_ref[cmd-index]
update_shallow_ref(cmd, si))
return shallow error;
- transaction = transaction_begin(err);
- if (!transaction ||
- transaction_update_ref(transaction, namespaced_name,
- new_sha1, old_sha1, 0, 1, push,
- err) ||
- transaction_commit(transaction, err)) {
- transaction_free(transaction);
+ if (!use_atomic_push) {
+ transaction = transaction_begin(err);
+ if (!transaction) {
+ rp_error(%s, err.buf);
+ strbuf_release(err);
+ return failed to start transaction;
+ }
+ }
+ if (transaction_update_ref(transaction,
+ namespaced_name,
+ new_sha1, old_sha1,
+ 0, 1, push,
+ err)) {
rp_error(%s, err.buf);
strbuf_release(err);
return failed to update ref;
}
-
- transaction_free(transaction);
+ if (!use_atomic_push) {
+ if (transaction_commit(transaction, err)) {
+ transaction_free(transaction);
+ rp_error(%s, err.buf);
+ strbuf_release(err);
+ return failed to update ref;
+ }
+ transaction_free(transaction);
+ }
strbuf_release(err);
return NULL; /* good */
}
@@ -1058,6 +1082,16 @@ static void execute_commands(struct command *commands,
return;
}
+ if (use_atomic_push) {
+ transaction = transaction_begin(err);
+ if (!transaction
[PATCH v2 1/7] receive-pack.c: add protocol support to negotiate atomic-push
This adds support to the protocol between send-pack and receive-pack to
* allow receive-pack to inform the client that it has atomic push capability
* allow send-pack to request atomic push back.
There is currently no setting in send-pack to actually request that atomic
pushes are to be used yet. This only adds protocol capability not ability
for the user to activate it.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
Documentation/technical/protocol-capabilities.txt | 12 ++--
builtin/receive-pack.c| 6 +-
send-pack.c | 6 ++
3 files changed, 21 insertions(+), 3 deletions(-)
diff --git a/Documentation/technical/protocol-capabilities.txt
b/Documentation/technical/protocol-capabilities.txt
index 0c92dee..26bc5b1 100644
--- a/Documentation/technical/protocol-capabilities.txt
+++ b/Documentation/technical/protocol-capabilities.txt
@@ -18,8 +18,9 @@ was sent. Server MUST NOT ignore capabilities that client
requested
and server advertised. As a consequence of these rules, server MUST
NOT advertise capabilities it does not understand.
-The 'report-status', 'delete-refs', 'quiet', and 'push-cert' capabilities
-are sent and recognized by the receive-pack (push to server) process.
+The 'atomic-push', 'report-status', 'delete-refs', 'quiet', and 'push-cert'
+capabilities are sent and recognized by the receive-pack (push to server)
+process.
The 'ofs-delta' and 'side-band-64k' capabilities are sent and recognized
by both upload-pack and receive-pack protocols. The 'agent' capability
@@ -244,6 +245,13 @@ respond with the 'quiet' capability to suppress
server-side progress
reporting if the local progress reporting is also being suppressed
(e.g., via `push -q`, or if stderr does not go to a tty).
+atomic-push
+---
+
+If the server sends the 'atomic-push' capability, it means it is
+capable of accepting atomic pushes. If the pushing client requests this
+capability, the server will update the refs in one single atomic transaction.
+
allow-tip-sha1-in-want
--
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ccea9dc..65d9a7e 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -40,6 +40,7 @@ static int transfer_unpack_limit = -1;
static int unpack_limit = 100;
static int report_status;
static int use_sideband;
+static int use_atomic_push;
static int quiet;
static int prefer_ofs_delta = 1;
static int auto_update_server_info;
@@ -171,7 +172,8 @@ static void show_ref(const char *path, const unsigned char
*sha1)
struct strbuf cap = STRBUF_INIT;
strbuf_addstr(cap,
- report-status delete-refs side-band-64k quiet);
+ report-status delete-refs side-band-64k quiet
+ atomic-push);
if (prefer_ofs_delta)
strbuf_addstr(cap, ofs-delta);
if (push_cert_nonce)
@@ -1178,6 +1180,8 @@ static struct command *read_head_info(struct sha1_array
*shallow)
use_sideband = LARGE_PACKET_MAX;
if (parse_feature_request(feature_list, quiet))
quiet = 1;
+ if (parse_feature_request(feature_list, atomic-push))
+ use_atomic_push = 1;
}
if (!strcmp(line, push-cert)) {
diff --git a/send-pack.c b/send-pack.c
index 949cb61..1ccc84c 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -294,6 +294,8 @@ int send_pack(struct send_pack_args *args,
int use_sideband = 0;
int quiet_supported = 0;
int agent_supported = 0;
+ int atomic_push_supported = 0;
+ int atomic_push = 0;
unsigned cmds_sent = 0;
int ret;
struct async demux;
@@ -314,6 +316,8 @@ int send_pack(struct send_pack_args *args,
agent_supported = 1;
if (server_supports(no-thin))
args-use_thin_pack = 0;
+ if (server_supports(atomic-push))
+ atomic_push_supported = 1;
if (args-push_cert) {
int len;
@@ -335,6 +339,8 @@ int send_pack(struct send_pack_args *args,
strbuf_addstr(cap_buf, side-band-64k);
if (quiet_supported (args-quiet || !args-progress))
strbuf_addstr(cap_buf, quiet);
+ if (atomic_push)
+ strbuf_addstr(cap_buf, atomic-push);
if (agent_supported)
strbuf_addf(cap_buf, agent=%s, git_user_agent_sanitized());
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 6/7] refs.c: add an err argument to create_reflog
Add err argument to create_reflog that can explain the reason for a
failure. This then eliminates the need to manage errno through this
function since we can just add strerror(errno) to the err string when
meaningful. No callers relied on errno from this function for anything
else than the error message.
log_ref_write is a private function that calls create_reflog. Update
this function to also take an err argument and pass it back to the caller.
This again eliminates the need to manage errno in this function.
Update the private function write_sha1_update_reflog to also take an
err argument.
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
---
builtin/checkout.c | 8 +++---
refs.c | 71 +++---
refs.h | 4 +--
3 files changed, 42 insertions(+), 41 deletions(-)
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 60a68f7..d9cb9c3 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -590,10 +590,12 @@ static void update_refs_for_switch(const struct
checkout_opts *opts,
if (opts-new_orphan_branch) {
if (opts-new_branch_log !log_all_ref_updates) {
char *ref_name = mkpath(refs/heads/%s,
opts-new_orphan_branch);
+ struct strbuf err = STRBUF_INIT;
- if (create_reflog(ref_name)) {
- fprintf(stderr, _(Can not do reflog
for '%s'\n),
- opts-new_orphan_branch);
+ if (create_reflog(ref_name, err)) {
+ fprintf(stderr, _(Can not do reflog
for '%s'. %s\n),
+ opts-new_orphan_branch, err.buf);
+ strbuf_release(err);
return;
}
}
diff --git a/refs.c b/refs.c
index c98d594..04ed571 100644
--- a/refs.c
+++ b/refs.c
@@ -2900,8 +2900,7 @@ static int copy_msg(char *buf, const char *msg)
return cp - buf;
}
-/* This function must set a meaningful errno on failure */
-int create_reflog(const char *refname)
+int create_reflog(const char *refname, struct strbuf *err)
{
int logfd, oflags = O_APPEND | O_WRONLY;
char logfile[PATH_MAX];
@@ -2912,9 +2911,8 @@ int create_reflog(const char *refname)
starts_with(refname, refs/notes/) ||
!strcmp(refname, HEAD)) {
if (safe_create_leading_directories(logfile) 0) {
- int save_errno = errno;
- error(unable to create directory for %s, logfile);
- errno = save_errno;
+ strbuf_addf(err, unable to create directory for %s.
+ %s, logfile, strerror(errno));
return -1;
}
oflags |= O_CREAT;
@@ -2927,20 +2925,16 @@ int create_reflog(const char *refname)
if ((oflags O_CREAT) errno == EISDIR) {
if (remove_empty_directories(logfile)) {
- int save_errno = errno;
- error(There are still logs under '%s',
- logfile);
- errno = save_errno;
+ strbuf_addf(err, There are still logs under
+ '%s', logfile);
return -1;
}
logfd = open(logfile, oflags, 0666);
}
if (logfd 0) {
- int save_errno = errno;
- error(Unable to append to %s: %s, logfile,
- strerror(errno));
- errno = save_errno;
+ strbuf_addf(err, Unable to append to %s: %s,
+ logfile, strerror(errno));
return -1;
}
}
@@ -2977,7 +2971,8 @@ static int log_ref_write_fd(int fd, const unsigned char
*old_sha1,
}
static int log_ref_write(const char *refname, const unsigned char *old_sha1,
-const unsigned char *new_sha1, const char *msg)
+const unsigned char *new_sha1, const char *msg,
+struct strbuf *err)
{
int logfd, result = 0, oflags = O_APPEND | O_WRONLY;
char log_file[PATH_MAX];
@@ -2986,7 +2981,7 @@ static int log_ref_write(const char *refname, const
unsigned char *old_sha1,
log_all_ref_updates = !is_bare_repository();
if (log_all_ref_updates !reflog_exists(refname))
- result = create_reflog(refname);
+ result = create_reflog(refname, err);
if (result)
return
Re: [PATCH 05/15] refs.c: update rename_ref to use a transaction
On Wed, Oct 29, 2014 at 11:43 AM, Junio C Hamano gits...@pobox.com wrote: Ronnie Sahlberg sahlb...@google.com writes: On Tue, Oct 28, 2014 at 2:12 PM, Junio C Hamano gits...@pobox.com wrote: More importantly, when you know that the end result you want to see is that the old and new log files are bit-for-bit identical, and if not there is some bug in either parsing or formatting, why parse the old and reformat into the new? What would happen when there were malformed entries in the old that makes your parsing fail? Fair enough. I will change it to ONLY use a transaction for the actual ref update and keep using rename() for the reflog handling. Only real change I will do for the reflog handling is to change the temporary file name used to be less collission prone if there are two renames happening at the same time so that they don't destroy each others reflogs. I think it is a good idea to make renaming the entire reflog a logical element of transaction (as you mentioned in our private discussion) to allow different backends implement in their best efficient robust way. Right. I have changed it to use an optimized function to read the whole existing reflog as a blob into a strbuf and then a new transaction function transaction_replace_reflog(... the-blob ...) to write the whole blob back to the new location. And for filesystem-backed backends, I actually think keep the original until we know we do not have to roll back, that follows the same pattern for the other transactional updates, is a good implementation of that best efficient robust way, compared to the original just rename it. It frees us from having to be worried about what happens if we cannot rename it back. Thanks. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/8] receive-pack.c: add protocol support to negotiate atomic-push
On Thu, Oct 30, 2014 at 12:59 PM, Junio C Hamano gits...@pobox.com wrote: Ronnie Sahlberg sahlb...@google.com writes: @@ -337,6 +341,8 @@ int send_pack(struct send_pack_args *args, strbuf_addstr(cap_buf, quiet); if (agent_supported) strbuf_addf(cap_buf, agent=%s, git_user_agent_sanitized()); + if (atomic_push) + strbuf_addstr(cap_buf, atomic-push); This is just aesthetics, but please leave the agent at the end. Done. Thanks. Will be in the re-roll. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/8] send-pack.c: add an --atomic-push command line argument
On Thu, Oct 30, 2014 at 1:04 PM, Junio C Hamano gits...@pobox.com wrote:
Ronnie Sahlberg sahlb...@google.com writes:
diff --git a/Documentation/git-send-pack.txt
b/Documentation/git-send-pack.txt
index 2a0de42..8f64feb 100644
--- a/Documentation/git-send-pack.txt
+++ b/Documentation/git-send-pack.txt
@@ -9,7 +9,7 @@ git-send-pack - Push objects over Git protocol to another
repository
SYNOPSIS
[verse]
-'git send-pack' [--all] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin]
[host:]directory [ref...]
+'git send-pack' [--all] [--dry-run] [--force]
[--receive-pack=git-receive-pack] [--verbose] [--thin] [--atomic-push]
[host:]directory [ref...]
DESCRIPTION
---
@@ -62,6 +62,11 @@ be in a separate packet, and the list must end with a
flush packet.
Send a thin pack, which records objects in deltified form based
on objects not included in the pack to reduce network traffic.
+--atomic-push::
+ With atomic-push all refs are updated in one single atomic transaction.
+ This means that if any of the refs fails then the entire push will
+ fail without changing any refs.
Whenever you say This means that, please read it twice to see if
everything before and including that phrase can be removed. It is a
sign that you found what you wrote before it is not understandable
and what follows is the version that would be understood by the
readers.
I reworded it for the next re-roll.
@@ -203,6 +203,13 @@ static int ref_update_to_be_sent(const struct ref *ref,
const struct send_pack_a
case REF_STATUS_REJECT_NEEDS_FORCE:
case REF_STATUS_REJECT_STALE:
case REF_STATUS_REJECT_NODELETE:
+ if (atomic_push_failed args-use_atomic_push) {
Hmph. Can we even get atomis_push_failed when args-use_atomic_push
is not set?
It can, but it looks wrong to have one variable passed in via an
argument and the other variable being a global.
I changed it to be
if (atomic_push_failed) {
instead and only pass in a non-NULL atomic_push_failed by the caller
iff args-use_atomic_push is true.
+ fprintf(stderr, Atomic push is not possible
+ for ref %s. Status:%d\n, ref-name,
+ ref-status);
+ *atomic_push_failed = 1;
+ }
+ /* fallthrough */
Is this is not possible (hence we do not let the user use it), or
failed (i.e. we tried and we failed)?
I reworded it to say failed instead.
Will be part of next re-roll.
Thanks
Ronnie Sahlberg
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] receive-pack.c: add a receive.preferatomicpush configuration variable
On Thu, Oct 30, 2014 at 1:11 PM, Junio C Hamano gits...@pobox.com wrote: Ronnie Sahlberg sahlb...@google.com writes: Add receive.preferatomicpush setting to receive-pack.c. This triggers a new capability prefer-atomic-push to be sent back to the send-pack client, requesting the client, if it supports it, to request an atomic push. I can understand a configuration that says We take only atomics when a push tries to update more than one, but this one is iffy. If the receiver accepts non-atomic from older send-pack, those with newer send-pack should have a way to say the receiving end may prefer atomic, but I choose not to. Is there a way to do so? There is no way to do so right now. I can add a --no-atomic-push argument to the client to make it ignore this hint from the server. And if there is such a way, what value does the preference add to the user experience and the server's operation? The reason for this preference was to make it possible to flag a repository to always try to make all pushes atomic, when possible. A preference for convenience. You could set on the server to try to make all pushes atomic so that clients do not have to specify --atomic-push all the time. Currently this is just a hint on from the server and is not enforced for backward compatibility reasons. If the client does not have atomic push support, the client will ignore this hint and just do a normal push instead. This is the least important part in this patch series so I am open for advice : 1, I can remove this patch/preference if you prefer. or 2a, I can add a --no-atomic-push flag to send-pack to have a way to force the client to do a normal nonatomic push even if the server indicates it wants atomic pushes. and/or 2b, I can add another preference receive.requireatomicpush to the server so that the server can reject any push outright if it is not atomic. At some stage it may becomes too many preferences and over-engineered. Maybe I should drop this patch and then just require the plain if you want a push to be atomic, then use --atomic-push. end. and we have simple and easy to understand semantics. Please advise. Regards Ronnie Sahlberg -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] receive-pack.c: add a receive.preferatomicpush configuration variable
On Thu, Oct 30, 2014 at 3:03 PM, Junio C Hamano gits...@pobox.com wrote: Ronnie Sahlberg sahlb...@google.com writes: At some stage it may becomes too many preferences and over-engineered. Maybe I should drop this patch and then just require the plain if you want a push to be atomic, then use --atomic-push. end. and we have simple and easy to understand semantics. As I still do not quite understand why you find that this could be a convenience preference on the server operator's end, that would be my preference, at least until I am convinced why this could be a good idea. Thanks. Ok, I dropped this patch. Thanks Ronnie Sahlberg -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] refs.c: update rename_ref to use a transaction
On Tue, Oct 28, 2014 at 2:12 PM, Junio C Hamano gits...@pobox.com wrote: Ronnie Sahlberg sahlb...@google.com writes: I timed a git branch -m for a branch with ~2400 log entries and it takes neglible time : real 0m0.008s user 0m0.000s sys 0m0.007s I really hate this line of reasoning. Small things tend to add up. More importantly, when you know that the end result you want to see is that the old and new log files are bit-for-bit identical, and if not there is some bug in either parsing or formatting, why parse the old and reformat into the new? What would happen when there were malformed entries in the old that makes your parsing fail? Fair enough. I will change it to ONLY use a transaction for the actual ref update and keep using rename() for the reflog handling. Only real change I will do for the reflog handling is to change the temporary file name used to be less collission prone if there are two renames happening at the same time so that they don't destroy each others reflogs. -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 09/15] refs.c: only write reflog update if msg is non-NULL
On Thu, Oct 23, 2014 at 11:32 AM, Junio C Hamano gits...@pobox.com wrote:
Ronnie Sahlberg sahlb...@google.com writes:
commit 020ed65a12838bdead64bc3c5de249d3c8f5cfd8 upstream.
When performing a reflog transaction update, only write to the reflog iff
msg is non-NULL. This can then be combined with REFLOG_TRUNCATE to perform
an update that only truncates but does not write.
Does any existing caller call this codepath with update-msg == NULL?
Will please truncate stay to be the only plausible special cause
to call into this codepath without having any meaningful message?
I am trying to make sure that this patch is not painting us into a
corner where we will find out another reason for doing something
esoteric in this codepath but need to find a way other than setting
msg to NULL for the caller to trigger that new codepath. Put it in
another way, please convince me that a new boolean field in update,
e.g. update-truncate_reflog, is way overkill for this.
This change only affects whether or not a reflog entry will be emitted
by the update.
msg==NULL means we will not create a new entry. This is orthogonal to
whether we truncate the log or not.
In order to truncate the reflog we do have a boolean in update-flags
REFLOG_TRUNCATE
which determines whether the update will truncate the log or not.
I see these two actions a) write a log entry and b) truncate the log
as orthogonal and thus think we should have separate knobs for them.
Currently, modulo bugs, the only caller that will use msg==NULL is
when we do reflog truncations. Thus that codepath BOTH sets msg==NULL
(to not write a new log entry) and also sets
update-flagsREFLOG_TRUNCATE (to truncate the log).
Having separate knobs for the two actions allow us the current behaviour with:
msg==NULL update-flagsREFLOG_TRUNCATE
but it will also allow a caller to do things like
msg=truncated by foo because ... update-flagsREFLOG_TRUNCATE
If there is some future usecase where we want to truncate the log and
then also generate a new initial log entry for the new log.
I will work on the commit message to make the distinction between
msg==NULL and update-flagsREFLOG_TRUNCATE more clear.
thanks
ronnie sahlberg
Change-Id: I44c89caa7e7c4960777b79cfb5d339a5aa3ddf7a
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 5 +++--
refs.h | 1 +
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index d54c3b9..f14b76e 100644
--- a/refs.c
+++ b/refs.c
@@ -3895,8 +3895,9 @@ int transaction_commit(struct transaction *transaction,
update-reflog_fd = -1;
continue;
}
- if (log_ref_write_fd(update-reflog_fd, update-old_sha1,
- update-new_sha1,
+ if (update-msg
+ log_ref_write_fd(update-reflog_fd,
+ update-old_sha1, update-new_sha1,
update-committer, update-msg)) {
error(Could write to reflog: %s. %s,
update-refname, strerror(errno));
diff --git a/refs.h b/refs.h
index 5075073..bf96b36 100644
--- a/refs.h
+++ b/refs.h
@@ -337,6 +337,7 @@ int transaction_delete_ref(struct transaction
*transaction,
/*
* Append a reflog entry for refname. If the REFLOG_TRUNCATE flag is set
* this update will first truncate the reflog before writing the entry.
+ * If msg is NULL no update will be written to the log.
*/
int transaction_update_reflog(struct transaction *transaction,
const char *refname,
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 10/15] refs.c: allow multiple reflog updates during a single transaction
On Thu, Oct 23, 2014 at 11:54 AM, Junio C Hamano gits...@pobox.com wrote:
Ronnie Sahlberg sahlb...@google.com writes:
@@ -3531,7 +3537,7 @@ struct ref_update {
enum transaction_update_type update_type;
unsigned char new_sha1[20];
unsigned char old_sha1[20];
- int flags; /* REF_NODEREF? */
+ int flags; /* REF_NODEREF? or private flags */
Not a very informative comment, I'd have to say. How are users of
this API expected to avoid stepping on each others' and API
implementation's toes?
That is an old and bitrotted comment.
I changed it to point to the canonical definition of these flags instead :
int flags; /* The flags to transaction_update_ref[log] are defined
* in refs.h
*/
@@ -3539,8 +3545,9 @@ struct ref_update {
/* used by reflog updates */
int reflog_fd;
- struct lock_file reflog_lock;
+ struct lock_file *reflog_lock;
What is this change about?
We have one update entry for each line we want to write to the reflog.
So for the first update to a reflog we allocate a lock_file structure.
For any subsequent reflog entry to the same ref we just let the
pointer point to the previous structure we already allocated.
I.e. a way to have only one lock_file structure and share it across
multiple struct ref_update structures.
Does the lifetime rule for struct lock_file described in
Documentation/technical/api-lockfile.txt, namely, once you call
hold_lock_file_* family on it, you cannot free it yourself, have
any implication on this?
Nope.
+ if (!(update-flags UPDATE_REFLOG_NOLOCK))
+ update-reflog_lock = xcalloc(1, sizeof(struct lock_file));
+
Hmph, does this mean that the caller needs to keep track of the refs
it ever touched inside a single transaction, call this without nolock
on the first invocation on a particular ref and with nolock on the
subsequent invocation?
Nope. This is not visible to the caller and is managed transparently
inside the transaction code.
Or is the caller just implementation detail of the API and higher level
callers do not have to care?
The latter,the higher level do not have to care.
regards
ronnie sahlberg
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 05/15] refs.c: update rename_ref to use a transaction
On Tue, Oct 28, 2014 at 12:56 PM, Junio C Hamano gits...@pobox.com wrote: Junio C Hamano gits...@pobox.com writes: Ronnie Sahlberg sahlb...@google.com writes: commit 0295e9cebc41020ee84da275549b164a8770ffba upstream. Change refs.c to use a single transaction to copy/rename both the refs and its reflog. Since we are no longer using rename() to move the reflog file we no longer need to disallow rename_ref for refs with a symlink for its reflog so we can remove that test from the testsuite. Do you mean that we used to do a single rename(2) to move the entire logfile, but now you copy potentially thousands of reflog entries one by one? H,... is that an improvement? I think so. It makes to code a lot simpler and more atomic. As a side effect it removes restrictions for symlink handling and eliminates the two renames colliding race. Though, a read and then rewrite thousands of reflog entries will be slower than a single rename() syscall. I see some value in keep the original while creating a new one, just in case we fail to fully recreate the new one so that we can roll back with less programming effort. But still, we should be able to copy the original to new without parsing and reformatting each and every entry, no? Is renaming a branch with a long history is such a frequent or time critical event? I timed a git branch -m for a branch with ~2400 log entries and it takes neglible time : real 0m0.008s user 0m0.000s sys 0m0.007s During the special rename case, we are deleting one ref and creating another. For cases such as m-m/m or the reverse we must delete the old file/directory before we can create the new one. The old rename code did this by renaming the file out to a common directory and then back to the new location. Which is fast (but a bit ...) The alternative is to read the old file into memory, delete it and then write the content back to the new location, which is kind of what the new code does. If this turns out to be a bottleneck we can change the io when writing the reflog entries to use fwrite(). Lets see if there is a problem first. regards ronnie sahlberg -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 01/15] refs.c make ref_transaction_create a wrapper to ref_transaction_update
On Thu, Oct 23, 2014 at 10:42 AM, Junio C Hamano gits...@pobox.com wrote:
Ronnie Sahlberg sahlb...@google.com writes:
Subject: Re: [PATCH 01/15] refs.c make ref_transaction_create a wrapper to
ref_transaction_update
Missing colon after refs.c
commit 03001144a015f81db5252005841bb78f57d62774 upstream.
Huh?
The ref_transaction_update function can already be used to create refs by
passing null_sha1 as the old_sha1 parameter. Simplify by replacing
transaction_create with a thin wrapper.
Nice code reduction.
Change-Id: I687dd47cc4f4e06766e8313b4fd1b07cd4a56c1a
Please don't leak local housekeeping details into the official
history.
Ah, Ok.
Do you want me to re-send the series with these lines deleted ?
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 27 ++-
1 file changed, 2 insertions(+), 25 deletions(-)
diff --git a/refs.c b/refs.c
index 0368ed4..ed0485e 100644
--- a/refs.c
+++ b/refs.c
@@ -3623,31 +3623,8 @@ int ref_transaction_create(struct ref_transaction
*transaction,
int flags, const char *msg,
struct strbuf *err)
{
- struct ref_update *update;
-
- assert(err);
-
- if (transaction-state != REF_TRANSACTION_OPEN)
- die(BUG: create called for transaction that is not open);
-
- if (!new_sha1 || is_null_sha1(new_sha1))
- die(BUG: create ref with null new_sha1);
-
- if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
- strbuf_addf(err, refusing to create ref with bad name %s,
- refname);
- return -1;
- }
-
- update = add_update(transaction, refname);
-
- hashcpy(update-new_sha1, new_sha1);
- hashclr(update-old_sha1);
- update-flags = flags;
- update-have_old = 1;
- if (msg)
- update-msg = xstrdup(msg);
- return 0;
+ return ref_transaction_update(transaction, refname, new_sha1,
+ null_sha1, flags, 1, msg, err);
}
int ref_transaction_delete(struct ref_transaction *transaction,
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/15] copy.c: make copy_fd preserve meaningful errno
On Thu, Oct 23, 2014 at 10:51 AM, Junio C Hamano gits...@pobox.com wrote:
Ronnie Sahlberg sahlb...@google.com writes:
commit 306805ccd147bfdf160b288a8d51fdf9b77ae0fa upstream.
Update copy_fd to return a meaningful errno on failure.
These two are good changes, but makes me wonder if more places
benefit from having error_errno() that does the save away errno,
use strerror(errno) to give an error message and restore errno
for us.
Not meant as a suggestion for further changes in this series, but as
a future discussion item.
That sounds like a good idea.
As a separate series once these are done, I can volunteer to go
through all the errno handling in git
and look into that.
Change-Id: I771f9703acc816902affcf35ff2a56d9426315ac
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
copy.c | 15 ++-
1 file changed, 10 insertions(+), 5 deletions(-)
diff --git a/copy.c b/copy.c
index f2970ec..a8d366e 100644
--- a/copy.c
+++ b/copy.c
@@ -8,12 +8,17 @@ int copy_fd(int ifd, int ofd)
if (!len)
break;
if (len 0) {
- return error(copy-fd: read returned %s,
- strerror(errno));
+ int save_errno = errno;
+ error(copy-fd: read returned %s, strerror(errno));
+ errno = save_errno;
+ return -1;
+ }
+ if (write_in_full(ofd, buffer, len) 0) {
+ int save_errno = errno;
+ error(copy-fd: write returned %s, strerror(errno));
+ errno = save_errno;
+ return -1;
}
- if (write_in_full(ofd, buffer, len) 0)
- return error(copy-fd: write returned %s,
- strerror(errno));
}
return 0;
}
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 08/15] refs.c: add a flag to allow reflog updates to truncate the log
commit 8d0a342375fbd926ae6ae93f9be42a436a787fb6 upstream.
Add a flag that allows us to truncate the reflog before we write the
update.
Change-Id: I187b22e0910499b1976689b46866698b6fd46eb3
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 17 +++--
refs.h | 10 +-
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/refs.c b/refs.c
index 100b3a3..d54c3b9 100644
--- a/refs.c
+++ b/refs.c
@@ -3873,7 +3873,12 @@ int transaction_commit(struct transaction *transaction,
}
}
- /* Update all reflog files */
+ /*
+* Update all reflog files
+* We have already done all ref updates and deletes.
+* There is not much we can do here if there are any reflog
+* update errors other than complain.
+*/
for (i = 0; i n; i++) {
struct ref_update *update = updates[i];
@@ -3881,7 +3886,15 @@ int transaction_commit(struct transaction *transaction,
continue;
if (update-reflog_fd == -1)
continue;
-
+ if (update-flags REFLOG_TRUNCATE)
+ if (lseek(update-reflog_fd, 0, SEEK_SET) 0 ||
+ ftruncate(update-reflog_fd, 0)) {
+ error(Could not truncate reflog: %s. %s,
+ update-refname, strerror(errno));
+ rollback_lock_file(update-reflog_lock);
+ update-reflog_fd = -1;
+ continue;
+ }
if (log_ref_write_fd(update-reflog_fd, update-old_sha1,
update-new_sha1,
update-committer, update-msg)) {
diff --git a/refs.h b/refs.h
index 8220d18..5075073 100644
--- a/refs.h
+++ b/refs.h
@@ -328,7 +328,15 @@ int transaction_delete_ref(struct transaction *transaction,
struct strbuf *err);
/*
- * Append a reflog entry for refname.
+ * Flags controlling transaction_update_reflog().
+ * REFLOG_TRUNCATE: Truncate the reflog.
+ *
+ * Flags = 0x100 are reserved for internal use.
+ */
+#define REFLOG_TRUNCATE 0x01
+/*
+ * Append a reflog entry for refname. If the REFLOG_TRUNCATE flag is set
+ * this update will first truncate the reflog before writing the entry.
*/
int transaction_update_reflog(struct transaction *transaction,
const char *refname,
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 07/15] refs.c: add a transaction function to append a reflog entry
commit de045215a52a6a9591b0e786488de2293d79d245 upstream.
Define a new transaction update type, UPDATE_LOG, and a new function
transaction_update_reflog. This function will lock the reflog and append
an entry to it during transaction commit.
Change-Id: I8cc935ef311688d561d447fa51b44ac98492693b
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 102 +++--
refs.h | 12
2 files changed, 112 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index 0e11b1c..100b3a3 100644
--- a/refs.c
+++ b/refs.c
@@ -3517,7 +3517,8 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
}
enum transaction_update_type {
- UPDATE_SHA1 = 0
+ UPDATE_SHA1 = 0,
+ UPDATE_LOG = 1
};
/**
@@ -3535,6 +3536,12 @@ struct ref_update {
struct ref_lock *lock;
int type;
char *msg;
+
+ /* used by reflog updates */
+ int reflog_fd;
+ struct lock_file reflog_lock;
+ char *committer;
+
const char refname[FLEX_ARRAY];
};
@@ -3581,6 +3588,7 @@ void transaction_free(struct transaction *transaction)
for (i = 0; i transaction-nr; i++) {
free(transaction-updates[i]-msg);
+ free(transaction-updates[i]-committer);
free(transaction-updates[i]);
}
free(transaction-updates);
@@ -3601,6 +3609,41 @@ static struct ref_update *add_update(struct transaction
*transaction,
return update;
}
+int transaction_update_reflog(struct transaction *transaction,
+ const char *refname,
+ const unsigned char *new_sha1,
+ const unsigned char *old_sha1,
+ const unsigned char *email,
+ unsigned long timestamp, int tz,
+ const char *msg, int flags,
+ struct strbuf *err)
+{
+ struct ref_update *update;
+
+ if (transaction-state != TRANSACTION_OPEN)
+ die(BUG: update_reflog called for transaction that is not
open);
+
+ update = add_update(transaction, refname, UPDATE_LOG);
+ hashcpy(update-new_sha1, new_sha1);
+ hashcpy(update-old_sha1, old_sha1);
+ update-reflog_fd = -1;
+ if (email) {
+ struct strbuf buf = STRBUF_INIT;
+ char sign = (tz 0) ? '-' : '+';
+ int zone = (tz 0) ? (-tz) : tz;
+
+ strbuf_addf(buf, %s %lu %c%04d, email, timestamp, sign,
+ zone);
+ update-committer = xstrdup(buf.buf);
+ strbuf_release(buf);
+ }
+ if (msg)
+ update-msg = xstrdup(msg);
+ update-flags = flags;
+
+ return 0;
+}
+
int transaction_update_ref(struct transaction *transaction,
const char *refname,
const unsigned char *new_sha1,
@@ -3773,7 +3816,28 @@ int transaction_commit(struct transaction *transaction,
}
}
- /* Perform updates first so live commits remain referenced */
+ /* Lock all reflog files */
+ for (i = 0; i n; i++) {
+ struct ref_update *update = updates[i];
+
+ if (update-update_type != UPDATE_LOG)
+ continue;
+ update-reflog_fd = hold_lock_file_for_append(
+ update-reflog_lock,
+ git_path(logs/%s, update-refname),
+ 0);
+ if (update-reflog_fd 0) {
+ const char *str = Cannot lock reflog for '%s'. %s;
+
+ ret = -1;
+ if (err)
+ strbuf_addf(err, str, update-refname,
+ strerror(errno));
+ goto cleanup;
+ }
+ }
+
+ /* Perform ref updates first so live commits remain referenced */
for (i = 0; i n; i++) {
struct ref_update *update = updates[i];
@@ -3809,6 +3873,40 @@ int transaction_commit(struct transaction *transaction,
}
}
+ /* Update all reflog files */
+ for (i = 0; i n; i++) {
+ struct ref_update *update = updates[i];
+
+ if (update-update_type != UPDATE_LOG)
+ continue;
+ if (update-reflog_fd == -1)
+ continue;
+
+ if (log_ref_write_fd(update-reflog_fd, update-old_sha1,
+update-new_sha1,
+update-committer, update-msg)) {
+ error(Could write to reflog: %s. %s,
+ update-refname, strerror(errno
[PATCH 12/15] refs.c: rename log_ref_setup to create_reflog
commit 8f78d5d7a1e62f04c75524bb3e87aaad24372250 upstream.
log_ref_setup is used to do several semi-related things :
* sometimes it will create a new reflog including missing parent directories
and cleaning up any conflicting stale directories in the path.
* fill in a filename buffer for the full path to the reflog.
* unconditionally re-adjust the permissions for the file.
This function is only called from two places: checkout.c where it is always
used to create a reflog and refs.c:log_ref_write where it sometimes are
used to create a reflog and sometimes just used to fill in the filename.
Rename log_ref_setup to create_reflog and change it to only take the
refname as argument to make its signature similar to delete_reflog and
reflog_exists. Change create_reflog to ignore log_all_ref_updates and
unconditionally create the reflog when called. Since checkout.c always
wants to create a reflog we can call create_reflog directly and avoid the
temp-and-log_all_ref_update dance.
In log_ref_write, only call create_reflog iff we want to create a reflog
and if the reflog does not yet exist. This means that for the common case
where the log already exists we now only need to perform a single lstat()
instead of a open(O_CREAT)+lstat()+close().
Change-Id: Ib9493b3b81a97a0e154cc44303a5ed7cdceaaca7
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
builtin/checkout.c | 8 +---
refs.c | 22 +-
refs.h | 8 +++-
3 files changed, 17 insertions(+), 21 deletions(-)
diff --git a/builtin/checkout.c b/builtin/checkout.c
index 5410dac..8550b6d 100644
--- a/builtin/checkout.c
+++ b/builtin/checkout.c
@@ -587,19 +587,13 @@ static void update_refs_for_switch(const struct
checkout_opts *opts,
if (opts-new_branch) {
if (opts-new_orphan_branch) {
if (opts-new_branch_log !log_all_ref_updates) {
- int temp;
- char log_file[PATH_MAX];
char *ref_name = mkpath(refs/heads/%s,
opts-new_orphan_branch);
- temp = log_all_ref_updates;
- log_all_ref_updates = 1;
- if (log_ref_setup(ref_name, log_file,
sizeof(log_file))) {
+ if (create_reflog(ref_name)) {
fprintf(stderr, _(Can not do reflog
for '%s'\n),
opts-new_orphan_branch);
- log_all_ref_updates = temp;
return;
}
- log_all_ref_updates = temp;
}
}
else
diff --git a/refs.c b/refs.c
index 3189f11..952cc66 100644
--- a/refs.c
+++ b/refs.c
@@ -2947,16 +2947,16 @@ static int copy_msg(char *buf, const char *msg)
}
/* This function must set a meaningful errno on failure */
-int log_ref_setup(const char *refname, char *logfile, int bufsize)
+int create_reflog(const char *refname)
{
int logfd, oflags = O_APPEND | O_WRONLY;
+ char logfile[PATH_MAX];
- git_snpath(logfile, bufsize, logs/%s, refname);
- if (log_all_ref_updates
- (starts_with(refname, refs/heads/) ||
-starts_with(refname, refs/remotes/) ||
-starts_with(refname, refs/notes/) ||
-!strcmp(refname, HEAD))) {
+ git_snpath(logfile, sizeof(logfile), logs/%s, refname);
+ if (starts_with(refname, refs/heads/) ||
+ starts_with(refname, refs/remotes/) ||
+ starts_with(refname, refs/notes/) ||
+ !strcmp(refname, HEAD)) {
if (safe_create_leading_directories(logfile) 0) {
int save_errno = errno;
error(unable to create directory for %s, logfile);
@@ -3025,16 +3025,20 @@ static int log_ref_write_fd(int fd, const unsigned char
*old_sha1,
static int log_ref_write(const char *refname, const unsigned char *old_sha1,
const unsigned char *new_sha1, const char *msg)
{
- int logfd, result, oflags = O_APPEND | O_WRONLY;
+ int logfd, result = 0, oflags = O_APPEND | O_WRONLY;
char log_file[PATH_MAX];
if (log_all_ref_updates 0)
log_all_ref_updates = !is_bare_repository();
- result = log_ref_setup(refname, log_file, sizeof(log_file));
+ if (log_all_ref_updates !reflog_exists(refname))
+ result = create_reflog(refname);
+
if (result)
return result;
+ git_snpath(log_file, sizeof(log_file), logs/%s, refname);
+
logfd = open(log_file, oflags);
if (logfd 0)
return 0;
diff --git a/refs.h b/refs.h
index 9f70b89..17e3a3c 100644
--- a/refs.h
+++ b/refs.h
@@ -207,11 +207,6
[PATCH 01/15] refs.c make ref_transaction_create a wrapper to ref_transaction_update
commit 03001144a015f81db5252005841bb78f57d62774 upstream.
The ref_transaction_update function can already be used to create refs by
passing null_sha1 as the old_sha1 parameter. Simplify by replacing
transaction_create with a thin wrapper.
Change-Id: I687dd47cc4f4e06766e8313b4fd1b07cd4a56c1a
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 27 ++-
1 file changed, 2 insertions(+), 25 deletions(-)
diff --git a/refs.c b/refs.c
index 0368ed4..ed0485e 100644
--- a/refs.c
+++ b/refs.c
@@ -3623,31 +3623,8 @@ int ref_transaction_create(struct ref_transaction
*transaction,
int flags, const char *msg,
struct strbuf *err)
{
- struct ref_update *update;
-
- assert(err);
-
- if (transaction-state != REF_TRANSACTION_OPEN)
- die(BUG: create called for transaction that is not open);
-
- if (!new_sha1 || is_null_sha1(new_sha1))
- die(BUG: create ref with null new_sha1);
-
- if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL)) {
- strbuf_addf(err, refusing to create ref with bad name %s,
- refname);
- return -1;
- }
-
- update = add_update(transaction, refname);
-
- hashcpy(update-new_sha1, new_sha1);
- hashclr(update-old_sha1);
- update-flags = flags;
- update-have_old = 1;
- if (msg)
- update-msg = xstrdup(msg);
- return 0;
+ return ref_transaction_update(transaction, refname, new_sha1,
+ null_sha1, flags, 1, msg, err);
}
int ref_transaction_delete(struct ref_transaction *transaction,
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 13/15] refs.c: make unlock_ref/close_ref/commit_ref static
commit a4369f77d1975566bcd29bfa46720d48372c241d upstream.
unlock|close|commit_ref can be made static since there are no more external
callers.
Change-Id: I743f9daaa1c101fdc781a76283c2e8b462591d07
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 24
refs.h | 9 -
2 files changed, 12 insertions(+), 21 deletions(-)
diff --git a/refs.c b/refs.c
index 952cc66..5e5f066 100644
--- a/refs.c
+++ b/refs.c
@@ -2096,6 +2096,16 @@ int refname_match(const char *abbrev_name, const char
*full_name)
return 0;
}
+static void unlock_ref(struct ref_lock *lock)
+{
+ /* Do not free lock-lk -- atexit() still looks at them */
+ if (lock-lk)
+ rollback_lock_file(lock-lk);
+ free(lock-ref_name);
+ free(lock-orig_ref_name);
+ free(lock);
+}
+
/* This function should make sure errno is meaningful on error */
static struct ref_lock *verify_lock(struct ref_lock *lock,
const unsigned char *old_sha1, int mustexist)
@@ -2894,7 +2904,7 @@ int rename_ref(const char *oldrefname, const char
*newrefname, const char *logms
return 1;
}
-int close_ref(struct ref_lock *lock)
+static int close_ref(struct ref_lock *lock)
{
if (close_lock_file(lock-lk))
return -1;
@@ -2902,7 +2912,7 @@ int close_ref(struct ref_lock *lock)
return 0;
}
-int commit_ref(struct ref_lock *lock)
+static int commit_ref(struct ref_lock *lock)
{
if (commit_lock_file(lock-lk))
return -1;
@@ -2910,16 +2920,6 @@ int commit_ref(struct ref_lock *lock)
return 0;
}
-void unlock_ref(struct ref_lock *lock)
-{
- /* Do not free lock-lk -- atexit() still looks at them */
- if (lock-lk)
- rollback_lock_file(lock-lk);
- free(lock-ref_name);
- free(lock-orig_ref_name);
- free(lock);
-}
-
/*
* copy the reflog message msg to buf, which has been allocated sufficiently
* large, while cleaning up the whitespaces. Especially, convert LF to space,
diff --git a/refs.h b/refs.h
index 17e3a3c..025e2cb 100644
--- a/refs.h
+++ b/refs.h
@@ -198,15 +198,6 @@ extern struct ref_lock *lock_any_ref_for_update(const char
*refname,
const unsigned char *old_sha1,
int flags, int *type_p);
-/** Close the file descriptor owned by a lock and return the status */
-extern int close_ref(struct ref_lock *lock);
-
-/** Close and commit the ref locked by the lock */
-extern int commit_ref(struct ref_lock *lock);
-
-/** Release any lock taken but not written. **/
-extern void unlock_ref(struct ref_lock *lock);
-
/** Reads log for the value of ref during at_time. **/
extern int read_ref_at(const char *refname, unsigned int flags,
unsigned long at_time, int cnt,
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 05/15] refs.c: add a function to append a reflog entry to a fd
commit 07baa71eda8d1047f003acc7ea32ad6b6600b179 upstream.
Break out the code to create the string and writing it to the file
descriptor from log_ref_write and into a dedicated function log_ref_write_fd.
For now this is only used from log_ref_write but later on we will call
this function from reflog transactions too which means that we will end
up with only a single place where we write a reflog entry to a file instead
of the current two places (log_ref_write and builtin/reflog.c).
Change-Id: I178e84a52af18c800844596af33dbc21bc076c1f
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 48 ++--
1 file changed, 30 insertions(+), 18 deletions(-)
diff --git a/refs.c b/refs.c
index 8803b95..0e11b1c 100644
--- a/refs.c
+++ b/refs.c
@@ -2990,15 +2990,37 @@ int log_ref_setup(const char *refname, char *logfile,
int bufsize)
return 0;
}
+static int log_ref_write_fd(int fd, const unsigned char *old_sha1,
+ const unsigned char *new_sha1,
+ const char *committer, const char *msg)
+{
+ int msglen, written;
+ unsigned maxlen, len;
+ char *logrec;
+
+ msglen = msg ? strlen(msg) : 0;
+ maxlen = strlen(committer) + msglen + 100;
+ logrec = xmalloc(maxlen);
+ len = sprintf(logrec, %s %s %s\n,
+ sha1_to_hex(old_sha1),
+ sha1_to_hex(new_sha1),
+ committer);
+ if (msglen)
+ len += copy_msg(logrec + len - 1, msg) - 1;
+
+ written = len = maxlen ? write_in_full(fd, logrec, len) : -1;
+ free(logrec);
+ if (written != len)
+ return -1;
+
+ return 0;
+}
+
static int log_ref_write(const char *refname, const unsigned char *old_sha1,
const unsigned char *new_sha1, const char *msg)
{
- int logfd, result, written, oflags = O_APPEND | O_WRONLY;
- unsigned maxlen, len;
- int msglen;
+ int logfd, result, oflags = O_APPEND | O_WRONLY;
char log_file[PATH_MAX];
- char *logrec;
- const char *committer;
if (log_all_ref_updates 0)
log_all_ref_updates = !is_bare_repository();
@@ -3010,19 +3032,9 @@ static int log_ref_write(const char *refname, const
unsigned char *old_sha1,
logfd = open(log_file, oflags);
if (logfd 0)
return 0;
- msglen = msg ? strlen(msg) : 0;
- committer = git_committer_info(0);
- maxlen = strlen(committer) + msglen + 100;
- logrec = xmalloc(maxlen);
- len = sprintf(logrec, %s %s %s\n,
- sha1_to_hex(old_sha1),
- sha1_to_hex(new_sha1),
- committer);
- if (msglen)
- len += copy_msg(logrec + len - 1, msg) - 1;
- written = len = maxlen ? write_in_full(logfd, logrec, len) : -1;
- free(logrec);
- if (written != len) {
+ result = log_ref_write_fd(logfd, old_sha1, new_sha1,
+ git_committer_info(0), msg);
+ if (result) {
int save_errno = errno;
close(logfd);
error(Unable to append to %s, log_file);
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 02/15] refs.c: make ref_transaction_delete a wrapper for ref_transaction_update
commit 0beeda259297c92d411ecc92fa508ec7cfd87cc5 upstream.
Change-Id: I685291986e544a8dc14f94c73b6a7c6400acd9d2
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 22 ++
refs.h | 2 +-
2 files changed, 3 insertions(+), 21 deletions(-)
diff --git a/refs.c b/refs.c
index ed0485e..c607ab7 100644
--- a/refs.c
+++ b/refs.c
@@ -3633,26 +3633,8 @@ int ref_transaction_delete(struct ref_transaction
*transaction,
int flags, int have_old, const char *msg,
struct strbuf *err)
{
- struct ref_update *update;
-
- assert(err);
-
- if (transaction-state != REF_TRANSACTION_OPEN)
- die(BUG: delete called for transaction that is not open);
-
- if (have_old !old_sha1)
- die(BUG: have_old is true but old_sha1 is NULL);
-
- update = add_update(transaction, refname);
- update-flags = flags;
- update-have_old = have_old;
- if (have_old) {
- assert(!is_null_sha1(old_sha1));
- hashcpy(update-old_sha1, old_sha1);
- }
- if (msg)
- update-msg = xstrdup(msg);
- return 0;
+ return ref_transaction_update(transaction, refname, null_sha1,
+ old_sha1, flags, have_old, msg, err);
}
int update_ref(const char *action, const char *refname,
diff --git a/refs.h b/refs.h
index 2bc3556..7d675b7 100644
--- a/refs.h
+++ b/refs.h
@@ -283,7 +283,7 @@ struct ref_transaction *ref_transaction_begin(struct strbuf
*err);
/*
* Add a reference update to transaction. new_sha1 is the value that
- * the reference should have after the update, or zeros if it should
+ * the reference should have after the update, or null_sha1 if it should
* be deleted. If have_old is true, then old_sha1 holds the value
* that the reference should have had before the update, or zeros if
* it must not have existed beforehand.
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 00/15] ref-transactions-reflog
List, Please find a patch that updates the reflog handling to use transactions. This patch series has previously been sent to the list but is now rebased on the current content of next which contains ref changes we depend on in this series. This series converts the reflog handling and builtin/reflog.c to use a transaction for both the ref as well as the reflog updates. As a side effect of this it simplifies the reflog marshalling code so that we only have one place where we marshall the entry. It also means that we can remove several functions from the public api towards the end of the series since we no longer need those functions. This series can also be found at https://github.com/rsahlberg/git/tree/ref-transactions-reflog Ronnie Sahlberg (15): refs.c make ref_transaction_create a wrapper to ref_transaction_update refs.c: make ref_transaction_delete a wrapper for ref_transaction_update refs.c: rename the transaction functions refs.c: add a new update_type field to ref_update refs.c: add a function to append a reflog entry to a fd copy.c: make copy_fd preserve meaningful errno refs.c: add a transaction function to append a reflog entry refs.c: add a flag to allow reflog updates to truncate the log refs.c: only write reflog update if msg is non-NULL refs.c: allow multiple reflog updates during a single transaction reflog.c: use a reflog transaction when writing during expire refs.c: rename log_ref_setup to create_reflog refs.c: make unlock_ref/close_ref/commit_ref static refs.c: remove lock_any_ref_for_update refs.c: allow deleting refs with a broken sha1 branch.c| 14 +- builtin/branch.c| 5 +- builtin/checkout.c | 8 +- builtin/commit.c| 10 +- builtin/fetch.c | 12 +- builtin/receive-pack.c | 13 +- builtin/reflog.c| 85 -- builtin/replace.c | 10 +- builtin/tag.c | 10 +- builtin/update-ref.c| 26 +-- cache.h | 7 + copy.c | 15 +- fast-import.c | 22 +-- refs.c | 401 +--- refs.h | 87 +- sequencer.c | 12 +- t/t1402-check-ref-format.sh | 8 + walker.c| 10 +- 18 files changed, 448 insertions(+), 307 deletions(-) -- 2.1.0.rc2.206.gedb03e5 -- To unsubscribe from this list: send the line unsubscribe git in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 10/15] refs.c: allow multiple reflog updates during a single transaction
commit 5fd1fa7d8520cf03e94fa7d0e9aa8685de0ef63f upstream.
Allow to make multiple reflog updates to the same ref during a transaction.
This means we only need to lock the reflog once, during the first update
that touches the reflog, and that all further updates can just write the
reflog entry since the reflog is already locked.
This allows us to write code such as:
t = transaction_begin()
transaction_reflog_update(t, foo, REFLOG_TRUNCATE, NULL);
loop-over-something...
transaction_reflog_update(t, foo, 0, message);
transaction_commit(t)
where we first truncate the reflog and then build the new content one line
at a time.
While this technically looks like O(n2) behavior it not that bad.
We only do this loop for transactions that cover a single ref during
reflog expire. This means that the linear search inside
transaction_update_reflog() will find the match on the very first entry
thus making it O(1) and not O(n) or our usecases. Thus the whole expire
becomes O(n) instead of O(n2). If in the future we start doing this for many
refs in one single transaction we might want to optimize this.
But there is no need to complexify the code and optimize for future usecases
that might never materialize at this stage.
Change-Id: Ibe703310e33a4db07e5bcc704310211268b788fc
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 46 +-
1 file changed, 37 insertions(+), 9 deletions(-)
diff --git a/refs.c b/refs.c
index f14b76e..f7e947f 100644
--- a/refs.c
+++ b/refs.c
@@ -31,6 +31,12 @@ static unsigned char refname_disposition[256] = {
*/
#define REF_ISPRUNING 0x0100
/*
+ * Only the first reflog update needs to lock the reflog file. Further updates
+ * just use the lock taken by the first update.
+ */
+#define UPDATE_REFLOG_NOLOCK 0x0200
+
+/*
* Try to read one refname component from the front of refname.
* Return the length of the component found, or -1 if the component is
* not legal. It is legal if it is something reasonable to have under
@@ -3521,7 +3527,7 @@ enum transaction_update_type {
UPDATE_LOG = 1
};
-/**
+/*
* Information needed for a single ref update. Set new_sha1 to the
* new value or to zero to delete the ref. To check the old value
* while locking the ref, set have_old to 1 and set old_sha1 to the
@@ -3531,7 +3537,7 @@ struct ref_update {
enum transaction_update_type update_type;
unsigned char new_sha1[20];
unsigned char old_sha1[20];
- int flags; /* REF_NODEREF? */
+ int flags; /* REF_NODEREF? or private flags */
int have_old; /* 1 if old_sha1 is valid, 0 otherwise */
struct ref_lock *lock;
int type;
@@ -3539,8 +3545,9 @@ struct ref_update {
/* used by reflog updates */
int reflog_fd;
- struct lock_file reflog_lock;
+ struct lock_file *reflog_lock;
char *committer;
+ struct ref_update *orig_update; /* For UPDATE_REFLOG_NOLOCK */
const char refname[FLEX_ARRAY];
};
@@ -3619,11 +3626,26 @@ int transaction_update_reflog(struct transaction
*transaction,
struct strbuf *err)
{
struct ref_update *update;
+ int i;
if (transaction-state != TRANSACTION_OPEN)
die(BUG: update_reflog called for transaction that is not
open);
update = add_update(transaction, refname, UPDATE_LOG);
+ update-flags = flags;
+ for (i = 0; i transaction-nr - 1; i++) {
+ if (transaction-updates[i]-update_type != UPDATE_LOG)
+ continue;
+ if (!strcmp(transaction-updates[i]-refname,
+ update-refname)) {
+ update-flags |= UPDATE_REFLOG_NOLOCK;
+ update-orig_update = transaction-updates[i];
+ break;
+ }
+ }
+ if (!(update-flags UPDATE_REFLOG_NOLOCK))
+ update-reflog_lock = xcalloc(1, sizeof(struct lock_file));
+
hashcpy(update-new_sha1, new_sha1);
hashcpy(update-old_sha1, old_sha1);
update-reflog_fd = -1;
@@ -3639,7 +3661,6 @@ int transaction_update_reflog(struct transaction
*transaction,
}
if (msg)
update-msg = xstrdup(msg);
- update-flags = flags;
return 0;
}
@@ -3822,10 +3843,15 @@ int transaction_commit(struct transaction *transaction,
if (update-update_type != UPDATE_LOG)
continue;
+ if (update-flags UPDATE_REFLOG_NOLOCK) {
+ update-reflog_fd = update-orig_update-reflog_fd;
+ update-reflog_lock = update-orig_update-reflog_lock;
+ continue;
+ }
update-reflog_fd = hold_lock_file_for_append(
- update-reflog_lock,
+ update
[PATCH 03/15] refs.c: rename the transaction functions
commit fd304abafb351f509b105da2d648298efd78df18 upstream.
Rename the transaction functions. Remove the leading ref_ from the
names and append _ref to the names for functions that create/delete/
update sha1 refs.
This also makes the names more appropriate for future changes that have been
discussed when the transactions could also operate on non ref objects. Such as
..git/config and .gitmodule.
Change-Id: Iffdc56536be71c5061caa23040ce0d89efd52196
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
branch.c | 14 -
builtin/commit.c | 10 +++
builtin/fetch.c| 12
builtin/receive-pack.c | 13 -
builtin/replace.c | 10 +++
builtin/tag.c | 10 +++
builtin/update-ref.c | 26 -
fast-import.c | 22 +++---
refs.c | 78 +-
refs.h | 36 +++
sequencer.c| 12
walker.c | 10 +++
12 files changed, 126 insertions(+), 127 deletions(-)
diff --git a/branch.c b/branch.c
index 4bab55a..d9b1174 100644
--- a/branch.c
+++ b/branch.c
@@ -279,17 +279,17 @@ void create_branch(const char *head,
log_all_ref_updates = 1;
if (!dont_change_ref) {
- struct ref_transaction *transaction;
+ struct transaction *transaction;
struct strbuf err = STRBUF_INIT;
- transaction = ref_transaction_begin(err);
+ transaction = transaction_begin(err);
if (!transaction ||
- ref_transaction_update(transaction, ref.buf, sha1,
- null_sha1, 0, !forcing, msg, err) ||
- ref_transaction_commit(transaction, err))
+ transaction_update_ref(transaction, ref.buf, sha1,
+ null_sha1, 0, !forcing, msg,
+ err) ||
+ transaction_commit(transaction, err))
die(%s, err.buf);
- ref_transaction_free(transaction);
- strbuf_release(err);
+ transaction_free(transaction);
}
if (real_ref track)
diff --git a/builtin/commit.c b/builtin/commit.c
index e108c53..f50b7df 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1673,7 +1673,7 @@ int cmd_commit(int argc, const char **argv, const char
*prefix)
struct stat statbuf;
struct commit *current_head = NULL;
struct commit_extra_header *extra = NULL;
- struct ref_transaction *transaction;
+ struct transaction *transaction;
struct strbuf err = STRBUF_INIT;
if (argc == 2 !strcmp(argv[1], -h))
@@ -1804,17 +1804,17 @@ int cmd_commit(int argc, const char **argv, const char
*prefix)
strbuf_insert(sb, 0, reflog_msg, strlen(reflog_msg));
strbuf_insert(sb, strlen(reflog_msg), : , 2);
- transaction = ref_transaction_begin(err);
+ transaction = transaction_begin(err);
if (!transaction ||
- ref_transaction_update(transaction, HEAD, sha1,
+ transaction_update_ref(transaction, HEAD, sha1,
current_head
? current_head-object.sha1 : NULL,
0, !!current_head, sb.buf, err) ||
- ref_transaction_commit(transaction, err)) {
+ transaction_commit(transaction, err)) {
rollback_index_files();
die(%s, err.buf);
}
- ref_transaction_free(transaction);
+ transaction_free(transaction);
unlink(git_path(CHERRY_PICK_HEAD));
unlink(git_path(REVERT_HEAD));
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 6ffd023..323ffd9 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -404,7 +404,7 @@ static int s_update_ref(const char *action,
{
char msg[1024];
char *rla = getenv(GIT_REFLOG_ACTION);
- struct ref_transaction *transaction;
+ struct transaction *transaction;
struct strbuf err = STRBUF_INIT;
int ret, df_conflict = 0;
@@ -414,23 +414,23 @@ static int s_update_ref(const char *action,
rla = default_rla.buf;
snprintf(msg, sizeof(msg), %s: %s, rla, action);
- transaction = ref_transaction_begin(err);
+ transaction = transaction_begin(err);
if (!transaction ||
- ref_transaction_update(transaction, ref-name, ref-new_sha1,
+ transaction_update_ref(transaction, ref-name, ref-new_sha1,
ref-old_sha1, 0, check_old, msg, err))
goto fail;
- ret = ref_transaction_commit(transaction, err);
+ ret = transaction_commit(transaction, err);
if (ret) {
df_conflict = (ret
[PATCH 09/15] refs.c: only write reflog update if msg is non-NULL
commit 020ed65a12838bdead64bc3c5de249d3c8f5cfd8 upstream.
When performing a reflog transaction update, only write to the reflog iff
msg is non-NULL. This can then be combined with REFLOG_TRUNCATE to perform
an update that only truncates but does not write.
Change-Id: I44c89caa7e7c4960777b79cfb5d339a5aa3ddf7a
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 5 +++--
refs.h | 1 +
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/refs.c b/refs.c
index d54c3b9..f14b76e 100644
--- a/refs.c
+++ b/refs.c
@@ -3895,8 +3895,9 @@ int transaction_commit(struct transaction *transaction,
update-reflog_fd = -1;
continue;
}
- if (log_ref_write_fd(update-reflog_fd, update-old_sha1,
-update-new_sha1,
+ if (update-msg
+ log_ref_write_fd(update-reflog_fd,
+update-old_sha1, update-new_sha1,
update-committer, update-msg)) {
error(Could write to reflog: %s. %s,
update-refname, strerror(errno));
diff --git a/refs.h b/refs.h
index 5075073..bf96b36 100644
--- a/refs.h
+++ b/refs.h
@@ -337,6 +337,7 @@ int transaction_delete_ref(struct transaction *transaction,
/*
* Append a reflog entry for refname. If the REFLOG_TRUNCATE flag is set
* this update will first truncate the reflog before writing the entry.
+ * If msg is NULL no update will be written to the log.
*/
int transaction_update_reflog(struct transaction *transaction,
const char *refname,
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 14/15] refs.c: remove lock_any_ref_for_update
commit 3a2f55c247ff290943fd552674e226062c13fd00 upstream.
No one is using this function so we can delete it.
Change-Id: I524dac239da500e5d4432328b997ed8ab34f6803
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
refs.c | 7 ---
refs.h | 9 +
2 files changed, 1 insertion(+), 15 deletions(-)
diff --git a/refs.c b/refs.c
index 5e5f066..5bd6d62 100644
--- a/refs.c
+++ b/refs.c
@@ -2352,13 +2352,6 @@ static struct ref_lock *lock_ref_sha1_basic(const char
*refname,
return NULL;
}
-struct ref_lock *lock_any_ref_for_update(const char *refname,
-const unsigned char *old_sha1,
-int flags, int *type_p)
-{
- return lock_ref_sha1_basic(refname, old_sha1, NULL, flags, type_p);
-}
-
/*
* Write an entry to the packed-refs file for the specified refname.
* If peeled is non-NULL, write it as the entry's peeled value.
diff --git a/refs.h b/refs.h
index 025e2cb..721e21f 100644
--- a/refs.h
+++ b/refs.h
@@ -181,8 +181,7 @@ extern int is_branch(const char *refname);
extern int peel_ref(const char *refname, unsigned char *sha1);
/*
- * Flags controlling lock_any_ref_for_update(), transaction_update_ref(),
- * transaction_create_ref(), etc.
+ * Flags controlling transaction_update_ref(), transaction_create_ref(), etc.
* REF_NODEREF: act on the ref directly, instead of dereferencing
* symbolic references.
* REF_DELETING: tolerate broken refs
@@ -191,12 +190,6 @@ extern int peel_ref(const char *refname, unsigned char
*sha1);
*/
#define REF_NODEREF0x01
#define REF_DELETING 0x02
-/*
- * This function sets errno to something meaningful on failure.
- */
-extern struct ref_lock *lock_any_ref_for_update(const char *refname,
- const unsigned char *old_sha1,
- int flags, int *type_p);
/** Reads log for the value of ref during at_time. **/
extern int read_ref_at(const char *refname, unsigned int flags,
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 15/15] refs.c: allow deleting refs with a broken sha1
commit 8d75baa250c0f84cc438568cb53ef1a9dd3dfec9 upstream.
Add back support to make it possible to delete refs that have a broken
sha1.
Add new internal flags REF_ALLOW_BROKEN and RESOLVE_REF_ALLOW_BAD_SHA1
to pass intent from branch.c that we are willing to allow
resolve_ref_unsafe and lock_ref_sha1_basic to allow broken refs.
Since these refs can not actually be resolved to a sha1, they instead resolve
to null_sha1 when these flags are used.
For example, the ref:
echo Broken ref .git/refs/heads/foo-broken-1
can now be deleted using git branch -d foo-broken-1
Change-Id: I4e744d9e7d8b7e81dde5479965819117d03c98db
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
builtin/branch.c| 5 +++--
cache.h | 7 +++
refs.c | 6 ++
refs.h | 6 --
t/t1402-check-ref-format.sh | 8
5 files changed, 28 insertions(+), 4 deletions(-)
diff --git a/builtin/branch.c b/builtin/branch.c
index 3b79c50..04f57d4 100644
--- a/builtin/branch.c
+++ b/builtin/branch.c
@@ -238,7 +238,8 @@ static int delete_branches(int argc, const char **argv, int
force, int kinds,
target = resolve_ref_unsafe(name,
RESOLVE_REF_READING
| RESOLVE_REF_NO_RECURSE
- | RESOLVE_REF_ALLOW_BAD_NAME,
+ | RESOLVE_REF_ALLOW_BAD_NAME
+ | RESOLVE_REF_ALLOW_BAD_SHA1,
sha1, flags);
if (!target) {
error(remote_branch
@@ -255,7 +256,7 @@ static int delete_branches(int argc, const char **argv, int
force, int kinds,
continue;
}
- if (delete_ref(name, sha1, REF_NODEREF)) {
+ if (delete_ref(name, sha1, REF_NODEREF|REF_ALLOW_BROKEN)) {
error(remote_branch
? _(Error deleting remote branch '%s')
: _(Error deleting branch '%s'),
diff --git a/cache.h b/cache.h
index 99ed096..61e61af 100644
--- a/cache.h
+++ b/cache.h
@@ -1000,10 +1000,17 @@ extern int read_ref(const char *refname, unsigned char
*sha1);
* resolved. The function returns NULL for such ref names.
* Caps and underscores refers to the special refs, such as HEAD,
* FETCH_HEAD and friends, that all live outside of the refs/ directory.
+ *
+ * RESOLVE_REF_ALLOW_BAD_SHA1 when this flag is set and the ref contains
+ * an invalid sha1, resolve_ref_unsafe will clear the sha1 argument,
+ * set the REF_ISBROKEN flag and return the refname.
+ * This allows using resolve_ref_unsafe to check for existence of such
+ * broken refs.
*/
#define RESOLVE_REF_READING 0x01
#define RESOLVE_REF_NO_RECURSE 0x02
#define RESOLVE_REF_ALLOW_BAD_NAME 0x04
+#define RESOLVE_REF_ALLOW_BAD_SHA1 0x08
extern const char *resolve_ref_unsafe(const char *ref, int resolve_flags,
unsigned char *sha1, int *flags);
extern char *resolve_refdup(const char *ref, int resolve_flags, unsigned char
*sha1, int *flags);
diff --git a/refs.c b/refs.c
index 5bd6d62..ae29d11 100644
--- a/refs.c
+++ b/refs.c
@@ -1584,6 +1584,10 @@ const char *resolve_ref_unsafe(const char *refname, int
resolve_flags, unsigned
(buffer[40] != '\0' !isspace(buffer[40]))) {
if (flags)
*flags |= REF_ISBROKEN;
+ if (resolve_flags RESOLVE_REF_ALLOW_BAD_SHA1)
{
+ hashclr(sha1);
+ return refname;
+ }
errno = EINVAL;
return NULL;
}
@@ -2265,6 +2269,8 @@ static struct ref_lock *lock_ref_sha1_basic(const char
*refname,
if (flags REF_NODEREF)
resolve_flags |= RESOLVE_REF_NO_RECURSE;
}
+ if (flags REF_ALLOW_BROKEN)
+ resolve_flags |= RESOLVE_REF_ALLOW_BAD_SHA1;
refname = resolve_ref_unsafe(refname, resolve_flags,
lock-old_sha1, type);
diff --git a/refs.h b/refs.h
index 721e21f..2e97f4f 100644
--- a/refs.h
+++ b/refs.h
@@ -185,11 +185,13 @@ extern int peel_ref(const char *refname, unsigned char
*sha1);
* REF_NODEREF: act on the ref directly, instead of dereferencing
* symbolic references.
* REF_DELETING: tolerate broken refs
+ * REF_ALLOW_BROKEN: allow locking refs that are broken.
*
* Flags = 0x100 are reserved for internal use.
*/
-#define REF_NODEREF0x01
-#define REF_DELETING 0x02
+#define REF_NODEREF0x01
+#define REF_DELETING 0x02
+#define REF_ALLOW_BROKEN 0x04
[PATCH 11/15] reflog.c: use a reflog transaction when writing during expire
commit 5ac378cd8fad09a836d17fec379780854838bde5 upstream.
Use a transaction for all updates during expire_reflog.
Change-Id: Ieb81b2660cefeeecf0bcb3cdbc1ef3cbb86e7eb8
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
builtin/reflog.c | 85
refs.c | 4 +--
refs.h | 2 +-
3 files changed, 40 insertions(+), 51 deletions(-)
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 2d85d26..6bb7454 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -32,8 +32,11 @@ struct cmd_reflog_expire_cb {
int recno;
};
+static struct strbuf err = STRBUF_INIT;
+
struct expire_reflog_cb {
- FILE *newlog;
+ struct transaction *t;
+ const char *refname;
enum {
UE_NORMAL,
UE_ALWAYS,
@@ -316,20 +319,18 @@ static int expire_reflog_ent(unsigned char *osha1,
unsigned char *nsha1,
if (cb-cmd-recno --(cb-cmd-recno) == 0)
goto prune;
- if (cb-newlog) {
- char sign = (tz 0) ? '-' : '+';
- int zone = (tz 0) ? (-tz) : tz;
- fprintf(cb-newlog, %s %s %s %lu %c%04d\t%s,
- sha1_to_hex(osha1), sha1_to_hex(nsha1),
- email, timestamp, sign, zone,
- message);
+ if (cb-t) {
+ if (transaction_update_reflog(cb-t, cb-refname, nsha1, osha1,
+ email, timestamp, tz, message, 0,
+ err))
+ return -1;
hashcpy(cb-last_kept_sha1, nsha1);
}
if (cb-cmd-verbose)
printf(keep %s, message);
return 0;
prune:
- if (!cb-newlog)
+ if (!cb-t)
printf(would prune %s, message);
else if (cb-cmd-verbose)
printf(prune %s, message);
@@ -353,29 +354,26 @@ static int expire_reflog(const char *ref, const unsigned
char *sha1, int unused,
{
struct cmd_reflog_expire_cb *cmd = cb_data;
struct expire_reflog_cb cb;
- struct ref_lock *lock;
- char *log_file, *newlog_path = NULL;
struct commit *tip_commit;
struct commit_list *tips;
int status = 0;
memset(cb, 0, sizeof(cb));
+ cb.refname = ref;
- /*
-* we take the lock for the ref itself to prevent it from
-* getting updated.
-*/
- lock = lock_any_ref_for_update(ref, sha1, 0, NULL);
- if (!lock)
- return error(cannot lock ref '%s', ref);
- log_file = git_pathdup(logs/%s, ref);
if (!reflog_exists(ref))
goto finish;
- if (!cmd-dry_run) {
- newlog_path = git_pathdup(logs/%s.lock, ref);
- cb.newlog = fopen(newlog_path, w);
+ cb.t = transaction_begin(err);
+ if (!cb.t) {
+ status |= error(%s, err.buf);
+ goto cleanup;
+ }
+ if (transaction_update_reflog(cb.t, cb.refname, null_sha1, null_sha1,
+ NULL, 0, 0, NULL, REFLOG_TRUNCATE,
+ err)) {
+ status |= error(%s, err.buf);
+ goto cleanup;
}
-
cb.cmd = cmd;
if (!cmd-expire_unreachable || !strcmp(ref, HEAD)) {
@@ -407,7 +405,10 @@ static int expire_reflog(const char *ref, const unsigned
char *sha1, int unused,
mark_reachable(cb);
}
- for_each_reflog_ent(ref, expire_reflog_ent, cb);
+ if (for_each_reflog_ent(ref, expire_reflog_ent, cb)) {
+ status |= error(%s, err.buf);
+ goto cleanup;
+ }
if (cb.unreachable_expire_kind != UE_ALWAYS) {
if (cb.unreachable_expire_kind == UE_HEAD) {
@@ -420,32 +421,20 @@ static int expire_reflog(const char *ref, const unsigned
char *sha1, int unused,
}
}
finish:
- if (cb.newlog) {
- if (fclose(cb.newlog)) {
- status |= error(%s: %s, strerror(errno),
- newlog_path);
- unlink(newlog_path);
- } else if (cmd-updateref
- (write_in_full(lock-lock_fd,
- sha1_to_hex(cb.last_kept_sha1), 40) != 40 ||
-write_str_in_full(lock-lock_fd, \n) != 1 ||
-close_ref(lock) 0)) {
- status |= error(Couldn't write %s,
- lock-lk-filename.buf);
- unlink(newlog_path);
- } else if (rename(newlog_path, log_file)) {
- status |= error(cannot rename %s to %s,
- newlog_path, log_file);
- unlink(newlog_path);
- } else if (cmd
[PATCH 06/15] copy.c: make copy_fd preserve meaningful errno
commit 306805ccd147bfdf160b288a8d51fdf9b77ae0fa upstream.
Update copy_fd to return a meaningful errno on failure.
Change-Id: I771f9703acc816902affcf35ff2a56d9426315ac
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
copy.c | 15 ++-
1 file changed, 10 insertions(+), 5 deletions(-)
diff --git a/copy.c b/copy.c
index f2970ec..a8d366e 100644
--- a/copy.c
+++ b/copy.c
@@ -8,12 +8,17 @@ int copy_fd(int ifd, int ofd)
if (!len)
break;
if (len 0) {
- return error(copy-fd: read returned %s,
-strerror(errno));
+ int save_errno = errno;
+ error(copy-fd: read returned %s, strerror(errno));
+ errno = save_errno;
+ return -1;
+ }
+ if (write_in_full(ofd, buffer, len) 0) {
+ int save_errno = errno;
+ error(copy-fd: write returned %s, strerror(errno));
+ errno = save_errno;
+ return -1;
}
- if (write_in_full(ofd, buffer, len) 0)
- return error(copy-fd: write returned %s,
-strerror(errno));
}
return 0;
}
--
2.1.0.rc2.206.gedb03e5
--
To unsubscribe from this list: send the line unsubscribe git in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 03/15] refs.c: use packed refs when deleting refs during a transaction
commit fb5fa1d338ce113b0fea3bb955b50bbb3e827805 upstream.
Make the deletion of refs during a transaction more atomic.
Start by first copying all loose refs we will be deleting to the packed
refs file and then commit the packed refs file. Then re-lock the packed refs
file to stop anyone else from modifying these refs and keep it locked until
we are finished.
Since all refs we are about to delete are now safely held in the packed refs
file we can proceed to immediately unlink any corresponding loose refs
and still be fully rollback-able.
The exception is for refs that can not be resolved. Those refs are never
added to the packed refs and will just be un-rollback-ably deleted during
commit.
By deleting all the loose refs at the start of the transaction we make make
it possible to both delete one ref and then re-create a different ref in
the same transaction even if the two refs would normally conflict.
Example: rename m-m/m
In that example we want to delete the file 'm' so that we make room so
that we can create a directory with the same name in order to lock and
write to the ref m/m and its lock-file m/m.lock.
If there is a failure during the commit phase we can rollback without losing
any refs since we have so far only deleted loose refs that that are
guaranteed to also have a corresponding entry in the packed refs file.
Once we have finished all updates for refs and their reflogs we can repack
the packed refs file and remove the to-be-deleted refs from the packed refs,
at which point all the deleted refs will disappear in one atomic rename
operation.
This also means that for an outside observer, deletion of multiple refs
in a single transaction will look atomic instead of one ref being deleted
at a time.
In order to do all this we need to change the semantics for the
repack_without_refs function so that we can lock the packed refs file,
do other stuff, and later be able to call repack_without_refs with the
lock already taken.
This means we need some additional changes in remote.c to reflect the
changes to the repack_without_refs semantics.
Change-Id: I1e4f58050acaabc6bcfa3409fbc0c2b866bb59aa
Signed-off-by: Ronnie Sahlberg sahlb...@google.com
Signed-off-by: Jonathan Nieder jrnie...@gmail.com
---
builtin/remote.c | 20 ++-
refs.c | 155 ++-
2 files changed, 138 insertions(+), 37 deletions(-)
diff --git a/builtin/remote.c b/builtin/remote.c
index 7f28f92..c25420f 100644
--- a/builtin/remote.c
+++ b/builtin/remote.c
@@ -1,4 +1,5 @@
#include builtin.h
+#include lockfile.h
#include parse-options.h
#include transport.h
#include remote.h
@@ -753,6 +754,15 @@ static int remove_branches(struct string_list *branches)
const char **branch_names;
int i, result = 0;
+ if (lock_packed_refs(0)) {
+ struct strbuf err = STRBUF_INIT;
+
+ unable_to_lock_message(git_path(packed-refs), errno, err);
+ error(%s, err.buf);
+ strbuf_release(err);
+ return -1;
+ }
+
branch_names = xmalloc(branches-nr * sizeof(*branch_names));
for (i = 0; i branches-nr; i++)
branch_names[i] = branches-items[i].string;
@@ -1337,9 +1347,15 @@ static int prune_remote(const char *remote, int dry_run)
delete_refs[i] = states.stale.items[i].util;
if (!dry_run) {
struct strbuf err = STRBUF_INIT;
- if (repack_without_refs(delete_refs, states.stale.nr,
- err))
+
+ if (lock_packed_refs(0)) {
+ unable_to_lock_message(git_path(packed-refs),
+ errno, err);
result |= error(%s, err.buf);
+ } else
+ if (repack_without_refs(delete_refs,
+ states.stale.nr, err))
+ result |= error(%s, err.buf);
strbuf_release(err);
}
free(delete_refs);
diff --git a/refs.c b/refs.c
index c088d36..662af9b 100644
--- a/refs.c
+++ b/refs.c
@@ -2660,6 +2660,9 @@ static int curate_packed_ref_fn(struct ref_entry *entry,
void *cb_data)
return 0;
}
+/*
+ * Must be called with packed refs already locked (and sorted)
+ */
int repack_without_refs(const char **refnames, int n, struct strbuf *err)
{
struct ref_dir *packed;
@@ -2674,14 +2677,12 @@ int repack_without_refs(const char **refnames, int n,
struct strbuf *err)
if (get_packed_ref(refnames[i]))
break;
- /* Avoid locking if we have nothing to do */
- if (i == n)
+ /* Avoid processing if we have nothing to do */
+ if (i == n) {
+ rollback_packed_refs
