zanmato1984 commented on code in PR #41695:
URL: https://github.com/apache/arrow/pull/41695#discussion_r1604305811
##########
cpp/src/arrow/compute/util_internal.h:
##########
@@ -38,21 +38,24 @@ class ARROW_EXPORT TempVectorStack {
friend class TempVectorHolder;
public:
+ TempVectorStack() = default;
+ ~TempVectorStack();
+
+ TempVectorStack(const TempVectorStack&) = delete;
+ TempVectorStack& operator=(const TempVectorStack&) = delete;
+
+ TempVectorStack(TempVectorStack&&) = default;
+ TempVectorStack& operator=(TempVectorStack&&) = default;
+
Status Init(MemoryPool* pool, int64_t size);
int64_t AllocatedSize() const { return top_; }
private:
- static int64_t EstimatedAllocationSize(int64_t size) {
- return PaddedAllocationSize(size) + 2 * sizeof(uint64_t);
- }
-
static int64_t PaddedAllocationSize(int64_t num_bytes);
void alloc(uint32_t num_bytes, uint8_t** data, int* id);
void release(int id, uint32_t num_bytes);
- static constexpr uint64_t kGuard1 = 0x3141592653589793ULL;
- static constexpr uint64_t kGuard2 = 0x0577215664901532ULL;
Review Comment:
Thanks @felipecrv , I'll add them back.
But I'm also putting down some notes here about what I've been thinking,
perhaps just as a memo to myself.
I'm removing these guards because ASAN can prevent both invalid reads and
writes to poisoned memory. I don't think I can replicate the exact issue that
#11335 was trying to fix, but I simply reverted #11335 and ran hash join ut,
seeing ASAN successfully caught write after poison.
However there could be more tricky cases like below:
1. `func1` allocates a temp vector `v1` and reads/writes to `v1`;
2. `func1` calls `func2` (note: w/o releasing `v1` - this is the essential
of a "stack" and the tricky part);
3. `func2` allocates another temp vector `v2` and reads/writes to `v2`;
4. `func2` releases `v2` and returns to `func1`;
5. `func1` reads/writes to `v1` again;
6. `func1` releases `v1` and returns to its caller.
A) Suppose writes to `v1` at `offset >= v1.size()` happen in 1, then ASAN
can catch it because that piece of memory is already poisoned.
B) Suppose writes to `v1` at `offset < 0` happen in 1, then whether ASAN can
catch it depends on the validity of the memory address lower than `v1`, but the
guards have a better chance.
C) Suppose writes to `v2` at `offset >= v2.size()` happen in 3, same as A.
D) Suppose writes to `v2` at `offset < 0` happen in 3, then ASAN has zero
chance to catch it as long as the offset is within `v1.size()`, but the guards
have a better chance.
E) Is it possible that `func1` writes to `v1` at `offset >= v1.size()` while
`v2` is still alive? So it is corrupting `v2`'s content without ASAN noticing
because `v2`'s memory is unpoisoned throughout `v2`'s lifespan. The answer is
NO, the control flow will be in `func2` as long as `v2` is alive so `func1`
won't be able to execute any code.
To summarize, ASAN poisoning is not able to 100% cover what the guards can
do. The guards have a better chance to catch invalid writes at negative offsets
(B and D). But other than that, ASAN is equally good as the guards.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
