reuvenlax commented on code in PR #40791:
URL: https://github.com/apache/beam/pull/40791#discussion_r4396367078
##########
sdks/java/core/src/main/java/org/apache/beam/sdk/values/RowWithGetters.java:
##########
@@ -63,28 +61,22 @@ public class RowWithGetters<T extends @NonNull Object>
extends Row {
@SuppressWarnings({"TypeParameterUnusedInFormals", "unchecked"})
public <W> W getValue(int fieldIdx) {
Field field = getSchema().getField(fieldIdx);
- boolean cacheField = cacheFieldType(field);
-
- if (cacheField && cache == null) {
- cache = new TreeMap<>();
- }
-
@Nullable Object fieldValue;
- if (cacheField) {
+ if (cacheFieldType(field)) {
+ @Nullable Object[] cache = this.cache;
if (cache == null) {
- cache = new TreeMap<>();
+ cache = new Object[getters.size()];
Review Comment:
This requires some thought, as it might negatively impact very wide schemas.
In the past, the cache memory usage was proportional to # fields accessed. Now
it always is as large as the whole schema. So if the schema has 1000 fields but
a DoFn only access one of them, we will still allocate enough space for the
entire field set.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
