Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Addressed Karthik's comments, rebased on master, resolved conflicts, and
squashed all but the conflict resolution commit.
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Thanks, Parth! Will make another commit to address small issues that
Karthik pointed out. Let's hold off the actual commit until Drill 1.12 ships.
I'll then commit the changes when we open things
Github user parthchandra commented on the issue:
https://github.com/apache/drill/pull/914
+1. Ship it!
---
Github user parthchandra commented on the issue:
https://github.com/apache/drill/pull/914
I was hoping that we would just get better encapsulation without losing
performance. This performance boost is rather serendipitous.
One possible explanation might be in the JVM optimizing
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
There has been discussion recently on Drill's goal of integrating with
Arrow. The work to use the `Drillbuf` highlights how we can integrate with
Arrow. Simply replace the `Drillbuf` usage with
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Regarding the use of memory addresses. The only reason to do so is
performance. To show the benefit of using addresses, I reran the
`PerformanceTool` class to test the original code, the code
Github user parthchandra commented on the issue:
https://github.com/apache/drill/pull/914
Just FYI, this (https://github.com/d-t-w/netty-buffers) seems to indicate
that some of Netty's internal fragmentation issues seem to have been addressed
since 4.0.37
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Rebased on master. Squashed all previous commits; left code review
revisions separate for now.
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Pushed a commit with the changes suggested by Parth's review comments.
@parthchandra, please review.
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Finally, a note on the fragmentation issue. As you noted, this is a subtle
issue. It is true that Netty maintains a memory pool, based on binary
allocations, that minimizes the normal kind of
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
@parthchandra, thank you for taking the time to review this large chunk of
code. Thanks for noticing the tests; I see those as the only way to ensure that
a change of this size actually works;
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Thank you, @bitblender, for your in-person review. I've pushed a commit
that reflects your comments.
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Rebased on to master with conflicts resolved.
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Added final development stage to this PR. This is a minor implementation
tweak. The result set loader internals for maps held redundant map vectors. In
the final vector container, each map must
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Rebased onto master to resolve conflict in `exec/jdbc-all/pom.xml`.
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
This next commit reintroduces a projection feature. With this change, a
client can:
* Define the set of columns to project
* Define the schema of the data source (table, file, etc.)
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
This commit introduces a feature to limit memory consumed by a batch.
### Batch Size Limits
With this change, the code now has three overlapping limits:
* The traditional
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Rebased onto latest master.
---
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
The second group of five commits refines the result set loader mechanism.
### Model Layer Revision
Drill is unique for a query engine in that it handles structured types as
Github user paul-rogers commented on the issue:
https://github.com/apache/drill/pull/914
Merged a set of enhancements & bug fixes. Compared to the first set of
commits:
* Provides tools to print the sometimes-complex tuple model, schema and
writer object used for complex
20 matches
Mail list logo
