Re: [PR] DRILL-8478. HashPartition memory leak when it allocate memory exception with OutOfMemoryException (#2874) (drill)

2024-01-23 Thread via GitHub
shfshihuafeng commented on code in PR #2875: URL: https://github.com/apache/drill/pull/2875#discussion_r1464211148 ## exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashPartition.java: ## @@ -157,11 +162,11 @@ public HashPartition(FragmentContext

Re: Parquet files with microsecond columns

2024-01-23 Thread James Turton
A reply on your actual topic now. I think that following implementation of the int96_as_timestamp option will result in the type conversion being done "deeply enough" for Drill. I sympathise a lot with the design thinking in your second option but I'd personally go the first route and only

Re: [PR] DRILL-8478. HashPartition memory leak when it allocate memory exception with OutOfMemoryException (#2874) (drill)

2024-01-23 Thread via GitHub
shfshihuafeng commented on code in PR #2875: URL: https://github.com/apache/drill/pull/2875#discussion_r1464211148 ## exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashPartition.java: ## @@ -157,11 +162,11 @@ public HashPartition(FragmentContext

Re: [PR] DRILL-8478. HashPartition memory leak when it allocate memory exception with OutOfMemoryException (#2874) (drill)

2024-01-23 Thread via GitHub
shfshihuafeng commented on code in PR #2875: URL: https://github.com/apache/drill/pull/2875#discussion_r1464222619 ## exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/AbstractHashBinaryRecordBatch.java: ## @@ -1312,7 +1312,9 @@ private void cleanup() {

Re: [PR] DRILL-8478. HashPartition memory leak when it allocate memory exception with OutOfMemoryException (#2874) (drill)

2024-01-23 Thread via GitHub
paul-rogers commented on code in PR #2875: URL: https://github.com/apache/drill/pull/2875#discussion_r1463921977 ## exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/AbstractHashBinaryRecordBatch.java: ## @@ -1312,7 +1312,9 @@ private void cleanup() { }

Re: [PR] DRILL-8474: Add Daffodil Format Plugin (drill)

2024-01-23 Thread via GitHub
mbeckerle commented on PR #2836: URL: https://github.com/apache/drill/pull/2836#issuecomment-1906827568 Ok, so the geo-ip UDF stuff has no special mechanisms or description about those resource files, so the generic code that "scans" must find them and drag them along automatically.

Re: [PR] DRILL-8478. HashPartition memory leak when it allocate memory exception with OutOfMemoryException (#2874) (drill)

2024-01-23 Thread via GitHub
shfshihuafeng commented on code in PR #2875: URL: https://github.com/apache/drill/pull/2875#discussion_r1464211148 ## exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashPartition.java: ## @@ -157,11 +162,11 @@ public HashPartition(FragmentContext

Re: [PR] DRILL-8478. HashPartition memory leak when it allocate memory exception with OutOfMemoryException (#2874) (drill)

2024-01-23 Thread via GitHub
shfshihuafeng commented on code in PR #2875: URL: https://github.com/apache/drill/pull/2875#discussion_r1464211148 ## exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashPartition.java: ## @@ -157,11 +162,11 @@ public HashPartition(FragmentContext

Re: [PR] DRILL-8478. HashPartition memory leak when it allocate memory exception with OutOfMemoryException (#2874) (drill)

2024-01-23 Thread via GitHub
shfshihuafeng commented on code in PR #2875: URL: https://github.com/apache/drill/pull/2875#discussion_r1464211148 ## exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashPartition.java: ## @@ -157,11 +162,11 @@ public HashPartition(FragmentContext

Re: [PR] DRILL-8478. HashPartition memory leak when it allocate memory exception with OutOfMemoryException (#2874) (drill)

2024-01-23 Thread via GitHub
shfshihuafeng commented on code in PR #2875: URL: https://github.com/apache/drill/pull/2875#discussion_r1464211148 ## exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashPartition.java: ## @@ -157,11 +162,11 @@ public HashPartition(FragmentContext

[jira] [Created] (DRILL-8479) mergejion memory leak when exception

2024-01-23 Thread shihuafeng (Jira)
shihuafeng created DRILL-8479: - Summary: mergejion memory leak when exception Key: DRILL-8479 URL: https://issues.apache.org/jira/browse/DRILL-8479 Project: Apache Drill Issue Type: Bug

[PR] Merge join leak (drill)

2024-01-23 Thread via GitHub
shfshihuafeng opened a new pull request, #2877: URL: https://github.com/apache/drill/pull/2877 # [DRILL-](https://issues.apache.org/jira/browse/DRILL-): PR Title (Please replace `PR Title` with actual PR Title) ## Description (Please describe the change. If more

Re: [PR] Merge join leak (drill)

2024-01-23 Thread via GitHub
shfshihuafeng closed pull request #2877: Merge join leak URL: https://github.com/apache/drill/pull/2877 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] mergejoin memory leak when Depleting incoming batches throw exception (drill)

2024-01-23 Thread via GitHub
shfshihuafeng commented on issue #2876: URL: https://github.com/apache/drill/issues/2876#issuecomment-1907421433 [https://github.com/apache/drill/issues/2871](url) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[PR] DRILL-8479: mergejoin leak when Depleting incoming batches throw exception… (drill)

2024-01-23 Thread via GitHub
shfshihuafeng opened a new pull request, #2878: URL: https://github.com/apache/drill/pull/2878 … (#2876) # [DRILL-8479](https://issues.apache.org/jira/browse/DRILL-8479): mergejoin leak when Depleting incoming batches throw exception ## Description when fragment

Re: [I] mergejoin memory leak when Depleting incoming batches throw exception (drill)

2024-01-23 Thread via GitHub
shfshihuafeng commented on issue #2876: URL: https://github.com/apache/drill/issues/2876#issuecomment-1907424021 scenario is reliably repeated by above test ,i fixed 。i can not find leak -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [I] mergejoin memory leak when Depleting incoming batches throw exception (drill)

2024-01-23 Thread via GitHub
shfshihuafeng commented on issue #2876: URL: https://github.com/apache/drill/issues/2876#issuecomment-1907419612 i fixed it see attachment [0001-mergejoin-leak.patch](https://github.com/apache/drill/files/14033613/0001-mergejoin-leak.patch) -- This is an automated message from the

[I] mergejoin memory leak when Depleting incoming batches throw exception (drill)

2024-01-23 Thread via GitHub
shfshihuafeng opened a new issue, #2876: URL: https://github.com/apache/drill/issues/2876 Before submitting a bug report, please verify that you are using the most current version of Drill. **Describe the bug** mergejoin memory leak when Depleting incoming batches throw

Re: [PR] DRILL-8478. HashPartition memory leak when it allocate memory exception with OutOfMemoryException (#2874) (drill)

2024-01-23 Thread via GitHub
shfshihuafeng commented on PR #2875: URL: https://github.com/apache/drill/pull/2875#issuecomment-1905599592 > [An unsued import crept in](https://github.com/apache/drill/actions/runs/7622586264/job/20762475705#step:6:1277), could you remove it please? removed it -- This is an

Re: [PR] DRILL-8478. HashPartition memory leak when it allocate memory exception with OutOfMemoryException (#2874) (drill)

2024-01-23 Thread via GitHub
jnturton commented on PR #2875: URL: https://github.com/apache/drill/pull/2875#issuecomment-1905598192 [An unsued import crept in](https://github.com/apache/drill/actions/runs/7622586264/job/20762475705#step:6:1277), could you remove it please? -- This is an automated message from the

Re: [PR] DRILL-8478. HashPartition memory leak when it allocate memory exception with OutOfMemoryException (#2874) (drill)

2024-01-23 Thread via GitHub
jnturton merged PR #2875: URL: https://github.com/apache/drill/pull/2875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: Parquet files with microsecond columns

2024-01-23 Thread Peter Franzen
Hi Paul, Thanks for your comments. I wasn’t aware that the Web UI doesn’t have sessions; when setting the option at the system level the Web UI behaves as expected. I’ll go ahead and create a pull request within the next few days. /Peter > On 22 Jan 2024, at 21:40, Paul Rogers wrote: > >

Re: Parquet files with microsecond columns

2024-01-23 Thread James Turton
My experience has been that if you've switched on authentication in Drill then web UI /does/ sustain a session. If not then it doesn't. On 2024/01/23 09:48, Peter Franzen wrote: Hi Paul, Thanks for your comments. I wasn’t aware that the Web UI doesn’t have sessions; when setting the option

Re: Parquet files with microsecond columns

2024-01-23 Thread James Turton
A quick test with authentication enabled, each statement issued separately in the web UI. set `window.enable` = false; -- disable window functions select row_number() over (order by 1); org.apache.drill.common.exceptions.UserRemoteException: UNSUPPORTED_OPERATION ERROR: Window functions are

Re: [PR] DRILL-8474: Add Daffodil Format Plugin (drill)

2024-01-23 Thread via GitHub
mbeckerle commented on PR #2836: URL: https://github.com/apache/drill/pull/2836#issuecomment-1906561549 > > @cgivre @paul-rogers is there an example of a Drill UDF that is not part of the drill repository tree? > > I'd like to understand the mechanisms for distributing any jar files and

Re: [PR] DRILL-8474: Add Daffodil Format Plugin (drill)

2024-01-23 Thread via GitHub
cgivre commented on PR #2836: URL: https://github.com/apache/drill/pull/2836#issuecomment-1906689793 > > > @cgivre @paul-rogers is there an example of a Drill UDF that is not part of the drill repository tree? > > > I'd like to understand the mechanisms for distributing any jar files