[jira] [Created] (FLINK-35585) Add documentation for distribution

2024-06-12 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-35585: -- Summary: Add documentation for distribution Key: FLINK-35585 URL: https://issues.apache.org/jira/browse/FLINK-35585 Project: Flink Issue Type: Sub-task

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-06-07 Thread Jim Hughes
ableEnvironment.loadPlan`, it looks like the compiled plan ought to be sufficient to run a job at a later time. I think the FLIP should be clear on the backwards support strategy here. The strategy for streaming is "forever". This may be the most interesting part of the FLIP to discuss

Re: [VOTE] FLIP-459: Support Flink hybrid shuffle integration with Apache Celeborn

2024-06-07 Thread Jim Hughes
HI all, +1 (non-binding) Cheers, Jim On Fri, Jun 7, 2024 at 4:03 AM Yuxin Tan wrote: > Hi everyone, > > Thanks for all the feedback about the FLIP-459 Support Flink > hybrid shuffle integration with Apache Celeborn[1]. > The discussion thread is here [2]. > > I'd like to start a vote for it.

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-05-13 Thread Jim Hughes
Hi Alexey, After some thought, I have a question about deprecations: 3. Can we remove old versions of batch operators eventually? Or do we need to keep them forever like we would for streaming operators? Cheers, Jim On Thu, May 9, 2024 at 11:29 AM Jim Hughes wrote: > Hi Ale

Re: Alignment with FlinkSQL Standards | Feedback requested

2024-05-13 Thread Jim Hughes
Hi Kanchi, One can already directly query Kafka topics using Flink SQL. The Flink Kafka connector allows one to use Kafka to back a Flink table. As a concrete example, this blog post should explain things:

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-05-09 Thread Jim Hughes
Hi Alexey, Overall, the FLIP looks good and makes sense to me. 1. For the testing, I'd call the tests "execution" tests rather than "restore" tests. For streaming execution, restore tests have the compiled plan and intermediate state; the tests verify that those can work together and continue

Re: [DISCUSS] FLIP-XXX: Introduce Flink SQL variables

2024-03-27 Thread Jim Hughes
Hi Ferenc, Looks like a good idea. I'd prefer sticking to the SQL standard if possible. Would it be possible / sensible to allow for each syntax, perhaps managed by a config setting? Cheers, Jim On Tue, Mar 26, 2024 at 6:59 AM Ferenc Csaky wrote: > Hello devs, > > I would like to start a

Re: [DISCUSS] FLIP-434: Support optimizations for pre-partitioned data sources

2024-03-11 Thread Jim Hughes
Hi Jeyhun, I like the idea! Given FLIP-376[1], I wonder if it'd make sense to generalize FLIP-434 to be about "pre-divided" data to cover "buckets" and "partitions" (and maybe even situations where a data source is partitioned and bucketed). Separate from that, the page mentions TPC-H Q1 as an

[jira] [Created] (FLINK-34173) Implement CatalogTable.Builder

2024-01-19 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-34173: -- Summary: Implement CatalogTable.Builder Key: FLINK-34173 URL: https://issues.apache.org/jira/browse/FLINK-34173 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-34172) Add support for altering a distribution via ALTER TABLE

2024-01-19 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-34172: -- Summary: Add support for altering a distribution via ALTER TABLE Key: FLINK-34172 URL: https://issues.apache.org/jira/browse/FLINK-34172 Project: Flink Issue

Re: [DISCUSS] FLIP-419: Optimize multi-sink query plan generation

2024-01-16 Thread Jim Hughes
Hi Jeyhun, Generally, I like the idea of speeding up the optimizer in the case of multiple queries! I am new to the optimizer, but I have a few comments / questions. 1. StreamOptimizeContext may still be needed to pass the fact that we are optimizing a streaming query. I don't think

[jira] [Created] (FLINK-34067) Fix javacc warnings in flink-sql-parser

2024-01-11 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-34067: -- Summary: Fix javacc warnings in flink-sql-parser Key: FLINK-34067 URL: https://issues.apache.org/jira/browse/FLINK-34067 Project: Flink Issue Type: Improvement

Re: [VOTE] FLIP-400: AsyncScalarFunction for asynchronous scalar function support

2023-12-21 Thread Jim Hughes
Hi Alan, +1 (non binding) Cheers, Jim On Wed, Dec 20, 2023 at 2:41 PM Alan Sheinberg wrote: > Hi everyone, > > I'd like to start a vote on FLIP-400 [1]. It covers introducing a new UDF > type, AsyncScalarFunction for completing invocations asynchronously. It > has been discussed in this

[jira] [Created] (FLINK-33918) Fix AsyncSinkWriterThrottlingTest test failure

2023-12-20 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33918: -- Summary: Fix AsyncSinkWriterThrottlingTest test failure Key: FLINK-33918 URL: https://issues.apache.org/jira/browse/FLINK-33918 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-33805) Implement restore tests for OverAggregate node

2023-12-12 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33805: -- Summary: Implement restore tests for OverAggregate node Key: FLINK-33805 URL: https://issues.apache.org/jira/browse/FLINK-33805 Project: Flink Issue Type: Sub

Re: Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-11 Thread Jim Hughes
ust mentioning that ahead of time Cheers, Jim > > [1] https://issues.apache.org/jira/browse/FLINK-24204 > > > > > > > -- > > Best! > Xuyang > > > > > > At 2023-12-09 06:25:30, "Jim Hughes" wrote: > >Hi Xuyang, > &

Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-08 Thread Jim Hughes
Hi Xuyang, As a clarification, since FLINK-24204 is finishing up work from FLIP-145[1], do we need to discuss anything before you work out the details of FLINK-24024 as a PR? Relatedly, as that goes up for a PR, as part of FLINK-33421 [2], Bonnie and I are working through migrating some of the

[jira] [Created] (FLINK-33777) ParquetTimestampITCase>FsStreamingSinkITCaseBase failing in CI

2023-12-07 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33777: -- Summary: ParquetTimestampITCase>FsStreamingSinkITCaseBase failing in CI Key: FLINK-33777 URL: https://issues.apache.org/jira/browse/FLINK-33777 Project: Fl

Re: [DISCUSS] FLIP-400: AsyncScalarFunction for asynchronous scalar function support

2023-12-06 Thread Jim Hughes
Hi Alan, Nicely written and makes sense. The only feedback I have is around the naming of the generalization, e.g. "Specifically, PythonCalcSplitRuleBase will be generalized into RemoteCalcSplitRuleBase." This naming seems to imply/suggest that all Async functions are remote. I wonder if we

[jira] [Created] (FLINK-33767) Implement restore tests for TemporalJoin node

2023-12-06 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33767: -- Summary: Implement restore tests for TemporalJoin node Key: FLINK-33767 URL: https://issues.apache.org/jira/browse/FLINK-33767 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-33758) Implement restore tests for TemporalSort node

2023-12-05 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33758: -- Summary: Implement restore tests for TemporalSort node Key: FLINK-33758 URL: https://issues.apache.org/jira/browse/FLINK-33758 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-33757) Implement restore tests for Rank node

2023-12-05 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33757: -- Summary: Implement restore tests for Rank node Key: FLINK-33757 URL: https://issues.apache.org/jira/browse/FLINK-33757 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-33756) Missing record with CUMULATE/HOP windows using an optimization

2023-12-05 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33756: -- Summary: Missing record with CUMULATE/HOP windows using an optimization Key: FLINK-33756 URL: https://issues.apache.org/jira/browse/FLINK-33756 Project: Flink

[jira] [Created] (FLINK-33676) Implement restore tests for WindowAggregate node

2023-11-28 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33676: -- Summary: Implement restore tests for WindowAggregate node Key: FLINK-33676 URL: https://issues.apache.org/jira/browse/FLINK-33676 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-33667) Implement restore tests for MatchRecognize node

2023-11-27 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33667: -- Summary: Implement restore tests for MatchRecognize node Key: FLINK-33667 URL: https://issues.apache.org/jira/browse/FLINK-33667 Project: Flink Issue Type: Sub

Re: [VOTE] FLIP-393: Make QueryOperations SQL serializable

2023-11-21 Thread Jim Hughes
+1 (non-binding) Thanks, Dawid! Jim On Tue, Nov 21, 2023 at 7:20 AM Gyula Fóra wrote: > +1 (binding) > > Gyula > > On Tue, 21 Nov 2023 at 13:11, xiangyu feng wrote: > > > +1 (non-binding) > > > > Thanks for driving this. > > > > Best, > > Xiangyu Feng > > > > > > Ferenc Csaky 于2023年11月21日周二

[jira] [Created] (FLINK-33601) Implement restore tests for Expand node

2023-11-20 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33601: -- Summary: Implement restore tests for Expand node Key: FLINK-33601 URL: https://issues.apache.org/jira/browse/FLINK-33601 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-33521) Implement restore tests for PythonCalc node

2023-11-10 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33521: -- Summary: Implement restore tests for PythonCalc node Key: FLINK-33521 URL: https://issues.apache.org/jira/browse/FLINK-33521 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-33488) Implement restore tests for Deduplicate node

2023-11-08 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33488: -- Summary: Implement restore tests for Deduplicate node Key: FLINK-33488 URL: https://issues.apache.org/jira/browse/FLINK-33488 Project: Flink Issue Type: Sub

Re: [VOTE] FLIP-376: Add DISTRIBUTED BY clause

2023-11-07 Thread Jim Hughes
Hi all, +1 (non-binding) Cheers, Jim On Mon, Nov 6, 2023 at 6:39 AM Timo Walther wrote: > Hi everyone, > > I'd like to start a vote on FLIP-376: Add DISTRIBUTED BY clause[1] which > has been discussed in this thread [2]. > > The vote will be open for at least 72 hours unless there is an

[jira] [Created] (FLINK-33470) Implement restore tests for Join node

2023-11-06 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33470: -- Summary: Implement restore tests for Join node Key: FLINK-33470 URL: https://issues.apache.org/jira/browse/FLINK-33470 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-33469) Implement restore tests for Limit node

2023-11-06 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-33469: -- Summary: Implement restore tests for Limit node Key: FLINK-33469 URL: https://issues.apache.org/jira/browse/FLINK-33469 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] [FLINK-32873] Add a config to allow disabling Query hints

2023-09-11 Thread Jim Hughes
Hi Jing and Jark! I can definitely appreciate the desire to have fewer configurations. Do you have a suggested alternative for platform providers to limit or restrict the hints that Bonnie is talking about? As one possibility, maybe one configuration could be set to control all hints. Cheers,

[jira] [Created] (FLINK-32691) SELECT fcn does not work with an unset catalog or database

2023-07-26 Thread Jim Hughes (Jira)
Jim Hughes created FLINK-32691: -- Summary: SELECT fcn does not work with an unset catalog or database Key: FLINK-32691 URL: https://issues.apache.org/jira/browse/FLINK-32691 Project: Flink Issue

Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement

2023-04-10 Thread Jim Hughes
Should this FLIP involve invalidating that information in some manner? Or does truncating a source table for a query cause undefined behavior on that query? Basically, I'm trying to think through the implementations of a truncate operation to streaming sources and queries. Cheers, Jim > Best regar

Re: [DISCUSS] FLIP-302: Support TRUNCATE TABLE statement

2023-04-10 Thread Jim Hughes
Hi Yuxia, Two questions: 1. Are you expecting all DynamicTableSinks to support Truncate? The FLIP could use some explanation for what supporting and not supporting the operation means. 2. How will truncate inoperate with streaming queries? That is, if I truncate an input to a streaming

Re: [DISCUSS] Add support for Apache Arrow format

2023-03-30 Thread Jim Hughes
Hi all, How do Flink formats relate to or interact with Paimon (formerly Flink-Table-Store)? If the Flink format interface is used there, then it may be useful to consider Arrow along with other columnar formats. Separately, from previous experience, I've seen the Arrow format be useful as an

Re: [DISCUSS] FLIP-275: Support Remote SQL Client Based on SQL Gateway

2022-11-30 Thread Jim Hughes
Hi Yu, Thanks for moving my comments to this thread! Also, thank you for answering my questions; it is helping me understand the SQL Gateway better. 5. > Our idea is to introduce a new session option (like 'sql-client.result.fetch-interval') to control the fetching requests sending frequency.

Re: SQL Gateway and SQL Client

2022-11-28 Thread Jim Hughes
n my team about my design. My teammates > > think it’s better to unify the local and remote mode, so I’ve > investigated > > and redesigned a new plan. I’ll inform you after the rewriting of FLIP > > finished (will be soon) and Shengkai will help to your PR. > > > > &

Re: SQL Gateway and SQL Client

2022-11-21 Thread Jim Hughes
PRs I see look pretty good. Are they going to be merged soon? Anything else I can do to help? Cheers, Jim On Mon, Nov 14, 2022 at 12:52 PM Jim Hughes wrote: > Hi Yu, > > The PR looks good to me; the only thing I noticed is that the > RemoveJarOperation may need to be added. > &g

Re: SQL Gateway and SQL Client

2022-11-14 Thread Jim Hughes
ink/table/gateway/rest/handler/AbstractSqlGatewayRestHandler.java#L98 > > Jim Hughes 于2022年11月12日周六 00:45写道: > > > Hi Shengkai, > > > > I think there is an additional case where a proxy is between the client > and > > gateway. In that case, being able to pass headers would

Re: SQL Gateway and SQL Client

2022-11-14 Thread Jim Hughes
at 10:45 PM yu zelin wrote: > Hi Jim, > > It would be nice if you can take a look on > https://github.com/apache/flink/pull/21133 and give me some feedback. > > Best, > Yu Zelin > > > 2022年11月12日 00:44,Jim Hughes 写道: > > > > Hi Shengkai, > > >

Re: SQL Gateway and SQL Client

2022-11-11 Thread Jim Hughes
an join us to finish the feature. I think the > modification about the SQL Gateway side is ready to review. > > Best, > Shengkai > > > Jim Hughes 于2022年11月11日周五 05:19写道: > > > Hi Yu Zelin, > > > > I have read through your draft and it looks good. I am n

Re: SQL Gateway and SQL Client

2022-11-10 Thread Jim Hughes
Hi Yu Zelin, I have read through your draft and it looks good. I am new to Flink, so I haven't learned about everything which needs to be done yet. One of the considerations that I'm interested in understanding is how to pass additional headers when sending REST requests. From looking at the