[jira] [Commented] (FLINK-11395) Support for Avro StreamingFileSink
[ https://issues.apache.org/jira/browse/FLINK-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915108#comment-16915108 ] vinoyang commented on FLINK-11395: -- [~gyfora] right. This issue is inactive, [~elango_ganesan] could you share more thought? > Support for Avro StreamingFileSink > -- > > Key: FLINK-11395 > URL: https://issues.apache.org/jira/browse/FLINK-11395 > Project: Flink > Issue Type: New Feature > Components: Connectors / FileSystem >Reporter: Elango Ganesan >Assignee: vinoyang >Priority: Major > > Current implementation for StreamingFileSink supports Rowformat for text and > json files . BulkWriter has support for Parquet files . Out of the box > RowFormat does not seem to be right fit for writing Avro files as avro files > need to persist metadata when opening new file. We are thinking of > implementing Avro writers similar to how Parquet is implemented . I am happy > to submit a PR if this approach sounds good . -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops
flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops URL: https://github.com/apache/flink/pull/9383#issuecomment-519130955 ## CI report: * 5d8448c4813f5b362f98f898998f1278f062d807 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122292142) * 4d628935e8899d6019566bfc93b5c688bc1835ec : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122941321) * d7c0bd5edc65110910d79ca7c7bf2139672f8c02 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123229382) * b7a19fe5d83ee271e7560f90fbf07a7703937273 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123235786) * 7650b3b19b05ed6a121566d7c19d5e7bc71489fa : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123332630) * 2493723ebd2c307f47bbdfcf154a31ab97cda312 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123335647) * f3f0fe6d16ef3bba35d06a797196f94f372701ff : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123336279) * c6ee15104ee678c239367670773723920e34c26d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123371348) * d0e4fbf25a8ff9982171ed982868b51ad851aaf0 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123472764) * 741386a495a5657bb654dcd0168f2d42873445e7 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/123476701) * 05e27c097851c65bd9a405b4aae376e2ef6c2b50 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123645059) * 8fba78f22bc6c0d042cb1dde270c02af08d98bbd : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123696981) * 681ac331e4c0b547e1d410b448bc34ff651dbc6a : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123698096) * e18739460fbcf7c59be3c9121fc26fc279e0353a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123703814) * 670affbc0ae2883e93bb2f6ca3c1300fb78f26c5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123726809) * 50a91cbc74c645576432d25d40e7a42190ac28e8 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123758034) * 2fba64d8e9fa4939c5a6c3fb3d758d55ca344b6c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123842040) * 0507aa67d2f7183c3a7e4556fbf7732414647ac7 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123855539) * 9a293ae331986d8fae16f619158f1e59572cd1e9 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123856922) * 2bb1bc290a3fc8e4dc843be781063515e86509d8 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/123859379) * 9ff3bc6cdeaff18473a075076ae4931bbdae7173 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124474961) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops
flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops URL: https://github.com/apache/flink/pull/9383#issuecomment-519130955 ## CI report: * 5d8448c4813f5b362f98f898998f1278f062d807 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122292142) * 4d628935e8899d6019566bfc93b5c688bc1835ec : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/122941321) * d7c0bd5edc65110910d79ca7c7bf2139672f8c02 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123229382) * b7a19fe5d83ee271e7560f90fbf07a7703937273 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123235786) * 7650b3b19b05ed6a121566d7c19d5e7bc71489fa : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123332630) * 2493723ebd2c307f47bbdfcf154a31ab97cda312 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123335647) * f3f0fe6d16ef3bba35d06a797196f94f372701ff : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123336279) * c6ee15104ee678c239367670773723920e34c26d : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123371348) * d0e4fbf25a8ff9982171ed982868b51ad851aaf0 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123472764) * 741386a495a5657bb654dcd0168f2d42873445e7 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/123476701) * 05e27c097851c65bd9a405b4aae376e2ef6c2b50 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123645059) * 8fba78f22bc6c0d042cb1dde270c02af08d98bbd : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123696981) * 681ac331e4c0b547e1d410b448bc34ff651dbc6a : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123698096) * e18739460fbcf7c59be3c9121fc26fc279e0353a : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123703814) * 670affbc0ae2883e93bb2f6ca3c1300fb78f26c5 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123726809) * 50a91cbc74c645576432d25d40e7a42190ac28e8 : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123758034) * 2fba64d8e9fa4939c5a6c3fb3d758d55ca344b6c : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/123842040) * 0507aa67d2f7183c3a7e4556fbf7732414647ac7 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123855539) * 9a293ae331986d8fae16f619158f1e59572cd1e9 : CANCELED [Build](https://travis-ci.com/flink-ci/flink/builds/123856922) * 2bb1bc290a3fc8e4dc843be781063515e86509d8 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/123859379) * 9ff3bc6cdeaff18473a075076ae4931bbdae7173 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops
flinkbot edited a comment on issue #9383: [FLINK-13248] [runtime] Adding processing of downstream messages in AsyncWaitOperator's wait loops URL: https://github.com/apache/flink/pull/9383#issuecomment-519129010 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 9ff3bc6cdeaff18473a075076ae4931bbdae7173 (Sat Aug 24 20:59:08 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13248).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[ https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915052#comment-16915052 ] Kostas Kloudas commented on FLINK-13832: Oh sorry, I just saw that you just deprecate it. The diff was a bit confusing. In this case, I think it is ok to just go on with the discussion without any FLIP. > DefaultRollingPolicy create() method should be renamed to builder() > --- > > Key: FLINK-13832 > URL: https://issues.apache.org/jira/browse/FLINK-13832 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Affects Versions: 1.9.0 >Reporter: Gyula Fora >Assignee: Gyula Fora >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The DefaultRollingPolicy.create() method returns an instance of > PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new > method named .builder() and deprecate "create". > Right now if we want to create a new instance with the default settings we > have to call: > DefaultRollingPolicy.create().build() > This nicely illustrates the problem with the naming of this method. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[
https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915051#comment-16915051
]
Kostas Kloudas commented on FLINK-13832:
The PR you have opened has the method {{DefaultRollingPolicy.create()}} renamed
to {{DefaultRollingPolicy.builder()}}, right? Or am I missing something? I am
referring to this change as breaking.
> DefaultRollingPolicy create() method should be renamed to builder()
> ---
>
> Key: FLINK-13832
> URL: https://issues.apache.org/jira/browse/FLINK-13832
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
>Affects Versions: 1.9.0
>Reporter: Gyula Fora
>Assignee: Gyula Fora
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The DefaultRollingPolicy.create() method returns an instance of
> PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new
> method named .builder() and deprecate "create".
> Right now if we want to create a new instance with the default settings we
> have to call:
> DefaultRollingPolicy.create().build()
> This nicely illustrates the problem with the naming of this method.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[ https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915049#comment-16915049 ] Gyula Fora commented on FLINK-13832: To be precise I am not suggesting to remove the method only to deprecate it for now. It is clearly a case of confusing naming and is a very good target for deprecation and future removal, in Flink 2.0 for example when we intend to remove other deprecated methods. > DefaultRollingPolicy create() method should be renamed to builder() > --- > > Key: FLINK-13832 > URL: https://issues.apache.org/jira/browse/FLINK-13832 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Affects Versions: 1.9.0 >Reporter: Gyula Fora >Assignee: Gyula Fora >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The DefaultRollingPolicy.create() method returns an instance of > PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new > method named .builder() and deprecate "create". > Right now if we want to create a new instance with the default settings we > have to call: > DefaultRollingPolicy.create().build() > This nicely illustrates the problem with the naming of this method. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[
https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915048#comment-16915048
]
Kostas Kloudas commented on FLINK-13832:
I am refering to renaming the {{create()}} to {{builder().}}
> DefaultRollingPolicy create() method should be renamed to builder()
> ---
>
> Key: FLINK-13832
> URL: https://issues.apache.org/jira/browse/FLINK-13832
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
>Affects Versions: 1.9.0
>Reporter: Gyula Fora
>Assignee: Gyula Fora
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The DefaultRollingPolicy.create() method returns an instance of
> PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new
> method named .builder() and deprecate "create".
> Right now if we want to create a new instance with the default settings we
> have to call:
> DefaultRollingPolicy.create().build()
> This nicely illustrates the problem with the naming of this method.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[ https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915043#comment-16915043 ] Kostas Kloudas commented on FLINK-13832: To some extent, I agree. But the discussion then goes to "what is a small change and what is a big one" when it comes to PublicEvolving. We did the same with the Program interface (which was a trivial discussion/FLIP/Vote). I am not so sure if a FLIP is required as this is pretty small but I would suggest to open a discussion and ask also this in the discussion thread. > DefaultRollingPolicy create() method should be renamed to builder() > --- > > Key: FLINK-13832 > URL: https://issues.apache.org/jira/browse/FLINK-13832 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Affects Versions: 1.9.0 >Reporter: Gyula Fora >Assignee: Gyula Fora >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The DefaultRollingPolicy.create() method returns an instance of > PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new > method named .builder() and deprecate "create". > Right now if we want to create a new instance with the default settings we > have to call: > DefaultRollingPolicy.create().build() > This nicely illustrates the problem with the naming of this method. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[ https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915040#comment-16915040 ] Gyula Fora edited comment on FLINK-13832 at 8/24/19 4:46 PM: - I don't mind opening a quick discussion about deprecating this method, but I don't think Flink would benefit from a FLIP for such a trivial case as it doesnt break current user code. There definitely should be a FLIP once we start removing deprecated methods for Flink 2.0 that should contain this as part of the work, but if we open a FLIP for every single method that should eventually be renamed for a better API we will ruin the wiki :D was (Author: gyfora): I don't mind opening a quick discussion about deprecating this method, but I don't think Flink would benefit a Flip for such a trivial case as it doesnt break current user code. There definitely should be a FLIP once we start removing deprecated methods for Flink 2.0 that should contain this as part of the work, but if we open a FLIP for every single method that should eventually be renamed for a better API we will ruin the wiki :D > DefaultRollingPolicy create() method should be renamed to builder() > --- > > Key: FLINK-13832 > URL: https://issues.apache.org/jira/browse/FLINK-13832 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Affects Versions: 1.9.0 >Reporter: Gyula Fora >Assignee: Gyula Fora >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The DefaultRollingPolicy.create() method returns an instance of > PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new > method named .builder() and deprecate "create". > Right now if we want to create a new instance with the default settings we > have to call: > DefaultRollingPolicy.create().build() > This nicely illustrates the problem with the naming of this method. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[ https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915040#comment-16915040 ] Gyula Fora commented on FLINK-13832: I don't mind opening a quick discussion about deprecating this method, but I don't think Flink would benefit a Flip for such a trivial case as it doesnt break current user code. There definitely should be a FLIP once we start removing deprecated methods for Flink 2.0 that should contain this as part of the work, but if we open a FLIP for every single method that should eventually be renamed for a better API we will ruin the wiki :D > DefaultRollingPolicy create() method should be renamed to builder() > --- > > Key: FLINK-13832 > URL: https://issues.apache.org/jira/browse/FLINK-13832 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Affects Versions: 1.9.0 >Reporter: Gyula Fora >Assignee: Gyula Fora >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The DefaultRollingPolicy.create() method returns an instance of > PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new > method named .builder() and deprecate "create". > Right now if we want to create a new instance with the default settings we > have to call: > DefaultRollingPolicy.create().build() > This nicely illustrates the problem with the naming of this method. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (FLINK-9294) Improve type inference for UDFs with composite parameter or result type
[
https://issues.apache.org/jira/browse/FLINK-9294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-9294:
--
Labels: pull-request-available (was: )
> Improve type inference for UDFs with composite parameter or result type
>
>
> Key: FLINK-9294
> URL: https://issues.apache.org/jira/browse/FLINK-9294
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Reporter: Rong Rong
>Assignee: Rong Rong
>Priority: Major
> Labels: pull-request-available
>
> Most of the UDF function signatures that includes composite types such as
> *{{MAP}}*, *{{ARRAY}}*, etc would require user to override
> *{{getParameterType}}* or *{{getResultType}}* method explicitly.
> It should be able to resolve the composite type based on the function
> signature, such as:
> {code:java}
> public String[] eval(Map mapArg) { /* ... */ }
> {code}
> The function catalog search should do either of the following:
> [Update]
> since we have backward compatibility issue with resolving to a different
> type, we will not go with the modify type option.
> - -Automatically resolve that:-
> -1. *{{ObjectArrayTypeInfo}}* to be the result type.-
> -2. *{{MapTypeInfo}}* to be the
> parameter type.-
> - Improved function mapping to find and locate function with such signatures
>
> [Update]
> This ticket should only cover *Map* and *Row* type, It does not cover
> * ObjectArrayType, since Array is actually resolved by eval method signature
> correctly.
> * Pojo types, Pojo will be addressed separately.
> This ticket should consolidate some discrepancy between how TableFunction,
> AggregateFunction and ScalarFunction resolves types. which at this moment
> goes through different code path.
> The rest of the optimization should go to follow up tickets in FLINK-9484
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] walterddr closed pull request #6472: [FLINK-9294] [table] Improve type inference for UDFs with composite parameter and/or result type
walterddr closed pull request #6472: [FLINK-9294] [table] Improve type inference for UDFs with composite parameter and/or result type URL: https://github.com/apache/flink/pull/6472 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] walterddr commented on issue #6472: [FLINK-9294] [table] Improve type inference for UDFs with composite parameter and/or result type
walterddr commented on issue #6472: [FLINK-9294] [table] Improve type inference for UDFs with composite parameter and/or result type URL: https://github.com/apache/flink/pull/6472#issuecomment-524564718 close in favor of [FLIP-37](https://cwiki.apache.org/confluence/display/FLINK/FLIP-37%3A+Rework+of+the+Table+API+Type+System) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[ https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914999#comment-16914999 ] Gyula Fora commented on FLINK-13832: Do you mean the eventual removal of the method will break user code? > DefaultRollingPolicy create() method should be renamed to builder() > --- > > Key: FLINK-13832 > URL: https://issues.apache.org/jira/browse/FLINK-13832 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Affects Versions: 1.9.0 >Reporter: Gyula Fora >Assignee: Gyula Fora >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The DefaultRollingPolicy.create() method returns an instance of > PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new > method named .builder() and deprecate "create". > Right now if we want to create a new instance with the default settings we > have to call: > DefaultRollingPolicy.create().build() > This nicely illustrates the problem with the naming of this method. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning
flinkbot edited a comment on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning URL: https://github.com/apache/flink/pull/9525#issuecomment-524508878 ## CI report: * 1af1d3eed520394012342a900558baf124bc7f44 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124447039) * 7342071adbe407c8215f89c4bbf35efed8767994 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124464599) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (FLINK-12573) ability to add suffix to part file created in Bucket (StreamingFileSink)
[ https://issues.apache.org/jira/browse/FLINK-12573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kostas Kloudas closed FLINK-12573. -- Resolution: Invalid This is redundant after merging https://issues.apache.org/jira/browse/FLINK-13428 > ability to add suffix to part file created in Bucket (StreamingFileSink) > > > Key: FLINK-12573 > URL: https://issues.apache.org/jira/browse/FLINK-12573 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Affects Versions: 1.8.0 >Reporter: yitzchak lieberman >Assignee: Louis Xu >Priority: Major > > a possibility to add suffix to part file path other than: > new Path(bucketPath, PART_PREFIX + -- subtaskIndex + partCounter); > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[ https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914985#comment-16914985 ] Kostas Kloudas commented on FLINK-13832: Hi [~gyfora]! Thanks for the contribution. The changes (from a first look) look good, but the problem is that this PR changes a class annotated as public evolving. This is not, strictly speaking, a public API, but such a change will break user code. This means that before merging such a change, there should be a discussion, a FLIP and a voting thread. I know that for such a minute change this is too much work, but in the long-run, such a process will let the project grow in a transparent way. > DefaultRollingPolicy create() method should be renamed to builder() > --- > > Key: FLINK-13832 > URL: https://issues.apache.org/jira/browse/FLINK-13832 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Affects Versions: 1.9.0 >Reporter: Gyula Fora >Assignee: Gyula Fora >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The DefaultRollingPolicy.create() method returns an instance of > PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new > method named .builder() and deprecate "create". > Right now if we want to create a new instance with the default settings we > have to call: > DefaultRollingPolicy.create().build() > This nicely illustrates the problem with the naming of this method. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-11931) Avro Bulk Writer Factory for StreamingFileSink Bulk Format
[ https://issues.apache.org/jira/browse/FLINK-11931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914983#comment-16914983 ] Gyula Fora commented on FLINK-11931: This is a duplicate of https://issues.apache.org/jira/browse/FLINK-11395 > Avro Bulk Writer Factory for StreamingFileSink Bulk Format > -- > > Key: FLINK-11931 > URL: https://issues.apache.org/jira/browse/FLINK-11931 > Project: Flink > Issue Type: New Feature > Components: Connectors / FileSystem >Reporter: Achyuth Narayan Samudrala >Priority: Major > > Currently the StreamingFileSink.BulkFormatBuilder expects a > BulkWriter.Factory implementation. By default there is only support for the > Parquet format. But there is no such implementation for Avro format. This > feature request would add the avro writer factory implementation. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Closed] (FLINK-11931) Avro Bulk Writer Factory for StreamingFileSink Bulk Format
[ https://issues.apache.org/jira/browse/FLINK-11931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyula Fora closed FLINK-11931. -- Resolution: Duplicate > Avro Bulk Writer Factory for StreamingFileSink Bulk Format > -- > > Key: FLINK-11931 > URL: https://issues.apache.org/jira/browse/FLINK-11931 > Project: Flink > Issue Type: New Feature > Components: Connectors / FileSystem >Reporter: Achyuth Narayan Samudrala >Priority: Major > > Currently the StreamingFileSink.BulkFormatBuilder expects a > BulkWriter.Factory implementation. By default there is only support for the > Parquet format. But there is no such implementation for Avro format. This > feature request would add the avro writer factory implementation. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-11395) Support for Avro StreamingFileSink
[ https://issues.apache.org/jira/browse/FLINK-11395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914982#comment-16914982 ] Gyula Fora commented on FLINK-11395: Wouldn't the Avro format be a Row based format implementing Encoder instead of a bulk writer? > Support for Avro StreamingFileSink > -- > > Key: FLINK-11395 > URL: https://issues.apache.org/jira/browse/FLINK-11395 > Project: Flink > Issue Type: New Feature > Components: Connectors / FileSystem >Reporter: Elango Ganesan >Assignee: vinoyang >Priority: Major > > Current implementation for StreamingFileSink supports Rowformat for text and > json files . BulkWriter has support for Parquet files . Out of the box > RowFormat does not seem to be right fit for writing Avro files as avro files > need to persist metadata when opening new file. We are thinking of > implementing Avro writers similar to how Parquet is implemented . I am happy > to submit a PR if this approach sounds good . -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning
flinkbot edited a comment on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning URL: https://github.com/apache/flink/pull/9525#issuecomment-524508878 ## CI report: * 1af1d3eed520394012342a900558baf124bc7f44 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124447039) * 7342071adbe407c8215f89c4bbf35efed8767994 : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/124464599) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning
flinkbot edited a comment on issue #9525: [FLINK-13363][docs] Add documentation for streaming aggregate performance tuning URL: https://github.com/apache/flink/pull/9525#issuecomment-524508878 ## CI report: * 1af1d3eed520394012342a900558baf124bc7f44 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124447039) * 7342071adbe407c8215f89c4bbf35efed8767994 : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9528: [FLINK-13829] [docs] Fix the base url of release 1.9 docs and mark 1.9 as stable
flinkbot edited a comment on issue #9528: [FLINK-13829] [docs] Fix the base url of release 1.9 docs and mark 1.9 as stable URL: https://github.com/apache/flink/pull/9528#issuecomment-524549611 ## CI report: * 358d971a69be296f264c555a14db8016b0f5b61a : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124461665) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13395) Add source and sink connector for Aliyun Log Service
[ https://issues.apache.org/jira/browse/FLINK-13395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ke Li updated FLINK-13395: -- Description: Aliyun Log Service is a big data service which has been widely used in Alibaba Group and thousands of customers of Alibaba Cloud. The core storage engine of Log Service is named Loghub which is a large scale distributed storage system which provides producer and consumer to push and pull data like Kafka, AWS Kinesis and Azure Eventhub does. There are a lot of users are using Log Service to collect and analysis data from both on premise and cloud data sources, and consuming data stored in Log Service from Flink or Blink for streaming computing. was: Aliyun Log Service is a big data service which has been widely used in Alibaba Group and thousands of companies on Alibaba Cloud. The core storage engine of Log Service is called Loghub which is a large scale distributed storage system and provides producer/consumer API like Kafka or AWS Kinesis. There are a lot of users of Flink are using Log Service to collect and analysis data from both on premise and cloud data sources, and consuming data stored in Log Service from Flink or Blink for streaming compute. > Add source and sink connector for Aliyun Log Service > > > Key: FLINK-13395 > URL: https://issues.apache.org/jira/browse/FLINK-13395 > Project: Flink > Issue Type: New Feature > Components: Connectors / Common >Reporter: Ke Li >Priority: Major > > Aliyun Log Service is a big data service which has been widely used in > Alibaba Group and thousands of customers of Alibaba Cloud. The core storage > engine of Log Service is named Loghub which is a large scale distributed > storage system which provides producer and consumer to push and pull data > like Kafka, AWS Kinesis and Azure Eventhub does. > There are a lot of users are using Log Service to collect and analysis data > from both on premise and cloud data sources, and consuming data stored in Log > Service from Flink or Blink for streaming computing. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9528: [FLINK-13829] [docs] Fix the base url of release 1.9 docs and mark 1.9 as stable
flinkbot edited a comment on issue #9528: [FLINK-13829] [docs] Fix the base url of release 1.9 docs and mark 1.9 as stable URL: https://github.com/apache/flink/pull/9528#issuecomment-524549611 ## CI report: * 358d971a69be296f264c555a14db8016b0f5b61a : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/124461665) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9528: [FLINK-13829] [docs] Fix the base url of release 1.9 docs and mark 1.9 as stable
flinkbot commented on issue #9528: [FLINK-13829] [docs] Fix the base url of release 1.9 docs and mark 1.9 as stable URL: https://github.com/apache/flink/pull/9528#issuecomment-524549611 ## CI report: * 358d971a69be296f264c555a14db8016b0f5b61a : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-13829) Incorrect version of 1.9 docs
[ https://issues.apache.org/jira/browse/FLINK-13829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914914#comment-16914914 ] Zhu Zhu edited comment on FLINK-13829 at 8/24/19 1:14 PM: -- The cause should that the _*baseurl**/**javadocs_baseurl/pythondocs_baseurl*_ **in _*docs/_config.yml_* still points to _*//ci.apache.org/projects/flink/flink-docs-master*_. Pointing them to _*//ci.apache.org/projects/flink/flink-docs-*__*release-1.9*_ should fix it. Besides that, I found that 1.9 is not marked as stable in *__config.yml_*, resulting in that the project templates in quickstart pages be incorrect. [~till.rohrmann], I opened a PR to fix it, could you or anyone else familiar with the release process help to take a look? was (Author: zhuzh): The cause should that the _*baseurl**/**javadocs_baseurl/pythondocs_baseurl_ *in _*docs/_config.yml_* still points to _*//ci.apache.org/projects/flink/flink-docs-master*_. Pointing them to _*//ci.apache.org/projects/flink/flink-docs-*__*release-1.9*_ should fix it. Besides that, I found that 1.9 is not marked as stable in *__config.yml_*, resulting in that the project templates in quickstart pages be incorrect. [~till.rohrmann], could you assign this issue to me so I can fix it? > Incorrect version of 1.9 docs > - > > Key: FLINK-13829 > URL: https://issues.apache.org/jira/browse/FLINK-13829 > Project: Flink > Issue Type: Task > Components: Documentation >Affects Versions: 1.9.0 >Reporter: Paul Lin >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The document of Flink 1.9 links to docs of master branch, which should be > 1.9-release branch. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot commented on issue #9528: [FLINK-13829] [docs] Fix the base url of release 1.9 docs and mark 1.9 as stable
flinkbot commented on issue #9528: [FLINK-13829] [docs] Fix the base url of release 1.9 docs and mark 1.9 as stable URL: https://github.com/apache/flink/pull/9528#issuecomment-524549337 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit 358d971a69be296f264c555a14db8016b0f5b61a (Sat Aug 24 13:14:39 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! * **This pull request references an unassigned [Jira ticket](https://issues.apache.org/jira/browse/FLINK-13829).** According to the [code contribution guide](https://flink.apache.org/contributing/contribute-code.html), tickets need to be assigned before starting with the implementation work. Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13829) Incorrect version of 1.9 docs
[ https://issues.apache.org/jira/browse/FLINK-13829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-13829: --- Labels: pull-request-available (was: ) > Incorrect version of 1.9 docs > - > > Key: FLINK-13829 > URL: https://issues.apache.org/jira/browse/FLINK-13829 > Project: Flink > Issue Type: Task > Components: Documentation >Affects Versions: 1.9.0 >Reporter: Paul Lin >Priority: Major > Labels: pull-request-available > > The document of Flink 1.9 links to docs of master branch, which should be > 1.9-release branch. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] zhuzhurk opened a new pull request #9528: [FLINK-13829] [docs] Fix the base url of release 1.9 docs and mark 1.9 as stable
zhuzhurk opened a new pull request #9528: [FLINK-13829] [docs] Fix the base url of release 1.9 docs and mark 1.9 as stable URL: https://github.com/apache/flink/pull/9528 ## What is the purpose of the change *The links in release 1.9 docs to pages of master docs, rather than pages of release-1.9 docs.* *Besides that, release 1.9 is not marked as stable in _config.yml, resulting in that the project templates in quickstart pages be incorrect.* ## Brief change log - *fix baseurl/javadocs_baseurl/pythondocs_baseurl in docs/_config.yml* - *set is_stable to true in docs/_config.yml* ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (FLINK-13829) Incorrect version of 1.9 docs
[ https://issues.apache.org/jira/browse/FLINK-13829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914914#comment-16914914 ] Zhu Zhu edited comment on FLINK-13829 at 8/24/19 1:05 PM: -- The cause should that the _*baseurl**/**javadocs_baseurl/pythondocs_baseurl_ *in _*docs/_config.yml_* still points to _*//ci.apache.org/projects/flink/flink-docs-master*_. Pointing them to _*//ci.apache.org/projects/flink/flink-docs-*__*release-1.9*_ should fix it. Besides that, I found that 1.9 is not marked as stable in *__config.yml_*, resulting in that the project templates in quickstart pages be incorrect. [~till.rohrmann], could you assign this issue to me so I can fix it? was (Author: zhuzh): The cause should that the _*baseurl**/**javadocs_baseurl/pythondocs_baseurl***_ in _*docs/_config.yml*_ still points to _*//ci.apache.org/projects/flink/flink-docs-master*_. Pointing them to _*//ci.apache.org/projects/flink/flink-docs-*__*release-1.9*_ should fix it. Hi [~till.rohrmann], could you assign this issue to me so I can fix it? > Incorrect version of 1.9 docs > - > > Key: FLINK-13829 > URL: https://issues.apache.org/jira/browse/FLINK-13829 > Project: Flink > Issue Type: Task > Components: Documentation >Affects Versions: 1.9.0 >Reporter: Paul Lin >Priority: Major > > The document of Flink 1.9 links to docs of master branch, which should be > 1.9-release branch. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (FLINK-13842) Improve Javadocs and web documentation of the StreamingFileSink
Gyula Fora created FLINK-13842: -- Summary: Improve Javadocs and web documentation of the StreamingFileSink Key: FLINK-13842 URL: https://issues.apache.org/jira/browse/FLINK-13842 Project: Flink Issue Type: Improvement Components: Connectors / FileSystem, Documentation Reporter: Gyula Fora Assignee: Gyula Fora Both the javadocs and the web docs of the StreamingFileSink and associated components should be improved for a smooth user experience with more detailed explanations and examples. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-13829) Incorrect version of 1.9 docs
[ https://issues.apache.org/jira/browse/FLINK-13829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914914#comment-16914914 ] Zhu Zhu commented on FLINK-13829: - The cause should that the _*baseurl**/**javadocs_baseurl/pythondocs_baseurl***_ in _*docs/_config.yml*_ still points to _*//ci.apache.org/projects/flink/flink-docs-master*_. Pointing them to _*//ci.apache.org/projects/flink/flink-docs-*__*release-1.9*_ should fix it. Hi [~till.rohrmann], could you assign this issue to me so I can fix it? > Incorrect version of 1.9 docs > - > > Key: FLINK-13829 > URL: https://issues.apache.org/jira/browse/FLINK-13829 > Project: Flink > Issue Type: Task > Components: Documentation >Affects Versions: 1.9.0 >Reporter: Paul Lin >Priority: Major > > The document of Flink 1.9 links to docs of master branch, which should be > 1.9-release branch. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9527: [FLINK-13832] Rename DefaultRollingPolicy create to builder
flinkbot edited a comment on issue #9527: [FLINK-13832] Rename DefaultRollingPolicy create to builder URL: https://github.com/apache/flink/pull/9527#issuecomment-524532599 ## CI report: * 5e573e0756c77c14e481717ab35eb6df5b537abd : FAILURE [Build](https://travis-ci.com/flink-ci/flink/builds/124455219) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot edited a comment on issue #9527: [FLINK-13832] Rename DefaultRollingPolicy create to builder
flinkbot edited a comment on issue #9527: [FLINK-13832] Rename DefaultRollingPolicy create to builder URL: https://github.com/apache/flink/pull/9527#issuecomment-524532599 ## CI report: * 5e573e0756c77c14e481717ab35eb6df5b537abd : PENDING [Build](https://travis-ci.com/flink-ci/flink/builds/124455219) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] flinkbot commented on issue #9527: [FLINK-13832] Rename DefaultRollingPolicy create to builder
flinkbot commented on issue #9527: [FLINK-13832] Rename DefaultRollingPolicy create to builder URL: https://github.com/apache/flink/pull/9527#issuecomment-524532599 ## CI report: * 5e573e0756c77c14e481717ab35eb6df5b537abd : UNKNOWN This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[ https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated FLINK-13832: --- Labels: pull-request-available (was: ) > DefaultRollingPolicy create() method should be renamed to builder() > --- > > Key: FLINK-13832 > URL: https://issues.apache.org/jira/browse/FLINK-13832 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Affects Versions: 1.9.0 >Reporter: Gyula Fora >Assignee: Gyula Fora >Priority: Minor > Labels: pull-request-available > > The DefaultRollingPolicy.create() method returns an instance of > PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new > method named .builder() and deprecate "create". > Right now if we want to create a new instance with the default settings we > have to call: > DefaultRollingPolicy.create().build() > This nicely illustrates the problem with the naming of this method. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] gyfora opened a new pull request #9527: [FLINK-13832] Rename DefaultRollingPolicy create to builder
gyfora opened a new pull request #9527: [FLINK-13832] Rename DefaultRollingPolicy create to builder URL: https://github.com/apache/flink/pull/9527 ## What is the purpose of the change Deprecate and rename DefaultRollingPolicy create() method to builder() ## Brief change log - Renamed and deprecated old method - Replaced existing usage - Added test to validate deprecated method still works ## Verifying this change This change is a trivial rework / code cleanup with minimum test coverage for the deprecated method ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: yes - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? no - If yes, how is the feature documented? JavaDocs This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[ https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gyula Fora reassigned FLINK-13832: -- Assignee: Gyula Fora > DefaultRollingPolicy create() method should be renamed to builder() > --- > > Key: FLINK-13832 > URL: https://issues.apache.org/jira/browse/FLINK-13832 > Project: Flink > Issue Type: Improvement > Components: Connectors / FileSystem >Affects Versions: 1.9.0 >Reporter: Gyula Fora >Assignee: Gyula Fora >Priority: Minor > > The DefaultRollingPolicy.create() method returns an instance of > PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new > method named .builder() and deprecate "create". > Right now if we want to create a new instance with the default settings we > have to call: > DefaultRollingPolicy.create().build() > This nicely illustrates the problem with the naming of this method. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9
flinkbot edited a comment on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9 URL: https://github.com/apache/flink/pull/9526#issuecomment-524521158 Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community to review your pull request. We will use this comment to track the progress of the review. ## Automated Checks Last check on commit acf89a30196f5f62d6d28f5b498b357db62aae31 (Sat Aug 24 07:06:24 UTC 2019) **Warnings:** * No documentation files were touched! Remember to keep the Flink docs up to date! Mention the bot in a comment to re-run the automated checks. ## Review Progress * ❓ 1. The [description] looks good. * ❓ 2. There is [consensus] that the contribution should go into to Flink. * ❓ 3. Needs [attention] from. * ❓ 4. The change fits into the overall [architecture]. * ❓ 5. Overall code [quality] is good. Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process. The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required Bot commands The @flinkbot bot supports the following commands: - `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`) - `@flinkbot approve all` to approve all aspects - `@flinkbot approve-until architecture` to approve everything until `architecture` - `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention - `@flinkbot disapprove architecture` to remove an approval you gave earlier This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] yanghua commented on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9
yanghua commented on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9 URL: https://github.com/apache/flink/pull/9526#issuecomment-524527111 cc @tillrohrmann This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13832) DefaultRollingPolicy create() method should be renamed to builder()
[
https://issues.apache.org/jira/browse/FLINK-13832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914848#comment-16914848
]
vinoyang commented on FLINK-13832:
--
+1 to rename the {{create}} method.
> DefaultRollingPolicy create() method should be renamed to builder()
> ---
>
> Key: FLINK-13832
> URL: https://issues.apache.org/jira/browse/FLINK-13832
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / FileSystem
>Affects Versions: 1.9.0
>Reporter: Gyula Fora
>Priority: Minor
>
> The DefaultRollingPolicy.create() method returns an instance of
> PolicyBuilder not a DefaultRollingPolicy. Therefore we should add a new
> method named .builder() and deprecate "create".
> Right now if we want to create a new instance with the default settings we
> have to call:
> DefaultRollingPolicy.create().build()
> This nicely illustrates the problem with the naming of this method.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[GitHub] [flink] flinkbot edited a comment on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9
flinkbot edited a comment on issue #9526: [FLINK-13614][Tests] Add MigrationVersion.v1_9 URL: https://github.com/apache/flink/pull/9526#issuecomment-524521471 ## CI report: * acf89a30196f5f62d6d28f5b498b357db62aae31 : SUCCESS [Build](https://travis-ci.com/flink-ci/flink/builds/124451445) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (FLINK-13839) Support to set yarn node label for flink jobmanager and taskmanager container
[ https://issues.apache.org/jira/browse/FLINK-13839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914842#comment-16914842 ] vinoyang commented on FLINK-13839: -- Hi [~fly_in_gis] Did you mean FLINK-7836? It seems this feature has been supported. > Support to set yarn node label for flink jobmanager and taskmanager container > - > > Key: FLINK-13839 > URL: https://issues.apache.org/jira/browse/FLINK-13839 > Project: Flink > Issue Type: New Feature > Components: Deployment / YARN >Reporter: Yang Wang >Priority: Major > > Yarn node label feature is introduced from 2.6. It is a way to group nodes > with similar characteristics and applications can specify where to run. In > the production or cloud environment, we want to the jobmanager running on > some more stable machines. The node label could help us to achieve that. > > However, the ResourceRequest.setNodeLabelExpression have not been supported > in the current hadoop version dependency(2.4.1). So we need to bump the > hadoop version to 2.6.5. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (FLINK-7151) Add a function SQL DDL
[
https://issues.apache.org/jira/browse/FLINK-7151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914838#comment-16914838
]
Zhenqiu Huang commented on FLINK-7151:
--
[~phoenixjiangnan]Sure. Thanks for the suggestion. I will discuss and draft a
design doc with shuyi early next week.
> Add a function SQL DDL
> --
>
> Key: FLINK-7151
> URL: https://issues.apache.org/jira/browse/FLINK-7151
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / API
>Reporter: yuemeng
>Assignee: Shuyi Chen
>Priority: Critical
> Fix For: 1.10.0
>
>
> Based on create function and table.we can register a udf,udaf,udtf use sql:
> {code}
> CREATE FUNCTION [IF NOT EXISTS] [catalog_name.db_name.]function_name AS
> class_name;
> DROP FUNCTION [IF EXISTS] [catalog_name.db_name.]function_name;
> ALTER FUNCTION [IF EXISTS] [catalog_name.db_name.]function_name RENAME TO
> new_name;
> {code}
> {code}
> CREATE function 'TOPK' AS
> 'com..aggregate.udaf.distinctUdaf.topk.ITopKUDAF';
> INSERT INTO db_sink SELECT id, TOPK(price, 5, 'DESC') FROM kafka_source GROUP
> BY id;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
