[jira] [Created] (FLINK-8218) move flink-connector-kinesis examples from /src to /test
Bowen Li created FLINK-8218: --- Summary: move flink-connector-kinesis examples from /src to /test Key: FLINK-8218 URL: https://issues.apache.org/jira/browse/FLINK-8218 Project: Flink Issue Type: Improvement Components: Kinesis Connector Affects Versions: 1.5.0 Reporter: Bowen Li Assignee: Bowen Li Priority: Minor Fix For: 1.5.0 I just saw there are runnable examples in src tree and traced back to this PR. I feel it's really bad to have runnable examples (with java main method) in a lib jar that Flink distributes. chatted with [~tzulitai] , we agreed that we should move those examples from /src to /test -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8217) Properly annotate APIs of flink-connector-kinesis
Bowen Li created FLINK-8217: --- Summary: Properly annotate APIs of flink-connector-kinesis Key: FLINK-8217 URL: https://issues.apache.org/jira/browse/FLINK-8217 Project: Flink Issue Type: Sub-task Components: Kinesis Connector Affects Versions: 1.5.0 Reporter: Bowen Li Assignee: Bowen Li Fix For: 1.5.0 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8216) Unify test utils in flink-connector-kinesis
Bowen Li created FLINK-8216: --- Summary: Unify test utils in flink-connector-kinesis Key: FLINK-8216 URL: https://issues.apache.org/jira/browse/FLINK-8216 Project: Flink Issue Type: Improvement Components: Kinesis Connector Affects Versions: 1.5.0 Reporter: Bowen Li Assignee: Bowen Li Priority: Minor Fix For: 1.5.0 currently there are a few ways to get a Properties object with required fields (aws access key, aws secret key, aws region) for KinesisConsumer and KinesisProducer in unit tests. We should unify them and provide a single util to get that Properties object -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Performance of docker-flink
Hi, Just copying a question I had posted to user mailing list as well: I wanted to explore docker-flink (using Ceph for state backend). before opting for a standalone cluster. Has there been any comparative studies on the performance of docker-flink? Would the states be consistent and performant if the docker containers go down and respawn frequently? Also, we want real time processing in some cases; would using docker hamper the performance?
[jira] [Created] (FLINK-8214) Do not use mockito for collecting results in StreamMockEnvironment
Piotr Nowojski created FLINK-8214: - Summary: Do not use mockito for collecting results in StreamMockEnvironment Key: FLINK-8214 URL: https://issues.apache.org/jira/browse/FLINK-8214 Project: Flink Issue Type: Sub-task Reporter: Piotr Nowojski Assignee: Piotr Nowojski -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8213) Prevent potential metric-related ClassCastExceptions
Chesnay Schepler created FLINK-8213:
---
Summary: Prevent potential metric-related ClassCastExceptions
Key: FLINK-8213
URL: https://issues.apache.org/jira/browse/FLINK-8213
Project: Flink
Issue Type: Bug
Components: Metrics, Streaming
Affects Versions: 1.5.0
Reporter: Chesnay Schepler
Assignee: Chesnay Schepler
Priority: Minor
There are a few places, in particular revolving around the {{StreamOperator}},
that may run into ClassCastExceptions.
For example, in the {{DataSinkTask}} you can find this line:
{{final Counter numRecordsIn = ((OperatorMetricGroup)
ctx.getMetricGroup()).getIOMetricGroup().getNumRecordsInCounter();}}
The cast here is necessary since the {{AbstractStreamOperator}} exposes the
{{OperatorMetricGroup}} as a plain {{MetricGroup}} since it is an internal
class. However there is obviously no guarantee that
{{StreamOperator#getMetricGroup()}} returns an actual {{OperatorMetricGroup}}.
In fact, a fall-back case in {{AbstractStreamOperator#setup}} causes exactly
this assumption to be violated.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[VOTE] Release 1.4.0, release candidate #3
Hi everyone, Please review and vote on release candidate #3 for the version 1.4.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], * the official Apache source release and binary convenience releases to be deployed to dist.apache.org[2], which are signed with the key with fingerprint F2A67A8047499BBB3908D17AA8F4FD97121D7293 [3], * all artifacts to be deployed to the Maven Central Repository [4], * source code tag "release-1.4.0-rc1" [5], * website pull request listing the new release [6]. Please have a careful look at the website PR because I changed some wording and we're now also releasing a binary without Hadoop dependencies. Please use this document for coordinating testing efforts: [7] The only change between RC1 and this RC2 is that the source release package does not include the erroneously included binary Ruby dependencies of the documentation anymore. Because of this I would like to propose a shorter voting time and close the vote around the time that RC1 would have closed. This would mean closing by end of Wednesday. Please let me know if you disagree with this. The vote is adopted by majority approval, with at least 3 PMC affirmative votes. Thanks, Your friendly Release Manager [1] https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12340533 [2] http://people.apache.org/~aljoscha/flink-1.4.0-rc3/ [3] https://dist.apache.org/repos/dist/release/flink/KEYS [4] https://repository.apache.org/content/repositories/orgapacheflink-1141 [5] https://git-wip-us.apache.org/repos/asf?p=flink.git;a=tag;h=8fb9635dd2e64dbb20887c84f646f02034b57cb1 [6] https://github.com/apache/flink-web/pull/95 [7] https://docs.google.com/document/d/1cOkycJwEKVjG_onnpl3bQNTq7uebh48zDtIJxceyU2E/edit?usp=sharing Pro-tip: you can create a settings.xml file with these contents: flink-1.4.0 flink-1.4.0 flink-1.4.0 https://repository.apache.org/content/repositories/orgapacheflink-1141/ archetype https://repository.apache.org/content/repositories/orgapacheflink-1141/ And reference that in you maven commands via --settings path/to/settings.xml. This is useful for creating a quickstart based on the staged release and for building against the staged jars.
[jira] [Created] (FLINK-8212) Pull EnvironmentInformation out of TaskManagerServices
Till Rohrmann created FLINK-8212:
Summary: Pull EnvironmentInformation out of TaskManagerServices
Key: FLINK-8212
URL: https://issues.apache.org/jira/browse/FLINK-8212
Project: Flink
Issue Type: Improvement
Components: Local Runtime
Affects Versions: 1.5.0
Reporter: Till Rohrmann
Fix For: 1.5.0
We should pull the {{EnvironmentInformation}} out of the
{{TaskManagerServices}} where it is used to get access to the memory settings
of the executing JVM. This unnecessarily couples the former with the latter and
makes testing extremely hard (one has to use {{PowerMockRunner}} and mock the
static {{EnvironmentInformation}}).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (FLINK-8211) Rename ResultPartitionWriter to OutputGate
Piotr Nowojski created FLINK-8211: - Summary: Rename ResultPartitionWriter to OutputGate Key: FLINK-8211 URL: https://issues.apache.org/jira/browse/FLINK-8211 Project: Flink Issue Type: Improvement Components: Network Reporter: Piotr Nowojski Assignee: Piotr Nowojski Fix For: 1.5.0 This will unify the naming convention with InputGate. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8210) Do not use mockito for collecting results in MockEnvironment
Piotr Nowojski created FLINK-8210: - Summary: Do not use mockito for collecting results in MockEnvironment Key: FLINK-8210 URL: https://issues.apache.org/jira/browse/FLINK-8210 Project: Flink Issue Type: Sub-task Reporter: Piotr Nowojski Assignee: Piotr Nowojski -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8209) Do not relay on specific method names in LocalBufferPoolTest
Piotr Nowojski created FLINK-8209: - Summary: Do not relay on specific method names in LocalBufferPoolTest Key: FLINK-8209 URL: https://issues.apache.org/jira/browse/FLINK-8209 Project: Flink Issue Type: Sub-task Reporter: Piotr Nowojski Assignee: Piotr Nowojski -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8208) Reduce mockito usage in RecordWriterTest
Piotr Nowojski created FLINK-8208: - Summary: Reduce mockito usage in RecordWriterTest Key: FLINK-8208 URL: https://issues.apache.org/jira/browse/FLINK-8208 Project: Flink Issue Type: Sub-task Reporter: Piotr Nowojski Assignee: Piotr Nowojski -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8206) Clean up and refactor RecordWriter Tests
Piotr Nowojski created FLINK-8206: - Summary: Clean up and refactor RecordWriter Tests Key: FLINK-8206 URL: https://issues.apache.org/jira/browse/FLINK-8206 Project: Flink Issue Type: Improvement Components: Network Reporter: Piotr Nowojski Assignee: Piotr Nowojski Fix For: 1.5.0 This is part of an effort to make the tests less implementations details specific and easier to maintain in the future. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (FLINK-8205) Multi key get
Martin Eden created FLINK-8205: -- Summary: Multi key get Key: FLINK-8205 URL: https://issues.apache.org/jira/browse/FLINK-8205 Project: Flink Issue Type: New Feature Components: Queryable State Affects Versions: 1.4.0 Environment: Any Reporter: Martin Eden Fix For: 1.4.0 Currently the Java queryable state api only allows for fetching one key at a time. It would be extremely useful and more efficient if a similar call exists for submitting multiple keys. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
