Re: [Disuss]Permission of checkpoint directory

2017-03-20 Thread Greg Hogan
Prior discussion at https://github.com/apache/flink/pull/3335 > On Mar 19, 2017, at 11:34 PM, Wangtao (WangTao) wrote: > > Hi All, > > Checkpoint directory will store user data and it is better to keep it with > minimum

Re: [DISCUSS] Project build time and possible restructuring

2017-03-20 Thread Greg Hogan
t; I would actually suggest to do only the library split initially, to see > what the challenges are in setting up the multi-repo build and release > tooling. Once we gathered experience there, we can probably easily see what > else we can split out. > > Stephan > > > On Fr

[DISCUSS] TravisCI status on GitHub Page

2017-03-20 Thread Greg Hogan
We are now showing the TravisCI build status on Flink’s GitHub page. I think Robert’s comment in Jira may have gone unnoticed when the PR was committed. https://issues.apache.org/jira/browse/FLINK-6122 If not yet seeing the benefit even if

Re: [DISCUSS] Project build time and possible restructuring

2017-03-17 Thread Greg Hogan
;> much complexity and too many repositories. >>>>> "flink" and "flink-libraries" are hopefully enough to get the build >>> time >>>>> significantly down. >>>>> We can also consider putting the connectors into the >> &quo

Re: [DISCUSS] Could we Improve tests time and stability?

2017-03-17 Thread Greg Hogan
Dmytro, This is a good idea and a nice speedup, though I notice that nearly half of the speedup (1104s of 2461s) is from job 7 which appears to have hung and timed out in the initial run. Could you test the two changes in isolation (increased maximum memory and garbage collector)? If the

Re: Bumping API stability check version

2017-03-16 Thread Greg Hogan
nModifications". Does > it fail the build even if somebody did a change that is non API breaking on > a @Public class? > > On Thu, Mar 16, 2017 at 3:37 PM, Greg Hogan <c...@greghogan.com> wrote: > >> Hi, >> >> I see in the parent pom.xml that 1.3-SNAP

Bumping API stability check version

2017-03-16 Thread Greg Hogan
Hi, I see in the parent pom.xml that 1.3-SNAPSHOT is checking for API stability against 1.1.4. Also, that this version was only bumped with FLINK-5617 late in the 1.2 development cycle. Should we bump this version as part of the release process, i.e. on the 1.2.0 release updating 1.3-SNAPSHOT

Re: [DISCUSS] Project build time and possible restructuring

2017-03-15 Thread Greg Hogan
we have library repository >>> depend >>>> on >>>>> snapshot Flink versions, we need to make sure that the snapshot >>>> deployment >>>>> always works. This also means that people working on a library >>> repository >>>&g

[jira] [Created] (FLINK-6009) Deprecate DataSetUtils#checksumHashCode

2017-03-09 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-6009: - Summary: Deprecate DataSetUtils#checksumHashCode Key: FLINK-6009 URL: https://issues.apache.org/jira/browse/FLINK-6009 Project: Flink Issue Type: Improvement

Re: FLINK-5734 : Code Generation for NormalizedKeySorter

2017-03-08 Thread Greg Hogan
Hi Pat, I’m still trying to understand the implications of Java’s Class Hierarchy Analysis [0]. Flink currently uses only a single implementation of InMemorySorter, which is NormalizedKeySorter. FLINK-4705 adds support for FixedLengthRecordSorter for Flink’s Value types and Tuples. This

[jira] [Created] (FLINK-5945) Close function in OuterJoinOperatorBase#executeOnCollections

2017-03-01 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5945: - Summary: Close function in OuterJoinOperatorBase#executeOnCollections Key: FLINK-5945 URL: https://issues.apache.org/jira/browse/FLINK-5945 Project: Flink Issue

Re: [DISCUSS] Gelly planning for release 1.3 and roadmap

2017-03-01 Thread Greg Hogan
sary), rather than high-level things (e.g. > algorithms, performance) on top of it. What if we can change both the > edges' values and vertices' values during an iteration one day? :) > > Best, > Xingcan > > > On Sat, Feb 25, 2017 at 2:43 AM, Vasiliki Kalavri <vasilikikal

Re: [DISCUSS] Gelly planning for release 1.3 and roadmap

2017-03-01 Thread Greg Hogan
On Fri, Feb 24, 2017 at 1:43 PM, Vasiliki Kalavri <vasilikikala...@gmail.com <mailto:vasilikikala...@gmail.com>> wrote: Hi Greg, On 24 February 2017 at 18:09, Greg Hogan <c...@greghogan.com <mailto:c...@greghogan.com>> wrote: > Thanks, Vasia, for starting the disc

Re: [DISCUSS] Code style / checkstyle

2017-02-27 Thread Greg Hogan
ve to go manually through >>> all >>>>> past commits until you find the commit which changed a given line >>> before >>>>> the reformatting. >>>>> >>>>> Cheers, >>>>> Till >>>>> >>>>> On Sun,

[jira] [Created] (FLINK-5913) Example drivers

2017-02-24 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5913: - Summary: Example drivers Key: FLINK-5913 URL: https://issues.apache.org/jira/browse/FLINK-5913 Project: Flink Issue Type: Sub-task Components: Gelly

[jira] [Created] (FLINK-5912) Inputs for CSV and graph generators

2017-02-24 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5912: - Summary: Inputs for CSV and graph generators Key: FLINK-5912 URL: https://issues.apache.org/jira/browse/FLINK-5912 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-5911) Command-line parameters

2017-02-24 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5911: - Summary: Command-line parameters Key: FLINK-5911 URL: https://issues.apache.org/jira/browse/FLINK-5911 Project: Flink Issue Type: Sub-task Components

[jira] [Created] (FLINK-5910) Framework for Gelly examples

2017-02-24 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5910: - Summary: Framework for Gelly examples Key: FLINK-5910 URL: https://issues.apache.org/jira/browse/FLINK-5910 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-5909) Interface for GraphAlgorithm results

2017-02-24 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5909: - Summary: Interface for GraphAlgorithm results Key: FLINK-5909 URL: https://issues.apache.org/jira/browse/FLINK-5909 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Gelly planning for release 1.3 and roadmap

2017-02-24 Thread Greg Hogan
Thanks, Vasia, for starting the discussion. I was expecting more changes from the recent discussion on restructuring the project, in particular regarding the libraries. Gelly has always collected algorithms and I have personally taken an algorithms-first approach for contributions. Is that

Re: [DISCUSS] Code style / checkstyle

2017-02-24 Thread Greg Hogan
I agree wholeheartedly with Ufuk. We cannot reformat the codebase, cannot pause while flushing the PR queue, and won't find a consensus code style. I think we can create a baseline code style for new and existing contributors for which reformatting on changed files will be acceptable for PR

Re: Visualizing topologies

2017-02-24 Thread Greg Hogan
Ken and Fabian, Is the use case to generate and act on the dot file from within the user program? Would it be more maintainable to make the plan JSON more accessible (through the CLI and web interface) which users could then pipe through a converter script? Greg On Fri, Feb 24, 2017 at 4:55 AM,

Re: [DISCUSS] Project build time and possible restructuring

2017-02-22 Thread Greg Hogan
An additional option for reducing time to build and test is parallel execution. This would help users more than on TravisCI since we're generally running on multi-core machines rather than VM slices. Is the idea that each user would only check out the modules that he or she is developing with?

[jira] [Created] (FLINK-5891) ConnectedComponents is broken when object reuse enabled

2017-02-22 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5891: - Summary: ConnectedComponents is broken when object reuse enabled Key: FLINK-5891 URL: https://issues.apache.org/jira/browse/FLINK-5891 Project: Flink Issue Type

[jira] [Created] (FLINK-5890) GatherSumApply broken when object reuse enabled

2017-02-22 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5890: - Summary: GatherSumApply broken when object reuse enabled Key: FLINK-5890 URL: https://issues.apache.org/jira/browse/FLINK-5890 Project: Flink Issue Type: Bug

Re: [DISCUSS] Code style / checkstyle

2017-02-22 Thread Greg Hogan
Will not the code style be applied on save to any user-modified file? So this will clutter PRs and overwrite history. On Wed, Feb 22, 2017 at 6:19 AM, Dawid Wysakowicz < wysakowicz.da...@gmail.com> wrote: > I also agree with Till and Chesnayl. Anyway as to "capture the current > style" I have

Re: KeyGroupRangeAssignment ?

2017-02-21 Thread Greg Hogan
Integer's hashCode is the identity function. Store your slot index in an Integer or IntValue and key off that field. On Tue, Feb 21, 2017 at 6:04 AM, Ovidiu-Cristian MARCU < ovidiu-cristian.ma...@inria.fr> wrote: > Hi, > > As in my example, each key is a window so I want to evenly distributed >

[jira] [Created] (FLINK-5838) Fix jobmanager.sh usage

2017-02-17 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5838: - Summary: Fix jobmanager.sh usage Key: FLINK-5838 URL: https://issues.apache.org/jira/browse/FLINK-5838 Project: Flink Issue Type: Bug Components

Re: FLINK-5734 : Code Generation for NormalizedKeySorter

2017-02-14 Thread Greg Hogan
Pat, Thanks for adding the new test results. This idea for this implementation was Gábor's from the FLINK-3722 description. Since you will be filing a FLIP I recommend including these benchmarks for consideration and discussion on the mailing list. In part because the PR is 4 months old and need

Re: [ANNOUNCE] Welcome Stefan Richter as a new committer

2017-02-10 Thread Greg Hogan
Welcome, Stefan, and thank you for your contributions! On Fri, Feb 10, 2017 at 5:00 AM, Ufuk Celebi wrote: > Hey everyone, > > I'm very happy to announce that the Flink PMC has accepted Stefan > Richter to become a committer of the Apache Flink project. > > Stefan is part of

Re: FLINK-5734 : Code Generation for NormalizedKeySorter

2017-02-08 Thread Greg Hogan
Hi Pat, Serkan, and Gábor, This looks very nice. I'll treat this like a pre-FLIP and ask my question here. Do I understand correctly that the generated code is only dependent on the length of the sort key? So we could separate the writing and reading of keys and records and from the generated

Re: [ANNOUNCE] Welcome Jark Wu and Kostas Kloudas as committers

2017-02-07 Thread Greg Hogan
Welcome Jark and Kostas! Thank you for your contributions and many more to come. On Tue, Feb 7, 2017 at 3:16 PM, Fabian Hueske wrote: > Hi everybody, > > I'm very happy to announce that Jark Wu and Kostas Kloudas accepted the > invitation of the Flink PMC to become committers

Re: [Discuss] Organizing Documentation for Configuration Options

2017-02-07 Thread Greg Hogan
mating the network buffer configuration in order to > get rid of any manual tuning for most users (because of the issues you > described + streaming and batch jobs require different tuning, which > complicates things even more). > > – Ufuk > > On 6 February 2017 at 19:21:28,

[jira] [Created] (FLINK-5733) Link to Bahir connectors

2017-02-07 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5733: - Summary: Link to Bahir connectors Key: FLINK-5733 URL: https://issues.apache.org/jira/browse/FLINK-5733 Project: Flink Issue Type: Improvement

[Discuss] Organizing Documentation for Configuration Options

2017-02-06 Thread Greg Hogan
Hi devs, Flink's Configuration page [1] has grown intimidatingly long and complex. Options are described across three main sections: common options (single section), advanced options (multiple sections), and full reference. The trailing "background" section further describes the most impactful

[jira] [Created] (FLINK-5680) Document env.ssh.opts

2017-01-27 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5680: - Summary: Document env.ssh.opts Key: FLINK-5680 URL: https://issues.apache.org/jira/browse/FLINK-5680 Project: Flink Issue Type: Improvement Components

Re: [DISCUSS] (Not) tagging reviewers

2017-01-27 Thread Greg Hogan
> I took a quick skim on the PRs and I noticed that only a few of them are actually in mergeable shapes (i.e., properly rebased and passing CI). Although TravisCI is quite unstable, Flink executes multiple tests with different configurations so you'll want to instead look at which tests are

[jira] [Created] (FLINK-5562) Driver fixes

2017-01-18 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5562: - Summary: Driver fixes Key: FLINK-5562 URL: https://issues.apache.org/jira/browse/FLINK-5562 Project: Flink Issue Type: Bug Components: Gelly Affects

[jira] [Created] (FLINK-5563) Add density to vertex metrics

2017-01-18 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5563: - Summary: Add density to vertex metrics Key: FLINK-5563 URL: https://issues.apache.org/jira/browse/FLINK-5563 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-5558) Replace TriangleCount with a Count analytic

2017-01-18 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5558: - Summary: Replace TriangleCount with a Count analytic Key: FLINK-5558 URL: https://issues.apache.org/jira/browse/FLINK-5558 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-5557) Fix link in library methods

2017-01-18 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5557: - Summary: Fix link in library methods Key: FLINK-5557 URL: https://issues.apache.org/jira/browse/FLINK-5557 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Time-based releases in Flink

2017-01-18 Thread Greg Hogan
I'm +0 on switching to a pre-determined schedule. It may be that the Flink codebase has reached a level of maturity allowing for a time-based release schedule, and I'm hopeful that a known schedule will improve communication about and expectations for new features. I'd like to hear a

[jira] [Created] (FLINK-5485) Mark compiled web frontend files as binary when processed by git diff

2017-01-13 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5485: - Summary: Mark compiled web frontend files as binary when processed by git diff Key: FLINK-5485 URL: https://issues.apache.org/jira/browse/FLINK-5485 Project: Flink

[jira] [Created] (FLINK-5461) Remove Superflous TypeInformation Declaration

2017-01-11 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5461: - Summary: Remove Superflous TypeInformation Declaration Key: FLINK-5461 URL: https://issues.apache.org/jira/browse/FLINK-5461 Project: Flink Issue Type

Re: [DISCUSS] Merging the FLIP-6 feature branch into the Master branch

2016-12-02 Thread Greg Hogan
Hi Stephan, How soon are you expecting the "release-1.2" fork? I am sure you have considered merging the FLIP-6 branch after the fork. Do we anticipate the new tests pushing Flink over Travis CI's new 50 minute limit? This might be a good opportunity to rebalance the test ranges as the most

[jira] [Created] (FLINK-5209) Fix TaskManager metrics

2016-11-29 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-5209: - Summary: Fix TaskManager metrics Key: FLINK-5209 URL: https://issues.apache.org/jira/browse/FLINK-5209 Project: Flink Issue Type: Bug Components

[DISCUSS] @Public libraries

2016-11-22 Thread Greg Hogan
Hi all, Should stable APIs in Flink's CEP, ML, and Gelly libraries be annotated @Public or restricted to use of @PublicEvolving? We would ensure that library APIs do not add restrictions to the core APIs. Libraries could use @PublicEvolving or @Internal core APIs within @Public or

Travis CI

2016-11-10 Thread Greg Hogan
We're getting the dreaded "The job exceeded the maximum time limit for jobs, and has been terminated." error for some recent Travis-CI builds. https://travis-ci.org/apache/flink/builds/174615801 The docs state that termination will occur when "A job takes longer than 50 minutes on

[jira] [Created] (FLINK-4996) Make CrossHint @Public

2016-11-02 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4996: - Summary: Make CrossHint @Public Key: FLINK-4996 URL: https://issues.apache.org/jira/browse/FLINK-4996 Project: Flink Issue Type: Improvement Components

[jira] [Created] (FLINK-4970) Parameterize vertex value for SSSP

2016-10-31 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4970: - Summary: Parameterize vertex value for SSSP Key: FLINK-4970 URL: https://issues.apache.org/jira/browse/FLINK-4970 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-4966) BetweennessCentrality

2016-10-30 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4966: - Summary: BetweennessCentrality Key: FLINK-4966 URL: https://issues.apache.org/jira/browse/FLINK-4966 Project: Flink Issue Type: New Feature Components

[jira] [Created] (FLINK-4965) AllPairsShortestPaths

2016-10-30 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4965: - Summary: AllPairsShortestPaths Key: FLINK-4965 URL: https://issues.apache.org/jira/browse/FLINK-4965 Project: Flink Issue Type: New Feature Components

[jira] [Created] (FLINK-4963) Tabulate edge direction for directed VertexMetrics

2016-10-28 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4963: - Summary: Tabulate edge direction for directed VertexMetrics Key: FLINK-4963 URL: https://issues.apache.org/jira/browse/FLINK-4963 Project: Flink Issue Type

[jira] [Created] (FLINK-4946) Load jar files from subdirectories of lib

2016-10-27 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4946: - Summary: Load jar files from subdirectories of lib Key: FLINK-4946 URL: https://issues.apache.org/jira/browse/FLINK-4946 Project: Flink Issue Type: Improvement

Contributing to flink-web

2016-10-27 Thread Greg Hogan
Should we align the process for contributing to apache/flink-web to mirror that for apache/flink? Flink's JIRA has an existing component for "Project Website". Commits to flink-web are sent to the commits mailing list. Does Jira require further integeration?

[jira] [Created] (FLINK-4936) Operator names for Gelly inputs

2016-10-26 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4936: - Summary: Operator names for Gelly inputs Key: FLINK-4936 URL: https://issues.apache.org/jira/browse/FLINK-4936 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-4934) Triadic Census

2016-10-26 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4934: - Summary: Triadic Census Key: FLINK-4934 URL: https://issues.apache.org/jira/browse/FLINK-4934 Project: Flink Issue Type: New Feature Components: Gelly

[jira] [Created] (FLINK-4896) PageRank algorithm for directed graphs

2016-10-24 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4896: - Summary: PageRank algorithm for directed graphs Key: FLINK-4896 URL: https://issues.apache.org/jira/browse/FLINK-4896 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-4869) Store record pointer after record keys

2016-10-20 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4869: - Summary: Store record pointer after record keys Key: FLINK-4869 URL: https://issues.apache.org/jira/browse/FLINK-4869 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-4861) Package optional project artifacts

2016-10-19 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4861: - Summary: Package optional project artifacts Key: FLINK-4861 URL: https://issues.apache.org/jira/browse/FLINK-4861 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-4860) Sort performance

2016-10-19 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4860: - Summary: Sort performance Key: FLINK-4860 URL: https://issues.apache.org/jira/browse/FLINK-4860 Project: Flink Issue Type: Improvement Reporter: Greg

Re: Removing flink-contrib/flink-operator-stats

2016-10-19 Thread Greg Hogan
Based on a cursory reading of FLINK-1297 I would lean toward dropping the code rather than moving to Apache Bahir. This looks to only be appropriate for batch and this module was not integrated into the runtime. If there is a way forward to make use this code in core Flink then that would be even

Re: [DISCUSS] Deprecate Hadoop source method from (batch) ExecutionEnvironment

2016-10-14 Thread Greg Hogan
+1 On Fri, Oct 14, 2016 at 5:29 AM, Fabian Hueske wrote: > Hi everybody, > > I would like to propose to deprecate the utility methods to read data with > Hadoop InputFormats from the (batch) ExecutionEnvironment. > > The motivation for deprecating these methods is reduce

Re: [DISCUSS] Drop Hadoop 1 support with Flink 1.2

2016-10-13 Thread Greg Hogan
Hi Robert, What are the benefits to Flink for dropping Hadoop 1 support? Is there significant code cleanup or would we simply be publishing one less set of artifacts? Greg On Thu, Oct 13, 2016 at 10:47 AM, Robert Metzger wrote: > Hi, > > The Apache Hadoop community has

Re: [DISCUSS] Support Suspending and Resuming of Flink Jobs

2016-10-12 Thread Greg Hogan
Sorry, I haven't followed this development, but roughly how much more costly is the new serialization for savepoints? On Wed, Oct 12, 2016 at 5:51 AM, SHI Xiaogang wrote: > Hi all, > > Currently, savepoints are exactly the completed checkpoints, and Flink > provides

Re: [Discuss] Upgrade JUnit to 4.12

2016-10-05 Thread Greg Hogan
Tests are passing with one additional change to an inner test class visibility. The ticket is FLINK-4740. On Wed, Oct 5, 2016 at 3:52 AM, Till Rohrmann <trohrm...@apache.org> wrote: > +1 for that :-) > > On Tue, Oct 4, 2016 at 10:11 PM, Greg Hogan <c...@greghogan.com> w

[jira] [Created] (FLINK-4740) Upgrade testing libraries

2016-10-04 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4740: - Summary: Upgrade testing libraries Key: FLINK-4740 URL: https://issues.apache.org/jira/browse/FLINK-4740 Project: Flink Issue Type: Improvement

Re: [Discuss] Upgrade JUnit to 4.12

2016-10-04 Thread Greg Hogan
test 1.10.19. Since the changes are more than a single version I'll create a ticket and PR so the test results can be discussed. Greg On Tue, Oct 4, 2016 at 3:19 PM, Stephan Ewen <se...@apache.org> wrote: > From my side +1, unless there are known issues with JUnit 4.12 > > On Tue, Oc

[Discuss] Upgrade JUnit to 4.12

2016-10-04 Thread Greg Hogan
JUnit 4.12 was released 4 Dec 2014. Flink is currently using JUnit 4.11 from 14 Nov 2012. https://github.com/junit-team/junit4/releases My use case is the support for assert equals on boolean arrays, but in general this looks to be an innocuous change and I could not find any prior discussion.

[jira] [Created] (FLINK-4736) Don't duplicate fields in Ordering

2016-10-04 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4736: - Summary: Don't duplicate fields in Ordering Key: FLINK-4736 URL: https://issues.apache.org/jira/browse/FLINK-4736 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-4734) Remove use of Tuple setField for fixed position

2016-10-04 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4734: - Summary: Remove use of Tuple setField for fixed position Key: FLINK-4734 URL: https://issues.apache.org/jira/browse/FLINK-4734 Project: Flink Issue Type

[jira] [Created] (FLINK-4729) Use optional VertexCentric CombineFunction

2016-10-03 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4729: - Summary: Use optional VertexCentric CombineFunction Key: FLINK-4729 URL: https://issues.apache.org/jira/browse/FLINK-4729 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-4728) Replace reference equality with object equality

2016-10-03 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4728: - Summary: Replace reference equality with object equality Key: FLINK-4728 URL: https://issues.apache.org/jira/browse/FLINK-4728 Project: Flink Issue Type

Duplicate sort keys

2016-10-03 Thread Greg Hogan
Is it correct to expect that Flink should remove duplicate sort keys? I'm working on instrumenting the FixedLengthRecordSorter (FLINK-4705) and the following test case from TypeHintITCase:200 is having an unexpected effect due to the keyPositions = {0, 0} being passed to TupleComparator. DataSet

[jira] [Created] (FLINK-4705) Instrument FixedLengthRecordSorter

2016-09-28 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4705: - Summary: Instrument FixedLengthRecordSorter Key: FLINK-4705 URL: https://issues.apache.org/jira/browse/FLINK-4705 Project: Flink Issue Type: Improvement

Re: 答复: [DISCUSS] add netty tcp/restful pushed source support

2016-09-27 Thread Greg Hogan
Apache Bahir's website only suggests support for additional frameworks, but there is a Flink repository at https://github.com/apache/bahir-flink On Tue, Sep 27, 2016 at 8:38 AM, shijinkui wrote: > Hey, Stephan Ewen > > 1. bahir's target is spark. The contributer are

[jira] [Created] (FLINK-4673) TypeFactory for Either type

2016-09-23 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4673: - Summary: TypeFactory for Either type Key: FLINK-4673 URL: https://issues.apache.org/jira/browse/FLINK-4673 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-4664) Add translator to NullValue

2016-09-22 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4664: - Summary: Add translator to NullValue Key: FLINK-4664 URL: https://issues.apache.org/jira/browse/FLINK-4664 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-4643) Average Clustering Coefficient

2016-09-20 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4643: - Summary: Average Clustering Coefficient Key: FLINK-4643 URL: https://issues.apache.org/jira/browse/FLINK-4643 Project: Flink Issue Type: New Feature

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
: > Thanks Greg."Your setting of 4096 is only 128 MiB."...Correct. Cz I > followed that formula :-)))I can bump it up to twice as much like what the > example is doing to for instance 300 MiB.Is this reasonable? what do you > suggest as a reasonable range?Thanks Greg > >

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
y, a metric travels from TaskManager -> WebInterface -> User. > FLINK-4389 was about the first arrow, which is a prerequisite step for the > second one. > > Regards, > Chesnay > > > On 19.09.2016 21:35, Greg Hogan wrote: > >> The nightly snapshots now inclu

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
ion: Configuration > > > | > | > | > | || > > | > > | > | > | | > Apache Flink 1.2-SNAPSHOT Documentation: Configuration >| | > > | > > | > > > > 4096 = (16x16)x4x4 where 16 is number of tasks per TM, 4 is # of TMs & 4 > is there in t

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
Hi Amir, You may see improved performance setting "taskmanager.memory.preallocate: true" in order to use off-heap memory. Also, your number of buffers looks quite low and you may want to increase "taskmanager.network.numberOfBuffers". Your setting of 4096 is only 128 MiB. As this is a only

[jira] [Created] (FLINK-4594) Validate lower bound in MathUtils.checkedDownCast

2016-09-07 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4594: - Summary: Validate lower bound in MathUtils.checkedDownCast Key: FLINK-4594 URL: https://issues.apache.org/jira/browse/FLINK-4594 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4583) NullPointerException in CliFrontend

2016-09-06 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4583: - Summary: NullPointerException in CliFrontend Key: FLINK-4583 URL: https://issues.apache.org/jira/browse/FLINK-4583 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4572) Convert to negative in LongValueToIntValue

2016-09-02 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4572: - Summary: Convert to negative in LongValueToIntValue Key: FLINK-4572 URL: https://issues.apache.org/jira/browse/FLINK-4572 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4571) Configurable little parallelism in Gelly drivers

2016-09-02 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4571: - Summary: Configurable little parallelism in Gelly drivers Key: FLINK-4571 URL: https://issues.apache.org/jira/browse/FLINK-4571 Project: Flink Issue Type

Re: why job submit timeout is 21474835 second

2016-08-29 Thread Greg Hogan
Could be rewritten as "val INFO_TIMEOUT = Integer.MAX_VALUE seconds"? On Mon, Aug 29, 2016 at 4:22 AM, 时金魁 wrote: > > > AkkaUtils.scala > val INF_TIMEOUT = 21474835 seconds > > > That is job submit timeout 248.55 days. > > > Why is this number? > > > >

Re: Additional project downloads

2016-08-25 Thread Greg Hogan
gt; > Maybe we should put a link to maven central. We could parameterize the > > link > > > so that it always links to the current release linked on our downloads > > > page. > > > > > > On Wed, Aug 24, 2016 at 5:04 PM, Greg Hogan <c...@gre

[jira] [Created] (FLINK-4481) Maximum results for pairwise algorithms

2016-08-24 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4481: - Summary: Maximum results for pairwise algorithms Key: FLINK-4481 URL: https://issues.apache.org/jira/browse/FLINK-4481 Project: Flink Issue Type: New Feature

Additional project downloads

2016-08-24 Thread Greg Hogan
Hi, Should Flink add-ons such as CEP, Gelly, ML, and the optional Metrics Reporters be available from the download page? Is the alternative to direct users to Maven Central? Greg

[jira] [Created] (FLINK-4452) TaskManager network buffer guages

2016-08-23 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4452: - Summary: TaskManager network buffer guages Key: FLINK-4452 URL: https://issues.apache.org/jira/browse/FLINK-4452 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-4447) Include NettyConfig options on Configurations page

2016-08-22 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4447: - Summary: Include NettyConfig options on Configurations page Key: FLINK-4447 URL: https://issues.apache.org/jira/browse/FLINK-4447 Project: Flink Issue Type

Re: [DISCUSS] Python API for Fllink libraries

2016-08-22 Thread Greg Hogan
Hi Ivan, My expectation would be that programs written for the Python API would be much slower than when implementing with Java or Scala. A performance comparison would be quite interesting. Gelly has both iterative and non-iterative algorithms. Greg On Sat, Aug 20, 2016 at 7:11 PM, Ivan

Re: [DISCUSS] API breaking change in DataStream Windows

2016-08-09 Thread Greg Hogan
I agree that expecting users to cast is undesirable. Upon changing the API, why would we not mark the next release as 2.0? The same issue arose with Gabor's addition of hash-combine in the Scala DataSet API where DataSet was returned rather than a specialized Operator. The solution was to add an

[jira] [Created] (FLINK-4265) Add a NoOpOperator

2016-07-25 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4265: - Summary: Add a NoOpOperator Key: FLINK-4265 URL: https://issues.apache.org/jira/browse/FLINK-4265 Project: Flink Issue Type: New Feature Components

[jira] [Created] (FLINK-4264) New GraphMetrics driver

2016-07-25 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4264: - Summary: New GraphMetrics driver Key: FLINK-4264 URL: https://issues.apache.org/jira/browse/FLINK-4264 Project: Flink Issue Type: New Feature Components

[jira] [Created] (FLINK-4257) Handle delegating algorithm change of class

2016-07-22 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4257: - Summary: Handle delegating algorithm change of class Key: FLINK-4257 URL: https://issues.apache.org/jira/browse/FLINK-4257 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-4231) Switch DistinctOperator from GroupReduceFunction to ReduceFunction

2016-07-18 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4231: - Summary: Switch DistinctOperator from GroupReduceFunction to ReduceFunction Key: FLINK-4231 URL: https://issues.apache.org/jira/browse/FLINK-4231 Project: Flink

[jira] [Created] (FLINK-4219) Quote PDSH opts in start-cluster.sh

2016-07-14 Thread Greg Hogan (JIRA)
Greg Hogan created FLINK-4219: - Summary: Quote PDSH opts in start-cluster.sh Key: FLINK-4219 URL: https://issues.apache.org/jira/browse/FLINK-4219 Project: Flink Issue Type: Bug

<    1   2   3   4   >