[GitHub] flink pull request: [FLINK-1944][gelly] Added GSA-PageRank example
GitHub user andralungu opened a pull request: https://github.com/apache/flink/pull/626 [FLINK-1944][gelly] Added GSA-PageRank example This PR also adds the getSuperStepNumber() method to the Gather, Sum and Apply Functions. You can merge this pull request into a Git repository by running: $ git pull https://github.com/andralungu/flink gsaPageRank Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/626.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #626 commit da73b7c74cabedafbc31fdf2403a4f01d77d46d0 Author: andralungu Date: 2015-04-26T22:26:41Z [FLINK-1944][gelly] Added GSA-PageRank example --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1944) Add a Gelly-GSA PageRank example
[ https://issues.apache.org/jira/browse/FLINK-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513308#comment-14513308 ] ASF GitHub Bot commented on FLINK-1944: --- GitHub user andralungu opened a pull request: https://github.com/apache/flink/pull/626 [FLINK-1944][gelly] Added GSA-PageRank example This PR also adds the getSuperStepNumber() method to the Gather, Sum and Apply Functions. You can merge this pull request into a Git repository by running: $ git pull https://github.com/andralungu/flink gsaPageRank Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/626.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #626 commit da73b7c74cabedafbc31fdf2403a4f01d77d46d0 Author: andralungu Date: 2015-04-26T22:26:41Z [FLINK-1944][gelly] Added GSA-PageRank example > Add a Gelly-GSA PageRank example > > > Key: FLINK-1944 > URL: https://issues.apache.org/jira/browse/FLINK-1944 > Project: Flink > Issue Type: Task >Affects Versions: 0.9 >Reporter: Vasia Kalavri >Assignee: Andra Lungu >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1615] [java api] SimpleTweetInputFormat
Github user Elbehery commented on the pull request: https://github.com/apache/flink/pull/621#issuecomment-96418152 @aljoscha I have checked the Travic log. The problem is in Flink Streaming, so why my branch failed ?!! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1615) Introduces a new InputFormat for Tweets
[ https://issues.apache.org/jira/browse/FLINK-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513171#comment-14513171 ] ASF GitHub Bot commented on FLINK-1615: --- Github user Elbehery commented on the pull request: https://github.com/apache/flink/pull/621#issuecomment-96418152 @aljoscha I have checked the Travic log. The problem is in Flink Streaming, so why my branch failed ?!! > Introduces a new InputFormat for Tweets > --- > > Key: FLINK-1615 > URL: https://issues.apache.org/jira/browse/FLINK-1615 > Project: Flink > Issue Type: New Feature > Components: flink-contrib >Affects Versions: 0.8.1 >Reporter: mustafa elbehery >Priority: Minor > > An event-driven parser for Tweets into Java Pojos. > It parses all the important part of the tweet into Java objects. > Tested on cluster and the performance in pretty well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [gelly][Refactoring] Removed example string
GitHub user andralungu opened a pull request: https://github.com/apache/flink/pull/625 [gelly][Refactoring] Removed example string @vasia, Just FYI :), in example/utils we have a class EdgeWithLongIdNullValueParser that is never used. The examples that correspond to the library methods, e.g. PageRankExample cannot be renamed because that will cause a collision with the method name, in this case, PageRank. I guess that's why we added "Example" to the class name in the first place. To me the examples look good as they are in this commit. If you have another idea for the library method examples, let me know, but in my view Example is better than Usage, or whatever different word we may come up with. You can merge this pull request into a Git repository by running: $ git pull https://github.com/andralungu/flink examples Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/625.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #625 commit d6414eae54a2bafdba4457ffb2aa0bd1d34ad57d Author: andralungu Date: 2015-04-26T18:09:04Z [gelly][Refactoring] Removed example string --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1886) how can handles left join ,right join , FULL OUTER JOIN in flink
[
https://issues.apache.org/jira/browse/FLINK-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513089#comment-14513089
]
hagersaleh commented on FLINK-1886:
---
how can handle left outer join for any two dataset this dataset inlcude any
filed number
example
data set one
ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
DataSet>
customer=env.readCsvFile("/home/hadoop/Desktop/Dataset/customer.csv")
.fieldDelimiter('|')
.includeFields("").ignoreFirstLine()
.types(Integer.class,String.class,String.class,String.class);
dataset two
ExecutionEnvironment orders = ExecutionEnvironment.getExecutionEnvironment();
DataSet
customer=env.readCsvFile("/home/hadoop/Desktop/Dataset/order.csv")
.fieldDelimiter('|')
.includeFields("").ignoreFirstLine()
.types(Integer.class,String.class,String.classs);
> how can handles left join ,right join , FULL OUTER JOIN in flink
>
>
> Key: FLINK-1886
> URL: https://issues.apache.org/jira/browse/FLINK-1886
> Project: Flink
> Issue Type: Bug
>Reporter: hagersaleh
>
> how can handles left join ,right join , FULL OUTER JOIN in flink
> how can write code for handle this
> example in inner join
> DataSet result =
>
> customerWithOrders.join(lineitems).where(0).equalTo(0)
> .with(
>
> new JoinFunction ShippingPriorityItem>() {
>
> @Override
>
> public ShippingPriorityItem join(ShippingPriorityItem i,
> Lineitem l) {
>
> i.setRevenue(l.getExtendedprice() * (1 -
> l.getDiscount()));
>
> return i;
>
> }
>
> })
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (FLINK-1944) Add a Gelly-GSA PageRank example
[ https://issues.apache.org/jira/browse/FLINK-1944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andra Lungu reassigned FLINK-1944: -- Assignee: Andra Lungu > Add a Gelly-GSA PageRank example > > > Key: FLINK-1944 > URL: https://issues.apache.org/jira/browse/FLINK-1944 > Project: Flink > Issue Type: Task >Affects Versions: 0.9 >Reporter: Vasia Kalavri >Assignee: Andra Lungu >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1670) Collect method for streaming
[ https://issues.apache.org/jira/browse/FLINK-1670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513019#comment-14513019 ] ASF GitHub Bot commented on FLINK-1670: --- Github user ggevay commented on the pull request: https://github.com/apache/flink/pull/581#issuecomment-96373660 I did the small change suggested by Gyula. > Collect method for streaming > > > Key: FLINK-1670 > URL: https://issues.apache.org/jira/browse/FLINK-1670 > Project: Flink > Issue Type: New Feature > Components: Streaming >Affects Versions: 0.9 >Reporter: Márton Balassi >Assignee: Gabor Gevay >Priority: Minor > > A convenience method for streaming back the results of a job to the client. > As the client itself is a bottleneck anyway an easy solution would be to > provide a socket sink with degree of parallelism 1, from which a client > utility can read. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1944) Add a Gelly-GSA PageRank example
Vasia Kalavri created FLINK-1944: Summary: Add a Gelly-GSA PageRank example Key: FLINK-1944 URL: https://issues.apache.org/jira/browse/FLINK-1944 Project: Flink Issue Type: Task Affects Versions: 0.9 Reporter: Vasia Kalavri Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1670] Made DataStream iterable
Github user ggevay commented on the pull request: https://github.com/apache/flink/pull/581#issuecomment-96373660 I did the small change suggested by Gyula. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (FLINK-1758) Extend Gelly's neighborhood methods
[ https://issues.apache.org/jira/browse/FLINK-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vasia Kalavri resolved FLINK-1758. -- Resolution: Implemented Fix Version/s: 0.9 > Extend Gelly's neighborhood methods > --- > > Key: FLINK-1758 > URL: https://issues.apache.org/jira/browse/FLINK-1758 > Project: Flink > Issue Type: Improvement > Components: Gelly >Affects Versions: 0.9 >Reporter: Vasia Kalavri >Assignee: Andra Lungu > Fix For: 0.9 > > > Currently, the neighborhood methods only allow returning a single value per > vertex. In many cases, it is desirable to return several or no value per > vertex. This is the case in clustering coefficient computation, > vertex-centric jaccard, algorithms where a vertex computes a value per edge > or when a vertex computes a value only for some of its neighbors. > This issue proposes to > - change the current reduceOnEdges/reduceOnNeighbors methods to use > combinable reduce operations where possible > - provide groupReduce-versions, which will use a Collector and allow > returning none or more values per vertex. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1758][gelly] Neighborhood Methods Exten...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/576 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (FLINK-1758) Extend Gelly's neighborhood methods
[ https://issues.apache.org/jira/browse/FLINK-1758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513018#comment-14513018 ] ASF GitHub Bot commented on FLINK-1758: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/576 > Extend Gelly's neighborhood methods > --- > > Key: FLINK-1758 > URL: https://issues.apache.org/jira/browse/FLINK-1758 > Project: Flink > Issue Type: Improvement > Components: Gelly >Affects Versions: 0.9 >Reporter: Vasia Kalavri >Assignee: Andra Lungu > > Currently, the neighborhood methods only allow returning a single value per > vertex. In many cases, it is desirable to return several or no value per > vertex. This is the case in clustering coefficient computation, > vertex-centric jaccard, algorithms where a vertex computes a value per edge > or when a vertex computes a value only for some of its neighbors. > This issue proposes to > - change the current reduceOnEdges/reduceOnNeighbors methods to use > combinable reduce operations where possible > - provide groupReduce-versions, which will use a Collector and allow > returning none or more values per vertex. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (FLINK-1942) Add configuration options to Gelly-GSA
[ https://issues.apache.org/jira/browse/FLINK-1942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andra Lungu reassigned FLINK-1942: -- Assignee: Andra Lungu > Add configuration options to Gelly-GSA > -- > > Key: FLINK-1942 > URL: https://issues.apache.org/jira/browse/FLINK-1942 > Project: Flink > Issue Type: Improvement > Components: Gelly >Affects Versions: 0.9 >Reporter: Vasia Kalavri >Assignee: Andra Lungu > > Currently, it is not possible to configure a GSA iteration. Similarly to > vertex-centric, we should allow setting the iteration name and degree of > parallelism, aggregators, broadcast variables and whether the solution set is > kept in unmanaged memory. > The docs should be updated accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1943) Add Gelly-GSA compiler and translation tests
Vasia Kalavri created FLINK-1943: Summary: Add Gelly-GSA compiler and translation tests Key: FLINK-1943 URL: https://issues.apache.org/jira/browse/FLINK-1943 Project: Flink Issue Type: Test Components: Gelly Affects Versions: 0.9 Reporter: Vasia Kalavri These should be similar to the corresponding Spargel tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1942) Add configuration options to Gelly-GSA
Vasia Kalavri created FLINK-1942: Summary: Add configuration options to Gelly-GSA Key: FLINK-1942 URL: https://issues.apache.org/jira/browse/FLINK-1942 Project: Flink Issue Type: Improvement Components: Gelly Affects Versions: 0.9 Reporter: Vasia Kalavri Currently, it is not possible to configure a GSA iteration. Similarly to vertex-centric, we should allow setting the iteration name and degree of parallelism, aggregators, broadcast variables and whether the solution set is kept in unmanaged memory. The docs should be updated accordingly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1941) Add documentation for Gelly-GSA
Vasia Kalavri created FLINK-1941: Summary: Add documentation for Gelly-GSA Key: FLINK-1941 URL: https://issues.apache.org/jira/browse/FLINK-1941 Project: Flink Issue Type: Task Components: Gelly Affects Versions: 0.9 Reporter: Vasia Kalavri Add a section in the Gelly guide to describe the newly introduced Gather-Sum-Apply iteration method. Show how GSA uses delta iterations internally and explain the differences of this model as compared to vertex-centric. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (FLINK-1514) [Gelly] Add a Gather-Sum-Apply iteration method
[ https://issues.apache.org/jira/browse/FLINK-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vasia Kalavri resolved FLINK-1514. -- Resolution: Implemented Fix Version/s: 0.9 > [Gelly] Add a Gather-Sum-Apply iteration method > --- > > Key: FLINK-1514 > URL: https://issues.apache.org/jira/browse/FLINK-1514 > Project: Flink > Issue Type: New Feature > Components: Gelly >Affects Versions: 0.9 >Reporter: Vasia Kalavri >Assignee: Daniel Bali > Fix For: 0.9 > > > This will be a method that implements the GAS computation model, but without > the "scatter" step. The phases can be mapped into the following steps inside > a delta iteration: > gather: a map on each < srcVertex, edge, trgVertex > that produces a partial > value > sum: a reduce that combines the partial values > apply: join with vertex set to update the vertex values using the results of > sum and the previous state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1514) [Gelly] Add a Gather-Sum-Apply iteration method
[ https://issues.apache.org/jira/browse/FLINK-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513014#comment-14513014 ] ASF GitHub Bot commented on FLINK-1514: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/408 > [Gelly] Add a Gather-Sum-Apply iteration method > --- > > Key: FLINK-1514 > URL: https://issues.apache.org/jira/browse/FLINK-1514 > Project: Flink > Issue Type: New Feature > Components: Gelly >Affects Versions: 0.9 >Reporter: Vasia Kalavri >Assignee: Daniel Bali > > This will be a method that implements the GAS computation model, but without > the "scatter" step. The phases can be mapped into the following steps inside > a delta iteration: > gather: a map on each < srcVertex, edge, trgVertex > that produces a partial > value > sum: a reduce that combines the partial values > apply: join with vertex set to update the vertex values using the results of > sum and the previous state. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] flink pull request: [FLINK-1514][Gelly] Add a Gather-Sum-Apply ite...
Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/408 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (FLINK-1940) StockPrice example cannot be visualized
Gyula Fora created FLINK-1940: - Summary: StockPrice example cannot be visualized Key: FLINK-1940 URL: https://issues.apache.org/jira/browse/FLINK-1940 Project: Flink Issue Type: Bug Components: Streaming Reporter: Gyula Fora The planvisualizer fails on the JSON generated by the StockPrice example -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (FLINK-1939) Add Parquet Documentation to Wiki
Felix Neutatz created FLINK-1939: Summary: Add Parquet Documentation to Wiki Key: FLINK-1939 URL: https://issues.apache.org/jira/browse/FLINK-1939 Project: Flink Issue Type: Task Components: Documentation Reporter: Felix Neutatz Assignee: Felix Neutatz Priority: Trivial Add documentation who to read and write Parquet :) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
