Re: Bug fix release 0.9.1
Hi, I did a (thin) fix for FLINK-2286 Window ParallelMerge sometimes swallows elements of the last window and also added a PR https://github.com/apache/flink/pull/994. Regards, Gábor On Thu, Aug 6, 2015 at 10:07 AM Maximilian Michels m...@apache.org wrote: Hi, The following commits have been added to the release-0.9 branch since the 0.9.0 release: c7e8684 [FLINK-2229] Add equals() and hashCode() to ObjectArrayTypeInfo 451eb82 [FLINK-2280] GenericTypeComparator.compare() respects ascending flag acd4317 [FLINK-2353] Respect JobConfigurable interface in Hadoop mapred wrappers 055997e [docs] fix loading of style sheet with protocol relative base URL 7c2a704 [FLINK-2293] [runtime] Fix estimation for the number of hash buckets on recursive builds 0789460 [FLINK-2285] [streaming] Removed duplicate call in close from GroupedActiveDiscretizer d946599 [FLINK-2298] Add option to pass a custom name for Flink on YARN 9835625 [docs] fix broken links in FAQ 69f858e [FLINK-2257] [streaming] Properly forward rich window function calls to wrapped functions d7cfa55 [docs] correct baseurl for 0.9 documentation b5702c2 [docs] correct yarn command-line example 9f0da5b [FLINK-2262][utils] rename method for default integer value in ParameterTool I'd propose to add all of them to the 0.9.1 release. They contain important fixes and should thus be included in the minor release. As for the JIRA issues, it would be great to fix and include all of them. Except for FLINK-2236 RowSerializer and CaseClassComparator are not in sync regarding Null-Values because we reverted null-value support for in the Table API for the 0.9.0 release. If the support on the master is fixed, we could include it again together with the fix for the RowSerializer and CaseClassComperator. Cheers, Max On Fri, Jul 31, 2015 at 12:40 PM, Fabian Hueske fhue...@gmail.com wrote: Thanks Ufuk for starting this discussion. We should also go through the commit logs of the master branch and see if we forgot to cherry-pick some fixes over to the release-0.9 branch. I can do that and compile a list of potential fixes. Cheers, Fabian 2015-07-31 11:34 GMT+02:00 Ufuk Celebi u...@apache.org: Hey all! I want to start a discussion about the next 0.9 release. Since 0.9.0 there have been 19 commits addressing 16 issues. Of these 16 issues, two were critical issues regarding the runtime, which require a new release urgently. What's do you think about this? --- If we agree about doing a new release, I can act as release manager. JIRA currently reports 5 open issues with fix version set to 0.9.1: FLINK-2450 IndexOutOfBoundsException in KryoSerializer FLINK-2442 PojoType fields not supported by field position keys FLINK-2356 Resource leak in checkpoint coordinator FLINK-2286 Window ParallelMerge sometimes swallows elements of the last window FLINK-2236 RowSerializer and CaseClassComparator are not in sync regarding Null-Values I propose the following to get the release going: 1. Please have a look at these issues [1] and provide comments/fixes where appropriate. 2. Please open new issues (with fixes if you like ;)) and tag them with fix version 0.9.1 if you think that there is something that should get into the release as well. I propose to do 1. and 2. for the next week. If there is a consensus about the issues after next week, we can get the release process/testing started. – Ufuk [1] https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20status%20in%20%28Open%2C%20%22In%20Progress%22%2C%20Reopened%29%20AND%20fixVersion%20%3D%200.9.1
Re: Replacing Checkpointed interface with field annotations
Wow, this looks pretty concise. I really like it!
On Mon, Jun 29, 2015 at 3:27 PM Gyula Fóra gyf...@apache.org wrote:
Hey all!
Just to add something new to the end of the discussion list. After some
discussion with Seif, and Paris, I have added a commit that replaces the
use of the Checkpointed interface with field annotations.
This is probably the most lightweight state declaration so far and it will
probably work very well to replace the Checkpointed interface:
public class StatefulMapper implements MapFunctionInteger,Integer {
@State
int counter;
@State
Serializable state;
Object notState
public Integer map(Integer input)[
counter++;
//update other state
/...
}
}
What do you think?
You can check it out here
https://github.com/gyfora/flink/commits/annotated_state.
Cheers,
Gyula
Re: New project website
Great! It looks way better than the current site :) On Mon, May 11, 2015 at 1:28 PM Stephan Ewen se...@apache.org wrote: I think we may have to remove the term Language Integrated Queries, I think that is trademarked by Microsoft. Otherwise, +1 On Mon, May 11, 2015 at 1:19 PM, Maximilian Michels m...@apache.org wrote: +1 very nice comprehensive overhaul of the website. I'd suggest to merge this as soon as possible. We can incrementally fix the remaining issues and add a twitter feed to the front page. On Mon, May 11, 2015 at 12:09 PM, Ted Dunning ted.dunn...@gmail.com wrote: If there is an active twitter entity for Flink, I recommend framing that on the home page. On Mon, May 11, 2015 at 8:51 AM, Ufuk Celebi u...@apache.org wrote: Hey all, I reworked the project website the last couple of days and would like to share the preview: http://uce.github.io/flink-web/ I would like to get this in asap. We can push incremental updates at any time, but I think this version is a big improvement over the current status quo. If I get some +1s I'll go ahead and update the website today. – Ufuk
Re: [Gelly] Vertex-centric iteration updateVertex does not get called
Hey, Thank you both for your help :) I was missing this point in the Pregel model. I thought it would be expected to receive empty message iterators. Yes, I know it is not the best use-case for vertex-centric iteration as there are well-defined steps, and also know the Flink examples. I only wanted this as a demonstration for the participants of the workshop (among other examples) of thinking in the vertex-centric Pregel model. Then I would show them the other solution that fits the problem more. Of course, I could implement triangle count with the vertex-centric iteration too, but I ran into this behavior and was not sure if it was expected. I might move on to other examples. Thanks anyway :) Cheers, Gabor
Re: Website documentation minor bug
+1 for the proposal!
On Tue, Mar 17, 2015 at 4:35 PM Aljoscha Krettek aljos...@apache.org
wrote:
+1
On Mar 17, 2015 12:28 PM, Maximilian Michels m...@apache.org wrote:
Another proposal in addition to the recent changes: How about we remove
the
fixed attribute from the navigation of the docs? Then the nav will just
scroll away and give space for the documentation. After all, how useful
is
it to constantly have the Java and Scala doc links available?
Actually, that's also the behavior for the rest of the website like in
http://flink.apache.org/community.html
On Tue, Mar 10, 2015 at 4:03 PM, Maximilian Michels m...@apache.org
wrote:
Thanks for the feedback. Merged.
On Tue, Mar 10, 2015 at 11:34 AM, Márton Balassi
balassi.mar...@gmail.com wrote:
+1
On Tue, Mar 10, 2015 at 11:28 AM, Hermann Gábor
reckone...@gmail.com
wrote:
Looks nice, +1 for the new one.
On Tue, Mar 10, 2015 at 11:24 AM Maximilian Michels m...@apache.org
wrote:
Seems like my smart data crawling web mail took the linked images
out.
So here we go again:
New
http://i.imgur.com/KK7fhiR.png
Old
http://i.imgur.com/kP2LPnY.png
On Tue, Mar 10, 2015 at 11:17 AM, Stephan Ewen se...@apache.org
wrote:
Looks the same to me ;-)
The mailing lists do not support attachments...
On Tue, Mar 10, 2015 at 11:15 AM, Maximilian Michels
m...@apache.org
wrote:
So here are the proposed changes.
New
Old
If there are no objections, I will merge this by the end of the
day.
Best regards,
Max
On Mon, Mar 9, 2015 at 4:22 PM, Hermann Gábor
reckone...@gmail.com
wrote:
Thanks Gyula, that helps a lot :D
Nice solution. Thank you Max!
I also support the reduced header size!
Cheers,
Gabor
On Mon, Mar 9, 2015 at 3:36 PM Márton Balassi
balassi.mar...@gmail.com
wrote:
+1 for the proposed solution from Max
+1 for decreasing the size: but let's have preview, I also
think
that
the
current one is a bit too large
On Mon, Mar 9, 2015 at 2:16 PM, Maximilian Michels
m...@apache.org
wrote:
We can fix this for the headings by adding the following
CSS
rule:
h1, h2, h3, h4 {
padding-top: 100px;
margin-top: -100px;
}
In the course of changing this, we could also reduce the
size of
the
navigation header in the docs. It is occupies too much
space
and
doesn't have a lot of functionality. I'd suggest to half
its
size.
The
positioning at the top is fine for me.
Kind regards,
Max
On Mon, Mar 9, 2015 at 2:08 PM, Hermann Gábor
reckone...@gmail.com
wrote:
I think the navigation looks nice this way.
It's rather a small CSS/HTML problem that the header
shades
the
title
when
clicking on an anchor link.
(It's that the content starts at top, but there is the
header
covering
it.)
I'm not much into web stuff, but I would gladly fix it.
Can someone help me with this?
On Sun, Mar 8, 2015 at 9:52 PM Stephan Ewen
se...@apache.org
wrote:
I agree, it is not optimal.
What would be a better way to do this? Have the main
navigation
(currently
on the left) at the top, and the per-page navigation
on
the
side?
Do you want to take a stab at this?
On Sun, Mar 8, 2015 at 7:08 PM, Hermann Gábor
reckone...@gmail.com
wrote:
Hey,
Currently following an anchor link (e.g.
#transformations
http://ci.apache.org/projects/flink/flink-docs-master/
programming_guide.html#transformations
)
results in the header occupying the top of the page,
thus
the
title
and
some of the first lines cannot be seen. This is not
a
big
deal,
but
it's
user-facing and a bit irritating.
Can someone fix it, please?
(I tried it on Firefox and Chromium on Ubuntu 14.10)
Cheers,
Gabor
Re: Website documentation minor bug
Great! Thanks Max!
On Wed, Mar 18, 2015 at 11:34 AM Maximilian Michels m...@apache.org wrote:
Alright, changed it for the master. I think it makes the docs much more
pleasant to read.
http://ci.apache.org/projects/flink/flink-docs-master/
On Wed, Mar 18, 2015 at 9:26 AM, Hermann Gábor reckone...@gmail.com
wrote:
+1 for the proposal!
On Tue, Mar 17, 2015 at 4:35 PM Aljoscha Krettek aljos...@apache.org
wrote:
+1
On Mar 17, 2015 12:28 PM, Maximilian Michels m...@apache.org wrote:
Another proposal in addition to the recent changes: How about we
remove
the
fixed attribute from the navigation of the docs? Then the nav will
just
scroll away and give space for the documentation. After all, how
useful
is
it to constantly have the Java and Scala doc links available?
Actually, that's also the behavior for the rest of the website like
in
http://flink.apache.org/community.html
On Tue, Mar 10, 2015 at 4:03 PM, Maximilian Michels m...@apache.org
wrote:
Thanks for the feedback. Merged.
On Tue, Mar 10, 2015 at 11:34 AM, Márton Balassi
balassi.mar...@gmail.com wrote:
+1
On Tue, Mar 10, 2015 at 11:28 AM, Hermann Gábor
reckone...@gmail.com
wrote:
Looks nice, +1 for the new one.
On Tue, Mar 10, 2015 at 11:24 AM Maximilian Michels
m...@apache.org
wrote:
Seems like my smart data crawling web mail took the linked
images
out.
So here we go again:
New
http://i.imgur.com/KK7fhiR.png
Old
http://i.imgur.com/kP2LPnY.png
On Tue, Mar 10, 2015 at 11:17 AM, Stephan Ewen
se...@apache.org
wrote:
Looks the same to me ;-)
The mailing lists do not support attachments...
On Tue, Mar 10, 2015 at 11:15 AM, Maximilian Michels
m...@apache.org
wrote:
So here are the proposed changes.
New
Old
If there are no objections, I will merge this by the end of
the
day.
Best regards,
Max
On Mon, Mar 9, 2015 at 4:22 PM, Hermann Gábor
reckone...@gmail.com
wrote:
Thanks Gyula, that helps a lot :D
Nice solution. Thank you Max!
I also support the reduced header size!
Cheers,
Gabor
On Mon, Mar 9, 2015 at 3:36 PM Márton Balassi
balassi.mar...@gmail.com
wrote:
+1 for the proposed solution from Max
+1 for decreasing the size: but let's have preview, I
also
think
that
the
current one is a bit too large
On Mon, Mar 9, 2015 at 2:16 PM, Maximilian Michels
m...@apache.org
wrote:
We can fix this for the headings by adding the
following
CSS
rule:
h1, h2, h3, h4 {
padding-top: 100px;
margin-top: -100px;
}
In the course of changing this, we could also reduce
the
size of
the
navigation header in the docs. It is occupies too
much
space
and
doesn't have a lot of functionality. I'd suggest to
half
its
size.
The
positioning at the top is fine for me.
Kind regards,
Max
On Mon, Mar 9, 2015 at 2:08 PM, Hermann Gábor
reckone...@gmail.com
wrote:
I think the navigation looks nice this way.
It's rather a small CSS/HTML problem that the
header
shades
the
title
when
clicking on an anchor link.
(It's that the content starts at top, but there is
the
header
covering
it.)
I'm not much into web stuff, but I would gladly fix
it.
Can someone help me with this?
On Sun, Mar 8, 2015 at 9:52 PM Stephan Ewen
se...@apache.org
wrote:
I agree, it is not optimal.
What would be a better way to do this? Have the
main
navigation
(currently
on the left) at the top, and the per-page
navigation
on
the
side?
Do you want to take a stab at this?
On Sun, Mar 8, 2015 at 7:08 PM, Hermann Gábor
reckone...@gmail.com
wrote:
Hey,
Currently following an anchor link (e.g.
#transformations
http://ci.apache.org/projects/flink/flink-docs-master/
programming_guide.html#transformations
)
results
Re: [DISCUSS] Issues with heterogeneity of the code
+1 for the stricter Java code styles. We should not forget about providing code formatter settings for Eclipse and Intellij IDEA (as mentioned above). That would help a lot. (Of course if we'll use Google Code Style, they already provide such files https://code.google.com/p/google-styleguide/source/browse/trunk/intellij-java-google-style.xml .) On Mon, Mar 16, 2015 at 2:45 PM Alexander Alexandrov alexander.s.alexand...@gmail.com wrote: +1 for not limiting the line length. 2015-03-16 14:39 GMT+01:00 Stephan Ewen se...@apache.org: +1 for not limiting the line length. Everyone should have a good sense to break lines. When in exceptional cases people violate this, it is usually for a good reason. On Mon, Mar 16, 2015 at 2:18 PM, Maximilian Michels m...@apache.org wrote: +1 for enforcing a more strict Java code style. However, let's not introduce a line legth of 100 like in Scala. I think that's hurting readability of the code. On Sat, Mar 14, 2015 at 4:41 PM, Ufuk Celebi u...@apache.org wrote: On Saturday, March 14, 2015, Aljoscha Krettek aljos...@apache.org wrote: I'm in favor of strict coding styles. And I like the google style. +1 I would like that. We essentially all agree that we want more homogeneity and I think strict rules are the only way to go. Since this is a very subjective matter it makes sense to go with something (somewhat) well established like the Google code style.
Re: [DISCUSS] Name of Expression API and DataSet abstraction
+1 for DataTable On Mon, Mar 16, 2015 at 4:11 PM Till Rohrmann trohrm...@apache.org wrote: +1 for DataTable On Mon, Mar 16, 2015 at 10:34 AM, Márton Balassi balassi.mar...@gmail.com wrote: +1 for Max's suggestion. On Mon, Mar 16, 2015 at 10:32 AM, Ufuk Celebi u...@apache.org wrote: On Fri, Mar 13, 2015 at 6:08 PM, Maximilian Michels m...@apache.org wrote: Thanks for starting the discussion. We should definitely not keep flink-expressions. I'm in favor of DataTable for the DataSet abstraction equivalent. For consistency, the package name should then be flink-table. At first sight, the name seems kind of plain but I think it is quite intuitive because the API enables you to work in a SQL like fashion. +1 I think this is a very good suggestion. :-) (There is an associated issue, we shouldn't forget to close: https://issues.apache.org/jira/browse/FLINK-1623)
Re: Website documentation minor bug
Looks nice, +1 for the new one.
On Tue, Mar 10, 2015 at 11:24 AM Maximilian Michels m...@apache.org wrote:
Seems like my smart data crawling web mail took the linked images out.
So here we go again:
New
http://i.imgur.com/KK7fhiR.png
Old
http://i.imgur.com/kP2LPnY.png
On Tue, Mar 10, 2015 at 11:17 AM, Stephan Ewen se...@apache.org wrote:
Looks the same to me ;-)
The mailing lists do not support attachments...
On Tue, Mar 10, 2015 at 11:15 AM, Maximilian Michels m...@apache.org
wrote:
So here are the proposed changes.
New
Old
If there are no objections, I will merge this by the end of the day.
Best regards,
Max
On Mon, Mar 9, 2015 at 4:22 PM, Hermann Gábor reckone...@gmail.com
wrote:
Thanks Gyula, that helps a lot :D
Nice solution. Thank you Max!
I also support the reduced header size!
Cheers,
Gabor
On Mon, Mar 9, 2015 at 3:36 PM Márton Balassi
balassi.mar...@gmail.com
wrote:
+1 for the proposed solution from Max
+1 for decreasing the size: but let's have preview, I also think
that
the
current one is a bit too large
On Mon, Mar 9, 2015 at 2:16 PM, Maximilian Michels m...@apache.org
wrote:
We can fix this for the headings by adding the following CSS rule:
h1, h2, h3, h4 {
padding-top: 100px;
margin-top: -100px;
}
In the course of changing this, we could also reduce the size of
the
navigation header in the docs. It is occupies too much space and
doesn't have a lot of functionality. I'd suggest to half its size.
The
positioning at the top is fine for me.
Kind regards,
Max
On Mon, Mar 9, 2015 at 2:08 PM, Hermann Gábor
reckone...@gmail.com
wrote:
I think the navigation looks nice this way.
It's rather a small CSS/HTML problem that the header shades the
title
when
clicking on an anchor link.
(It's that the content starts at top, but there is the header
covering
it.)
I'm not much into web stuff, but I would gladly fix it.
Can someone help me with this?
On Sun, Mar 8, 2015 at 9:52 PM Stephan Ewen se...@apache.org
wrote:
I agree, it is not optimal.
What would be a better way to do this? Have the main navigation
(currently
on the left) at the top, and the per-page navigation on the
side?
Do you want to take a stab at this?
On Sun, Mar 8, 2015 at 7:08 PM, Hermann Gábor
reckone...@gmail.com
wrote:
Hey,
Currently following an anchor link (e.g. #transformations
http://ci.apache.org/projects/flink/flink-docs-master/
programming_guide.html#transformations
)
results in the header occupying the top of the page, thus the
title
and
some of the first lines cannot be seen. This is not a big
deal,
but
it's
user-facing and a bit irritating.
Can someone fix it, please?
(I tried it on Firefox and Chromium on Ubuntu 14.10)
Cheers,
Gabor
Re: Website documentation minor bug
I think the navigation looks nice this way. It's rather a small CSS/HTML problem that the header shades the title when clicking on an anchor link. (It's that the content starts at top, but there is the header covering it.) I'm not much into web stuff, but I would gladly fix it. Can someone help me with this? On Sun, Mar 8, 2015 at 9:52 PM Stephan Ewen se...@apache.org wrote: I agree, it is not optimal. What would be a better way to do this? Have the main navigation (currently on the left) at the top, and the per-page navigation on the side? Do you want to take a stab at this? On Sun, Mar 8, 2015 at 7:08 PM, Hermann Gábor reckone...@gmail.com wrote: Hey, Currently following an anchor link (e.g. #transformations http://ci.apache.org/projects/flink/flink-docs-master/ programming_guide.html#transformations ) results in the header occupying the top of the page, thus the title and some of the first lines cannot be seen. This is not a big deal, but it's user-facing and a bit irritating. Can someone fix it, please? (I tried it on Firefox and Chromium on Ubuntu 14.10) Cheers, Gabor
Re: Website documentation minor bug
Thanks Gyula, that helps a lot :D
Nice solution. Thank you Max!
I also support the reduced header size!
Cheers,
Gabor
On Mon, Mar 9, 2015 at 3:36 PM Márton Balassi balassi.mar...@gmail.com
wrote:
+1 for the proposed solution from Max
+1 for decreasing the size: but let's have preview, I also think that the
current one is a bit too large
On Mon, Mar 9, 2015 at 2:16 PM, Maximilian Michels m...@apache.org wrote:
We can fix this for the headings by adding the following CSS rule:
h1, h2, h3, h4 {
padding-top: 100px;
margin-top: -100px;
}
In the course of changing this, we could also reduce the size of the
navigation header in the docs. It is occupies too much space and
doesn't have a lot of functionality. I'd suggest to half its size. The
positioning at the top is fine for me.
Kind regards,
Max
On Mon, Mar 9, 2015 at 2:08 PM, Hermann Gábor reckone...@gmail.com
wrote:
I think the navigation looks nice this way.
It's rather a small CSS/HTML problem that the header shades the title
when
clicking on an anchor link.
(It's that the content starts at top, but there is the header covering
it.)
I'm not much into web stuff, but I would gladly fix it.
Can someone help me with this?
On Sun, Mar 8, 2015 at 9:52 PM Stephan Ewen se...@apache.org wrote:
I agree, it is not optimal.
What would be a better way to do this? Have the main navigation
(currently
on the left) at the top, and the per-page navigation on the side?
Do you want to take a stab at this?
On Sun, Mar 8, 2015 at 7:08 PM, Hermann Gábor reckone...@gmail.com
wrote:
Hey,
Currently following an anchor link (e.g. #transformations
http://ci.apache.org/projects/flink/flink-docs-master/
programming_guide.html#transformations
)
results in the header occupying the top of the page, thus the title
and
some of the first lines cannot be seen. This is not a big deal, but
it's
user-facing and a bit irritating.
Can someone fix it, please?
(I tried it on Firefox and Chromium on Ubuntu 14.10)
Cheers,
Gabor
[DISCUSS] Documentation Java/Scala order
Hey all, The default language for the source codes in the documentation is Java (see the Programming Guide http://ci.apache.org/projects/flink/flink-docs-master/programming_guide.html), but in blog posts (like Introducing Flink Streaming http://flink.apache.org/news/2015/02/09/streaming-example.html) Scala is the default. I think it should be consistent. Scala is more concise, thus it might be better for longer source codes, but Java is more widely used. What do you think? It came up because we've been dealing with FLINK-1429 https://issues.apache.org/jira/browse/FLINK-1429 (streaming Scala docs) with Marton. Cheers, Gabor
Re: Gelly is in!
Cool! Congrats! :) On Wed Feb 11 2015 at 12:45:17 Max Michels m...@data-artisans.com wrote: Great contribution! Thanks to everyone who made it possible. On Wed, Feb 11, 2015 at 12:21 PM, Gyula Fóra gyf...@apache.org wrote: Great! Thank you guys :) On Wed, Feb 11, 2015 at 12:17 PM, Vasiliki Kalavri vasilikikala...@gmail.com wrote: Hello everyone, I'm very happy ^^ Thank you for reviewing and merging! And to everyone that was at the hackathon in Stockholm :) There's a lot more exciting stuff coming in Gelly soon! Cheers, V. On 11 February 2015 at 12:11, Márton Balassi balassi.mar...@gmail.com wrote: Woot! :) On Wed, Feb 11, 2015 at 11:53 AM, Stephan Ewen se...@apache.org wrote: Hi everyone! I am happy to say that the graph library Gelly is finally in the code :-) Thanks Vasia, Daniel, Andra, and Carsten for the great work! Greetings, Stephan
Streaming fault tolerance with event sending
Hey, We've been implementing a simple Storm-like fault tolerance system with persisting source records and keeping track of all the records (whether they've been processed), and replaying them if they fail. The straightforward way to do this was creating a special AbstractJobVertex (and AbstractInvokable) that deals with replaying and keeps track of the records. So there is a so called FTLayerVertex that needs a two-way communication with the tasks: 1. it needs to be notified about a processed record or a newly created record 2. it needs to resend failed records to the appropriate tasks We've decided to use events for 1. and regular message sending (i.e. BufferReader/Writer) for 2. So we need to send events backward on a BufferReader. The problem is that backward events are only sent and received when the channel is initialized (in case of one channel for now), there's a living connection (i.e. it's not UNKNOWN, please correct me if I'm wrong). But sometimes the connection is not yet initialized when we're sending these events from the tasks to the FTLayerVertex. We've tried a workaround: sending an initializer forward event from FT to all the tasks, and waiting for this event at the beginning of every task's invoke, thus the backward events will only be sent when the channel is surely up. That did not work out, because in order to receive an event, we need to read that input, and we only want to read and process that input after we made sure we can use the channel. Another workaround were sending an initializer record instead of an event. That did not work out either, because we wanted to read that input (from the FTLayerVertex) as a regular input, so the given BufferReader were also passed to a UnionBufferReader, and reading the given BufferReader separately (wrapped in a RecordReader) resulted in the UnionBufferReader getting stuck. (It seemed as the UnionBufferReader put the given BufferReader in it's queue because of this initializer message, but the message had been already read from the BufferReader separately, so it got stuck waiting for a new record from the BufferReader). These workarounds don't seem natural either, because we're pulling this synchronization of connection initialization a layer up. It would be nice if we had some kind of events that are made sure to get to the destination. Is that possible? Would that be useful for other purposes too? Does anyone know some workaround that might work? Are we missing something? (Please correct me if I see something wrong.) Should we use something else instead of events? (I think we cannot avoid this two-way communication between vertices.) All suggestions are welcome! Best regards, Gabor
Re: Question on Streaming Dependency
Hey! As I can see, it was mistakenly added there during summer (probably by interns working on connectors). Thanks for the fix! Regards, Gábor On Sat, Jan 10, 2015 at 4:50 PM, Stephan Ewen se...@apache.org wrote: Just figured out that it is a transitive dependency of the streaming connectors. Will move it there with a comment. On Sat, Jan 10, 2015 at 4:43 PM, Stephan Ewen se...@apache.org wrote: I am currently going through the code checking that all license-related issues are well (and cleaning up unused dependencies on the way). I found that the streaming core project references the following dependency, which seems never used in the code: groupIdorg.apache.sling/groupId artifactIdorg.apache.sling.commons.json/artifactId version2.0.6/version Was this accidentally added? IIRC sling is a web framework, so I was also a bit surprised to see it as a dependency in the streaming code. Can anyone help me? Was this added because another dependency relies on it, but does not include it by itself? Greetings, Stephan
