IIRC it is also exciting in that the final window is smaller than the union
of its component windows. I think we have a JIRA open to decide if that is
even allowed.
On Wed, Sep 20, 2017 at 12:15 AM, Etienne Chauchot
wrote:
> Hi
> Indeed, the query builds a lot of maps, it is thus expensive. I to
Hi
Indeed, the query builds a lot of maps, it is thus expensive. I totally
agree with your point, adding a query with a simple merge such as the
one that is done in the ValidatesRunner bellow is a good idea. I'll add
2 tickets, one for the migration of winningBids to state API and one for
the
I think adding a new query for window merging makes sense and would be easier
to follow and maintain.
Regards
JB
On Sep 19, 2017, 18:09, at 18:09, Reuven Lax wrote:
>On Tue, Sep 19, 2017 at 7:29 AM, Etienne Chauchot
>wrote:
>
>> Hi all,
>>
>> I'm resuming my work on Nexmark a bit, starting to
On Tue, Sep 19, 2017 at 7:29 AM, Etienne Chauchot
wrote:
> Hi all,
>
> I'm resuming my work on Nexmark a bit, starting to do some maintenance on
> the tickets
>
> @Reuven: I have some comments inline below.
>
> Le 14/05/2017 à 14:29, Reuven Lax a écrit :
>
>> Great to hear! A couple of comments:
Hi all,
I'm resuming my work on Nexmark a bit, starting to do some maintenance
on the tickets
@Reuven: I have some comments inline below.
Le 14/05/2017 à 14:29, Reuven Lax a écrit :
Great to hear! A couple of comments:
When Query 10 was written, the file-based sinks did not supported unboun
@Reuven,
Tell me if I can help on that
Etienne
Le 15/09/2017 à 06:44, Reuven Lax a écrit :
It's being worked on. Turns out there are some modifications still needed
to the NexMark queries.
Reuven
On Thu, Sep 14, 2017 at 9:33 PM, Pei HE wrote:
Could any Googlers help to run NexMark on Dataf
It's being worked on. Turns out there are some modifications still needed
to the NexMark queries.
Reuven
On Thu, Sep 14, 2017 at 9:33 PM, Pei HE wrote:
> Could any Googlers help to run NexMark on Dataflow streaming and share the
> numbers with the community?
> --
> Pei
>
> On Fri, Aug 25, 2017
Could any Googlers help to run NexMark on Dataflow streaming and share the
numbers with the community?
--
Pei
On Fri, Aug 25, 2017 at 11:28 PM, Lukasz Cwik
wrote:
> Etienne, cut some JIRAs for improvements like ValidatesRunner for the
> Nexmark suite that you think are worthy. Some of them might
Etienne, cut some JIRAs for improvements like ValidatesRunner for the
Nexmark suite that you think are worthy. Some of them might be good
'starter' tasks as well.
On Fri, Aug 25, 2017 at 1:43 AM, Etienne Chauchot
wrote:
> Hi guys,
>
> There is also some points to discuss:
>
> - I think some of t
Hi guys,
There is also some points to discuss:
- I think some of the tests in this test suite should be generalized as
validatesRunner tests like it was done for example for custom window
merging
(https://github.com/apache/beam/blob/5181e619f17e1f69fabe8d5bdfc7a3a6a2142cde/sdks/java/core/src/
Yeah, was looking forward to this.
On Thu, Aug 24, 2017 at 9:20 AM, Tyler Akidau
wrote:
> Awesome news, thank you! :-D
>
> On Thu, Aug 24, 2017 at 12:40 AM Etienne Chauchot
> wrote:
>
> > Hi all,
> >
> > I wanted to let you know that the Nexmark PR is merged into master. Feel
> > free to use it
Awesome news, thank you! :-D
On Thu, Aug 24, 2017 at 12:40 AM Etienne Chauchot
wrote:
> Hi all,
>
> I wanted to let you know that the Nexmark PR is merged into master. Feel
> free to use it (e.g. performance testing, release testing ...).
>
> Etienne
>
> Le 12/05/2017 à 10:55, Etienne Chauchot a
Hi all,
I wanted to let you know that the Nexmark PR is merged into master. Feel
free to use it (e.g. performance testing, release testing ...).
Etienne
Le 12/05/2017 à 10:55, Etienne Chauchot a écrit :
Hi guys,
I wanted to let you know that I have just submitted a PR around
NexMark. This
Thrilled to see this!
Hopefully we'll get to automating the execution fully in Jenkins, and have
the results readily available to the whole community. (Happy to help
myself!)
On Sun, May 14, 2017 at 5:29 AM, Reuven Lax
wrote:
> Great to hear! A couple of comments:
>
> When Query 10 was written,
Great to hear! A couple of comments:
When Query 10 was written, the file-based sinks did not supported unbounded
input. Now that in Beam FileBasedSink supports windowed output files, I
think we should just rip out the custom IO code in Query 10 and replace it
with AvroIO - this is closer to what
Hello,
Thanks Etienne for opening the Pull Request and starting the
discussion for the review process. I also want to thank publicly all
the people that somehow contributed to this:
- Mark Shields and the original people at google who worked at nexmark
for contributing this in the first place.
-
I think these are valuable enough that we should get them into apache/master
On Fri, May 12, 2017 at 4:34 AM, Jean-Baptiste Onofré
wrote:
> Hi,
>
> PR or even a feature branch could work. Up to you.
>
> Regards
> JB
>
>
> On 05/12/2017 10:55 AM, Etienne Chauchot wrote:
>
>> Hi guys,
>>
>> I want
Hi,
PR or even a feature branch could work. Up to you.
Regards
JB
On 05/12/2017 10:55 AM, Etienne Chauchot wrote:
Hi guys,
I wanted to let you know that I have just submitted a PR around NexMark. This is
a port of the NexMark queries to Beam, to be used as integration tests.
This can also be
Hi guys,
I wanted to let you know that I have just submitted a PR around NexMark.
This is a port of the NexMark queries to Beam, to be used as integration
tests.
This can also be used as A-B testing (no-regression or performance
comparison between 2 versions of the same engine or of the same r
This is great! Having a variety of realistic-ish pipelines running on all
runners complements the validation suite and IO IT work.
If I recall, some of these involve heavy and esoteric uses of state, so
definitely give me a ping if you hit any trouble.
Kenn
On Tue, Mar 21, 2017 at 9:38 AM, Etien
Hi Etienne,
That's a great news and good job !
By "having Nexmark on Beam", I guess you mean the translation of the NEXMark
queries in Beam, not NEXMark itself, right ?
If you mean the later, I'm not sure as NEXMark is not Beam related (it's more
generic) and it could be tricky in terms of l
Not a deep response, but this is awesome! We'd really like to have some
good benchmarks, and I'm excited you're updating Nexmark. This will be
great!
On Tue, Mar 21, 2017 at 9:38 AM, Etienne Chauchot
wrote:
> Hi all,
>
> Ismael and I are working on upgrading the Nexmark implementation for Beam.
Hi all,
Ismael and I are working on upgrading the Nexmark implementation for
Beam. See https://github.com/iemejia/beam/tree/BEAM-160-nexmark and
https://issues.apache.org/jira/browse/BEAM-160. We are continuing the
work done by Mark Shields. See https://github.com/apache/beam/pull/366
for the
23 matches
Mail list logo
