y. It looks like this is due to a transitive
dependency.
Diffs
-
samza-yarn/src/main/java/org/apache/samza/job/yarn/SamzaTaskManager.java
c2a9ac0740101ce9dc73f80220dc6ec828407409
Diff: https://reviews.apache.org/r/38832/diff/
Testing
---
Thanks,
Jake Maes
rn/src/test/java/org/apache/samza/job/yarn/util/MockContainerListener.java
PRE-CREATION
samza-yarn/src/test/java/org/apache/samza/job/yarn/util/MockContainerRequestState.java
PRE-CREATION
Diff: https://reviews.apache.org/r/38966/diff/
Testing
---
./gradlew clean samza-yarn:test
Th
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39355/#review102824
---
Ship it!
Ship It!
- Jake Maes
On Oct. 15, 2015, 6:58 p.m
-management.md (line 227)
<https://reviews.apache.org/r/39252/#comment161125>
Thanks for documenting this.
- Jake Maes
On Oct. 16, 2015, 11:44 p.m., Navina Ramesh wrote:
>
> ---
> This is an automatically generated e-mail.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39806/#review104814
---
Ship it!
Ship It!
- Jake Maes
On Nov. 2, 2015, 9:56 p.m
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40857/#review108656
---
Ship it!
- Jake Maes
On Dec. 2, 2015, 7:53 a.m., Aleksandar
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40934/#review108912
---
Ship it!
- Jake Maes
On Dec. 3, 2015, 11:37 p.m., Jagadish
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/40933/#review108915
---
Ship it!
Ship It!
- Jake Maes
On Dec. 3, 2015, 11:10 p.m., Yi
/ProcessJobFactory.scala
(line 38)
<https://reviews.apache.org/r/41106/#comment169146>
nit: this message will no longer be valid after we fully deprecate
yarn.container.count. It would be more future-proof to say "Container count
larger than 1..."
- Jake Maes
On Dec. 9, 2015
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41106/#review109687
---
Ship it!
Ship It!
- Jake Maes
On Dec. 10, 2015, 4:34 a.m., Tao
> On Jan. 5, 2016, 12:21 a.m., Jake Maes wrote:
> > Ship It!
Any idea why the checkstyle behavior changed with the newer gradle?
- Jake
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.or
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41874/#review112705
---
Ship it!
Ship It!
- Jake Maes
On Jan. 4, 2016, 9:59 a.m
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41912/#review112970
---
Ship it!
Ship It!
- Jake Maes
On Jan. 5, 2016, 1:21 a.m
ab3061eae2cfc2da6681ce2034492b165d0d8b96
samza-yarn/src/main/java/org/apache/samza/job/yarn/HostAwareContainerAllocator.java
ff22dbfe5537ba2a3c55c4e6063680ff4c9e55f4
Diff: https://reviews.apache.org/r/43074/diff/
Testing
---
Thanks,
Jake Maes
/
Testing (updated)
---
Unit tests still pass.
Thanks,
Jake Maes
/HostAwareContainerAllocator.java
ff22dbfe5537ba2a3c55c4e6063680ff4c9e55f4
Diff: https://reviews.apache.org/r/43074/diff/
Testing
---
Unit tests still pass.
Thanks,
Jake Maes
66114aabf76a36d3b7ca4c5810628e0c77
>
> samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerRequestState.java
> ab3061eae2cfc2da6681ce2034492b165d0d8b96
>
> samza-yarn/src/main/java/org/apache/samza/job/yarn/HostAwareContainerAllocator.java
> ff22dbfe5537ba2a3c55c4e6063680ff4c9e55f4
>
> Diff: https://reviews.apache.org/r/43074/diff/
>
>
> Testing
> ---
>
> Unit tests still pass.
>
>
> Thanks,
>
> Jake Maes
>
>
://reviews.apache.org/r/43074/diff/
Testing
---
Unit tests still pass.
Thanks,
Jake Maes
hould notify the
AM which will try to restart the container on the same host. It will get a
connection error and at that point should retry on a DIFFERENT host.
Thanks,
Jake Maes
ediately after launch, just that we were able to instruct the NM to
launch it).
If we can't reach any node on the cluster, there's something horribly wrong and
our (AM's) container is likely to go down next.
If the necessary resources aren't available, the RM won't all
Host Affinity. When the RM detects the outage, it should notify the
AM which will try to restart the container on the same host. It will get a
connection error and at that point should retry on a DIFFERENT host.
Thanks,
Jake Maes
now.
- Jake
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43350/#review118604
---
On Feb. 10, 2016, 2:40 a.m., J
--
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43350/#review118610
---
On Feb. 10, 2016, 2:40 a.m., Jake Maes wrote:
>
> ---
Host Affinity. When the RM detects the outage, it should notify the
AM which will try to restart the container on the same host. It will get a
connection error and at that point should retry on a DIFFERENT host.
Thanks,
Jake Maes
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43550/#review119120
---
Ship it!
Ship It!
- Jake Maes
On Feb. 13, 2016, 12:13 a.m
--
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43350/#review118935
---
On Feb. 10, 2016, 3:57 p.m., Jake Maes wrote:
>
> ---
eExceptions are). It should be handled
explicitly.
- Jake
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43350/#review118935
---
On
and running the container together, you should never do
one without the other.
- Jake
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43350/#review119488
-----
uses Host Affinity. When the RM detects the outage, it should notify the
AM which will try to restart the container on the same host. It will get a
connection error and at that point should retry on a DIFFERENT host.
Thanks,
Jake Maes
l get a
connection error and at that point should retry on a DIFFERENT host.
Thanks,
Jake Maes
kill the NM for one of the containers of a job
that uses Host Affinity. When the RM detects the outage, it should notify the
AM which will try to restart the container on the same host. It will get a
connection error and at that point should retry on a DIFFERENT host.
Thanks,
Jake Maes
like "neededContainers" are
correct throughout the test.
Thanks,
Jake Maes
good hosts
(since it is already aware of the bad hosts)
e. Container is started on a good host.
In both cases, the job stays alive and the full set of containers is eventually
running on good hosts. Also, the state counters, like "neededContainers" are
correct throughout the test.
function to the getOrElse() method from
Java.
- Jake
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43350/#review120393
------
54db5e5b2b1b109d202e814809adbfd2bc84fb4b
samza-yarn/src/main/java/org/apache/samza/job/yarn/ContainerUtil.java
91fae98c074e1648e7168fb8e76d6e1e656816fc
Diff: https://reviews.apache.org/r/44241/diff/
Testing
---
Manual testing and log verification (made sure they weren't verbose)
Thanks,
Jake Maes
/
Testing
---
Manual testing and log verification (made sure they weren't verbose)
Thanks,
Jake Maes
og message from Kafka
33 | "ContainerRequestState [INFO] Got a new container: ..." | Approx equal to
the number of containers in the normal case. All other logs updated in
ContainerRequestState will be of similar or lower frequency.
Thanks,
Jake Maes
testing and log verification (made sure they weren't verbose)
Thanks,
Jake Maes
------
On March 2, 2016, 2:24 a.m., Jake Maes wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/44241/
>
://reviews.apache.org/r/44293/diff/
Testing
---
Manual testing and log verification (made sure they weren't verbose)
Thanks,
Jake Maes
2f9669f8b7e77abb65b244ccd067ae7ab1f245c3
Diff: https://reviews.apache.org/r/44772/diff/
Testing
---
Verified that the tests fail without the logic fix and pass with it.
I think I fixed some of the sporadic failures in the tests too. Ran them many
times to confirm.
Thanks,
Jake Maes
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44775/#review123477
---
Ship it!
Ship It!
- Jake Maes
On March 14, 2016, 6:02 a.m
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44820/#review123572
---
Ship it!
Ship It!
- Jake Maes
On March 15, 2016, 1:06 a.m
: https://reviews.apache.org/r/44866/diff/
Testing
---
Built with java 7
Also searched the code to make sure there was no other usage of getOrDefault()
Thanks,
Jake Maes
/44920/#comment186211>
Same terminology question
- Jake Maes
On March 16, 2016, 6:23 p.m., Jagadish Venkatraman wrote:
>
> ---
> This is an automatically genera
introduced in SAMZA-867
Diffs (updated)
-
samza-yarn/src/main/java/org/apache/samza/job/yarn/HostAwareContainerAllocator.java
979719687be864bf24354aea0a7dc51b5f11a712
Diff: https://reviews.apache.org/r/44772/diff/
Testing (updated)
---
Verified with a job.
Thanks,
Jake Maes
.
Thanks,
Jake Maes
https://reviews.apache.org/r/45136/diff/
Testing
---
New unit tests and a test job. With the test job set to autocommit, I was able
to see the OFFSET file get written every commit.
Thanks,
Jake Maes
xpected to reduce task
reassignment as the container count was adjusted from 4->3->5 on subsequent
runs.
Thanks,
Jake Maes
: https://reviews.apache.org/r/45136/diff/
Testing
---
New unit tests and a test job. With the test job set to autocommit, I was able
to see the OFFSET file get written every commit.
Thanks,
Jake Maes
modelled according to the LocalityManager that
> obtains the container locality. Never mind.
There are some comments to that effect in the unit tests. I could copy them
over, but they sort of dominate the file.
About immutability; the one way to do it is to remove the map
ly missing) is added the
first time the job is run. It is then used as expected to reduce task
reassignment as the container count was adjusted from 4->3->5 on subsequent
runs.
Thanks,
Jake Maes
below.
samza-core/src/main/scala/org/apache/samza/coordinator/StreamPartitionCountMonitor.scala
(line 67)
<https://reviews.apache.org/r/44405/#comment187850>
This exception shouldn't just be caught:
http://www.yegor256.com/2015/10/20/interrupted-exception.html
- Jake Maes
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45246/#review125112
---
Ship it!
Ship It!
- Jake Maes
On March 23, 2016, 7:58 p.m
/diff/
Testing
---
New unit tests and a test job. With the test job set to autocommit, I was able
to see the OFFSET file get written every commit.
Thanks,
Jake Maes
s adjusted from 4->3->5 on subsequent
runs.
Thanks,
Jake Maes
xpected to reduce task
reassignment as the container count was adjusted from 4->3->5 on subsequent
runs.
Thanks,
Jake Maes
teTaskCountPerContainer()
I'll add an assertion to make sure.
> On March 24, 2016, 11:13 p.m., Yi Pan (Data Infrastructure) wrote:
> > samza-core/src/main/java/org/apache/samza/container/grouper/task/GroupByContainerCount.java,
> > line 231
> > <
A bunch of new unit tests have been added.
Also tested with a test job. The task mapping (initially missing) is added the
first time the job is run. It is then used as expected to reduce task
reassignment as the container count was adjusted from 4->3->5 on subsequent
runs.
Thanks,
Jake Maes
227>
> >
> > What if taskNamesToAssign.isEmpty()==true here?
>
> Jake Maes wrote:
> It can't if the math is right in calculateTaskCountPerContainer()
>
> I'll add an assertion to make sure.
Actually I feel like an assertion is going to significantly harm re
unit tests have been added.
Also tested with a test job. The task mapping (initially missing) is added the
first time the job is run. It is then used as expected to reduce task
reassignment as the container count was adjusted from 4->3->5 on subsequent
runs.
Thanks,
Jake Maes
unit tests have been added.
Also tested with a test job. The task mapping (initially missing) is added the
first time the job is run. It is then used as expected to reduce task
reassignment as the container count was adjusted from 4->3->5 on subsequent
runs.
Thanks,
Jake Maes
ml
But I made it explicit for readability
- Jake
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45144/#review125426
----
,
Jake Maes
rg/apache/samza/coordinator/stream/TestCoordinatorStreamSystemConsumer.java
0e73e18bd55e343e1a5122be7e8f3c666b797dc5
Diff: https://reviews.apache.org/r/45339/diff/
Testing
---
Added a unit test which fails before the change and passes after.
Ran check-all.sh
This patch fixed my test job fo
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45144/#review125481
---
On March 25, 2016, 5:30 p.m., Jake Maes wrote:
>
>
-yarn/src/test/java/org/apache/samza/job/yarn/util/MockContainerUtil.java
2f9669f8b7e77abb65b244ccd067ae7ab1f245c3
Diff: https://reviews.apache.org/r/45190/diff/
Testing
---
Thanks,
Jake Maes
function. This
> > is deviation that I think is not necessary.
> >
> > The TaskAssignmentManager could be instantiated in JobCoordinator and
> > passed to the "balance" call. Do we really need to couple them together?
>
> Jake Maes wrote:
> Explained abov
function. This
> > is deviation that I think is not necessary.
> >
> > The TaskAssignmentManager could be instantiated in JobCoordinator and
> > passed to the "balance" call. Do we really need to couple them together?
>
> Jake Maes wrote:
> Explained abov
function. This
> > is deviation that I think is not necessary.
> >
> > The TaskAssignmentManager could be instantiated in JobCoordinator and
> > passed to the "balance" call. Do we really need to couple them together?
>
> Jake Maes wrote:
> Explained abov
function. This
> > is deviation that I think is not necessary.
> >
> > The TaskAssignmentManager could be instantiated in JobCoordinator and
> > passed to the "balance" call. Do we really need to couple them together?
>
> Jake Maes wrote:
> Explained abov
/resources/log4j-console.xml
a6ee7eec9cb94050a7122321bf7f06eb4b681040
samza-test/src/main/resources/log4j.xml
f93e4dd889d9b20ad05502cd69206b2d681ef159
Diff: https://reviews.apache.org/r/45388/diff/
Testing
---
Thanks,
Jake Maes
---
Add milliseconds and thread name to log4j config
Diffs
-
src/main/resources/log4j.xml 129149ffe84c9605d929deb8edec553955535c34
Diff: https://reviews.apache.org/r/45387/diff/
Testing
---
Thanks,
Jake Maes
.314 [main] AbstractConnector [INFO] Started
SelectChannelConnector@0.0.0.0:54065
2016-03-28 11:29:32.314 [main] HttpServer [INFO] Started HttpServer on:
http://jmaes-mn1:54065/
Thanks,
Jake Maes
ty is false, these exceptions are much more
problematic.
Thanks,
Jake Maes
---
Thanks,
Jake Maes
subsequent
runs.
Thanks,
Jake Maes
---
Thanks,
Jake Maes
/util/MockContainerUtil.java
2f9669f8b7e77abb65b244ccd067ae7ab1f245c3
Diff: https://reviews.apache.org/r/45190/diff/
Testing
---
Thanks,
Jake Maes
hanks,
Jake Maes
o I couldn't push all the assertions to the
main thread, but the latch still makes it cleaner and more reliable.
- Jake
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45190/#r
x27;t see any sporadic
failures anymore.
Thanks,
Jake Maes
should have doc. Many of these are pretty self
explanatory, but a little doc can still help.
samza-core/src/main/java/org/apache/samza/config/ClusterManagerConfig.java
(line 107)
<https://reviews.apache.org/r/44920/#comment191526>
Same nit: javadoc for public methods, just to hel
the condition being satisfied, the assertions are run, but
the 2 behaviors should not be conflated.
- Jake
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45190/#review12818
x27;t see any sporadic
failures anymore.
Thanks,
Jake Maes
eviews.apache.org/r/45464/#review128508
---
On March 30, 2016, 12:30 a.m., Jake Maes wrote:
>
> ---
> This is an automatically generated e-mail. To re
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45601/#review128501
-------
On April 1, 2016, 7:49 p.m., Jake Maes wrote:
>
> -
;. It is not clear from the context without
> > reading the SAMZA-886.
>
> Jake Maes wrote:
> I don't want to duplicate the hadoop documentation that is linked in the
> first sentence, so can you please specify what is unclear?
>
> Would it help if I said "D
: https://reviews.apache.org/r/45601/diff/
Testing
---
Thanks,
Jake Maes
,
Jake Maes
: samza
Description
---
test commit
Diffs (updated)
-
build.gradle 16facbbf4dff378c561461786ff186bd9eed
Diff: https://reviews.apache.org/r/46129/diff/
Testing
---
Thanks,
Jake Maes
Diffs
-
build.gradle 16facbbf4dff378c561461786ff186bd9eed
Diff: https://reviews.apache.org/r/46130/diff/
Testing
---
Thanks,
Jake Maes
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45258/#review128910
---
Ship it!
Ship It!
- Jake Maes
On March 24, 2016, 1:10 a.m
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46287/#review129210
---
Ship it!
Ship It!
- Jake Maes
On April 15, 2016, 11:17 p.m
ike we might be able to omit the serverPort
altogether and simplify the JMXServiceURL to
"service:jmx:rmi:///jndi/rmi://localhost:" + registryPort + "/jmxrmi"
http://stackoverflow.com/questions/2768087/explain-jmx-url
This needs to be tested, though.
- Jake M
://reviews.apache.org/r/46546/diff/
Testing
---
Thanks,
Jake Maes
Testing (updated)
---
Ran the test a number of times and haven't seen the assertion fail.
Thanks,
Jake Maes
you'll see that it has in fact changed, all the
way down to the native method that actually kills the process.
- Jake
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org
scala
f47f8189bd92c4071ae76ae323e066823f3a6f61
Diff: https://reviews.apache.org/r/46856/diff/
Testing
---
Added a test.
Ran check-all.sh
Thanks,
Jake Maes
ll.sh
Thanks,
Jake Maes
1 - 100 of 279 matches
Mail list logo
