[mllib] Share the simple benchmark result about the cast cost from Spark vector to Breeze vector
Hi all, I wondered the cast cost from Spark Vectors to Breeze vector is high or low. So I benchmarked the simple operation about addition, multiplication and division of RDD[Vector] or RDD[BV[Double]]. I share the simple benchmark result with you. In conclusion, the cast cost was lower than I had expected. For more information, please read the below report, if you are interested in it. https://github.com/yu-iskw/benchmark-breeze-on-spark/blob/master/doc%2Fbenchmark-result.md Best, Yu Ishikawa - -- Yu Ishikawa -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Share-the-simple-benchmark-result-about-the-cast-cost-from-Spark-vector-to-Breeze-vector-tp8793.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: Unit testing Master-Worker Message Passing
What's happening when I do this is that the Worker tries to get the Master actor by calling context.actorSelection(), and the RegisterWorker message gets sent to the dead letters mailbox instead of being picked up by expectMsg. I'm new to Akka and I've tried various ways to registering a mock master to no avail. I would think there would be at least some kind of test for master - worker message passing, no? On Wed, Oct 15, 2014 at 11:28 AM, Nan Zhu zhunanmcg...@gmail.com wrote: I don’t think there are test cases for Worker itself You can val actorRef = TestActorRef[Master](Props(classOf[Master], ...))( actorSystem) actorRef.underlyingActor.receive(Heartbeat) and use expectMsg to test if Master can reply correct message by assuming Worker is absolutely correct Then in another test case to test if Worker can send register message to Master after receiving Master’s “re-register” instruction, (in this test case assuming that the Master is absolutely right) Best, -- Nan Zhu On Wednesday, October 15, 2014 at 2:04 PM, Matthew Cheah wrote: Thanks, the example was helpful. However, testing the Worker itself is a lot more complicated than WorkerWatcher, since the Worker class is quite a bit more complex. Are there any tests that inspect the Worker itself? Thanks, -Matt Cheah On Tue, Oct 14, 2014 at 6:40 PM, Nan Zhu zhunanmcg...@gmail.com wrote: You can use akka testkit Example: https://github.com/apache/spark/blob/ef4ff00f87a4e8d38866f163f01741c2673e41da/core/src/test/scala/org/apache/spark/deploy/worker/WorkerWatcherSuite.scala -- Nan Zhu On Tuesday, October 14, 2014 at 9:17 PM, Matthew Cheah wrote: Hi everyone, I’m adding some new message passing between the Master and Worker actors in order to address https://issues.apache.org/jira/browse/SPARK-3736 . I was wondering if these kinds of interactions are tested in the automated Jenkins test suite, and if so, where I could find some examples to help me do the same. Thanks! -Matt Cheah
short jenkins downtime -- trying to get to the bottom of the git fetch timeouts
i'm going to be downgrading our git plugin (from 2.2.7 to 2.2.2) to see if that helps w/the git fetch timeouts. this will require a short downtime (~20 mins for builds to finish, ~20 mins to downgrade), and will hopefully give us some insight in to wtf is going on. thanks for your patience... shane
Re: short jenkins downtime -- trying to get to the bottom of the git fetch timeouts
I support this effort. :thumbsup: On Wed, Oct 15, 2014 at 4:52 PM, shane knapp skn...@berkeley.edu wrote: i'm going to be downgrading our git plugin (from 2.2.7 to 2.2.2) to see if that helps w/the git fetch timeouts. this will require a short downtime (~20 mins for builds to finish, ~20 mins to downgrade), and will hopefully give us some insight in to wtf is going on. thanks for your patience... shane
Re: Unit testing Master-Worker Message Passing
You can call resolve method on ActorSelection.resolveOne() to see if the actor is still there or the path is correct. The method returns a future and you can wait for it with timeout. This way, you know the actor is live or already dead or incorrect. Another way, is to send Identify method to ActorSystem, if it returns with correct identified message; then you can act on it, otherwise, ... hope this helps Chester On Wed, Oct 15, 2014 at 1:38 PM, Matthew Cheah matthew.c.ch...@gmail.com wrote: What's happening when I do this is that the Worker tries to get the Master actor by calling context.actorSelection(), and the RegisterWorker message gets sent to the dead letters mailbox instead of being picked up by expectMsg. I'm new to Akka and I've tried various ways to registering a mock master to no avail. I would think there would be at least some kind of test for master - worker message passing, no? On Wed, Oct 15, 2014 at 11:28 AM, Nan Zhu zhunanmcg...@gmail.com wrote: I don’t think there are test cases for Worker itself You can val actorRef = TestActorRef[Master](Props(classOf[Master], ...))( actorSystem) actorRef.underlyingActor.receive(Heartbeat) and use expectMsg to test if Master can reply correct message by assuming Worker is absolutely correct Then in another test case to test if Worker can send register message to Master after receiving Master’s “re-register” instruction, (in this test case assuming that the Master is absolutely right) Best, -- Nan Zhu On Wednesday, October 15, 2014 at 2:04 PM, Matthew Cheah wrote: Thanks, the example was helpful. However, testing the Worker itself is a lot more complicated than WorkerWatcher, since the Worker class is quite a bit more complex. Are there any tests that inspect the Worker itself? Thanks, -Matt Cheah On Tue, Oct 14, 2014 at 6:40 PM, Nan Zhu zhunanmcg...@gmail.com wrote: You can use akka testkit Example: https://github.com/apache/spark/blob/ef4ff00f87a4e8d38866f163f01741c2673e41da/core/src/test/scala/org/apache/spark/deploy/worker/WorkerWatcherSuite.scala -- Nan Zhu On Tuesday, October 14, 2014 at 9:17 PM, Matthew Cheah wrote: Hi everyone, I’m adding some new message passing between the Master and Worker actors in order to address https://issues.apache.org/jira/browse/SPARK-3736 . I was wondering if these kinds of interactions are tested in the automated Jenkins test suite, and if so, where I could find some examples to help me do the same. Thanks! -Matt Cheah
Re: short jenkins downtime -- trying to get to the bottom of the git fetch timeouts
ok, we're up and building... :crossesfingersfortheumpteenthtime: On Wed, Oct 15, 2014 at 1:59 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: I support this effort. :thumbsup: On Wed, Oct 15, 2014 at 4:52 PM, shane knapp skn...@berkeley.edu wrote: i'm going to be downgrading our git plugin (from 2.2.7 to 2.2.2) to see if that helps w/the git fetch timeouts. this will require a short downtime (~20 mins for builds to finish, ~20 mins to downgrade), and will hopefully give us some insight in to wtf is going on. thanks for your patience... shane -- You received this message because you are subscribed to the Google Groups amp-infra group. To unsubscribe from this group and stop receiving emails from it, send an email to amp-infra+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: short jenkins downtime -- trying to get to the bottom of the git fetch timeouts
four builds triggered and no timeouts. :crossestoes: :) On Wed, Oct 15, 2014 at 2:19 PM, shane knapp skn...@berkeley.edu wrote: ok, we're up and building... :crossesfingersfortheumpteenthtime: On Wed, Oct 15, 2014 at 1:59 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: I support this effort. :thumbsup: On Wed, Oct 15, 2014 at 4:52 PM, shane knapp skn...@berkeley.edu wrote: i'm going to be downgrading our git plugin (from 2.2.7 to 2.2.2) to see if that helps w/the git fetch timeouts. this will require a short downtime (~20 mins for builds to finish, ~20 mins to downgrade), and will hopefully give us some insight in to wtf is going on. thanks for your patience... shane -- You received this message because you are subscribed to the Google Groups amp-infra group. To unsubscribe from this group and stop receiving emails from it, send an email to amp-infra+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Unit testing Master-Worker Message Passing
I think on a higher level I also want to ask why such unit testing has not actually been done in this codebase. If it's not a common practice to test message passing then I'm fine with leaving out the unit test, however I'm more curious as to why such testing was not done before. On Wed, Oct 15, 2014 at 2:18 PM, Chester Chen ches...@alpinenow.com wrote: You can call resolve method on ActorSelection.resolveOne() to see if the actor is still there or the path is correct. The method returns a future and you can wait for it with timeout. This way, you know the actor is live or already dead or incorrect. Another way, is to send Identify method to ActorSystem, if it returns with correct identified message; then you can act on it, otherwise, ... hope this helps Chester On Wed, Oct 15, 2014 at 1:38 PM, Matthew Cheah matthew.c.ch...@gmail.com wrote: What's happening when I do this is that the Worker tries to get the Master actor by calling context.actorSelection(), and the RegisterWorker message gets sent to the dead letters mailbox instead of being picked up by expectMsg. I'm new to Akka and I've tried various ways to registering a mock master to no avail. I would think there would be at least some kind of test for master - worker message passing, no? On Wed, Oct 15, 2014 at 11:28 AM, Nan Zhu zhunanmcg...@gmail.com wrote: I don’t think there are test cases for Worker itself You can val actorRef = TestActorRef[Master](Props(classOf[Master], ...))( actorSystem) actorRef.underlyingActor.receive(Heartbeat) and use expectMsg to test if Master can reply correct message by assuming Worker is absolutely correct Then in another test case to test if Worker can send register message to Master after receiving Master’s “re-register” instruction, (in this test case assuming that the Master is absolutely right) Best, -- Nan Zhu On Wednesday, October 15, 2014 at 2:04 PM, Matthew Cheah wrote: Thanks, the example was helpful. However, testing the Worker itself is a lot more complicated than WorkerWatcher, since the Worker class is quite a bit more complex. Are there any tests that inspect the Worker itself? Thanks, -Matt Cheah On Tue, Oct 14, 2014 at 6:40 PM, Nan Zhu zhunanmcg...@gmail.com wrote: You can use akka testkit Example: https://github.com/apache/spark/blob/ef4ff00f87a4e8d38866f163f01741c2673e41da/core/src/test/scala/org/apache/spark/deploy/worker/WorkerWatcherSuite.scala -- Nan Zhu On Tuesday, October 14, 2014 at 9:17 PM, Matthew Cheah wrote: Hi everyone, I’m adding some new message passing between the Master and Worker actors in order to address https://issues.apache.org/jira/browse/SPARK-3736 . I was wondering if these kinds of interactions are tested in the automated Jenkins test suite, and if so, where I could find some examples to help me do the same. Thanks! -Matt Cheah
Re: Unit testing Master-Worker Message Passing
There are some end-to-end integration tests of Master - Worker fault-tolerance in https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/FaultToleranceTest.scala I’ve actually been working to develop a more generalized Docker-based integration-testing framework for Spark in order to test Master - Worker interactions. I’d like to eventually clean up my code and release it publicly. On October 15, 2014 at 2:39:22 PM, Matthew Cheah (matthew.c.ch...@gmail.com) wrote: I think on a higher level I also want to ask why such unit testing has not actually been done in this codebase. If it's not a common practice to test message passing then I'm fine with leaving out the unit test, however I'm more curious as to why such testing was not done before. On Wed, Oct 15, 2014 at 2:18 PM, Chester Chen ches...@alpinenow.com wrote: You can call resolve method on ActorSelection.resolveOne() to see if the actor is still there or the path is correct. The method returns a future and you can wait for it with timeout. This way, you know the actor is live or already dead or incorrect. Another way, is to send Identify method to ActorSystem, if it returns with correct identified message; then you can act on it, otherwise, ... hope this helps Chester On Wed, Oct 15, 2014 at 1:38 PM, Matthew Cheah matthew.c.ch...@gmail.com wrote: What's happening when I do this is that the Worker tries to get the Master actor by calling context.actorSelection(), and the RegisterWorker message gets sent to the dead letters mailbox instead of being picked up by expectMsg. I'm new to Akka and I've tried various ways to registering a mock master to no avail. I would think there would be at least some kind of test for master - worker message passing, no? On Wed, Oct 15, 2014 at 11:28 AM, Nan Zhu zhunanmcg...@gmail.com wrote: I don’t think there are test cases for Worker itself You can val actorRef = TestActorRef[Master](Props(classOf[Master], ...))( actorSystem) actorRef.underlyingActor.receive(Heartbeat) and use expectMsg to test if Master can reply correct message by assuming Worker is absolutely correct Then in another test case to test if Worker can send register message to Master after receiving Master’s “re-register” instruction, (in this test case assuming that the Master is absolutely right) Best, -- Nan Zhu On Wednesday, October 15, 2014 at 2:04 PM, Matthew Cheah wrote: Thanks, the example was helpful. However, testing the Worker itself is a lot more complicated than WorkerWatcher, since the Worker class is quite a bit more complex. Are there any tests that inspect the Worker itself? Thanks, -Matt Cheah On Tue, Oct 14, 2014 at 6:40 PM, Nan Zhu zhunanmcg...@gmail.com wrote: You can use akka testkit Example: https://github.com/apache/spark/blob/ef4ff00f87a4e8d38866f163f01741c2673e41da/core/src/test/scala/org/apache/spark/deploy/worker/WorkerWatcherSuite.scala -- Nan Zhu On Tuesday, October 14, 2014 at 9:17 PM, Matthew Cheah wrote: Hi everyone, I’m adding some new message passing between the Master and Worker actors in order to address https://issues.apache.org/jira/browse/SPARK-3736 . I was wondering if these kinds of interactions are tested in the automated Jenkins test suite, and if so, where I could find some examples to help me do the same. Thanks! -Matt Cheah
Re: Unit testing Master-Worker Message Passing
Thanks Josh! These tests seem to cover the cases I'm looking for already =). What's interesting though is that we still ran into SPARK-3736 despite such integration tests being in place to catch it - specifically, the case when the master disconnects and reconnects, the workers should reconnect to the master after the master restarts. Are the tests here run regularly, i.e. Jenkins build or nightly build, and if so how did that test case pass while SPARK-3736 apparently still exists? At any rate, I think I'll submit my fix PR but with no particular extra automated test written for it, since it seems like FaultToleranceTest sufficiently covers what I need. On Wed, Oct 15, 2014 at 2:42 PM, Josh Rosen rosenvi...@gmail.com wrote: There are some end-to-end integration tests of Master - Worker fault-tolerance in https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/FaultToleranceTest.scala I’ve actually been working to develop a more generalized Docker-based integration-testing framework for Spark in order to test Master - Worker interactions. I’d like to eventually clean up my code and release it publicly. On October 15, 2014 at 2:39:22 PM, Matthew Cheah ( matthew.c.ch...@gmail.com) wrote: I think on a higher level I also want to ask why such unit testing has not actually been done in this codebase. If it's not a common practice to test message passing then I'm fine with leaving out the unit test, however I'm more curious as to why such testing was not done before. On Wed, Oct 15, 2014 at 2:18 PM, Chester Chen ches...@alpinenow.com wrote: You can call resolve method on ActorSelection.resolveOne() to see if the actor is still there or the path is correct. The method returns a future and you can wait for it with timeout. This way, you know the actor is live or already dead or incorrect. Another way, is to send Identify method to ActorSystem, if it returns with correct identified message; then you can act on it, otherwise, ... hope this helps Chester On Wed, Oct 15, 2014 at 1:38 PM, Matthew Cheah matthew.c.ch...@gmail.com wrote: What's happening when I do this is that the Worker tries to get the Master actor by calling context.actorSelection(), and the RegisterWorker message gets sent to the dead letters mailbox instead of being picked up by expectMsg. I'm new to Akka and I've tried various ways to registering a mock master to no avail. I would think there would be at least some kind of test for master - worker message passing, no? On Wed, Oct 15, 2014 at 11:28 AM, Nan Zhu zhunanmcg...@gmail.com wrote: I don’t think there are test cases for Worker itself You can val actorRef = TestActorRef[Master](Props(classOf[Master], ...))( actorSystem) actorRef.underlyingActor.receive(Heartbeat) and use expectMsg to test if Master can reply correct message by assuming Worker is absolutely correct Then in another test case to test if Worker can send register message to Master after receiving Master’s “re-register” instruction, (in this test case assuming that the Master is absolutely right) Best, -- Nan Zhu On Wednesday, October 15, 2014 at 2:04 PM, Matthew Cheah wrote: Thanks, the example was helpful. However, testing the Worker itself is a lot more complicated than WorkerWatcher, since the Worker class is quite a bit more complex. Are there any tests that inspect the Worker itself? Thanks, -Matt Cheah On Tue, Oct 14, 2014 at 6:40 PM, Nan Zhu zhunanmcg...@gmail.com wrote: You can use akka testkit Example: https://github.com/apache/spark/blob/ef4ff00f87a4e8d38866f163f01741c2673e41da/core/src/test/scala/org/apache/spark/deploy/worker/WorkerWatcherSuite.scala -- Nan Zhu On Tuesday, October 14, 2014 at 9:17 PM, Matthew Cheah wrote: Hi everyone, I’m adding some new message passing between the Master and Worker actors in order to address https://issues.apache.org/jira/browse/SPARK-3736 . I was wondering if these kinds of interactions are tested in the automated Jenkins test suite, and if so, where I could find some examples to help me do the same. Thanks! -Matt Cheah
Re: short jenkins downtime -- trying to get to the bottom of the git fetch timeouts
ok, we've had about 10 spark pull request builds go through w/o any git timeouts. it seems that the git timeout issue might be licked. i will be definitely be keeping an eye on this for the next few days. thanks for being patient! shane On Wed, Oct 15, 2014 at 2:27 PM, shane knapp skn...@berkeley.edu wrote: four builds triggered and no timeouts. :crossestoes: :) On Wed, Oct 15, 2014 at 2:19 PM, shane knapp skn...@berkeley.edu wrote: ok, we're up and building... :crossesfingersfortheumpteenthtime: On Wed, Oct 15, 2014 at 1:59 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: I support this effort. :thumbsup: On Wed, Oct 15, 2014 at 4:52 PM, shane knapp skn...@berkeley.edu wrote: i'm going to be downgrading our git plugin (from 2.2.7 to 2.2.2) to see if that helps w/the git fetch timeouts. this will require a short downtime (~20 mins for builds to finish, ~20 mins to downgrade), and will hopefully give us some insight in to wtf is going on. thanks for your patience... shane -- You received this message because you are subscribed to the Google Groups amp-infra group. To unsubscribe from this group and stop receiving emails from it, send an email to amp-infra+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Issues with ALS positive definite
Hi,
If I take the Movielens data and run the default ALS with regularization as
0.0, I am hitting exception from LAPACK that the gram matrix is not
positive definite. This is on the master branch.
This is how I run it :
./bin/spark-submit --total-executor-cores 1 --master spark://
tusca09lmlvt00c.uswin.ad.vzwcorp.com:7077 --jars
/Users/v606014/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar
--class org.apache.spark.examples.mllib.MovieLensALS
./examples/target/spark-examples_2.10-1.1.0-SNAPSHOT.jar --rank 20
--numIterations 20 --lambda 0.0 --kryo
hdfs://localhost:8020/sandbox/movielens/
Error from LAPACK:
WARN TaskSetManager: Lost task 0.0 in stage 11.0 (TID 22,
tusca09lmlvt00c.uswin.ad.vzwcorp.com):
org.jblas.exceptions.LapackArgumentException: LAPACK DPOSV: Leading minor
of order i of A is not positive definite.
From the maths it's not expected right ?
||r - wi'hj||^{2} has to be positive definite...
I think the tests are not running any 0.0 regularization tests otherwise we
should have caught it as well...
For the sparse coding NMF variant that I am running, I have to turn off L2
regularization when I run a L1 on products to extract sparse topics...
Thanks.
Deb
Re: Issues with ALS positive definite
Hi Debaish,
I think ||r - wi'hj||^{2} is semi-positive definite.
Thanks,
Liquan
On Wed, Oct 15, 2014 at 4:57 PM, Debasish Das debasish.da...@gmail.com
wrote:
Hi,
If I take the Movielens data and run the default ALS with regularization as
0.0, I am hitting exception from LAPACK that the gram matrix is not
positive definite. This is on the master branch.
This is how I run it :
./bin/spark-submit --total-executor-cores 1 --master spark://
tusca09lmlvt00c.uswin.ad.vzwcorp.com:7077 --jars
/Users/v606014/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar
--class org.apache.spark.examples.mllib.MovieLensALS
./examples/target/spark-examples_2.10-1.1.0-SNAPSHOT.jar --rank 20
--numIterations 20 --lambda 0.0 --kryo
hdfs://localhost:8020/sandbox/movielens/
Error from LAPACK:
WARN TaskSetManager: Lost task 0.0 in stage 11.0 (TID 22,
tusca09lmlvt00c.uswin.ad.vzwcorp.com):
org.jblas.exceptions.LapackArgumentException: LAPACK DPOSV: Leading minor
of order i of A is not positive definite.
From the maths it's not expected right ?
||r - wi'hj||^{2} has to be positive definite...
I think the tests are not running any 0.0 regularization tests otherwise we
should have caught it as well...
For the sparse coding NMF variant that I am running, I have to turn off L2
regularization when I run a L1 on products to extract sparse topics...
Thanks.
Deb
--
Liquan Pei
Department of Physics
University of Massachusetts Amherst
Re: Issues with ALS positive definite
But do you expect the mllib code to fail if I run with 0.0 regularization ?
I think ||r - wi'hj||^{2} is positive definite...It can become positive
semi definite only if there are dependent rows in the matrix...
@sean is that right ? We had this discussion before as well...
On Wed, Oct 15, 2014 at 5:01 PM, Liquan Pei liquan...@gmail.com wrote:
Hi Debaish,
I think ||r - wi'hj||^{2} is semi-positive definite.
Thanks,
Liquan
On Wed, Oct 15, 2014 at 4:57 PM, Debasish Das debasish.da...@gmail.com
wrote:
Hi,
If I take the Movielens data and run the default ALS with regularization
as
0.0, I am hitting exception from LAPACK that the gram matrix is not
positive definite. This is on the master branch.
This is how I run it :
./bin/spark-submit --total-executor-cores 1 --master spark://
tusca09lmlvt00c.uswin.ad.vzwcorp.com:7077 --jars
/Users/v606014/.m2/repository/com/github/scopt/scopt_2.10/3.2.0/scopt_2.10-3.2.0.jar
--class org.apache.spark.examples.mllib.MovieLensALS
./examples/target/spark-examples_2.10-1.1.0-SNAPSHOT.jar --rank 20
--numIterations 20 --lambda 0.0 --kryo
hdfs://localhost:8020/sandbox/movielens/
Error from LAPACK:
WARN TaskSetManager: Lost task 0.0 in stage 11.0 (TID 22,
tusca09lmlvt00c.uswin.ad.vzwcorp.com):
org.jblas.exceptions.LapackArgumentException: LAPACK DPOSV: Leading minor
of order i of A is not positive definite.
From the maths it's not expected right ?
||r - wi'hj||^{2} has to be positive definite...
I think the tests are not running any 0.0 regularization tests otherwise
we
should have caught it as well...
For the sparse coding NMF variant that I am running, I have to turn off L2
regularization when I run a L1 on products to extract sparse topics...
Thanks.
Deb
--
Liquan Pei
Department of Physics
University of Massachusetts Amherst
Re: short jenkins downtime -- trying to get to the bottom of the git fetch timeouts
A quick scan through the Spark PR board https://spark-prs.appspot.com/ shows no recent failures related to this git checkout problem. Looks promising! Nick On Wed, Oct 15, 2014 at 6:10 PM, shane knapp skn...@berkeley.edu wrote: ok, we've had about 10 spark pull request builds go through w/o any git timeouts. it seems that the git timeout issue might be licked. i will be definitely be keeping an eye on this for the next few days. thanks for being patient! shane On Wed, Oct 15, 2014 at 2:27 PM, shane knapp skn...@berkeley.edu wrote: four builds triggered and no timeouts. :crossestoes: :) On Wed, Oct 15, 2014 at 2:19 PM, shane knapp skn...@berkeley.edu wrote: ok, we're up and building... :crossesfingersfortheumpteenthtime: On Wed, Oct 15, 2014 at 1:59 PM, Nicholas Chammas nicholas.cham...@gmail.com wrote: I support this effort. :thumbsup: On Wed, Oct 15, 2014 at 4:52 PM, shane knapp skn...@berkeley.edu wrote: i'm going to be downgrading our git plugin (from 2.2.7 to 2.2.2) to see if that helps w/the git fetch timeouts. this will require a short downtime (~20 mins for builds to finish, ~20 mins to downgrade), and will hopefully give us some insight in to wtf is going on. thanks for your patience... shane -- You received this message because you are subscribed to the Google Groups amp-infra group. To unsubscribe from this group and stop receiving emails from it, send an email to amp-infra+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
