@Ted, here's the old jira, HBASE-14167. Use that. On Fri, Oct 14, 2016 at 12:02 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> I just ran the tests in hbase-spark module using 'mvn verify'. > > All passed. > > I am testing a patch locally where hbase-spark tests are run in test phase. > > If the tests pass, I will log a JIRA. > > Thanks > > > On Oct 14, 2016, at 11:41 AM, Andrew Purtell <apurt...@apache.org> > wrote: > > > > The hbase-spark integration tests run (and fail) for me locally whenever > I > > build master with 'mvn clean install -DskipITs' . > > > > HBaseConnectionCacheSuite: > > - all test cases *** FAILED *** > > 2 did not equal 1 (HBaseConnectionCacheSuite.scala:92) > > > > Saw it but had to ignore/triage to get something else done. > > > > We have a weird situation where integration tests run when they shouldn't > > locally yet no tests run at all for patch process? > > > > I would like to see Spark behave like the other modules. I remember > filing > > a JIRA asking that hbase-spark honor -DskipITs. It still doesn't. > > Meanwhile, it does its own thing with '-DskipSparkTests', which is not > > appropriate given that none of the other modules have their own distinct > > control parameters. There also doesn't seem to be a distinction between > > unit tests and integration tests. The 'test' target does nothing. > > Everything happens during the 'integration-test' phase. Is this a Spark > > limitation? > > > > > >> On Fri, Oct 14, 2016 at 11:27 AM, Sean Busbey <bus...@cloudera.com> > wrote: > >> > >> Do the HBase Spark tests only run during the maven verify command? > >> We'll need to update our personality to say that that command should > >> be used for unit tests when in the hbase spark module. ugh. > >> > >> On Thu, Oct 13, 2016 at 7:42 PM, Apekshit Sharma <a...@cloudera.com> > >> wrote: > >>> Our patch process isn't running hbase-spark tests. See this for > example: > >>> > >>> https://builds.apache.org/job/PreCommit-HBASE-Build/3842/ > >>> https://builds.apache.org/job/PreCommit-HBASE-Build/3842/ > >> artifact/patchprocess/patch-unit-hbase-spark.txt/*view*/ > >>> > >>> Found it when trying to debug cause of trunk failures. Part of the > cause > >> is > >>> hbase-spark's HBaseConnectionCacheSuite test failure ( > >>> https://builds.apache.org/view/All/job/HBase-Trunk_ > >> matrix/jdk=JDK%201.8%20(latest),label=yahoo-not-h2/1776/consoleFull > >> <https://builds.apache.org/view/All/job/HBase-Trunk_ > matrix/jdk=JDK%201.8%20%28latest%29,label=yahoo-not-h2/1776/consoleFull> > >> ) > >>> which was added in HBASE-16638. However, to be fair, QA was green and > >>> reported passing hbase-spark tests for that jira. > >>> > >>>> On Mon, Sep 19, 2016 at 12:57 PM, Stack <st...@duboce.net> wrote: > >>>> > >>>> childCustomWorkspace seems to be just the ticket. Nice find Appy. > >>>> St.Ack > >>>> > >>>> On Mon, Sep 19, 2016 at 10:03 AM, Sean Busbey <bus...@cloudera.com> > >> wrote: > >>>> > >>>>> Option 2c looks to be working really well. Thanks for tackling this > >> Appy! > >>>>> > >>>>> We still have some failures on the master build, but it looks like > >>>>> actual problems (or perhaps a flakey). There are several passing > >>>>> builds. > >>>>> > >>>>> This should be pretty easy to replicate on the other jobs. I don't > see > >>>>> a downside. Anyone else have concerns? > >>>>> > >>>>> > >>>>> On Fri, Sep 16, 2016 at 6:15 PM, Apekshit Sharma <a...@cloudera.com> > >>>>> wrote: > >>>>>> So this all started with spaces-in-path issue, right? I think it > >> has > >>>>>> gobbled up a lot of time of a lot of people. > >>>>>> Let's discuss our options and try to fix it for good. Here are what > >> i > >>>> can > >>>>>> think of, and my opinion about them. > >>>>>> > >>>>>> 1. Not use matrix build > >>>>>> Temporary fix. Not preferred since not applicable to other > >>>>>> branches' builds. > >>>>>> > >>>>>> 2. Use matrix build > >>>>>> > >>>>>> a. Use tool environment trick > >>>>>> I applied this few days ago. Seemed to work until we > >>>>> discovered > >>>>>> scalatest issue. While the solution looks legitimate, we can't trust > >>>> that > >>>>>> all tools will use JAVA_HOME instead of directly using java command. > >>>>>> > >>>>>> b. Use JDK axix > >>>>>> Doesn't work right now. I don't have much idea of what's > >> the > >>>>> cost > >>>>>> for fixing it. > >>>>>> > >>>>>> c. Use JDK axis with custom child workspace > >>>>>> > >>>>>> https://github.com/jenkinsci/matrix-project-plugin/blob/ > >>>>> master/src/main/resources/hudson/matrix/MatrixProject/ > >>>>> help-childCustomWorkspace.html > >>>>>> Just found this one, and it might solve things for good. I > >>>> have > >>>>>> updated the job to use this. Let's see how it works. > >>>>>> > >>>>>> What do others think? > >>>>>> > >>>>>>> On Fri, Sep 16, 2016 at 3:31 PM, Stack <st...@duboce.net> wrote: > >>>>>>> > >>>>>>> The profile (or define) skipSparkTests looks like it will skip > >> spark > >>>>> tests. > >>>>>>> Setting skipIntegrationTests to true will skip it. > >>>>>>> S > >>>>>>> > >>>>>>> On Fri, Sep 16, 2016 at 1:40 PM, Dima Spivak < > >> dimaspi...@apache.org> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Doesn't seem we need a matrix project for master anymore since > >> we're > >>>>> just > >>>>>>>> doing JDK 8 now, right? Also, it looks like the hbase-spark > >>>>>>>> integration-test phase is what's tripping up the build. Why not > >> just > >>>>>>>> temporarily disable that to unblock testing? > >>>>>>>> > >>>>>>>> On Friday, September 16, 2016, Apekshit Sharma < > >> a...@cloudera.com> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>>> So the issue is, we set JAVA_HOME to jdk8 based on matrix param > >>>> and > >>>>>>> using > >>>>>>>>> tool environment. Since mvn uses the env variable, it compiles > >>>> with > >>>>> jdk > >>>>>>>> 8. > >>>>>>>>> But i suspect that scalatest isn't using the env variable, > >> instead > >>>>> it > >>>>>>>> might > >>>>>>>>> be directly using 'java' cmd, which can be jdk 7 or 8, and can > >>>> vary > >>>>> by > >>>>>>>>> machine. > >>>>>>>>> Build succeed if 'java' points to jdk 8, otherwise fails. > >>>>>>>>> Note that we didn't have this issue earlier since we were using > >>>>> jenkins > >>>>>>>>> 'JDK' axis which would set the 'java' to the appropriate > >> version. > >>>>> But > >>>>>>>> that > >>>>>>>>> methods had spaces-in-path issue, so i had to change it. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Fri, Sep 16, 2016 at 3:46 AM, aman poonia < > >>>>> aman.poonia...@gmail.com > >>>>>>>>> <javascript:;>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> I am not sure if this will help. But it looks like it is > >> because > >>>>> of > >>>>>>>>> version > >>>>>>>>>> mismatch, that is, it is expecting JDK1.7 and we are > >> compiling > >>>>> with > >>>>>>>>> jdk1.8. > >>>>>>>>>> That means there is some library which has to be compiled > >> with > >>>>> jdk8 > >>>>>>> or > >>>>>>>>>> needs to be updated to a jdk8 compatible version. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> *With Regards:-* > >>>>>>>>>> *Aman Poonia* > >>>>>>>>>> > >>>>>>>>>> On Fri, Sep 16, 2016 at 2:40 AM, Apekshit Sharma < > >>>>> a...@cloudera.com > >>>>>>>>> <javascript:;>> > >>>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> And....everything is back to red. > >>>>>>>>>>> Because something is plaguing our builds again. :( > >>>>>>>>>>> > >>>>>>>>>>> If anyone knows what's problem in this case, please reply > >> on > >>>>> this > >>>>>>>>> thread, > >>>>>>>>>>> otherwise i'll try to fix it later sometime today. > >>>>>>>>>>> > >>>>>>>>>>> [INFO] *--- scalatest-maven-plugin:1.0:test > >>>> (integration-test) > >>>>> @ > >>>>>>>>>>> hbase-spark --- > >>>>>>>>>>> * [36mDiscovery starting. [0m > >>>>>>>>>>> [31m*** RUN ABORTED *** [0m > >>>>>>>>>>> [31m java.lang.UnsupportedClassVersionError: > >>>>>>>>>>> org/apache/hadoop/hbase/spark/example/hbasecontext/ > >>>>>>>>>>> JavaHBaseDistributedScan > >>>>>>>>>>> : Unsupported major.minor version 52.0 [0m > >>>>>>>>>>> [31m at java.lang.ClassLoader.defineClass1(Native > >> Method) > >>>> [0m > >>>>>>>>>>> [31m at java.lang.ClassLoader. > >> defineClass(ClassLoader.java: > >>>>> 803) > >>>>>>>> [0m > >>>>>>>>>>> [31m at java.security.SecureClassLoader.defineClass( > >>>>>>>>>>> SecureClassLoader.java:142) > >>>>>>>>>>> [0m > >>>>>>>>>>> [31m at java.net.URLClassLoader. > >> defineClass(URLClassLoader. > >>>>>>>> java:449) > >>>>>>>>>> [0m > >>>>>>>>>>> [31m at java.net.URLClassLoader. > >> access$100(URLClassLoader. > >>>>>>> java:71) > >>>>>>>>> [0m > >>>>>>>>>>> [31m at java.net.URLClassLoader$1.run( > >>>>> URLClassLoader.java:361) > >>>>>>> [0m > >>>>>>>>>>> [31m at java.net.URLClassLoader$1.run( > >>>>> URLClassLoader.java:355) > >>>>>>> [0m > >>>>>>>>>>> [31m at java.security.AccessController.doPrivileged( > >> Native > >>>>>>> Method) > >>>>>>>>> [0m > >>>>>>>>>>> [31m at java.net.URLClassLoader. > >>>>> findClass(URLClassLoader.java: > >>>>>>> 354) > >>>>>>>>> [0m > >>>>>>>>>>> [31m at java.lang.ClassLoader. > >> loadClass(ClassLoader.java: > >>>> 425) > >>>>>>> [0m > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> On Mon, Sep 12, 2016 at 5:01 PM, Mikhail Antonov < > >>>>>>>> olorinb...@gmail.com > >>>>>>>>> <javascript:;>> > >>>>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Great work indeed! > >>>>>>>>>>>> > >>>>>>>>>>>> Agreed, occasional failed runs may not be that bad, but > >>>> fairly > >>>>>>>>> regular > >>>>>>>>>>>> failed runs ruin the idea of CI. Especially for released > >> or > >>>>>>>> otherwise > >>>>>>>>>>>> supposedly stable branches. > >>>>>>>>>>>> > >>>>>>>>>>>> -Mikhail > >>>>>>>>>>>> > >>>>>>>>>>>> On Mon, Sep 12, 2016 at 4:53 PM, Sean Busbey < > >>>>>>> bus...@cloudera.com > >>>>>>>>> <javascript:;>> > >>>>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> awesome work Appy! > >>>>>>>>>>>>> > >>>>>>>>>>>>> That's certainly good news to hear. > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Mon, Sep 12, 2016 at 2:14 PM, Apekshit Sharma < > >>>>>>>>> a...@cloudera.com <javascript:;>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> On a separate note: > >>>>>>>>>>>>>> Trunk had 8 green runs in last 3 days! ( > >>>>>>>>>>>>>> https://builds.apache.org/job/HBase-Trunk_matrix/) > >>>>>>>>>>>>>> This was due to fixing just the mass failures on > >> trunk > >>>>> and no > >>>>>>>>>> change > >>>>>>>>>>> in > >>>>>>>>>>>>>> flaky infra. Which made me to conclude two things: > >>>>>>>>>>>>>> 1. Flaky infra works. > >>>>>>>>>>>>>> 2. It relies heavily on the post-commit build's > >>>> stability > >>>>>>>> (which > >>>>>>>>>>> every > >>>>>>>>>>>>>> project should anyways strive for). If the build > >> fails > >>>>>>>>>>> catastrophically > >>>>>>>>>>>>>> once in a while, we can just exclude that one run > >> using > >>>> a > >>>>>>> flag > >>>>>>>>> and > >>>>>>>>>>>>>> everything will work, but if it happens frequently, > >> then > >>>>> it > >>>>>>>> won't > >>>>>>>>>>> work > >>>>>>>>>>>>>> right. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> I have re-enabled Flaky tests job ( > >>>>>>>>>>>>>> https://builds.apache.org/ > >>>> view/H-L/view/HBase/job/HBASE- > >>>>>>>>>> Flaky-Tests/ > >>>>>>>>>>> ) > >>>>>>>>>>>>> which > >>>>>>>>>>>>>> was disabled for almost a month due to trunk being on > >>>>> fire. > >>>>>>>>>>>>>> I will keep an eye on how things are going. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Mon, Sep 12, 2016 at 2:02 PM, Apekshit Sharma < > >>>>>>>>>> a...@cloudera.com <javascript:;>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> @Sean, Mikhail: I found the alternate solution. > >> Using > >>>>> user > >>>>>>>>> defined > >>>>>>>>>>>> axis, > >>>>>>>>>>>>>>> tool environment and env variable injection. > >>>>>>>>>>>>>>> See latest diff to https://builds.apache.org/job/ > >>>>>>>>>>> HBase-Trunk_matrix/ > >>>>>>>>>>>>> job > >>>>>>>>>>>>>>> for reference. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Tue, Aug 30, 2016 at 7:39 PM, Mikhail Antonov < > >>>>>>>>>>>> olorinb...@gmail.com <javascript:;>> > >>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> FYI, I did the same for branch-1.3 builds. I've > >>>>> disabled > >>>>>>>>>> hbase-1.3 > >>>>>>>>>>>> and > >>>>>>>>>>>>>>>> hbase-1.3-IT jobs and instead created > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> https://builds.apache.org/job/HBase-1.3-JDK8 and > >>>>>>>>>>>>>>>> https://builds.apache.org/job/HBase-1.3-JDK7 > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> This should work for now until we figure out how to > >>>> move > >>>>>>>>> forward. > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> Mikhail > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> On Wed, Aug 17, 2016 at 1:41 PM, Sean Busbey < > >>>>>>>>>> bus...@cloudera.com <javascript:;>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> /me smacks forehead > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> these replacement jobs, of course, also have > >> special > >>>>>>>>> characters > >>>>>>>>>>> in > >>>>>>>>>>>>>>>>> their names which then show up in the working > >> path. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> renaming them to skip spaces and parens. > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> On Wed, Aug 17, 2016 at 1:34 PM, Sean Busbey < > >>>>>>>>>>>> sean.bus...@gmail.com <javascript:;>> > >>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>> FYI, it looks like essentially our entire CI > >> suite > >>>>> is > >>>>>>>> red, > >>>>>>>>>>>> probably > >>>>>>>>>>>>>>>> due > >>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>> parts of our codebase not tolerating spaces or > >>>> other > >>>>>>>>> special > >>>>>>>>>>>>>>>> characters > >>>>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>> the working directory. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> I've made a stop-gap non-multi-configuration > >> set > >>>> of > >>>>>>> jobs > >>>>>>>>> for > >>>>>>>>>>>>> running > >>>>>>>>>>>>>>>> unit > >>>>>>>>>>>>>>>>>> tests for the 1.2 branch against JDK 7 and JDK > >> 8: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> https://builds.apache.org/ > >>>>>>> view/H-L/view/HBase/job/HBase% > >>>>>>>>>>>>>>>>> 201.2%20(JDK%201.7)/ > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> https://builds.apache.org/ > >>>>>>> view/H-L/view/HBase/job/HBase% > >>>>>>>>>>>>>>>>> 201.2%20(JDK%201.8)/ > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> Due to the lack of response from infra@ I > >> suspect > >>>>> our > >>>>>>>> only > >>>>>>>>>>>> options > >>>>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>> continuing on ASF infra is to fix whatever > >> part of > >>>>> our > >>>>>>>>> build > >>>>>>>>>>>>> doesn't > >>>>>>>>>>>>>>>>>> tolerate the new paths, or stop using > >>>>>>> multiconfiguration > >>>>>>>>>>>>> deployments. > >>>>>>>>>>>>>>>> I > >>>>>>>>>>>>>>>>> am > >>>>>>>>>>>>>>>>>> obviously less than thrilled at the idea of > >> having > >>>>>>>> several > >>>>>>>>>>>>> multiples > >>>>>>>>>>>>>>>> of > >>>>>>>>>>>>>>>>>> current jobs. > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> On Wed, Aug 10, 2016 at 6:28 PM, Sean Busbey < > >>>>>>>>>>>> bus...@cloudera.com <javascript:;>> > >>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> Ugh. > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> I sent a reply to Gav on builds@ about maybe > >>>>> getting > >>>>>>>>> names > >>>>>>>>>>> that > >>>>>>>>>>>>>>>> don't > >>>>>>>>>>>>>>>>>>> have spaces in them: > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> https://lists.apache.org/thread.html/ > >>>>>>>>>>>>> 8ac03dc62f9d6862d4f3d5eb37119c > >>>>>>>>>>>>>>>>>>> 9c73b4059aaa3ebba52fc63bb6@%3C > >> builds.apache.org > >>>> %3E > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> In the mean time, is this an issue we need > >> file > >>>>> with > >>>>>>>>> Hadoop > >>>>>>>>>> or > >>>>>>>>>>>>>>>>>>> something we need to fix in our own code? > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> On Wed, Aug 10, 2016 at 6:04 PM, Matteo > >> Bertozzi > >>>>>>>>>>>>>>>>>>> <theo.berto...@gmail.com <javascript:;>> > >> wrote: > >>>>>>>>>>>>>>>>>>>> There are a bunch of builds that have most > >> of > >>>> the > >>>>>>> test > >>>>>>>>>>>> failing. > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Example: > >>>>>>>>>>>>>>>>>>>> https://builds.apache.org/job/ > >>>>>>>>>> HBase-Trunk_matrix/1392/jdk= > >>>>>>>>>>>>>>>>>>> JDK%201.7%20(latest),label= > >>>>>>>> yahoo-not-h2/testReport/junit/ > >>>>>>>>>>>>>>>>>>> org.apache.hadoop.hbase/ > >> TestLocalHBaseCluster/ > >>>>>>>>>>>>> testLocalHBaseCluster/ > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> from the stack trace looks like the problem > >> is > >>>>> with > >>>>>>>> the > >>>>>>>>>> jdk > >>>>>>>>>>>> name > >>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>> has > >>>>>>>>>>>>>>>>>>>> spaces: > >>>>>>>>>>>>>>>>>>>> the hadoop FsVolumeImpl calls > >>>> setNameFormat(... + > >>>>>>>>>>>>>>>> fileName.toString() > >>>>>>>>>>>>>>>>> + > >>>>>>>>>>>>>>>>>>> ...) > >>>>>>>>>>>>>>>>>>>> and this seems to not be escaped > >>>>>>>>>>>>>>>>>>>> so we end up with JDK%25201.7%2520(latest) > >> in > >>>> the > >>>>>>>> string > >>>>>>>>>>>> format > >>>>>>>>>>>>>>>> and we > >>>>>>>>>>>>>>>>>>> get > >>>>>>>>>>>>>>>>>>>> a IllegalFormatPrecisionException: 7 > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> 2016-08-10 22:07:46,108 WARN [DataNode: > >>>>>>>>>>>>>>>>>>>> [[[DISK]file:/home/jenkins/ > >>>>>>>>> jenkins-slave/workspace/HBase- > >>>>>>>>>>>>>>>>>>> Trunk_matrix/jdk/JDK%25201.7% > >>>>>>>>> 2520(latest)/label/yahoo-not- > >>>>>>>>>>>>>>>>>>> h2/hbase-server/target/test- > >>>>>>>> data/e7099624-ecfa-4674-87de- > >>>>>>>>>>>>>>>>>>> a8733d13b582/dfscluster_ > >> 10fdcfc3-cd1b-45be-9b5a- > >>>>>>>>>>>>>>>>>>> 9c88f385e6f1/dfs/data/data1/, > >>>>>>>>>>>>>>>>>>>> [DISK]file:/home/jenkins/ > >>>>>>>> jenkins-slave/workspace/HBase- > >>>>>>>>>>>>>>>>>>> Trunk_matrix/jdk/JDK%25201.7% > >>>>>>>>> 2520(latest)/label/yahoo-not- > >>>>>>>>>>>>>>>>>>> h2/hbase-server/target/test- > >>>>>>>> data/e7099624-ecfa-4674-87de- > >>>>>>>>>>>>>>>>>>> a8733d13b582/dfscluster_ > >> 10fdcfc3-cd1b-45be-9b5a- > >>>>>>>>>>>>>>>>>>> 9c88f385e6f1/dfs/data/data2/]] > >>>>>>>>>>>>>>>>>>>> heartbeating to localhost/127.0.0.1:34629] > >>>>>>>>>>>>>>>>>>>> datanode.BPServiceActor(831): Unexpected > >>>>> exception > >>>>>>> in > >>>>>>>>>> block > >>>>>>>>>>>> pool > >>>>>>>>>>>>>>>> Block > >>>>>>>>>>>>>>>>>>>> pool <registering> (Datanode Uuid > >> unassigned) > >>>>>>> service > >>>>>>>> to > >>>>>>>>>>>>>>>>>>>> localhost/127.0.0.1:34629 > >>>>>>>>>>>>>>>>>>>> java.util.IllegalFormatPrecisionException: > >> 7 > >>>>>>>>>>>>>>>>>>>> at java.util.Formatter$ > >>>>>>>>> FormatSpecifier.checkText( > >>>>>>>>>>>>>>>>>>> Formatter.java:2984) > >>>>>>>>>>>>>>>>>>>> at java.util.Formatter$ > >>>>>>>> FormatSpecifier.<init>( > >>>>>>>>>>>>>>>>>>> Formatter.java:2688) > >>>>>>>>>>>>>>>>>>>> at java.util.Formatter.parse( > >>>>>>>>> Formatter.java:2528) > >>>>>>>>>>>>>>>>>>>> at java.util.Formatter.format( > >>>>>>>>>> Formatter.java:2469) > >>>>>>>>>>>>>>>>>>>> at java.util.Formatter.format( > >>>>>>>>>> Formatter.java:2423) > >>>>>>>>>>>>>>>>>>>> at java.lang.String.format( > >>>>>>> String.java:2792) > >>>>>>>>>>>>>>>>>>>> at com.google.common.util. > >> concurrent. > >>>>>>>>>>>>> ThreadFactoryBuilder. > >>>>>>>>>>>>>>>>>>> setNameFormat(ThreadFactoryBuilder.java:68) > >>>>>>>>>>>>>>>>>>>> at org.apache.hadoop.hdfs.server. > >>>>>>>>>>>>> datanode.fsdataset.impl. > >>>>>>>>>>>>>>>>>>> FsVolumeImpl.initializeCacheExecutor( > >>>>>>>>> FsVolumeImpl.java:140) > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> Matteo > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>> On Tue, Aug 9, 2016 at 9:55 AM, Stack < > >>>>>>>> st...@duboce.net > >>>>>>>>> <javascript:;>> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> Good on you Sean. > >>>>>>>>>>>>>>>>>>>>> S > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>> On Mon, Aug 8, 2016 at 9:43 PM, Sean > >> Busbey < > >>>>>>>>>>>> bus...@apache.org <javascript:;> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> I updated all of our jobs to use the > >> updated > >>>>> JDK > >>>>>>>>>> versions > >>>>>>>>>>>>> from > >>>>>>>>>>>>>>>>> infra. > >>>>>>>>>>>>>>>>>>>>>> These have spaces in the names, and those > >>>>> names > >>>>>>> end > >>>>>>>>> up > >>>>>>>>>> in > >>>>>>>>>>>> our > >>>>>>>>>>>>>>>>>>>>>> workspace path, so try to keep an eye > >> out. > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> On Mon, Aug 8, 2016 at 10:42 AM, Sean > >>>> Busbey < > >>>>>>>>>>>>>>>> bus...@cloudera.com <javascript:;>> > >>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>> running in docker is the default now. > >>>>> relying > >>>>>>> on > >>>>>>>>> the > >>>>>>>>>>>>> default > >>>>>>>>>>>>>>>>> docker > >>>>>>>>>>>>>>>>>>>>>>> image that comes with Yetus means that > >> our > >>>>>>> protoc > >>>>>>>>>>> checks > >>>>>>>>>>>>> are > >>>>>>>>>>>>>>>>>>>>>>> failing[1]. > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> [1]: https://issues.apache.org/ > >>>>>>>>>> jira/browse/HBASE-16373 > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> On Sat, Aug 6, 2016 at 5:03 PM, Sean > >>>> Busbey > >>>>> < > >>>>>>>>>>>>>>>> bus...@apache.org <javascript:;>> > >>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>> Hi folks! > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> this morning I merged the patch that > >>>>> updates > >>>>>>> us > >>>>>>>> to > >>>>>>>>>>> Yetus > >>>>>>>>>>>>>>>>> 0.3.0[1] > >>>>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>> updated the precommit job appropriately. > >> I > >>>>> also > >>>>>>>>> changed > >>>>>>>>>>> it > >>>>>>>>>>>> to > >>>>>>>>>>>>>>>> use > >>>>>>>>>>>>>>>>> one > >>>>>>>>>>>>>>>>>>> of > >>>>>>>>>>>>>>>>>>>>>> the Java versions post the puppet > >> changes to > >>>>> asf > >>>>>>>>> build. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> The last three builds look normal > >> (#2975 > >>>> - > >>>>>>>> #2977). > >>>>>>>>>> I'm > >>>>>>>>>>>>> gonna > >>>>>>>>>>>>>>>> try > >>>>>>>>>>>>>>>>>>>>>> running things in docker next. I'll email > >>>>> again > >>>>>>>> when > >>>>>>>>> I > >>>>>>>>>>> make > >>>>>>>>>>>>> it > >>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>> default. > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> [1]: https://issues.apache.org/ > >>>>>>>>>>> jira/browse/HBASE-15882 > >>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>> On 2016-06-16 10:43 (-0500), Sean > >> Busbey > >>>> < > >>>>>>>>>>>>> bus...@apache.org <javascript:;>> > >>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>> FYI, today our precommit jobs started > >>>>> failing > >>>>>>>>>> because > >>>>>>>>>>>> our > >>>>>>>>>>>>>>>>> chosen > >>>>>>>>>>>>>>>>>>> jdk > >>>>>>>>>>>>>>>>>>>>>>>>> (1.7.0.79) disappeared (mentioned on > >>>>>>>>> HBASE-16032). > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Initially we were doing something > >> wrong, > >>>>>>> namely > >>>>>>>>>>>> directly > >>>>>>>>>>>>>>>>>>> referencing > >>>>>>>>>>>>>>>>>>>>>>>>> the jenkins build tools area without > >>>>> telling > >>>>>>>>>> jenkins > >>>>>>>>>>> to > >>>>>>>>>>>>> give > >>>>>>>>>>>>>>>>> us an > >>>>>>>>>>>>>>>>>>>>> env > >>>>>>>>>>>>>>>>>>>>>>>>> variable that stated where the jdk is > >>>>>>> located. > >>>>>>>>>>> However, > >>>>>>>>>>>>>>>> after > >>>>>>>>>>>>>>>>>>>>>>>>> attempting to switch to the > >> appropriate > >>>>>>> tooling > >>>>>>>>>>>> variable > >>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>> jdk > >>>>>>>>>>>>>>>>>>>>>>>>> 1.7.0.79, I found that it didn't > >> point > >>>> to > >>>>> a > >>>>>>>> place > >>>>>>>>>>> that > >>>>>>>>>>>>>>>> worked. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> I've now updated the job to rely on > >> the > >>>>>>> latest > >>>>>>>>> 1.7 > >>>>>>>>>>> jdk, > >>>>>>>>>>>>>>>> which > >>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>>>>> currently 1.7.0.80. I don't know how > >>>> often > >>>>>>>>> "latest" > >>>>>>>>>>>>> updates. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> Personally, I think this is a sign > >> that > >>>> we > >>>>>>> need > >>>>>>>>> to > >>>>>>>>>>>>>>>> prioritize > >>>>>>>>>>>>>>>>>>>>>>>>> HBASE-15882 so that we can switch > >> back > >>>> to > >>>>>>> using > >>>>>>>>>>>> Docker. I > >>>>>>>>>>>>>>>> won't > >>>>>>>>>>>>>>>>>>> have > >>>>>>>>>>>>>>>>>>>>>>>>> time this week, so if anyone else > >> does > >>>>> please > >>>>>>>>> pick > >>>>>>>>>> up > >>>>>>>>>>>> the > >>>>>>>>>>>>>>>>> ticket. > >>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>> On Thu, Mar 17, 2016 at 5:19 PM, > >> Stack < > >>>>>>>>>>>> st...@duboce.net <javascript:;> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks Sean. > >>>>>>>>>>>>>>>>>>>>>>>>>> St.Ack > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Mar 16, 2016 at 12:04 PM, > >> Sean > >>>>>>>> Busbey < > >>>>>>>>>>>>>>>>>>> bus...@cloudera.com <javascript:;> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> FYI, I updated the precommit job > >>>> today > >>>>> to > >>>>>>>>>> specify > >>>>>>>>>>>> that > >>>>>>>>>>>>>>>> only > >>>>>>>>>>>>>>>>>>>>> compile > >>>>>>>>>>>>>>>>>>>>>> time > >>>>>>>>>>>>>>>>>>>>>>>>>>> checks should be done against jdks > >>>>> other > >>>>>>>> than > >>>>>>>>>> the > >>>>>>>>>>>>> primary > >>>>>>>>>>>>>>>>> jdk7 > >>>>>>>>>>>>>>>>>>>>>> instance. > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 7, 2016 at 8:43 PM, > >> Sean > >>>>>>> Busbey > >>>>>>>> < > >>>>>>>>>>>>>>>>>>> bus...@cloudera.com <javascript:;>> > >>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> I tested things out, and while > >>>>>>>> YETUS-297[1] > >>>>>>>>> is > >>>>>>>>>>>>> present > >>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>> default runs > >>>>>>>>>>>>>>>>>>>>>>>>>>>> all plugins that can do multiple > >>>> jdks > >>>>>>>>> against > >>>>>>>>>>>> those > >>>>>>>>>>>>>>>>> available > >>>>>>>>>>>>>>>>>>>>>> (jdk7 and > >>>>>>>>>>>>>>>>>>>>>>>>>>>> jdk8 in our case). > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> We can configure things to only > >> do > >>>> a > >>>>>>>> single > >>>>>>>>>> run > >>>>>>>>>>> of > >>>>>>>>>>>>> unit > >>>>>>>>>>>>>>>>>>> tests. > >>>>>>>>>>>>>>>>>>>>>> They'll be > >>>>>>>>>>>>>>>>>>>>>>>>>>>> against jdk7, since that is our > >>>>> default > >>>>>>>> jdk. > >>>>>>>>>>> That > >>>>>>>>>>>>> fine > >>>>>>>>>>>>>>>> by > >>>>>>>>>>>>>>>>>>>>>> everyone? It'll > >>>>>>>>>>>>>>>>>>>>>>>>>>>> save ~1.5 hours on any build > >> that > >>>>> hits > >>>>>>>>>>>> hbase-server. > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 7, 2016 at 1:22 PM, > >>>>> Stack < > >>>>>>>>>>>>>>>> st...@duboce.net <javascript:;>> > >>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hurray! > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> It looks like YETUS-96 is in > >> there > >>>>> and > >>>>>>> we > >>>>>>>>> are > >>>>>>>>>>>> only > >>>>>>>>>>>>>>>>> running > >>>>>>>>>>>>>>>>>>> on > >>>>>>>>>>>>>>>>>>>>>> jdk build > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> now, the default (but testing > >>>>> compile > >>>>>>>>> against > >>>>>>>>>>>>>>>> both).... > >>>>>>>>>>>>>>>>> Will > >>>>>>>>>>>>>>>>>>>>>> keep an > >>>>>>>>>>>>>>>>>>>>>>>>>>> eye. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> St.Ack > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Mar 7, 2016 at 10:27 > >> AM, > >>>>> Sean > >>>>>>>>> Busbey > >>>>>>>>>> < > >>>>>>>>>>>>>>>>>>>>>> bus...@cloudera.com <javascript:;>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FYI, I've just updated our > >>>>> precommit > >>>>>>>> jobs > >>>>>>>>>> to > >>>>>>>>>>>> use > >>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>> 0.2.0 > >>>>>>>>>>>>>>>>>>>>>> release of > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yetus > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that came out today. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> After keeping an eye out for > >>>>>>>> strangeness > >>>>>>>>>>> today > >>>>>>>>>>>>> I'll > >>>>>>>>>>>>>>>>> turn > >>>>>>>>>>>>>>>>>>>>>> docker mode > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> back > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on by default tonight. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Jan 13, 2016 at 10:14 > >>>> AM, > >>>>>>> Sean > >>>>>>>>>>> Busbey < > >>>>>>>>>>>>>>>>>>>>>> bus...@apache.org <javascript:;>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FYI, I added a new > >> parameter > >>>> to > >>>>> the > >>>>>>>>>>> precommit > >>>>>>>>>>>>> job: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> * USE_YETUS_PRERELEASE - > >>>> causes > >>>>> us > >>>>>>> to > >>>>>>>>> use > >>>>>>>>>>> the > >>>>>>>>>>>>>>>> HEAD of > >>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>> apache/yetus > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> repo rather than our chosen > >>>>> release > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> It defaults to inactive, > >> but > >>>>> can be > >>>>>>>>> used > >>>>>>>>>> in > >>>>>>>>>>>>>>>>>>>>>> manually-triggered runs > >>>>>>>>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test a solution to a > >> problem > >>>> in > >>>>> the > >>>>>>>>> yetus > >>>>>>>>>>>>>>>> library. At > >>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>> moment, > >>>>>>>>>>>>>>>>>>>>>>>>>>> I'm > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> using it to test a > >> solution to > >>>>>>>> default > >>>>>>>>>>> module > >>>>>>>>>>>>>>>>> ordering > >>>>>>>>>>>>>>>>>>> as > >>>>>>>>>>>>>>>>>>>>>> seen in > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HBASE-15075. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Jan 8, 2016 at 7:58 > >>>> AM, > >>>>>>> Sean > >>>>>>>>>>> Busbey < > >>>>>>>>>>>>>>>>>>>>>> bus...@cloudera.com <javascript:;>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FYI, I just pushed > >>>> HBASE-13525 > >>>>>>>>> (switch > >>>>>>>>>> to > >>>>>>>>>>>>> Apache > >>>>>>>>>>>>>>>>> Yetus > >>>>>>>>>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>>>>>>>>>>> precommit > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tests) > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and updated our jenkins > >>>>> precommit > >>>>>>>>> build > >>>>>>>>>>> to > >>>>>>>>>>>>> use > >>>>>>>>>>>>>>>> it. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Jenkins job has some > >>>>> explanation: > >>>>>>>>>>>>>>>>>>>>>>>>>>> https://builds.apache.org/ > >>>>>>>>>>>>> view/PreCommit%20Builds/job/ > >>>>>>>>>>>>>>>>>>>>>> PreCommit-HBASE-Build/ > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Release note from > >>>> HBASE-13525 > >>>>>>> does > >>>>>>>> as > >>>>>>>>>>> well. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The old job will stick > >>>> around > >>>>>>> here > >>>>>>>>> for > >>>>>>>>>> a > >>>>>>>>>>>>> couple > >>>>>>>>>>>>>>>> of > >>>>>>>>>>>>>>>>>>> weeks, > >>>>>>>>>>>>>>>>>>>>>> in case > >>>>>>>>>>>>>>>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> need > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to refer back to it: > >>>>>>>>>>>>>>>>>>>>>>>>>>> https://builds.apache.org/ > >>>>>>>>>>>>> view/PreCommit%20Builds/job/ > >>>>>>>>>>>>>>>>>>>>>> PreCommit-HBASE-Build-deprecated/ > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> If something looks awry, > >>>>> please > >>>>>>>> drop > >>>>>>>>> a > >>>>>>>>>>> note > >>>>>>>>>>>>> on > >>>>>>>>>>>>>>>>>>>>> HBASE-13525 > >>>>>>>>>>>>>>>>>>>>>> while > >>>>>>>>>>>>>>>>>>>>>>>>>>> it > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> remains > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> open (and make a new > >> issue > >>>>>>> after). > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Wed, Dec 2, 2015 at > >> 3:22 > >>>>> PM, > >>>>>>>>> Stack < > >>>>>>>>>>>>>>>>>>> st...@duboce.net <javascript:;>> > >>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As part of my continuing > >>>>>>> advocacy > >>>>>>>> of > >>>>>>>>>>>>>>>>>>> builds.apache.org > >>>>>>>>>>>>>>>>>>>>>> and that > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> their > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> results are now worthy > >> of > >>>> our > >>>>>>>> trust > >>>>>>>>>> and > >>>>>>>>>>>>>>>> nurture, > >>>>>>>>>>>>>>>>> here > >>>>>>>>>>>>>>>>>>>>> are > >>>>>>>>>>>>>>>>>>>>>> some > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> highlights > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> from the last few days > >> of > >>>>>>> builds: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> + hadoopqa is now > >> finding > >>>>>>> zombies > >>>>>>>>>> before > >>>>>>>>>>>> the > >>>>>>>>>>>>>>>>> patch is > >>>>>>>>>>>>>>>>>>>>>> committed. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> HBASE-14888 showed "-1 > >> core > >>>>>>> tests. > >>>>>>>>> The > >>>>>>>>>>>> patch > >>>>>>>>>>>>>>>>> failed > >>>>>>>>>>>>>>>>>>>>> these > >>>>>>>>>>>>>>>>>>>>>> unit > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> tests:" > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> but > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> didn't have any failed > >>>> tests > >>>>>>>> listed > >>>>>>>>>> (I'm > >>>>>>>>>>>>>>>> trying to > >>>>>>>>>>>>>>>>>>> see > >>>>>>>>>>>>>>>>>>>>> if > >>>>>>>>>>>>>>>>>>>>>> I can > >>>>>>>>>>>>>>>>>>>>>>>>>>> do > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> anything > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> about this...). Running > >> our > >>>>>>> little > >>>>>>>>>>>>>>>>>>>>>>>>>>> ./dev-tools/findHangingTests.py > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> against > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the consoleText, it > >> showed > >>>> a > >>>>>>>> hanging > >>>>>>>>>>> test. > >>>>>>>>>>>>>>>> Running > >>>>>>>>>>>>>>>>>>>>>> locally, I see > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> same > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> hang. This is before the > >>>>> patch > >>>>>>>>> landed. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> + Our branch runs are > >> now > >>>>> near > >>>>>>>>> totally > >>>>>>>>>>>>> zombie > >>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>> flakey > >>>>>>>>>>>>>>>>>>>>>> free -- > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> still > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> some > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work to do -- but a > >> recent > >>>>> patch > >>>>>>>>> that > >>>>>>>>>>>> seemed > >>>>>>>>>>>>>>>>> harmless > >>>>>>>>>>>>>>>>>>>>> was > >>>>>>>>>>>>>>>>>>>>>>>>>>> causing a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reliable flake fail in > >> the > >>>>>>>> backport > >>>>>>>>> to > >>>>>>>>>>>>>>>> branch-1* > >>>>>>>>>>>>>>>>>>>>>> confirmed by > >>>>>>>>>>>>>>>>>>>>>>>>>>> local > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> runs. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The flakeyness was > >> plain to > >>>>> see > >>>>>>> up > >>>>>>>>> in > >>>>>>>>>>>>>>>>>>> builds.apache.org > >>>>>>>>>>>>>>>>>>>>> . > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> + In the last few days > >> I've > >>>>>>>>> committed > >>>>>>>>>> a > >>>>>>>>>>>>> patch > >>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>> included > >>>>>>>>>>>>>>>>>>>>>>>>>>> javadoc > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> warnings even though > >>>> hadoopqa > >>>>>>> said > >>>>>>>>> the > >>>>>>>>>>>> patch > >>>>>>>>>>>>>>>>>>> introduced > >>>>>>>>>>>>>>>>>>>>>> javadoc > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> issues > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> (I > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> missed it). This messed > >> up > >>>>> life > >>>>>>>> for > >>>>>>>>>>> folks > >>>>>>>>>>>>>>>>>>> subsequently > >>>>>>>>>>>>>>>>>>>>> as > >>>>>>>>>>>>>>>>>>>>>> their > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> patches > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> now > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reported javadoc > >> issues.... > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In short, I suggest that > >>>>>>>>>>>> builds.apache.org > >>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>> worth > >>>>>>>>>>>>>>>>>>>>>> keeping an > >>>>>>>>>>>>>>>>>>>>>>>>>>> eye > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> on, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> make > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sure you get a clean > >> build > >>>>> out > >>>>>>> of > >>>>>>>>>>> hadoopqa > >>>>>>>>>>>>>>>> before > >>>>>>>>>>>>>>>>>>>>>> committing > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> anything, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> lets all work together > >> to > >>>> try > >>>>>>> and > >>>>>>>>> keep > >>>>>>>>>>> our > >>>>>>>>>>>>>>>> builds > >>>>>>>>>>>>>>>>>>> blue: > >>>>>>>>>>>>>>>>>>>>>> it'll > >>>>>>>>>>>>>>>>>>>>>>>>>>> save > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> us > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> work in the long run. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> St.Ack > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Tue, Nov 4, 2014 at > >> 9:38 > >>>>> AM, > >>>>>>>>> Stack > >>>>>>>>>> < > >>>>>>>>>>>>>>>>>>> st...@duboce.net <javascript:;> > >>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Branch-1 and master > >> have > >>>>>>>>> stabilized > >>>>>>>>>>> and > >>>>>>>>>>>>> now > >>>>>>>>>>>>>>>> run > >>>>>>>>>>>>>>>>>>> mostly > >>>>>>>>>>>>>>>>>>>>>> blue > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> (give or > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> take > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the odd failure) > >> [1][2]. > >>>>>>> Having > >>>>>>>> a > >>>>>>>>>>> mostly > >>>>>>>>>>>>> blue > >>>>>>>>>>>>>>>>>>> branch-1 > >>>>>>>>>>>>>>>>>>>>>> has > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> helped us > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> identify at least one > >>>>>>>>> destabilizing > >>>>>>>>>>>>> commit in > >>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>> last > >>>>>>>>>>>>>>>>>>>>>> few > >>>>>>>>>>>>>>>>>>>>>>>>>>> days, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> maybe > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> two; > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this is as it should > >> be > >>>>>>> (smile). > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Lets keep our builds > >>>> blue. > >>>>> If > >>>>>>>> you > >>>>>>>>>>>> commit a > >>>>>>>>>>>>>>>>> patch, > >>>>>>>>>>>>>>>>>>> make > >>>>>>>>>>>>>>>>>>>>>> sure > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> subsequent > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> builds stay blue. You > >> can > >>>>>>>>> subscribe > >>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>> bui...@hbase.apache.org <javascript:;> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> get > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> notice of failures if > >> not > >>>>>>>> already > >>>>>>>>>>>>> subscribed. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> St.Ack > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 1. > >>>>>>>>>>>>>>>>>>>>>>>>>>> https://builds.apache.org/ > >>>>>>>>>>>>> view/H-L/view/HBase/job/HBase- > >>>>>>>>>>>>>>>>> 1.0/ > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://builds.apache.org/view > >>>>>>>>>>>>>>>> /H-L/view/HBase/job/HBase- > >>>>>>>>>>>>>>>>>>> TRUNK/ > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Oct 13, 2014 > >> at > >>>>> 4:41 > >>>>>>> PM, > >>>>>>>>>>> Stack < > >>>>>>>>>>>>>>>>>>>>>> st...@duboce.net <javascript:;>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> wrote: > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> A few notes on > >> testing. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Too long to read, > >> infra > >>>> is > >>>>>>> more > >>>>>>>>>>> capable > >>>>>>>>>>>>> now > >>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>> after > >>>>>>>>>>>>>>>>>>>>>> some > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> work, we > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seeing branch-1 and > >>>> trunk > >>>>>>>> mostly > >>>>>>>>>>>> running > >>>>>>>>>>>>>>>> blue. > >>>>>>>>>>>>>>>>>>> Lets > >>>>>>>>>>>>>>>>>>>>>> try and > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> keep it > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> way going forward. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Apache Infra has new, > >>>> more > >>>>>>>>> capable > >>>>>>>>>>>>> hardware. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> A recent spurt of > >> test > >>>>> fixing > >>>>>>>>>>> combined > >>>>>>>>>>>>> with > >>>>>>>>>>>>>>>>> more > >>>>>>>>>>>>>>>>>>>>>> capable > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> hardware > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> seems > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to have gotten us to > >> a > >>>> new > >>>>>>>> place; > >>>>>>>>>>> tests > >>>>>>>>>>>>> are > >>>>>>>>>>>>>>>>> mostly > >>>>>>>>>>>>>>>>>>>>>> passing now > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> on > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> branch-1 > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and master. Lets try > >>>> and > >>>>>>> keep > >>>>>>>> it > >>>>>>>>>>> this > >>>>>>>>>>>>> way > >>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>> start > >>>>>>>>>>>>>>>>>>>>>> to trust > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> our > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> test > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> runs > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> again. Just a few > >>>> flakies > >>>>>>>>> remain. > >>>>>>>>>>>> Lets > >>>>>>>>>>>>> try > >>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>> nail > >>>>>>>>>>>>>>>>>>>>>> them. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Our tests now run in > >>>>> parallel > >>>>>>>>> with > >>>>>>>>>>>> other > >>>>>>>>>>>>>>>> test > >>>>>>>>>>>>>>>>>>> suites > >>>>>>>>>>>>>>>>>>>>>> where > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> previous > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ran alone. You can > >> see > >>>>> this > >>>>>>>>>> sometimes > >>>>>>>>>>>>> when > >>>>>>>>>>>>>>>> our > >>>>>>>>>>>>>>>>>>> zombie > >>>>>>>>>>>>>>>>>>>>>> detector > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> reports > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tests from another > >>>> project > >>>>>>>>>> altogether > >>>>>>>>>>>> as > >>>>>>>>>>>>>>>>> lingerers > >>>>>>>>>>>>>>>>>>>>> (To > >>>>>>>>>>>>>>>>>>>>>> be > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> fixed). > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Some > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our tests are failing > >>>>>>> because a > >>>>>>>>>>>>> concurrent > >>>>>>>>>>>>>>>>> hbase > >>>>>>>>>>>>>>>>>>> run > >>>>>>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>>>>>>> undoing > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> classes > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> data from under it. > >>>> Also, > >>>>>>> lets > >>>>>>>>> fix. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Our tests are > >> brittle. > >>>> It > >>>>>>> takes > >>>>>>>>>>>> 75minutes > >>>>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>>> them to > >>>>>>>>>>>>>>>>>>>>>>>>>>> complete. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Many > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> are > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> heavy-duty > >> integration > >>>>> tests > >>>>>>>>>> starting > >>>>>>>>>>>> up > >>>>>>>>>>>>>>>>> multiple > >>>>>>>>>>>>>>>>>>>>>> clusters and > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> mapreduce > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> all in the one JVM. > >> It > >>>> is > >>>>> a > >>>>>>>>> miracle > >>>>>>>>>>>> they > >>>>>>>>>>>>>>>> pass > >>>>>>>>>>>>>>>>> at > >>>>>>>>>>>>>>>>>>> all. > >>>>>>>>>>>>>>>>>>>>>> Usually > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> integration > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tests have been cast > >> as > >>>>> unit > >>>>>>>>> tests > >>>>>>>>>>>>> because > >>>>>>>>>>>>>>>>> there > >>>>>>>>>>>>>>>>>>> was > >>>>>>>>>>>>>>>>>>>>>> no where > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> else > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> for > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> them > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to get an airing. We > >>>> have > >>>>>>> the > >>>>>>>>>>> hbase-it > >>>>>>>>>>>>>>>> suite > >>>>>>>>>>>>>>>>> now > >>>>>>>>>>>>>>>>>>>>>> which would > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> be a > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> more > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> apt > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> place but until these > >>>> are > >>>>> run > >>>>>>>> on > >>>>>>>>> a > >>>>>>>>>>>>> regular > >>>>>>>>>>>>>>>>> basis > >>>>>>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>>>>>> public for > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> all > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> see, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the fat integration > >>>> tests > >>>>>>>>> disguised > >>>>>>>>>>> as > >>>>>>>>>>>>> unit > >>>>>>>>>>>>>>>>> tests > >>>>>>>>>>>>>>>>>>>>> will > >>>>>>>>>>>>>>>>>>>>>> remain. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> A > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> review of > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our current unit > >> tests > >>>>>>> weeding > >>>>>>>>> the > >>>>>>>>>>> old > >>>>>>>>>>>>> cruft > >>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>> no longer > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> relevant > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> duplicates would be a > >>>> nice > >>>>>>>>>>> undertaking > >>>>>>>>>>>> if > >>>>>>>>>>>>>>>>> someone > >>>>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>> looking > >>>>>>>>>>>>>>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> contribute. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Alex Newman has been > >>>>> working > >>>>>>> on > >>>>>>>>>>> making > >>>>>>>>>>>>> our > >>>>>>>>>>>>>>>>> tests > >>>>>>>>>>>>>>>>>>> work > >>>>>>>>>>>>>>>>>>>>>> up on > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> travis > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> and > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> circle-ci. That'll > >> be > >>>>> sweet > >>>>>>>> when > >>>>>>>>>> it > >>>>>>>>>>>> goes > >>>>>>>>>>>>>>>>>>> end-to-end. > >>>>>>>>>>>>>>>>>>>>>> He also > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> added > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> some "type" > >>>>> categorizations > >>>>>>> -- > >>>>>>>>>>> client, > >>>>>>>>>>>>>>>> filter, > >>>>>>>>>>>>>>>>>>>>>> mapreduce -- > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> alongside > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> our > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> old "sizing" > >>>>> categorizations > >>>>>>> of > >>>>>>>>>>>>>>>>>>> small/medium/large. > >>>>>>>>>>>>>>>>>>>>>> His > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> thinking > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> we can run these > >>>>>>>> categorizations > >>>>>>>>> in > >>>>>>>>>>>>> parallel > >>>>>>>>>>>>>>>>> so we > >>>>>>>>>>>>>>>>>>>>>> could run > >>>>>>>>>>>>>>>>>>>>>>>>>>> the > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> total > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suite in about the > >> time > >>>> of > >>>>>>> the > >>>>>>>>>>> longest > >>>>>>>>>>>>> test, > >>>>>>>>>>>>>>>>> say > >>>>>>>>>>>>>>>>>>>>>> 20-30minutes? > >>>>>>>>>>>>>>>>>>>>>>>>>>>>> We > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> could > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> even change Apache to > >>>> run > >>>>>>> them > >>>>>>>>> this > >>>>>>>>>>>> way. > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FYI, > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> St.Ack > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sean > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>>>>>>>>>>>>>> busbey > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>>>>>>>>>>>> busbey > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>>>>>>>>>>> busbey > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>>>>>>> busbey > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>>> busbey > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>>> Sean > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>>> busbey > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> Michael Antonov > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> -- > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> -- Appy > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> -- > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> -- Appy > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> -- > >>>>>>>>>>>>> busbey > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> -- > >>>>>>>>>>>> Thanks, > >>>>>>>>>>>> Michael Antonov > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> -- > >>>>>>>>>>> > >>>>>>>>>>> -- Appy > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> > >>>>>>>>> -- Appy > >>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> -Dima > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> > >>>>>> -- Appy > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> busbey > >>> > >>> > >>> > >>> -- > >>> > >>> -- Appy > >> > >> > >> > >> -- > >> busbey > > > > > > > > -- > > Best regards, > > > > - Andy > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > > (via Tom White) > > -- -- Appy