Re: Review Request 64290: WIP: Update to Thrift 0.10.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64290/#review192672 --- Master (89338dd) is red with this patch. ./build-support/jenkins/build.sh INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. INFO] Reached consecutive success limit. --- Captured log call http_signaler.py52 DEBUGDEBUG] Using timeout 1.0 secs. health_checker.py 211 DEBUGDEBUG] Health checker thread started. health_checker.py 132 DEBUGDEBUG] Health checks enabled. Performing health check. health_checker.py 143 WARNING WARN] Health check failure: failure-1 health_checker.py 184 WARNING WARN] Ignoring failure of attempt: 1 health_checker.py 132 DEBUGDEBUG] Health checks enabled. Performing health check. health_checker.py 143 WARNING WARN] Health check failure: failure-2 health_checker.py 184 WARNING WARN] Ignoring failure of attempt: 2 health_checker.py 132 DEBUGDEBUG] Health checks enabled. Performing health check. health_checker.py 167 INFO INFO] Reached consecutive success limit. 1 failed, 794 passed, 6 skipped in 272.94 seconds FAILURE Waiting for background workers to finish. 07:51:32 05:12 [complete] FAILURE I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On Dec. 4, 2017, 7:27 a.m., John Sirois wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64290/ > --- > > (Updated Dec. 4, 2017, 7:27 a.m.) > > > Review request for Aurora, Stephan Erb and Bill Farner. > > > Repository: aurora > > > Description > --- > > Update to Thrift 0.10.0 (https://github.com/apache/thrift/blob/master/CHANGES) > > Included changes: > > * Update to the latest pants version. This was necesary to get ./pants gen > working. > * The Java hashcode option has been removed as it is now the default. > > .isort.cfg > | 1 + > 3rdparty/python/requirements.txt > | 2 +- > api/src/main/thrift/org/apache/aurora/gen/BUILD > | 10 +- > api/src/main/thrift/org/apache/thermos/BUILD > | 2 +- > build-support/jenkins/build.sh > | 3 +- > build-support/packer/build.sh > | 4 +- > build-support/python/checkstyle-check > | 2 +- > build-support/python/isort > | 32 - > build-support/python/isort-check > | 2 +- > build-support/python/isort-run > | 2 +- > build-support/release/make-python-sdists > | 2 +- > build-support/thrift/.gitignore > | 3 +- >
Re: Review Request 62590: WIP: Update to Thrift 0.10.0
> On Sept. 26, 2017, 5:53 p.m., Bill Farner wrote: > > ``` > > /bin/sh: cmake: command not found > > ``` > > > > But now i need to install cmake, so i'm not sure this pays off. > > Bill Farner wrote: > (this = the switch to cmake) > > Stephan Erb wrote: > Bison on MacOs is 10 years old. I assumed they have a good reason for it > and considered an added dependency a less risky endeavor. (Or is it just > negligence?) > > Bill Farner wrote: > What i meant to say - currently, i can build aurora on a stock macOS > machine with only a modern JDK and xcode command line utilities. Thrift > 0.10.0 seems to put us in the position of choosing between: > > a.) adding another step to our bootstrap routine to pre-build bison > b.) adding bison as a build-time dependency > c.) adding cmake as a build-time dependency > d.) other options? (i'd like to float the idea of hosting thrift binaries > like how pants does) > > Stephan Erb wrote: > Giving it some thought, you are right that just requring cmake does not > improve the situation in any way. > > I also like that we can build Aurora on a stock MacOS without much hassle > (or administrator rights). I think we should retain this property. This would > restrict us to options a) and d). > > Bill Farner wrote: > I would support (d). We can place unofficial binaries in svn for dev > platforms as needed, and could support using thrift from the `PATH` as a > fallback. This has the bonus of making from-scratch builds much faster. > > Stephan Erb wrote: > I saw your patch to https://github.com/morimekta/providence. Did you > manage to get something working with it? > > Bill Farner wrote: > I'm about 20% along. A very large mechanical patch is needed for Aurora. > The only untested piece is binary format compatibility (which providence > aims to achieve). There is some incompatibility i need to investigate > further (trivial details like a round-tripped `null` collection turning into > an empty collection), but so far it looks good! I'm proceeding with high > confidence, as there is also the future promise of a more approachable > HTTP/JSON interface via thrift IDL when using providence. > > Bill Farner wrote: > I should have clarified - i don't think we should hold back on upgrading > thrift. It's not yet guaranteed that the migration to providence will be > successful or timely. > > As for my proposed (d) above, we could store prebuilt binaries similarly > to how we store mesos eggs: https://svn.apache.org/repos/asf/aurora/3rdparty/ > > Stephan Erb wrote: > I have just pushed another patch that I have been sitting one for a few > days. It suffers from the same pants issues as > https://reviews.apache.org/r/63750/ The still incomplete solution: https://reviews.apache.org/r/64290/ - John --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62590/#review186342 --- On Dec. 3, 2017, 1:09 p.m., Stephan Erb wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62590/ > --- > > (Updated Dec. 3, 2017, 1:09 p.m.) > > > Review request for Aurora and Bill Farner. > > > Repository: aurora > > > Description > --- > > Update to Thrift 0.10.0 (https://github.com/apache/thrift/blob/master/CHANGES) > > Included changes: > > * Update to the latest pants version. This was necesary to make `./pants gen` > working. Unfortunately this breaks a few things. For details see > https://reviews.apache.org/r/63750/ > * The Java `hashcode` option has been removed as it is now the default. > > > Diffs > - > > 3rdparty/python/requirements.txt 155930dc6b808fb2f573f427a4508e388ee04b5a > build-support/packer/build.sh 85444125abc0c7e600a09933411e57c0d74051ac > build-support/thrift/AURORA-1727.compiler.cpp.Makefile.in.patch > 77c966caa3d1f644241bcc2b1968bc9306c56689 > > build-support/thrift/AURORA-1727.compiler.cpp.src.generate.t_java_generator.cc.patch > 42300b43a8f72e45c96b975e5d3a6a7bd0283529 > build-support/thrift/AURORA-1727.lib.py.setup.py.patch > 11c7b13341e156f3686511cb40ab13c1256203a6 > build-support/thrift/BUILD ab19f1f68682d88f731a463c15591e45a317e760 > build-support/thrift/Makefile f440b610afe321af663e393a29eebda7af7bd7a8 > build-support/thrift/prepare_binary.sh > 4ad997bf039294f7940b93a76ebf014689f8f618 > build-support/thrift/thriftw c8debd07bc9da97fb58db795e67c9ac82cc30bc1 > build.gradle af119910e84c48f75f2573ababcfa287c3b986fc > buildSrc/src/main/groovy/org/apache/aurora/build/ThriftPlugin.groovy > fc2bc9082dae2c63aa578c05dc89feb346260a67 > pants.ini 0671d9ab6381e5b9c324dc09a891a639cbfb2ccc >
Re: Review Request 63750: pants for high-sierra
> On Nov. 14, 2017, 5:13 p.m., Aurora ReviewBot wrote: > > Master (4fecf1f) is red with this patch. > > ./build-support/jenkins/build.sh > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/build_graph/build_file_aliases.py:52: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/aurora/client/cli:auroraversion will > > be removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > context_aware_object_factory(parse_context, *args, **kwargs) > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/engine/legacy/graph.py:173: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/thermos/observer:observer will be > > removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > return target_cls(build_graph=self, **kwargs) > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/engine/legacy/graph.py:173: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/aurora/client/cli:auroraversion will > > be removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > return target_cls(build_graph=self, **kwargs) > > > > > > 00:13:35 00:00 [main] > >(To run a reporting server: ./pants server) > > 00:13:35 00:00 [setup] > > 00:13:36 00:01 [parse] > >Executing tasks in goals: gen -> pyprep -> test > > 00:13:36 00:01 [gen] > > 00:13:36 00:01 [thrift-py] > > 00:13:36 00:01 [cache] > >No cached artifacts for 4 targets. > >Invalidated 4 targets.INFO] Attempting to fetch thrift > > binary from: > > /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/thrift/serve/bin/thrift/linux/x86_64/0.9.1/thrift > > ... > > 2% . 180 KB 4% .. 350 KB 6% ... 520 KB 8% 700 KB 10% . 870 KB 12% .. 1040 KB 14% ...1210 KB 16% 1390 KB 18% . 1560 KB 20% .. 1730 KB 22% ...1900 KB 24% 2080 KB 26% . 2250 KB 28% .. 2420 KB 30% ...2590 KB 32% 2770 KB 34% . 2940 KB 36% .. 3110 KB 38% ...3280 KB 40% 3460 KB 42% . 3630 KB 44% .. 3800 KB 46% ...3970 KB 48% 4150 KB 50% . 4320 KB 52% .. 4490 KB 54% ...4660 KB 56% 4840 KB 58% . 5010 KB 60% .. 5180 KB 62% ...5350 KB 64% 5530 KB 66% . 5700 KB 68% .. 5870 KB 70% ...6040 KB 72% 6220 KB 74% . 6390 KB 76% .. 6560 KB 78% ...6740 KB 80% 6910 KB 82% . 7080 KB 84% .. 7250 KB 86% ...7430 KB 88%
Review Request 64290: WIP: Update to Thrift 0.10.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64290/ --- Review request for Aurora, Stephan Erb and Bill Farner. Repository: aurora Description --- Update to Thrift 0.10.0 (https://github.com/apache/thrift/blob/master/CHANGES) Included changes: * Update to the latest pants version. This was necesary to get ./pants gen working. * The Java hashcode option has been removed as it is now the default. .isort.cfg | 1 + 3rdparty/python/requirements.txt | 2 +- api/src/main/thrift/org/apache/aurora/gen/BUILD | 10 +- api/src/main/thrift/org/apache/thermos/BUILD | 2 +- build-support/jenkins/build.sh | 3 +- build-support/packer/build.sh | 4 +- build-support/python/checkstyle-check | 2 +- build-support/python/isort | 32 - build-support/python/isort-check | 2 +- build-support/python/isort-run | 2 +- build-support/release/make-python-sdists | 2 +- build-support/thrift/.gitignore | 3 +- build-support/thrift/AURORA-1727.compiler.cpp.Makefile.in.patch | 370 -- build-support/thrift/AURORA-1727.compiler.cpp.src.generate.t_java_generator.cc.patch | 15 -- build-support/thrift/AURORA-1727.lib.py.setup.py.patch | 45 -- build-support/thrift/BUILD | 27 build-support/thrift/Makefile | 63 - build-support/thrift/prepare_binary.sh | 72 -- build-support/thrift/thriftw | 80 --- build.gradle | 2 +- buildSrc/src/main/groovy/org/apache/aurora/build/ThriftPlugin.groovy | 2 +- pants.ini | 18 +-- src/main/python/apache/aurora/client/cli/BUILD | 4 +- src/main/python/apache/thermos/observer/BUILD | 8 +- src/test/python/apache/aurora/__init__.py | 13 -- src/test/python/apache/aurora/admin/__init__.py | 13 -- src/test/python/apache/aurora/client/__init__.py | 13 -- src/test/python/apache/aurora/client/api/__init__.py | 13 -- src/test/python/apache/aurora/client/binding_helpers/__init__.py | 13 -- src/test/python/apache/aurora/client/cli/__init__.py | 13 -- src/test/python/apache/aurora/client/docker/__init__.py | 13 -- src/test/python/apache/aurora/common/health_check/__init__.py | 13 -- src/test/python/apache/aurora/executor/common/__init__.py | 13 -- src/test/python/apache/thermos/core/test_process.py | 38 +++--- 34 files changed, 110 insertions(+), 816 deletions(-) Diffs - .isort.cfg f53ccba7d1d45adcb4753cfd05c53bbec5c89dde 3rdparty/python/requirements.txt 155930dc6b808fb2f573f427a4508e388ee04b5a api/src/main/thrift/org/apache/aurora/gen/BUILD 835b0cc6f51a592f7d5b6129dbe9519455eb6555 api/src/main/thrift/org/apache/thermos/BUILD b58338e76bdee3481071d8f27235a9dbb6bf07b5 build-support/jenkins/build.sh f59fe278aea55fc3ffa1b5dff7b686cfcfae1ee7 build-support/packer/build.sh 85444125abc0c7e600a09933411e57c0d74051ac build-support/python/checkstyle-check 0c8800232e20347b2be9a538ffed1d47a7815241 build-support/python/isort 37e6cd23d2437055a15abfec6924c1bdd051de20 build-support/python/isort-check a1bb94c99250f54f8fc261fb311cd34ce8355c78 build-support/python/isort-run 1b39d41212d6e870c55d420a9d844fd67dd40eae build-support/release/make-python-sdists 51e11dbc1929609374031c9b23a96f035eba902b build-support/thrift/.gitignore 82c5bc5cd773ecf3ec194de41a340a3ef9b83aa5 build-support/thrift/AURORA-1727.compiler.cpp.Makefile.in.patch
Re: Review Request 64287: Clean up some lint rules
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64287/#review192668 --- Ship it! Ship It! - Stephan Erb On Dez. 4, 2017, 12:39 vorm., Bill Farner wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64287/ > --- > > (Updated Dez. 4, 2017, 12:39 vorm.) > > > Review request for Aurora and Stephan Erb. > > > Repository: aurora > > > Description > --- > > Many of these were completely stale, and there were a few simple code fixes > as well. > > > Diffs > - > > config/pmd/main.xml b90c221fefece5b62ae23007a87b308f7fed789a > config/spotbugs/excludeFilter.xml 51790cce8d9047e40741f05ee55af15dbdc3065e > src/main/java/org/apache/aurora/scheduler/http/Utilization.java > efacd8cdaa31c904370a76cba20b7e85a262b9f3 > src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java > f92306761706764a043b5d4384785ff20c435326 > > src/test/java/org/apache/aurora/scheduler/http/api/security/AuthorizeHeaderTokenTest.java > 9e956deb20bd4ec4821f4793d1a1cd18ed361c53 > > > Diff: https://reviews.apache.org/r/64287/diff/1/ > > > Testing > --- > > > Thanks, > > Bill Farner > >
Re: Review Request 64288: Add a SQL persistence implementation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64288/#review192666 --- Do we have any idea how this performs at scale? I'd like to avoid repeating the MyBatis work, where we delayed the scale testing until after we had committed a bunch of patches. - David McLaughlin On Dec. 4, 2017, 1:09 a.m., Bill Farner wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64288/ > --- > > (Updated Dec. 4, 2017, 1:09 a.m.) > > > Review request for Aurora, David McLaughlin, Jordan Ly, and Stephan Erb. > > > Repository: aurora > > > Description > --- > > Introduces a `Persistence` implementation that uses a SQL database via JDBC. > I've opted to lean towards MySQL SQL dialect, as unfortunately there are > vendor differences for even the very simple SQL used here. > > I chose [HikariCP](https://github.com/brettwooldridge/HikariCP) to serve as a > `DataSource` (connection pool) implementation. We don't really need much out > of a connection pool aside from general connection lifecycle management (i.e. > not for concurrency). I chose this library based on recent development > activity and several positive comparisons to other pools. > > Note that the implementation is not yet wired into the scheduler application. > That will come in a follow-up. > > > Diffs > - > > build.gradle af119910e84c48f75f2573ababcfa287c3b986fc > config/spotbugs/excludeFilter.xml 51790cce8d9047e40741f05ee55af15dbdc3065e > > src/main/java/org/apache/aurora/scheduler/storage/durability/DurableStorage.java > 85b2113631586f43d854c4d2812f43b7b864d452 > > src/main/java/org/apache/aurora/scheduler/storage/durability/Persistence.java > 9eb862c01bf451252bfbcc7a2eac60d2c965c9f0 > src/main/java/org/apache/aurora/scheduler/storage/log/LogPersistence.java > e70e6051582ca90ae72014626b983bbf4b8d5b48 > src/main/java/org/apache/aurora/scheduler/storage/sql/SqlPersistence.java > PRE-CREATION > > src/main/java/org/apache/aurora/scheduler/storage/sql/SqlPersistenceModule.java > PRE-CREATION > src/main/resources/org/apache/aurora/scheduler/storage/sql/schema.sql > PRE-CREATION > > src/test/java/org/apache/aurora/scheduler/storage/sql/SqlPersistenceTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/64288/diff/1/ > > > Testing > --- > > > Thanks, > > Bill Farner > >
Re: Review Request 64287: Clean up some lint rules
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64287/#review192665 --- Ship it! Ship It! - David McLaughlin On Dec. 3, 2017, 11:39 p.m., Bill Farner wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64287/ > --- > > (Updated Dec. 3, 2017, 11:39 p.m.) > > > Review request for Aurora and Stephan Erb. > > > Repository: aurora > > > Description > --- > > Many of these were completely stale, and there were a few simple code fixes > as well. > > > Diffs > - > > config/pmd/main.xml b90c221fefece5b62ae23007a87b308f7fed789a > config/spotbugs/excludeFilter.xml 51790cce8d9047e40741f05ee55af15dbdc3065e > src/main/java/org/apache/aurora/scheduler/http/Utilization.java > efacd8cdaa31c904370a76cba20b7e85a262b9f3 > src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java > f92306761706764a043b5d4384785ff20c435326 > > src/test/java/org/apache/aurora/scheduler/http/api/security/AuthorizeHeaderTokenTest.java > 9e956deb20bd4ec4821f4793d1a1cd18ed361c53 > > > Diff: https://reviews.apache.org/r/64287/diff/1/ > > > Testing > --- > > > Thanks, > > Bill Farner > >
Re: Review Request 64288: Add a SQL persistence implementation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64288/#review192664 --- Ship it! Master (89338dd) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On Dec. 3, 2017, 5:09 p.m., Bill Farner wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64288/ > --- > > (Updated Dec. 3, 2017, 5:09 p.m.) > > > Review request for Aurora, David McLaughlin, Jordan Ly, and Stephan Erb. > > > Repository: aurora > > > Description > --- > > Introduces a `Persistence` implementation that uses a SQL database via JDBC. > I've opted to lean towards MySQL SQL dialect, as unfortunately there are > vendor differences for even the very simple SQL used here. > > I chose [HikariCP](https://github.com/brettwooldridge/HikariCP) to serve as a > `DataSource` (connection pool) implementation. We don't really need much out > of a connection pool aside from general connection lifecycle management (i.e. > not for concurrency). I chose this library based on recent development > activity and several positive comparisons to other pools. > > Note that the implementation is not yet wired into the scheduler application. > That will come in a follow-up. > > > Diffs > - > > build.gradle af119910e84c48f75f2573ababcfa287c3b986fc > config/spotbugs/excludeFilter.xml 51790cce8d9047e40741f05ee55af15dbdc3065e > > src/main/java/org/apache/aurora/scheduler/storage/durability/DurableStorage.java > 85b2113631586f43d854c4d2812f43b7b864d452 > > src/main/java/org/apache/aurora/scheduler/storage/durability/Persistence.java > 9eb862c01bf451252bfbcc7a2eac60d2c965c9f0 > src/main/java/org/apache/aurora/scheduler/storage/log/LogPersistence.java > e70e6051582ca90ae72014626b983bbf4b8d5b48 > src/main/java/org/apache/aurora/scheduler/storage/sql/SqlPersistence.java > PRE-CREATION > > src/main/java/org/apache/aurora/scheduler/storage/sql/SqlPersistenceModule.java > PRE-CREATION > src/main/resources/org/apache/aurora/scheduler/storage/sql/schema.sql > PRE-CREATION > > src/test/java/org/apache/aurora/scheduler/storage/sql/SqlPersistenceTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/64288/diff/1/ > > > Testing > --- > > > Thanks, > > Bill Farner > >
Re: Review Request 64288: Add a SQL persistence implementation
> On Dec. 3, 2017, 5:22 p.m., Santhosh Kumar Shanmugham wrote: > > Can you list the details that will benefit us for pikcing HikariCP? Beyond what i wrote in the description, minimal dependencies (just slf4j) was the biggest. The benchmarks look favorable (and compare to the options i chose between), and there are lots of configuration knobs. Frankly, i think any pool implementation is likely to serve us well. Since it's easy to swap a connection pool, i didn't bother digging deep until i have first-hand experience of shortcomings in our use case. - Bill --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64288/#review192662 --- On Dec. 3, 2017, 5:09 p.m., Bill Farner wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64288/ > --- > > (Updated Dec. 3, 2017, 5:09 p.m.) > > > Review request for Aurora, David McLaughlin, Jordan Ly, and Stephan Erb. > > > Repository: aurora > > > Description > --- > > Introduces a `Persistence` implementation that uses a SQL database via JDBC. > I've opted to lean towards MySQL SQL dialect, as unfortunately there are > vendor differences for even the very simple SQL used here. > > I chose [HikariCP](https://github.com/brettwooldridge/HikariCP) to serve as a > `DataSource` (connection pool) implementation. We don't really need much out > of a connection pool aside from general connection lifecycle management (i.e. > not for concurrency). I chose this library based on recent development > activity and several positive comparisons to other pools. > > Note that the implementation is not yet wired into the scheduler application. > That will come in a follow-up. > > > Diffs > - > > build.gradle af119910e84c48f75f2573ababcfa287c3b986fc > config/spotbugs/excludeFilter.xml 51790cce8d9047e40741f05ee55af15dbdc3065e > > src/main/java/org/apache/aurora/scheduler/storage/durability/DurableStorage.java > 85b2113631586f43d854c4d2812f43b7b864d452 > > src/main/java/org/apache/aurora/scheduler/storage/durability/Persistence.java > 9eb862c01bf451252bfbcc7a2eac60d2c965c9f0 > src/main/java/org/apache/aurora/scheduler/storage/log/LogPersistence.java > e70e6051582ca90ae72014626b983bbf4b8d5b48 > src/main/java/org/apache/aurora/scheduler/storage/sql/SqlPersistence.java > PRE-CREATION > > src/main/java/org/apache/aurora/scheduler/storage/sql/SqlPersistenceModule.java > PRE-CREATION > src/main/resources/org/apache/aurora/scheduler/storage/sql/schema.sql > PRE-CREATION > > src/test/java/org/apache/aurora/scheduler/storage/sql/SqlPersistenceTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/64288/diff/1/ > > > Testing > --- > > > Thanks, > > Bill Farner > >
Re: Review Request 64288: Add a SQL persistence implementation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64288/#review192662 --- Can you list the details that will benefit us for pikcing HikariCP? - Santhosh Kumar Shanmugham On Dec. 3, 2017, 5:09 p.m., Bill Farner wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64288/ > --- > > (Updated Dec. 3, 2017, 5:09 p.m.) > > > Review request for Aurora, David McLaughlin, Jordan Ly, and Stephan Erb. > > > Repository: aurora > > > Description > --- > > Introduces a `Persistence` implementation that uses a SQL database via JDBC. > I've opted to lean towards MySQL SQL dialect, as unfortunately there are > vendor differences for even the very simple SQL used here. > > I chose [HikariCP](https://github.com/brettwooldridge/HikariCP) to serve as a > `DataSource` (connection pool) implementation. We don't really need much out > of a connection pool aside from general connection lifecycle management (i.e. > not for concurrency). I chose this library based on recent development > activity and several positive comparisons to other pools. > > Note that the implementation is not yet wired into the scheduler application. > That will come in a follow-up. > > > Diffs > - > > build.gradle af119910e84c48f75f2573ababcfa287c3b986fc > config/spotbugs/excludeFilter.xml 51790cce8d9047e40741f05ee55af15dbdc3065e > > src/main/java/org/apache/aurora/scheduler/storage/durability/DurableStorage.java > 85b2113631586f43d854c4d2812f43b7b864d452 > > src/main/java/org/apache/aurora/scheduler/storage/durability/Persistence.java > 9eb862c01bf451252bfbcc7a2eac60d2c965c9f0 > src/main/java/org/apache/aurora/scheduler/storage/log/LogPersistence.java > e70e6051582ca90ae72014626b983bbf4b8d5b48 > src/main/java/org/apache/aurora/scheduler/storage/sql/SqlPersistence.java > PRE-CREATION > > src/main/java/org/apache/aurora/scheduler/storage/sql/SqlPersistenceModule.java > PRE-CREATION > src/main/resources/org/apache/aurora/scheduler/storage/sql/schema.sql > PRE-CREATION > > src/test/java/org/apache/aurora/scheduler/storage/sql/SqlPersistenceTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/64288/diff/1/ > > > Testing > --- > > > Thanks, > > Bill Farner > >
Review Request 64288: Add a SQL persistence implementation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64288/ --- Review request for Aurora, David McLaughlin, Jordan Ly, and Stephan Erb. Repository: aurora Description --- Introduces a `Persistence` implementation that uses a SQL database via JDBC. I've opted to lean towards MySQL SQL dialect, as unfortunately there are vendor differences for even the very simple SQL used here. I chose [HikariCP](https://github.com/brettwooldridge/HikariCP) to serve as a `DataSource` (connection pool) implementation. We don't really need much out of a connection pool aside from general connection lifecycle management (i.e. not for concurrency). I chose this library based on recent development activity and several positive comparisons to other pools. Note that the implementation is not yet wired into the scheduler application. That will come in a follow-up. Diffs - build.gradle af119910e84c48f75f2573ababcfa287c3b986fc config/spotbugs/excludeFilter.xml 51790cce8d9047e40741f05ee55af15dbdc3065e src/main/java/org/apache/aurora/scheduler/storage/durability/DurableStorage.java 85b2113631586f43d854c4d2812f43b7b864d452 src/main/java/org/apache/aurora/scheduler/storage/durability/Persistence.java 9eb862c01bf451252bfbcc7a2eac60d2c965c9f0 src/main/java/org/apache/aurora/scheduler/storage/log/LogPersistence.java e70e6051582ca90ae72014626b983bbf4b8d5b48 src/main/java/org/apache/aurora/scheduler/storage/sql/SqlPersistence.java PRE-CREATION src/main/java/org/apache/aurora/scheduler/storage/sql/SqlPersistenceModule.java PRE-CREATION src/main/resources/org/apache/aurora/scheduler/storage/sql/schema.sql PRE-CREATION src/test/java/org/apache/aurora/scheduler/storage/sql/SqlPersistenceTest.java PRE-CREATION Diff: https://reviews.apache.org/r/64288/diff/1/ Testing --- Thanks, Bill Farner
Re: Review Request 63750: pants for high-sierra
> On Nov. 14, 2017, 4:13 p.m., Aurora ReviewBot wrote: > > Master (4fecf1f) is red with this patch. > > ./build-support/jenkins/build.sh > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/build_graph/build_file_aliases.py:52: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/aurora/client/cli:auroraversion will > > be removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > context_aware_object_factory(parse_context, *args, **kwargs) > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/engine/legacy/graph.py:173: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/thermos/observer:observer will be > > removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > return target_cls(build_graph=self, **kwargs) > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/engine/legacy/graph.py:173: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/aurora/client/cli:auroraversion will > > be removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > return target_cls(build_graph=self, **kwargs) > > > > > > 00:13:35 00:00 [main] > >(To run a reporting server: ./pants server) > > 00:13:35 00:00 [setup] > > 00:13:36 00:01 [parse] > >Executing tasks in goals: gen -> pyprep -> test > > 00:13:36 00:01 [gen] > > 00:13:36 00:01 [thrift-py] > > 00:13:36 00:01 [cache] > >No cached artifacts for 4 targets. > >Invalidated 4 targets.INFO] Attempting to fetch thrift > > binary from: > > /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/thrift/serve/bin/thrift/linux/x86_64/0.9.1/thrift > > ... > > 2% . 180 KB 4% .. 350 KB 6% ... 520 KB 8% 700 KB 10% . 870 KB 12% .. 1040 KB 14% ...1210 KB 16% 1390 KB 18% . 1560 KB 20% .. 1730 KB 22% ...1900 KB 24% 2080 KB 26% . 2250 KB 28% .. 2420 KB 30% ...2590 KB 32% 2770 KB 34% . 2940 KB 36% .. 3110 KB 38% ...3280 KB 40% 3460 KB 42% . 3630 KB 44% .. 3800 KB 46% ...3970 KB 48% 4150 KB 50% . 4320 KB 52% .. 4490 KB 54% ...4660 KB 56% 4840 KB 58% . 5010 KB 60% .. 5180 KB 62% ...5350 KB 64% 5530 KB 66% . 5700 KB 68% .. 5870 KB 70% ...6040 KB 72% 6220 KB 74% . 6390 KB 76% .. 6560 KB 78% ...6740 KB 80% 6910 KB 82% . 7080 KB 84% .. 7250 KB 86% ...7430 KB 88%
Re: Review Request 63750: pants for high-sierra
> On Nov. 14, 2017, 5:13 p.m., Aurora ReviewBot wrote: > > Master (4fecf1f) is red with this patch. > > ./build-support/jenkins/build.sh > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/build_graph/build_file_aliases.py:52: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/aurora/client/cli:auroraversion will > > be removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > context_aware_object_factory(parse_context, *args, **kwargs) > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/engine/legacy/graph.py:173: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/thermos/observer:observer will be > > removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > return target_cls(build_graph=self, **kwargs) > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/engine/legacy/graph.py:173: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/aurora/client/cli:auroraversion will > > be removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > return target_cls(build_graph=self, **kwargs) > > > > > > 00:13:35 00:00 [main] > >(To run a reporting server: ./pants server) > > 00:13:35 00:00 [setup] > > 00:13:36 00:01 [parse] > >Executing tasks in goals: gen -> pyprep -> test > > 00:13:36 00:01 [gen] > > 00:13:36 00:01 [thrift-py] > > 00:13:36 00:01 [cache] > >No cached artifacts for 4 targets. > >Invalidated 4 targets.INFO] Attempting to fetch thrift > > binary from: > > /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/thrift/serve/bin/thrift/linux/x86_64/0.9.1/thrift > > ... > > 2% . 180 KB 4% .. 350 KB 6% ... 520 KB 8% 700 KB 10% . 870 KB 12% .. 1040 KB 14% ...1210 KB 16% 1390 KB 18% . 1560 KB 20% .. 1730 KB 22% ...1900 KB 24% 2080 KB 26% . 2250 KB 28% .. 2420 KB 30% ...2590 KB 32% 2770 KB 34% . 2940 KB 36% .. 3110 KB 38% ...3280 KB 40% 3460 KB 42% . 3630 KB 44% .. 3800 KB 46% ...3970 KB 48% 4150 KB 50% . 4320 KB 52% .. 4490 KB 54% ...4660 KB 56% 4840 KB 58% . 5010 KB 60% .. 5180 KB 62% ...5350 KB 64% 5530 KB 66% . 5700 KB 68% .. 5870 KB 70% ...6040 KB 72% 6220 KB 74% . 6390 KB 76% .. 6560 KB 78% ...6740 KB 80% 6910 KB 82% . 7080 KB 84% .. 7250 KB 86% ...7430 KB 88%
Re: Review Request 64287: Clean up some lint rules
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64287/#review192659 --- Ship it! Master (89338dd) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On Dec. 3, 2017, 3:39 p.m., Bill Farner wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64287/ > --- > > (Updated Dec. 3, 2017, 3:39 p.m.) > > > Review request for Aurora and Stephan Erb. > > > Repository: aurora > > > Description > --- > > Many of these were completely stale, and there were a few simple code fixes > as well. > > > Diffs > - > > config/pmd/main.xml b90c221fefece5b62ae23007a87b308f7fed789a > config/spotbugs/excludeFilter.xml 51790cce8d9047e40741f05ee55af15dbdc3065e > src/main/java/org/apache/aurora/scheduler/http/Utilization.java > efacd8cdaa31c904370a76cba20b7e85a262b9f3 > src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java > f92306761706764a043b5d4384785ff20c435326 > > src/test/java/org/apache/aurora/scheduler/http/api/security/AuthorizeHeaderTokenTest.java > 9e956deb20bd4ec4821f4793d1a1cd18ed361c53 > > > Diff: https://reviews.apache.org/r/64287/diff/1/ > > > Testing > --- > > > Thanks, > > Bill Farner > >
Review Request 64287: Clean up some lint rules
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64287/ --- Review request for Aurora and Stephan Erb. Repository: aurora Description --- Many of these were completely stale, and there were a few simple code fixes as well. Diffs - config/pmd/main.xml b90c221fefece5b62ae23007a87b308f7fed789a config/spotbugs/excludeFilter.xml 51790cce8d9047e40741f05ee55af15dbdc3065e src/main/java/org/apache/aurora/scheduler/http/Utilization.java efacd8cdaa31c904370a76cba20b7e85a262b9f3 src/main/java/org/apache/aurora/scheduler/http/api/ApiBeta.java f92306761706764a043b5d4384785ff20c435326 src/test/java/org/apache/aurora/scheduler/http/api/security/AuthorizeHeaderTokenTest.java 9e956deb20bd4ec4821f4793d1a1cd18ed361c53 Diff: https://reviews.apache.org/r/64287/diff/1/ Testing --- Thanks, Bill Farner
Re: Review Request 63750: pants for high-sierra
> On Nov. 15, 2017, 1:13 a.m., Aurora ReviewBot wrote: > > Master (4fecf1f) is red with this patch. > > ./build-support/jenkins/build.sh > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/build_graph/build_file_aliases.py:52: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/aurora/client/cli:auroraversion will > > be removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > context_aware_object_factory(parse_context, *args, **kwargs) > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/engine/legacy/graph.py:173: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/thermos/observer:observer will be > > removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > return target_cls(build_graph=self, **kwargs) > > > > WARN] > > /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pants/setup/bootstrap-Linux-x86_64/1.4.0.dev20/local/lib/python2.7/site-packages/pants/engine/legacy/graph.py:173: > > DeprecationWarning: DEPRECATED: The `resources=` Python target argument > > found on target src/main/python/apache/aurora/client/cli:auroraversion will > > be removed in version 1.5.0.dev0. > > Depend on resources targets instead. > > return target_cls(build_graph=self, **kwargs) > > > > > > 00:13:35 00:00 [main] > >(To run a reporting server: ./pants server) > > 00:13:35 00:00 [setup] > > 00:13:36 00:01 [parse] > >Executing tasks in goals: gen -> pyprep -> test > > 00:13:36 00:01 [gen] > > 00:13:36 00:01 [thrift-py] > > 00:13:36 00:01 [cache] > >No cached artifacts for 4 targets. > >Invalidated 4 targets.INFO] Attempting to fetch thrift > > binary from: > > /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/thrift/serve/bin/thrift/linux/x86_64/0.9.1/thrift > > ... > > 2% . 180 KB 4% .. 350 KB 6% ... 520 KB 8% 700 KB 10% . 870 KB 12% .. 1040 KB 14% ...1210 KB 16% 1390 KB 18% . 1560 KB 20% .. 1730 KB 22% ...1900 KB 24% 2080 KB 26% . 2250 KB 28% .. 2420 KB 30% ...2590 KB 32% 2770 KB 34% . 2940 KB 36% .. 3110 KB 38% ...3280 KB 40% 3460 KB 42% . 3630 KB 44% .. 3800 KB 46% ...3970 KB 48% 4150 KB 50% . 4320 KB 52% .. 4490 KB 54% ...4660 KB 56% 4840 KB 58% . 5010 KB 60% .. 5180 KB 62% ...5350 KB 64% 5530 KB 66% . 5700 KB 68% .. 5870 KB 70% ...6040 KB 72% 6220 KB 74% . 6390 KB 76% .. 6560 KB 78% ...6740 KB 80% 6910 KB 82% . 7080 KB 84% .. 7250 KB 86% ...7430 KB 88%
Re: Review Request 62590: WIP: Update to Thrift 0.10.0
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62590/#review192648
---
Master (89338dd) is red with this patch.
./build-support/jenkins/build.sh
20:23:58 00:00 [setup]
20:23:58 00:00 [parse]
Executing tasks in goals: gen -> pyprep -> test
20:23:58 00:00 [gen]
20:23:58 00:00 [thrift-py]
20:23:58 00:00 [cache]
No cached artifacts for 1 target.
Invalidated 1 target..
20:23:58 00:00 [pyprep]
20:23:58 00:00 [interpreter]
20:24:02 00:04 [requirements]
20:24:02 00:04 [cache]
No cached artifacts for 34 targets.
Invalidated 34 targets. Failed to install
pykerberos-1.1.14 (caused by: NonZeroExit("received exit code 1 during
execution of `[u'/usr/bin/python2.7', '-', 'bdist_wheel',
'--dist-dir=/tmp/tmpVbU5Nf']` while trying to execute `[u'/usr/bin/python2.7',
'-', 'bdist_wheel', '--dist-dir=/tmp/tmpVbU5Nf']`",)
):
stdout:
running bdist_wheel
running build
running build_ext
building 'kerberos' extension
creating build
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes
-fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g
-fstack-protector-strong -Wformat -Werror=format-security -fPIC
-I/usr/include/python2.7 -c src/kerberos.c -o
build/temp.linux-x86_64-2.7/src/kerberos.o
stderr:
In file included from src/kerberos.c:19:0:
src/kerberosbasic.h:17:27: fatal error: gssapi/gssapi.h: No such file or
directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Waiting for background workers to finish.
20:24:21 00:23 [complete]
FAILURE
Exception caught: ()
Exception message: Package
SourcePackage(u'file:///home/jenkins/jenkins-slave/workspace/AuroraBot/.pants.d/python-setup/resolved_requirements/CPython-2.7.12/pykerberos-1.1.14.tar.gz')
is not translateable by ChainedTranslator(WheelTranslator, EggTranslator,
SourceTranslator)
I will refresh this build result if you post a review containing "@ReviewBot
retry"
- Aurora ReviewBot
On Dec. 3, 2017, 12:09 p.m., Stephan Erb wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62590/
> ---
>
> (Updated Dec. 3, 2017, 12:09 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> ---
>
> Update to Thrift 0.10.0 (https://github.com/apache/thrift/blob/master/CHANGES)
>
> Included changes:
>
> * Update to the latest pants version. This was necesary to make `./pants gen`
> working. Unfortunately this breaks a few things. For details see
> https://reviews.apache.org/r/63750/
> * The Java `hashcode` option has been removed as it is now the default.
>
>
> Diffs
> -
>
> 3rdparty/python/requirements.txt 155930dc6b808fb2f573f427a4508e388ee04b5a
> build-support/packer/build.sh 85444125abc0c7e600a09933411e57c0d74051ac
> build-support/thrift/AURORA-1727.compiler.cpp.Makefile.in.patch
> 77c966caa3d1f644241bcc2b1968bc9306c56689
>
> build-support/thrift/AURORA-1727.compiler.cpp.src.generate.t_java_generator.cc.patch
> 42300b43a8f72e45c96b975e5d3a6a7bd0283529
> build-support/thrift/AURORA-1727.lib.py.setup.py.patch
> 11c7b13341e156f3686511cb40ab13c1256203a6
> build-support/thrift/BUILD ab19f1f68682d88f731a463c15591e45a317e760
> build-support/thrift/Makefile f440b610afe321af663e393a29eebda7af7bd7a8
> build-support/thrift/prepare_binary.sh
> 4ad997bf039294f7940b93a76ebf014689f8f618
> build-support/thrift/thriftw c8debd07bc9da97fb58db795e67c9ac82cc30bc1
> build.gradle af119910e84c48f75f2573ababcfa287c3b986fc
> buildSrc/src/main/groovy/org/apache/aurora/build/ThriftPlugin.groovy
> fc2bc9082dae2c63aa578c05dc89feb346260a67
> pants.ini 0671d9ab6381e5b9c324dc09a891a639cbfb2ccc
> src/main/python/apache/aurora/executor/BUILD
> 486230db34a22ea5dd0f68da911c0afb1afbcac0
>
>
> Diff: https://reviews.apache.org/r/62590/diff/2/
>
>
> Testing
> ---
>
> ./build-support/jenkins/build.sh
>
>
> Thanks,
>
> Stephan Erb
>
>
Re: Review Request 62590: WIP: Update to Thrift 0.10.0
> On Sept. 27, 2017, 1:53 a.m., Bill Farner wrote: > > ``` > > /bin/sh: cmake: command not found > > ``` > > > > But now i need to install cmake, so i'm not sure this pays off. > > Bill Farner wrote: > (this = the switch to cmake) > > Stephan Erb wrote: > Bison on MacOs is 10 years old. I assumed they have a good reason for it > and considered an added dependency a less risky endeavor. (Or is it just > negligence?) > > Bill Farner wrote: > What i meant to say - currently, i can build aurora on a stock macOS > machine with only a modern JDK and xcode command line utilities. Thrift > 0.10.0 seems to put us in the position of choosing between: > > a.) adding another step to our bootstrap routine to pre-build bison > b.) adding bison as a build-time dependency > c.) adding cmake as a build-time dependency > d.) other options? (i'd like to float the idea of hosting thrift binaries > like how pants does) > > Stephan Erb wrote: > Giving it some thought, you are right that just requring cmake does not > improve the situation in any way. > > I also like that we can build Aurora on a stock MacOS without much hassle > (or administrator rights). I think we should retain this property. This would > restrict us to options a) and d). > > Bill Farner wrote: > I would support (d). We can place unofficial binaries in svn for dev > platforms as needed, and could support using thrift from the `PATH` as a > fallback. This has the bonus of making from-scratch builds much faster. > > Stephan Erb wrote: > I saw your patch to https://github.com/morimekta/providence. Did you > manage to get something working with it? > > Bill Farner wrote: > I'm about 20% along. A very large mechanical patch is needed for Aurora. > The only untested piece is binary format compatibility (which providence > aims to achieve). There is some incompatibility i need to investigate > further (trivial details like a round-tripped `null` collection turning into > an empty collection), but so far it looks good! I'm proceeding with high > confidence, as there is also the future promise of a more approachable > HTTP/JSON interface via thrift IDL when using providence. > > Bill Farner wrote: > I should have clarified - i don't think we should hold back on upgrading > thrift. It's not yet guaranteed that the migration to providence will be > successful or timely. > > As for my proposed (d) above, we could store prebuilt binaries similarly > to how we store mesos eggs: https://svn.apache.org/repos/asf/aurora/3rdparty/ I have just pushed another patch that I have been sitting one for a few days. It suffers from the same pants issues as https://reviews.apache.org/r/63750/ - Stephan --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62590/#review186342 --- On Dec. 3, 2017, 9:09 p.m., Stephan Erb wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62590/ > --- > > (Updated Dec. 3, 2017, 9:09 p.m.) > > > Review request for Aurora and Bill Farner. > > > Repository: aurora > > > Description > --- > > Update to Thrift 0.10.0 (https://github.com/apache/thrift/blob/master/CHANGES) > > Included changes: > > * Update to the latest pants version. This was necesary to make `./pants gen` > working. Unfortunately this breaks a few things. For details see > https://reviews.apache.org/r/63750/ > * The Java `hashcode` option has been removed as it is now the default. > > > Diffs > - > > 3rdparty/python/requirements.txt 155930dc6b808fb2f573f427a4508e388ee04b5a > build-support/packer/build.sh 85444125abc0c7e600a09933411e57c0d74051ac > build-support/thrift/AURORA-1727.compiler.cpp.Makefile.in.patch > 77c966caa3d1f644241bcc2b1968bc9306c56689 > > build-support/thrift/AURORA-1727.compiler.cpp.src.generate.t_java_generator.cc.patch > 42300b43a8f72e45c96b975e5d3a6a7bd0283529 > build-support/thrift/AURORA-1727.lib.py.setup.py.patch > 11c7b13341e156f3686511cb40ab13c1256203a6 > build-support/thrift/BUILD ab19f1f68682d88f731a463c15591e45a317e760 > build-support/thrift/Makefile f440b610afe321af663e393a29eebda7af7bd7a8 > build-support/thrift/prepare_binary.sh > 4ad997bf039294f7940b93a76ebf014689f8f618 > build-support/thrift/thriftw c8debd07bc9da97fb58db795e67c9ac82cc30bc1 > build.gradle af119910e84c48f75f2573ababcfa287c3b986fc > buildSrc/src/main/groovy/org/apache/aurora/build/ThriftPlugin.groovy > fc2bc9082dae2c63aa578c05dc89feb346260a67 > pants.ini 0671d9ab6381e5b9c324dc09a891a639cbfb2ccc > src/main/python/apache/aurora/executor/BUILD > 486230db34a22ea5dd0f68da911c0afb1afbcac0 > > > Diff:
Re: Review Request 62590: WIP: Update to Thrift 0.10.0
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62590/ --- (Updated Dec. 3, 2017, 9:09 p.m.) Review request for Aurora and Bill Farner. Changes --- Use the Thrift binary bootstrapped by Pants also for the Java build. Repository: aurora Description (updated) --- Update to Thrift 0.10.0 (https://github.com/apache/thrift/blob/master/CHANGES) Included changes: * Update to the latest pants version. This was necesary to make `./pants gen` working. Unfortunately this breaks a few things. For details see https://reviews.apache.org/r/63750/ * The Java `hashcode` option has been removed as it is now the default. Diffs (updated) - 3rdparty/python/requirements.txt 155930dc6b808fb2f573f427a4508e388ee04b5a build-support/packer/build.sh 85444125abc0c7e600a09933411e57c0d74051ac build-support/thrift/AURORA-1727.compiler.cpp.Makefile.in.patch 77c966caa3d1f644241bcc2b1968bc9306c56689 build-support/thrift/AURORA-1727.compiler.cpp.src.generate.t_java_generator.cc.patch 42300b43a8f72e45c96b975e5d3a6a7bd0283529 build-support/thrift/AURORA-1727.lib.py.setup.py.patch 11c7b13341e156f3686511cb40ab13c1256203a6 build-support/thrift/BUILD ab19f1f68682d88f731a463c15591e45a317e760 build-support/thrift/Makefile f440b610afe321af663e393a29eebda7af7bd7a8 build-support/thrift/prepare_binary.sh 4ad997bf039294f7940b93a76ebf014689f8f618 build-support/thrift/thriftw c8debd07bc9da97fb58db795e67c9ac82cc30bc1 build.gradle af119910e84c48f75f2573ababcfa287c3b986fc buildSrc/src/main/groovy/org/apache/aurora/build/ThriftPlugin.groovy fc2bc9082dae2c63aa578c05dc89feb346260a67 pants.ini 0671d9ab6381e5b9c324dc09a891a639cbfb2ccc src/main/python/apache/aurora/executor/BUILD 486230db34a22ea5dd0f68da911c0afb1afbcac0 Diff: https://reviews.apache.org/r/62590/diff/2/ Changes: https://reviews.apache.org/r/62590/diff/1-2/ Testing (updated) --- ./build-support/jenkins/build.sh Thanks, Stephan Erb
Re: Review Request 64284: Expose thrift workload stats on pruneTasks() invocations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64284/#review192646 --- Ship it! Ship It! - Bill Farner On Dec. 3, 2017, 5:20 a.m., Jing Chen wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64284/ > --- > > (Updated Dec. 3, 2017, 5:20 a.m.) > > > Review request for Aurora, Mehrdad Nurolahzade and Stephan Erb. > > > Bugs: AURORA-1901 > https://issues.apache.org/jira/browse/AURORA-1901 > > > Repository: aurora > > > Description > --- > > Expose thrift workload stats on pruneTasks() invocations > > > Diffs > - > > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > a519b0728b02b5ccdaaa5566566aae4691f07ffa > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 883738446a093e464630d06f1241832129165fcc > > > Diff: https://reviews.apache.org/r/64284/diff/1/ > > > Testing > --- > > curl 192.168.33.7:8081/vars | grep thrift_workload_pruneTasks > % Total% Received % Xferd Average Speed TimeTime Time > Current > Dload Upload Total SpentLeft Speed > 100 260530 260530 0 4074k 0 --:--:-- --:--:-- --:--:-- 4240k > thrift_workload_pruneTasks 0 > > > Thanks, > > Jing Chen > >
Re: Review Request 64284: Expose thrift workload stats on pruneTasks() invocations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64284/#review192645 --- Ship it! Ship It! - Stephan Erb On Dez. 3, 2017, 2:20 nachm., Jing Chen wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64284/ > --- > > (Updated Dez. 3, 2017, 2:20 nachm.) > > > Review request for Aurora, Mehrdad Nurolahzade and Stephan Erb. > > > Bugs: AURORA-1901 > https://issues.apache.org/jira/browse/AURORA-1901 > > > Repository: aurora > > > Description > --- > > Expose thrift workload stats on pruneTasks() invocations > > > Diffs > - > > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > a519b0728b02b5ccdaaa5566566aae4691f07ffa > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 883738446a093e464630d06f1241832129165fcc > > > Diff: https://reviews.apache.org/r/64284/diff/1/ > > > Testing > --- > > curl 192.168.33.7:8081/vars | grep thrift_workload_pruneTasks > % Total% Received % Xferd Average Speed TimeTime Time > Current > Dload Upload Total SpentLeft Speed > 100 260530 260530 0 4074k 0 --:--:-- --:--:-- --:--:-- 4240k > thrift_workload_pruneTasks 0 > > > Thanks, > > Jing Chen > >
Re: Review Request 64284: Expose thrift workload stats on pruneTasks() invocations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64284/#review192644 --- Ship it! Master (cea43db) is green with this patch. ./build-support/jenkins/build.sh I will refresh this build result if you post a review containing "@ReviewBot retry" - Aurora ReviewBot On Dec. 3, 2017, 1:20 p.m., Jing Chen wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64284/ > --- > > (Updated Dec. 3, 2017, 1:20 p.m.) > > > Review request for Aurora, Mehrdad Nurolahzade and Stephan Erb. > > > Bugs: AURORA-1901 > https://issues.apache.org/jira/browse/AURORA-1901 > > > Repository: aurora > > > Description > --- > > Expose thrift workload stats on pruneTasks() invocations > > > Diffs > - > > > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java > a519b0728b02b5ccdaaa5566566aae4691f07ffa > > src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java > 883738446a093e464630d06f1241832129165fcc > > > Diff: https://reviews.apache.org/r/64284/diff/1/ > > > Testing > --- > > curl 192.168.33.7:8081/vars | grep thrift_workload_pruneTasks > % Total% Received % Xferd Average Speed TimeTime Time > Current > Dload Upload Total SpentLeft Speed > 100 260530 260530 0 4074k 0 --:--:-- --:--:-- --:--:-- 4240k > thrift_workload_pruneTasks 0 > > > Thanks, > > Jing Chen > >
Review Request 64284: Expose thrift workload stats on pruneTasks() invocations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64284/ --- Review request for Aurora, Mehrdad Nurolahzade and Stephan Erb. Bugs: AURORA-1901 https://issues.apache.org/jira/browse/AURORA-1901 Repository: aurora Description --- Expose thrift workload stats on pruneTasks() invocations Diffs - src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a519b0728b02b5ccdaaa5566566aae4691f07ffa src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 883738446a093e464630d06f1241832129165fcc Diff: https://reviews.apache.org/r/64284/diff/1/ Testing --- curl 192.168.33.7:8081/vars | grep thrift_workload_pruneTasks % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 260530 260530 0 4074k 0 --:--:-- --:--:-- --:--:-- 4240k thrift_workload_pruneTasks 0 Thanks, Jing Chen
Re: Review Request 64283: Remove redundant transaction recorder
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64283/#review192643 --- Ship it! Ship It! - Stephan Erb On Dez. 3, 2017, 6:53 vorm., Bill Farner wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64283/ > --- > > (Updated Dez. 3, 2017, 6:53 vorm.) > > > Review request for Aurora and Jordan Ly. > > > Repository: aurora > > > Description > --- > > I missed this in https://reviews.apache.org/r/64234/. The intention of the > newly-introduced `TransactionRecorder` was to replace the code removed here. > > > Diffs > - > > src/main/java/org/apache/aurora/scheduler/storage/log/LogPersistence.java > a0a6b6c7270d2b7f5eae2ea218e17b929f9bbd53 > src/main/java/org/apache/aurora/scheduler/storage/log/StreamManager.java > 18da32daf4a86966c67fd5c6673b32497ccb8f35 > > src/main/java/org/apache/aurora/scheduler/storage/log/StreamManagerImpl.java > c5b107fe98597bb38589d654bbd619e7ad15b62b > > src/main/java/org/apache/aurora/scheduler/storage/log/StreamTransaction.java > a51fd18ad00537bf244419442078548d8545a841 > src/test/java/org/apache/aurora/scheduler/storage/log/LogManagerTest.java > cb38f107385997b091c3b65db3a13f9cca0fa42d > > > Diff: https://reviews.apache.org/r/64283/diff/1/ > > > Testing > --- > > > Thanks, > > Bill Farner > >
