<snip/>
b.
org.apache.accumulo.core.security.crypto.BlockedIOStreamTest.testGiantWrite.
This test assumes a max heap of about 1GB. This fails on IBM JRE,
since the default max heap is not specified, and on IBM JRE this depends
on the OS (see
http://www-01.ibm.com/support/knowledgecenter/SSYKE2_6.0.0/com.ibm.java.doc.diagnostics.60/diag/appendixes/defaults.html?lang=en
).
Proposal: add -Xmx1g to the surefire maven plugin reference in
parent maven pom.
This might be https://issues.apache.org/jira/browse/ACCUMULO-2774
Yup! I actually bumped this up to 1G already after I started seeing
failures (again) from the ACCUMULO-2774 patch which set a 768M heap.
Pull the upstream changes and feel free to submit something to address
any problem you still have.
> c. Both org.apache.accumulo.core.security.crypto.CrypoTest &
org.apache.accumulo.core.file.rfile.RFileTest have lots of failures due
to
calls to SEcureRandom with Random Number Generator Provider hard-coded as
Sun. The IBM JRE has it's own built in RNG Provider called IBMJCE. 2
issues - hard-coded calls to SecureRandom.getInstance(<algo>,"SUN") and
also default value in Property class is "SUN".
Proposal: Add mechanism to override default Property through
System property through new annotator in Property class. Only usage will
be by Property.CRYPTO_SECURE_RNG_PROVIDER
I'm not sure about adding new annotators to Property. However, the
CryptoTest should be getting the value from the conf instead of hard-coding
it. Then you can specify the correct value in accumulo-site.xml
I think another part of the issue is in
CryptoModuleFactory::fillParamsObjectFromStringMap because it looks like
that ignores the default setting.
>
2. Environment/Configuration
a. The generated configuration files contain references to GC
params that are specific to Sun JVM. In accumulo-env.sh, the
ACCUMULO_TSERVER_OPTS contains -XX:NewSize and -XX:MaxNewSize , and also
in ACCUMULO_GENERAL_OPTS,
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 are used.
b. in bin/accumulo, get ClassNotFoundException due to
specification of JAXP Doc Builder:
-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl
.
The Sun implementation of Document Builder Factory does not
exists
in IBM JDK, so a ClassNotFoundException is thrown on running accumulo
script
c. MiniAccumuloCluster - in the MiniAccumuloClusterImpl,
Sun-speciifc GC params are passed as params to the java process (similar
to section a. )
Single proposal for solving all three above issues:
Enhance bootstrap_config.sh with request to select Java vendor.
Selecting this will set correct values for GC params (they differ between
IBM and Sun), inclusion/ommision of JAXP setting. The
MiniAccumuloClusterImpl can read the same env variable that was set in
code for the GC Params, and use in the exec command.
I don't know enough about the IBM JDK to comment on this part
intelligently. Go ahead and generate a patch, and we can use that as a
starting point for discussion.
I'm a little hesitant to remove the CMS configuration (as it really
helps). My first thought about how to address this is you can submit
some example Accumulo configurations that work with IBM JDK or you can
add something to the configuration template/script (conf/examples and
conf/templates with bin/bootstrap_config.sh, respectively). I think
you're on the right path.
>
So far, my work has been focused on getting unit tests working for all
Java vendors in a clean manner. I have not yet run intensive testing of
real clusters following these changes, and would be happy to get pointers
to what else might need treatment.
Unit tests is a good first pass. Integration tests (mvn verify) is probably
the minimum that you want on your continuous integration once you have
things set up.
Accumulo also comes with a set of longer running, cluster based tests,
since we know that there are some pieces too complex for unit tests to
catch. have a look in the test module for the Continuous Ingest test. Once
you get to that point, we can help you set it up if the README is unclear.
I would also like to hear if these changes make sense, and if so, should
I go ahead and create some JIRAs, and attach my patches for commit
approval?
Filing JIRAs is going to be the most straightforward path, yes.
> Looking forward to hearing feedback!
Likewise. Looking forward to applying some patches!
