Does anyone have any code examples for how they persist join data across
multiple input splits and how they test it? Currently I populate a singleton in
the setup method of my mapper (along with having jvm reuse turned on for this
job) but with no way to have dependency injection into the mapper I am really
having a hard time with wrapping a UT around the code. I could have a package
scoped setter simply for testing purposes but that just feels dirty to be
honest. Any help is greatly appreciated and I have both MRUnit and Mockito at
my disposal.
private BitPackedMarkerMap markerMap =
BitPackedMarkerMapSingleton.getInstance().getMarkerMap();
private int numberOfIndividuals = -999;
private int numberOfAlleles = -999;
@Override
protected void setup(Context context) throws IOException,
InterruptedException {
LongPackedDoubleInteger inputSizes;
if(markerMap.getSize() == 0){
FileInputStream scoresInputStream = null;
try{
Path[] cacheFiles =
DistributedCache.getLocalCacheFiles(context.getConfiguration());
if (cacheFiles != null && cacheFiles.length > 0){
scoresInputStream = new FileInputStream(cacheFiles[0].toString());
inputSizes = markerMap.parse(scoresInputStream);
numberOfIndividuals = inputSizes.getInt1();
numberOfAlleles = inputSizes.getInt2();
}
} catch (IOException e){
System.err.println("Exception reading DistributedCache: " + e);
throw e;
}finally {
if(scoresInputStream != null){
scoresInputStream.close();
}
}
}
}
Matt
This e-mail message may contain privileged and/or confidential information, and
is intended to be received only by persons entitled
to receive such information. If you have received this e-mail in error, please
notify the sender immediately. Please delete it and
all attachments from any servers, hard drives or any other media. Other use of
this e-mail by you is strictly prohibited.
All e-mails and attachments sent and received are subject to monitoring,
reading and archival by Monsanto, including its
subsidiaries. The recipient of this e-mail is solely responsible for checking
for the presence of "Viruses" or other "Malware".
Monsanto, along with its subsidiaries, accepts no liability for any damage
caused by any such code transmitted by or accompanying
this e-mail or any attachment.
The information contained in this email may be subject to the export control
laws and regulations of the United States, potentially
including but not limited to the Export Administration Regulations (EAR) and
sanctions regulations issued by the U.S. Department of
Treasury, Office of Foreign Asset Controls (OFAC). As a recipient of this
information you are obligated to comply with all
applicable U.S. export laws and regulations.
