Hey John,
Could you give an example of one of the ranges you are using which causes
this to happen?
On Fri, Aug 22, 2014 at 11:02 PM, John Yost soozandjohny...@gmail.com
wrote:
Hey Everyone,
The AccumuloMultiTableInputFormat is an awesome addition to the Accumulo
API and I am really
Hey Corey,
Sure thing! Here is my code:
MapString,InputTableConfig configs = new
HashMapString,InputTableConfig();
ListRange ranges = Lists.newArrayList(new Range(104587),new
Range(105255));
The table configs get serialized as base64 and placed in the job's
Configuration under the key AccumuloInputFormat.ScanOpts.TableConfigs.
Could you verify/print what's being placed in this key in your
configuration?
On Sat, Aug 23, 2014 at 12:15 AM, JavaHokie soozandjohny...@gmail.com
wrote:
Hey Corey,
Gotcha, i get a null when I attempt to log the value:
log.debug(configuration.get(AccumuloInputFormat.ScanOpts.TableConfigs));
--John
--
View this message in context:
The tests I'm running aren't using the native Hadoop libs either. If you
don't mind, a little more code as to how you are setting up your job would
be useful. That's weird the key in the config would be null. Are you using
the job.getConfiguration()?
On Sat, Aug 23, 2014 at 12:31 AM, JavaHokie
Sure thing, here's my run method implementation:
Configuration configuration = new Configuration();
configuration.set(fs.defaultFS, hdfs://127.0.0.1:8020);
configuration.set(mapreduce.job.tracker, localhost:54311);
Also, if you don't mind me asking, why isn't your job setup class extending
Configured? That was you are picking up configurations injected from the
environment.
You would do MyJobSetUpClass extends Configured Then use getConf()
instead of newing up a new configuration.
On Sat, Aug 23, 2014 at
Hey Corey,
Argh, that's right, sorry about that. OK, the value is still null with the
following:
log.debug(job.getConfiguration().get(AccumuloInputFormat.ScanOpts.TableConfigs));
Certainly explains the IOException...readFields does not like null DataInput
objects. :)
--John
--
View this
Agreed, should have used getConf(), I cleaned that up, so now things look
like this:
Job job = Job.getInstance(getConf());
/*
* Set the basic stuff
*/
job.setJobName(TwitterJoin Query);
H...the byte[] array is generated OK.
byte[] bytes =
Base64.decodeBase64(configString.getBytes(StandardCharsets.UTF_8));
I wonder what's golng wrong with one of these lines below?
ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
mapWritable.readFields(new
That code I posted should be able to validate where you are getting hung
up. Can you try running that on the machine and seeing if it prints the
expected tables/ranges?
Also, are you running the job live? What does the configuration look like
for the job on your resource manager? Can you see if
11 matches
Mail list logo
