You are using the default timezone. If the timezones differ, this could account for the discrepancy. Based on your (de)serialization code, the long value should be the same; its interpretation is different.
stan
On Tue, Jun 26, 2012 at 10:20 AM, Dave Shine <
dave.sh...@channelintelligence.com> wrote:
> After about a week of researching, logging, etc. I have finally discovered
> what is happening, but I have no idea why.****
>
> ** **
>
> I have created my own WritableComparable object so I can emit it as the
> key from my Mapper. The object contains several Longs, one String, and one
> Date property. The following code snippets are from the object****
>
> ** **
>
> private Date SummaryDate;****
>
> ** **
>
> /******
>
> * @return the summaryDate****
>
> */****
>
> public Date getSummaryDate() {****
>
> return SummaryDate;****
>
> }****
>
> ** **
>
> /******
>
> * @param summaryDate the summaryDate to set****
>
> */****
>
> public void setSummaryDate(Date summaryDate) {****
>
> Calendar cal = Calendar.getInstance();****
>
> cal.setTime(summaryDate);****
>
> cal.set(Calendar.HOUR, 0);****
>
> cal.set(Calendar.MINUTE, 0);****
>
> cal.set(Calendar.SECOND, 0);****
>
> cal.set(Calendar.MILLISECOND, 0);****
>
> cal.set(Calendar.AM_PM, Calendar.AM);****
>
> SummaryDate = cal.getTime();****
>
> }****
>
> ** **
>
> @Override****
>
> public void readFields(DataInput arg0) throws IOException {
> ****
>
> ....****
>
> getSummaryDate().setTime(arg0.readLong());
> ****
>
> }****
>
> ** **
>
> @Override****
>
> public void write(DataOutput arg0) throws IOException {***
> *
>
> ....****
>
> arg0.writeLong(getSummaryDate().getTime());
> ****
>
> }****
>
> ** **
>
> The intent is for the Summary date to be always be as of midnight, thus
> the use of the Calendar object in the setSummaryDate() method. ****
>
> ** **
>
> I have proven via logging that the Mapper is storing the correct value in
> the SummaryDate property, but sometimes the value received by the Reducer
> is the previous day. Does anyone have any idea how this could happen? **
> **
>
> ** **
>
> My only theory is precision on the Long where the epoch time is actually
> stored, that it somehow loses a tick and becomes 1 millisecond before
> midnight, then my code drops the time and the date portion is left with a
> date that is one day earlier. Has anyone else seen anything like this?***
> *
>
> ** **
>
> I ready to go change my code to just store the date as a formatted
> string. But I’d really like to know if this is a known Java or Hadoop
> problem. FWIW, I’m using CDH3U4.****
>
> ** **
>
> Thanks,****
>
> *Dave Shine*****
>
> Sr. Software Engineer****
>
> 321.939.5093 direct | 407.314.0122 mobile****
>
> ** **
>
> [image: cid:D34AFA33-EA7B-4B08-9DD4-2C8DFBE66338]****
>
> *CI Boost™ Clients* *Outperform Online™ *www.ciboost.com****
>
> facebook platform | where-to-buy | product search engines | shopping
> engines****
>
> ** **
>
> ** **
>
> ** **
> ------------------------------
>
> The information contained in this email message is considered confidential
> and proprietary to the sender and is intended solely for review and use by
> the named recipient. Any unauthorized review, use or distribution is
> strictly prohibited. If you have received this message in error, please
> advise the sender by reply email and delete the message.****
>
> --
>
>
> ****
>
<<image001.png>>
