bq. (successfulAuction.timestampNanos - auction.timestampNanos) < 10000000L &&
Have you included the above condition into consideration when inspecting timestamps of the results ? On Tue, Jan 26, 2016 at 1:10 PM, Nkechi Achara <nkach...@googlemail.com> wrote: > > > down votefavorite > <http://stackoverflow.com/questions/35009560/anyone-had-issues-with-subtraction-of-a-long-within-an-rdd?noredirect=1#> > > I am having an issue with the subtraction of a long within an RDD to > filter out items in the RDD that are within a certain time range. > > So my code filters an RDD of case class auctions, with an object of > successfulAuctions(Long, Int, String): > > auctions.filter(it => relevantAuctions(it, successfulAuctions)) > > The successfulAuctions object is made up of a timestamp: Long, an itemID: > Int, and a direction: String (BUY/SELL). > > The relevantAuctions function basically uses tail recursion to find the > auctions in a time range for the exact item and direction. > > @tailrec > def relevantAuctions(auction: Auction, successfulAuctions: List[(Long, > String, String)]): Boolean = successfulAuctions match { > case sample :: xs => if (isRelevantAuction(auction, sample) ) true > else relevantAuctions(auction, xs) > case Nil => false > } > > This then feeds into another method in the if statement that checks the > timestamp in the sample is within a 10ms range, and the item ID is the > same, as is the direction. > > def isRelevantAuction(auction: Auction, successfulAuction: (Long, String, > String)): Boolean = {(successfulAuction.timestampNanos - > auction.timestampNanos) >= 0 && > (successfulAuction.timestampNanos - auction.timestampNanos) < 10000000L && > auction.itemID == successfulAuction.itemID && > auction.direction== successfulAuction.direction > } > > I am having issues where the range option is not entirely working. The > timestamps I am receiving back are not within the required range. Although > the Item ID and direction seems to be working successfully. > > The results I am getting are as follows, when I have a timestamp of > 1431651108749267459 for the successful auction, I am receiving other > auctions of a time GREATER than this, where it should be less. > > The auctions I am receiving have the timestamps of: > > 143165110874932660314316511087493307321431651108749537901 > > Has anyone experienced this phenomenon? > > Thanks! >
