[
https://issues.apache.org/jira/browse/BIGTOP-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14326391#comment-14326391
]
Konstantin Boudnik commented on BIGTOP-1680:
--------------------------------------------
But BPS compilation is happening before Spark package is installed, is it? If
so, then Scala RT redistributed in Spark package is irrelevant to this
conversation.
However, in order to build Spark package in the first place we need to have
Scale dev. env. And that's already provided via
{{bigtop_toolchain/manifests/scala.pp}}. As I said - I barked on a wrong tree.
Sorry.
> [BigPetStore] BPS-Spark Unit tests for transaction generation
> -------------------------------------------------------------
>
> Key: BIGTOP-1680
> URL: https://issues.apache.org/jira/browse/BIGTOP-1680
> Project: Bigtop
> Issue Type: Test
> Components: blueprints
> Affects Versions: backlog
> Reporter: jay vyas
> Assignee: RJ Nowling
> Fix For: backlog
>
>
> The following code could be split up into a method, i think, and would be
> greatly served by a companion unit test.
> Its the reference impl of the data generator on spark: i.e. this is where we
> magically generate all the customer transactions .
> {noformat}
> 155 val transactionRDD = customerRDD.mapPartitionsWithIndex{
> 156 (index, custIter) =>
> 157 // Create a new RNG
> 158 val seedFactory = new SeedFactory(nextSeed ^ index)
> 159 val transactionIter = custIter.map{
> 160 customer =>
> 161 val products = productBC.value
> 162 //Create a new purchasing profile.
> 163 val profileGen = new PurchasingProfileGenerator(products,
> seedFactory)
> 164 val profile = profileGen.generate()
> 165 val transGen = new TransactionGenerator(customer, profile,
> storesBC.value, products, seedFact ory)
> 166 var transactions : List[Transaction] = List()
> 167 var transaction = transGen.generate()
> 168
> 169 //Create a list of this customer's transactions for the time
> period
> 170 while(transaction.getDateTime() < simulationLength) {
> 171 transactions = transaction :: transactions
> 172 transaction = transGen.generate()
> 173 }
> 174 //The final result, we return the list of transactions produced
> above.
> 175 transactions
> 176 }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
