Thanks, Added a JIRA. Let me know if I can assist further. Ron. On Fri, May 22, 2015 at 7:03 PM, Josh Wills <[email protected]> wrote:
> Hey Ron, > > It's a little tricky, but yeah, I think we could add a mode that would > simulate object re-use (at the very least during the reduce phase, where it > tends to cause most problems.) File a JIRA for it? > > Thanks, > J > > On Fri, May 22, 2015 at 7:18 AM, Ron Hashimshony < > [email protected]> wrote: > >> Hi, >> We love Crunch for the great Unit-Testing capabilities, which gives us a >> good confidence when running the pipeline on the real data. >> However, we did find one place in which the Unit-Tests failed us - when >> we need to add *getDettachedValue* call on *iterable*, unit-tests did >> behave differently and did not reuse the same objects, as happened in the >> production pipeline on the whole data. >> Is there any way to incorporate this validation in unit-tests? >> We have tests that run a set of *DoFn*s setting the input *MemCollection* >> s and check the *MemCollection*s out using *MemPipeline*, and other >> tests for a single *DoFn*'s *process* using *InMemoryEmitter*. >> Thanks, >> Ron Hashimshony >> > > > > -- > Director of Data Science > Cloudera <http://www.cloudera.com> > Twitter: @josh_wills <http://twitter.com/josh_wills> > -- *Ron Hashimshony* Back-End developer Mobile: +972-54-6617722 | [email protected] | www.myheritage.com MyHeritage Ltd., 3 Ariel Sharon St., Terminal Park, Or Yehuda 60250, Israel <http://www.myheritage.com/> <http://blog.myheritage.com/> <https://www.facebook.com/myheritage> <https://twitter.com/myheritage>
