[
https://issues.apache.org/jira/browse/PIG-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13857790#comment-13857790
]
Rohini Palaniswamy commented on PIG-3645:
-----------------------------------------
FileLocalizer.setR(new Random(1331L)); - If you do this in setup instead of
setUpBeforeClass in TestMRCompiler, I think you will not need to regenerate the
golden files. But if you still need to regenerate the golden files, is it
possible for you to copy over what is done in TestTezCompiler setup() method.
That makes each test's golden file independent of another and allows tests to
be run in any order or just run individually from eclipse without having to run
previous tests. Doing that in a separate jira is fine as well. Everything else
looks good. Will do a +1 once you confirm full unit test suite passes.
> Move FileLocalizer.setR() calls to unit tests
> ---------------------------------------------
>
> Key: PIG-3645
> URL: https://issues.apache.org/jira/browse/PIG-3645
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Reporter: Cheolsoo Park
> Assignee: Cheolsoo Park
> Priority: Minor
> Fix For: 0.13.0
>
> Attachments: PIG-3645-1.patch, PIG-3645-2.patch, PIG-3645-3.patch
>
>
> Currently, temporary paths are generated by FileLocalizer using
> Random.nextInt(). To provide strong randomness, MapReduceLauncher resets the
> Random object every time when compiling physical plan to MR plan:
> {code}
> MRCompiler comp = new MRCompiler(php, pc);
> comp.randomizeFileLocalizer(); // This in turn calls FileLocalizer.setR(new
> Random()).
> {code}
> Besides, there are a couple of places calling FileLocalizer.setR() (e.g.
> MRCompiler) with some random seed.
> I think-
> # Randomizing Random seed is unnecessary if we switch to UUID.
> # Setting Random objects in code like this is error-prone because it can be
> easily broken by having or missing a FileLocalizer.setR() somewhere else. See
> an example [here|http://search-hadoop.com/m/2nxTzQXfHw1].
> So I propose that we remove all this "randomizing Random seed" code and use
> UUID instead in temporary paths.
> For unit tests that compare the results against gold files, we should still
> allow to set Random seed through FileLocalizer.setR(). But this method will
> be annotated as "VisibleForTesting" to ensure it is not used nowhere else
> other than in unit tests.
> Regarding the existing gold files, they can be easily regenerated by
> TestMRCompiler as follows-
> {code}
> FileOutputStream fos = new FileOutputStream(expectedFile + "_new");
> PrintWriter pw = new PrintWriter(fos);
> pw.write(compiledPlan);
> {code}
> I assume there won't be any kind of regressions due to this change. But
> please let me know if I am wrong.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
