Dear Wiki user, You have subscribed to a wiki page or wiki category on "Pig Wiki" for change notification.
The following page has been changed by AlanGates: http://wiki.apache.org/pig/PigMix ------------------------------------------------------------------------------ == Runs == - == November 19, 2008 == - PigMix was run on a 25 machine hadoop cluster. The cluster was running hadoop version 0.18.1. The tests were run against two + The following table includes runs done of the pig mix. All of these runs have been done on a cluster with 26 slaves plus one machine acting as the name node and job tracker. The cluster was running + hadoop version 0.18.1. (TODO: Need to get specific hardware info on those machines). + + The tests were run against two versions of pig: top of trunk, and top of types branch both as of Nov 21 2008. - So far only the latency tests have been run, not the scalability tests. Equivalent map reduce programs have not yet been run. - The tests were run three times for each version and the results averaged. - || Query || Top of Trunk || Top of Types Branch || - || L1 explode || 261 || 283 || - || L2 fr join || 1665 || 253 || - || L3 join || 1912 || 320 || - || L4 distinct agg || 254 || 193 || - || L5 anti-join || 1535 || 281 || - || L6 large group by key || 294 || 226 || - || L7 nested split || 243 || 204 || - || L8 group all || 462 || 194 || - || L9 order by 1 field || 5294 || 867 || - || L10 order by multiple fields || 1403 || 565 || - || L11 distinct + union || 316 || 255 || - || L12 multi-store || fails || 781 || + tot = top of trunk + totb = top of types branch + + || Version || Map Reduce Java Code || tot 11/21/08 || totb 11/21/08 || totb 1/20/09 || + || Date Run || 11/22/08 || 11/21/08 || 11/21/08 || 1/20/09 || + || L1 explode || 116 || 261 || 283 || 218 || + || L2 fr join || 41 || 1665 || 253 || 168 || + || L3 join || 97 || 1912 || 320 || 258 || + || L4 distinct agg || 68 || 254 || 193 || 110 || + || L5 anti-join || 90 || 1535 || 281 || 209 || + || L6 large group by key || 61 || 294 || 226 || 126 || + || L7 nested split || 72 || 243 || 204 || 107 || + || L8 group all || 56 || 462 || 194 || 104 || + || L9 order by 1 field || 286 || 5294 || 867 || 851 || + || L10 order by multiple fields || 634 || 1403 || 565 || 469 || + || L11 distinct + union || 120 || 316 || 255 || 164 || + || L12 multi-store || 150 || fails || 781 || 499 || + || Total time || 1791 || 13638 || 4420 || 3284 || + || Compared to hadoop || 1.0 || 7.6 || 2.5 || 1.8 || + + The totb run of 1/20/09 includes the change to make BufferedPositionedInputStream use a buffer instead of relying on hadoop to buffer. == Features Tested == Based on a sample of user queries, PigMix includes tests for the following features.