I am using mahout-distribution 0.6. My first test programm of mahout FPGrowth
with a small data set
worked well (example1.txt).
In my second test programm I get this exception.
"Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 5
at org.apache.mahout.fpm.pfpgrowth.convertors.TransactionIterator$
1.apply(TransactionIterator.java:48)
at
org.apache.mahout.fpm.pfpgrowth.convertors.TransactionIterator$1.apply(TransactionIterator.java:42)
at com.google.common.collect.Iterators$8.next(Iterators.java:765)
at
com.google.common.collect.ForwardingIterator.next(ForwardingIterator.java:48)
at
org.apache.mahout.fpm.pfpgrowth.fpgrowth.FPGrowth.generateTopKFrequentPatterns(FPGrowth.java:290)
at
org.apache.mahout.fpm.pfpgrowth.fpgrowth.FPGrowth.generateTopKFrequentPatterns(FPGrowth.java:174)
at fpgrowth.Fpgrowth.frequentPatternMining(Fpgrowth.java:82)
at Main.main(Main.java:119)"
Then I added items to the itemsets in that way that each transaction has the
same length.
This also worked well. (exmaple2.txt).
I have two questions.
Is it neccessary to use itemsets with equal length?
(In my first data set I didn't use itemsets with equal length... )
Is it possible to use itemsets with duplicates in mahout FPGrowth?
https://cwiki.apache.org/confluence/display/MAHOUT/Mailing+Lists,+IRC+and+Archives
"Also, please send questions to this list to verify your problem before filing
issues in JIRA."
-> I don't think so, but is my problem perhaps a bug in mahout FPGrowth?
best regards
Jens
[[I1, I2, I5], [I2, I4], [I2, I3], [I1, I2, I4], [I1, I3], [I2, I3], [I1, I3],
[I1, I2, I3, I5], [I1, I2, I3]]
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Number of unique items 5
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Number of unique pruned items 5
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Number of Nodes in the FP Tree: 10
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 4
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Found 1 Patterns with Least Support 2
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 3
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Found 1 Patterns with Least Support 2
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 2
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Found 4 Patterns with Least Support 6
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 1
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Found 5 Patterns with Least Support 6
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 0
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Found 6 Patterns with Least Support 7
12/05/29 14:14:20 INFO fpgrowth.FPGrowth: Tree Cache: First Level: Cache
hits=10 Cache Misses=5
element and frequent itemsets with that element
I3=[([I3],6), ([I1, I3],4), ([I2, I3],4), ([I1, I2, I3],2)]
I4=[([I2, I4],2)]
I1=[([I1],6), ([I1, I2],4), ([I1, I3],4), ([I1, I2, I5],2), ([I1, I2, I3],2)]
I2=[([I2],7), ([I2, I3],4), ([I1, I2],4), ([I1, I2, I5],2), ([I1, I2, I3],2),
([I2, I4],2)]
I5=[([I1, I2, I5],2)]
//Input lines have equal length and no duplicates
[register request, examine casually, check ticket, decide, reinitiate request,
examine thoroughly, check ticket, decide, pay compensation]
[register request, check ticket, examine casually, decide, pay compensation,
test6, test7, test8, test9]
[register request, examine thoroughly, check ticket, decide, reject request,
test6, test7, test8, test9]
[register request, examine casually, check ticket, decide, pay compensation,
test6, test7, test8, test9]
[register request, examine casually, check ticket, decide, reinitiate request,
check ticket, examine casually, decide, reinitiate request]
[register request, check ticket, examine thoroughly, decide, reject request,
test6, test7, test8, test9]
12/05/29 14:47:16 INFO fpgrowth.FPGrowth: Number of unique items 8
12/05/29 14:47:16 INFO fpgrowth.FPGrowth: Number of unique pruned items 8
12/05/29 14:47:16 INFO fpgrowth.FPGrowth: Number of Nodes in the FP Tree: 18
12/05/29 14:47:16 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 7
12/05/29 14:47:17 INFO fpgrowth.FPGrowth: Found 1 Patterns with Least Support 4
12/05/29 14:47:17 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 6
12/05/29 14:47:17 INFO fpgrowth.FPGrowth: Found 1 Patterns with Least Support 4
12/05/29 14:47:17 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 5
12/05/29 14:47:17 INFO fpgrowth.FPGrowth: Found 1 Patterns with Least Support 4
12/05/29 14:47:17 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 4
12/05/29 14:47:17 INFO fpgrowth.FPGrowth: Found 1 Patterns with Least Support 4
12/05/29 14:47:17 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 3
12/05/29 14:47:17 INFO fpgrowth.FPGrowth: Found 1 Patterns with Least Support 5
12/05/29 14:47:17 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 2
12/05/29 14:47:18 INFO fpgrowth.FPGrowth: Found 4 Patterns with Least Support 6
12/05/29 14:47:18 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 1
12/05/29 14:47:18 INFO fpgrowth.FPGrowth: Found 5 Patterns with Least Support 8
12/05/29 14:47:18 INFO fpgrowth.FPGrowth: Mining FTree Tree for all patterns
with 0
12/05/29 14:47:20 INFO fpgrowth.FPGrowth: Found 5 Patterns with Least Support 8
12/05/29 14:47:20 INFO fpgrowth.FPGrowth: Tree Cache: First Level: Cache
hits=22 Cache Misses=14
element and frequent itemsets with that element
decide=[([check ticket, decide],8), ([check ticket, decide, register
request],6), ([check ticket, check ticket, decide, examine casually, register
request],5), ([check ticket, decide, register request, test6, test7, test8,
test9],4), ([check ticket, check ticket, decide, register request],4)]
examine casually=[([check ticket, check ticket, decide, examine casually,
register request],5)]
test8=[([check ticket, decide, register request, test6, test7, test8, test9],4)]
check ticket=[([check ticket, decide],8), ([check ticket, decide, register
request],6), ([check ticket, check ticket, decide, examine casually, register
request],5), ([check ticket, decide, register request, test6, test7, test8,
test9],4), ([check ticket, check ticket, decide, register request],4)]
test9=[([check ticket, decide, register request, test6, test7, test8, test9],4)]
test6=[([check ticket, decide, register request, test6, test7, test8, test9],4)]
test7=[([check ticket, decide, register request, test6, test7, test8, test9],4)]
register request=[([check ticket, decide, register request],6), ([check ticket,
check ticket, decide, examine casually, register request],5), ([check ticket,
decide, register request, test6, test7, test8, test9],4), ([check ticket, check
ticket, decide, register request],4)]
//Input lines have equal length but duplicates
[register request, examine casually, check ticket, decide, reinitiate request,
examine thoroughly, check ticket, decide, pay compensation]
[register request, check ticket, examine casually, decide, pay compensation,
Test, Test, Test, Test]
[register request, examine thoroughly, check ticket, decide, reject request,
Test, Test, Test, Test]
[register request, examine casually, check ticket, decide, pay compensation,
Test, Test, Test, Test]
[register request, examine casually, check ticket, decide, reinitiate request,
check ticket, examine casually, decide, reinitiate request]
[register request, check ticket, examine thoroughly, decide, reject request,
Test, Test, Test, Test]
12/05/29 14:51:28 INFO fpgrowth.FPGrowth: Number of unique items 5
12/05/29 14:51:28 INFO fpgrowth.FPGrowth: Number of unique pruned items 5
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 5
at
org.apache.mahout.fpm.pfpgrowth.convertors.TransactionIterator$1.apply(TransactionIterator.java:48)
at
org.apache.mahout.fpm.pfpgrowth.convertors.TransactionIterator$1.apply(TransactionIterator.java:42)
at com.google.common.collect.Iterators$8.next(Iterators.java:765)
at
com.google.common.collect.ForwardingIterator.next(ForwardingIterator.java:48)
at
org.apache.mahout.fpm.pfpgrowth.fpgrowth.FPGrowth.generateTopKFrequentPatterns(FPGrowth.java:290)
at
org.apache.mahout.fpm.pfpgrowth.fpgrowth.FPGrowth.generateTopKFrequentPatterns(FPGrowth.java:174)
at fpgrowth.Fpgrowth.frequentPatternMining(Fpgrowth.java:82)
at Main.main(Main.java:119)
//Input lines don't have equal length and no duplicates
[register request, examine casually, check ticket, decide, reinitiate request,
examine thoroughly, check ticket, decide, pay compensation]
[register request, check ticket, examine casually, decide, pay compensation]
[register request, examine thoroughly, check ticket, decide, reject request]
[register request, examine casually, check ticket, decide, pay compensation]
[register request, examine casually, check ticket, decide, reinitiate request,
check ticket, examine casually, decide, reinitiate request, examine casually,
check ticket, decide, reject request]
[register request, check ticket, examine thoroughly, decide, reject request]
12/05/29 14:57:05 INFO fpgrowth.FPGrowth: Number of unique items 4
12/05/29 14:57:05 INFO fpgrowth.FPGrowth: Number of unique pruned items 4
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 4
at
org.apache.mahout.fpm.pfpgrowth.convertors.TransactionIterator$1.apply(TransactionIterator.java:48)
at
org.apache.mahout.fpm.pfpgrowth.convertors.TransactionIterator$1.apply(TransactionIterator.java:42)
at com.google.common.collect.Iterators$8.next(Iterators.java:765)
at
com.google.common.collect.ForwardingIterator.next(ForwardingIterator.java:48)
at
org.apache.mahout.fpm.pfpgrowth.fpgrowth.FPGrowth.generateTopKFrequentPatterns(FPGrowth.java:290)
at
org.apache.mahout.fpm.pfpgrowth.fpgrowth.FPGrowth.generateTopKFrequentPatterns(FPGrowth.java:174)
at fpgrowth.Fpgrowth.frequentPatternMining(Fpgrowth.java:82)
at Main.main(Main.java:116)
