[jira] [Commented] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-18 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13479402#comment-13479402 ] Koji Noguchi commented on PIG-2975: --- bq. Result incorrect (when order-by used). [0.11 and

[jira] [Commented] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-18 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13479404#comment-13479404 ] Koji Noguchi commented on PIG-2975: --- Silly me. Result of above script was {noformat} (0)

[jira] [Commented] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480103#comment-13480103 ] Koji Noguchi commented on PIG-2975: --- bq. I don't think we need to sacrifice performance if

[jira] [Commented] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480153#comment-13480153 ] Koji Noguchi commented on PIG-2975: --- bq. I am going to roll up my sleeves and do this

[jira] [Updated] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-2975: -- Attachment: PIG-2975-0_jco-v2.patch Thanks Jonathan! bq. 1. we could just do the comparison via calls

[jira] [Updated] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-2975: -- Attachment: pig-2975-trunk_v03-unionapproach.txt bq. 2. We could just make a custom WritableComparator

[jira] [Commented] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480296#comment-13480296 ] Koji Noguchi commented on PIG-2975: --- Forgot to mention one more alternative. Take out the

[jira] [Commented] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480301#comment-13480301 ] Koji Noguchi commented on PIG-2975: --- bq. As far as the switch approach, I don't think it

[jira] [Commented] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480340#comment-13480340 ] Koji Noguchi commented on PIG-2975: --- bq. by including the 4 bytes in the comparison I am

[jira] [Updated] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-2975: -- Attachment: pig-2975-trunk_v04-purerawcompare.txt ||patch ||correctness||byte

[jira] [Commented] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480413#comment-13480413 ] Koji Noguchi commented on PIG-2975: --- {quote} pig-2975-trunk_v04-purerawcompare.txt

[jira] [Commented] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13480442#comment-13480442 ] Koji Noguchi commented on PIG-2975: --- bq. One thing I worry about this new sort order is,

[jira] [Updated] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-22 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-2975: -- Attachment: pig-2975-trunk_v05-BinInterSedesRawComparatorAndlightweight-withouttest.txt bq. or we can

[jira] [Updated] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-22 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-2975: -- Attachment: pig-2975-trunk_v05-BinInterSedesRawComparatorAndlightweight-withtest.txt Attaching patch

[jira] [Updated] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-22 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-2975: -- Attachment: pig-2975-trunk_v05-BinInterSedesRawComparatorAndlightweight-withtest2.txt bq. statically

[jira] [Commented] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-22 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481761#comment-13481761 ] Koji Noguchi commented on PIG-2975: --- bq. Would it have been the same before? Yes. My test

[jira] [Created] (PIG-2999) Regression after PIG-2975: BinInterSedesTupleRawComparator secondary sort failing

2012-10-22 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-2999: - Summary: Regression after PIG-2975: BinInterSedesTupleRawComparator secondary sort failing Key: PIG-2999 URL: https://issues.apache.org/jira/browse/PIG-2999 Project: Pig

[jira] [Commented] (PIG-2975) TestTypedMap.testOrderBy failing with incorrect result

2012-10-22 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13482077#comment-13482077 ] Koji Noguchi commented on PIG-2975: --- bq. Now I am running all the tests with your fix and

[jira] [Updated] (PIG-2999) Regression after PIG-2975: BinInterSedesTupleRawComparator secondary sort failing

2012-10-22 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-2999: -- Attachment: pig-2999-v1.txt During the performance test in pig-2975, I made a change inside

[jira] [Updated] (PIG-2999) Regression after PIG-2975: BinInterSedesTupleRawComparator secondary sort failing

2012-10-22 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-2999: -- Assignee: Jonathan Coveney Regression after PIG-2975: BinInterSedesTupleRawComparator secondary

[jira] [Commented] (PIG-2999) Regression after PIG-2975: BinInterSedesTupleRawComparator secondary sort failing

2012-10-22 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13482088#comment-13482088 ] Koji Noguchi commented on PIG-2999: --- Thanks Jonathan, Gianmarco. bq. Can you post the

[jira] [Commented] (PIG-2999) Regression after PIG-2975: BinInterSedesTupleRawComparator secondary sort failing

2012-10-22 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13482095#comment-13482095 ] Koji Noguchi commented on PIG-2999: --- bq. and more.

[jira] [Commented] (PIG-2999) Regression after PIG-2975: BinInterSedesTupleRawComparator secondary sort failing

2012-10-23 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13482381#comment-13482381 ] Koji Noguchi commented on PIG-2999: --- After patch 'pig-2999-v1.txt' is applied, previous

[jira] [Updated] (PIG-2999) Regression after PIG-2975: BinInterSedesTupleRawComparator secondary sort failing

2012-10-23 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-2999: -- Attachment: pig-2999-v2.txt In WritableComparator, it converts to 'int' and then compare. {noformat}

[jira] [Commented] (PIG-2999) Regression after PIG-2975: BinInterSedesTupleRawComparator secondary sort failing

2012-10-23 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13482934#comment-13482934 ] Koji Noguchi commented on PIG-2999: --- bq. less than tuple with greater value expected:-1

[jira] [Commented] (PIG-2999) Regression after PIG-2975: BinInterSedesTupleRawComparator secondary sort failing

2012-10-24 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13483832#comment-13483832 ] Koji Noguchi commented on PIG-2999: --- Thanks for fixing the mess I've made! bq. I am

[jira] [Updated] (PIG-2968) ColumnMapKeyPrune fails to prune a subtree inside foreach

2012-10-29 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-2968: -- Fix Version/s: 0.11 I'd like to see this fixed in 0.11. ColumnMapKeyPrune fails to

[jira] [Updated] (PIG-2968) ColumnMapKeyPrune fails to prune a subtree inside foreach

2012-10-29 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-2968: -- Attachment: pig-2968-trunk_v03.txt bq. Actually, can you please fix the indentation of

[jira] [Commented] (PIG-2474) DuplicateForEachColumnRewrite fails in some pig scripts

2012-10-31 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487980#comment-13487980 ] Koji Noguchi commented on PIG-2474: --- bq. Duplicate of PIG-2119. Was this fixed in

[jira] [Commented] (PIG-2970) Nested foreach getting incorrect schema when having unrelated inner query

2012-11-02 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489578#comment-13489578 ] Koji Noguchi commented on PIG-2970: --- bq. Can you give more info on why you would like to

[jira] [Commented] (PIG-2970) Nested foreach getting incorrect schema when having unrelated inner query

2012-11-02 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489602#comment-13489602 ] Koji Noguchi commented on PIG-2970: --- bq. for optimization purposes (and not for

[jira] [Created] (PIG-3051) java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning

2012-11-15 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3051: - Summary: java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning Key: PIG-3051 URL: https://issues.apache.org/jira/browse/PIG-3051 Project: Pig

[jira] [Commented] (PIG-3051) java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning

2012-11-15 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13498150#comment-13498150 ] Koji Noguchi commented on PIG-3051: --- Stack dump from task failure using trunk {noformat}

[jira] [Commented] (PIG-3051) java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning

2012-11-15 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13498165#comment-13498165 ] Koji Noguchi commented on PIG-3051: --- Tracking the LogicalPlan change. Original.

[jira] [Commented] (PIG-3051) java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning

2012-11-15 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13498166#comment-13498166 ] Koji Noguchi commented on PIG-3051: --- Sorry, After ColumnMapKeyPrune, I pasted the wrong

[jira] [Updated] (PIG-3051) java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning

2012-11-15 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3051: -- Attachment: pig-3051-v1-withouttest.txt There must be a better way/method than this but I hope the patch

[jira] [Commented] (PIG-3051) java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning

2012-11-15 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13498190#comment-13498190 ] Koji Noguchi commented on PIG-3051: --- Sorry, I missed Bill's comment. bq. We've seen

[jira] [Commented] (PIG-3051) java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning

2012-11-16 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499345#comment-13499345 ] Koji Noguchi commented on PIG-3051: --- bq. I couldn't reproduce this with short pig code

[jira] [Created] (PIG-3056) CLONE - PigStorageSchema need to deal with missing field

2012-11-19 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3056: - Summary: CLONE - PigStorageSchema need to deal with missing field Key: PIG-3056 URL: https://issues.apache.org/jira/browse/PIG-3056 Project: Pig Issue Type: Bug

[jira] [Updated] (PIG-3056) CLONE - PigStorageSchema need to deal with missing field

2012-11-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3056: -- Fix Version/s: (was: 0.10.0) CLONE - PigStorageSchema need to deal with missing field

[jira] [Commented] (PIG-2127) PigStorageSchema need to deal with missing field

2012-11-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500468#comment-13500468 ] Koji Noguchi commented on PIG-2127: --- bq. Vivek, bq. Please open a new jira linked to this

[jira] [Updated] (PIG-3051) java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning

2012-11-26 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3051: -- Attachment: pig-3051-v1.1-withe2etest.txt Added an e2e test since I needed columnprune + limitoptimizer.

[jira] [Updated] (PIG-3051) java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning

2012-11-26 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3051: -- Fix Version/s: 0.11 Status: Patch Available (was: Open) Requesting for 0.11 if possible.

[jira] [Created] (PIG-3072) Pig job reporting negative progress

2012-12-03 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3072: - Summary: Pig job reporting negative progress Key: PIG-3072 URL: https://issues.apache.org/jira/browse/PIG-3072 Project: Pig Issue Type: Bug Components:

[jira] [Updated] (PIG-3072) Pig job reporting negative progress

2012-12-03 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3072: -- Attachment: pig-3072-v01.txt 'pos' inside TFileRecordReader was never updated. Using fileIn.getPos()

[jira] [Updated] (PIG-3072) Pig job reporting negative progress

2012-12-03 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3072: -- Attachment: pig-3072-v02.txt bq. Can we create the basic.tfile using Util.createTempFileDelOnExit().

[jira] [Updated] (PIG-3072) Pig job reporting negative progress

2012-12-03 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3072: -- Status: Patch Available (was: Open) Pig job reporting negative progress

[jira] [Updated] (PIG-3072) Pig job reporting negative progress

2012-12-03 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3072: -- Attachment: pig-3072-v03.txt (Minor changes to previous v02 patch.) * Added comment in

[jira] [Updated] (PIG-3072) Pig job reporting negative progress

2012-12-04 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3072: -- Attachment: pig-3072-v04.txt bq. Can you use HadoopShims to create the TaskAttemptContext in your test.

[jira] [Resolved] (PIG-2802) Wrong Schema generated when there is a dangling alias

2012-12-12 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-2802. --- Resolution: Duplicate Wrong Schema generated when there is a dangling alias

[jira] [Updated] (PIG-3051) java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning

2012-12-20 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3051: -- Attachment: pig-3051-v2.1-withe2etest.txt Thanks Rohini for the review. bq .But found an issue with

[jira] [Commented] (PIG-3100) If a .pig_schema file is present, can get an index out of bounds error

2012-12-20 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13537287#comment-13537287 ] Koji Noguchi commented on PIG-3100: --- I should have commented on PIG-3056, but when our

[jira] [Commented] (PIG-3100) If a .pig_schema file is present, can get an index out of bounds error

2012-12-20 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13537302#comment-13537302 ] Koji Noguchi commented on PIG-3100: --- bq. Perhaps there can be a flag or setting for

[jira] [Created] (PIG-3102) Option for PigStorage load to error out when input record is incomplete (instead of filling in null)

2012-12-20 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3102: - Summary: Option for PigStorage load to error out when input record is incomplete (instead of filling in null) Key: PIG-3102 URL: https://issues.apache.org/jira/browse/PIG-3102

[jira] [Created] (PIG-3147) Spill failing with java.lang.RuntimeException: InternalCachedBag.spill() should not be called

2013-01-29 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3147: - Summary: Spill failing with java.lang.RuntimeException: InternalCachedBag.spill() should not be called Key: PIG-3147 URL: https://issues.apache.org/jira/browse/PIG-3147

[jira] [Commented] (PIG-3147) Spill failing with java.lang.RuntimeException: InternalCachedBag.spill() should not be called

2013-01-29 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13565698#comment-13565698 ] Koji Noguchi commented on PIG-3147: --- Dumping a stacktrace when InternalCacheBag is added

[jira] [Commented] (PIG-2923) Lazily register bags with SpillableMemoryManager

2013-01-29 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13565705#comment-13565705 ] Koji Noguchi commented on PIG-2923: --- Hi Dmitriy, I'm seeing a weird error when pig 0.11

[jira] [Updated] (PIG-3147) Spill failing with java.lang.RuntimeException: InternalCachedBag.spill() should not be called

2013-01-29 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3147: -- Attachment: pig-3147-v01.txt Reading PIG-975, InternalCachedBag should not register with

[jira] [Created] (PIG-3148) OutOfMemory exception while spilling stale DefaultDataBag. Extra option to gc() before spilling large bag.

2013-01-29 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3148: - Summary: OutOfMemory exception while spilling stale DefaultDataBag. Extra option to gc() before spilling large bag. Key: PIG-3148 URL: https://issues.apache.org/jira/browse/PIG-3148

[jira] [Commented] (PIG-3148) OutOfMemory exception while spilling stale DefaultDataBag. Extra option to gc() before spilling large bag.

2013-01-29 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13565975#comment-13565975 ] Koji Noguchi commented on PIG-3148: --- I cannot attach the original query, but to give you

[jira] [Updated] (PIG-3148) OutOfMemory exception while spilling stale DefaultDataBag. Extra option to gc() before spilling large bag.

2013-01-29 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3148: -- Attachment: pig-3148-v01.patch Uploading a patch that adds a feature that would call System.gc() when

[jira] [Created] (PIG-3178) Print a stacktrace when ExecutableManager hits an OOM

2013-02-11 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3178: - Summary: Print a stacktrace when ExecutableManager hits an OOM Key: PIG-3178 URL: https://issues.apache.org/jira/browse/PIG-3178 Project: Pig Issue Type:

[jira] [Updated] (PIG-3178) Print a stacktrace when ExecutableManager hits an OOM

2013-02-11 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3178: -- Attachment: pig-3178-trunk-v01.patch Adding printStackTrace call. Since it's only a logging change, no

[jira] [Updated] (PIG-3178) Print a stacktrace when ExecutableManager hits an OOM

2013-02-11 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3178: -- Status: Patch Available (was: Open) Print a stacktrace when ExecutableManager hits an OOM

[jira] [Commented] (PIG-3178) Print a stacktrace when ExecutableManager hits an OOM

2013-02-11 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13575869#comment-13575869 ] Koji Noguchi commented on PIG-3178: --- bq. Doesn't LOG.error(t); log the stacktrace? I

[jira] [Updated] (PIG-3178) Print a stacktrace when ExecutableManager hits an OOM

2013-02-11 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3178: -- Status: Open (was: Patch Available) Print a stacktrace when ExecutableManager hits an OOM

[jira] [Updated] (PIG-3178) Print a stacktrace when ExecutableManager hits an OOM

2013-02-11 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3178: -- Attachment: pig-3178-trunk-v02.patch bq. Can you just add a message - LOG.error(Error running blah blah,

[jira] [Updated] (PIG-3178) Print a stacktrace when ExecutableManager hits an OOM

2013-02-11 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3178: -- Status: Patch Available (was: Open) Print a stacktrace when ExecutableManager hits an OOM

[jira] [Updated] (PIG-3178) Print a stacktrace when ExecutableManager hits an OOM

2013-02-11 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3178: -- Attachment: pig-3178-trunk-v03.patch Sorry for the spam. Rohini pointed out that last patch was based

[jira] [Created] (PIG-3179) Task Information Header only prints out the first split for each task

2013-02-11 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3179: - Summary: Task Information Header only prints out the first split for each task Key: PIG-3179 URL: https://issues.apache.org/jira/browse/PIG-3179 Project: Pig

[jira] [Updated] (PIG-3179) Task Information Header only prints out the first split for each task

2013-02-11 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3179: -- Attachment: pig-3179-v01.patch Added for-loop to print all the splits. Task

[jira] [Updated] (PIG-3179) Task Information Header only prints out the first split for each task

2013-02-11 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3179: -- Attachment: pig-3179-v02.patch Changed based on Rohini's suggestion. Added extra line printing out the

[jira] [Updated] (PIG-3179) Task Information Header only prints out the first split for each task

2013-02-11 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3179: -- Attachment: pig-3179-v03.patch bq. Hi Koji Noguchi, minor thing - calling toString on Path might be

[jira] [Updated] (PIG-3179) Task Information Header only prints out the first split for each task

2013-02-13 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3179: -- Attachment: pig-3179-v04.patch bq. Can you reuse the StringBuilder. i.e move StringBuilder sb = new

[jira] [Commented] (PIG-3148) OutOfMemory exception while spilling stale DefaultDataBag. Extra option to gc() before spilling large bag.

2013-02-13 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13577743#comment-13577743 ] Koji Noguchi commented on PIG-3148: --- Rohini asked me to clarify why I'm adding extra param

[jira] [Commented] (PIG-3148) OutOfMemory exception while spilling stale DefaultDataBag. Extra option to gc() before spilling large bag.

2013-02-14 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13578507#comment-13578507 ] Koji Noguchi commented on PIG-3148: --- Thanks Dmitriy, Rohini! I like the fixed ratio

[jira] [Updated] (PIG-3148) OutOfMemory exception while spilling stale DefaultDataBag. Extra option to gc() before spilling large bag.

2013-03-01 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3148: -- Attachment: pig-3148-v02.patch Sorry for the delay. Attaching a patch with suggested change.

[jira] [Commented] (PIG-2597) Move grunt from javacc to ANTRL

2013-03-15 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13603583#comment-13603583 ] Koji Noguchi commented on PIG-2597: --- bq. Jonathan, any update on this? I'm interested in

[jira] [Created] (PIG-3251) Bzip2TextInputFormat requires double the memory of maximum record size

2013-03-18 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3251: - Summary: Bzip2TextInputFormat requires double the memory of maximum record size Key: PIG-3251 URL: https://issues.apache.org/jira/browse/PIG-3251 Project: Pig

[jira] [Updated] (PIG-3251) Bzip2TextInputFormat requires double the memory of maximum record size

2013-03-18 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3251: -- Attachment: pig-3251-trunk-v01.patch In Bzip2TextInputFormat, it says {code} /** *

[jira] [Commented] (PIG-3251) Bzip2TextInputFormat requires double the memory of maximum record size

2013-03-19 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13606445#comment-13606445 ] Koji Noguchi commented on PIG-3251: --- bq. Let me know if you find any problem in your

[jira] [Updated] (PIG-3251) Bzip2TextInputFormat requires double the memory of maximum record size

2013-03-20 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3251: -- Attachment: pig-3251-trunk-v02.patch (1) Current status (before any patch) ||hadoop version ||

[jira] [Commented] (PIG-3251) Bzip2TextInputFormat requires double the memory of maximum record size

2013-03-20 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13607860#comment-13607860 ] Koji Noguchi commented on PIG-3251: --- bq. With HADOOP-7823, can we remove

[jira] [Commented] (PIG-3251) Bzip2TextInputFormat requires double the memory of maximum record size

2013-03-20 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13607886#comment-13607886 ] Koji Noguchi commented on PIG-3251: --- bq. With HADOOP-7823, can we remove

[jira] [Updated] (PIG-3251) Bzip2TextInputFormat requires double the memory of maximum record size

2013-03-20 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3251: -- Attachment: pig-3251-trunk-v03.patch bq. Makes sense, we shall move to the new approach for Hadoop

[jira] [Commented] (PIG-3255) Avoid extra byte array copy in streaming deserialize

2013-03-20 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13608227#comment-13608227 ] Koji Noguchi commented on PIG-3255: --- +1 Looks good to me. Probably another Jira, but I

[jira] [Created] (PIG-3266) Pig takes forever to parse scripts with foreach + multi level binconds

2013-04-02 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3266: - Summary: Pig takes forever to parse scripts with foreach + multi level binconds Key: PIG-3266 URL: https://issues.apache.org/jira/browse/PIG-3266 Project: Pig

[jira] [Commented] (PIG-3266) Pig takes forever to parse scripts with foreach + multi level binconds

2013-04-02 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620098#comment-13620098 ] Koji Noguchi commented on PIG-3266: --- If I revert the change from PIG:1387, parsing speed

[jira] [Commented] (PIG-3266) Pig takes forever to parse scripts with foreach + multi level binconds

2013-04-02 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13620158#comment-13620158 ] Koji Noguchi commented on PIG-3266: --- bq. Does it finish in the end, or never? I would

[jira] [Commented] (PIG-3266) Pig takes forever to parse scripts with foreach + multi level binconds

2013-04-03 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13621115#comment-13621115 ] Koji Noguchi commented on PIG-3266: --- Does it finish in the end, or never? I would guess

[jira] [Commented] (PIG-3261) User set PIG_CLASSPATH entries must be prepended to the CLASSPATH, not appended

2013-04-03 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13621155#comment-13621155 ] Koji Noguchi commented on PIG-3261: --- I prefer with PIG_USER_CLASSPATH_FIRST. I've seen

[jira] [Commented] (PIG-3266) Pig takes forever to parse scripts with foreach + multi level binconds

2013-04-03 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13621178#comment-13621178 ] Koji Noguchi commented on PIG-3266: --- bq. I assume there is an infinite loop. Next time

[jira] [Commented] (PIG-3266) Pig takes forever to parse scripts with foreach + multi level binconds

2013-04-03 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13621230#comment-13621230 ] Koji Noguchi commented on PIG-3266: --- bq. Koji Noguchi, I think this was fixed. I don't see

[jira] [Resolved] (PIG-3266) Pig takes forever to parse scripts with foreach + multi level binconds

2013-04-03 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi resolved PIG-3266. --- Resolution: Duplicate Release Note: Found it. This is a duplicate of PIG-2769. Sorry Xuefu for

[jira] [Updated] (PIG-3266) Pig takes forever to parse scripts with foreach + multi level binconds

2013-04-03 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3266: -- Release Note: (was: Found it. This is a duplicate of PIG-2769. Sorry Xuefu for wasting your time on

[jira] [Commented] (PIG-2769) a simple logic causes very long compiling time on pig 0.10.0

2013-04-04 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13622731#comment-13622731 ] Koji Noguchi commented on PIG-2769: --- bq. We should put this into 0.11 branch, maybe there

[jira] [Created] (PIG-3270) Union onschema failing at runtime when merging incompatible types

2013-04-09 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3270: - Summary: Union onschema failing at runtime when merging incompatible types Key: PIG-3270 URL: https://issues.apache.org/jira/browse/PIG-3270 Project: Pig Issue

[jira] [Created] (PIG-3271) POSplit ignoring error from input processing giving empty results

2013-04-09 Thread Koji Noguchi (JIRA)
Koji Noguchi created PIG-3271: - Summary: POSplit ignoring error from input processing giving empty results Key: PIG-3271 URL: https://issues.apache.org/jira/browse/PIG-3271 Project: Pig Issue

[jira] [Updated] (PIG-3271) POSplit ignoring error from input processing giving empty results

2013-04-09 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3271: -- Attachment: pig-3271-v01.patch I'm having hard time tracking the code but this seems to catch the error.

[jira] [Updated] (PIG-3271) POSplit ignoring error from input processing giving empty results

2013-04-10 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3271: -- Attachment: pig-3271-v02.patch bq. Ready to go with a testcase. I'm lost in this. Original example

  1   2   3   4   5   6   7   8   9   10   >