[jira] [Updated] (PIG-3144) Erroneous map entry alias resolution leading to Duplicate schema alias errors
[
https://issues.apache.org/jira/browse/PIG-3144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kai Londenberg updated PIG-3144:
Description:
The following code illustrates a problem concerning alias resolution in pig
The schema of D2 will incorrectly be described as containing two age fields.
And the last step in the following script will lead to a Duplicate schema
alias error message.
I only encountered this bug when using aliases for map fields.
{code}
DATA = LOAD 'file:///whatever' as (a:map[chararray], b:chararray);
D1 = FOREACH DATA GENERATE a#'name' as name, a#'age' as age, b;
D2 = FOREACH D1 GENERATE name, age, b;
DESCRIBE D2;
{code}
Output:
{code}
D2: {
age: chararray,
age: chararray,
b: chararray
}
{code}
{code}
D3 = FOREACH D2 GENERATE *;
DESCRIBE D3;
{code}
Output:
{code}
file file:///.../pig-bug-example.pig, line 20, column 16 Duplicate schema
alias: age
{code}
This error occurs in this form in Apache Pig version 0.11.0-SNAPSHOT (r6408). A
less severe variant of this bug is also present in pig 0.10.1. In 0.10.1, the
Duplicate schema alias error message won't occur, but the schema of D2 (see
above) will still have wrong duplicate alias entries.
was:
The following code illustrates a problem concerning alias resolution in pig
0.10.x. The schema of D2 will incorrectly be described as containing two age
fields. And the last step in the following script will lead to a Duplicate
schema alias error message.
I only encountered this bug when using aliases for map fields.
{code}
DATA = LOAD 'file:///whatever' as (a:map[chararray], b:chararray);
D1 = FOREACH DATA GENERATE a#'name' as name, a#'age' as age, b;
D2 = FOREACH D1 GENERATE name, age, b;
DESCRIBE D2;
{code}
Output:
{code}
D2: {
age: chararray,
age: chararray,
b: chararray
}
{code}
{code}
D3 = FOREACH D2 GENERATE *;
DESCRIBE D3;
{code}
Output:
{code}
file file:///.../pig-bug-example.pig, line 20, column 16 Duplicate schema
alias: age
{code}
Affects Version/s: (was: 0.10.0)
0.10.1
0.11
Erroneous map entry alias resolution leading to Duplicate schema alias
errors
---
Key: PIG-3144
URL: https://issues.apache.org/jira/browse/PIG-3144
Project: Pig
Issue Type: Bug
Affects Versions: 0.11, 0.10.1
Reporter: Kai Londenberg
The following code illustrates a problem concerning alias resolution in pig
The schema of D2 will incorrectly be described as containing two age
fields. And the last step in the following script will lead to a Duplicate
schema alias error message.
I only encountered this bug when using aliases for map fields.
{code}
DATA = LOAD 'file:///whatever' as (a:map[chararray], b:chararray);
D1 = FOREACH DATA GENERATE a#'name' as name, a#'age' as age, b;
D2 = FOREACH D1 GENERATE name, age, b;
DESCRIBE D2;
{code}
Output:
{code}
D2: {
age: chararray,
age: chararray,
b: chararray
}
{code}
{code}
D3 = FOREACH D2 GENERATE *;
DESCRIBE D3;
{code}
Output:
{code}
file file:///.../pig-bug-example.pig, line 20, column 16 Duplicate schema
alias: age
{code}
This error occurs in this form in Apache Pig version 0.11.0-SNAPSHOT (r6408).
A less severe variant of this bug is also present in pig 0.10.1. In 0.10.1,
the Duplicate schema alias error message won't occur, but the schema of D2
(see above) will still have wrong duplicate alias entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2994) Grunt shortcuts
[ https://issues.apache.org/jira/browse/PIG-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566554#comment-13566554 ] Cheolsoo Park commented on PIG-2994: Hi Prasanth, Sorry for the delay. I will review it today. In the meantime, do you mind updating docs? I think we can add a table of shortcuts to this section: http://pig.apache.org/docs/r0.10.0/start.html#interactive-mode. What do you think? Grunt shortcuts --- Key: PIG-2994 URL: https://issues.apache.org/jira/browse/PIG-2994 Project: Pig Issue Type: New Feature Components: grunt Reporter: Prasanth J Assignee: Prasanth J Priority: Minor Attachments: PIG-2994.1.git.patch This feature is aimed at providing shortcuts for frequently used commands like illustrate, dump, explain, describe, quit, help etc. This feature is inspired from postgres(psql) shortcuts. I tried implementing a simple shortcut for quitting the grunt shell using \q with very minimal changes. I think this feature will help save many keystrokes for users. If this feature looks useful I can submit the current patch for review and go ahead with implementing the following shortcuts \i alias - illustrate \e alias - explain \de alias - describe \du alias - dump \h - help This will also be useful to view information about tables/statistics stored in HCatalog similar to the way psql does. \dt alias - display table \dm - display metadata etc.. except \t, \r and \n delimiters we should be able to use all other characters as shortcuts. Please let me know your thoughts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (PIG-3149) e2e build.xml still refers to jython 2.5.0 jar even though it's replaced by jython standalone 2.5.2 jar
Cheolsoo Park created PIG-3149: -- Summary: e2e build.xml still refers to jython 2.5.0 jar even though it's replaced by jython standalone 2.5.2 jar Key: PIG-3149 URL: https://issues.apache.org/jira/browse/PIG-3149 Project: Pig Issue Type: Bug Components: e2e harness Affects Versions: 0.11 Reporter: Cheolsoo Park Assignee: Cheolsoo Park Fix For: 0.12 PIG-2665 replaced jython-2.5.0.jar with jython-standalone-2.5.2.jar. But e2e build.xml still refers to the former. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3147) Spill failing with java.lang.RuntimeException: InternalCachedBag.spill() should not be called
[
https://issues.apache.org/jira/browse/PIG-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566640#comment-13566640
]
Dmitriy V. Ryaboy commented on PIG-3147:
Yeah that's my fault, fix looks right. +1, will commit.
Spill failing with java.lang.RuntimeException: InternalCachedBag.spill()
should not be called
---
Key: PIG-3147
URL: https://issues.apache.org/jira/browse/PIG-3147
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.11
Reporter: Koji Noguchi
Priority: Blocker
Attachments: pig-3147-v01.txt
Tried 0.11 jar with spilling, my job failed to spill with the following stack
trace. Anyone else seeing this?
{noformat}
java.lang.RuntimeException: InternalCachedBag.spill() should not be called
at
org.apache.pig.data.InternalCachedBag.spill(InternalCachedBag.java:167)
at
org.apache.pig.impl.util.SpillableMemoryManager.handleNotification(SpillableMemoryManager.java:243)
at
sun.management.NotificationEmitterSupport.sendNotification(NotificationEmitterSupport.java:138)
at sun.management.MemoryImpl.createNotification(MemoryImpl.java:171)
at
sun.management.MemoryPoolImpl$PoolSensor.triggerAction(MemoryPoolImpl.java:272)
at sun.management.Sensor.trigger(Sensor.java:120)
Exception in thread Low Memory Detector java.lang.InternalError: Error in
invoking listener
at
sun.management.NotificationEmitterSupport.sendNotification(NotificationEmitterSupport.java:141)
at sun.management.MemoryImpl.createNotification(MemoryImpl.java:171)
at
sun.management.MemoryPoolImpl$PoolSensor.triggerAction(MemoryPoolImpl.java:272)
at sun.management.Sensor.trigger(Sensor.java:120)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3149) e2e build.xml still refers to jython 2.5.0 jar even though it's replaced by jython standalone 2.5.2 jar
[ https://issues.apache.org/jira/browse/PIG-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3149: --- Attachment: PIG-3149.patch Attached is a patch that parameterized jython.version and jruby.version and load them from ivy/libraries.properties. e2e build.xml still refers to jython 2.5.0 jar even though it's replaced by jython standalone 2.5.2 jar --- Key: PIG-3149 URL: https://issues.apache.org/jira/browse/PIG-3149 Project: Pig Issue Type: Bug Components: e2e harness Affects Versions: 0.11 Reporter: Cheolsoo Park Assignee: Cheolsoo Park Fix For: 0.12 Attachments: PIG-3149.patch PIG-2665 replaced jython-2.5.0.jar with jython-standalone-2.5.2.jar. But e2e build.xml still refers to the former. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3149) e2e build.xml still refers to jython 2.5.0 jar even though it's replaced by jython standalone 2.5.2 jar
[ https://issues.apache.org/jira/browse/PIG-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3149: --- Status: Patch Available (was: Open) e2e build.xml still refers to jython 2.5.0 jar even though it's replaced by jython standalone 2.5.2 jar --- Key: PIG-3149 URL: https://issues.apache.org/jira/browse/PIG-3149 Project: Pig Issue Type: Bug Components: e2e harness Affects Versions: 0.11 Reporter: Cheolsoo Park Assignee: Cheolsoo Park Fix For: 0.12 Attachments: PIG-3149.patch PIG-2665 replaced jython-2.5.0.jar with jython-standalone-2.5.2.jar. But e2e build.xml still refers to the former. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3142) Fixed-width load and store functions for the Piggybank
[ https://issues.apache.org/jira/browse/PIG-3142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566673#comment-13566673 ] Alan Gates commented on PIG-3142: - Is this patch ready for review? If so, you want to click the submit patch button so committers know to review it. Fixed-width load and store functions for the Piggybank -- Key: PIG-3142 URL: https://issues.apache.org/jira/browse/PIG-3142 Project: Pig Issue Type: New Feature Components: piggybank Affects Versions: 0.11 Reporter: Jonathan Packer Attachments: fixed-width.patch Adds load/store functions for fixed width data to the Piggybank. They use the syntax of the unix cut command to specify column positions, and have an option to skip the header row when loading or to write a header row when storing. The header handling works properly with multiple small files each with a header being combined into one split, or a large file with a single header being split into multiple splits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-3147) Spill failing with java.lang.RuntimeException: InternalCachedBag.spill() should not be called
[
https://issues.apache.org/jira/browse/PIG-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitriy V. Ryaboy resolved PIG-3147.
Resolution: Fixed
Fix Version/s: 0.12
0.11
Assignee: Koji Noguchi
committed to 0.11 and trunk.
way to catch that before we roll the release! :)
Spill failing with java.lang.RuntimeException: InternalCachedBag.spill()
should not be called
---
Key: PIG-3147
URL: https://issues.apache.org/jira/browse/PIG-3147
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.11
Reporter: Koji Noguchi
Assignee: Koji Noguchi
Priority: Blocker
Fix For: 0.11, 0.12
Attachments: pig-3147-v01.txt
Tried 0.11 jar with spilling, my job failed to spill with the following stack
trace. Anyone else seeing this?
{noformat}
java.lang.RuntimeException: InternalCachedBag.spill() should not be called
at
org.apache.pig.data.InternalCachedBag.spill(InternalCachedBag.java:167)
at
org.apache.pig.impl.util.SpillableMemoryManager.handleNotification(SpillableMemoryManager.java:243)
at
sun.management.NotificationEmitterSupport.sendNotification(NotificationEmitterSupport.java:138)
at sun.management.MemoryImpl.createNotification(MemoryImpl.java:171)
at
sun.management.MemoryPoolImpl$PoolSensor.triggerAction(MemoryPoolImpl.java:272)
at sun.management.Sensor.trigger(Sensor.java:120)
Exception in thread Low Memory Detector java.lang.InternalError: Error in
invoking listener
at
sun.management.NotificationEmitterSupport.sendNotification(NotificationEmitterSupport.java:141)
at sun.management.MemoryImpl.createNotification(MemoryImpl.java:171)
at
sun.management.MemoryPoolImpl$PoolSensor.triggerAction(MemoryPoolImpl.java:272)
at sun.management.Sensor.trigger(Sensor.java:120)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2994) Grunt shortcuts
[ https://issues.apache.org/jira/browse/PIG-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566681#comment-13566681 ] Cheolsoo Park commented on PIG-2994: I like your suggestion better. Thanks! Grunt shortcuts --- Key: PIG-2994 URL: https://issues.apache.org/jira/browse/PIG-2994 Project: Pig Issue Type: New Feature Components: grunt Reporter: Prasanth J Assignee: Prasanth J Priority: Minor Attachments: PIG-2994.1.git.patch This feature is aimed at providing shortcuts for frequently used commands like illustrate, dump, explain, describe, quit, help etc. This feature is inspired from postgres(psql) shortcuts. I tried implementing a simple shortcut for quitting the grunt shell using \q with very minimal changes. I think this feature will help save many keystrokes for users. If this feature looks useful I can submit the current patch for review and go ahead with implementing the following shortcuts \i alias - illustrate \e alias - explain \de alias - describe \du alias - dump \h - help This will also be useful to view information about tables/statistics stored in HCatalog similar to the way psql does. \dt alias - display table \dm - display metadata etc.. except \t, \r and \n delimiters we should be able to use all other characters as shortcuts. Please let me know your thoughts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3145) Parameters in core-site.xml and mapred-site.xml are not correctly substituted
[
https://issues.apache.org/jira/browse/PIG-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cheolsoo Park updated PIG-3145:
---
Attachment: PIG-3145.patch
Attached is a patch that includes proposed changes.
Note that I also removed 4 lines of {{properties.setProperty()}} for local
mode. This is because they're redundant as those properties are loaded from
*-default.xml files.
I ran commit-test with both hadoop 20 and 23 and saw no failures. To be safe, I
will run a full suite of unit test as well.
Parameters in core-site.xml and mapred-site.xml are not correctly substituted
-
Key: PIG-3145
URL: https://issues.apache.org/jira/browse/PIG-3145
Project: Pig
Issue Type: Bug
Reporter: Cheolsoo Park
Assignee: Cheolsoo Park
Attachments: PIG-3145.patch
To reproduce the issue, please do the following:
# Parameterize the address of name node in core-site.xml.
{code}
property
namefs.default.name/name
valuehdfs://${foo}:8020/value
/property
{code}
# Set the value of foo via -D option.
{code}
export PIG_OPTS=-Dfoo=mr1-0.cheolsoo.com
{code}
# Pig fails with the following error.
{code}
2013-01-28 18:54:02,786 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to hadoop file system at: hdfs://${foo}:8020
2013-01-28 18:54:02,805 [main] ERROR org.apache.pig.Main - ERROR 2999:
Unexpected internal error. null
Details at logfile: /home/cheolsoo/pig-cdh/pig_1359428042522.log
{code}
Note that the parameter $\{foo\} in core-site.xml is not expanded. This is
because the addresses of name node and job tracker are read directly from
core-site.xml instead of reading via Configuration.get().
{code:title=HExecutionEngine.java}
// properties is Java Properties
cluster = properties.getProperty(JOB_TRACKER_LOCATION);
nameNode = properties.getProperty(FILE_SYSTEM_LOCATION);
{code}
Replacing these lines with Configuration.get() fixes the issue.
{code:title=HExecutionEngine.java}
// jc is Hadoop Configuration
cluster = jc.get(JOB_TRACKER_LOCATION);
nameNode = jc.get(FILE_SYSTEM_LOCATION);
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3142) Fixed-width load and store functions for the Piggybank
[ https://issues.apache.org/jira/browse/PIG-3142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Packer updated PIG-3142: - Status: Patch Available (was: Open) Submitted patch for review, thanks. Fixed-width load and store functions for the Piggybank -- Key: PIG-3142 URL: https://issues.apache.org/jira/browse/PIG-3142 Project: Pig Issue Type: New Feature Components: piggybank Affects Versions: 0.11 Reporter: Jonathan Packer Attachments: fixed-width.patch Adds load/store functions for fixed width data to the Piggybank. They use the syntax of the unix cut command to specify column positions, and have an option to skip the header row when loading or to write a header row when storing. The header handling works properly with multiple small files each with a header being combined into one split, or a large file with a single header being split into multiple splits. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2994) Grunt shortcuts
[
https://issues.apache.org/jira/browse/PIG-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566790#comment-13566790
]
Cheolsoo Park commented on PIG-2994:
[~prasanth_j] Thank you very much for the patch! This is very convenient. I
have two suggestions.
* Can we simplify the grammar?
From:
{code}
DUMP
(
t1 = IDENTIFIER
{processDump(t1.image);}
|
{processDump(null);}
)
|
DUMP_SHORT
(
t1 = IDENTIFIER
{processDump(t1.image);}
|
{processDump(null);}
)
{code}
To:
{code}
( DUMP | DUMP_SHORT )
(
t1 = IDENTIFIER
{processDump(t1.image);}
|
{processDump(null);}
)
{code}
The same goes to all the shortcuts that you're adding. I tested it myself, and
it seems to work. Please correct me if I am wrong because I am no expert on
JAVACC.
* Can we throw an exception if dump/explain/illustrate is called with no alias
there is no previous alias in the Pig context?
For example, if I have a dummy script such as follows:
{code}
DUMP;
{code}
Since dump, explain, and illustrate are now allowed with no alias, my script
fails with the following error at logical plan validation:
{code}
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open
iterator for alias null
{code}
However, it used to fail faster at parsing:
{code}
ERROR 1000: Error during parsing. Encountered IDENTIFIER d at line 2,
column 1.
{code}
I think the original error is more intuitive. We can probably add a null check
after getLastAlias(). For example,
{code}
alias = mPigServer.getPigContext().getLastAlias();
if (alias == null) {
throw new ParseException('illustrate' statement must be on an alias or on
a script.);
}
{code}
Please let me know what you think. Thanks!
Grunt shortcuts
---
Key: PIG-2994
URL: https://issues.apache.org/jira/browse/PIG-2994
Project: Pig
Issue Type: New Feature
Components: grunt
Reporter: Prasanth J
Assignee: Prasanth J
Priority: Minor
Attachments: PIG-2994.1.git.patch
This feature is aimed at providing shortcuts for frequently used commands
like illustrate, dump, explain, describe, quit, help etc. This feature is
inspired from postgres(psql) shortcuts. I tried implementing a simple
shortcut for quitting the grunt shell using \q with very minimal changes. I
think this feature will help save many keystrokes for users. If this feature
looks useful I can submit the current patch for review and go ahead with
implementing the following shortcuts
\i alias - illustrate
\e alias - explain
\de alias - describe
\du alias - dump
\h - help
This will also be useful to view information about tables/statistics stored
in HCatalog similar to the way psql does.
\dt alias - display table
\dm - display metadata
etc..
except \t, \r and \n delimiters we should be able to use all other characters
as shortcuts.
Please let me know your thoughts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Pig 11.0
Olga if you've got your hands full I'll volunteer to wear the release manager hat this round. Alan/Olga is this doc current? *https://cwiki.apache.org/PIG/howtorelease.html* On Tue, Jan 29, 2013 at 7:26 PM, Alan Gates ga...@hortonworks.com wrote: Olga had previously volunteered to be RM for this release. Olga, did you still want to do this or are you open to someone else taking it on? BTW, I don't have time to drive it right now either, though I'm happy to coach a first time RM through it if Olga doesn't want to do it. Alan. On Jan 29, 2013, at 7:00 PM, Daniel Dai wrote: My hands are full these several days, I can try after several days but no promise. Thanks, Daniel On Tue, Jan 29, 2013 at 11:40 AM, Dmitriy Ryaboy dvrya...@gmail.com wrote: Sounds like we can roll a release. Who wants to do the honors? I think Olga and Daniel have done it in the past, not sure if they have the time? On Mon, Jan 28, 2013 at 11:37 PM, Bill Graham billgra...@gmail.com wrote: I've just committed the 2 documentation jiras and all Pig 0.11 issues are resolved once again. On Mon, Jan 28, 2013 at 7:22 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: e2e tests are fine on my end too except for hcat tests (Did not have hcat setup) and few transient Errors test failures which is a known issue. Ran with Hadoop 0.23. Regards, Rohini On Mon, Jan 28, 2013 at 4:44 PM, Julien Le Dem jul...@twitter.com wrote: It sounds like we are ready to go. There are two remaining documentation JIRAs that will be committed soon. Daniel, do you want to build the release? Julien On Mon, Jan 28, 2013 at 10:12 AM, Daniel Dai da...@hortonworks.com wrote: All tests pass for me, no single failures or aborts. Daniel On Mon, Jan 28, 2013 at 8:33 AM, Cheolsoo Park cheol...@cloudera.com wrote: Here are my results: Hadoop-1.0.x: [exec] Final results ,PASSED: 612 FAILED: 2 SKIPPED: 24 ABORTED: 1FAILED DEPENDENCY: 0 - The failures are simply because I didn't install Hcatalog. Hadoop-2.0.x: [exec] Final results ,PASSED: 567 FAILED: 3 SKIPPED: 27 ABORTED: 42 FAILED DEPENDENCY: 0 - The failures seem due to issues in my cluster rather than Pig issues. I will re-run them to verify. On Fri, Jan 25, 2013 at 5:31 PM, Cheolsoo Park cheol...@cloudera.com wrote: I will also run e2e on Hadoop-1.x and Hadoop-2.x. On Fri, Jan 25, 2013 at 5:02 PM, Daniel Dai da...@hortonworks.com wrote: I will run e2e tests on Hadoop 1.x over the weekend. Thanks, Daniel On Fri, Jan 25, 2013 at 4:27 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: Thats good :). Unit tests have all been passing. I haven't run e2e tests on pig 0.11 for sometime. Will kick off one this weekend. It would be nice if Cheolsoo and Daniel can also kick off one run. Regards, Rohini On Fri, Jan 25, 2013 at 4:08 PM, Julien Le Dem jul...@twitter.com wrote: It looks like all the tickets for Pig 0.11 have been resolved as of today. See: https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%20%220.11%22%20AND%20project%20%3D%20PIG%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20updated%20DESC%2C%20created%20ASC%2C%20priority%20DESC I propose we make the release 0.11.0 next week. Julien -- *Note that I'm no longer using my Yahoo! email address. Please email me at billgra...@gmail.com going forward.* -- *Note that I'm no longer using my Yahoo! email address. Please email me at billgra...@gmail.com going forward.*
[jira] [Commented] (PIG-3135) HExecutionEngine should look for resources in user passed Properties
[
https://issues.apache.org/jira/browse/PIG-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566853#comment-13566853
]
Cheolsoo Park commented on PIG-3135:
[~prkommireddi] Overall looks good to me. I have 3 questions.
* Can we call the property pig.use.overriden.hadoop.configs? What we're
overriding here are basically Hadoop confs, so I think that a more specific
name is better. Do you agree?
* On a related note, can you update the following comment in
testJobConfGeneration?
From:
{code}
// This should fail as pig expects classpath to be set
{code}
To something like:
{code}
// This should fail as pig expects Hadoop configs are present in classpath.
{code}
* Can you add this new property to conf/pig.properties with some explanation,
so people can know about it? It would be nice if we could mention that this is
a Mr-mode-specific property too. Please let me know if you have a better
suggestion regarding how to document this.
Thanks!
HExecutionEngine should look for resources in user passed Properties
Key: PIG-3135
URL: https://issues.apache.org/jira/browse/PIG-3135
Project: Pig
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Prashant Kommireddi
Assignee: Prashant Kommireddi
Attachments: PIG-3135.patch
Looking at this snippet:
{code}
private void init(Properties properties) throws ExecException {
.
.
.
// Check existence of hadoop-site.xml or core-site.xml
Configuration testConf = new Configuration();
ClassLoader cl = testConf.getClassLoader();
URL hadoop_site = cl.getResource( HADOOP_SITE );
URL core_site = cl.getResource( CORE_SITE );
if( hadoop_site == null core_site == null ) {
throw new ExecException(Cannot find hadoop configurations in
classpath (neither hadoop-site.xml nor core-site.xml was found in the
classpath). +
If you plan to use local mode, please put -x local
option in command line,
4010);
}
{code}
This assumes the resources (*-site.xml) are set on the classpath, but this
will not always be the case when run with Pig's Java APIs. One could want to
programatically set the resources and the code here should additionally check
if they are available in there.
Example: When a Configuration object is created and resources are added
before passing it on to Pig.
{code}
Configuration conf = new Configuration(false);
conf.addResource(foo/core-site.xml);
conf.addResource(bar/hadoop-site.xml);
PigServer pServer = new PigServer(ExecType.MAPREDUCE, conf);
{code}
The above conf is not used right now to obtain resources.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3145) Parameters in core-site.xml and mapred-site.xml are not correctly substituted
[
https://issues.apache.org/jira/browse/PIG-3145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566856#comment-13566856
]
Jarek Jarcec Cecho commented on PIG-3145:
-
+1 (non-binding)
The change seems reasonable to me.
Jarcec
Parameters in core-site.xml and mapred-site.xml are not correctly substituted
-
Key: PIG-3145
URL: https://issues.apache.org/jira/browse/PIG-3145
Project: Pig
Issue Type: Bug
Reporter: Cheolsoo Park
Assignee: Cheolsoo Park
Attachments: PIG-3145.patch
To reproduce the issue, please do the following:
# Parameterize the address of name node in core-site.xml.
{code}
property
namefs.default.name/name
valuehdfs://${foo}:8020/value
/property
{code}
# Set the value of foo via -D option.
{code}
export PIG_OPTS=-Dfoo=mr1-0.cheolsoo.com
{code}
# Pig fails with the following error.
{code}
2013-01-28 18:54:02,786 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting
to hadoop file system at: hdfs://${foo}:8020
2013-01-28 18:54:02,805 [main] ERROR org.apache.pig.Main - ERROR 2999:
Unexpected internal error. null
Details at logfile: /home/cheolsoo/pig-cdh/pig_1359428042522.log
{code}
Note that the parameter $\{foo\} in core-site.xml is not expanded. This is
because the addresses of name node and job tracker are read directly from
core-site.xml instead of reading via Configuration.get().
{code:title=HExecutionEngine.java}
// properties is Java Properties
cluster = properties.getProperty(JOB_TRACKER_LOCATION);
nameNode = properties.getProperty(FILE_SYSTEM_LOCATION);
{code}
Replacing these lines with Configuration.get() fixes the issue.
{code:title=HExecutionEngine.java}
// jc is Hadoop Configuration
cluster = jc.get(JOB_TRACKER_LOCATION);
nameNode = jc.get(FILE_SYSTEM_LOCATION);
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2994) Grunt shortcuts
[
https://issues.apache.org/jira/browse/PIG-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566951#comment-13566951
]
Prasanth J commented on PIG-2994:
-
Great suggestions Cheolsoo! Thanks for your review.
Regarding your 2nd suggestion, one thing I noticed from the grunt parser code
is that only ILLUSTRATE and EXPLAIN operator supports -script option wherein
we can pass a pig script as an argument. Hence these commands are valid even
when last alias is not defined yet.
For example:
{code}grunt \i -script /illustrate.pig{code}
is a valid operation even when there is no previously defined alias. Similarly
for explain.
Hence just checking 'alias == null' after will make the above operation
invalid. So my suggestion is to do the following
{code}alias = mPigServer.getPigContext().getLastAlias();
if (alias == null script == null) {
throw new ParseException('illustrate' statement must be on an alias or on
a script.);
}{code}
Please let me know if this looks good. I will resubmit the patch incorporating
the changes you had mentioned along with documentation.
Grunt shortcuts
---
Key: PIG-2994
URL: https://issues.apache.org/jira/browse/PIG-2994
Project: Pig
Issue Type: New Feature
Components: grunt
Reporter: Prasanth J
Assignee: Prasanth J
Priority: Minor
Attachments: PIG-2994.1.git.patch
This feature is aimed at providing shortcuts for frequently used commands
like illustrate, dump, explain, describe, quit, help etc. This feature is
inspired from postgres(psql) shortcuts. I tried implementing a simple
shortcut for quitting the grunt shell using \q with very minimal changes. I
think this feature will help save many keystrokes for users. If this feature
looks useful I can submit the current patch for review and go ahead with
implementing the following shortcuts
\i alias - illustrate
\e alias - explain
\de alias - describe
\du alias - dump
\h - help
This will also be useful to view information about tables/statistics stored
in HCatalog similar to the way psql does.
\dt alias - display table
\dm - display metadata
etc..
except \t, \r and \n delimiters we should be able to use all other characters
as shortcuts.
Please let me know your thoughts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2994) Grunt shortcuts
[ https://issues.apache.org/jira/browse/PIG-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566964#comment-13566964 ] Cheolsoo Park commented on PIG-2994: Sounds good to me. Thanks for the quick response! Grunt shortcuts --- Key: PIG-2994 URL: https://issues.apache.org/jira/browse/PIG-2994 Project: Pig Issue Type: New Feature Components: grunt Reporter: Prasanth J Assignee: Prasanth J Priority: Minor Attachments: PIG-2994.1.git.patch This feature is aimed at providing shortcuts for frequently used commands like illustrate, dump, explain, describe, quit, help etc. This feature is inspired from postgres(psql) shortcuts. I tried implementing a simple shortcut for quitting the grunt shell using \q with very minimal changes. I think this feature will help save many keystrokes for users. If this feature looks useful I can submit the current patch for review and go ahead with implementing the following shortcuts \i alias - illustrate \e alias - explain \de alias - describe \du alias - dump \h - help This will also be useful to view information about tables/statistics stored in HCatalog similar to the way psql does. \dt alias - display table \dm - display metadata etc.. except \t, \r and \n delimiters we should be able to use all other characters as shortcuts. Please let me know your thoughts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3135) HExecutionEngine should look for resources in user passed Properties
[
https://issues.apache.org/jira/browse/PIG-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13567030#comment-13567030
]
Prashant Kommireddi commented on PIG-3135:
--
Thanks for the quick review. Your comments make sense, I will make the above
changes and upload a new patch.
HExecutionEngine should look for resources in user passed Properties
Key: PIG-3135
URL: https://issues.apache.org/jira/browse/PIG-3135
Project: Pig
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Prashant Kommireddi
Assignee: Prashant Kommireddi
Attachments: PIG-3135.patch
Looking at this snippet:
{code}
private void init(Properties properties) throws ExecException {
.
.
.
// Check existence of hadoop-site.xml or core-site.xml
Configuration testConf = new Configuration();
ClassLoader cl = testConf.getClassLoader();
URL hadoop_site = cl.getResource( HADOOP_SITE );
URL core_site = cl.getResource( CORE_SITE );
if( hadoop_site == null core_site == null ) {
throw new ExecException(Cannot find hadoop configurations in
classpath (neither hadoop-site.xml nor core-site.xml was found in the
classpath). +
If you plan to use local mode, please put -x local
option in command line,
4010);
}
{code}
This assumes the resources (*-site.xml) are set on the classpath, but this
will not always be the case when run with Pig's Java APIs. One could want to
programatically set the resources and the code here should additionally check
if they are available in there.
Example: When a Configuration object is created and resources are added
before passing it on to Pig.
{code}
Configuration conf = new Configuration(false);
conf.addResource(foo/core-site.xml);
conf.addResource(bar/hadoop-site.xml);
PigServer pServer = new PigServer(ExecType.MAPREDUCE, conf);
{code}
The above conf is not used right now to obtain resources.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Pig 11.0
Thanks Bill, that will be great. Yes, the doc is up to date: https://cwiki.apache.org/confluence/display/PIG/HowToRelease Thanks, Daniel On Wed, Jan 30, 2013 at 11:13 AM, Bill Graham billgra...@gmail.com wrote: Olga if you've got your hands full I'll volunteer to wear the release manager hat this round. Alan/Olga is this doc current? *https://cwiki.apache.org/PIG/howtorelease.html* On Tue, Jan 29, 2013 at 7:26 PM, Alan Gates ga...@hortonworks.com wrote: Olga had previously volunteered to be RM for this release. Olga, did you still want to do this or are you open to someone else taking it on? BTW, I don't have time to drive it right now either, though I'm happy to coach a first time RM through it if Olga doesn't want to do it. Alan. On Jan 29, 2013, at 7:00 PM, Daniel Dai wrote: My hands are full these several days, I can try after several days but no promise. Thanks, Daniel On Tue, Jan 29, 2013 at 11:40 AM, Dmitriy Ryaboy dvrya...@gmail.com wrote: Sounds like we can roll a release. Who wants to do the honors? I think Olga and Daniel have done it in the past, not sure if they have the time? On Mon, Jan 28, 2013 at 11:37 PM, Bill Graham billgra...@gmail.com wrote: I've just committed the 2 documentation jiras and all Pig 0.11 issues are resolved once again. On Mon, Jan 28, 2013 at 7:22 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: e2e tests are fine on my end too except for hcat tests (Did not have hcat setup) and few transient Errors test failures which is a known issue. Ran with Hadoop 0.23. Regards, Rohini On Mon, Jan 28, 2013 at 4:44 PM, Julien Le Dem jul...@twitter.com wrote: It sounds like we are ready to go. There are two remaining documentation JIRAs that will be committed soon. Daniel, do you want to build the release? Julien On Mon, Jan 28, 2013 at 10:12 AM, Daniel Dai da...@hortonworks.com wrote: All tests pass for me, no single failures or aborts. Daniel On Mon, Jan 28, 2013 at 8:33 AM, Cheolsoo Park cheol...@cloudera.com wrote: Here are my results: Hadoop-1.0.x: [exec] Final results ,PASSED: 612 FAILED: 2 SKIPPED: 24 ABORTED: 1FAILED DEPENDENCY: 0 - The failures are simply because I didn't install Hcatalog. Hadoop-2.0.x: [exec] Final results ,PASSED: 567 FAILED: 3 SKIPPED: 27 ABORTED: 42 FAILED DEPENDENCY: 0 - The failures seem due to issues in my cluster rather than Pig issues. I will re-run them to verify. On Fri, Jan 25, 2013 at 5:31 PM, Cheolsoo Park cheol...@cloudera.com wrote: I will also run e2e on Hadoop-1.x and Hadoop-2.x. On Fri, Jan 25, 2013 at 5:02 PM, Daniel Dai da...@hortonworks.com wrote: I will run e2e tests on Hadoop 1.x over the weekend. Thanks, Daniel On Fri, Jan 25, 2013 at 4:27 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: Thats good :). Unit tests have all been passing. I haven't run e2e tests on pig 0.11 for sometime. Will kick off one this weekend. It would be nice if Cheolsoo and Daniel can also kick off one run. Regards, Rohini On Fri, Jan 25, 2013 at 4:08 PM, Julien Le Dem jul...@twitter.com wrote: It looks like all the tickets for Pig 0.11 have been resolved as of today. See: https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%20%220.11%22%20AND%20project%20%3D%20PIG%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20updated%20DESC%2C%20created%20ASC%2C%20priority%20DESC I propose we make the release 0.11.0 next week. Julien -- *Note that I'm no longer using my Yahoo! email address. Please email me at billgra...@gmail.com going forward.* -- *Note that I'm no longer using my Yahoo! email address. Please email me at billgra...@gmail.com going forward.*
[jira] [Commented] (PIG-3149) e2e build.xml still refers to jython 2.5.0 jar even though it's replaced by jython standalone 2.5.2 jar
[ https://issues.apache.org/jira/browse/PIG-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13567124#comment-13567124 ] Rohini Palaniswamy commented on PIG-3149: - +1 e2e build.xml still refers to jython 2.5.0 jar even though it's replaced by jython standalone 2.5.2 jar --- Key: PIG-3149 URL: https://issues.apache.org/jira/browse/PIG-3149 Project: Pig Issue Type: Bug Components: e2e harness Affects Versions: 0.11 Reporter: Cheolsoo Park Assignee: Cheolsoo Park Fix For: 0.12 Attachments: PIG-3149.patch PIG-2665 replaced jython-2.5.0.jar with jython-standalone-2.5.2.jar. But e2e build.xml still refers to the former. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3149) e2e build.xml still refers to jython 2.5.0 jar even though it's replaced by jython standalone 2.5.2 jar
[ https://issues.apache.org/jira/browse/PIG-3149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13567128#comment-13567128 ] Johnny Zhang commented on PIG-3149: --- +1 (non-binding) e2e build.xml still refers to jython 2.5.0 jar even though it's replaced by jython standalone 2.5.2 jar --- Key: PIG-3149 URL: https://issues.apache.org/jira/browse/PIG-3149 Project: Pig Issue Type: Bug Components: e2e harness Affects Versions: 0.11 Reporter: Cheolsoo Park Assignee: Cheolsoo Park Fix For: 0.12 Attachments: PIG-3149.patch PIG-2665 replaced jython-2.5.0.jar with jython-standalone-2.5.2.jar. But e2e build.xml still refers to the former. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2994) Grunt shortcuts
[ https://issues.apache.org/jira/browse/PIG-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated PIG-2994: Attachment: PIG-2994.2.git.patch Grunt shortcuts --- Key: PIG-2994 URL: https://issues.apache.org/jira/browse/PIG-2994 Project: Pig Issue Type: New Feature Components: grunt Reporter: Prasanth J Assignee: Prasanth J Priority: Minor Attachments: PIG-2994.1.git.patch, PIG-2994.2.git.patch This feature is aimed at providing shortcuts for frequently used commands like illustrate, dump, explain, describe, quit, help etc. This feature is inspired from postgres(psql) shortcuts. I tried implementing a simple shortcut for quitting the grunt shell using \q with very minimal changes. I think this feature will help save many keystrokes for users. If this feature looks useful I can submit the current patch for review and go ahead with implementing the following shortcuts \i alias - illustrate \e alias - explain \de alias - describe \du alias - dump \h - help This will also be useful to view information about tables/statistics stored in HCatalog similar to the way psql does. \dt alias - display table \dm - display metadata etc.. except \t, \r and \n delimiters we should be able to use all other characters as shortcuts. Please let me know your thoughts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2994) Grunt shortcuts
[ https://issues.apache.org/jira/browse/PIG-2994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13567134#comment-13567134 ] Prasanth J commented on PIG-2994: - Hi Cheolsoo Submitted the new patch with the changes incorporated. Added few more testcases and updated documentation. Please let me know if the patch looks good. Grunt shortcuts --- Key: PIG-2994 URL: https://issues.apache.org/jira/browse/PIG-2994 Project: Pig Issue Type: New Feature Components: grunt Reporter: Prasanth J Assignee: Prasanth J Priority: Minor Attachments: PIG-2994.1.git.patch, PIG-2994.2.git.patch This feature is aimed at providing shortcuts for frequently used commands like illustrate, dump, explain, describe, quit, help etc. This feature is inspired from postgres(psql) shortcuts. I tried implementing a simple shortcut for quitting the grunt shell using \q with very minimal changes. I think this feature will help save many keystrokes for users. If this feature looks useful I can submit the current patch for review and go ahead with implementing the following shortcuts \i alias - illustrate \e alias - explain \de alias - describe \du alias - dump \h - help This will also be useful to view information about tables/statistics stored in HCatalog similar to the way psql does. \dt alias - display table \dm - display metadata etc.. except \t, \r and \n delimiters we should be able to use all other characters as shortcuts. Please let me know your thoughts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: Pig 11.0
Cool. I hope to be able to get to this later in the week. On Wed, Jan 30, 2013 at 3:05 PM, Daniel Dai da...@hortonworks.com wrote: Thanks Bill, that will be great. Yes, the doc is up to date: https://cwiki.apache.org/confluence/display/PIG/HowToRelease Thanks, Daniel On Wed, Jan 30, 2013 at 11:13 AM, Bill Graham billgra...@gmail.com wrote: Olga if you've got your hands full I'll volunteer to wear the release manager hat this round. Alan/Olga is this doc current? *https://cwiki.apache.org/PIG/howtorelease.html* On Tue, Jan 29, 2013 at 7:26 PM, Alan Gates ga...@hortonworks.com wrote: Olga had previously volunteered to be RM for this release. Olga, did you still want to do this or are you open to someone else taking it on? BTW, I don't have time to drive it right now either, though I'm happy to coach a first time RM through it if Olga doesn't want to do it. Alan. On Jan 29, 2013, at 7:00 PM, Daniel Dai wrote: My hands are full these several days, I can try after several days but no promise. Thanks, Daniel On Tue, Jan 29, 2013 at 11:40 AM, Dmitriy Ryaboy dvrya...@gmail.com wrote: Sounds like we can roll a release. Who wants to do the honors? I think Olga and Daniel have done it in the past, not sure if they have the time? On Mon, Jan 28, 2013 at 11:37 PM, Bill Graham billgra...@gmail.com wrote: I've just committed the 2 documentation jiras and all Pig 0.11 issues are resolved once again. On Mon, Jan 28, 2013 at 7:22 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: e2e tests are fine on my end too except for hcat tests (Did not have hcat setup) and few transient Errors test failures which is a known issue. Ran with Hadoop 0.23. Regards, Rohini On Mon, Jan 28, 2013 at 4:44 PM, Julien Le Dem jul...@twitter.com wrote: It sounds like we are ready to go. There are two remaining documentation JIRAs that will be committed soon. Daniel, do you want to build the release? Julien On Mon, Jan 28, 2013 at 10:12 AM, Daniel Dai da...@hortonworks.com wrote: All tests pass for me, no single failures or aborts. Daniel On Mon, Jan 28, 2013 at 8:33 AM, Cheolsoo Park cheol...@cloudera.com wrote: Here are my results: Hadoop-1.0.x: [exec] Final results ,PASSED: 612 FAILED: 2 SKIPPED: 24 ABORTED: 1FAILED DEPENDENCY: 0 - The failures are simply because I didn't install Hcatalog. Hadoop-2.0.x: [exec] Final results ,PASSED: 567 FAILED: 3 SKIPPED: 27 ABORTED: 42 FAILED DEPENDENCY: 0 - The failures seem due to issues in my cluster rather than Pig issues. I will re-run them to verify. On Fri, Jan 25, 2013 at 5:31 PM, Cheolsoo Park cheol...@cloudera.com wrote: I will also run e2e on Hadoop-1.x and Hadoop-2.x. On Fri, Jan 25, 2013 at 5:02 PM, Daniel Dai da...@hortonworks.com wrote: I will run e2e tests on Hadoop 1.x over the weekend. Thanks, Daniel On Fri, Jan 25, 2013 at 4:27 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: Thats good :). Unit tests have all been passing. I haven't run e2e tests on pig 0.11 for sometime. Will kick off one this weekend. It would be nice if Cheolsoo and Daniel can also kick off one run. Regards, Rohini On Fri, Jan 25, 2013 at 4:08 PM, Julien Le Dem jul...@twitter.com wrote: It looks like all the tickets for Pig 0.11 have been resolved as of today. See: https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%20%220.11%22%20AND%20project%20%3D%20PIG%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20updated%20DESC%2C%20created%20ASC%2C%20priority%20DESC I propose we make the release 0.11.0 next week. Julien -- *Note that I'm no longer using my Yahoo! email address. Please email me at billgra...@gmail.com going forward.* -- *Note that I'm no longer using my Yahoo! email address. Please email me at billgra...@gmail.com going forward.* -- *Note that I'm no longer using my Yahoo! email address. Please email me at billgra...@gmail.com going forward.*
[jira] Subscription: PIG patch available
Issue Subscription Filter: PIG patch available (28 issues) Subscriber: pigdaily Key Summary PIG-3149e2e build.xml still refers to jython 2.5.0 jar even though it's replaced by jython standalone 2.5.2 jar https://issues.apache.org/jira/browse/PIG-3149 PIG-3142Fixed-width load and store functions for the Piggybank https://issues.apache.org/jira/browse/PIG-3142 PIG-3137fix Piggybank test to not using /tmp dir https://issues.apache.org/jira/browse/PIG-3137 PIG-3136Introduce a syntax making declared aliases optional https://issues.apache.org/jira/browse/PIG-3136 PIG-3123Simplify Logical Plans By Removing Unneccessary Identity Projections https://issues.apache.org/jira/browse/PIG-3123 PIG-3122Operators should not implicitly become reserved keywords https://issues.apache.org/jira/browse/PIG-3122 PIG-3114Duplicated macro name error when using pigunit https://issues.apache.org/jira/browse/PIG-3114 PIG-3108HBaseStorage returns empty maps when mixing wildcard- with other columns https://issues.apache.org/jira/browse/PIG-3108 PIG-3105Fix TestJobSubmission unit test failure. https://issues.apache.org/jira/browse/PIG-3105 PIG-3098Add another test for the self join case https://issues.apache.org/jira/browse/PIG-3098 PIG-3088Add a builtin udf which removes prefixes https://issues.apache.org/jira/browse/PIG-3088 PIG-3069Native Windows Compatibility for Pig E2E Tests and Harness https://issues.apache.org/jira/browse/PIG-3069 PIG-3028testGrunt dev test needs some command filters to run correctly without cygwin https://issues.apache.org/jira/browse/PIG-3028 PIG-3027pigTest unit test needs a newline filter for comparisons of golden multi-line https://issues.apache.org/jira/browse/PIG-3027 PIG-3026Pig checked-in baseline comparisons need a pre-filter to address OS-specific newline differences https://issues.apache.org/jira/browse/PIG-3026 PIG-3025TestPruneColumn unit test - SimpleEchoStreamingCommand perl inline script needs simplification https://issues.apache.org/jira/browse/PIG-3025 PIG-3024TestEmptyInputDir unit test - hadoop version detection logic is brittle https://issues.apache.org/jira/browse/PIG-3024 PIG-3015Rewrite of AvroStorage https://issues.apache.org/jira/browse/PIG-3015 PIG-3010Allow UDF's to flatten themselves https://issues.apache.org/jira/browse/PIG-3010 PIG-2959Add a pig.cmd for Pig to run under Windows https://issues.apache.org/jira/browse/PIG-2959 PIG-2955 Fix bunch of Pig e2e tests on Windows https://issues.apache.org/jira/browse/PIG-2955 PIG-2878Pig current releases lack a UDF equalIgnoreCase.This function returns a Boolean value indicating whether string left is equal to string right. This check is case insensitive. https://issues.apache.org/jira/browse/PIG-2878 PIG-2873Converting bin/pig shell script to python https://issues.apache.org/jira/browse/PIG-2873 PIG-2834MultiStorage requires unused constructor argument https://issues.apache.org/jira/browse/PIG-2834 PIG-2661Pig uses an extra job for loading data in Pigmix L9 https://issues.apache.org/jira/browse/PIG-2661 PIG-1942script UDF (jython) should utilize the intended output schema to more directly convert Py objects to Pig objects https://issues.apache.org/jira/browse/PIG-1942 PIG-1914Support load/store JSON data in Pig https://issues.apache.org/jira/browse/PIG-1914 PIG-1237Piggybank MutliStorage - specify field to write in output https://issues.apache.org/jira/browse/PIG-1237 You may edit this subscription at: https://issues.apache.org/jira/secure/FilterSubscription!default.jspa?subId=13225filterId=12322384
