date:20100114

[jira] Commented: (PIG-1186) Pig do not take values in pig-cluster-hadoop-site.xml

2010-01-14 Thread Daniel Dai (JIRA)


[ 
https://issues.apache.org/jira/browse/PIG-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800285#action_12800285
 ] 

Daniel Dai commented on PIG-1186:
-

I didn't include unit test because it is very hard to write a unit test for 
this. I tested it manually and it works.

 Pig do not take values in pig-cluster-hadoop-site.xml
 ---

 Key: PIG-1186
 URL: https://issues.apache.org/jira/browse/PIG-1186
 Project: Pig
  Issue Type: Bug
  Components: impl
Affects Versions: 0.6.0
Reporter: Daniel Dai
Assignee: Daniel Dai
 Fix For: 0.6.0

 Attachments: PIG-1186-1.patch




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-1178) LogicalPlan and Optimizer are too complex and hard to work with

2010-01-14 Thread Ying He (JIRA)

[
https://issues.apache.org/jira/browse/PIG-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800289#action_12800289
]

Ying He commented on PIG-1178:
--

LogicalPlan and Optimizer are too complex and hard to work with
---

Key: PIG-1178
URL: https://issues.apache.org/jira/browse/PIG-1178
Project: Pig
Issue Type: Improvement
Reporter: Alan Gates
Assignee: Ying He
Attachments: expressions.patch, lp.patch, PIG_1178.patch

The current implementation of the logical plan and the logical optimizer in
Pig has proven to not be easily extensible. Developer feedback has indicated
that adding new rules to the optimizer is quite burdensome. In addition, the
logical plan has been an area of numerous bugs, many of which have been
difficult to fix. Developers also feel that the logical plan is difficult to
understand and maintain. The root cause for these issues is that a number of
design decisions that were made as part of the 0.2 rewrite of the front end
have now proven to be sub-optimal. The heart of this proposal is to revisit a
number of those proposals and rebuild the logical plan with a simpler design
that will make it much easier to maintain the logical plan as well as extend
the logical optimizer.
See http://wiki.apache.org/pig/PigLogicalPlanOptimizerRewrite for full
details.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-1187) UTF-8 (international code) breaks with loader when load with schema is specified

2010-01-14 Thread Viraj Bhat (JIRA)


[ 
https://issues.apache.org/jira/browse/PIG-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12800315#action_12800315
 ] 

Viraj Bhat commented on PIG-1187:
-

Hi Jeff,
 This is specific to the data we are using and it looks like parser failed when 
it is trying to interpret some characters. As such we have tested this with 
Chinese characters and it works.
Viraj

 UTF-8 (international code) breaks with loader when load with schema is 
 specified
 

 Key: PIG-1187
 URL: https://issues.apache.org/jira/browse/PIG-1187
 Project: Pig
  Issue Type: Bug
Affects Versions: 0.6.0
Reporter: Viraj Bhat
 Fix For: 0.6.0


 I have a set of Pig statements which dump an international dataset.
 {code}
 INPUT_OBJECT = load 'internationalcode';
 describe INPUT_OBJECT;
 dump INPUT_OBJECT;
 {code}
 Sample output
 (756a6196-ebcd-4789-ad2f-175e5df65d55,{(labelAaÂâÀ),(labelあいうえお1),(labelஜார்க2),(labeladfadf)})
 It works and dumps results but when I use a schema for loading it fails.
 {code}
 INPUT_OBJECT = load 'internationalcode' AS (object_id:chararray, labels: bag 
 {T: tuple(label:chararray)});
 describe INPUT_OBJECT;
 {code}
 The error message is as follows:2010-01-14 02:23:27,320 FATAL 
 org.apache.hadoop.mapred.Child: Error running child : 
 org.apache.pig.data.parser.TokenMgrError: Error: Bailing out of infinite loop 
 caused by repeated empty string matches at line 1, column 21.
   at 
 org.apache.pig.data.parser.TextDataParserTokenManager.TokenLexicalActions(TextDataParserTokenManager.java:620)
   at 
 org.apache.pig.data.parser.TextDataParserTokenManager.getNextToken(TextDataParserTokenManager.java:569)
   at 
 org.apache.pig.data.parser.TextDataParser.jj_ntk(TextDataParser.java:651)
   at 
 org.apache.pig.data.parser.TextDataParser.Tuple(TextDataParser.java:152)
   at 
 org.apache.pig.data.parser.TextDataParser.Bag(TextDataParser.java:100)
   at 
 org.apache.pig.data.parser.TextDataParser.Datum(TextDataParser.java:382)
   at 
 org.apache.pig.data.parser.TextDataParser.Parse(TextDataParser.java:42)
   at 
 org.apache.pig.builtin.Utf8StorageConverter.parseFromBytes(Utf8StorageConverter.java:68)
   at 
 org.apache.pig.builtin.Utf8StorageConverter.bytesToBag(Utf8StorageConverter.java:76)
   at 
 org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POCast.getNext(POCast.java:845)
   at 
 org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(POForEach.java:250)
   at 
 org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(POForEach.java:204)
   at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:249)
   at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:240)
   at 
 org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.map(PigMapOnly.java:65)
   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
   at org.apache.hadoop.mapred.Child.main(Child.java:159)
 Viraj

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

58 matches

Mail list logo