Seems like the functionality I need can only be achieved with a SPLIT. Basically some records could have a Superset/subset relation in which 1 record would be stored in 2 places. With bincond that might not be achievable.
On Wed, Dec 7, 2011 at 8:30 PM, Prashant Kommireddi <[email protected]>wrote: > Im not sure if I am making any obvious mistake here with the syntax, but > the statement errors out with the following message > > Pig Stack Trace > --------------- > ERROR 1200: <file platform-login.pig, line 21, column 45> Syntax error, > unexpected symbol at or near 'loginType' > > org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error > during parsing. <file platform-login.pig, line 21, column 45> Syntax > error, unexpected symbol at or near 'loginType' > at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1639) > at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1584) > at org.apache.pig.PigServer.registerQuery(PigServer.java:584) > at > org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942) > at > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188) > at > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164) > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81) > at org.apache.pig.Main.run(Main.java:553) > at org.apache.pig.Main.main(Main.java:108) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > Caused by: Failed to parse: <file platform-login.pig, line 21, column 45> > Syntax error, unexpected symbol at or near 'loginType' > at > org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:222) > at > org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:164) > at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1631) > ... 14 more > > > > On Wed, Dec 7, 2011 at 7:45 PM, Jonathan Coveney <[email protected]>wrote: > >> yes, you can nest them >> >> 2011/12/7 Prashant Kommireddi <[email protected]> >> >> > Is using nested bicond valid? >> > >> > MergeAll = FOREACH pruneFields GENERATE *, ( loginType matches 'I' ? >> > 'LoginAPI' : >> > loginType matches 'A' ? 'LoginWeb' : >> > loginType matches 'R' ? 'LoginPartner' : >> > loginType matches '2' ? 'LoginInternal' : >> > ( loginType matches 'i' OR loginType matches >> > '6' ) ? : >> > loginType matches '3' ? 'LoginPortal' : >> > ( loginType matches '5' OR loginType matches >> > '8' OR loginType matches 'b' >> > OR loginType matches 'c' OR >> > loginType matches 'h' ) ? 'LoginSSO' : 'LoginOther') as featureName; >> > >> > On Wed, Dec 7, 2011 at 2:46 PM, Xiaomeng Wan <[email protected]> >> wrote: >> > >> > > use nested bincond? >> > > >> > > Shawn >> > > >> > > On Wed, Dec 7, 2011 at 3:05 PM, Prashant Kommireddi < >> [email protected] >> > > >> > > wrote: >> > > > I have a query and I want to improve on the following steps: >> > > > >> > > > LFV is an alias to a Custom UDF. >> > > > >> > > > Step 1: >> > > > pruneFields = FOREACH logs GENERATE >> > > > LFV(row, 'organizationId') as orgId, >> > > > LFV(row, 'userId') as userId, >> > > > (int)LFV(row, 'runTime') as runTime, >> > > > (int)LFV(row, 'cpuTime') as cpuTime, >> > > > (int)LFV(row, 'oracleStatCpuTime') as oct, >> > > > LFV(row, 'logRecordType') as logRecordType, >> > > > LFV(row, 'loginType') as loginType, >> > > > LFV(row, 'timestamp') as ts; >> > > > >> > > > Step 2: >> > > > SPLIT pruneFields INTO LoginAPI IF loginType matches 'I', >> > > > LoginWeb IF loginType matches 'A', >> > > > LoginPartner IF loginType matches 'R', >> > > > LoginInternal IF loginType matches '2', >> > > > LoginOauth IF ( loginType matches 'i' OR >> > loginType >> > > > matches '6' ), >> > > > LoginPortal IF loginType matches '3', >> > > > LoginSSO IF ( loginType matches '5' OR >> loginType >> > > > matches '8' OR loginType matches 'b' >> > > > OR loginType matches 'c' >> OR >> > > > loginType matches 'h' ), >> > > > LoginOther IF loginType matches '.*.'; >> > > > >> > > > Step 3: >> > > > --Append featureName >> > > > LoginAPI = FOREACH LoginAPI GENERATE *, 'LoginAPI' as featureName; >> > > > LoginWeb = FOREACH LoginWeb GENERATE *, 'LoginWeb' as featureName; >> > > > LoginPartner = FOREACH LoginPartner GENERATE *, 'LoginPartner' as >> > > > featureName; >> > > > LoginInternal = FOREACH LoginInternal GENERATE *, 'LoginInternal' as >> > > > featureName; >> > > > LoginOauth = FOREACH LoginOauth GENERATE *, 'LoginOauth' as >> > featureName; >> > > > LoginPortal = FOREACH LoginPortal GENERATE *, 'LoginPortal' as >> > > featureName; >> > > > LoginSSO = FOREACH LoginSSO GENERATE *, 'LoginSSO' as featureName; >> > > > LoginOther = FOREACH LoginOther GENERATE *, 'LoginOther' as >> > featureName; >> > > > >> > > > --Merge the various login types >> > > > MergeAll = UNION LoginAPI, LoginWeb, LoginPartner, LoginInternal, >> > > > LoginOauth, LoginPortal, LoginSSO, LoginOther; >> > > > >> > > > >> > > > I am having to split "pruneFields" to be able to append a >> featureName >> > in >> > > > Step 3. Any better way this could be done? >> > > > >> > > > Thanks, >> > > > Prashant >> > > >> > >> > >
