I have a query and I want to improve on the following steps:

LFV is an alias to a Custom UDF.

Step 1:
pruneFields = FOREACH logs GENERATE
                    LFV(row, 'organizationId') as orgId,
                    LFV(row, 'userId') as userId,
                    (int)LFV(row, 'runTime') as runTime,
                    (int)LFV(row, 'cpuTime') as cpuTime,
                    (int)LFV(row, 'oracleStatCpuTime') as oct,
                    LFV(row, 'logRecordType') as logRecordType,
                    LFV(row, 'loginType') as loginType,
                    LFV(row, 'timestamp') as ts;

Step 2:
SPLIT pruneFields INTO LoginAPI IF loginType matches 'I',
                       LoginWeb IF loginType matches 'A',
                       LoginPartner IF loginType matches 'R',
                       LoginInternal IF loginType matches '2',
                       LoginOauth IF ( loginType matches 'i' OR loginType
matches '6' ),
                       LoginPortal IF loginType matches '3',
                       LoginSSO IF ( loginType matches '5' OR loginType
matches '8' OR loginType matches 'b'
                                            OR loginType matches 'c' OR
loginType matches 'h' ),
                       LoginOther IF loginType matches '.*.';

Step 3:
--Append featureName
LoginAPI = FOREACH LoginAPI GENERATE *, 'LoginAPI' as featureName;
LoginWeb = FOREACH LoginWeb GENERATE *, 'LoginWeb' as featureName;
LoginPartner = FOREACH LoginPartner GENERATE *, 'LoginPartner' as
featureName;
LoginInternal = FOREACH LoginInternal GENERATE *, 'LoginInternal' as
featureName;
LoginOauth = FOREACH LoginOauth GENERATE *, 'LoginOauth' as featureName;
LoginPortal = FOREACH LoginPortal GENERATE *, 'LoginPortal' as featureName;
LoginSSO = FOREACH LoginSSO GENERATE *, 'LoginSSO' as featureName;
LoginOther = FOREACH LoginOther GENERATE *, 'LoginOther' as featureName;

--Merge the various login types
MergeAll = UNION LoginAPI, LoginWeb, LoginPartner, LoginInternal,
LoginOauth, LoginPortal, LoginSSO, LoginOther;


I am having to split "pruneFields" to be able to append a featureName in
Step 3. Any better way this could be done?

Thanks,
Prashant

Reply via email to