Yes. That's a feature that the RLS returns the results as LFN:PFN pairs even if it means returning the LFN names multiple times.
I think you could expect returning your entire LRC catalog to take a several minutes. Since 800k entries in 10 minutes is approx 1300~ per second, that is about the rate that I'd expect. I do agree that we could use that get_all_lfn() function. If you get a stack trace for your crash that'd be good. You can add a record to bugzilla.globus.org. rob ________________________________ From: Ole Weidner [mailto:[EMAIL PROTECTED] Sent: Friday, January 18, 2008 10:21 AM To: Robert Schuler Cc: [email protected] Subject: Re: [gt-user] RLS: list LFNs Hi Rob, A get_all_lfn() function would IMHO be really useful ;-) But I don't think that the std::vector should be a problem: I'm not talking about seconds when I say slow - I'm talking about ~ 10 minutes on a LRC server with ~ 800.000 entries... - even if I comment out the push_back() line. I'll try to produce a useful stacktrace for the mmalloc issue. Is the fact that If a LFN has n different PFNs, the LFN shows up n-times in the result list a bug or a feature? Right now I'm processing the result list for redundant entries which is another time consuming task :-( Thanks for your help, Ole On Jan 18, 2008, at 11:50 AM, Robert Schuler wrote: Hi, Ole, The ...get_lfn_wc() function is probably the only way to get all the LFNs from a LRC. In terms of why it is running slow, you might reconsider your use of the "std:vector<...>" template. Whenever you are using "push_back(...)" it may need to reallocate the underlying array. In such case, the operation becomes very expensive and may (I'm not sure) have something to do with why your code is running so slow. Also, the crash (...mmalloc... related) could be related to that vector of yours but we'd need more detail from the stack trace to know. I might add a get_all_lfn(...) type function to a list of feature enhancements, at least so I can discuss it with other users. But for now the wildcard approach is the only way to get the info you want. rob -----Original Message----- From: [EMAIL PROTECTED] on behalf of Ole Weidner Sent: Thu 1/17/2008 1:12 PM To: [email protected] Subject: [gt-user] RLS: list LFNs Aloha, I'm implementing a replica management abstraction on top of the Globus RLS C API. So far, I was quite successful but I got stuck when I tried to retrieve a list of all available LFNs from a LRC. AFAIK there's no API call to do that directly, so I used globus_rls_client_lrc_get_lfn_wc() function with "*" as a wildcard which should give me a list of all LFNs. The code looks like this: std::vector<std::string> LFNList; globus_result_t result = GLOBUS_FALSE; int offset=0; globus_list_t * result_list; result = globus_rls_client_lrc_get_lfn_wc(RLSHandle, "*", rls_pattern_unix, &offset, 200, // reslimit &result_list); if (result != GLOBUS_RLS_SUCCESS) { throw globus_rls_replica_adaptor::exception(result); } else { globus_list_t *p; globus_rls_string2_t *str2; for (p = result_list; p; p = globus_list_rest(p)) { str2 = (globus_rls_string2_t *) globus_list_first(p); LFNList.push_back(std::string(str2->s1)); } globus_rls_client_free_list(p); globus_rls_client_free_list(result_list); } It seems to work but I'm not very satisfied with the results: * It's _incredibly_ sloooooow * If a LFN has n different PFNs, the LFN shows up n-times in the result list * If I don't use "*" as wildcard string (e.g. "/MY/REPLICAS/*") the app crashes after some time: test_rls_replica(9291,0xa0672f60) malloc: *** mmap(size=2097152) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug test_rls_replica(9291,0xa0672f60) malloc: *** mmap(size=2097152) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug terminate called after throwing an instance of 'St9bad_alloc' what(): St9bad_alloc Has anybody experienced something similar or does anybody know a better solution to list LFNs? Any help is much appreciated! Cheers, Ole
