Hey Jeff, So you do bursts of imports, on a scheduled task or something?
cheers, Scott. On 14/01/2014, at 9:38 AM, Jeff Coughlin <[email protected]> wrote: > I have scripts that import thousands of rows at once for daily imports using > createData(). The trick is to batch them so as not to fill up all the RAM in > java - the more you do per batch, the slower it gets. Also, if indexing them > in solr, make sure to set commit to false and do the solr commit command > after all the imports are done (or after each batch if you prefer). Doing > both of these suggestions above will make your imports super fast and you > won't have to worry about missing and fields, custom functions, or any ORM > issues, and instead allow FarCry to do what it does best. > > -- > Jeff Coughlin > > On Jan 13, 2014, at 6:02 PM, Scott Mebberson <[email protected]> wrote: > >> Hey guys, >> >> Yeah. We don't always import using createData or setData. When doing it on >> 1000s of rows (daily) it's WAYYYYY to slow. >> >> On one instance, we wrote a bunch of import scripts based largely in bulk >> inserts within MYSQL (CSV straight to MYSQL) and then filled in the gaps >> that the framework usually does via createData and setData. >> >> Lots of testing to get that right though! >> >> My current instance is slightly different though. But it's all working sweet >> now, thanks. >> >> cheers, >> Scott. >> >> On 14/01/2014, at 9:28 AM, Jeff Coughlin <[email protected]> wrote: >> >>>> Yeah, the label property will only get set by the framework if setData() >>>> is called >>> >>> or when using createData() - which is what you'd normally use when manually >>> importing data into FarCry. >>> >>> ...Unless you are setting the label in beforeSave() in which case manually >>> calling createData() does not call beforeSave(). This is a known bug >>> that's just been around for years (can't find the bug number). We just >>> always make sure to call beforeSave() manually before using createData(). >>> >>> And always make sure to call application.fc.factory.farFU.setSystemFU() to >>> create the friendly url as well after using createData(), as createData() >>> doesn't call that either if also using displaySystemFU.cfm (bug number >>> FC-2268). So just to be safe, I always call the aforementioned method for >>> FU's after calling createData(). >>> >>> Good luck. >>> >>> -- >>> Jeff Coughlin >>> >>> On Jan 13, 2014, at 5:36 PM, Justin Carter <[email protected]> >>> wrote: >>> >>>> Yeah, the label property will only get set by the framework if setData() >>>> is called, so if it was a direct SQL insert then it will bypass the >>>> framework :) >>>> >>>> cheers, >>>> Justin >>>> >>>> -- >>>> Justin Carter >>>> http://www.madfellas.com/blog >>>> http://twitter.com/justincarter >>>> >>>> >>>> On Tue, Jan 14, 2014 at 9:31 AM, Scott Mebberson >>>> <[email protected]> wrote: >>>> Hey Jeff, >>>> >>>> Of course. I'll give that a shot, and verify it that it doesn't happen for >>>> new instances. I'm working with data imported by someone else, so chances >>>> are they didn't script label field up when inserting the data. >>>> >>>> I think everyone can ignore this post! Sorry. >>>> >>>> cheers, >>>> Scott. >>>> >>>> On 14/01/2014, at 8:57 AM, Jeff Coughlin <[email protected]> wrote: >>>> >>>>> I haven't seen this, but I'm still on FC 6.3 at the moment (except for >>>>> some testing I'm doing in 7). Until a fix is found, one thing you can do >>>>> is force it to save the title to the label by creating a custom >>>>> beforeSave() method inside the custom type. If anything it will at least >>>>> give you a temporary workaround. But if this is a bug in FC7, then it's >>>>> a pretty serious one (IMO). >>>>> >>>>> Example: >>>>> >>>>> <cffunction name="beforeSave" access="public" output="false" >>>>> returntype="struct"> >>>>> <cfargument name="stProperties" required="true" type="struct" /> >>>>> <cfargument name="stFields" required="true" type="struct" /> >>>>> <cfargument name="stFormPost" required="false" type="struct" /> >>>>> >>>>> <cfset arguments.stProperties.label = arguments.stProperties.title /> >>>>> >>>>> <cfreturn super.beforeSave(argumentCollection=arguments) /> >>>>> </cffunction> >>>>> >>>>> If you feel comfortable sharing your CFC, go ahead and post it (maybe >>>>> we'll see something you're missing - sometimes it just takes a second set >>>>> of eyes to see something we've overlooked). If not, you are welcome to >>>>> send it to just me and I'll be happy to see if something jumps out. >>>>> >>>>> -- >>>>> Jeff Coughlin >>>>> >>>>> On Jan 13, 2014, at 5:12 PM, Scott Mebberson <[email protected]> >>>>> wrote: >>>>> >>>>>> Has anyone run into a problem in which labels for a custom object type >>>>>> aren't working on all occasions. Works for some instances of the object >>>>>> type, but not others. The type does have a title property, which always >>>>>> contains the real value. >>>>>> >>>>>> I've tried the following: >>>>>> updateApp (a few dozen times, and after each of the following) >>>>>> added bLabel to the title cfproperty tag >>>>>> added displayLabel.cfm to the type folder in webskins and am outputting >>>>>> stObj.title >>>>>> But still, no label. The label doesn't exist in the stObj that is passed >>>>>> around to the various webskins. >>>>>> >>>>>> Any ideas? >>>>>> >>>>>> I'm running FarCry 7 (I did git pull on both core and farcrycms this >>>>>> morning so they're fresh!). >>>>>> >>>>>> Thanks, >>>>>> Scott. > > > -- > You received this message cos you are subscribed to "farcry-dev" Google group. > To post, email: [email protected] > To unsubscribe, email: [email protected] > For more options: http://groups.google.com/group/farcry-dev > -------------------------------- > Follow us on Twitter: http://twitter.com/farcry > --- > You received this message because you are subscribed to a topic in the Google > Groups "farcry-dev" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/farcry-dev/yrrRHhE0H-4/unsubscribe. > To unsubscribe from this group and all of its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. -- You received this message cos you are subscribed to "farcry-dev" Google group. To post, email: [email protected] To unsubscribe, email: [email protected] For more options: http://groups.google.com/group/farcry-dev -------------------------------- Follow us on Twitter: http://twitter.com/farcry --- You received this message because you are subscribed to the Google Groups "farcry-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
