@Kaan, thank you for continuing to respond. I think it's time to move beyond this thread and take action to have the potential issue looked at properly.
If you think you've found an issue, you should create a Public Issue Tracker thread <https://code.google.com/p/googleappengine/issues/list> with one of the following: 1. A timeframe on an affected instance, or 2. An example that can be used to reproduce, or 3. At least your custom task monitoring solution and leave the creation of a massive burst of tasks to the engineer that attempts to reproduce 4. You could even, in theory, request the thread you create to be set to private so you can upload your whole codebase The bottom line is this: no issue can be looked at or worked on, if it is claimed by somebody who saw it in the wild, until it's reproducible or at least has some kind of trace in logs somewhere. We're happy to look up logs on an affected version/module/instance/etc. during a given timeframe, and try to determine what happened, if anything indeed happened. Either of these will suffice, but anything less is not really valid when reporting a bug. Every software project in the galaxy beyond hobbyist-size will have these requirements for bug reporting. At present, no viable data has been produced on how to observe the behaviour claimed to occur. All we have is the fact that a custom implementation of task-counting produced the numbers "11476" and "11507" as output. I hope you can see the gap between what's produced and what's needed. If nobody who has observed the issue (if indeed there is such an issue to observe) and wants to see a fix (I assume this is you) wants to point towards a reproducing example, the very least information requested was a time-frame on an affected instance. All that's needed is to say: > On my project primal-buttress-1337, instance 1 of module devel, version > v1, began enqueuing 1,000,000 tasks at [TIMESTAMP], and you can see a log > line for each task being added, up until the last task is added at > [TIMESTAMP]. The last task was executed at [TIMESTAMP]. The number of > Flobbb entities created in the Datastore is only 999,998. Therefore I > believe this indicates 2 tasks were enqueued but never executed. There are > no records in the logs of any task failures. If and when such information is produced in a Public Issue Tracker thread, you'll be pleased to see it taken up, worked on, and a solution will come in a course of time appropriate to the severity of the issue. Up to this point, such viable data has not been received, and hence the deadlocked nature of this thread. Please proceed to open a Public Issue Tracker thread with the information necessary to report an issue. On Sunday, March 1, 2015 at 1:23:12 PM UTC-5, Kaan Soral wrote: > > I just ran my tasks, in 2 stages > > In the first stage, task sequences executed as expected, I only noticed a > single additional incoming task, thought it was a glitch / repetition in > the counting logic > I also experienced a ndb.delete_multi lock issue that is not related to > the taskqueue, but works to count/combine some side metrics, this deadlock > prevented that operation, caused some data loss, yet the taskqueue's were > unaffected (first time I experienced this issue) > > ---- > > I just ran another set of task sequences, this time I noticed a > significant increase in additionally executed tasks > > "Workers Out: 11476 In: 11507" > > 31 tasks re-executed, since these tasks are unique by name, they don't > cause task explosions, yet I verify that these additional tasks really > executed, because the total number of elements visited are proportionally > increased > > (I don't see any task error logs, these issues / executions are all silent) > > ---- > > To sum up, I didn't notice any missing tasks this time around, yet I > noticed some invisibly re-executed tasks > > To sum up this entire flurry, the main issue is that, tasks re-execute, > fail silently, these are all counted as retries, yet they might happen 100 > times and count as 100 retries, these "retry"'s should actually be user > failures and not internal ones > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/057d423c-95f8-4f61-b7e7-d48c68f61c1e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
