Re: Batch iteration
Thanks Andrus.
Mark
> On 17 Oct 2016, at 06:28, Andrus Adamchik wrote:
>
> Hi Mark,
>
> Scenario B is now the default after a recent change per [1]. We decoupled the
> transaction handling open iterator (and all its connections) from any
> transactions started by context commits within the iterator. Here is a
> related mailing list thread [2].
>
> Andrus
>
> [1] https://issues.apache.org/jira/browse/CAY-2111
> [2]
> https://lists.apache.org/thread.html/f1ccb1ac0f0aa974ffe32217fe4fcfe4b3d9a11ef2c8513f2f0e8436@%3Cdev.cayenne.apache.org%3E
>
>> On Oct 16, 2016, at 11:45 PM, Mark Wardle wrote:
>>
>> Thanks John. Interestingly, I did some testing and peer contexts aren’t
>> committed until the batch iterator JDBC ResultSet is closed at the end of
>> iteration either. This is fine but for a long running task, I thought that
>> there was a problem (but there wasn’t). I think the only workarounds are a)
>> to not care [which I think is probably fine or b) do other processing in
>> another connection.
>>
>> Mark
>>
>>> On 15 Oct 2016, at 22:34, John Huss wrote:
>>>
>>> I would use a peer context.
>>>
>>> You can create a peer context like this if you runtime has been bound to
>>> the current thread using
>>> CayenneRuntime.bindThreadInjector(runtime.getInjector())
>>>
>>> public static ObjectContext newObjectContext() {
>>> ObjectContextFactory factory = CayenneRuntime.getThreadInjector
>>> ().getInstance(ObjectContextFactory.class);
>>> return (factory != null) ? factory.createContext() : null;
>>> }
>>>
>>>
>>>
>>> On Sat, Oct 15, 2016 at 2:37 PM Mark Wardle wrote:
>>>
Sorry… meant to say using Cayenne M4.0.M3…
Mark
> On 15 Oct 2016, at 20:29, Mark Wardle wrote:
>
> I’m using ResultBatchIterator like this:
>
> Consumer forEach = ...
> try (ResultBatchIterator iterator = query.batchIterator(context,
batchSize)) {
>for(List batch : iterator) {
>for (T c : batch) {
>forEach.accept(c);
>}
>}
>}
>
> but I also want to do some processing, inserting rows, editing objects
etc. I see lots of SQL in my logs, but the transaction doesn’t get
committed until the batch iterator completes, whether I use
context.commitChanges() or context.performGenericQuery() on some custom SQL
created using SQLTemplate.
>
> Is there a way of committing the transaction earlier, or should I use a
peer object context? Is there an easy of getting a new peer editing context
given a single object context or do I need to inject a ServerRuntime here?
>
> Thank you,
>
> Mark
>
>
>>
>
Re: Batch iteration
Hi Mark,
Scenario B is now the default after a recent change per [1]. We decoupled the
transaction handling open iterator (and all its connections) from any
transactions started by context commits within the iterator. Here is a related
mailing list thread [2].
Andrus
[1] https://issues.apache.org/jira/browse/CAY-2111
[2]
https://lists.apache.org/thread.html/f1ccb1ac0f0aa974ffe32217fe4fcfe4b3d9a11ef2c8513f2f0e8436@%3Cdev.cayenne.apache.org%3E
> On Oct 16, 2016, at 11:45 PM, Mark Wardle wrote:
>
> Thanks John. Interestingly, I did some testing and peer contexts aren’t
> committed until the batch iterator JDBC ResultSet is closed at the end of
> iteration either. This is fine but for a long running task, I thought that
> there was a problem (but there wasn’t). I think the only workarounds are a)
> to not care [which I think is probably fine or b) do other processing in
> another connection.
>
> Mark
>
>> On 15 Oct 2016, at 22:34, John Huss wrote:
>>
>> I would use a peer context.
>>
>> You can create a peer context like this if you runtime has been bound to
>> the current thread using
>> CayenneRuntime.bindThreadInjector(runtime.getInjector())
>>
>> public static ObjectContext newObjectContext() {
>> ObjectContextFactory factory = CayenneRuntime.getThreadInjector
>> ().getInstance(ObjectContextFactory.class);
>> return (factory != null) ? factory.createContext() : null;
>> }
>>
>>
>>
>> On Sat, Oct 15, 2016 at 2:37 PM Mark Wardle wrote:
>>
>>> Sorry… meant to say using Cayenne M4.0.M3…
>>>
>>> Mark
>>>
On 15 Oct 2016, at 20:29, Mark Wardle wrote:
I’m using ResultBatchIterator like this:
Consumer forEach = ...
try (ResultBatchIterator iterator = query.batchIterator(context,
>>> batchSize)) {
for(List batch : iterator) {
for (T c : batch) {
forEach.accept(c);
}
}
}
but I also want to do some processing, inserting rows, editing objects
>>> etc. I see lots of SQL in my logs, but the transaction doesn’t get
>>> committed until the batch iterator completes, whether I use
>>> context.commitChanges() or context.performGenericQuery() on some custom SQL
>>> created using SQLTemplate.
Is there a way of committing the transaction earlier, or should I use a
>>> peer object context? Is there an easy of getting a new peer editing context
>>> given a single object context or do I need to inject a ServerRuntime here?
Thank you,
Mark
>>>
>>>
>
Re: Batch iteration
Thanks John. Interestingly, I did some testing and peer contexts aren’t
committed until the batch iterator JDBC ResultSet is closed at the end of
iteration either. This is fine but for a long running task, I thought that
there was a problem (but there wasn’t). I think the only workarounds are a) to
not care [which I think is probably fine or b) do other processing in another
connection.
Mark
> On 15 Oct 2016, at 22:34, John Huss wrote:
>
> I would use a peer context.
>
> You can create a peer context like this if you runtime has been bound to
> the current thread using
> CayenneRuntime.bindThreadInjector(runtime.getInjector())
>
> public static ObjectContext newObjectContext() {
> ObjectContextFactory factory = CayenneRuntime.getThreadInjector
> ().getInstance(ObjectContextFactory.class);
> return (factory != null) ? factory.createContext() : null;
> }
>
>
>
> On Sat, Oct 15, 2016 at 2:37 PM Mark Wardle wrote:
>
>> Sorry… meant to say using Cayenne M4.0.M3…
>>
>> Mark
>>
>>> On 15 Oct 2016, at 20:29, Mark Wardle wrote:
>>>
>>> I’m using ResultBatchIterator like this:
>>>
>>> Consumer forEach = ...
>>> try (ResultBatchIterator iterator = query.batchIterator(context,
>> batchSize)) {
>>> for(List batch : iterator) {
>>> for (T c : batch) {
>>> forEach.accept(c);
>>> }
>>> }
>>> }
>>>
>>> but I also want to do some processing, inserting rows, editing objects
>> etc. I see lots of SQL in my logs, but the transaction doesn’t get
>> committed until the batch iterator completes, whether I use
>> context.commitChanges() or context.performGenericQuery() on some custom SQL
>> created using SQLTemplate.
>>>
>>> Is there a way of committing the transaction earlier, or should I use a
>> peer object context? Is there an easy of getting a new peer editing context
>> given a single object context or do I need to inject a ServerRuntime here?
>>>
>>> Thank you,
>>>
>>> Mark
>>>
>>>
>>
>>
Re: Batch iteration
I would use a peer context.
You can create a peer context like this if you runtime has been bound to
the current thread using
CayenneRuntime.bindThreadInjector(runtime.getInjector())
public static ObjectContext newObjectContext() {
ObjectContextFactory factory = CayenneRuntime.getThreadInjector
().getInstance(ObjectContextFactory.class);
return (factory != null) ? factory.createContext() : null;
}
On Sat, Oct 15, 2016 at 2:37 PM Mark Wardle wrote:
> Sorry… meant to say using Cayenne M4.0.M3…
>
> Mark
>
> > On 15 Oct 2016, at 20:29, Mark Wardle wrote:
> >
> > I’m using ResultBatchIterator like this:
> >
> > Consumer forEach = ...
> > try (ResultBatchIterator iterator = query.batchIterator(context,
> batchSize)) {
> > for(List batch : iterator) {
> > for (T c : batch) {
> > forEach.accept(c);
> > }
> > }
> > }
> >
> > but I also want to do some processing, inserting rows, editing objects
> etc. I see lots of SQL in my logs, but the transaction doesn’t get
> committed until the batch iterator completes, whether I use
> context.commitChanges() or context.performGenericQuery() on some custom SQL
> created using SQLTemplate.
> >
> > Is there a way of committing the transaction earlier, or should I use a
> peer object context? Is there an easy of getting a new peer editing context
> given a single object context or do I need to inject a ServerRuntime here?
> >
> > Thank you,
> >
> > Mark
> >
> >
>
>
Re: Batch iteration
Sorry… meant to say using Cayenne M4.0.M3…
Mark
> On 15 Oct 2016, at 20:29, Mark Wardle wrote:
>
> I’m using ResultBatchIterator like this:
>
> Consumer forEach = ...
> try (ResultBatchIterator iterator = query.batchIterator(context,
> batchSize)) {
> for(List batch : iterator) {
> for (T c : batch) {
> forEach.accept(c);
> }
> }
> }
>
> but I also want to do some processing, inserting rows, editing objects etc. I
> see lots of SQL in my logs, but the transaction doesn’t get committed until
> the batch iterator completes, whether I use context.commitChanges() or
> context.performGenericQuery() on some custom SQL created using SQLTemplate.
>
> Is there a way of committing the transaction earlier, or should I use a peer
> object context? Is there an easy of getting a new peer editing context given
> a single object context or do I need to inject a ServerRuntime here?
>
> Thank you,
>
> Mark
>
>
Batch iteration
I’m using ResultBatchIterator like this:
Consumer forEach = ...
try (ResultBatchIterator iterator = query.batchIterator(context, batchSize))
{
for(List batch : iterator) {
for (T c : batch) {
forEach.accept(c);
}
}
}
but I also want to do some processing, inserting rows, editing objects etc. I
see lots of SQL in my logs, but the transaction doesn’t get committed until the
batch iterator completes, whether I use context.commitChanges() or
context.performGenericQuery() on some custom SQL created using SQLTemplate.
Is there a way of committing the transaction earlier, or should I use a peer
object context? Is there an easy of getting a new peer editing context given a
single object context or do I need to inject a ServerRuntime here?
Thank you,
Mark
