Re: Cayenne web module deprecated

[John Huss]

The web module provides a very small amount of functionality that is pretty
easy to copy into your own application. I'd suggest looking at the code of
CayenneFilter

to see how it works.

With two servlet options now - javax and jakarta and the limitations
stemming from defining the runtime fully in the web.xml file, it was
decided that the web module would be better left as sample code for users
to implement in their own projects.



On Tue, Jan 31, 2023 at 1:06 AM Markus Reich  wrote:

> Hi,
>
> I saw that the web module is deprecated since 4.2
> I checked the DB First docu of 4.2. but there's the module still in use?
> What is the new way to use cayenne with J2EE apps?
>
> best regards
> Meex
>

Re: Which to pick: Cayenne 4.2 vs. 5.0?

[John Huss]

The main feature of 5.0 at this point is the removal of deprecated features
(like the ROP platform), so either one is a good choice I'd say. But I'd go
with 4.2.

On Mon, Nov 21, 2022 at 3:44 PM Hugi Thordarson  wrote:

> Hi guys!
> I'm about to be rid of a legacy DB that's been keeping my most important
> application stuck on a heavily modified Cayenne 4.1 stack. So, finally
> looking into joining modernity and upgrading Cayenne (yay!).
>
> I see there's a lot of stuff happening in 5.0 at the moment and I'd like
> to take my "non business-critical" stuff there so I can partake in the fun.
>
> But then there's the more serious (read: paying customer) software. Seeing
> as how v4.2 is still in the RC stage, with 5.0 receiving active
> development, would you recommend migrating "real apps" to 4.2 now—or just
> not bothering, holding out and moving directly to 5.0? Is anyone using the
> 4.2 RC in production yet?
>
> Cheers,
> - hugi

Re: How do I install and use Apache Cayenne?

[John Huss]

Start here and then please check back when you have more questions:
https://cayenne.apache.org/docs/4.1/getting-started-guide/


On Tue, Jul 26, 2022 at 9:52 AM Turritopsis Dohrnii Teo En Ming <
tdtemc...@gmail.com> wrote:

> Subject: How do I install and use Apache Cayenne?
>
> Good day from Singapore,
>
> How do I install and use Apache Cayenne?
>
> Thank you.
>
> Regards,
>
> Mr. Turritopsis Dohrnii Teo En Ming
> Targeted Individual in Singapore
> 26 July 2022 Tuesday
> Blogs:
> https://tdtemcerts.blogspot.com
> https://tdtemcerts.wordpress.com
>

Re: Apache cayenne 4.0.2 bug

[John Huss]

What version of Java are you using? There wasn't support for JDK 17 yet in
that version, so that could be the problem.

On Wed, Mar 23, 2022 at 12:25 PM Mustafa Said Tozluoğlu 
wrote:

> Hello,
>
> I just bought new macbook pro M1 chip. And I have to install apache
> cayenne 4.0.2 version becase my company use this version. But this version
> does not open in my mac. When I click the cayenne modeller app icon just
> appear in dock but after a couple seconds it is gone. Do you know what
> might be the problem or this version is available apple chip which is M1.
> Please help me! Thanks.
>
>
> Best Regards
>
>
> Mustafa Said Tozluoglu

Re: DI container has no binding for key

[John Huss]

You have to bind the DI injector to the thread (and unbind it later):

CayenneRuntime.*bindThreadInjector*(cayenneRuntime.getInjector());

If you are using servlets, then CayenneFilter will do this for you.
Otherwise you can bind it at the start of a request and unbind it at the
end.

On Wed, Mar 23, 2022 at 9:10 AM Stefan Stegic 
wrote:

> Hi Andrus,
>
> Of course:
>
> org.apache.cayenne.di.DIRuntimeException: DI container has no binding for
> key 
> at
>
> org.apache.cayenne.di.spi.DefaultInjector.getProvider(DefaultInjector.java:158)
> ~[?:?]
> at
>
> org.apache.cayenne.di.spi.DefaultInjector.getProvider(DefaultInjector.java:144)
> ~[?:?]
> at
>
> org.apache.cayenne.di.spi.DefaultInjector.getInstance(DefaultInjector.java:134)
> ~[?:?]
> at
>
> org.apache.cayenne.configuration.CayenneRuntime.newContext(CayenneRuntime.java:124)
> ~[?:?]
> at
>
> io.github.phuskus.firstspigotplugin.StatisticsController.reportPlayerConnectionEvent(StatisticsController.java:20)
> ~[?:?]
> at
>
> io.github.phuskus.firstspigotplugin.FirstSpigotPlugin.onEnable(FirstSpigotPlugin.java:29)
> ~[?:?]
> at
> org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:264)
> ~[spigot-api-1.18.1-R0.1-SNAPSHOT.jar:?]
> at
>
> org.bukkit.plugin.java.JavaPluginLoader.enablePlugin(JavaPluginLoader.java:342)
> ~[spigot-api-1.18.1-R0.1-SNAPSHOT.jar:?]
> at
>
> org.bukkit.plugin.SimplePluginManager.enablePlugin(SimplePluginManager.java:480)
> ~[spigot-api-1.18.1-R0.1-SNAPSHOT.jar:?]
> at
>
> org.bukkit.craftbukkit.v1_18_R1.CraftServer.enablePlugin(CraftServer.java:521)
> ~[spigot-1.18.1-R0.1-SNAPSHOT.jar:3443-Spigot-699290c-2c1e499]
> at
>
> org.bukkit.craftbukkit.v1_18_R1.CraftServer.enablePlugins(CraftServer.java:435)
> ~[spigot-1.18.1-R0.1-SNAPSHOT.jar:3443-Spigot-699290c-2c1e499]
> at
> net.minecraft.server.MinecraftServer.loadWorld0(MinecraftServer.java:612)
> ~[spigot-1.18.1-R0.1-SNAPSHOT.jar:3443-Spigot-699290c-2c1e499]
> at
> net.minecraft.server.MinecraftServer.loadLevel(MinecraftServer.java:414)
> ~[spigot-1.18.1-R0.1-SNAPSHOT.jar:3443-Spigot-699290c-2c1e499]
> at
> net.minecraft.server.dedicated.DedicatedServer.e(DedicatedServer.java:262)
> ~[spigot-1.18.1-R0.1-SNAPSHOT.jar:3443-Spigot-699290c-2c1e499]
> at net.minecraft.server.MinecraftServer.w(MinecraftServer.java:994)
> ~[spigot-1.18.1-R0.1-SNAPSHOT.jar:3443-Spigot-699290c-2c1e499]
> at
> net.minecraft.server.MinecraftServer.lambda$0(MinecraftServer.java:304)
> ~[spigot-1.18.1-R0.1-SNAPSHOT.jar:3443-Spigot-699290c-2c1e499]
> at java.lang.Thread.run(Thread.java:833) [?:?]
>
> On Wed, Mar 23, 2022 at 11:49 AM Andrus Adamchik 
> wrote:
>
> > Hi Stefan,
> >
> > Could you include a full stack trace please?
> >
> > Thanks,
> > Andrus
> >
> > > On Mar 23, 2022, at 1:04 AM, Stefan Stegic 
> > wrote:
> > >
> > > Hi,
> > >
> > >
> > > First some context: I'm working on a custom Minecraft server (Spigot)
> > > plugin. It's just a JAR that you export and place somewhere in the
> > server's
> > > folder structure.
> > >
> > > I wrote a simple insert query via Cayenne's API. It works when I run
> the
> > > example JAR directly via executing Main from the command line (java -cp
> > > myPlugin.jar path.to.Main), but it fails with the following error when
> > > executed by the server in my plugin's OnEnable lifecycle hook:
> > >
> > > org.apache.cayenne.di.DIRuntimeException: DI container has no binding
> for
> > > key 
> > >
> > >
> > > My initial theory was that it's being executed from a different thread,
> > so
> > > the DI system might not have access to that dependency for some reason,
> > but
> > > it seems like the server executes this lifecycle hook from the main
> > thread
> > > as well (though they've named the thread "Server thread", and I don't
> > know
> > > how to check if it's the main thread for sure). Not sure if this is a
> > > promising direction to pursue, but I got the hunch by looking at the
> > > threadInjector bits of Cayenne's DI container docs.
> > >
> > > My example looks like this:
> > >
> > > ServerRuntime cayenneRuntime = ServerRuntime.builder()
> > >.addConfig("cayenne-project.xml")
> > >.build();ObjectContext ctx =
> cayenneRuntime.newContext();
> > > PlayerConnectionEvent newEvent =
> > > ctx.newObject(PlayerConnectionEvent.class);
> > > newEvent.setEventType(eType);
> > > newEvent.setPlayerName(playerName);
> > > newEvent.setIpAddress(ipAddress);
> > > newEvent.setTimestampUnixSeconds(unixSeconds);
> > >
> > > ctx.commitChanges();
> > >
> > > Some help would be greatly appreciated, thanks in advance!
> > > --
> > > S poštovanjem,
> > > *Stefan Stegić*
> >
> >
>
> --
> S poštovanjem,
> *Stefan Stegić*
>

Re: Dealing with a commit exception....

[John Huss]

Actually most (or all?) DBs that cayenne supports will do a batch bind for
multiple inserts, which is quite a bit faster than issuing separate sql
statements.

On Thu, Jun 24, 2021 at 10:14 AM Jurgen Doll  wrote:

> I'm not sure about this, but I don't think Cayenne does SQL batch insert
> statements - it does each insert (even in a batch) as single SQL insert
> statements. If this is the case then there's no real advantage, I think,
> to doing batch commits with Cayenne in this particular case (or am I
> mistaken?).
>
> Cayenne context creations are purposefully cheap; so for each insert get
> a
> new context, create a new object in that context, and commit. Surround
> with the usual try block, inside your loop, to catch the unique clashes
> and continue with the next record 
>
> Just to be clear the select query of the data that's being processed is
> still in "batch" mode, like a 1000 as suggested.
>
>
> On Thu, 24 Jun 2021 16:35:46 +0200, Michael Gentry 
> wrote:
>
> > 10 is a rather small batch size. I'd suggest at least 100, perhaps even
> > 1000, unless you are expecting tons of problem records.
> >
> > For your hashing concerns, could you maybe store that in an H2 DB you use
> > for this migration?
> >
> > On Wed, Jun 23, 2021 at 12:54 PM Tony Giaccone 
> wrote:
> >
> >> Yeah, that's not a solution that's going to work. I need to move about a
> >> million records from one database to a second, and the delay associated
> >> with querying the database for every record would take a problem that's
> >> already going to take more than a day to complete and turn it into
> >> several
> >> days.  That's why I'm batching them into groups of 10 at a time.  The
> >> problem is there are duplicate records, they hash to the same value, and
> >> when that happens the insert on the other side is going to fail the
> >> Hash is
> >> a way to identify uniqueness.   I'm sure you're thinking, well keep a
> >> record of the hash values and only insert ones that you haven't seen.
> >> The
> >> problem with this solution is that I have to be able to restart the
> >> application and pick up where the process ended. If I had an in memory
> >> cache of the hash values, on restart I'd have to read all the previously
> >> transferred records and inflate the cache.  Another process that
> really
> >> is
> >> going to be too odious and time consuming.
> >>
> >>
> >>
> >> On Wed, Jun 23, 2021 at 12:45 PM John Huss  wrote:
> >>
> >> > I think it would be better to figure out the "problem" objects before
> >> > committing by querying the DB and the object context.
> >> >
> >> > On Wed, Jun 23, 2021 at 9:47 AM Tony Giaccone 
> >> wrote:
> >> >
> >> > > I have a list of 10 new objects that I've inserted into the
> >> objectcontex
> >> > > and am about to do a commit changes on the object context.
> >> > >
> >> > > One, or more, of those entries violates a constraint and causes the
> >> > commit
> >> > > changes to throw an exception.
> >> > >
> >> > > Now most of them are  probably ok, so I want to make sure they get
> >> > > inserted. How do I handle this?
> >> > >
> >> > > My first thought was to invalidate the 10 items.. then
> individually
> >> add
> >> > > each one back into the context and do a commit changes after each
> >> add.
> >> > Is
> >> > > that a reasonable path? Obviously the one that failed before will
> >> fail
> >> > > again, and then I can just log that, invalidate it again and keep
> >> going.
> >> > >
> >> > > Is there a better faster way to do this?
> >> > >
> >> > >
> >> > >
> >> > > Tony Giaccone
> >> > >
> >> >
>

Re: Dealing with a commit exception....

[John Huss]

I think it would be better to figure out the "problem" objects before
committing by querying the DB and the object context.

On Wed, Jun 23, 2021 at 9:47 AM Tony Giaccone  wrote:

> I have a list of 10 new objects that I've inserted into the objectcontex
> and am about to do a commit changes on the object context.
>
> One, or more, of those entries violates a constraint and causes the commit
> changes to throw an exception.
>
> Now most of them are  probably ok, so I want to make sure they get
> inserted. How do I handle this?
>
> My first thought was to invalidate the 10 items.. then individually add
> each one back into the context and do a commit changes after each add.  Is
> that a reasonable path? Obviously the one that failed before will fail
> again, and then I can just log that, invalidate it again and keep going.
>
> Is there a better faster way to do this?
>
>
>
> Tony Giaccone
>

Re: About relationship refresh.

[John Huss]

If you want related objects to be refreshed you have to use prefetches to
explicitly refresh them.

InvalidateObjects doesn't refresh related objects in the object graph,
only the objects you explicitly pass to it.

On Sun, May 9, 2021 at 1:14 PM yazmin georgina quintal
 wrote:

> Hi, We are using 4.0.B2 and found that object relationships are not
> refreshed. We have tried:
> 1. invalidateObjects(object): simple properties are refreshed from db, but
> relationShips remain untouched.2. RelationshipQuery: better, but only
> refresh entities already in the objectContext, does not requery db for new
> o deleted related entities.
> This issue araised now that we begin using more than one running session
> of out app, where other stations can add or delete related entities to a
> common parent.
> The workaround for now is not usuing object relationships but to requery
> related entities with manual ObjectSelect... this is not OO from the domain
> model's view point.
>
> Summary.
> objectContext.invalidateObjects(parent) does not invalidate parent's
> relationships ...  so parent.getRelatedEntities() does not have the current
> state.
>
> Does somebody has more info on this ?... does a more recent version
> invalidate full object graph ?.
> Thanks.
>
> Saludos.Atte. Ing. Juan Manuel Díaz LaraCel. 9932 293338
>
>  Atte. Yazmín Georgina Quintal Moo
> Local 63 y 64 de Plaza de la Tecnología. Villahermosa, Tab.
> Tel 993 1311688
> Cel. 9932 493007
>

Re: Heisenbug-ish NullPointerExceptions in DataContext.flushToParent()

[John Huss]

On Fri, May 7, 2021 at 8:18 AM Hugi Thordarson  wrote:

> Well I'll be… Over two decades of full time Java and I somehow didn't know
> about that stack trace omission. Yikes. Thanks!
>
> Now, looking at the first occurrence of the exception after app startup, I
> see the NPE is happening in @PreUpdate on a listener registered to my
> DataDomain. The offending code looks a bit like this:
>
> @PreUpdate({ BaseDataObject.class })
> public void handleUpdate( final BaseDataObject dataObject ) {
> final DataContext dc = (DataContext)dataObject.getObjectContext();
> final ObjectId objectId = dataObject.getObjectId();
> final ObjEntity entity = dc.getEntityResolver().getObjEntity(
> objectId.getEntityName() );
> final DataRow snapshot = dc.getObjectStore().getSnapshot( objectId
> );
>
> for( final ObjAttribute objAttribute : entity.getAttributes() ) {
> final Object originalValue = snapshot.get(
> objAttribute.getDbAttributeName() );
> [ ... irrelevant rest of code omitted ... ]
> }
> }
>
> …the NPE gets thrown by the first line in the for-loop (…snapshot.get(
> objAttribute.getDbAttributeName() )).
>
> So it seems either 'snapshot' or 'objAttribute' is null, probably the
> snapshot, right? Any ideas why that might be?
>

Yeah, it would be the snapshot is null. The snapshot cache is limited in
size, so it may have purged that entry if you have a busy server. You could
try increasing the size of the snapshot cache:

org.apache.cayenne.configuration.Constants.SNAPSHOT_CACHE_SIZE_PROPERTY




>
> - hugi
>
>
>
> > On 7 May 2021, at 12:53, John Huss  wrote:
> >
> > Or if you have older logs, you can look back farther to find one without
> > the cause omitted.
> >
> > On Fri, May 7, 2021 at 7:52 AM John Huss  wrote:
> >
> >> It will omit the trace if it has happened a very large number of times.
> If
> >> you restart the server it will start printing it again when it happens.
> >>
> >> On Fri, May 7, 2021 at 7:26 AM Hugi Thordarson 
> wrote:
> >>
> >>> One would think so, but there's just no stack trace printed for the
> >>> NullPointerException.
> >>>
> >>>
> >>>> On 7 May 2021, at 12:24, John Huss  wrote:
> >>>>
> >>>> The first line of the stack after the Caused By line would tell you
> the
> >>>> line with the problem, wouldn't it? What was that line?
> >>>>
> >>>> On Fri, May 7, 2021 at 7:16 AM Hugi Thordarson 
> >>> wrote:
> >>>>
> >>>>> Hi all,
> >>>>>
> >>>>> occasionally my users encounter errors when committing changes to the
> >>> DB.
> >>>>> The relevant part of the stack trace looks like this:
> >>>>>
> >>>>> -
> >>>>> org.apache.cayenne.CayenneRuntimeException: [v.4.1 Jul 14 2020
> >>> 10:26:08]
> >>>>> Commit Exception
> >>>>>   at
> >>>>>
> >>>
> org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:774)
> >>>>>   at
> >>>>>
> >>>
> org.apache.cayenne.access.DataContext.commitChanges(DataContext.java:691)
> >>>>>   [...removed rest of trace…]
> >>>>> Caused by: java.lang.NullPointerException
> >>>>> -
> >>>>>
> >>>>> Unfortunately I've been unable to reproduce this error, so before I
> >>> dive
> >>>>> into it I'm wondering if this looks familiar to anyone, if there are
> >>>>> obvious places to look or if there are recommendations for strategies
> >>> to
> >>>>> debug?
> >>>>>
> >>>>> Cheers,
> >>>>> - hugi
> >>>
> >>>
>
>

Re: Heisenbug-ish NullPointerExceptions in DataContext.flushToParent()

[John Huss]

On Fri, May 7, 2021 at 8:18 AM Hugi Thordarson  wrote:

> Well I'll be… Over two decades of full time Java and I somehow didn't know
> about that stack trace omission. Yikes. Thanks!


You’ve just never caused an error that bad before right? 


>
> Now, looking at the first occurrence of the exception after app startup, I
> see the NPE is happening in @PreUpdate on a listener registered to my
> DataDomain. The offending code looks a bit like this:
>
> @PreUpdate({ BaseDataObject.class })
> public void handleUpdate( final BaseDataObject dataObject ) {
> final DataContext dc = (DataContext)dataObject.getObjectContext();
> final ObjectId objectId = dataObject.getObjectId();
> final ObjEntity entity = dc.getEntityResolver().getObjEntity(
> objectId.getEntityName() );
> final DataRow snapshot = dc.getObjectStore().getSnapshot( objectId
> );
>
> for( final ObjAttribute objAttribute : entity.getAttributes() ) {
> final Object originalValue = snapshot.get(
> objAttribute.getDbAttributeName() );
> [ ... irrelevant rest of code omitted ... ]
> }
> }
>
> …the NPE gets thrown by the first line in the for-loop (…snapshot.get(
> objAttribute.getDbAttributeName() )).
>
> So it seems either 'snapshot' or 'objAttribute' is null, probably the
> snapshot, right? Any ideas why that might be?
>
> - hugi
>
>
>
> > On 7 May 2021, at 12:53, John Huss  wrote:
> >
> > Or if you have older logs, you can look back farther to find one without
> > the cause omitted.
> >
> > On Fri, May 7, 2021 at 7:52 AM John Huss  wrote:
> >
> >> It will omit the trace if it has happened a very large number of times.
> If
> >> you restart the server it will start printing it again when it happens.
> >>
> >> On Fri, May 7, 2021 at 7:26 AM Hugi Thordarson 
> wrote:
> >>
> >>> One would think so, but there's just no stack trace printed for the
> >>> NullPointerException.
> >>>
> >>>
> >>>> On 7 May 2021, at 12:24, John Huss  wrote:
> >>>>
> >>>> The first line of the stack after the Caused By line would tell you
> the
> >>>> line with the problem, wouldn't it? What was that line?
> >>>>
> >>>> On Fri, May 7, 2021 at 7:16 AM Hugi Thordarson 
> >>> wrote:
> >>>>
> >>>>> Hi all,
> >>>>>
> >>>>> occasionally my users encounter errors when committing changes to the
> >>> DB.
> >>>>> The relevant part of the stack trace looks like this:
> >>>>>
> >>>>> -
> >>>>> org.apache.cayenne.CayenneRuntimeException: [v.4.1 Jul 14 2020
> >>> 10:26:08]
> >>>>> Commit Exception
> >>>>>   at
> >>>>>
> >>>
> org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:774)
> >>>>>   at
> >>>>>
> >>>
> org.apache.cayenne.access.DataContext.commitChanges(DataContext.java:691)
> >>>>>   [...removed rest of trace…]
> >>>>> Caused by: java.lang.NullPointerException
> >>>>> -
> >>>>>
> >>>>> Unfortunately I've been unable to reproduce this error, so before I
> >>> dive
> >>>>> into it I'm wondering if this looks familiar to anyone, if there are
> >>>>> obvious places to look or if there are recommendations for strategies
> >>> to
> >>>>> debug?
> >>>>>
> >>>>> Cheers,
> >>>>> - hugi
> >>>
> >>>
>
>

Re: Heisenbug-ish NullPointerExceptions in DataContext.flushToParent()

[John Huss]

Or if you have older logs, you can look back farther to find one without
the cause omitted.

On Fri, May 7, 2021 at 7:52 AM John Huss  wrote:

> It will omit the trace if it has happened a very large number of times. If
> you restart the server it will start printing it again when it happens.
>
> On Fri, May 7, 2021 at 7:26 AM Hugi Thordarson  wrote:
>
>> One would think so, but there's just no stack trace printed for the
>> NullPointerException.
>>
>>
>> > On 7 May 2021, at 12:24, John Huss  wrote:
>> >
>> > The first line of the stack after the Caused By line would tell you the
>> > line with the problem, wouldn't it? What was that line?
>> >
>> > On Fri, May 7, 2021 at 7:16 AM Hugi Thordarson 
>> wrote:
>> >
>> >> Hi all,
>> >>
>> >> occasionally my users encounter errors when committing changes to the
>> DB.
>> >> The relevant part of the stack trace looks like this:
>> >>
>> >> -
>> >> org.apache.cayenne.CayenneRuntimeException: [v.4.1 Jul 14 2020
>> 10:26:08]
>> >> Commit Exception
>> >>at
>> >>
>> org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:774)
>> >>at
>> >>
>> org.apache.cayenne.access.DataContext.commitChanges(DataContext.java:691)
>> >>[...removed rest of trace…]
>> >> Caused by: java.lang.NullPointerException
>> >> -
>> >>
>> >> Unfortunately I've been unable to reproduce this error, so before I
>> dive
>> >> into it I'm wondering if this looks familiar to anyone, if there are
>> >> obvious places to look or if there are recommendations for strategies
>> to
>> >> debug?
>> >>
>> >> Cheers,
>> >> - hugi
>>
>>

Re: Heisenbug-ish NullPointerExceptions in DataContext.flushToParent()

[John Huss]

It will omit the trace if it has happened a very large number of times. If
you restart the server it will start printing it again when it happens.

On Fri, May 7, 2021 at 7:26 AM Hugi Thordarson  wrote:

> One would think so, but there's just no stack trace printed for the
> NullPointerException.
>
>
> > On 7 May 2021, at 12:24, John Huss  wrote:
> >
> > The first line of the stack after the Caused By line would tell you the
> > line with the problem, wouldn't it? What was that line?
> >
> > On Fri, May 7, 2021 at 7:16 AM Hugi Thordarson  wrote:
> >
> >> Hi all,
> >>
> >> occasionally my users encounter errors when committing changes to the
> DB.
> >> The relevant part of the stack trace looks like this:
> >>
> >> -
> >> org.apache.cayenne.CayenneRuntimeException: [v.4.1 Jul 14 2020 10:26:08]
> >> Commit Exception
> >>at
> >>
> org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:774)
> >>at
> >>
> org.apache.cayenne.access.DataContext.commitChanges(DataContext.java:691)
> >>[...removed rest of trace…]
> >> Caused by: java.lang.NullPointerException
> >> -
> >>
> >> Unfortunately I've been unable to reproduce this error, so before I dive
> >> into it I'm wondering if this looks familiar to anyone, if there are
> >> obvious places to look or if there are recommendations for strategies to
> >> debug?
> >>
> >> Cheers,
> >> - hugi
>
>

Re: Heisenbug-ish NullPointerExceptions in DataContext.flushToParent()

[John Huss]

The first line of the stack after the Caused By line would tell you the
line with the problem, wouldn't it? What was that line?

On Fri, May 7, 2021 at 7:16 AM Hugi Thordarson  wrote:

> Hi all,
>
> occasionally my users encounter errors when committing changes to the DB.
> The relevant part of the stack trace looks like this:
>
> -
> org.apache.cayenne.CayenneRuntimeException: [v.4.1 Jul 14 2020 10:26:08]
> Commit Exception
> at
> org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:774)
> at
> org.apache.cayenne.access.DataContext.commitChanges(DataContext.java:691)
> [...removed rest of trace…]
> Caused by: java.lang.NullPointerException
> -
>
> Unfortunately I've been unable to reproduce this error, so before I dive
> into it I'm wondering if this looks familiar to anyone, if there are
> obvious places to look or if there are recommendations for strategies to
> debug?
>
> Cheers,
> - hugi

Re: Configuring multiple DataNodes

[John Huss]

No, it supports System Properties passed to the Java process with
-Dproperty.name=value but not reading from environment variables directly.

On Fri, Apr 23, 2021 at 12:42 PM Tony Giaccone  wrote:

> Andrus,
>
> with these properties, you set an environment variable like this:
>
> $ export cayenne.jdbc.driver.eps.oracleDb=oracle.jdbc.OracleDriver
>
> and that sets the driver on that sdt of domain and node name?
>
>
> Tony
>
> On Fri, Apr 23, 2021 at 11:52 AM Andrus Adamchik 
> wrote:
>
> > also per-node connection params can be passed as properties:
> >
> >
> >
> https://cayenne.apache.org/docs/4.1/cayenne-guide/#appendix-a-configuration-properties
> > <
> >
> https://cayenne.apache.org/docs/4.1/cayenne-guide/#appendix-a-configuration-properties
> > >
> >
> >
> >
> > > On Apr 23, 2021, at 10:58 AM, John Huss  wrote:
> > >
> > > When you create your ServerRuntime you need to supply a custom DI
> module
> > to
> > > pass the DB connection parameters, something like this:
> > >
> > > *import* org.apache.cayenne.configuration.Constants;
> > >
> > > *import* org.apache.cayenne.di.Binder;
> > >
> > > *import* org.apache.cayenne.di.MapBuilder;
> > >
> > > *import* org.apache.cayenne.di.Module;
> > >
> > >
> > > *public* *class* MyModule *implements* Module {
> > >
> > > *public* *void* configure(Binder binder) {
> > >
> > > MapBuilder mapBuilder = binder.bindMap(String.*class*,
> Constants.
> > > *PROPERTIES_MAP*);
> > >
> > > mapBuilder.put(Constants.*JDBC_URL_PROPERTY* +
> > > ".YourDataDomainName.YourDataNodeName", System.*getenv*("url"));
> > >
> > > ...
> > >
> > > }
> > >
> > > }
> > >
> > > On Fri, Apr 23, 2021 at 9:32 AM Tony Giaccone 
> wrote:
> > >
> > >> I have a datadomain with two datamaps, and two datasource each talking
> > to a
> > >> different database.
> > >>
> > >> Right now, I have had the datasource configured in the config file,
> but
> > I
> > >> need to remove that and set it programmatically, from an environment
> > >> variable.  Simple enough to do with a defauldata source, unclear how
> to
> > do
> > >> when you have multiple data sources and data nodes.
> > >>
> > >> After browsing through the code and the app as it's running, it's
> > unclear
> > >> to me how to do this.
> > >>
> > >> Does anyone know the secret sauce?
> > >>
> > >>
> > >>
> > >> Tony
> > >>
> >
> >
>

Re: Configuring multiple DataNodes

[John Huss]

When you create your ServerRuntime you need to supply a custom DI module to
pass the DB connection parameters, something like this:

*import* org.apache.cayenne.configuration.Constants;

*import* org.apache.cayenne.di.Binder;

*import* org.apache.cayenne.di.MapBuilder;

*import* org.apache.cayenne.di.Module;


*public* *class* MyModule *implements* Module {

*public* *void* configure(Binder binder) {

MapBuilder mapBuilder = binder.bindMap(String.*class*, Constants.
*PROPERTIES_MAP*);

mapBuilder.put(Constants.*JDBC_URL_PROPERTY* +
".YourDataDomainName.YourDataNodeName", System.*getenv*("url"));

...

}

}

On Fri, Apr 23, 2021 at 9:32 AM Tony Giaccone  wrote:

> I have a datadomain with two datamaps, and two datasource each talking to a
> different database.
>
> Right now, I have had the datasource configured in the config file, but I
> need to remove that and set it programmatically, from an environment
> variable.  Simple enough to do with a defauldata source, unclear how to do
> when you have multiple data sources and data nodes.
>
> After browsing through the code and the app as it's running, it's unclear
> to me how to do this.
>
> Does anyone know the secret sauce?
>
>
>
> Tony
>

Re: PK Generation and FrontBase

[John Huss]

I don't think FrontBase is very commonly used. I would recommend submitting
a minimal example of the problem and someone should be able to fix the
issue then.

On Thu, Apr 15, 2021 at 3:44 PM Jérémy DE ROYER 
wrote:

> Hello,
>
> I switched my model from FrontBase to PostgreSQL and it works great.
>
> Does someone use Cayenne 4.1 with FrontbaseSQL successfully ?
>
> Is there something specific to know ?
>
> Thank’s for any tip,
>
> Jérémy
>
> > Le 14 avr. 2021 à 23:05, Jérémy DE ROYER 
> a écrit :
> >
> > Hi all,
> >
> > I’m trying to find a solution for the (temporary ?) lack of horizontal
> inheritance so I did model vertical inheritance following :
> >
> > https://cayenne.apache.org/docs/3.0/modeling-vertical-inheritance.html
> >
> > using a Frontbase database.
> >
> > I did :
> > - create the model,
> > - generate the database,
> > - fix lack of jars and other things like username, password, jdbc driver
> >
> > The demo code below starts well and connects to database...
> >
> > try {
> > ServerRuntime cayenneRuntime = ServerRuntime.builder()
> > .addConfig("cayenne-CayenneTest.xml")
> > .build();
> >
> > ObjectContext context = cayenneRuntime.newContext();
> >
> > Book _newBook = context.newObject(Book.class);
> > _newBook.setTitle("Nouveau Book");
> > _newBook.setCreated(new Date());
> >
> > context.commitChanges();
> >
> > EBook _newEBook = context.newObject(EBook.class);
> > _newEBook.setTitle("Nouveau EBook");
> > _newEBook.setCreated(new Date());
> > _newEBook.setDownloadUrl("https://www.google.fr/;);
> >
> > context.commitChanges();
> >
> > PaperBook _newPaperBook = context.newObject(PaperBook.class);
> > _newPaperBook.setTitle("Nouveau PaperBook");
> > _newPaperBook.setCreated(new Date());
> > _newPaperBook.setShippingWeight(1000);
> >
> > context.commitChanges();
> >
> > List _books = ObjectSelect.query(Book.class).select(context);
> >
> > System.out.println("** ALL **");
> >
> > for (Book currentBook : _books)
> > {
> > System.out.println(currentBook.getTitle() + " created on " +
> currentBook.getCreated());
> > }
> > }
> > catch (Exception e) {
> > System.out.println("An error occured : " + e.getMessage());
> > e.printStackTrace();
> > }
> >
> > …but throws this exception
> >
> > An error occured : [v.4.1 Jul 14 2020 10:40:45] Commit Exception
> > org.apache.cayenne.CayenneRuntimeException: [v.4.1 Jul 14 2020 10:40:45]
> Commit Exception
> > at
> org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:774)
> > at
> org.apache.cayenne.access.DataContext.commitChanges(DataContext.java:691)
> > at your.app.Application.(Application.java:50)
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> > at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> > at java.lang.Class.newInstance(Class.java:442)
> > at com.webobjects.appserver.WOApplication.main(WOApplication.java:547)
> > at your.app.Application.main(Application.java:25)
> > Caused by: java.lang.NullPointerException
> > at
> org.apache.cayenne.access.jdbc.SQLTemplateAction.execute(SQLTemplateAction.java:242)
> > at
> org.apache.cayenne.access.jdbc.SQLTemplateAction.runWithNamedParametersBatch(SQLTemplateAction.java:179)
> > at
> org.apache.cayenne.access.jdbc.SQLTemplateAction.performAction(SQLTemplateAction.java:111)
> > at
> org.apache.cayenne.access.DataNodeQueryAction.runQuery(DataNodeQueryAction.java:97)
> > at org.apache.cayenne.access.DataNode.performQueries(DataNode.java:273)
> > at
> org.apache.cayenne.dba.frontbase.FrontBasePkGenerator.longPkFromDatabase(FrontBasePkGenerator.java:143)
> > at
> org.apache.cayenne.dba.JdbcPkGenerator.generatePk(JdbcPkGenerator.java:220)
> > at
> org.apache.cayenne.access.DataDomainInsertBucket.createPermIds(DataDomainInsertBucket.java:168)
> > at
> org.apache.cayenne.access.DataDomainInsertBucket.appendQueriesInternal(DataDomainInsertBucket.java:76)
> > at
> org.apache.cayenne.access.DataDomainSyncBucket.appendQueries(DataDomainSyncBucket.java:78)
> > at
> org.apache.cayenne.access.DataDomainFlushAction.preprocess(DataDomainFlushAction.java:185)
> > at
> org.apache.cayenne.access.DataDomainFlushAction.flush(DataDomainFlushAction.java:143)
> > at org.apache.cayenne.access.DataDomain.onSyncFlush(DataDomain.java:624)
> > at
> org.apache.cayenne.access.DataDomain.onSyncNoFilters(DataDomain.java:594)
> > at
> org.apache.cayenne.access.DataDomain$DataDomainSyncFilterChain.onSync(DataDomain.java:822)
> > at
> org.apache.cayenne.tx.TransactionFilter.lambda$onSync$0(TransactionFilter.java:61)
> > at
> org.apache.cayenne.tx.DefaultTransactionManager$BaseTransactionHandler.performInTransaction(DefaultTransactionManager.java:183)
> > at
> 

Re: Executing saved SQL

[John Huss]

If you mean it takes a generic type parameter, then you would probably want
to use DataRow if it is a raw SQL query mapped in the modeler.

On Fri, Apr 9, 2021 at 11:35 AM Tony Giaccone  wrote:

> No, it's not a column  in a table, it's a composite value it doesn't belong
> to any object.
>
> My current guess is this:
>
> QueryResult result =  MappedExec.*query*("calcClob").execute(context);
>
>
> but java wants a parameter type for the QueryResult.  And I'm not sure that
> the MappedExec is the right choice either.
>
> MappedSelect says it can return a SQLTemplate, which is what this is. But
> it seems to need a mapped object which I don't have in this case.
>
>
>
> Tony
>
> On Fri, Apr 9, 2021 at 12:21 PM John Huss  wrote:
>
> > Perhaps this?
> >
> > Selecting individual columns:
> >
> >
> https://cayenne.apache.org/docs/4.1/cayenne-guide/#selecting-individual-columns
> >
> > On Fri, Apr 9, 2021 at 11:03 AM Tony Giaccone  wrote:
> >
> > > So I have a query that does a complicated select, and returns a list
> > > containing a single clob for each row.
> > >
> > > What I'm not sure about is when I execute that query, How to handle the
> > > result.
> > >
> > > How do I explain to Cayenne that the result is not a mapped object but
> > just
> > > a list of clobs?
> > >
> > >
> > > Tony
> > >
> >
>

Re: Executing saved SQL

[John Huss]

Perhaps this?

Selecting individual columns:
https://cayenne.apache.org/docs/4.1/cayenne-guide/#selecting-individual-columns

On Fri, Apr 9, 2021 at 11:03 AM Tony Giaccone  wrote:

> So I have a query that does a complicated select, and returns a list
> containing a single clob for each row.
>
> What I'm not sure about is when I execute that query, How to handle the
> result.
>
> How do I explain to Cayenne that the result is not a mapped object but just
> a list of clobs?
>
>
> Tony
>

Re: Managing data from two databases.

[John Huss]

Yep, it’s that easy. Just assign each DataMap to the correct DataNode.

On Thu, Apr 8, 2021 at 5:50 PM Tony Giaccone  wrote:

> So I have a model, it has two datanodes and two datamaps. Data map/node
> one, represents an oracledb, datamap/node two represents a postgres DB.  I
> want to read from Oracle, and write to the Postgres.  The data entities in
> each database are unique. So MyReadClass lives in dataMapOracle and
> MyNewData lives in Postgres.
>
> Do I just spin up the runtime, and fetch from one and create the other and
> the write stuff ends up going to the correct database? Is it really
> just that easy? Or am I missing something?
>
>
> Tony
>

Re: use Date values in ExpressionFactory.exp

[John Huss]

Isn’t there a version of .exp that takes args? You can pass a date object
directly there without worrying about formatting.

On Sat, Jan 2, 2021 at 7:49 AM Andrea Biasillo  wrote:

> Hi Mike!
>
>
>
> I have tried with:
>
>
>
> Expression expDate = ExpressionFactory.exp("modifiedDate>
> TO_DATE('2003-07-17','-mm-dd')");
>
>
>
> but I get this exception:
>
>
>
> Caused by: org.apache.cayenne.exp.ExpressionException: [v.4.2.M1 Nov 26
> 2020 09:20:26] Encountered " "(" "( "" at line 1, column 22.
>
> Was expecting one of:
>
> 
>
> "or" ...
>
> "and" ...
>
> "|" ...
>
> "^" ...
>
> "&" ...
>
> "<<" ...
>
> ">>" ...
>
> "+" ...
>
> "-" ...
>
> "/" ...
>
> "*" ...
>
>
>
> and I really would want to have all my expression in one string, without
> use greaterExp.
>
> Like this:
>
>
>
> Expression exp = ExpressionFactory.exp("businessPartnerName='andrea' and
> businessPartnerCode!='rossi' or ((businessPartnerCode!='bianchi' and
> maxAdvancePaymentPercent>9) or businessPartnerCode!='pippo' or
> (maxAdvancePaymentPercent>100 or maxAdvancePaymentPercent<10)) and
> modifiedDate> TO_DATE('2003-07-17','-mm-dd')");
>
>
>
> But maybe it is not possible.
>
>
>
> Andrea
>
>
>
> On 2021/01/02 12:45:49, Mike Kienenberger  wrote:
>
> > Oracle expects SQL like the following to compare dates.
>
> >
>
> >MODIFIED_DATE > TO_DATE('2003-07-17','-mm-dd')
>
> >
>
> > You either need to use that in your exp() method, or use greaterExp(),
>
> > which can automatically generate the needed sql for the current database.
>
> >
>
> > Date date = new GregorianCalendar(2003, Calendar.JULY,
>
> > 17).getTime() ;
>
> > Expression matchDateGreaterThanDate1 =
>
> > ExpressionFactory.greaterExp("MODIFIED_DATE", date);
>
> >
>
> >
>
> >
>
> >
>
> > On Sat, Jan 2, 2021 at 3:45 AM Andrea Biasillo  wrote:
>
> >
>
> > > Hi!
>
> > >
>
> > > Is it possible to use Date values in ExpressionFactory.exp?
>
> > >
>
> > > This works very well:
>
> > >
>
> > > Expression exp = ExpressionFactory.exp("businessPartnerName='andrea'
> and
>
> > > businessPartnerCode!='rossi' or ((businessPartnerCode!='bianchi' and
>
> > > maxAdvancePaymentPercent>9) or businessPartnerCode!='pippo' or
>
> > > (maxAdvancePaymentPercent>100 or maxAdvancePaymentPercent<10))");
>
> > >
>
> > > but if I use a Date value like this:
>
> > >
>
> > > Expression expDate = ExpressionFactory.exp("modifiedDate>
> '2003-07-17'");
>
> > >
>
> > > The database, Oracle in my case, will complain:
>
> > >
>
> > > Literal does not match format string
>
> > > (the database expects something like this:  MODIFIED_DATE > DATE
>
> > > '2003-07-17')
>
> > >
>
> > > Any input?
>
> > >
>
> > > Andrea
>
> > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> > >
>
> >
>
>

Re: ObjectSelect to select uncommitted objects

[John Huss]

There isn't anything built-in. I use this method in my own DataContext
subclass to do it. Use at your own risk, etc.

*public*  List
selectIncludingPendingChanges(ObjectSelect query) {

Set newAndModified = *new* HashSet<>(newObjects());

newAndModified.addAll(modifiedObjects());

Set inMemoryResults = newAndModified.stream()

.filter(o -> o.getClass().equals(query.getEntityType()))

.filter(o -> query.getWhere().match(o))

.map(o -> (T)o)

.collect(Collectors.*toSet*());

List dbResults = select(query);

dbResults.removeAll(newAndModified); // these have already been checked

dbResults.removeAll(deletedObjects());

*if* (inMemoryResults.isEmpty()) {

*return* dbResults; // performance optimization

}

inMemoryResults.addAll(dbResults); // combines and removes duplicates

List results = *new* ArrayList<>(inMemoryResults);

*if* (query.getOrderings() != *null* && !query.getOrderings().isEmpty()) {

Ordering.*orderList*(results, *new* ArrayList<>(query.getOrderings()));

}

*if* (query.getLimit() > 0) {

results = results.subList(0, query.getLimit());

}

*return* (List) results;

}


*public* *static* *class* Factory *extends* DataContextFactory {

@Override

*protected* MyDataContext newInstance(DataChannel parent, ObjectStore
objectStore) {

*return* *new* MyDataContext(parent, objectStore);

}

}


*public* *static* *final* Module *MODULE* = binder ->

binder.bind(ObjectContextFactory.*class*).to(MyDataContext.Factory.*class*);



On Sat, Nov 14, 2020 at 10:20 AM giulio.ces...@gmail.com <
giulio.ces...@gmail.com> wrote:

> Hello,
>
> I have multiple places where I have to search data either on the current
> ObjectStore (newly inserted or modified objects) or directly in the DB.
>
> At the moment I have two distinct blocks of code, one expressed as a
> Predicate (used inside a stream filter) and the other with an ObjectSelect
> (on which I call its select method).
>
> As both structure encode the same "logic" criteria, it would be nice to be
> able to derive one from the other.
>
> One way to solve this would be to be able to transform on ObjectSelect
> instance into a Predicate instance.
> Or, the ability to run an ObjectSelect not only on the DB, but also into
> the uncommitted objects in the ObjectContext.
>
> Is there anything I am missing that could help me dealing with this need?
>
> Cheers,
>
> Giulio Cesare
>

Re: Bug in DataRowsUtils

[John Huss]

I also dislike this behavior, but I think it is intentional.

As I workaround I lowercase all the DbAttribute names in my DataMap at
startup and make sure all my MappedSelects use lowercase names.

On Wed, Nov 11, 2020 at 3:38 AM Mark Stobbe  wrote:

> Hi all,
>
>
>
> I was using MappedSelect to query for objects and I was surprised to see I
>
> was returned a HOLLOW object. After some investigation I found that
>
> in DataRowUtils.refreshObjectWithSnapshot there is a check if we
>
> fetched the whole object using a map.
>
>
>
> However, the check ("snapshot.containsKey(dbAttrPath)") does not take into
>
> account the casing difference between database and datamap. This does work
>
> in the rest of Cayenne, but not here.
>
>
>
> Mark
>
>

Re: about mysql support

[John Huss]

Yes, all common DBs that have a JDBC driver are supported.

On Wed, Nov 11, 2020 at 3:11 AM 郑岩  wrote:

> Hello , I want to know if the Cayenne4 can support mysql8.0 Database？
>
>
>
>
>
>

Re: Query timeout in QueryMetadata

[John Huss]

The answer might vary depending on your specific use case. One way to do it
would be to:

1) start an explicit transaction,
2) run a query to set the timeout
SQLExec.*query*("set statement_timeout = 6").execute(context
);
3) do your work
4) run a query to reset or unset the timeout
5) commit

TransactionFactory txFactory = CayenneRuntime.*getThreadInjector*
().getInstance(TransactionFactory.*class*);

Transaction tx = txFactory.createTransaction();

tx.begin();

*try* {

... // do stuff


tx.commit();

} *catch* (Exception e) {

tx.rollback();

*throw* e;

}


On Tue, Nov 10, 2020 at 7:50 AM Mark Stobbe  wrote:

> Hi all,
>
> Can we have an option to set query timeout on the metadata of the query,
> similar to the statementFetchSize?
>
> The workaround seems to involve a lot of subclasses via the adapter; or via
> the connection but then I need to have the same timeout for all statements
> or parse SQL to identify query..
>
> Best regards,
> Mark
>

Re: Spinning up Cayenne in a unit test.

[John Huss]

Looks like it doesn't have an adaptor, so you should try specifying one
(this is line 103):

*boolean* supportsGeneratedKeys = node
.getAdapter().supportsGeneratedKeys();


On Thu, Oct 29, 2020 at 9:51 AM Tony Giaccone  wrote:

> Andrus, we're using Bootique, but this portion of the code is in it's own
> jar file, it's just the model and some basic services on the objects.
> Because it's included in a bootique app, we can't use Bootique to do the
> testing here. It needs to be just Cayenne. So any ideas about what might be
> going wrong here?
>
>
> Tony
>
> On Thu, Oct 29, 2020 at 1:56 AM Andrus Adamchik 
> wrote:
>
> > And while we are at it, I absolutely must plug a mention of Bootique 2.0
> /
> > JUnit 5 testing facilities :)
> >
> > https://bootique.io/docs/2.x/bootique-docs/#_testing
> > https://bootique.io/docs/2.x/bootique-jdbc-docs/#jdbc-testing
> >
> > They are on a totally different level:
> >
> > * Support for Docker/Testcontainers, so you'd test against a real DB, not
> > Derby or H2
> > * Liquibase migrations in tests, so you can use a real up-to-date schema
> > * "Global" scope for test databases and "apps", so you can share them
> > across test classes, improving performance
> > * Improved "Table" API for datatsets preparation and data assertions
> >
> > Andrus
> >
> >
> > > On Oct 29, 2020, at 8:48 AM, Andrus Adamchik 
> > wrote:
> > >
> > > You can still start a BQRuntime in tests, even if the module doesn't
> use
> > Bootique. Then you'd define a DataSource within BQRuntime, and
> > "bootique-cayenne" would create a DataNode for it automatically and
> > transparently. YMMV with older Bootique versions. E.g. Agrest (that has
> no
> > Bootique dependency itself) is using Bootique for tests.
> > >
> > > Andrus
> > >
> > >> On Oct 29, 2020, at 7:27 AM, Tony Giaccone  wrote:
> > >>
> > >> This is what I have now for the spin up:
> > >>
> > >> runtime = ServerRuntime.builder()
> > >>   .addConfig("cayenne-PriceIncrease.xml")
> > >>   .addModule(binder -> {
> > >>   ServerModule.contributeProperties(binder)
> > >>   .put("cayenne.jdbc.driver", "org.h2.Driver")
> > >>   .put("cayenne.jdbc.url", "jdbc:h2:mem:CohortDB");
> > >>   })
> > >>   .build();
> > >> DataNode node = new DataNode("testNode");
> > >> runtime.getDataDomain().setDefaultNode(node);
> > >>
> > >>
> > >> And now on commit changes I get this:
> > >>
> > >>
> > >> Caused by: java.lang.NullPointerException
> > >> at
> > >>
> >
> org.apache.cayenne.access.DataDomainInsertBucket.createPermIds(DataDomainInsertBucket.java:103)
> > >> at
> > >>
> >
> org.apache.cayenne.access.DataDomainInsertBucket.appendQueriesInternal(DataDomainInsertBucket.java:73)
> > >> at
> > >>
> >
> org.apache.cayenne.access.DataDomainSyncBucket.appendQueries(DataDomainSyncBucket.java:78)
> > >> at
> > >>
> >
> org.apache.cayenne.access.DataDomainFlushAction.preprocess(DataDomainFlushAction.java:185)
> > >> at
> > >>
> >
> org.apache.cayenne.access.DataDomainFlushAction.flush(DataDomainFlushAction.java:143)
> > >>
> > >>
> > >>
> > >> On Wed, Oct 28, 2020 at 4:08 PM John Huss 
> wrote:
> > >>
> > >>> You could add a node at runtime:
> > >>>
> > >>> DataNode node = *new* DataNode("node");
> > >>>
> > >>> ... // configure node
> > >>>
> > >>> runtime.getDataDomain().addNode(node);
> > >>>
> > >>> runtime.getDataDomain().setDefaultNode(node);
> > >>>
> > >>> On Wed, Oct 28, 2020 at 2:52 PM Tony Giaccone 
> > wrote:
> > >>>
> > >>>> So I have a project which is running in Bootique, but the data model
> > is
> > >>>> used in several different apps. To facilitate this, we created a jar
> > file
> > >>>> that includes the generated classes, and the code and the
> > >>>> cayenne-project.xml file in one jar file.  The result of this is
> tests
> > >>> run
> > >>>> for this jar, don't have access to the boo

Re: Spinning up Cayenne in a unit test.

[John Huss]

You could add a node at runtime:

DataNode node = *new* DataNode("node");

... // configure node

runtime.getDataDomain().addNode(node);

runtime.getDataDomain().setDefaultNode(node);

On Wed, Oct 28, 2020 at 2:52 PM Tony Giaccone  wrote:

> So I have a project which is running in Bootique, but the data model is
> used in several different apps. To facilitate this, we created a jar file
> that includes the generated classes, and the code and the
> cayenne-project.xml file in one jar file.  The result of this is tests run
> for this jar, don't have access to the bootique runtime, as they are part
> of a jar that's going to be included in other apps.
>
> We're using an older version of Bootique and so the version of Cayenne is
> 4.0.B2
>
> The cayenne project, only has a datamap there is no datanode defined in the
> xml.
>
> So I'm trying to spin up a cayenne runtime, using this code.
>
> runtime = ServerRuntime.builder()
> .addConfig("cayenne-CohortManager.xml")
> .addModule(binder -> {
> ServerModule.contributeProperties(binder)
> .put("cayenne.jdbc.driver", "org.h2.Driver")
> .put("cayenne.jdbc.url", "jdbc:h2:mem:CohortDB");
> })
> .build();
>
>
> The problem is that later, when I try to commit changes, I get this error:
>
> org.apache.cayenne.CayenneRuntimeException: [v.4.0.B2 Sep 26 2017
> 10:05:06] No DataNode configured for DataMap 'datamap' and no default
> DataNode set
>
> I'm successfully able to run other tests, because they never do a
> commitobjects.
>
>
> Any suggestions on how to resolve my problem.
>

Re: MariaDB column with auto increment

[John Huss]

I'd turn on SQL logging and check that it is refetching the row via SQL.
Also query the DB to verify the value is set there.

log4j.logger.org.apache.cayenne.log.JdbcEventLogger = INFO

On Wed, Oct 28, 2020 at 2:35 AM Markus Reich  wrote:

> Hi,
>
> I'm using Cayenne 4.1.
> I have an an object A with an one to one relation to another object B,
> in object B I got an column with an autoincrement field (no PK).
> Now when I create an new object A / B the value of the column stays at 0.
> I tried to refresh with object contexts invalidateObjects method and with a
> RefreshQuery but it stays 0?
>
> regards
>
> --
> *Markus Reich*
> Waldweg 62
> 6393 St. Ulrich am Pillersee
> www.markusreich.at / www.meeximum.at
> markus.re...@markusreich.at
>

Re: Expression to compare a property against another property

[John Huss]

The generated Property instances in the model classes support this out of
the box, but maybe you are using an older version of Cayenne?

On Thu, Sep 10, 2020 at 4:00 PM Faizel Dakri  wrote:

> Is there a facility in Cayenne to create an expression that would compare
> a property of one object against another property (maybe via a relationship
> path rooted at the same object) rather than against a scalar value?
>
> I’m trying to port over some old EOF code and we used to have a
> EOKeyComparisonQualifier which would allow you to create a qualifier to
> compare a property’s value with that of another property (or property key
> path).
>
> For example, given a model consisting of employees, where each employee
> can have a relationship to a manager and each employee has a salary
> property, you could use a key comparison qualifier to find those employees
> that are making more than their manager:
>
> salary > manager.salary
>
> I’ve looked in ExpressionFactory and through some of the source, but to me
> it looks like all comparison nodes in an expression operate on a key and a
> scalar. Am I missing something?
>
> Thanks for any help or pointers,
>
> F
>
> --
> Faizel Dakri
>
>
>

Re: Cayenne 4.2—Modifying SQL generation to always use COALESCE when updating a certain column

[John Huss]

Since you are already at the DB level (outside of cayenne) making this
change, I would just use a BEFORE UPDATE row-level trigger in Postgres. You
can change the value there.

Something like this would work:

CREATE OR REPLACE FUNCTION company_default()

  RETURNS trigger AS $$

BEGIN

NEW.company = coalesce(NEW.company, );

RETURN NEW;

END;

$$ LANGUAGE plpgsql;

CREATE TRIGGER mytable_update
BEFORE UPDATE ON MyTable
FOR EACH ROW
WHEN (NEW.company IS NULL)
EXECUTE FUNCTION company_default();


On Tue, Sep 8, 2020 at 3:13 PM Hugi Thordarson  wrote:

> Working with old DB designs really results in the weirdest questions…
>
> So… I've been working around a design problem in a customer DB by using my
> own BatchTranslatorFactory. The functionality was that if a column is
> called "company", every update wraps the column's new value in a coalesce
> function to ensure that it's never set to null (for… reasons). This has
> worked great as a workaround for our problem.
>
> However, SQL generation in Cayenne 4.2 is all new so my current solution (
> https://gist.github.com/hugith/a33a20fc7da7fd8f709f59ce3a30a96a <
> https://gist.github.com/hugith/a33a20fc7da7fd8f709f59ce3a30a96a> )
> doesn't really port.
> Before I start considering migration to 4.2, is this possible to do there?
> Or should I just bite the bullet and start fixing up that bloody DB before
> upgrading?
>
> Cheers,
> - hugi

Re: Cayenne 4.2/Postgres - difference in generated SQL based on nesting in expressions

[John Huss]

I see. It’s a good question, but I would just stick with .andExp in that
case. I prefer using that one anyway due to historical usage and also to
keep things consistent if I have Expressions defined apart from queries.

On Tue, Aug 18, 2020 at 4:30 PM Hugi Thordarson  wrote:

> Hi John,
>
> the grouping of the expressions (the locations of the parentheses) only
> relate to the Cayenne expression. The main issue is the different SQL
> generated by Cayenne 4.2 from those expressions due to the grouping.
>
> If you look at the SQL generated for (2), the last condition in the where
> clause (that uses date_time) is applied with a "=" rather than "AND". It's
> something I honestly just haven't seen or used before (and I'm embarrassed
> not knowing how or why it works, reading up right now). But I assume it
> means it first applies the first condition and then applies the second
> condition to the result of that to return a final result. In my case, that
> would make sense performance wise—I have indexes on "date_time" so if
> date_time is not included in the first query, we're going to have a bad
> time.
>
> What I don't get is why that "=" is generated instead of and "AND". If
> there's an "AND" the index gets used, regardless of the order of the
> expressions.
>
> - hugi
>
>
>
>
> > On 18 Aug 2020, at 18:10, John Huss  wrote:
> >
> > I would tend to think the postgres query planner is smart enough to
> handle
> > the differences in the placement of parentheses if that is the only
> > difference. But in any case, try adding "explain analyze" to the
> beginning
> > of the query and then compare the query plans between the fast version
> and
> > the slow version. You can post them here too. That would be the best
> place
> > to start I think.
> >
> > On Tue, Aug 18, 2020 at 11:57 AM Hugi Thordarson 
> wrote:
> >
> >> Hi all,
> >> I'm currently migrating a large project to Cayenne 4.2. Code wise the
> >> transition was easy enough, apart from one thing.
> >>
> >> Consider the following two expressions. Both are generated from the same
> >> three conditions.
> >>
> >> In the first case they're combined using exp1.andExp( exp2 ).andExp(
> exp3
> >> ).
> >>
> >> In the second case, they're combined using:
> >>
> >> ObjectSelect
> >>.query( SomeClass.class )
> >>.where( exp1 )
> >>.and( exp2 )
> >>.and( exp3 )
> >>
> >> The two different methods yield:
> >> 1) (customer = )
> >> and (key.number = "161-400") and (date > 2015-08-18)
> >> 2) ((customer = )
> >> and (key.number = "161-400")) and (date > 2015-08-18)
> >>
> >> Note the different nesting. In Cayenne 4.1 this was not a problem since
> >> the two expressions generated more or less the same SQL.
> >>
> >> However in 4.2.M2-SNAPSHOT, the generated SQL is quite different,
> >> resulting in a large performance loss in the latter case (as in, queries
> >> that previously took a few of milliseconds now take minutes). Here's the
> >> SQL generated — note the comparison that happens with the last
> (date_time)
> >> condition in (2):
> >>
> >> SQL generated by 1:
> >> - SELECT DISTINCT "t0"."amount" FROM "nb_movement" "t0" JOIN "nb_key"
> "t1"
> >> ON ( "t0"."company" = "t1"."company" ) AND ( "t0"."gl_number" =
> >> "t1"."gl_number" ) WHERE ( ( ( "t0"."company" = ? ) AND (
> >> "t0"."customer_no" = ? ) AND ( "t1"."gl_number" = ? ) AND (
> >> "t0"."date_time" > ? ) ) ) LIMIT 10 [bind: 1:'017294', 2:40,
> >> 3->gl_number:'161-400', 4->date_time:2015-08-18]
> >>
> >> SQL generated by 2:
> >> - SELECT DISTINCT "t0"."amount" FROM "nb_movement" "t0" JOIN "nb_key"
> "t1"
> >> ON ( "t0"."company" = "t1"."company" ) AND ( "t0"."gl_number" =
> >> "t1"."gl_number" ) WHERE ( ( ( ( "t0"."company" = ? ) AND (
> >> "t0"."customer_no" = ? ) AND ( "t1"."gl_number" = ? ) ) = (
> >> "t0"."date_time" > ? ) ) ) LIMIT 10 [bind: 1:'017294', 2:40,
> >> 3->gl_number:'161-400', 4->date_time:2015-08-18]
> >>
> >> Any ideas?
> >>
> >> I'd like to add the disclaimer that I have not been able to identify the
> >> exact conditions that result in the generation of the slower SQL, but it
> >> certainly does not happen in all cases (i.e. will only happen with some
> >> combinations of conditions).
> >>
> >> But I'd be happy to try to create a reproducible standalone test-case if
> >> required, just give me a shout. I just wanted to start out by throwing
> this
> >> out there.
> >>
> >> Cheers,
> >> - hugi
>
>

Re: Cayenne 4.2/Postgres - difference in generated SQL based on nesting in expressions

[John Huss]

I would tend to think the postgres query planner is smart enough to handle
the differences in the placement of parentheses if that is the only
difference. But in any case, try adding "explain analyze" to the beginning
of the query and then compare the query plans between the fast version and
the slow version. You can post them here too. That would be the best place
to start I think.

On Tue, Aug 18, 2020 at 11:57 AM Hugi Thordarson  wrote:

> Hi all,
> I'm currently migrating a large project to Cayenne 4.2. Code wise the
> transition was easy enough, apart from one thing.
>
> Consider the following two expressions. Both are generated from the same
> three conditions.
>
> In the first case they're combined using exp1.andExp( exp2 ).andExp( exp3
> ).
>
> In the second case, they're combined using:
>
> ObjectSelect
> .query( SomeClass.class )
> .where( exp1 )
> .and( exp2 )
> .and( exp3 )
>
> The two different methods yield:
> 1) (customer = )
> and (key.number = "161-400") and (date > 2015-08-18)
> 2) ((customer = )
> and (key.number = "161-400")) and (date > 2015-08-18)
>
> Note the different nesting. In Cayenne 4.1 this was not a problem since
> the two expressions generated more or less the same SQL.
>
> However in 4.2.M2-SNAPSHOT, the generated SQL is quite different,
> resulting in a large performance loss in the latter case (as in, queries
> that previously took a few of milliseconds now take minutes). Here's the
> SQL generated — note the comparison that happens with the last (date_time)
> condition in (2):
>
> SQL generated by 1:
> - SELECT DISTINCT "t0"."amount" FROM "nb_movement" "t0" JOIN "nb_key" "t1"
> ON ( "t0"."company" = "t1"."company" ) AND ( "t0"."gl_number" =
> "t1"."gl_number" ) WHERE ( ( ( "t0"."company" = ? ) AND (
> "t0"."customer_no" = ? ) AND ( "t1"."gl_number" = ? ) AND (
> "t0"."date_time" > ? ) ) ) LIMIT 10 [bind: 1:'017294', 2:40,
> 3->gl_number:'161-400', 4->date_time:2015-08-18]
>
> SQL generated by 2:
> - SELECT DISTINCT "t0"."amount" FROM "nb_movement" "t0" JOIN "nb_key" "t1"
> ON ( "t0"."company" = "t1"."company" ) AND ( "t0"."gl_number" =
> "t1"."gl_number" ) WHERE ( ( ( ( "t0"."company" = ? ) AND (
> "t0"."customer_no" = ? ) AND ( "t1"."gl_number" = ? ) ) = (
> "t0"."date_time" > ? ) ) ) LIMIT 10 [bind: 1:'017294', 2:40,
> 3->gl_number:'161-400', 4->date_time:2015-08-18]
>
> Any ideas?
>
> I'd like to add the disclaimer that I have not been able to identify the
> exact conditions that result in the generation of the slower SQL, but it
> certainly does not happen in all cases (i.e. will only happen with some
> combinations of conditions).
>
> But I'd be happy to try to create a reproducible standalone test-case if
> required, just give me a shout. I just wanted to start out by throwing this
> out there.
>
> Cheers,
> - hugi

Re: SpringBoot and Cayenne

[John Huss]

Unless you have special needs, yes, there should only be one.

On Mon, Aug 10, 2020 at 1:51 PM Tony Giaccone  wrote:

> So I'm using the example that was posted here a while ago, and I've
> modified it to read from a springboot yml file. I noticed that when the
> application spins up, each time a new instance of the CayenneService gets
> created, the code to spin up the cayenne runtime gets called.
>
> I have a strong feeling that's not the right behavior.  So I've moved the
> runtime to a class side property and then see if it's null before
> spinning up the runtime. That seems to work.
>
> My question is, in a well behaved Cayenne based application connected to
> only one instance of a database, should there only be one instance of the
> runtime, and one instance of the datasource?
>
> Also the code has a shared context, which gets overridden each time the
> init method is called.  That doesn't seem like a good idea either.
>

Re: 4.1 final release

[John Huss]

I'm happy with 4.1 going out!

On Mon, Jul 13, 2020 at 8:29 AM Hugi Thordarson  wrote:

> I think at least my hundreds of daily users (that will soon have enjoyed 3
> virtually problem free years in production with 4.1) approve :).
>
> Although then I'll have to start migrating to 4.2… Can't get caught using
> an officially stable final release, that's just lame!
>
> - hugi
>
>
>
> > On 10 Jul 2020, at 13:46, Nikita Timofeev 
> wrote:
> >
> > Hi all,
> >
> > I think that 4.1 is fully ready for the final release.
> > But first I wanted to check if I missing something.
> >
> > Are there any problems that are not in Jira?
> > Or everything is good and I could go with the release?
> >
> > --
> > Best regards,
> > Nikita Timofeev
>
>

Re: Types.OTHER

[John Huss]

Yeah, sounds good! Thanks!

On Mon, Jun 15, 2020 at 7:22 AM Malcolm Edgar 
wrote:

> That sounds like a great approach to me.
>
> regards Malcolm
>
> On Mon, Jun 15, 2020 at 9:49 PM Andrus Adamchik 
> wrote:
>
> >
> >
> > > On Jun 9, 2020, at 5:54 PM, John Huss  wrote:
> > >
> > > 2) Regarding skipping parsing when just going straight back out again
> to
> > > serialization - I have felt the reluctance (like you) of parsing this
> in
> > > those cases. But one thing to think about is that the parser/serializer
> > > (let's say Jackson in this case) may have special rules for how types
> are
> > > outputted like pretty/minified, sorted/unsorted, string/number (for
> > > BigDecimal). So skipping that parse/serialization may not actually be
> > > desirable, at least not in every case.
> >
> > Was discussing this offline with Nikita and stumbled on what can be a
> good
> > solution:
> >
> > 1. Cayenne would provide a set of "unparsed" wrapper types (Wkt, Json,
> > etc.) and the JDBC/SQL machinery around them (ExtendedType,
> > SqlTreeProcessor).
> > 2. A user who needs a parsed version would create their own types (e.g.
> > WktParsed, JsonParsed), and would connect them to Cayenne via a custom
> > ValueObjectType [1] (e.g. ValueObjectType).
> >
> > Such a two-tier approach would allow the users to map persistent
> > properties to anything they want without much effort and use their
> > preferred third-party parsers. While Cayenne would do all the DB-side
> heavy
> > lifting. The two parts are cleanly separated.
> >
> > Andrus
> >
> >
> > [1] https://cayenne.apache.org/docs/4.2/cayenne-guide/#value-object-type
>

Re: Types.OTHER

[John Huss]

On Wed, Jun 10, 2020 at 1:25 AM Andrus Adamchik 
wrote:

> >  Would this also address the use case where the DB doesn't have a special
> > type (like JSONB) and just stores it as a plain varchar or byte[], but
> you
> > still want the Java side to treat it as a complex object?
>
> Yep. I did this in my implementation for Wkt type, which can be mapped as
> either (VAR)CHAR or GEOMETRY. Of course in the former case DB-side geometry
> operations will not be available, but I suppose that's expected. (I know
> Nikita had some thoughts on how to expose custom operators for these new
> types, so that we could use them in queries).
>
> > 2) Regarding skipping parsing when just going straight back out again to
> > serialization - I have felt the reluctance (like you) of parsing this in
> > those cases. But one thing to think about is that the parser/serializer
> > (let's say Jackson in this case) may have special rules for how types are
> > outputted like pretty/minified, sorted/unsorted, string/number (for
> > BigDecimal). So skipping that parse/serialization may not actually be
> > desirable, at least not in every case.
>
> Could you clarify this point please? I understand the part about special
> settings in the ObjectMapper of Jackson, but to me that's an argument in
> favor of keeping this logic outside Cayenne, and letting the user control
> it themselves.
>

By not parsing a JSON value from a String to an object, you avoid having to
re-serialize it back to a String to return it from a web service. This is
good since that provides a performance bump. But you also miss out on the
"opportunity" for whatever mapper you are using to serialize data for your
web service to examine the deeper structure of this Object and perhaps make
different choices about how to serialize it.

The broader point is that the way your DB chooses to store a JSON value
might differ from the way you want to present it in your web service. If
that is the case, then preserving the same String form of the value from
the DB through the output of the web service doesn't help you - you'll need
to parse it and then re-serialize it with your special rules.

I think either approach is valid here. Based on my current use cases I
don't mind the performance hit taken from deserializing the value from the
DB into an object every time, and I appreciate the simplicity of just
getting the object I want (rather than a JSON string) without any other
ceremony. But I definitely understand the desire to avoid that
deserialization in cases where it appears to be unnecessary.


> Andrus
>
>
>
> > On Jun 9, 2020, at 5:54 PM, John Huss  wrote:
> >
> > Thanks Andrus, that's great. It would be very useful to have more
> built-in
> > or robust support in Cayenne for these types.
> >
> > A few things I thought of:
> > 1) Would this also address the use case where the DB doesn't have a
> special
> > type (like JSONB) and just stores it as a plain varchar or byte[], but
> you
> > still want the Java side to treat it as a complex object? I would think
> > that should be fairly easy to do given what you said. For example,
> storing
> > JSON in a MySQL varchar column and mapping it in cayenne to some kind of
> > java.util.Map
> >
> > 2) Regarding skipping parsing when just going straight back out again to
> > serialization - I have felt the reluctance (like you) of parsing this in
> > those cases. But one thing to think about is that the parser/serializer
> > (let's say Jackson in this case) may have special rules for how types are
> > outputted like pretty/minified, sorted/unsorted, string/number (for
> > BigDecimal). So skipping that parse/serialization may not actually be
> > desirable, at least not in every case.
> >
> >
> >
> > On Tue, Jun 9, 2020 at 1:59 AM Andrus Adamchik 
> > wrote:
> >
> >> Hi John, Malcom --
> >>
> >> (Don't know if Malcom is on dev@, so posting it here; we can move to
> dev@)
> >>
> >> Looks like the need for JSON type and other non-standard (from JDBC
> >> standpoint) types is there. Many people need it. I am hoping the effort
> >> that started independently by at least 3 community members can result in
> >> something that we can bring to the Cayenne core. Let's discuss how this
> >> might work.
> >>
> >> So... We'd need to create a taxonomy of "value object" types that map to
> >> "java.sql.Types.OTHER" (e.g. Jsonb, Wkt, TimestampRange, etc.). Then
> we'd
> >> create ExtendedTypes for each one of them, and register them inside the
> >> corresponding DbAdapters. More than one adapter can support 

Re: Types.OTHER

[John Huss]

Thanks Andrus, that's great. It would be very useful to have more built-in
or robust support in Cayenne for these types.

A few things I thought of:
1) Would this also address the use case where the DB doesn't have a special
type (like JSONB) and just stores it as a plain varchar or byte[], but you
still want the Java side to treat it as a complex object? I would think
that should be fairly easy to do given what you said. For example, storing
JSON in a MySQL varchar column and mapping it in cayenne to some kind of
java.util.Map

2) Regarding skipping parsing when just going straight back out again to
serialization - I have felt the reluctance (like you) of parsing this in
those cases. But one thing to think about is that the parser/serializer
(let's say Jackson in this case) may have special rules for how types are
outputted like pretty/minified, sorted/unsorted, string/number (for
BigDecimal). So skipping that parse/serialization may not actually be
desirable, at least not in every case.



On Tue, Jun 9, 2020 at 1:59 AM Andrus Adamchik 
wrote:

> Hi John, Malcom --
>
> (Don't know if Malcom is on dev@, so posting it here; we can move to dev@)
>
> Looks like the need for JSON type and other non-standard (from JDBC
> standpoint) types is there. Many people need it. I am hoping the effort
> that started independently by at least 3 community members can result in
> something that we can bring to the Cayenne core. Let's discuss how this
> might work.
>
> So... We'd need to create a taxonomy of "value object" types that map to
> "java.sql.Types.OTHER" (e.g. Jsonb, Wkt, TimestampRange, etc.). Then we'd
> create ExtendedTypes for each one of them, and register them inside the
> corresponding DbAdapters. More than one adapter can support a given type.
> Also we need to add adapter-specific logic to db-import Types.OTHER as the
> right value object type using DB-specific metadata of a given column. Same
> for SQL schema generation - for Types.OTHER we can derive the true type
> based on the value object it is mapped to.
>
> An important question is the internal structure of the value object types.
> Most of these types require parsing of a String or a byte[] into some
> special Java object. And there are two problems with it:
>
> 1. Performance. You may not always need that special object in many cases.
> E.g. your JSON is written directly as String to a REST response. No need to
> parse it by default.
> 2. External dependencies - Jsonb would require Jackson, Wkt - JTS libs to
> parse it to geometry, etc.
>
> I solved both problems by structuring the types as unparsed wrappers of
> String or byte[]. If there is a third-party lib dependency, it up to the
> calling code to decide when and how to parse the data and which library to
> use. If there are no dependencies (e.g. Postgres "tsrange" can be parsed to
> a pair of LocalDateTime objects using JDK classes) a parser can be included
> in the type, but the parsing should happen lazily.
>
> Also I was able to avoid the dependency on driver-specific code (PgObject
> in John's example) by providing a custom "sql tree processor" (a 4.2
> feature), so that String or byte[] can be bound to PreparedStatement, and
> the tree processor would alter the SQL to wrap the parameter in a Postgres
> function doing the type conversion in DB.
>
> I am planning to open-source the code that implements parts of the
> solution above. Hopefully soon.
>
> Thoughts?
>
>
> Also a minor note on John's example:
>
> > public static  void registerWithRuntime(ServerRuntime runtime,
> Class javaClass){
> >
> >   JsonbType jsonbType = *new* JsonbType(javaClass);
> >   Collection nodes =
> runtime.getDataDomain().getDataNodes();
> >   for(DataNode node : nodes) {
> >
>  node.getAdapter().getExtendedTypes().registerType(jsonbType);
> >   }
> > }
>
> Unless you have a custom DbAdapter as discussed above, an idiomatic way to
> register an ExtendedType would be to write a DI module and add it to
> runtime on startup:
>
>   Module m = b -> ServerModule.contributeUserTypes(b).add(jsonbType);
>
>
> Andrus
>
>
>
>
> > On May 26, 2020, at 5:37 PM, John Huss  wrote:
> >
> > For JSONB support I use this class (basically). But in addition you will
> > probably want an immutable (or optionally immutable) version of Map
> because
> > changes to the Map attribute itself won't be detected by Cayenne unless
> you
> > reset the attribute value on the entity, like
> > myEntityObject.setMyAttribute(new JsonMap(map)). In the Cayenne data
> > map/model you use the OTHER jdbc type.
> >
> > *import* java.sql.CallableStatement;
> >
> > *im

Re: Building Master Branch

[John Huss]

What command are you running to build? I would just start with "mvn clean
install"

For JSONB support I use this class (basically). But in addition you will
probably want an immutable (or optionally immutable) version of Map because
changes to the Map attribute itself won't be detected by Cayenne unless you
reset the attribute value on the entity, like
myEntityObject.setMyAttribute(new JsonMap(map)). In the Cayenne data
map/model you use the OTHER jdbc type.

*import* java.sql.CallableStatement;

*import* java.sql.PreparedStatement;

*import* java.sql.ResultSet;

*import* java.util.Collection;


*import* org.apache.cayenne.access.DataNode;

*import* org.apache.cayenne.access.types.ExtendedType;

*import* org.apache.cayenne.configuration.server.ServerRuntime;

*import* org.apache.cayenne.eof.JsonList;

*import* org.codehaus.jackson.map.ObjectMapper;

*import* org.postgresql.util.PGobject;


*public* *class* JsonbType *implements* ExtendedType {


*private* *static* *final* String *DB_TYPE* = "jsonb";


/**

* Call this at startup time to register with the Cayenne runtime.

* *@param* javaClass with user-created properties (getters and setters)

*/

*public* *static*  *void* registerWithRuntime(ServerRuntime runtime,
Class javaClass){

JsonbType jsonbType = *new* JsonbType(javaClass);


Collection nodes = runtime.getDataDomain().getDataNodes();

*for* (DataNode node : nodes) {

node.getAdapter().getExtendedTypes().registerType(jsonbType);

}

}


*private* *final* Class clazz;


*public* JsonbType(Class clazz) {

*this*.clazz = clazz;

*try* {

clazz.getConstructor(*new* Class[0]);

} *catch* (NoSuchMethodException | SecurityException e) {

*throw* *new* RuntimeException("Json type " + clazz.getName() + " does not
have a no-args constructor");

}

}


@Override

*public* String getClassName() {

*return* clazz.getName();

}


@Override

*public* T materializeObject(ResultSet rs, *int* index, *int* type) *throws*
Exception {

String json = rs.getString(index);

*return* materialize(json);

}


@Override

*public* T materializeObject(CallableStatement rs, *int* index, *int* type)
*throws* Exception {

String json = rs.getString(index);

*return* materialize(json);

}


*private* T materialize(String json) {

*if* (json != *null*) {

*try* {

T value = *new* ObjectMapper().readValue(json, clazz);

*if* (value *instanceof* JsonList) {

((JsonList)value).freeze();

} *else* *if* (value *instanceof* JsonMap) {

((JsonMap)value).freeze();

}

*return* value;

} *catch* (Exception e) {

*throw* *new* RuntimeException("Failed to deserialize value of type " +
clazz.getSimpleName() + " from " + json + ". " + e.getMessage(), e);

}

} *else* {

*return* *null*;

}

}


@Override

*public* *void* setJdbcObject(

PreparedStatement statement,

T value,

*int* pos,

*int* type,

*int* scale) *throws* Exception {


*if* (value == *null*) {

statement.setNull(pos, type);

} *else* {

String json;

*try* {

json = *new* ObjectMapper().writeValueAsString(value);

} *catch* (Exception e) {

*throw* *new* RuntimeException("Failed to serialize value of type " +
clazz.getSimpleName()
+ " from " + value + ". " + e.getMessage());

}


PGobject dataObject = *new* PGobject();

dataObject.setType(*DB_TYPE*);

dataObject.setValue(json);

statement.setObject(pos, dataObject);

}

}


@Override

*public* String toString(T value) {

*return* value.toString();

}


}


On Tue, May 26, 2020 at 5:17 AM Malcolm Edgar 
wrote:

> Hi All,
>
> I am wanting to build the Cayenne master branch to explore adding support
> for Postgres JSON, JSONB data types.  I am getting the following build
> error resolving the cayenne-maven-plugin when building on Windows or Linux
> / WSL.
>
> Does anyone have suggestions for how to resolve this?
>
> Thanks for your help.
>
> regards Malcolm
>
> [INFO]
> 
> [INFO] BUILD FAILURE
> [INFO]
> 
> [INFO] Total time:  5.206 s
> [INFO] Finished at: 2020-05-26T11:16:12+10:00
> [INFO]
> 
> [ERROR] Plugin
> org.apache.cayenne.plugins:cayenne-maven-plugin:4.2.M2-SNAPSHOT or one of
> its dependencies could not be resolved: Could not find artifact
> org.apache.cayenne.plugins:cayenne-maven-plugin:jar:4.2.M2-SNAPSHOT ->
> [Help 1]
> org.apache.maven.plugin.PluginResolutionException: Plugin
> org.apache.cayenne.plugins:cayenne-maven-plugin:4.2.M2-SNAPSHOT or one of
> its dependencies could not be resolved: Could not find artifact
> org.apache.cayenne.plugins:cayenne-maven-plugin:jar:4.2.M2-SNAPSHOT
> at
> org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolve
> (DefaultPluginDependenciesResolver.java:131)
> at
>
> org.apache.maven.plugin.internal.DefaultMavenPluginManager.getPluginDescriptor
> (DefaultMavenPluginManager.java:182)
> at
>
> 

Re: From 3.1.2 to 4.0.2

[John Huss]

Not at a computer at the moment - try MappedSelect.

On Thu, May 7, 2020 at 11:51 AM Sébastien Pérès-Labourdette <
sebastien.peres-labourde...@omerin.com> wrote:

> Hi.
>
> In 4.0.2, what happened to getEntityResolver().lookupQuery(queryName) ?
>
> How to fetch my queries from my datamap xml in 4.0.2 ?
>
> Cheers
> Sébastien.
>
>

Re: Can I access metadata (InfoExtension) from my velocity templates

[John Huss]

For accessing the metadata, I looked into this a bit. At runtime in an app
this information isn't accessible by default, so if you wanted to use it at
runtime you would have to configure the runtime to access it with a DI
module like this:

*public* *class* MyModule *implements* Module {

*public* *void* configure(Binder binder) {

binder.bind(HandlerFactory.*class*).to(ExtensionAwareHandlerFactory.
*class*);

binder.bind(DataChannelMetaData.*class*
).to(DefaultDataChannelMetaData.*class*);

ProjectModule.*contributeExtensions*(binder)

.add(InfoExtension.*class*);

}

}


Then you can get the metadata like so:


DataChannelMetaData metadata = localRuntime
.getInjector().getInstance(DataChannelMetaData.*class*);

String comment = ObjectInfo.*getFromMetaData*(metadata, attribute,
ObjectInfo.*COMMENT*);


In an entity template for cgen that part is all you need. Coding these two
lines directly in a template is difficult and ugly, so this would be a good
place to use a custom velocity tool to simplify this API.


COMMENT is the only metadata type defined currently, but I would bet you
could add your own, although I don't know if Modeler would preserve them if
you use it for editing.




On Tue, May 5, 2020 at 1:13 AM Faizel Dakri  wrote:

> No worries. I never did find a way to pass properties to the cgen goal in
> the plugin, so I ended up experimenting with the plugin source and modified
> it to allow specifying system properties from the pom.xml file (in the
> goal’s configuration section).
>
> Once the plugin was able to set system properties, it was relatively
> painless to get cgen to pick up my new tool using maven (I just had to
> specify it as a dependency in the cayenne plugin declaration in my
> application’s POM)—no need to muck with classpaths explicitly, thankfully.
>
> After all this, I still can’t find a way to access the metadata from
> within my own tool. I am starting to think that it can only be done by
> supplying the metadata (e.g. metaDataUtils) from the ClassGenerationAction
> class.
>
> Thanks again for the pointers.
>
> F
>
>
> > On May 4, 2020, at 9:56 AM, John Huss  wrote:
> >
> > Unfortunately I'm not a maven user, so I can't really help here. But
> > hopefully someone else could chime in.
> >
> >> On Sun, May 3, 2020 at 11:39 AM Faizel Dakri  wrote:
> >>
> >> Hi John,
> >>
> >> Thank you for the pointers. I didn’t see any way to easily get the
> >> metadata into my templates via a custom tool, however I thought it
> would be
> >> a good exercise to figure out how to do it anyway. I’ve tried to follow
> >> your advice (as well as looking at the cayenne source),   but I’m having
> >> difficulty getting my properties file loaded via the
> >> org.apache.velocity.tools property.
> >>
> >> I’m running cgen via my maven build and I can confirm the cgen goal is
> >> running (I can see that it is picking up my custom templates), however
> I’ve
> >> been unsuccessful in getting it to pick up my custom properties file
> where
> >> my tool is specified.  Is there a sanctioned way of telling the cgen
> goal
> >> about this property?  I’ve tried using both the maven properties plugin
> and
> >> passing on the command line, but I have yet to succeed in loading my
> file.
> >> At least it appears unsuccessful. When I debug my maven build, I see
> that
> >> there is an unresolved reference for my tool when velocity is
> processing my
> >> templates.
> >>
> >> Assuming I’m eventually successful in loading my properties, I have one
> >> more question. You mention that any custom velocity tool (and properties
> >> files) must be on the class path. I’m assuming that this is the class
> path
> >> in effect while the cgen goal is running. How does one go about altering
> >> the class path for a maven plugin? Or does the plugin pickup the project
> >> dependencies? (Apologies if this is a basic question, I’m not yet a
> Maven
> >> expert).
> >>
> >> Thanks again for your help,
> >>
> >> F
> >>
> >>
> >>>> On Apr 29, 2020, at 02:14 PM, John Huss  wrote:
> >>>
> >>> I haven't used the metadata / comments yet, so I can't help with that
> >>> specifically.
> >>>
> >>> For the using a custom tool with cgen you need to define a
> >>> "tools.properties" file in your class path (I would just put it at the
> >>> root) with contents like this:
> >>>
> >>>
> >>> tools.toolbox = application
> >>>
> >&g

Re: MySQL JSON Support

[John Huss]

I have been using the Postgres JSONB type as an entity attribute (mapped as
jdbc type OTHER) for some time, both with Lists and Maps. You can define an
ExtendedType subclass to read the value as a string and parse into a
map/list using Jackson's ObjectMapper.

On Mon, May 4, 2020 at 11:23 PM Malcolm Edgar 
wrote:

> Hi All,
>
> I was hoping to use the JSON features in MySQL 5.7 and wondered if anyone
> had used with with Cayenne.  I was hoping to use Named Queries to perform
> JSON specific queries, but treat it as a String otherwise with the Cayenne
> classes.
>
> Any comments or recommendations would be appreciated.
>
> regards Malcolm
>

Re: Can I access metadata (InfoExtension) from my velocity templates

[John Huss]

Unfortunately I'm not a maven user, so I can't really help here. But
hopefully someone else could chime in.

On Sun, May 3, 2020 at 11:39 AM Faizel Dakri  wrote:

> Hi John,
>
> Thank you for the pointers. I didn’t see any way to easily get the
> metadata into my templates via a custom tool, however I thought it would be
> a good exercise to figure out how to do it anyway. I’ve tried to follow
> your advice (as well as looking at the cayenne source),   but I’m having
> difficulty getting my properties file loaded via the
> org.apache.velocity.tools property.
>
> I’m running cgen via my maven build and I can confirm the cgen goal is
> running (I can see that it is picking up my custom templates), however I’ve
> been unsuccessful in getting it to pick up my custom properties file where
> my tool is specified.  Is there a sanctioned way of telling the cgen goal
> about this property?  I’ve tried using both the maven properties plugin and
> passing on the command line, but I have yet to succeed in loading my file.
> At least it appears unsuccessful. When I debug my maven build, I see that
> there is an unresolved reference for my tool when velocity is processing my
> templates.
>
> Assuming I’m eventually successful in loading my properties, I have one
> more question. You mention that any custom velocity tool (and properties
> files) must be on the class path. I’m assuming that this is the class path
> in effect while the cgen goal is running. How does one go about altering
> the class path for a maven plugin? Or does the plugin pickup the project
> dependencies? (Apologies if this is a basic question, I’m not yet a Maven
> expert).
>
> Thanks again for your help,
>
> F
>
>
> > On Apr 29, 2020, at 02:14 PM, John Huss  wrote:
> >
> > I haven't used the metadata / comments yet, so I can't help with that
> > specifically.
> >
> > For the using a custom tool with cgen you need to define a
> > "tools.properties" file in your class path (I would just put it at the
> > root) with contents like this:
> >
> >
> > tools.toolbox = application
> >
> > tools.application.*myUtils* = com.something.tools.MyCgenUtils
> >
> >
> > The referenced class (and the properties file) needs to be on the class
> > path when you invoke cgen. The class should contain public instance (not
> > static) methods in bean-style that work on parts of the DataMap, like an
> > ObjEntity or a ObjAttribute (or their properties). For example:
> >
> >
> > *public* *boolean* hasReverse(ObjRelationship rel) {}
> >
> > Then when you invoke cgen you need to set a system property to point to
> the
> > location of your tools.properties file, like this if it is in the class
> > path root:
> >
> > -Dorg.apache.velocity.tools=/tools.properties
> >
> >
> > Your template can reference an instance of your util class using the
> short
> > name defined for it in the tools.properties file:
> >
> >
> > ${*myUtils*.getGwtType($attr.Type)}
> >
> >
> > This process has multiple points of failure, so it can be hard to
> > setup correctly, but hopefully you can get it working.
> >
> > On Wed, Apr 29, 2020 at 11:29 AM Faizel Dakri  wrote:
> >
> >> Hello all,
> >>
> >> Being fairly new to Cayenne, this may be an obvious question. I would
> like
> >> to know how I can access the metadata stored on an
> >> attribute/relationship/entity in the datamap from within my velocity
> >> templates (I think in CayenneModeler, this is how the comment field is
> >> stored for a datamap item). I think this is the metadata stored via the
> >> newish InfoExtension facility.
> >>
> >> I see that the DataChannelMetaData can be injected into a DataDomain
> (and
> >> I’m doing that at runtime in my server app), however I cannot see how to
> >> get to this metadata from an attribute or relationship or entity in the
> >> context of a velocity template. Is this possible? I would think it is
> since
> >> Modeler is able to read/write those comments, but I am having a hard
> time
> >> doing so in a template.
> >>
> >> I did see that there is a hook to provide my own tool into the cgen
> tool.
> >> Would this be a potential path to look into if the metadata is not
> >> accessible directly in the templates as is? If so, any pointers on
> where to
> >> start?
> >>
> >> Thanks for any advice.
> >>
> >> F
> >>
> >> --
> >> Faizel Dakri
> >> list...@dakri.com
> >>
> >>
> >>
> >>
>
>

Re: Can I access metadata (InfoExtension) from my velocity templates

[John Huss]

I haven't used the metadata / comments yet, so I can't help with that
specifically.

For the using a custom tool with cgen you need to define a
"tools.properties" file in your class path (I would just put it at the
root) with contents like this:


tools.toolbox = application

tools.application.*myUtils* = com.something.tools.MyCgenUtils


The referenced class (and the properties file) needs to be on the class
path when you invoke cgen. The class should contain public instance (not
static) methods in bean-style that work on parts of the DataMap, like an
ObjEntity or a ObjAttribute (or their properties). For example:


*public* *boolean* hasReverse(ObjRelationship rel) {}

Then when you invoke cgen you need to set a system property to point to the
location of your tools.properties file, like this if it is in the class
path root:

-Dorg.apache.velocity.tools=/tools.properties


Your template can reference an instance of your util class using the short
name defined for it in the tools.properties file:


${*myUtils*.getGwtType($attr.Type)}


This process has multiple points of failure, so it can be hard to
setup correctly, but hopefully you can get it working.

On Wed, Apr 29, 2020 at 11:29 AM Faizel Dakri  wrote:

> Hello all,
>
> Being fairly new to Cayenne, this may be an obvious question. I would like
> to know how I can access the metadata stored on an
> attribute/relationship/entity in the datamap from within my velocity
> templates (I think in CayenneModeler, this is how the comment field is
> stored for a datamap item). I think this is the metadata stored via the
> newish InfoExtension facility.
>
> I see that the DataChannelMetaData can be injected into a DataDomain (and
> I’m doing that at runtime in my server app), however I cannot see how to
> get to this metadata from an attribute or relationship or entity in the
> context of a velocity template. Is this possible? I would think it is since
> Modeler is able to read/write those comments, but I am having a hard time
> doing so in a template.
>
> I did see that there is a hook to provide my own tool into the cgen tool.
> Would this be a potential path to look into if the metadata is not
> accessible directly in the templates as is? If so, any pointers on where to
> start?
>
> Thanks for any advice.
>
> F
>
> --
> Faizel Dakri
> list...@dakri.com
>
>
>
>

Re: Template path strip out home directory,

[John Huss]

I can't directly answer your question as I don't use modeler directly for
class generation, but use a build task instead. I recommend using a build
task because you get consistency and you don't have to re-select the
templates over and over. I use *ant* for this, but it can be done with
maven too:





























On Fri, Jan 17, 2020 at 6:47 PM Juan Manuel Diaz Lara
 wrote:

> I am using Cayenn 4.1.RC and trying to user custom templates. I created a
> new template, an selected in preferences, when generating I choose Advance
> and selecte my new template, generations fails with "Unable to find
> resource ..." and the template path, but this path does not have my home
> directory, it shows as a relative path, but in preferencis it has an
> absolute path.
> I revies my data map xml in a text editor and then template path is
> relative, without my home directory as prefix.
> So, I run CayenneModeler from my home directory and the template is found
> correctly. This is strange, because en Preferences path are absolute, but
> when generating they are used as relative to my home directory.
> I do not know if this is a bug o I am missing something.
> Thanks.
>
> Atte. Juan Manuel Díaz Lara

Re: New "Property" classes requiring Persistent

[John Huss]

For a very quick reply, I would recommend using the Java 8 streams API
.filter, etc instead of Property for doing things with POJOs. This should
perform better and be more debuggable, and be only slightly more verbose.

On Thu, Nov 14, 2019 at 4:50 PM Lon Varscsak  wrote:

> Hey all,
>
> I use Property-type classes for all sorts of things, usually for Cayenne
> related activities...but sometimes they're really useful on POJOs too.  The
> new Property related classes (like EntityProperty) require the class to be
> a subclass of Persistent.  I can't see where this is actually
> relevant...but I might be missing something.  Any reason we couldn't remove
> this restriction?
>
> Thanks,
>
> Lon
>
> P.S. -- Still wishing these also implemented Serializable :P
>

Re: Help with understanding caching possibly.

[John Huss]

Queries will always read from the database unless you explicitly indicate
you want a cached result. Prefetches are necessary to ensure related
objects are also refreshed by the query, otherwise they may be stale.

On Tue, Oct 29, 2019 at 8:05 PM Andrew Willerding 
wrote:

> I'm not sure what this means in terms of using Cayenne objects. I've
> tried to find some examples and found this reference for Cayenne 3.1
> where exactly the same question I'm asking is sort of addressed.
>
> https://stackoverflow.com/questions/29781605/cayenne-3-1-disable-cache
>
> but let's say I have this
>
>  return ObjectSelect.query(NotifyEvent.class)
>  .orderBy(STATUS_ID.asc())
>  .orderBy(D_TSTART.asc())
>  .select(DBClientBase.getObjectContext());
>
> How would I make sure all the objects that are referenced in NotifyEvent
> (including NotifyEvent) are reread from the database?
>
> There is a .prefetch(String path, int semantics) option but I don't know
> what to put into the parameters.
>
> Are there some better examples?
>
> Or should I just do something like this?
>
> List list = ObjectSelect.query(NotifyEvent.class)
>  .orderBy(STATUS_ID.asc())
>  .orderBy(D_TSTART.asc())
>  .select(DBClientBase.getObjectContext());
> DBClientBase.getObjectContext().invalidateObjects(list);
> return list;
>
>
> On 2019-10-29 11:22 a.m., Andrus Adamchik wrote:
> > You can always use queries (with prefetches for relationships) instead
> of keeping references to objects.
> >
> > Andrus
> >
> >> On Oct 29, 2019, at 5:55 PM, Andrew Willerding <
> awillerd...@itsurcom.com> wrote:
> >>
> >> Thank you Andrus.  I was chasing down the rabbit hole of looking for
> something like "reload" or "refresh" .
> >>
> >> Is there a way to always force reads from the database?  I'm not
> dealing with a high traffic applications so the caching is not important to
> me (at this point) and the retrieval of up-to-date data from the database
> is my priority.
> >>
> >> Andrew
> >>
> >>
> >> On 2019-10-26 7:58 a.m., Andrus Adamchik wrote:
> >>> Yes, of course:
> >>>
> >>>context.invalidateObjects(o);
> >>>
> >>> Andrus
> >>>
> >>>
>  On Oct 25, 2019, at 10:41 PM, Andrew Willerding <
> awillerd...@itsurcom.com> wrote:
> 
>  Is there a way to force a Cayenne object to refresh itself from the
> underlying database?  I have a situation where two *independent*
> Cayenne-based applications are accessing and updating the database and it
> seems that an update from one application is not reflected in the other.
> I'd like to manually force a refresh.
> 
>  Thanks,
> 
>  Andrew
> 
> 
>  On 2019-10-11 10:03 a.m., Andrus Adamchik wrote:
> >>   I think the docs for 4.1 still indicate it's set to true by
> default
> > Yes, this was wrong. I changed it right after I wrote my reply.
> Should republish soon.
> >
> > Andrus
> >
> >
> >> On Oct 11, 2019, at 9:43 AM, Andrew Willerding <
> awillerd...@itsurcom.com> wrote:
> >>
> >> Thank you Andrus!
> >>
> >> This has been driving me crazy.  I didn't realize this change was
> made.  I think the docs for 4.1 still indicate it's set to true by default
> >>
> >>
> https://cayenne.apache.org/docs/4.1/cayenne-guide/#appendix-a-configuration-properties
> >>
> >> *
> >>
> >>|cayenne.server.contexts_sync_strategy| defines whether peer
> >>ObjectContexts should receive snapshot events after commits from
> >>other contexts. If true (/default/), the contexts would
> >>automatically synchronize their state with peers.
> >>
> >>  o
> >>
> >>Possible values: true, false
> >>
> >>  o
> >>
> >>Default value: true
> >>
> >> On 2019-10-11 7:21 a.m., Andrus Adamchik wrote:
> >>> Hi Andrew,
> >>>
> >>> So I assume those two NotifyQueue are in different ObjectContexts?
> >>>
> >>> Since 4.1 we stopped doing cross-context state synchronization by
> default. So when you commit in one context, the same object in other
> contexts does not get refreshed until e.g. you run a query. This improves
> both performance and consistency in a typical multi-user application. But
> of course each app has different requirements. So you may turn
> synchronization back on when bootstrapping Cayenne:
> >>>
> >>> SererRuntime runtime = ServerRuntime
> >>>  .builder()
> >>>  ...
> >>>  .addModule(b ->
> ServerModule.contributeProperties(b).put(Constants.SERVER_CONTEXTS_SYNC_PROPERTY,
> "true"))
> >>>  .build();
> >>>
> >>> Andrus
> >>>
> >>>
>  On Oct 3, 2019, at 2:43 PM, Andrew Willerding <
> awillerd...@itsurcom.com> wrote:
> 
>  Hi,
> 
>  I am getting some strange results in writing/reading with Threads
> using Cayenne and I'm 

Re: 4.1 config file path migration question

[John Huss]

You'll have to subclass ResourceLocator and implement it:

/**

 * Finds a collection of matching resources for a given name. The name
components must

 * be separated by forward slashes.

 */

Collection findResources(String name);



For example, The name portion could just be the file name
"cayenne-blah.xml", and your subclass could hardcode the directory path.
The logic could be copied from FilesystemResourceLocator:


File resourceFile = *new* File(root, name);

*if* (resourceFile.exists()) {

*try* {

resources.add(*new* URLResource(resourceFile
.toURI().toURL()));

}

*catch* (MalformedURLException e) {

*throw* *new* CayenneRuntimeException("Can't convert
file to URL: %s", e,  resourceFile.getAbsolutePath());

}

}



Binding it using DI looks like this:


// a locator of resources, such as XML descriptors

binder.bind(ResourceLocator.*class*).to(MyResourceLocator.*class*);

binder.bind(Key.*get*(ResourceLocator.*class*, Constants.
*SERVER_RESOURCE_LOCATOR*)).to(MyResourceLocator.*class*);


On Fri, Oct 11, 2019 at 8:57 AM Andrew Willerding 
wrote:

> I have hopefully a quick question on how Cayenne can be configured to
> use a configuration file outside of a JAR/WAR.
>
> I currently use this to reference an absolute file path for my Cayenne
> files but I see that FilesystemResourceLocator is now deprecated ...
>
>  Module myModule = (org.apache.cayenne.di.Binder binder) -> {
>  binder.bind(ResourceLocator.class).toInstance(new
> FilesystemResourceLocator(filePathForCayenne));
>  ...
>
> What is the "new" way to replicate the same functionality.  The
> deprecated notes just provide the same example I'm already using.
>
> Thanks,
>
> Andrew
>
>
>
>

MappedSelect/SQLTemplate #chain and #chunk directives

[John Huss]

Looking at the docs
 for
MappedSelect/SQLTemplate I'm not clear on how to supply the parameters for
the #chain and #chunk directives from the java side. How should I structure
this data and pass it to the SQLTemplate query?

SQL:
#chain('OR' 'AND')
#chunk($thing1) thing1  = #bind($thing1, 'INTEGER') #end
#chunk($thing2) thing2 >= #bind($thing2, 'TIMESTAMP') #end
#end"

Java:

Map queryParameters = *new* HashMap<>(2);

queryParameters.put(*?*);


MappedSelect query = MappedSelect.*query*(MyDataMap.
*MYQUERY_QUERYNAME*, MyEntity.*class*).params(queryParameters);



Thanks.

Re: entity Qualifiers and enums

[John Huss]

Some more info would probably help us to understand what you're doing.

Is this a java enum? Or just a group of java constants?

How are you storing it in the DB? With the string value of the enum
constants? Or with the ordinal value or int constant value?

I believe the answers are:
1) Yes, you have to set it yourself, which is true of all non-primitive
fields.
2) If you set to the wrong value
 - If using enums, it should blow up when you read the row back from
the DB
 - If using int constants, it will just be wrong, but no exceptions.


On Tue, Oct 1, 2019 at 3:23 PM Tony Giaccone  wrote:

> I have two objects, that are based on the same table. I want to
> differentiate them by using a qualifier.
>
> I'm using a qualifer that looks like accountType = AccountType.hdaccount
>
> where hdaccount is one of the enumerated values.
>
> When i create an object of this type, the field is set to null. Is that
> typical? Do I have to explicitly set it myself, if so what happens if I set
> it to the wrong value?
>
> Tony Giaccone
>

Re: 4.2 modeler change

[John Huss]

The way to handle testing for the UI is to write the app in an MVP style,
sort of like this:
https://stackoverflow.com/questions/11367250/concrete-code-example-of-mvp

The gist is to define an interface contract for the view so that you can
replace the actual Swing UI View with a mock while testing.
But that doesn't help much with an existing application that wasn't written
in that style.


On Tue, Oct 1, 2019 at 1:00 AM Andrus Adamchik 
wrote:

> Yeah, I was thinking how do we even approach testing of Java UI. Any
> suggestions are welcome.
>
> And to complicate things we've been postponing a dive into JavaFX, while
> the Swing app keeps adding functionality. So investing effort in a test
> framework should take this pending decision into account.
>
> Andrus
>
> > On Oct 1, 2019, at 7:11 AM, Aristedes Maniatis  wrote:
> >
> > I've been down that path before, trying to test Swing and JavaFX. Its
> not easy to do. The best tool I found (and that was about 8 years ago) was
> https://www.froglogic.com/squish/editions/automated-java-gui-testing/ but
> I don't know if they have any licensing available for open source projects.
> >
> > Emerson, if you have any experience with this, let us know what has
> worked for you.
> >
> >
> > Ari
> >
> >
> > On 30/9/19 12:21am, Emerson Castañeda wrote:
> >> Wonder if these bugs would be into the kind of things that a good GUI
> test
> >> suite for the modeler could prevent.
> >>
> >> EmeCas
> >>
> >> On Thu, Sep 26, 2019 at 12:08 PM Lon Varscsak 
> >> wrote:
> >>
> >>> Okay, cool.  Another bug (I just found) is on the add relationship
> dialog
> >>> (on object entity) is that it seems to ignore the "delete" rule and
> just is
> >>> always the default.  Easily worked around by just editing the added
> >>> relationship after the fact.
> >>>
> >>> On Thu, Sep 26, 2019 at 12:22 AM Andrus Adamchik <
> and...@objectstyle.org>
> >>> wrote:
> >>>
>  Screenshots are stripped by the list management software, but the
>  description is pretty clear. I am not using 4.2 myself, but we do
> need to
>  fix it before we release 4.2.M1.
> 
>  Andrus
> 
> > On Sep 26, 2019, at 1:08 AM, Lon Varscsak 
>  wrote:
> > Hey all,
> >
> > It looks like when adding a db-relationship in the Modeler
>  (4.2.M1-SNAPSHOT from today) the potential target entities is not
> sorted
>  (which is only mildly annoying), but doesn't contain any target
> entities
>  outside of the current data map (blocker).  Am i missing something?
> > Thanks!
> >
> > -Lon
> >
> > Here's a screenshot for reference:
> >
> 
>
>

Re: Sending data across the wire.

[John Huss]

I guess I'm not clear on what you are using the DTO for. If I was using
DTOs it would be the same class on the server *and* the client (GWT). If
you're only creating DTOs in order to serialize them to a non-java
destination that just seems like a waste of effort. I suspect any good
serialization library should be able to handle customizing the
serialization of the regular DataObjects themselves without needing another
layer.

Jackson (JSON) supports using MixIn classes to tweak serialization so that
your regular cayenne classes are not affected ("polluted") in any way.

I strongly recommend against modeling circular relationships (two way
relationships) in your cayenne model to avoid headaches with serialization.
If you avoid circular relationships then you can use MixIns to fairly
easily trim your object graph to a reasonable size. Jackson also has Views
which can be used if you need to have more than one representation for a
class depending on the situation.




On Wed, Sep 18, 2019 at 7:52 AM Tony  wrote:

> Obviously the complexity of thiscould go off the charts. And you’re right
> the obvious complex solution would be agrest. I’m thinking of the opposite
> extreme. One DTO object per entity.
>
> Double bonus points if the DTO has two methods, entityToDTO and
> dtoToEntity.
>
> Relationship data can be as complicated and varied as use cases but a
> check box on the entity that created a DTO with just the attributes would
> certainly make my life easier.
>
> Mike Gentry’s idea of using Jackson I personally don’t like.   DTO and
> entity perform two different tasks and I don’t want to intertwine them. I
> recognize they are awfully close in nature and using Jackson is tempting
> but we still create DTOs. I’ll have to think on that more.
>
> Case by case you have to decide which relationships you should bundle back
> but more and more I like links to fetch related data.  Again not always
> appropriate but if you have DTOs readily made you can bundle them up in the
> payload as well.
>
> The other advantage of this is that you could market AGRest as the next
> step beyond the DTO solution that comes in Cayenne.
>
> Edge cases are always where the devil lives. :-) Thankfully we’re here to
> solve those problems.
>
> Tony Giaccone
>
> > On Sep 18, 2019, at 2:21 AM, Andrus Adamchik 
> wrote:
> >
> > Ignoring serialization mechanics (and even the presence of Cayenne) for
> a moment, I was always wondering how do people even approach this problem.
> Every persistent object is potentially an entry point to a near-infinite
> graph of objects coming from DB. So you need to have a set of clear rules
> of how deep and wide you want to go when you start serializing.
> >
> > DTO is one way to define such rules. But unless your rules are very
> simple ("send the attributes, don't send the relationships"), if you start
> generating DTOs, they will have the same set of relationships. And you'd
> end up with the same issue as you had with serializing the original
> DataObjects.
> >
> > So what parts of the graph do you need to serialize?
> >
> > And if this needs to be flexible, I would probably write a small library
> that allows a user to define a subset of model to serialize, and then
> applies that "submodel" to DataObjects directly (this is the Agrest.io
> approach on a smaller scale). But as with any generic solution, this will
> require a bit of effort, dealing with edge cases, etc.
> >
> > Andrus
> >
> >
> >
> >> On Sep 18, 2019, at 12:29 AM, John Huss  wrote:
> >>
> >> It's a small effort to create a custom entity template to generate any
> kind
> >> of entity class you want, for instance for a DTO. You can copy a lot of
> >> stuff right out of the regular templates if you want.
> >>
> >> Then you can use maven or ant to easily regenerate those classes the
> same
> >> way CayenneModeler does without having to change the custom template
> >> selections all the time.
> >>
> >>> On Tue, Sep 17, 2019 at 1:49 PM Tony Giaccone 
> wrote:
> >>>
> >>> So typically I create a Data Transfer Object to return values across
> the
> >>> wire that I have fetched and copy attributes from my model object to
> the
> >>> DTO.  This gets tedious quickly.  I wondered if there's not another way
> >>> that people are doing this, because this seems entirely too painful.
> >>>
> >>> However, if* this is* what most people do, then perhaps a change could
> be
> >>> made to the modeler, that would all you to create automatically a set
> of
> >>> DTO objects with when you generate the model objects.  I recognize that
> >>> this would be a rather significant effort and I'm guessing that there's
> >>> probably a better way, but I'm not aware of what it is.
> >>>
> >>> Any suggestions?
> >>>
> >>>
> >>> Tony Giaccone
> >>>
> >
>

Re: Sending data across the wire.

[John Huss]

Here's a link to a default template that you could copy some things from:
https://github.com/apache/cayenne/blob/master/cayenne-cgen/src/main/resources/templates/v1_2/singleclass.vm



On Tue, Sep 17, 2019 at 4:29 PM John Huss  wrote:

> It's a small effort to create a custom entity template to generate any
> kind of entity class you want, for instance for a DTO. You can copy a lot
> of stuff right out of the regular templates if you want.
>
> Then you can use maven or ant to easily regenerate those classes the same
> way CayenneModeler does without having to change the custom template
> selections all the time.
>
> On Tue, Sep 17, 2019 at 1:49 PM Tony Giaccone  wrote:
>
>> So typically I create a Data Transfer Object to return values across the
>> wire that I have fetched and copy attributes from my model object to the
>> DTO.  This gets tedious quickly.  I wondered if there's not another way
>> that people are doing this, because this seems entirely too painful.
>>
>> However, if* this is* what most people do, then perhaps a change could be
>> made to the modeler, that would all you to create automatically a set of
>> DTO objects with when you generate the model objects.  I recognize that
>> this would be a rather significant effort and I'm guessing that there's
>> probably a better way, but I'm not aware of what it is.
>>
>> Any suggestions?
>>
>>
>> Tony Giaccone
>>
>

Re: Sending data across the wire.

[John Huss]

It's a small effort to create a custom entity template to generate any kind
of entity class you want, for instance for a DTO. You can copy a lot of
stuff right out of the regular templates if you want.

Then you can use maven or ant to easily regenerate those classes the same
way CayenneModeler does without having to change the custom template
selections all the time.

On Tue, Sep 17, 2019 at 1:49 PM Tony Giaccone  wrote:

> So typically I create a Data Transfer Object to return values across the
> wire that I have fetched and copy attributes from my model object to the
> DTO.  This gets tedious quickly.  I wondered if there's not another way
> that people are doing this, because this seems entirely too painful.
>
> However, if* this is* what most people do, then perhaps a change could be
> made to the modeler, that would all you to create automatically a set of
> DTO objects with when you generate the model objects.  I recognize that
> this would be a rather significant effort and I'm guessing that there's
> probably a better way, but I'm not aware of what it is.
>
> Any suggestions?
>
>
> Tony Giaccone
>

Re: fetching from cache without sql transaction

[John Huss]

LOCAL_CACHE will only return cached data for queries using the *same*
objectContext, which is usually the same HTTP request). If you have
different objectContexts (requests) then you have to use SHARED_CACHE
instead. Then you have to figure out when to invalidate/expire the cached
data.

On Tue, Sep 17, 2019 at 2:29 AM abapseres  wrote:

> Hello,
>
> I would to know if it's possible to fetch newObject directely from cache
> with ObjectSelect.
>
> For now i don't manage to do this, for example when I make something like
> that :
>
>
> List list1 = ObjectSelect.query(MyTable.class)
>
> .cacheStrategy(QueryCacheStrategy.LOCAL_CACHE,"MyGroup")
>
> .select(getObjectContext());
>
>
> MyTable o1 = getObjectContext().newObject(MyTable.class);
>
> o1.setName("nouveau");
>
> getObjectContext().commitChanges();
>
>
>
> List list2 = ObjectSelect.query(MyTable.class)
>
> .cacheStrategy(QueryCacheStrategy.LOCAL_CACHE,"MyGroup")
>
> .select(getObjectContext());
>
>
> For result list1 and list2 have same size, i have to switch
> QueryCacheStrategy to LOCAL_CACHE_REFRESH to get new row, but there is a
> database transaction.
>
>
> Thanks
>
>
>

Re: Cayenne And SpringBoot

[John Huss]

You don't want to construct multiple ServerRuntimes, especially not one per
context. You just need one runtime and you can create many contexts from it.

On Tue, Aug 13, 2019 at 1:09 PM Emerson Castañeda  wrote:

> I got this working using this approach:
>
> *dependencies:*
>
> //for persistency
> compile 'org.apache.cayenne:cayenne-server:X.Y'
> compile 'org.apache.cayenne:cayenne-java8:X.Y'
> compile 'com.oracle:ojdbc.x.y.b'
>
>
> *application.yml*
>
> spring:
> application:
> name: app-api
> evnpath:
> cayenne_datasource:
> fpa:
> username: user
> password: pwd
> url: jdbc:oracle:thin:@server.org:1521/DEV.org
> driver: oracle.jdbc.driver.OracleDriver
>
>
> *Configuration class:*
>
> @Component
> public class DataProviderService {
> private static final Logger logger =
> LoggerFactory.getLogger(DataProviderService.class);
> List cayenneContexts = null;
> @Autowired
> private MyBeanPropertyReader myBeanPropertyReader;
>
> private  ObjectContext cayenneContext(String user, String password,String
> url, String driver) {
> ServerRuntimeBuilder b = ServerRuntime.builder();
> b = b.url(url);
> b = b.jdbcDriver(driver);
> b = b.user(user);
> b = b.password(password);
> b = b.addConfigs("cayenne-ExistentFacultyPortfolio.xml");
> ServerRuntime cayenneRuntime = b.build();
> return cayenneRuntime.newContext();
> }
>
>
> @PostConstruct
> private void init(){
> cayenneContexts =
>
> Arrays.asList(cayenneContext(myBeanPropertyReader.getUser(),myBeanPropertyReader.getPassword(),
> myBeanPropertyReader.getUrl(),myBeanPropertyReader.getDriver()));
>
> logger.debug("DataProviderService init APPLICATION");
>
> cayenneContexts.forEach(
> oc -> oc.getEntityResolver()
> .getDataMaps()
> .forEach(dm -> logger.debug("DATAMAP : "+dm.getName(;
> }
>
> //any other method using the context
> }
>
> *Usage:*
>
> @Autowired
> private DataProviderService dataProviderService;
>
>
> *Note:*
>
> datanode and datamap are located in the usual place: src/main/resourcesMy
>
> On Tue, Aug 13, 2019 at 12:29 PM Tony Giaccone  wrote:
>
> > I want to look into using Cayenne with SpringBoot.  I was able to get a
> > basic cayenne stack up and running by implementing a ContextListener  and
> > on the create event starting up a Cayenne Runtime.  I was using an in
> > memory database and I had problems getting the ;create=true working.  My
> > hack was to set the strategy on the DataNode after the runtime after it
> > was  spun up.
> >
> > Has anyone done this before?  Are there any  suggestions  on what I
> should
> > be certain to do or avoid? Should I just spin up the standard Cayenne web
> > filter?  Are there other choices?
> >
> > Thanks for any help you can provide.
> >
> >
> > Tony
> >
>

Re: Cayenne And SpringBoot

[John Huss]

Cayenne is pretty generic as far as integrations go - it should work well
and easily with any framework. Just create your ServerRuntime and define a
way to retrieve it (using ServletContext.setAttribute is typical). Then
you'll want to bind the runtime to each request that comes in, which is all
that CayenneFilter does. If CayenneFilter has worked for you, then just use
that. CayenneFilter is very minimal so you copy it and customize it if
needed.

On Tue, Aug 13, 2019 at 11:29 AM Tony Giaccone  wrote:

> I want to look into using Cayenne with SpringBoot.  I was able to get a
> basic cayenne stack up and running by implementing a ContextListener  and
> on the create event starting up a Cayenne Runtime.  I was using an in
> memory database and I had problems getting the ;create=true working.  My
> hack was to set the strategy on the DataNode after the runtime after it
> was  spun up.
>
> Has anyone done this before?  Are there any  suggestions  on what I should
> be certain to do or avoid? Should I just spin up the standard Cayenne web
> filter?  Are there other choices?
>
> Thanks for any help you can provide.
>
>
> Tony
>

Re: ServerRuntime.builder().addConfig

[John Huss]

Gradle and Maven both have great support for loading resources from the
classpath by convention. *These resources should be placed in
"src/main/resources".*

The current/working directory makes no difference since everything is
loaded from the classpath by default.

I'd recommend setting your database connection info using a module for BOTH
test AND production in order to externalize this information from the
project file.

If you insist on loading the project file from the filesystem directly
instead, it is supported as well, but I don't have experience doing it.
This way is deprecated, but here you go:

A ResourceLocator that can locate resources in the filesystem. Instances of
FilesystemResourceLocator are explicitly created by the user and then bound
to a DI registry. E.g.:

 class MyModule implements Module {

 public void configure(Binder binder) {
File dir1 = ...
File dir2 = ...
binder.bind(ResourceLocator.class).
   toInstance(new FilesystemResourceLocator(dir1, dir2);
 }



On Thu, Jul 18, 2019 at 12:12 PM Joe Baldwin 
wrote:

> Mike,
>
> status: tried everyone’s recommendations and gradle & cayenne are still
> fighting each other
> root problem: appears to be connection properties & cayenne config file
> path (gradle & cayenne fighting each other)
> Question: has anyone actually combined gradle and cayenne together for
> both unit & integrated test successfully - if so, what is the best practice
> (easy config, and durable)?
>
> What I have done so far (based on community comments):
>
> ---
>
> > under Cayenne 3.1, I am using this to provide different connection
> properties depending on whether it's a production system, a dev system,
> running tests, etc.
>
> Yes, that was my original strategy, but then John Huss commented:
>
> "You should only have one project file. You can change the DB connection
> properties using a module passed to the runtime. Here's some info:
> https://stackoverflow.com/questions/18948418/set-database-path-in-cayenne;
>
> Of course that link is really old and the code doesn’t work with 4.0.1.
>  So I hacked a demo with the 4.0.1 classes.   Then extrapolated what I
> think John Huss was suggesting (in his short comment), and put my
> connection properties in a gradle properties file.  I could then quickly
> set test/integ-test config values from the properties file.
>
> If I understand John’s implied strategy, here is working-code I hobbled
> together (note: CayenneConfPath & ServerModule):
>
> ServerRuntime cayenneRuntime = ServerRuntime.builder()
> .addConfig(CayenneConfPath)
> .addModule(binder ->
> ServerModule.contributeProperties(binder)
> .put(Constants.JDBC_DRIVER_PROPERTY,
> driverStr)
> .put(Constants.JDBC_URL_PROPERTY, urlStr)
> .put(Constants.JDBC_USERNAME_PROPERTY,
> userStr)
> .put(Constants.JDBC_PASSWORD_PROPERTY,
> passwordStr)
> )
> .build();
>
> Seemed ‘all good’ but not so … Gradle has a “convention over
> configuration” stated-philosphy, while Cayenne as a “just dump it into the
> classpath root” policy (both are "opinionated"), so the relative paths are
> set to the test directories, and if you try to put the cayenne config files
> in a single dir (say “main”), then cayenne (via junit5) can’t find the
> project files (i.e. CayenneConfPath - which by convention points
> Cayenne-Runtime to the Datsource Map).
>
> OK, so now I have to go “spelunking” …  I print out the Gradle “test" &
> “main" (main is the production conventions).  The paths can be overridden
> but it is not recommended - and doing so is problematic (i.e. it is easy to
> make mistakes given Gradle-path-convention-behavior).
>
> So currently, I have added in John’s “Module” suggestion, (which appears
> to assume the .addConfig() files are still in the CLASSPATH so as to load
> the project-MAP).   Thus, I still have to have two versions of the cayenne
> config files because Gradle enforces a test-path policy, while Cayenne
> enforces a relative-classpath-only policy.
>
> It appears I am back where I started from.  So then I thought this is ‘in
> - fricking - sane’ .. there has to be a better way.:)
>
> The root-cause of the problem (based on experimentation), is that Gradle
> has a relative-path convention that does not play-well with cayenne’s
> root-classpath convention.  (The only way I have found around this prob

Re: ServerRuntime.builder().addConfig

[John Huss]

You should only have one project file. You can change the DB connection
properties using a module passed to the runtime. Here's some info:
https://stackoverflow.com/questions/18948418/set-database-path-in-cayenne


On Wed, Jul 17, 2019 at 11:01 AM Joe Baldwin 
wrote:

> John,
>
> > What are you trying to override for testing? If it's the database
> connection properties there are better ways to do that.
>
>
> The context is automated testing.  (i.e. unit and integrated testing).  I
> am using the project file to set the connection properties (i.e. mysql,
> server-ip).   The unit testing will be on a different server from
> integrated testing.
>
> The initial idea is to have gradle perform the unit tests, while have
> jenkins orchestrate the integrated testing.
>
> I assumed that this could be handled via a simple change to the project
> file connection parameters.  However, with the addConfig assumption of
> classpath (disallowing explicit path), coupled with gradle defaults of
> adding both main resources AND test resources, cayenne runtime is
> complaining that it is finding two project files.   In the short run, I am
> relying on Cayenne preferring the first project file found.
>
> However, this is a short term solution - I would like to replace it with a
> more durable solution.   I would like to be able to build various testing
> databases as the need requires and apply connection properties (in a
> deterministic but simple-to-modify manner).
>
> > The typical approach is to put the project file in the root of the
> classpath.
>
>
> That is indeed an “approach” but as I summarized in the previous email, it
> disallows explicitly setting the path - so it looks like the *only*
> approach - at least based on my experimentation with addConfig() - and one
> that cayenne runtime is complaining about in this case.
>
> I am trying to come up a connection configuration strategy that uses
> cayanne best practices, as well as conforms to gradle and jenkins
> capabilities.  Just relying on “classpath” is causing problems right now,
> and it does not look reliable (based on the complaints by cayenne runtime).
>
> Joe
>
>
>
> > On Jul 17, 2019, at 10:54 AM, John Huss  wrote:
> >
> > The typical approach is to put the project file in the root of the
> > classpath.
> >
> > What are you trying to override for testing? If it's the database
> > connection properties there are better ways to do that.
> >
> > On Wed, Jul 17, 2019 at 9:25 AM Joe Baldwin 
> wrote:
> >
> >> Goal:
> >> My goal is to provide an unambiguous (deterministic) path to my project
> >> xml file to ServerRuntime during automated testing (the plan is to have
> >> multiple stages of testing).
> >>
> >> My understanding  is that 4.0.1 uses:
> >> ServerRuntime.builder().addConfig()
> >> to accomplish this normally.
> >>
> >> Problems:
> >> 1. While trying to understand the behavior of .addConfig(), I have had
> to
> >> rely primarily on experiments (i.e. hacking), since I have found no
> >> documentation in JavaDocs (addConfig() JavaDoc has no comments).
> >>
> >> 2. So it appears from my experiments that addConfig() assumes CLASSPATH
> >> referencing and will not accept an explicit path.
> >> Example:
> >> org.apache.cayenne.configuration.server.DataDomainLoadException:
> [v.4.0.1
> >> Dec 20 2018 11:02:32] Configuration resource
> >> "/webdev/cms/src/main/resources/config/cayenne/cayenne-CMSDomain.xml" is
> >> not found.
> >> Note:
> >> If I understand the behavior, addConfig() *only* allows
> project-file-path
> >> specification relative to one of the paths already specified in the
> >> CLASSPATH, and will not accept a full path entry.
> >>
> >> 3. To further complicate this, during testing, cayenne is complaining
> that
> >> multiple project files are found in the classpath (gradle, by default,
> >> appears to create a complex classpath that include multiple resource
> >> directories that contain the two project files).
> >>
> >> Note:
> >> Currently, I have the test-resources earlier in the classpath (cayenne
> >> runtime appears to select based on classpath positioning).  I would
> prefer
> >> a more explicit solution.
> >>
> >>
> >> Questions:
> >> 1. With not much info in the JavaDocs on usage-options, does anyone have
> >> an online reference that could provide more insight (i.e. alternatives,
> >> use-case examples, etc)?
> >>
> >> 2. Does anyone know of a simpler way to accomplish my goal (i.e.
> >> deterministically or explicitly provide the project file to cayenne
> >> runtime)?
> >>
> >>
> >> Thanks
> >> Joe
> >>
> >>
>
>

Re: ServerRuntime.builder().addConfig

[John Huss]

The typical approach is to put the project file in the root of the
classpath.

What are you trying to override for testing? If it's the database
connection properties there are better ways to do that.

On Wed, Jul 17, 2019 at 9:25 AM Joe Baldwin  wrote:

> Goal:
> My goal is to provide an unambiguous (deterministic) path to my project
> xml file to ServerRuntime during automated testing (the plan is to have
> multiple stages of testing).
>
> My understanding  is that 4.0.1 uses:
>  ServerRuntime.builder().addConfig()
> to accomplish this normally.
>
> Problems:
> 1. While trying to understand the behavior of .addConfig(), I have had to
> rely primarily on experiments (i.e. hacking), since I have found no
> documentation in JavaDocs (addConfig() JavaDoc has no comments).
>
> 2. So it appears from my experiments that addConfig() assumes CLASSPATH
> referencing and will not accept an explicit path.
> Example:
> org.apache.cayenne.configuration.server.DataDomainLoadException: [v.4.0.1
> Dec 20 2018 11:02:32] Configuration resource
> "/webdev/cms/src/main/resources/config/cayenne/cayenne-CMSDomain.xml" is
> not found.
> Note:
> If I understand the behavior, addConfig() *only* allows project-file-path
> specification relative to one of the paths already specified in the
> CLASSPATH, and will not accept a full path entry.
>
> 3. To further complicate this, during testing, cayenne is complaining that
> multiple project files are found in the classpath (gradle, by default,
> appears to create a complex classpath that include multiple resource
> directories that contain the two project files).
>
> Note:
> Currently, I have the test-resources earlier in the classpath (cayenne
> runtime appears to select based on classpath positioning).  I would prefer
> a more explicit solution.
>
>
> Questions:
> 1. With not much info in the JavaDocs on usage-options, does anyone have
> an online reference that could provide more insight (i.e. alternatives,
> use-case examples, etc)?
>
> 2. Does anyone know of a simpler way to accomplish my goal (i.e.
> deterministically or explicitly provide the project file to cayenne
> runtime)?
>
>
> Thanks
> Joe
>
>

Re: Testing & Map

[John Huss]

https://stackoverflow.com/questions/18948418/set-database-path-in-cayenne

On Sat, Jul 6, 2019 at 8:36 PM Joe Baldwin  wrote:

> I am trying to genericize the map.xml for multiple instances with multiple
> dbnames (looking for a “use ” analogue for cayenne config files -
> or a better idea if you have one)
>
> Details ...
>
> Context:
>   cayenne: 4.0.1
>   db: mysql 8.0.16
>   dbname: “cms" / “cmstest" / "cms_” (all
> referring to the same schema)
>
> Goals:
>   - migrate to cicd (gradle, git, jenkins, automated testing,etc)
>   - support testing configurations, as well as multiple instances of the
> same app but with unique dbnames
>
> Prob:
> While trying to migrate to fully automated testing (using junit-5) I hit a
> snag:
>   - the cayenne map.xml has the datasource name (“cms” in this instance)
> embedded in the map.xml config file:
>
> 
> …
> 
>
> So the general idea I had was to create a tmp-test database (cmstest), run
> the junit tests for unit & complex unit testing configured using gradle,
> then move on to a more complex testing config downstream.   When I deploy
> it will be to multiple instances (with unique dbnames).
>
> But I cannot figure out how to genericize this map.xml so that I can
> easily change the datasource name - I could throw sed/awk at it - to change
> all the schema-name-refs, but I was hoping for something a bit more
> cayenne-oriented (and less brittle than sed/awk).
>
> Ideally, I am searching for a cayenne-analogue to mysql “use ” so
> that I could change the datasource name in one location (vs 25).
>
> Thanks
> Joe
> PS if you have any other gradle tricks that you rely on, please do send a
> link
>
>

Re: Deprecate SelectQuery in 4.2

[John Huss]

On Mon, Jul 1, 2019 at 2:24 PM Joe Baldwin  wrote:

> Arseni,
>
> What is the ObjectSelect analog to
>
> RefreshQuery
>

Not sure what equivalent for this one is.


> and
> setCacheStrategy
>

ObjectSelect has .cacheStrategy(QueryCacheStrategy strategy, String
cacheGroup)


>
> Thanks
> Joe
>
>
> > On Jun 28, 2019, at 9:47 AM, Arseni Bulatski 
> wrote:
> >
> > Hi all,
> >
> > SelectQuery is planned to be deprecated in 4.2.
> > ObjectSelect has better api and is recommended to use.
> >
> > PR: https://issues.apache.org/jira/browse/CAY-2523
> >
> > Please, write your thoughts about it.
>
>

Re: Deprecate SelectQuery in 4.2

[John Huss]

On Mon, Jul 1, 2019 at 11:29 AM Joe Baldwin  wrote:

> Arseni,
>
> Disclaimer:
> I *just* (this week) upgraded from 4.0.M2 to 4.0.1.  I migrated to using
> the newer syntax - ex:
>
> table1.property1.asc()
>
> > SelectQuery is planned to be deprecated in 4.2.
>
> My project makes extensive use of “.andExp” to perform
> conditional-expression-building.  So the only question I have is concerning
> discrete expression component building (ie. combining expression-components
> via “.andExp").  Would this be available or would it be deprecated as well?
>

ObjectSelect has a .and(Expression...) method that you can use. Or you can
use Expression.andExp(Expression...)


> BTW, I do like the newer syntax, as it it much easier to read and is less
> verbose.  Excellent work!
>
> Thanks
> Joe
>
>
> > On Jun 28, 2019, at 9:47 AM, Arseni Bulatski 
> wrote:
> >
> > Hi all,
> >
> > SelectQuery is planned to be deprecated in 4.2.
> > ObjectSelect has better api and is recommended to use.
> >
> > PR: https://issues.apache.org/jira/browse/CAY-2523
> >
> > Please, write your thoughts about it.
>
>

Re: Deleting a bunch of interrelated objects

[John Huss]

If the column name is the same in all the tables it would be pretty easy to
loop over all your entities and execute the same delete statement (via raw
SQLExec). Put all the the entities with that column in a Set and loop over
them, removing one when it succeeds until the set is empty.

On Wed, May 29, 2019 at 7:35 AM Hugi Thordarson  wrote:

> Hi all.
>
> I have a pretty large model where every to-many relationship's delete rule
> is "Deny" (I like being explicit about the data I delete, especially since
> the DB in question still doesn't have well specified FKs).
>
> Almost every table in this DB is somehow related to a "Customer" table
> (sometimes through a chain of tables) that "owns" data.
> I'm now implementing an operation to nuke a Customer (as in "delete
> everything related to that customer")—but the data  "owned" by the customer
> is hugely interrelated, even often creating circular relations.
>
> Do I have any nice options, given a set of DataObjects, to just tell
> Cayenne to nuke that darn data without any consideration for normal delete
> rules? As in "usually I wouldn't allow this and Ashwood is very angry at
> you—but since you really, really want it…".
>
> Note that the whole operation results in a consistent database (even if
> there are temporary in-memory inconsistencies due to the circular
> relationships) since the interrelated data is all deleted within the
> operation.
>
> Cheers,
> - Hugi

Re: Further on modeler problems and a Question..

[John Huss]

Just by querying the table at startup you can load it into the Object cache
(if your shared cache is enabled, which it is by default). However if you
exceed the size of the shared cache it will eventually evict those objects
and re-fetch them one at a time as needed (which is bad).

On Mon, May 20, 2019 at 11:09 AM Tony Giaccone  wrote:

> I also have Cayenne Modeler at version 4.01 and that starts up with out a
> problem. So I am even more confused.  The 4.0 release version seems to work
> fine and keeps the same version number in the model files so I'll be using
> that.
>
> Now the question that started this whole process.  I have a table in the
> database that's essentially a look up table, it has several thousand rows,
> and I'd like to just read it in once and then use it across different
> incoming requests.  Is there a best practice for this use case?
>
>
> Tony
>

Re: Property class and Serialization

[John Huss]

Does this help?

https://stackoverflow.com/questions/910374/why-does-java-have-transient-fields


On Tue, May 14, 2019 at 5:20 PM Lon Varscsak  wrote:

> Any reason we can't have Property be Serializable?  I have situations where
> I will keep references to properties in a web page (like for dynamic
> columns) and my framework serializes those things (or errors out if it's
> not serializable).  I originally solved this with having my own Subclass be
> Serializable, but now with protected constructors I'm getting errors.
>
> Thoughts?
>
> -Lon
>

Re: Migrate Horizontal Inheritance (EOF) to Cayenne

[John Huss]

On Sat, Apr 13, 2019 at 9:26 AM Jérémy DE ROYER 
wrote:

> Hello,
>
> I’m trying to migrate (eo)models using horizontal inheritance (over
> multiple frameworks) but without any success : it’s not possible to
> generate the Java classes using Cayenne Modeler.
>
> I did a test with only one entity (and table) : that works great (and
> easy).
>
> I’ve created a second entity (and  table) : that works great and easy.
>
> I’ve set up an horizontal inheritance between the two entities (and table)
> : Cayenne forces me to use the same table for both entities
>
> In the 3.0's doc, I’ve read : “As of this writing Cayenne does not support
> horizontal inheritance. It may in the future”.


> Is it still true in the 4.0 ? 4.1 ? Is the game over ? (as I can not use
> vertical inheritance) 洛
>
Unfortunately I believe this is still true.


>
> Thank’s,
>
> Jérémy
>

Re: Checking a value on a DataObject and modifying it before commit if required

[John Huss]

There can be issues if you modify other objects inside lifecycle callbacks
(a callback won't be called on that object). I would recommend doing the
mods in validateForSave instead.

On Thu, Apr 4, 2019 at 6:44 AM Hugi Thordarson  wrote:

> Yes, this would have been a good idea :). Unfortunately the model
> (containing dozens of tables and hundreds of relationships) is already
> completely modeled using relationships and makes heavy use of them (for
> expressions, ordering, prefetching etc. etc.)
>
> The Cayenne-project was originally created as the reporting-part of the
> system, so I didn't really hit these issues until now, when we're moving
> the actual "write" part of the system there as well.
>
> Dang…
>
> Cheers,
> - hugi
>
>
>
> > On 4 Apr 2019, at 11:10, Ken Anderson  wrote:
> >
> > My guess would be “no”.
> >
> > I suggest you don’t model the relationships until the foreign systems
> are fixed or removed. In the meanwhile, I would implement methods as the
> relationships, that would know if there’s a fictitious value or not and
> return NULL for the relationship. When the other systems are fixed, remove
> the method and implement the relationship normally.
> >
> >> On Apr 4, 2019, at 6:39 AM, Hugi Thordarson  wrote:
> >>
> >> I spoke a little too soon—my solution isn't sufficient, who'd have
> thunk it :).
> >>
> >> Some of the columns with "fictional null values" are used in
> relationships. For example, I can have an invoice with customer_id "-1" to
> indicate that there's no related customer.
> >>
> >> As I'm done cleaning up the DB, these will eventually end up as actual
> foreign keys with nulls instead of that -1, so I'd like to model the
> relationship, hide the FK in the ObjEntity and use the "customer"
> relationship (I've actually modeled it like that already and it works fine
> for read operations, but of course everything explodes once I try to write
> and object with a null customer to the DB and "customer_id" is set to null).
> >>
> >> So, I kind of need to be doing this on the DbEntity level (if customer
> is null, write "-1" to the customer_id).
> >>
> >> Is this at all possible?
> >>
> >> Cheers,
> >> - hugi
> >>
> >>
> >>> On 4 Apr 2019, at 10:09, Hugi Thordarson  wrote:
> >>>
> >>> Hi all,
> >>> I'm currently working on a legacy system. The DB has a lot of columns
> where null should actually be allowed, but instead those fields are
> non-nullable and get other values (empty string, "0", "-1" etc) written to
> them to indicate the absence of a value. Yay.
> >>>
> >>> Unfortunately I can't just do the right thing; make the fields
> nullable and start writing nulls, since that would make other (non Cayenne)
> parts of the system explode, so I have to create a temporary workaround
> while I work through each column and make it nullable and fix (or kill) the
> legacy apps.
> >>>
> >>> What I'd like to do is write my application logic as if the column was
> nullable, setting nulls when I want to, but when the object goes to the DB,
> "legacy nulls" are written to the DB instead (empty string or "-1" or
> whatever).
> >>>
> >>> I have a working solution, I just wanted to check if you guys would
> have a better or more performant solution or see anything potentially wrong
> with what I'm doing.
> >>>
> >>> What I'm doing is creating an interface (HasFictionalNullValues) which
> can be implemented by entity classes that require it. It defines a single
> method that can be implemented like…
> >>>
> >>> public class SomeEntityClass extends _SomeEntityClass implements
> HasFictionalNullValues {
> >>>
> >>> @Override
> >>> Map fictionalNullValues) {
> >>> Map();
> >>> map.put( NAME, "" );
> >>> map.put( AGE, -1 );
> >>> return map;
> >>> }
> >>>
> >>> [...rest of class body...]
> >>> }
> >>>
> >>> …then I add a listener to my DataDomain to catch new and updated
> objects
> >>>
> >>> public class FictionalNullValuesListener {
> >>>
> >>> @PrePersist( { BaseDataObject.class } )
> >>> @PreUpdate( { BaseDataObject.class } )
> >>> public void onPrePersist( BaseDataObject object ) {
> >>> if( object instanceof HasFictionalNullValues ) {
> >>>
>  ((HasFictionalNullValues)object).fictionalNullValues().forEach( (
> property, replacementNullValue ) -> {
> >>> if( property.getFrom( object ) == null ) {
> >>> property.setIn( object,
> replacementNullValue );
> >>> }
> >>> } );
> >>> }
> >>> }
> >>> }
> >>>
> >>> Any comments on this practice or something I'm missing?
> >>>
> >>> Cheers,
> >>> - hugi
> >>
> >
>
>

Re: Timeout long running selects

[John Huss]

Set the statement timeout using the SQL syntax or connection string
property provided by your database.

https://stackoverflow.com/questions/24092463/psql-set-default-statement-timeout-as-a-user-in-postgres#24093305

On Sat, Mar 23, 2019 at 1:15 PM Mark Stobbe  wrote:

> Hi all,
>
> Does anyone have a solution for killing long running SelectQuerys in
> cayenne?
>
> Best regards,
> Mark
>

Re: Increasing memory leads to OOM error

[John Huss]

I would try creating a new ObjectContext every so often, not just
invalidating the objects.

On Tue, Dec 4, 2018 at 2:58 PM Tony Giaccone  wrote:

> We're processing a request and generating about 130,000 new objects. Once
> these objects are created and saved we'll never need to access them again
> in this process.  The heap steadily grows as we do this. Even thought after
> each insert we invalidate the object we just inserted.
>
> What should I do to minimize the amount of memory that's held by Cayenne as
> we do this large number of inserts?   Am I missing something? Are there
> snapshots for each of these objects? If there are is there an way to purge
> those?
>
>
> Tony
>

Re: Get a TransactionFactory without a server runtime

[John Huss]

Usually you would access the injector bound to the thread. I think the API
is CayenneRuntime.bindThreadInjector.
On Fri, Nov 16, 2018 at 6:34 AM Hugi Thordarson  wrote:

> Hi all.
> I have a library containing some utility code. Most of the utility methods
> only accept an ObjectContext as an argument and don't know anything else
> about the environment they're running in. However, some of that code needs
> to run in transactions and for that I need TransactionFactories, normally
> obtained from the ServerRuntime's injector:
>
> someServerRuntime.getInjector().getInstance( TransactionFactory.class )
>
> So, I guess the question boils down to: Is there some way for me to obtain
> a TransactionFactory (or an Injector) from an ObjectContext, without having
> a ServerRuntime?
>
> Cheers,
> - hugi

Re: Iterate thought all queries, apply template, get raw SQL

[John Huss]

On Tue, Nov 13, 2018 at 11:34 AM Alexander Petrossian (PAF) <
alexander.petross...@gmail.com> wrote:

> вт, 13 нояб. 2018 г. в 20:02, Giaccone, Tony  >:
>
> > This is going to be a pretty big task, because of the number of tables no
> > matter what you do. Is the problem that there are no foreign key
> > relationships between your tables?
> >
>
> Yep. This is my primary problem. No doc/FK/relationships in cayenne
> mappings.
> Guys only did hundreds of queries.
>
>
> > If you have the mapping file, you can always open that map with the
> Cayenne
> > Modeler. It with that you can generate all the SQL for all the schema, of
> > your database. You can even create a diagram of the database and the
> > relationships between the tables using the Cayenne Modeler.
>
>
> Can't do that. There are no relationships defined in mapping files ;)
>
>
> > If you have the java classes that the modeler creates, you can also start
> > looking in the java class files for the relationships between the
> objects.
> >
>
> If there were no relationships defined inside mapping files I feel there
> would be nothing defined in code?
>
>
> > Again a daunting task, but certainly possible.  I'm sure you can find a
> > java tool, maybe class-visualizer, to document the relationships between
> > the objects.
> >
>
> I have no access to source code, but one can reverse .class files, I guess.
> Haven't tried that yet. Thanks for the pointer.
>
>
> > If you don't have a mapping file with classes, but and just have a huge
> map
> > with many sql statements in it, you could always pull them out of the map
> > directly.
> >
>
> Tony, as I've indicated, I can give SQL statements to analyser (part of
> jailer, done with JSQLParser project).
> But I have hard time fighting with SQL templates.
> Maybe there is some easy [ ;) ] way to ask Cayenne to explode all those
> templates for me?
> remove #result, solve all #chain and #chunk and do #ifs (all via true side)
> and anything else I forgot here.
> I need "some valid SQL" queries.
>

I think your best path forward would be to use an XML parser to pull out
SQL Templates from the .map.xml and then use Regular Expressions to replace
the Velocity template commands (#if, etc) with whatever you want.


>
> Alexander
>
> > https://github.com/JSQLParser/JSqlParser/issues/710
> > https://sourceforge.net/p/jailer/feature-requests/52/
>
>
>
>
> > Tony
> >
> > On Tue, Nov 13, 2018 at 3:25 AM Alexander Petrossian (PAF) <
> > alexander.petross...@gmail.com> wrote:
> >
> > > Friends, please share your vision.
> > >
> > > I got a project with lots of Apache Cayenne mappings (2.0), and
> database
> > is
> > > not documented at all.
> > >
> > > All insights about links between tables (about 2 thousands of tables) I
> > can
> > > get is only from queries found in Cayenne mappings (several megabytes
> of
> > > those).
> > >
> > > Those tables I can quick analyse with JSQLParser. But it needs raw
> SQLs.
> > >
> > > Question: Is there some easy way to programmatically iterate through
> all
> > > queries and apply templates to get those raw SQLs?
> > >
> > > (Any hints would be very appreciated!)
> > >
> > >
> > >
> > > https://github.com/JSQLParser/JSqlParser/issues/710
> > >
> > > https://sourceforge.net/p/jailer/feature-requests/52/
> > >
> > > --
> > > PAF
> > >
> > > P.S. tried to discuss it here first:
> > > https://jira.apache.org/jira/browse/CAY-2495
> > >
> >
>
>
> --
> PAF
>

Re: Iterate thought all queries, apply template, get raw SQL

[John Huss]

I'm not really sure what you want.

The DB should have foreign keys to define the related tables.

Similarly, the Cayenne .map.xml file should have relationships defined
between tables to indicate their relationships (via foreign keys).

If you have unit tests, running those with SQL logging enabled in your
database might help. Cayenne can also log the SQL it is generating during
runtime, which you can enable with a log4j property like this:

log4j.logger.org.apache.cayenne.log.CommonsJdbcEventLogger=INFO


On Tue, Nov 13, 2018 at 2:25 AM Alexander Petrossian (PAF) <
alexander.petross...@gmail.com> wrote:

> Friends, please share your vision.
>
> I got a project with lots of Apache Cayenne mappings (2.0), and database is
> not documented at all.
>
> All insights about links between tables (about 2 thousands of tables) I can
> get is only from queries found in Cayenne mappings (several megabytes of
> those).
>
> Those tables I can quick analyse with JSQLParser. But it needs raw SQLs.
>
> Question: Is there some easy way to programmatically iterate through all
> queries and apply templates to get those raw SQLs?
>
> (Any hints would be very appreciated!)
>
>
>
> https://github.com/JSQLParser/JSqlParser/issues/710
>
> https://sourceforge.net/p/jailer/feature-requests/52/
>
> --
> PAF
>
> P.S. tried to discuss it here first:
> https://jira.apache.org/jira/browse/CAY-2495
>

Re: Controlling object insert order

[John Huss]

Without having more details my general recommendation would be to handle
the details up front before you get to the Cayenne level. Put things in
order in memory and then create your Cayenne Objects and commit them one at
a time.

On Thu, Nov 1, 2018 at 9:35 AM Hugi Thordarson  wrote:

> Hi all.
> I have a set of new (uncommitted) objects that I need to insert into the
> DB in a certain order. For… Reasons. (let's hear it for legacy DBs with
> meaningful PKs, yay!)
>
> The action is not performance sensitive and the whole thing occurs within
> a transaction, so performing multiple commits is OK. My initial though was
> that when saving, I could just iterate through the new objects, localizing
> each one in a new ObjectContext and committing that on each iteration—but I
> haven't found a way to get that to work.
>
> Any ideas?
>
> Cheers,
> - hugi

Re: Commit Exception

[John Huss]

That's a database error, not an ORM thing. And it's database specific. And
all the database gives you is the error string. So you HAVE to parse it if
that's what you want. The less hacky approach would be to detect the
prevent the duplicate key error upfront with custom logic.

On Fri, Sep 28, 2018 at 9:28 AM Tony Giaccone  wrote:

> Yeah, that's pretty much what I ended up doing. Even reading the file line
> by line and doing an insert after each object is created only made the run
> time go to 4 minutes and I can live with that. What I really wanted to do
> was find a way to recover from a larger commit. It seems that's not really
> possible. The one feature that would make that failure easier to deal with
> would be some kind of data value in commit error that would identify the
> class and key value of the object that caused the commit exception. I
> recognize that the value is there in the text, but parsing through that
> text message to find the value is a serious hack. It would be better if the
> framework included in the commit exception, the class type and the key
> value of the entity that caused the problem.
>
> Now maybe in the larger scheme of things, it doesn't make sense to identify
> which item in the set of items being committed caused the problem. It's
> clear it makes sense in my use case, but in the general use case, maybe
> not..
>
>
> Tony
>
> On Thu, Sep 27, 2018 at 5:10 PM John Huss  wrote:
>
> > Commit the ObjectContext after each object/row and rollback the
> > ObjectContext on failure.
> >
> > On Thu, Sep 27, 2018 at 3:57 PM Tony Giaccone  wrote:
> >
> > > So the question isn't as much about who to manage the transaction. It's
> > > more about how to recover and eliminate the offending object so that
> the
> > > commit can be made again.
> > >
> > > On Thu, Sep 27, 2018 at 3:52 PM John Huss  wrote:
> > >
> > > > I'd just wrap the whole thing in a database transaction. Then commit
> > your
> > > > ObjectContexts as often as you want to, but the real DB commit won't
> > > happen
> > > > until the end.
> > > >
> > > > TransactionManager transactionManager =
> > > CayenneRuntime.*getThreadInjector*
> > > > ().getInstance(TransactionManager.*class*);
> > > >
> > > > transactionManager.performInTransaction(*new*
> > > > TransactionalOperation() {
> > > >
> > > > @Override
> > > >
> > > > *public* Void perform() {
> > > >
> > > > *return* *null*;
> > > >
> > > > }
> > > >
> > > > });
> > > >
> > > >
> > > >
> > > > On Thu, Sep 27, 2018 at 2:36 PM Tony Giaccone 
> > wrote:
> > > >
> > > > > I'm processing a large number of rows, over 600,000 and the key
> value
> > > > > should be unique in this file but I'd like to ensure that. I also
> > want
> > > > this
> > > > > to happen with some rapidity.  To speed this process upI'm going to
> > > read
> > > > > lines from the file, create objects and commit the changes after
> 500
> > > have
> > > > > been created.
> > > > >
> > > > > The problem with this is that if I have a duplicate value I won't
> > catch
> > > > it
> > > > > till I do the commit.
> > > > >
> > > > > When I insert a second key value the first exception is a db level
> :
> > > > > org.postgresql.util.PSQLException
> > > > >
> > > > > eventually this gets wrapped by a Cayenne Commit error.
> > > > >
> > > > > So I'd like to get a sense of what folks think. Given that I want
> to
> > > > > balance these conflicting goals of speed and accuracy.
> > > > >
> > > > > Can I easily figure out what object or objects caused the error and
> > > can I
> > > > > exclude them from the context and redo the commit? f
> > > > >
> > > > > Is this a reasonable path to follow.
> > > > >
> > > > >
> > > > >
> > > > > Tony Giaccone
> > > > >
> > > >
> > >
> >
>

Re: Commit Exception

[John Huss]

Commit the ObjectContext after each object/row and rollback the
ObjectContext on failure.

On Thu, Sep 27, 2018 at 3:57 PM Tony Giaccone  wrote:

> So the question isn't as much about who to manage the transaction. It's
> more about how to recover and eliminate the offending object so that the
> commit can be made again.
>
> On Thu, Sep 27, 2018 at 3:52 PM John Huss  wrote:
>
> > I'd just wrap the whole thing in a database transaction. Then commit your
> > ObjectContexts as often as you want to, but the real DB commit won't
> happen
> > until the end.
> >
> > TransactionManager transactionManager =
> CayenneRuntime.*getThreadInjector*
> > ().getInstance(TransactionManager.*class*);
> >
> > transactionManager.performInTransaction(*new*
> > TransactionalOperation() {
> >
> > @Override
> >
> > *public* Void perform() {
> >
> > *return* *null*;
> >
> > }
> >
> > });
> >
> >
> >
> > On Thu, Sep 27, 2018 at 2:36 PM Tony Giaccone  wrote:
> >
> > > I'm processing a large number of rows, over 600,000 and the key value
> > > should be unique in this file but I'd like to ensure that. I also want
> > this
> > > to happen with some rapidity.  To speed this process upI'm going to
> read
> > > lines from the file, create objects and commit the changes after 500
> have
> > > been created.
> > >
> > > The problem with this is that if I have a duplicate value I won't catch
> > it
> > > till I do the commit.
> > >
> > > When I insert a second key value the first exception is a db level :
> > > org.postgresql.util.PSQLException
> > >
> > > eventually this gets wrapped by a Cayenne Commit error.
> > >
> > > So I'd like to get a sense of what folks think. Given that I want to
> > > balance these conflicting goals of speed and accuracy.
> > >
> > > Can I easily figure out what object or objects caused the error and
> can I
> > > exclude them from the context and redo the commit? f
> > >
> > > Is this a reasonable path to follow.
> > >
> > >
> > >
> > > Tony Giaccone
> > >
> >
>

Re: Commit Exception

[John Huss]

I'd just wrap the whole thing in a database transaction. Then commit your
ObjectContexts as often as you want to, but the real DB commit won't happen
until the end.

TransactionManager transactionManager = CayenneRuntime.*getThreadInjector*
().getInstance(TransactionManager.*class*);

transactionManager.performInTransaction(*new*
TransactionalOperation() {

@Override

*public* Void perform() {

*return* *null*;

}

});



On Thu, Sep 27, 2018 at 2:36 PM Tony Giaccone  wrote:

> I'm processing a large number of rows, over 600,000 and the key value
> should be unique in this file but I'd like to ensure that. I also want this
> to happen with some rapidity.  To speed this process upI'm going to read
> lines from the file, create objects and commit the changes after 500 have
> been created.
>
> The problem with this is that if I have a duplicate value I won't catch it
> till I do the commit.
>
> When I insert a second key value the first exception is a db level :
> org.postgresql.util.PSQLException
>
> eventually this gets wrapped by a Cayenne Commit error.
>
> So I'd like to get a sense of what folks think. Given that I want to
> balance these conflicting goals of speed and accuracy.
>
> Can I easily figure out what object or objects caused the error and can I
> exclude them from the context and redo the commit? f
>
> Is this a reasonable path to follow.
>
>
>
> Tony Giaccone
>

Re: DataObject changes - before commit

[John Huss]

Or I suppose you could create fields for each of the original values in the
same way that the current values are stored.

On Wed, Sep 12, 2018 at 4:32 PM John Huss  wrote:

> Cayenne stores the original values, but they are in DbEntity
> representation rather than ObjEntity representation which makes comparing
> the two difficult. Without writing a lot of code to convert and compare
> your best bet would be to use a more efficient data structure without any
> extra unneeded space. That would be an array (Object[]) that is smart
> enough to map a fixed set of string keys into array indexes. To do this
> you'll need a HashMap per ObjEntity(table) that maps from
> string key to array index. Then all of your CayenneDataObjects will need an
> Object[] with the original values.
>
> On Tue, Sep 11, 2018 at 10:58 AM Matt R Watson  wrote:
>
>> Thanks mrg.
>>
>> I am aware that Cayenne knows about the original values, but trying to
>> find an efficient way to access them. Or at least more efficient than the
>> Map we are currently using, which is heavy on memory yet quick on access
>> time (and easy to read code). I’ve seen your utilities in the past when
>> searching for a solution for this, but I don’t see anything thats
>> comparable to finding the original value of a specific property on a
>> specific object. I also don’t see where you are grabbing the original value
>> of a relationship. I know the snapshot can show me the original DataRow
>> values, but I don’t think those are always the same “type” as the actual
>> DataObject properties.
>>
>> Still hoping for solution for our current code (that doesn’t have to loop
>> over all modified objects, every time):
>>
>> inventoryAdjustment.isPropertyModified(“date”)
>> inventoryAdjustment.getOriginalProperty(“date”)
>>
>> with something like:
>>
>> Cayenne.isPropertyModified(inventoryAdjustment, "date”)
>> Cayenne.getOriginalProperty(inventoryAdjustment, "date”)
>>
>> Thanks,
>> Matt
>>
>> > On Sep 11, 2018, at 5:31 AM, Michael Gentry 
>> wrote:
>> >
>> > Hi Matt,
>> >
>> > Cayenne already keeps track of the original values for you.  This is
>> needed
>> > in order to do optimistic locking, plus it allows Cayenne to only send
>> over
>> > changes in a record instead of the entire record when you are doing an
>> > UPDATE.
>> >
>> > I haven't tried this approach on the field-based version of Cayenne yet,
>> > but these two utility methods are similar and illustrate how to find the
>> > underlying values and detect the changes, at least in 3.x:
>> >
>> >
>> https://gist.github.com/mrg/4dce22b67175c27f4047#file-cayenneutils-java-L124
>> >
>> >
>> https://gist.github.com/mrg/4dce22b67175c27f4047#file-cayenneutils-java-L184
>> >
>> > I believe you could adapt for your purposes/needs pretty readily.
>> >
>> > mrg
>> >
>> >
>> > On Mon, Sep 10, 2018 at 8:04 PM Matt R Watson 
>> wrote:
>> >
>> >> Now that the DataObject classes are “field based”, I would like to
>> find a
>> >> way to replace the heavy Map we have placed on all of our DataObject
>> >> classes.
>> >>
>> >> We extend Cayenne’s default class and added a HashMap to keep track of
>> the
>> >> original values (as it changes) since the object has been loaded and
>> before
>> >> committing. (see our class below)
>> >>
>> >> This allows us to check if something is modified (
>> >> inventoryAdjustment.isPropertyModified(“date”), or
>> >> inventoryAdjustment.getOriginalProperty(“date”) )
>> >>
>> >> I’d like to find a Cayenne friendly way of doing this. Any suggestions?
>> >>
>> >> Thanks,
>> >> Matt
>> >>
>> >> 
>> >>
>> >>
>> >> public class CayenneBaseDataObject extends CayenneDataObject {
>> >>
>> >>/**
>> >> * Stores the original values for this object's properties. Uses
>> >> the
>> >> * property name string as the key for each entry. Its purpose
>> is
>> >> to keep
>> >> * track of any original values since the object was initialized
>> >> or last
>> >> * committed.
>> >> */
>> >>private final Map originalPropertyMap = New.map();
>> >>
>&

Re: DataObject changes - before commit

[John Huss]

Cayenne stores the original values, but they are in DbEntity representation
rather than ObjEntity representation which makes comparing the two
difficult. Without writing a lot of code to convert and compare your best
bet would be to use a more efficient data structure without any extra
unneeded space. That would be an array (Object[]) that is smart enough to
map a fixed set of string keys into array indexes. To do this you'll need a
HashMap per ObjEntity(table) that maps from string key to
array index. Then all of your CayenneDataObjects will need an Object[] with
the original values.

On Tue, Sep 11, 2018 at 10:58 AM Matt R Watson  wrote:

> Thanks mrg.
>
> I am aware that Cayenne knows about the original values, but trying to
> find an efficient way to access them. Or at least more efficient than the
> Map we are currently using, which is heavy on memory yet quick on access
> time (and easy to read code). I’ve seen your utilities in the past when
> searching for a solution for this, but I don’t see anything thats
> comparable to finding the original value of a specific property on a
> specific object. I also don’t see where you are grabbing the original value
> of a relationship. I know the snapshot can show me the original DataRow
> values, but I don’t think those are always the same “type” as the actual
> DataObject properties.
>
> Still hoping for solution for our current code (that doesn’t have to loop
> over all modified objects, every time):
>
> inventoryAdjustment.isPropertyModified(“date”)
> inventoryAdjustment.getOriginalProperty(“date”)
>
> with something like:
>
> Cayenne.isPropertyModified(inventoryAdjustment, "date”)
> Cayenne.getOriginalProperty(inventoryAdjustment, "date”)
>
> Thanks,
> Matt
>
> > On Sep 11, 2018, at 5:31 AM, Michael Gentry  wrote:
> >
> > Hi Matt,
> >
> > Cayenne already keeps track of the original values for you.  This is
> needed
> > in order to do optimistic locking, plus it allows Cayenne to only send
> over
> > changes in a record instead of the entire record when you are doing an
> > UPDATE.
> >
> > I haven't tried this approach on the field-based version of Cayenne yet,
> > but these two utility methods are similar and illustrate how to find the
> > underlying values and detect the changes, at least in 3.x:
> >
> >
> https://gist.github.com/mrg/4dce22b67175c27f4047#file-cayenneutils-java-L124
> >
> >
> https://gist.github.com/mrg/4dce22b67175c27f4047#file-cayenneutils-java-L184
> >
> > I believe you could adapt for your purposes/needs pretty readily.
> >
> > mrg
> >
> >
> > On Mon, Sep 10, 2018 at 8:04 PM Matt R Watson  wrote:
> >
> >> Now that the DataObject classes are “field based”, I would like to find
> a
> >> way to replace the heavy Map we have placed on all of our DataObject
> >> classes.
> >>
> >> We extend Cayenne’s default class and added a HashMap to keep track of
> the
> >> original values (as it changes) since the object has been loaded and
> before
> >> committing. (see our class below)
> >>
> >> This allows us to check if something is modified (
> >> inventoryAdjustment.isPropertyModified(“date”), or
> >> inventoryAdjustment.getOriginalProperty(“date”) )
> >>
> >> I’d like to find a Cayenne friendly way of doing this. Any suggestions?
> >>
> >> Thanks,
> >> Matt
> >>
> >> 
> >>
> >>
> >> public class CayenneBaseDataObject extends CayenneDataObject {
> >>
> >>/**
> >> * Stores the original values for this object's properties. Uses
> >> the
> >> * property name string as the key for each entry. Its purpose is
> >> to keep
> >> * track of any original values since the object was initialized
> >> or last
> >> * committed.
> >> */
> >>private final Map originalPropertyMap = New.map();
> >>
> >>/**
> >> * Initializes the originalPropertyMap. Used as a callback during
> >> the
> >> */
> >>public void initializeOriginalPropertyMap() {
> >>originalPropertyMap.clear();
> >>}
> >>
> >>@Override
> >>protected void beforePropertyWrite(String propName, Object oldValue,
> >> Object newValue) {
> >>if (!originalPropertyMap.containsKey(propName)) {
> >>originalPropertyMap.put(propName, oldValue);
> >>}
> >>
> >>super.beforePropertyWrite(propName, oldValue, newValue);
> >>}
> >>
> >>/**
> >> * Overrides
> >> * {@link org.apache.cayenne.DataObject#writeProperty(String,
> >> Object)} by
> >> * saving a {@link CayenneBaseDataObject}'s existing property
> >> value, if any,
> >> * to the {@link #originalPropertyMap} prior to updating the
> >> * {@link CayenneBaseDataObject}'s property with a new value.
> >> *
> >> * @param relationship
> >> *  the name of the to-one-relationship
> >> * @param value
> >> *  the 

Re: Problem with Callback not getting called in parent/child setup

[John Huss]

Are you taking about parent/child ObjectContexts? I see you're calling
commitChangesToParent not commitChanges. Does it work when you call
commitChanges instead?

Here's the code I have for the callbacks:

callbackRegistry =
runtime.getDataDomain().getEntityResolver().getCallbackRegistry();

callbackRegistry.addCallback(LifecycleEvent.PRE_UPDATE,
ICSDataObject.class, "willUpdate");

public void willUpdate() {
}

On Tue, Aug 7, 2018 at 12:47 PM Lon Varscsak  wrote:

> Hey all,
>
> I've got a data object who has a preUpdate callback so that I can set
> required values (like a timestamp) before update.  According to the docs
> this should work in the parent-child setup, but that doesn't seem to be the
> case.  Thoughts?  Here's a snippet of the stacktrace:
>
> Validation failure for
> com.smarthealth.businesslogic.personalization.ArchiveImage: 'timestamp' is
> a required value.
>
> Validation failure for
> com.smarthealth.businesslogic.personalization.ArchiveImage.timestamp:
> "timestamp"  is required.
>
> Validation failure for
> com.smarthealth.businesslogic.personalization.ArchiveImage: 'timestamp' is
> a required value.
>
> at org.apache.cayenne.access.ObjectStoreGraphDiff.validateAndCheckNoop(
> ObjectStoreGraphDiff.java:113)
>
> at
> org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:734)
>
> at org.apache.cayenne.access.DataContext.commitChangesToParent(
> DataContext.java:682)
>
> at com.smarthealth.wicket.designpal.components.EditorPage.save(
> EditorPage.java:127)
>
>
> The preUpdate callback is never fired.
>
> Thanks,
>
> Lon
>

Re: SelectById vs SQLSelect cache behavior

[John Huss]

If you want fresh data you shouldn't use .localCache() or you should pass a
cache strategy to refresh the cache.

You can also force objects to be refreshed on the next access or query by
using ObjectContext.invalidateObjects.

On Tue, Jul 31, 2018 at 6:35 PM kw...@tybera.com  wrote:

> I have a situation where multiple applications can make changes to the
> database, and I need to be able pull data from the database and not use
> cached data.
>
> We were running with cayenne 3.1M3 but I found documentation that
> indicated that SelectById may be better suited to do what I wanted than
> Cayenne.objectForPK which we were using - but I needed to update.  So I
> downloaded version 4.0.RC1 and changed this:
> Employee  existingUserInfo = Cayenne.objectForPK(context,
> Employee.class, userInfo.getId());
> to this:
> ObjectContext context = cayenneRuntime.getContext();
> Employee  existingUserInfo = SelectById.query(Employee.class,
> userInfo.getId()).localCache().selectOne(context);
>
> This appears to have solved my problem.  However, I found that where I was
> calling SQLTemplate to get the Employee, that I am not getting the latest
> information because of the cache.  I switched these to use SQLSelect rather
> than SQLTemplate hoping that SQLSelect would act like SelectById, but it is
> not.
>
> Here is what I put in to try to test this:
> String sql = "SELECT * from Employee "
> + " WHERE login_Id =
> #bind($loginId) "
> + " and historical_Date is null ";
>
> ObjectContext context =
> cayenneRuntime.getContext();
> Employee emp = SQLSelect.query(Employee.class,
> sql).params("loginId", userName).localCache().selectOne(context);
> if (emp == null) return null;
> System.out.println("emp middleName=" +
> emp.getMiddleName());
> Employee emp2 = SelectById.query(Employee.class,
> emp.getObjectId()).localCache().selectOne(context);
> System.out.println("emp2 middleName=" +
> emp.getMiddleName());
>
> I access the Employee in my application, and then I change the middle name
> of the employee in the database.  The first call uses SQLSelect to get
> Employee emp based on their loginId.   Then I use SelectById to get the
> same Employee in emp2 based on the objectId for emp.  The calls follow the
> same pattern of specifying localCache and selectOne with the same context.
> However SQLSelect is not picking up the changed middle name, while
> SelectById does.
>
> What is the best approach to handling this?  I see that SQLSelect is a
> replacement for SQLTemplate with the new fluid approach, but I am hoping
> that I can switch to ObjectSelect and it will behave as SelectById.  I also
> wonder if I should be using ObjectContext context = runtime.newContext();
> rather than runtime.getContext();
>
> I am planning on updating to the new 4.1.M2 release.  I don't know if this
> gives me more/better options.
>
>
>
>
>
>

Re: Redacting db user name and password from XML

[John Huss]

On Wed, Jul 18, 2018 at 1:26 PM Meeks, Andrew  wrote:

> I feel as though a recent bit of code is closely related to the topic in
> this old thread.   The Cayenne  xml element supports a
> password with a path external to the application.  The observed environment
> properties appear to only include JDBC_PASSWORD_PROPERTY which seems to
> only support a literal password.  For Spring developers, an integration
> suggestion is included below.
>
> I would like to say, in agreement with the earlier thread, that the
> documentation at:
> https://cayenne.apache.org/docs/4.1/cayenne-guide/#appendix-a-configuration-properties
> is unclear in one respect.  I think it needs to state that all
> cayenne.jdbc.* properties must be defined in order for environment
> properties to override (or correctly override) the definition of
> data-source in the xml file.  That would have saved me a lot of stepping
> into code before I found what was happening in DelegatingDataSourceFactory.
> shouldConfigureDataSourceFromProperties(DataNodeDescriptor
> dataNodeDescriptor);
>

I believe only the URL and driver have to be specified via properties in
order for them to override the values in the cayenne project file. This is
still not really intuitive, but it's better than all of them being required.


>
> If you have a Spring Project and would like to control your 
> through a more spring-like configuration, here are relevant snippets from
> files of how I set it up based on examples in the cayenne documentation:
>
> -- file: application.properties --
> cayenne.jdbc.driver=oracle.jdbc.driver.OracleDriver
> cayenne.jdbc.max_connections=20
> cayenne.jdbc.min_connections=1
> cayenne.jdbc.username=SCOTT
>
> --file: application-dev.properties--
>
> cayenne.jdbc.passwordSource=C:\\home\\apps-config\\testProject\\cayenne_pw.txt
> cayenne.jdbc.url=jdbc:oracle:thin:@//localhost:1521/xe
>
> --file: application-prod.properties--
>
> cayenne.jdbc.passwordSource=C:\\home\\apps-config\\testProject\\cayenne_pw_prod.txt
> cayenne.jdbc.url=jdbc:oracle:thin:@//prod.db.example.com:1521/prod_svc
>
> --file: SpringMvcConfig.java--
> @Configuration
> public class MvcConfig extends WebMvcConfigurerAdapter {
> @Value("${cayenne.jdbc.url}")
> String cayenneClientUrl;
>
> @Value("${cayenne.jdbc.driver}")
> String cayenneDriverProperty;
>
> @Value("${cayenne.jdbc.passwordSource}")
> String cayennePasswordSource;
>
> @Value("${cayenne.jdbc.username}")
> String cayenneUsername;
>
> @Value("${cayenne.jdbc.min_connections}")
> String cayenneMinConnections;
>
> @Value("${cayenne.jdbc.max_connections}")
> String cayenneMaxConnections;
>
> @Bean(name="serverRuntime")
> public ServerRuntime serverRuntime(){
>
> String url = cayenneClientUrl;
> String driver = cayenneDriverProperty;
> String username = cayenneUsername;
> String pwSource = cayennePasswordSource;
> String maxConnections = cayenneMaxConnections;
> String minConnections = cayenneMinConnections;
>
> String actualPassword = getPassword(pwSource);
>
> Module cayenneModule = binder ->
>
> ServerModule.contributeProperties(binder).put(Constants.JDBC_DRIVER_PROPERTY,
> driver).put(Constants.JDBC_URL_PROPERTY, url)
>   .put(Constants.JDBC_USERNAME_PROPERTY,
> username).put(Constants.JDBC_PASSWORD_PROPERTY, actualPassword)
>   .put(Constants.JDBC_MIN_CONNECTIONS_PROPERTY,
> minConnections).put(Constants.JDBC_MAX_CONNECTIONS_PROPERTY,
> maxConnections);
>
> ServerRuntimeBuilder builder = ServerRuntime.builder();
> builder.addConfig("cayenne-testProject.xml");
> builder.addModule(cayenneModule);
> ServerRuntime createdRuntime = builder.build();
>
> try {
> // this will tell us on startup if it cannot
> connect to the database
> createdRuntime.getDataSource().getConnection();
> } catch (SQLException e) {
> e.printStackTrace();
> }
>
> return createdRuntime;
> }
>
> private String getPassword(String passwordSourcePath) {
> try {
> return new
> String(Files.readAllBytes(Paths.get(passwordSourcePath)));
> } catch (IOException e) {
> return null;
> }
> }
> }
>
> --End of code snippets--
> The primary advantage with this is that server admins can play around with
> the database parameters and developers will not need to build with a new
> cayenne xml, while still leaving the database password in an external
> file.  Once each Spring profile has its default values, then the server
> admins need only specify a parameter rather 

Re: Accessing a sequence number in java

[John Huss]

long myId = ((Number)SQLSelect.dataRowQuery("select nextval('mytable_seq'
as id").selectOne(ec).get("id")).longValue();


On Wed, May 23, 2018 at 3:52 PM Tony Giaccone  wrote:

> Imagine I have a field in a persistent object, the field is not the primary
> key, and I want it to be unique.  So the obvious choice to me is to use a
> sequence value from the db, so that I can avoid trying to calculate a value
> in java.
>
> Is there an easy way to access that sequence value, from java, using
> Cayenne?
>
>
> Tony
>

Re: Cayenne firing faults when it shouldn’t be (or at least I don’t think it should be)

[John Huss]

Yes

On Sat, May 12, 2018 at 7:12 AM Lon Varscsak  wrote:

> I have a situation where I fetched a large amount of objects (50k) and
> prefetched a bunch of relationships…this all worked as expected (although I
> didn’t realize that this set of data was that large at the time)…but what
> caught my eye is that even though it pre-fetched the relationships it was
> firing some of the those same relationship’s faults.  I _think_ they were
> all to-one (not sure that matters), but something that I thought of.
>
> Is faulting in any way tied to object cache and since the result set is so
> large that some things have fallen out of cache and get re-faulted?
>
> -Lon
>

Re: Cayenne context inside weapplication

[John Huss]

On Sun, Apr 15, 2018 at 12:47 PM JK  wrote:

> Hello,
>
> I have a bit of confusion around how Cayenne contex works inside web
> application.
> I dont really need to have session scoped context- however, there is a
> specific need to have isolated scopes .
>
> 1) What is the best way to manage Cayenne context inside a web application
> ? Is creating a new context for every http request a good practice ? any
> known performance degradation ?

You need the runtime (or injector) to create contexts, so you have to get
it from somewhere, either the servletcontext or a static variable. Creating
a context for each request is typical and performs well.


> 2) Is CayenneFilter good enough for this purpose ?

Typically, yes


> 3) My application requires isolated scopes and I do create multiple child
> context ( but only 1 level ). What happens to the child context  . What
> time do they get removed from memory ?or do I need to force release these
> somehow ?
>
They get garbage collected like everything else, there isn't manual
attention needed except managing cache size if you are caching.

>
>

Re: Staged saves pattern ?

[John Huss]

is
> would
> >>>>   be how I'd try to solve it.
> >>>>
> >>>>
> >>>>   On Wed, Mar 28, 2018 at 9:29 PM, Ken Anderson <k...@anderhome.com>
> wrote:
> >>>>> Hugi,
> >>>>>
> >>>>> That’s correct - it’s not like we’re just importing a bunch of
> records.  Each row in the file could affect the same set of objects.
> >>>>>
> >>>>> So, we did the child context, but obviously if we created an object
> in a prior child and then saved it to the parent, we won’t be able to
> easily find it in the next child context.  If you get a localObject in your
> new child context, it is “hollow”, so not connected to all the other
> objects floating around in the parent.  We also can’t fire relationships,
> because those relationships will go to the database instead of the parent
> context.
> >>>>>
> >>>>> Ken
> >>>>>
> >>>>>> On Mar 28, 2018, at 7:11 PM, Hugi Thordarson <h...@karlmenn.is>
> wrote:
> >>>>>>
> >>>>>>> That's exactly what we want to do - save once at the end.
> However, we have 2 problems:
> >>>>>>>
> >>>>>>> 1. How do we find the objects that we already created but haven't
> saved yet
> >>>>>>
> >>>>>> You can go through yur ObjectContext's newObjects() and filter that
> to your liking—pretty much the same as you'd do with EOF.
> >>>>>>
> >>>>>>> 2. How do we roll back each line if there's an error?  Not a DB
> error, but the logic gets so far, and then determines that there's no way
> to continue so we must skip this line.
> >>>>>>
> >>>>>> As you tried yourself, I'd use a child context and commit to the
> parent once you're sure everything is in place. Can you explain further
> what was problematic with that (that you need to "access the same objects
> multiple times")? Do you mean that each row of the file is in some way
> looking at data from other rows?
> >>>>>>
> >>>>>> Cheers,
> >>>>>> - hugi
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> Ken
> >>>>>>>
> >>>>>>> On 3/28/18, 6:07 PM, "John Huss" <johnth...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> Well, you could just save once at the end.  Why do you need to save
> >>>>>>> multiple times during the processing?  Validation exceptions and
> Optimistic
> >>>>>>> Locking errors could be handled in the save with some custom logic
> and a
> >>>>>>> retry.
> >>>>>>>
> >>>>>>> Or if this isn't a super long process you can use a database
> transaction to
> >>>>>>> allow saving multiple times without actually having that data be
> visible
> >>>>>>> outside of the transaction.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Mar 28, 2018 at 6:56 AM Ken Anderson <
> ken.ander...@amphorainc.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> All,
> >>>>>>>>
> >>>>>>>> We have a process that reads in a file and, for each line,
> creates or
> >>>>>>>> edits objects in the object graph.  We only want to commit to the
> database
> >>>>>>>> once at the end.
> >>>>>>>>
> >>>>>>>> We have a finite set of lines, so memory is not an issue.  We
> need to save
> >>>>>>>> only once because saving will actually fire triggers that will
> start doing
> >>>>>>>> other things to the database, which will then lead to optimistic
> lock
> >>>>>>>> exceptions for us if we have data that overlaps (which we do).
> >>>>>>>>
> >>>>>>>> Please don’t suggest we change how the trigger pattern works –
> it’s a big
> >>>>>>>> system and we don’t have control over it.
> >>>>>>>>
> >>>>>>>> So, what we’ve toyed with is using a parent/child context
> arrangement,
> >>>>>>>> where each line is processed in a child, and assuming everything
> goes OK,
> >>>>>>>> we commit only to the parent.  This works well as long as we
> don’t need to
> >>>>>>>> access the same objects multiple times, but unfortunately, we
> do.  We can
> >>>>>>>> reach into the parent context’s unsaved objects, but those
> objects do not
> >>>>>>>> have any relationships since they were built in the child
> context.  This
> >>>>>>>> makes things painful.
> >>>>>>>>
> >>>>>>>> In EOF, I might consider using a single context and undo, but it
> doesn’t
> >>>>>>>> seem like Cayenne has this kind of functionality.
> >>>>>>>>
> >>>>>>>> Thoughts?  Suggestions?  In EOF, I had once written a layer that
> >>>>>>>> intercepted all queries and tried to find the correct object in
> unsaved
> >>>>>>>> objects, but I don’t have nearly enough experience with Cayenne
> to do that.
> >>>>>>>>
> >>>>>>>> Thanks!
> >>>>>>>> Ken
> >>>>>>>>
> >>>>>>>> Confidentiality Notice: This e-mail and accompanying documents
> contain
> >>>>>>>> confidential information intended for a specific individual and
> purpose.
> >>>>>>>> This e-mailed information is private and protected by law. If you
> are not
> >>>>>>>> the intended recipient, you are hereby notified that any
> disclosure,
> >>>>>>>> copying, or distribution, or the taking of any action based on
> the contents
> >>>>>>>> of this information, is strictly prohibited.
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Confidentiality Notice: This e-mail and accompanying documents
> contain confidential information intended for a specific individual and
> purpose. This e-mailed information is private and protected by law. If you
> are not the intended recipient, you are hereby notified that any
> disclosure, copying, or distribution, or the taking of any action based on
> the contents of this information, is strictly prohibited.
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>> Confidentiality Notice: This e-mail and accompanying documents
> contain confidential information intended for a specific individual and
> purpose. This e-mailed information is private and protected by law. If you
> are not the intended recipient, you are hereby notified that any
> disclosure, copying, or distribution, or the taking of any action based on
> the contents of this information, is strictly prohibited.
> >>
>
>

Re: Staged saves pattern ?

[John Huss]

Well, you could just save once at the end.  Why do you need to save
multiple times during the processing?  Validation exceptions and Optimistic
Locking errors could be handled in the save with some custom logic and a
retry.

Or if this isn't a super long process you can use a database transaction to
allow saving multiple times without actually having that data be visible
outside of the transaction.


On Wed, Mar 28, 2018 at 6:56 AM Ken Anderson 
wrote:

> All,
>
> We have a process that reads in a file and, for each line, creates or
> edits objects in the object graph.  We only want to commit to the database
> once at the end.
>
> We have a finite set of lines, so memory is not an issue.  We need to save
> only once because saving will actually fire triggers that will start doing
> other things to the database, which will then lead to optimistic lock
> exceptions for us if we have data that overlaps (which we do).
>
> Please don’t suggest we change how the trigger pattern works – it’s a big
> system and we don’t have control over it.
>
> So, what we’ve toyed with is using a parent/child context arrangement,
> where each line is processed in a child, and assuming everything goes OK,
> we commit only to the parent.  This works well as long as we don’t need to
> access the same objects multiple times, but unfortunately, we do.  We can
> reach into the parent context’s unsaved objects, but those objects do not
> have any relationships since they were built in the child context.  This
> makes things painful.
>
> In EOF, I might consider using a single context and undo, but it doesn’t
> seem like Cayenne has this kind of functionality.
>
> Thoughts?  Suggestions?  In EOF, I had once written a layer that
> intercepted all queries and tried to find the correct object in unsaved
> objects, but I don’t have nearly enough experience with Cayenne to do that.
>
> Thanks!
> Ken
>
> Confidentiality Notice: This e-mail and accompanying documents contain
> confidential information intended for a specific individual and purpose.
> This e-mailed information is private and protected by law. If you are not
> the intended recipient, you are hereby notified that any disclosure,
> copying, or distribution, or the taking of any action based on the contents
> of this information, is strictly prohibited.
>

Re: About Improved Transaction API on 4.0

[John Huss]

To rollback you have to use the more explicit API:

TransactionFactory txFactory =
CayenneRuntime.getThreadInjector().getInstance(TransactionFactory.class);
Transaction tx = txFactory.createTransaction();
tx.begin();
try {
// do stuff

context.commitChanges();
tx.commit();
} catch (Exception e) {
tx.rollback();
throw e;
}




On Tue, Mar 20, 2018 at 11:07 AM Maik Musall <m...@selbstdenker.ag> wrote:

> First an additional answer: you can return whatever you want. That will
> then become the return value of the performInTransaction() call.
>
> But I have a related question. How is an explicit rollback supposed to
> work there? If I halfway through decide to roll back the entire
> transaction, after a few commitChanges() already have happened, how do I do
> that?
>
> I tried
>
> context.rollbackChangesLocally();
> BaseTransaction.getThreadTransaction().rollback();
>
> but then I get
>
> java.lang.IllegalStateException: Transaction must have ‘STATUS_ACTIVE’ to
> be committed. Current status: STATUS_ROLLEDBACK
>
> What seems to work is throwing a CayenneRuntimeException, but that will
> get rethrown after the rollback, so I would have to catch that again
> outside the performInTransaction() call, which doesn't feel how it's
> supposed to work.
>
> Maik
>
>
>
> > Am 20.03.2018 um 16:40 schrieb John Huss <johnth...@gmail.com>:
> >
> > performInTransaction just encapsulates a database transaction, so it has
> no
> > affect on the object (ORM) level, it only affects what is visible in the
> > database.  So the behavior with commitChanges() is the same as it would
> be
> > outside a performInTransaction block.
> >
> >
> > On Tue, Mar 20, 2018 at 10:25 AM Juan Manuel Diaz Lara
> > <jmdia...@yahoo.com.invalid> wrote:
> >
> >>  runtime.performInTransaction(() -> {
> >>// ... do some changes
> >>context.commitChanges();
> >>
> >>// ... do more changes
> >>context.commitChanges();
> >>
> >>return true;
> >> });   Previous code is an example on "Guide to 4.0 Features".
> >> My questions are:
> >>   - What is then persistence state of objects after first
> commitChanges()
> >> ?
> >>
> >>   - If second commitChanges()  fails, what happen to the persistence
> >> state coming from firs commitChages() ?
> >>   - What happen if we return false?
> >> Atte. Juan Manuel Díaz Lara
>
>

Re: About Improved Transaction API on 4.0

[John Huss]

performInTransaction just encapsulates a database transaction, so it has no
affect on the object (ORM) level, it only affects what is visible in the
database.  So the behavior with commitChanges() is the same as it would be
outside a performInTransaction block.


On Tue, Mar 20, 2018 at 10:25 AM Juan Manuel Diaz Lara
 wrote:

>   runtime.performInTransaction(() -> {
> // ... do some changes
> context.commitChanges();
>
> // ... do more changes
> context.commitChanges();
>
> return true;
> });   Previous code is an example on "Guide to 4.0 Features".
> My questions are:
>- What is then persistence state of objects after first commitChanges()
> ?
>
>- If second commitChanges()  fails, what happen to the persistence
> state coming from firs commitChages() ?
>- What happen if we return false?
> Atte. Juan Manuel Díaz Lara

Re: Object not cached/SELECT on each access

[John Huss]

Hard to say.

Is the shared cache enabled?  Is it sized big enough?

Is the app under memory pressure when this is happening, or is it happening
all the time?

On Wed, Jan 10, 2018 at 6:35 AM Pascal Robert  wrote:

> Hi,
>
> I’m trying to find out why it takes between 200 and 500 ms to convert an
> entity coming from FileMaker and commit the « new » entity to MySQL. Found
> out that every time I access a property of the fetched object, Cayenne does
> a SELECT query. Example:
>
>try(ResultBatchIterator batch =
> ObjectSelect.query(ContactsPrincipaux.class, null,
> tris).batchIterator(context, 100)) {
>   for (List contactsPrincipaux: batch) {
> for (ContactsPrincipaux contactPrincipalFileMaker:
> contactsPrincipaux) {
>   Utilisateur contactPrincipalMySQL =
> mysqlContext.newObject(Utilisateur.class);
>
>
> contactPrincipalMySQL.setNoFilemaker(contactPrincipalFileMaker.getKpCodeContact().longValue());
>
>   LocalDateTime dateCreation =
> transformeDate(contactPrincipalFileMaker.getZCreationDate(),
> contactPrincipalFileMaker.getZCreationHeure());
>   LocalDateTime dateModification =
> transformeDate(contactPrincipalFileMaker.getZModifDate(),
> contactPrincipalFileMaker.getZModifHeure());
>
> That would generate 5 SELECT for the same object, because I’m called 5
> properties on contactPrincipalFileMaker. The SELECT statements are exactly
> the same (same columns, same primary key). I was reading the guide, and
> looks like the object should not be refetched by default, so I don’t
> understand why it’s happening.
>
>
>

Re: Truncating a table

[John Huss]

It's going to depend on which cache you mean.  The query cache can be
cleared by setting a cache group on the query that fetches the objects and
then removing that cache group later.

The shared object cache can be cleared by finding the objects you want in
context.getGraphManager().registeredNodes() and then invalidating them one
by one. It would better to use the query cache.

On Thu, Dec 21, 2017 at 6:48 AM Musall, Maik  wrote:

> Hi Michael,
>
> how to deal with the caches is basically my actual question. Ideally, I'd
> like to call something like myentity.truncateTable(), and just have all
> data deleted and all caches purged by that, but of course that doesn't
> exist yet.
>
> Maik
>
>
> > Am 21.12.2017 um 13:13 schrieb Michael Gentry :
> >
> > Hi Maik,
> >
> > Raw SQL would certainly be the most efficient way.  Even if you didn't
> use
> > raw SQL, though, how were you planning on dealing with Cayenne's
> caches?  I
> > think this issue would, regardless of how you truncated the table.  There
> > are various options, I'm just trying to get a feel for your use can and
> > thoughts.
> >
> > Thanks,
> >
> > mrg
> >
> >
> > On Thu, Dec 21, 2017 at 5:10 AM, Musall, Maik 
> wrote:
> >
> >> Hi,
> >>
> >> I have a lookup table with >400k rows that I want to periodically refill
> >> from external sources. Since it also contains precomputed values that
> are
> >> not part of the external source, my plan is to read the external data
> and
> >> batch-insert it all into the table.
> >>
> >> How can I truncate the entire table to prepare it for new inserts? The
> >> only thing that comes to mind is raw SQL, but that would obviously leave
> >> stale data in Cayenne's various caches.
> >>
> >> Thanks
> >> Maik
> >>
> >>
>
>

Re: Cayenne on Android?

[John Huss]

I tried a few years ago and got a very basic version running.  It was
challenging since Android wasn't a full implementation of java.  For
example the java.beans package was missing.  I think Android may have
improved this situation since then though... depends how old of an android
version you're targeting maybe.

Try it first, and if you have problems there is some info in the JIRA about
what I did.

On Mon, Dec 11, 2017 at 10:10 AM Ken Anderson  wrote:

> Has anyone used Cayenne on Android?  I’m considering making an android
> equivalent of an iOS app that uses Core Data.  It would be great to have an
> ORM on the Android side.
>
> Thanks for any thoughts!
>
> Ken

Re: QueryCache scope

[John Huss]

It is connected to the context in the sense that query results that are
localCached in one context will not be retrieved from the cache by another
context (but instead hit the DB).  However, the objects themselves may
continue to reside in memory indefinitely until memory pressure causes them
to be removed or the cache expires due to a timeout.

On Thu, Nov 9, 2017 at 10:40 AM Lon Varscsak  wrote:

> In the case of using a more specific cache (like discussed in the other
> thread), the problem is these “small” caches get abandoned.  The
> documentation says “localCache” is bound to the ObjectContext, but that
> doesn’t appear to be true.
>
> -Lon
>
> On Wed, Nov 8, 2017 at 11:22 PM, Andrus Adamchik 
> wrote:
>
> > NestedQueryCache is created per ObjectContext. The underlying data is
> > stored in the common cache, prefixed with unique key. While that data is
> > not actively removed from the cache when the ObjectContext (and its
> > NestedQueryCache) go out of scope, it should not affect the overall cache
> > performance. Since most caches have an LRU policy, those abandoned
> entries
> > will eventually be pushed out of the cache, replaced by more recent
> entries
> > from other contexts.
> >
> > Andrus
> >
> > > On Nov 9, 2017, at 2:08 AM, Lon Varscsak 
> wrote:
> > >
> > > Hey all,
> > >
> > > I would have bet large sums of money that a custom implementation of
> > > QueryCache would result in a new QueryCache object per context (local).
> > > However, it looks like a NestedQueryContext gets created and only one
> > > instance of the specified QueryCache is created (in
> DataContextFactory).
> > >
> > > I'd really love to have local caches dumped when the ObjectContext goes
> > > away (assuming it's a localCache).  Is this possible?  Thoughts?
> > >
> > > Thanks!
> > >
> > > -Lon
> >
> >
>

Re: Cache Groups

[John Huss]

Yes, invalidating the cache group will remove the cached data for all users
or query parameters, etc. It's not great.  You can make the cache group
more specific (like by adding a username or something to it) and that will
work, but you'll need to have these cache names configured in your cache
provider already or have a default that makes sense.  That might be
difficult to do.

I'm interested to hear how others handle this.  I see this being a fairly
big limitation of the query cache.

On Wed, Nov 8, 2017 at 11:23 AM Lon Varscsak  wrote:

> Yeah, I’m not syncing between contexts (as much as I’d like to :P)…this is
> about query caches.
>
> On Wed, Nov 8, 2017 at 3:56 AM, Musall, Maik  wrote:
>
> > Hi Lon,
> >
> > have you read this? https://cayenne.apache.org/docs/4.0/cayenne-guide/
> > performance-tuning.html#turning-off-synchronization-of-objectcontexts <
> >
> https://cayenne.apache.org/docs/4.0/cayenne-guide/performance-tuning.html#
> > turning-off-synchronization-of-objectcontexts>
> >
> > I added this module to my server runtime builder (stripped down to the
> > relevant bit for this discussion):
> >
> > Module cachePropertiesModule = new Module() {
> > @Override
> > public void configure( Binder binder ) {
> > MapBuilder props = binder.bindMap( String.class,
> > Constants.PROPERTIES_MAP );
> > props.put( Constants.SERVER_CONTEXTS_SYNC_PROPERTY,
> > "false" );
> > }
> > };
> >
> > Maik
> >
> >
> > > Am 08.11.2017 um 01:40 schrieb Lon Varscsak :
> > >
> > > Hey all,
> > >
> > > I’ve been using cache groups to do refreshing of object lists and it
> > > occurred to me today that this refresh is across all object contexts in
> > my
> > > application.  So if User A has a list of orders for their account and
> > that
> > > is cached with the group “orderHistory” and User B also has a list of
> > > orders in “orderHistory”, running a refresh query will result in ALL
> > users
> > > refreshing their orderHistory caches.
> > >
> > > I’m not sure that this is what I want.  How have you solved this
> > > situation?  Do you care? :P  I was thinking about doing something like
> > > WEB_SESSION_ID+cacheGroup…maybe this has some downsides I’m unaware of.
> > >
> > > Any feedback is appreciated.
> > >
> > > -Lon
> >
> >
>

Re: Order of insert/update/delete operations

[John Huss]

You're looking for the ashwood entity sorter
On Tue, Aug 1, 2017 at 5:40 PM Lon Varscsak  wrote:

> I think in EOF I handled this with a custom ordering of operations, where I
> had a master list of entities in the order I wanted operations to perform
> (and then reversed them for deletes).  Maybe there’s some way to do this
> with Cayenne, but it’s not immediately evident to me where.
>
> -Lon
>
> On Tue, Aug 1, 2017 at 3:37 PM, Lon Varscsak 
> wrote:
>
> > Hey guys, so Cayenne does db operations in insert/update/delete
> > order…which makes sense in most cases.  But in Sybase I don’t have
> deferred
> > constraints (it’s not an option) and if I delete an object and then
> > recreate it will the same primary key I end up with a database error
> > because it tries to insert a duplicate row into the database.
> >
> > Originally I had changed the order of operations in my fork to
> > update/delete/Insert which solves this problems and works in all cases
> > (that I use) except for one.  That’s where you need to insert a row to
> get
> > it’s id (identity) which would then be used in an update operation for a
> FK.
> >
> > It’s possible I’m the only one that uses Sybase, but just looking for
> some
> > thoughts on how I could go about solving this.  I can’t think of a
> > straightforward case.
> >
> > Thanks,
> >
> > Lon
> >
>

Re: Ordering using non-property values

[John Huss]

Expressions will work with non-modeled properties as long as the methods
look like getters - have the "get" prefix and no arguments.

But I would use this:

Collections.sort(list, (a,b) -> a.getSomeProp().compareTo(b.getSomeProp()));



On Mon, Jul 17, 2017 at 8:08 AM Musall, Maik  wrote:

> Hi all,
>
> I have a UI with a list of objects and user-configurable filters and
> sorting criteria. Most of the sorting criteria are plain db path
> expressions, so I can just feed them into Ordering.orderedList(objects).
>
> A couple of them however are path expressions with the last segment being
> not a Property, but a method name that is supposed to be invoked, which
> would then return a String, after which the list is to be ordered. This
> worked fine in the past using EOF and EOSortOrdering, but Cayenne's
> ordering throws an ExpressionException on evaluate().
>
> What is the usual way in Cayenne to deal with this?
>
> Thanks
> Maik
>
>

Re: Object cache - shared vs local

[John Huss]

Thanks Andrus, that was very helpful.

I've never seen much documentation on how caching works, particularly for
the Object caches.  So I'm going to write what I've learned here and maybe
it can help someone else. If there is anything amiss please correct me.


OBJECT CACHE

The "Shared Object Cache" is a DataRowStore (snapshot cache) that can be
consulted by EVERY ObjectContext to fulfill requests primarily for
relationship faults (lazy loading). If the shared Object cache is enabled
then any fired fault will end up in the cache and will be available to
every ObjectContext in the app that may fire the fault in the future. It
won't refetch the row from the database then unless you have explicitly
requested a refresh (or an explicit prefetch). However, the size of the
cache is limited by the Domain configuration (in Cayenne Modeler) property
"size of object cache". So rows will be purged from the cache if it gets
full.  This requires caution since you can't count on any shared data
staying put.  Making the cache extremely large may avoid having your data
evicted, but will waste memory as every object you fetch without an
explicit Local cache strategy ends up in this cache.

The "Local Object Cache" is an ObjectStore that each ObjectContext has a
separate instance of. It is tied directly to an individual ObjectContext.
This allows you to hold on to data that you've prefetched into the context
(or explicitly fetched and lost reference to) but haven't accessed
otherwise yet. It disappears when the ObjectContext disappears. It is
actually backed by it's own DataRowStore, which is inconsequential EXCEPT
for the fact that this means it also it affected by the "size of object
cache" property given above. If this size is smaller than the numbers of
rows returned by a single query plus prefetches it will ignore your
prefetched data and fault in these relationships one at a time.

My recommendations for users are to:
1) Disable the Shared Object Cache. Otherwise you'll have to be vigilant to
avoid stale data in cases where you forgot a prefetch to refresh related
data. It's better to over-fetch than to return invalid (stale) data. That
makes it a performance problem instead of correctness problem, which is a
better default behavior.

If you ARE going to use the Shared Object Cache, then you should use a
Local cache strategy on all your queries that you don't want to end up in
the Shared cache, and you should be very careful to prefetch every
relationship that you need to be fresh.

2) After you've disabled the Shared Object cache, set the size of the
Object cache (this is just for the *Local* Object Cache now) to MAX_INT
(2147483647). Otherwise you risk having Cayenne ignore data that you have
explicitly prefetched and having it fall back to horrendous
one-row-at-a-time fetches. A separate cache is created for each
ObjectContext and only lives as only as the context does, so you don't
really have to worry about the potentially large size of the cache as long
as your contexts are all short-lived.

3) In the small number of cases where you actually WANT to have a shared
cache (like for read-only lookup tables) you can implement a
DataChannelFilter to act as a cache for specific entities. This will ensure
that reads of relationships to these lookup tables will always hit the
cache. This takes a bit of effort, but it works.
https://cayenne.apache.org/docs/4.0/cayenne-guide/lifecycle-events.html#comining-listeners-with-datachannelfilters


QUERY CACHE

The Query cache
<https://cayenne.apache.org/docs/4.0/cayenne-guide/performance-tuning.html#caching-and-fresh-data>
is completely separate from both the Shared Object Cache and the Local
Object Cache. However it also has Shared and Local versions, where Shared
query results are available to EVERY ObjectContext and Local query results
are only available to a single ObjectContext. If you are using the query
cache you should set it up explicitly with a custom cache provider like
EhCache. While there may a learning curve for your cache provider,
Cayenne's behavior with the cache won't surprise you - the configuration is
up to you.

A "Local" query cache with no expiration (no cache group) is useful as a
way to treat an explicit query like a relationship fault since the query
will only be executed once during an ObjectContext's lifetime.

A "Shared" query cache is useful for data where you want to avoid having to
fetch every single time and where some amount of staleness is ok. The
cached query result can be figured to expire after a fixed time period or
when a triggering event occurs.


On Tue, Jun 27, 2017 at 9:39 AM Andrus Adamchik <and...@objectstyle.org>
wrote:

>
> > On Jun 27, 2017, at 10:14 AM, John Huss <johnth...@gmail.com> wrote:
> >
> > I would be ok with disabling the shared cache except for a few entities.
> Is
> > there a way wi

Re: Object cache - shared vs local

[John Huss]

I would be ok with disabling the shared cache except for a few entities. Is
there a way with listeners to intercept queries for specific entities and
return something manually and skip the query?

Using the query cache is great, except if you are firing relationships that
weren't prefetched -- in that case you can't avoid using the shared cache
and getting stale data.

On Tue, Jun 27, 2017 at 8:39 AM Andrus Adamchik <and...@objectstyle.org>
wrote:

>
> > On Jun 21, 2017, at 4:10 PM, John Huss <johnth...@gmail.com> wrote:
>
> > A cache shared between ObjectContexts has a fixed upper limit. 1 is
> the
> > default maximum number of entries, which can be changed in the Modeler. A
> > cache attached to each ObjectContext (also referred to as "local cache"
> > elsewhere in this chapter), which only stores the objects that were
> > accessed via this context, has no upper limit.
>
> This sounds about right, even in 4.0.
>
> > So there is a Shared cache and a Local cache.  The default behavior for
> > relationship faulting (lazy loading) is to place these objects into the
> > *Shared* cache. Same with Cayenne.objectForPK.
>
> More generally, every query, implicit or explicit, would result in
> selected objects placed in both shared cache and local cache of a given
> context.
>
> > This can cause the shared cache to get large, and in some cases I've had
> > objects that I really wanted to be cached forever to get pushed out of
> the
> > Shared cache by poorly written code that fires a ton of lazy
> relationships.
> >
> > Also this can cause stale data to be returned when not carefully guarding
> > against it with prefetches that refresh any needed relationships.
> >
> > I'd prefer that objects from lazily loaded relationships be placed in the
> > *Local* cache to eliminate both of these problems.  I really only want to
> > ever use the Shared cache *explicitly*, never implicitly.  So this
> default
> > behavior seems backwards to me.
> >
> > Is there a way to change this?
>
> Not easily. It is unmanaged (you can't set per-entity caching policies,
> expiration times, etc.), and this is certainly a big limitation. Some
> people turn it off completely by unchecking "Use Shared Cache" in the
> Modeler, but that's another extreme.
>
> Having said that, I never bothered tweaking shared *object* cache, cause I
> base all my refresh policies on *query* cache instead. Query cache is of
> course fully configurable, and local vs shared can be specified explicitly.
> So my typical approach is to leave object cache alone to do what it does
> behind the scenes, but manage query cache that would update objects in the
> shared cache as a side effect.
>
> Andrus

Object cache - shared vs local

[John Huss]

The old docs describe the Object cache(s) like so:

A cache shared between ObjectContexts has a fixed upper limit. 1 is the
default maximum number of entries, which can be changed in the Modeler. A
cache attached to each ObjectContext (also referred to as "local cache"
elsewhere in this chapter), which only stores the objects that were
accessed via this context, has no upper limit.

https://cayenne.apache.org/docs/3.0/individual-object-caching.html


So there is a Shared cache and a Local cache.  The default behavior for
relationship faulting (lazy loading) is to place these objects into the
*Shared* cache. Same with Cayenne.objectForPK.

This can cause the shared cache to get large, and in some cases I've had
objects that I really wanted to be cached forever to get pushed out of the
Shared cache by poorly written code that fires a ton of lazy relationships.

Also this can cause stale data to be returned when not carefully guarding
against it with prefetches that refresh any needed relationships.

I'd prefer that objects from lazily loaded relationships be placed in the
*Local* cache to eliminate both of these problems.  I really only want to
ever use the Shared cache *explicitly*, never implicitly.  So this default
behavior seems backwards to me.

Is there a way to change this?  It seems like a DI property could switch
between these two modes.

Thanks,
John

Re: Bootique, Cayenne and production

[John Huss]

Cayenne is not going to read your System properties directly.  If you set
it in System properties to still have to pass it explicitly to the Cayenne
module like Nikita's code sample does.

On Mon, Jun 12, 2017 at 9:32 AM Pascal Robert  wrote:

> Tried that, no luck (at least with the Bootique app):
>
> java
> -Dcayenne.jdbc.url="jdbc:mysql://xx.xx.xx.xx:3306/filemaker?connectTimeout=0=true"
> -jar RevendeursWeb-1.0.jar --config=config.yml —server
>
> Still connect to the URL defined in the model.
>
> > Le 12 juin 2017 à 09:32, Nikita Timofeev  a
> écrit :
> >
> > Hi Pascal,
> >
> > Yes you can setup DataSource via PropertyDataSourceFactory[1] that can
> > read all information from runtime properties.
> > All you need is to provide those properties via env variables or
> > directly in your code like this (this code for the latest 4.0
> > version):
> >
> > ServerRuntime cayenneRuntime = ServerRuntime.builder()
> >.addConfig("cayenne-project.xml")
> >.addModule(binder -> {
> >ServerModule.contributeProperties(binder)
> >.put("cayenne.jdbc.driver", "com.mysql.Driver")
> >.put("cayenne.jdbc.url", "jdbc:mysql://
> 127.0.0.1:3306/");
> >})
> >.build();
> >
> > [1]
> http://cayenne.apache.org/docs/4.0/api/org/apache/cayenne/configuration/server/PropertyDataSourceFactory.html
> >
> > On Mon, Jun 12, 2017 at 3:59 PM, Pascal Robert 
> wrote:
> >>
> >>> Le 9 juin 2017 à 11:37, Andrus Adamchik  a
> écrit :
> >>>
> >>> The easiest thing is to remove the DataNode from CayenneModel
> completely, and use Bootique connections locally, in production and in
> other environments.
> >>
> >> And can I set the connection information in another way for
> non-Bootique apps?
> >>
> >>> Andrus
> >>>
>  On Jun 9, 2017, at 6:14 PM, Pascal Robert  wrote:
> 
>  Hi,
> 
>  I’m deploying my first Bootique app, which use Cayenne. In my
> config.yml file, I set the datasource like this:
> 
>  jdbc:
>  mysql:
>   driverClassName: com.mysql.jdbc.Driver
>   url:
> "jdbc:mysql://10.6.xx.xx:3306/filemaker?connectTimeout=0=true"
>   initialSize: 1
>   username: x
>   password: x
> 
>  cayenne:
>  datasource: mysql
> 
>  But when I make a request to the app, it tries to connect to the
> server defined in the data model, not from the configuration file.
> 
>  INFO  [2017-06-09 15:08:32,255] bootique-http-36
> o.a.c.c.XMLDataChannelDescriptorLoader: Loading XML configuration resource
> from jar:file:/opt/bin/RevendeursWeb-1.0.jar!/cayenne-filemaker.xml
>  INFO  [2017-06-09 15:08:32,278] bootique-http-36
> o.a.c.c.XMLDataChannelDescriptorLoader: Loading XML DataMap resource from
> jar:file:/opt/bin/RevendeursWeb-1.0.jar!/mysql.map.xml
>  INFO  [2017-06-09 15:08:32,332] bootique-http-36
> o.a.c.c.XMLDataChannelDescriptorLoader: Loading XML DataMap resource from
> jar:file:/opt/bin/RevendeursWeb-1.0.jar!/revendeurs.map.xml
>  INFO  [2017-06-09 15:08:32,338] bootique-http-36
> o.a.c.c.XMLDataChannelDescriptorLoader: Loading XML DataMap resource from
> jar:file:/opt/bin/RevendeursWeb-1.0.jar!/utilisateurs.map.xml
> 
>  INFO  [2017-06-09 15:08:32,366] bootique-http-36
> o.a.c.d.DriverDataSource: Connecting to
> 'jdbc:mysql://legestionnaire.druide:3306/filemaker' as 'filemaker'
>  INFO  [2017-06-09 15:08:32,413] bootique-http-36
> o.a.c.d.DriverDataSource: *** Connecting: FAILURE.
> 
>  What’s the trick?
> >>>
> >>
> >
> >
> >
> > --
> > Best regards,
> > Nikita Timofeev
>
>

Re: Best method to fetch a lot of objects

[John Huss]

Andrus, does this handle memory growth too?  If you use the same context
for each batch won't you still run out of memory?

On Fri, May 19, 2017 at 8:49 AM Andrus Adamchik 
wrote:

> I concur with Mike on the suggestion. Though I would recommend using
> vastly improved 4.0 API:
>
>
> http://cayenne.apache.org/docs/4.0/cayenne-guide/performance-tuning.html#iterated-queries
>
> > As you iterate over your entire record set, you can convert the DataRows
> > into Cayenne objects
>
> In 4.0 you can iterate over objects.
>
> > Gather up 50 or 100 or 1000
>
> In 4.0 you can use batch iterator to receive the stream already split in
> batches. Docs example actually has a typo. Batch iterator looks like this:
>
> try(ResultBatchIterator it =
> ObjectSelect.query(Artist.class).batchIterator(context, batchSize)) {
>
> for(List list : it) {
>...
>context.commitChanges();
> }
> }
>
> Andrus
>
>
> > On May 19, 2017, at 4:39 PM, Michael Gentry  wrote:
> >
> > Hi Pascal,
> >
> > I suspect you need to utilize an iterated query:
> >
> >
> http://cayenne.apache.org/docs/3.1/cayenne-guide/performance-tuning.html#iterated-queries
> >
> > As you iterate over your entire record set, you can convert the DataRows
> > into Cayenne objects (see the section in the documentation above the
> > iterated queries documentation) in a *different* DataContext.  Gather up
> 50
> > or 100 or 1000 (whatever number feels good to you) in that second
> > DataContext and then commit them, throw away that DataContext and create
> a
> > new one.  Repeat.  This should keep your memory usage fairly constant and
> > allow you to process arbitrarily large record sizes.
> >
> > mrg
> >
> >
> > On Fri, May 19, 2017 at 9:27 AM, Pascal Robert 
> wrote:
> >
> >> Hi,
> >>
> >> I’m still in my FileMaker -> MySQL migration project. This time, I want
> to
> >> migrate a FileMaker table who have 445 244 records in it. If I fetch
> >> everything into an object entity for each row, I’m getting a Java heap
> >> space problem, which is somewhat expected by the size of the result set.
> >>
> >> If I call setFetchLimit() with a 10 000 limit, works fine. FileMaker
> >> doesn’t support fetch limits, so I can’t do something on that side.
> >>
> >> Any tips?
>
>

Re: Stacktrace

[John Huss]

The screenshot didn't come through for me.

On Tue, May 2, 2017 at 5:17 AM Markus Reich 
wrote:

> Hi,
>
> I have a question for a stacktrace I took on our prod. system.
> We are analyzing performance issues and found the method
> Invocation.equals(), when we drill down we get to the cayenne events.
>
> Is there someone who could explain those Invocation objects, is this the
> shared cache?
>
> I added a screenshot of the stacktrace
>
> best regards
> Markus[image: cayenne_stacktrace_2.PNG]
>