date:20180308

Re: Appropriate use of lazy fetching with fetchSize + fetchStream ?

2018-03-08 Thread Lukas Eder

>
> For my culture : is this the PortalSuspended and multiple Execute limit 1
> stuff that indicates if it works ?
>

Probably :)
If you want to be sure, I think that the PostgreSQL mailing lists, or Stack
Overflow are more appropriate channels...

-- 
You received this message because you are subscribed to the Google Groups "jOOQ 
User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jooq-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Appropriate use of lazy fetching with fetchSize + fetchStream ?

2018-03-08 Thread LMey

Alright thanks. I'm seeing this when doing chunks of 1 :

-2018-03-08 09:52:59.811 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  FE=> 
Parse(stmt=null,query="REDACTED",oids={1043,0,0})
-2018-03-08 09:52:59.811 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  FE=> Bind(stmt=REDACTED>)
-2018-03-08 09:52:59.811 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  FE=> Describe(portal=C_5)
-2018-03-08 09:52:59.811 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  FE=> Execute(portal=C_5,limit=1)
-2018-03-08 09:52:59.811 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  FE=> Sync
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  <=BE ParseComplete [null]
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  <=BE BindComplete [C_5]
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  <=BE RowDescription(7)
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   : 
Field(fieldNameRedacted,TEXT,65535,T)
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   : 
Field(fieldNameRedacted,TIMESTAMPTZ,8,T)
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   : 
Field(fieldNameRedacted,INT8,8,T)
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   : 
Field(fieldNameRedacted,UUID,16,T)
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   : 
Field(fieldNameRedacted,BPCHAR,65535,T)
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   : 
Field(fieldNameRedacted,NUMERIC,65535,T)
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   : 
Field(fieldNameRedacted,NUMERIC,65535,T)
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  <=BE DataRow(len=79)
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  <=BE PortalSuspended
-2018-03-08 09:52:59.813 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  <=BE ReadyForQuery(T)
-2018-03-08 09:52:59.843 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  FE=> Execute(portal=C_5,limit=1)
-2018-03-08 09:52:59.843 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  FE=> Sync
-2018-03-08 09:52:59.843 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  <=BE DataRow(len=80)
-2018-03-08 09:52:59.843 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  <=BE PortalSuspended
-2018-03-08 09:52:59.843 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  <=BE ReadyForQuery(T)
-2018-03-08 09:52:59.844 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  FE=> Execute(portal=C_5,limit=1)
-2018-03-08 09:52:59.844 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  FE=> Sync
-2018-03-08 09:52:59.844 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  <=BE CommandStatus(SELECT 0)
-2018-03-08 09:52:59.844 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  <=BE ReadyForQuery(T)
-2018-03-08 09:52:59.987 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :   simple execute, 
handler=org.postgresql.jdbc.PgConnection$TransactionCommandHandler@6658f08a, 
maxRows=0, fetchSize=0, flags=22
-2018-03-08 09:52:59.988 TRACE 28417 --- [   main] 
o.postgresql.core.v3.QueryExecutorImpl   :  FE=> ClosePortal(C_5)

For my culture : is this the PortalSuspended and multiple Execute limit 1 
stuff that indicates if it works ?

Le jeudi 8 mars 2018 09:39:14 UTC+1, Lukas Eder a écrit :
>
> Hello,
>
> The API usage is correct:
>
>- fetchSize() overrides the JDBC driver's default, which in the case 
>of PostgreSQL is 0 (reading the source code), meaning that all rows are 
>fetched in one go by default.
>- You're using jOOQ's fetchStream(), which keeps an open JDBC 
>ResultSet internally, so jOOQ also doesn't fetch everything in memory
>- You're correctly wrapping the Stream in a try-with-resources 
>statement, to be sure that there are no resource leaks
>
> If you want to validate the behaviour, put some breakpoints into the 
> source code of the PostgreSQL JDBC driver, or turn on debug logging of the 
> driver. It should emit some debug information, e.g. in 
> org.postgresql.core.v3.QueryExecutorImpl.sendExecute()
>
> I hope this hel

Re: Appropriate use of lazy fetching with fetchSize + fetchStream ?

2018-03-08 Thread Lukas Eder

Hello,

The API usage is correct:

   - fetchSize() overrides the JDBC driver's default, which in the case of
   PostgreSQL is 0 (reading the source code), meaning that all rows are
   fetched in one go by default.
   - You're using jOOQ's fetchStream(), which keeps an open JDBC ResultSet
   internally, so jOOQ also doesn't fetch everything in memory
   - You're correctly wrapping the Stream in a try-with-resources
   statement, to be sure that there are no resource leaks

If you want to validate the behaviour, put some breakpoints into the source
code of the PostgreSQL JDBC driver, or turn on debug logging of the driver.
It should emit some debug information, e.g. in
org.postgresql.core.v3.QueryExecutorImpl.sendExecute()

I hope this helps,
Lukas

2018-03-07 18:15 GMT+01:00 LMey :

> Hi,
>
> I'm not sure I'm using lazy fetching with streams correctly... I'm using
> Jooq 3.9.6 en Postgresql 9.5.
>
> This is my (simplified) repo method :
>
> public Stream findLazy() {
> return dslContext.select(...)
> .fetchSize(100)
> .fetchStream()
> .map(ExampleMapper::apply);
> }
>
>
> And my service method (running in a transaction) :
>
> try (Stream stream = exampleRepository.findLazy()) {
> stream.forEach(magic::doStuff);
> }
>
>
> Will this chunk appropriately or load everything into memory ? How does
> this work exactly ? Sadly with postgresql I have not found any way of
> checking if this works.
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "jOOQ User Group" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jooq-user+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups "jOOQ 
User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jooq-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Appropriate use of lazy fetching with fetchSize + fetchStream ?

Re: Appropriate use of lazy fetching with fetchSize + fetchStream ?

Re: Appropriate use of lazy fetching with fetchSize + fetchStream ?

3 matches

Site Navigation

Mail list logo

Footer information