t;> Hi Alex,
>>
>> Did you get a chance to look into it ? thanks.
>>
>> Regards,
>> Anil
>>
>>
>>
>> ------
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://apache-ignite-users.70518.x6.nabbl
Hi Alex,
Did you get a chance to look into it ? thanks.
Regards,
Anil
;
>> On 21 June 2017 at 09:23, Anil <[hidden email]
>> <http:///user/SendEmail.jtp?type=node=14058=1>> wrote:
>>
>>> Thanks Alex. I will test it in my local and share the results.
>>>
>>> Did you get a chance to look at the Jdbc driver's nex
Socket closed is the very frequent. May i now which causes the following
exception ? thanks,
Some more log -
2017-06-23 02:33:34 488 ERROR TcpDiscoverySpi:495 - Failed to send message:
TcpDiscoveryClientHeartbeatMessage [super=TcpDiscoveryAbstractMessage
[sndNodeId=null,
HI Alex,
i tried XX:G1NewSizePercent=30 and ignite client is getting restarted very
frequently, for each export operation.
-Xmx6144m -XX:MetaspaceSize=512m -XX:+UnlockExperimentalVMOptions
-XX:G1NewSizePercent=30 -XX:+UseTLAB -XX:+UseG1GC -XX:MaxGCPauseMillis=500
-XX:+ScavengeBeforeFullGC
Thanks Alex. I will test it in my local and share the results.
Did you get a chance to look at the Jdbc driver's next() issue ? Thanks.
Thanks,
Anil
t;>> resIter = res.iterator();
>>>>>>>>>> }else {
>>>>>>>>>> // incase of split query scenario
>>>>>>>>>> }
>>>>&
gt; Query cursor is iterator which does column value mapping per page.
>>>>>>>> But still all records of query are still in memory. correct?
>>>>>>>>
>>>>>>>> Please correct me if I am wrong. thanks.
>>>>>>>>
>>>>>&g
t 15:53, Anil <[hidden email]
>>>>>>> <http:///user/SendEmail.jtp?type=node=13626=1>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> jvm parameters used -
>>>>>>>>
>>>>>>>> -Xmx6144m -XX:NewSize=512m -XX:+UseTLAB -XX:+UseG1GC
>>>>>>&
rintAdaptiveSizePolicy
>>>>>>> -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:+HeapDumpAfterFullGC
>>>>>>> -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC -XX:+AlwaysPreTouch
>>>>>>> -XX:+PrintFlagsFinal -XX:HeapDumpPath=C:/Anil/dumps
>
gt;>>
>>>>>> Thanks.
>>>>>>
>>>>>> On 10 June 2017 at 15:06, Anil <[hidden email]
>>>>>> <http:///user/SendEmail.jtp?type=node=13626=2>> wrote:
>>>>>>
>>>>>>> HI,
>>
;>>>>
>>>>>> I have implemented export feature of ignite data using JDBC Interator
>>>>>>
>>>>>> ResultSet rs = statement.executeQuery();
>>>>>>
>>>>>> while (rs.next()){
>>>>>> // do operations
>>>>>>
;>>>>
>>>>>> while (rs.next()){
>>>>>> // do operations
>>>>>>
>>>>>> }
>>>>>>
>>>>>> and fetch size is 200.
>>>>>>
>>>>>> when i run export operati
;>
>>>>> and fetch size is 200.
>>>>>
>>>>> when i run export operation twice for 4 L records whole 6B is filled
>>>>> up and never getting released.
>>>>>
>>>>> Initially i thought that operations transforting result
t;>>> I just did follwoing and still the memory is growing and not getting
>>>> released
>>>>
>>>> while (rs.next()){
>>>> // nothing
>>>> }
>>>>
>>>> num #instances #bytes class name
>&g
Do you have any advice on implementing large records export from ignite ?
I could not use ScanQuery right as my whole application built around Jdbc
driver and writing complex queries in scan query is very difficult.
Thanks
On 10 June 2017 at 18:48, Anil wrote:
> I
I understand from the code that there is no cursor from h2 db (or ignite
embed h2 db) internally and all mapper response consolidated at reducer. It
means when exporting large number of records, all data is in memory.
if (send(nodes,
oldStyle ?
jvm parameters used -
-Xmx6144m -XX:NewSize=512m -XX:+UseTLAB -XX:+UseG1GC
-XX:MaxGCPauseMillis=500 -XX:+ScavengeBeforeFullGC -XX:+DisableExplicitGC
-Xloggc:C:/Anil/dumps/gc-client.log -XX:+HeapDumpOnOutOfMemoryError
-XX:+PrintGCCause -XX:+PrintGCDetails -XX:+PrintAdaptiveSizePolicy
HI,
I have implemented export feature of ignite data using JDBC Interator
ResultSet rs = statement.executeQuery();
while (rs.next()){
// do operations
}
and fetch size is 200.
when i run export operation twice for 4 L records whole 6B is filled up and
never getting released.
Initially i
19 matches
Mail list logo