Hello, I have not saw shortcomings in your code, but me was not clear which size of data moved between nodes or are data collect?
Can you create working example? So I was able to reproduce the problem. On Tue, Sep 13, 2016 at 12:06 PM, 胡永亮/Bob <[email protected]> wrote: > > Only 118 jobs for this test. > > ------------------------------ > Bob > > > *From:* Taras Ledkov <[email protected]> > *Date:* 2016-09-13 14:52 > *To:* [email protected] > *Subject:* Re: Re: Increase Ignite instances can't increase the speed of > compute > Hi, > > How many MatchingJobs do you submit? > > > On Tue, Sep 13, 2016 at 12:29 PM, 胡永亮/Bob <[email protected]> wrote: > >> Hello, Vladislav <[email protected]> >> >> The following is some code. >> >> ... >> IgniteCompute compute = ignite.compute();//.withAsync(); >> >> compute.run(new MatchingJob(m_mapReadyDataPara)); >> } >> >> private static class MatchingJob implements IgniteRunnable{ >> >> private Map<String, Map<Long, std>> m_mapReadyData; >> private IgniteCache<String, Map<Long, List<baselink>>> mapMatchingData; >> *//This is a cache in Ignite cluster.* >> ... >> >> public void run() { >> ... >> Iterator<Entry<String, Map<Long, std>>> entryKeyIterator1 >> = m_mapReadyData.entrySet().iterator(); *//**m_mapReadyData is the >> input data, its size is 5000 for every job now.* >> Map<String, Map<Long, List<baselink>>> local_writeCache = >> new HashMap<String, Map<Long, List<baselink>>>(); >> ... >> *//Then the job read detail data from m_mapReadyData, and compute.* >> while (entryKeyIterator1.hasNext()) { >> Entry<String, Map<Long, std>> eKey1 = entryK >> eyIterator1.next(); >> String carKey = eKey1.getKey(); >> Map<Long, std> value1 = eKey1.getValue(); >> >> * //local node cache* >> Map<Long, List<baselink>> existMapbaselink = >> local_mapMatchingData.get(carKey); >> if(existMapbaselink == null){ >> existMapbaselink = mapMatchingData.get(carKey); * //Read >> data to compute with it from Ignite cache. This data's size is 154M for * >> PARTITIONED *mode.* >> if(existMapbaselink != null) >> local_mapMatchingData.putIfAbsent(ca >> rKey, existMapbaselink); >> } >> >> * //some compute logic code* >> >> mapbaselink = local_writeCache.get(carKey); >> if(mapbaselink == null){ >> mapbaselink = new TreeMap<Long, List<baselink>>(); >> } >> mapbaselink.put(stdtime, ListBaseLink); >> local_writeCache.put(carKey, mapbaselink); >> } >> >> *//batch to write data into Ignite.* >> Iterator<Entry<String, Map<Long, List<baselink>> >> >> it = local_writeCache.entrySet().iterator(); >> while(it.hasNext()){ >> Entry<String, Map<Long, List<baselink>>> ent >> ry = it.next(); >> String carKey = entry.getKey(); >> final Map<Long, List<baselink>> value = entry.getValue(); >> >> if(!mapMatchingData.containsKey(carKey)){ >> mapMatchingData.put(carKey, value); >> }else{ >> mapMatchingData.invoke(carKey, new >> EntryProcessor<String, Map<Long, List<baselink>>, Void>() { >> @Override >> public Void process(MutableEntry<Strin >> g, Map<Long, List<baselink>>> entry, Object... args) { >> Map<Long, List<baselink>> map = en >> try.getValue(); >> map.putAll(value); >> entry.setValue(map); >> return null; >> } >> }); >> } >> >> } >> >> >> ------------------------------ >> bob >> >> >> *From:* Vladislav Pyatkov <[email protected]> >> *Date:* 2016-09-12 18:37 >> *To:* [email protected] >> *Subject:* Re: Increase Ignite instances can't increase the speed of >> compute >> Hello, >> >> I don't understand, what do you try to measure, without code. >> Size of calculation task, size of data moved into network have importance. >> >> Could you please provide code example? >> >> On Mon, Sep 12, 2016 at 12:33 PM, 胡永亮/Bob <[email protected]> wrote: >> >>> Hi, everyone: >>> >>> I am using Ignite for computing and cache. >>> >>> I use the same input data and the same compute logic. >>> When my ignite cluster's node is 2 in 2 machines, the total cost >>> time is 38s. >>> >>> But, when I increase the Ignite cluster nodes to 3 in 3 machines, >>> the cost time is 32s/51s/41s >>> 4 instances in 4 machines, the cost time is 32s/40s. >>> >>> The compute speed can't change faster, what may the reason be? >>> >>> Thanks. >>> >>> Bob >>> >>> ------------------------------------------------------------ >>> --------------------------------------- >>> Confidentiality Notice: The information contained in this e-mail and any >>> accompanying attachment(s) >>> is intended only for the use of the intended recipient and may be >>> confidential and/or privileged of >>> Neusoft Corporation, its subsidiaries and/or its affiliates. If any >>> reader of this communication is >>> not the intended recipient, unauthorized use, forwarding, printing, >>> storing, disclosure or copying >>> is strictly prohibited, and may be unlawful.If you have received this >>> communication in error,please >>> immediately notify the sender by return e-mail, and delete the original >>> message and all copies from >>> your system. Thank you. >>> ------------------------------------------------------------ >>> --------------------------------------- >>> >> >> >> >> -- >> Vladislav Pyatkov >> >> >> ------------------------------------------------------------ >> --------------------------------------- >> Confidentiality Notice: The information contained in this e-mail and any >> accompanying attachment(s) >> is intended only for the use of the intended recipient and may be >> confidential and/or privileged of >> Neusoft Corporation, its subsidiaries and/or its affiliates. If any >> reader of this communication is >> not the intended recipient, unauthorized use, forwarding, printing, >> storing, disclosure or copying >> is strictly prohibited, and may be unlawful.If you have received this >> communication in error,please >> immediately notify the sender by return e-mail, and delete the original >> message and all copies from >> your system. Thank you. >> ------------------------------------------------------------ >> --------------------------------------- >> > > > ------------------------------------------------------------ > --------------------------------------- > Confidentiality Notice: The information contained in this e-mail and any > accompanying attachment(s) > is intended only for the use of the intended recipient and may be > confidential and/or privileged of > Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader > of this communication is > not the intended recipient, unauthorized use, forwarding, printing, > storing, disclosure or copying > is strictly prohibited, and may be unlawful.If you have received this > communication in error,please > immediately notify the sender by return e-mail, and delete the original > message and all copies from > your system. Thank you. > ------------------------------------------------------------ > --------------------------------------- > -- Vladislav Pyatkov
