Doubts

[Juan Carlos Fiorenzano]

Hello everybody I am a little lost, I need some help, first of all I need
to know what is the module which implement the DataGrid I want to know how
the DataGrid works, and my other question is how do you test a specific
module? Do you defined specific unit test for the module and include the
test files in the gitignore list or do you do other thing? I know that is a
silly question, but I want to know what is the best way for do.

Thanks in advance

Juan C Fiorenzano

Re: Halfway to Ignite 1.6

[Andrey Gura]

Guys,

I've finished with improvements for JDBC drivers [1]. I hope somebody will
review and merge it before code freeze.
Also I created issue related with query execution from client nodes [2]. It
affects system behaviour that related with [1] and [3]. I don't think that
it is critical because walk around exists.

[1] https://issues.apache.org/jira/browse/IGNITE-2382
[2] https://issues.apache.org/jira/browse/IGNITE-3136
[3] https://issues.apache.org/jira/browse/IGNITE-2455

On Fri, May 13, 2016 at 6:24 PM, Anton Vinogradov  wrote:

> Igniters,
>
> We postponed code freeze till monday UTC 19-00.
>
> On Thu, May 12, 2016 at 1:48 PM, Anton Vinogradov  wrote:
>
> > Typo fix:
> > Pushes to ignite-1.6 are allowed till Friday (*May *13) 19:00 UTC.
> >
> >
> >
> > On Thu, May 12, 2016 at 1:41 PM, Anton Vinogradov  wrote:
> >
> >> Igniters,
> >>
> >> Ignite 1.6 almost ready to be finally tested.
> >>
> >> Branch ignite-1.6 now contains all changes scheduled for 1.6.
> >>
> >> In case you want to add something else to this release, please note that
> >> pushes to ignite-1.6 allowed till Friday (Aug 13) 19:00 UTC.
> >> After that date only fixes will allowed.
> >>
> >> On Fri, May 6, 2016 at 4:18 PM, Anton Vinogradov  wrote:
> >>
> >>> Igniters,
> >>>
> >>> We're going to release Ignite 1.6 in the nearest future.
> >>> Branch ignite-1.6 was created and already contains changes scheduled
> for
> >>> 1.6.
> >>>
> >>> And from now on, I propose to start testing and fixing ignite-1.6
> source
> >>> code to provide stable release.
> >>>
> >>> As you know we have a huge amount of tests and some of them now failing
> >>> at release branch.
> >>> Fails can be found here:
> >>>
> >>>
> http://149.202.210.143:8111/project.html?projectId=IgniteTests=projectOverview_IgniteTests=ignite-1.6
> >>>
> >>> So, feel free to start investigations and fix issues :)
> >>>
> >>
> >>
> >
>



-- 
Andrey Gura
GridGain Systems, Inc.
www.gridgain.com

[jira] [Created] (IGNITE-3136) Cache initialization failed on client node in case of dynamic cache start and not binary marshaller

[Andrey Gura (JIRA)]

Andrey Gura created IGNITE-3136:
---

 Summary: Cache initialization failed on client node in case of 
dynamic cache start and not binary marshaller
 Key: IGNITE-3136
 URL: https://issues.apache.org/jira/browse/IGNITE-3136
 Project: Ignite
  Issue Type: Bug
Reporter: Andrey Gura
Priority: Minor


Cache can't be created on client if cluster uses not {{BinaryMarshaller}} and 
indexing configured for cache with value classes not in client classpath.

There are at least two failing cases:

1. Automatic cache creation in case of executing cross cache query on client 
node
2. Automatic cache creation in case of using JDBC driver with out specified 
cache name.

Steps to reproduce:

1. Both client and server use OptimizedMarshaller.
2. Start server node with cache that should use some non primitive values and 
have indexing configuring for fields of this value classes.
3. Start client node that doesn't have mentioned value classes in class path 
and doesn't start cache.
4. On client node execute sql query (e.g. via some default cache) that uses 
cache name started only on server node.

Exception will be thrown:

{noformat}
SEVERE: Failed to wait for completion of partition map exchange (preloading 
will not start): GridDhtPartitionsExchangeFuture [dummy=false, 
forcePreload=false, reassign=false, discoEvt=DiscoveryCustomEvent 
[customMsg=null, affTopVer=AffinityTopologyVersion [topVer=6, minorTopVer=1], 
super=DiscoveryEvent [evtNode=TcpDiscoveryNode 
[id=655f9a15-50d1-41fe-9437-65594a280455, addrs=[127.0.0.1, 192.168.0.192], 
sockAddrs=[/192.168.0.192:0, /127.0.0.1:0, /192.168.0.192:0], discPort=0, 
order=6, intOrder=0, lastExchangeTime=1463343172508, loc=true, 
ver=1.6.0#20160513-sha1:85eaa9c3, isClient=true], topVer=6, nodeId8=655f9a15, 
msg=null, type=DISCOVERY_CUSTOM_EVT, tstamp=1463343174650]], 
remaining=[61462236-c2f2-4f31-be78-495105a066d6, 
a07f59c3-348b-42d8-b237-cbb48006c698, 03224102-9daf-4477-ba6a-21ef0105c533], 
crd=TcpDiscoveryNode [id=03224102-9daf-4477-ba6a-21ef0105c533, 
addrs=[127.0.0.1, 192.168.0.192], sockAddrs=[/192.168.0.192:47500, 
/127.0.0.1:47500, /192.168.0.192:47500], discPort=47500, order=1, intOrder=1, 
lastExchangeTime=1463343173861, loc=false, ver=1.6.0#20160513-sha1:85eaa9c3, 
isClient=false], exchId=GridDhtPartitionExchangeId 
[topVer=AffinityTopologyVersion [topVer=6, minorTopVer=1], nodeId=655f9a15, 
evt=DISCOVERY_CUSTOM_EVT], added=true, initFut=GridFutureAdapter [resFlag=2, 
res=false, startTime=1463343174650, endTime=1463343174670, 
ignoreInterrupts=false, state=DONE], init=false, topSnapshot=null, 
lastVer=null, partReleaseFut=null, affChangeMsg=null, skipPreload=true, 
clientOnlyExchange=false, initTs=1463343174650, centralizedAff=false, 
oldest=03224102-9daf-4477-ba6a-21ef0105c533, oldestOrder=1, evtLatch=0, 
remaining=[61462236-c2f2-4f31-be78-495105a066d6, 
a07f59c3-348b-42d8-b237-cbb48006c698, 03224102-9daf-4477-ba6a-21ef0105c533], 
super=GridFutureAdapter [resFlag=1, res=class o.a.i.IgniteCheckedException: 
Failed to find value class in the node classpath (use default marshaller to 
enable binary objects) : o.a.i.zeppelin.Person, startTime=1463343174650, 
endTime=1463343174670, ignoreInterrupts=false, state=DONE]]
class org.apache.ignite.IgniteCheckedException: Failed to find value class in 
the node classpath (use default marshaller to enable binary objects) : 
org.apache.ignite.zeppelin.Person
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.initializeCache(GridQueryProcessor.java:249)
at 
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:462)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1043)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1714)
at 
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1605)
at 
org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:382)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:562)
at 
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:445)
at 
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1333)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:744)
{noformat}





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Ignite Release Schedules - a suggestion

[Konstantin Boudnik]

I don't think the extrapolation works in this particular case, although it
looks scary ;) However, I agree, shorter bug fix & smaller feature releases
might make a lot of sense, considering how actively the userbase and community
grow. However, major features take longer time to mature and be integrated
into the master, hence they should be carried on the branches (an good example
such is IGNITE-1371). And this might lead, in the extreme case, to the
multi-release universe.

Hence coming Dmitriy's quesion about branching model. So I would like to, once
again, offer 
http://nvie.com/posts/a-successful-git-branching-model/

which effectively and transparently allow to support multiple release trains.

But in general, releases shouldn't be a big deal and any committer can call a
release at any given moment. All it needs to pass is >3 PMC votes to become an
official one. Easier the release process is and more release managers (RMs) we
have - shorter the release spam might become.

Cos

On Fri, May 13, 2016 at 02:21AM, endianignite wrote:
> Dear Igniters,
> 
> First of all, congratulations on heading towards the 1.6 release.  Each
> Ignite release is a huge amount of work, done to a very high standard by a
> great group of people.  I raise my hat to each and every one of you!
> 
> I would like to open a discussion regarding release schedules, and how we
> currently perform releases.  Right now, we have very large waterfall
> releases, that are a combination of substantial numbers of bug fixes, along
> with significant new features - binary marshaling from 1.5 is one that I
> witnessed first hand.  The integration of all of these changes can be
> challenging, and from my observation tends to extend the time taken to go
> from the final code freeze to the point of release.  Some data from the
> release dates:
> 
> Release,Date,DaysSinceLastRelease
> 1.0,02-Apr-15,
> 1.1,28-May-15,56
> 1.2,29-Jun-15,32
> 1.3,21-Jul-15,22
> 1.4,28-Sep-15,69
> 1.5,04-Jan-16,98
> 1.6*,31-May-16,148
> 1.7+,03-Dec-16,186
> 1.8+,17-Jul-17,226.7
> 1.9+,11-Apr-18,267.4
> 2.0+,13-Feb-19,308.1
> 
> *estimated date
> +extrapolated date.  Equation: date=407*releaseVersion-505.9 taken from
> Excel best-fit line, R^2=0.9906.  
> 
> As you can see, the time between releases is trending upwards since 1.3, and
> on a very steep gradient.  Doing some (admittedly silly) extrapolation, at
> the current it could take 308 days to go from a future version 1.9 to a 2.0
> release, just assuming the current growth rate.  This could mean that a bug
> found in April 2018 would not be released until February 2019!  Currently
> I'm facing a bug (https://issues.apache.org/jira/browse/IGNITE-2080) that
> was found in Dec-15, fixed in Feb-16, but will not be released until
> May/June 2015.  
> 
> In my opinion this is not sustainable if we want Ignite to grow in userbase
> and functionality, whilst retaining a reputation for stability and being
> bug-free, as I'm sure we all do.  Therefore, as a sign of Ignite's growing
> maturity, I would like to suggest that we adopt a release schedule where bug
> fixes are released on a timed-release basis, and new features are included
> in less frequent major releases.  There are many examples of this kind of
> release schedule: Intel's tick-tock cycle, Ubuntu's timed-release cycle,
> Redhat's major/minor/asynch cycle.  
> 
> I propose that the release schedule for Ignite would be separated in to two
> streams: bug fixes and major features.  The first stream, bug fixes, would
> close for code freeze every month, undergo testing and QA and then be
> released, hopefully within two weeks of the code freeze.  Bugs not fixed in
> time for the code freeze would be pushed to the next monthly cycle.  The
> Major Features stream would be over a significantly longer period, although
> I would still suggest that it be less than the current 3-5 month interval.  
> 
> Finally, if you got this far then well done, you have got staying power.  I
> hope that you will take the above thoughts and analysis in the way that they
> are offered, as a positive, and not as criticism (which they are not).
> 
> I'm really interested to hear your thoughts on this subject.  I'm happy to
> try and lead a release cycle change if the community would like me to, or to
> assist someone within GridGain if that makes more sense.
> 
> Kind regards
> Mike
> 
> 
> 
> 
> --
> View this message in context: 
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Release-Schedules-a-suggestion-tp8867.html
> Sent from the Apache Ignite Developers mailing list archive at Nabble.com.


signature.asc
Description: Digital signature

Re: HDFS CacheStore

[Konstantin Boudnik]

Cristian,

Great idea!
a couple of comments:
 - HDFS supports truncate since at least v. 2.7.0 (see HDFS-3107)
 - integration with HBase would increase the operational complexity, which is
   already high in HBase, to new levels. Besides, using HBase as the
   compaction mechanism shouldn't be called HDFS CacheStore
 - do you think this can be combined with IGFS on top of HDFS?

Thoughts?
  Cos

On Fri, May 13, 2016 at 10:35AM, Cristian Malinescu wrote:
> Hello folks - reloading on the topic to add a HDFS CacheStore module, is
> anybody interested in brainstorming upon the idea? I want to be consistent
> and make sure I capture most of the ideas, opinions and options before
> proceeding brute force :). As long as HDFS paradigm is append-only, maybe a
> plain integration with HBase would be a better option as going this path
> Ignite will offload the need to manage HDFS compaction to HBase.
> Cheers
> Cris


signature.asc
Description: Digital signature

Re: 1.5.0.final is breaking packaging: osgi dependency is non-existent

[Konstantin Boudnik]

Thanks for checking out Andrey. Do you mind to elaborate on 'something'? The
way I see it, is that rpmbuilder performs certain checks to declare the
package dependencies, and the osgi once are popping up from somewhere. Which
wasn't the case before OSGI was added. Which lead me to believe that the issue
might be rooting from missing jars or something similar. Hence, my this email
thread.

Were my assumptions incorrect.
Thanks
  Cos

On Thu, May 05, 2016 at 04:39AM, Andrey Kornev wrote:
> Anton,
> 
> My first impression after having taken a look at bigtop's JIRA ticket is
> that the issue is not so much with OSGi per se, but rather it has something
> to do with Ignite's HDFS accelerator yum packaging.
> 
> Regards
> Andrey
> 
> > Date: Thu, 5 May 2016 13:55:26 +0300
> > Subject: Re: 1.5.0.final is breaking packaging: osgi dependency is 
> > non-existent
> > From: avinogra...@gridgain.com
> > To: dev@ignite.apache.org
> > 
> > Igniters,
> > 
> > Issue is critical for Bigtop project.
> > In case someone familiar with OSGI please have a look.
> > 
> > On Wed, May 4, 2016 at 11:15 AM, Anton Vinogradov 
> > wrote:
> > 
> > > Roman, Raul,
> > >
> > > As far as I remember you're familiar with OSGI. Could you please have a
> > > look at this?
> > >
> > > On Wed, May 4, 2016 at 6:09 AM, Konstantin Boudnik 
> > > wrote:
> > >
> > >> Hey guys
> > >>
> > >> For your information: adding osgi into the product is breaking the
> > >> dependency matrix of the bigtop packages. For more info see
> > >> https://issues.apache.org/jira/browse/BIGTOP-2421
> > >>
> > >> --
> > >> Take care,
> > >> Cos
> > >> 2CAC 8312 4870 D885 8616  6115 220F 6980 1F27 E622
> > >> Cos' pubkey: http://people.apache.org/~cos/cos.asc
> > >>
> > >>   Wisdom of the hour 
> > >>
> > >>
> > >
> 


signature.asc
Description: Digital signature