Whatever approach is best for you is fine I guess. But an idea would be to fork the MM GitHub mirror. That way we can quite easily produce a diff to apply to our ASF Git repo (the GitHub is "just" a mirror). Den 13/09/2014 10.21 skrev "Alberto Rodriguez" <ardl...@gmail.com>:
> Hi Kasper, > > Your idea is great I will make a "first functional version" and then we > might improve it with native ES queries and filters. > > I haven´t made an ICLA for ASF yet. I will send the signed form ASAP. > > I am also going to fork the MM project, do you want me to create a new > branch from master on the forked project and start making my changes there? > > Thank you! > > Kind regards, > > Alberto > > > 2014-09-12 22:39 GMT+02:00 Kasper Sørensen <i.am.kasper.soren...@gmail.com > >: > > > Hi Alberto, > > > > We agree completely :-) it's great that you push as much querying down to > > the backend. I was only suggesting you start with making something that > is > > "slow and stupid, but functional" and then optimize. But I think you're > > already doing much better :-) > > > > Consider making this as a fork to the MetaModel source. At least, I think > > we could integrate this quite soon into the main MM source tree. > > > > I do think however we would need a contributor license agreement from you > > to accept it (it's not just a patch, but more substantial). Have you > > already made an ICLA for ASF? > > > > Best regards, > > Kasper > > > > 2014-09-12 22:10 GMT+02:00 Alberto Rodriguez <ardl...@gmail.com>: > > > > > Hi Kasper, > > > > > > Ok got it! thank you for explaining to me. > > > > > > I will implement the materializeMainSchemaTable but as you said I think > > is > > > a good idea to take advantage of the very powerful ElasticSearch API > > which > > > has a DSL that support natively queries and filters, have a look at > them: > > > Query > > > DSL > > > < > > > > > > http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/query-dsl-queries.html > > > > > > > , Query dsl filters > > > < > > > > > > http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/query-dsl-filters.html > > > >. > > > By using these APIs we would also get a huge performance improvement > > > because we were using Apache Lucene under-the-hood. > > > > > > I will let you know if there is any further update. Thank you!! > > > > > > Kind regards, > > > > > > Alberto > > > > > > > > > > > > > > > 2014-09-12 14:52 GMT+02:00 Kasper Sørensen < > > i.am.kasper.soren...@gmail.com > > > >: > > > > > > > Hi Alberto, > > > > > > > > Yes you are right that these ones do override executeQuery(Query). In > > the > > > > case of JDBC, it's a completely different story so let's leave that > out > > > :-) > > > > In the case of MongoDB this method is overridden to catch a few > > optimized > > > > corner cases, which is fine. But in case those corner cases are not > > met, > > > it > > > > delegates to super.executeQuery(Query), which in turn will delegate > to > > > > materializeMainSchemaTable(...) then! That is why, if you implement > > > > materializeMainSchemaTable(...) then at least you are 100% > functional, > > > > because MM can then always revert to doing any filtering and > > aggregation > > > in > > > > memory! It may not be performing well at all, but it's a great start > to > > > be > > > > functional and compatible with the query model. > > > > > > > > I would in deed recommend you catch the corner cases of the WHERE > items > > > > that you can support natively. And also to implement > > > executeCountQuery(...) > > > > would be good. > > > > > > > > Best regards, > > > > Kasper > > > > > > > > 2014-09-12 14:35 GMT+02:00 Alberto Rodriguez <ardl...@gmail.com>: > > > > > > > > > Hi Kasper, > > > > > > > > > > thank you for the feedback! > > > > > > > > > > I am not sure if I fully understand your piece of advice related to > > not > > > > > implement the executeQuery(query) method. I've been having a look > at > > > the > > > > > different modules (jdbc, mongo...) and they do override this > method. > > > I've > > > > > just taken a look at the materializeTable method signature and it > > does > > > > not > > > > > include, for instance, the FilterItem list so I wouldn't have acces > > to > > > > the > > > > > filters within this method. > > > > > > > > > > I've just commited some changes that allow to add simple > > where/isEquals > > > > > clause to the query (although they are inside the executeQuery). > > > > > > > > > > I have also just upgraded the dependency to the latest version > (thank > > > > you!) > > > > > > > > > > Kind regards, > > > > > > > > > > Alberto > > > > > > > > > > 2014-09-12 13:58 GMT+02:00 Kasper Sørensen < > > > > i.am.kasper.soren...@gmail.com > > > > > >: > > > > > > > > > > > Hi Alberto, > > > > > > > > > > > > I gave you project a quick look, and it's already quite awesome > > work! > > > > > :-) I > > > > > > like the way you discover metadata ... I didn't know of that > > > approach, > > > > > so I > > > > > > am looking forward to trying it out. > > > > > > > > > > > > A quick thing I noticed is that your DataContext overrides the > > > > > > executeQuery(Query) method. You probably shouldn't do that, since > > > your > > > > > > superclass does all the hard work. Simply implement the > > > > > > materializeTable(...) method and you're much better off. > > > > > > > > > > > > If you upgrade your dependency to rely on the latest version of > MM > > > > (4.2) > > > > > > then you will even have a lot more helper methods to implement in > > > your > > > > > > DataContext. Such as executePrimaryKeyLookupQuery and others. > Doing > > > > that > > > > > > will quickly get your going to a very broad query support and > > better > > > > > > performance. > > > > > > > > > > > > Best regards, > > > > > > Kasper > > > > > > > > > > > > > > > > > > 2014-09-12 13:27 GMT+02:00 Hans Drexler < > > > > hans.drex...@humaninference.com > > > > > >: > > > > > > > > > > > > > Just a small correction. MetaModel is not my project. I am not > > > nearly > > > > > > > smart enough for that. But I'll do my best to give you some > > > > worthwhile > > > > > > > feedback. > > > > > > > > > > > > > > Cheers, > > > > > > > > > > > > > > Hans > > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Alberto Rodriguez [mailto:ardl...@gmail.com] > > > > > > > Sent: Friday, September 12, 2014 1:03 PM > > > > > > > To: dev@metamodel.incubator.apache.org > > > > > > > Subject: Re: ElasticSearch Module > > > > > > > > > > > > > > Hi Hans, > > > > > > > > > > > > > > no worries!! I think your project is awesome and I would like > to > > > > > > > contribute somehow. > > > > > > > > > > > > > > I will keep working on the module. > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > Alberto > > > > > > > > > > > > > > 2014-09-12 12:58 GMT+02:00 Hans Drexler < > > > > > hans.drex...@humaninference.com > > > > > > >: > > > > > > > > > > > > > > > Hi Alberto, > > > > > > > > > > > > > > > > What great news. I will checkout your code and try to give > > > > comments. > > > > > > > > Thank you for your contributon. > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > > > > > Hans > > > > > > > > > > > > > > > > Op 12 sep. 2014 11:13 schreef Alberto Rodriguez < > > > ardl...@gmail.com > > > > >: > > > > > > > > Hi there, > > > > > > > > > > > > > > > > as per our previous discussion I have been taking a look at > the > > > > > > > > elasticsearch module for metamodel. > > > > > > > > > > > > > > > > I have been working on it and I have a very simple version > > which > > > is > > > > > > > > already fetching the metadata from ES. I have also managed to > > > make > > > > > > > > simple queries work (queries without filters and where > > clauses). > > > > > > > > > > > > > > > > Could you please have a look at the project on my github > > account > > > > and > > > > > > > > let me know what you think about it?: Metamodel-elasticsearch > > > > module > > > > > > > > <https://github.com/ardlema/metamodel-elasticsearch> > > > > > > > > > > > > > > > > Thank you in advance. > > > > > > > > > > > > > > > > Kind regards, > > > > > > > > > > > > > > > > Alberto > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
