Re: Add datastore for Elasticsearch. Outreachy Week 11 Report
Hi Maria, Thank you for hard work Maria. Can you raise a PR, once you are comfortable with changes? Regards Kevin On Thu, Feb 25, 2021 at 10:06 AM Maria Podorvanova < podorvanova.ma...@gmail.com> wrote: > Hi John, > > Thanks for your comment. I am working on it. > > Regards, > Maria > > On Wed, 24 Feb 2021 at 17:50, John Mora wrote: > >> Hi Maria. >> >> Thanks for the update. >> >> Unfortunately, looping through all possible values in the range is not a >> practical solution. >> >> You should use the range query feature for this: >> >> https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html >> >> I think you should manually add a special field in the elasticsearch >> record that you can range query (you can add it to the mapping file as a >> 'mock' primary key field). It will be basically a copy of the '_id' field. >> >> Here, you can find a similar workaround in the Redis DataStore where >> Sorted Sets were as secondary indexes for range queries. >> >> >> https://github.com/apache/gora/blob/master/gora-redis/src/main/java/org/apache/gora/redis/store/RedisStore.java#L299 >> >> Best, >> John >> >> El sáb, 20 feb 2021 a las 3:01, Maria Podorvanova (< >> podorvanova.ma...@gmail.com>) escribió: >> >>> Hi, >>> >>> Report #11 >>> Week 11: February, 14 - February, 20 >>> Activities: >>> - Added scaling_factor support [1] >>> - Removed unsupported Elasticsearch data types [2] >>> - Implemented Metadata Analyzer for Elasticsearch Store [3] >>> - Tried to fix range query by “_id” field [4] >>> - Wrote documentation for Apache Gora website [5] >>> - Polished and sent my CV for reviewing >>> >>> Question: >>> >>>1. I tried to fix the issue, where Elasticsearch "_id" field does >>>not support range queries. I've tried treating "_id" as a number, but one >>>of the test "_id" field values is "http://foo.com/;. So >>>my approach did not work, but I decided to commit[4] my work on this >>> issue >>>in order to show you what I tried to do. >>> >>> >>> [1] >>> https://github.com/apache/gora/commit/670a04c51f4a6d169df319ed5fd3d1d0abd81870 >>> [2] >>> https://github.com/apache/gora/commit/55020d722f9424021fefe8d94b6bf3ece213226d >>> [3] >>> https://github.com/apache/gora/commit/c491a6447d197b0509473294ee844834b1623a63 >>> [4] >>> https://github.com/apache/gora/commit/a870ca8a2075af7cbab75b9341a94de4966fbf7a >>> [5] >>> https://docs.google.com/document/d/1AF6MG3pqe6A5Z0KtLooEKQlipuyeObbYlAa4O7rFXqM/edit?usp=sharing >>> >>> Regards, >>> Maria >>> >>
Re: Add datastore for Elasticsearch. Outreachy Week 11 Report
Hi John, Thanks for your comment. I am working on it. Regards, Maria On Wed, 24 Feb 2021 at 17:50, John Mora wrote: > Hi Maria. > > Thanks for the update. > > Unfortunately, looping through all possible values in the range is not a > practical solution. > > You should use the range query feature for this: > > https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html > > I think you should manually add a special field in the elasticsearch > record that you can range query (you can add it to the mapping file as a > 'mock' primary key field). It will be basically a copy of the '_id' field. > > Here, you can find a similar workaround in the Redis DataStore where > Sorted Sets were as secondary indexes for range queries. > > > https://github.com/apache/gora/blob/master/gora-redis/src/main/java/org/apache/gora/redis/store/RedisStore.java#L299 > > Best, > John > > El sáb, 20 feb 2021 a las 3:01, Maria Podorvanova (< > podorvanova.ma...@gmail.com>) escribió: > >> Hi, >> >> Report #11 >> Week 11: February, 14 - February, 20 >> Activities: >> - Added scaling_factor support [1] >> - Removed unsupported Elasticsearch data types [2] >> - Implemented Metadata Analyzer for Elasticsearch Store [3] >> - Tried to fix range query by “_id” field [4] >> - Wrote documentation for Apache Gora website [5] >> - Polished and sent my CV for reviewing >> >> Question: >> >>1. I tried to fix the issue, where Elasticsearch "_id" field does not >>support range queries. I've tried treating "_id" as a number, but one of >>the test "_id" field values is "http://foo.com/;. So my approach did >>not work, but I decided to commit[4] my work on this issue in order to >> show >>you what I tried to do. >> >> >> [1] >> https://github.com/apache/gora/commit/670a04c51f4a6d169df319ed5fd3d1d0abd81870 >> [2] >> https://github.com/apache/gora/commit/55020d722f9424021fefe8d94b6bf3ece213226d >> [3] >> https://github.com/apache/gora/commit/c491a6447d197b0509473294ee844834b1623a63 >> [4] >> https://github.com/apache/gora/commit/a870ca8a2075af7cbab75b9341a94de4966fbf7a >> [5] >> https://docs.google.com/document/d/1AF6MG3pqe6A5Z0KtLooEKQlipuyeObbYlAa4O7rFXqM/edit?usp=sharing >> >> Regards, >> Maria >> >
Re: Outreachy 2020-2021 - Neo4j - Weekly reports.
Hi Gaby Thanks for your report The delete method should return true when a record is actually deleted. If the key does not exist it should return false. Try to use the executeUpdate method of PreparedStatement. It returns the number of affected rows, if this number is greater than zero you can return true or false otherwise. Best, John El lun, 22 feb 2021 a las 1:09, gabriela ortiz () escribió: > Hi all. > > I wanted to inform the tasks I worked on this week: Feb 13 - Feb 19 . > > * Develop method newQuery. > * Develop classes Neo4jResult and Neo4jQuery. > * Develop method execute(query). > * Activate tests: > testTruncateSchema > testDeleteSchema > testQuery > testQueryStartKey > testQueryEndKey > testQueryKeyRange > testQueryWebPageSingleKey > testQueryWebPageSingleKeyDefaultFields > testQueryWebPageQueryEmptyResults > testDelete > testGetPartitions > testResultSize > testResultSizeStartKey > testResultSizeEndKey > testResultSizeKeyRange > testResultSizeWithLimit > testResultSizeStartKeyWithLimit > testResultSizeEndKeyWithLimit > testResultSizeKeyRangeWithLimit > > Also, I have a question, When the method Neo4jStore#delete(key) should > return true or false?, I found out that I had to always return true in > order to pass the tests, is that correct? > > My code is here: https://github.com/mgov88/gora/tree/GORA-663 > > Regards, > Gaby > > El lun, 15 de feb. de 2021 a la(s) 19:01, John Mora (jhnmora...@gmail.com) > escribió: > >> Hi Gaby >> >> Thanks for the update. >> >> Overall the code looks good, I do not have specific feedback for you this >> week. >> >> According to your proposed timeline you should start working on the Query >> features, let's do it. Let me know if you have questions. >> >> >> Thanks, >> John >> >> El sáb, 13 feb 2021 a las 0:57, gabriela ortiz () >> escribió: >> >>> Hi all. >>> >>> I wanted to inform the tasks I worked on this week: Feb 06 - Feb 12 . >>> >>> * Enhance variable names. >>> * Add enum for neo4j protocols. >>> * Enhance getUnionSchema method for Maps. >>> * Implement partitons. >>> * Activate tests: >>> testUpdate >>> testGetRecursive >>> testGetDoubleRecursive >>> testGetWebPage >>> testGetWebPageDefaultFields >>> >>> Also, I started working on my C.V. >>> >>> My code is here: https://github.com/mgov88/gora/tree/GORA-663 >>> >>> Regards, >>> Gaby >>> >>> El mié, 10 de feb. de 2021 a la(s) 21:33, gabriela ortiz ( >>> arqgabyor...@gmail.com) escribió: >>> Hi John. Thanks for the feedback I will work on your comments. Regards, Gaby El mié, 10 de feb. de 2021 a la(s) 12:04, John Mora ( jhnmora...@gmail.com) escribió: > Hi Gaby > > Thanks for the update. > > BTW, I am sorry that I did not provide feedback on your code last > week, I have been busy. > > Some comments: > > Please use more descriptive variable names: > > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L368 > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L165 > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L171 > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L193 > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L194 > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L200 > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L206 > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L216 > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L216 > > Typo: > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L216 > > Avoid string concatenation: > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L307 > > Use an Enum instead of string literals, > > https://github.com/mgov88/gora/blob/GORA-663/gora-neo4j/src/main/java/org/apache/gora/neo4j/store/Neo4jStore.java#L129 > > > regards > John > > El lun, 8 feb 2021 a las 2:08, gabriela ortiz () > escribió: > >> Hi all. >> >> I wanted to inform the tasks I worked on this week: Jan 30 - Feb 05 . >> >> * Enhance the deleteSchema method (delete existing nodes when >> deleting the schema constraints) >> * Enhance Map, Record, Array and Bytes serialization /
