Custom FieldType not getting called while querying to get the field with custom fieldtype
Hi All, I am trying to fetch values (docValues to be exact) from external redis during query time using a custom written FieldType which I am loading in my solr instance. I notice that my custom FieldType is being initialized based on the schema when loading the core. I also notice that during query time, my custom field type code is called but the results back from solr do not populate the custom field even if I explicitly request it. Also, I am asking the results to be sorted by the custom field but results are not sorted. Solr version: 7.3.1 Here is my implementation: https://gist.github.com/vermaharsh/042e1cf07070c6d9b3b6cc7eaaf0b49c Here is my solrconfig.xml: https://gist.github.com/vermaharsh/97d7310b242fd7ba3d8d3a3bda209ac3 Here is my managed-schema: https://gist.github.com/vermaharsh/8a89195377802a6bbcdde9215a2fdaf5 Query that I am making: /solr/redis/select?fl=hits=*:*=hits%20asc Response that I get back: https://gist.github.com/vermaharsh/2f63282b10320c4c35a9f85016fe30c0 Another query with debug flag: solr/redis/select?debugQuery=on=*,hits=*:*=hits%20desc Response: https://gist.github.com/vermaharsh/f7f74a65a5403ecec9310ceb81cb674c Sort order should be - 10, 9, 8, 6, 5, 4, 3, 2, 1, 7; because, corresponding hits value from redis are - 1, 900, 800, 600, 500, 400, 300, 200, 100, 7. Can someone help me identify what am I missing? Thanks, Harsh Copy of discussion on #solr irc channel. I am trying to write a custom solr FieldType which fetches value from external redis instance. I got Solr to load my custom jars and have defined the schema with custom fieldType class. But at query time, I cannot get values in the response. [08:38] I do not see any errors in the logs as well, so cannot tell if something failed [08:38] here is the code for my custom FieldType - https://gist.github.com/vermaharsh/042e1cf07070c6d9b3b6cc7eaaf0b49c [08:39] this is my solrconfig.xml for my custom configset - https://gist.github.com/vermaharsh/97d7310b242fd7ba3d8d3a3bda209ac3 [08:42] I basically placed the necessary jars under contrib/redis-field [08:43] and my managed-schema - https://gist.github.com/vermaharsh/8a89195377802a6bbcdde9215a2fdaf5 [08:43] I am using solr version 7.3.1 [08:44] any idea why I am not getting the value for my custom field type back? [08:45] <@elyograg> harsh: I'm not familiar with the API to that level. FYI, this should go in #solr -- this channel is for development of Solr itself. [08:46] <@elyograg> the field is not marked as stored. I wonder if that might tell Solr that it shouldn't be returned at all. (I don't know whether setting stored=true might require something more in your code, though) [08:46] I am using it as docValue, but I can try stored. Though, as you mentioned, not sure if that would need more to be implemented in the code [08:47] I will try #solr channel for the question as well [08:47] <@elyograg> ah, I didn't scroll to the right enough to see that part. :) [08:48] <@elyograg> I wonder if you might need useDocValuesAsStored="true". [08:49] I thought that is the default value [08:51] for completeness, this is the query that I am using - /solr/redis/select?fl=hits=*:*=hits%20asc [08:52] and response that I got back - https://gist.github.com/vermaharsh/2f63282b10320c4c35a9f85016fe30c0 [08:53] == dataminion [~le...@c-69-181-118-61.hsd1.ca.comcast.net] has joined #solr-dev [08:53] <@elyograg> this one's probably going to need to go to the mailing list. with all the pastes you've mentioned here. [08:57] == dataminion [~le...@c-69-181-118-61.hsd1.ca.comcast.net] has quit [Ping timeout: 264 seconds] [08:59] <@elyograg> you may be right about that being default. The code in FieldType.java seems to support that. [09:00] <@elyograg> if (schemaVersion >= 1.6f) properties |= USE_DOCVALUES_AS_STORED; [09:02] alright, thanks elyograg for looking into it. I will send this to mailing list as well.
RE: Custom FieldType not getting called while querying to get the field with custom fieldtype
Not sure if I am on the mailing list. Trying again. Also additional info. I added more logs to my custom field type and now I see that my code is called. But, still I cannot get the value to be populated back in response. Also, the sorting is not working, even though I see that correct values are fetched from redis. Here is sample response for query - solr/redis/select?debugQuery=on=*,hits=*:*=hits%20desc Response - https://gist.github.com/vermaharsh/f7f74a65a5403ecec9310ceb81cb674c Sort order should be - 10, 9, 8, 6, 5, 4, 3, 2, 1, 7; because, corresponding hits value from redis are - 1, 900, 800, 600, 500, 400, 300, 200, 100, 7. From: Harsh Verma Sent: Wednesday, June 27, 2018 11:40 AM To: dev@lucene.apache.org Subject: Custom FieldType not getting called while querying to get the field with custom fieldtype Hi All, I am trying to fetch values (docValues to be exact) from external redis during query time using a custom written FieldType which I am loading in my solr instance. I notice that my custom FieldType is being initialized based on the schema when loading the core. However, at query time, my custom code is not called and hence the response do not have the field from redis populated. Solr version: 7.3.1 Here is my implementation: https://gist.github.com/vermaharsh/042e1cf07070c6d9b3b6cc7eaaf0b49c Here is my solrconfig.xml: https://gist.github.com/vermaharsh/97d7310b242fd7ba3d8d3a3bda209ac3 Here is my managed-schema: https://gist.github.com/vermaharsh/8a89195377802a6bbcdde9215a2fdaf5 Query that I am making: /solr/redis/select?fl=hits=*:*=hits%20asc Response that I get back: https://gist.github.com/vermaharsh/2f63282b10320c4c35a9f85016fe30c0 Can someone help me identify what am I missing? Thanks, Harsh Copy of discussion on #solr irc channel. I am trying to write a custom solr FieldType which fetches value from external redis instance. I got Solr to load my custom jars and have defined the schema with custom fieldType class. But at query time, I cannot get values in the response. [08:38] I do not see any errors in the logs as well, so cannot tell if something failed [08:38] here is the code for my custom FieldType - https://gist.github.com/vermaharsh/042e1cf07070c6d9b3b6cc7eaaf0b49c [08:39] this is my solrconfig.xml for my custom configset - https://gist.github.com/vermaharsh/97d7310b242fd7ba3d8d3a3bda209ac3 [08:42] I basically placed the necessary jars under contrib/redis-field [08:43] and my managed-schema - https://gist.github.com/vermaharsh/8a89195377802a6bbcdde9215a2fdaf5 [08:43] I am using solr version 7.3.1 [08:44] any idea why I am not getting the value for my custom field type back? [08:45] <@elyograg> harsh: I'm not familiar with the API to that level. FYI, this should go in #solr -- this channel is for development of Solr itself. [08:46] <@elyograg> the field is not marked as stored. I wonder if that might tell Solr that it shouldn't be returned at all. (I don't know whether setting stored=true might require something more in your code, though) [08:46] I am using it as docValue, but I can try stored. Though, as you mentioned, not sure if that would need more to be implemented in the code [08:47] I will try #solr channel for the question as well [08:47] <@elyograg> ah, I didn't scroll to the right enough to see that part. :) [08:48] <@elyograg> I wonder if you might need useDocValuesAsStored="true". [08:49] I thought that is the default value [08:51] for completeness, this is the query that I am using - /solr/redis/select?fl=hits=*:*=hits%20asc [08:52] and response that I got back - https://gist.github.com/vermaharsh/2f63282b10320c4c35a9f85016fe30c0 [08:53] == dataminion [~le...@c-69-181-118-61.hsd1.ca.comcast.net] has joined #solr-dev [08:53] <@elyograg> this one's probably going to need to go to the mailing list. with all the pastes you've mentioned here. [08:57] == dataminion [~le...@c-69-181-118-61.hsd1.ca.comcast.net] has quit [Ping timeout: 264 seconds] [08:59] <@elyograg> you may be right about that being default. The code in FieldType.java seems to support that. [09:00] <@elyograg> if (schemaVersion >= 1.6f) properties |= USE_DOCVALUES_AS_STORED; [09:02] alright, thanks elyograg for looking into it. I will send this to mailing list as well.
Custom FieldType not getting called while querying to get the field with custom fieldtype
Hi All, I am trying to fetch values (docValues to be exact) from external redis during query time using a custom written FieldType which I am loading in my solr instance. I notice that my custom FieldType is being initialized based on the schema when loading the core. However, at query time, my custom code is not called and hence the response do not have the field from redis populated. Solr version: 7.3.1 Here is my implementation: https://gist.github.com/vermaharsh/042e1cf07070c6d9b3b6cc7eaaf0b49c Here is my solrconfig.xml: https://gist.github.com/vermaharsh/97d7310b242fd7ba3d8d3a3bda209ac3 Here is my managed-schema: https://gist.github.com/vermaharsh/8a89195377802a6bbcdde9215a2fdaf5 Query that I am making: /solr/redis/select?fl=hits=*:*=hits%20asc Response that I get back: https://gist.github.com/vermaharsh/2f63282b10320c4c35a9f85016fe30c0 Can someone help me identify what am I missing? Thanks, Harsh Copy of discussion on #solr irc channel. I am trying to write a custom solr FieldType which fetches value from external redis instance. I got Solr to load my custom jars and have defined the schema with custom fieldType class. But at query time, I cannot get values in the response. [08:38] I do not see any errors in the logs as well, so cannot tell if something failed [08:38] here is the code for my custom FieldType - https://gist.github.com/vermaharsh/042e1cf07070c6d9b3b6cc7eaaf0b49c [08:39] this is my solrconfig.xml for my custom configset - https://gist.github.com/vermaharsh/97d7310b242fd7ba3d8d3a3bda209ac3 [08:42] I basically placed the necessary jars under contrib/redis-field [08:43] and my managed-schema - https://gist.github.com/vermaharsh/8a89195377802a6bbcdde9215a2fdaf5 [08:43] I am using solr version 7.3.1 [08:44] any idea why I am not getting the value for my custom field type back? [08:45] <@elyograg> harsh: I'm not familiar with the API to that level. FYI, this should go in #solr -- this channel is for development of Solr itself. [08:46] <@elyograg> the field is not marked as stored. I wonder if that might tell Solr that it shouldn't be returned at all. (I don't know whether setting stored=true might require something more in your code, though) [08:46] I am using it as docValue, but I can try stored. Though, as you mentioned, not sure if that would need more to be implemented in the code [08:47] I will try #solr channel for the question as well [08:47] <@elyograg> ah, I didn't scroll to the right enough to see that part. :) [08:48] <@elyograg> I wonder if you might need useDocValuesAsStored="true". [08:49] I thought that is the default value [08:51] for completeness, this is the query that I am using - /solr/redis/select?fl=hits=*:*=hits%20asc [08:52] and response that I got back - https://gist.github.com/vermaharsh/2f63282b10320c4c35a9f85016fe30c0 [08:53] == dataminion [~le...@c-69-181-118-61.hsd1.ca.comcast.net] has joined #solr-dev [08:53] <@elyograg> this one's probably going to need to go to the mailing list. with all the pastes you've mentioned here. [08:57] == dataminion [~le...@c-69-181-118-61.hsd1.ca.comcast.net] has quit [Ping timeout: 264 seconds] [08:59] <@elyograg> you may be right about that being default. The code in FieldType.java seems to support that. [09:00] <@elyograg> if (schemaVersion >= 1.6f) properties |= USE_DOCVALUES_AS_STORED; [09:02] alright, thanks elyograg for looking into it. I will send this to mailing list as well.
