Re: Wiki Error JSON syntax
On Mon, Jul 25, 2011 at 12:24 PM, Stefan Matheis
matheis.ste...@googlemail.com wrote:
Hi Remy,
so you may open an Issue for this on the github Project? i mean .. just
creating another client, because i have one problem, does not sound like a
good plan?
Agreed, and thanks for calling my attention to this thread, Stefan.
Regards
Stefan
Am 25.07.2011 10:56, schrieb Remy Loubradou:
Hey Stephan,
Thanks, but I already used this solr client and I got an error when I add
too much documents FATAL ERROR: JS Allocation failed - process out of
memory.
I didn't find the source of the problem in the solr client. So I decided
to
write my own without this error hopefully and also I'm using JSON
documents
and not XML documents. I read a post saying that I can get better
performance using JSON documents.
I will release this client as an npm module.
How many documents are you attempting to add at once when you get that
error? Would it possible to chunk them into smaller groups?
I'm happy to work with you on enhancing node-solr to meet your needs.
The only reason updates are via XML rather than JSON is that 3.1 was
new or not yet released (don't quite remember which) when I first
wrote node-solr. Even now I imagine many people may still be using a
version of Solr that doesn't handle JSON updates. Maybe a flag
parameter could be added to the Client object to switch from XML to
JSON?
The node-solr client has fairly complete test coverage, a history of
commits from the Node community, and versions aligning with several
versions of Node. I would appreciate your contributions, either via
issues or pull requests.
Regards,
Remy
2011/7/25 Stefan Matheismatheis.ste...@googlemail.com
Remy,
didn't use it myself .. but you know about https://github.com/gsf/node-**
solrhttps://github.com/gsf/node-solr ?
Regards
Stefan
Am 20.07.2011 20:05, schrieb Remy Loubradou:
I think I can trust you but this is weird.
Funny things if you try to validate on http://jsonlint.com/ this JSON,
duplicates keys are automatically removed. But the thing is, how can you
possibly generate this json with Javascript Object?
It will be really nice to combine both ways that you show on the page.
Something like:
{
add: [
{
doc: {
id: DOC1,
my_boosted_field: {
boost: 2.3,
value: test
},
my_multivalued_field: [
aaa,
bbb
]
}
},
{
commitWithin: 5000,
overwrite: false,
boost: 3.45,
doc: {
f1: v2
}
}
],
commit: {},
optimize: {
waitFlush: false,
waitSearcher: false
},
delete: [
{
id: ID
},
{
query: QUERY
}
]
}
Thanks you for you previous response Yonik.
2011/7/20 Yonik
Seeleyyonik@lucidimagination.**comyo...@lucidimagination.com
On Wed, Jul 20, 2011 at 12:16 PM, Remy Loubradou
remyloubra...@gmail.com wrote:
Hi,
I was writing a Solr Client API for Node and I found an error on this
page
http://wiki.apache.org/solr/**UpdateJSONhttp://wiki.apache.org/solr/UpdateJSON,on
the section Update Commands
the
JSON is not valid because there are duplicate keys and two times with
add
and delete.
It's a common misconception that it's invalid JSON. Duplicate keys
are in fact legal.
I can't resist addressing this side conversation.
While I understand the desire for a straightforward mapping between
the XML and JSON update formats, I think the use of duplicate keys is
a bad idea. As noted in the spec
(http://www.ietf.org/rfc/rfc4627.txt), The names within an object
SHOULD be unique. I'm not sure the reasons here justify ignoring that
recommendation.
The fact that you need to keep reminding people that duplicate names
are legal is a sign that it's more trouble than it's worth. Also, most
JSON parsers just punt on duplicate names (see the third paragraph in
A word about design at
http://planet.plt-scheme.org/package-source/dherman/json.plt/3/0/planet-docs/json/index.html
for one take on the situation). I really don't want to write a new
JavaScript JSON parser just for node-solr.
Re: Wiki Error JSON syntax
Hi,
2011/7/25 Gabriel Farrell gsf...@gmail.com
On Mon, Jul 25, 2011 at 12:24 PM, Stefan Matheis
matheis.ste...@googlemail.com wrote:
Hi Remy,
so you may open an Issue for this on the github Project? i mean .. just
creating another client, because i have one problem, does not sound like
a
good plan?
Agreed, and thanks for calling my attention to this thread, Stefan.
Yes I'm agree too, I'm not used to, pull, submit issue it's new for me :). I
will publish an issue.
Regards
Stefan
Am 25.07.2011 10:56, schrieb Remy Loubradou:
Hey Stephan,
Thanks, but I already used this solr client and I got an error when I
add
too much documents FATAL ERROR: JS Allocation failed - process out of
memory.
I didn't find the source of the problem in the solr client. So I decided
to
write my own without this error hopefully and also I'm using JSON
documents
and not XML documents. I read a post saying that I can get better
performance using JSON documents.
I will release this client as an npm module.
How many documents are you attempting to add at once when you get that
error? Would it possible to chunk them into smaller groups?
I add a document and commit after. But it's a very XML document ~130MB and
that's happening only with big XML file.
I got this error FATAL ERROR: JS Allocation failed - process out of memory .
I'm happy to work with you on enhancing node-solr to meet your needs.
The only reason updates are via XML rather than JSON is that 3.1 was
new or not yet released (don't quite remember which) when I first
wrote node-solr. Even now I imagine many people may still be using a
version of Solr that doesn't handle JSON updates. Maybe a flag
parameter could be added to the Client object to switch from XML to
JSON?
Yes, this could be a good solution. So we can try to merge your solr-client
with my client.
Before to release my client I want to have full test coverage with vows
(have some fun :) )
The node-solr client has fairly complete test coverage, a history of
commits from the Node community, and versions aligning with several
versions of Node. I would appreciate your contributions, either via
issues or pull requests.
Cool. I will be pleased to do that!
Regards,
Remy
2011/7/25 Stefan Matheismatheis.ste...@googlemail.com
Remy,
didn't use it myself .. but you know about
https://github.com/gsf/node-**
solrhttps://github.com/gsf/node-solr ?
Regards
Stefan
Am 20.07.2011 20:05, schrieb Remy Loubradou:
I think I can trust you but this is weird.
Funny things if you try to validate on http://jsonlint.com/ this
JSON,
duplicates keys are automatically removed. But the thing is, how can
you
possibly generate this json with Javascript Object?
It will be really nice to combine both ways that you show on the page.
Something like:
{
add: [
{
doc: {
id: DOC1,
my_boosted_field: {
boost: 2.3,
value: test
},
my_multivalued_field: [
aaa,
bbb
]
}
},
{
commitWithin: 5000,
overwrite: false,
boost: 3.45,
doc: {
f1: v2
}
}
],
commit: {},
optimize: {
waitFlush: false,
waitSearcher: false
},
delete: [
{
id: ID
},
{
query: QUERY
}
]
}
Thanks you for you previous response Yonik.
2011/7/20 Yonik
Seeleyyonik@lucidimagination.**comyo...@lucidimagination.com
On Wed, Jul 20, 2011 at 12:16 PM, Remy Loubradou
remyloubra...@gmail.com wrote:
Hi,
I was writing a Solr Client API for Node and I found an error on
this
page
http://wiki.apache.org/solr/**UpdateJSON
http://wiki.apache.org/solr/UpdateJSON,on
the section Update Commands
the
JSON is not valid because there are duplicate keys and two times
with
add
and delete.
It's a common misconception that it's invalid JSON. Duplicate keys
are in fact legal.
I can't resist addressing this side conversation.
While I understand the desire for a straightforward mapping between
the XML and JSON update formats, I think the use of duplicate keys is
a bad idea. As noted in the spec
(http://www.ietf.org/rfc/rfc4627.txt), The names within an object
SHOULD be unique. I'm not sure the reasons here justify ignoring that
recommendation.
The fact that you need to keep reminding people that duplicate names
are legal is a sign that it's more trouble than it's worth. Also, most
JSON parsers just punt on duplicate names (see the third paragraph in
A word about design at
Re: Wiki Error JSON syntax
Hey Stephan,
Thanks, but I already used this solr client and I got an error when I add
too much documents FATAL ERROR: JS Allocation failed - process out of
memory.
I didn't find the source of the problem in the solr client. So I decided to
write my own without this error hopefully and also I'm using JSON documents
and not XML documents. I read a post saying that I can get better
performance using JSON documents.
I will release this client as an npm module.
Regards,
Remy
2011/7/25 Stefan Matheis matheis.ste...@googlemail.com
Remy,
didn't use it myself .. but you know about https://github.com/gsf/node-**
solr https://github.com/gsf/node-solr ?
Regards
Stefan
Am 20.07.2011 20:05, schrieb Remy Loubradou:
I think I can trust you but this is weird.
Funny things if you try to validate on http://jsonlint.com/ this JSON,
duplicates keys are automatically removed. But the thing is, how can you
possibly generate this json with Javascript Object?
It will be really nice to combine both ways that you show on the page.
Something like:
{
add: [
{
doc: {
id: DOC1,
my_boosted_field: {
boost: 2.3,
value: test
},
my_multivalued_field: [
aaa,
bbb
]
}
},
{
commitWithin: 5000,
overwrite: false,
boost: 3.45,
doc: {
f1: v2
}
}
],
commit: {},
optimize: {
waitFlush: false,
waitSearcher: false
},
delete: [
{
id: ID
},
{
query: QUERY
}
]
}
Thanks you for you previous response Yonik.
2011/7/20 Yonik
Seeleyyonik@lucidimagination.**comyo...@lucidimagination.com
On Wed, Jul 20, 2011 at 12:16 PM, Remy Loubradou
remyloubra...@gmail.com wrote:
Hi,
I was writing a Solr Client API for Node and I found an error on this
page
http://wiki.apache.org/solr/**UpdateJSONhttp://wiki.apache.org/solr/UpdateJSON,on
the section Update Commands
the
JSON is not valid because there are duplicate keys and two times with
add
and delete.
It's a common misconception that it's invalid JSON. Duplicate keys
are in fact legal.
-Yonik
http://www.lucidimagination.**com http://www.lucidimagination.com
I tried with an array and it doesn't work as well, I got error
400, I think that's because the syntax is bad.
I don't really know if I am at the good place to talk about that but ...
that the only place I found. Sorry if it's not.
Thanks,
And I love Solr :)
Re: Wiki Error JSON syntax
Hi Remy,
so you may open an Issue for this on the github Project? i mean .. just
creating another client, because i have one problem, does not sound like
a good plan?
Regards
Stefan
Am 25.07.2011 10:56, schrieb Remy Loubradou:
Hey Stephan,
Thanks, but I already used this solr client and I got an error when I add
too much documents FATAL ERROR: JS Allocation failed - process out of
memory.
I didn't find the source of the problem in the solr client. So I decided to
write my own without this error hopefully and also I'm using JSON documents
and not XML documents. I read a post saying that I can get better
performance using JSON documents.
I will release this client as an npm module.
Regards,
Remy
2011/7/25 Stefan Matheismatheis.ste...@googlemail.com
Remy,
didn't use it myself .. but you know about https://github.com/gsf/node-**
solrhttps://github.com/gsf/node-solr ?
Regards
Stefan
Am 20.07.2011 20:05, schrieb Remy Loubradou:
I think I can trust you but this is weird.
Funny things if you try to validate on http://jsonlint.com/ this JSON,
duplicates keys are automatically removed. But the thing is, how can you
possibly generate this json with Javascript Object?
It will be really nice to combine both ways that you show on the page.
Something like:
{
add: [
{
doc: {
id: DOC1,
my_boosted_field: {
boost: 2.3,
value: test
},
my_multivalued_field: [
aaa,
bbb
]
}
},
{
commitWithin: 5000,
overwrite: false,
boost: 3.45,
doc: {
f1: v2
}
}
],
commit: {},
optimize: {
waitFlush: false,
waitSearcher: false
},
delete: [
{
id: ID
},
{
query: QUERY
}
]
}
Thanks you for you previous response Yonik.
2011/7/20 Yonik Seeleyyonik@lucidimagination.**comyo...@lucidimagination.com
On Wed, Jul 20, 2011 at 12:16 PM, Remy Loubradou
remyloubra...@gmail.com wrote:
Hi,
I was writing a Solr Client API for Node and I found an error on this
page
http://wiki.apache.org/solr/**UpdateJSONhttp://wiki.apache.org/solr/UpdateJSON,on the
section Update Commands
the
JSON is not valid because there are duplicate keys and two times with
add
and delete.
It's a common misconception that it's invalid JSON. Duplicate keys
are in fact legal.
-Yonik
http://www.lucidimagination.**comhttp://www.lucidimagination.com
I tried with an array and it doesn't work as well, I got error
400, I think that's because the syntax is bad.
I don't really know if I am at the good place to talk about that but ...
that the only place I found. Sorry if it's not.
Thanks,
And I love Solr :)
Re: Wiki Error JSON syntax
Remy,
didn't use it myself .. but you know about
https://github.com/gsf/node-solr ?
Regards
Stefan
Am 20.07.2011 20:05, schrieb Remy Loubradou:
I think I can trust you but this is weird.
Funny things if you try to validate on http://jsonlint.com/ this JSON,
duplicates keys are automatically removed. But the thing is, how can you
possibly generate this json with Javascript Object?
It will be really nice to combine both ways that you show on the page.
Something like:
{
add: [
{
doc: {
id: DOC1,
my_boosted_field: {
boost: 2.3,
value: test
},
my_multivalued_field: [
aaa,
bbb
]
}
},
{
commitWithin: 5000,
overwrite: false,
boost: 3.45,
doc: {
f1: v2
}
}
],
commit: {},
optimize: {
waitFlush: false,
waitSearcher: false
},
delete: [
{
id: ID
},
{
query: QUERY
}
]
}
Thanks you for you previous response Yonik.
2011/7/20 Yonik Seeleyyo...@lucidimagination.com
On Wed, Jul 20, 2011 at 12:16 PM, Remy Loubradou
remyloubra...@gmail.com wrote:
Hi,
I was writing a Solr Client API for Node and I found an error on this
page
http://wiki.apache.org/solr/UpdateJSON ,on the section Update Commands
the
JSON is not valid because there are duplicate keys and two times with
add
and delete.
It's a common misconception that it's invalid JSON. Duplicate keys
are in fact legal.
-Yonik
http://www.lucidimagination.com
I tried with an array and it doesn't work as well, I got error
400, I think that's because the syntax is bad.
I don't really know if I am at the good place to talk about that but ...
that the only place I found. Sorry if it's not.
Thanks,
And I love Solr :)
Wiki Error JSON syntax
Hi, I was writing a Solr Client API for Node and I found an error on this page http://wiki.apache.org/solr/UpdateJSON ,on the section Update Commands the JSON is not valid because there are duplicate keys and two times with add and delete.I tried with an array and it doesn't work as well, I got error 400, I think that's because the syntax is bad. I don't really know if I am at the good place to talk about that but ... that the only place I found. Sorry if it's not. Thanks, And I love Solr :)
Re: Wiki Error JSON syntax
On Wed, Jul 20, 2011 at 12:16 PM, Remy Loubradou remyloubra...@gmail.com wrote: Hi, I was writing a Solr Client API for Node and I found an error on this page http://wiki.apache.org/solr/UpdateJSON ,on the section Update Commands the JSON is not valid because there are duplicate keys and two times with add and delete. It's a common misconception that it's invalid JSON. Duplicate keys are in fact legal. -Yonik http://www.lucidimagination.com I tried with an array and it doesn't work as well, I got error 400, I think that's because the syntax is bad. I don't really know if I am at the good place to talk about that but ... that the only place I found. Sorry if it's not. Thanks, And I love Solr :)
Re: Wiki Error JSON syntax
I think I can trust you but this is weird.
Funny things if you try to validate on http://jsonlint.com/ this JSON,
duplicates keys are automatically removed. But the thing is, how can you
possibly generate this json with Javascript Object?
It will be really nice to combine both ways that you show on the page.
Something like:
{
add: [
{
doc: {
id: DOC1,
my_boosted_field: {
boost: 2.3,
value: test
},
my_multivalued_field: [
aaa,
bbb
]
}
},
{
commitWithin: 5000,
overwrite: false,
boost: 3.45,
doc: {
f1: v2
}
}
],
commit: {},
optimize: {
waitFlush: false,
waitSearcher: false
},
delete: [
{
id: ID
},
{
query: QUERY
}
]
}
Thanks you for you previous response Yonik.
2011/7/20 Yonik Seeley yo...@lucidimagination.com
On Wed, Jul 20, 2011 at 12:16 PM, Remy Loubradou
remyloubra...@gmail.com wrote:
Hi,
I was writing a Solr Client API for Node and I found an error on this
page
http://wiki.apache.org/solr/UpdateJSON ,on the section Update Commands
the
JSON is not valid because there are duplicate keys and two times with
add
and delete.
It's a common misconception that it's invalid JSON. Duplicate keys
are in fact legal.
-Yonik
http://www.lucidimagination.com
I tried with an array and it doesn't work as well, I got error
400, I think that's because the syntax is bad.
I don't really know if I am at the good place to talk about that but ...
that the only place I found. Sorry if it's not.
Thanks,
And I love Solr :)
