[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13610419#comment-13610419 ] Commit Tag Bot commented on SOLR-4112: -- [branch_4x commit] Shalin Shekhar Mangar http://svn.apache.org/viewvc?view=revisionrevision=1432048 SOLR-4112: Fixed DataImportHandler ZKAwarePropertiesWriter implementation so import works fine with SolrCloud clusters Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: Shalin Shekhar Mangar Priority: Blocker Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch, SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
Hi Erick, There is a difference in *.fdt and *.fdx files between 14G and 7G files. I have enclosed the sizes. I also did add fl=score to the queries and lazy load is configured in solrconfig.xml but still the performance issues exist. I am measuring QTime. I am seeing a huge CPU spike in the machine when searching 100 QPS here is the load average: load average: 86.90, 134.85, 79.98 Here is a comparison between v4.0 and v4.1 in a cloud setup QPS v4.0 (msec) v4.1 (msec) 35 27 46.7 70 24 290 Some timeout errorserrors 128 26 Error 240 26.4Did not try Please let us know how to solve this issue. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/jira-Created-SOLR-4112-Dataimporting-with-SolrCloud-Fails-tp4022365p4033952.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
Here are the *.fdt *.fdx file details: 14G files -rw-rw 1 root root 7784904347 Jan 11 18:38 _12n7.fdt -rw-rw 1 root root 82947778 Jan 11 18:38 _12n7.fdx 7G files -rw-rw 1 root root 3162548974 Jan 11 18:57 _43v.fdt -rw-rw 1 root root1052674 Jan 11 18:56 _43v.fdx -rw-rw 1 root root 157718173 Jan 11 18:57 _44y.fdt -rw-rw 1 root root 46891 Jan 11 18:57 _44y.fdx -rw-rw 1 root root 749326180 Jan 11 18:56 _50u.fdt -rw-rw 1 root root 225013 Jan 11 18:56 _50u.fdx -rw-rw 1 root root 64541680 Jan 11 18:56 _545.fdt -rw-rw 1 root root 19206 Jan 11 18:56 _545.fdx -rw-rw 1 root root 115060275 Jan 11 18:56 _57i.fdt -rw-rw 1 root root 35375 Jan 11 18:56 _57i.fdx -rw-rw 1 root root 49801042 Jan 11 18:56 _5ct.fdt -rw-rw 1 root root 15291 Jan 11 18:56 _5ct.fdx -rw-rw 1 root root2555043 Jan 11 18:57 _5fw.fdt -rw-rw 1 root root684 Jan 11 18:57 _5fw.fdx -rw-rw 1 root root 125076704 Jan 11 18:57 _5gq.fdt -rw-rw 1 root root 36049 Jan 11 18:57 _5gq.fdx -rw-rw 1 root root 55056217 Jan 11 18:56 _5kx.fdt -rw-rw 1 root root 16925 Jan 11 18:56 _5kx.fdx -rw-rw 1 root root5057535 Jan 11 18:56 _5lg.fdt -rw-rw 1 root root 1387 Jan 11 18:56 _5lg.fdx -rw-rw 1 root root2465839 Jan 11 18:57 _5n5.fdt -rw-rw 1 root root671 Jan 11 18:57 _5n5.fdx -rw-rw 1 root root3121301 Jan 11 18:57 _5ne.fdt -rw-rw 1 root root801 Jan 11 18:57 _5ne.fdx -rw-rw 1 root root3640966 Jan 11 18:57 _5nz.fdt -rw-rw 1 root root925 Jan 11 18:57 _5nz.fdx -rw-rw 1 root root1407426 Jan 11 18:57 _5o2.fdt -rw-rw 1 root root368 Jan 11 18:57 _5o2.fdx -rw-rw 1 root root1291281 Jan 11 18:56 _5o6.fdt -rw-rw 1 root root342 Jan 11 18:56 _5o6.fdx -rw-rw 1 root root 118825712 Jan 11 18:56 _5o9.fdt -rw-rw 1 root root 36276 Jan 11 18:56 _5o9.fdx -rw-rw 1 root root1172483 Jan 11 18:56 _5oc.fdt -rw-rw 1 root root364 Jan 11 18:56 _5oc.fdx -rw-rw 1 root root 960418 Jan 11 18:56 _5og.fdt -rw-rw 1 root root277 Jan 11 18:57 _5og.fdx -rw-rw 1 root root 742292 Jan 11 18:56 _5om.fdt -rw-rw 1 root root215 Jan 11 18:56 _5om.fdx -rw-rw 1 root root1047978 Jan 11 18:57 _5or.fdt -rw-rw 1 root root287 Jan 11 18:57 _5or.fdx -rw-rw 1 root root2570733 Jan 11 18:56 _5ot.fdt -rw-rw 1 root root696 Jan 11 18:56 _5ot.fdx -rw-rw 1 root root 672436 Jan 11 18:57 _5p0.fdt -rw-rw 1 root root199 Jan 11 18:57 _5p0.fdx -rw-rw 1 root root2938914 Jan 11 18:56 _5p2.fdt -rw-rw 1 root root794 Jan 11 18:56 _5p2.fdx -rw-rw 1 root root 401757 Jan 11 18:57 _5p3.fdt -rw-rw 1 root root128 Jan 11 18:56 _5p3.fdx -- View this message in context: http://lucene.472066.n3.nabble.com/jira-Created-SOLR-4112-Dataimporting-with-SolrCloud-Fails-tp4022365p4033954.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
One more update we did try a master slave setup with v4.0 but the query performance is as bad as the Solr4.1 CPU just spikes after a few minutes of running our test the load average is more than 300. load average: 353.46, 242.38, 141.74 Please suggest how this can be improved. Thanks, -- View this message in context: http://lucene.472066.n3.nabble.com/jira-Created-SOLR-4112-Dataimporting-with-SolrCloud-Fails-tp4022365p4033958.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
The other thing is that we have configured large JVM memory for the Solr process. -XX:+UseParallelGC -XX:NewSize=30g -Xms60g -Xmx60g -Xss256k -server -- View this message in context: http://lucene.472066.n3.nabble.com/jira-Created-SOLR-4112-Dataimporting-with-SolrCloud-Fails-tp4022365p4034006.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551095#comment-13551095 ] Erick Erickson commented on SOLR-4112: -- Shalin: Thanks! Since you did the hard part, want me to check it in? Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: Shalin Shekhar Mangar Priority: Blocker Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch, SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551115#comment-13551115 ] Shalin Shekhar Mangar commented on SOLR-4112: - I'm about to do just that. I was waiting for the whole suite run to finish :) Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: Shalin Shekhar Mangar Priority: Blocker Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch, SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551120#comment-13551120 ] Erick Erickson commented on SOLR-4112: -- Two minor nits then TestZKPropertiesWriter.java has an uncommented System.out.println in it, probably want to take that out/comment? CHANGES.txt could use an entry. FWIW, Erick Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: Shalin Shekhar Mangar Priority: Blocker Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch, SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13551124#comment-13551124 ] Shalin Shekhar Mangar commented on SOLR-4112: - Oops, I committed it already with a change log entry in both trunk and branch_4x. Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: Shalin Shekhar Mangar Priority: Blocker Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch, SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
Sausarkar: When you say the index went from 14G to 7G, did you notice whether the difference was tin the *.fdt and *.fdx files? That would be due to compression of stored fields which is now the default If you could, would you let us know the sizes of the files with those two extensions before after? I'm trying to gather real-world examples... But about your slowdown, does the same thing happen if you specify fl=score (and insure that lazy load is configured in solrconfig.xml)? I don't think that would be reading the fields off disk and decompressing them... what are you measuring? Total time to return to the client? It'd also help pin this down if you looked just at QTime in the responses, that should be exclusive of time to assemble the documents, it's purely searching. Thanks, Erick On Wed, Jan 9, 2013 at 8:50 PM, sausarkar sausar...@ebay.com wrote: We are using solr-meter for generating query load of around 110 Queries per second per node. With 4.1 with the average query time is 300 msec if we switch to 4.0 the average query time is around 11 msec. We used the same load test params and same 10 million records, only differences are the version and index files, 4.1 has 7GB and 4.0 has 14GB. -- View this message in context: http://lucene.472066.n3.nabble.com/jira-Created-SOLR-4112-Dataimporting-with-SolrCloud-Fails-tp4022365p4032084.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
We are using solr-meter for generating query load of around 110 Queries per second per node. With 4.1 with the average query time is 300 msec if we switch to 4.0 the average query time is around 11 msec. We used the same load test params and same 10 million records, only differences are the version and index files, 4.1 has 7GB and 4.0 has 14GB. -- View this message in context: http://lucene.472066.n3.nabble.com/jira-Created-SOLR-4112-Dataimporting-with-SolrCloud-Fails-tp4022365p4032084.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
We are also hitting this DIH issue in 4.1 this issue was not happening in the 4.0 release. Please let us know when this will be resolved, this is delaying our SolrCloud implementation release. btw... We tried the night release from 1/1/2013 but this issue is not resolved. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/jira-Created-SOLR-4112-Dataimporting-with-SolrCloud-Fails-tp4022365p4030158.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
You might try the supplied patch against your 4.1 checkout. Manual verification that it works might prompt a faster commit. Hopefully Shalin or Erick or James will be able to get to it soon. - Mark On Jan 2, 2013, at 6:29 PM, sausarkar sausar...@ebay.com wrote: We are also hitting this DIH issue in 4.1 this issue was not happening in the 4.0 release. Please let us know when this will be resolved, this is delaying our SolrCloud implementation release. btw... We tried the night release from 1/1/2013 but this issue is not resolved. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/jira-Created-SOLR-4112-Dataimporting-with-SolrCloud-Fails-tp4022365p4030158.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
Hi Mark, We have applied the patch and reindexed 10 million records. We did a load test and it seems that the query time per second has increased about 10 times in the 4.1x version. In the 4.0 version we also noticed that the size of the index used to around 14 GB but we noticed this is now 7GB in the 4.1x release. So is the index compressed in 4.1? Can we turn off that for performance improvement. Thanks. From: Mark Miller-3 [via Lucene] ml-node+s472066n4031336...@n3.nabble.commailto:ml-node+s472066n4031336...@n3.nabble.com Date: Monday, January 7, 2013 11:40 AM To: Sarkar, Sauvik sausar...@ebay.commailto:sausar...@ebay.com Subject: Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails You might try the supplied patch against your 4.1 checkout. Manual verification that it works might prompt a faster commit. Hopefully Shalin or Erick or James will be able to get to it soon. - Mark On Jan 2, 2013, at 6:29 PM, sausarkar [hidden email]/user/SendEmail.jtp?type=nodenode=4031336i=0 wrote: We are also hitting this DIH issue in 4.1 this issue was not happening in the 4.0 release. Please let us know when this will be resolved, this is delaying our SolrCloud implementation release. btw... We tried the night release from 1/1/2013 but this issue is not resolved. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/jira-Created-SOLR-4112-Dataimporting-with-SolrCloud-Fails-tp4022365p4030158.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: [hidden email]/user/SendEmail.jtp?type=nodenode=4031336i=1 For additional commands, e-mail: [hidden email]/user/SendEmail.jtp?type=nodenode=4031336i=2 - To unsubscribe, e-mail: [hidden email]/user/SendEmail.jtp?type=nodenode=4031336i=3 For additional commands, e-mail: [hidden email]/user/SendEmail.jtp?type=nodenode=4031336i=4 If you reply to this email, your message will be added to the discussion below: http://lucene.472066.n3.nabble.com/jira-Created-SOLR-4112-Dataimporting-with-SolrCloud-Fails-tp4022365p4031336.html To unsubscribe from [jira] [Created] (SOLR-4112) Dataimporting with SolrCloud Fails, click herehttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4022365code=c2F1c2Fya2FyQGViYXkuY29tfDQwMjIzNjV8LTE0MTU2ODg5MDk=. NAMLhttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml -- View this message in context: http://lucene.472066.n3.nabble.com/jira-Created-SOLR-4112-Dataimporting-with-SolrCloud-Fails-tp4022365p4031398.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
On Jan 7, 2013, at 7:13 PM, sausarkar sausar...@ebay.com wrote: Hi Mark, We have applied the patch and reindexed 10 million records. We did a load test and it seems that the query time per second has increased about 10 times in the 4.1x version. In the 4.0 version we also noticed that the size of the index used to around 14 GB but we noticed this is now 7GB in the 4.1x release. So is the index compressed in 4.1? Can we turn off that for performance improvement. Interesting. It would be good to try and see if this slowdown is indeed due to the compression. It's now the default and you can read more about it here: http://blog.jpountz.net/post/35667727458/stored-fields-compression-in-lucene-4-1 http://blog.jpountz.net/post/33247161884/efficient-compressed-stored-fields-with-lucene If we can confirm that is indeed the issue, we should create a JIRA issue. What are the details of the query load test you are doing? - Mark - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
On 1/7/2013 5:39 PM, Mark Miller wrote: Interesting. It would be good to try and see if this slowdown is indeed due to the compression. It's now the default and you can read more about it here: http://blog.jpountz.net/post/35667727458/stored-fields-compression-in-lucene-4-1 http://blog.jpountz.net/post/33247161884/efficient-compressed-stored-fields-with-lucene If we can confirm that is indeed the issue, we should create a JIRA issue. For me, 4.1 seems to be faster than 3.5.0 - I have no 4.0 data to compare against. I did think when I saw my index drop in size due to compression that an option should be available in Solr to turn compression off but otherwise leave the 4.1 index format alone - I would probably not use it, but I could see instances where others might need to. Thanks, Shawn - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542634#comment-13542634 ] Sauvik Sarkar commented on SOLR-4112: - We are stuck with this issue after we upgraded from 4.0 to 4.1 Please let me know when will this be committed. Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: Shalin Shekhar Mangar Priority: Blocker Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13541364#comment-13541364 ] Erick Erickson commented on SOLR-4112: -- I _really_ want to insure that this gets into 4.1, or explicitly decide not to as part of the release process... so moving back to blocker. If nothing else, we can commit w/o tests. Honest, I spent some time trying to make some tests, but between not knowing much about either ZK or DIH tests, I got pretty lost. Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: James Dyer Priority: Blocker Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13541372#comment-13541372 ] Shalin Shekhar Mangar commented on SOLR-4112: - I can take this up and write some tests. Will make sure that it goes into 4.1 Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: James Dyer Priority: Blocker Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13539270#comment-13539270 ] Erick Erickson commented on SOLR-4112: -- I'm going to go ahead and commit this in a day or two unless someone objects. If I do before some angel creates tests, I'll open up a new JIRA for tests referring back to this one. Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: James Dyer Priority: Blocker Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13532471#comment-13532471 ] Erick Erickson commented on SOLR-4112: -- I've been working with a client that has this exact problem, and this patch cures it on last night's 4x code. If I get the time this weekend, I'd like to 1 try it on trunk 2 make sure all tests work in both 3 commit it to both trunk and 4x. Other then the perennial make some tests too, any objections? James: Since I have the driving need, should I just assign this to myself? Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: James Dyer Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13532498#comment-13532498 ] James Dyer commented on SOLR-4112: -- Erick, I certainly mean for this to be committed prior to a 4.1 (possibly it should be a Blocker), and if you want to commit, then go ahead. My only concern is that if it gets committed without test coverage that a new issue be opened to add a test later. I'm tired of breaking DIH every time I commit a change because the test coverage was inadequate and I was lulled into a false sense of security by All Tests Pass. ZKPropertiesWriter is a case in point. This was committed back in February with no test coverage and then I come along and break it in November. I've been trying lately to write better tests for DIH but if people's attitude is DIH is a low-quality contrib mod...I don't need to write a test, then what's the point? (we all have stories about working with developers who created negative productivity. I'm trying to personally stay out of anyone's story like that here!) Then again my concern about coverage shouldn't hold this hostage so it misses the next release. I also wonder if SOLR-3911 means changing how ZKPropertiesWriter works entirely, or if it is even necessary at all? I don't know enough about SolrCloud (yet...its on my to-do list) to understand how SOLR-3911 might change what needs to be done here, but maybe there is a different, better approach now? Mark, you said something about setting up a base test class to build on, but I was just looking at AbstrackDistribZKTestCase and isn't such a base class already there? Isn't all we need to do is to set up a 1-node, 1-shard cluster and have it import 1 document then open the Properties file with ZKPropertiesWriter and see if it exists and can be read? Someday we can get fancy and expand TestSqlEntityProcessorDelta to sometimes run in full distributed mode but for now something quick and simple would suffice, right? Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: James Dyer Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13532530#comment-13532530 ] Mark Miller commented on SOLR-4112: --- Yup, we already have some base classes that I would use to build on. I would just help get it setup to run from the DIH module and offer any help around getting DIH configured and running with it. If you can easily just figure that out, then by all means, go on ahead. If you have any questions, let me know. Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: James Dyer Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13505762#comment-13505762 ] James Dyer commented on SOLR-4112: -- It looks like all we need for this now is a unit test for ZKPropertiesWriter, which I do not think has any test coverage? (After breaking a bunch of things in the DIH included with Solr 3.6, I'm really trying to improve the test situation with DIH. But I don't know how to write a good test for SolrCloud stuff...) Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: James Dyer Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13505774#comment-13505774 ] Mark Miller commented on SOLR-4112: --- bq. don't know how to write a good test for SolrCloud stuff And I don't know DIH at all - but perhaps I can setup some base test class for you to build on. Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Assignee: James Dyer Fix For: 4.1, 5.0 Attachments: SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4112) Dataimporting with SolrCloud Fails
[ https://issues.apache.org/jira/browse/SOLR-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13504279#comment-13504279 ] Deniz Durmus commented on SOLR-4112: As a feedback, would like to say that the patch that James Dyer prepared seems to solve the exceptions Dataimporting with SolrCloud Fails -- Key: SOLR-4112 URL: https://issues.apache.org/jira/browse/SOLR-4112 Project: Solr Issue Type: Bug Affects Versions: 5.0 Reporter: Deniz Durmus Attachments: SOLR-4112.patch While trying to import data from db on cloud, it shows this in logs: SEVERE: Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:336) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:418) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) Caused by: org.apache.solr.common.cloud.ZooKeeperException: ZkSolrResourceLoader does not support getConfigDir() - likely, what you are trying to do is not supported in ZooKeeper mode at org.apache.solr.cloud.ZkSolrResourceLoader.getConfigDir(ZkSolrResourceLoader.java:100) at org.apache.solr.handler.dataimport.SimplePropertiesWriter.init(SimplePropertiesWriter.java:91) at org.apache.solr.handler.dataimport.ZKPropertiesWriter.init(ZKPropertiesWriter.java:45) at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:334) ... 3 more Exception in thread Thread-306 java.lang.NullPointerException at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:427) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:487) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:468) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
