Re: Unable to index documents using DataImportHandler with MSSQL

2011-11-28 Thread Ian Grainger 
Right.
This is REALLY weird - I've now started from scratch on another
machine (this time Windows 7), and got _exactly_ the same problem !?


On Mon, Nov 28, 2011 at 7:37 AM, Husain, Yavar yhus...@firstam.com wrote:
 Hi Ian

 I am having exactly the same problem what you are having on Win 7 and 2008 
 Server http://lucene.472066.n3.nabble.com/DIH-Strange-Problem-tc3530370.html

 I still have not received any replies which could solve my problem till now. 
 Please do let me know if you have arrived at some solution for your problem.

 Thanks.

 Regards,
 Yavar

 -Original Message-
 From: Ian Grainger [mailto:i...@isfluent.com]
 Sent: Friday, November 25, 2011 10:59 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Unable to index documents using DataImportHandler with MSSQL

 Update on this: I've established:
 * It's not a problem in the DB (I can index from this DB into a Solr
 instance on another server)
 * It's not Tomcat (I get the same problem in Jetty)
 * It's not the schema (I have simplified it to one field)

 That leaves SolrConfig.xml and data-config.

 Only thing changed in SolrConfig.xml is adding:

  lib dir=D:/Software/Solr/example/solr/dist/
 regex=apache-solr-cell-\d.*\.jar /
  lib dir=D:/Software/Solr/example/solr/dist/
 regex=apache-solr-clustering-\d.*\.jar /
  lib dir=D:/Software/Solr/example/solr/dist/
 regex=apache-solr-dataimporthandler-\d.*\.jar /
 requestHandler name=/dataimport
    class=org.apache.solr.handler.dataimport.DataImportHandler
  lst name=defaults
    str name=configD:/Software/Solr/example/solr/conf/data-config.xml/str
  /lst
 /requestHandler

 And data-config.xml is pretty much as attached - except simpler.

 Any help or any advice on how to diagnose would be appreciated!


 On Fri, Nov 25, 2011 at 12:29 PM, Ian Grainger i...@isfluent.com wrote:
 Hi I have copied my Solr config from a working Windows server to a new
 one, and it can't seem to run an import.

 They're both using win server 2008 and SQL 2008R2. This is the data
 importer config

    dataConfig
      dataSource type=JdbcDataSource  name=ds1
            driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
            url=jdbc:sqlserver://localhost;databaseName=DB
            user=Solr
            password=pwd/
      document name=datas
        entity name=data dataSource=ds1 pk=key
        query=EXEC SOLR_COMPANY_SEARCH_DATA
        deltaImportQuery=SELECT * FROM Company_Search_Data WHERE
 [key]='${dataimporter.delta.key}'
        deltaQuery=SELECT [key] FROM Company_Search_Data WHERE modify_dt
 '${dataimporter.last_index_time}'
              field column=WorkDesc_Comments
 name=WorkDesc_Comments_Split /
              field column=WorkDesc_Comments name=WorkDesc_Comments_Edge 
 /
        /entity
      /document
    /dataConfig

 I can use MS SQL Profiler to watch the Solr user log in successfully,
 but then nothing. It doesn't seem to even try and execute the stored
 procedure. Any ideas why this would be working one server and not on
 another?

 FTR the only thing in the tomcat catalina log is:

    org.apache.solr.handler.dataimport.JdbcDataSource$1 call
    INFO: Creating a connection for entity data with URL:
 jdbc:sqlserver://localhost;databaseName=CATLive

 --
 Ian

 i...@isfluent.com
 +44 (0)1223 257903




 --
 Ian

 i...@isfluent.com
 +44 (0)1223 257903
 **
 This message may contain confidential or proprietary information intended 
 only for the use of the
 addressee(s) named above or may contain information that is legally 
 privileged. If you are
 not the intended addressee, or the person responsible for delivering it to 
 the intended addressee,
 you are hereby notified that reading, disseminating, distributing or copying 
 this message is strictly
 prohibited. If you have received this message by mistake, please immediately 
 notify us by
 replying to the message and delete the original message and any copies 
 immediately thereafter.

 Thank you.-
 **
 FAFLD





-- 
Ian

i...@isfluent.com
+44 (0)1223 257903

RE: Unable to index documents using DataImportHandler with MSSQL

2011-11-28 Thread Husain, Yavar 
Hi Ian

I downloaded and build latest Solr (3.4) from sources and finally hit following 
line of code in Solr (where I put my debug statement) :

if(url != null){
   LOG.info(Yavar: getting handle to driver manager:);
   c = DriverManager.getConnection(url, initProps);
   LOG.info(Yavar: got handle to driver manager:);
}

The call to Driver Manager was not returning. Here was the error!! The Driver 
we were using was Microsoft Type 4 JDBC driver for SQL Server. I downloaded 
another driver called jTDS jDBC driver and installed that. Problem got fixed!!!

So please follow the following steps:

1. Download jTDS jDBC driver from http://jtds.sourceforge.net/
2. Put the driver jar file into your Solr/lib directory where you had put 
Microsoft JDBC driver.
3. In the data-config.xml use this statement: 
driver=net.sourceforge.jtds.jdbc.Driver
4. Also in data-config.xml mention url like this: 
url=jdbc:jTDS:sqlserver://localhost:1433;databaseName=XXX
5. Now run your indexing.

It should solve the problem.

Regards,
Yavar

-Original Message-
From: Ian Grainger [mailto:i...@isfluent.com] 
Sent: Monday, November 28, 2011 4:11 PM
To: Husain, Yavar
Cc: solr-user@lucene.apache.org
Subject: Re: Unable to index documents using DataImportHandler with MSSQL

Right.
This is REALLY weird - I've now started from scratch on another
machine (this time Windows 7), and got _exactly_ the same problem !?


On Mon, Nov 28, 2011 at 7:37 AM, Husain, Yavar yhus...@firstam.com wrote:
 Hi Ian

 I am having exactly the same problem what you are having on Win 7 and 2008 
 Server http://lucene.472066.n3.nabble.com/DIH-Strange-Problem-tc3530370.html

 I still have not received any replies which could solve my problem till now. 
 Please do let me know if you have arrived at some solution for your problem.

 Thanks.

 Regards,
 Yavar

 -Original Message-
 From: Ian Grainger [mailto:i...@isfluent.com]
 Sent: Friday, November 25, 2011 10:59 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Unable to index documents using DataImportHandler with MSSQL

 Update on this: I've established:
 * It's not a problem in the DB (I can index from this DB into a Solr
 instance on another server)
 * It's not Tomcat (I get the same problem in Jetty)
 * It's not the schema (I have simplified it to one field)

 That leaves SolrConfig.xml and data-config.

 Only thing changed in SolrConfig.xml is adding:

  lib dir=D:/Software/Solr/example/solr/dist/
 regex=apache-solr-cell-\d.*\.jar /
  lib dir=D:/Software/Solr/example/solr/dist/
 regex=apache-solr-clustering-\d.*\.jar /
  lib dir=D:/Software/Solr/example/solr/dist/
 regex=apache-solr-dataimporthandler-\d.*\.jar /
 requestHandler name=/dataimport
    class=org.apache.solr.handler.dataimport.DataImportHandler
  lst name=defaults
    str name=configD:/Software/Solr/example/solr/conf/data-config.xml/str
  /lst
 /requestHandler

 And data-config.xml is pretty much as attached - except simpler.

 Any help or any advice on how to diagnose would be appreciated!


 On Fri, Nov 25, 2011 at 12:29 PM, Ian Grainger i...@isfluent.com wrote:
 Hi I have copied my Solr config from a working Windows server to a new
 one, and it can't seem to run an import.

 They're both using win server 2008 and SQL 2008R2. This is the data
 importer config

    dataConfig
      dataSource type=JdbcDataSource  name=ds1
            driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
            url=jdbc:sqlserver://localhost;databaseName=DB
            user=Solr
            password=pwd/
      document name=datas
        entity name=data dataSource=ds1 pk=key
        query=EXEC SOLR_COMPANY_SEARCH_DATA
        deltaImportQuery=SELECT * FROM Company_Search_Data WHERE
 [key]='${dataimporter.delta.key}'
        deltaQuery=SELECT [key] FROM Company_Search_Data WHERE modify_dt
 '${dataimporter.last_index_time}'
              field column=WorkDesc_Comments
 name=WorkDesc_Comments_Split /
              field column=WorkDesc_Comments name=WorkDesc_Comments_Edge 
 /
        /entity
      /document
    /dataConfig

 I can use MS SQL Profiler to watch the Solr user log in successfully,
 but then nothing. It doesn't seem to even try and execute the stored
 procedure. Any ideas why this would be working one server and not on
 another?

 FTR the only thing in the tomcat catalina log is:

    org.apache.solr.handler.dataimport.JdbcDataSource$1 call
    INFO: Creating a connection for entity data with URL:
 jdbc:sqlserver://localhost;databaseName=CATLive

 --
 Ian

 i...@isfluent.com
 +44 (0)1223 257903




 --
 Ian

 i...@isfluent.com
 +44 (0)1223 257903
 **
 This message may contain confidential or proprietary information intended 
 only for the use of the
 addressee(s) named above or may contain information that is legally 
 privileged. If you are
 not the intended addressee, or the person responsible for delivering

Re: Unable to index documents using DataImportHandler with MSSQL

2011-11-28 Thread Ian Grainger 
Hah, I've just come on here to suggest you do the same thing! Thanks
for getting back to me - and interesting we both came up with the same
solution!

Now I have the problem that running a delta update updates the
'dataimport.properties' file - but then just re-fetches all the data
regardless! Weird!


On Mon, Nov 28, 2011 at 11:59 AM, Husain, Yavar yhus...@firstam.com wrote:
 Hi Ian

 I downloaded and build latest Solr (3.4) from sources and finally hit 
 following line of code in Solr (where I put my debug statement) :

 if(url != null){
               LOG.info(Yavar: getting handle to driver manager:);
               c = DriverManager.getConnection(url, initProps);
               LOG.info(Yavar: got handle to driver manager:);
 }

 The call to Driver Manager was not returning. Here was the error!! The Driver 
 we were using was Microsoft Type 4 JDBC driver for SQL Server. I downloaded 
 another driver called jTDS jDBC driver and installed that. Problem got 
 fixed!!!

 So please follow the following steps:

 1. Download jTDS jDBC driver from http://jtds.sourceforge.net/
 2. Put the driver jar file into your Solr/lib directory where you had put 
 Microsoft JDBC driver.
 3. In the data-config.xml use this statement: 
 driver=net.sourceforge.jtds.jdbc.Driver
 4. Also in data-config.xml mention url like this: 
 url=jdbc:jTDS:sqlserver://localhost:1433;databaseName=XXX
 5. Now run your indexing.

 It should solve the problem.

 Regards,
 Yavar

 -Original Message-
 From: Ian Grainger [mailto:i...@isfluent.com]
 Sent: Monday, November 28, 2011 4:11 PM
 To: Husain, Yavar
 Cc: solr-user@lucene.apache.org
 Subject: Re: Unable to index documents using DataImportHandler with MSSQL

 Right.
 This is REALLY weird - I've now started from scratch on another
 machine (this time Windows 7), and got _exactly_ the same problem !?


 On Mon, Nov 28, 2011 at 7:37 AM, Husain, Yavar yhus...@firstam.com wrote:
 Hi Ian

 I am having exactly the same problem what you are having on Win 7 and 2008 
 Server http://lucene.472066.n3.nabble.com/DIH-Strange-Problem-tc3530370.html

 I still have not received any replies which could solve my problem till now. 
 Please do let me know if you have arrived at some solution for your problem.

 Thanks.

 Regards,
 Yavar

 -Original Message-
 From: Ian Grainger [mailto:i...@isfluent.com]
 Sent: Friday, November 25, 2011 10:59 PM
 To: solr-user@lucene.apache.org
 Subject: Re: Unable to index documents using DataImportHandler with MSSQL

 Update on this: I've established:
 * It's not a problem in the DB (I can index from this DB into a Solr
 instance on another server)
 * It's not Tomcat (I get the same problem in Jetty)
 * It's not the schema (I have simplified it to one field)

 That leaves SolrConfig.xml and data-config.

 Only thing changed in SolrConfig.xml is adding:

  lib dir=D:/Software/Solr/example/solr/dist/
 regex=apache-solr-cell-\d.*\.jar /
  lib dir=D:/Software/Solr/example/solr/dist/
 regex=apache-solr-clustering-\d.*\.jar /
  lib dir=D:/Software/Solr/example/solr/dist/
 regex=apache-solr-dataimporthandler-\d.*\.jar /
 requestHandler name=/dataimport
    class=org.apache.solr.handler.dataimport.DataImportHandler
  lst name=defaults
    str 
 name=configD:/Software/Solr/example/solr/conf/data-config.xml/str
  /lst
 /requestHandler

 And data-config.xml is pretty much as attached - except simpler.

 Any help or any advice on how to diagnose would be appreciated!


 On Fri, Nov 25, 2011 at 12:29 PM, Ian Grainger i...@isfluent.com wrote:
 Hi I have copied my Solr config from a working Windows server to a new
 one, and it can't seem to run an import.

 They're both using win server 2008 and SQL 2008R2. This is the data
 importer config

    dataConfig
      dataSource type=JdbcDataSource  name=ds1
            driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
            url=jdbc:sqlserver://localhost;databaseName=DB
            user=Solr
            password=pwd/
      document name=datas
        entity name=data dataSource=ds1 pk=key
        query=EXEC SOLR_COMPANY_SEARCH_DATA
        deltaImportQuery=SELECT * FROM Company_Search_Data WHERE
 [key]='${dataimporter.delta.key}'
        deltaQuery=SELECT [key] FROM Company_Search_Data WHERE modify_dt
 '${dataimporter.last_index_time}'
              field column=WorkDesc_Comments
 name=WorkDesc_Comments_Split /
              field column=WorkDesc_Comments 
 name=WorkDesc_Comments_Edge /
        /entity
      /document
    /dataConfig

 I can use MS SQL Profiler to watch the Solr user log in successfully,
 but then nothing. It doesn't seem to even try and execute the stored
 procedure. Any ideas why this would be working one server and not on
 another?

 FTR the only thing in the tomcat catalina log is:

    org.apache.solr.handler.dataimport.JdbcDataSource$1 call
    INFO: Creating a connection for entity data with URL:
 jdbc:sqlserver://localhost;databaseName=CATLive

 --
 Ian

 i...@isfluent.com
 +44 (0)1223 257903

RE: Unable to index documents using DataImportHandler with MSSQL

2011-11-27 Thread Husain, Yavar 
Hi Ian

I am having exactly the same problem what you are having on Win 7 and 2008 
Server http://lucene.472066.n3.nabble.com/DIH-Strange-Problem-tc3530370.html

I still have not received any replies which could solve my problem till now. 
Please do let me know if you have arrived at some solution for your problem. 

Thanks.

Regards,
Yavar

-Original Message-
From: Ian Grainger [mailto:i...@isfluent.com] 
Sent: Friday, November 25, 2011 10:59 PM
To: solr-user@lucene.apache.org
Subject: Re: Unable to index documents using DataImportHandler with MSSQL

Update on this: I've established:
* It's not a problem in the DB (I can index from this DB into a Solr
instance on another server)
* It's not Tomcat (I get the same problem in Jetty)
* It's not the schema (I have simplified it to one field)

That leaves SolrConfig.xml and data-config.

Only thing changed in SolrConfig.xml is adding:

  lib dir=D:/Software/Solr/example/solr/dist/
regex=apache-solr-cell-\d.*\.jar /
  lib dir=D:/Software/Solr/example/solr/dist/
regex=apache-solr-clustering-\d.*\.jar /
  lib dir=D:/Software/Solr/example/solr/dist/
regex=apache-solr-dataimporthandler-\d.*\.jar /
requestHandler name=/dataimport
class=org.apache.solr.handler.dataimport.DataImportHandler
  lst name=defaults
str name=configD:/Software/Solr/example/solr/conf/data-config.xml/str
  /lst
/requestHandler

And data-config.xml is pretty much as attached - except simpler.

Any help or any advice on how to diagnose would be appreciated!


On Fri, Nov 25, 2011 at 12:29 PM, Ian Grainger i...@isfluent.com wrote:
 Hi I have copied my Solr config from a working Windows server to a new
 one, and it can't seem to run an import.

 They're both using win server 2008 and SQL 2008R2. This is the data
 importer config

    dataConfig
      dataSource type=JdbcDataSource  name=ds1
            driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
            url=jdbc:sqlserver://localhost;databaseName=DB
            user=Solr
            password=pwd/
      document name=datas
        entity name=data dataSource=ds1 pk=key
        query=EXEC SOLR_COMPANY_SEARCH_DATA
        deltaImportQuery=SELECT * FROM Company_Search_Data WHERE
 [key]='${dataimporter.delta.key}'
        deltaQuery=SELECT [key] FROM Company_Search_Data WHERE modify_dt
 '${dataimporter.last_index_time}'
              field column=WorkDesc_Comments
 name=WorkDesc_Comments_Split /
              field column=WorkDesc_Comments name=WorkDesc_Comments_Edge 
 /
        /entity
      /document
    /dataConfig

 I can use MS SQL Profiler to watch the Solr user log in successfully,
 but then nothing. It doesn't seem to even try and execute the stored
 procedure. Any ideas why this would be working one server and not on
 another?

 FTR the only thing in the tomcat catalina log is:

    org.apache.solr.handler.dataimport.JdbcDataSource$1 call
    INFO: Creating a connection for entity data with URL:
 jdbc:sqlserver://localhost;databaseName=CATLive

 --
 Ian

 i...@isfluent.com
 +44 (0)1223 257903




-- 
Ian

i...@isfluent.com
+44 (0)1223 257903
**
 
This message may contain confidential or proprietary information intended only 
for the use of the 
addressee(s) named above or may contain information that is legally privileged. 
If you are 
not the intended addressee, or the person responsible for delivering it to the 
intended addressee, 
you are hereby notified that reading, disseminating, distributing or copying 
this message is strictly 
prohibited. If you have received this message by mistake, please immediately 
notify us by 
replying to the message and delete the original message and any copies 
immediately thereafter. 

Thank you.- 
**
FAFLD

Unable to index documents using DataImportHandler with MSSQL

2011-11-25 Thread Ian Grainger 
Hi I have copied my Solr config from a working Windows server to a new
one, and it can't seem to run an import.

They're both using win server 2008 and SQL 2008R2. This is the data
importer config

dataConfig
  dataSource type=JdbcDataSource  name=ds1
driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
url=jdbc:sqlserver://localhost;databaseName=DB
user=Solr
password=pwd/
  document name=datas
entity name=data dataSource=ds1 pk=key
query=EXEC SOLR_COMPANY_SEARCH_DATA
deltaImportQuery=SELECT * FROM Company_Search_Data WHERE
[key]='${dataimporter.delta.key}'
deltaQuery=SELECT [key] FROM Company_Search_Data WHERE modify_dt
 '${dataimporter.last_index_time}'
  field column=WorkDesc_Comments
name=WorkDesc_Comments_Split /
  field column=WorkDesc_Comments name=WorkDesc_Comments_Edge /
/entity
  /document
/dataConfig

I can use MS SQL Profiler to watch the Solr user log in successfully,
but then nothing. It doesn't seem to even try and execute the stored
procedure. Any ideas why this would be working one server and not on
another?

FTR the only thing in the tomcat catalina log is:

org.apache.solr.handler.dataimport.JdbcDataSource$1 call
INFO: Creating a connection for entity data with URL:
jdbc:sqlserver://localhost;databaseName=CATLive

-- 
Ian

i...@isfluent.com
+44 (0)1223 257903

Re: Unable to index documents using DataImportHandler with MSSQL

2011-11-25 Thread Ian Grainger 
Update on this: I've established:
* It's not a problem in the DB (I can index from this DB into a Solr
instance on another server)
* It's not Tomcat (I get the same problem in Jetty)
* It's not the schema (I have simplified it to one field)

That leaves SolrConfig.xml and data-config.

Only thing changed in SolrConfig.xml is adding:

  lib dir=D:/Software/Solr/example/solr/dist/
regex=apache-solr-cell-\d.*\.jar /
  lib dir=D:/Software/Solr/example/solr/dist/
regex=apache-solr-clustering-\d.*\.jar /
  lib dir=D:/Software/Solr/example/solr/dist/
regex=apache-solr-dataimporthandler-\d.*\.jar /
requestHandler name=/dataimport
class=org.apache.solr.handler.dataimport.DataImportHandler
  lst name=defaults
str name=configD:/Software/Solr/example/solr/conf/data-config.xml/str
  /lst
/requestHandler

And data-config.xml is pretty much as attached - except simpler.

Any help or any advice on how to diagnose would be appreciated!


On Fri, Nov 25, 2011 at 12:29 PM, Ian Grainger i...@isfluent.com wrote:
 Hi I have copied my Solr config from a working Windows server to a new
 one, and it can't seem to run an import.

 They're both using win server 2008 and SQL 2008R2. This is the data
 importer config

    dataConfig
      dataSource type=JdbcDataSource  name=ds1
            driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
            url=jdbc:sqlserver://localhost;databaseName=DB
            user=Solr
            password=pwd/
      document name=datas
        entity name=data dataSource=ds1 pk=key
        query=EXEC SOLR_COMPANY_SEARCH_DATA
        deltaImportQuery=SELECT * FROM Company_Search_Data WHERE
 [key]='${dataimporter.delta.key}'
        deltaQuery=SELECT [key] FROM Company_Search_Data WHERE modify_dt
 '${dataimporter.last_index_time}'
              field column=WorkDesc_Comments
 name=WorkDesc_Comments_Split /
              field column=WorkDesc_Comments name=WorkDesc_Comments_Edge 
 /
        /entity
      /document
    /dataConfig

 I can use MS SQL Profiler to watch the Solr user log in successfully,
 but then nothing. It doesn't seem to even try and execute the stored
 procedure. Any ideas why this would be working one server and not on
 another?

 FTR the only thing in the tomcat catalina log is:

    org.apache.solr.handler.dataimport.JdbcDataSource$1 call
    INFO: Creating a connection for entity data with URL:
 jdbc:sqlserver://localhost;databaseName=CATLive

 --
 Ian

 i...@isfluent.com
 +44 (0)1223 257903




-- 
Ian

i...@isfluent.com
+44 (0)1223 257903