[jira] [Updated] (SOLR-11402) DataImportHandler dataimport.properties should write to data dir by default

2017-09-26 Thread Jamie Jackson (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SOLR-11402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jamie Jackson updated SOLR-11402:
-
Description: 
Currently, DIH drops the {{dataimport.properties}} file in the core directory 
by default, but the data directory seems to be the logical choice.

* The core directory tends to be read-only.
* The data directory is the write area, and the {{dataimport.properties}} file 
is tied to the index, rather than the core configurations.

Docker is a use case where the current behavior is glaringly problematic: The 
core directory lives in the container layer, and any files that Solr writes 
there disappear when the container is restarted (forcing a subsequent full 
index). The data directory, on the other hand, is already persisted to a volume 
(according to normal practice), so if it were the default location to write 
{{dataimport.properties}}, it would behave as one would expect.

It's possible to work around this (using PropertyWriter, symlinks, or other 
tricks), but this shouldn't be necessary.

* Downstream Solr Docker ticket: 
https://github.com/docker-solr/docker-solr/issues/150
* SOLR-1970, in which others make the same argument

  was:
Currently, DIH drops the {{dataimport.properties}} file in the cores directory 
by default, but the data directory seems to be the logical choice.

* The core directory tends to be read-only.
* The data directory is the write area, and the {{dataimport.properties}} file 
is tied to the index, rather than the core configurations.

Docker is a use case where the current behavior is glaringly problematic: The 
cores directory lives in the container layer, and any files that Solr writes 
there disappear when the container is restarted (forcing a subsequent full 
index). The data directory, on the other hand, is already persisted to a volume 
(according to normal practice), so if it were the default location to write 
{{dataimport.properties}}, it would behave as one would expect.

It's possible to work around this (using PropertyWriter, symlinks, or other 
tricks), but this shouldn't be necessary.

* Downstream Solr Docker ticket: 
https://github.com/docker-solr/docker-solr/issues/150
* SOLR-1970, in which others make the same argument


> DataImportHandler dataimport.properties should write to data dir by default
> ---
>
> Key: SOLR-11402
> URL: https://issues.apache.org/jira/browse/SOLR-11402
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 4.10, 5.5, 6.6
>Reporter: Jamie Jackson
>Priority: Minor
>
> Currently, DIH drops the {{dataimport.properties}} file in the core directory 
> by default, but the data directory seems to be the logical choice.
> * The core directory tends to be read-only.
> * The data directory is the write area, and the {{dataimport.properties}} 
> file is tied to the index, rather than the core configurations.
> Docker is a use case where the current behavior is glaringly problematic: The 
> core directory lives in the container layer, and any files that Solr writes 
> there disappear when the container is restarted (forcing a subsequent full 
> index). The data directory, on the other hand, is already persisted to a 
> volume (according to normal practice), so if it were the default location to 
> write {{dataimport.properties}}, it would behave as one would expect.
> It's possible to work around this (using PropertyWriter, symlinks, or other 
> tricks), but this shouldn't be necessary.
> * Downstream Solr Docker ticket: 
> https://github.com/docker-solr/docker-solr/issues/150
> * SOLR-1970, in which others make the same argument



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11402) DataImportHandler dataimport.properties should write to data dir by default

2017-09-25 Thread Jamie Jackson (JIRA) 
Jamie Jackson created SOLR-11402:


 Summary: DataImportHandler dataimport.properties should write to 
data dir by default
 Key: SOLR-11402
 URL: https://issues.apache.org/jira/browse/SOLR-11402
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: contrib - DataImportHandler
Affects Versions: 6.6, 5.5, 4.10
Reporter: Jamie Jackson
Priority: Minor


Currently, DIH drops the {{dataimport.properties}} file in the cores directory 
by default, but the data directory seems to be the logical choice.

* The core directory tends to be read-only.
* The data directory is the write area, and the {{dataimport.properties}} file 
is tied to the index, rather than the core configurations.

Docker is a use case where the current behavior is glaringly problematic: The 
cores directory lives in the container layer, and any files that Solr writes 
there disappear when the container is restarted (forcing a subsequent full 
index). The data directory, on the other hand, is already persisted to a volume 
(according to normal practice), so if it were the default location to write 
{{dataimport.properties}}, it would behave as one would expect.

It's possible to work around this (using PropertyWriter, symlinks, or other 
tricks), but this shouldn't be necessary.

* Downstream Solr Docker ticket: 
https://github.com/docker-solr/docker-solr/issues/150
* SOLR-1970, in which others make the same argument



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9725) Allow Variables for All Data Import Handler Data Source Configuration Values

2016-11-10 Thread Jamie Jackson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SOLR-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655406#comment-15655406
 ] 

Jamie Jackson edited comment on SOLR-9725 at 11/10/16 10:44 PM:


I barely know what I'm doing here, but it's a combination of 
[this|https://github.com/apache/lucene-solr/pull/46] and 
[this|https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=blobdiff;f=solr/contrib/dataimporthandler/src/test/org/apache/solr/handler/dataimport/TestJdbcDataSource.java;h=3b1568f4892661fa52ad9b50f8667f1f8c11b060;hp=08a936ae48c510838dce86c404fd9e569d2f2d39;hb=c2aea1b;hpb=59b4fc0bb0105ec25285f763fde86739433a38b1].
 

Notes:
* I haven't actually tested it in situ (variables actually in a DIH config 
file, running a core), since I don't have time today to figure out building and 
configuring. 
* It targets {{branch_5_5}}, so I'm not sure the best way to get it up to 
master. Separate pull request?
* {{testRetrieveFromJndiWithCredentialsEncryptedAndResolved()}} replaces 
{{testRetrieveFromJndiWithCredentialsWithEncryptedPwd()}}, as I didn't think 
the old name made as much sense anymore.
* Master has 
{{testRetrieveFromJndiWithCredentialsWithEncryptedAndResolvedPwd()}} (note the 
trailing {{pwd}}). IMO, my new test will replace this one.


was (Author: jamiejackson):
I barely know what I'm doing here, but it's a combination of 
[this|https://github.com/apache/lucene-solr/pull/46] and 
[this|https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=blobdiff;f=solr/contrib/dataimporthandler/src/test/org/apache/solr/handler/dataimport/TestJdbcDataSource.java;h=3b1568f4892661fa52ad9b50f8667f1f8c11b060;hp=08a936ae48c510838dce86c404fd9e569d2f2d39;hb=c2aea1b;hpb=59b4fc0bb0105ec25285f763fde86739433a38b1].
 

Notes:
* I haven't actually tested it in situ (variables actually in a DIH config 
file, running a core), since I don't have time today to figure out building and 
configuring. 
* It targets {{branch_5_5}}, so I'm not sure the best way to get it up to 
master. Separate pull request?
* {{testRetrieveFromJndiWithCredentialsEncryptedAndResolved()}} replaces 
{{testRetrieveFromJndiWithCredentialsWithEncryptedPwd()}}, as I didn't think 
the old name made as much sense anymore.

> Allow Variables for All Data Import Handler Data Source Configuration Values
> 
>
> Key: SOLR-9725
> URL: https://issues.apache.org/jira/browse/SOLR-9725
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 5.5.3
>Reporter: Jamie Jackson
>Priority: Minor
>  Labels: patch
>
> I need to be able to use a variable for a password when also using 
> {{encryptKeyFile}}.
> For instance:
> {code:xml}
>  driver="${custom.dataimporter.datasource.driver}"
>   url="${custom.dataimporter.datasource.url}"
>   user="${custom.dataimporter.datasource.user}"
>   password="${custom.dataimporter.datasource.password}"
>   encryptKeyFile="/opt/solr/credentials/encrypt.key"
>   />
> {code}
> Because I need to change certain variables based on the environment. I'd 
> start like this:
> {code}
>  -a
>   -Dcustom.dataimporter.datasource.driver=org.mariadb.jdbc.Driver
>   
> -Dcustom.dataimporter.datasource.url=jdbc:mysql://local.mysite.com:3306/mysite
>   -Dcustom.dataimporter.datasource.user=root
>   
> -Dcustom.dataimporter.datasource.password=U2FsdGVkX1/dqwTb8RBfFq82SM37DkDRGeWMOndftHY=
> {code}
> If I hardcode the password, it works; if I use a variable reference, it 
> doesn't.
> As far as I know [this pull 
> request|https://github.com/apache/lucene-solr/pull/46] was submitted to 
> address this issue, but it didn't come with a Jira ticket or a full 
> explanation.
> Also, note that I'm not using a variable for the value of {{encryptKeyFile}}, 
> because [it's not possible in 5.x, though it seems to be fixed in 
> 6.1|https://issues.apache.org/jira/browse/SOLR-8610]. Presumably, the above 
> patch would encompass {{encryptKeyFile}}'s value, as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9725) Allow Variables for All Data Import Handler Data Source Configuration Values

2016-11-10 Thread Jamie Jackson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SOLR-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655406#comment-15655406
 ] 

Jamie Jackson commented on SOLR-9725:
-

I barely know what I'm doing here, but it's a combination of 
[this|https://github.com/apache/lucene-solr/pull/46] and 
[this|https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=blobdiff;f=solr/contrib/dataimporthandler/src/test/org/apache/solr/handler/dataimport/TestJdbcDataSource.java;h=3b1568f4892661fa52ad9b50f8667f1f8c11b060;hp=08a936ae48c510838dce86c404fd9e569d2f2d39;hb=c2aea1b;hpb=59b4fc0bb0105ec25285f763fde86739433a38b1].
 

Notes:
* I haven't actually tested it in situ (variables actually in a DIH config 
file, running a core), since I don't have time today to figure out building and 
configuring. 
* It targets {{branch_5_5}}, so I'm not sure the best way to get it up to 
master. Separate pull request?
* {{testRetrieveFromJndiWithCredentialsEncryptedAndResolved()}} replaces 
{{testRetrieveFromJndiWithCredentialsWithEncryptedPwd()}}, as I didn't think 
the old name made as much sense anymore.

> Allow Variables for All Data Import Handler Data Source Configuration Values
> 
>
> Key: SOLR-9725
> URL: https://issues.apache.org/jira/browse/SOLR-9725
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 5.5.3
>Reporter: Jamie Jackson
>Priority: Minor
>  Labels: patch
>
> I need to be able to use a variable for a password when also using 
> {{encryptKeyFile}}.
> For instance:
> {code:xml}
>  driver="${custom.dataimporter.datasource.driver}"
>   url="${custom.dataimporter.datasource.url}"
>   user="${custom.dataimporter.datasource.user}"
>   password="${custom.dataimporter.datasource.password}"
>   encryptKeyFile="/opt/solr/credentials/encrypt.key"
>   />
> {code}
> Because I need to change certain variables based on the environment. I'd 
> start like this:
> {code}
>  -a
>   -Dcustom.dataimporter.datasource.driver=org.mariadb.jdbc.Driver
>   
> -Dcustom.dataimporter.datasource.url=jdbc:mysql://local.mysite.com:3306/mysite
>   -Dcustom.dataimporter.datasource.user=root
>   
> -Dcustom.dataimporter.datasource.password=U2FsdGVkX1/dqwTb8RBfFq82SM37DkDRGeWMOndftHY=
> {code}
> If I hardcode the password, it works; if I use a variable reference, it 
> doesn't.
> As far as I know [this pull 
> request|https://github.com/apache/lucene-solr/pull/46] was submitted to 
> address this issue, but it didn't come with a Jira ticket or a full 
> explanation.
> Also, note that I'm not using a variable for the value of {{encryptKeyFile}}, 
> because [it's not possible in 5.x, though it seems to be fixed in 
> 6.1|https://issues.apache.org/jira/browse/SOLR-8610]. Presumably, the above 
> patch would encompass {{encryptKeyFile}}'s value, as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9725) Allow Variables for All Data Import Handler Data Source Configuration Values

2016-11-10 Thread Jamie Jackson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SOLR-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15655388#comment-15655388
 ] 

Jamie Jackson commented on SOLR-9725:
-

I gave up on Windows; I can't get tests to run there.

I tried on Linux and was able to run tests. 
[This|https://gist.github.com/jamiejackson/063a67499370084cdf154f893f7006f3#file-solr_test_linux-txt-L14]
 is what the output looks like on Linux. After the highlighted line is where it 
had been hanging in Windows.

> Allow Variables for All Data Import Handler Data Source Configuration Values
> 
>
> Key: SOLR-9725
> URL: https://issues.apache.org/jira/browse/SOLR-9725
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 5.5.3
>Reporter: Jamie Jackson
>Priority: Minor
>  Labels: patch
>
> I need to be able to use a variable for a password when also using 
> {{encryptKeyFile}}.
> For instance:
> {code:xml}
>  driver="${custom.dataimporter.datasource.driver}"
>   url="${custom.dataimporter.datasource.url}"
>   user="${custom.dataimporter.datasource.user}"
>   password="${custom.dataimporter.datasource.password}"
>   encryptKeyFile="/opt/solr/credentials/encrypt.key"
>   />
> {code}
> Because I need to change certain variables based on the environment. I'd 
> start like this:
> {code}
>  -a
>   -Dcustom.dataimporter.datasource.driver=org.mariadb.jdbc.Driver
>   
> -Dcustom.dataimporter.datasource.url=jdbc:mysql://local.mysite.com:3306/mysite
>   -Dcustom.dataimporter.datasource.user=root
>   
> -Dcustom.dataimporter.datasource.password=U2FsdGVkX1/dqwTb8RBfFq82SM37DkDRGeWMOndftHY=
> {code}
> If I hardcode the password, it works; if I use a variable reference, it 
> doesn't.
> As far as I know [this pull 
> request|https://github.com/apache/lucene-solr/pull/46] was submitted to 
> address this issue, but it didn't come with a Jira ticket or a full 
> explanation.
> Also, note that I'm not using a variable for the value of {{encryptKeyFile}}, 
> because [it's not possible in 5.x, though it seems to be fixed in 
> 6.1|https://issues.apache.org/jira/browse/SOLR-8610]. Presumably, the above 
> patch would encompass {{encryptKeyFile}}'s value, as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9725) Allow Variables for All Data Import Handler Data Source Configuration Values

2016-11-10 Thread Jamie Jackson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SOLR-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15654718#comment-15654718
 ] 

Jamie Jackson commented on SOLR-9725:
-

The command hangs 
[here|https://gist.github.com/jamiejackson/d3ba71d5439899799dbf8776fdd0983a].

> Allow Variables for All Data Import Handler Data Source Configuration Values
> 
>
> Key: SOLR-9725
> URL: https://issues.apache.org/jira/browse/SOLR-9725
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 5.5.3
>Reporter: Jamie Jackson
>Priority: Minor
>  Labels: patch
>
> I need to be able to use a variable for a password when also using 
> {{encryptKeyFile}}.
> For instance:
> {code:xml}
>  driver="${custom.dataimporter.datasource.driver}"
>   url="${custom.dataimporter.datasource.url}"
>   user="${custom.dataimporter.datasource.user}"
>   password="${custom.dataimporter.datasource.password}"
>   encryptKeyFile="/opt/solr/credentials/encrypt.key"
>   />
> {code}
> Because I need to change certain variables based on the environment. I'd 
> start like this:
> {code}
>  -a
>   -Dcustom.dataimporter.datasource.driver=org.mariadb.jdbc.Driver
>   
> -Dcustom.dataimporter.datasource.url=jdbc:mysql://local.mysite.com:3306/mysite
>   -Dcustom.dataimporter.datasource.user=root
>   
> -Dcustom.dataimporter.datasource.password=U2FsdGVkX1/dqwTb8RBfFq82SM37DkDRGeWMOndftHY=
> {code}
> If I hardcode the password, it works; if I use a variable reference, it 
> doesn't.
> As far as I know [this pull 
> request|https://github.com/apache/lucene-solr/pull/46] was submitted to 
> address this issue, but it didn't come with a Jira ticket or a full 
> explanation.
> Also, note that I'm not using a variable for the value of {{encryptKeyFile}}, 
> because [it's not possible in 5.x, though it seems to be fixed in 
> 6.1|https://issues.apache.org/jira/browse/SOLR-8610]. Presumably, the above 
> patch would encompass {{encryptKeyFile}}'s value, as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9725) Allow Variables for All Data Import Handler Data Source Configuration Values

2016-11-08 Thread Jamie Jackson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SOLR-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15648239#comment-15648239
 ] 

Jamie Jackson edited comment on SOLR-9725 at 11/8/16 5:46 PM:
--

I tried for a couple hours to get the project set up and to [simply run a test 
suite|https://gist.github.com/jamiejackson/b2d7b48205c9366dbdf8d9fef5d6b319], 
and I wasn't getting substantive answers on IRC. I don't have the time to spin 
my wheels just to get the project set up.


was (Author: jamiejackson):
I tried for a couple hours to get the project set up and to [simply run a test 
suite](https://gist.github.com/jamiejackson/b2d7b48205c9366dbdf8d9fef5d6b319), 
and I wasn't getting substantive answers on IRC. I don't have the time to spin 
my wheels just to get the project set up.

> Allow Variables for All Data Import Handler Data Source Configuration Values
> 
>
> Key: SOLR-9725
> URL: https://issues.apache.org/jira/browse/SOLR-9725
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 5.5.3
>Reporter: Jamie Jackson
>Priority: Minor
>  Labels: patch
>
> I need to be able to use a variable for a password when also using 
> {{encryptKeyFile}}.
> For instance:
> {code:xml}
>  driver="${custom.dataimporter.datasource.driver}"
>   url="${custom.dataimporter.datasource.url}"
>   user="${custom.dataimporter.datasource.user}"
>   password="${custom.dataimporter.datasource.password}"
>   encryptKeyFile="/opt/solr/credentials/encrypt.key"
>   />
> {code}
> Because I need to change certain variables based on the environment. I'd 
> start like this:
> {code}
>  -a
>   -Dcustom.dataimporter.datasource.driver=org.mariadb.jdbc.Driver
>   
> -Dcustom.dataimporter.datasource.url=jdbc:mysql://local.mysite.com:3306/mysite
>   -Dcustom.dataimporter.datasource.user=root
>   
> -Dcustom.dataimporter.datasource.password=U2FsdGVkX1/dqwTb8RBfFq82SM37DkDRGeWMOndftHY=
> {code}
> If I hardcode the password, it works; if I use a variable reference, it 
> doesn't.
> As far as I know [this pull 
> request|https://github.com/apache/lucene-solr/pull/46] was submitted to 
> address this issue, but it didn't come with a Jira ticket or a full 
> explanation.
> Also, note that I'm not using a variable for the value of {{encryptKeyFile}}, 
> because [it's not possible in 5.x, though it seems to be fixed in 
> 6.1|https://issues.apache.org/jira/browse/SOLR-8610]. Presumably, the above 
> patch would encompass {{encryptKeyFile}}'s value, as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9725) Allow Variables for All Data Import Handler Data Source Configuration Values

2016-11-08 Thread Jamie Jackson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SOLR-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15648239#comment-15648239
 ] 

Jamie Jackson commented on SOLR-9725:
-

I tried for a couple hours to get the project set up and to [simply run a test 
suite](https://gist.github.com/jamiejackson/b2d7b48205c9366dbdf8d9fef5d6b319), 
and I wasn't getting substantive answers on IRC. I don't have the time to spin 
my wheels just to get the project set up.

> Allow Variables for All Data Import Handler Data Source Configuration Values
> 
>
> Key: SOLR-9725
> URL: https://issues.apache.org/jira/browse/SOLR-9725
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 5.5.3
>Reporter: Jamie Jackson
>Priority: Minor
>  Labels: patch
>
> I need to be able to use a variable for a password when also using 
> {{encryptKeyFile}}.
> For instance:
> {code:xml}
>  driver="${custom.dataimporter.datasource.driver}"
>   url="${custom.dataimporter.datasource.url}"
>   user="${custom.dataimporter.datasource.user}"
>   password="${custom.dataimporter.datasource.password}"
>   encryptKeyFile="/opt/solr/credentials/encrypt.key"
>   />
> {code}
> Because I need to change certain variables based on the environment. I'd 
> start like this:
> {code}
>  -a
>   -Dcustom.dataimporter.datasource.driver=org.mariadb.jdbc.Driver
>   
> -Dcustom.dataimporter.datasource.url=jdbc:mysql://local.mysite.com:3306/mysite
>   -Dcustom.dataimporter.datasource.user=root
>   
> -Dcustom.dataimporter.datasource.password=U2FsdGVkX1/dqwTb8RBfFq82SM37DkDRGeWMOndftHY=
> {code}
> If I hardcode the password, it works; if I use a variable reference, it 
> doesn't.
> As far as I know [this pull 
> request|https://github.com/apache/lucene-solr/pull/46] was submitted to 
> address this issue, but it didn't come with a Jira ticket or a full 
> explanation.
> Also, note that I'm not using a variable for the value of {{encryptKeyFile}}, 
> because [it's not possible in 5.x, though it seems to be fixed in 
> 6.1|https://issues.apache.org/jira/browse/SOLR-8610]. Presumably, the above 
> patch would encompass {{encryptKeyFile}}'s value, as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9725) Allow Variables for All Data Import Handler Data Source Configuration Values

2016-11-08 Thread Jamie Jackson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SOLR-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15647787#comment-15647787
 ] 

Jamie Jackson commented on SOLR-9725:
-

That looks pretty digestible. I'll take a crack at it if I get a chance.

> Allow Variables for All Data Import Handler Data Source Configuration Values
> 
>
> Key: SOLR-9725
> URL: https://issues.apache.org/jira/browse/SOLR-9725
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 5.5.3
>Reporter: Jamie Jackson
>Priority: Minor
>  Labels: patch
>
> I need to be able to use a variable for a password when also using 
> {{encryptKeyFile}}.
> For instance:
> {code:xml}
>  driver="${custom.dataimporter.datasource.driver}"
>   url="${custom.dataimporter.datasource.url}"
>   user="${custom.dataimporter.datasource.user}"
>   password="${custom.dataimporter.datasource.password}"
>   encryptKeyFile="/opt/solr/credentials/encrypt.key"
>   />
> {code}
> Because I need to change certain variables based on the environment. I'd 
> start like this:
> {code}
>  -a
>   -Dcustom.dataimporter.datasource.driver=org.mariadb.jdbc.Driver
>   
> -Dcustom.dataimporter.datasource.url=jdbc:mysql://local.mysite.com:3306/mysite
>   -Dcustom.dataimporter.datasource.user=root
>   
> -Dcustom.dataimporter.datasource.password=U2FsdGVkX1/dqwTb8RBfFq82SM37DkDRGeWMOndftHY=
> {code}
> If I hardcode the password, it works; if I use a variable reference, it 
> doesn't.
> As far as I know [this pull 
> request|https://github.com/apache/lucene-solr/pull/46] was submitted to 
> address this issue, but it didn't come with a Jira ticket or a full 
> explanation.
> Also, note that I'm not using a variable for the value of {{encryptKeyFile}}, 
> because [it's not possible in 5.x, though it seems to be fixed in 
> 6.1|https://issues.apache.org/jira/browse/SOLR-8610]. Presumably, the above 
> patch would encompass {{encryptKeyFile}}'s value, as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9725) Allow Variables for All Data Import Handler Data Source Configuration Values

2016-11-07 Thread Jamie Jackson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SOLR-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15645715#comment-15645715
 ] 

Jamie Jackson commented on SOLR-9725:
-

Probably not from me, unless someone's prepared to do a lot of hand-holding. At 
a quick glance, it doesn't look too easy (for _me_, anyway) to add this test to 
[{{TestSqlEntityProcessor}}|https://github.com/apache/lucene-solr/tree/branch_5x/solr/contrib/dataimporthandler/src/test/org/apache/solr/handler/dataimport].

> Allow Variables for All Data Import Handler Data Source Configuration Values
> 
>
> Key: SOLR-9725
> URL: https://issues.apache.org/jira/browse/SOLR-9725
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 5.5.3
>Reporter: Jamie Jackson
>Priority: Minor
>  Labels: patch
>
> I need to be able to use a variable for a password when also using 
> {{encryptKeyFile}}.
> For instance:
> {code:xml}
>  driver="${custom.dataimporter.datasource.driver}"
>   url="${custom.dataimporter.datasource.url}"
>   user="${custom.dataimporter.datasource.user}"
>   password="${custom.dataimporter.datasource.password}"
>   encryptKeyFile="/opt/solr/credentials/encrypt.key"
>   />
> {code}
> Because I need to change certain variables based on the environment. I'd 
> start like this:
> {code}
>  -a
>   -Dcustom.dataimporter.datasource.driver=org.mariadb.jdbc.Driver
>   
> -Dcustom.dataimporter.datasource.url=jdbc:mysql://local.mysite.com:3306/mysite
>   -Dcustom.dataimporter.datasource.user=root
>   
> -Dcustom.dataimporter.datasource.password=U2FsdGVkX1/dqwTb8RBfFq82SM37DkDRGeWMOndftHY=
> {code}
> If I hardcode the password, it works; if I use a variable reference, it 
> doesn't.
> As far as I know [this pull 
> request|https://github.com/apache/lucene-solr/pull/46] was submitted to 
> address this issue, but it didn't come with a Jira ticket or a full 
> explanation.
> Also, note that I'm not using a variable for the value of {{encryptKeyFile}}, 
> because [it's not possible in 5.x, though it seems to be fixed in 
> 6.1|https://issues.apache.org/jira/browse/SOLR-8610]. Presumably, the above 
> patch would encompass {{encryptKeyFile}}'s value, as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8610) DIH - Resolve variables in encryptKeyFile

2016-11-03 Thread Jamie Jackson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SOLR-8610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633824#comment-15633824
 ] 

Jamie Jackson commented on SOLR-8610:
-

[~sashevsky], I made a [ticket|SOLR-9725] of your issue.

> DIH - Resolve variables in encryptKeyFile
> -
>
> Key: SOLR-8610
> URL: https://issues.apache.org/jira/browse/SOLR-8610
> Project: Solr
>  Issue Type: Improvement
>  Components: contrib - DataImportHandler
>Affects Versions: 5.4.1
>Reporter: Kristine Jetzke
>Assignee: Mikhail Khludnev
> Fix For: 6.1, master (7.0)
>
> Attachments: SOLR-8610.patch, SOLR-8610.patch
>
>
> I would like to use a variable like {{$\{path.to.file\}}} for  
> {{encryptKeyFile}} in my DIH config files so that I don't have to specify the 
> concrete absolute path in all files. This is currently not possible since 
> variables are not resolved. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9725) Allow Variables for All Data Import Handler Data Source Configuration Values

2016-11-03 Thread Jamie Jackson (JIRA) 
Jamie Jackson created SOLR-9725:
---

 Summary: Allow Variables for All Data Import Handler Data Source 
Configuration Values
 Key: SOLR-9725
 URL: https://issues.apache.org/jira/browse/SOLR-9725
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: contrib - DataImportHandler
Affects Versions: 5.5.3
Reporter: Jamie Jackson
Priority: Minor


I need to be able to use a variable for a password when also using 
{{encryptKeyFile}}.

For instance:
{code:xml}

{code}

Because I need to change certain variables based on the environment. I'd start 
like this:

{code}
 -a
  -Dcustom.dataimporter.datasource.driver=org.mariadb.jdbc.Driver
  
-Dcustom.dataimporter.datasource.url=jdbc:mysql://local.mysite.com:3306/mysite
  -Dcustom.dataimporter.datasource.user=root
  
-Dcustom.dataimporter.datasource.password=U2FsdGVkX1/dqwTb8RBfFq82SM37DkDRGeWMOndftHY=
{code}

If I hardcode the password, it works; if I use a variable reference, it doesn't.

As far as I know [this pull 
request|https://github.com/apache/lucene-solr/pull/46] was submitted to address 
this issue, but it didn't come with a Jira ticket or a full explanation.

Also, note that I'm not using a variable for the value of {{encryptKeyFile}}, 
because [it's not possible in 5.x, though it seems to be fixed in 
6.1|https://issues.apache.org/jira/browse/SOLR-8610]. Presumably, the above 
patch would encompass {{encryptKeyFile}}'s value, as well.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4392) DIH - Need to externalize or encrypt username/password stored within data-config.xml

2016-10-06 Thread Jamie Jackson (JIRA) 

[ 
https://issues.apache.org/jira/browse/SOLR-4392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15552385#comment-15552385
 ] 

Jamie Jackson commented on SOLR-4392:
-

As far as I know, the only place where encrypted passwords are documented is in 
https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler,
 under the "Configuring the DIH Configuration File", in a comment of a sample 
XML file:

{code:xml}

{code}

Unfortunately, [I can't get it to 
work|http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201610.mbox/%3CCA%2BonWPfGakOUO0VJqBYfnALfq61HfysL0Uv%2BG8p-TRL0P44NOQ%40mail.gmail.com%3E].

> DIH - Need to externalize or encrypt username/password stored within 
> data-config.xml
> 
>
> Key: SOLR-4392
> URL: https://issues.apache.org/jira/browse/SOLR-4392
> Project: Solr
>  Issue Type: New Feature
>  Components: contrib - DataImportHandler
>Affects Versions: 4.0, 4.1
>Reporter: Senthuran Sivananthan
>Assignee: Noble Paul
> Fix For: 5.2, 6.0
>
> Attachments: SOLR-4392.patch, SOLR-4392.patch
>
>
> Today, the connection (database or otherwise) credentials is wide open in 
> data-config.xml.  Not really an issue until someone sends out the config file 
> outside of the server.
> We should look into externalizing the database lookup or providing a way to 
> encrypt the username and password.
> The needs are:
> 1/  Some projects want to enable multi-tenancy where data for each core is 
> situated in different database servers w/ their own credentials.  We need a 
> way to expose hooks that will allow implementations to be plugged in.  It can 
> be done though the "type" attribute on the dataSource, but providing a 
> factory might work better.
> 2/  Most orgs are very protective of their credentials and weary of 
> plain-text settings.
> {code:xml}
>  url="jdbc:oracle:thin:@//hostname:port/SID" user="db_username" 
> 
> 
> password="U2FsdGVkX18QMjY0yfCqlfBMvAB4d3XkwY96L7gfO2o=" 
> 
> encryptKeyFile="/location/of/encryptionkey"
> />
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org