Hi Alejandro,
Hmm, my test case worked fine. Here's what I did, and the simple history.
Of course, I can't tell if the acls are correct because that's not logged,
but certainly the data seems to be indexed and all queries run:
>>>>>>
1.
Seeding query: SELECT idcol AS $(IDCOLUMN) FROM crawltest
Version check query: SELECT idcol AS $(IDCOLUMN), versioncol AS
$(VERSIONCOLUMN) FROM crawltest WHERE cast(idcol AS VARCHAR(32)) IN
$(IDLIST)
Access token query: SELECT idcol AS $(IDCOLUMN), aclvalue AS
$(TOKENCOLUMN) FROM crawlacltest WHERE cast(idcol AS VARCHAR(32)) IN
$(IDLIST)
Data query: SELECT idcol AS $(IDCOLUMN), urlcol AS $(URLCOLUMN),
datacol AS $(DATACOLUMN) FROM crawltest WHERE cast(idcol AS VARCHAR(32)) IN
$(IDLIST)
Security: Enabled
No access tokens specified
<<<<<<
>>>>>>
11-01-2014 03:10:07.145 output notification (null) OK 0
1
11-01-2014 03:09:57.266 job end 1414759801884(test)
0 1
11-01-2014 03:09:49.074 document ingest (null) http://3
OK 17 1
11-01-2014 03:09:49.067 fetch 3
OK 17 12
11-01-2014 03:09:49.061 document ingest (null) http://2
OK 17 1
11-01-2014 03:09:49.058 fetch 2
OK 17 7
11-01-2014 03:09:49.047 document ingest (null) http://1
OK 20 1
11-01-2014 03:09:49.030 fetch 1
OK 20 26
11-01-2014 03:09:49.027 external query SELECT idcol AS lcf__id,
urlcol AS lcf__url, datacol AS lcf__...
data FROM crawltest WHERE cast(idcol AS VARCHAR(32)) IN (?,?...
,?) ; arguments = ('1','3','2')
OK 0 1
11-01-2014 03:09:49.017 external query SELECT idcol AS lcf__id,
aclvalue AS lcf__token FROM crawlacl...
test WHERE cast(idcol AS VARCHAR(32)) IN (?,?,?) ; arguments...
= ('1','3','2')
OK 0 5
11-01-2014 03:09:49.014 external query SELECT idcol AS lcf__id,
versioncol AS lcf__version FROM craw...
ltest WHERE cast(idcol AS VARCHAR(32)) IN (?,?,?) ; argument...
s = ('1','3','2')
OK 0 1
11-01-2014 03:09:47.131 external query SELECT idcol AS lcf__id FROM
crawltest; arguments = ()
OK 0 1
<<<<<<
What are you doing that differs in any significant way from this? For
example, are you supplying a version query, or not?
Thanks,
Karl
On Fri, Oct 31, 2014 at 8:37 PM, Karl Wright <[email protected]> wrote:
> Hi Alejandro,
> Is there anything in the logs or in the simple history that might give us
> a hint?
>
> Karl
>
> Sent from my Windows Phone
> ------------------------------
> From: Alejandro Calbazana
> Sent: 10/31/2014 4:49 PM
> To: [email protected]
> Subject: Re: JDBC Repository Connector + Access Token Question
>
> Hi Karl,
>
> Thanks for the response and the patch! I pulled this down, built,
> deployed, configured, and ran it. I see the new fields for the ACL query
> and supplied some test data. Unfortunately, I don't see anything in the
> output document. My setup is similar to my earlier post with the exception
> of the added access token query. I tried adding a static token to the job,
> and that does not show up when I use the access token query. I did define
> a new job to test static tokens, and that seems to work okay, so I'm not
> sure of the issue.
>
> Your help is appreciated!
>
> Thanks,
>
> Alejandro
>
>
> On Fri, Oct 31, 2014 at 8:55 AM, Karl Wright <[email protected]> wrote:
>
>> Hi Alejandro,
>>
>> I've completed development of this feature. Remember that this work is
>> currently based on unreleased MCF 2.0 code, so you will want to try it out
>> in a way that does not impact any existing instances of MCF you may have
>> running. To try it out, please do the following:
>>
>> (1) Check out the branch:
>>
>> svn co
>> https://svn.apache.org/repos/asf/manifoldcf/branches/CONNECTORS-1089
>>
>> (2) Build the branch:
>>
>> cd CONNECTORS-1089
>> ant make-core-deps build
>>
>> (3) Start the simple example:
>>
>> cd dist/example
>> start.bat (or start.sh)
>>
>> (4) Go to the UI and create your connections and jobs. Then crawl.
>>
>> Please let me know if you find any problems. If everything looks good, I
>> will merge this branch back to trunk and to the dev_1x branch.
>>
>>
>> Thanks!
>> Karl
>>
>>
>> On Thu, Oct 30, 2014 at 1:38 PM, Alejandro Calbazana <
>> [email protected]> wrote:
>>
>>> Hi Karl,
>>>
>>> Thanks for the quick reply.
>>>
>>> (a) What authority(s) do you hope to use to secure JDBC-based document
>>> content?
>>> I was hoping to tie together 3 authorities in one group. One for user
>>> level ACLs, one for roles, and another for "group" level authorization
>>> (although its possible that group could be a variation of a role in my
>>> context). All of these would be JDBC bound. Please let me know if I missed
>>> your question.
>>>
>>> (b) Do you need support for both "allow" and "deny" ACLs?
>>> No. Allow meets my use case.
>>>
>>> (c) Are there multiple levels of ACL required (e.g. document, folder,
>>> share)?
>>> No. Just documents.
>>>
>>> (d) What is your preferred schema for access tokens in the database? a
>>> joined table, or a delimiter-separated list?
>>> A joined table would be preferable.
>>>
>>> Thanks,
>>>
>>> Alejandro
>>>
>>>
>>> On Thu, Oct 30, 2014 at 12:55 PM, Karl Wright <[email protected]>
>>> wrote:
>>>
>>>> Hi Alejandro,
>>>>
>>>> The reason you are confused about document security in JDBC is because,
>>>> heretofore, nobody has implemented document security using JDBC. We've
>>>> been waiting for quite a while for a user who needs this functionality, so
>>>> that we develop it properly.
>>>>
>>>> If you are hoping to get security tokens from each document row in the
>>>> database, then we should talk further and I will develop a patch for you
>>>> that works in a way that is consistent with your problem. I've created a
>>>> ticket, CONNECTORS-1089, for tracking work on this feature.
>>>>
>>>> For a start, can you let me know the following:
>>>>
>>>> (a) What authority(s) do you hope to use to secure JDBC-based document
>>>> content?
>>>> (b) Do you need support for both "allow" and "deny" ACLs?
>>>> (c) Are there multiple levels of ACL required (e.g. document, folder,
>>>> share)?
>>>> (d) What is your preferred schema for access tokens in the database? a
>>>> joined table, or a delimiter-separated list?
>>>>
>>>> Thanks,
>>>> Karl
>>>>
>>>>
>>>> On Thu, Oct 30, 2014 at 12:37 PM, Alejandro Calbazana <
>>>> [email protected]> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I'm getting started with ManifoldCF and I'm scratching my head a
>>>>> little with how to get access tokens into my indexed content using a JDBC
>>>>> repository. Here is my setup:
>>>>>
>>>>> - JDBC repository connector, indicating which authority group to use
>>>>> - Solr output connector
>>>>> - Auth connector, specifying user query and access token query
>>>>> - Job tying the connectors together indicating the queries for content
>>>>> IDs and content data
>>>>>
>>>>> Where do I instruct Manifold on how where obtain access tokens for the
>>>>> content that it is crawling? I see that there is a "security" tab in the
>>>>> job setup that allows me to specify access tokens, but this is static. My
>>>>> content will have different tokens associated with it per row. How do I
>>>>> accomplish this? Is this a place where I need to add custom code to fetch
>>>>> the authorization detail and apply it to content?
>>>>>
>>>>> Any guidance is greatly appreciated!
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Alejandro
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
