Right. curl takes the ManifoldSecurityFilter completely out of the picture, which is why I tried it. That doesn't mean that ManifoldSecurityFilter has no bugs, but in this case that's not the primary problem.
Karl On Tue, Apr 26, 2011 at 1:32 PM, Kadri Atalay <atalay.ka...@gmail.com> wrote: > Karl, > > Never mind the previous comment.. mcf-authority is not complaining as long > as there is any domain name attached to the user name regardless what it > is.. > > Thx > > > On Tue, Apr 26, 2011 at 1:26 PM, Kadri Atalay <atalay.ka...@gmail.com> > wrote: >> >> Hi Karl, >> >> It appears that mcf-authority is working as expected and returning correct >> response per user >> So, problem actually might be in the ManifoldSecurityFilter. >> following is the manifoldcf log: >> >> [2011-04-26 11:51:24,614]WARN Authority error: Username is in unexpected >> form (no @): 'Fred' >> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Username is in >> unexpected form (no @): 'Fred' >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.parseUser(ActiveDirectoryAuthority.java:495) >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.getAuthorizationResponse(ActiveDirectoryAuthority.java:167) >> at >> org.apache.manifoldcf.authorities.system.AuthCheckThread.run(AuthCheckThread.java:92) >> [2011-04-26 11:51:24,614]WARN Authority 'TEQA-DC' is unreachable for user >> 'Fred' >> [2011-04-26 11:51:33,708]WARN Authority error: Username is in unexpected >> form (no @): 'katalay' >> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Username is in >> unexpected form (no @): 'katalay' >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.parseUser(ActiveDirectoryAuthority.java:495) >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.getAuthorizationResponse(ActiveDirectoryAuthority.java:167) >> at >> org.apache.manifoldcf.authorities.system.AuthCheckThread.run(AuthCheckThread.java:92) >> [2011-04-26 11:51:33,708]WARN Authority 'TEQA-DC' is unreachable for user >> 'katalay' >> [2011-04-26 11:51:37,724]WARN Authority error: Username is in unexpected >> form (no @): 'katalay_admin' >> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Username is in >> unexpected form (no @): 'katalay_admin' >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.parseUser(ActiveDirectoryAuthority.java:495) >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.getAuthorizationResponse(ActiveDirectoryAuthority.java:167) >> at >> org.apache.manifoldcf.authorities.system.AuthCheckThread.run(AuthCheckThread.java:92) >> [2011-04-26 11:51:37,724]WARN Authority 'TEQA-DC' is unreachable for user >> 'katalay_admin' >> [2011-04-26 11:52:33,163]WARN Authority error: Username is in unexpected >> form (no @): 'joe' >> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Username is in >> unexpected form (no @): 'joe' >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.parseUser(ActiveDirectoryAuthority.java:495) >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.getAuthorizationResponse(ActiveDirectoryAuthority.java:167) >> at >> org.apache.manifoldcf.authorities.system.AuthCheckThread.run(AuthCheckThread.java:92) >> [2011-04-26 11:52:33,163]WARN Authority 'TEQA-DC' is unreachable for user >> 'joe' >> [2011-04-26 11:52:38,601]WARN Authority 'TEQA-DC' is unreachable for user >> 'joe@' >> [2011-04-26 11:56:07,106]WARN Authority error: Username is in unexpected >> form (no @): 'joe' >> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Username is in >> unexpected form (no @): 'joe' >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.parseUser(ActiveDirectoryAuthority.java:495) >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.getAuthorizationResponse(ActiveDirectoryAuthority.java:167) >> at >> org.apache.manifoldcf.authorities.system.AuthCheckThread.run(AuthCheckThread.java:92) >> [2011-04-26 11:56:07,122]WARN Authority 'TEQA-DC' is unreachable for user >> 'joe' >> [2011-04-26 12:01:56,755]WARN Authority error: Username is in unexpected >> form (no @): 'joe' >> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Username is in >> unexpected form (no @): 'joe' >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.parseUser(ActiveDirectoryAuthority.java:495) >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.getAuthorizationResponse(ActiveDirectoryAuthority.java:167) >> at >> org.apache.manifoldcf.authorities.system.AuthCheckThread.run(AuthCheckThread.java:92) >> [2011-04-26 12:01:56,755]WARN Authority 'TEQA-DC' is unreachable for user >> 'joe' >> [2011-04-26 12:12:29,662]WARN Authority error: Username is in unexpected >> form (no @): 'joe' >> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Username is in >> unexpected form (no @): 'joe' >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.parseUser(ActiveDirectoryAuthority.java:495) >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.getAuthorizationResponse(ActiveDirectoryAuthority.java:167) >> at >> org.apache.manifoldcf.authorities.system.AuthCheckThread.run(AuthCheckThread.java:92) >> [2011-04-26 12:12:29,662]WARN Authority 'TEQA-DC' is unreachable for user >> 'joe' >> [2011-04-26 12:40:27,518]WARN Authority error: Username is in unexpected >> form (no @): 'joe' >> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Username is in >> unexpected form (no @): 'joe' >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.parseUser(ActiveDirectoryAuthority.java:495) >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.getAuthorizationResponse(ActiveDirectoryAuthority.java:167) >> at >> org.apache.manifoldcf.authorities.system.AuthCheckThread.run(AuthCheckThread.java:92) >> [2011-04-26 12:40:27,518]WARN Authority 'TEQA-DC' is unreachable for user >> 'joe' >> [2011-04-26 12:41:37,316]WARN Authority error: Username is in unexpected >> form (no @): 'fakeuser' >> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Username is in >> unexpected form (no @): 'fakeuser' >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.parseUser(ActiveDirectoryAuthority.java:495) >> at >> org.apache.manifoldcf.authorities.authorities.activedirectory.ActiveDirectoryAuthority.getAuthorizationResponse(ActiveDirectoryAuthority.java:167) >> at >> org.apache.manifoldcf.authorities.system.AuthCheckThread.run(AuthCheckThread.java:92) >> [2011-04-26 12:41:37,316]WARN Authority 'TEQA-DC' is unreachable for user >> 'fakeuser' >> >> >> On Tue, Apr 26, 2011 at 12:54 PM, Karl Wright <daddy...@gmail.com> wrote: >>> >>> If a completely unknown user still comes back as existing, then it's >>> time to look at how your domain controller is configured. >>> Specifically, what do you have it configured to trust? What version >>> of Windows is this? >>> >>> The way LDAP tells you a user does not exist in Java is by an >>> exception. So this statement: >>> >>> NamingEnumeration answer = ctx.search(searchBase, searchFilter, >>> searchCtls); >>> >>> will throw the NameNotFoundException if the name doesn't exist, which >>> the Active Directory connector then catches: >>> >>> catch (NameNotFoundException e) >>> { >>> // This means that the user doesn't exist >>> return userNotFoundResponse; >>> } >>> >>> Clearly this is not working at all for your setup. Maybe you can look >>> at the DC's event logs, and see what kinds of decisions it is making >>> here? It's not making much sense to me at this point. >>> >>> Karl >>> >>> On Tue, Apr 26, 2011 at 12:45 PM, Kadri Atalay <atalay.ka...@gmail.com> >>> wrote: >>> > Get the same result with user doesn't exist >>> > C:\OPT\security_example>curl >>> > >>> > "http://localhost:8345/mcf-authority-service/UserACLs?username=fakeuser@fakedomain" >>> > AUTHORIZED:TEQA-DC >>> > TOKEN:TEQA-DC:S-1-1-0 >>> > >>> > BTW, is there a command to get all users available in Active Directory, >>> > from >>> > mcf-authority service, or other test commands to see if it's working >>> > correctly ? >>> > >>> > Also, I set the logging level to finest from Solr Admin for >>> > ManifoldCFSecurityFilter,but I don't see any logs created.. Is there >>> > any >>> > other settings need to be tweaked ? >>> > >>> > Thanks >>> > >>> > Kadri >>> > >>> > On Tue, Apr 26, 2011 at 12:38 PM, Karl Wright <daddy...@gmail.com> >>> > wrote: >>> >> >>> >> One other quick note. You might want to try a user that doesn't exist >>> >> and see what you get. It should be a USERNOTFOUND response. >>> >> >>> >> If that's indeed what you get back, then this is a relatively minor >>> >> issue with Active Directory. Basically the S-1-1-0 SID is added by >>> >> the active directory authority, so the DC is actually returning an >>> >> empty list of SIDs for the user with an unknown domain. It *should* >>> >> tell us the user doesn't exist, I agree, but that's clearly a problem >>> >> only Active Directory can solve; we can't make that decision in the >>> >> active directory connector because the DC may be just one node in a >>> >> hierarchy. Perhaps there's a Microsoft knowledge-base article that >>> >> would clarify things further. >>> >> >>> >> Please let me know what you find. >>> >> Karl >>> >> >>> >> On Tue, Apr 26, 2011 at 12:27 PM, Karl Wright <daddy...@gmail.com> >>> >> wrote: >>> >> > The method code from the Active Directory authority that handles the >>> >> > LDAP query construction is below. It looks perfectly reasonable to >>> >> > me: >>> >> > >>> >> > /** Parse a user name into an ldap search base. */ >>> >> > protected static String parseUser(String userName) >>> >> > throws ManifoldCFException >>> >> > { >>> >> > //String searchBase = >>> >> > "CN=Administrator,CN=Users,DC=qa-ad-76,DC=metacarta,DC=com"; >>> >> > int index = userName.indexOf("@"); >>> >> > if (index == -1) >>> >> > throw new ManifoldCFException("Username is in unexpected form >>> >> > (no @): '"+userName+"'"); >>> >> > String userPart = userName.substring(0,index); >>> >> > String domainPart = userName.substring(index+1); >>> >> > // Start the search base assembly >>> >> > StringBuffer sb = new StringBuffer(); >>> >> > sb.append("CN=").append(userPart).append(",CN=Users"); >>> >> > int j = 0; >>> >> > while (true) >>> >> > { >>> >> > int k = domainPart.indexOf(".",j); >>> >> > if (k == -1) >>> >> > { >>> >> > sb.append(",DC=").append(domainPart.substring(j)); >>> >> > break; >>> >> > } >>> >> > sb.append(",DC=").append(domainPart.substring(j,k)); >>> >> > j = k+1; >>> >> > } >>> >> > return sb.toString(); >>> >> > } >>> >> > >>> >> > So I have to conclude that your Active Directory domain controller >>> >> > is >>> >> > simply not caring what the DC= fields are, for some reason. No idea >>> >> > why. >>> >> > >>> >> > If you want to confirm this picture, you might want to create a >>> >> > patch >>> >> > to add some Logging.authorityConnectors.debug statements at >>> >> > appropriate places so we can see the actual query it's sending to >>> >> > LDAP. I'm happy to commit this debug output patch eventually if you >>> >> > also want to create a ticket. >>> >> > >>> >> > Thanks, >>> >> > Karl >>> >> > >>> >> > On Tue, Apr 26, 2011 at 12:17 PM, Kadri Atalay >>> >> > <atalay.ka...@gmail.com> >>> >> > wrote: >>> >> >> Yes, ManifoldCF is running with JCIFS connector, and using Solr 3.1 >>> >> >> >>> >> >> response to first call: >>> >> >> C:\OPT\security_example>curl >>> >> >> "http://localhost:8345/mcf-authority-service/UserACLs?username=joe" >>> >> >> UNREACHABLEAUTHORITY:TEQA-DC >>> >> >> TOKEN:TEQA-DC:DEAD_AUTHORITY >>> >> >> >>> >> >> response to fake domain call: >>> >> >> C:\OPT\security_example>curl >>> >> >> >>> >> >> >>> >> >> "http://localhost:8345/mcf-authority-service/UserACLs?username=joe@fakedomain" >>> >> >> AUTHORIZED:TEQA-DC >>> >> >> TOKEN:TEQA-DC:S-1-1-0 >>> >> >> >>> >> >> response to actual domain account call: >>> >> >> C:\OPT\security_example>curl >>> >> >> >>> >> >> >>> >> >> "http://localhost:8345/mcf-authority-service/UserACLs?username=katalay_admin@teqa" >>> >> >> AUTHORIZED:TEQA-DC >>> >> >> TOKEN:TEQA-DC:S-1-1-0 >>> >> >> >>> >> >> Looks like as long as there is a domain suffix, return is >>> >> >> positive.. >>> >> >> >>> >> >> Thanks >>> >> >> >>> >> >> Kadri >>> >> >> >>> >> >> >>> >> >> On Tue, Apr 26, 2011 at 12:10 PM, Karl Wright <daddy...@gmail.com> >>> >> >> wrote: >>> >> >>> >>> >> >>> So you are trying to extend the example in the book, correct, to >>> >> >>> run >>> >> >>> against active directory and the JCIFS connector? And this is >>> >> >>> with >>> >> >>> Solr 3.1? >>> >> >>> >>> >> >>> The book was written for Solr 1.4.1, so it's entirely possible >>> >> >>> that >>> >> >>> something in Solr changed in relation to the way search components >>> >> >>> are >>> >> >>> used. So I think we're going to need to do some debugging. >>> >> >>> >>> >> >>> (1) First, to confirm sanity, try using curl against the mcf >>> >> >>> authority >>> >> >>> service. Try some combination of users to see how that works, >>> >> >>> e.g.: >>> >> >>> >>> >> >>> curl >>> >> >>> >>> >> >>> "http://localhost:8345/mcf-authority-service/UserACLs?username=joe" >>> >> >>> >>> >> >>> ...and >>> >> >>> >>> >> >>> curl >>> >> >>> >>> >> >>> >>> >> >>> "http://localhost:8345/mcf-authority-service/UserACLs?username=joe@fakedomain" >>> >> >>> >>> >> >>> ...and also the real domain name, whatever that is. See if the >>> >> >>> access >>> >> >>> tokens that come back look correct. If they don't then we know >>> >> >>> where >>> >> >>> there's an issue. >>> >> >>> >>> >> >>> If they *are* correct, let me know and we'll go to the next stage, >>> >> >>> which would be to make sure the authority service is actually >>> >> >>> getting >>> >> >>> called and the proper query is being built and run under Solr 3.1. >>> >> >>> >>> >> >>> Thanks, >>> >> >>> Karl >>> >> >>> >>> >> >>> On Tue, Apr 26, 2011 at 11:59 AM, Kadri Atalay >>> >> >>> <atalay.ka...@gmail.com> >>> >> >>> wrote: >>> >> >>> > Hi Karl, >>> >> >>> > >>> >> >>> > I followed the instructions, and for testing purposes set >>> >> >>> > "stored=true" >>> >> >>> > to >>> >> >>> > be able to see the ACL values stored in Solr. >>> >> >>> > >>> >> >>> > But, when I run the search in following format I get peculiar >>> >> >>> > results.. >>> >> >>> > >>> >> >>> > >>> >> >>> > >>> >> >>> > :http://10.1.200.155:8080/solr/select/?q=*%3A*&AuthenticatedUserName=username >>> >> >>> > >>> >> >>> > Any user name without a domain name ie >>> >> >>> > AuthenticatedUserName=joe >>> >> >>> > does >>> >> >>> > not >>> >> >>> > return any results (which is correct) >>> >> >>> > But any user name with ANY domain name returns all the indexes >>> >> >>> > ie >>> >> >>> > AuthenticatedUserName=joe@fakedomain (which is not correct) >>> >> >>> > >>> >> >>> > Any thoughts ? >>> >> >>> > >>> >> >>> > Thanks >>> >> >>> > >>> >> >>> > Kadri >>> >> >>> > >>> >> >>> > On Sun, Apr 24, 2011 at 7:08 PM, Karl Wright >>> >> >>> > <daddy...@gmail.com> >>> >> >>> > wrote: >>> >> >>> >> >>> >> >>> >> Solr 3.1 is being clever here; it's seeing arguments coming in >>> >> >>> >> that >>> >> >>> >> do >>> >> >>> >> not correspond to known schema fields, and presuming they are >>> >> >>> >> "automatic" fields. So when the schema is unmodified, you see >>> >> >>> >> these >>> >> >>> >> fields that Solr creates for you, with the attr_ prefix. They >>> >> >>> >> are >>> >> >>> >> created as being "stored", which is not good for access tokens >>> >> >>> >> since >>> >> >>> >> then you will see them in the response. I don't know if they >>> >> >>> >> are >>> >> >>> >> indexed or not, but I imagine not, which is also not good. >>> >> >>> >> >>> >> >>> >> So following the instructions is still the right thing to do, I >>> >> >>> >> would >>> >> >>> >> say. >>> >> >>> >> >>> >> >>> >> Karl >>> >> >>> >> >>> >> >>> >> On Fri, Apr 22, 2011 at 3:24 PM, Kadri Atalay >>> >> >>> >> <atalay.ka...@gmail.com> >>> >> >>> >> wrote: >>> >> >>> >> > Hi Karl, >>> >> >>> >> > >>> >> >>> >> > There is one thing I noticed while following the example in >>> >> >>> >> > chapter >>> >> >>> >> > 4.: >>> >> >>> >> > Prior to making any changes into the schema.xml, I was able >>> >> >>> >> > to >>> >> >>> >> > see >>> >> >>> >> > the >>> >> >>> >> > following security information in query responses: >>> >> >>> >> > ie: >>> >> >>> >> > >>> >> >>> >> > <doc> >>> >> >>> >> > - >>> >> >>> >> > <arr name="attr_allow_token_document"> >>> >> >>> >> > <str>TEQA-DC:S-1-3-0</str> >>> >> >>> >> > <str>TEQA-DC:S-1-5-13</str> >>> >> >>> >> > <str>TEQA-DC:S-1-5-18</str> >>> >> >>> >> > <str>TEQA-DC:S-1-5-32-544</str> >>> >> >>> >> > <str>TEQA-DC:S-1-5-32-545</str> >>> >> >>> >> > <str>TEQA-DC:S-1-5-32-547</str> >>> >> >>> >> > </arr> >>> >> >>> >> > - >>> >> >>> >> > <arr name="attr_allow_token_share"> >>> >> >>> >> > <str>TEQA-DC:S-1-1-0</str> >>> >> >>> >> > <str>TEQA-DC:S-1-5-2</str> >>> >> >>> >> > - >>> >> >>> >> > <str> >>> >> >>> >> > TEQA-DC:S-1-5-21-1212545812-2858578934-3563067286-1480 >>> >> >>> >> > </str> >>> >> >>> >> > </arr> >>> >> >>> >> > - >>> >> >>> >> > <arr name="attr_content"> >>> >> >>> >> > - >>> >> >>> >> > <str> >>> >> >>> >> > Autonomy ODBC Fetch Technical >>> >> >>> >> > Brief >>> >> >>> >> > 0506 >>> >> >>> >> > Technical Brief >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> > But, after I modified the schema/xml, and added the following >>> >> >>> >> > fields, >>> >> >>> >> > <!-- Security fields --> >>> >> >>> >> > <field name="allow_token_document" type="string" >>> >> >>> >> > indexed="true" >>> >> >>> >> > stored="false" multiValued="true"/> >>> >> >>> >> > <field name="deny_token_document" type="string" >>> >> >>> >> > indexed="true" >>> >> >>> >> > stored="false" multiValued="true"/> >>> >> >>> >> > <field name="allow_token_share" type="string" >>> >> >>> >> > indexed="true" >>> >> >>> >> > stored="false" multiValued="true"/> >>> >> >>> >> > <field name="deny_token_share" type="string" >>> >> >>> >> > indexed="true" >>> >> >>> >> > stored="false" multiValued="true"/> >>> >> >>> >> > >>> >> >>> >> > I longer see neither the attr_allow_token_document or the >>> >> >>> >> > allow_token_document fields.. >>> >> >>> >> > >>> >> >>> >> > Since same fields exist with attr_ prefix, should we need to >>> >> >>> >> > add >>> >> >>> >> > these >>> >> >>> >> > new >>> >> >>> >> > field names into the schema file, or can we simply change >>> >> >>> >> > ManifoldSecurity >>> >> >>> >> > to use attr_ fields ? >>> >> >>> >> > >>> >> >>> >> > Also, when Solr is running under Tomcat, I have to re-start >>> >> >>> >> > the >>> >> >>> >> > Solr >>> >> >>> >> > App, or >>> >> >>> >> > re-start Tomcat to see the newly added indexes.. >>> >> >>> >> > >>> >> >>> >> > Any thoughts ? >>> >> >>> >> > >>> >> >>> >> > Thanks >>> >> >>> >> > >>> >> >>> >> > Kadri >>> >> >>> >> > >>> >> >>> >> > On Fri, Apr 22, 2011 at 12:53 PM, Karl Wright >>> >> >>> >> > <daddy...@gmail.com> >>> >> >>> >> > wrote: >>> >> >>> >> >> >>> >> >>> >> >> I don't believe Solr has yet officially released document >>> >> >>> >> >> access >>> >> >>> >> >> control, so you will need to use the patch for ticket 1895. >>> >> >>> >> >> Alternatively, the ManifoldCF in Action chapter 4 example >>> >> >>> >> >> has an >>> >> >>> >> >> implementation based on this ticket. You can get the code >>> >> >>> >> >> for >>> >> >>> >> >> it at >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> https://manifoldcfinaction.googlecode.com/svn/trunk/edition_1/security_example. >>> >> >>> >> >> >>> >> >>> >> >> Thanks, >>> >> >>> >> >> Karl >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> On Fri, Apr 22, 2011 at 11:45 AM, Kadri Atalay >>> >> >>> >> >> <atalay.ka...@gmail.com> >>> >> >>> >> >> wrote: >>> >> >>> >> >> > Hello, >>> >> >>> >> >> > >>> >> >>> >> >> > Does anyone know which version of Solr have implements the >>> >> >>> >> >> > Document >>> >> >>> >> >> > Level >>> >> >>> >> >> > Access Control, or has it implemented (partially or fully) >>> >> >>> >> >> > ? >>> >> >>> >> >> > Particularly issue #s 1834, 1872, 1895 >>> >> >>> >> >> > >>> >> >>> >> >> > Thanks >>> >> >>> >> >> > >>> >> >>> >> >> > Kadri >>> >> >>> >> >> > >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> > >>> >> >>> > >>> >> >> >>> >> >> >>> >> > >>> > >>> > >> > >