[ 
https://issues.apache.org/jira/browse/SOLR-14653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17159595#comment-17159595
 ] 

Erick Erickson commented on SOLR-14653:
---------------------------------------

After looking this over, I decided to just go with some simple string 
manipulation in a utility class. ZK paths have nothing really to do with URIs, 
Files, Paths, etc. they're just simple strings with no sensitivity to the file 
system etc. I toyed a bit with a utility method that would take a bunch of 
fragments and assemble them into a Znode path, but then realized that was a 
whole can of worms. There are lots and lots of places where we construct a path 
just by putting together strings, look at calls to SolrZkClient.create for 
instance.

So this patch takes a minimalist approach. There are three cases where 
hadoop.fs.Path was brought in outside of something having hdfs in the class 
name:
 * to get the parent from a string. These now call 
ZkMaintenanceUtils.getZkParent.
 * to pull in Path.SEPARATOR to construct a path. This is silly. These now just 
hard-code "/" like we do everywhere else.
 * SolrSnapshotsTool and DiskChecker (a test) which I left alone.

If we're going to do something more robust when it comes to creating ZK paths, 
it'll be a much larger undertaking to retrofit all the places we construct them 
inline now. I think this address the immediate concern.

All tests pass, I'll commit tomorrow baring objections.

> Replace usages of org.apache.hadoop.fs.Path that are used to construct 
> Zookeeper paths
> --------------------------------------------------------------------------------------
>
>                 Key: SOLR-14653
>                 URL: https://issues.apache.org/jira/browse/SOLR-14653
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>            Reporter: Erick Erickson
>            Assignee: Erick Erickson
>            Priority: Major
>         Attachments: SOLR-14653.patch
>
>
> See SOLR-13939. We construct Zookeeper paths at least in 
> ShardLeaderElectionContextBase.java by using methods in 
> org.apache.hadoop.fs.Path e.g. "String parent = new 
> Path(leaderPath).getParent().toString()". This works becauseĀ  the Hadoop 
> method uses forward slashes even on Windows machines.
> If those calls are changed to use nio Paths.get or similar, we can't run on 
> Windows. Relying on hdfs classes to construct these paths in code unrelated 
> to HDFS is suspicious at best.
> We should examine all inclusions of org.apache.hadoop.fs.Path and replace 
> them with something else. Uwe suggests java.net.URI as one possibility.
> We should check to see what Zookeeper provides.
> There are 5 files that import org.apache.hadoop... that do _not_ have HDFS in 
> their name, and 4 of those are probably legitimate (SolrSnapshotsTool for 
> instance) so checking all of them won't take long.
> From Uwe's comment in SOLR-13939:
> {quote}"Hi,
>  Windows is happy again. So your fix helped. I responded to mailing list, 
> here again:
> In Zookeeper all paths are forward slashes (as it is a "path" inside the 
> zookeeper environment). If you replace that with a NIO Windows Path it 
> creates backslashes. This makes Zookeeper fail (of course).
> Actually using the Hadoop class is broken workaround, it may be better to use 
> the more abstract java.net.URI class for correct "path" transformations. The 
> risk with java.nio.file.Path is also that if the path exists in file system 
> it handles the resolving different. So if some user has for some reason a 
> local path exactly looking like a zookeper node path it would use different 
> behaviour.
> So my suggestion: Use java.net.URI to build the path and use URI.getPath() or 
> similar to make a String out of it. The alternative to replace all 
> backslashes in the path to forwards slashes is a bad idea for the reasons I 
> explained before.
> Now we should replace the hadoop class IMHO by java.net.URI.
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to