Thanks Tim. Yeah I was able to get it to work by pointing directly to the hdfs:// URL for the data we want to query. For our initial Impala experiments we should be able to proceed. Do you know if there’s a jira tracking the long term fix? (or is that going to continue to be Impala-77?). Based on our initial experiments, we might be open to helping out with the fix, we can chime in on the jira in the next couple of weeks as well.
Thanks, -- Piyush From: Tim Armstrong <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Monday, December 4, 2017 at 8:35 PM To: "[email protected]" <[email protected]> Subject: Re: Using Impala with a federated HDFS setup Hi Piyush, You're right that we don't support ViewFileSystem at the moment. It looks like IMPALA-77 was resolved by failing more gracefully for viewfs. I suspect it just needs some targeted code changes and testing - the logic for different filesystems is mostly the same with some tweaks. I'm not sure when this will happen - it will likely largely depend on when someone steps forward to do it. I believe pointing directly to the hdfs:// URLs should work. - Tim On Thu, Nov 30, 2017 at 11:39 AM, Piyush Narang <[email protected]<mailto:[email protected]>> wrote: Hi folks, Our company is looking to experiment with setting up Impala for some of our adhoc query workloads. I was working on setting up Impala to test things out and I ran into the following errors on startup, “Currently configured default filesystem: ViewFileSystem. fs.defaultFS (viewfs://root) is not supported.”. Noticed that this has been implemented as part of this jira: https://issues.apache.org/jira/browse/IMPALA-77<https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_IMPALA-2D77&d=DwMFaQ&c=nxfEpP1JWHVKAq835DW4mA&r=3Ka-O_qIfLiCDaGELmIN3BcChZatNdPOwe36odQXFYo&m=qT5IbyXEu2Sy58jS9NeuJSnIOLpbD0hRJb4qI6t5_IY&s=GMdxzTI4bDF8FUK-J5SzOIyWrzk-S6NsEH0rVKTHKLg&e=>. Is this still currently not on the Impala roadmap? Are there any possible workarounds for users with federated namenodes? The data we want to query as of now resides in one namenode’s namespace. A potential workaround for us might be to just expose that hdfs:// mount point directly. Not sure if anyone’s tried this and what kinds of issues they’ve run into. Thanks, Piyush
