[
https://issues.apache.org/jira/browse/TRAFODION-2091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15361874#comment-15361874
]
ASF GitHub Bot commented on TRAFODION-2091:
-------------------------------------------
GitHub user sandhyasun opened a pull request:
https://github.com/apache/incubator-trafodion/pull/574
Fix for [TRAFODION-2091] Lobglobals and hdfsFS handle fix
hdfsFS handling is now done at the Context level and is maintained as
hashqueue for each hdfsPort/hdfsSever combination.
The first time a component connects, it checks thae global list, if the
handle is not found a new entry is created and added to the list.
All calls to individual callers making hdfs Disconnect calls are removed
since they affect other components usage of hdfs API.
Cleanup in how LobGlobals are allocated and maintained. They are part of
ex_globals and TCBs get a pointer to the ExLobGlobals when they need to use the
LOB interface.
Fixed dataModCheck and emptyDirectory to change how lobGlobals
initialization happens.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sandhyasun/incubator-trafodion lobglobals_fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-trafodion/pull/574.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #574
----
commit 7975a58ece0c6758a0070c15f8e9da2458d8c346
Author: Sandhya Sundaresan <sand...@apache.org>
Date: 2016-07-01T21:17:07Z
Changes to move hdfFs handing to the context globals level and remove all
disconnects from hdfs from the various components.
commit 5ecd3c60282ffdc114ac4885eab10cf04680482d
Author: Sandhya Sundaresan <sand...@apache.org>
Date: 2016-07-01T22:02:58Z
Merge remote branch 'origin/master' into lobglobals_fix
Conflicts:
core/sql/cli/Context.cpp
core/sql/exp/ExpLOBaccess.cpp
commit ac1b131c86105a0e95c7061adba58d4c36510af6
Author: Sandhya Sundaresan <sand...@apache.org>
Date: 2016-07-04T00:44:36Z
Regression fix and fixes for emptyDirectory().
----
> hdfsFS filesystem usage causies abnormal behavior in Trafodion
> ---------------------------------------------------------------
>
> Key: TRAFODION-2091
> URL: https://issues.apache.org/jira/browse/TRAFODION-2091
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-general
> Affects Versions: 2.1-incubating
> Reporter: Sandhya Sundaresan
> Assignee: Sandhya Sundaresan
>
> Several components use the libhdfs API to perform operations on hdfs files in
> Trafodion. Some places use the JNI API as well to access hdfs.
> The inconsistency observed was ehen one component cached and kept the hdfsFS
> hande for it's use another part of the code would perform a "close" or a
> dsconnect and prematurely close the hdfsFS handle which is currently being
> used. The reason is that the hdfs layer seems to cache the hdfsFS handle
> underneathThe solution now is to keep a list of hdfsFS handles at the global
> context level. All disconnect calls and JNI close calls are remmoved. The
> hdfHS handles will get cleaned up either at context destructor time or at
> endSession time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
