[
https://issues.apache.org/jira/browse/FALCON-94?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13791191#comment-13791191
]
Srikanth Sundarrajan commented on FALCON-94:
--------------------------------------------
Generic CatalogService (AbstractCatalogService) seems to return HCatPartition,
which would mean that the CatalogService is tied to HCatalog. If that was the
intention, you might consider renaming AbstractCatalogService to
AbstractHCatalogService.
{code}
+ public abstract List<HCatPartition> listPartitionsByFilter(String
catalogUrl, String database,
+ String
tableName, String filter)
+ throws FalconException;
{code}
We are requiring some hcatalog related jars to be copied to oozie shared lib
directory. I am assuming this will be covered in documentation. Also this would
mean that Off the shelf Oozie patched with falcon config wont work any more. We
further require the shared lib dirs to be setup and the contents copied. Are
there any challenges in making these jars available in the retention lib path
by default and not requiring shared libs ?
{code}
+ <!-- HCatalog jars -->
+ <property>
+ <name>oozie.use.system.libpath</name>
+ <value>true</value>
+ </property>
+ <property>
+ <name>oozie.action.sharelib.for.java</name>
+ <value>hcatalog</value>
+ </property>
{code}
Is "feedStorageType" a new property added to oozie coordinator for retention.
If so, can this be prefixed with "falcon.". The older ones without prefix need
to be fixed some time in future (Since that would be an incompatible change, we
need to hold off on that). Meanwhile, all new property additions can be
prefixed appropriately.
{code}
+ } else if ("feedStorageType".equals(property.getName())) {
+ storageType = property.getValue();
{code}
Can ${ & ?{ be defined as constants?
{code}
+ Assert.assertEquals(feedDataPath,
storage.getUriTemplate().replaceAll("\\$\\{", "\\?\\{"));
{code}
Since the feed storage is abstracted in Storage class either as
FileSystemStorage or CatalogStorage, moreover, behaviors for listing partitions
and drop partitions are listed there, why do we need to hardcode the eviction
behavior and instance deletion discovery for filesystem need to happen in
FeedEvictor ? Why can't be implemented in appropriate Storage implementation.
That way FeedEvictor would simpler and lot cleaner. Thoughts ?
> Retention to handle hive table eviction
> ---------------------------------------
>
> Key: FALCON-94
> URL: https://issues.apache.org/jira/browse/FALCON-94
> Project: Falcon
> Issue Type: Sub-task
> Affects Versions: 0.3
> Reporter: Venkatesh Seetharam
> Assignee: Venkatesh Seetharam
> Attachments: FALCON-94.patch, FALCON-94-r1.patch, FALCON-94-r2.patch
>
>
> Must handle both hive managed and external tables.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
