[jira] [Commented] (GORA-666) Add support for the Kudu datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223964#comment-17223964 ] Alfonso Nishikawa commented on GORA-666: Merged https://github.com/apache/gora/pull/232 to master. Thank you, [~arqgabyortiz]! > Add support for the Kudu datastore in GoraExplorer > -- > > Key: GORA-666 > URL: https://issues.apache.org/jira/browse/GORA-666 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Assignee: Gaby Ortiz >Priority: Major > Fix For: 1.0 > > > Add support for the Kudu datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (GORA-666) Add support for the Kudu datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alfonso Nishikawa resolved GORA-666. Resolution: Fixed > Add support for the Kudu datastore in GoraExplorer > -- > > Key: GORA-666 > URL: https://issues.apache.org/jira/browse/GORA-666 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Assignee: Gaby Ortiz >Priority: Major > Fix For: 1.0 > > > Add support for the Kudu datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (GORA-666) Add support for the Kudu datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alfonso Nishikawa updated GORA-666: --- Fix Version/s: 1.0 > Add support for the Kudu datastore in GoraExplorer > -- > > Key: GORA-666 > URL: https://issues.apache.org/jira/browse/GORA-666 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Assignee: Gaby Ortiz >Priority: Major > Fix For: 1.0 > > > Add support for the Kudu datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alfonso Nishikawa resolved GORA-557. Resolution: Fixed > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Assignee: Mariia Podorvanova >Priority: Minor > Labels: gsoc2019 > Fix For: 1.0 > > Time Spent: 1h > Remaining Estimate: 0h > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223960#comment-17223960 ] Alfonso Nishikawa commented on GORA-557: Merged https://github.com/apache/gora/pull/231 to master. Thank you, [~mariia.podorvanova]! > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Assignee: Mariia Podorvanova >Priority: Minor > Labels: gsoc2019 > Fix For: 1.0 > > Time Spent: 1h > Remaining Estimate: 0h > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alfonso Nishikawa updated GORA-557: --- Fix Version/s: 1.0 > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Assignee: Mariia Podorvanova >Priority: Minor > Labels: gsoc2019 > Fix For: 1.0 > > Time Spent: 1h > Remaining Estimate: 0h > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-665) Add support for the Ignite datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223891#comment-17223891 ] Alfonso Nishikawa commented on GORA-665: Merged to master. Thank you, [~arqgabyortiz]! > Add support for the Ignite datastore in GoraExplorer > > > Key: GORA-665 > URL: https://issues.apache.org/jira/browse/GORA-665 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Assignee: Gaby Ortiz >Priority: Major > Fix For: 1.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Add support for the Ignite datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (GORA-665) Add support for the Ignite datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alfonso Nishikawa resolved GORA-665. Resolution: Fixed > Add support for the Ignite datastore in GoraExplorer > > > Key: GORA-665 > URL: https://issues.apache.org/jira/browse/GORA-665 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Assignee: Gaby Ortiz >Priority: Major > Fix For: 1.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Add support for the Ignite datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (GORA-665) Add support for the Ignite datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alfonso Nishikawa updated GORA-665: --- Fix Version/s: 1.0 > Add support for the Ignite datastore in GoraExplorer > > > Key: GORA-665 > URL: https://issues.apache.org/jira/browse/GORA-665 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Assignee: Gaby Ortiz >Priority: Major > Fix For: 1.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Add support for the Ignite datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [gora] jhnmora000 commented on a change in pull request #231: GORA-557 Add support for MongoDB to GoraExplorer
jhnmora000 commented on a change in pull request #231:
URL: https://github.com/apache/gora/pull/231#discussion_r514816245
##
File path:
gora-mongodb/src/main/java/org/apache/gora/mongodb/store/MongoStoreMetadataAnalyzer.java
##
@@ -0,0 +1,67 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.mongodb.store;
+
+import com.mongodb.client.MongoClient;
+import com.mongodb.client.MongoDatabase;
+import org.apache.gora.store.impl.DataStoreMetadataAnalyzer;
+import org.apache.gora.util.GoraException;
+
+import java.util.ArrayList;
+import java.util.List;
+
+public class MongoStoreMetadataAnalyzer extends DataStoreMetadataAnalyzer {
+
+private MongoDatabase mongoDatabase;
+private MongoClient mongoClient;
+
+@Override
+public void initialize() {
+MongoStoreParameters parameters =
MongoStoreParameters.load(properties, getConf());
+mongoClient = MongoStore.getClient(parameters);
+mongoDatabase = mongoClient.getDatabase(parameters.getDbname());
+}
+
+@Override
+public String getType() {
+return "MONGODB";
+}
+
+@Override
+public List getTablesNames() throws GoraException {
+try {
+return mongoDatabase.listCollectionNames().into(new ArrayList<>());
+} catch (Exception e) {
+throw new GoraException(e);
+}
+}
+
+@Override
+public MongoStoreCollectionMetadata getTableInfo(String tableName) {
+MongoStoreCollectionMetadata collectionMetadata = new
MongoStoreCollectionMetadata();
+
collectionMetadata.getCollectionDocumentKeys().addAll(mongoDatabase.getCollection(tableName).find().first().keySet());
Review comment:
This will work if there is at least one document in the collection. But,
since we do not enforce a schema in the collection this approach should do the
job.
In the future we should give a look to:
https://docs.mongodb.com/manual/core/schema-validation/ maybe a Jira issue
should be created.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [gora] jhnmora000 commented on a change in pull request #232: Add support for the Kudu datastore in GoraExplorer
jhnmora000 commented on a change in pull request #232: URL: https://github.com/apache/gora/pull/232#discussion_r514782899 ## File path: gora-kudu/pom.xml ## @@ -93,7 +93,7 @@ org.apache.maven.plugins maven-surefire-plugin - true + false Review comment: The tests of Kudu fail on Jenkins (https://issues.apache.org/jira/browse/GORA-636), please undo this change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] alfonsonishikawa commented on a change in pull request #228: [GORA-665] - Add support for the Ignite datastore in GoraExplorer
alfonsonishikawa commented on a change in pull request #228:
URL: https://github.com/apache/gora/pull/228#discussion_r514614524
##
File path:
gora-ignite/src/main/java/org/apache/gora/ignite/store/IgniteTableMetadata.java
##
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.ignite.store;
+
+import java.util.Map;
+
+public class IgniteTableMetadata {
+
+ //Primary key
+ private String primaryKey;
Review comment:
Is it possible to get the Primary Key type too?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [gora] alfonsonishikawa commented on a change in pull request #232: Add support for the Kudu datastore in GoraExplorer
alfonsonishikawa commented on a change in pull request #232:
URL: https://github.com/apache/gora/pull/232#discussion_r514616517
##
File path:
gora-kudu/src/main/java/org/apache/gora/kudu/store/KuduStoreMetadataAnalyzer.java
##
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.kudu.store;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Properties;
+import org.apache.gora.kudu.utils.KuduParameters;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.gora.store.impl.DataStoreMetadataAnalyzer;
+import org.apache.gora.util.GoraException;
+import org.apache.kudu.ColumnSchema;
+import org.apache.kudu.client.KuduClient;
+import org.apache.kudu.client.KuduException;
+import org.apache.kudu.client.KuduTable;
+
+public class KuduStoreMetadataAnalyzer extends DataStoreMetadataAnalyzer {
+
+ private KuduClient client;
+
+ @Override
+ public void initialize() throws GoraException {
+Properties createProps = DataStoreFactory.createProps();
+KuduParameters kuduParameters = KuduParameters.load(createProps,
getConf());
+KuduClient.KuduClientBuilder kuduClientBuilder = new
KuduClient.KuduClientBuilder(kuduParameters.getMasterAddresses());
+if (kuduParameters.getBossCount() != null) {
+ kuduClientBuilder.bossCount(kuduParameters.getBossCount());
+}
+if (kuduParameters.getDefaultAdminOperationTimeoutMs() != null) {
+
kuduClientBuilder.defaultAdminOperationTimeoutMs(kuduParameters.getDefaultAdminOperationTimeoutMs());
+}
+if (kuduParameters.getDefaultOperationTimeoutMs() != null) {
+
kuduClientBuilder.defaultOperationTimeoutMs(kuduParameters.getDefaultOperationTimeoutMs());
+}
+if (kuduParameters.getDefaultSocketReadTimeoutMs() != null) {
+
kuduClientBuilder.defaultSocketReadTimeoutMs(kuduParameters.getDefaultSocketReadTimeoutMs());
+}
+if (kuduParameters.getWorkerCount() != null) {
+ kuduClientBuilder.workerCount(kuduParameters.getWorkerCount());
+}
+if (kuduParameters.isClientStatistics() != null &&
!kuduParameters.isClientStatistics()) {
+ kuduClientBuilder.disableStatistics();
+}
+client = kuduClientBuilder.build();
+ }
+
+ @Override
+ public String getType() {
+return "KUDU";
+ }
+
+ @Override
+ public List getTablesNames() throws GoraException {
+try {
+ return client.getTablesList().getTablesList();
+} catch (KuduException ex) {
+ throw new GoraException(ex);
+}
+ }
+
+ @Override
+ public Object getTableInfo(String tableName) throws GoraException {
Review comment:
This should return KuduTableMetadata :)
##
File path:
gora-kudu/src/main/java/org/apache/gora/kudu/store/KuduTableMetadata.java
##
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.kudu.store;
+
+import java.util.Map;
+
+public class KuduTableMetadata {
+
+ //Primary key
+ private String primaryKey;
Review comment:
From the example mapping in tests, the primary key column has a type.
Can only be "STRING" or we can get the type from kudu?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please
[GitHub] [gora] alfonsonishikawa commented on pull request #231: GORA-557 Add support for MongoDB to GoraExplorer
alfonsonishikawa commented on pull request #231: URL: https://github.com/apache/gora/pull/231#issuecomment-719069057 Oh! And we would need a test for MongoStoreMetadataAnalyzer :S This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] mgov88 opened a new pull request #232: Add support for the Kudu datastore in GoraExplorer
mgov88 opened a new pull request #232: URL: https://github.com/apache/gora/pull/232 Add metadata analyzer feature in KUDU for GoraExplorer support. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] alfonsonishikawa commented on pull request #231: GORA-557 Add support for MongoDB to GoraExplorer
alfonsonishikawa commented on pull request #231: URL: https://github.com/apache/gora/pull/231#issuecomment-719066358 > Since MongoDB has collections instead of tables and these collections do not necessarily have a schema, I decided to use random collection document's key set. I see that in the gora-mongodb-mapping.xml from tests we have: ``` ``` so, there is something like "fields" with type? The document keyset you get are those names in "docfield="? is it possible to get the "type" too, if any? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] alfonsonishikawa removed a comment on pull request #231: GORA-557 Add support for MongoDB to GoraExplorer
alfonsonishikawa removed a comment on pull request #231: URL: https://github.com/apache/gora/pull/231#issuecomment-719065268 This pull request looks good to me. If no one comments anything I will merge it tomorrow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] alfonsonishikawa commented on pull request #231: GORA-557 Add support for MongoDB to GoraExplorer
alfonsonishikawa commented on pull request #231: URL: https://github.com/apache/gora/pull/231#issuecomment-719065268 This pull request looks good to me. If no one comments anything I will merge it tomorrow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] mgov88 commented on pull request #228: [GORA-665] - Add support for the Ignite datastore in GoraExplorer
mgov88 commented on pull request #228: URL: https://github.com/apache/gora/pull/228#issuecomment-718981167 @alfonsonishikawa I developed the Metadata Analyzer feature and a test, I had to update the ignite version, I can not find the schema information of the tables in the older version. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] podorvanova opened a new pull request #231: GORA-557 Add support for MongoDB to GoraExplorer
podorvanova opened a new pull request #231: URL: https://github.com/apache/gora/pull/231 Since MongoDB has collections instead of tables and these collections do not necessarily have a schema, I decided to use random collection document's key set. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] alfonsonishikawa commented on pull request #228: [GORA-665] - Add support for the Ignite datastore in GoraExplorer
alfonsonishikawa commented on pull request #228: URL: https://github.com/apache/gora/pull/228#issuecomment-718213725 @mgov88 I find we will need the metadata analyzer for Ignite too to have fully support of the features related to GoraExplorer. Thank you! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] mgov88 commented on pull request #228: [GORA-665] - Add support for the Ignite datastore in GoraExplorer
mgov88 commented on pull request #228: URL: https://github.com/apache/gora/pull/228#issuecomment-718227118 @alfonsonishikawa Hi, I took a look of HBase metadata analyzer https://github.com/apache/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStoreMetadataAnalyzer.java I there are two methods to be implemented: getTablesNames and getTableInfo. getTablesNames is just a list of the tables and getTableInfo is a list of the columns of a table is it correct? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] alfonsonishikawa edited a comment on pull request #228: [GORA-665] - Add support for the Ignite datastore in GoraExplorer
alfonsonishikawa edited a comment on pull request #228: URL: https://github.com/apache/gora/pull/228#issuecomment-718234821 > @alfonsonishikawa Hi, I took a look of HBase metadata analyzer https://github.com/apache/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStoreMetadataAnalyzer.java I there are two methods to be implemented: > getTablesNames and getTableInfo. > > getTablesNames is just a list of the tables > and getTableInfo is a list of the columns of a table > > is it correct? It is correct but that is specific for HBase. The first method, `getTablesNames` is needed for the type of mapping HBase has in the tag ``. The signature for this method returns `List` because I expected all datastores to have some "table" defined in their mappings. The second method, `getTableInfo` returns specific data of the table and for HBase is an instance of `HBaseTableMetadata`, so in Ignite should be something like `IgniteTableMetadata`. The class IgniteTableMetadata that you have to create should have the needed information you can recover from the native store to help creating the mapping for that table. The rule of thumb is that `IgniteTableMetadata` should return metadata about a table in Ignite. In the case of HBase are things called "Column Families", but that is absolutely related to HBase, not Ignite. It could be, columns and types of each column, for example. Or columns, type, sizes, etc... As I see from https://github.com/apache/gora/blob/master/gora-ignite/src/test/resources/gora-ignite-mapping.xml it looks Ignite needs the column name and it's type. So, that should be the minimum metadata returned in a (still not implemented) IgniteTableMetadata.java This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] alfonsonishikawa commented on pull request #228: [GORA-665] - Add support for the Ignite datastore in GoraExplorer
alfonsonishikawa commented on pull request #228: URL: https://github.com/apache/gora/pull/228#issuecomment-718235012 Oh! And the primary key! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [gora] alfonsonishikawa commented on pull request #228: [GORA-665] - Add support for the Ignite datastore in GoraExplorer
alfonsonishikawa commented on pull request #228: URL: https://github.com/apache/gora/pull/228#issuecomment-718234821 > @alfonsonishikawa Hi, I took a look of HBase metadata analyzer https://github.com/apache/gora/blob/master/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStoreMetadataAnalyzer.java I there are two methods to be implemented: > getTablesNames and getTableInfo. > > getTablesNames is just a list of the tables > and getTableInfo is a list of the columns of a table > > is it correct? It is correct but that is specific for HBase. The first method, `getTablesNames` is needed for the type of mapping HBase has in the tag ``. The signature for this method returns `List` because I expected all datastores to have some "table" defined in their mappings. The sencond method, `getTableInfo` returns specific data of the table and for HBase is an instance of `HBaseTableMetadata`, so in Ignite should be something like `IgniteTableMetadata`. The class IgniteTableMetadata that you have to create should have the needed information you can recover from the native store to help creating the mapping for that table. The rule of thumb is that `IgniteTableMetadata` should return metadata about a table in Ignite. In the case of HBase are things called "Column Families", but that is absolutely related to HBase, not Ignite. It could be, columns and types of each column, for example. Or columns, type, sizes, etc... As I see from https://github.com/apache/gora/blob/master/gora-ignite/src/test/resources/gora-ignite-mapping.xml it looks Ignite needs the column name and it's type. So, that should be the minimum metadata returned in a (still not implemented) IgniteTableMetadata.java This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Assigned] (GORA-665) Add support for the Ignite datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alfonso Nishikawa reassigned GORA-665: -- Assignee: Gaby Ortiz > Add support for the Ignite datastore in GoraExplorer > > > Key: GORA-665 > URL: https://issues.apache.org/jira/browse/GORA-665 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Assignee: Gaby Ortiz >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Add support for the Ignite datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (GORA-666) Add support for the Kudu datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alfonso Nishikawa reassigned GORA-666: -- Assignee: Gaby Ortiz > Add support for the Kudu datastore in GoraExplorer > -- > > Key: GORA-666 > URL: https://issues.apache.org/jira/browse/GORA-666 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Assignee: Gaby Ortiz >Priority: Major > > Add support for the Kudu datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221860#comment-17221860 ] Alfonso Nishikawa commented on GORA-557: Hi, [~mariia.podorvanova]. Here is some documentation about the metadata analyzer: https://translate.google.com/translate?sl=es=en=https://redmine.nishilua.com/projects/goraexplorer/wiki/Obtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas (I Googletranslated it from Spanish). One of the constructors of DataStoreMetadataFactory receives a Properties (that, probably, will be gora.properties or set by code), and as you say, DataStoreMetadataAnalyzer does not seems to have that field. Probably you have a good reason to add that field at DataStoreMetadataAnalyzer and populate it when the factory instantiates a MongoDB analyzer :) > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Assignee: Mariia Podorvanova >Priority: Minor > Labels: gsoc2019 > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221857#comment-17221857 ] Mariia Podorvanova commented on GORA-557: - Hi, [~alfonso.nishikawa], thank you for the prompt response. I was talking about the retrieval of native information from MongoDB. I am working on MongoStoreDataAnalyzer, which should connect to a database while initializing. HBaseStoreDataAnalyzer uses hadoop configuration, but MongoDB requires properties. You are saying that properties are configured when creating a connection, but I am struggling to find how to access properties from MongoStoreDataAnalyzer. Do you know how I can access them? > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Assignee: Mariia Podorvanova >Priority: Minor > Labels: gsoc2019 > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221842#comment-17221842 ] Alfonso Nishikawa commented on GORA-557: Hi, [~mariia.podorvanova]. What do you refer exactly as "the second task of the ticket"? The second bullet point or "A plus would be implement the support in GoraExplorer so it will be possible to manage MongoDB data visually"? If it is the "blahblahblah data visually", when creating a connection there is a "hadoop configuration" option that you can ignore. It is there because HBase uses a file hbase-site.xml configuration that is hadoop-style, but if mongodb only needs gora.properties, that is configured when creating a connection too. The issue is that GoraExplorer does not support mongodb. If you want to implement that part, you can take a look at what [~arqgabyortiz] has been doing: https://bitbucket.org/alfonsonishikawa/goraexplorer/pull-requests/1/add-support-for-the-ignite-datastore-in/diff#comment-184402931 Here is how to prepare the development environment: https://bitbucket.org/alfonsonishikawa/goraexplorer/wiki/Building Here is a video list on what is GoraExplorer (development): https://www.youtube.com/watch?v=TTtoRh5aaDI=PLfaBSywymrO86u9SOqRCwGB6bz5ru1al7 And here is a live demo of the application: https://www.goraexplorer.com/livedemo/ Know that GoraExplorer is an independen project different from Apache Gora! > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Assignee: Mariia Podorvanova > Priority: Minor > Labels: gsoc2019 > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-666) Add support for the Kudu datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221502#comment-17221502 ] Gaby Ortiz commented on GORA-666: - Hello [~alfonso.nishikawa] [~jhnmora000] Thanks for your answers. I will try to implement the missing feature. > Add support for the Kudu datastore in GoraExplorer > -- > > Key: GORA-666 > URL: https://issues.apache.org/jira/browse/GORA-666 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Priority: Major > > Add support for the Kudu datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221308#comment-17221308 ] Mariia Podorvanova commented on GORA-557: - Hi, [~alfonso.nishikawa] I have a question about the second task of the ticket. I do not quite get how does the configuration in Gora Explorer work. Hadoop configuration is being used for HBaseStore, is it expected of me to also use Hadoop configuration to open connection with MongoDB? > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Assignee: Mariia Podorvanova >Priority: Minor > Labels: gsoc2019 > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-666) Add support for the Kudu datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221042#comment-17221042 ] Alfonso Nishikawa commented on GORA-666: Hi, [~arqgabyortiz]. Yes, there are something missing for Gora Kudu support. There are some classes that retrieve native information from the underlying datastore that at this moment only is implemented for HBase. Kudu is missing that feature. GoraExplorer uses that native information to help creating mappings with a Wizard (and it is specific for each datastore). Here is some documentation about it: https://redmine.nishilua.com/projects/goraexplorer/wiki/Obtenci%C3%B3n_de_metadatos_de_las_BBDD_nativas (it is in spanish, but I suspect you won't have any issue with it ;P) Probably it is not the best design :( I made it in quite a hurry. If you see anything improvable, let me know :) > Add support for the Kudu datastore in GoraExplorer > -- > > Key: GORA-666 > URL: https://issues.apache.org/jira/browse/GORA-666 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Priority: Major > > Add support for the Kudu datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-665) Add support for the Ignite datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220404#comment-17220404 ] John Mora commented on GORA-665: Hi [~arqgabyortiz] , Thanks for your interest in this issue. I think [~carlosrmng] developed the Ignite datastore, maybe he can check the changes you made. Best, John > Add support for the Ignite datastore in GoraExplorer > > > Key: GORA-665 > URL: https://issues.apache.org/jira/browse/GORA-665 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Add support for the Ignite datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-666) Add support for the Kudu datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220392#comment-17220392 ] John Mora commented on GORA-666: Hi [~arqgabyortiz] Thanks for your interest in this issue, I was not aware of that. I think [~alfonso.nishikawa] knows more about Gora Explorer than I do. Alfonso, is the support for Kudu complete? > Add support for the Kudu datastore in GoraExplorer > -- > > Key: GORA-666 > URL: https://issues.apache.org/jira/browse/GORA-666 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Priority: Major > > Add support for the Kudu datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-665) Add support for the Ignite datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220236#comment-17220236 ] Gaby Ortiz commented on GORA-665: - HI. I took a look to the last datastore added to GoraExplorer : KUDU in these commits [Commit 1|https://bitbucket.org/alfonsonishikawa/goraexplorer/commits/29ca730b703bfbf18986da8d48426b93af92c944] and [Commit 2 |https://bitbucket.org/alfonsonishikawa/goraexplorer/commits/064892aed37025848febcd5f4952dd4ad5ea0d87] and I tried to reproduce those changes for Ignite, but I got this error: "Mapping file 'gora-ignite-mapping.xml' could not be found!" After some research and comparison with KUDU I noticed that the method for reading the mapping was a little different in Ignite, so I changed it and it solved the problem, I sent a [pull request|https://github.com/apache/gora/pull/228] for this, please check it if you can, I hope I did not break anything. > Add support for the Ignite datastore in GoraExplorer > > > Key: GORA-665 > URL: https://issues.apache.org/jira/browse/GORA-665 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Add support for the Ignite datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [gora] mgov88 opened a new pull request #228: [GORA-665] - Add support for the Ignite datastore in GoraExplorer
mgov88 opened a new pull request #228: URL: https://github.com/apache/gora/pull/228 To add support for Apache Ignite in Gora Explorer the mapping XML must be read from gora.mapping in the same way as is implemented in the Kudu datastore. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (GORA-666) Add support for the Kudu datastore in GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17220228#comment-17220228 ] Gaby Ortiz commented on GORA-666: - Hi [~jhnmora000] I took a look to Gora Explorer and it seems that Kudu is available in the Connectors list, are there pending tasks for a complete support?. > Add support for the Kudu datastore in GoraExplorer > -- > > Key: GORA-666 > URL: https://issues.apache.org/jira/browse/GORA-666 > Project: Apache Gora > Issue Type: New Feature >Reporter: John Mora >Priority: Major > > Add support for the Kudu datastore in GoraExplorer - > https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Accessing GoraExplorer
Hi, Nilothpal. Write me to alfonso.nishik...@gmail.com with your IP and I remove the firewall rules. The past week I updated te firewall rules and it seems they are too much restrictive :S Thank you! :) Best regards, Alfonso Nishikawa El mié., 21 oct. 2020 a las 4:28, Nilothpal Pillai (< nilothpal.pil...@gmail.com>) escribió: > Hi, > > I'm unable to access https://www.goraexplorer.com. The link times out and > never loads. Can someone please help me out? Thank you. > > -- > Nilothpal Pillai > Chennai, Tamil Nadu, India >
Accessing GoraExplorer
Hi, I'm unable to access https://www.goraexplorer.com. The link times out and never loads. Can someone please help me out? Thank you. -- Nilothpal Pillai Chennai, Tamil Nadu, India
[jira] [Commented] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216496#comment-17216496 ] Alfonso Nishikawa commented on GORA-557: Hi, [~mariia.podorvanova]. Thank you for noticing it. I believe there was some misunderstanding in GORA-271. Let's put this on hold until we clarify it. > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Assignee: Mariia Podorvanova >Priority: Minor > Labels: gsoc2019 > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17216428#comment-17216428 ] Mariia Podorvanova commented on GORA-557: - Hi, Alfonso While working on the first task of the ticket (Allow to load mappings from properties instead only from files) I found out that the example at HBaseStore was changed. Looks like developers decided to not to support loading mappings from properties. So, the question is whether it is still necessary to implement loading mappings from properties in MongoStore? The ticket seems quite old, and the changes I described above were made after the ticket creation. > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Assignee: Mariia Podorvanova >Priority: Minor > Labels: gsoc2019 > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214566#comment-17214566 ] Alfonso Nishikawa commented on GORA-557: Hi, Maria! Great! Thank you very much and welcome! > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Assignee: Mariia Podorvanova >Priority: Minor > Labels: gsoc2019 > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alfonso Nishikawa reassigned GORA-557: -- Assignee: Mariia Podorvanova > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Assignee: Mariia Podorvanova >Priority: Minor > Labels: gsoc2019 > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17213598#comment-17213598 ] Mariia Podorvanova commented on GORA-557: - Hi I am Maria, an applicant for the Outreachy internship Dec 2020 program. I have started working on this issue. > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Priority: Minor > Labels: gsoc2019 > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (GORA-666) Add support for the Kudu datastore in GoraExplorer
John Mora created GORA-666: -- Summary: Add support for the Kudu datastore in GoraExplorer Key: GORA-666 URL: https://issues.apache.org/jira/browse/GORA-666 Project: Apache Gora Issue Type: New Feature Reporter: John Mora Add support for the Kudu datastore in GoraExplorer - https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (GORA-665) Add support for the Ignite datastore in GoraExplorer
John Mora created GORA-665: -- Summary: Add support for the Ignite datastore in GoraExplorer Key: GORA-665 URL: https://issues.apache.org/jira/browse/GORA-665 Project: Apache Gora Issue Type: New Feature Reporter: John Mora Add support for the Ignite datastore in GoraExplorer - https://www.goraexplorer.com/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Add GoraExplorer at Related Projects section
Hi! Done! Added GoraExplorer to the Related Projects :) Thank you!! Best Regards, Alfonso Nishikawa El jue., 5 sept. 2019 a las 15:30, carlos muñoz () escribió: > Hi, > > +1 > > Cheers, > Carlos > > El jue., 5 sept. 2019 a las 6:34, Cihad Guzel () > escribió: > > > +1 > > > > Cihad Güzel > > > > > > Furkan KAMACI , 5 Eyl 2019 Per, 07:58 tarihinde > > şunu yazdı: > > > > > Hi Alfonso, > > > > > > Sure! +1. > > > > > > Kind Regards, > > > Furkan KAMACI > > > > > > 5 Eyl 2019 Per, saat 07:48 tarihinde Kevin Ratnasekera < > > > djkevincr1...@gmail.com> şunu yazdı: > > > > > > > +1 > > > > > > > > On Thu, Sep 5, 2019 at 2:17 AM Alfonso Nishikawa < > > > > alfonso.nishik...@gmail.com> wrote: > > > > > > > > > Hi! > > > > > > > > > > I have been thiking about adding GoraExplorer [1] to the Related > > > Projects > > > > > section of the web, although it is in alpha stage. > > > > > Since I would be adding my own project, is everyone OK with that? > :) > > > > > > > > > > [1] https://www.goraexplorer.com/ > > > > > [2] http://gora.apache.org/related.html > > > > > > > > > > Thank you! > > > > > > > > > > Regards, > > > > > > > > > > Alfonso Nishikawa > > > > > > > > > > > > > > >
Re: Add GoraExplorer at Related Projects section
Hi, +1 Cheers, Carlos El jue., 5 sept. 2019 a las 6:34, Cihad Guzel () escribió: > +1 > > Cihad Güzel > > > Furkan KAMACI , 5 Eyl 2019 Per, 07:58 tarihinde > şunu yazdı: > > > Hi Alfonso, > > > > Sure! +1. > > > > Kind Regards, > > Furkan KAMACI > > > > 5 Eyl 2019 Per, saat 07:48 tarihinde Kevin Ratnasekera < > > djkevincr1...@gmail.com> şunu yazdı: > > > > > +1 > > > > > > On Thu, Sep 5, 2019 at 2:17 AM Alfonso Nishikawa < > > > alfonso.nishik...@gmail.com> wrote: > > > > > > > Hi! > > > > > > > > I have been thiking about adding GoraExplorer [1] to the Related > > Projects > > > > section of the web, although it is in alpha stage. > > > > Since I would be adding my own project, is everyone OK with that? :) > > > > > > > > [1] https://www.goraexplorer.com/ > > > > [2] http://gora.apache.org/related.html > > > > > > > > Thank you! > > > > > > > > Regards, > > > > > > > > Alfonso Nishikawa > > > > > > > > > >
Re: Add GoraExplorer at Related Projects section
Hi Alfonso, Sure! +1. Kind Regards, Furkan KAMACI 5 Eyl 2019 Per, saat 07:48 tarihinde Kevin Ratnasekera < djkevincr1...@gmail.com> şunu yazdı: > +1 > > On Thu, Sep 5, 2019 at 2:17 AM Alfonso Nishikawa < > alfonso.nishik...@gmail.com> wrote: > > > Hi! > > > > I have been thiking about adding GoraExplorer [1] to the Related Projects > > section of the web, although it is in alpha stage. > > Since I would be adding my own project, is everyone OK with that? :) > > > > [1] https://www.goraexplorer.com/ > > [2] http://gora.apache.org/related.html > > > > Thank you! > > > > Regards, > > > > Alfonso Nishikawa > > >
Re: Add GoraExplorer at Related Projects section
+1 On Thu, Sep 5, 2019 at 2:17 AM Alfonso Nishikawa < alfonso.nishik...@gmail.com> wrote: > Hi! > > I have been thiking about adding GoraExplorer [1] to the Related Projects > section of the web, although it is in alpha stage. > Since I would be adding my own project, is everyone OK with that? :) > > [1] https://www.goraexplorer.com/ > [2] http://gora.apache.org/related.html > > Thank you! > > Regards, > > Alfonso Nishikawa >
Add GoraExplorer at Related Projects section
Hi! I have been thiking about adding GoraExplorer [1] to the Related Projects section of the web, although it is in alpha stage. Since I would be adding my own project, is everyone OK with that? :) [1] https://www.goraexplorer.com/ [2] http://gora.apache.org/related.html Thank you! Regards, Alfonso Nishikawa
[jira] [Updated] (GORA-557) Add support for MongoDB to GoraExplorer
[ https://issues.apache.org/jira/browse/GORA-557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alfonso Nishikawa updated GORA-557: --- Issue Type: Improvement (was: New Feature) > Add support for MongoDB to GoraExplorer > --- > > Key: GORA-557 > URL: https://issues.apache.org/jira/browse/GORA-557 > Project: Apache Gora > Issue Type: Improvement > Components: gora-mongodb >Reporter: Alfonso Nishikawa >Priority: Minor > Labels: gsoc2019 > > After merging GoraExplorer interfaces and methods for the core, MongoDB > module has to be updated with the following: > - Allow to load mappings from properties instead only from files. [Example at > HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] > (where the static final constant much probably will have to be refactorized). > - Implement the retrieval of native information from MongoDB. ([Technical > details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) > A plus would be implement the support in > [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage > MongoDB data visually. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (GORA-558) Add support for Cassandra to GoraExplorer
Alfonso Nishikawa created GORA-558: -- Summary: Add support for Cassandra to GoraExplorer Key: GORA-558 URL: https://issues.apache.org/jira/browse/GORA-558 Project: Apache Gora Issue Type: Improvement Components: gora-cassandra Reporter: Alfonso Nishikawa After merging GoraExplorer interfaces and methods for the core, Cassandra module has to be updated with the following: - Allow to load mappings from properties instead only from files. [Example at HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] (where the static final constant much probably will have to be refactorized). - Implement the retrieval of native information from Cassandra. ([Technical details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) A plus would be implement the support in [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage Cassandra's data visually. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (GORA-557) Add support for MongoDB to GoraExplorer
Alfonso Nishikawa created GORA-557: -- Summary: Add support for MongoDB to GoraExplorer Key: GORA-557 URL: https://issues.apache.org/jira/browse/GORA-557 Project: Apache Gora Issue Type: New Feature Components: gora-mongodb Reporter: Alfonso Nishikawa After merging GoraExplorer interfaces and methods for the core, MongoDB module has to be updated with the following: - Allow to load mappings from properties instead only from files. [Example at HBaseStore|https://github.com/apache/gora/blob/6ce47dcdea60e4b0d3abe03ddac01a65716cc173/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java#L145] (where the static final constant much probably will have to be refactorized). - Implement the retrieval of native information from MongoDB. ([Technical details|https://translate.google.es/translate?sl=es=en=y=_t=es=UTF-8=http%3A%2F%2Fredmine.nishilua.com%2Fprojects%2Fgoraexplorer%2Fwiki%2FObtenci%25C3%25B3n_de_metadatos_de_las_BBDD_nativas=]) A plus would be implement the support in [GoraExplorer|https://www.goraexplorer.com] so it will be possible to manage MongoDB data visually. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [gora] djkevincr commented on issue #135: Goraexplorer needed changes
djkevincr commented on issue #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#issuecomment-474604466 @alfonsonishikawa Lets create a new ticket and address separately :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] alfonsonishikawa commented on issue #135: Goraexplorer needed changes
alfonsonishikawa commented on issue #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#issuecomment-474603882 Oh! Thank you for the pull! I was working on the Gora-Pig tests as @djkevincr suggested. I will continue working on them anyway but will open a Jira issue. Thank you both for the reviews! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr commented on issue #135: Goraexplorer needed changes
djkevincr commented on issue #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#issuecomment-474603912 @alfonsonishikawa I have merged you PR, thank you very much for your hard work on this :) I went through your design documents and all looked very impressive. It s really nice to see Apache Pig integrated to our project. Let's continue this work :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr merged pull request #135: Goraexplorer needed changes
djkevincr merged pull request #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] lewismc commented on issue #135: Goraexplorer needed changes
lewismc commented on issue #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#issuecomment-474597518 > Do you have any concern over your review on this PR? No This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes
alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#discussion_r266680620 ## File path: gora-pig/src/test/java/org/apache/gora/pig/GoraStorageTest.java-disabled ## @@ -0,0 +1,352 @@ +package org.apache.gora.pig; Review comment: > Were you able to try the same with HBase 2 upgrade? Hi! No. I will try, though. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr commented on issue #135: Goraexplorer needed changes
djkevincr commented on issue #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#issuecomment-474034153 @alfonsonishikawa One concern I do have is, I noticed record.vm velocity template changes, hopefully I think you have regenerated all the AVRO databean classes again to avoid any inconsistent updates, due to multiple updates to velocity template. This is really great :) as first step. We can continue this work, with the improvements you suggested to me offline. @lewismc Do you have any concern over your review on this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr commented on a change in pull request #135: Goraexplorer needed changes
djkevincr commented on a change in pull request #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#discussion_r266567600 ## File path: gora-pig/src/test/java/org/apache/gora/pig/GoraStorageTest.java-disabled ## @@ -0,0 +1,352 @@ +package org.apache.gora.pig; Review comment: Were you able to try the same with HBase 2 upgrade? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr commented on issue #135: Goraexplorer needed changes
djkevincr commented on issue #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#issuecomment-474029378 Locally tested the PR, build passes without any test failures. [INFO] [INFO] Reactor Summary: [INFO] [INFO] Apache Gora SUCCESS [ 2.381 s] [INFO] Apache Gora :: Compiler SUCCESS [ 3.077 s] [INFO] Apache Gora :: Compiler-CLI SUCCESS [ 1.385 s] [INFO] Apache Gora :: Core SUCCESS [01:56 min] [INFO] Apache Gora :: Pig . SUCCESS [ 3.235 s] [INFO] Apache Gora :: Accumulo SUCCESS [08:07 min] [INFO] Apache Gora :: HBase ... SUCCESS [03:24 min] [INFO] Apache Gora :: Cassandra - CQL . SUCCESS [01:53 min] [INFO] Apache Gora :: GoraCI .. SUCCESS [ 3.998 s] [INFO] Apache Gora :: Infinispan .. SUCCESS [01:22 min] [INFO] Apache Gora :: JCache .. SUCCESS [01:21 min] [INFO] Apache Gora :: OrientDB SUCCESS [01:48 min] [INFO] Apache Gora :: Dynamodb SUCCESS [ 4.441 s] [INFO] Apache Gora :: CouchDB . SUCCESS [ 4.872 s] [INFO] Apache Gora :: Maven Plugin SUCCESS [ 3.021 s] [INFO] Apache Gora :: MongoDB . SUCCESS [02:04 min] [INFO] Apache Gora :: Solr SUCCESS [02:59 min] [INFO] Apache Gora :: Aerospike ... SUCCESS [ 2.849 s] [INFO] Apache Gora :: Ignite .. SUCCESS [02:56 min] [INFO] Apache Gora :: Tutorial SUCCESS [ 6.486 s] [INFO] Apache Gora :: Sources-Dist SUCCESS [ 0.364 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 28:32 min [INFO] Finished at: 2019-03-18T23:14:41+05:30 [INFO] Final Memory: 101M/1679M [INFO] This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes
alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#discussion_r266256724 ## File path: gora-pig/src/test/examples/README.md ## @@ -0,0 +1,46 @@ + Review comment: The Gora-Pig example README.txt renamed to README.md and updated to include steps. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes
alfonsonishikawa commented on a change in pull request #135: Goraexplorer
needed changes
URL: https://github.com/apache/gora/pull/135#discussion_r266256105
##
File path: gora-pig/src/test/examples/example.pig
##
@@ -0,0 +1,4 @@
+set job.name 'GoraStorage test' ;
+register gora/*.jar ;
+webpage = LOAD '.' using
org.apache.gora.pig.GoraStorage('java.lang.String','admin.WebPage','baseUrl,status,content')
;
+dump webpage;
Review comment:
Deleted `webpage.jar` and updated README.md to describe the steps to execute
an example.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes
alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#discussion_r266256014 ## File path: gora-core/src/examples/java/org/apache/gora/examples/generated/package-info.java ## @@ -1,20 +0,0 @@ -/** Review comment: Now the package-info.java files are moved to the parent directory temporarily at `compile-examples.sh` before deleting the generated classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes
alfonsonishikawa commented on a change in pull request #135: Goraexplorer
needed changes
URL: https://github.com/apache/gora/pull/135#discussion_r266255944
##
File path: conf/log4j.properties
##
@@ -24,7 +24,7 @@ log4j.threshhold=ALL
log4j.rootLogger=${gora.root.logger}
#special logging requirements for some commandline tools
-log4j.logger.org.apache.gora.compiler.GoraCompiler=INFO,console
+log4j.logger.org.apache.gora.compiler.GoraCompiler=INFO
Review comment:
Logs in console about compiled files appeared twice because the root,logger
is already configured to log to console.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes
alfonsonishikawa commented on a change in pull request #135: Goraexplorer
needed changes
URL: https://github.com/apache/gora/pull/135#discussion_r266249623
##
File path: gora-core/src/main/java/org/apache/gora/store/DataStoreFactory.java
##
@@ -412,7 +484,7 @@ public static String getMappingFile(Properties properties,
DataStore store
return mappingFilename;
}
- private static String getDefaultDataStore(Properties properties) {
+ public static String getDefaultDataStore(Properties properties) {
Review comment:
Sadly the MetadataAnalyzer (to retrieve native information from the backend)
depends on the name of the DataStore.
I acknowledge I went into a rush about implementing the feature, so any
suggestions about are welcomed :)
I see you found out why by yourself!
(https://github.com/apache/gora/pull/135/files/2974126a934bddc526830182b402767c0f1eefd0?file-filters%5B%5D=.java=true#diff-d9701987f6844457a897d30a5786e5a4)
:D
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes
alfonsonishikawa commented on a change in pull request #135: Goraexplorer
needed changes
URL: https://github.com/apache/gora/pull/135#discussion_r222506186
##
File path:
gora-compiler/src/main/velocity/org/apache/gora/compiler/templates/record.vm
##
@@ -180,6 +180,11 @@ public class ${this.mangle($schema.getName())}#if
($schema.isError()) extends or
return new #if
($schema.getNamespace())$schema.getNamespace().#end${this.mangle($schema.getName())}.Builder(other);
}
+ @Override
Review comment:
A clone method. I used it
[here](https://bitbucket.org/alfonsonishikawa/goraexplorer/src/12f226408e7f9686aa6b5975692edf270b94182e/src/main/java/com/nishilua/goraexplorer/tables/TableEntityServiceImpl.java#lines-121)
because `get()` reuses the instance.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [gora] alfonsonishikawa commented on issue #135: Goraexplorer needed changes
alfonsonishikawa commented on issue #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#issuecomment-472663392 Hi! I am updating this pull request with the fixes from your comments. Will finish it this week. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes
alfonsonishikawa commented on a change in pull request #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#discussion_r265370854 ## File path: gora-core/src/examples/java/org/apache/gora/examples/generated/package-info.java ## @@ -1,20 +0,0 @@ -/** Review comment: It seems that compile-examples.sh just deletes generated/* to regenerate the classes. I will update bin/compile-examples.sh to just delete .java files This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr edited a comment on issue #135: Goraexplorer needed changes
djkevincr edited a comment on issue #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#issuecomment-472522370 @alfonsonishikawa Are you done with updating PR as per the reviews? Is there anything else that we should address or Do you need further time on this? Its awesome if we can get this to the release coming shortly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr edited a comment on issue #135: Goraexplorer needed changes
djkevincr edited a comment on issue #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#issuecomment-472522370 @alfonsonishikawa Are you done with updating PR as per the reviews? Is there anything else that we should address or do you need further time on this? Its awesome if we can get to the release coming shortly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [gora] djkevincr commented on issue #135: Goraexplorer needed changes
djkevincr commented on issue #135: Goraexplorer needed changes URL: https://github.com/apache/gora/pull/135#issuecomment-472522370 @alfonsonishikawa Are you done with updating PR as per the reviews? Is there anything else that we should address or do you need further time on this? Its awesome if we can get to the release coming shortly? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user alfonsonishikawa commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r225001067 --- Diff: pom.xml --- @@ -832,6 +833,12 @@ 1.0.1 2.5 + +0.16.0 +1.11-8 --- End diff -- Was for testing purposes. Deleted. ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user alfonsonishikawa commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r225000854
--- Diff: gora-pig/src/test/examples/example.pig ---
@@ -0,0 +1,4 @@
+set job.name 'GoraStorage test' ;
+register gora/*.jar ;
+webpage = LOAD '.' using
org.apache.gora.pig.GoraStorage('java.lang.String','admin.WebPage','baseUrl,status,content')
;
+dump webpage;
--- End diff --
This is actually an usage example. It could be completely deleted and
commented on the README.md how to create the file. Or even delete the folder
completely.
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user alfonsonishikawa commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r225000361
--- Diff:
gora-pig/src/main/java/org/apache/gora/pig/mapreduce/PigGoraOutputFormat.java
---
@@ -0,0 +1,86 @@
+package org.apache.gora.pig.mapreduce;
+
+import java.io.IOException;
+
+import org.apache.gora.mapreduce.GoraRecordWriter;
+import org.apache.gora.mapreduce.NullOutputCommitter;
+import org.apache.gora.persistency.Persistent;
+import org.apache.gora.pig.StorageConfiguration;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.hadoop.conf.Configurable;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapreduce.JobContext;
+import org.apache.hadoop.mapreduce.OutputCommitter;
+import org.apache.hadoop.mapreduce.OutputFormat;
+import org.apache.hadoop.mapreduce.RecordWriter;
+import org.apache.hadoop.mapreduce.TaskAttemptContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class PigGoraOutputFormat extends
OutputFormat implements Configurable {
+
+ public static final Logger LOG =
LoggerFactory.getLogger(PigGoraOutputFormat.class);
+
+ protected Configuration conf ;
+
+ /**
+ * The GoraStorage configuration setted at constructor (converted from
json to bean)
+ */
+ protected StorageConfiguration storageConfiguration ;
+
+ protected DataStore dataStore ;
+
+ @Override
+ @SuppressWarnings({ "unchecked", "rawtypes" })
+ public RecordWriter getRecordWriter(TaskAttemptContext context)
+ throws IOException, InterruptedException {
+
+if ( this.dataStore == null ) {
+ this.dataStore = DataStoreFactory.getDataStore(
+ this.storageConfiguration.getKeyClass(),
+ this.storageConfiguration.getPersistentClass(),
+ this.storageConfiguration.getGoraPropertiesAsProperties(),
+ this.conf
+ ) ;
+}
+
+return new GoraRecordWriter(this.dataStore, context);
+ }
+
+
+ @Override
+ public void checkOutputSpecs(JobContext context) throws IOException,
InterruptedException {
+// Nothing
--- End diff --
Implementing an mandatory abstract method. We just don't check and take it
as free of errors. The output will fail always if an
IOException/InterrupedException is thrown.
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user alfonsonishikawa commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r225000105
--- Diff: gora-pig/pom.xml ---
@@ -0,0 +1,180 @@
+
+http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;
+xmlns="http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;>
+4.0.0
+
+
+org.apache.gora
+gora
+0.9-SNAPSHOT
+../
+
+
+gora-pig
+bundle
+
+Apache Gora :: Pig
+http://gora.apache.org
+Pig -> Gora -> DataStore connector.
+2018
+
+The Apache Software Foundation
+http://www.apache.org/
+
+
+
+*
+
org.apache.gora.pig*;version="${project.version}";-noimport:=true
+
+
+
+target
+target/classes
+${project.artifactId}-${project.version}
+target/test-classes
+src/test/java
+src/main/java
+
+
+
+org.apache.maven.plugins
+maven-jar-plugin
+
+
+
+test-jar
+
+
+
+
+
+
+
+
+
+org.slf4j
+jcl-over-slf4j
+
+
+
+
+org.apache.gora
+gora-core
+compile
+
+
+
--- End diff --
They are commented because they were the ones used for -the not working-
JUnit tests. Anyway, they will get rotten, so you are right :) I will delete
them.
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user alfonsonishikawa commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r224999854 --- Diff: gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java --- @@ -80,15 +81,20 @@ * DataStore for HBase. Thread safe. * */ -public class HBaseStore extends DataStoreBase --- End diff -- Yes, it seems it was only for HBaseStore, from the initial import of the source code 8 years ago. ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user alfonsonishikawa commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r224998986
--- Diff:
gora-compiler/src/main/velocity/org/apache/gora/compiler/templates/record.vm ---
@@ -180,6 +180,11 @@ public class ${this.mangle($schema.getName())}#if
($schema.isError()) extends or
return new #if
($schema.getNamespace())$schema.getNamespace().#end${this.mangle($schema.getName())}.Builder(other);
}
+ @Override
--- End diff --
Oh, the line to reference is the 184, just the code hidden after.
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r222543455 --- Diff: gora-pig/src/test/examples/README.txt --- @@ -0,0 +1,9 @@ + --- End diff -- Can you make this an .md file? ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r222542930 --- Diff: gora-pig/src/main/java/org/apache/gora/pig/ResourceFieldSchemaWithIndex.java --- @@ -0,0 +1,31 @@ +package org.apache.gora.pig; --- End diff -- License header. ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r222540948 --- Diff: gora-compiler/src/main/java/org/apache/gora/compiler/GoraCompiler.java --- @@ -86,6 +88,31 @@ public static void compileSchema(File[] srcFiles, File dest, LicenseHeaders lice } } + /** --- End diff -- I like this ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r222541609
--- Diff:
gora-core/src/main/java/org/apache/gora/store/DataStoreFactory.java ---
@@ -251,6 +254,70 @@ D getDataStore( Class dataStoreClass, Class
keyClass,
}
}
+ /**
+ * Instantiate a new {@link DataStore}. Uses default properties. Uses
'null' schema.
+ *
+ * @param dataStoreClass The datastore implementation class as
string.
--- End diff --
Not required this is not a parameter
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r222542624 --- Diff: gora-pig/pom.xml --- @@ -0,0 +1,180 @@ + --- End diff -- License header. ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r222540708 --- Diff: bin/compile-examples.sh --- @@ -36,8 +36,57 @@ DIR=$GORA_HOME/$MODULE/src/examples/avro/ OUTDIR=$GORA_HOME/$MODULE/src/examples/java GORA_BIN=$GORA_HOME/bin/gora +rm -r $OUTDIR/org/apache/gora/examples/generated/* --- End diff -- Yep OK ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r222543677 --- Diff: pom.xml --- @@ -757,9 +758,9 @@ gora-maven-plugin gora-mongodb gora-solr +gora-aerospike --- End diff -- Good catch. ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r222542669
--- Diff: gora-pig/pom.xml ---
@@ -0,0 +1,180 @@
+
+http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;
+xmlns="http://maven.apache.org/POM/4.0.0;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;>
+4.0.0
+
+
+org.apache.gora
+gora
+0.9-SNAPSHOT
+../
+
+
+gora-pig
+bundle
+
+Apache Gora :: Pig
+http://gora.apache.org
+Pig -> Gora -> DataStore connector.
+2018
+
+The Apache Software Foundation
+http://www.apache.org/
+
+
+
+*
+
org.apache.gora.pig*;version="${project.version}";-noimport:=true
+
+
+
+target
+target/classes
+${project.artifactId}-${project.version}
+target/test-classes
+src/test/java
+src/main/java
+
+
+
+org.apache.maven.plugins
+maven-jar-plugin
+
+
+
+test-jar
+
+
+
+
+
+
+
+
+
+org.slf4j
+jcl-over-slf4j
+
+
+
+
+org.apache.gora
+gora-core
+compile
+
+
+
--- End diff --
Just remove commented out dependencies.
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r222541705
--- Diff:
gora-core/src/main/java/org/apache/gora/store/DataStoreFactory.java ---
@@ -412,7 +484,7 @@ public static String getMappingFile(Properties
properties, DataStore store
return mappingFilename;
}
- private static String getDefaultDataStore(Properties properties) {
+ public static String getDefaultDataStore(Properties properties) {
--- End diff --
Why the change here?
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r222541077
--- Diff:
gora-compiler/src/main/velocity/org/apache/gora/compiler/templates/record.vm ---
@@ -180,6 +180,11 @@ public class ${this.mangle($schema.getName())}#if
($schema.isError()) extends or
return new #if
($schema.getNamespace())$schema.getNamespace().#end${this.mangle($schema.getName())}.Builder(other);
}
+ @Override
--- End diff --
You should have referenced line 121 not 118.
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r222543835
--- Diff: pom.xml ---
@@ -1546,6 +1553,40 @@
${orientqb.version}
+
+
+ org.apache.pig
+ pig
+ h2
+ ${pig.version}
+
+
+ org.apache.avro
+ avro
+
+
+
+
+
+ dk.brics.automaton
+ automaton
+ ${automaton.version}
+ test
+
+
+
+ org.antlr
+ antlr-runtime
+ ${antlr.version}
+ test
+
+
+
+ com.linkedin.datafu
--- End diff --
Can you not use org.apache.datafu artifact?
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r222543728 --- Diff: pom.xml --- @@ -832,6 +833,12 @@ 1.0.1 2.5 + +0.16.0 +1.11-8 --- End diff -- Is this required? What do you use it for? ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r222542415
--- Diff:
gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java ---
@@ -734,14 +756,13 @@ private void setField(T persistent, Field field, List
list) {
}
@SuppressWarnings("unchecked")
- private HBaseMapping readMapping(String filename) throws IOException {
+ private HBaseMapping readMapping(InputStream mappingStream) throws
IOException {
--- End diff --
I like this. The quicker work with ByteStream the better.
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r222543239
--- Diff:
gora-pig/src/main/java/org/apache/gora/pig/mapreduce/PigGoraOutputFormat.java
---
@@ -0,0 +1,86 @@
+package org.apache.gora.pig.mapreduce;
+
+import java.io.IOException;
+
+import org.apache.gora.mapreduce.GoraRecordWriter;
+import org.apache.gora.mapreduce.NullOutputCommitter;
+import org.apache.gora.persistency.Persistent;
+import org.apache.gora.pig.StorageConfiguration;
+import org.apache.gora.store.DataStore;
+import org.apache.gora.store.DataStoreFactory;
+import org.apache.hadoop.conf.Configurable;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapreduce.JobContext;
+import org.apache.hadoop.mapreduce.OutputCommitter;
+import org.apache.hadoop.mapreduce.OutputFormat;
+import org.apache.hadoop.mapreduce.RecordWriter;
+import org.apache.hadoop.mapreduce.TaskAttemptContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class PigGoraOutputFormat extends
OutputFormat implements Configurable {
+
+ public static final Logger LOG =
LoggerFactory.getLogger(PigGoraOutputFormat.class);
+
+ protected Configuration conf ;
+
+ /**
+ * The GoraStorage configuration setted at constructor (converted from
json to bean)
+ */
+ protected StorageConfiguration storageConfiguration ;
+
+ protected DataStore dataStore ;
+
+ @Override
+ @SuppressWarnings({ "unchecked", "rawtypes" })
+ public RecordWriter getRecordWriter(TaskAttemptContext context)
+ throws IOException, InterruptedException {
+
+if ( this.dataStore == null ) {
+ this.dataStore = DataStoreFactory.getDataStore(
+ this.storageConfiguration.getKeyClass(),
+ this.storageConfiguration.getPersistentClass(),
+ this.storageConfiguration.getGoraPropertiesAsProperties(),
+ this.conf
+ ) ;
+}
+
+return new GoraRecordWriter(this.dataStore, context);
+ }
+
+
+ @Override
+ public void checkOutputSpecs(JobContext context) throws IOException,
InterruptedException {
+// Nothing
--- End diff --
throw unsupported exception?
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r222543267 --- Diff: gora-pig/src/main/java/org/apache/gora/pig/udf/MapMerge.java --- @@ -0,0 +1,33 @@ +package org.apache.gora.pig.udf; --- End diff -- License header. ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request:
https://github.com/apache/gora/pull/135#discussion_r222541967
--- Diff:
gora-core/src/main/java/org/apache/gora/store/DataStoreMetadataFactory.java ---
@@ -0,0 +1,93 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.gora.store;
+
+import java.util.Properties;
+
+import org.apache.gora.store.impl.DataStoreMetadataAnalyzer;
+import org.apache.gora.util.GoraException;
+import org.apache.gora.util.ReflectionUtils;
+import org.apache.hadoop.conf.Configuration;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Factory to create Metadata Analyzers for the backends.
+ * It uses the properties from gora.properties, but can be configured with
other properties.
+ *
+ * The returned Metadata Analyzer defaults to *StoreMetadataAnalyzer, but
can be configured. By default detects the backend storage
+ * taking it from the Gora Properties (which defaults to gora.properties).
+ */
+public class DataStoreMetadataFactory {
+
+ public static final Logger log =
LoggerFactory.getLogger(DataStoreMetadataFactory.class);
+
+ private DataStoreMetadataFactory() {
+ }
+
+ /**
+ * Creates a metadata analyzer given the Hadoop Configuration instance.
Uses the default gora.properties where it takes the
+ * base name from the default store to infer the metadata analyzer to
create.
+ * @throws GoraException
+ * @throws ClassNotFoundException - Exception thrown when does not
exists a suitable metadata analyzer class
+ */
+ public static DataStoreMetadataAnalyzer createAnalyzer(Configuration
configuration) throws GoraException, ClassNotFoundException {
+return createAnalyzer(configuration, DataStoreFactory.createProps());
+ }
+
+ /**
+ * Creates a metadata analyzer given the Hadoop Configuration instance
and the gora properties to use. It uses this properties
+ * to infer the metadata analyzer to create.
+ * @throws GoraException
+ * @throws ClassNotFoundException - Exception thrown when does not
exists a suitable metadata analyzer class
+ */
+ public static DataStoreMetadataAnalyzer createAnalyzer(Configuration
configuration, Properties properties) throws GoraException,
ClassNotFoundException {
+String metadataAnalyzerClassName =
DataStoreFactory.getDefaultDataStore(properties) + "MetadataAnalyzer";
--- End diff --
Aha, I now see why you want this to be made public.
---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r222542504 --- Diff: gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseTableMetadata.java --- @@ -0,0 +1,27 @@ +package org.apache.gora.hbase.store; --- End diff -- License header please. ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r222541246 --- Diff: gora-core/src/examples/java/org/apache/gora/examples/generated/package-info.java --- @@ -1,20 +0,0 @@ -/** --- End diff -- Why remove? ---
[GitHub] gora pull request #135: Goraexplorer needed changes
Github user lewismc commented on a diff in the pull request: https://github.com/apache/gora/pull/135#discussion_r222542987 --- Diff: gora-pig/src/main/java/org/apache/gora/pig/StorageConfiguration.java --- @@ -0,0 +1,140 @@ +package org.apache.gora.pig; --- End diff -- License header. ---
