[
https://issues.apache.org/jira/browse/BEAM-14081?focusedWorklogId=753160&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753160
]
ASF GitHub Bot logged work on BEAM-14081:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Apr/22 00:39
Start Date: 06/Apr/22 00:39
Worklog Time Spent: 10m
Work Description: chamikaramj commented on code in PR #17104:
URL: https://github.com/apache/beam/pull/17104#discussion_r843340624
##########
sdks/java/io/cdap/build.gradle:
##########
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+plugins {
+ id 'java'
+ id 'org.apache.beam.module'
+}
+
+applyJavaNature(
+ exportJavadoc: false,
+ automaticModuleName: 'org.apache.beam.sdk.io.cdap',
+)
+provideIntegrationTestingDependencies()
+enableJavaPerformanceTesting()
+
+description = "Apache Beam :: CDAP :: Java"
+ext.summary = """Apache Beam SDK provides a simple, Java-based
+interface for integration with CDAP plugins."""
+
+/** Define the list of runners which execute a precommit test.
+ * Some runners are run from separate projects, see the preCommit task below
+ * for details.
+ */
+
+allprojects {
+ repositories {
+ maven { url 'https://jitpack.io' }
+ }
+}
+
+dependencies {
+ implementation project(path: ":sdks:java:core", configuration: "shadow")
Review Comment:
Same blockers as https://github.com/apache/beam/pull/17051 should apply here.
##########
sdks/java/io/cdap/src/main/java/org/apache/beam/sdk/io/cdap/PluginConfigInstantiationUtils.java:
##########
@@ -0,0 +1,134 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.cdap;
+
+import io.cdap.cdap.api.annotation.Name;
+import io.cdap.cdap.api.plugin.PluginConfig;
+import java.lang.reflect.Constructor;
+import java.lang.reflect.Field;
+import java.lang.reflect.InvocationTargetException;
+import java.lang.reflect.Modifier;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import javax.annotation.Nullable;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Class for getting any filled {@link io.cdap.cdap.api.plugin.PluginConfig}
configuration object.
+ */
+@SuppressWarnings({"unchecked", "assignment.type.incompatible"})
+public class PluginConfigInstantiationUtils {
Review Comment:
Seems like this just duplicates
https://github.com/apache/beam/pull/17051/files so not reviewing here.
##########
sdks/java/io/cdap/src/main/java/org/apache/beam/sdk/io/cdap/context/BatchContextImpl.java:
##########
@@ -0,0 +1,220 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.cdap.context;
+
+import io.cdap.cdap.api.data.DatasetInstantiationException;
+import io.cdap.cdap.api.data.schema.Schema;
+import io.cdap.cdap.api.dataset.Dataset;
+import io.cdap.cdap.api.dataset.DatasetManagementException;
+import io.cdap.cdap.api.dataset.DatasetProperties;
+import io.cdap.cdap.api.metadata.Metadata;
+import io.cdap.cdap.api.metadata.MetadataEntity;
+import io.cdap.cdap.api.metadata.MetadataException;
+import io.cdap.cdap.api.metadata.MetadataScope;
+import io.cdap.cdap.api.plugin.PluginProperties;
+import io.cdap.cdap.etl.api.FailureCollector;
+import io.cdap.cdap.etl.api.Lookup;
+import io.cdap.cdap.etl.api.StageMetrics;
+import io.cdap.cdap.etl.api.action.SettableArguments;
+import io.cdap.cdap.etl.api.batch.BatchContext;
+import io.cdap.cdap.etl.api.lineage.field.FieldOperation;
+import java.net.URL;
+import java.sql.Timestamp;
+import java.util.List;
+import java.util.Map;
+import javax.annotation.Nullable;
+
+/**
+ * Class OperationContext is a common class for Batch, Sink and Stream CDAP
wrapper classes that use
+ * it to provide common details.
+ */
+@SuppressWarnings({"TypeParameterUnusedInFormals", "nullness"})
+public class BatchContextImpl implements BatchContext {
+
+ private final FailureCollectorWrapper failureCollector = new
FailureCollectorWrapper();
+
+ private final Timestamp startTime = new
Timestamp(System.currentTimeMillis());
+
+ @Override
+ public String getStageName() {
+ return null;
Review Comment:
Should these methods be implemented ? What does the default implementation
that mostly return null provide ? If this class is not usable as is can we make
it an abstract class ?
##########
sdks/java/io/cdap/src/main/java/org/apache/beam/sdk/io/cdap/ConfigWrapper.java:
##########
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.sdk.io.cdap;
+
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import io.cdap.cdap.api.plugin.PluginConfig;
+import java.io.File;
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.Map;
+import javax.annotation.Nonnull;
+import javax.annotation.Nullable;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/** Class for building {@link PluginConfig} object of the specific class
{@param <T>}. */
+public class ConfigWrapper<T extends PluginConfig> {
Review Comment:
Seems like this just duplicates
https://github.com/apache/beam/pull/17051/files so not reviewing here.
Issue Time Tracking
-------------------
Worklog Id: (was: 753160)
Time Spent: 6.5h (was: 6h 20m)
> [CdapIO] Design and implement Context wrapper class
> ---------------------------------------------------
>
> Key: BEAM-14081
> URL: https://issues.apache.org/jira/browse/BEAM-14081
> Project: Beam
> Issue Type: Task
> Components: io-java-cdap
> Reporter: Elizaveta Lomteva
> Assignee: Igor Krasavin
> Priority: P2
> Labels: cdap-io-sprint-2
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> h3. Context:
> CDAP plugins use context classes (ex.
> {{{}io.cdap.cdap.etl.api.batch.BatchSourceContext{}}}) to provide the updates
> and support additional logic (ex.
> [FailureCollector|https://github.com/data-integrations/salesforce/blob/d0d83fd82bbdbbb7ef07168f3b1698eebdc90c3e/src/main/java/io/cdap/plugin/salesforce/plugin/source/batch/SalesforceBatchSource.java#L96]
> or [getting
> timestamp|https://github.com/data-integrations/salesforce/blob/d0d83fd82bbdbbb7ef07168f3b1698eebdc90c3e/src/main/java/io/cdap/plugin/salesforce/plugin/source/batch/SalesforceBatchSource.java#L111])
> in Source or Sink class methods (ex. {{{}SalesforceBatchSink{}}}).
> h3. Task Description:
> Required to design wrapper class(es) (it could be an heir of the original
> class) for the CDAP context classes so that the CDAP context instance can be
> created in the Apache Beam connector and used when calling the Source/Sink
> methods that require the context as an argument (ex.
> {{prepareRun(BatchSinkContext context)}}
> [method|https://github.com/data-integrations/salesforce/blob/d0d83fd82bbdbbb7ef07168f3b1698eebdc90c3e/src/main/java/io/cdap/plugin/salesforce/plugin/sink/batch/SalesforceBatchSink.java#L67]).
> h3. Acceptance criteria:
> Class diagram and source code of CDAP context logic wrapper class(es) that
> will provide an original context class logic
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
