[GitHub] [samza] mynameborat commented on a change in pull request #938: SAMZA-1531: Support run.id in standalone for batch processing.

[GitBox]

mynameborat commented on a change in pull request #938: SAMZA-1531: Support 
run.id in standalone for batch processing.
URL: https://github.com/apache/samza/pull/938#discussion_r262814180
 
 

 ##
 File path: 
samza-core/src/main/java/org/apache/samza/runtime/LocalApplicationRunner.java
 ##
 @@ -280,4 +368,47 @@ private void setApplicationFinalStatus() {
   }
 }
   }
+
+
+  /**
+   * Defines a specific implementation of {@link CoordinationSessionListener} 
for local {@link CoordinationUtils}
+   */
+  private final class LocalCoordinationSessionListener implements 
CoordinationSessionListener {
+
+/**
+ * If the coordination utils session has reconnected, check if global 
runid differs from local runid
+ * if it differs then shut down processor and throw exception
+ * else recreate ephemeral node corresponding to this processor inside the 
read write lock for runid
+ */
+@Override
+public void handleReconnect() {
+  LOG.info("Reconnected to coordination utils");
+  if(coordinationUtils == null) {
+return;
+  }
+  DistributedDataAccess runIdAccess = coordinationUtils.getDataAccess();
+  String globalRunId = (String) runIdAccess.readData(RUNID_PATH);
+  if( runId != globalRunId){
+processors.forEach(StreamProcessor::stop);
+cleanup();
+appStatus = ApplicationStatus.UnsuccessfulFinish;
+String msg = String.format("run.id %s on processor %s differs from the 
global run.id %s", runId, uid, globalRunId);
+throw new SamzaException(msg);
+  } else if(runIdLock != null) {
+String msg = String.format("Processor {} failed to get the lock for 
run.id", uid);
+try {
+  // acquire lock to recreate active processor ephemeral node
+  DistributedReadWriteLock.AccessType lockAccess = 
runIdLock.lock(LOCK_TIMEOUT, LOCK_TIMEOUT_UNIT);
+  if(lockAccess == DistributedReadWriteLock.AccessType.WRITE || 
lockAccess == DistributedReadWriteLock.AccessType.READ) {
+LOG.info("Processor {} creates its active processor ephemeral node 
in read write lock again" + uid);
+runIdLock.unlock();
+  } else {
+throw new SamzaException(msg);
+  }
+} catch (TimeoutException e) {
+  throw new SamzaException(msg, e);
 
 Review comment:
   Refer to my previous comment about throwing the exception. If the intent is 
to stop the LAR, we need update the appStatus with failure cause.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] mynameborat commented on a change in pull request #938: SAMZA-1531: Support run.id in standalone for batch processing.

[GitBox]

mynameborat commented on a change in pull request #938: SAMZA-1531: Support 
run.id in standalone for batch processing.
URL: https://github.com/apache/samza/pull/938#discussion_r262817813
 
 

 ##
 File path: 
samza-core/src/main/java/org/apache/samza/coordinator/DistributedReadWriteLock.java
 ##
 @@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.samza.coordinator;
+
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+
+/**
+ * Lock to acquire read access or write access
+ * At any point in time, only one processor can hold the lock
+ * The processor that has acquired the lock, holds it until it does `unlock` 
explicitly
+ * The first processor to acquire the lock should be given a WRITE access
+ * All other subsequent requests should acquire only READ access
+ */
+public interface DistributedReadWriteLock {
+
+  /**
+   * Try to acquire lock: returns the type of access lock was acquired for
+   * @param timeout Duration of lock acquiring timeout.
+   * @param unit Time Unit of the timeout defined above.
+   * @return AccessType.READ if lock is acquired for read, AcessType.WRITE if 
acquired for write and AccessType.NONE if not acquired
+   * @throws TimeoutException if could not acquire the lock.
+   */
+  AccessType lock(long timeout, TimeUnit unit) throws TimeoutException;
+
+  /**
+   * Release the lock
+   */
+  void unlock();
+
+  /**
+   * Clean state of the lock
+   */
+  void cleanState();
 
 Review comment:
   Can you elaborate on what the state and what the expectations from framework?
   Another useful thing to document would be about the lifecycle of the lock 
and where `cleanState` falls in it.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] mynameborat commented on a change in pull request #938: SAMZA-1531: Support run.id in standalone for batch processing.

[GitBox]

mynameborat commented on a change in pull request #938: SAMZA-1531: Support 
run.id in standalone for batch processing.
URL: https://github.com/apache/samza/pull/938#discussion_r262814871
 
 

 ##
 File path: 
samza-core/src/main/java/org/apache/samza/runtime/LocalApplicationRunner.java
 ##
 @@ -280,4 +368,47 @@ private void setApplicationFinalStatus() {
   }
 }
   }
+
+
+  /**
+   * Defines a specific implementation of {@link CoordinationSessionListener} 
for local {@link CoordinationUtils}
+   */
+  private final class LocalCoordinationSessionListener implements 
CoordinationSessionListener {
+
+/**
+ * If the coordination utils session has reconnected, check if global 
runid differs from local runid
+ * if it differs then shut down processor and throw exception
+ * else recreate ephemeral node corresponding to this processor inside the 
read write lock for runid
+ */
+@Override
+public void handleReconnect() {
+  LOG.info("Reconnected to coordination utils");
+  if(coordinationUtils == null) {
+return;
+  }
+  DistributedDataAccess runIdAccess = coordinationUtils.getDataAccess();
+  String globalRunId = (String) runIdAccess.readData(RUNID_PATH);
+  if( runId != globalRunId){
+processors.forEach(StreamProcessor::stop);
+cleanup();
+appStatus = ApplicationStatus.UnsuccessfulFinish;
+String msg = String.format("run.id %s on processor %s differs from the 
global run.id %s", runId, uid, globalRunId);
+throw new SamzaException(msg);
+  } else if(runIdLock != null) {
+String msg = String.format("Processor {} failed to get the lock for 
run.id", uid);
+try {
+  // acquire lock to recreate active processor ephemeral node
+  DistributedReadWriteLock.AccessType lockAccess = 
runIdLock.lock(LOCK_TIMEOUT, LOCK_TIMEOUT_UNIT);
+  if(lockAccess == DistributedReadWriteLock.AccessType.WRITE || 
lockAccess == DistributedReadWriteLock.AccessType.READ) {
 
 Review comment:
   We would need to differentiate between session expiration vs reconnect 
scenario. In the former, it is guaranteed that the node is cleaned up in the 
majority of the quorum, vs the latter where the node still exists and you don't 
need to do anything since the session reconnect reuses the old session id.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] mynameborat commented on a change in pull request #938: SAMZA-1531: Support run.id in standalone for batch processing.

[GitBox]

mynameborat commented on a change in pull request #938: SAMZA-1531: Support 
run.id in standalone for batch processing.
URL: https://github.com/apache/samza/pull/938#discussion_r262816170
 
 

 ##
 File path: 
samza-core/src/main/java/org/apache/samza/coordinator/CoordinationUtils.java
 ##
 @@ -28,6 +28,9 @@
  *   - LeaderElection
  *   - Latch
  *   - LockWithState (does not lock if state is set)
+ *   - ReadWriteLock (lock to acquire read or write access)
+ *   - DataAccess (to read or write data)
+ *   - setCoordinationSessionListener to listen to coordination utils session 
state changes
  */
 @InterfaceStability.Evolving
 public interface CoordinationUtils {
 
 Review comment:
   Can we add java documentation to the new APIs and provide contracts and 
framework's expectations from the implementers? 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] mynameborat commented on a change in pull request #938: SAMZA-1531: Support run.id in standalone for batch processing.

[GitBox]

mynameborat commented on a change in pull request #938: SAMZA-1531: Support 
run.id in standalone for batch processing.
URL: https://github.com/apache/samza/pull/938#discussion_r262815650
 
 

 ##
 File path: 
samza-core/src/main/java/org/apache/samza/coordinator/CoordinationUtils.java
 ##
 @@ -39,6 +42,12 @@
 
   DistributedLockWithState getLockWithState(String lockId);
 
+  DistributedReadWriteLock getReadWriteLock();
+
+  DistributedDataAccess getDataAccess();
+
+  void setCoordinationSessionListener(CoordinationSessionListener 
sessionListener);
 
 Review comment:
   This is listener seems very zookeeper specific. Is it possible to push the 
logic down to implementer and not have this part of coordination utils?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] mynameborat commented on a change in pull request #938: SAMZA-1531: Support run.id in standalone for batch processing.

[GitBox]

mynameborat commented on a change in pull request #938: SAMZA-1531: Support 
run.id in standalone for batch processing.
URL: https://github.com/apache/samza/pull/938#discussion_r262815895
 
 

 ##
 File path: 
samza-core/src/main/java/org/apache/samza/coordinator/CoordinationUtils.java
 ##
 @@ -39,6 +42,12 @@
 
   DistributedLockWithState getLockWithState(String lockId);
 
+  DistributedReadWriteLock getReadWriteLock();
 
 Review comment:
   Why does this not take a lock id? Are we imposing that you can only have one 
read write lock per coordination utils instance?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] mynameborat commented on a change in pull request #938: SAMZA-1531: Support run.id in standalone for batch processing.

[GitBox]

mynameborat commented on a change in pull request #938: SAMZA-1531: Support 
run.id in standalone for batch processing.
URL: https://github.com/apache/samza/pull/938#discussion_r262817152
 
 

 ##
 File path: 
samza-core/src/main/java/org/apache/samza/coordinator/DistributedReadWriteLock.java
 ##
 @@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.samza.coordinator;
+
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+
+/**
+ * Lock to acquire read access or write access
+ * At any point in time, only one processor can hold the lock
+ * The processor that has acquired the lock, holds it until it does `unlock` 
explicitly
+ * The first processor to acquire the lock should be given a WRITE access
+ * All other subsequent requests should acquire only READ access
+ */
+public interface DistributedReadWriteLock {
 
 Review comment:
   Can we expect a predicate that decides who gets the read lock vs write? By 
this, you can reuse the read write locks for different scenarios in future as 
opposed to having it only purposed for handing the write access to first 
processor that acquires the lock and read access to the rest .


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] mynameborat commented on a change in pull request #938: SAMZA-1531: Support run.id in standalone for batch processing.

[GitBox]

mynameborat commented on a change in pull request #938: SAMZA-1531: Support 
run.id in standalone for batch processing.
URL: https://github.com/apache/samza/pull/938#discussion_r262818100
 
 

 ##
 File path: 
samza-core/src/main/java/org/apache/samza/coordinator/DistributedReadWriteLock.java
 ##
 @@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.samza.coordinator;
+
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+
+/**
+ * Lock to acquire read access or write access
+ * At any point in time, only one processor can hold the lock
+ * The processor that has acquired the lock, holds it until it does `unlock` 
explicitly
+ * The first processor to acquire the lock should be given a WRITE access
+ * All other subsequent requests should acquire only READ access
+ */
+public interface DistributedReadWriteLock {
+
+  /**
+   * Try to acquire lock: returns the type of access lock was acquired for
+   * @param timeout Duration of lock acquiring timeout.
+   * @param unit Time Unit of the timeout defined above.
+   * @return AccessType.READ if lock is acquired for read, AcessType.WRITE if 
acquired for write and AccessType.NONE if not acquired
 
 Review comment:
   What is the scenario where you fail to acquire the lock but not timeout?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] mynameborat commented on a change in pull request #938: SAMZA-1531: Support run.id in standalone for batch processing.

[GitBox]

mynameborat commented on a change in pull request #938: SAMZA-1531: Support 
run.id in standalone for batch processing.
URL: https://github.com/apache/samza/pull/938#discussion_r262813679
 
 

 ##
 File path: 
samza-core/src/main/java/org/apache/samza/runtime/LocalApplicationRunner.java
 ##
 @@ -280,4 +368,47 @@ private void setApplicationFinalStatus() {
   }
 }
   }
+
+
+  /**
+   * Defines a specific implementation of {@link CoordinationSessionListener} 
for local {@link CoordinationUtils}
+   */
+  private final class LocalCoordinationSessionListener implements 
CoordinationSessionListener {
+
+/**
+ * If the coordination utils session has reconnected, check if global 
runid differs from local runid
+ * if it differs then shut down processor and throw exception
+ * else recreate ephemeral node corresponding to this processor inside the 
read write lock for runid
+ */
+@Override
+public void handleReconnect() {
+  LOG.info("Reconnected to coordination utils");
+  if(coordinationUtils == null) {
+return;
+  }
+  DistributedDataAccess runIdAccess = coordinationUtils.getDataAccess();
+  String globalRunId = (String) runIdAccess.readData(RUNID_PATH);
+  if( runId != globalRunId){
+processors.forEach(StreamProcessor::stop);
+cleanup();
+appStatus = ApplicationStatus.UnsuccessfulFinish;
+String msg = String.format("run.id %s on processor %s differs from the 
global run.id %s", runId, uid, globalRunId);
+throw new SamzaException(msg);
 
 Review comment:
   Can we populate the failure cause instead of throwing an exception? This 
exception will be thrown in the zk client thread and that wouldn't impact the 
main thread. It should be sufficient to stop the processors and update the 
status with unsuccessful finish and log the exception too. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Re: [DISCUSS] Samza 1.1.0 release

[Yi Pan]

+1 (binding)

On Wed, Mar 6, 2019 at 10:08 PM Daniel Chen  wrote:

> Hello everyone,
>
> We have added couple of major features to master since 1.0.0 that warrants
> a major release.
>
> Within LinkedIn, some of these features have already been tested as part of
> our test suites. We plan to continue our testing in coming weeks to
> validate the stability prior to release.
>
> Here is the highlighted list of features that are part of the new release
> (in chronological order)
> SAMZA-1981
> Consolidate table descriptors to samza-api
> SAMZA-1985
> Implement Startpoints model and StartpointManager
> SAMZA-1998
> Table API refactoring
> SAMZA-2012
> Add API for wiring an external context through to application processing
> code
> SAMZA-2041
> Add system descriptors for HDFS and Kinesis
> SAMZA-2043
> Consolidate ReadableTable and ReadWriteTable
> SAMZA-2106
> Samza App & Job Config Refactor
> SAMZA-2081
> Samza SQL : Type system for Samza SQL
>
> You can find a complete list of features here:
>
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fissues%2F%3Fjql%3Dproject%2520%253D%2520SAMZA%2520AND%2520resolution%2520%2520%253D%2520Fixed%2520%2520AND%2520(fixVersion%2520%253E%253D%25201.1%2520)%2520ORDER%2520BY%2520createdDate%2520%2520DESCdata=02%7C01%7Cdchen1%40linkedin.com%7C01251a7438ea4324f3f608d6a2c11a53%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636875347611087937sdata=ZDMaQj5vX6Vlm%2B8vpGhrNygxpI2vvNnYGi1USWe%2FD5A%3Dreserved=0
>
> Here is my proposal on our release schedule and timelines.
>
>1. Cut a release version 1.1.0 from master
>2. Target a release vote on the week March 13th (next week)
>
> Thoughts?
>
> Thanks,
> Daniel
>

Re: Trouble running samza 0.14 in standalone mode

[Yi Pan]

Hi, Anoop,

1. Please provide the full log file if possible. Just listing out a single
log line reporting the failure does not help.
2. Is there any reason that you still stay with Samza 0.14? I would highly
recommend to upgrade to 1.0 since there are tons of API and standalone
related improvements went in to that version after 0.14.

-Yi

On Wed, Mar 6, 2019 at 1:54 PM Anoop Krishnakumar <
anoop.krishnaku...@gmail.com> wrote:

> I am prototyping an application where I have two input topics and two
> output topics. The application also uses a rocksdb state store that is
> backed by kafka.
>
> I followed WikipediaZkLocalApplication example, but the application always
> exits without listening for messages from topic. I could see the execution
> plan and job coordinator communicating with zookeeper. I could see the
> following line in logs
> StreamProcessor [INFO] Container is not instantiated for stream processor:
> 5f7e2a46-fb7a-4054-bf5c-c62423ca2e35.
> I could not figure out why the container wouldn't start. Any help would be
> appreciated.
>
> Attached is the log and the job configuration.
>
> -anoop
>

[DISCUSS] Samza 1.1.0 release

[Daniel Chen]

Hello everyone,

We have added couple of major features to master since 1.0.0 that warrants
a major release.

Within LinkedIn, some of these features have already been tested as part of
our test suites. We plan to continue our testing in coming weeks to
validate the stability prior to release.

Here is the highlighted list of features that are part of the new release
(in chronological order)
SAMZA-1981
Consolidate table descriptors to samza-api
SAMZA-1985
Implement Startpoints model and StartpointManager
SAMZA-1998
Table API refactoring
SAMZA-2012
Add API for wiring an external context through to application processing
code
SAMZA-2041
Add system descriptors for HDFS and Kinesis
SAMZA-2043
Consolidate ReadableTable and ReadWriteTable
SAMZA-2106
Samza App & Job Config Refactor
SAMZA-2081
Samza SQL : Type system for Samza SQL

You can find a complete list of features here:
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fissues%2F%3Fjql%3Dproject%2520%253D%2520SAMZA%2520AND%2520resolution%2520%2520%253D%2520Fixed%2520%2520AND%2520(fixVersion%2520%253E%253D%25201.1%2520)%2520ORDER%2520BY%2520createdDate%2520%2520DESCdata=02%7C01%7Cdchen1%40linkedin.com%7C01251a7438ea4324f3f608d6a2c11a53%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636875347611087937sdata=ZDMaQj5vX6Vlm%2B8vpGhrNygxpI2vvNnYGi1USWe%2FD5A%3Dreserved=0

Here is my proposal on our release schedule and timelines.

   1. Cut a release version 1.1.0 from master
   2. Target a release vote on the week March 13th (next week)

Thoughts?

Thanks,
Daniel

[DISCUSS] Samza 1.1.0 release

[Daniel Chen]

Hello everyone,

We have added couple of major features to master since 1.0.0 that warrants
a major release.

Within LinkedIn, some of these features have already been tested as part of
our test suites. We plan to continue our testing in coming weeks to
validate the stability prior to release.

Here is the highlighted list of features that are part of the new release (in 
chronological order)
SAMZA-1981
Consolidate table descriptors to samza-api
SAMZA-1985
Implement Startpoints model and StartpointManager
SAMZA-1998
Table API refactoring
SAMZA-2012
Add API for wiring an external context through to application processing code
SAMZA-2041
Add system descriptors for HDFS and Kinesis
SAMZA-2043
Consolidate ReadableTable and ReadWriteTable
SAMZA-2106
Samza App & Job Config Refactor
SAMZA-2081
Samza SQL : Type system for Samza SQL

You can find a complete list of features here: 
https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20resolution%20%20%3D%20Fixed%20%20AND%20(fixVersion%20%3E%3D%201.1%20)%20ORDER%20BY%20createdDate%20%20DESC

Here is my proposal on our release schedule and timelines.

   1. Cut a release version 1.1.0 from master
   2. Target a release vote on the week March 13th (next week)

Thoughts?

Thanks,
Daniel

[GitHub] [samza] rmatharu opened a new pull request #942: Bugfix: Recent CSM refactor was causing some metrics to not be emitted. Fixed -restore-time metric.

[GitBox]

rmatharu opened a new pull request #942: Bugfix: Recent CSM refactor was 
causing some metrics to not be emitted. Fixed -restore-time metric.
URL: https://github.com/apache/samza/pull/942
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] xinyuiscool merged pull request #920: SAMZA-2106: Samza app and job config refactor

[GitBox]

xinyuiscool merged pull request #920: SAMZA-2106: Samza app and job config 
refactor
URL: https://github.com/apache/samza/pull/920
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] cameronlee314 opened a new pull request #941: SAMZA-2120: Enable custom handling of ConsumerRecords consumed by Kafka

[GitBox]

cameronlee314 opened a new pull request #941: SAMZA-2120: Enable custom 
handling of ConsumerRecords consumed by Kafka 
URL: https://github.com/apache/samza/pull/941
 
 
   This includes the code from https://github.com/apache/samza/pull/940.
   For this PR, it would be practical to just look at commit 
https://github.com/apache/samza/commit/e09f03022802b0757146915647a20a56478a7e4e.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Re: Trouble running samza 0.14 in standalone mode

[Bharath Kumarasubramanian]

Can you please attach the container logs? I noticed only job configurations 
were attached.

--
Bharath

From: Anoop Krishnakumar 
Reply-To: "dev@samza.apache.org" 
Date: Wednesday, March 6, 2019 at 1:54 PM
To: "dev@samza.apache.org" 
Subject: Trouble running samza 0.14 in standalone mode

I am prototyping an application where I have two input topics and two output 
topics. The application also uses a rocksdb state store that is backed by kafka.

I followed WikipediaZkLocalApplication example, but the application always 
exits without listening for messages from topic. I could see the execution plan 
and job coordinator communicating with zookeeper. I could see the following 
line in logs
StreamProcessor [INFO] Container is not instantiated for stream processor: 
5f7e2a46-fb7a-4054-bf5c-c62423ca2e35.
I could not figure out why the container wouldn't start. Any help would be 
appreciated.

Attached is the log and the job configuration.

-anoop

[GitHub] [samza] prateekm commented on issue #897: SEP-19 : Refactoring to remove localityManager from JobModel

[GitBox]

prateekm commented on issue #897: SEP-19 : Refactoring to remove 
localityManager from JobModel
URL: https://github.com/apache/samza/pull/897#issuecomment-470334092
 
 
   @rmatharu We also want to move the JobModel to samza-api (where 
ContainerModel and TaskModel already are).


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] Sanil15 commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor

[GitBox]

Sanil15 commented on a change in pull request #920: SAMZA-2106: Samza app and 
job config refactor
URL: https://github.com/apache/samza/pull/920#discussion_r263162324
 
 

 ##
 File path: samza-core/src/main/java/org/apache/samza/util/JobConfigUtil.java
 ##
 @@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.samza.util;
+
+import java.util.HashMap;
+import java.util.Map;
+import org.apache.samza.SamzaException;
+import org.apache.samza.config.ApplicationConfig;
+import org.apache.samza.config.JobConfig;
+import org.apache.samza.config.MapConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+public class JobConfigUtil {
 
 Review comment:
   will do 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] Sanil15 commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor

[GitBox]

Sanil15 commented on a change in pull request #920: SAMZA-2106: Samza app and 
job config refactor
URL: https://github.com/apache/samza/pull/920#discussion_r263162260
 
 

 ##
 File path: 
samza-core/src/main/java/org/apache/samza/runtime/RemoteApplicationRunner.java
 ##
 @@ -84,7 +85,7 @@ public void kill() {
 // since currently we only support single actual remote job, we can get 
its status without
 // building the execution plan.
 try {
-  JobConfig jc = new JobConfig(appDesc.getConfig());
+  JobConfig jc = new 
JobConfig(JobConfigUtil.generateJobIdAndName(appDesc.getConfig()));
 
 Review comment:
   will do


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] Sanil15 commented on a change in pull request #920: SAMZA-2106: Samza app and job config refactor

[GitBox]

Sanil15 commented on a change in pull request #920: SAMZA-2106: Samza app and 
job config refactor
URL: https://github.com/apache/samza/pull/920#discussion_r263162204
 
 

 ##
 File path: 
samza-core/src/main/java/org/apache/samza/execution/ExecutionPlanner.java
 ##
 @@ -113,7 +115,7 @@ private void validateConfig() {
*/
   /* package private */
   JobGraph createJobGraph(ApplicationDescriptorImpl appDesc) {
-JobGraph jobGraph = new JobGraph(config, appDesc);
+JobGraph jobGraph = new JobGraph(new MapConfig(config), appDesc);
 
 Review comment:
   yes that should not exist


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Trouble running samza 0.14 in standalone mode

[Anoop Krishnakumar]

I am prototyping an application where I have two input topics and two
output topics. The application also uses a rocksdb state store that is
backed by kafka.

I followed WikipediaZkLocalApplication example, but the application always
exits without listening for messages from topic. I could see the execution
plan and job coordinator communicating with zookeeper. I could see the
following line in logs
StreamProcessor [INFO] Container is not instantiated for stream processor:
5f7e2a46-fb7a-4054-bf5c-c62423ca2e35.
I could not figure out why the container wouldn't start. Any help would be
appreciated.

Attached is the log and the job configuration.

-anoop
job.name=iva-stream-application
job.default.system=kafka
job.coordinator.factory=org.apache.samza.zk.ZkJobCoordinatorFactory
job.coordinator.zk.connect=localhost:2181
# Kafka System
systems.kafka.samza.factory=org.apache.samza.system.kafka.KafkaSystemFactory
systems.kafka.consumer.zookeeper.connect=localhost:2181
systems.kafka.producer.bootstrap.servers=localhost:9092
systems.kafka.producer.max.request.size=15000180
systems.kafka.default.stream.replication.factor=1
# Input Streams
streams.sensor-audit.samza.system=kafka
streams.sensor-audit.samza.physical.name=rsdata.public.issues
streams.risk-updates.samza.system=kafka
streams.risk-updates.samza.physical.name=rsdata.public.rs_updates_issues_timestamps2
# State store
serializers.registry.string.class=org.apache.samza.serializers.StringSerdeFactory
serializers.registry.assetlabelserde.class=com.rootsecure.insights.serde.AssetLabelSerdeFactory
stores.AssetLabelStore.factory=org.apache.samza.storage.kv.RocksDbKeyValueStorageEngineFactory
stores.AssetLabelStore.changelog=kafka.asset-labels-changelog
stores.AssetLabelStore.key.serde=string
stores.AssetLabelStore.msg.serde=assetlabelserde

[GitHub] [samza] sborya merged pull request #939: SAMZA-2122: Fix the task caught-up logic which doesn't handle no incoming messages

[GitBox]

sborya merged pull request #939: SAMZA-2122: Fix the task caught-up logic which 
doesn't handle no incoming messages
URL: https://github.com/apache/samza/pull/939
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] cameronlee314 opened a new pull request #940: SAMZA-2121: Add checkpoint offset field to IncomingMessageEnvelope

[GitBox]

cameronlee314 opened a new pull request #940: SAMZA-2121: Add checkpoint offset 
field to IncomingMessageEnvelope
URL: https://github.com/apache/samza/pull/940
 
 
   This is the first step for implementing 
https://issues.apache.org/jira/browse/SAMZA-2120.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] xinyuiscool commented on a change in pull request #939: SAMZA-2122: Fix the task caught-up logic which doesn't handle no incoming messages

[GitBox]

xinyuiscool commented on a change in pull request #939: SAMZA-2122: Fix the 
task caught-up logic which doesn't handle no incoming messages
URL: https://github.com/apache/samza/pull/939#discussion_r263086035
 
 

 ##
 File path: 
samza-core/src/main/scala/org/apache/samza/container/TaskInstance.scala
 ##
 @@ -321,6 +323,25 @@ class TaskInstance(
 }
   }
 
+  def initCaughtUpMapping() {
 
 Review comment:
   Added the comments.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] xinyuiscool commented on a change in pull request #939: SAMZA-2122: Fix the task caught-up logic which doesn't handle no incoming messages

[GitBox]

xinyuiscool commented on a change in pull request #939: SAMZA-2122: Fix the 
task caught-up logic which doesn't handle no incoming messages
URL: https://github.com/apache/samza/pull/939#discussion_r263086086
 
 

 ##
 File path: 
samza-core/src/main/scala/org/apache/samza/container/TaskInstance.scala
 ##
 @@ -321,6 +323,25 @@ class TaskInstance(
 }
   }
 
+  def initCaughtUpMapping() {
+if (taskContext.getStreamMetadataCache != null) {
+  systemStreamPartitions.foreach(ssp => {
+val partitionMetadata = taskContext
+  .getStreamMetadataCache
+  .getSystemStreamMetadata(ssp.getSystemStream, false)
+  .getSystemStreamPartitionMetadata.get(ssp.getPartition)
+
+val upcomingOffset = partitionMetadata.getUpcomingOffset
+val startingOffset = offsetManager.getStartingOffset(taskName, ssp)
+  .getOrElse(throw new SamzaException("No offset defined for 
SystemStreamPartition: %s" format ssp))
+
+if(Objects.equals(upcomingOffset, startingOffset)) {
+  ssp2CaughtupMapping(ssp) = true
 
 Review comment:
   Added.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] sborya commented on a change in pull request #939: SAMZA-2122: Fix the task caught-up logic which doesn't handle no incoming messages

[GitBox]

sborya commented on a change in pull request #939: SAMZA-2122: Fix the task 
caught-up logic which doesn't handle no incoming messages
URL: https://github.com/apache/samza/pull/939#discussion_r263078743
 
 

 ##
 File path: 
samza-core/src/main/scala/org/apache/samza/container/TaskInstance.scala
 ##
 @@ -321,6 +323,25 @@ class TaskInstance(
 }
   }
 
+  def initCaughtUpMapping() {
+if (taskContext.getStreamMetadataCache != null) {
+  systemStreamPartitions.foreach(ssp => {
+val partitionMetadata = taskContext
+  .getStreamMetadataCache
+  .getSystemStreamMetadata(ssp.getSystemStream, false)
+  .getSystemStreamPartitionMetadata.get(ssp.getPartition)
+
+val upcomingOffset = partitionMetadata.getUpcomingOffset
+val startingOffset = offsetManager.getStartingOffset(taskName, ssp)
+  .getOrElse(throw new SamzaException("No offset defined for 
SystemStreamPartition: %s" format ssp))
+
+if(Objects.equals(upcomingOffset, startingOffset)) {
+  ssp2CaughtupMapping(ssp) = true
 
 Review comment:
   comment here.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] sborya commented on a change in pull request #939: SAMZA-2122: Fix the task caught-up logic which doesn't handle no incoming messages

[GitBox]

sborya commented on a change in pull request #939: SAMZA-2122: Fix the task 
caught-up logic which doesn't handle no incoming messages
URL: https://github.com/apache/samza/pull/939#discussion_r263078808
 
 

 ##
 File path: 
samza-core/src/main/scala/org/apache/samza/container/TaskInstance.scala
 ##
 @@ -321,6 +323,25 @@ class TaskInstance(
 }
   }
 
+  def initCaughtUpMapping() {
 
 Review comment:
   Comment for the method.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

[GitHub] [samza] asfgit closed pull request #934: SAMZA-1935 : Refactor TaskContextImpl to not include access to objects that are only used internally

[GitBox]

asfgit closed pull request #934: SAMZA-1935 : Refactor TaskContextImpl to not 
include access to objects that are only used internally
URL: https://github.com/apache/samza/pull/934
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Re: [POSSIBLE PHISHING] Task Partition Commit Failed After Upgrade

[Prateek Maheshwari]

Hi Jeremiah,

The configuration you want to look for is:
'job.systemstreampartition.grouper.factory'. It should default to:
'org.apache.samza.container.grouper.stream.GroupByPartitionFactory'.
Can you check if you see this value in the configuration logged by
SamzaContainer during container start? You can grep for: "Using
configuration".

For context, there are two groupers for a Samza job. One that groups input
partitions into tasks (this one), and one that groups tasks into containers
(the one you mentioned above).

Thanks,
Prateek



On Wed, Mar 6, 2019 at 8:14 AM Jeremiah Adams 
wrote:

> It appears that the issue is related to the KafkaCheckpointLogKey.java
> constructor. grouperFactoryClassName here is null.  THe documentation
> indicates that task.name.grouper.factory config setting has a default value
> of
>  org.apache.samza.container.grouper.task.GroupByContainerCountFactory. I
> wouldn't expect it to be null here.
>
> If I specify GroupByContainerCountFactory for the
> task.name.grouper.factory in my properties file, I get a
> NoSuchMethodException:
>
> Exception in thread "main" java.lang.InstantiationException:
> org.apache.samza.container.grouper.task.GroupByContainerCount
> at java.lang.Class.newInstance(Class.java:427)
> at org.apache.samza.util.Util$.getObj(Util.scala:80)
> at
> org.apache.samza.coordinator.JobModelManager$.readJobModel(JobModelManager.scala:261)
> at
> org.apache.samza.coordinator.JobModelManager$.getJobModelManager(JobModelManager.scala:155)
> at
> org.apache.samza.coordinator.JobModelManager$.apply(JobModelManager.scala:117)
> at
> org.apache.samza.coordinator.JobModelManager.apply(JobModelManager.scala)
> at
> org.apache.samza.clustermanager.ClusterBasedJobCoordinator.buildJobModelManager(ClusterBasedJobCoordinator.java:241)
> at
> org.apache.samza.clustermanager.ClusterBasedJobCoordinator.(ClusterBasedJobCoordinator.java:152)
> at
> org.apache.samza.clustermanager.ClusterBasedJobCoordinator.main(ClusterBasedJobCoordinator.java:297)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.samza.container.grouper.task.GroupByContainerCount.()
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.newInstance(Class.java:412)
> ... 8 more
>
>
>
> Jeremiah Adams
> Software Engineer
> www.helixeducation.com 
> Blog  | Twitter <
> https://twitter.com/HelixEducation> | Facebook <
> https://www.facebook.com/HelixEducation> | LinkedIn <
> http://www.linkedin.com/company/3609946>
>
>
> On 3/4/19, 2:48 PM, "Jeremiah Adams"  wrote:
>
> I am updating dependencies and moving from Samza V0.13.0 to V0.14.0.
> I develop locally using the grid app in the hello-samza project to spin up
> local yarn/zookeeper/kafka instances.
>
> Grid is running these versions:
> kafka_2.11-0.10.2.1.tgz
> hadoop-2.6.1.tar.gz
> zookeeper-3.4.3.tar.gz
>
>
> My job is now failing with the NPE below. anyone have ideas on the
> cause of this error?
>
>
> 2019-03-04 14:13:49 AsyncRunLoop [ERROR] Task Partition 0 commit failed
> java.lang.NullPointerException
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:782)
> at
> org.apache.samza.checkpoint.kafka.KafkaCheckpointLogKey.(KafkaCheckpointLogKey.java:46)
> at
> org.apache.samza.checkpoint.kafka.KafkaCheckpointManager.writeCheckpoint(KafkaCheckpointManager.scala:136)
> at
> org.apache.samza.checkpoint.OffsetManager.writeCheckpoint(OffsetManager.scala:259)
> at
> org.apache.samza.container.TaskInstance.commit(TaskInstance.scala:205)
> at
> org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker$5.run(AsyncRunLoop.java:494)
> at
> org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.commit(AsyncRunLoop.java:513)
> at
> org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.run(AsyncRunLoop.java:379)
> at
> org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.access$300(AsyncRunLoop.java:314)
> at
> org.apache.samza.task.AsyncRunLoop.runTasks(AsyncRunLoop.java:228)
> at
> org.apache.samza.task.AsyncRunLoop.run(AsyncRunLoop.java:157)
> at
> org.apache.samza.container.SamzaContainer.run(SamzaContainer.scala:728)
> at
> org.apache.samza.runtime.LocalContainerRunner.run(LocalContainerRunner.java:102)
> at
> org.apache.samza.runtime.LocalContainerRunner.main(LocalContainerRunner.java:147)
> 2019-03-04 14:13:49 AsyncRunLoop [ERROR] Caught throwable and stopping
> run loop
>
>
>
> Jeremiah Adams
> Software Engineer
>
> https://url.emailprotection.link/?ahfhEufaAWbezBrUFPG98ZJcterGfIerU3ZwsA3Gv_C0~
> <
> https://url.emailprotection.link/?a49H2rNGIIBtQOw6md8OcHp-qKE3Xn2gNiZ3dlqAeSDA~
> >
> Blog<
> 

Re: [POSSIBLE PHISHING] Task Partition Commit Failed After Upgrade

[Jeremiah Adams]

It appears that the issue is related to the KafkaCheckpointLogKey.java 
constructor. grouperFactoryClassName here is null.  THe documentation indicates 
that task.name.grouper.factory config setting has a default value of 
org.apache.samza.container.grouper.task.GroupByContainerCountFactory. I 
wouldn't expect it to be null here.

If I specify GroupByContainerCountFactory for the task.name.grouper.factory in 
my properties file, I get a NoSuchMethodException:

Exception in thread "main" java.lang.InstantiationException: 
org.apache.samza.container.grouper.task.GroupByContainerCount
at java.lang.Class.newInstance(Class.java:427)
at org.apache.samza.util.Util$.getObj(Util.scala:80)
at 
org.apache.samza.coordinator.JobModelManager$.readJobModel(JobModelManager.scala:261)
at 
org.apache.samza.coordinator.JobModelManager$.getJobModelManager(JobModelManager.scala:155)
at 
org.apache.samza.coordinator.JobModelManager$.apply(JobModelManager.scala:117)
at 
org.apache.samza.coordinator.JobModelManager.apply(JobModelManager.scala)
at 
org.apache.samza.clustermanager.ClusterBasedJobCoordinator.buildJobModelManager(ClusterBasedJobCoordinator.java:241)
at 
org.apache.samza.clustermanager.ClusterBasedJobCoordinator.(ClusterBasedJobCoordinator.java:152)
at 
org.apache.samza.clustermanager.ClusterBasedJobCoordinator.main(ClusterBasedJobCoordinator.java:297)
Caused by: java.lang.NoSuchMethodException: 
org.apache.samza.container.grouper.task.GroupByContainerCount.()
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.newInstance(Class.java:412)
... 8 more

 

Jeremiah Adams
Software Engineer
www.helixeducation.com 
Blog  | Twitter 
 | Facebook 
 | LinkedIn 

 

On 3/4/19, 2:48 PM, "Jeremiah Adams"  wrote:

I am updating dependencies and moving from Samza V0.13.0 to V0.14.0.  I 
develop locally using the grid app in the hello-samza project to spin up local 
yarn/zookeeper/kafka instances.

Grid is running these versions:
kafka_2.11-0.10.2.1.tgz
hadoop-2.6.1.tar.gz
zookeeper-3.4.3.tar.gz


My job is now failing with the NPE below. anyone have ideas on the cause of 
this error?


2019-03-04 14:13:49 AsyncRunLoop [ERROR] Task Partition 0 commit failed
java.lang.NullPointerException
at 
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:782)
at 
org.apache.samza.checkpoint.kafka.KafkaCheckpointLogKey.(KafkaCheckpointLogKey.java:46)
at 
org.apache.samza.checkpoint.kafka.KafkaCheckpointManager.writeCheckpoint(KafkaCheckpointManager.scala:136)
at 
org.apache.samza.checkpoint.OffsetManager.writeCheckpoint(OffsetManager.scala:259)
at 
org.apache.samza.container.TaskInstance.commit(TaskInstance.scala:205)
at 
org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker$5.run(AsyncRunLoop.java:494)
at 
org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.commit(AsyncRunLoop.java:513)
at 
org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.run(AsyncRunLoop.java:379)
at 
org.apache.samza.task.AsyncRunLoop$AsyncTaskWorker.access$300(AsyncRunLoop.java:314)
at 
org.apache.samza.task.AsyncRunLoop.runTasks(AsyncRunLoop.java:228)
at org.apache.samza.task.AsyncRunLoop.run(AsyncRunLoop.java:157)
at 
org.apache.samza.container.SamzaContainer.run(SamzaContainer.scala:728)
at 
org.apache.samza.runtime.LocalContainerRunner.run(LocalContainerRunner.java:102)
at 
org.apache.samza.runtime.LocalContainerRunner.main(LocalContainerRunner.java:147)
2019-03-04 14:13:49 AsyncRunLoop [ERROR] Caught throwable and stopping run 
loop



Jeremiah Adams
Software Engineer

https://url.emailprotection.link/?ahfhEufaAWbezBrUFPG98ZJcterGfIerU3ZwsA3Gv_C0~

Blog
 | 
Twitter
 | 
Facebook
 | 
LinkedIn

4 Apache Events in 2019: DC Roadshow soon; next up Chicago, Las Vegas, and Berlin!

[Rich Bowen]

Dear Apache Enthusiast,

(You’re receiving this because you are subscribed to one or more user
mailing lists for an Apache Software Foundation project.)

TL;DR:
 * Apache Roadshow DC is in 3 weeks. Register now at
https://apachecon.com/usroadshowdc19/
 * Registration for Apache Roadshow Chicago is open.
http://apachecon.com/chiroadshow19
 * The CFP for ApacheCon North America is now open.
https://apachecon.com/acna19
 * Save the date: ApacheCon Europe will be held in Berlin, October 22nd
through 24th.  https://apachecon.com/aceu19


Registration is open for two Apache Roadshows; these are smaller events
with a more focused program and regional community engagement:

Our Roadshow event in Washington DC takes place in under three weeks, on
March 25th. We’ll be hosting a day-long event at the Fairfax campus of
George Mason University. The roadshow is a full day of technical talks
(two tracks) and an open source job fair featuring AWS, Bloomberg, dito,
GridGain, Linode, and Security University. More details about the
program, the job fair, and to register, visit
https://apachecon.com/usroadshowdc19/

Apache Roadshow Chicago will be held May 13-14th at a number of venues
in Chicago’s Logan Square neighborhood. This event will feature sessions
in AdTech, FinTech and Insurance, startups, “Made in Chicago”, Project
Shark Tank (innovations from the Apache Incubator), community diversity,
and more. It’s a great way to learn about various Apache projects “at
work” while playing at a brewery, a beercade, and a neighborhood bar.
Sign up today at https://www.apachecon.com/chiroadshow19/

We’re delighted to announce that the Call for Presentations (CFP) is now
open for ApacheCon North America in Las Vegas, September 9-13th! As the
official conference series of the ASF, ApacheCon North America will
feature over a dozen Apache project summits, including Cassandra,
Cloudstack, Tomcat, Traffic Control, and more. We’re looking for talks
in a wide variety of categories -- anything related to ASF projects and
the Apache development process. The CFP closes at midnight on May 26th.
In addition, the ASF will be celebrating its 20th Anniversary during the
event. For more details and to submit a proposal for the CFP, visit
https://apachecon.com/acna19/ . Registration will be opening soon.

Be sure to mark your calendars for ApacheCon Europe, which will be held
in Berlin, October 22-24th at the KulturBrauerei, a landmark of Berlin's
industrial history. In addition to innovative content from our projects,
we are collaborating with the Open Source Design community
(https://opensourcedesign.net/) to offer a track on design this year.
The CFP and registration will open soon at https://apachecon.com/aceu19/ .

Sponsorship opportunities are available for all events, with details
listed on each event’s site at http://apachecon.com/.

We look forward to seeing you!

Rich, for the ApacheCon Planners
@apachecon