EmbeddedKafka Server for testing kafka related code

2016-10-13 Thread Chinmay Kolhatkar
Dear Community, While testing kafka related endpoint code for Apex-Calcite integration, I found an interesting embedded kafka implementation over the internet and I used it in my code. The implementation is based on ready to use EmbeddedZookeer and KafkaServer from mvn dependency

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572169#comment-15572169 ] Chandni Singh commented on APEXMALHAR-2284: --- 1. put needs to done only after you retrieve

Re: (APEXMALHAR-2290) JDBC operator does not deploy after failure in certain cases

2016-10-13 Thread Chinmay Kolhatkar
+1 On Thu, Oct 13, 2016 at 7:48 PM, Hitesh Kapoor wrote: > Hi All, > > Thank you for your suggestions. > Following the approach as suggested by Chinmay in option 1. > Will make the necessary code changes, test it and create a PR. > > --Hitesh > > On Thu, Oct 13, 2016 at

Re: (APEXMALHAR-2290) JDBC operator does not deploy after failure in certain cases

2016-10-13 Thread Hitesh Kapoor
Hi All, Thank you for your suggestions. Following the approach as suggested by Chinmay in option 1. Will make the necessary code changes, test it and create a PR. --Hitesh On Thu, Oct 13, 2016 at 11:20 AM, Pradeep A. Dalvi wrote: > +1 for Option 1 to use conn.getMetaData() >

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572183#comment-15572183 ] Chandni Singh commented on APEXMALHAR-2284: --- I think the proper fix for this may take time.

[GitHub] apex-malhar pull request #420: APEXMALHAR-2242 Added user documentation for ...

2016-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/apex-malhar/pull/420 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

Re: EmbeddedKafka Server for testing kafka related code

2016-10-13 Thread Sanjay Pujare
Strong +1 for using embedded kafka. Found embedded servers very useful in ActiveMQ and SQS (ElasticMQ) unit tests. My only question/concern is how well can you simulate different versions of Kafka. On 10/13/16, 7:25 AM, "Chinmay Kolhatkar" wrote: Dear Community,

[jira] [Commented] (APEXCORE-559) Support quit which should behave the same as "exit"

2016-10-13 Thread Sanjay M Pujare (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572630#comment-15572630 ] Sanjay M Pujare commented on APEXCORE-559: -- bash is a Linux/Unix command line shell and

[jira] [Commented] (APEXMALHAR-2220) Move the FunctionOperator to Malhar library

2016-10-13 Thread Dongming Liang (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572382#comment-15572382 ] Dongming Liang commented on APEXMALHAR-2220: [~siyuan] Did you have a chance to look into

[jira] [Created] (APEXCORE-557) Upgrade netlet dependency to 1.3.0

2016-10-13 Thread Vlad Rozov (JIRA)
Vlad Rozov created APEXCORE-557: --- Summary: Upgrade netlet dependency to 1.3.0 Key: APEXCORE-557 URL: https://issues.apache.org/jira/browse/APEXCORE-557 Project: Apache Apex Core Issue Type:

[jira] [Resolved] (APEXMALHAR-2242) Add documentation for 0.9 version of Kafka Input Operator.

2016-10-13 Thread Munagala V. Ramanath (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Munagala V. Ramanath resolved APEXMALHAR-2242. -- Resolution: Fixed Fix Version/s: 3.6.0 Merged. > Add

[jira] [Created] (APEXCORE-558) Do not use yellow color to display command strings in help output

2016-10-13 Thread Sanjay M Pujare (JIRA)
Sanjay M Pujare created APEXCORE-558: Summary: Do not use yellow color to display command strings in help output Key: APEXCORE-558 URL: https://issues.apache.org/jira/browse/APEXCORE-558 Project:

[jira] [Commented] (APEXMALHAR-2242) Add documentation for 0.9 version of Kafka Input Operator.

2016-10-13 Thread ASF GitHub Bot (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572645#comment-15572645 ] ASF GitHub Bot commented on APEXMALHAR-2242: Github user asfgit closed the pull request

Re: EmbeddedKafka Server for testing kafka related code

2016-10-13 Thread Thomas Weise
+1, how hard is it to change the existing tests in the Kafka module to use it? Different versions are IMO not an issue for the existing tests, they already run against a fixed version (the broker is launched as part of the test). On Thu, Oct 13, 2016 at 9:59 AM, Sanjay Pujare

[jira] [Created] (APEXCORE-559) Support quit which should behave the same as "exit"

2016-10-13 Thread Sanjay M Pujare (JIRA)
Sanjay M Pujare created APEXCORE-559: Summary: Support quit which should behave the same as "exit" Key: APEXCORE-559 URL: https://issues.apache.org/jira/browse/APEXCORE-559 Project: Apache Apex

[jira] [Commented] (APEXCORE-558) Do not use yellow color to display command strings in help output

2016-10-13 Thread Munagala V. Ramanath (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572586#comment-15572586 ] Munagala V. Ramanath commented on APEXCORE-558: --- Yes, agree, the yellow has inadequate

[jira] [Commented] (APEXCORE-559) Support quit which should behave the same as "exit"

2016-10-13 Thread Munagala V. Ramanath (JIRA)
[ https://issues.apache.org/jira/browse/APEXCORE-559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572610#comment-15572610 ] Munagala V. Ramanath commented on APEXCORE-559: --- I agree that this improves usability

[jira] [Commented] (APEXMALHAR-2283) Issues with Kafka output operator

2016-10-13 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572849#comment-15572849 ] Thomas Weise commented on APEXMALHAR-2283: -- It also appears that the user has to specify the

[jira] [Created] (APEXMALHAR-2297) Readme files for Demos

2016-10-13 Thread Sandesh (JIRA)
Sandesh created APEXMALHAR-2297: --- Summary: Readme files for Demos Key: APEXMALHAR-2297 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2297 Project: Apache Apex Malhar Issue Type:

Re: [jira] [Updated] (APEXCORE-553) Add google analytics tracking to apex website

2016-10-13 Thread Michelle Xiao
I suggest to install Google Tag Manager on Apex website, and add Google Analytics there. GA is the analytics tool that provides reports about activity on the site. GTM is a tag manager that can output tags based on defined rules. The main benefit of GTM is that we can use it for multiple tags,

[jira] [Commented] (APEXMALHAR-2220) Move the FunctionOperator to Malhar library

2016-10-13 Thread Siyuan Hua (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15573091#comment-15573091 ] Siyuan Hua commented on APEXMALHAR-2220: [~d9liang] The malhar-library should be depends on

[jira] [Commented] (APEXMALHAR-2190) Use reusable buffer to serial spillable data structure

2016-10-13 Thread bright chen (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15573426#comment-15573426 ] bright chen commented on APEXMALHAR-2190: - Add new features - Add ByteStreamMonitor to

Re: EmbeddedKafka Server for testing kafka related code

2016-10-13 Thread Chinmay Kolhatkar
I can look into possibility of using this in existing tests as well. Basically the idea behind this EmbeddedKafka class is to provide a single object which spawns up JVM-local Kafka Cluster. Possibly there are extensions (like multiple brokers etc) which needs to be done in this class for making

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Thomas Weise (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574244#comment-15574244 ] Thomas Weise commented on APEXMALHAR-2284: -- That's fine. Can you summarize how the issue

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574274#comment-15574274 ] Chaitanya commented on APEXMALHAR-2284: --- Right now the operator is broken. >

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571020#comment-15571020 ] Chaitanya commented on APEXMALHAR-2284: --- This was failing due to the below merged PR:

[jira] [Comment Edited] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571020#comment-15571020 ] Chaitanya edited comment on APEXMALHAR-2284 at 10/13/16 6:46 AM: -

[jira] [Comment Edited] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571074#comment-15571074 ] Chandni Singh edited comment on APEXMALHAR-2284 at 10/13/16 6:57 AM:

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571291#comment-15571291 ] Chaitanya commented on APEXMALHAR-2284: --- [~csingh] If you feel that adding the variable

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571119#comment-15571119 ] Chandni Singh commented on APEXMALHAR-2284: --- Chaitanya, Please feel free to assign the

Re: can operators emit on a different from the operator itself thread?

2016-10-13 Thread Amol Kekre
Vlad, I agree that the check should be ON by default. Ability to turn it off for entire app is fine, per port is not needed. Thks Amol On Wed, Oct 12, 2016 at 10:34 PM, Tushar Gosavi wrote: > +1 for on by default and ability to turn it off for entire application. > > -

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571074#comment-15571074 ] Chandni Singh commented on APEXMALHAR-2284: --- Are you sure about that? The tests were

[jira] [Comment Edited] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chandni Singh (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571119#comment-15571119 ] Chandni Singh edited comment on APEXMALHAR-2284 at 10/13/16 7:27 AM:

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571170#comment-15571170 ] Chaitanya commented on APEXMALHAR-2284: --- getAsync() method has two parts. First it looks into

[jira] [Commented] (APEXMALHAR-2284) POJOInnerJoinOperatorTest fails in Travis CI

2016-10-13 Thread Chaitanya (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15571181#comment-15571181 ] Chaitanya commented on APEXMALHAR-2284: --- Changes which are reverted in PR(#444) are related to

S3RecordReader design review

2016-10-13 Thread AJAY GUPTA
Hi Apex Dev community, Kindly provide me with feedback for the S3RecordReader design below The design will be a merge of approaches from S3BlockReader to read blocks from S3 and FSRecordReader to emit records from these blocks. *S3RecordReader-for Delimited Records* FSRecordReader uses

[jira] [Created] (APEXMALHAR-2292) Add generic endpoint to sql support of Apex

2016-10-13 Thread Chinmay Kolhatkar (JIRA)
Chinmay Kolhatkar created APEXMALHAR-2292: - Summary: Add generic endpoint to sql support of Apex Key: APEXMALHAR-2292 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2292 Project: Apache

[jira] [Updated] (APEXMALHAR-2292) Add generic endpoint to sql support of Apex

2016-10-13 Thread Chinmay Kolhatkar (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar updated APEXMALHAR-2292: -- Issue Type: New Feature (was: Improvement) > Add generic endpoint to sql

[jira] [Created] (APEXMALHAR-2293) Add sql line cli to sql support of apex

2016-10-13 Thread Chinmay Kolhatkar (JIRA)
Chinmay Kolhatkar created APEXMALHAR-2293: - Summary: Add sql line cli to sql support of apex Key: APEXMALHAR-2293 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2293 Project: Apache

[jira] [Updated] (APEXMALHAR-2292) Add generic endpoint to malhar-sql

2016-10-13 Thread Chinmay Kolhatkar (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar updated APEXMALHAR-2292: -- Summary: Add generic endpoint to malhar-sql (was: malhar-sql: Add generic

[jira] [Updated] (APEXMALHAR-2292) malhar-sql: Add generic endpoint to sql support of Apex

2016-10-13 Thread Chinmay Kolhatkar (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar updated APEXMALHAR-2292: -- Summary: malhar-sql: Add generic endpoint to sql support of Apex (was: Add

[jira] [Updated] (APEXMALHAR-2292) Apex SQL: Add generic endpoint to malhar-sql

2016-10-13 Thread Chinmay Kolhatkar (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar updated APEXMALHAR-2292: -- Summary: Apex SQL: Add generic endpoint to malhar-sql (was: Add generic

[jira] [Created] (APEXMALHAR-2294) Apex SQL: Make operator names user friendly for user to configure via properties.xml

2016-10-13 Thread Chinmay Kolhatkar (JIRA)
Chinmay Kolhatkar created APEXMALHAR-2294: - Summary: Apex SQL: Make operator names user friendly for user to configure via properties.xml Key: APEXMALHAR-2294 URL:

[jira] [Updated] (APEXMALHAR-2293) Apex SQL: Add sql line cli to sql support of apex

2016-10-13 Thread Chinmay Kolhatkar (JIRA)
[ https://issues.apache.org/jira/browse/APEXMALHAR-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chinmay Kolhatkar updated APEXMALHAR-2293: -- Summary: Apex SQL: Add sql line cli to sql support of apex (was: Add sql

[jira] [Created] (APEXMALHAR-2295) Apex SQL: Replace POJOInnerJoinOperator with WindowedJoinOperator

2016-10-13 Thread Chinmay Kolhatkar (JIRA)
Chinmay Kolhatkar created APEXMALHAR-2295: - Summary: Apex SQL: Replace POJOInnerJoinOperator with WindowedJoinOperator Key: APEXMALHAR-2295 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2295

[jira] [Created] (APEXMALHAR-2296) Apex SQL: Add support for SQL GROUP BY (Aggregate RelNode)

2016-10-13 Thread Chinmay Kolhatkar (JIRA)
Chinmay Kolhatkar created APEXMALHAR-2296: - Summary: Apex SQL: Add support for SQL GROUP BY (Aggregate RelNode) Key: APEXMALHAR-2296 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2296